CJSmith dot me

I dump stuff I find useful here

Category: SQL Server 2012

Sage Line 500 password policy implementation and force password reset

Published / by Chris Smith / Leave a Comment

I wrote a simple script to aid users moving to a new password policy, users were put on a policy which would expire a password after a day. Once they had changed their password the policy is amended to the new policy. I now use this for forcing password resets. Replace and with your Sage Line 500/1000 password policies

-- =============================================
-- Author:           Christopher Smith
-- Create date: 18/07/2018
-- Description:      Implement the new password policy in stages.
-- 1. Change the password policy first to enforce a password change in a short time period
-- 2. Backup and truncate the table [csmaster].[scheme].[pwdhistm]
-- 3. Run this routine nightly, it will pickup users who have changed their password and set the new policy
-- =============================================
-- Change Log:
-- 1.0 - 18/07/2018 - First Version
-- =============================================
-- exec dbo.usp_SageImplementPWPolicy
-- exec [dbo].[usp_ForceSagePWChange] 'csmithte'
-- =============================================
CREATE PROCEDURE [dbo].[usp_SageImplementPWPolicy]
AS
BEGIN
       SET NOCOUNT ON;
-- =============================================
print 'Declare Variables and setup'
-- =============================================
  declare @spdate1 date
  declare @spdate2 int
  declare @secgroup char(3)
  declare @oldsecgroup char(4)
-- Set variable to 2 days ago
  set @spdate1 = CAST(GETDATE() as date)
  print @spdate1
-- Convert the date to one which matches the Sage table
  set @spdate2 = DATEDIFF(d,'01 jan 1900',@spdate1)
  print 'Sage Date Code for Today'
  print @spdate2
  print 'New user security group'
  set @secgroup = ''
  set @oldsecgroup = ''
  print @secgroup
  print @oldsecgroup
-- =============================================
print 'Populate working table'
-- =============================================
       select t1.name, dateadd(d,t2."setup",'01 jan 1900') as pw_date, t1.sec_group, 'NO ' as pw_changed into #pwworking
              from scheme.[usermastm](nolock) t1
                     INNER JOIN scheme.passexpm(nolock) t2
                           on t1.name = t2.user_id
              where t1.sec_group = @oldsecgroup
       select * from #pwworking
-- =============================================
print 'Check if password recently changed'
-- =============================================
       update #pwworking
              set pw_changed = 'YES'
                     where name IN
                           (select "username" from [csmaster].[scheme].[pwdhistm] group by "username")
       select * from #pwworking
-- =============================================
print 'Change security group for those who have a new password on the new policy'
-- =============================================
       update scheme.[usermastm]
              set sec_group = @secgroup
                     where name IN (select name from #pwworking where pw_changed = 'YES')
drop table #pwworking
END
GO

Below is the code used to force users to enter a new password upon login

USE [csmaster]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE procedure [dbo].[usp_ForceSagePWChange] @user char(8)
--EXEC usp_ForceSagePWChange ''
AS
--IF @user = 'ALL' GOTO ALLUSERS
--IF @user != 'ALL' GOTO SINGLEUSER
BEGIN
/*ALLUSERS:
update scheme.passexpm
set setup = '10001'
GOTO FINISH
SINGLEUSER:*/

update csmaster.scheme.passexpm
set setup = '10001'
where user_id = @user
update csmaster.[scheme].[usermastm]
set [sec_group] = ''
where name = @user
GOTO FINISH
FINISH:
END

Setup MS SQL Server Alerts via TSQL

Published / by Chris Smith / Leave a Comment

Amend TSQL to your environment, run the TSQL, restart the agent service and test (I find it easy to enter an incorrect username and password).

USE [msdb]
GO
EXEC msdb.dbo.sp_set_sqlagent_properties @email_save_in_sent_folder=1,                              @databasemail_profile=N'<TSQL_MAIL_PROFILE>',                              @use_databasemail=1
GO

USE [msdb]
GO
EXEC msdb.dbo.sp_add_alert @name=N'Login Mismatch',                               @message_id=18456,                               @severity=0,                               @enabled=1,                               @delay_between_responses=0,                               @include_event_description_in=1,                              @job_id=N'00000000-0000-0000-0000-000000000000'GO

USE [msdb]
GO 
/****** Object:  Alert [Login failed : State 5]    Script Date: 24/03/2020 12:50:49 ******/
EXEC msdb.dbo.sp_add_alert @name=N'Login failed : State 5',                               @message_id=0,                               @severity=0,                               @enabled=1,                               @delay_between_responses=0,                               @include_event_description_in=5,                              @category_name=N'[Uncategorized]',                              @wmi_namespace=N'\\.\root\Microsoft\SqlServer\ServerEvents\MSSQLSERVER',                              @wmi_query=N'SELECT * FROM AUDIT_LOGIN_FAILED WHERE State = 1 OR State=  5 OR State =  8 OR State= 14 OR State=  38',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO USE [msdb]GO /****** Object:  Alert [Error 823]    Script Date: 24/03/2020 12:51:12 ******/EXEC msdb.dbo.sp_add_alert @name=N'Error 823',                               @message_id=823,                               @severity=0,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'
GO  
USE [msdb]
GO 
/****** Object:  Alert [Error 824]    Script Date: 24/03/2020 12:51:33 ******/
EXEC msdb.dbo.sp_add_alert @name=N'Error 824',                               @message_id=824,                               @severity=0,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Error 825]    Script Date: 24/03/2020 12:51:51 ******/EXEC msdb.dbo.sp_add_alert @name=N'Error 825',                               @message_id=825,                               @severity=0,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 016]    Script Date: 24/03/2020 12:52:05 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 016',                               @message_id=0,                               @severity=16,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'
GO  
USE [msdb]
GO 
/****** Object:  Alert [Severity 017]    Script Date: 24/03/2020 12:52:14 ******/
EXEC msdb.dbo.sp_add_alert @name=N'Severity 017',                               @message_id=0,                               @severity=17,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 018]    Script Date: 24/03/2020 12:52:22 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 018',                               @message_id=0,                               @severity=18,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 019]    Script Date: 24/03/2020 12:52:34 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 019',                               @message_id=0,                               @severity=19,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'
GO  
USE [msdb]
GO 
/****** Object:  Alert [Severity 020]    Script Date: 24/03/2020 12:52:44 ******/
EXEC msdb.dbo.sp_add_alert @name=N'Severity 020',                               @message_id=0,                               @severity=20,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 021]    Script Date: 24/03/2020 12:52:52 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 021',                               @message_id=0,                               @severity=21,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO   USE [msdb]GO /****** Object:  Alert [Severity 022]    Script Date: 24/03/2020 12:53:02 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 022',                               @message_id=0,                               @severity=22,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'
GO  
USE [msdb]
GO 
/****** Object:  Alert [Severity 023]    Script Date: 24/03/2020 12:53:23 ******/
EXEC msdb.dbo.sp_add_alert @name=N'Severity 023',                               @message_id=0,                               @severity=23,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 024]    Script Date: 24/03/2020 12:53:32 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 024',                               @message_id=0,                               @severity=24,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'GO  USE [msdb]GO /****** Object:  Alert [Severity 025]    Script Date: 24/03/2020 12:53:52 ******/EXEC msdb.dbo.sp_add_alert @name=N'Severity 025',                               @message_id=0,                               @severity=25,                               @enabled=1,                               @delay_between_responses=60,                               @include_event_description_in=1,                              @category_name=N'[Uncategorized]',                              @job_id=N'00000000-0000-0000-0000-000000000000'
GO 

Powershell issue with output characters changing to ?

Published / by Chris Smith / Leave a Comment

On PowerShell command pipe to export CSV add Encoding Unicode if characters change to ?
Credit: Idera

execute xp_cmdshell 'powershell -Command "Get-ChildItem -File -Path @path -filter *.* -Recurse | ForEach-Object {[PsCustomObject]@{Fullname = "$($_.FullName)"; LastWriteTime = "$($_.LastWriteTime)"}} | Sort Fullname -Descending | Export-Csv -Path @CSVFile -NoTypeInformation -Encoding unicode"'

If using BULK INSERT ensure widechar is used (It will work but TSQL will add notices which may add pointless text to log files)

BULK INSERT #bcp_file_mem
                FROM @CSVFile
                                WITH (FORMAT = 'CSV', DATAFILETYPE = 'widechar', FIELDQUOTE = '"', FIRSTROW = 2, FIELDTERMINATOR = ',', ROWTERMINATOR = '\n', TABLOCK);

Sage EDI IN Intermediary Email

Published / by Chris Smith / Leave a Comment

I have found users don’t always read the logs from the Adaptus EDI Module anymore in Sage Line 500.
There are a few reasons why orders get stuck, the most common is a customer has sent a duplicate EDI file although one was caused by incorrect product supersession setup causing an endless loop.

USE <Affected Sage Company MSSQL Database>
GO
CREATE procedure [dbo].[usp_email_edi_held_orders]
AS
-- =============================================
-- Author:		Christopher Smith
-- Create date: 01/08/2017
-- Description:	The procedure looks for orders held in the EDI IN Intermediary Tables
-- Changes:
-- 1.0.1 - 02/08/2017 - Added dateformat dmy as when run was reporting "Conversion failed when converting date and/or time from character string.".  Ensured all date values were cast as datetime as well.
-- 1.0.2 - 11/04/2019 - Corrected the Sales Order lookup to check the invoice customer instead of delivery customer on opheadm
-- =============================================
-- exec [dbo].[usp_email_edi_held_orders]
-- =============================================
--------------------------------------------------------------------------------------------------------------------------
print 'Change to DMY format'
--------------------------------------------------------------------------------------------------------------------------
SET DATEFORMAT dmy;
--------------------------------------------------------------------------------------------------------------------------
print 'Populate temp table'
--------------------------------------------------------------------------------------------------------------------------
Create Table #results(
	customer VARCHAR(8),
	existingorderno varchar(10),
	received_date char(8),
	customer_order VARCHAR(20),
	"filename" char(30),
	warehouse char(2),
	product char(20),
	long_description char(40),
	required_quantity int
	)
insert into #results (customer,received_date,customer_order,"filename",warehouse,product,long_description,required_quantity) 
	SELECT customer,received_date,customer_order,"filename",warehouse,product,long_description,required_quantity
	FROM "scheme"."eiorhdm" t1
		INNER JOIN "scheme"."eiordtm" t2
			ON t1.key_code = LEFT(t2.key_code,LEN(RTRIM(t1.key_code)))
	WHERE t1.update_status = 'h' and CAST(received_date as datetime) BETWEEN CAST(GETDATE() -7 as datetime) and CAST(GETDATE() as datetime)
--------------------------------------------------------------------------------------------------------------------------
print 'Check if need to send email'
--------------------------------------------------------------------------------------------------------------------------
if (select count(*) AS nos from #results) = 0 
begin
print 'No Email to Send'
goto END1
end 
--------------------------------------------------------------------------------------------------------------------------
print 'Check if existing SOP number exists and update table'
--------------------------------------------------------------------------------------------------------------------------
update t1
	set t1.existingorderno = t2.order_no
		from #results t1
			INNER JOIN scheme.opheadm(nolock) t2
				ON t1.customer_order = t2.customer_order_no and t1.customer = t2.invoice_customer
update #results
	set existingorderno = 'NOMATCH'
		from #results
		where existingorderno IS NULL
--------------------------------------------------------------------------------------------------------------------------
print 'Create HTML for Email'
--------------------------------------------------------------------------------------------------------------------------

DECLARE @tableHTML  NVARCHAR(MAX) ;

SET @tableHTML =
    N'<H1>Orders held in the EDI IN Intermediary Tables</H1>' +
	N'<font size="5" face="arial" color="red"> <H2>Please review orders held in the EDI IN module, if the order held does not match an existing order further checks may need to be done</H2></font>' +
    N'<table border="1">' +
    N'<tr><th>Customer No</th><th>Existing SO No</th>' +
    N'<th>Customer Order No</th><th>EDI Filename</th><th>Received Date</th>' +
	N'<th>WH</th><th>Product</th>' +
	N'<th>Description</th><th>Qty</th>' +
    CAST ( ( select td =   customer,'',
	td = existingorderno,'',
	td = customer_order,'',
	td = "filename",'',
	td = received_date,'',
	td = warehouse,'',
	td = product,'',
	td = long_description,'',
	td = required_quantity
from #results
order by customer, customer_order
FOR XML PATH('tr'), TYPE ) AS NVARCHAR(MAX) ) + 
N'</table>'  + 
N'<p> </p>' + 
N'<p> </p>' + 
N'<p>Regards.</p>' + 
N'<p> </p>' +
N'<p>usp_email_edi_held_orders</p>' 
--------------------------------------------------------------------------------------------------------------------------
print 'Send Email'
--------------------------------------------------------------------------------------------------------------------------
    EXEC msdb.dbo.sp_send_dbmail
    @profile_name = '<SQL Email Profile>',
	@recipients = '<Recipients>',
    @copy_recipients = '<Copy_Address>',
	@body = @tableHTML,
    @body_format = 'HTML',
	@subject = 'Orders held in the EDI IN Intermediary Tables'
END1:

Use T-Sql to find triggers on specific database

Published / by Chris Smith / Leave a Comment

Credit: Joe Stefanelli on Stack Overflow

I wanted a simple way to find triggers on a specific database, Joe Stefanelli posted SQL which did exactly what I wanted.

SELECT 
     sysobjects.name AS trigger_name 
    ,USER_NAME(sysobjects.uid) AS trigger_owner 
    ,s.name AS table_schema 
    ,OBJECT_NAME(parent_obj) AS table_name 
    ,OBJECTPROPERTY( id, 'ExecIsUpdateTrigger') AS isupdate 
    ,OBJECTPROPERTY( id, 'ExecIsDeleteTrigger') AS isdelete 
    ,OBJECTPROPERTY( id, 'ExecIsInsertTrigger') AS isinsert 
    ,OBJECTPROPERTY( id, 'ExecIsAfterTrigger') AS isafter 
    ,OBJECTPROPERTY( id, 'ExecIsInsteadOfTrigger') AS isinsteadof 
    ,OBJECTPROPERTY(id, 'ExecIsTriggerDisabled') AS [disabled] 
FROM sysobjects
INNER JOIN sysusers 
    ON sysobjects.uid = sysusers.uid
INNER JOIN sys.tables t 
    ON sysobjects.parent_obj = t.object_id
INNER JOIN sys.schemas s 
    ON t.schema_id = s.schema_id
WHERE sysobjects.type = 'TR'

Find unused MSSQL indexs on a database

Published / by Chris Smith / Leave a Comment

Credit: Basit’s SQL Server Tips

The following is very handy to check for any unused index’s on a MSSQL database, this should only be run once the database has been up for at least a week or when all scheculed jobs have been allowed to run at least once. The only change I made is to display the table and index name.

SELECT o.name,i.name,u.*
FROM [sys].[indexes] i
INNER JOIN [sys].[objects] o ON (i.OBJECT_ID = o.OBJECT_ID)
LEFT JOIN [sys].[dm_db_index_usage_stats] u ON (i.OBJECT_ID = u.OBJECT_ID)
    AND i.[index_id] = u.[index_id]
    AND u.[database_id] = DB_ID() --returning the database ID of the current database
WHERE o.[type] <> 'S' --shouldn't be a system base table
    AND i.[type_desc] <> 'HEAP'
    AND i.[name] NOT LIKE 'PK_%'
    AND u.[user_seeks] + u.[user_scans] + u.[user_lookups] = 0
    AND u.[last_system_scan] IS NOT NULL
ORDER BY 1 ASC

SQL Server 2008 – Find which stored procedure is part of a scheduled

Published / by Chris Smith / Leave a Comment

Find a stored procedure (or any text) inside a scheduled on on SQL Server 2008 and 2012 by querying the sysjobs and sysjobsteps system tables.
The following provides the name of the job and the step name. It uses the standard % wildcard so replace myjob in “%myjob%” below

SELECT name, "description", "enabled", database_name,step_name,step_id,command
       FROM "msdb"."dbo"."sysjobsteps" t1
              INNER JOIN "msdb"."dbo"."sysjobs" t2 on t1.job_id = t2.job_id
       where t1."command" LIKE '%myjob%'

More info on SQL system tables here:
MS Docs
MS Docs

Maintain MSDB email tables

Published / by Chris Smith / Leave a Comment

Credits:
SQL Server Central
SQL Authority
Microsoft Docs
Microsoft Docs

We recently had an issue where MSDB was growing very quickly, it turned out it was due to a huge increase in data we was directly emailing to users.
We now run a weekly job just to maintain the MSDB email tables as whole copies of emails, attachments and logs are kept. I first run a stored procedure designed to maintain the tables and then directly delete from the tables (I would expect these to delete nothing).

use msdb
go
declare @DeleteToDate datetime
--I have set this to keep 60 days
set @DeleteToDate = DATEADD(d, -60, getdate())
--Print the date for logs
print @DeleteToDate
print 'Using inbuilt script to maintain tables'
EXECUTE msdb.dbo.sysmail_delete_mailitems_sp @sent_before = @DeleteToDate
--Line below is SQL Server 2008 only
--EXECUTE msdb.dbo.Sysmail_delete_log_sp @logged_before = @DeleteToDate
--maintain sysmail_allitems
print 'maintain sysmail_allitems'
DELETE FROM msdb.dbo.sysmail_allitems
   where sent_date < @DeleteToDate
--maintain log file
print 'maintain sysmail_log'
DELETE FROM msdb.dbo.sysmail_log
   where log_date < @DeleteToDate
--remove old emails
print 'maintain sysmail_mailitems'
delete from msdb.dbo.sysmail_mailitems
   where sent_date < @DeleteToDate
--remove retries (Hopefully this shouldn;'t be to high a number)
print 'maintain sysmail_send_retries'
delete from msdb.dbo.sysmail_send_retries
       where last_send_attempt_date < @DeleteToDate
--remove attachments (If you are removing the email no point in keeping the attachments)
print 'maintain sysmail_attachments'
DELETE FROM dbo.sysmail_attachments
       WHERE last_mod_date < @DeleteToDate

Find processes which have been using tempdb

Published / by Chris Smith / Leave a Comment

Credit: Microsoft TechNet
On some occasions a few MS SQL databases have exhausted tempdb space, this was due to users leaving computers running for weeks on end and the program using different isolation levels or open transactions.
I use the following in SQL in see which process ID have been running the longest, as its a select statement it can of course be amended to your requirements (Adding an order by clause or just selecting the columns you require).

SELECT *
FROM sys.dm_tran_active_snapshot_database_transactions 
ORDER BY elapsed_time_seconds DESC;

You should see something like the following, rather than kill of the transaction if it is a user try and educate the user to come out of transactions when finished.

Add a user to DatabaseMailUserRole on msdb

Published / by Chris Smith / Leave a Comment

I had a user unable to use a script which generated an email, I found the user didn’t have permission, Microsoft state:
To reduce the surface area of SQL Server, Database Mail stored procedures are disabled by default.
To send Database Mail, you must be a member of the DatabaseMailUserRole database role in the msdb database.

You can add the user manually by:
Opening SQL Server Management Studio
Connecting to the server in question
Expand Databases
Expand System Databases
Expand msdb
Expand Security
Expand Users, right click the user or group and click properties
If the user is not in the list right click users and add the user or group
On the Membership tab ensure the user or group is a member of DatabaseMailUserRole

If you are lazy use a script like the one below (It will return a 1 if SQL can’t find the user or the user already has access, 0 means it was successful)

EXEC msdb.sys.sp_addrolemember @rolename = 'DatabaseMailUserRole', @membername = '<AD_or_SQL_Username>'

Source: Microsoft TechNet
Microsoft MSDN

Notes:
I found using Windows Active Directory Groups required an extra step, by default all users have access to the Public Profile but not groups on SQL Mail.
Groups need to be added via “Manage Profile Security” –> “Private Profiles” tab.
As the group would be added select the AD group from the drop down box.
Ensure Access is ticked and click Next and OK.
As I have only done this the once I haven’t looked up if this can be done directly in T-SQL.