In this post I am going to explain the steps we should follow to create an SSIS package that run’s every night. The package will ensure already imported records are updated and new records are created in the destination system.
For demonstration I am assuming a source system, say CRM exists and it has a view as shown below
The job we are going to create will import this data into destination system, say Employee table as shown below
Now open SQL Business Intelligence Studio. Create new project and select “Integration Services Project” as shown below
From toolbox, drop a “Data Flow Task”
Double click to go inside the Data Flow Task and add a new OLE DB source control
Double click OLE DB Source control.
First we need to setup a new connection. For that click on the New button, enter the source database information. Once completed, select the connection. And set other properties as shown below
Next step is to add a Lookup transformation. This will help us to identify whether the record exists in the destination or not.
Before that we should add a new connection to the destination.
Now we can setup the lookup transformation as shown below
Lookup transformation by default fail if no match is found. In our case for new records this scenario will come up. To ensure new records pass through the pipeline we should set the Error Output properly.
Next, add a conditional split control
Drop the data flow arrow into conditional split control, which brings up following dialog. Select “Lookup Match Output”
Double click Conditional Split control and set following values
Now we should add the ole db destination and a ole db command controls. To “OLE DB Destination” we should select the input “New_Records” from Conditional Split. To “OLE DB Command” we should select the input “Modified_Records” from conditional split.
Make following changes to OLE DB Destination
Make following changes to OLE DB Command control
Now the package is ready for testing, hit F5 to run the package
In my next post I will show how to schedule this package to run every night.
I use following query to generate a set of “SELECT” statement which can later be executed to search for a given string in the database
select 'select distinct ''' + tab.name + '.' + col.name + ''' from [' + tab.name + '] where [' + col.name + '] like ''%TEXT_TO_SEARCH%'' union ' from sys.tables tab join sys.columns col on (tab.object_id = col.object_id) join sys.types types on (col.system_type_id = types.system_type_id) where tab.type_desc ='USER_TABLE' and types.name IN ('CHAR', 'NCHAR', 'VARCHAR', 'NVARCHAR');
Replace TEXT_TO_SEARCH with the term you are looking for. You can further enhance this query to search for any type other than string.
Sometimes we might want to create comma separated list of values from a SQL table column. This can be easily achieved using below query
DECLARE @listStr VARCHAR(MAX) SELECT @listStr = COALESCE(@listStr+''',''' ,'') + ColumnName FROM TableName SELECT @listStr
Query which returns all Triggers in a database
USE YourDBName GO SELECT trigger_name = sysobjects.name, trigger_owner = 'x', table_schema = s.name, table_name = OBJECT_NAME(parent_obj), isupdate = OBJECTPROPERTY( id, 'ExecIsUpdateTrigger'), isdelete = OBJECTPROPERTY( id, 'ExecIsDeleteTrigger'), isinsert = OBJECTPROPERTY( id, 'ExecIsInsertTrigger'), isafter = OBJECTPROPERTY( id, 'ExecIsAfterTrigger'), isinsteadof = OBJECTPROPERTY( id, 'ExecIsInsteadOfTrigger'), [disabled] = OBJECTPROPERTY(id, 'ExecIsTriggerDisabled') FROM sysobjects INNER JOIN sys.tables t ON sysobjects.parent_obj = t.object_id INNER JOIN sys.schemas s ON t.schema_id = s.schema_id WHERE sysobjects.type = 'TR'
Query which returns all tables in a particular database
USE YourDBName GO SELECT * FROM sys.Tables
Following query uses Information schema
SELECT * FROM DatabaseName.information_schema.routines WHERE routine_type = 'PROCEDURE'
On one of my recent project I got a requirement to proper case a SQL table column values. The data got into this table is through an SSIS package from a 3rd party database. To tackle this I created a user defined function
CREATE FUNCTION ProperCase(@inpStr nvarchar(8000)) RETURNS nvarchar(8000) AS BEGIN DECLARE @Result nvarchar(8000) SET @inpStr = LOWER(@inpStr) + ' ' SET @Result = '' WHILE 1=1 BEGIN IF PATINDEX('% %',@inpStr) = 0 BREAK SET @Result = @Result + UPPER(Left(@inpStr,1))+ SubString (@inpStr,2,CharIndex(' ',@inpStr)-1) SET @inpStr = SubString(@inpStr, CharIndex(' ',@inpStr)+1,Len(@inpStr)) END SET @Result = Left(@Result,Len(@Result)) RETURN @Result END