SSIS 2012 : work around for missing milliseconds in SSIS datetime variable+ Solution for incremental load using datetime column

In many cases for incremental load we will consider datetime column . when we want to implement incremental load based on datetime column in SSIS we should be very careful because the milliseconds will be truncated in SSIS datetime variable . In This case we will face issue if the data is in milliseconds also .


Workaround  :  To avoid this milliseconds issue  create variable sof string instead of Datetime and load the value using the fallowing query

select CONVERT(VARCHAR(23), max(LastUpdated), 121) as MaxLastUpdated from [Schema].[OPS_CDR_Scrapper_Data]


and

use that varible in  OLEDB source  like below


select FileName, Folder, LastUpdated from scraper_log_file_details
where LastUpdated > CONVERT(DATETIME, ?, 121)

and map the parameter .

Comments

  1. AmyJuly 29, 2015 at 9:54 AM

    You just saved me a lot of rework! Thanks!!!

    ReplyDelete
  2. gaurav kumar@nitkMay 25, 2016 at 9:50 AM

    Thank you, but I am getting very very slow performance. Are you facing same ?

    ReplyDelete
    Replies
    1. Surendra ThotaMay 25, 2016 at 12:48 PM

      Hi Gaurav, we are not facing any performance issue , do you have index on datetime column ?

      Delete
  3. UnknownAugust 23, 2016 at 12:36 PM

    Very helpful! Helped me resolve my problem in 5 minutes. Thank you.

    ReplyDelete
  4. UnknownMarch 9, 2018 at 7:32 AM

    Excellent! Thanks!

    ReplyDelete

Post a Comment

Popular posts from this blog

SSAS : A connection cannot be made to redirector. Ensure that 'SQL Browser' service is running. (Microsoft.AnalysisServices.AdomdClient). + Solution

Problem : have installed Analysis service on my server machine through remote login from my service account. I am able to open Analysis services from there. When I open SSMS from the same service account on my local machine, I am not able to connect to that Analysis service. It is giving me error by saying : "A connection cannot be made to redirector. Ensure that 'SQL Browser' service is running. (Microsoft.AnalysisServices.AdomdClient). A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connection host has failed to respond 10.245.192.111:2382(System)." Solution : Based on my research, the issue was caused by the Browser Service account was changed to the domain account. In your scenario, you can try to connect to the Analysis Service using <Servername>:2383 instead of instance name. Or you can change the Browser Service account to "Local System".
Read more

SSIS Error : Unable to infer the XSD from the XML file. The XML contains multiple namespaces

Image
Problem : When we have Multiple Namespaces in the Xml Source which we need to load the Data from   as we shown below Then we will face a issue in generating XSD for that XML Error : Unable to infer the XSD from the XML file. The XML contains multiple namespaces. (Microsoft Visual Studio) Solution : First we need to avoid multiple namespaces to transform your source file to a format that doesn’t refer to the namespaces. SSIS has an XML task that can do the transformation. 1) Add the XML task to an SSIS   Control Flow and edit it. 2) Change the OperationType property value to XSLT, the SourceType to File connection and the Source to your source file that has the problem. 3) Set the SaveOperationResult property to True and expand the OperationResult branch. Set DestinationType to File Connection and the Destination to a new XML file ( ModifiedInput.XML). 4) Add the below script to a new file and save it with an xslt file extension (Input.XSLT) <
Read more