A SQL Server Blog

  • Extending SSIS webcast later today

    At 1PM PST today, I'll be doing an MSDN Webcast on how to extend Integration Services including how to write new tasks and create new components. Why would you want to do that? Well, the simplest example would be if you have binary files that only you know how to read or have other existing business logic, you can integrate that with the product fairly easily.

    In our presentations, we've been talking about the fact that Integration Services is a change in mindset from DTS in that while DTS was a useful tool, SSIS is really a platform for enterprise level data integration. This presentation is where the rubber hits the road and we talk about how do you plug into the platform yourself.

    The webcast will be recorded for posterity.

    http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?EventID=1032276860&EventCategory=4&culture=en-US&CountryCode=US

    regards,

    ash

     

  • New Blog Location

    Hi there,

    I'm a member of the SQL Server Integration Services team and previously had a blog here: sqljunkies.

    Last week I ran into issues posting there and thought I'd try out this site instead. Since I work to integrate technologies, getting blogs over from sqljunkies over to this site is a great problem to have. Of course since I'm lazy I want to use an existing tool to do that but haven't been able to find anything as yet.

    The first part of moving data is to get access to it. Since sqljunkies uses CommunityServer as does this site, and CommunityServer supports MetaWebLog so I should be all set, right? Sorry, no cigar. While MSDN has MetaWebLog API enabled, I haven't been able to find the URL for the sqljunkies service so here we are twiddling our thumbs.

    The other way to get data out would be to use RSS or Atom but that only goes back 4 months for sqljunkies. I want to get all the content!

    Hmm.

    That sent me down the path of googling my options and I ran into several blogging tools and technologies.

    Man, I've been out of it! w.bloggar seemed pretty good, and that's what I used for posting the first version of this blog. I was naive and didn't realize the editor was HTML, not wysiwyg. oh well. 

    Well, no answer to the migration as yet. If you have any wisdom to share in this area, let me know. I would like to stay away from scraping HTML contents to get the content out.

    Regards,

    ash

  • SQL: Performance of incremental bulk insert

    Bulk loading into a table that already has records is a common scenario. A lot of folks ask how can they optimize this, and besides the standard 'it depends' answer, there's a BOL topic that gives useful information on what to do depending on how large your new load is relative to how much data you have, and the type of indices on the table.

    Sunil (who worked with my father for a brief time earlier in his career!) and Man have a TechNet article basically answering this question with a lot more rigor than I have seen in the past... the article discusses this topic in good amount of detail and provides experimental findings as well. While the doc was written in Sept. 2004, I just came across it today.

    Check it out!

    http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/incbulkload.mspx

     

     

  • SSIS: Logging in pipeline components

    Ethan from AMB Dataminers had a great question about how to do logging through his pipeline component.
     
    One way to do it is to call the Fire* methods that's on the IDTSComponentMetadata90. However, that does incur the cost of basically firing an event and handling an associated event handler. That's appropriate at times, while what's required at other times is the ability to just log something as quickly as you can. In addition, some situations call for such log entries to be separated from other log entries that other components might raise. I.e. the OnInformation log event can be pretty chatty, and might not be necessarily what you might want to force the user to use and get a plethora of other events they might not be interested in.
     
    So, here's another way to do logging in the components, thanks to Ted and Matt with their help in this investigation!
     
    IDTSComponentMetadata90 also exposes another method (in addition to the Fire* ones) called PostLogMessage.
     
    Here's the steps for this route:

    1. Register your log entry with the runtime environments so that it's available as a log entry in the
    logging dialog for the pipeline task.
    2. Add your log entry!
     
    As an example of doing this, lets add a log entry that will describe the SQL statement used by the ADOSource sample component that gets installed by Setup.
     

    Registering log entries
     
    In the component, I define a couple of consts:
     
    private const string MyLogEntryName = "AdoNet Source Log Entry";
    private const string MyLogEntryDescription = "Log entry from the AdoNet Custom Source";

    and then overrode the 'RegisterLogEntries' method like this:
     
    this.LogEntryInfos.Add(MyLogEntryName, MyLogEntryDescription,
     
    Microsoft.SqlServer.Dts.Runtime.Wrapper.DTSLogEntryFrequency.DTSLEF_CONSISTENT);
     
    DTSLogEntryFrequency provides a hint to the runtime about how frequently will the event be logged:
     
    DTSLEF_OCCASIONAL: only sometimes, not during every execution
    DTSLEF_CONSISTENT: constant number of times during every execution
    DTSLEF_PROPORTIONAL: proportional to amount of work completed
     
    The call above, then, creates a log entry with the name and description we want. I'll be logging once per execution, so am using DTSLEF_CONSISTENT.
     
    This magically means that when I add a new component to the pipeline task, the 'Logging' dialog in the designer shows a new log entry now available for the pipeline task with the name "AdoNet Source Log Entry". Cool!
     
    Logging
     
    Now time to create the log entry. In this case, I just want to log the SQL statement that'll be used, and see that the existing code gets this statement in PreExecute. So, the following go at the end of that function:
     
    DateTime now = DateTime.Now;
    byte[] additionalData = null;
    ComponentMetaData.PostLogMessage(MyLogEntryName, ComponentMetaData.Name, "Command Sent was: " + oledbCommand.CommandText, now, now, 0, ref additionalData);
     

    Testing

    After registering the component, I configure the statement in the source to be 'select * from INFORMATION_SCHEMA.TABLES', ask this log event to go to a file, and get the following:
     
    User:AdoNet Source Log Entry,ASHVINIS-MOBILE,REDMOND\ashvinis,Ado Source CS Sample,{0000004D-0000-0000-0000-000000000000},{9FD94716-EEB8-48A8-A396-79E3CB705A0A},7/13/2005 5:15:38 PM,7/13/2005 5:15:38 PM,0,(null),Command Sent was: select * from INFORMATION_SCHEMA.TABLES
     
    Hope this helps, thanks again, Ethan, for asking such an interesting question!
     
    regards,
    ash
More Posts « Previous page

© 2009 Microsoft Corporation. All rights reserved. Terms of Use  |  Trademarks  |  Privacy Statement
Microsoft
Page view tracker