Jason Niver's Blog

SQL Server Index Maintenance

2008-11-04T23:09:17Z

After working on a problem where the transaction logs for a large database were filling to sizes larger then the actual database every night it turned out to be a server maintenance plan that rebuilt and reorganized indexes every night. While working on designing a custom plan that would only rebuild or reorganize when actually necessary I found many references out on the internet. None of the scripts I found out there really accomplished what I was after.

Starting with finding the indexes that need maintenance done on them I eventually came up with this SQL:

select

    sch.name [schema_name]

    ,obj.name [table_name]

    ,idx.name [index_name]

    ,idx.type_desc [index_type]

    ,part.used_page_count*8 [used_kb] --each page is 8k

    ,part.row_count

    ,CONVERT(DECIMAL(9,2),frag.avg_fragmentation_in_percent) [fragmentation]

    from sys.dm_db_partition_stats part

        join sys.objects obj on part.object_id = obj.object_id

        join sys.schemas sch on obj.schema_id = sch.schema_id

        left join sys.indexes idx on part.object_id = idx.object_id and part.index_id = idx.index_id

        left join sys.dm_db_index_physical_stats(db_id(),null,null,null,null) frag on part.object_id = frag.object_id

            and part.index_id = frag.index_id

    where

        obj.is_ms_shipped = 0

        and frag.page_count > 100

    order by [schema_name],[table_name],[index_name];

This gave me a nice view to see how big the tables were, and where the fragmentation problems were. We decided that if it's less800KB (based on frag.page_count) we didn't care about the fragmentation.

Armed with this info it was a matter of basically executing the rebuild or reorganize depending on your preference. I decided to use a in memory table and just did a insert-select into it. My temp table looked like:

declare @IndexStatus table(

    schema_name varchar(250)

    ,table_name varchar(250)

    ,index_name varchar(250)

    ,index_type varchar(250)

    ,used_kb int

    ,row_count int

    ,fragmentation decimal(9,2));

From there I just took the results into a cursor and executed dynamic sql. My logic was <10% fragmentation I didn't care, greater then 30 gets a rebuild, 10-30 gets a reorganize

declare maintCursor cursor for

    select 'alter index ['+[index_name]+'] on ['+[schema_name]+'].['+[table_name]+']', fragmentation

        from @IndexStatus

        where fragmentation > 10;

declare @stmt varchar(max);

declare @frag float;

open maintCursor

fetch next from maintCursor into @stmt, @frag

while @@fetch_status = 0

begin

    if @frag > 30

        set @stmt = @stmt+' rebuild'; --if this is enterprise you can add 'with (online=on)'

    else

        set @stmt = @stmt+' reorganize';

    print (@stmt)

    exec (@stmt)

    fetch next from maintCursor into @stmt, @frag

end

close maintCursor

deallocate maintCursor

Sending Encrypted E-Mails in C#

2008-08-19T01:14:26Z

So I was faced with the problem of sending an encrypted email to a group of people. Not really thinking, I responded with sure we can do that no problem. Getting back to my desk I started working on the project to find out that it turns out to be more difficult then I had thought.

Doing some internet searches, I found several commercial products that allow you to encrypt emails. I also found several discussions where people got close to what they were after, most were just after digital signatures without attachments. Nothing close to a full package.

After a few hours worth of digging I finally was able to come up with a solution that met my needs, multiple addressees and multiple attachments. Below is the code, note that you need to add a reference to System.Security in your Visual Studio Project to compile this code.

using System;

using System.Text;

using System.Net.Mail;

using System.IO;

using System.Security.Cryptography.Pkcs;

using System.Security.Cryptography.X509Certificates;

namespace CommonUtilities

    //requires reference to System.Security

    class EmailUtil

        public static void SendEncryptedEmail(string[] to, string from, string subject, string body, string[] attachments)

            MailMessage message = new MailMessage();

            message.From = new MailAddress(from);

            message.Subject = subject;

            if (attachments != null && attachments.Length > 0)

                StringBuilder buffer = new StringBuilder();

                buffer.Append("MIME-Version: 1.0\r\n");

                buffer.Append("Content-Type: multipart/mixed; boundary=unique-boundary-1\r\n");

                buffer.Append("\r\n");

                buffer.Append("This is a multi-part message in MIME format.\r\n");

                buffer.Append("--unique-boundary-1\r\n");

                buffer.Append("Content-Type: text/plain\r\n");  //could use text/html as well here if you want a html message

                buffer.Append("Content-Transfer-Encoding: 7Bit\r\n\r\n");

                buffer.Append(body);

                if (!body.EndsWith("\r\n"))

                    buffer.Append("\r\n");

                buffer.Append("\r\n\r\n");

                foreach (string filename in attachments)

                    FileInfo fileInfo = new FileInfo(filename);

                    buffer.Append("--unique-boundary-1\r\n");

                    buffer.Append("Content-Type: application/octet-stream; file=" + fileInfo.Name + "\r\n");

                    buffer.Append("Content-Transfer-Encoding: base64\r\n");

                    buffer.Append("Content-Disposition: attachment; filename=" + fileInfo.Name + "\r\n");

                    buffer.Append("\r\n");

                    byte[] binaryData = File.ReadAllBytes(filename);

                    string base64Value = Convert.ToBase64String(binaryData, 0, binaryData.Length);

                    int position = 0;

                    while (position < base64Value.Length)

                        int chunkSize = 100;

                        if (base64Value.Length - (position + chunkSize) < 0)

                            chunkSize = base64Value.Length - position;

                        buffer.Append(base64Value.Substring(position, chunkSize));

                        buffer.Append("\r\n");

                        position += chunkSize;

                    buffer.Append("\r\n");

                body = buffer.ToString();

            else

                body = "Content-Type: text/plain\r\nContent-Transfer-Encoding: 7Bit\r\n\r\n" + body;

            byte[] messageData = Encoding.ASCII.GetBytes(body);

            ContentInfo content = new ContentInfo(messageData);

            EnvelopedCms envelopedCms = new EnvelopedCms(content);

            CmsRecipientCollection toCollection = new CmsRecipientCollection();

            foreach (string address in to)

                message.To.Add(new MailAddress(address));

                X509Certificate2 certificate = null; //Need to load from store or from file the client's cert

                CmsRecipient recipient = new CmsRecipient(SubjectIdentifierType.SubjectKeyIdentifier, certificate);

                toCollection.Add(recipient);

            envelopedCms.Encrypt(toCollection);

            byte[] encryptedBytes = envelopedCms.Encode();

            //add digital signature:

            SignedCms signedCms = new SignedCms(new ContentInfo(encryptedBytes));

            X509Certificate2 signerCertificate = null; //Need to load from store or from file the signer's cert

            CmsSigner signer = new CmsSigner(SubjectIdentifierType.SubjectKeyIdentifier, signerCertificate);

            signedCms.ComputeSignature(signer);

            encryptedBytes = signedCms.Encode();

            //end digital signature section

            MemoryStream stream = new MemoryStream(encryptedBytes);

            AlternateView view = new AlternateView(stream, "application/pkcs7-mime; smime-type=signed-data;name=smime.p7m");

            message.AlternateViews.Add(view);

            SmtpClient client = new SmtpClient("your.smtp.mailhost");

            //add authentication info if required by your smtp server etc...

            //client.Credentials = CredentialCache.DefaultCredentials;

            client.Send(message);

This should get you pretty much everything you need. The only thing left is to load the certificates from somewhere. I used a function to load them from the current users certificate store on the machine. I have a previous blog posting on how to load the certificates from the store. The trick comes in to fetching the certificates for unknown parties etc.

Using certificates from the Windows certificate store

2008-07-16T21:15:00Z

I recently had to setup some web services that used certificates to communicate back and forth and one thing I found is that pretty much every site I found references on was using a file on the file system to access the client certificate. What we wanted to do was access the certificates directly from the store. After a little playing around we got it to work, so I though I would put this out in case someone else wants to do the same. There is another X509Certificate2Collection item at MSDN, however the one for the X509Certificate2 only shows getting the file.

Assuming:
using System.Security.Cryptography.X509Certificates;

private X509Certificate GetCertificate()
{
    X509Store store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
    store.Open(OpenFlags.ReadOnly);
    X509Certificate2Collection certs = store.Certificates.Find(X509FindType.FindBySerialNumber, "123456", true);
    store.Close();
    return certs[0];
}

There are several ways to do the search, I like the FindBySerialNumber, but you can also search by DN, SubjectName, etc.

Downloading files from the internet in PowerShell (with progress)

2008-06-13T16:42:00Z

Ok, so the easiest way (that I know of) to download files in powershell from the internet is to use the .net WebClient. The simple way I started with was the two liner:

$client = New-Object "System.Net.WebClient"
$client.DownloadFile("http://somesite.com/largefile.zip","c:\temp\largefile.zip"

However I was working on a script that required some pretty large files to be downloaded, and using the DownloadFile method has no progress indicator. I figured some users might thing the program died. So I decided to create a method that still uses the webclient to download the files, however give a status of where it is in the download. Here is what I came up with, hopefully someone else will find it useful.

function downloadFile($url, $targetFile)
{
    "Downloading $url"
    $uri = New-Object "System.Uri" "$url"
    $request = [System.Net.HttpWebRequest]::Create($uri)
    $request.set_Timeout(15000) #15 second timeout
    $response = $request.GetResponse()
    $totalLength = [System.Math]::Floor($response.get_ContentLength()/1024)
    $responseStream = $response.GetResponseStream()
    $targetStream = New-Object -TypeName System.IO.FileStream -ArgumentList $targetFile, Create
    $buffer = new-object byte[] 10KB
    $count = $responseStream.Read($buffer,0,$buffer.length)
    $downloadedBytes = $count
    while ($count -gt 0)
    {
        [System.Console]::CursorLeft = 0
        [System.Console]::Write("Downloaded {0}K of {1}K", [System.Math]::Floor($downloadedBytes/1024), $totalLength)
        $targetStream.Write($buffer, 0, $count)
        $count = $responseStream.Read($buffer,0,$buffer.length)
        $downloadedBytes = $downloadedBytes + $count
    }
    "`nFinished Download"
    $targetStream.Flush()
    $targetStream.Close()
    $targetStream.Dispose()
    $responseStream.Dispose()
}

This would be used like
downloadFile "http://somesite/largefile.zip" "c:\temp\largefile.zip"

Importing a File Share into Document Library via PowerShell

2008-05-22T04:24:53Z

So today I was asked for a way to automate importing files from a file share into a document library. For this, ACL's were not an issue, however using PS's Get-ACL function can get you that as well to match the insert.

Here is what I came up with as a quick and dirty response. I though it might be nice to post for other people to use as a base script for things like synchronizing file shares with document libraries etc.

     #Variables

    $targetLibrary = "Imported Documents"

    $sourceDirectory = "\\somemachine\someshare"

    $siteUrl = "http://localhost/sites/somesite"

    #open the SPWeb/SPSite

    [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")

    $site = New-Object "Microsoft.SharePoint.SPSite" $siteUrl

    $web = $site.OpenWeb()

    #get all the files and directories on the share and iterate them

    foreach ($file in get-childitem $sourceDirectory -recurse)

    {

        $fileName = ($file | Select-Object Name).Name    #get the base file nname

        $parentPath = ($file | Select-Object PSParentPath) #get the path

        #eliminate everything in the path prior to the root folder

        $parentPath = $parentPath.Substring($parentPath.PSParentPath.ToLower().IndexOf($sourceDirectory.ToLower())+$sourceDirectory.Length)

        $parentPath = $parentPath.Replace("\","/")    #make the path url friendly

        $targetPath = "/$targetLibrary$parentPath/$fileName" #combine for a full url

        #check if this is a directory or file

        if (($file.GetType() | Select-Object Name).Name -eq "DirectoryInfo")

        {

            "Adding Folder: $targetPath"

            $web.get_Folders().Add($targetPath)    

        }

        else

        {

            $parentFile = ($file | Select-Object FullName).FullName #gets the full path to the file

            "Adding File: $targetPath"

            $fileBytes = [System.IO.File]::OpenRead("$parentFile") #read the file to a byte arry

            $web.get_Files().Add($targetPath,$fileBytes)

        }

    }

    #dispose of the web/site

    $web.Dispose()

    $site.Dispose()

Web Parts the Simple Way - Use User Controls

2008-05-13T16:39:00Z

One of the first questions I usually get about SharePoint development is how to make the WebPart development process simpler. My answer to that is use UserControls. There is a very good solution on CodePlex called Smart Parts http://www.codeplex.com/smartpart However I have found in some environments the customer does not want to add a third party product into the mix. In these cases the fastest way I have found to create webparts is to use user controls and wrap them with the webpart.

We start with a very simple WebPart class that only overrides the CreateChildControls method, we use this to load our user control.

public class InboxListWebPart: WebPart

{

private Control control = null;

protected override void CreateChildControls()

{

Controls.Clear();

control = this.Page.LoadControl("/_controltemplates/MyCustomControls/InboxListControl.ascx");

Controls.Add(control);

}

From there everything is driven from my usercontrol InboxListControl.ascx. The control goes into the program files\common files\microsoft shared\web server extensions\12\templates\controltemplates directory (as shown above I created a sub directory called MyCustomControls). I usually will deploy the code behind into the GAC, but they can be deployed into the same directory as well.

If you need to do more advanced webpart stuff (like connections or properties) you can get to your webpart from your user control. For example if I had a property that I wanted to present to the user (say like Inbox Name) in my .cs file for the webpart I would do the following (using the 3.5 property syntax):

[Personalizable(PersonalizationScope.User), WebBrowsable(true), WebDisplayName("Inbox Name"), WebDescription("some useful description")]

public string InboxName

{

get;

set;

}

In my user control I usually define a parent webpart property that returns the webpart I used to wrap my user control:

private InboxListWebPart ParentWebPart

{

get

{

return this.Parent as InboxListWebPart;

}

From here I can now define my InboxName on my user control:

public String InboxName

{

get

{

return ParentWebPart.InboxName;

}

set

{

ParentWebPart.InboxName = value;

}

It takes a little more code when you want to do things like properties and connections, but I think the usability of having your UI in an ascx file more then makes up for this small inconvenience.