Paul's MCS Developer Blog

PDF View Components for Windows Store apps (WinRT) (XAML/C#)

paulwhit — Fri, 15 Feb 2013 18:04:58 GMT

There is no native C#/XAML PDF viewing component for Windows Store applications, but there are a growing number of options, and various apps have demonstrated success in different areas of PDF rasterization (converting PDF pages to images) and viewing. Here is a breakdown of all the components that I have seen. These are all expensive, so they might not work well for a hobbyist scenario. But if you have a need in your commercial or LOB application, you may find one of these to be useful.

If you learn of others, please post a reply or send me a tweet @paulwhit. I’m actively monitoring this topic.

Also some developers appear to have ported MuPDF for use in Windows Store applications. I believe this is GPL licensed so you would either need to open source your project or buy a license, so use caution there.

If you have the possibility to rasterize the images using a tool external to your Windows Store application, you can create a decent reader experience from a directory of images. Your users will take a bandwidth hit because downloading the rasterized contents can inflate the download size a lot, but at least you can provide the functionality. That is what I did with Army Field Manuals.

In fact, I used GhostScript with a high resolution raster and then used ImageMagick to do an adaptive resize and convert to grayscale. This gave decent quality although some of the scanned pages are still too large due to scanner ghosting in the page background. I think I need to do something to increase the contrast on those particular documents. Anyway, here’s a rundown of the command line arguments I used with those tools in case you want to do something similar.

gswin64c.exe -q -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 -sDEVICE=png16m -dTextAlphaBits=4 -dGraphicsAlphaBits=4 -r288 -o fm1_0/out_%d.png {pdffilename}

convert {filename} -adaptive-resize 1224x1584 sm_{filename}

mogrify -type grayscale {filename}

Unzipping files with a progress bar in Windows Store applications (XAML/C#)

paulwhit — Fri, 25 Jan 2013 00:30:34 GMT

Here’s a method for unzipping a Zip file in a Windows Store application. ZipArchive is part of System.IO.Compression.

private async Task UnzipWithProgress(StorageFolder outFolder, Stream zipFileStream, ProgressBar pb)
{
    bool bDelete = false;

    using (var archive = new ZipArchive(zipFileStream))
    {
        try
        {
            // Ensure the UI update part runs in the UI thread
            var ignore = this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
            {
                this.tbDownloadStatus.Text = "Unzipping";
                pb.Value = 0;
                pb.Maximum = archive.Entries.Count;
            });

            foreach (ZipArchiveEntry entry in archive.Entries)
            {
                if (entry.Name != "")
                {
                    string fileName = entry.FullName.Replace("/", @"\");
                    StorageFile newFile = await CreateFile(outFolder, fileName);
                    Stream newFileStream = await newFile.OpenStreamForWriteAsync();
                    Stream fileData = entry.Open();
                    byte[] data = new byte[entry.Length];
                    int len = await fileData.ReadAsync(data, 0, data.Length);
                    while (len > 0)
                    {
                        await newFileStream.WriteAsync(data, 0, len);
                        len = await fileData.ReadAsync(data, 0, data.Length);
                    }
                    await newFileStream.FlushAsync();

                    var ignore2 = this.Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
                    {
                        pb.Value++;
                    });
                }
            }

            var itemId = OriginalUriString;
            this.Frame.Navigate(typeof(ItemDetailPage), itemId);
        }
        catch (Exception ex)
        {
            bDelete = true;
        }
        finally
        {
            if (bDelete)
                outFolder.DeleteAsync();
        }
    }
}

To use it, call it this way. In my case I called it from the LoadState method.

StorageFile file;
try
{
    file = await ApplicationData.Current.LocalFolder.GetFileAsync(OriginalUriString);
}
catch (FileNotFoundException)
{
    file = null;
}

if (file != null)
{
    using (var ws = await file.OpenReadAsync())
    using (var forReadRandomStream = ws.AsStreamForRead(0))
    {
        string path = ApplicationData.Current.LocalFolder.Path + @"\" + Path.GetFileNameWithoutExtension(file.Name) + @"\";
        await UnzipWithProgress(await StorageFolder.GetFolderFromPathAsync(path), forReadRandomStream, progressBar1);
    }
}

Apologize that this is not a File->New demo, but I thought it might be useful and Scott Hanselman made me do it.

Office 2013 release day questions

paulwhit — Tue, 17 Jul 2012 02:13:49 GMT

So, being the good dogfooder that I am, I'm posting this using Word 2013 on release day.

Looks like this may be the future of Windows Live Writer (according to the tea leaves). It feels a lot like Writer initially. It can do a lot out of the box but I'll spare you the needless includes from Flickr and other locations. Although I don't see Insert->Code, maybe cut and paste from Visual Studio is formatted well enough.

Even though I'm a Microsoft employee, I haven't been privy to anything with this release—in fact, I don't think I was able to glean anything other than what's been posted online. I'm looking forward to learning more in the next few weeks and months. I'm particularly intrigued by the sync story with Skydrive and the investment in web versions of the apps. Also fascinating to think about is 3^rd party apps integrated with Office 365, Skydrive, and Office Web Apps. Looks like there's an Office store where folks can acquire these add-ons.

Here are a few questions that I have after an hour working with some of the release. There seems to be a cohesive connected-device picture coming together.

Word 2013 Questions

Will the sharing features (such as blog posting) be available in the web app?
Will there be an RSS reader online to compete with Google Reader?
Will real-time collaboration be backward compatible with Office 2010?
How well does the PDF editor work?

Outlook 2013 Questions

I imported my Google Reader RSS feeds OPML file. Now, when I click on RSS Feeds, it goes to my Google Reader. Is this a fluke?
Anything new in people merging? Will my merged contacts sync with the Metro Mail App and Windows Phone? It's a lot of work to redo these associations.
I see you can add Facebook and LinkedIn social connectors. Where's Twitter?
Are live tiles coming?

OneNote 2013 Questions

Can OneNote edit PDFs like Word can?

Other/Random

Why aren't we seeing the Metro style versions in this beta release?
Any new authentication mechanisms planned for Skydrive (Facebook?)

I'll probably figure some of this out, but if anyone has some of this information already, feel free to include it in the comments.

Thanks!

p.s. Man this thing posts a LOT faster than Writer.

Vietnam Wall for Windows Phone

paulwhit — Wed, 13 Jun 2012 02:23:59 GMT

Choosing the Platform

I’d been looking into developing an app for Windows Phone since I picked up the Samsung Focus 18 months ago – to learn the technology and showcase what the platform can do. As a Microsoft employee, I feel tied to the platform, but also felt like it could be approachable since I am familiar with the language and some of the tools already. I also liked what I had heard about the Windows Phone 7.5 update and wanted to try out some of the features. I recently finished a work project using WPF in which I picked up some XAML experience, so I felt energized about wrapping my head around the Silverlight-based development stack, and becoming a published app author.

Behind the Idea

So I set out to brainstorm ideas. I figured that good mobile app either has an interesting data set or an interesting algorithm, and is something that is useful away from your computer. I thought about places where you might go and need an app, and somehow remembered about a trip that my dad and I took to the Vietnam Veterans Memorial Wall in Washington, DC. While at the Wall, my dad tried to remember some of the folks he knew about who died in the war. We thumbed through some arcane books where the names were listed alphabetically and by hometown, and found one of the fallen GIs. But we had trouble finding others. Given that most of the soldiers, sailors, airmen, marines and coast guardsmen perished or were declared M.I.A. over 40 years ago, it seems like recall could be a problem worth trying to solve.

The Tiny Wall

So grew the idea of having a portable version of the Vietnam Wall database. I contacted multiple organizations that maintain the database of names behind the wall (there are a few parallel copies maintained by different organizations). There was a lot of interest. Even though the number of potential users is relatively small compared to iPhone or Android, the consensus was that the additional interest and exposure for the wall, and the memory of those listed on it, would justify the efforts. I chose to partner with the good folks at the Virtual Wall®, who tirelessly maintain a virtual memorial for each war casualty and M.I.A., and provide updates to their copy of the data. With their agreement to provide a regularly-updated dump of the wall names (surprisingly, even after 40 years, the data is updated monthly with dozens of changes), I agreed to develop the app free of charge and not include ads or solicit donations. The focus would be on providing a new way to search the database, and providing additional information about the wall to visitors including a link to The Virtual Wall®’s memorials.

We decided on an initial set of features:

Support searching the wall by First and Last Name
Support searching by Home of Record (City/State/Country)
Enable “sounds like” searching for names
When searching by Home of Record, include nearby locations
Indicate where on the wall the name is located

I chose some technical requirements in order to make the platform shine:

Use the Windows Phone embedded database included with Windows Phone 7.5
Adhere to Metro design language principles as best as possible, and take design cues from others (related, make it look good)
Learn and use the standard tools: Visual Studio, Windows Phone Tools, Expression Blend, various CodePlex projects
Avoid common mistakes: slow scrolling list boxes, inconsistent fonts, ignoring user color choices

The Published App

After about 200 hours of work, Vietnam Wall was published just after Memorial Day 2012. I was shooting for a release for that holiday, but it didn’t quite get approved in time.

http://www.windowsphone.com/en-US/apps/0b758fa2-6414-4381-923e-557172cb3edc?wa=wsignin1.0

Tools and Resources

Here is a collection of tools and resources I used in the process of creating the Vietnam Wall app. In retrospect, I’d say the tools are robust and capable, if you know where to look. In order to create a clean and compelling app, it’s necessary to be conscious about all your pixels. Vietnam Wall isn’t a perfect design by any means, but I think it looks clean and works well, thanks mostly to Scott Hanselman and Jeff Wilcox, as well as the other links I’ve shared below.

Essential Reading

These resources are key to getting started and knowing about the tools and processes to develop Windows Phone apps. Scott Hanselman’s volume is quite helpful, although after the 10th or so 6-hour session working on Vietnam Wall, I begin to curse his blog post’s title. Ha!

From Concept to Code in 6 hours: Shipping my first Windows Phone App by Scott Hanselman
Charles Petzold’s free e-Book, Programming Windows Phone 7
App Hub community forum’s “look here to find the resources that you need” post
80% of Windows Phone Developers Not Using the Free Tools

Database

I used a Mango Local database. Turns out it’s not quite SQL CE 3.5, and you have to create and populate it in a very specific way, by creating a Windows Phone app and using the emulator to pump the data in using LINQ-to-SQL, then copying it from local storage to your “real” app. It seems like a rather convoluted process, but it works OK. In the end, it’s nice to have full LINQ access to a data store and not have to distribute any libraries (since the SQLCE drivers are included in the Windows Phone 7.5 OS).

Windows Phone Mango Local Database (SQL CE) by WindowsPhoneGeek (awesome series)
LINQ-to-SQL CompiledQuery
MSDN: Query Examples (LINQ to SQL)
How to Migrate/Downsize a SQL Server database to SQL Server Compact 4.0 (and 3.5)

Interface Controls

I used with Telerik RadControls. As a Microsoft employee, I was able to snag a free (normally $99) license. I was happy with their support and the tile, progress bar and animation helper controls (as part of their custom Root Application Frame) were slick. In particular, I like how transition animations work when paging away from the tile controls (see it in action in Vietnam Wall). I also used their Diagnostics and Picker controls, which I think saved some effort over the CodePlex versions.

Omni-present progress bar (template for Silverlight)
WP7 LongListSelector in depth | Part2: Data binding scenarios
Telerik Windows Phone 7 Controls Forums
Telerik RadControls for Windows Phone documentation

Design

I figure I spent 60% of my time getting this app to look decent. Jeff Wilcox’s design guides for developers were infinitely helpful here, and I set out to apply his guidance as best I could. I also thought the MSDN resources were decent.

Conclusion

I had a pretty good experience developing my first app for Windows Phone. I bit off a fairly complex project (using a Local DB and providing some advanced search functionality), and ended up spending considerable time learning and applying the Metro design language and various Windows Phone design guidelines. I’d say the tools are fairly easy and good to work with (especially the Visual Studio/Blend pass-off), and the documentation is abundant and easy to find. Now that Vietnam Wall is out the door, developing additional apps will have a lot smaller learning curve.

Goal setting at Microsoft

paulwhit — Wed, 14 Sep 2011 19:56:42 GMT

When reading about how Microsoft does business, it’s quickly evident that we’re a metrics-driven organization. Even my wife, who works in strategic planning, mentioned that the instructor at a recent training class cited Steve Ballmer as a core example of someone who leads by scorecard.

On the surface, one might think this would be frustrating. Feeling micromanaged, like your time revolves around the numbers. If the numbers became out of sync with business reality, or if they were unreasonable, I might feel this way. But Microsoft leadership seems to be responsive to changes and suggestions each year, and the process gets me regularly thinking about creating goals that help grow our business, increase my impact, and help others succeed (among other things).

I work in a super competitive place. My compensation is partially awarded in the context of others in my pay grade, as well as the one above me. This means some very experienced, passionate, and intelligent people are vying for the same recognition that I am.

So each year I sit down a few times with my manager to tweak my “SMART” goals to help me stand out above my peers. In some ways, it’s a personal exercise rather than an organizational one. For instance, trying to figure out how to put a personal metric on increasing my exposure and influence is a little tough, and the output seems a little goofy. Statements about number of technical discussions outside my immediate team, for instance, represent my intention rather than a metric tied directly to business accomplishments. I think the point is more to consistently think about accomplishing the intention of goals, rather than the language itself.

The process isn’t flawless, and some of the metrics that get assigned to me may not fit exactly with my work conditions. Also, the goals are not explicitly examined as if they were an exam, so some aggressive goals I set (such as a new MCITP or MCPD certification) get lost in the shuffle if I don’t highlight them specifically. But, generally speaking, goal setting at MCS is a motivational process and has helped lead me down an interesting path towards a more visible presence within my team and at my customers.

Career Inspiration at PASS Summit

paulwhit — Thu, 11 Nov 2010 21:29:28 GMT

I’m at the Professional Association for SQL Server (PASS) Summit, an event that brings together a few thousand professionals that work with Microsoft’s SQL Server for technical training, inspiration and networking.

My justification for attending was mostly technical: to fill some gaps and learn new skills, although I also expected to make new contacts and friends. I’ve done these things, but I also gained some unexpected insight. Namely, I was inspired to focus and document my career plans.

This “aha” moment began with a “Birds of a Feather” lunch group. This type of lunch group at PASS involves choosing a lunch table assigned a topic. My choice was “Why you need to distinguish yourself from the rest,” a group facilitated by DBA and professional development guru Edwin Sarmiento (@bassplayerdoc). We discussed the motivations to stand out, and the topic quickly shifted to figuring out how to stand out. We had great discussions about how to change perceptions that others have of you, and how to do some personal branding in order to be recognized for your work.

I attended some great technical presentations, and although all the presenters have impressive backgrounds, two presenters stood out to me as role models for interesting careers and deep focus: Stacia Misner (@staciamisner) and Joy Mundy. They both have an intense passion and deep focus on the Business Intelligence stack and many years of experience.

Finally, Andy Warren (@sqlandy) presented on developing a Professional Development Plan. In the talk, he outlined the need to discover and document the resources you plan to use to better your career, and described how to create a written plan. This was excellent and is something I have started working on. Importantly, he made points about balance and avoiding burn-out, and I plan to use this to develop insights into my own development plan.

My head is swimming in new thoughts and ideas, and I’m working to develop a story for my next 5-10 years. I have some role models to keep tabs on and some tools to help me move forward. Thanks PASS!

Next steps in learning as a professional consultant

paulwhit — Tue, 22 Jun 2010 23:05:09 GMT

I’ve decided to pursue a Master of Information Management (MIM) program from University of Maryland.

As a consultant for Microsoft, I am required to become an expert on technologies as they emerge. Since I work with customers that are sometimes slow to adopt technologies, I need to ramp up on older technologies as well. Continuously learning multiple generations of technology can be a fun challenge, although sometimes daunting!

Microsoft provides me with amazing tools to stay fresh. MCS allows for 4 weeks of paid training per year, and every 18 months, I attend TechReady, a week-long training in Seattle. I also have free access to everything Microsoft Learning and Microsoft Press creates. I try to take full advantage of these tools and always add certification exams to my yearly commitments in order to keep myself on track.

With all the need for me to learn technologies, there seems to be little time to cycle back to foundational knowledge that can help me make broader decisions, and help customers make more impactful technology choices that are led by business needs.

In the next 5 years, my impact will increase. I’ve made my technology skills and aptitude for running with new technology challenges evident on every project I’ve been assigned to, and I plan to continue to do so.

With the knowledge I gain from MIM program as well as my growing impact on the job, I am hoping to position myself to more fully affect change at a higher level.

In the next 5 years, I hope to shift from the “tactical” impact of designing and implementing specific systems to the world of “strategic” thinking, and I think this degree program will help me get to that point.

Recent Microsoft certification exams as a differentiator

paulwhit — Mon, 06 Jul 2009 23:12:17 GMT

I’m coming off of a stint of a few exams in a row to earn my Microsoft Certified Professional Developer (MCPD) credentials for ASP.NET 3.5, which I obtained last week!

In the past year, I have also passed a few Microsoft Certified Technical Specialist (MCTS) exams for SharePoint, and today I failed (eep!) the MB2-634 CRM 4.0 Extending Microsoft Dynamics exam (but plan to re-take it).

Throughout this experience I’ve been considering the impact having these “titles” might have on my employer’s confidence in me and how I’m presented to customers. I refer to the certifications as titles because they’re often displayed in email signatures, including mine.

In my limited experience as an interviewer (I’ve interviewed a dozen or so candidates for various positions within and outside of Microsoft), I have found that the old certifications weren’t terribly useful as a distinguisher. Most of the senior-level candidates had a pile of them (MCSD, MCAD, MCDBA, etc), and they were usually old versions of tests that showed they took a bunch of tests at some point in their career. It was nice to be able to fill in the certification check box, but didn’t always indicate relevant exposure or experience.

Things are slightly different for me now that there are more clearly versioned exams (such as the ASP.NET Developer 3.5 or SQL Server 2008 exams), and I feel that the titles can be helpful to distinguish developers on a few factors:

Active in Microsoft Learning community

With the version-specific exams, I think the most obvious and clear conclusion about someone who takes newer exams is that they make a conscious and committed effort to Microsoft training. Everyone committed to a career that touches Microsoft software (or any other vendor for that matter) may want to pay attention to what the company itself feels is important to learn.

While the learning community is a clear marketing tool for Microsoft to push earlier adoption and drive product sales, it also serves the purpose of producing better Microsoft developers.

In interviews, I usually ask what resources candidates use to keep up on the latest happenings. I like to hear things like blogs, newsgroups, and email distribution lists. Being conscientious of what Microsoft itself pushes out for training shows an additional level of commitment.
Up-to-speed on the latest technologies, to some extent

I feel strongly that Microsoft focuses its exams on showing the power of its technologies, and mentioning common pain points.

Especially with the developer exams, it’s clearly in Microsoft’s best interest to point out gaps and have developers make the best use of its technologies. If ISVs and business application developers can make Microsoft technologies scream, the company will sell more software.

There are limitations to what can be gleaned from a certification list, of course. In my experience it’s helpful to take practice tests to pass exams, but these sometimes seem to promote memorization of question types rather than actual learning. In the end, though, I’ve always learned valuable lessons from the exams and at least have been exposed to much of the power and pain points of the various technologies covered by the tests.
Has specialized or broad knowledge

It seems like the impetus of changes to the title structure for exams in the past few years is to help people indicate what they’ve focused on.

Although I’ve not yet realized the impact of my own certifications, I consider it a bonus for my marketability as a consultant to list the titles on my email signature and on my internal résumé. I think it will help the partners and customers I work with to see my focus areas, and help my management to allocate me appropriately in the future.

Additionally, I want to indicate that I’m focused on being knowledgeable in a variety of areas, so I will continue to take exams in .NET development, SQL Server, SharePoint, CRM, and any other technology that I feel will help me solve customer problems in a more effective manner. I’m hoping my employer will see this commitment and allow me to grow into a more senior role as a consultant and/or architect.

Conclusion

There’s no doubt that Microsoft benefits monetarily when people become certified. Passing active and recent exams helps developers and allows employers to distinguish ones who stay current. It’s not a foolproof method, since passing an exam doesn’t always indicate practical knowledge or experience, but the titles can effectively communicate commitment to learning and staying current.

Disclosure: I work for Microsoft. I get exams, study materials, and practice tests for Microsoft exams for free. I also have access to free exam prep classes (schedule and budget permitting) at internal training events (such as TechReady, an internal training conference) and instructor-led training. However, I still have the same test standards as everyone else and no insight into exam answers.

Walkthrough: Quick Start with SharePoint Web Services

paulwhit — Wed, 08 Apr 2009 11:10:00 GMT

Introduction

Integrating external applications with SharePoint data and functionality is pretty easy, but the documentation is scattered, so I thought it might be helpful to provide a complete solution that covers a few main scenarios.

I’ve used the SharePoint web services to provide a custom search interface and to populate lookup tables. I’ve also seen it used as a workaround in a few scenarios where the SharePoint Object Model was behaving erratically.

This post will focus on how to quickly get started with the web services interface in WSS, and particularly how to access SharePoint data and search results quickly, in a new project. It’s not very polished, but it gets the job done.

Does this apply to me?

First, when communicating with WSS and MOSS, you have to decide which remote access tool best fits your needs.

The SharePoint Object Model (i.e. from the WSS SDK) is great when your application is written in .NET and is deployed to a server running MOSS or WSS,
RPC over HTTP is apparently good for large file uploads,
WebDAV allows for browsing via Windows Explorer and drive mapping, and
The web services interface is best when you need a connection that doesn’t depend on the WSS/MOSS SDK, when you need to connect from outside .NET code (such as via JavaScript).

As Alvin Bruney points out in his book Programming Excel Services (Safari), the web services interface is also “a way to access the resources of the server with reduced risk of instability to the server resources.”

Secondly, figure out if you need to do this from scratch at all.

Check into LINQ to SharePoint, which can use either the SharePoint Object model or the web services interface. It looks like a really slick and robust way to interface with your existing SharePoint data. On the down side, it appears to not have been updated since November 2007 and Bart De Smet, the project’s author (blog), notes that it is an alpha release and not ready for production. I’ve steered clear of it for that reason and due to client restrictions, but you might save yourself a lot of time if you can use it!

Check out this video, from the project’s CodePlex page, for a good quick start:

LINQ to SharePoint quick start video (5:37, 3.18MB, WMV)

If you decide against LINQ to SharePoint, you can still easily add web references and consume the SharePoint web services in your custom code.

Walkthrough: Accessing List data

As it turns out, the web host for Hands on DC (a volunteer organization with which I work) does some hokey things with NTLM authentication that aren’t supported by WCF. Namely, it seems like they’re doing Transport level authentication over plaintext HTTP instead of HTTP over SSL. I can see why this isn’t supported, for security reasons. Despite Dan Rigby’s excellent post about impersonation in WCF, I couldn’t get anything to work.

But alas, with no slight of hand whatsoever, the scenario works in VS2005 and a regular old-school ASMX-based web service proxy (web reference). Here’s a quick (and quite dirty) example, initially based on Ishai Sagi’s (blog) response on MSDN:

Video hint: Click the full screen icon (looks like a TV)

Download source code (39KB, ZIP file)

The query that we send to the list service is written in CAML. When we want to get more complex in our query, such as only pulling back certain columns, check out the U2U CAML Query Builder. It provides an awesome interface into creating your CAML queries.

In the WCF world, Kirk Evans has an awesome walkthrough, Calling SharePoint Lists Web Service using WCF, which includes how to streamline the XML access by using an XPathNavigator, and later using a DataTable that reads from an XmlNodeReader. Better than querying the fields directly for sure!

Walkthrough: Querying WSS Search Results

Search is a little more challenging. As is the case with every other web service, we send the query service XML, and it returns XML. We can take a shortcut and generate the request XML with a tool such as the Search Query Web Service Test Tool for MOSS (or we could also import the schema and generate a C# class from it).

Since I’m a big fan of using XSD.EXE to generate a C# class, I chose to do so with the search result support classes. It was mostly productive, although the Results node is free-form (can take any node type) and the generator doesn’t seem to support that. In the end, we can use the generated classes to get statistics about the data set, and can navigate the Documents using regular XML methods.

Here is a complete walkthrough of adding very basic search results, including a total result count and the first page of results, to our application:

Video hint: Click the full screen icon (looks like a TV)

Download source code (56KB, ZIP file)

Conclusion

Use SharePoint web services when you have the need to separate concerns and reduce risk inherent with new code deployments to the SharePoint farm, you want remote access to SharePoint from a non-SharePoint server, or you need to access SharePoint from something other than .NET (such as JavaScript).

This post and associated videos walked you through creating a Windows Forms application, from scratch, that pulls data in from SharePoint, including List-based data and search results. The process is similar to any other web service, but there are a few gotchas and pain points that I hope have been cleared up in this resource.

I collected some additional links in the process of putting this post together, and added them on delicious with the tag “sharepoint-webservices”.

Walkthrough: full example of using MvcContrib grid with jQuery datatable

paulwhit — Sun, 29 Mar 2009 06:57:00 GMT

Note: This walkthrough is based on the 1.0 release of ASP.NET MVC, MvcContrib version 1.0.0.916 with a minor code change, jQuery 1.2.6 (included in the MVC release) and jQuery datatables 1.4.

Introduction

For the past few months (and 100+ volunteer hours!) I’ve been creating a web application for Hands on DC that calculates volunteers, gallons of paint, and materials for work projects for their annual Work-a-Thon event. After the encouragement of a few coworkers who did some initial work on the project, I committed to using ASP.NET MVC, technology which has been out over a year but just reached a production 1.0 release at Mix 09 this year.

Getting up and running with MVC wasn’t an easy task. The project was also my first foray into LINQ to SQL, and really .NET 3.5 in general, so it was a little intimidating at first! There’s not much documentation and it’s split across the many release versions of MVC. The main site will get you up doing very basic things (but is seriously lacking content), though Phil Haack’s webcast and Scott Hanselman, et. al.’s free e-Book are helpful.

In the process, I discovered some important companion pieces in MvcContrib and jQuery, including the validation plugin and the datatable plugin. I want to highlight work that I did to combine the MvcContrib data grid with the datatable for sorting, paging and filtering. This was something I struggled with for several hours, so I’m hoping there is some value in posting the full example.

Figure 1. Example of using MvcContrib with jQuery datatable plugin.

Walkthrough

Here is a complete from-scratch example.

Create a new ASP.NET MVC Web Application. Press F5 to make sure it runs.
Drag and drop the “media” folder from the datatable project directory (where you unzipped the project) into the new Solution, like so:

Figure 2. Solution after copying the datatable media folder.

Add a reference to the MvcContrib.dll.
Open the Views/Home/Index.aspx file and add the following under the page directive:

<%@ Import Namespace="MvcContrib.UI.Grid" %>
<%@ Import Namespace="MvcContrib.UI.Grid.ActionSyntax" %>

Open the Views/Shared/Site.Master file and add references to jquery, jquery.dataTables and the demos.css file, like so:

Hide Code [-]

<%@ Master Language="C#" Inherits="System.Web.Mvc.ViewMasterPage" %> 

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 
<html xmlns="http://www.w3.org/1999/xhtml"> 
<head runat="server"> 
<title>
<asp:ContentPlaceHolder ID="TitleContent" runat="server" /></title> 
<link href="../../Content/Site.css" rel="stylesheet" type="text/css" /> 
<link href="../../media/css/demos.css" rel="stylesheet" type="text/css" /> 
<script src="../../media/js/jquery.js" type="text/javascript"></script> 
<script src="../../media/js/jquery.dataTables.js" type="text/javascript"></script> 
</head>

Now set up some dummy data in the HomeController.cs file like so (add to the body of the class):

Hide Code [-]

public enum Medal
{
    Gold,
    Silver,
    Bronze
}

public class MedalWinner
{
    public string Location { get; set; }
    public string Year { get; set; }
    public string Sport { get; set; }
    public Medal Medal { get; set; }
    public string Country { get; set; }
    public string Name { get; set; }

    public MedalWinner(string l, string y, string s, Medal m, string c, string n)
    {
        Location = l;
        Year = y;
        Sport = s;
        Medal = m;
        Country = c;
        Name = n;
    }
}

public ActionResult Index()
{
    ViewData["Message"] = "Welcome to ASP.NET MVC!";

    var medalWinners = new List<MedalWinner>();
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Gold, "Croatia", "LOSERT, Veni"));
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Gold, "Croatia", "BALIC, Ivano"));
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Gold, "Croatia", "ZRNIC, Vedran"));
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Silver, "Germany", "JANSEN, Torsten"));
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Silver, "Germany", "KRETZSCHMAR, Stefan"));
    medalWinners.Add(
        new MedalWinner("Athens", "2004", "Handball",
            Medal.Silver, "Germany", "VON BEHREN, Frank "));

    ViewData["MedalWinners"] = medalWinners;

    return View();
}

Now we have some data to work with. In the view (ViewData[“MedalWinners”]). Test that theory by adding the following code to the “indexContent” content block in Views/Home/Index.aspx:

Hide Code [-]

<ol>
<% foreach (HomeController.MedalWinner winner 
       in (List<HomeController.MedalWinner>)ViewData["MedalWinners"] )
{ %>
    <li><%= winner.Name %>, <%= winner.Country %></li>
<% } %>
</ol>

You should see output like this:

Assuming that worked, we’re ready to do some grid work! Once you get the basic syntax, the MvcContrib grid makes display a cinch. This is the “action syntax” so make sure your imports are right (shown at the top of the walkthrough.

Hide Code [-]

<% Html.Grid((List<HomeController.MedalWinner>)ViewData["MedalWinners"])
       .Columns(column =>
       {
           column.For(c => c.Year);
           column.For(c => c.Location);
           column.For(c => c.Name);
           column.For(c => c.Country);
           column.For(c => c.Medal.ToString());
           column.For(c => c.Sport);
       }).Render();
%>

Run the application; you should see output like so:
That’s pretty easy, but not terribly useful yet. For fun, let’s add a Live search on the name field so we can see who these athletes are. The “Action Syntax” makes it simple to do powerful clean custom code for each cell. Pretty slick! Note that you need to specify the table cell tags or you’ll get messed up output.

Hide Code [-]

<% Html.Grid((List<HomeController.MedalWinner>)ViewData["MedalWinners"])
       .Columns(column =>
       {
           column.For(c => c.Year);
           column.For(c => c.Location);
           column.For(c => c.Name).Named("Athlete Name").Action(a => {%>
               <td><a href=
               "http://search.live.com/results.aspx?q=<%= HttpUtility.UrlEncode(a.Name) %>&form=QBLH">
               <%= HttpUtility.HtmlEncode(a.Name) %></a></td>
           <%
           });
           column.For(c => c.Country);
           column.For(c => c.Medal.ToString());
           column.For(c => c.Sport);
       }).Render();
%>

So now our output name is linked to a Live Search.
Now we’re ready to incorporate the jquery datatable stuff. If you see in the directory that you unzipped the datatable project to, there is an “eaxmple_zero_config.html” file. Let’s steal the code from the top of that.

In order to get this to work, we need to add an id attribute to the grid, as is shown below.

Let’s also style it a little by putting it in a container. Otherwise the search box doesn’t align with the table and looks a little wonky:

Hide Code [-]

<script type="text/javascript" charset="utf-8">
    $(document).ready(function() {
        $('#example').dataTable();
    });
</script>

<style>
#example { width: 100%; }
#container { width: 600px; }
</style>
    
<div id="container">
<% Html.Grid((List<HomeController.MedalWinner>)ViewData["MedalWinners"])
       .Columns(column =>
       {
           column.For(c => c.Year);
           column.For(c => c.Location);
           column.For(c => c.Name).Named("Athlete Name").Action(a => {%>
               <td><a href=
               "http://search.live.com/results.aspx?q=<%= HttpUtility.UrlEncode(a.Name) %>&form=QBLH">
               <%= HttpUtility.HtmlEncode(a.Name) %></a></td>
           <%
           });
           column.For(c => c.Country);
           column.For(c => c.Medal.ToString());
           column.For(c => c.Sport);
       }).Attributes(id => "example").Render();
%>
</div>

Now we can do sorting by clicking the column names, filtering by entering text in the search box, and pagination by selecting the drop-down for “show X entries”.

That’s pretty much it! Before we part, let’s make a quick foray into the syntax of making changes to the datatable initialization code. We can disable columns so they can’t be sorted, create a custom sort order, and change the default length by setting some properties. Let’s disable sorting by the first and sixth columns, sort by name by default, and set the default paging to 25. Just change the javascript block to this:

Hide Code [-]

$(document).ready(function() {
    $('#example').dataTable({
        "iDisplayLength": 25,
        "aaSorting": [[2, "asc"]],
        "aoColumns": [{ "bSortable": false }, null,
            null, null, null, { "bSortable": false}]
    });
});

And voilá!

[Download complete source code] (394KB, ZIP file)

Get common dates in T-SQL

paulwhit — Wed, 25 Jun 2008 21:10:00 GMT

A few years ago I created an article around Reporting Services and dates. It could have been written more generically, because I reference this quite a bit to get common dates like "the beginning of this week", "midnight last night", etc, in my SQL queries. It's a fairly comprehensive list of relative dates that one might want to get in T-SQL for reporting, scheduling, etc.

It can get pretty complex, such as this function for getting the end of the current week

CREATE FUNCTION get_week_end (@date datetime)
RETURNS datetime AS
BEGIN
   return dateadd(yyyy, datepart(yyyy,
      dateadd(weekday,7-datepart(weekday, @date),@date))-1900, 0)
    + dateadd(ms, -3,
      dateadd(dy, datepart(dy,
     dateadd(weekday,7-datepart(weekday, @date),@date)),0) )
END

If you don't find what you need, you can typically use the dateadd function to tweak one of these. Here is the complete list outlined in the article:

Start of this week
End of this week
Start of last week
End of last week
Start of this month
End of this month
Start of last month
End of last month
Start of the day yesterday
End of the day yesterday
Start of the day today
End of the day today
Start of the day this Monday
End of the day this Monday
Start of the year
Tomorrow at noon
Today at noon

New SharePoint Developer Resource

paulwhit — Sat, 14 Jun 2008 19:00:00 GMT

As part of the Capital Area .NET's new SharePoint SIG, I led a discussion group last week which worked together to create a WSS/MOSS Developer Productivity Wiki Page. Check it out on WSSWiki.com, and please feel free to add to it!

Personal Twitter Client Counter

paulwhit — Wed, 30 Apr 2008 06:10:00 GMT

Get a count and percentage of how many updates you've had with each twitter tool.

Download C# 3.5 WPF Source Code
Note: This source code is mostly a quick hack to get the data I wanted. Also, if you run this on your own data, note that you may need to wait a few minutes for results to come back.

In testing out various twitter clients, I wanted to get a feel for my personal usage (how much I've used each client), so I decided I'd write some code around it. To begin, here are the results:

web, 1770, 41.549%
im, 1214, 28.498%
txt, 713, 16.737%
SnapFoo, 153, 3.592%
twigadge, 146, 3.427%
Twadget, 128, 3.005%
TwitterFox, 61, 1.432%
Snitter, 21, 0.493%
Jott, 17, 0.399%
TwitBin, 16, 0.376%
twitterfeed, 7, 0.164%
Spaz, 6, 0.141%
Twitter Opera widget, 3, 0.070%
twhirl, 3, 0.070%
Twittearth, 2, 0.047%

I snagged an API code sample and fired up my trusty Visual Studio 2005. Thankfully (so I could step out of my box a bit), the code sample was in C# 3.5! So... I popped open my laptop and set out coding (finally, for the first time outside of dragging some controls around) in Visual Studio 2008.

As part of this, I wrote my first LINQ-to-XML query!

This code makes a request to twitter (you have to be authenticated first) and gets an anonymous type list called sc that has the key (the name of the source, as shown above) and the count from that page. After that I work some magic to aggregate them across all pages, though I'm investigating how to do the aggregation by first combining the XElement-s from each page (though that would take up a lot of memory).

url = "http://twitter.com/account/archive.xml?page=" + x.ToString();

WebRequest Request = HttpWebRequest.Create(url);
Request.Credentials = new NetworkCredential(userName, password);
WebResponse Response = Request.GetResponse();
Stream stream = Response.GetResponseStream();
TextReader reader = new StringReader(new StreamReader(stream).ReadToEnd());
XElement statuses = XElement.Load(reader);

// My first LINQ query!
var sourceNames = from source in statuses.Descendants("source")
                  group source by source.Value into sc
                  orderby sc.Count() descending
                  select new { sc.Key, MyCount = sc.Count() };

I also got to play around in WPF just a little. Nothing fancy at all, but it was nice to see the new IDE in action. It feels much more powerful.

I may play around with this some more, add graphing, toy around with it as a Silverlight 2 app. Or it might just sit there and do nothing ;-)

If you'd like a more full-featured statistics tool for your twitter status updates, check out TweetStats (my stats).

Latest tool: XPath query field extractor for InfoPath

paulwhit — Wed, 16 Apr 2008 10:55:25 GMT

Update: This tool seems to be only relevant with InfoPath 2003. A co-worker pointed out that due the customer pains around a lack of a feature like this, in InfoPath 2007 you can right-click a node in the data source and choose "Copy XPath". Cool! Thanks to Scott Heim for pointing this out.

So, I love making tools. One-off things that process something. Batch files that lessen monotony. Sexy little WinForm apps for displaying data. XSL transformations that regurgitate and pre-process XML.

My latest tool is a small WinForms app that pulls out a list of fields form an InfoPath document so they can be referenced as XPath. This is pretty neat because my client needed a documentation artifact, and this saved quite a bit of tedium. Getting this data also proved difficult because the Namespace URI for the "my" namespace is always changing because it includes a timestamp (see solution below).

Here's a sample of the output. You can use each of these lines as XPath queries against the document:

my:MyFields/my:secMain my:MyFields/my:secMain/my:secPhysicalDescriptionHeader my:MyFields/my:secMain/my:secDualCitizenshipMain my:MyFields/my:secMain/my:secDualCitizenshipMain/my:secDualCitizenshipName my:MyFields/my:secMain/my:secDualCitizenshipMain/my:secDualCitizenship
.
.
.

The tool uses the CAB SDK's extract.exe via a command shell to pull out the template.xml file from the InfoPath XSN file.

The code for using the SDK was pretty straightforward. In this case (since we want the XML template in the InfoPath, the variable ExtractionFileName is defined as:

private const string ExtractionFileName = "template.xml";

And fileName is the location of the XSN file. The code for the extraction is:

// Use the CAB SDK to grab the template file from the InfoPath XSN package
Process proc = new Process();
proc.StartInfo.FileName = ".\\EXTRACT.EXE";
proc.StartInfo.Arguments = string.Format("\"{0}\" {1}", fileName, ExtractionFileName);
proc.StartInfo.UseShellExecute = true;
proc.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
proc.Start();
proc.WaitForExit();

Dealing with the ever-changing "my" namespace

One particular challenge arose. Since the "my" namespace in InfoPath is defined with a timestamp, every time you change the schema the namespace URI changes. So, I had to do a little preprocessing before I could read the fields (since they're all in the "my" namespace). Thankfully the timestamp is of fixed length.

XmlDocument doc = new XmlDocument();
doc.Load(extractionFilePath);

// Use a low-tech way to get the "my:" namespace URI, since it 
// differs by form and when changes are made
string fileContents = File.ReadAllText(extractionFilePath);
int myIndex = fileContents.IndexOf("xmlns:my=") + 10;
string myNamespace = fileContents.Substring(myIndex, 75);

XmlNamespaceManager mgr = new XmlNamespaceManager(doc.NameTable);
mgr.AddNamespace("my", myNamespace);

XmlNode myFieldsNode = doc.SelectSingleNode("//my:myFields", mgr);

After you get the "myFieldsNode" you can recursively iterate through the nodes and output the fields.

Synchronize XSD from an InfoPath document with a SharePoint project for server-side XSD validation

paulwhit — Tue, 15 Apr 2008 10:41:00 GMT

In order to implement server-side XSD validation against an InfoPath form submission, I had to pull the XSD from the InfoPath document archive (XSN file) and use that to update an embedded resource in a SharePoint solution project (including automated checkout/checkin in TFS). I was able to do it using a batch file.

Pretty specialized need, but maybe somebody could make use of it someday ;).

Note: If you want to use this code, you'll have to combine some of the lines that were too long to display.

@echo off 

rem ***********************************************************
rem This batch file is used to extract the XSD files from each 
rem of the InfoPath forms, and copy them into the source tree
rem Download and install the Microsoft CAB SDK from 
rem http://support.microsoft.com/kb/310618 to c:\cabsdk
rem *********************************************************** 

rem IMPORTANT: Set these values to your system. Do not include 
rem trailing slashes, and include values in quotes if necessary 

set CABSDK_PATH=c:\cabsdk\bin
set ROOT=C:\projects\projectname
set TFS_USERNAME="domain\user.name"
set TFS_PASSWORD=password 

rem These should be relatively fixed
set FORMS_ROOT=%ROOT%\Project.Namespace\12\TEMPLATE\Layouts\infopathForms
set XSD_ROOT=%ROOT%\Project.Namespace\12\Resources
set TF_PATH=C:\Program Files\Microsoft Visual Studio 8\Common7\IDE 

echo.
echo ********* InfoPath Form 1 *********
echo.
set FORM_PATH=%FORMS_ROOT%\Form 1\Form1.xsn
set XSD_PATH=%XSD_ROOT%\Form1.xsd
set RETURN=FINISH
goto UPDATE_SCHEMA 

:FINISH
goto FINISH_BATCH

:UPDATE_SCHEMA
echo Checking file out...
"%TF_PATH%\tf.exe" edit "%XSD_PATH%" 
    /login:%TFS_USERNAME%,%TFS_PASSWORD% /noprompt >/nil 

echo Extracting file...
del myschema.xsd
"%CABSDK_PATH%\extract.exe" "%FORM_PATH%" myschema.xsd >/nil 

echo Replacing file...
copy myschema.xsd "%XSD_PATH%" >/nil 

echo Checking file in...
"%TF_PATH%\tf.exe" checkin "%XSD_PATH%" 
    /login:%TFS_USERNAME%,%TFS_PASSWORD% /noprompt 
    /comment:"Automated checkin from XsdExtractor script" >/nil 

goto %RETURN% 

:FINISH_BATCH

Live Writer: Word+Social media, for bloggers

paulwhit — Tue, 15 Apr 2008 05:28:19 GMT

I just started using Windows Live Writer to update my blogs*. I'm thoroughly impressed with what I see, both from the practical sense and just how much cool seems to surround this thing.

1) My Apartment Building, Washington, DC

Software+Services is Microsoft's answer to the debate about where the future of computing is going in the near future. My take on what this means is a badass WinForms interface with a sexy back-end built around some Web 2.0-ish stuff. That's the technical explanation.

Live Writer epitomizes all of this. It's a lightweight, feature- and graphics-rich editing environment, and it's very closely integrated with a bunch of "Web 2.0" sites, including a pile of blog packages and social media sites.

There are some neat plug-ins "out of the box", including one that took me exactly 12 seconds to show you an image of my apartment building (see image 1).

Plus, there seems to be an active plug-in community developing things to better pretty up source code, integrate with Flickr, Facebook and other sites. Some of em are crap, but the architecture seems like it must be easy enough to be accessible. One cool one is the Amazon Book Lookup plugin (see example below)

Some of the interface features I'm falling in love with include the ability to match the style to the blog's style (so I can switch between the three blogs and see how it'll look in each one), and the ability to pull up old posts from the "Open" menu.

Party of One: The Loners' Manifesto
by Anneli Rufus

Read more about this book...

* My blogs:

Use the DescriptionAttribute with an Enum to display status messages.

paulwhit — Tue, 01 Apr 2008 04:29:00 GMT

I've been working on various forms of displaying status messages from enums, and here's the latest preferred iteration of how to do this. Regurgitated and tweaked from WayneHartman.com.

public enum XmlValidationResult
{
    [Description("Success.")]
    Success,
    [Description("Could not load file.")]
    FileLoadError,
    [Description("Could not load schema.")]
    SchemaLoadError,
    [Description("Form XML did not pass schema validation.")]
    SchemaError
}

private string GetEnumDescription(Enum value)
{
    // Get the Description attribute value for the enum value
    FieldInfo fi = value.GetType().GetField(value.ToString());
    DescriptionAttribute[] attributes = 
        (DescriptionAttribute[])fi.GetCustomAttributes(
            typeof(DescriptionAttribute), false);

    if (attributes.Length > 0)
    {
        return attributes[0].Description;
    }
    else
    {
        return value.ToString();
    }
}

It's possible to do something even cooler like cache the values or add a ToDescription() method (in C#3.0), but I just wanted an simple, repeatable way to do this.