UberDemo

Oomph: A Microformats Toolkit

2009-08-26T02:29:00Z

Mix Oline presents their newest project: Oomph2: A Microformats Toolkit. This new version adds support for the hMedia format, plus a lot of other enhancments/improvements. Check it out at: http://www.visitmix.com/Lab/oomph/.

"Microformats are about enhancing the web, representing data in HTML and moving that data around. Oomph: A Microformats Toolkit is for web developers, designers and users, making it easier to create, consume, and style Microformats. Oomph makes consuming and producing content for the web just a little easier, more efficient, more fun. Now featuring Oomph2!"

Circumventing an IE/Flash Display Issue in the WPF WebBrowser control on x64 machines

2009-07-09T19:29:59Z

Recently I wrote some code with the intent to display an embedded YouTube Flash video in Window Presentation Foundation’s WebBrowser control. I ran into an interesting problem. Internet Explorer displayed a broken object icon on some machines, but not others. After some investigation it appeared that the Flash video was displaying properly on x86 machines and failing to appear on x64 machines. Apparently the x64 build of the WPF application spawns an x64 instance of IE. The Flash object does not display, since IE attempts to load the x64 version of Flash, which incidentally does not yet exist for x64 versions of Windows.

To solve the problem, set the compiler to explicitly compile the app to x86, and it becomes a non-issue. The downside is that the app can no longer take advantage of x64 capabilities, but for many cases this is probably fine.

Incidentally here is the code that I used:

private string embedded = "<html style=\"margin:0 0 0 0; background-color:Black; overflow:hidden\">" +
"<script event=onload for=window>document.body.scroll=\"no\"</script>"+ "<object width=\"425\" height=\"344\">" +
"<param name=\"movie\" value=\"http://www.youtube.com/v/{0}&hl=en&fs=1\"></param>" +
"<param name=\"allowFullScreen\" value=\"true\"></param>" + "<param name=\"allowscriptaccess\" value=\"always\"></param>" +
"<embed src=\"http://www.youtube.com/v/{0}&hl=en&fs=1\" type=\"application/x-shockwave-flash\" allowscriptaccess=\"always\" allowfullscreen=\"true\" width=\"425\" height=\"344\"></embed>" +
"</object></html>";

WebBrowser wb = new WebBrowser();
wb.Width = 425;
wb.Height = 344;
wb.NavigateToString(string.Format(embeded, passInPathToVideo));

Using Custom Namespaces at the Root of a XAML File

2009-04-29T21:02:00Z

Here’s something I didn’t know about Silverlight, and I did not find any documentation on it. It’s possible to create custom namespaces at the root of a XAML file. I always knew that you could insert namespaces inline in XAML, but apparently when you define a namespace attribute in XAML it also applies to the root. Here’s a simple example:

<myType:TreeMapPanelItem

x:Class="MIXOnline.Descry.Controls.CustomTreeMapItem"

xmlns=http://schemas.microsoft.com/winfx/2006/xaml/presentation

xmlns:x=http://schemas.microsoft.com/winfx/2006/xaml

xmlns:myType="clr-namespace:MIXOnline.Descry;assembly=Inaugural" >

<TextBlock Text="Hello" />

</myType:TreeMapPanelItem>

This allows replacing of custom elements programmatically as was done in the CPP Custom Treemap sample in Descry which needed to inherit from a base TreeMapPanelItem class to pick up some of its default properties and methods. This could not have been done with a standard Silverlight class since they cannot inherit from custom classes. See the CPP TreeMap sample in Descry for an example and look to the CustomTreeMapItem class for an example of an implementation. Note the assignment of the PanelItemType property in the Page.xaml.cs file which lets the TreeMapItem know which class to use for rendering.

Glimmer Released!

2009-04-29T01:23:41Z

The Mix Online_ team has just released their latest work: Glimmer. Glimmer is a WPF tool that generates jQuery so that you can easily create effects for your website! Check it out at http://www.visitmix.com/lab/glimmer

Regular Expression support in Visual Studio 2008

2009-04-14T20:46:00Z

Since I write a lot of Silverlight and WPF applications, I make heavy use of Visual Studio, and when I work with large XML files it is very time saving to know how to use Visual Studio’s Regular Expressions search support. Every time I use it though I invariably forget how to write RegEx searches, since I use it so seldom. So as the proverb goes: the weakest ink is better than the best memory. Perhaps you may get some use out of it.

Regular Expression search is not on by default. To turn it on you need to expand the options section in the Find and Replace dialog. RegEx search cannot be on by default since certain RegEx characters have completely different meaning that the literal characters and so would cause conflict if both were enabled.

The Complete Character List is conveniently linked off the menu shown when clicking the arrows to the right of the find and replace Textboxes. I tend to learn much more quickly by example, and this page does provide *some* examples, so while useful, it doesn’t contain as many as I’d like. So here are two more for now:

\<xdate\>(.*)

Finds and selects the following node to the end of the line: <xdate>11/14/2008</xdate>

This is handy if you need to remove or modify an XML node. Note that the angle brackets are escaped with backslashes. The parentheses' group the period which allows any character and the asterisk which signifies zero or more characters before the carriage return.

\n~((.+)\n)

Finds and selects carriage returns that have empty lines following them. This is handy if you want to compact empty vertical space. Essentially the logic is: Find a carriage return, but only one that precedes a line with a blank carriage return. This search will not find a line if it has spaces contained in the empty line, so you would need to modify it.

I only provided two examples, but if you know of any good RegEx resources please provide a link in the comment, or you can tweet me on twitter.

MIX09’ing it up with a Tetris Version of Flotzam

2009-03-20T20:23:00Z

For MIX09, our team decided to create a new look for Flotzam that was to be shown prior to the keynotes. We’d thought about possibly creating a Vegas slot machine version, but we ended up choosing Tetris, since it was very much in line with the #MIX09 theme. It sounded like an exciting project, so I jumped on it! Tim Aidlin designed the user experience and the look and feel, and Karsten Januszewski modified the data object model to support binding a single data template type to any one of the available Flotzam data sources collected from the various feeds: Twitter, Flickr, RSS, etc.

We brainstormed about what we wanted it to do, and settled on creating a somewhat simplified version of Tetris that avoided rotating the blocks. Flotzam Tetris needed to specifically target the 16x9 format used in the MIX09 keynote screens. This translated into 1280px by 720px dimensions, a fairly low resolution by today’s standards, which forced us to experiment with shape sizes. We needed to keep a careful balance between ensuring readability of the text in the back row of the keynote sessions while looking esthetically pleasing; the less blocks on screen, the less compelling it appears.

The Tetris logic itself was simple enough; I found several samples on the web to draw from. I chose to borrow some ideas from the CodePlex WPF Tetris that Karsten showed me, but this only got us about halfway. Absent was how to programmatically solve which Tetris blocks should fall based on the pieces that had already fallen. There are probably a few ways to solve this problem. I chose to take a commonly used game approach of using a buffer for the Tetris shapes defined as an array of integers, which I’ll call a table. Zero represents an empty space, and a one represents a space occupied by a block, the fundamental unit that makes up Tetris shapes. It turned out to be convenient to use integers since it very was handy to add up columns and rows for calculating when a row was full or calculating the total height of a stack of shapes for a particular column.

The shapes are defined in the TetrisShape class. The individual blocks that compose a shape are represented in a Point array offset from a (0,0) origin. The TetrisShape class also contains a Position property that maintains the absolute position of a TetrisShape shape in the table.

There are two other interesting parts to the TetrisShape class. The first is a Weight property that allows us to define how important the shape is, so that we can get a better spread of shape usage. It’s used in determining the probability that it will be chosen as the “next” shape to be dropped. The second is the ShapeSignature() method which returns a series of numbers, as a string, that define the “signature” of the bottom of a shape. So for example a 2 x 2 square would have a signature of “00”, or a 3 x 2 Tee shape would have a signature of “101”. The numbers represent the distance to the bottom block from an imaginary line running under the shape. This reason that this signature is returned as a string is so that string operators can be used to match the signature of a shape to the signature of all the other shapes settled in the table.

Fig. 1: Example of the signature of a Square shape vs. a Tee shape.

The GetShapeNeededInTable() method returns the signature of entire width of the bottom of the table. This is a function that examines the height of the columns of the settled shapes in the table to obtain the signature. Once we have this, we can see if a shape can fit in the table by simply calling: ShapeNeeded.Contains(ts.ShapeSignature()). If the call returns true it’s a suitable shape and the shape is stored it in a temporary list that gets sorted based on the TetrisShape Weight property. If there’s more than one shape in the temporary list it randomly chooses one of the shapes with the two highest weights and returns the chosen shape.

The shape chosen then gets bound to the data source and gets added to the VisualTree and displayed on screen. When a row is completed it causes all the “settled” shapes to drop down one level. If a shape drops below the bottom threshold it gets removed from the VisualTree and also the table shape list to prevent a memory leak.

The main loop is the RunTetrisFlotzam() method, which is called at regular intervals by a timer (.7 seconds in by default.) This method contains a method that determines whether a shape can drop down or whether it is “settled” it also calls a function that animates the shapes from one position to the next in a programmatic manner. We wanted the animation to have easing, but it would take a bit of reorganization of the code to make this happen.

You’ll find the source code for Flotzam and Tetris Flotzam here. This was a fun project and perhaps you can make some interesting modifications to it. If you do, be sure to let us know!

A VirtualizedStackPanel Control in Silverlight 2.0

2009-02-18T18:44:00Z

When putting together the code for the Descry - Social Timeline visualization, I quickly discovered that when I subscribed to FriendFeed friends that contained large numbers of items, I eventually encountered performance issues. Scobleizer’s feed was a great acid test, since he had about 13K+ subscribers. We needed the Social Timeline to display, at a maximum, 3 months of subscriber data. I found as the total number of FriendFeed items approached 10K, scrolling and animating databound StackPanels became unacceptably slow. My opinion is that quick response time not only is a critical determining factor in user satisfaction, but also contributes to faster user feedback time and continuity, so that user can make better informed decisions.

It can be argued that what I was attempting to do was an architectural design flaw, since we were attempting to use StackPanels to represent one days’ worth of stacked FriendFeed items; a total of 93 StackPanels for 3 months of data. However, we wished to retain the resizing and scaling properties of the StackPanels’ parent Grid control and it’s ColumnDefinitions, so that we would could avoid writing messy sizing/positioning logic, that the Grid control, conveniently, already has built-in. As a result we would need to find another method to gain performance. We ended up choosing control virtualization, since the sheer number of FriendFeed items is what was causing the slowness.

So, what do I mean by “virtualized control?” A virtualized control is one that utilizes resources efficiently. You might ask: “Shouldn’t all controls utilize resources efficiently?” The answer to that is yes, but there are performance and behavior tradeoffs between a typical control and a virtualized control. Non-virtualized controls have all the databound UI already in memory. Virtualized controls are specialized controls geared for managing large amounts of data that need to bind UI on the fly, and they take a performance hit when doing this. Also, you’ll note that when scrolling the content of the virtual StackPanel control in this project that it does not smoothly scroll up a pixel as a time as a normal ListBox does, instead it scrolls up one item at a time. This is not a limitation of a virtualized control, but rather a limitation of this implementation. To implement this would take a bit more thought.

In other words, when a typical Silverlight control is databound to a data source that contains ten’s of thousand objects, it will literally contain ten’s of thousand UI elements in its’ visual tree; potentially using a vast amount of resources. The visual tree is maintained in memory (in addition to the original data source) whether the visual tree is onscreen or off! Conversely, a virtualized control while having access the to the data source in memory, would only contain the elements that need to be physically displayed within the viewable area of the control, resulting in a dramatically smaller set of items in the visual tree.

This sample shows a virtualization technique for creating a scalable and performant ListBox-like control. There are two key classes: the VirtualizedStackPanel and the VirtualizedItemsControl. The source for this simple virtualized control is here.

VirtualizedStackPanel Role

The VirtualizedStackPanel is a control that simply limits the number of children contained in its visual tree to only the items in the viewable area. This control alone is unable to display the data properly, since a scrollbar is needed to manage the position of the currently viewable area; the VirtualizedItemsControl takes care of that need.

The VirtualizedStackPanel behaves as a normal StackPanel until a data generator is attached. The IDataGenerator implemented in the custom VirtualizedItemsControl lets the VirtualizedStackPanel know what children are currently in the active viewable area, based on control size and scrollbar position settings. As soon as the size of the control, or scrollbar position changes, the MeasureOverride method is called, and the control recalculates which children are to be added or removed from the visual tree. A ContentPresenter is acquired from the assigned data generator, so that a common DataTemplate can be used to define item UI. When the ContentPresenter Content property is set, the data is bound to the UI defined in the DataTemplate.

The main logic in the VirtualizedStackPanel is in the MeasureOverride method, creates new containers if they are needed when the control height increases, or reuses containers if no new ones are needed, and also removes unnecessary containers of the control height decreases. The ArrangeOverride simply positions all the children in the StackPanel vertically.

VirtualizedItemsControl Role

The VirtualizedItemsControl’s role is to act as a container for one or more Virtualized StackPanel’s. It manages the data source, a common DataTemplate that children use, and it also manages the Scrollbar positioning logic. The VirtualizedItemsControl implements the IDataGenerator interface that assists in communicating with its VirtualizedStackPanel children. This sample shows a single VirtualizedStackPanel child for brevity. But you may refer to the VirtualizedTimelineItemsControl in the Social Timeline project on CodePlex for one that contains multiple children.

Incidentally, the VirtualizedItemsControl’s content is defined in the projects /Themes/Generic.xaml file, it could have been just as well defined procedurally, but it uses an external resource file for better designer / developer workflow. It is critical that this file exist in the project in the Themes folder, and note that this file needs to be defined as a resource in the project to avoid errors. I made the Silverlight 2 newbie mistake of not doing this, and I literally spent days trying to figure out why I couldn't get the project to work.

Resources

I found that writing a virtualized control wasn’t quite as complex as I had anticipated, after I broke down a much more complex sample. There is a good example of a full Implementation of a VirtualListBox in the Silverlight Extensions project on CodePlex. Another related WPF VirtualizedCanvas sample by Chris Lovett that I found useful is located here.

Descry and Codeplex

Visit CodePlex to download a more fully developed version of a virtualized control that supports mouse drag animation with physics, plus a VirtualizedListBox as well. And visit here to get the entire Descry project source!

Descry

2009-01-31T08:12:23Z

“Ever wonder what goes into building an effective visualization? Look no further. We decided to roll up our sleeves and explore the topic. We're calling it Project Descry.”

Descry is the new culmination of our “Innovative Web” teams efforts! It’s worth a check!

Announcing MIX ONLINE!

2008-10-22T18:52:59Z

My team members have been working on a entirely new version of the VisitMIX site for the past couple of months. There's going to be some very interesting activity around this site.

MIX Online is a community site for web designers and developers who are building and believe in the innovative web. In the past, the site has given a varied perspective of what is happening on the web, a view into our conference called MIX, and interviews with amazing people with incredible ideas and stories on how designers and developers can take advantage of the web. MIX Online has always been a year-round companion to the event. [more...]

hCard

2008-10-20T22:10:00Z

Hans Hugli

Microsoft

http://www.microsoft.com

Hans Hugli

1 Microsoft Way

Redmond WA , 98052

USA

425.882.8080

Capturing a WebCam stream to a WMV file from within a WPF application

2008-03-28T00:10:00Z

I was asked to help out with a demo, my assignment was to capture some video from a WebCam that was to be used for the Cirque du Soleil demo for MIX08. There were a few ways to do this, none which were obviously attractive. There are many 3^rd party tools that capture video and output to a file, but none I found were extendable and usable from within an application. Another thought was to use the DirectX API’s to do the capture, but this required some fairly involved interop. Windows Media Encoder was not an obvious choice, but to my delight ended up being perfect for the job. I was satisfied with the results. Windows Media Encoder Series 9 has type libraries that expose its COM API’s and can be added to a Visual Studio project directly as references. Visual Studio automatically generates Runtime Callable Wrappers for them.

This is a not a production quality sample, but I thought it still lhad some value getting it out there, since I spent quite some time finding out how to go about this.

A few things to mention up front

· The Windows Media Encoder Series 9 (WME9 from here on) has a bug on Windows Vista that requires a hotfix to be installed. It needs to be installed as an Administrator of the machine. You can run it as a non-Admin, but the hotfix will not install properly and will not report errors. If you find that anytime you access WME9 code (on Vista) and the application crashes without error, you can be sure that the hotfix is not installed properly.

· This sample shows at a basic level how to access audio level information via a Managed DirectX API (Microsoft.DirectX.DirectSound is the DLL referenced) and so requires installing some managed DLL’s. I happened to find these DLLs in the Microsoft LifeCam install, so that’s how I installed them, instead of installing the entire Managed DirectX SDK. There may be a faster/better way to install these DLL’s but I haven’t taken the time to look around for it. If you decide not to install this software, you will need to comment the relevant code.

· You will need a WebCam that functions on your PC to run this sample. This means that you need to install your WebCam’s software and ensure that it works outside of the context of this sample. I myself used the a Microsoft Vx6000 LifeCam, but I’ve tested this app with two other WebCams without issue. The application detects and lists the WebCam in a ComboBox on application launch.

· This application was built in Visual Studio 2008, and will also open properly in Expression Blend 2.0 Beta (February). A DesignTimeHelper class has been added to make it easier to see some of the application elements in the designer at design-time.

· To run this project in Visaul Studio you will need to uncheck the “Loader Lock” thrown checkbox due to the audio level code. To get to this dialog choose Debug -> Exceptions from the Visual Studio menu and expand the “Managed Debugging Assistants” node.

Installing and running the sample

1. Install WME9 from: http://www.microsoft.com/downloads/details.aspx?FamilyID=5691ba02-e496-465a-bba9-b2f1182cdf24&DisplayLang=en

2. [optional] Install the WME9 SDK from: http://www.microsoft.com/downloads/details.aspx?FamilyID=000a16f5-d62b-4303-bb22-f0c0861be25b&DisplayLang=en (The WME9 SDK ended up being very helpful, although I found that the samples in the “samples” directories were not as helpful as the actual content in the CHM file. There are complete samples hidden in the text of the CHM files that were enlightening.)

3. [required for Windows Vista only] Copy the WME9 hotfix locally from: http://download.microsoft.com/download/0/3/d/03d35c05-67da-40e0-9e45-3ea0ca6329a4/windowsmedia9-kb929182-intl.exe (Right-click and choose “Run as Administrator” to install)

4. Install the Microsoft LifeCam Software from: http://download.microsoft.com/download/1/9/5/195512A9-1C1E-4429-BFF0-613D8D92E122/LC14.exe

5. Copy the sample project bits from http://www.hugli.org/code/videoencoder/videoencoder.zip expand it and place the VideoEncoder directory in a directory of your choice. Typically I place content in a c:\users\public\demos directory, since this directory is accessible from all profiles without security issues.

6. Install any necessary software for your WebCam and plug it in.

7. Launch the project in Visual Studio 2008.

8. Modify the OUTPUT_PATH to be a directory that your identity has write access to. The path c:\users\public is the default and will work on Vista, however for XP you will need to modify this path.

9. Press F5. After the application launches you should see two ComboBox’s and a clickable image.

10. Choose your WebCam from the “Video Source” Combobox and use the default profile (384 Kbps.)

11. Click on the image and a video dialog will fade/animate in.

12. Press “Begin Recording.” There will be a slight delay and a video preview will appear on the video dialog. Note that there are also crosshairs, recording status and progress bars to indicate audio level that overlay the video.

13. Press the “Record” button and you will see timecode start advancing in the video overlay.

14. Press the “Save” button and the video strean will be closed and saved to disk in the directory specified in OUTPUT_PATH.

15. The video is then added to a WrapPanel as a MediaElement

16. End Sample.

How was sample done?

Now I’m going to cover the application and some of its hidden subtlties. I wrote this sample in a couple of days since the demo had to be put together in short order. If I had more time, I’d break out the audio related content into its own class and spend some time cleaning up the VideoEncoder class that wraps up all the encoding logic. Another thing that I would do is break out the VideoWindow code into its own UserControl. UserControls cannot draw outside of its own bounds so you would need to allow the UserControl to span the width/height of the entire application to allow for the video fadein animation. There are some problems in the code at present. The most troublesome is that sometimes I find that when capturing video the encoder captures only blank video, and I haven’t found the cause of it yet.

WME9 archiving

One thing that confused me about WME9 initially was that as soon as the Encoder was started the video started capturing to disk and it was only then that a a preview would show up in the preview drawing surface (both happened simultaneously). What I really wanted instead was to see the video preview *before* I began capturing the video, so that I could center my WebCam on my subject, and then and only then start the video capture to disk. Well, it turns out that when the Encoder is started, the video starts capturing (Archiving as they refer to it) immediately, by default. To change the behaviour to what I wanted, i.e. to see a preview before the video captured begins, I needed to set the Encoder.EnableAutoArchive property to false (from its default true), and then start the Encoder. At this point the preview begins displaying in the application, and then we can optionally start the video capture by clicking on the “Begin Recording” button with this call:

Encoder.Archive(WMENC_ARCHIVE_TYPE.WMENC_ARCHIVE_LOCAL, WMENC_ARCHIVE_OPERATION.WMENC_ARCHIVE_START);

Encoder.PrepareToEncode(true);

and end the video capture with this call:

Encoder.Archive(WMENC_ARCHIVE_TYPE.WMENC_ARCHIVE_LOCAL, WMENC_ARCHIVE_OPERATION.WMENC_ARCHIVE_STOP);

It’s also good to know that if you want to capture to an entirely new “archive” file, you will need to stop the encoder, redefine the capture file info and then restart the encoder.

Interop

To get the Video to display in a WPF application took some simple interop. It was interesting for me to find out that you can get a Handle from all WPF elements using IntPtr handle = ((System.Windows.Interop.HwndSource)myElement).Handle so that I could have the video draw on a specific element, but all elements returned the same Handle (the parent windows Handle.) This wasn’t very useful since the video ended up being drawn inside the entire Window no matter which element I chose. To workaround this, I instead created a WindowsFormsHost element and added it to the VisualTree. I then added a System.Windows.Forms.Panel as a child to that WindowsFormsHost (note it is important that the WindowsFormsHost has a child, since the WindowsFormHost element itself gives you the handle of the parent Window, which again is not what we want. The differentiating factor is that System.Windows.Forms controls have a Handle property which makes it very convenient to get the Handle. All System.Windows.Forms.Panel controls are Windows themselve’s. This makes it easy to grab a handle and draw within the bounds of the panel. It should be noted that having all controls be windows can lead to resource and performace problems for large numbers of children, this is not an issue in WPF. The Handle property returns a System.IntPtr which we then pass directly to the Preview.SetViewProperties method along with the video stream itself. Doing this draws on the panel.

When you draw directly on a surface using this technique, there is a side-effect in a WPF application; you no longer can see WPF elements residing on top of this drawing surface anywhere in the VisualTree. I call it a “black hole”, since it obscures or cuts a hole into any window-less elements you try to place on top of it, whether a child element or an element with a higher z-order. This is essentially because the OS can only address something with a handle (a Window in this case) and so it draws the video after all the other elements in the WPF window are drawn.

Video overlay

So how then was the video overlay done? It was accomplished by placing yet another Window on top of the video. The WPF Popup element just happens to be a Window with its own handle. There is a side-effect to this technique: if the parent window is moved, you will find that the Popup doesn’t move along with the parent. This can be remedied by subcribing to the Window.LocationChanged event and in that event modify the position of the Popup. Another side effect is that when the parent Window loses focus, the popup ends up on top of other applications. Again this can be remedied by subscribing to the proper focus events, both of these issues are beyond the scope of this writing.

Animation

Note that the video preview Grid (videoCaptureGrid which also contains the Popup) are not part of the animated videoWindow Grid. I found that trying to animate an element that contained the video rendered through interop performed poorly and drew improperly, so I simply hid videoCaptureGrid until the videoWindow Grid finished fading and animating in.

Possible Exercises

Try to get a Thumbnail of a video. I spent some time on trying to do this and ended up falling back to having a media element display the video for lack of time. Two potential ways to do this are: Use RenderTargetBitmap or grab the Thumbnail that Windows itself generates. The code for using RenderTargetBitmap is in the project in the RasterizeVisualAt96dpi method, but is not used at present. I found that it was tricky to grab the thumb of the video since the first frame(s) of the video tend to be black. Grabbing a video thumbnail from Windows will require the use of interop. I’m not sure if this is possible or feasible.
WME9 is able to capture the desktop of your computer. Create a WPF application that runs in your systray that captures your desktop to a WMV file.
Calibrate the audio levels. Right now the Audio levels are cosmetic and gratuitous. They seem to be a bit delayed and levels seem higher than they should appear. Experiment to get the indicators working properly.
Modify the Popup position when the application is moved or loses focus.

Debug Monitoring Tool

2007-12-07T03:14:58Z

In Building a recipe application using Vista and .NET 3.0 (Part IV: ThumbnailProvider interface) I reference the DBMon.exe tool, and state that it is located in the Windows SDK, however, as of the lastest Windows SDK: Microsoft® Windows® Software Development Kit Update for Windows Vista™ (released 3/22/07), DBMon no longer ships in the SDK. I was unable to find it in any other Microsoft release.

In my search I found this tool that very nicely replaces DBMon: DebugView v4.74 (by SysInternals.) This tool has the same basic functionality, plus many, many more features, such as the abilty to connect to remote machines, persisting sessions to disk, filter entries, and capture from many debug sources.

It is available here on the Microsoft.com TechNet site: (published 11/27/07)

Security Integration of SharePoint 2007 and Community Server 2007

2007-11-14T19:55:00Z

Although I'm certainly not a SharePoint expert, MSDN just posted an article I authored that shows how to cleanly integrate the security model between these two platforms using ASP.NET Roles. I put this together for the Office team from something that came out of a need for our team: http://msdn2.microsoft.com/en-us/library/bb892784.aspx.

“Linkifying” a String

2007-09-28T22:49:00Z

I had a need to create a string “Linkify” function for a web site that I am working on; a function that takes a string, parses it for URL’s, and replaces those URL’s with HTML anchors. The end result is when the string is passed to a web client it would allow the user to browse to the URL.

I started out thinking that it could be done with some simple string manipulation, but quickly realized that finding the end of a URL was not trivial. It then occurred to me that Regular Expressions would solve the problem from my experience with Perl. Regular expressions are exceedingly powerful, but with its’ power comes complexity.

The .NET framework has a namespace dedicated to Regular Expressions: System.Text.RegularExpressions. The RegEx class has a constructor that takes a pattern and options as parameters. The RegEx.Matches(string) method takes a string as an input and returns a MatchCollection, which contains a lot of buried information. For my purposes I was able to avoid having to drill down deep into the collections contained within the object, but it’s good to know that it has the ability to capture the individual groups that a pattern returns.

The trickiest part for me, was recalling how to construct a pattern for this type of query. RegEx pattern syntax is a language of its own, that takes me time to wrap my head around every time I need to work with Regular Expressions, which is not all too often.

So how did I overcome my cerebrally-challenged problem? In searching on Live.com… incidentally in Windows Vista, by default, hit the Windows key, type your search word(s), press the down arrow and press enter… I came across a free 3^rd party tool Regular Expression Designer by Rad Software, (this tool is not endorsed by Microsoft) that helped me to quickly construct and try out the pattern that I needed. It also has a handy quick reference to show the meaning of the esoteric symbols used in RegEx patterns.

Construction of the pattern

Anything within parentheses in a RegEx pattern is called a “group.” In this application want 3 groups; the first: the protocol, the second: the domain, and the third: the path. For the protocol group we want to find an occurrence of http or and ftp. The | operator indicates an Or, followed by the literal ://. The domain group is one or more matches + that do not contain a forward slash, a carriage return, a close paren, or a quote mark [^/\r\n\")], note that some are escaped by preceding backslashes, and the path group needs to have zero or more matches * that begin with a forward slash , but not a carriage return, close paren or a quote mark [^\r\n)\"], and lastly we want to find *all* matches in the string with a ?

Putting it all together we arrive at this: "(http|ftp)://([^/\r\n\")]+)(/[^\r\n\")]*)?"

We could have given the groups friendly names as in: "(?<protocol>http|ftp)://(<?<domain>[^/\r\n\")]+)(?<path>/[^\r\n\")]*)?", if we had wanted to access the individual values by name, for some other purpose. Without assigning names, by default, the groups are numbered in order of occurrence within the pattern starting with 0.

Following is a method call that wraps it all up:

using System.Text.RegularExpressions;

/// <summary>

/// This method takes a string and looks for URL’s within it.

/// If a URL is found it makes a HTML Anchor out of it and embeds it

/// in the output string. This method assumes that we are not passing

/// in HTML. This method would have to be revised to support that.

/// </summary>

/// <param name="input"></param>

/// <returns>Clickified String</returns>

public static string LinkifyString(string input)

{

Regex regex = new Regex("(http|ftp)://([^/\r\n)\"]+)(/[^\r\n)\"]*)?", RegexOptions.IgnoreCase);

MatchCollection mc = regex.Matches(input);

foreach (Match m in mc)

{

string url = m.Value;

string link = Globals.CreateAnchorTargetBlank(url, url);

input = input.Replace(url, link);

}

return input;

}

public static string CreateAnchorTargetBlank(string href, string description)

{

return string.Format("<a href=\"{0}\" target=\"_blank\">{1}</a>", href, description);

}

MSDN has some RegEx examples that may be worth taking a look at.

Be advised that this LinkifyString function will corrupt HTML if you pass an HTML’ified string to it, since we are not checking for anchor elements surrounding the URL’s. To do this one would have to add additional groups that ignore matches inside anchors. i.e. <a href=” http://www.foo.com”>http://www.foo.com</a>.

-Hans Hugli

Creating a “TagFilter” Web Part that interacts with pages in SharePoint 2007

2007-07-25T22:10:00Z

Abstract

Becoming familiar with SharePoint 2007 web parts, CAML and implementing a custom web part that interacts with a SharePoint page’s CAML. This article also covers setting up a development environment, building, deploying, installing, and debugging a web part and includes a project linked at the bottom.

Intro

It is our teams’ goal this year to create an internal site for our Technical Evangelists to go to as a first stop resource for demos. We wanted to organize these demos so that they are easy to find, and we wanted to have the site in a blog-like format. It’s a demos’ nature to become stale with time, and so we thought that a blog format was a good fit for documenting demos. We really like the “tag clouds” concept since it surface’s keywords and their frequency of use and popularity.

We took a look at a few blogging solutions. The first was an ASP.NET sample “Blog Starter Kit.” While it’s a great example of how to code a blog server from the ground up, it proved to be too much work to get where we wanted in a short time.

I next took a look at Community Server 2.0. This is very robust and easy to use blogging software and contains a wealth of other community capabilities, but it ended up being overkill for our simple blogging needs. It also didn’t have the ability to easily customize the UI layout via its web interface; I wanted to be able to add announcements and contact information without having to manage any HTML, plus we needed to be able to add custom properties to the blog entries, such as demo storage location, which Community Server was not designed to enable without code, this meant becoming familiar with their architecture and writing custom code.

Having had some familiarity with SharePoint, and knowing its great support for being able to customize lists, I then examined SharePoint 2007’s blog capabilities, and while they are certainly not as developed as Community Servers’, they were a good starting point for our proposed solution. Out-of-the-box SharePoint 2007 has the capability of creating blog sites with ease. It creates a home page that contains a Blog Post List web part (based on ListViewWebPart.) These blogs can have multiple categories assigned to them with items from the “Category” List, and they can also have multiple comments attached with items from the “Comments” List. The homepage contains a list of all posts in a summary view layout. A second, but equally important, reason for having chosen SharePoint is that I wanted to have a better understanding of its inner workings.

From the Category List, the user can drill down into the category page which out-of-box only displays items from a single category. I found this to be too constraining for our needs, and so I set out to modify Category.aspx to allow filtering with multiple categories, plus create a way to navigate the page in a more efficient and meaningful way with a “Tag Filter” control that would look something like this:

I started out with the “Tag Cloud” web part that I found on www.codeplex.com as a starting point which was very useful, but I needed to change it dramatically to meet our requirements since they were quite different. See resources for more info on the “Tag Cloud” web parts

Definitions

Not the most exciting, but necessary never-the-less to understand later portions of this writing.

CAML – Collaborative Application Markup Language – not to be confused with the Caml programming language, is an XML based language that contains tags to define and display data. In this post we will focus on CAML’s query language and query language schema.

SharePoint Designer 2007 – A tool to edit SharePoint pages along with CAML embedded in those pages. Since the web part CAML markup is stored in SQL, opening files via UNC (e.g. \\<my share point server name>\blogs\default.aspx) from a SharePoint Server within normal editing tools, will not show the CAML markup, so it’s necessary to edit pages with SharePoint Designer 2007 to surface and edit the CAML. Incidentally here’s the trial version of SharePoint Web Designer 2007.

SharePoint Web Part – This is the fundamental unit that can access SharePoint information and display itself on a SharePoint page. With SharePoint 2007, the Office team has made it much easier to develop web parts, since they are now leveraging ASP.NET’s web part infrastructure. See resources section for required reading on web parts that will help you along with development.

Getting Started

The Blog Home page

The default.aspx SharePoint blog home page shows all blog posts that have been created. The two web parts that we will find most interesting on this page are the Category web part and the Posts web part. The Category part enumerates all categories that have been defined. For example: a category of “MyTag” would link to “/<blogsitename>/lists/categories/category.aspx?Name=MyTag”. Note that “Category” and “Tag” will sometimes be used interchangeably in this writing. The Posts part shows all Blog posts ordered by PublishedDate and ID. How do I know how the Posts are ordered? Well, I could edit the page in the browser, go into the View Settings, and look at the settings for its current view, but that won’t help us later when we are trying to dissect the category.aspx page. So let’s instead open the default.aspx page in the SharePoint Designer and take a look at the web part titled “Posts”.

CAML

The web part XML that you will find contains all the default and user-defined settings for the Posts part. A large portion of these settings are exposed through the web management UI, but the ListViewXml property is not. Instead the Web based View Settings UI manages most of the capabilities of this property.

Notice that embedded in the CAML (escaped to avoid collisions in SharePoint Designer) inside the ListViewXml property is a query node. That looks like this:

<Query>

</OrderBy>

</Query>

Converted to XML, it appears much more readable as:

<Query>

</OrderBy>

</Query>

The Category.aspx Page

So now that we know where the CAML and its Queries are serialized, we can move on to taking a look at the Category.aspx page. The Category.aspx page shows only Post items that have been tagged with a name that is passed as a parameter to the page. How does the Category.aspx page capture a request parameter? The answer lies in the CAML of the Posts web part which is also included on the category.aspx page. Inspecting the Category.aspx page with the SharePoint Designer this is how the Posts part markup appears inline. Taking a look at ListViewXml reveals this CAML XML. We find the Query node and see now that it is more complex. Looking at just the Query node again, we now see a “Where” clause that compares the “PostCategory” value with the “Name” request variable:

<Query>

</OrderBy>

<Where>

<Eq>

</Value>

</Eq>

</Where>

</Query>

If we make some minor changes to this query, we can get some much more interesting results, for example: if I wanted to find all Posts that contain the word “Demos” and “WPF”. I could pass the following to the Category.aspx page: “Category.aspx?N1=Demos&N2=WPF” and have the following CAML embedded process the request:

<Query>

</OrderBy>

<Where>

<And>

<Eq>

</Value>

</Eq>

<Eq>

</Value>

</Eq>

</And>

</Where>

</Query>

And then taken to the extreme; we’d like to retain the original “Name” parameter, for backwards compatibility, so that pre-existing links don’t break, and we’d like to have the ability to filter up to 6 Categories deep. (Note that if values are omitted for any of the parameters, no results will be returned. The drawback to this approach, due to the AND query logic, is that we need to always pass a parameter for all request variables; otherwise the query returns no results. So, for this sample to work we need to create a common category “All” and (important for this to work) tag every single Blog Post with it. So our final query will be:

<Query>

</OrderBy>

<Where>

<Or>

<And>

<Eq>

</Value>

</Eq>

<Eq>

</Value>

</Eq>

</And>

<Eq>

</Value>

</Eq>

</And>

<Eq>

</Value>

</Eq>

</And>

<Eq>

</Value>

</Eq>

</And>

<Eq>

</Value>

</Eq>

</And>

<Eq>

</Value>

</Eq>

</Or>

</Where>

</Query>

So now a request for all Posts that contain “Demos” and “WPF” would be: “Category.aspx?N1=Demos&N2=WPF&N3=All&N4=All&N5=All&N6=All”

Setting up a dev environment

First things first, we need to set up the development environment for building/debugging a SharePoint web part. Optimally, it‘s best to set up the Visual Studio 2005 development environment on the SharePoint Server itself, so that you can debug the SharePoint web part. The resources section below contains links that will walk you through some of the basics of creating a web part, though I’d like to provide some basic and additional comments.

You will need to copy the Microsoft.Sharepoint.dll (9Mb) from the SharePoint server into your web part project root to get it to properly compile. I found mine in “C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI” Note that some of the documentation points you to other directories for earlier versions of SharePoint. There was a naming convention change, and so for SharePoint 2007 the correct files are located in the \12\ISAPI directory standing for Office 2007 (or Office “12”). The \60\ISAPI directory is for Office 2003 (or version 6.0).

Registering and installing a web part

Manually registering and installing a web part is not a trivial task, but I’ve taken the following steps to make it easier to version the web part.

1. Make sure that your web part is “strong name signed”. If you use the project that I’ve provided, the build process will take care of this (essentially you just need to add a key file to your solution, and in project property settings choose it to sign the assembly.) If you happen to create a new key, you will have to determine its new PublicKeyToken and adjust this value anywhere it occurs (safecontrol entry and dwp file). See article 1 in the resources section below for direction.

2. In a default installation of SharePoint server the files for the site itself will be contained in the following directory: “C:\Inetpub\wwwroot\wss\VirtualDirectories\80.” Yours may vary, but the 80 stands for port 80. In your web part project you will want to make sure that the assembly that is created, when building the project, gets copied to the BIN directory in your SharePoint Site. This means adding something like the following to the PostBuildEvent property in your project:

xcopy /y $(TargetPath) "C:\Inetpub\wwwroot\wss\VirtualDirectories\80\bin\"

3. Add the following line to the “C:\Inetpub\wwwroot\wss\VirtualDirectories\80\web.config” file in the <SafeControls> section: <SafeControl Assembly="UberdemoWebparts, Version=1.0.0.0, Culture=neutral, PublicKeyToken=d4dfd62c364ed482" Namespace="UberdemoWebparts" TypeName="*" Safe="True"/>

4. Make the following change in the trust level in the same web.config file from WSS_Minimal to WSS_Medium in the trust node so that it appears so: <trust level="WSS_Medium" originUrl="" /> (Note that this is a very insecure way to install web parts. Anyone can install them if they have permissions on the machine, but it is also the least pain free way for doing development. See article 3 in the resources section below, how to properly deploy a web part in a production environment)

5. You must create a DWP file for each web part in an assembly, yet one assembly can contain multiple web parts, and be declared once as safe in the Web.config file. See Resource article 1 below on how to properly format a DWP file. (note that the project provided has a TagFilter.dwp file in the root solution folder.)

6. You should now build the web part in Visual Studio 2005, and ensure that the DLL has been copied into the site’s BIN directory.

7. Now you need to add a reference the web part in a page. Visit the page that you would like to install the web part on and click “Site Actions->Edit Page”

8. Click on “Add a Web Part” on the page, and in the dialog that appears click the “Advanced Web Part Gallery and options” link.

9. Click on the “Browse” sub-heading just below the “Add Web Parts” heading, and click the “Import” menu item in the menu that appears.

10. Click the browse button and browse to the DWP file for the web part that you would like to install and click upload.

11. This step will determine whether you have the web part installed properly: drag the web part from the ToolBox menu onto the page anywhere. If you get an error at this time it will most likely be something to the effect of “your web part is not registered as safe”. Check steps 1-6 to ensure that all are correct and then repeat steps 7-11.

12. You should now see the added web part displaying on your page.

Debugging a web part

Debugging can be tricky at first, but once you get it set up it becomes much easier. In some of the below articles referenced in the Resources section it states that you need to attach to the w3wp.exe process, but it doesn’t say that there might be 3 running, and which one to choose from. I solved this by going into the IIS Manager and stopping the 2 SharePoint management sites.

1. Launch “inetmgr” from the command prompt and open the “Web Sites”node.

2. Stop all sites except for the main one. In my case the main is “SharePoint - 80”

3. You may need to reboot the machine to insure the other w3wp.exe processes die. “iisreset” did not work. Perhaps “net stop w3svc” and then “net start w3svc” will work, but I haven’t tried.

4. Once you have only one w3wp.exe running you should build your project.

5. Load up the web part once in the SharePoint page by visiting the page, to get it to load the version that you just built.

6. Set a breakpoint inside the web part constructor or RenderWebPart method.

7. Now attach to the w3wp.exe process in Visual Studio 2005 by choosing: “Debug->Attach to Process” and selecting the w3wp.exe process in the dialog and click Attach. The first time you do this it may take a while to load up all the debug files. In addition if you see the breakpoint turn hollow with a little yield sign that says something to the effect of “The breakpoint will currently not be hit. No symbols have been loaded”, can mean that the web page did not properly load your web part. Try refreshing the page and reattaching the VS2005.

Briefly examining the TagFilter web part

So now that we understand the inner workings of the SharePoint Post web part, we now need to create our own web part that displays tags based on what posts are displayed and to construct the requesting URLs for those tags properly.

The TagFilter web part inherits from Microsoft.SharePoint.WebPartPages.WebPart which in turn inherits from System.Web.UI.WebControls.WebParts.WebPart which makes it very nice for developers that are familiar with the ASP.NET WebPart programming model. It also abstracts away the implementations of the System.Web.UI.INamingContainer, System.Web.UI.IAttributeAccessor and Microsoft.SharePoint.WebPartPages.IConnectionData Interfaces.

[ToolboxData("<{0}:TagFilter runat=server></{0}:TagFilter>")]

[XmlRoot(Namespace = "UberdemoWebparts")]

public class TagFilter : WebPart

{…}

Each web part must contain a RenderWebPart override which renders any controls that you want the webpart to display. At its simplest we could send “<div>Hello World</div>” down to the browser with:

protected override void RenderWebPart(HtmlTextWriter output)

{

HtmlGenericControl container = new HtmlGenericControl("div");

container.Controls.Add(new LiteralControl("Hello World"));

container.RenderControl(output);

}

The most important thing to understand about this particular web part is how it accesses the Posts list and Categories that are contained within that list:

SPListCollection lists = SPControl.GetContextWeb(this.Context).Lists;

foreach (SPList list in lists)

{

foreach (SPListItem item in list.Items)

{

string[] tagArray = item.GetFormattedValue("Category").Trim().Split(';');

}

Custom properties are very useful and only require you to define some special attributes in order to surface these properties in the web based web part page editor. Properties are persisted in the page CAML when they are modified.

[Bindable(true),

Category("TagCloud Filter Settings"),

DefaultValue(""),

Description("Background Color for Tag Cloud"),

FriendlyName("Background Color")]

public string TagCloudBackgroundColor

{

get { return tagFilterBackgroundColor; }

set { tagFilterBackgroundColor = value; }

}

The rest from here is straightforward business logic. There is a Generics Tags List and a Tag Class that contains the list of remaining tags for a selection. I’ve also created a StyleGrade List to manage Style settings that can be set in the web part properties. I also created a breadcrumb and added filtering logic. This web part was coded to work only with the Category.aspx page.

Here is the project in a zip file (14Kb)

Hopefully this has provided some hard to find educational value above and beyond the articles listed in the following resource section.

Other SharePoint Resources

MSDN

SharePoint Server 2007 Developer Portal

Web Part Development

1) A developers introduction to web parts

2) Walkthrough Creating a Basic SharePoint Web Part

3) Microsoft Windows SharePoint Services and Code Access Security

4) Configuring IntelliSense with CAML Files When Developing for Windows SharePoint Services 3.0

Web part Solutions and Samples

www.codeplex.com/cks, TagCloud 1.0 contains a generic Tag Cloud part and 3 other web parts.