The Visual Basic Team : Matt Gertz

There’s No Such Thing As Too Much Euchre! (Matt Gertz)

VBTeam — Fri, 18 Dec 2009 14:26:00 GMT

“Oh, no!” I hear you cry, “not another blog about Euchre!”

Well, I do like the game, to be sure. But, more importantly, the Euchre scenario enables me to try out any new technologies on a reasonably complex problem (using Visual Basic, of course). It’s my way of checking out our product to make sure that it works before we ship it out to you guys.

For my app building this time around, I chose to focus my energies in two areas – WPF, and Windows Mobile. Porting my Euchre game to these platforms was challenging in several different ways, and I thought that I’d share what I’d learned with you. I won’t list out the code, most of which hasn’t changed, but you can find it at my Temple of VB site.

Porting VBEuchre to Windows Mobile

I actually hadn’t intended to do this one, actually. I mean, I’d always meant to do such a port so that I could play Euchre on my phone, but I’d kept putting it off since there were always other important things to do. Yesterday, however, I found myself sick at home and utterly bored, so I decided to tackle the port. It turned out to be easy (for the most part) – the hardest part is actually redoing the resources. But I’m getting ahead of myself…

I started out by creating a Windows Mobile application, and then adding the resources to the default form. The process of adding controls was nearly identical to creating the original game. The only differences were:

· I used PictureBoxes instead of Labels for the cards, since Labels don’t have images in the compact framework (I should have been using PictureBoxes in the original game, anyway).

· I didn’t bother with a Logo splash, the fancy graphical scoring, nor with the player names – real estate is at a premium on a smartphone.

· GroupBoxes don’t exist, so I use labels to indicate who the dealer is.

· I added a new button to launch the game, for reasons I’ll explain later.

· There are only two menu items in the strip (Restart Game and Exit).

· There is no tooltip control.

· The status area is now a label, and only shows the most recent action.

It took me about 30 minutes to set up the form. The two bid user controls were next, and were essentially identical to their desktop counterparts (though the text in them is abbreviated). I didn’t bother with a “Rules” dialog nor an “About” dialog, so that just left the options dialog. Speech isn’t easily available on a hand-held, and the player personality choices take up too much space as well, so that just left the rule variations to add into it (Stick the Dealer, SuperEuchre, Quiet Dealer, and Nine of Hearts) and the effects (Peek At Cards and Sound Effects).

I then copied the old code and resources over wholesale, and worked my way through the errors. There were darned few of them, actually. Here’s what I ran across:

· “My” support is limited in mobile applications. For some things (like My.Application.DoEvents), simply taking off the “My” is sufficient. For other things, such as playing sounds, I had to use the System.Media.SoundPlayer object directly.

· The Timer control uses “Enabled = True/False” instead of “Start/Stop”

· I ripped out or short-circuited all code relevant to tooltips, the F2 button, default buttons, and text-to-speech, since they have no meaning in this world.

· Images have no rotation methods, so I created some by hand so that cards would be oriented correctly.

· I shortened a lot of the text messages to fit into the available display space.

That was pretty much it – or so I thought. When I ran the game, everything seemed to work fine. However, “New Game” would only work once, and clicking on it again would do nothing. This is because (as I found out) events in the .NET Compact Framework do not support asynchronous events, and my game (as it existed) relied upon that functionality in order to process the “New Game” command in the middle of a game. Thus, I now use a button to start the initial game, and then “New Game” (now renamed “Restart Game”) simply sets a flag that the event loop catches to crawl back up the stack and restart, as before.

Side note: Reentrancy (essentially recursing into event or messagehandling) is dangerous, because you can easily get into a deadlock if you mess it up. DoEvents() is therefore a controversial method to use because it’s so easy to get it wrong, and many people even feel that it should be entirely ripped out of the framework. In my case, the requirement to drive the game and still allow the user to exit it makes it sort of a necessity (unless I were to implement some fairly significant threading and message handling, and that wasn’t the point of my exercise). I worked hard to make sure that I ran into no reentrancy problems when I developed the original program, but you should be very careful whenever you use it.

My total elapsed time at this point, from the start of the project, was about one hour – not bad! I still wasn’t quite done, however. My card images, although beautiful on the desktop version, looked pretty crummy when scaled down to 31 x 24 boxes, and furthermore they were wasting memory and causing time delays due to the scaling and orientation required on each showing. I had to re-write all of my cards into the proper sizes, making them much simple looking in the process so that they were legible.

After a couple of hours of tweaking those images and adding the setup project to generate the CAB, I was almost done. One final gameplay experience encouraged me to decrease the timer interval to 100ms – that seemed to pace the game better on the phone. The final result (developed using VS2008) is available on the Temple of VB site.

Porting Euchre to WPF

As easy as porting the game to Windows Mobile was, porting it to WPF was another thing entirely. WPF and WinForms are really meant for different scenarios, and therefore they require different types of coding skills. (I actually did the WPF port before the Windows Mobile port, but since the WPF one was more complicated, I saved it for last in this blog.)

I used VS2010 Beta 2 for this port, since I was interested to see if I could find any bugs in the product. I’m happy to say that I didn’t run into any major bugs – what I did run into was a lack of knowledge on my part, and so it took me quite a bit longer (a couple of weeks, in fact) to do this port.

As in the case of the Windows Mobile port, I first started by recreating the forms in a WPF Windows Application. The WPF designer is pretty easy to use (even when dropping in 53 controls!), though I ran into a few necessary changes from what I’d done in WinForms:

· I had to be careful to position cards over the group boxes rather than in them. This is because group boxes will, when invisible, hide anything inside them. (I had to go into the XAML to do this change, because my cards kept snapping into the group box.)

· The cards became Images (the moral equivalent of a WinForms PictureBox) instead of Labels, which they really have ought to have been originally anyway. Labels in WPF don’t support background images, so this change was required. (I started out using Labels in this WPF version, and then when I realized that that wouldn’t work, I shifted them to PictureBoxes simply by replacing the class info in XAML – very cool!)

· Controls in WPF have a Z-order, but you can’t change it programmatically. Therefore, it’s important to lay down the controls in the correct order, and to make sure that you make liberal use of “IsHitTestVisible = False (or True)” to avoid inadvertently covering up a control which is supposed to be active and clickable.

· Because of the Z-order issue, it’s easier to work with user controls by simply laying them down rather than creating them dynamically. However, your user controls won’t appear in your toolbox until you’ve built them once. Additionally, accessing them programmatically requires prefixing them with a namespace, something you should keep in mind.

· The tooltip control doesn’t exist and isn’t required – each control has a tooltip, which actually makes things a lot easier.

· Controls don’t have a cursor setting, so I had to set this by hand whenever state changed.

· Timer controls don’t exist, but this is not a big deal, since you can just use the System.Timers.Timer object in the code, which works essentially identically.

· Image controls do not natively support images accessed by My.Resources – they instead prefer files (URIs, actually). To dynamically set these on-the-fly, I had to stream the bitmaps into an in-memory file, and then assign them to the image. I had to do something similar for the RichText edit control on the “rules” page.

· Menus were trickier to set up, since you can’t auto-populate them with standard items. Once you get the hang of it, they essentially work like those in WinForms.

· Font sizes in WPF refer to pixels and not points, which means you’ll likely see a font appear about 30% smaller than you expected if you’re used to WinForms – just something to be aware of. (My code doesn’t correct for this yet, incidentally, so the fonts will appear a bit smaller.)

· The windowing controls (minimize, maximize, etc) are not set individually – to have only a minimize and a close box, the window needs to have ResizeMode set to “CanMinimize,” for example.

· Other controls work as you’d expect, except that radio buttons and checkboxes have both a Checked and Unchecked event, unlike WinForms where both are handled in one event.

The game logic was, of course, identical to the WinForms version, except where I had to retool cursors, timers, and tooltips. Nevertheless, there were a few changes that had to be made to managing the controls and dialogs:

· Instead of using the speech SDK, I took advantage of the built-in speech APIs in the framework. This resulted in much more elegant code, and a smaller package as well.

· There is no equivalent to “DoEvents” in WPF. As I was unwilling to totally rewrite the project to use worker threads and coordination messages, I leveraged a trick using DispatcherFrames that I’d found on the Internet (this is attributed properly in the code sample).

· Modal dialogs cannot be reused, and since I needed to reuse the options dialog and wanted to persist state between uses, I hooked the “closing” event to cancel the Close and instead just hide the window – this necessitated a minor tweak to my “new game” logic as well to check for the state of the options dialog.

· Disabled visible controls do not grey out, which gave me the opportunity to play around with the opacity property, which was kind of fun.

· There were numerous little changes that had to be made due to slight variations in naming conventions – e.g., Enable vs. IsEnabled – and I had to make sure that the IsHitTestVisible property was used correctly as well wherever the visibility of a control changed, due to the aforementioned Z-order limitation.

· Unlike the WinForms “Visible” property, the WPF “Visibility” is not a Boolean but instead supports an enum. Furthermore, the “Visible” part of that enum maps to 0, and the “Hidden” to 1, so if you accidentally use Booleans as argument to Visibility, you’ll get things backwards. For code readability, I threw in some If() calls to disambiguate the situation.

That work got me to parity with the WinForms version, and it works great! I’m not quite finished with the port, though, and so you won’t find an installer package for this one from me yet – now that I’ve got things working, I want to add some of the WPF special effects to the game! However, I’ve posted the code on the Temple of VB site, and the solution also includes the old code so that you can more easily compare the two.

‘Til next time,

--Matt--*

Linking Zune media items with LinQ, Part 2 (Matt Gertz)

VBTeam — Sat, 07 Nov 2009 16:00:00 GMT

In this post, I’ll continue on with coding the new playlist shuffler. If you haven’t read part 1 yet, I highly recommend it so that this post will make more sense. J

Code for the controls (continued)

The Title TextBox

When the title changes, we’ll want to indicate that the playlist has changed, and we’ll want to cache the change and update the menus. This is pretty simple:

Private Sub edtTitle_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles edtTitle.TextChanged

playlistTitle = edtTitle.Text

FileChanged = True

ResetMenus()

End Sub

The ListBox

The listbox has only two events that we need to handle. The first is just handling whenever the selection changes – we’ll have to update the buttons in such a case:

Private Sub ListBox1_SelectedIndexChanged(ByVal sender As Object, _

ByVal e As System.EventArgs) Handles ShuffleListBox.SelectedIndexChanged

ResetButtons()

End Sub

The other event is the DrawItem event, which is called whenever a listbox element needs to be draw. Normally, this event is handled by the base class, but for this application, we’ll handle it so that we can decorate items which are linked. (We set this up by setting the DrawMode property of the listbox to OwnerDrawFixed.) Now, it even gets called when the listbox is empty, in order to draw the focus rectangle or whatever else should go into an empty list. Drawing isn’t too difficult. First, I’ll let it draw the background as usual:

Private Sub ShuffleListBox_DrawItem(ByVal sender As Object, _

ByVal e As System.Windows.Forms.DrawItemEventArgs) _

Handles ShuffleListBox.DrawItem

e.DrawBackground()

Then, assuming that the index is non-zero and we actually have some text, we’ll check to see if the item is linked. Anything linked with be written in bold, with the header node written in green, the tail node written in red, and anything in-between written in blue. The default, however, will be black text:

If e.Index >= 0 Then

Dim myBrush As Brush = Brushes.Black

which we will use if there are no links involved:

If elem.NextLinkedElement Is Nothing AndAlso elem.PrevLinkedElement Is Nothing Then

e.Graphics.DrawString(ShuffleListBox.Items(e.Index).ToString(), _

e.Font, myBrush, e.Bounds, StringFormat.GenericDefault)

but if there are links, we’ll have to use a bold font:

Else

Dim linkFont As Font = New Font(e.Font.Name, e.Font.Size, _

FontStyle.Bold, e.Font.Unit)

and set the colors appropriately:

If elem.NextLinkedElement IsNot Nothing _

AndAlso elem.PrevLinkedElement Is Nothing Then

myBrush = Brushes.Green ' Starting items are green

ElseIf elem.NextLinkedElement IsNot Nothing _

AndAlso elem.PrevLinkedElement IsNot Nothing Then

myBrush = Brushes.Blue ' Middle items are blue

ElseIf elem.NextLinkedElement Is Nothing _

AndAlso elem.PrevLinkedElement IsNot Nothing Then

myBrush = Brushes.Red ' End items are red

End If

e.Graphics.DrawString(ShuffleListBox.Items(e.Index).ToString(), _

linkFont, myBrush, e.Bounds, StringFormat.GenericDefault)

End If

Finally, regardless of what we drew, we’ll need to draw the focus rectangle (and dotted line around any selections:

e.DrawFocusRectangle()

End Sub

The Link and Unlink buttons

I’ve been chattering a lot about linked files, but without walking through the process of actually linking them, it may be hard to see how they work in practice. When Link is chosen, we’ll take all of the selected files, move them together, and point them at each other as a doubly-linked list. We can then use the existence of these links to change the behavior the program when drawing the items (as we have already done), shuffling them, or even saving them (more on that later).

To link the files, first we need to know how many:

Private Sub LinkBtn_Click(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles LinkBtn.Click

Dim numToLink As Integer = ShuffleListBox.SelectedItems.Count

Then, we need to iterate through each of those and link them.:

For index As Integer = 0 To numToLink - 1

Dim elem As xmlMediaEntry = ShuffleListBox.SelectedItems(index)

elem.NextLinkedElement = If(index <> numToLink - 1,_

CType(ShuffleListBox.SelectedItems(index + 1), xmlMediaEntry), _

Nothing)

elem.PrevLinkedElement = If(index <> 0, _

CType(ShuffleListBox.SelectedItems(index - 1), _

xmlMediaEntry), Nothing)

Note that I am using the If() ternary function above, which takes three arguments. The first is the condition to test, the second is code to run if the condition is true, and the third is code to run if the condition is false. In this case, I’m checking to see if we are at the first or last of the selected items, and linking accordingly. Since header nodes shouldn’t have a “previous” and tail nodes shouldn’t have a “next”, I set these to Nothing under those conditions – otherwise, they are set to reference the adjacent node in the appropriate direction.

Now, I want to physically move the linked tracks together so that they will actually play together. I’ll get the first one where it is, and move the others up right behind it. I do this by removing them from the listbox and adding them back in at the right spot, and then remind the listbox that they should still be selected:

If index <> 0 Then

ShuffleListBox.Items.RemoveAt(ShuffleListBox.SelectedIndices(index))

ShuffleListBox.Items.Insert(ShuffleListBox.SelectedIndices(index - 1) + 1, elem)

ShuffleListBox.SetSelected(ShuffleListBox.SelectedIndices(index - 1) + 1, True)

End If

Be careful here; we are making two assumptions in all of this. First, we will be assuming that the list of selected indices is ordered from lowest to highest, and also that the list of selected objects is in the same order as the selected indices (although we could work around the latter point). These are safe assumptions to make, as near as I can tell. Second, we need to remember that the contents of SelectedIndices are indices into the Items() collection – that is, the contents of the 0^th index of the SelectedIndices might refer to the 4^th entry of Items() – that can get confusing.

Anyway, once we’ve moved them together and linked them, we need to tell the listbox to repaint so that it redraws the linked items appropriately. To do this, we invalidate the control, and then tell it to update the invalidated regions:

ShuffleListBox.Invalidate()

ShuffleListBox.Update()

Finally, we need to update the buttons and menus, because we have changed the playlist and we are in a condition where we could unlink now:

FileChanged = True

ResetMenus()

ResetButtons()

End Sub

Unlink is sort of the opposite of Link, as you might expect, but there is a catch – we might not have the entire chain in the selection. This means that we’ll have to navigate back to the head node for a chain before navigating through and unlinking. We’ll use a helper function to do this; the unlink handler simply checks each selected item to see if it is part of a chain and calls the helper function if so, then invalidates/repaints/updates:

Private Sub UnlinkBtn_Click(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles UnlinkBtn.Click

For Each item As xmlMediaEntry In ShuffleListBox.SelectedItems

If item.PrevLinkedElement IsNot Nothing _

OrElse item.NextLinkedElement IsNot Nothing Then

FlushLinkSetFromBeginningToEnd(item)

End If

ShuffleListBox.Invalidate()

ShuffleListBox.Update()

FileChanged = True

ResetMenus()

ResetButtons()

End Sub

Note that FlushLinkSetFromBeginning to End will clean out the whole linked chain for that item, so even if other members of that chain were part of the selection, it won’t be called a second time when those members are hit in the loop – they’ll already have had their links cleaned out. To prove that, here’s the code, which is identical to the link-crawling code we all learned in college (if we’re that old yet J):

Private Sub FlushLinkSetFromBeginningToEnd(ByVal elem As xmlMediaEntry)

Dim current As xmlMediaEntry = elem

' Rewind to the head of the linkage

While current.PrevLinkedElement IsNot Nothing

current = current.PrevLinkedElement

End While

' Now destroy the links in this chain, front to back:

Do While current.NextLinkedElement IsNot Nothing

current.PrevLinkedElement = Nothing

current = current.NextLinkedElement

current.PrevLinkedElement.NextLinkedElement = Nothing

Loop

current.PrevLinkedElement = Nothing

End Sub

Move Up and Move Down buttons

After moving the items during the Link, this code will look pretty tame. We’ll create a helper function to do the removal in either direction. We’ll verify that the helper function is told to move up or down exactly one, and return without doing anything otherwise:

Private Sub MoveEntry(ByVal direction As Integer)

If direction <> 1 AndAlso direction <> -1 Then Return

Now we’ll get the item and see if it’s linked. Moving a linked item will put the links out-of-sync with the playlist order, so we will flush the links out, since the assumption is that it was a bad chain to begin with:

Dim index As Integer = ShuffleListBox.SelectedIndices(0)

Dim item As xmlMediaEntry = ShuffleListBox.Items(index)

If item.PrevLinkedElement IsNot Nothing OrElse item.NextLinkedElement IsNot Nothing Then

FlushLinkSetFromBeginningToEnd(item)

End If

Then we remove and insert the item appropriately, keep it selected, and do the usual updating:

ShuffleListBox.Items.RemoveAt(index)

ShuffleListBox.Items.Insert(index + direction, item)

ShuffleListBox.SetSelected(index + direction, True)

ShuffleListBox.Invalidate()

ShuffleListBox.Update()

FileChanged = True

ResetMenus()

ResetButtons()

End Sub

We can then call this helper function from the event handlers, specifying -1 for “up” and +1 for “down”:

Private Sub MoveUpBtn_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MoveUpBtn.Click

MoveEntry(-1) ' Move in the negative direction

End Sub

Private Sub MoveDownBtn_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MoveDownBtn.Click

MoveEntry(1) ' Move in the positive direction

End Sub

The Shuffle Button

If you’ve read the posts that I did on my earlier attempts to shuffle a playlist, then this should be pretty familiar territory – we’ll create a new list, do an insertion from the first list to a random place in the second list (bringing along linked items), and then point the playlist to the new list. The big difference in this case is that we won’t be removing items from the first list until we’re all done. Why? Because the listbox (i.e., the original list) is an active control, and we don’t want it to have it flashing at the user during the shuffle.

First,we’ll create the storage list into which we will insert, and get the number of songs to move:

Private Sub Shuffle_Click(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles ShuffleBtn.Click

Dim newList As New List(Of xmlMediaEntry)

Dim numberOfSongs As Integer = ShuffleListBox.Items.Count

Next, we walk through the original list and start moving things to random locations in the new list. There are two cases: the item is a header node for a linked set, or it is not. If it is a header node, we’ll move all of the associated files (middle and tail nodes) together with it, and increment the loop counter appropriately past all of those:

For index As Integer = 0 To numberOfSongs - 1

Dim mediaItem As xmlMediaEntry = ShuffleListBox.Items(index)

If mediaItem.NextLinkedElement IsNot Nothing Then

Dim placeToInsert As Integer = FindInsertionLocation(newList)

Dim count = 0

Insert(placeToInsert + count, mediaItem, newList)

count += 1

mediaItem = mediaItem.NextLinkedElement

Loop While mediaItem IsNot Nothing

index += count

(FindInsertionLocation() and Insert() are helper functions; we’ll get to those in a second.) In the normal unlinked case, we just insert the file to a random location in the new list:

Else

Insert(mediaItem, newList)

End If

(Again, bear with me – I’ll describe Insert() in a bit.) The new list is ready to go now, so we’ll flush all of the old items all at once using Clear(), and then reinsert the items in the new order.

ShuffleListBox.Items.Clear()

For Each m In newList

ShuffleListBox.Items.Add(m)

' This is a playlist change, so react accordingly:

FileChanged = True

ResetMenus()

ResetButtons()

End Sub

(You could also simply remove and insert items to random location in the listbox without creating a second list – that would save some memory, but not enough to counter the elegance of operating beginning-to-end with minimal flickering in the listbox. If you are interested in that sort of randomization, check out the shuffle routine in my Euchre game series of posts.)

Now, to our helper functions: FindInsertionPoint needs to find a location in the new list that isn’t in the middle of a set of linked songs. We’ll do this in a simple way: generatea number, and if it’s in the middle of a list, then just rewind and use the location before the chain of songs. Of course, if the list is empty, we’d insert at 0. This is fundamentally identical to the version of this function from my earlier posts, so I won’t go into too much detail:

Private Function FindInsertionLocation(ByVal newList As List(Of xmlMediaEntry)) As Integer

If newList.Count = 0 Then

Return 0

Else

Dim placeToInsert As Integer = _

Math.Truncate(Microsoft.VisualBasic.Rnd() * (newList.Count + 1))

If placeToInsert = newList.Count Then Return placeToInsert ' Insert at end

Dim mediaItem As xmlMediaEntry = newList.Item(placeToInsert)

Dim itemEntry As xmlMediaEntry = mediaItem.PrevLinkedElement

If itemEntry IsNot Nothing Then

While itemEntry IsNot Nothing

itemEntry = itemEntry.PrevLinkedElement

placeToInsert -= 1

End While

Return If(placeToInsert >= 0, placeToInsert, 0)

End If

Return placeToInsert

End If

End Function

Insert() just wraps a normal insertion by checking for the condition where the index is at the end of the list, so that Add is used instead in that case:

Private Sub Insert(ByVal place As Integer, ByVal mediaItem As xmlMediaEntry, _

ByVal newList As List(Of xmlMediaEntry))

If place = newList.Count Then

newList.Add(mediaItem)

Else

newList.Insert(place, mediaItem)

End If

End Sub

And its overload (used in the simple file case) just calls FindInsertionLocation to get an index, then calls into its sibling:

Private Sub Insert(ByVal mediaItem As xmlMediaEntry, ByVal newList As List(Of xmlMediaEntry))

Dim place As Integer = FindInsertionLocation(newList)

Insert(place, mediaItem, newList)

End Sub

Persisting Linkage Information from Session to Session

We’re almost done now. Using what we’ve got, we can load in a playlist, link music items, shuffle it, and save it out. The linkage information, however, won’t persist – the next time we load in the playlist, it will be gone. In order to preserve this information so that the PC or Zune won’t stomp on it, we’ll avoid using metadata and instead we’ll persist a second hidden file which keeps this information handy. The information will be stored as XML in this format:

<body>

<mlink>

<media…/>

</mlink>

<mlink>

<media…/>

</mlink>

</body>

So, each <mlink> tag set will contains an arc of linked songs in sequential order.

We’ll save this information to a hidden file with a name identical to that of the playlist, but with the suffix “.links” added to it. After loading the link file after the playlist file, we’ll use this information to relink the files in the listbox. If Zune has changed the corresponding playlist (i.e., if the user has removed, added, or rearranged songs from within Zune), that’s OK; we’ll ignore any songs that we can’t identify.

First, let’s concentrate on saving the information. We’ll create a trivial helper function to create the hidden file’s name:

Private Function GetHiddenFileName() As String

Return FilePath & ".links"

End Function

Now, in EmitXML, we’ll add the following code at the end of the method:

Dim hiddenLinkInfo = <?xml version="1.0"?>

<body>

<%= GetLinkInfo() %>

</body>

This is similar to the other emit code – I create a schema for my XML document, but defer to a helper function to fill in the actual XML elements. Next, I need to determine if a previous version of the hidden file already exists. If it does, I’ll need to un-hide and delete it:

Dim hiddenPath As String = GetHiddenFileName()

If My.Computer.FileSystem.FileExists(hiddenPath) Then

File.SetAttributes(hiddenPath, FileAttributes.Normal)

My.Computer.FileSystem.DeleteFile(hiddenPath, FileIO.UIOption.OnlyErrorDialogs, FileIO.RecycleOption.DeletePermanently)

End If

Finally, I’ll save it and hide it:

' Save it and hide it.

hiddenLinkInfo.Save(hiddenPath)

File.SetAttributes(hiddenPath, File.GetAttributes(hiddenPath) Or FileAttributes.Hidden)

Next, I’ll create the helper function. To show how nicely this all nests, I’ll use one helper function for the <mlink> tags, and then one for the media items within those tags. Here’s GetLinkInfo:

Private Function GetLinkInfo() As IEnumerable(Of XElement)

Dim linkInfos As New List(Of XElement)

For Each m As xmlMediaEntry In ShuffleListBox.Items

If m.NextLinkedElement IsNot Nothing And m.PrevLinkedElement Is Nothing Then

Dim linkInfo = <mlink><%= GetLinkSet(m) %></mlink>

linkInfos.Add(linkInfo)

End If

Return linkInfos

End Function

As you can see, the code iterates through the listbox and finds the next header node for a linked arc of songs. It generates a <mlink> tag for the set, and then uses yet another helper function to get eh media items, called GetLinkSet, which does a standard link crawl and returns the XELements in a list:

Private Function GetLinkSet(ByVal m As xmlMediaEntry) As IEnumerable(Of XElement)

Dim linkSet As New List(Of XElement)

linkSet.Add(m.xmlElem)

m = m.NextLinkedElement

Loop While m IsNot Nothing

Return linkSet

End Function

I really like this solution; it is extremely elegant! But now we have to load it all back in when opening a playlist, so we add the following code to Open() right before the FileChanged = False line:

Try

Dim hiddenPath As String = GetHiddenFileName()

Dim xmlLinks As XElement = XElement.Load(hiddenPath)

Dim linkSets = From links In xmlLinks...<mlink> _

Select links

That opens the hidden file and gets the all of the <mlink> elemets. Next, for each of the <mlink> elements, we’ll retrieve the media elements and see if they exist in the listbox that we just populated earlier in Open(). If they do, we can throw them into a list to link up again as if the user had selected them. If we can’t find a particular song, we just skip it and work with whatever we’ve got.

For Each linkSet In linkSets

Dim linkedFiles = From media In linkSet.<media> _

Select media

Dim listOfElements As New List(Of Integer)

For Each m In linkedFiles

Dim entry As Integer = ShuffleListBox.FindStringExact(xmlMediaEntry.CreateMediaString(m))

If entry <> -1 Then

listOfElements.Add(entry)

End If

Link(listOfElements)

(Link() is a helper function which I’ll describe in a second.) Note that we are using the publically shared method of getting a file’s identifier from its title and artist. That should hopefully be enough to identify the song uniquely – if not, we can change CreateMediaString to return some other value, bearing in mind that whatever we choose will show up in the listbox. (Or, we can slowly step through the listbox and look for an exact match on all of a song’s attributes, but that’s not much fun…)

If we can’t open the hidden file, it might not exist, so we’ll just ignore that condition:

Catch ex As Exception

End Try

The final thing we need to do is define the Link() helper function. It’s very similar to the code we wrote to link selected items, except of course the items are not selected, and forcing a temporary multi-selection just for the purposes of linking would be an ugly hack. The biggest difference is that the items are drawn from a list of indices that we created in Open, we don’t have a SelectedItems list to help us, and we don’t need to set any file state – that’s done by the “Open” command. Otherwise, it’s very similar, and if we worked hard enough at it, we could probably combine the two functions (I’m feeling lazy):

Private Sub Link(ByVal linkList As List(Of Integer))

Dim numToLink As Integer = linkList.Count

For index As Integer = 0 To numToLink - 1

Dim elem As xmlMediaEntry = ShuffleListBox.Items(linkList(index))

elem.NextLinkedElement = If(index <> numToLink - 1, _

CType(ShuffleListBox.Items(linkList(index + 1)), xmlMediaEntry), _

Nothing)

elem.PrevLinkedElement = If(index <> 0, _

CType(ShuffleListBox.Items(linkList(index - 1)), xmlMediaEntry), _

Nothing)

If index <> 0 Then

ShuffleListBox.Items.RemoveAt(linkList(index))

ShuffleListBox.Items.Insert(linkList(index - 1) + 1, elem)

End If

ShuffleListBox.Invalidate()

ShuffleListBox.Update()

End Sub

And that’s it! As usual, the full code can be found on my Temple of VB site.

‘Til next time,

--Matt--*

Linking Zune media items with LinQ, Part 1 (Matt Gertz)

VBTeam — Fri, 06 Nov 2009 22:17:00 GMT

Way back in October 2007, I wrote up a few posts (here and here) on my experiments with the Windows Media Player object model. The problem I was trying to solve was that, when I had a playlist set to “shuffle,” WMP would break up songs that should always play together (for example, Jackson Browne’s “The Load-Out” and “Stay”, or Pink Floyd’s “Brain Damage” and “Total Eclipse”). In those posts, I worked around this by building my own shuffler which would emit a new randomized playlist which preserved those links, and then playing that playlist. A few months later, after I bought my Zune 80, I updated the program to copy the playlist to a ZPL format, since the format was essentially identical to WPL.

Although this was a good investigation into the WMP object model, I’ve never been happy with the program. First of all, for the program to work, I had to add metadata to the actual music tracks, and anything could come along and blow that metadata away. Second, the tracks had to already be adjacent in the parent playlist for it to work at all. Third, I had to bring up the WMP just to shuffle a playlist, which is pretty heavyweight. Finally, since I mostly care about Zune playlists these days and not WMP at all, it seemed the height of hackery to bring up the WMP to generate something for Zune (since the Zune object model is not public).

So, last week, after seeing my metadata blown away again when I carelessly updated my tracks after upgrading to Windows 7, I decided to work on a more permanent solution. Why bring up the WMP at all (or Zune, for that matter), when the playlists themselves are just XML, and the track just entries into those?

After some experiments with parsing the XML by following the parent/child chain, which seemed kludge to me, I came across a post by Avner in which he discusses porting iTunes playlists to Zune, leveraging LinQ to help with the data parsing. I realized right away that I could do something very similar in this case, and came up with what I think is a pretty elegant solution.

So here’s the plan: the application will:

(1) Load in a playlist that the user chooses

(2) Allow the user to rearrange the playlist

(3) Allow the user to link certain tracks together (and unlink them)

(4) Shuffle the playlist on command, preserving the linkages

(5) Allow the user to change the title of the playlist

(6) Allow the user to cave the playlist (or a copy of it)

(7) And, most important, persist the linkage data between sessions in such a way that the information won’t get blown away.

Let’s get started!

The form

Create a Windows Form Application (about 430 x 300) containing the following:

(1) A ListBox, about 290 x 180 and positioned on the left side, with the SelectionMode property set to “MultiExtended” and the DrawMode set to “OwnerDrawFixed.”

(2) Above that, a Label and EditBox, with the Label text set to “&Title”.

(3) To the right of those, five Buttons going down the right side, labeled “Link”, “Unlink”, “Move Up”, “Move Down”, and “Shuffle”.

(4) A MenuStrip. Select it, and on the right side of the strip, click the tiny right-pointing triangle icon and choose “Insert Standard Items” from the resulting popup. Now, go back and remove all of the items except File (Open, Close, Save, Save As, Exit) and Help (About). The Edit and Tools menu can be completely removed; we won’t be using those.

(5) An OpenFileDialog and a SaveFileDialog. The Filter property of both should be set to “Zune playlists|*.zpl” (without the quotes).

In my example, I also disabled the Maximize box in the form, added an icon and title to the form and application, etc., but those aren’t strictly necessary for the exercise.

The Zune playlist

The format of a Zune playlist (version 2.0) looks like this (memorize this, it will be useful later):

<smil>

<head>

<title> (some title) </title>

</head>

<body>

<seq>

<media … /> (etc… one for each track)

</seq>

</body>

</smil>

Code for the controls

The rest of this post will be organized based on the controls, starting from the top.

The Form

The only code associated with the form itself is the event handler for Load, a few member variables, and a helper class. Let’s take those in reverse order:

The helper class, xmlMediaEntry

We’ll be reading the Zune playlist using the XDocument class, and then we’ll be using LinQ to retrieve individual <media>…</media> tag pairs representing each track. These will be handed to us as XElements, and as we will want to add each of the tracks to the listbox, we’ll want to wrap these in an object that will expose the proper information to the ListBox – namely, a human-readable string. We will also want to have a way to link a track to another track, and we’ll be using a doubly-linked list to do this. So:

Class xmlMediaEntry

Public xmlElem As XElement

Public PrevLinkedElement As xmlMediaEntry = Nothing

Public NextLinkedElement As xmlMediaEntry = Nothing

That code allows us to cache the XElement and also point to other elements of this class backwards and forward. The header node for a chain of linked songs will have PrevLinkedElement remaining as Nothing; the tail node will have NextLinkedElement remain as Nothing; songs in the middle of the chain will have both of those set to some other node.

The following constructor allows us to initialize the object with the XElement to be cached.

Sub New(ByVal xelem As XElement)

xmlElem = xelem

End Sub

This code, exposed publically as a shared method, allows us to create a canonical (and readable) way to refer to a track in an XElement. To use one of my previous examples, the resulting value might be “The Load-Out (Jackson Browne)”:

Public Shared Function CreateMediaString(ByVal elem As XElement) As String

Return elem.Attribute("trackTitle").Value & _

" (" & elem.Attribute("trackArtist").Value & ")"

End Function

I can then leverage that function to override the ToString for the xmlMediaEntry:

Public Overrides Function ToString() As String

Return CreateMediaString(xmlElem)

End Function

End Class

And that’s all I need in that class. Once I add an instance of it to the ListBox, it will show up with nice readable text, and I’ll be able to navigate between it and any other linked songs (if any).

Member Variables

I’ll need a few variables to keep track of the state of the file, and these should all be self-explanatory:

Private FileLoaded As Boolean = False ' Do we have a file loaded into the listbox?

Private FileChanged As Boolean = False ' Has the playlist changed since it was last loaded/saved?

Private FilePath As String ' Where was the file last loaded from/saved to?

Private FileDirectory As String ' What directory was the file last loaded from/saved to?

We’ll also need to keep track of some of the playlist’s metadata. Each playlist has a GUID associated with it (if you don’t know what a GUID is, don’t worry about it – think of it as a unique identifier); if we do a “Save As” to a different file, we’ll want a different GUID. The user might also change the title of the playlist, and we should cache our creatorId format since it might be unique to the user’s situation:

Private playlistGuid As String ' What is the GUID of this playlist?

Private playlistTitle As String ' What is the title of this playlist?

Private playlistCreatorId As XElement ' What is the creator information of this playlist?

Events

The Form only has two events to handle that we care about – the Load event, and the FormClosing event. For the Load event, we’ll want to make sure that our menu items and buttons are enabled appropriately, and that we appropriately generate random numbers later on:

Private Sub VBShuffleForm_Load(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles MyBase.Load

Microsoft.VisualBasic.Randomize()

ResetButtons()

ResetMenus()

End Sub

We will be using those helper functions quite a lot, so it’s worth going over them. For ResetMenus, the “Open” commands will always be available, so we’ll apply a little overkill and ensure that. “Save As” and “Close” will be available whenever a file is loaded (otherwise, we’d have nothing to save or close!), and “Save” will be available if a file is loaded and has been changed since the last time that it was saved or loaded:

Private Sub ResetMenus()

OpenToolStripMenuItem.Enabled = True

CloseToolStripMenuItem.Enabled = FileLoaded

SaveAsToolStripMenuItem.Enabled = FileLoaded

SaveToolStripMenuItem.Enabled = FileLoaded AndAlso FileChanged

edtTitle.Enabled = FileLoaded

End Sub

Somewhat more complicated is ResetButtons. Each button reacts to one of three situations – nothing is selected in the listbox, exactly one thing is selected in the listbox, or multiple things are selected in the listbox. The breakdown is as follows:

· “Link” is only available if multiple things are selected. You can’t link if nothing is selected, and you can’t link a file to itself. It’s also OK to link files that are already linked, since you might be adding more files to the chain of songs.

· “Unlink” is available if at least one item in the selection has a link. You don’t have to select an entire link set to unlink it; it’s sufficient just to select one of the members of the link chain. If the selection contains elements from multiple link chains, then all of the associated chains will be unlinked. We can examine the “PrevLinkedElement” and “NextLinkedElement” values to determine if an item has a link.

· “Move Up” or “Move Down” moves one song, so they are only available if exactly one song is selected.

· “Shuffle” is available whenever a file is loaded.

So:

Private Sub ResetButtons()

If ShuffleListBox.SelectedIndices.Count = 0 Then

LinkBtn.Enabled = False

UnlinkBtn.Enabled = False

MoveDownBtn.Enabled = False

MoveUpBtn.Enabled = False

ElseIf ShuffleListBox.SelectedIndices.Count = 1 Then

LinkBtn.Enabled = False

MoveDownBtn.Enabled = (ShuffleListBox.SelectedIndices(0) <> _

ShuffleListBox.Items.Count() - 1)

MoveUpBtn.Enabled = (ShuffleListBox.SelectedIndices(0) <> 0)

UnlinkBtn.Enabled = (CType(ShuffleListBox.SelectedItems(0), _

xmlMediaEntry).PrevLinkedElement IsNot Nothing OrElse _

CType(ShuffleListBox.SelectedItems(0), xmlMediaEntry).NextLinkedElement _

IsNot Nothing)

Else

LinkBtn.Enabled = True

MoveDownBtn.Enabled = False

MoveUpBtn.Enabled = False

Dim enableBn As Boolean = True

For Each item In ShuffleListBox.SelectedItems

If CType(item, xmlMediaEntry).PrevLinkedElement Is Nothing AndAlso _

CType(item, xmlMediaEntry).NextLinkedElement Is Nothing Then

enableBn = False

Exit For

End If

UnlinkBtn.Enabled = enableBn

End If

ShuffleBtn.Enabled = FileLoaded

End Sub

The FormClosing event handler needs to prompt the user if there are unsaved changes before closing the application. (Note that this is the event generated from the clicking the “X” on the form’s title bar – not the “Close” menu item! We’ll implement that other one later.) The user will have an opportunity to save his/her work before closing (if applicable), or else canceling the Close operation altogether.

First, we’ll define the code which determines if it’s OK to close:

Private Function OKToUnload() As Boolean

If Not FileLoaded OrElse Not FileChanged Then

Return True

End If

Dim result As MsgBoxResult = _

MsgBox("Do you want to save your changes?", _

MsgBoxStyle.YesNoCancel, "Save changes?")

If result = MsgBoxResult.Cancel Then

Return False

Else

If result = MsgBoxResult.Yes Then

Return SaveAs()

End If

Return True

End Function

If there is no file loaded, or if the file is loaded but hasn’t changed, then it’s of course OK to proceed, and we return True. Otherwise, we pop up a MessageBox prompting the user if they want to save their changes. If the user chooses “Cancel,” then we simply return False to indicate that it is not OK to proceed. If the user chooses “Yes,” then we call SaveAs() (which we’ll define later) and return whatever the results are from that call. That just leaves “No,” which means the user doesn’t want to save changes, so we return True indicating that it’s OK to proceed. (The strings for the message box should be stored in the resources so that they can be localized, of course, but that makes the blog less readable. J)

Now, we can use that helper function in the event handler for Close:

Private Sub VBShuffleForm_FormClosing(ByVal sender As Object, _

ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles Me.FormClosing

If Not OKToUnload() Then

e.Cancel = True

End If

End Sub

In other words, if it’s not OK to close the application, then we set e.Cancel to True indicating that the event should be canceled.

The Menu Strip

Now, we start getting into the meat of the code. Let’s go through these one by one:

FileàOpen

Open is pretty trivial to implement, as you can see:

Private Sub OpenToolStripMenuItem_Click(ByVal sender As Object, _

ByVal e As System.EventArgs) Handles OpenToolStripMenuItem.Click

Open()

End Sub

OK, I’m kidding. I moved all of the handling for this into a helper function, in case I ever wanted to open the file in some other way than the menu click. Here’s the actual implementation:

Private Function Open() As Boolean

Try

If Not OKToUnload() Then

Return False ' User canceled; just return.

End If

As you can see, we first leverage the “OKToUnload” helper function to make sure that the user doesn’t lose any work in progress. We’ll also wrap the functionality in a Try construct in case something goes wrong with the opening of the file.

Next, we’ll get the name of the playlist to be opened. If we’ve recently loaded from (or saved to) a directory, we’ll start out there; otherwise, we’ll default to the Music directory as our initial location with no particular file in mind to load:

If String.IsNullOrEmpty(FileDirectory) Then

ShuffleOpenFileDialog.InitialDirectory = My.Computer.FileSystem.SpecialDirectories.MyMusic

Else

ShuffleOpenFileDialog.InitialDirectory = FileDirectory

End If

ShuffleOpenFileDialog.FileName = ""

Dim result As DialogResult = ShuffleOpenFileDialog.ShowDialog

If result = Windows.Forms.DialogResult.Cancel Then

Return False ' User canceled; just return

End If

And we’ll need to cache that information for later usage. (I cache the directory separately, so that I can easily use it to initialize the Save dialog’s initial location later on):

FilePath = ShuffleOpenFileDialog.FileName

Dim fileData As FileInfo = My.Computer.FileSystem.GetFileInfo(FilePath)

FileDirectory = fileData.DirectoryName

Next, we’ll close any open playlist. We’ll use a helper function to do this:

ClosePlaylist()

Where ClosePlaylist() is defined to just flush out our caches, and the listbox:

Private Sub ClosePlaylist()

' Get rid of all items and clear the member variables.

ShuffleListBox.Items.Clear()

FileLoaded = False

FileChanged = False

edtTitle.Text = ""

playlistGuid = ""

playlistTitle = ""

playlistCreatorId = Nothing

End Sub

Finally, we get to the part where we load the file! As I mentioned before, we’ll be using XElements to store the items, and reading in the file in such a case is really easy:

Dim xmlElements As XElement = XElement.Load(FilePath)

If you’re debugging this, take a moment to look at the result in the watch window. You’ll see that this is a fairly deep structure that is difficult to plumb, and so digging though it to find the media items would be quite a chore. Fortunately, LinQ to XML is here to help us, and we can use it liberally. First, let’s get and cache the GUID, so that we can spit it out again if we save back to the same file:

Dim guids = From guidEntry In xmlElements...<guid> _

Select guidEntry

playlistGuid = guids(0).Value

Looks trivial, doesn’t it? But we just saved ourselves about twenty lines of code that we would have had to write to navigate the XElement structure. Basically, I told link to search for the GUID element, nested somewhere under the root of the document (hence the “…” before the <guid> tag) and return me a list (actually an IEnumeration) of all such tags. I happen to know that there’s only one in this schema, so I’ll cache the first (and only) one returned to me.

I can do something similar for the title:

Dim titles = From titleEntry In xmlElements...<title> _

Select titleEntry.Value

playlistTitle = titles(0)

edtTitle.Text = playlistTitle

But creatorId is tricker – it’s not a tag itself, but an attribute on another tag called “<meta>.” There are several other <meta> tags which could be in any order, so I can’t simply grad the first and be done with it. Fortunately, I can use “Where” and the “@” qualifiers to narrow down the field to just the one that I want:

' Get the creatorId metatag

Dim cIds = From metaEntry In xmlElements...<meta> _

Where metaEntry.@name = "creatorId" _

Select metaEntry

playlistCreatorId = cIds(0)

But it’s the media entries that we’re most interested in. We can retrieve those similar to the way we got the title and guid information:, but this time we’ll save the results to a list that we can iterate over. Once we have that list, we will create an xmlMediaEntry for each XElement in that list, and throw it into the ListBox:

Dim mediaEntries = From media In xmlElements...<media> _

Select media

Dim mediaList As List(Of XElement) = mediaEntries.ToList

For Each m In mediaList

ShuffleListBox.Items.Add(New xmlMediaEntry(m))

The files are loaded, so we can update our state appropriately and return a “success:”

FileChanged = False

FileLoaded = True

ResetButtons()

ResetMenus()

Return True

Of course, if something went wrong, we need to tell the user, and return a failure code:

Catch ex As Exception

MsgBox("Error opening file: " & ex.Message, MsgBoxStyle.OkOnly)

Return False ' Oops, problems loading the playlist file.

End Try

End Function

FileàSave and FileàSaveAs

Here’s the other half of the file work. As with Open, we’ll do all of the work in a helper function, which we’ll then be able to leverage in other conditions:

Private Sub SaveToolStripMenuItem_Click(ByVal sender As Object, _

ByVal e As System.EventArgs) Handles SaveToolStripMenuItem.Click

Save()

End Sub

This, in turn, will call another function to emit the XML, after which it will clear the “FileChanged” flag and reset the menus, catching any errors that happen:

Private Function Save() As Boolean

Try

EmitXML()

FileChanged = False

ResetMenus()

Return True

Catch ex As Exception

MsgBox("Error saving file: " & ex.Message, MsgBoxStyle.OkOnly)

Return False

End Try

End Function

EmitXML, as you might guess, is where all of the work happens. But the work itself is pretty simple. Rather than opening a file and spitting in XML tags line by line, we’ll leverage an XML literal to build up the document and then we’ll save it wholesale to the file:

Private Sub EmitXML()

Dim playlist = <?xml version="1.0"?>

<?zpl version="2.0"?><smil>

<head>

<%= playlistCreatorId %>

</head>

<body>

<seq>

<%= GetMediaElements() %>

</seq>

</body>

</smil>

playlist.Save(FilePath)

End Sub

How cool is that? The playlistGuid, playlistCreatorId, and playlistTitle will automatically be replaced by whatever we cached, and the media will be generated within the context of the XML by a helper function called GetMediaElements(), the form of which is very simple:

Private Function GetMediaElements() As IEnumerable(Of XElement)

Dim media As New List(Of XElement)

For Each m As xmlMediaEntry In ShuffleListBox.Items

media.Add(m.xmlElem)

Return media

End Function

In other words, we just iterate through the listbox, pull the XElement out of each item, and add it to a list which gets passed back to the XML. That’s it; that’s all there is to it.

Now, we can leverage this for the Save As command; first the event handler:

Private Sub SaveAsToolStripMenuItem_Click(ByVal sender As Object, _

ByVal e As System.EventArgs) Handles SaveAsToolStripMenuItem.Click

SaveAs()

End Sub

And then the helper function. First, we get the name of the file to save, setting the initial directory to what we cached earlier, and defaulting the filename to the name of the title (instead of the original filename, reasoning that, if the user was saving to the same file, they would have used “Save” and not “SaveAs”):

Private Function SaveAs() As Boolean

ShuffleSaveFileDialog.FileName = playlistTitle

ShuffleSaveFileDialog.InitialDirectory = FileDirectory

Dim result As DialogResult = ShuffleSaveFileDialog.ShowDialog()

If result = Windows.Forms.DialogResult.Cancel Then

Return False

End If

We should supply a new GUID if the filename is going to change:

If FilePath <> ShuffleSaveFileDialog.FileName Then

playlistGuid = "{" & System.Guid.NewGuid.ToString() & "}"

End If

Then we cache the name and directory:

FilePath = ShuffleSaveFileDialog.FileName

Dim fileData As FileInfo = My.Computer.FileSystem.GetFileInfo(FilePath)

FileDirectory = fileData.DirectoryName

And having done that, we can just call Save() to do the rest of the work:

Return Save()

End Function

That’s it for the file work – for now. We’ll revisit Open and Save later when we persist the linkage info.

FileàClose and File->Exit

In the case of Close, we’re closing the playlist, not the application; for Exit, we close the application as well. The code for this is pretty trivial, as we have already written both of the helper functions involved:

Private Sub CloseToolStripMenuItem_Click(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles CloseToolStripMenuItem.Click

If OKToUnload() Then

ClosePlaylist()

End If

End Sub

Private Sub ExitToolStripMenuItem_Click(ByVal sender As Object, _

ByVal e As System.EventArgs) Handles ExitToolStripMenuItem.Click

If OKToUnload() Then

Close()' This is Form.Close, of course.

End If

End Sub

HelpàAbout

This is pretty trivial to handle, and I’ve described it before in earlier posts, but here goes: right-click on the project, choose “Add New Item,” and add an About Dialog. Then, right-click the project again and choose “Properties.” In the resulting properties document, select the Application tab (if it isn’t already selected) and click the “Assembly Information…” button. Fill in the information however you prefer. Then, we can hook up the dialog to the command pretty easily as follows:

Private Sub AboutToolStripMenuItem_Click(ByVal sender As System.Object, _

ByVal e As System.EventArgs) Handles AboutToolStripMenuItem.Click

Dim dlg As New VBSmartShuffleAbout

dlg.ShowDialog()

End Sub

That’s all I’m going to cover in this post… it’s getting too long, and I’ve got nine more pages to go! I’ll post the next set in a separate post, and we’ll wrap this up then.

‘ Til next time,

--Matt--*

Getting Loopy (Matt Gertz)

VBTeam — Thu, 08 Oct 2009 00:50:00 GMT

In my last post, I talked about the hidden costs that can occur whenever you call out to methods, particularly in loops. In looking at my examples, reader KG2V commented that another thing that folks need to be aware of is avoiding the assumption that the world (or, in this case, a list) is a static thing. It’s a good point and it deserves some attention, particularly given that different languages will react in different ways to change.

To illustrate this point, let’s assume that we have a form with a ListBox on it, and that we also have a button on the form that, when clicked, runs the following VB code:

' Bad -- don't do this

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

For i As Integer = 0 To ListBox1.Items.Count - 1

ListBox1.Items.RemoveAt(i)

End Sub

Ostensibly, we want to remove all of the items from the ListBox. (Yes, normally we’d use Clear() to do that, but stay with me for a moment; I’ll switch to more “real” cases in a few paragraphs.) But if we run this code, we’ll get an exception. Why? Because whenever we remove an object, all of the indices of the objects after it will decrease by 1. Let’s assume that there are three objects in the list - A, B, and C – so that Count=3:

· When i = 0, we’ll remove the object at index 0, which is A. B now moves to index 0, and C now moves to index 1.

· When i = 1, we’ll remove the object at index 1, which is C. There is nothing after C, so nothing else changes.

· When i =2, we’ll remove the object at index 2, which is… nothing. There’s only one object left, which is B, and it’s at index 0. So, we get an out-of-bounds exception.

In VB (as was pointed out to me in the comments section of my last point), the value of Count is only assessed once, right when the loop is encountered, so we will always loop through (in this case) three times.

But is the lack of reevaluation of Count the only issue? Maybe not. Let’s try the same method in C#, in which Count is evaluated on every iteration:

// Bad -- don't do this.

private void button1_Click(object sender, EventArgs e)

{

int i;

for (i = 0; i < listBox1.Items.Count; i++)

{

listBox1.Items.RemoveAt(i);

}

· When i = 0, we’ll remove the object at index 0, which is A. B now moves to index 0, and C now moves to index 1.

o Now, we’ll loop back to the top. The value of i becomes 1, and the value of Count becomes 2. The stop condition is not met, so we keep going.

· When i = 1, we’ll remove the object at index 1, which is C. There is nothing after C, so nothing else changes.

o Again, we loop back to the top. The value of i becomes 2, and the value of Count becomes 1. The stop condition is met, and we exit the loop.

So, although no exception is thrown, the method also fails in C# -- B is left behind. So, the problem isn’t whether or not Count is evaluated; it’s that the elements in the list are changing indices on each deletion.

Sometimes doing things backwards is good

This is why, in the case where multiple removals need to be made on an indexed list, a smart programmer (all of whom are yawning right now as they read this) will work backwards. That is, they start with the largest index and work down towards zero. For example, imagine that we want to remove the 4^th, 7^th, and 9^th element from a list. We don’t want to do it this way:

' Bad -- don't do this

ListBox1.Items.RemoveAt(4)

ListBox1.Items.RemoveAt(7)

ListBox1.Items.RemoveAt(9)

Once we’ve removed the 4^th item, then the element at 7 will have moved to 6 and the element at 9 will have moved to 8. After we’ve removed the new 7^th item, the element at 8 will move to 7, and so on. We’d either be removing the wrong elements or we’d crash. It’s far better to do them in reverse order:

ListBox1.Items.RemoveAt(9)

ListBox1.Items.RemoveAt(7)

ListBox1.Items.RemoveAt(4)

None of the indices up front will be affected by changes that are occurring behind them in the list – their indices won’t change, and this will work fine.

This issue comes up when you are (for example) deleting items from a listbox that supports multiselection. You basically have two ways to get the selected items in a listbox: you can use the SelectedIndices member function, which returns a collection of indices, or you can use SelectedItems, which returns a collection of selected items. Given either collection, there’s no way to say “remove all of these.” There is a “Remove” command on the resulting collections, but if you use it, you’ll just be removing the item from *that* collection and not from the ListBox, which has its own collection of items – essentially, you’ll just deselect the items.

So, what do you do? At first, it looks like you have two options; you could try removing them from the master list based on item, or based on object. Let’s look at removal based on the item:

' Bad -- don't do this

For Each index As Integer In ListBox1.SelectedIndices

ListBox1.Items.RemoveAt(index)

This won’t work. The selected indices are in numerical order (no matter in which order you selected them), so you’ll be sliding indices on each deletion and will subsequently delete the wrong things. You won’t crash if you overreach on an index, because the For…Each loop will check for Nothing when it does a MoveNext internally and will exit gracefully, but this is actually worse since it may take longer before you notice a problem!

This won’t work either, and will potentially crash:

' Bad -- don't do this

For index As Integer = 0 To ListBox1.SelectedIndices.Count - 1

ListBox1.Items.RemoveAt(ListBox1.SelectedIndices(index))

Again, the indices for the downstream objects will decrease, you’ll delete the wrong things, and you’ll likely crash.

You can, however, iterate down instead, as long as you are confident that the selected indices are in numerical order (as they will be in this case):

' This works:

For index As Integer = ListBox1.SelectedIndices.Count - 1 To 0 Step -1

ListBox1.Items.RemoveAt(ListBox1.SelectedIndices(index))

Having found something that works via iteration, let’s now consider the enumeration angle instead. In general, I prefer to enumerate lists whenever possible instead of iterating through them. This is not one of those times, though. Consider this code, which will fail:

' Bad -- don't do this

For Each item In ListBox1.SelectedItems

ListBox1.Items.Remove(item)

“Well, c’mon, why doesn’t this work?” I hear you cry. “I don’t care about indices in this case, I’m not even using them; I’m just removing specific items.” True, but if you run this code, you will indeed get an exception essentially stating that the collection has been modified and the list of selected items is no longer valid for enumeration. We can’t work around this by assigning the collection of selected items to another variable and enumerating on that, since that just creates a reference to the same collection. In order for it to work, you’ve have to get the references out of the collection itself so that the ListBox is no longer part of the equation on the removal loop’s assessment.

' Really, this is just silly -- don't ever do this

Dim selItems As New Collection

For Each item In ListBox1.SelectedItems

selItems.Add(item)

For Each item In selItems

ListBox1.Items.Remove(item)

Functionally, this will work; however, you shouldn’t do it for two reasons:

· First of all, it’s stupid – you have to loop through twice. Occasionally, I do need to clone lists, like when creating an editable list from a read-only list – I’ve done that in a previous blogpost on this site, in fact. But my motives are not so noble here; I’m using this for no other reason than allowing enumeration to work.

· Second, each call to RemoveItem has a hidden cost – namely, the list has got to go find that item. In this case, you can bet that there’s good hashing going on that doesn’t make that as painful as it might be, but do you really want to count on that, given the better numeric alternative above?

No, just there’s really no good way to do removal by enumeration that I’ve ever found. And since each object is totally clueless about whether it itself is selected or not, you can’t even enumerate through the collection yourself and delete it if it’s marked as selected.

Of course, I’ve just talked about deletions, since they are more interesting than additions due to the crash potential. However, additions can also mess you up in a loop due to similar index changes, and I’ve actually run into cases where a list may have both insertions and deletions done, so that I wasn’t even able to leverage the change in list size as a clue to figure out what had happened to the data. The bottom line is, whenever doing an operation within a loop, you need to understand whether or not that operation will collide with your loop conditions and assumptions. Generally, you control this in your code, but in multithreaded environments, where changes to an existing store of data might be made from a different thread “simultaneously” with your thread, you really need to understand how thread-safe your code is with respect to data changes. (But that is a story for another time.)

‘Til next time,

--Matt--*

Hidden Costs (Matt Gertz)

VBTeam — Tue, 06 Oct 2009 22:54:00 GMT

(Note: there was a grievous error in this post based on a bad assumption on my art regarding VB. Not feeling the need to hide my ignorance :-), I have instead made a number of edits in this post to bring it back to some semblance of reality.)

One thing that gets me annoyed with myself is realizing that the product or service I’ve just bought has some hidden costs that I didn’t anticipate. It might be as complicated as realizing that my plane ticket has all sorts of byzantine surcharges and luggage costs, or as simple as making budgeting for a new killer gaming PC and forgetting about the sales tax – it seems that there are always extra costs out there that will come back to bite you. Being (as I have said in past postings to this blog) an inherently lazy person, and being fully caught up in our “I must have the sparkly thing right now” culture, I’m apt to overlook the fine print in any purchase agreement unless I really force myself to slow down and pay attention to the details.

One of the times I really have to pay attention is when I’m reviewing code here at Microsoft. As a member of the release team that makes the final decisions as to what last-minute fixes are safe enough to be applied to an imminent release, it’s my job to take a look at the proposed fix and make sure that it doesn’t introduce more problems than it purports to solve – i.e., that it doesn’t add hidden costs.

On the rare occasion that I find a problem in someone’s code, it’s usually something that’s obviously dysfunctional – a case where a variable isn’t initialized properly, or where a different team has simultaneously made a change that will render this change dangerous. But some of the problems are more subtle, and might not be obvious if only minimal testing and review has been done. Consider the following code (very similar to a case I reviewed here at work the other day):

Public MyList As New List(Of String)

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click

Dim s As New System.Text.StringBuilder

For i As Integer = 0 To MyList.Count - 1

s.Append(MyList(i))

End Sub

Edit: This is an error in my blog post – I hadn’t realized that VB, unlike many other languages, only gets the value of the stop condition once (thanks to Phil Wells for pointing this out). To make my point, I would have to be using code like this:

Dim s As New System.Text.StringBuilder

Dim i As Integer = 0

Do While i < MyList.Count() - 1

s.Append(MyList(i))

i += 1

Loop

The method Count() might still have a hidden cost, but in the For Loop case, I’m at least only hitting it once. The fact that VB only analyzes the stop condition once leads to a different issue to be aware of that I’ll cover in my next blog.

Seems pretty simple, yes? When a button is clicked, the code iterates through the strings in a list and appends them to a StringBuilder, creating (for whatever reason) one long string containing all of the contents. And, to cursory appearances, it will function correctly.

However, the code has hidden costs. You see, written this way, the value MyList.Count is going to be reevaluated on every iteration of the “For” loop, even though the value itself isn’t changing. Furthermore, my indexing into MyList, which is always with reference to the beginning of the list, will also have a non-zero impact. This impacts the performance of the code and will make my customers unhappy over the sloth-like operation of my application.

In this case, the cost isn’t really high, as Count() is a property wrapping something that’s stored as a private integer member, and index referencing also has tricks to speed it up, but there’s still a cost and it can add up. It’s even worse if the Count() call has to be calculated! Consider this really awful code where I’ve created my own “List” type:

Public Class MyLameObject

' For some unknown reason, I'm creating my own linked list instead of using a List(Of) generic.

' Please don't do this for real!

Public data As String

Public NextMLO As MyLameObject

Public Sub Insert(ByVal mlo As MyLameObject)

If NextMLO IsNot Nothing Then

NextMLO = mlo

Else

mlo.NextMLO = NextMLO.NextMLO

NextMLO.NextMLO = mlo

End If

End Sub

Public Function Count() As Integer

Dim ct As Integer = 0

Dim current As MyLameObject = Me

Do While current IsNot Nothing

ct += 1

current = current.NextMLO

Loop

Return ct

End Function

Public Function Item(ByVal index As Integer) As MyLameObject

If index >= Count() OrElse index < 0 Then Return Nothing

Dim current As MyLameObject = Me

For i As Integer = 0 To index - 1

current = current.NextMLO

Return current

End Function

End Class

Public MyLameObjectHeader As New MyLameObject

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) _

Handles Button1.Click

Dim s As New System.Text.StringBuilder

For i As Integer = 0 To MyLameObjectHeader.Count - 1

s.Append(MyLameObjectHeader.Item(i))

End Sub

Perhaps as a school assignment, I’ve created a singly-linked list which contains objects of type “MyLameObject” (instead of using optimized collections that we provide). I’ve also created a member function called “Count” which returns the number of objects in my list (starting from the current node) and another member function called “Item” which returns me a reference to a specific object in the list, as indexed from the current node. The code will compile and it will “work.” But, oh! It has so many hidden costs, and if the definition of MyLameObject exists in an assembly for which I don’t have the source code, I might not even realize it!

First, let’s look at button click handler. It’s calling Count() on every single iteration, and there will be a number of iterations equal to the value returned from Count(). Count(), in turn, walks the entire list every time it’s called. So, that means that if my list contains 1000 items, that means that I’ll be referencing a MyLameObject one million times! (This is what they refer to as an “O(n²) problem” in computer science classes.)

But wait, it gets worse: ~~the code for Item() calls Count() also on each iteration to verify that the index is not out of bounds – that brings us up to a count of 1001000 MyLameObject references~~.

Edit: As noted above, that isn’t true for VB, though it would be for many other languages

And, of course, Item() itself needs to walk the list to actual get to the desired item, on every iteration – that adds 500500 more object hits (using the n(n+1)/2 formula) , for a grand total of ~~1501500~~ 1500500 object hits. And all that just to concatenate 1000 strings! That, I fear, is going to slow down my code’s performance somewhat.

Now, of course I could add code to get track of values for Count(), and that would reduce my cost to a third of what it was. I could use some clever hashing algorithm to more easily get to the items while still being able to use method calls. I could enumerate the list myself instead of relying on “Count” or on “Item,” and that would really cut my costs down dramatically. Or, best of all, I could avoid reinventing the wheel and using the classes that VB.NET already supplies for me, since we do our best to optimize them.

But that’s not the point that I’m trying to make here. The point is, whenever you call a function, you are at its mercy performance-wise – you are stuck with whatever hidden costs it contains. You might not even be able to determine what these hidden costs are if you don’t own the code you’re calling, except by experimentation. And if you call that function iteratively, you’re just compounding the problem.

Now, you can’t avoid making calls into code you don’t own – at some level, we all use libraries or assemblies that are handed to us. But you can be smart about how you use them~~. Consider *this* code:~~

~~Dim s As New System.Text.StringBuilder~~

~~Dim count As Integer = MyList.Count~~

~~For i As Integer = 0 To count - 1~~

~~s.Append(MyList(i))~~

~~Next~~

~~Now, I’m only calling Count() once, so I minimize the impact of its cost! Or,~~

Edit: As above, this doesn’t actually make a difference in VB. Of course, you still have the cost of “Count(),” whatever it may be. In this case, it’s just a property read; if it’s some home-grown collection or data repository, Count() (or it’s analog) could be a lot more costly.

Consider this even better example that you can use on collections that support IEnumerable (as all System.Collections types do):

Dim s As New System.Text.StringBuilder

For Each data As String In MyList

s.Append(data)

Yeah! Now I’m firing on all cylinders! Now I don’t have to worry about indexing costs either – I’m just walking the list, collecting my strings in the fastest way possible. Really, the only cost I have to deal with is the cost of actually appending the data (which is in fact why I’m using the highly-optimized StringBuilder class instead of just concatenating strings using a much more costly s &= data line).

So, rule of thumb: whenever calling any method, expect hidden costs (magnified if you’re in a loop) and work to minimize them. Call the methods outside of the loop if they are providing data that will remain the same throughout the loop, such as a count, and avoid them altogether if there’s an alternative way to iterate through the data that doesn’t implicitly involve resetting your position. Your customers will appreciate the time savings!

‘Til next time,

--Matt--*

It’s a Small World, After All, part 2 – VB, C#, and GPS (Matt Gertz)

VBTeam — Wed, 05 Aug 2009 22:30:00 GMT

In my previous post, we fixed up a C# GPS library to provide more support, and wrote all of the UI for a VB GPS application, all based on (but modified from) a Mobile GPS sample in the Windows Mobile 6.0 SDK. In this post, we’ll finish up the app by enabling a bunch of cool functionality not exposed in the original sample.

The C# sample uses a helper function called UpdateData – I will too (as noted above), except that the code is going to deviate significantly. We’ll start out by checking is the GPS is actually running, just in case we called outside of an event:

Protected Sub UpdateData(ByVal sender As Object, ByVal args As System.EventArgs)

If vbgps.Opened Then

First, we’ll update the screen with whatever cached device data we have:

If device IsNot Nothing Then

Me.Text = "VBGPS: " & device.FriendlyName.ToString

Me.StatusLabel.Text = device.DeviceState.ToString

End If

This will change the title bar to include the name of the GPS device in use (in case you have multiple devices) and will set the StatusLabel to whatever the state of the GPS is (e.g., “On”).

All other interesting information comes from the cached position object:

If position IsNot Nothing Then

We’ll start with the altitude. There are two ways of calculating altitude – sea level (self-explanatory), and ellipsoid (which calculates altitude against a “perfect” ellipsoid representation of the otherwise pear-shaped earth). We’re going to go with ellipsoid. We’ll need to check if the information for it is reported as being valid, and if so, assigned the value to the appropriate label. The value is returned to us in meters, and we’ll also show it in feet by multiplying the result by 3.281 feet/meters. Again, we’ll format the results so that we only have two decimal places:

If position.EllipsoidAltitudeValid Then

Me.AltLabel.Text = CDbl(position.EllipsoidAltitude).ToString("####0.00") _

& " m (" & CDbl(position.EllipsoidAltitude * 3.281).ToString("####0.00") _

& "ft)"

End If

Next is velocity. The speed value will be in meters per second; We’ll show it in kph and mph. I’ll also show the heading (if valid), and separate the two by an “@” sign (e.g., “88.00 kph (54.69 mph) @ 45.23°”:

If position.SpeedValid Then

Me.VelLabel.Text = CDbl(position.Speed * 1.852).ToString("###0.00") _

& " kph (" & CDbl(position.Speed * 1.151).ToString("###0.00") & " mph)"

If position.HeadingValid Then

Me.VelLabel.Text = Me.VelLabel.Text & " @ " _

& position.Heading.ToString("##0.00") & "°"

End If

Latitude and longitude are next. We’ll be showing them in both DMS and DM format, using the methods we wrote in C#. The only interesting thing here is converting negative values to W or S indicators:

If position.LatitudeValid Then

Me.LatLabel.Text = position.LatitudeInDegreesMinutesSeconds.DMSString() _

& " (" & position.LatitudeInDegreesMinutesSeconds.DMString & ")"

If position.Latitude < 0 Then

Me.LatLabel.Text = Me.LatLabel.Text & " S"

Else

Me.LatLabel.Text = Me.LatLabel.Text & " N"

End If

If position.LongitudeValid Then

Me.LongLabel.Text = position.LongitudeInDegreesMinutesSeconds.DMSString() _

& " (" & position.LongitudeInDegreesMinutesSeconds.DMString & ")"

If position.Longitude < 0 Then

Me.LongLabel.Text = Me.LongLabel.Text & " W"

Else

Me.LongLabel.Text = Me.LongLabel.Text & " E"

End If

Satellites are a bit more complicated, and if you do a straight port of the C# sample, you’ll get runtime errors. This is because the C# code references values which may be NULL even if the values are valid – my VB code checks to see if the values are Nothing before referencing them. (A value of Nothing is equivalent to zero satellites in this case, since the C# library uses the existence and size of the satellite array to generate a value.)

There are three satellite components that we need to use – the number of satellites used to create a positioning solution, the number of them in view, and the total count of satellites – the resulting string should look something like (for example) “4/7 (8)”, depending on the actual number of satellites involved.

If position.SatellitesInSolutionValid AndAlso _

position.SatellitesInViewValid AndAlso _

position.SatelliteCountValid Then

Dim SatSol As Satellite() = position.GetSatellitesInSolution()

Dim SatView As Satellite() = position.GetSatellitesInView()

Dim SatStr As String

If SatSol IsNot Nothing Then

SatStr = SatSol.Length & "/"

Else

SatStr = "0/"

End If

If SatView IsNot Nothing Then

SatStr += SatView.Length & " ("

Else

SatStr += "0 ("

End If

Me.SatLabel.Text = SatStr & position.SatelliteCount & ")"

End If

You can actually get more information on each satellite (such as its location in orbit, etc.) simply by exploring the methods and fields on the satellite object, but for now this will do fine for our project.

Now that we’ve shown the information as to where we are, we need to tell the user how to get to where they are going. Creating maps and routes is just a bit beyond the scope of this blog, but we can certainly indicate direction and distance. Let’s assume the existence of a helper function called UpdateTargetInfo at this point – we’ll define it later:

UpdateTargetInfo()

End If

End Sub

Now, you’ll recall that we added fields to allow the user to specify the target point. We’ll want to translate the users entries into actual lat/long coordinates, and it would also be really cool if we could update the distance and direction in real-time. There are six fields and two combo boxes that are involved with the target, so let’s create a handler to deal with all of their change events (TextChanged for the text boxes, SelectedIndexChanged for the combo boxes):

Private Sub Target_TextChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles _

LatDegTB.TextChanged, LatMinTB.TextChanged, LatSecTB.TextChanged, _

LongDegTB.TextChanged, LongMinTB.TextChanged, LongSecTB.TextChanged, _

LatPosCB.SelectedIndexChanged, LongPosCB.SelectedIndexChanged

I’m going to use a helper function to collect the data, because collecting latitude data is identical to collecting longitude data, so we might as well treat them generically. The helper method will be called “CacheCoordinate” and we’ll define it later. As arguments, we’ll pass through the relevant controls and caches, and then update the distance and direction afterwards:

CacheCoordinate(LatDegTB, LatMinTB, LatSecTB, LatPosCB, TargetLatitude)

CacheCoordinate(LongDegTB, LongMinTB, LongSecTB, LongPosCB, TargetLongitude)

UpdateTargetInfo()

Now, the user could enter all sorts of errors accidentally into the text fields, any of which could cause an exception to be thrown. I could check and recheck for text and oddly-formed numbers being inserted into those fields, but for this exercise, I’m going to go with the moral equivalent of duct tape and simply use a Try block around the code we just wrote. If there’s an error, I’ll simply tell the user that their entered target information is invalid, and so the distance and direction cannot be determined:

Try

CacheCoordinate(LatDegTB, LatMinTB, LatSecTB, LatPosCB, TargetLatitude)

CacheCoordinate(LongDegTB, LongMinTB, LongSecTB, LongPosCB, TargetLongitude)

UpdateTargetInfo()

Catch ex As Exception

Me.DistLabel.Text = "Invalid target."

Me.DirLabel.Text = "Invalid target."

TargetLatitude = Nothing

TargetLongitude = Nothing

End Try

End Sub

Now, we’ll start to parse together the data, starting with latitude. The combo box for it has two values – “N” or “E” (at the 0 position) and “S” or “W”(at the 1 position) – so a simple check on the index will tell us if we’ll be using positive or negative coordinates:

Private Sub CacheCoordinate(ByVal DegField As TextBox, ByVal MinField As TextBox, _

ByVal SecField As TextBox, ByVal PosField As ComboBox, _

ByRef cache As DegreesMinutesSeconds)

Dim isPositive As Boolean ' South and west are negative values

isPositive = (PosField.SelectedIndex = 0)

Note that the cache needs to be passed in as ByRef, since we’ll be allocating new memory around it!

So, a user might want to enter data in three ways: degrees with a decimal portion; degrees and minutes, with minutes having a decimal portion; degrees, minutes, and seconds, with seconds having a decimal portion. To help the user, we’ll hide all subsequent fields if a decimal point is used in one of the earlier fields. That is, if the user enters “47.” in the degrees field, then we know that he/she won’t need the minutes or seconds fields, so we’ll just hide them:

If DegField.Text.Contains(".") Then

MinField.Hide()

SecField.Hide()

Next, we’ll get the degrees value and makes sure that it is between 0 and 180 degrees:

Dim ddeg As Double = CDbl(DegField.Text)

If ddeg > 180 OrElse ddeg < 0 Then

Throw New Exception()

End If

Now, if the coordinate is south or west, we’ll have to multiply by -1 to indicate that:

If Not isPositive Then

ddeg *= -1.0

End If

And then finally cache the coordinate:

cache = New DegreesMinutesSeconds(ddeg)

Now, if the degrees field didn’t have a decimal, we’ll need to remember to show the minutes field in case it had been hidden earlier:

Else

MinField.Show()

We also know that degrees will be a whole number, so let’s cache that away:

Dim deg As UInteger = CUInt(DegField.Text)

The next case is the degree-minutes case – if the minutes field has a decimal, we’ll know that seconds won’t be needed, so we’ll hide it:

If MinField.Text.Contains(".") Then

SecField.Hide()

We then validate both the degrees (0…180) and the minutes (0…60), and then cache the target using the appropriate DegreesMinutesSeconds constructor that we created earlier:

Dim dmin As Double = CDbl(MinField.Text)

If deg > 180 OrElse deg < 0 _

OrElse dmin > 60 OrElse dmin < 0 Then

Throw New Exception()

End If

cache = New DegreesMinutesSeconds(isPositive, deg, dmin)

If we’ve gotten this far without finding a decimal place, then we know that we’re using all of the fields (degrees, minutes, and seconds), so we should verify that seconds is showing, do the validation, and cache the target:

SecField.Show()

Dim min As UInteger = CUInt(MinField.Text)

Dim dsec As Double = CDbl(SecField.Text)

If deg > 180 OrElse deg < 0 _

OrElse min > 180 OrElse min < 0 _

OrElse dsec > 60 OrElse dsec < 0 Then

Throw New Exception()

End If

' Cache this data for later use

cache = New DegreesMinutesSeconds(isPositive, deg, min, dsec)

End If

End Sub

It’s time for us to write this UpdateTargetInfo() code. That code will be responsible for calculating both the distance and direction from our current position to our desired position. We’ll be using the Math library to do these calculations. The functions in the math library use radians, not degrees, but we can whip up a quick translation between the two:

Private Function DegToRad(ByVal deg As Double) As Double

Return (deg * Math.PI) / 180

End Function

Private Function RadToDeg(ByVal rad As Double) As Double

Return (rad * 180) / Math.PI

End Function

To calculate distance, we’ll be using the Haversine formula that we all learned in high-school trigonometry. I refreshed my memory on this formula (and the direction formula) at http://www.movable-type.co.uk/scripts/latlong.html. These formulae do calculations based on a great-circle, which (if you’ll recall) is actually the path to follow for the shortest distance between two points on a sphere. (Thus, if your destination is on the same line of latitude as you, the direction you want to go will not actually be due west or east unless you are at the equator.)

Assuming our current position is valid, we’ll go ahead & get out starting point, as well as define the radius of the earth:

Private Sub UpdateTargetInfo()

If position IsNot Nothing AndAlso position.LatitudeValid AndAlso _

position.LongitudeValid AndAlso _

TargetLatitude IsNot Nothing AndAlso TargetLongitude IsNot Nothing Then

Dim radius As Double = 6371 ' kilometers

Dim rlat1 As Double = DegToRad(position.Latitude)

Dim rlat2 As Double = DegToRad(TargetLatitude.ToDecimalDegrees)

Dim rlong1 As Double = DegToRad(position.Longitude)

Dim rlong2 As Double = DegToRad(TargetLongitude.ToDecimalDegrees)

We’ll also get the difference in latitude and longitude handy:

Dim rdeltaLat As Double = rlat2 - rlat1

Dim rdeltaLong As Double = rlong2 - rlong1

Now, we can calculate distance based on Haversine:

Dim a As Double = Math.Pow(Math.Sin(rdeltaLat / 2), 2) + _

Math.Cos(rlat1) * Math.Cos(rlat2) * Math.Pow(Math.Sin(rdeltaLong / 2), 2)

Dim c As Double = 2.0 * Math.Atan2(Math.Sqrt(a), Math.Sqrt(1 - a))

Dim distance As Double = radius * c

[Edit 8/7/09: Fixed typo in calculation of a -- used rlong2 instead of rlat2 accidentally. Will repost on Temple of VB site.]

The distance is described in kilometers at this point. If we’re under one kilometer, it’d be easier for the user if we switch to meters instead – and of course, we’d like to show miles and feet as well, all formatted nicely using ToString formatting:

If distance > 1.0 Then

Me.DistLabel.Text = distance.ToString("####0.00") & " km (" & CDbl(distance * 0.621).ToString("####0.00") & "mi)"

Else

Me.DistLabel.Text = (distance * 1000.0).ToString("####0.00") & " m (" & CDbl(distance * 3280.84).ToString("####0.00") & "ft)"

End If

Next is direction:

Dim theta As Double

theta = Math.Atan2(Math.Sin(rdeltaLong) * Math.Cos(rlat2), _

Math.Cos(rlat1) * Math.Sin(rlat2) - _

Math.Sin(rlat1) * Math.Cos(rlat2) * Math.Cos(rdeltaLong))

Dim degTheta As Double = (RadToDeg(theta) + 360.0) Mod 360

Me.DirLabel.Text = degTheta.ToString("##0.00") & "°"

And of course if the position information isn’t valid, then we need to tell the user:

Else

Me.DistLabel.Text = "No GPS data available."

Me.DirLabel.Text = "No GPS data available."

End If

End Sub

And that’s it! Debugging it is a bit tricky, since you have to take it outside to try it out (I was basically running in and out of my house to try out new ideas I’d just coded), but it’s fully functional and would be useful for (for instance) geocaching or surveying. Going forward, I’ll probably add more functional to mark areas that I was interested in persisting – once I get some more free time. J The current version’s code is available at my Temple of VB site – it also includes a setup project for CABbing up the file, as well as a console application to test out the direction/distance functionality. Enjoy!

‘Til next time,

--Matt--*

It’s a Small World, After All, part 1 – VB, C#, and GPS (Matt Gertz)

VBTeam — Wed, 05 Aug 2009 00:34:00 GMT

The title of this blog post has sort of a triple meaning for me. First of all, I just got back from a long (and wonderful!) cruise in the Caribbean, followed by a stop at Disney World. And of course, if you go to Disney World with kids, it’s the law that you have to go through the “It’s a Small World” ride, thereby having the accompanying song burned into your brain for the next six months.

Secondly, “It’s a Small World” also reminds me of how new technologies are bringing us closer together. It’s amazing to me that I can do a full presentation in (for example) Asia without having to leaving my office. For such scenarios to work, a number of different technologies have got to work together, and that’s tricky to get right. In this blog post, I’m going to be working with both VB and C# and, while they are not really different technologies in the same sense as a phone and a video camera, it’s nevertheless very cool that you can work with both to achieve a cohesive “whole.”

Finally, in the third sense, “It’s a Small World” came to mind while I was working on my latest hobby project (which I’ll detail in this blog) – a GPS program for my Smartphone. Verizon Wireless recently unlocked the GPS functionality for the Samsung Saga (one of which I own), and since I’ve always been really interested in GPS ever since it was made available to civilians, this was a great opportunity for me to jump in and try my hand at the GPS. So, in this two-part series, I’ll be walking you through a GPS app which uses both C# and VB to support its functionality.

“Um, isn’t there already a GPS sample for Smartphones?”

Well, yes, and you can get it from the Windows Mobile 6 SDK, which you can download from here. In fact, that’s where I’m going to start this project. (This will work on both VS2005 and VS2008, by the way.)

If you download the SDK and run the C# “GPS” sample (not the “Mobile GPS” sample; that’s for C++), you’ll see that it’s pretty simple. There are two pieces; a class library which wraps the native GPS calls into managed calls, and a mobile application which displays the GPS info. The UI for the mobile application is very simple – there is a menu item to exit the program, another menu item pair to start and stop the GPS, and a label. The GPS device generates the events, the UI handles them and appends the text to the label – that’s it. It’s a good place to start, but it really doesn’t fit a daily usage bill. I want my GPS program to always display at-a-glance:

· The state of the GPS

· The number of satellites in the solution and how many of those are available

· Latitude and longitude – both in DMS and DM format (since many geocaching sites are given in DM format)

· My altitude

· My velocity (speed and direction)

· A way to specify a target location, and the distance/direction to it.

That’s a lot to cram onto a phone screen – but I managed to fit it in. (I plan on adding more later, particularly a way to mark locations and cache them to use as targets later, but this is a good start.) Here’s how I went about it:

Creating a new GPS UI

First, open the GPS sample solution (if you haven’t already) and remove the “GpsSample” project. (You might want to create a copy of the solution first, so that you have the old one for reference.) We’re going to write a new, better UI that uses VB instead, so add a VB Smart Device Project to the solution, calling it “VBGPS” or something like that. (Right-click the solution, choose “Add Project,” “Visual Basic,” “Smart Device,” type in “VBGPS” for the name, and press OK.) In the next dialog, select the “Device Application” project type and press “OK.”

In the new project, you’re going to need to add a reference to the Microsoft.WindowsMobile.Samples.Location class library, so right-click the VBGPS project and choose “Add Reference…” Navigate to the “Projects” tab and select the aforementioned class library, then press “OK.” Now, you are hooked up to the C# code which wraps the native GPS functionality.

Double-click “Form1” in the VBGPS” project to bring up the form (if it’s not visible already). In the property grid, change the FormFactor property to match the form factor of your phone. (I have a square screen on my phone, so I’ve chosen “Windows Mobile 6 Professional Square.”) You can also change the Text of the form to something like “VBGPS” so that the title of the window doesn’t say “Form1.”

Now, we’ll start adding controls. First, we’ll add the menu controls. These will be the same as the C# version of the sample. The menu control already exists by default, so click on the left-hand bottom menu and change its text to “Exit” in the property grid – you can also give the control a better name (e.g. “ExitMenuItem”) if you like. Click on the right-hand bottom menu and change its text to “GPS,” again changing the control name if you like (e.g. “GPSMenuItem”). With “GPS” still selected, click the “Type Here” above it and add two menu items – “Start GPS” and “Stop GPS.” While each is selected, change their control names to something more intuitive in the property grid (e.g. “StartGPSMenuItem” and “StopGPSMenuItem” respectively).

Now, we’ll add the labels and such. For the following functionality, I’ll drag out two labels – one for the name of the GPS property, and one next to it to contain its value:

· Status

· Satellites

· Lat (i.e., latitude)

· Long (i.e., longitude)

· Alt (i.e., altitude)

· Vel (i.e., velocity)

Status and Satellites should share a line (for a total of four labels on one line); the others have a line each to themselves. All of them should be set to 8pt font in order to fit on the screen, and the value labels should have meaningful names set for them in the property grid. (I used “StatusLabel,” “SattLabel,” “LatLabel,” “LongLabel,” “AltLabel,” and “VelLabel.)

We also want to be able to specify a target location. So, farther down on the form, we’ll drag out a label and change its text to “TargetLat:”. To the right of it, we add three small text boxes – these will hold degrees, minutes, and seconds. Finally, to the right of those, we’ll drag out a small combo box. In the property grid, make sure its Type is “DropDownList,” and in the Items collection, add “N” and “S” as possibilities. All of these controls should be set to 8pt fonts using the property grid, and all of the text and combo boxes should have meaningful names. (I chose “LatDegTB,” “LatMinTB,” “LatSecTB,” and LatPosCB.)

Now, we’ll select the entire line of controls and, while holding down the “Control” key, drag a new copy of them below the existing ones. These will be the longitude target controls, so make the appropriate name and text changes in the copies, and also change the copied combo box items to “E” and “W.”

Finally, we’ll add two more label pairs below that like we did at the beginning. One will be for “Distance” and the other for “Direction” – make sure that the value labels have good names like “DistLabel” and “DirLabel.”

That’s the UI. It just barely fits on the screen, and it we want to add any more functionality, we’d have to bring up another window. But this will get us going, so let’s write some code.

The C# code

First, let’s make a few changes in the class library. As I mentioned above, we’re going to keep the class library as a C# endeavor. There’s no reason why it couldn’t be changed to VB code, but since it’s just a bunch of wrappers and they all work pretty well, there’s no need to invest the time to change the code. However, there are a couple of helper methods that I want to either change or add, and they (will) exist in the DegreesMinutesSeconds helper class defined in DegreesMinutesSeconds.cs. (We’ll get back to VB code shortly.)

DegreesMinutesSeconds is a class that allows for easy manipulation of latitude and longitude data. Not only does it store a particular coordinate, but it gives you two ways to interact with the data – as pure degrees (e.g., -82.673529) or as degrees/minutes/seconds (e.g., -82° 40’ 24.7044”). In this model, North and East are positive values; West and South are negative values.

This is all well and good, but I like to go geocaching, and in geocaching the coordinates are often described in terms of degrees and minutes (e.g., -82° 40.41174’). I would like to therefore support that format as well. Currently, the members of this class are:

uint degrees; /// In VB, this would be Private degrees As UInteger

uint minutes; /// In VB, this would be Private minutes As UInteger

double seconds; /// In VB, this would be Private seconds As Double

and each of these have properties associated with them; for example:

public uint Degrees /// VB: ReadOnly Property Degs() As UInteger

{

get { return degrees; } /// VB: Get: Return degrees: End Get

}

However, we want to be able to express coordinates that have a fractional minute, and so we add the following:

double dminutes;

public double DMinutes

{

get { return dminutes; }

}

Now we have to use it. There are already a couple of constructors, and we’ll need to fix them up to set dminutes -- additions are underlined. The code is close enough to VB that I won’t bother providing a helpful translation in the blog – just remember that “(uint) x” is equivalent to “CUInt(x)”, etc:

public DegreesMinutesSeconds(double decimalDegrees)

{

isPositive = (decimalDegrees > 0);

degrees = (uint)Math.Abs(decimalDegrees);

dminutes = (Math.Abs(decimalDegrees) - Math.Abs((double)degrees)) * 60.0;

minutes = (uint)dminutes;

seconds = (dminutes - (double)minutes) * 60.0;

}

public DegreesMinutesSeconds(bool isPositive, uint degrees, uint minutes, double seconds)

{

this.isPositive = isPositive;

this.degrees = degrees;

this.minutes = minutes;

this.seconds = seconds;

this.dminutes = minutes + seconds / 60.0;

}

I also need to add an entirely new constructor to take decimal minutes as an argument:

public DegreesMinutesSeconds(bool isPositive, uint degrees, double dminutes)

{

this.isPositive = isPositive;

this.degrees = degrees;

this.dminutes = dminutes;

minutes = (uint)dminutes;

seconds = (dminutes - (double)minutes) * 60.0;

}

So, that’s the “read” changes that we need to make; now, we need to update the “write” routine. There is a “ToString” routine already, but I don’t like it because it doesn’t use the degree character (°) and it only prints out in DMS format – I want DM as well. So, I’ll create two new methods:

public string DMSString()

{

return degrees + "° " + minutes + "' " +

seconds.ToString("##0.00") + "\"";

}

public string DMString()

{

return degrees + "° " + dminutes.ToString("##0.00") + "' ";

}

These are nearly identical to their VB counterparts – replace “+” with “&”, get rid of the semicolons and braces, rearrange the signature, and you’re pretty much there. Note that we’re making good use of the formatting ability of “ToString” – the format “##0.00” (identical to what we’d use if writing this in VB) means that we only want 2 decimal places, and the we don’t want to see leading zeroes in the tens’ or hundreds’ place. (We can’t have more than four digits left of the decimal place, since coordinates only go up to 180°.)

Finally, change ToString() to return the DMSString() function instead of the code already there, and we’re done with the C# code changes. C# code does not compile automatically, so we’ll right-click the class library project and choose “Build” – the class library is now ready for use to use.

Back to VB

It’s time to write the support code for our UI, so right-click on some unused part of the VBGPS form and choose “View Code.”

We need to add some members. First, we need to create an object that represents the GPS device. The class library exposes one, and creating it is pretty easy:

Dim vbgps As Gps = New Gps() ' The GPS device

The GPS device will be sending us events, and we need a way to handle those. We can use an EventHandler to do this:

Private updateDataHandler As EventHandler ' Handle GPS events

There are two kinds of events – “state” events (is the GPS on, etc.) and “position” events (where are we, how fast as we going, etc.). We will cache this information whenever it is sent to us for later reference:

Dim device As GpsDeviceState ' Passed to us by GPS device events

Dim position As GpsPosition ' Passed to us by GPS device events

(This is all identical to the original sample, except translated to VB.) I also want to be able to cache the target location where I’m heading to, and I can leverage the DegreesMinutesSeconds class for this:

Dim TargetLatitude As DegreesMinutesSeconds ' Where do we want to go?

Dim TargetLongitude As DegreesMinutesSeconds ' Where do we want to go?

Now let’s start with the menu events, because that’s where everything begins or ends. These are all simple – we start or stop the GPS device and update a few menu items:

Private Sub startGpsMenuItem_Click(ByVal sender As Object, ByVal e As EventArgs) Handles StartGPSMenuItem.Click

If Not vbgps.Opened Then

vbgps.Open()

End If

StartGPSMenuItem.Enabled = False

StopGPSMenuItem.Enabled = True

End Sub

Private Sub stopGpsMenuItem_Click(ByVal sender As Object, ByVal e As EventArgs) Handles StopGPSMenuItem.Click

If vbgps.Opened Then

vbgps.Close()

End If

StartGPSMenuItem.Enabled = True

StopGPSMenuItem.Enabled = False

ResetValues()

End Sub

Private Sub exitMenuItem_Click(ByVal sender As Object, ByVal e As EventArgs) Handles ExitMenuItem.Click, MyBase.Closed

If vbgps.Opened Then

vbgps.Close()

End If

Close()

End Sub

Note how the Closed and Exit events are both handled by the same method, which simply shuts down the GPS and exits the program. The other two methods also have to update the menu items to indicate whether stopping or starting is possible. (Again, these are identical in the C# versions in the sample.)

Next thing to do is to create the handlers for the GPS events. These are very trivial, and a straight port from the C# example:

Protected Sub gps_LocationChanged(ByVal sender As Object, ByVal args As LocationChangedEventArgs)

position = args.Position

Invoke(updateDataHandler)

End Sub

Protected Sub gps_DeviceStateChanged(ByVal sender As Object, ByVal args As DeviceStateChangedEventArgs)

device = args.DeviceState

Invoke(updateDataHandler)

End Sub

Basically, we’re just caching the information that the GPS device is sending us, and then invoking our overall handler (which we defined above) to deal with the information. We could call a helper method directly instead of using Invoke, but this gives us more wiggle room – we might want to re-route the events elsewhere to a different handler in certain situations, and this setup allows us the leeway to do that programmatically rather than hard-coded.

Of course, I haven’t turned on the handlers yet. I do that in the Form1_Load handler (which you can add by double-clicking on an empty area on the form). The relevant lines are:

AddHandler vbgps.DeviceStateChanged, New DeviceStateChangedEventHandler(AddressOf gps_DeviceStateChanged)

AddHandler vbgps.LocationChanged, New LocationChangedEventHandler(AddressOf gps_LocationChanged)

Now the GPS events will be received. I also need to define the updateDataHandler – I’ll point it to a function called UpdateData (which I haven’t written yet)

updateDataHandler = New EventHandler(AddressOf UpdateData)

While I’m in Form1_Load, I might as well initialize the various fields and target data:

ResetValues()

StopGPSMenuItem.Enabled = False ' The C# sample forgets to do this

' Zero out the text boxes and set the target to 0 degrees N and E.

Me.LatDegTB.Text = "0"

Me.LatMinTB.Text = "0"

Me.LatSecTB.Text = "0"

Me.LongDegTB.Text = "0"

Me.LongMinTB.Text = "0"

Me.LongSecTB.Text = "0"

Me.LatPosCB.SelectedIndex = 0

Me.LongPosCB.SelectedIndex = 0

TargetLatitude = New DegreesMinutesSeconds(False, 0, 0, 0.0)

TargetLongitude = New DegreesMinutesSeconds(False, 0, 0, 0.0)

ResetValues() is a function which does initialization that I might have to do more than once – for example, when stopping the GPS, I need to reset the feedback to the user to indicate that no data is available. It looks like this:

Private Sub ResetValues()

Me.Text = "VBGPS"

Me.StatusLabel.Text = "Off"

Me.AltLabel.Text = "---"

Me.VelLabel.Text = "---"

Me.LatLabel.Text = "---"

Me.LongLabel.Text = "---"

Me.SatLabel.Text = "Unavailable"

Me.DistLabel.Text = "---"

Me.DirLabel.Text = "---"

End Sub

That’s it for this post. In tomorrow’s concluding post, I’ll be adding all of the “meat” of the functionality (updating fields, calculating paths and distances) – all using VB, of course!

‘Til next time,

--Matt--*

We Need to Talk (Matt Gertz)

VBTeam — Thu, 25 Jun 2009 21:41:00 GMT

By now, many of you will have downloaded the recent beta that we posted to the net, and (hopefully) are getting excited about the features that we’ve been developing. Recently, Beth Massi interviewed me about some of the work that I’ve been doing as a part of that effort. I always enjoy interviews, and Beth is great at pulling out some of the more interesting aspects of a person’s work, but I’m always somehow left with the feeling afterwards that there’s something more that I wanted to say -- some different issue to which I forgot to draw folks’ attention. You should therefore think of this blog entry as a follow-up to that interview, where I get to spend some time discussing the people-aspects of the job.

“People, process, product.”

(variously attributed in current management theory)

In the course of the aforementioned interview, I spent a lot of time discussing process, and of course this blog site spends a lot of time focused on product. I haven't talked much about the “people” aspect of it – and yet, that’s a really important part of the triad. The product needs to address what people (our customers) need and what other people (our employees) can deliver. The process, in turn, needs to be defined in such a way as to deliver the product to the people. Process for process’ sake will inevitably result in the wrong product, which in turn will greatly annoy both the people developing the product as well as the people for whom the product is intended.

That’s not to imply that we don’t spend a lot of time on the “people” issue – in fact, it's the needs of our customers that drive us to create the best software we can. I remember the first time I picked up a Visual Basic magazine and thumbed through it. As I read it, I noted that many of the articles and “letters to the editor” involved features that I had written or that I somehow was responsible for. And as I continued to read, I was suddenly struck by an almost frightening realization: the effectiveness of my work, and of my team’s work, had a direct impact on the daily lives of our customers.

Not just “customers.” Millions of customers. Yep, that’s a big number.

This was something that I suppose I’d always understood in an abstract sense, but reading this magazine really brought it home for me. Previously, I’d joked about how I was getting paid by Microsoft for doing what was essentially a hobby for me – writing software for fun and profit, what a blast! Suddenly, I realized that there was something bigger going on – keeping electric plants working efficiently, allowing the initiatives of various governments to run smoothly, enabling small businesses to keep on an even keel – it’s just an incredibly diverse ecosystem that uses our software. I got a lot more serious about my attitude towards coding after that personal epiphany, believe me. It’s a sobering thought as to how the work that any of us does impacts others, and it can be highly embarrassing when we get something wrong. This is why, over the years, we’ve worked closely with not only MVPs but with a broad spectrum of our customer base to determine where we need to focus our attention.

Most of my “people” work these days, however, is focused within Microsoft – how we as engineers interact with each other in such a way that we can produce a better product. Currently, within our division, we have over 30 product groups that somehow need to work together as seamlessly as possible for us to stay on track. Visual Basic is a big part of that, but there are many other parts as well, and they all connect together in interesting ways to cover the broad spectrum of scenarios that we support.

“What we’ve got here… is a failure to communicate.”

(Luke, in Cool Hand Luke, mocking the prison captain’s earlier words)

Over a dozen years ago, we started the first efforts to bring together the various pieces that would one day make up Visual Studio. The first big step that we took was to start focusing attention on the “Vegas” shell (our internal codename for the shell that we still use in VS today) – all new products would leverage this shell for Visual Studio 6.0, which other established products (notably Visual C++ and Visual Basic) would use their old shells but would have some of their functionality supported by the new shell (which would, for example, wrapping their build functionality via special project types). This pulling together of different groups meant that the “build manager” position for each team suddenly went up in importance. Previously, this person simply needed to make sure that dependencies were in place and that daily builds got built. Now, this person instead had to work directly with peer teams to pass code around and integrate it. The problem was, each of these teams had different cultures and different processes doing things, and they’d never been pressed to evolve the skills required for this new sort of relationship.

At the time, in addition to my coding duties, I’d acquired the build manager position for the VS Enterprise Edition product unit (comprising source code control and deployment teams), and thus the responsibility for coordinating with my peers on these other product units. I can best describe those days as “dark.” This was a challenging time for us, and most of the challenges arose because of a lack of communication between the teams. Inevitably, whenever I would integrate some other teams’ code into our codebase, their code would have massive conflicts with our code, and I would spend 2-3 days getting things working before being able to return to my “normal” programming duties. There were days where I and others worked very, very late to get things working again.

The other teams didn’t mean to break my team’s code. Our team didn’t mean to break their code. The problem was simply that it never occurred to anyone that their change would have a ripple effect with other teams in this shared environment, nor that they needed to communicate such changes. Now, you could claim that there was a process problem involved – and you’d be right. Our processes changed substantially as a result of this, believe me. But the fundamental problem was a cultural (i.e. people) problem – each team was still acting as if they were on their own island, and every divisional problem took a backseat to their drive to provide their own core functionality. There was no common language to describe the importance of one team’s scenario with respect to another’s.

Now, since then, we’ve made huge strides in inter-team communication. Leaders from across the division, in all disciplines, regularly meet to discuss changes coming down the line. The build manager position has evolved into the Build Facilitator Developer position (BFD), and they (sponsored by my team) also regularly meet to discuss policies and practices. The division as a whole devises specific goals for a given product that teams need to incorporate into their feature planning. As a result of improved processes and better communication, our downtime for build issues is the merest fraction of what it was a dozen years ago. We’ve continued to evolve these practices which, along with new processes, means that even the difference between now and a few years ago is stunningly better with respect to build efficiency.

“I feel that if a person can’t communicate, the very least he can do is to shut up.”

(Tom Lehrer, That Was the Year That Was)

So, in the previous section, the communication I discussed was mostly tactical in nature – i.e. here’s what’s coming, and here’s what we’re going to do to avoid problems. Another aspect of communication, though, is letting people where we are right now. It’s a critical point, because the best way to avoid mistakes, or to recognize the things that worked and which should be made policy, is to fully understand how our past work led us to this point.

Now, my first attempts along these lines, back when I was the Visual Basic box lead, were not as successful as I would have liked. In my eagerness to get all of the information out there, I evolved what started out as a small weekly bug report into this huge, massive thing, comprising about 6 pages of data which covered every single aspect of the product’s state. If you wanted to know *anything* about the status of Visual Basic, you could find it in that weekly mail.

The problem was, no one read the whole thing. It was simply too much information -- you couldn’t look at the report and know what data was important. It wasn’t communication at all – it was death by drowning (in data). Added to this was the problem that I (still) tend to be a very verbose person, using two words where one would suffice. It actually *hurt* communication because I still had people asking me for data hidden away in my mail – I was wasting my time and theirs.

Eventually, something clicked. I found out that people were at least reading the first paragraph, where I’d provide a brief recap of the previous week and thank the people who’d specifically helped out with the problems therein. They read that part because it was *personal* -- it related to them, not in an esoteric “here’s how many bugs you have” context, but instead in a familiar “I did this” sort of way. So, a few years later when I assumed my current job (which eventually included, among other duties, providing daily status to the divisional staff), I decided to re-think how I did my status reports. The status I or others on my team send out is devised as follows:

- What happened yesterday, why it happened, and how does it affect you

- What’s happening today, and what the status is

- What the most important resources are currently (i.e., builds and tools), and how you can access them

- What are the most important issues to address today

That’s it – the whole thing takes less than a printed page of text. It is a person-oriented mail, which gives kudos to folks that did something particularly well, and points out (in a clear but respectful way) areas where things could have gone better. Additionally, I sometimes add a “Question of the Day” which answers (in 1-2 small paragraphs) a process-relevant question which one or more people have asked me recently. This gets the focus on the issues that really matter to people in the division, and not just the ones that I think matter, so that everyone is made aware of them. This format of status mail has been received much more favorably than the previous one.

Let’s talk

Hopefully, some of what I’ve said above is relevant to the work that you do. I, in turn, would be fascinated to learn more about how you, our customer, tackle the issue of communication both inside and outside your own business. I’m certain that we’ve got a lot more to learn from each other beyond coding – the process of coding is really important, but without understanding how to communicate with the people involved, the code you and I create might just be the best thing that no one’s ever heard of.

‘Til next time,

--Matt--*

How We Do It: Building the Visual Studio Product Line (Beth Massi, Matt Gertz)

VBTeam — Fri, 22 May 2009 22:51:00 GMT

Ever wonder how we build Visual Studio and the .NET Framework? I mean literally. You know, the build process that spits out a product at the other end? Well I sure did and it's not just hitting F5 people ;-).

In this interview I sit down with Matt Gertz, frequent VB blogger and also the guy in charge of the builds in Developer Division (he has a long title, but I forget what it is :-)). He explains how the build and test processes work on a team of two thousand people all on Team Foundation Server. I must say it's pretty darn impressive. Many thanks to Matt for taking time out of his busy schedule to do this interview!

How We Do It: Building the Visual Studio Product Line

Enjoy,
-Beth Massi, Visual Studio Community

Phoning Home, revisited (Matt Gertz)

VBTeam — Thu, 21 May 2009 20:43:00 GMT

In my post “M. G. Phone Home,” I discussed how to write a simple phone application for Windows Mobile, using Visual Basic, which accesses the Bluetooth service. What I didn’t discuss was how to deploy it to the phone via CAB. Since I purchased a new phone yesterday (a Samsung Saga, running Windows Mobile 6.1), and I’ve discovered that I need the same app on it, I thought this would be an opportune time to discuss how to do this, since it’s not obvious and slightly different from the distribution of “normal” applications. If you want to follow along, you can find the code at my Temple of VB site (direct release link is here).

(By the way, I’ve modified the app’s code slightly from what I wrote in that previous blog -- I have assigned an icon to the application in the project’s properties, because I got tired of seeing the default, non-intuitive icon. If you do this also, make sure that, after doing so, you select the icon in the project’s files and, in the property grid, set its “Build Action” to “Embedded Resource.” If you don’t do that last step, you won’t see the icon on the phone.)

Once you’ve got your phone app completed and debugged (as discussed in the aforementioned blog), choose “File-> Add->New Project.” Select “Smart Device CAB Project” under the “Other Project Types->Setup and Deployment” tree node, and give the project some meaningful name (ResetBluetooth, for example). Select OK so that the project is created.

In the property grid for the new project, make sure that “Manufacturer” and “ProductName” are set however you want them to be, since they will be visible in the program list on the phone.

Now, in the Solution Explorer, right-click the new project and choose “Add->Project Output.” In the resulting dialog, make sure that the first project is selected (the one with the actual code), choose “Primary Output” and click OK. The CAB project will now always contact that project to get its outputs when creating the CAB.

You’re not quite finished, though. You need to add a couple of shortcuts so that you can actually interact with the program on your phone once it’s installed. To do this, bring up the CAB project’s File System Editor if it’s not already visible (right-click the project, choose “View->File System”). In the left-hand pane, you should see “Application Folder” and “Program Files Folder.” “Application Folder” is where the code will end up; don’t change anything about it.

However, “Program Files Folder” is empty, and you need to add a shortcut to the program in it. Select that folder, and in the right-hand pane, right-click anywhere and choose “Create New Shortcut.” In the resulting dialog, navigate into the Application Folder and select “Primary Output from ResetBluetooth (Active)” (or whatever you called your original program). Click “OK.” The Shortcut will be selected for editing, and it will have an atrociously long, descriptive name – give it a more sensible name like “Reset Bluetooth” or something.

You’re still not done, though -- you need to add one more folder. Right-click the “File System of Target Machine” node in the left-hand pane and choose “Add Special Folder->Start Menu Folder.” Inside that new folder, add the same shortcut to the primary output that you added in the “Program Files” folder, and give it the same sensible name.

Now you’re done. Build the CAB project, and you’ll find the CAB file in its Release subdirectory. You can copy that over to your phone via the usual ActiveSync mechanism, and then run it on the phone to install it. (I usually map a phone key to that program so that it’s trivial to launch the application – check your phone’s manual for instructions on how to do that.) And that’s it!

‘Til next time,

--Matt--*

How Do We Learn to Be Good Programmers? (Matt Gertz)

VBTeam — Wed, 06 May 2009 02:20:00 GMT

I’ve always enjoyed writing. Putting pen to paper (or, these days, fingers to keyboard) is something that brings a lot of joy into my life, even more than music or any other hobby that I have. I’ve been writing stories since earlier than I can remember, have faithfully kept a daily diary for many years, and have published countless journals, conference papers, and other articles on this or that technology.

So, when I was in my early twenties, it seemed the most obvious thing to try start writing professionally. I wrote out a number of stories (mostly science fiction), which I sent in to various magazines. All of them were rejected. This was a source of frustration to me for quite some time, and as I personally found my stories to be engaging and eloquent (why not?), I couldn’t understand why these obvious “gems” were being overlooked.

I could have given up, and I was tempted to. But I instead decided to get into it from another angle; why not practice? There was this new thing called “USENET” which leveraged the still-nascent Internet, which was read by people with very similar backgrounds to mine. I started out by writing short parodies, figuring out what worked and what didn’t. (Some of my Star Trek and Black Adder parodies are still lurking out there on the ‘Net, much to my embarrassment. If you come across them, have mercy on me. J)

I got a lot of healthy feedback from friends and ‘Net acquaintances on these attempts. Gradually I came to realize that there was a lot of difference between having a good idea and describing it well. There’s a fine line between too much description and not enough description, and I tended to err on the side of “too much.” It’s a problem I still have to some degree even now – for the technical writing that makes up much of my authoring, it’s not a big deal, but it’s deadly for any writer who’s trying to keep the action moving. This was a good thing to learn, I’ve gone on to publish one piece of fiction, and I do plan (if I ever get enough time) to try my hand at fiction again – and in the meantime, I have learned many things that have helped me in the writing that I do daily.

So, what has this got to do with programming? Well, this is going to be a slightly odd column – not my usual coding example, but more getting into how we learn and improve our skills. It’s motivated by a comment from “jerry” (which you can find here), in which he asks:

“[…]I've been trying and trying, searching endlessly on the internet, but i can't find anything on the internet that really teaches me more. It's either easy stuff, or articles like these *gulp* that are way too far beyond me. And just experimenting takes forever, and usually doesn't show much results, because i don't know what to look for. How can i learn?”

This is a great question; one that I wanted to spend some time on, rather than just ad-lib a few reassuring sentences in a comments section. How exactly do we learn to be good programmers?

The Right Stuff

Before I get into “ways of learning,” there are some basics I should touch on first. There are a few talents that I believe all good programmers share, the benefits of which extend beyond programming. Three of these include:

· Pattern matching: A good programmer sees patterns in how work gets done. These patterns can be used to generate the spec for a program (i.e., tailoring usability to match a cultural meme), or to generate the basic architecture for their code (i.e., boilerplate). A truly great programmer, however, will intuitively know when to ignore these patterns and move forward to something new so as not get trapped in an obsolete way of doing things – this is referred to as “thinking outside of the box.”

· Efficiency: A good programmer is capable of reducing a complex problem to a simpler one. (Some skill at pattern matching is a prerequisite for this talent.) This talent is exposed through the crafting and reuse of good code. A truly great programmer will be reevaluating plans and code even after they are complete, and for as long as they are relevant, and have the desire to accept honest feedback on the work to make sure that it’s as efficient as possible.

· People skills: Really? In a programmer? Well, yes. Without this, you can’t accurately gauge what the customer needs, or communicate your ideas, or…well, do anything that involves people. There are people who can write good code and have good ideas but you’ll never know it. A great programmer can sell new ideas to their team because they know how to explain the ideas in such a way that people immediately understand them.

Given these talents, my opinion is that you should be in a good position to grow your skills as a programmer, no matter your current expertise is. (Not coincidentally, interviews at Microsoft are set up to gauge these three talents, as well as general know-how.) So now, let’s take a look at some learning opportunities.

Example-based learning

Back in the early 80’s, there were a lot of magazines that one could purchase that had lengthy programs written in them, all ready for you to type into your C-64 or TRS-80. (I personally read Compute! magazine myself, being a C-64 user.) The programs were fully printed out due to necessity – 5.25” floppies disks were still a new thing then (many people, like me, were still using tape drives), and publishers didn’t bundle disks with magazines until later in the decade – too expensive!

The side-effect of this was that the user was essentially forced to *be* a programmer by virtue of typing in the program, unless they bought pre-packaged code (of which there wasn’t much). Anyone paying attention to what they were typing – if they had the talent – would be recognizing the patterns and thinking “Hmm, what if I changed this? How about if I do that instead?” In fact, I’d be willing to bet that that’s how 90% of programmers in my generation got their start – they began with someone else’s example (even if they didn’t understand it), and started tweaking the code, little by little (as well as coding around typos inadvertently introduced into the article). Gradually, they’d recognize patterns, and start applying them to their own code.

This sort of example-based learning is something that I still do today. When I became the lead of the VB Compiler Team back in 2003, I knew nothing about compilers beyond what I’d learned in college nearly 16 years prior, so I grabbed the compiler code, wrote a simple program, and stepped through the compiler line-by-line to learn what it was exactly that my new team owned. In more recent days, when I’m working on “hobby” (i.e., non-Microsoft) code, I’ll sometimes stop at open source sites, download the code, and do pretty much the same thing I always did – step through the code, tweak it, and see what happens so that I can learn the basics. (Edit and Continue makes this pretty simple to do.)

The epitome of example-based programming is to take code written in one programming language and porting it to a different language. You can learn a lot that way, especially if you’re currently weak in one of the languages. I’ve done this a lot in the past to get past the learning curve – for example, the very first code I posted in my VB blogs was a VB port I’d done in 2000 from some C++ code I’d written in 1998. It was the first full program I’d ever written in Visual Basic! I learned a lot about the differences between the languages, their strengths, and their weaknesses.

Oddly enough, these blog entries of mine are a sort of homage to the old Compute! magazine, with code interspersed with discussion – like other authors on this blog, I’m motivated to both present the code and talk about it as well, so that it becomes a good learning experience. Our own MSDN Code Gallery is good place to find code samples, and there are still magazines out there which publish code for you to try as well. And if you run across a programming concept that seems too hard – ask the author! You can’t expect the author to write your programs for you, but he or she will likely be very flattered to get questions from you and happy to clarify issues that might be too vague. At the very least, it gives the author something to write about for his or her next article!

Learning from the community

Occasionally, you get the opportunity to build on someone else’s code, and that’s a great learning experience as well. A gentleman by the name of Ken Mattern has done this with the MIDI code sample app that I posted a few weeks ago (here and here). You can find his extensions to my code here. Basically, he’s taken my code and added a keyboard and other controls to the form, which makes it a lot easier to enter the notes (you can even hear the notes as you press them) and is a big improvement on the interface that I had created. He was gracious enough to point me towards his extension, and he and I have exchanged mail on ways that the keyboard code could be made even more efficient. Win-win!

Visual Basic in particular has always had a great community from which to learn, and one sure way to learn more about coding is to join a VB User Group. There are many of these around the globe, and you can search the ‘Net to see if there’s one in your area. I’ve visited with some of them, and I’ve always been really impressed with them – the members are really helpful to each other with respect to learning new concepts. You can also get involved with “group projects” at these meetings – collaboration is a sure means of getting feedback on your skills and learning new tricks, as I’ve discovered at Microsoft.

Lessons on programming

And sometimes, you just need to take lessons. For example, I’d gotten back into golf recently after an injury got me out of the game a few years back, but was doing really poorly. Despite helpful advice from friends who are good golfers, and despite watching them play, my game didn’t turn around until I got lessons from a pro (and boy, did that help!). The same thing works for programming – sometimes you just need to study with a master.

This doesn’t necessarily mean taking classes (although there is certainly something motivational about getting graded on your programming, having an instructor at hand, etc.). It can be as simple as buying a book at your local bookstore. There are literally hundreds of books which can teach you to be a better programmer, and buying 2-3 of them is definitely a good investment for anyone seeking to improve their programming skills. (My shelves at work regularly contain books on COM, .NET, international programming, security, and so on.) If you go this route, here are a few things to consider when shopping for a book:

- Does the book provide clear descriptive text for the code?

- If a true textbook, does it provide a solution for the problems it sets you to work on?

- Does it explore different aspects of coding – data, forms, web use, etc?

- What level of programmer is it aimed at?

Read through the first chapter at the bookstore before buying the book – that should give you a good idea of where the author is going.

Community colleges and night schools also often have inexpensive courses in programming. Sometimes these are tilted in a particular direction (web development, rich client apps, coding in C#, and so on). You want to make sure that the course is about programming applications instead of using applications, of course!

And more…

At this point, I’m going to throw it open to everyone else to comment. Have I missed some obvious learning mechanisms? What else could a person wanting to improve their programming skills do? I’m looking forward to hearing back from everyone on what’s worked for them!

‘Til next time,

--Matt--*

P.S. I’ve uploaded yet another version of the VBScreenSaver code to fix a bug I found last weekend. It turned out that, in the unlikely event that the same random number is chosen twice in a row, the code was attempting to access the same picture before it released it, leading to a resource contention exception. I’ve fixed this by simply returning from DrawPicture() without doing anything if the random number is the same , keeping the current picture up until the next time event. It’s not a frequent enough occurrence to warrant anything more complicated than that, such as releasing the picture (which would cause a flicker) or generating new random numbers until we get a unique one.

Private Sub DrawPicture()

' Which picture? Get a new random number

Dim index As Integer = Me.m_Random.Next(m_fileCount)

If m_files(index) = m_currentFile Then

' We are already looking at this file and will get an exception

' if we try to open it before we close it.

' We could call for another random number, but why bother?

Return

End If

You can find the updated code at my Temple of VB site.

Keep the Customer Satisfied (Matt Gertz)

VBTeam — Tue, 14 Apr 2009 06:21:00 GMT

As I’ve written elsewhere, the toughest critics for any work you do can always be found in your own family. Pleased at the immense work that I’d done in scanning in and tagging all of my photos and media (as noted in this post), and with my ego sufficiently boosted by writing a screensaver in VB that would display not only the pictures but the tags associated with them, it was a deflating experience to have my middle child, Aidan, pepper me with comments like “That’s not me; that’s Brandon” or “You didn’t include Grandpa in the tags, but I can see his ear in this photo.”

In order to preserve the peace (and find the inevitable set of photos with bad tagging or poor rotation), I needed to open up the code to give the kids a way to let me know which photos needed to be re-examined. But how to do this? I’ve currently got the photos set to display for 30 seconds, which is a limited amount of time for the kids to get over to my desk, pull out the keyboard tray, and somehow record a flaw without turning off the screensaver.

“X” marks the spot

So, I considered my customers. My kids would likely be coming over from the couch, which is to the left of my desk. Pulling out the keyboard tray could jiggle the mouse and end the screensaver, so I wanted a key that they could reach without moving anything. Furthermore, it needed to be a memorable key, and so I settled on “X” (as in “X” marks the spot), which is of course located on the left side of the keyboard.

Now, this sounds a lot like overthinking, doesn’t it? But really, all design decisions need to take in account customer abilities and cognitive associations. I could just as easily have chosen “F5” as the keystroke, but besides creating an added potential for moving the mouse disruptively (an accessibility issue), the fact is that the key “F5” would have no association for my kids whatsoever. “X,” however, is an “interesting” letter and likely to stick in their minds.

So, anyhow, here’s how I went about it code-wise: I opened up the project that’s presented in the aforementioned post and navigated to the code for the frmScreenSaver object. Inside that code, I added an event handler for “X” thusly:

Private Sub frmScreenSaver_KeyPress(ByVal sender As Object, _

ByVal e As System.Windows.Forms.KeyPressEventArgs) Handles Me.KeyPress

First thing to do is check to see if “X” was pressed:

If e.KeyChar = "x" OrElse e.KeyChar = "X" Then

OK, “X” was pressed. Now, do I actually have a file that I’m currently showing? If my screensaver couldn’t find any files, then I wouldn’t have one. I can leverage the m_currentFile variable to check:

If Not String.IsNullOrEmpty(m_currentFile) Then

OK, I’ve got a file. So, I’m going to log this filename to a text file, appending it to whatever is already in that file (or creating it if none have been logged yet). (Every Saturday, I check this file for the accumulated complaints, examine the photos, make the required fixes, and delete the log file.)

My.Computer.FileSystem.WriteAllText( _

My.Computer.FileSystem.SpecialDirectories.MyDocuments _

& "\FilesToFix.txt", m_currentFile & vbCrLf, True)

Pretty simple! But I need to provide feedback to my kids that their keystroke was accepted, and so I add a simple “beep:”

My.Computer.Audio.PlaySystemSound(Media.SystemSounds.Exclamation)

End If

End Sub

And that’s it. I build the project in retail, rename it to be a *.scr file, and copy it up the the system directory.

“Is that really necessary to look at, dear?”

The second complaint I received about the screensaver came not from my kids but from my wife, the crux of the problem being that the screensaver didn’t discriminate between good pictures and bad pictures. This could be somewhat embarrassing when friends were visiting. For example, I’ve got a set of photos from when I was in Dubai on a recruiting trip a few years back. The photos weren’t just taken by me, but by my co-workers as well. Evidently, one of them was highly impressed with the bidet in their hotel bathroom and decided to commemorate it with a photo -- not something that my wife was crazy about seeing on the screen. And, while I am often impressed by the artistic abilities of my kids, the fact of the matter is that their photos are often punctuated by close-ups of the dog’s butt, their best friend’s eyeball, or some incredibly blurry thing that can’t be made out at all.

So, to address this problem, I decided to add a “nuke” feature to the screensaver. When the “N” key (another easy-to-reach key with a good mnemonic -- "N" stands for "nuke") is pressed for a given photo in the collection, I want to eliminate it and move to the next picture. Sounds simple, no? It’s just a modification of the “X” code from above.

But here’s the problem: my kids are wonderfully smart, but I don’t trust them to be good judges of whether or not a photo should be permanently deleted (and, depending on the timing, they might use the key just as the photo changed, thus deleting the wrong photo). I therefore needed to construct a way to send the photo to a “penalty box” until I had the time to review the photo. Furthermore, this “penalty box” needed to be able to persist between sessions.

Here’s how I set about it: first, in the same event I created above, I check for the “N” key and for the existence of the file:

ElseIf e.KeyChar = "n" OrElse e.KeyChar = "N" Then

If Not String.IsNullOrEmpty(m_currentFile) Then

I log the picture just as I did before, although to a different file this time:

My.Computer.FileSystem.WriteAllText( _

My.Computer.FileSystem.SpecialDirectories.MyDocuments _

& "\FilesToNuke.txt", m_currentFile & vbCrLf, True)

My.Computer.Audio.PlaySystemSound(Media.SystemSounds.Exclamation)

I then remove the photo from the list of files by finding its index and calling “RemoveAt()”:

Dim index As Integer = m_files.IndexOf(m_currentFile)

If index >= 0 Then

m_files.RemoveAt(index)

m_fileCount -= 1

End If

And then I call “DrawPicture(),” which forces a move to the next picture without waiting for the timer:

DrawPicture()

End If

Unfortunately, if the screensaver is interrupted before I get a chance to review the “photos to nuke” on Saturday, it will re-create the file list with the bad photo(s) still in it the next time that it restarts. Therefore, I need to adjust how I create the file list in the frmScreenSaver_Load() event handler – it needs to read from the “FilesToNuke.txt” file (if it exists) and remove such photos from the photo collection that it generates.

Here’s how I do that – first, I change m_file’s type to be a normal collection rather than a read-only collection, so that I can alter it:

Private m_files As New Collection(Of String) ' List of files

Then, in the Load event handler, I make a few minor modifications (in bold):

Dim files As ReadOnlyCollection(Of String) ' List of files

If m_Options.UseSubdirectories = True Then

files = My.Computer.FileSystem.GetFiles(m_Options.Directory, _

FileIO.SearchOption.SearchAllSubDirectories, "*.jpg")

Else

files = My.Computer.FileSystem.GetFiles(m_Options.Directory, _

FileIO.SearchOption.SearchTopLevelOnly, "*.jpg")

End If

That code is the same as before, except that I read in the collection for the new variable “files” rather than m_files.

Now, we encounter a pet peeve of mine – the method GetFiles returns a read-only collection rather than just a normal collection. This means that we can’t remove anything from the collection we get back from GetFiles(), and as noted above, I need to remove the questionable files from the collection until such time as I get around to reviewing and deleting the files permanently. (One day, I’m going to write a blog which discussed the correct reasons for using read-only objects – I'm all for security, but it's too easy go overboard with it sometimes, in my opinion.)

I have two choices – either write my own (thorough!) routine for enumerating files, or else bite the bullet and copy the read-only collection to a modifiable collection. (Of course, I could alter the Timer handler so that I check the random result against “bad” filenames every time the timer fires, but that would be taking on a perf hit on every event, not just the initialization.) Being inherently lazy, I’ll just copy the collection (ooh, I feel so dirty!) into m_files, which is the version that I ultimately cache & use internally:

' Copy to a writeable collection (why is the other collection read-only?)

For Each f As String In files

m_files.Add(f)

The worst (ethically) is over; I will now load in the set of questionable files from the "FilesToNuke.txt" file and remove them from the collection. Fortunately, the files being separated in the file by a carriage return, it’s easy for me to read them in and separate them into a “nuke” list, which I can then iterate to remove the corresponding items from the main collection. (I could also have read this list a priori and prevented the items from being copied into the modifiable collection above, but that’s actually less performant, as I would have far more misses than hits, even if I removed items from the “nuke” list after the corresponding entry was found.)

' Load files to ignore

Try

Dim separator() As String = {vbCrLf}

Dim sFilesToIgnore As String = _

My.Computer.FileSystem.ReadAllText( _

My.Computer.FileSystem.SpecialDirectories.MyDocuments _

& "\FilesToNuke.txt")

Dim sarrFilesToIgnore() As String = _

sFilesToIgnore.Split(separator, StringSplitOptions.RemoveEmptyEntries)

For Each s As String In sarrFilesToIgnore

Dim index As Integer = m_files.IndexOf(s)

If index >= 0 Then

m_files.RemoveAt(index) ' Don't want to see this file anymore

End If

Catch ex As Exception ' File not found

End Try

Done -- build, rename, copy to the system directory. Now, my wife and kids and can easily log files that *might* need to be deleted and, furthermore, be spared the sight of them, no matter how many times the screensaver stops and starts, until I get a chance to make the permanent deletion.

That’s it; all done (until my family requires me to add more functionalityJ). I’ve uploaded the new code to my “Temple of VB” site (see the link below -- I'm going to stop uploading files to this blog since it's problematic to maintain the code in two sites).

‘Til next time,

--Matt--*

Veni, MIDI, Vici: Generating a simple MIDI file using VB, part 2 (Matt Gertz)

VBTeam — Sun, 01 Mar 2009 21:00:00 GMT

In part 1 of this series, I constructed a pair of classes to supporting persisting MIDI data to files. In this entry, I’ll now leverage that code to support an (admittedly limited) music editor experience.

Caveat: As I mentioned in the first post, I’m sure many readers will be far more knowledgeable about MIDI than I am, and will justifiably roll their eyes at this code due to its limited scope. I will only be generating simple type-1 MIDI files that only use the Note On and Note Off events, and which don’t use metadata, targeting a “Save” scenario only. The idea of this exercise was to understand MIDI and how to code against it in VB, not to generate a fully-functional music generation program. Coding up “Load” and supporting more of the MIDI functionality would be a more time-consuming process which maybe I’ll get to one day.

I would love to be able to write a full-fledged editor with all of the music printed out, all WYSIWYG, but unfortunately I have a day job which would be impacted and so I’m opting for the Commodore-128 experience I mentioned in the previous post. Basically, I will (arbitrarily) support four tracks of music, each track containing notes, and each note derived from the following format:

dTv

Where:

· d is the duration of the note (a floating-point value; i.e., since 4/4 time is assumed, 1.0 would be a quarter note, 1.5 would be a dotted quarter note, 0.5 an eighth note, etc.)

· T is the tone of the note (i.e., anything from '''''C through G'''''' or R (for rest))

· v is the volume from 0 to 127 (i.e., &H0 through &HFF), which is never used for a rest.

So, in my design, a track which played a whole-note (4 beats) middle C, rested for a half-note (2 beats), and then dropped down to 'G for dotted quarter-note (1.5 beats), all at a volume of 60, would look like this:

4.0C60, 2.0R, 1.5'G60

That’s not nearly as nice to look at as the actual musical score, but it will allow me to give an example of parsing text data, so on we go…

The form

On my Windows application, I have four combo boxes in a vertical pattern, with a long text box adjacent to each. The combos’ DropDownStyle properties are all set to DropDownList, and the Items property for each is preset to be the numbers 1 through 15 (these numbers indicate MIDI channels), plus “blank” (a simple carriage return) – blank is the default value (i.e., unused track). The text boxes (which will contain the “dTv” notes) all start out as ReadOnly = True, and switch to editable only if the corresponding combo channel is non-blank. I also have a “Save” button to generate the MIDI output, and I added a SaveDialog object as well. That’s pretty much the extent of the form.

Back in the code, I’ll add an instance of the MIDI object I defined in the previous post:

Public Class Form1

Private Song As New MIDI

Now, I need to deal with the notes!

Specifying the notes that can be used

In order to support the parsing of the track information, I’ll need to translate the tone values from characters to the numbers that MIDI expects (of which there are 128). The best way to do this is to create a collection mapping the text value to the numeric value, so we can create the mapping once and be done with it:

Private Notes As New Collection

Const NumberOfNotes As Integer = 128

Now, I could do something lame like:

Notes.Add(60, "C")

Notes.Add(61, "C#")

And so on, but there are 128 potential values, and some of them have multiple names (for every sharp, there’s a corresponding flat), so that would be a lot of work. So, I wrote some helper routines to try to do this in a smarter way.

First, I need to find out how many single quote marks I need to add to the note. Each quote preceding the note indicates an octave one lower than the one occupied by middle C; conversely, a quote following a note is indicative of a higher octave. (That is, ‘C indicates a note that is one octave lower than middle C, whereas C’ indicates a note that is one octave higher.) There are twelve distinct notes in an octave, including accidentals (flats and sharps), so for a given value, I divide by twelve to determine its octave, and use the remainder (modulus) to determine its position within that octave:

Private Sub InitializeNotes()

For i As Integer = 0 To NumberOfNotes - 1

Dim octave As Integer = i \ 12

Dim tone As Integer = i Mod 12

Then, for each number, I call a helper function to do the actual work of creating the textual representation of the corresponding note. “C” is the first note in MIDI, and so would have a remainder of zero:

Select Case tone

Case 0

AddNote(i, octave, "C")

AddNote(i, octave - 1, "B#") ' A sharped B is technically in the next lower octave

Case 1

AddNote(i, octave, "C#")

AddNote(i, octave, "Db")

Case 2

AddNote(i, octave, "D")

(And so on.) Note that I am adding two notes for each accidental (one for its sharp representation, and one for its flat representation), since I don’t want to force the user to use a specific one. I also accommodate B# (= C), E# (= F), Fb ( = E), and Cb (= B), though these are rarely used in most music. (I don’t currently support double-flats or double-sharps, though it would be easy enough to do – just add more entries.)

At the end of these 128 additions, I also add an entry for rests directly:

Case 11

AddNote(i, octave, "B")

AddNote(i, octave + 1, "Cb") ' A flatted C is technically in the next higher octave

End Select

Notes.Add(NumberOfNotes, "R")

End Sub

AddNote() is a simple method that determine the quotes (if any) for the note and appends them or prepends them in the right order. (The 5^th octave is the middle octave and therefore has no quotes.)

Private Sub AddNote(ByVal value As Integer, ByVal octave As Integer, ByVal noteName As String)

Dim octaveQuotes As String = GetOctaveQuotes(octave)

If octave < 5 Then

Notes.Add(value, octaveQuotes & noteName)

ElseIf octave > 5 Then

Notes.Add(value, noteName & octaveQuotes)

Else

Notes.Add(value, noteName)

End If

End Sub

You’ll note that I use yet another helper function (GetOctaveQuotes) to get the actual number of quotes which are appended or prepended. The number of quotes increases as you move away from the middle octave:

Private Function GetOctaveQuotes(ByVal octave As Integer) As String

Dim octaveQuotes As New StringBuilder

If octave < 5 Then

octaveQuotes.Append("'", 5 - octave)

Else

octaveQuotes.Append("'", octave - 5)

End If

Return octaveQuotes.ToString

End Function

And now we have everything we need to initialize the notes. My form’s Load() event calls that initialization, and also creates the four tracks that we’ll support:

Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load

Song.AddTrack()

InitializeNotes()

End Sub

Form events

There are only two interesting events in my code – changing the channel on a track, and saving the music to a MIDI file.

Track changes

When the track changes, I want to disable or enable the corresponding text box based on whether or not the channel is blank. If it is blank, then I disable the text box and clear its contents; otherwise, I enable it. Rather than create a separate event handler for each combo box, I address them all in one handler:

Private Sub Track1Channel_SelectedIndexChanged(ByVal sender As System.Object, ByVal e As System.EventArgs) _

Handles Track1Channel.SelectedIndexChanged, Track2Channel.SelectedIndexChanged, Track3Channel.SelectedIndexChanged, Track4Channel.SelectedIndexChanged

By casting the sender object to a ComboBox, I’ll be able to determine which combo box sent the event, and which text box corresponds with it. (The index value will be used later to identify the song’s specific track which corresponds to a given combo/text set.)

Dim combo As ComboBox = CType(sender, ComboBox)

Dim text As TextBox

Dim index As Integer = 0

If combo Is Track1Channel Then

text = Track1

index = 0

ElseIf combo Is Track2Channel Then

text = Track2

index = 1

ElseIf combo Is Track3Channel Then

text = Track3

index = 2

Else

text = Track4

index = 3

End If

Now, it’s pretty straightforward to the activations or deactivations. The “blank” choice is in index 0 in my combobox, so if the selected index is 0, I turn everything off (after a prompt, of course), otherwise, I turn everything on. In either case, I update the appropriate track with its new channel:

If combo.SelectedIndex = 0 Then

If MsgBox("Delete all of the track information?") = MsgBoxResult.Ok Then

text.ReadOnly = True

text.Text = ""

Song.Tracks(index).TrackData.Clear()

Song.Tracks(index).Channel = -1

Else

' Change the index back -- Channel must be non-zero

combo.SelectedIndex = Song.Tracks(index).Channel - 1

End If

Else

text.ReadOnly = False

Song.Tracks(index).Channel = combo.SelectedIndex - 1

End If

Note that I’m cheating a little in that code – I happen to know that channel 0 is in index 1, channel 1 is in index 2, etc. This allows me to take a shortcut and a priori know that the channel is simply the selected index minus 1, rather than getting the text from the combobox and converting it to an integer.

Saving the music to a MIDI file

In order to save the music the user has specified, I need to translate the text he or she entered into MIDI format bytes and insert them into the tracks. I do the bulk of this work via a function called GenerateTrack, which takes a textbox and it’s corresponding track index as arguments, and which returns False if the track generation failed (because of a bad format or because it was not a valid track). Throughout this method, I will throw an exception any time I see a badly formatted music string so I can handle those errors all in one place.

Private Function GenerateTrack(ByVal trackTextBox As TextBox, ByVal index As Integer) As Boolean

The first thing to do verify that that track is valid (i.e., has a valid channel assigned to it), and clear out any existing track information:

If Not Song.Tracks(index).ValidTrack Then

Return False

End If

Song.Tracks(index).TrackData.Clear()

Next, I retrieve the music text and remove any spaces in the text:

Dim musicText As String = trackTextBox.Text

musicText = musicText.Replace(" ", "") ' Remove whitespace characters

I’ve then defined a couple of index counters to allow me to walk through the musicText, as well as a Double which will allow me to remember rests (which, in my application, are just delays between two notes and not really a thing in and of themselves):

Dim currentIndex As Integer = 0

Dim count As Integer = 0

Dim restBeats As Double = 0.0

Now, I loop over the characters in the string to begin parsing them:

While currentIndex < musicText.Length()

Step 1 is to get the beats associated with a note. I start with the current character and keep stepping until I find something that isn’t a number or a decimal point. (And If I find a decimal point, I make sure that I haven’t already found one on this note.) If I find no numbers, or too many decimals, or a decimal without a fractional part, I throw an error. Otherwise, I convert the number to a true Double. During all of this, currentIndex points to the presumed beginning of the beats, and count contains the number of characters to consume in the beats:

Dim beats As Double = 0.0

Dim dot As Boolean = False

While (musicText(currentIndex + count) >= "0" _

AndAlso musicText(currentIndex + count) <= "9") _

OrElse musicText(currentIndex + count) = "."

If musicText(currentIndex + count) = "." Then

If dot = True Then

Throw New ApplicationException("Bad beat format in track " & _

index.ToString())

Else

dot = True

End If

count += 1

End While

' Now get the beats!

If count = 0 OrElse (count = 1 AndAlso dot = True) Then

Throw New ApplicationException("No beats found for a note in track " & _

index.ToString())

End If

beats = Double.Parse(musicText.Substring(currentIndex, count))

Having gotten this far, I update currentIndex point past the beats and reset count to zero – time to get the note!

currentIndex = currentIndex + count

count = 0

Dim note As Integer = 0

Getting the note is very similar to getting the beats, except in this case I’m paying attention to quotes and accidentals instead of decimal points. Quotes can appear on either the left side or the right, but not both, so I need to have a Boolean to indicate if I’ve already found quotes on the left side:

Dim foundQuotes As Boolean = False

While musicText(currentIndex + count) = "'"

count += 1

foundQuotes = True

End While

Next I’ll look for the note. Since the note (excluding the # or b) only takes one character, this is an easy check to see if the next character is A through G (notes) or R (a rest):

If (musicText(currentIndex + count) < "A" _

OrElse musicText(currentIndex + count) > "G") _

AndAlso musicText(currentIndex + count) <> "R" Then

Throw New ApplicationException("Unrecognized note in track " & _

index.ToString())

End If

count += 1

And then I check for a flat or sharp and update the counter if I find one.

If musicText(currentIndex + count) = "#" _

OrElse musicText(currentIndex + count) = "b" Then

count += 1

End If

Now I check again for quote marks, remembering that if I found them on the left side, I’d better not find them on the right side (a note can’t be both higher and lower than the middle octave):

While musicText(currentIndex + count) = "'"

If foundQuotes = True Then

Throw New ApplicationException("Quotes on both side of a note in track " & _

index.ToString())

End If

count += 1

End While

Now I can get the note and look it up in my collection, and then update the counters again to grab the volume:

Dim key As String = musicText.Substring(currentIndex, count)

note = Notes(key)

currentIndex = currentIndex + count

count = 0

Okay, that’s two down. The final thing I need to retrieve from a given “dTv” triplet is the volume, which is an integer. However, rests don’t have volumes, and since I’ve defined a rest as NumberOfNotes, I can just check that value to see if I need to read a volume:

Dim volume As Integer = 0

If note <> NumberOfNotes Then ' Rests don't have volumes

(If the user provided a volume for a rest, I’ll catch that later and correctly throw an error when I try to parse the next note.)

Since this is just an integer, the rest of the code is easy – I keep crawling while the current character is 0 through 9, and if I get to the end and have found nothing, I’ll throw an error. Otherwise, I’ll just translate the volume to a true Integer and update the counters for the next thing to find:

While currentIndex + count < musicText.Length() _

AndAlso musicText(currentIndex + count) >= "0" _

AndAlso musicText(currentIndex + count) <= "9"

count += 1

End While

If currentIndex = 0 Then

Throw New ApplicationException("No volume found for a note in track " & index.ToString())

End If

volume = Integer.Parse(musicText.Substring(currentIndex, count))

currentIndex = currentIndex + count

count = 0

End If

That’s it for the parsing of that note; now I can update the track with the results. There are two cases here: either I have a note, or I have a rest. In the case of a rest, all I do is cache the duration of the rest. If it’s a note, I turn the note on after any cached rest, turn it off after a certain number of beats, and then clear the rest cache:

If note = NumberOfNotes Then

restBeats = beats ' A rest; just have the next note start later

Else

Song.Tracks(index).AddNoteOnOffEvent(restBeats,_

MIDI.Track.NoteEvent.NoteOn, CByte(note), CByte(volume))

Song.Tracks(index).AddNoteOnOffEvent(beats, _

MIDI.Track.NoteEvent.NoteOff, CByte(note), 0)

restBeats = 0

End If

I’ve finished working with the note, so I’ll see if there’s another note by checking the value of currentIndex compared to the length of the string. If I’m at the end, I can exit the while loop:

If currentIndex >= musicText.Length Then

Exit While ' No more notes!

End If

Otherwise, I’ll verify that the next character is a comma. If it is, then I’ll increment appropriately and loop back. Otherwise, it’s an error:

If musicText(currentIndex) = "," Then

currentIndex += 1

Else

Throw New ApplicationException ("Missing comma in track " & index.ToString())

End If

End While

Being out of the While loop, I need to verify that data got saved. If currentIndex is non-zero, then I know that I saved data (because it I ran into trouble later, I would have thrown an error). If I didn’t save data, then I need to return False:

If currentIndex = 0 Then

Return False ' Nothing got written out

End If

If we’ve gotten this far, then the generation was successful, and we’re done:

Return True

End Function

Now, finally, I can support the “Save” button. I’ll double click on it to generate the handler, and then start adding code. First, I need try to generate each track. If all of them returned False, then there was nothing to save, and I’ll abort the operation. But, since GenerateTrack() can also throw exceptions, I’ll add a Try-Catch structure to catch those. The catch handler will display the error to the user, and then abort the save:

Private Sub Save_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Save.Click

Try

If GenerateTrack(Track1, 0) = False And _

GenerateTrack(Track2, 1) = False And _

GenerateTrack(Track3, 2) = False And _

GenerateTrack(Track4, 3) = False Then

MsgBox("No tracks had valid data to write out")

Return

End If

Catch ex As Exception

MsgBox("Error while saving: " & ex.Message.ToString)

Return

End Try

It’s important to use “And” for that instead of “AndAlso,” because otherwise you’ll short-circuit tracks and not write them in.

Next, I need to pop up the save dialog box to determine where to save the file to. I’ll pre-populate this to call the file “Untitled.mid” and have it point to “My Documents:”

Me.SaveMIDIDialog.DefaultExt = "MID"

Me.SaveMIDIDialog.FileName = _

My.Computer.FileSystem.CombinePath(My.Computer.FileSystem.SpecialDirectories.MyDocuments, _

"Untitled.mid")

Me.SaveMIDIDialog.InitialDirectory = _

My.Computer.FileSystem.SpecialDirectories.MyDocuments

Me.SaveMIDIDialog.Filter = "VB MIDI files (*.MID)|*.MID"

Then, assuming that the user didn’t cancel, I just call Save on the MIDI song:

Dim result As DialogResult = Me.SaveMIDIDialog.ShowDialog

If result = DialogResult.OK Then

Song.Save(SaveMIDIDialog.FileName)

End If

End Sub

And that’s it! Launch the app, and you can start inserting music, save it, and then double-click on the resulting file to play it in Windows Media Player (or whatever you’ve got MIDI file set to). Here’s a couple samples from Howard Shore’s “The Fellowship of the Ring” score:

Sample 1:

First track, channel 0: 1B'60,1.75C''65, 0.25B'70, 0.25C''75, 0.25B'80, 0.25A'85, 0.25C''90, 1.0B'95, 3.0E'80

Second track, channel 1: 4.0G60, 4.0A60

Third track, channel 2: 4.0'D#60, 4.0'F#60

Fourth track, channel 3: 4.0''C60, 2.0'''B80, 1.0''C#60, 1.0''D60

Sample 2:

First track, channel 0: 2.0D60, 1.5C70, 0.25C70,0.25C70, 3.5D70, 0.25G60, 0.25A60, 1.5Bb60, 0.25A70, 0.25G70, 1.5F70, 0.25G70, 0.25A70, 2.0G70, 1.0F65, 1.0E60

Second track, channel 12.0'A60, 1.5'A60, 0.25'A70,0.25'A70, 3.5'A70,0.25'G60, 0.25'A60, 2.0'F60, 2.0'A70, 2.0'G70, 1.0'F65, 1.0'G60

Third track, channel 2: 2.0'F#60, 1.5'F60, 0.25'F70,0.25'F70, 4.0'F#70, 2.0'D60, 2.0'F70, 2.0'D70, 1.0'D65, 1.0'E60

Fourth track, channel 3: 2.0''D60, 1.5''F60, 0.25''F70,0.25''F70,4.0''D70, 2.0''Bb60, 2.0''A70, 2.0''D70, 1.0''Bb65, 1.0'C60

This is obviously just the tip of the iceberg – by adding to this application, I could assign different instruments to different tracks (via the “ProgramChange” event), persist metadata containing lyrics, and so on. (I don’t plan on doing that myself, since for me I simply wanted to learn more about MIDI, but who knows? Maybe later....) The completed code (along with another method which describes setting metadata) is attached to this blog post, and will also be posted on my Temple of VB site.

‘Til next time,

--Matt--*

Veni, MIDI, Vici: Generating a simple MIDI file using VB, part 1 (Matt Gertz)

VBTeam — Fri, 27 Feb 2009 21:01:00 GMT

As I’ve alluded to in previous blogs, music has always been a big part of my life, particularly performance music. I’ve been a clarinet and saxophone player for many years, am an avid singer, and (with the help of friends) I’ve done my best to teach myself piano and drums.

Composition of music, however, has always been a problem for me. I’ve written a few small pieces, but any attempt to write something larger has always ended up with me giving up after fifty or so measures. Programs to help a person create their own compositions have certainly come a long way, but they all have their idiosyncrasies. For example, one aspect of this that I’ve never been able to wrap my head around are MIDI channels and instruments – invariably, I always get these set wrong as my brain seems incapable of grokking the nuances of those areas. I’m still waiting for a low-cost composition program in which I can say “Piano plays this, clarinet plays this, and bass plays that,” and ignore the details of channels and whatnot.

It was simpler (though far less powerful!) when I had a Commodore 128. This was a gift I’d received for graduation, a fun machine that had three different operating systems (C64, C128, and CP/M) built into it. In the C128-mode, the version of BASIC included in the OS had a “Play” command which took a string of characters representing note pitches and durations. There were three voices which you could set to different instruments (well, envelopes, really), and the Play() call was asynchronous, so you could make chords by coding three Play() commands in a row. In practice, there was still a certain lag between the voices – it was not a speedy machine – but it was still a lot of fun to set up music in it.

I’ve decided that I’d enjoy the composition process more, given the tools that I have, if I understood how MIDI actually worked. After searching around on the Internet, I found a good article at Skytopia (http://www.skytopia.com/project/articles/midi.html) which listed out the basics of the MIDI format and also provided “for further information” links. Given that, I set out to create a MIDI-generation program which would mimic my C-128 experience, as well as give me some grounding in the MIDI format.

Caveat: I’m sure many readers will be far more knowledgeable about MIDI than I am, and will justifiably roll their eyes at this code due to its limited scope. I will only be generating simple type-1 MIDI files that only use the Note On and Note Off events, and which don’t use metadata, targeting a “Save” scenario only. The idea of this exercise was to understand MIDI and how to code against it in VB, not to generate a fully-functional music generation program. Coding up “Load” and supporting more of the MIDI functionality would be a more time-consuming process which maybe I’ll get to one day.

I started out by creating a Windows Application called VBMidi. But before I coded up the form, I took some time to craft out some support code to help me, as covered in the next section.

The gist of MIDI

MIDI files are composed of a well-defined header containing general information about the music, followed by a series of tracks containing actual notes, durations, and volumes for each track. To capture this, I added a new class to my application called MIDI. (Right-click the project, choose “Add,” and then choose “Class…”)

In this class, I set up some of the boilerplate information:

Public Class MIDI

' These are fixed data:

Dim MIDIHeader() As Byte = {&H4D, &H54, &H68, &H64, &H0, &H0, &H0, &H6}

Dim SubFormatType() As Byte = {&H0, &H1} ' Type-1 MIDI file (as opposed to Type-0)

MIDIHeader is always the first thing in the MIDI file, and just identifies this as a MIDI file. SubFormatType, on the other hand, is a two-byte field which, in my case, contains the bytes to identify this as a type-1 MIDI file (i.e., supporting multiple tracks, unlike type-0). Note that I’m using hexadecimal to define the bytes. When working with byte information, it’s traditional to use hex instead of decimal, as it’s a lot easier to see what will happen when bytes are concatenated or split apart into “nibbles” (groups of 4 bits).

The speed of the music is something I’ll also want to track at a high level, and I define that here:

Const ticksPerBeat = &H80

' These could be changed (in theory) by the program

Dim Speed() As Byte = {&H0, ticksPerBeat } ' Default to 128 ticks per beat

Speed can support two bytes, but as I’m defaulting to 128 (= &H80) ticks per beat, the first byte is 0.

All of the other data is owned by the tracks. I opted to create a nested class called “Track” to hold it. One of the first things I did in Track was to define the events that could exist:

Public Class Track

Public Enum NoteEvent

NoteOff = &H8

NoteOn = &H9

' Advanced

AfterTouch = &HA

ControlChange = &HB

ProgramChange = &HC

ChannelPressure = &HD

PitchWheel = &HE

End Enum

In this example, I’ll only be using the first two events – turning a note on, and turning it off. Each Track also has header and exit information:

' These are fixed data

Dim TrackHeader() As Byte = {&H4D, &H54, &H72, &H6B}

Dim TrackOut() As Byte = {&H0, &HFF, &H2F, &H0}

And then there’s all of the stuff in-between, such as the actual note data (stored in bytes) and metadata (also stored in bytes). I’ll be defining a container for metadata for the sake of completeness, even though I won’t be using it in this example:

' These can be changed by the program

Public TrackData As New List(Of Byte)

Dim TrackMetadata As New List(Of Byte)

Music can be played on one of 16 channels (&H0 through &HF). I’ve opted to create a “one channel per track” rule in my code, although this isn’t a requirement in the MIDI format. I’ve used channel = -1 to indicate that the track isn’t being used (even using a signed byte, I still have space for this, since the channel information only takes half of a byte), and I created a function to tell me if that’s the case:

Public Channel As SByte = -1

Public Function ValidTrack() As Boolean

Return Channel >= 0

End Function

Now, I need a way to actually add a note to the track. That function looks like this:

Public Sub AddNoteOnOffEvent(ByVal beatOffset As Double, ByVal ev As NoteEvent, _

ByVal note As Byte, ByVal volume As Byte)

This method requires a bit of explanation. First, let’s cover the arguments passed in:

· The caller will pass in the note’s beat offset (the difference between this event and the previous one) via a Double -- so, for example, assuming we’re in 4/4 time with a quarter note taking one beat, an eighth-note offset would be passed in as 0.5.

· The NoteEvent determines whether I’m turning on or off a note (there are other event possibilities, but as I’ve mentioned before, I’m going to ignore them for this application).

· The Note is just an index into the list of possible tones (for example, middle C is a 60 = &H3C).

· Finally, volume is just a value between 0 and 127.

Now, into the code for the function. If the track currently has no channel set for it, then I don’t bother adding the note and instead just return, since the note information has to be combined with the channel information.

If Not ValidTrack() Then Return

The next thing I do in that code is translate the note’s beatOffset to something MIDI can understand. The duration is with respect to the ticks per beat (defined earlier in the constant called rate as being 128 ticks per beat), so I multiply the two numbers together to get the number of ticks, casting it as an unsigned integer into something called tickOffset.

Dim tickOffset As UInt32 = CType(beatOffset * rate, UInt32)

Now I can actually start adding bytes to the TrackData. Assuming that the NoteEvent is either an On or Off, I’ll first add the tickOffset into the data. However, I can’t just blast in the value of tickOffset. MIDI requires a special format for durations which is a little screwy. Instead of saying “the duration data lasts this many bytes, and here they are,” they instead require a format where the final byte in the duration sequence is less than 128 (i.e., less than &H80). So, 127 ticks would be formatted as &H7F, but 128 ticks would be formatted as &H80 &H00, and this increases up to maximum possible value of &HFF &HFF &HFF &H7F, representing 268435455 ticks. I have a helper function to create this formatting for me, which I found at http://jedi.ks.uiuc.edu/~johns/links/music/midifile.htm and which I translated to VB:

Private Function TranslateTickTime(ByVal ticks As UInt32) As Byte()

Dim value As UInt32 = ticks

Dim buffer As UInt32

buffer = ticks And &H7F

value = value >> 7

While value > 0

buffer = buffer << 8

buffer = buffer Or ((value And &H7F) Or &H80)

value = value >> 7

End While

' The encoded values are now in the buffer backwards, so retrieve them...

Dim blist As New List(Of Byte)

While True

blist.Add(CByte(&HFF And buffer))

If (buffer And &H80) > 0 Then

buffer = buffer >> 8

Else

Exit While

End If

End While

Return blist.ToArray

End Function

I’m not going to get too deep into that function; suffice to say that I step through each byte in the ticks variable, retrieve up to &H7F bytes worth, increment the adjacent byte to a base of &H80 if it goes over, and continue through the ticks in that way. When I’m done, the bytes will be in reverse order in a buffer, so I pop bytes off that buffer and into a list of bytes, from which I generate a byte array to pass back. Back in the AddNoteOnOffEvent method, I then call it thusly:

If ev = NoteEvent.NoteOn OrElse ev = NoteEvent.NoteOff Then

TrackData.AddRange(TranslateTickTime(tickOffset))

which will add all of the bytes in the array that I returned into the TrackData. The next byte that needs to go in contains both the event and the channel – each of those takes up one nibble in the byte, so I’ll shift the event four places to the left and “or” the result with the channel. Since Channel is currently a signed byte and I need it to be a nibble, I need to explicitly cast it to Byte. (I also “and” it with &HF to eliminate the upper nibble in the resulting channel byte, though that really isn’t required since the upper nibble should already be zero.)

TrackData.Add((ev << 4) Or (CByte(Channel) And &HF))

Note and Volume are, thankfully, already in the correct format (byte), so I just add them directly.

TrackData.Add(note)

TrackData.Add(volume)

End If

End Sub

TrackData now contains the music, and I now create a Save() function for the track object to write it out to the MIDI file. This is pretty straightforward except for TrackSize. Since that value is an integer and I don’t necessarily know what order its bytes will be persisted in (it varies based on chip architecture), I’m explicitly saving it out byte-by-byte, using shifting and casting to make it work correctly:

Public Sub Save(ByVal filepath As String)

If ValidTrack() Then

My.Computer.FileSystem.WriteAllBytes(filepath, TrackHeader, True)

Dim TrackSize As UInt32 = TrackData.Count() + TrackMetadata.Count() + TrackOut.Count()

Dim byteTrackSize(0 To 3) As Byte

byteTrackSize(0) = CByte((TrackSize >> 24) And &HFF)

byteTrackSize(1) = CByte((TrackSize >> 16) And &HFF)

byteTrackSize(2) = CByte((TrackSize >> 8) And &HFF)

byteTrackSize(3) = CByte((TrackSize And &HFF))

My.Computer.FileSystem.WriteAllBytes(filepath, byteTrackSize, True)

My.Computer.FileSystem.WriteAllBytes(filepath, TrackData.ToArray, True)

My.Computer.FileSystem.WriteAllBytes(filepath, TrackMetadata.ToArray, True)

My.Computer.FileSystem.WriteAllBytes(filepath, TrackOut, True)

End If

End Sub

That’s it for the Track subclass. Back in the MIDI class, I can add code to set up the tracks:

Public Tracks As New List(Of Track)

Public Function AddTrack() As Track

Dim t As New Track

Tracks.Add(t)

Return t

End Function

And then a Save() method to save the MIDI header information and all of the tracks. Again, this is pretty straightforward. I save out the header and format information to the file, determine how many tracks are valid and persist that information out to the file in the appropriate byte-order, save out the speed data for the MIDI file as well, and then iterate through each track and let each one save its data (the track will handle the case where’s it’s invalid). I’ve wrapped the whole thing up in a try-catch structure in case there’s a problem accessing the file at any time.

Public Sub Save(ByVal filepath As String)

Try

My.Computer.FileSystem.WriteAllBytes(filepath, MIDIHeader, False)

My.Computer.FileSystem.WriteAllBytes(filepath, SubFormatType, True)

Dim numTracks As UShort = 0

For Each t In Tracks

If t.ValidTrack Then

numTracks += 1

End If

Dim byteTracks(0 To 1) As Byte

byteTracks(0) = CByte((numTracks >> 8) And &HFF)

byteTracks(1) = CByte((numTracks And &HFF))

My.Computer.FileSystem.WriteAllBytes(filepath, byteTracks, True)

My.Computer.FileSystem.WriteAllBytes(filepath, Speed, True)

For Each t In Tracks

t.Save(filepath)

Catch ex As Exception

MsgBox("Unable to save the file due to the following error:" & vbCrLf & ex.Message)

Return

End Try

End Sub

One thing I should note: you’ll see that the first call to WriteAllBytes uses “False” for its third parameter, whereas the other all use “True.” A value of True means that the data will be appended to the file instead of overwriting the file, so it’s the first call I make that overwrites any existing file, and the subsequent calls append data to the new incarnation of that file.

So, that’s it for the support code. In part two of this series, I’ll use this code to drive the music editing functionality.

‘Til next time,

--Matt--*