Non-Destructive Media Edits

For this column, I decided to implement something I've seen a need for before, but never taken the next step to actually create. If you have audio and video files on your computer (and who doesn't these days!), perhaps there are sections of those files that you don't care for as much. It could be commercials in a Windows XP Media Center Edition recorded TV file, or explicit lyrics in a song. For some files you can edit the files in a program such as Windows Movie Maker or various audio editing utilities, but then you would need to either keep two copies of the file, or overwrite the original. You are also limited by what you can modify with DRM-protected formats, or files which are just too big to edit effectively. Wouldn't it be great if you could just mark out the offensive sections of your media and skip or mute the sections automatically during playback without modifying the actual media file? I thought so!

This article requires Visual C# 2005 Express Edition or Visual Basic 2005 Express Edition. You can find the Express Edition downloads at http://lab.msdn.microsoft.com/express/. You can also install the Windows Media Player 10 SDK for samples and help files, though this isn't a requirement. Code downloads for this article are available in both C# and Visual Basic.

To simplify the code, I started with the .NET sample included with the Media Player 10 SDK. If you choose to install the SDK, you can find the original code at C:\WMSDK\WMPSDK10\samples\dotNet\csharp (assuming you install in the default location). This application demonstrates interacting with the Windows Media Player media library, standard play/pause/stop/forward/reverse buttons, seeking within a file, and embedding a viewer control in a managed form. Most of the original code is intact with only small changes, with the rest of the code added on in new methods.

Figure 1: Original user interface

I decided that the easiest way to structure the user interface was to allow the user to mark sections of the media during viewing. The start of a section is marked as A, with the end of the section being B. The section within these marks is then either muted or skipped based on an action indicator. The sample covered many basic Windows Media playback needs, so turned out to be a great starting point. The next step was to add the ability to capture the time codes for the A and B marks, and a drop-down box to indicate the action to take. Finally, the marks can be added to a list, with basic add and remove capabilities.

As the media plays, the current position is evaluated against the list of marks. If the current position is found to be within a pair of marks, the position is either advanced to skip, or the sound is muted to the next mark. This provides a very seamless viewing experience, and depending on the source media, may not even be noticeable. New marks can be added or removed even during playback, and individual marks can be set active or inactive without needing to remove them from the list. All marks are saved to disk at the same location as the original file ({filename}.marks) and loaded automatically when the media is subsequently loaded.

Diagram 2: The improved user interface with marking controls

How it Works

Windows Media Player has a rich object and event model, making it easy to tie into what it is doing at any time. Using the OpenStateChange event, you can detect media events such as media connecting, media waiting, playlist changed, and other event related to opening and changing streams. The PlayStateChange event lets you know events such as player is ready, playing, reconnecting, scanning, or stopped.

The Player object contains a property, currentPosition, representing the number of seconds into the media. If the user clicks the Mark A or Mark B button, the current position is saved to the corresponding label in the UI. The Action drop-down box can then be used to skip or mute the corresponding section, and the mark can be added to the list. Each change to marks results in a save.

To indicate the current position in the file using a slider, a timer is triggered every 250ms. When the timer fires, the current position is retrieved from the player control, compared against the media duration, and a new position for the slider is set. This turns out to also be a good place to evaluate the list of marks to see if the media has played to a marked section. This is checked in the EvaluateMarks() section.

Visual C#

private void EvaluateMarks()
{
    if (Player.playState == WMPPlayState.wmppsPlaying
    || Player.playState == WMPPlayState.wmppsScanForward
    || Player.playState == WMPPlayState.wmppsScanReverse)
    {
        double pos = Player.Ctlcontrols.currentPosition;
        bool inMark = false;

        foreach (MediaMarkListViewItem mark in lvMarks.Items)
        {
            if (mark.Active && (pos >= mark.MarkA && pos <= mark.MarkB))
            { 
                switch (mark.Action)
                {
                    case MediaMarkAction.SKIP:
                        Player.Ctlcontrols.currentPosition = mark.MarkB;
                        break;

                    case MediaMarkAction.MUTE:
                        Player.settings.mute = true;
                        currentlyMuting = true;
                        break;
                }
                inMark = true;
            }
        }                
        if (!inMark)
        {
            if (currentlyMuting)
            {
                Player.settings.mute = false;
                currentlyMuting = false;
            }
        }
    }
}

Visual Basic

Private Sub EvaluateMarks()
    If (Player.playState = WMPPlayState.wmppsPlaying) Then
        Dim pos As Double = Player.Ctlcontrols.currentPosition
        Dim inMark As Boolean = False

        For Each mark As MediaMarkListViewItem In lvMarks.Items
            If (mark.Active AndAlso ((pos >= mark.MarkA) _
                        AndAlso (pos <= mark.MarkB))) Then

                Select Case (mark.Action)
                    Case MediaMarkAction.SKIP
                        Player.Ctlcontrols.currentPosition = mark.MarkB
                    Case MediaMarkAction.MUTE
                        Player.settings.mute = True
                        currentlyMuting = True

                End Select
                inMark = True
            End If
        Next

        ' In not in a mark, but muting, then un-mute
        If Not inMark Then
            If currentlyMuting Then

                Player.settings.mute = False
                currentlyMuting = False
            End If
        End If

    End If
End Sub

The first check is to confirm that a clip is currently playing. If the user drags the scrollbar, it could cause strange behavior if the scrollbar snapped to positions based on marks. So then, the current position is retrieved and set aside. The "For Each" loop just compares the current position with each mark range, and skips or mutes accordingly, finally un-muting if not in any mark.

Internally, marks are saved as MediaMark objects, which are then added to MediaMarkListViewItem objects to be displayed in the ListView control. This may seem confusing, but by subclassing the ListViewItem object, it simplifies other application code and provides better object encapsulation. When a MediaMark object is created, it is embedded in a MediaMarkListViewItem for display. The original MediaMark object is also stored in a generic List object for easier serializing.

The MediaMark object must be marked with the Serializable attribute, but then saving and loading is trivial.

Visual C#

XmlSerializer s = new XmlSerializer(typeof(List<MediaMark>));
TextWriter w = new StreamWriter(Player.currentMedia.sourceURL + ".marks");
s.Serialize(w, marks);
w.Close();

Visual Basic

Dim s As XmlSerializer = New XmlSerializer(GetType(List(Of MediaMark)))
Dim w As TextWriter = New StreamWriter( Player.currentMedia.sourceURL & ".marks")
s.Serialize(w, marks)
w.Close()

Loading them is just as easy. Just tie into the OpenStateChange event, look for the wmposOpeningUnknownURL state, then load the marks to ensure that they are available as soon as a new media file prepares to play.

Extending the Application

This application was fun to write, and provides some great opportunities to extend it. For one thing, the labels for time codes would probably make more sense as textboxes in order to make minor changes. It would also make sense to allow labels to describe sections. This is actually in the object model now, but not supported in the user interface.

Being able to export the actual media in cut sections based on the marks would be a nice benefit for people transferring tapes or LPs to digital files, but would definitely add some complexity. Calling out to a command-line utility based on marks might be a workable solution.

Supporting DVDs and Internet streams would also be helpful. Obviously, the marks would need to be saved to a common location since the original locations are not writeable. Perhaps using user profile directories would make sense.

Finally, taking the basic idea and integrating it into a Media Player or XP Media Center Edition plug-in would make for a more seamless viewing experience. Instead of requiring the special player, this would be more natural during playback.

Conclusion

Get started with Visual Studio 2005 Express and Windows Media Player interoperability today! It's easier than you would have guessed, and can lead to some very rich user interactions. Download the Visual Studio Express Edition of your choice at http://lab.msdn.microsoft.com/express/, and use your imagination! Once you start working with Windows Media Player, you will see how easy it is, and ideas for projects will keep coming. Now see what you can come up with!