Michael Hopcroft

Home Automation Part 3 - Using Regular Expressions to Parse Sensor Data

2007-01-08T10:02:00Z

In the previous installment I managed to set up basic communications with the sensor network and try out some Link45 commands using my Super-Simple Serial Console. In order to actually do anything with the sensor data, I would need a function that could crack the Link45 responses into sensor IDs and temperature and humidity values.

This seemed like a job for System.Text.RegularExpressions. My only problem was that no matter how carefully I constructed my regular expression, there would always be a bug. The edit-compile-test cycle was getting tedious so I decided to take a detour and code up a simple interactive regular expression tester using WinForms. This post describes the regular expression tester and then explains how I used it to develop the regular expression necessary to extract the sensor ID, temperature, and humidity values from the Link45 response string.

An Interactive Regular Expression Tester

My goal was to create a simple application that could show the results of a regular expression in real time as I was editing the expression. I also wanted to prototype the C# code that I would need to navigate the Regex's resulting hierarchy of Matches, Groups, and Captures to pull out the values matched by my regular expression. If you develop a lot of regular expressions you might consider using a more sophisticated and full featured application such as Chris Sells' excellent RegexDesigner.NET.

My simple application provides two text boxes, one for entering sample data and one for the regular expression (see Figure 1). Edits to either the sample data or the regular expression trigger an application of the regular expression, the results of which are shown in the bottom pane in real time.

Figure 1. A very simple interactive regular expression evaluator.

The application is built in WinForms using C#. You can find the complete source code and executable in an attachment at the end of this post. Here's a rough outline of the steps you would need to take to write a similar application:

From Visual Studio, use File -> New Project to create a Visual C# Windows Application.
In Form1.cs [design]
1. Set the window title to "Interactive Regular Expression Evaluator"
2. Add a Label control with Text = "Sample Data"
3. Add a TextBox control named SampleData to hold the sample input string.
4. Add another Label with Text = "Regular Expression"
5. Add a TextBox named RegularExpression to hold the text of the regular expression.
6. Add another Label with Text = "Results"
7. Add a read-only TextBox named Results to display the results of the expression application.
8. Set all TextBox anchors to include Top, Left, and Right. Set the anchor for the Results TextBox to include Bottom.
Wire up the TextChanged event handlers for SampleData and RegularExpression to invoke a new method called ApplyRegularExpression().
Modify the constructor of Form1 to call ApplyRegularExpression() method on application startup.
Write the ApplyRegularExpression() method.

The heart of the application is the ApplyRegularExpression() method which is shown in Listing 1. Whenever the user changes either the sample data or the regular expression, the ApplyRegularExpression() method is invoked to update the results panel. Let's walk through the code.

Listing 1. The ApplyRegularExpression() method updates the results panel whenever the sample data or regular expression changes.

   1:  void ApplyRegularExpression()

   2:  {

   3:      try

   4:      {

   5:          Results.Text = "";

   6:          string results = "";

7:

   8:          Regex r = new Regex(RegularExpression.Text);

   9:          MatchCollection matches = r.Matches(SampleData.Text);

  10:          for (int i=0; i < matches.Count; ++i)

  11:          {

  12:              Match m = matches[i];

  13:              results += "Matches[" + i.ToString() + "]";

  14:              results += "\n\r\n\r";

  15:              for (int j = 0; j < m.Groups.Count; ++j)

  16:              {

  17:                  Group g = m.Groups[j];

18:

  19:                  results += "    .Groups[";

  20:                  string name = r.GroupNameFromNumber(j);

21:

  22:                  if (name == j.ToString())

  23:                      results += name;

  24:                  else

  25:                      results += "\"" + name + "\"";

  26:                  results += "]\n\r\n\r";

27:

  28:                  for (int k=0; k < g.Captures.Count; ++k)

  29:                  {

  30:                      Capture c = g.Captures[k];

  31:                      results += "        .Captures["+k+"]=\"" + c.Value + "\"\n\r\n\r";

  32:                  }

33:

  34:              }

35:

  36:          }

  37:          Results.Text = results;

  38:          RegularExpression.ForeColor = Color.Black;

  39:      }

  40:      catch (System.Exception e)

  41:      {

  42:          Results.Text = e.Message;

  43:          RegularExpression.ForeColor = Color.Red;

  44:      }

  45:  }

The try-catch block (lines 3-44) is used to catch errors that occur during the application of the regular expression. If an error is detected, the error message is shown in the results pane (lines 40-44) and the regular expression text is colored red.

Normally the code enters the try block at line 3. The string called results (line 6) is used to accumulate the text that will go into the results panel. Accumulating the results into a string seems to be more performant than accumulating them directly into Results.Text, probably because each change to Results.Text forces the TextBox to redo its text layout computations. In this code, all the changes are accumulated into results and only at the end is the string copied into Results.Text.

The regular expression is applied to the sample text in line 8 and a MatchCollection returns the set of Matches in line 9. The nested for-loops in lines 10-36, 15-34, 28-32 walk through the Match/Group/Capture hierarchy, converting it into human readable text which is accumulated in the results string. The results string is used to set the Text property of the results region on line 37. The only part that is not completely straight forward is the if statement on lines 22-25 which checks to see if the Group is named in order to determine whether to output a numerical or string indexer.

To understand how to actually extract matched values from an input string, it helps to understand the hierarchical Match/Group/Capture data structure created by the Regex in line 9. The root of the hierarchy is a MatchCollection with one Match for each portion of the input string that matches the entire regular expression string (line 9). Each Match contains one or more Groups (line 15). The first Group always corresponds to the regular expression in its entirety. Subsequent Groups correspond to grouping constructs within the regular expression. A grouping construct is just a parenthesized sub-expression of a regular expression. Grouping constructs are sometimes used to capture the input text matching specific portions of a larger match. As an example, one could use a grouping construct to capture the area code from a larger expression that matches phone numbers:

(\d\d\d)-\d\d\d-\d\d\d\d

Grouping constructs can also used when applying quantifiers in regular expression. As an example, one could match the string "abcabcabc" using a quantifier in combination with an expression that matches "abc":

(abc){3}

Each Group contains one or more Captures, each of which corresponds to one of the allowed matches for an expression with a quantifier. As an example, when the expression

(dog|cat){3}

is applied to the string "dogdogcat", the resulting subexpression group will contain three Captures, one each for "dog", "dog", and "cat" (see Figure 2).

Figure 2. An subexpression with a quantifier maps to a group with multiple captures.

As you can see, the Interactive Regular Expression Evaluator makes it easy to quickly examine the Match/Group/Capture hierarchy for a particular combination of sample data and regular expression. I encourage you to follow along in the evaluator as you read through the next section which describes the regular expression that extracts useful values from the Link45 data.

Building An Expression to Extract Sensor ID, Temperature, and Humidity

In the previous installment we saw that a "D" command to the Link45 would trigger a response that looks something like

2621D453000000F1 19,17.81,64.00,44
262FD453000000E2 19,18.12,64.56,43
EOD

Basically, each sensor returns a line of data of the form

xxxxxxxxxxxxxxxx ss,ccc.cc,fff.ff,hhh

where

xxxxxxxxxxxxxxxx is a 16 digit hexadecimal sensor ID
ss is the sensor type
ccc.cc is the temperature in degrees Celsius
fff.ff is the temperature in degrees Fahrenheit
hhh is the humidity

After the last sensor, the result set is terminated by a line containing the string "EOD".

Matching the Sensor ID. Let’s start by building the portion of the regular expression that extracts the sensor ID into a named Group called “Id”. We will start by using the [] character class construct to match a single hexadecimal digit. The []-construct will match a single instance of any character that appears between the square brackets. As an example, [abc] would match ‘a’, ‘b’, or ‘c’. We can match a single hexadecimal digit with

[0123456789ABCDEF]

The – operator allows us to specify ranges of Unicode characters in a character class, leading to more compact regular expressions. As an example, [a-z] will match any lower case English letter. Using – , we can write a the following equivalent but more compact expression to match a single hexadecimal digit:

[0-9A-F]

Sensor IDs consist of a sequence of 16 hexadecimal digits. We can use the {} quantifier construct to specify the number of adjacent digits to match. The {} construct modifies a regular expression to match multiple copies of its original matching input. As an example, the regular expression a{3} will match “aaa” and is equivalent to the regular expressions aa{2} and aaa. We can match exactly 16 hexadecimal digits with

[0-9A-F]{16}

Now that we can match the sensor ID, we’d like to actually extract it from the input string. I use a parenthesized subexpression to put the sensor ID into a match Group:

([0-9A-F]{16})

Figure 3 shows the match/group/capture hierarchy for this expression when applied to sample data from the Link45. Matches[0].Groups[0] is the match for the entire regular expression. Matches[0].Groups[1] is the match for the parenthesized subexpression. Since the entire regular expression is parenthesized, the two Groups correspond to the same matched substring.

Figure 3. This regular expression extracts the first sensor ID.

Accessing the extracted sensor ID from code is simple:

System.Console.WriteLine(Matches[0].Groups[1].Captures[0].Value);

This code is somewhat brittle because it assumes that the sensor ID will always be in the second Group. Any change to the regular expression that introduces another subexpression before the sensor ID subexpression will break the code. We can fix this problem by assigning a name to the sensor ID subexpression using the ?<> construct. Here’s an expression that captures a sequence of 16 hexadecimal digits into a Group called “Id”:

(?<Id>[0-9A-F]{16})

The code to access the sensor ID now becomes:

System.Console.WriteLine(Matches[0].Groups["Id"].Captures[0].Value);

This code is better because it does not depend on the position of the sensor ID subexpression relative to other subexpressions. Now let's extend the regular expression to grab the temperature.

Matching the Temperature and Humidity. I want to use Fahrenheit temperatures so after matching the sensor ID, my expression will need to skip over the sensor type and Celsius temperature before matching the Fahrenheit temperature. Skipping over the sensor type is easy - we just need to skip over a space, a two digit sensor type, and a comma. This can be done by appending

\s\d\d,

to the end of the regular expression for the sensor ID. In this expression, \s denotes a character class that matches any whitespace and \d denotes a character class that matches any decimal digit in the range 0 to 9.

The Celsius temperature is a bit more complex because when the temperature is below freezing, it starts with a minus sign. After that there are 1-3 decimal digits followed by a decimal point followed two more decimal digits.

We can use the ? quantifier to handle the minus sign. ? modifies an expression to match 0 or 1 instances, so the expression -? matches the minus sign when its cold but doesn't complain when its hot.

Earlier I used the {16} quantifier to match exactly 16 hexadecimal digits in the sensor ID. Another form of the {} quantifier allows you to specify an upper and lower bound for the number of matches. A quantifier of the form {x,y} will match at least x, but no more than y instances. Putting it all together, we can match the Celsius temperature with

-?\d{1,3}\.\d{2}

In this expression, the period is preceded by a backslash to indicate a match to a period. Without the backslash, period corresponds to a character class that matches any character.

We can use the identical expression to match the Fahrenheit temperature, but we'll add a named subexpression to allow us to capture the value into a group:

(?<Temp>-?\d{1,3}\.\d{2})

After this, matching the humidity is a snap - it is just a pair of decimal digits. There is one catch, though. The sensor network could contain sensors that only report temperature. In this case, the humidity value and the comma that separates it from the temperature would not appear in the Link45 data. We can use the ? quantifier to make the entire humidity expression optional:

(,(?<Humidity>\d{2}))?

This expression is less efficient than it could be because it captures the humidity value twice, once with the comma and once without. The reason is that I had to add a second parenthesized subexpression in order to make the ? quantifier apply to the sequence containing the comma and the humidity. We can improve the expression by using ?: to denote a non-capturing group:

(?:,(?<Humidity>\d{2}))?

Now we're ready to put it all together and write one giant expression to grab the sensor IDs, temperatures, and humidity values from the Link45 data. We do this by appending the following expression fragments:

Match sensor ID: (?<Id>[0-9A-F]{16})
Skip space, sensor type, and comma: \s\d\d,
Skip Celsius temperature and comma: -?\d{1,3}\.\d{2},
Match Fahrenheit temperature: (?<Temp>-?\d{1,3}\.\d{2})
Match optional humidity: (?:,(?<Humidity>\d{2}))?

Here's the final, almost incomprehensible expression:

(?<Id>[0-9A-F]{16})\s\d\d,-?\d{1,3}\.\d{2},(?<Temp>-?\d{1,3}\.\d{2})(?:,(?<Humidity>\d{2}))?

Figure 4 shows the expression in action.

Figure 4. The final regular expression applied to real Link45 data.

Now that we have the regular expression, lets modify the Super-Simple Serial Console from my previous installment to poll the sensor network and print out the sensor IDs, temperatures, and humidity levels.

Polling the Sensor Network. The goal here is to write a small console application that initializes the sensor network and then every 5 seconds, takes a set of readings and prints them to the console. This application will form the heart of a data logging service that we will implement in a future installment.

It turns out that System.IO.Ports.SerialPort has a feature that allows us to greatly simplify the code from the Super-Simple Serial Console. Recall that the Super-Simple Serial Console needed a bit of multiple-thread coordination to avoid printing the prompt before the Link45 had finished responding. Since our new application only runs the Link45 "D" command, we can take advantage of the fact that all of the Link45 responses will be terminated by the characters "EOD", allowing us to use the serial port's ReadTo() method.

ReadTo() takes a single string argument and blocks until this string appears in the serial port's input buffer or a read timeout limit has been exceeded. When ReadTo() detects the termination string, it returns the string of characters received up until the termination string. The termination string itself is not returned, but it is removed from the serial port's input buffer. Listing 2 shows the sensor reading application.

Listing 2. This application reads the sensor data and prints it to the console.

   1:  using System;

   2:  using System.IO.Ports;

   3:  using System.Text.RegularExpressions;

   4:  using System.Threading;

5:

   6:  namespace SensorConsole

   7:  {

   8:      // SensorConsole reads from the sensor network every 5 seconds and prints the

   9:      // sensor id, temperature, and humidity for each sensor.

  10:      class SensorConsole

  11:      {

  12:          private SerialPort port;

13:

  14:          public void Run()

  15:          {

  16:              Console.WriteLine("Temperature and Humidity Logger");

  17:              try

  18:              {

  19:                  Open();

20:

  21:                  Regex r = new Regex(@"(?<Id>[0-9A-F]{16})\s\d\d,-?\d{1,3}\.\d{2},(?<Temp>-?\d{1,3}\.\d{2})(?:,(?<Humidity>\d{2}))?");

22:

  23:                  while (true)

  24:                  {

  25:                      try

  26:                      {

  27:                          port.Write("D");

  28:                          string s = port.ReadTo("EOD");

29:

  30:                          MatchCollection matches = r.Matches(s);

  31:                          foreach (Match m in matches)

  32:                          {

  33:                              Int64 id = Int64.Parse(m.Groups["Id"].Value, System.Globalization.NumberStyles.HexNumber);

  34:                              float temp = float.Parse(m.Groups["Temp"].Value);

  35:                              int humidity = int.Parse(m.Groups["Humidity"].Value);

  36:                              Console.WriteLine("{0:X}: {1} degrees F and {2}% humidity", id, temp, humidity);

  37:                          }

  38:                          Console.WriteLine();

  39:                      }

  40:                      catch (System.Exception e)

  41:                      {

  42:                          if (e is System.TimeoutException)

  43:                          {

  44:                              Console.WriteLine("Error: timeout reading sensor data.");

  45:                          }

  46:                          else

  47:                              throw e;

  48:                      }

49:

  50:                      // Wait 5s between sensor readings.

  51:                      Thread.Sleep(5000);

  52:                  }

  53:              }

  54:              finally

  55:              {

  56:                  if (port != null && port.IsOpen)

  57:                      port.Close();

  58:              }

  59:          }

60:

  61:          void Open()

  62:          {

  63:              port = new SerialPort("COM1", 9600, Parity.None, 8, StopBits.One);

64:

  65:              // The iButton Link45 needs DTR and RTS enabled.

  66:              port.DtrEnable = true;

  67:              port.RtsEnable = true;

68:

  69:              // Bump up the read timeout from the default 500ms to ensure time for

  70:              // reporting from large sensor networks.

  71:              port.ReadTimeout = 5000;

72:

  73:              // The serial port is now configured and ready to be opened.

  74:              port.Open();

75:

  76:              // Shock the Link45 to life by sending a break.

  77:              port.BreakState = true;

  78:              port.BreakState = false;

  79:          }

80:

  81:          void Close()

  82:          {

  83:              port.Close();

  84:          }

  85:      }

86:

  87:      class Program

  88:      {

  89:          static void Main(string[] args)

  90:          {

  91:              SensorConsole sc = new SensorConsole();

  92:              sc.Run();

  93:          }

  94:      }

  95:  }

Let's walk through the code in execution order, starting at the entry point on line 89. Main() just creates a SensorConsole and invokes its Run() method. The body of the Run() method is protected by a try-finally block (line 17-58) that ensures the serial port is closed before returning from Run().

The first order of business inside the try block is to call the Open() method (lines 61-79) which initializes the serial port. The Open() method is nearly the same as the one in the Super-Simple Serial Console. The differences are that we no longer need to wire up the serial port's DataReceived event because we are using ReadTo() and we bumped up the serial port's ReadTimeout from 500ms to 5000ms to be on the safe side in case the Link45 takes more than a half a second to return data from a large sensor network.

After returning from Open(), the regular expression is built using the Regex class (line 21). Note the @-sign immediately before the regular expression string. The @-sign indicates to the compiler that the following string should be treated literally with no special meaning for the backslashes. If I didn't use the @-sign, I would have had to escape each of the backslashes with "\\" which would have led to an even less comprehensible regular expression.

The main loop spans from line 23 to 52. Right inside the loop a try block (lines 25-48) is used to separate serial port timeout exceptions from other potential problems. As this code morphs into a more sophisticated always-on data logger, I will need to extend this error handling so that the logger stays up and running in the presence of intermittent communication or sensor failures.

Inside the try block, we issue a "D" command to the Link45 on line 27 and then wait for a response on Line 28. Assuming we didn't get a timeout exception, control passes to line 30 where the our regular expression is applied to the Link45 data. The foreach loop on lines 31-37 walks over each of the Matches, extracting the sensor ID, temperature, and humidity. Since the expression is designed to return a single Capture per Group, I've used the shorthand of m.Groups["Id"].Value instead of the more verbose m.Groups["Id"].Captures[0].Value. The Group values are returned as strings so I pass the ID, temperature, and humidity to Int64.Parse(), float.Parse(), and int.Parse() in order to get the data into the correct types. After converting the strings, we print the values back out to the console (line 36) and head to the top of the foreach loop (line 31) to process the next sensor's results. When we run out of sensors, we print out a blank line (line 38), sleep for 5 seconds (line 51) and then head to the top of the while loop (line 23) to do it all over again.

We can now read the sensor network and extract useful fields like sensor ID, temperature, and humidity. Our next step will be to log the sensor values to a SQL database.

Next Installment: Logging sensor values to a database.

Building a traffic signal with Christmas lights

2006-12-19T11:12:16Z

It started out as a simple project. My children had become interested in basic electrical circuits so I bought them some LEDs, batteries, wire, and solderless breadboard to play around with. I soon found that these components weren't ideal for a number of reasons. While the solderless breadboard makes it easy to wire up circuits, it hides many of the electrical connections inside making it harder for children to grasp the concept of a circuit. In order to light up, the LEDs must be wired with the correct polarity and their uninsulated leads must not make a short. These factors combined to make it hard for the children to consistently build circuits that would light up.

A simple idea. I had a moment of inspiration one evening while walking my dog through the neighborhood in the days leading up to Christmas. I realized that miniature Christmas lights actually only require a few volts to light up. A string of Christmas lights runs on 120v, but the bulbs are wired in series so an individual bulb needs only 2-3 volts, a safe amount that can be obtained through a pair of flashlight batteries. (you can find out everything you ever wanted to know about Christmas lights here)

Christmas lights have a number of advantages over LEDs. They come in sockets with insulated wires already attached, they come in many colors, they all use the same voltage, and they don't care about polarity. Another benefit is cost. At most hardware stores you can purchase five replacement bulbs for about a dollar, but they don't come in sockets (see Figure 1).

Figure 1. Five bulbs and no sockets for 99¢ at the local hardware store.

A better deal is to purchase 100 bulbs and 100 sockets and 100 feet of wire for about three dollars (see Figure 2) and if you wait until the day after Christmas you can probably do even better.

Figure 2. These bulbs cost about 3¢ each and come in convenient pre-wired sockets.

I ended up getting a few strings of lights along with a battery holder, some alligator clips, and a knife switch. After cutting a light out of the string I was able to make a simple circuit (see Figure 3) and I felt that it illustrated the concepts better than the LEDs and solderless breadboard. Everything in this circuit is visible and can be manipulated and arranged. I like how the knife switch really gets at the essence of closing a circuit.

Figure 3. This circuit is easy to understand because everything is visible.

To be safe, I cut off both 120v plugs from the string of lights and then gave the lights to my kids along with a wire cutter/stripper and the batteries, alligator clamps, and knife switches. They quickly made simple circuits and then proceeded to wire up lights for their Lego creations and dioramas. I probably should have stopped here.

A simple idea grows into a bigger project. As I watched the kids play with the lights I got the idea to make them a stoplight. I knew it would be too complex for them to wire up by themselves, but I figured they could make a nice cardboard box for it and put it to good use. I decided to use a 2-pole 6-throw rotary switch from Radio Shack to control 12 lamps. This would allow green, yellow, and red lamps for roads coming from 4 directions. Before getting out the soldering iron, I wrote down the a simple state table for the 6 positions of the rotary switch (see table 1). After writing down the state table, I realized that a four-position switch would have worked better since the light cycles through four states before repeating. I only had a six-position switch so I went with the circuit in figure 4.

Table 1. State diagram for 2 roads crossing.

Position	North/South Lamps	East/West Lamps
1	Green	Red
2	Yellow	Red
3	Red	Green
4	Red	Yellow
5	Green	Red
6	Yellow	Red

Figure 4. Circuit for 12 lights on a 6-position rotary switch.

The circuit looks pretty simple, but it actually took me a while to build. The time consuming parts were scavenging and testing 12 lamps (figure 5), and soldering in close quarters on the rotary switch (figure 6).

Figure 5. A stoplight seems pretty simple, but it actually has 12 lamps inside.

Figure 6. Wiring the rotary switch was tricky.

By the time I was done, I was really glad that I didn't attempt this project with the kids. They would have become bored long before the first solder melted. Figure 7 shows the completed circuit. I taped the bulbs to a board to help keep the wires under control and I made a long wire harness between the switch and the bulbs to simplify mounting in whatever enclosure the kids decide to invent.

Figure 7. What started as a simple project got complex.

Overall, I was fairly satisfied with the stoplight, but am already making plans for a better one. It turned out that the rotary switch presented a number of challenges. One was that is was very hard for the kids to turn - they just couldn't get a tight enough grip on the shaft. I can fix this by putting a knob on the switch. The other problem is that the switch has a stop after position 6. I would have preferred a switch that could advance from position 6 back to position 1. I thought about cutting off the metal stop inside the switch, but decided against it because of the risk of breaking the switch that took so long to wire up.

This may be getting out of hand. If I make another stoplight, I am going to place an emphasis on ease of assembly. I will probably use LEDs soldered directly onto a prototyping board as shown in figure 8. Instead of a complicated wiring harness, I'll use 8-conductor ribbon wire with a single inline plug.

Figure 8. An easier-to-assemble design.

I'll also replace the rotary switch with a simple sequencing circuit or perhaps a micro-controller such as a Basic Stamp. I like the micro-controller idea because I'd get to write software for features like flashing red, sequenced lights, and even pressure sensors. If I do this I'm sure you will read about it here.

Have a wonderful holiday and a happy New Year.

-Mike

Save the date for PDC '07

2006-12-14T10:52:33Z

Microsoft has just announced that the next Professional Developers Conference (PDC) will be October 2-5, 2007 in Los Angeles, with two days of pre-conference on September 30 and October 1. The last two PDCs were fantastic and this one is likely to be just as exciting and informative. You can find more information here.

Hope this helps.

-Mike

Implicit and Explicit Interface Implementations

2006-12-13T10:59:13Z

As I was putting together a post on IEnumerable and IEnumerator I was reminded of the subtleties of implicit and explicit interface implementations. C# does not support multiple inheritance, but a class has the option of implementing one or more interfaces. One challenge with interfaces is that they may include methods that have the same signatures as existing class members or members of other interfaces. Explicit interface implementations can be used to disambiguate class and interface methods that would otherwise conflict. Explicit interfaces can also be used to hide the details of an interface that the class developer considers private.

Disambiguating Methods

Let's look at an example of method disambiguation. In Listing 1 we have started to write a class called C that implements interfaces I1 and I2, each of which defines a method A().

Listing 1. Class C implements interfaces I1 and I2.

interface I1
{
    void A();
}

interface I2
{
    void A();
}

class C : I1, I2
{
    public void A()
    {
        Console.WriteLine("C.A()");
    }
}

In this case, A() is a public class member that implicitly implements a member of both interfaces. A() can be invoked through either interface or through the class itself as follows:

Listing 2. A() can be invoked from I1, I2, or C.

C c = new C();
I1 i1 = (I1)c;
I2 i2 = (I2)c;

i1.A();
i2.A();
c.A();

The output from this code is

C.A()
C.A()
C.A()

This works fine if you want A() to do the same thing in both interfaces. In most cases, however, methods in different interfaces have distinct purposes requiring wholly different implementations. This is where explicit interface implementations come in handy. To explicitly implement an interface member, just use its fully qualified name in the declaration. A fully qualified interface name takes the form

InterfaceName.MemberName

In Listing 3 we add an explicit implementation of I1's A() method.

Listing 3. Class C explicitly implements I1.A().

class C : I1, I2
{
    public void A()
    {
        Console.WriteLine("C.A()");
    }

    void I1.A()
    {
        Console.WriteLine("I1.A()");
    }
}

Now when we run the statements from Listing 2 we get

I1.A()
C.A()
C.A()

When an interface method is explicitly implemented, it is no longer visible as a public member of the class. The only way to access it is through the interface. As an example, suppose we deleted the implicit implementation of A() as shown in Listing 4:

Listing 4. Class C does not implicitly implement A()

class C : I1, I2
{
    void I1.A()
    {
        Console.WriteLine("I2.A()");
    }
}

In this case we would get a compile error saying that C fails to implement I2.A(). We could fix this error by changing the first line to

class C : I1

but we'd get another compile error when trying to invoke A() as a member of C:

C c = new C();
c.A();

This time the compiler would report that class C does not contain a definition for method A(). We get the error because the explicit implementation of I1.A() hides A() from the class. The only way to call I1.A() now is through C's I1 interface:

C c = new C();
I1 i1 = (I1)c;
i1.A();

Explicit interface implementations are sometimes necessary when classes implement multiple interfaces with conflicting member definitions. A common example involves collection enumerators that implement System.Collections.IEnumerator and System.Collections.Generic.IEnumerator. In this case both interfaces specify a get-accessor for the Current property. To create a class that compiles, at least one of the two get-accessors must be explicitly implemented.

Hiding Interface Details.

In some cases explicit interface implementation can be useful even when disambiguation is unnecessary. One example is to use an explicit implementation to hide the details of an interface that the class developer considers private. While the level of privacy is not as great as that afforded by the private keyword, it can be useful in some circumstances. Listing 5 shows a common pattern involving IDisposable. In this case, the Dispose() method is hidden by explicit implementation because the method is really an implementation detail that is not germane to the users of class MyFile. At the very least, the explicit implementation keeps the interface members out of the class' Intellisense list.

Listing 5. Using explicit implementation to hide the details of IDisposable.

interface IDisposable
{
    void Dispose();
}

class MyFile : IDisposable
{
    void IDisposable.Dispose()
    {
        Close();
    }

    public void Close()
    {
        // Do what's necessary to close the file
        System.GC.SuppressFinalize(this);
    }
}

You can read more about implicit and explicit interface implementations in this MSDN Tutorial or in Version 1.2 of the C# Language Specification.

Hope this helps.

-Mike Hopcroft

MSDN Community Content

2006-12-08T10:04:57Z

MSDN recently added Wiki functionality to the portion of the site dealing with Visual Studio 2005 and .NET Framework 2.0. This already looks like it is going to be a great resource - at the time of this post, there were almost 1000 topics with Community Content from over 300 contributors. You can read more about it at MSDN Community Content FAQ.

Here's a sample screenshot of a community content block on an MSDN library page (click the image to see a full size version).

Hope this helps.

-Mike

Home Automation Part 2 – Making Basic Measurements

2006-12-06T12:39:00Z

In the previous installment I described my decision to build a temperature and humidity sensor network using modules from IButtonLink.com. The heart of the system is the Link45 which connects the PC’s serial port to the sensor network and provides an ASCII interface to the sensors.

In previous home automation projects using .NET 1.1 I had no built-in class to access the serial port and therefore had to rely on solutions like VB6’s MsComm32.ocx or Corrado Cavalli’s excellent CRs232 class. For this project I decided to use .NET Framework 2.0 so that I could try out the new System.IO.Ports.SerialPort class.

I needed to familiarize myself with the Link45 and learn how to enumerate the sensors on the network and take a reading from an individual sensor. With this in mind, I started by implementing a Super-Simple Serial Console that would let me type commands to the Link45 and see its responses in the console window. Most of the code was straight forward (see Listing 1, below) but I did run into a few gotchas related to serial port settings, Link45 initialization, and multi-threading.

Listing 1. The Super-Simple Serial Console consists of about 100 lines of code.

   1:  using System;

   2:  using System.IO.Ports;

   3:  using System.Threading;

4:

   5:  namespace s3c

   6:  {

   7:      // Super-simple Serial Console

   8:      // Provides an interactive console session with an iButtonLink Link45.

   9:      class S3C

  10:      {

  11:          private int nCharsReceived;

  12:          private SerialPort port;

13:

  14:          public void Run()

  15:          {

  16:              Console.WriteLine("Super-Simple Serial Console");

  17:              try

  18:              {

  19:                  Open();

20:

  21:                  while (true)

  22:                  {

  23:                      // Show > prompt, read line from console and transmit to serial port.

  24:                      Console.Write(">");

  25:                      String line = System.Console.ReadLine();

  26:                      if (line.Length == 0)

  27:                          return;

28:

  29:                      nCharsReceived = 0;

  30:                      port.Write(line);

31:

  32:                      // Wait until 200ms after the last data received from port before

  33:                      // prompting for next line.

  34:                      while (true)

  35:                      {

  36:                          int n = nCharsReceived;

  37:                          Thread.Sleep(200);

  38:                          if (n == nCharsReceived)

  39:                              break;

  40:                      }

  41:                  }

  42:              }

  43:              finally

  44:              {

  45:                  if (port != null && port.IsOpen)

  46:                      port.Close();

  47:              }

  48:          }

49:

  50:          public void Open()

  51:          {

  52:              port = new SerialPort("COM1", 9600, Parity.None, 8, StopBits.One);

53:

  54:              // The iButton Link45 needs DTR and RTS enabled.

  55:              port.DtrEnable = true;

  56:              port.RtsEnable = true;

57:

  58:              // Hook up callback to display data received from Link45

  59:              port.DataReceived += new SerialDataReceivedEventHandler(port_DataReceived);

60:

  61:              // The serial port is now configured and ready to be opened.

  62:              port.Open();

63:

  64:              // Shock the Link45 to life by sending a break.

  65:              port.BreakState = true;

  66:              port.BreakState = false;

  67:          }

68:

  69:          public void Close()

  70:          {

  71:              port.Close();

  72:          }

73:

  74:          void port_DataReceived(object sender, SerialDataReceivedEventArgs e)

  75:          {

  76:              string s = port.ReadExisting();

  77:              if (s != null)

  78:              {

  79:                  nCharsReceived += s.Length;

  80:                  Console.Write(s);

  81:              }

  82:          }

  83:      }

84:

  85:      class Program

  86:      {

87:

  88:          static void Main(string[] args)

  89:          {

  90:              S3C s3c = new S3C();

  91:              s3c.Run();

  92:          }

93:

  94:      }

  95:  }

Let's walk through the code in execution-order. The program entry point is the Main() function at at line 88. Main() just news up an S3C object (S3C is short for Super-Simple Serial Console) and calls it's Run() method. The Run() method does four things:

prints a welcome message at line 16.
initializes the serial port by calling Open() at line 19.
performs the main input and serial port write loop in lines 21-41.
closes the serial port in lines 43-47.

Initializing the Serial Port. My first challenge was getting the serial port settings correct in the Open() method, lines 50-67. The Link45 had some of the port settings conveniently printed on the side of the device. These included 9600 baud, no parity and 8 bits of data. I figured out the stop bits by trial and error. All of these values were passed to the constructor of the System.IO.Ports.SerialPort in line 52. I also found that I needed to enable DTR and RTS which I do on lines 55 and 56. If you want to learn more about serial port esoterics in general, check out Wikipedia RS323.

After creating and configuring the serial port, I wire up a handler for the port's DataReceived event on line 59. This is where the multithreading comes in. Serial port communications are inherently asynchronous - once you transmit some data your main thread continues on to do other work and at some point in the future when a response arrives back at the port, the DataReceived event is fired on another thread, which results in a call to port_DataReceived() which is defined on line 74.

Once the event handler is wired up, it is time to actually open the serial port (line 62). I thought I was done configuring the port at this point, but I found in my experiments that some Link45 commands such as the help command would sometimes fail if issued before a sensor read command. I ended up fixing this problem by toggling the serial port's break state to true and then back to false (lines 65-66). This seems to have the effect of resetting the communication channel to the Link45 and ensures that all of the Link45 commands work consistently the first time.

The Main Loop. After initializing the serial port, we enter the main loop on line 21. The loop runs until an empty input line is detected (lines 26-27) or an exception is thrown (lines 43-47). At the top of the loop we print a prompt and get user input (lines 24-25) and then send the user's text to the serial port (line 30). After sending the text, we wait a brief period for a response to be displayed (lines 34-40) and then head back to the top of the loop to prompt the user for more input.

A Bit of Multithreading. Responses from the Link45 trigger the serial port's DataReceived event. For the Super-Simple Serial Console, the role of the DataReceived event handler is simple - it just reads any data that is available in the serial port's input buffer and then prints that data to the console. This all happens on lines 74-82. The only part that isn't completely straight forward is line 79:

   79:                  nCharsReceived += s.Length;

This is where the DataReceived thread communicates with the main thread. The reason we need some sort of handshaking between the threads is that we'd like the serial port responses to be interleaved with the user input - basically a repeating prompt-input-response pattern. Without this handshaking, the prompt for the next command would be printed before the response from the current command.

One way to accomplish this interleaving is to blindly Sleep() for a fixed amount of time in the main loop after every command is transmitted to the serial port. This approach will work with a long enough sleep, but it leads to a sluggish user experience because every Sleep() needs to be long enough to wait for the slowest Link45 command. Experiments showed that the Link45 was always pretty responsive, but that some commands resulted in a verbose response that could take a second or more to receive.

My approach was to wait 200ms beyond the time the last character was received. If, after 200ms, no additional characters had been received, I would assume the response was complete and head back up to the top of the loop for more user input. I used the nCharsReceived variable to track the number of characters received while waiting. I clear nCharsReceived (line 29) immediately before writing to the serial port. The DataReceived event handler updates nCharsReceived on line 79. Lines 34-39 in the main loop alternately Sleep() for 200ms and check nCharsReceived for more data until a Sleep() occurs with no new data received. Technically speaking one should probably protect nCharsReceived with a lock in case we ever encounter an architecture where integer writes aren't atomic.

Putting It All Together. Once I had the Super-Simple Serial Console working, I proceeded to wire up a pair of sensors to the serial port in the back of my laptop. The connections were easy since the sensors have RJ-45 connectors. Figure 1 shows the setup.

Figure 1. CAT-5 networking cable connects a pair of sensors to the serial port via the Link45.

After plugging in the sensors I fired up the Super-Simple Serial Console. I began by typing the character h to see the list of commands available. The transcript in Figure 2 shows the response from the Link45. You can read more about the available commands in the Link45 Users Manual.

Figure 2. First communications with the Link45.

To get the list of sensors on the network I typed the character I, requesting that the Link45 report the sensor inventory. The results are shown in Figure 3. Each sensor appears on a line consisting of a 16 hex digit sensor ID followed by a 2 digit sensor type. My network shows two temperature+humidity sensors. Here’s the cheat sheet for the sensor types:

00 – temperature
19 – temperature + humidity
1A – temperature + voltage
1B – temperature + light

Figure 3. The I command reports the list of sensors on the network.

Finally, I used the D command to request temperature and humidity readings from all sensors on the network. Figure 4 shows the results. According to the Link45 documentation, the data returned for each sensor is of the form

xxxxxxxxxxxxxxxx ss,ccc.cc,fff.ff,hhh

where

xxxxxxxxxxxxxxxx is a 16 digit hexadecimal sensor ID
ss is the sensor type
ccc.cc is the temperature in degrees Celsius
fff.ff is the temperature in degrees Fahrenheit
hhh is the humidity

My network shows that the temperatures near my computer are in the mid 60s and with humidity around 44%.

Figure 4. The D command takes temperature and humidity readings from all sensors.

The Link45 has a lot of functionality and accepts many more commands than the ones I've shown here. You can learn more from the Link45 Users Manual.

With the code from the Super-Simple Serial Console and the Link45 commands for sensor inventory and sensor reading, we have all the building blocks we need to write a program to monitor the temperature and humidity.

Next installment: Using Regular Expressions to Parse Sensor Data

MSDN Online Chat: LINQ to SQL

2006-12-04T06:45:20Z

Members of the C# team will be hosting an online chat session about LINQ to SQL on Tuesday, December 12, 2006. Here's the description:

The LINQ Project is a codename for a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. LINQ to SQL is a component of LINQ which allows access to data stored in relational databases. We’re actively designing and developing this technology - show up and join the LINQ to SQL team to participate in that process or just get a better understanding!

For more information go to http://msdn.microsoft.com/chats/.

New For VS2005 - Abstract Stub Generation

2006-12-01T05:52:00Z

Visual Studio 2005 includes a new feature to generate stubs for methods in an abstract base class. Here's an example of how it works.

Suppose you want to create a class called MyDerivedClass that inherits from an abstract class called MyAbstractClass:

Before Visual Studio 2005, you would have had to look up and then type in the function prototypes for each of the abstract methods in the inheritance chain for MyDerivedClass. Just finding these methods to implement can be hard because the inheritance chain may include multiple abstract classes and any ancestor class can implement any abstract method. If you had the source code to all of the classes in the inheritance chain you could figure out which methods were still abstract for your class and then go to their definitions and copy-paste the function prototypes, replacing the abstract keyword with override. If you didn't have the source code for the entire inheritance chain, you might end up using the Object viewer to identify and copy the function definitions. Either way, you would be in for a bit of work.

In Visual Studio 2005, a small blue rectangle called a smart tag appears whenever you start to declare a class that derives from an abstract type:

Hovering over the tag or pressing Shift+Alt+F10 will produce a menu of alternatives for implementing the abstract methods:

Clicking on the down arrow opens the menu, which in this example shows a single option:

Invoking this option implements stubs for the overrides of the abstract methods as follows:

It’s that easy. Now all you need to do is fill in the implementation for each method and you are up and running.

Hope this helps.

Mike

VS 2005 IDE Tour

2006-11-30T07:57:24Z

Karen Liu, a C# IDE program manager recently sent me the specs for all of the new C# IDE features in Visual Studio 2005. As part of my rampup as the new C# PUM, I plan to make my way through this list and blog about some of the more interesting changes. Watch this space for articles tagged 'VS 2005' and 'IDE'.

Hope this helps.

Mike

Home Automation Part 1 - Burning $200 of propane

2006-11-28T06:50:00Z

I like to use my knowledge of technology to solve problems outside of work. In this series I explain how I used C# and the Internet to remotely monitor a propane stove in my cabin. While reading about my adventures you will learn about C#, embedded technology, regular expressions, databases, web servers, web cams and much more.

A few years back my wife and I bought a cabin in the mountains and it is heated by a propane gas hearth (see figure 1). I’m guessing that the hearth was added as an afterthought because when I bought the place, there were no thermostat wires in the walls to control the hearth. The previous owner used a wireless remote control thermostat that looks like the thing you use to change the channel on the cable box. The thermostat sends signals by IR or radio or telepathy to a little box near the hearth and the little box turns the flame on and off. At least this is how it is supposed to work. The problem is that when the batteries in the controller or the little box get low, the system switches into a failsafe mode where the hearth never shuts off. I don’t know if this behavior is by design or just random and I suppose it has the advantage of keeping the pipes from freezing, but one fall I arrived at the cabin to find the hearth burning away in a balmy 100 degree room and shortly thereafter had to pay $200 to refill the propane tank. I have no idea how long the hearth had been burning.

Figure 1: The hearth at my cabin in the Cascades

My first action was to replace the wireless thermostat with a hard wired unit. Fortunately, the hearth was next to an outside wall so I was able to stand on a ladder in a blizzard and run the thermostat wire under some trim on the outside of the wall while looking in at my wife and kids reading stories on the couch. The new thermostat increased the reliability of the system, but I was still concerned because the consequences of a failure were too costly. If the hearth failed to ignite I could end up with frozen pipes. If it failed to extinguish, I’d burn through all my propane and then have the pipes freeze. Also, the propane tank lies under ten feet of snow most of the winter, making a refill problematic.

I decided to build a monitoring system that could alert me to problems at the cabin and even let me inspect the premises over the web. My first order of business was to find a simple way to monitor the temperature. I wanted something that was unobtrusive, reliable, inexpensive, could easily interface with a PC, and did not require its own power supply. A brief search of the web turned up an amazing device called a 1-wire sensor. It’s manufactured by Dallas Semiconductor and is basically a CPU with a sensor and serial I/O on a die the size of a wheat kernel. The 1-wire sensor gets its power from the data wire used for communication with the outside world. Since each sensor has a unique 64 bit serial number, it is possible to connect multiple sensors to a single data line and then interrogate each in turn by sending its serial number and a command over the wire. Dallas Semiconductor sells 1-wire sensor chips and IButtons which enclose the chip in a round stainless steel can. Other vendors provide 1-wire devices in modules with all sorts of sensors including temperature, humidity, water, light, voltage and current. I ended up purchasing temperature+humidity modules from iButtonLink.com. These sensor modules consist of a very small printed circuit holding the 1-wire chip and a pair of RJ-45 jacks for the communication line. Building a network of sensors is as easy as plugging all the modules together with CAT-5 wire. Figure 2 shows an iButtonLink temperature+humidity module. Newer versions come in nice a plastic enclosure.

Figure 2: The temperature+humidity sensor module I purchased from iButtonLink.com.

I also purchased an iButtonLink Link45 which provides an ASCII interface to the sensor network via an RS-232 port (see figure 3). The Link45 is about an inch and a half long and includes a DE-9 connector at one end for the serial port and an RJ-45 on the other for the sensor network.

Figure 3: The Link45 provides an ASCII interface to the sensor network via an RS-232 port.

With a Link45, a handful of sensor modules, 100 feet of CAT-5, and an old PC, I was ready to start coding.

Next installment: Making Basic Measurements

(edited: updated link to next installment)

Welcome

2006-11-27T21:01:50Z

Hi. I’m Michael Hopcroft, the new Product Unit Manager for Visual C#. I started the job a few weeks ago and this is my first blog post. Working with the C# team is a dream job for me. I’ve liked C# since the day it came out and used it extensively in my previous job as Development Manager for Windows Presentation Foundation. One of my first tasks as Product Unit Manager is to really learn the technology in depth while getting to know the team. I hope you will join me in the coming weeks as I explore the product and its uses.