Laurent Ellerbach

Ce blog est principalement destiné à publier des informations relatives à Microsoft, à ses technologies, aux outils Visual Studio et à ses versions Express notamment Visual Basic

May, 2012

  • Laurent Ellerbach

    Creating an efficient HTTP Web Server for .NET Microframework (NETMF)

    • 4 Comments

    That’s not the first post I’m doing on incorporating a Web Server in .NET Microframework (NETMF). In some of my previous posts, I’ve explain how to do it using the existing .NET classes for this. And it is working very well!

    The main concerns I have is that I’m using a netduino board. This board is great, I love it. But there is a very limited amount of memory available and limited amount of space to store the programs. Total storage is 64Kb and what is left when the code is in is about 48K… So very little amount of memory. And the http .NET classes are great and use stream but they are intense in terms of memory and also the main http class is huge…

    So I had to find a solution and it was to redevelop a web server like IIS or Apache. OK, I can’t compare Sourire I just want a web server which handle GET answers and respond a simple web page. The other challenge I have is to be able to reuse the code I4ve already written for my Sprinkler, to pilot my Lego city and my Lego infrared receiver like my Lego trains…

    So I was searching for code to reuse on the Internet and found some code. So I did a mix of existing code and spend some time testing various solutions Sourire Most of the existing code is not really robust. It does fail if there is a network problem, if 2 requests arrive at the same time, etc. I’m not saying my code is perfect but it is working and working well for the last month with no problem at all.

    A web server is simple, it’s just a connection on a socket and a protocol of communication which is HTTP. It is also very simple as it is text based. What is interesting is to see all what you can do with such a simple protocol and such a simple markup language like HTML and some javascript.

    OK, so let start with what is necessary: a thread that will run all the time and handle socket requests. So we need also a socket. And a way to stop the thread.

     
    private bool cancel = false;
    private Thread serverThread = null;
    
    public WebServer(int port, int timeout)
    {
        this.Timeout = timeout;
        this.Port = port;
        this.serverThread = new Thread(StartServer);
        Debug.Print("Web server started on port " + port.ToString());
    }
    

    As you can see, it is quite simple, the WebServer object is initialize with a specific port and a timeout. By default, the http port is 80 but it can be anything. There is no limitation. And as it’s easy to implement, let make the code generic enough to be able to be use with different ports. And a new Thread is created to point on function StartServer. I will detail it later. I will explain also why we need a timeout later.

    Now we have this object initialize, let start the Webserver:

    public bool Start()
    {
        bool bStarted = true;
        // start server           
        try
        {
            cancel = false;
            serverThread.Start();
            Debug.Print("Started server in thread " + serverThread.GetHashCode().ToString());
        }
        catch
        {   //if there is a problem, maybe due to the fact we did not wait engouth
            cancel = true;
            bStarted = false;
        }
        return bStarted;
    }
    
    
    
    

    That is where the fun being! We start listening and initialize a variable we will use later to stop the server if needed. The catch can contain something to retry to start, here, it just return if it is started or not. At this stage, it should work with no problem as it is only a thread starting. But who knows Sourire

    private void StartServer()
    {
        using (Socket server = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
        {
            //set a receive Timeout to avoid too long connection 
            server.ReceiveTimeout = this.Timeout;
            server.Bind(new IPEndPoint(IPAddress.Any, this.Port));
            server.Listen(int.MaxValue);
            while (!cancel)
            {
                try
                {
    
                    using (Socket connection = server.Accept())
                    {
                        if (connection.Poll(-1, SelectMode.SelectRead))
                        {
                            // Create buffer and receive raw bytes.
                            byte[] bytes = new byte[connection.Available];
                            int count = connection.Receive(bytes);
                            Debug.Print("Request received from " 
    + connection.RemoteEndPoint.ToString() + " at " + DateTime.Now.ToString("dd MMM yyyy HH:mm:ss")); //stup some time for send timeout as 10s. //necessary to avoid any problem when multiple requests are done the same time. connection.SendTimeout = this.Timeout; ; // Convert to string, will include HTTP headers. string rawData = new string(Encoding.UTF8.GetChars(bytes)); string mURI; // Remove GET + Space // pull out uri and remove the first / if (rawData.Length > 5) { int uriStart = rawData.IndexOf(' ') + 2; mURI = rawData.Substring(uriStart, rawData.IndexOf(' ', uriStart) - uriStart); } else mURI = ""; // return a simple header string header = "HTTP/1.1 200 OK\r\nContent-Type: text/html
    ; charset=utf-8\r\nConnection: close\r\n\r\n"
    ; connection.Send(Encoding.UTF8.GetBytes(header), header.Length, SocketFlags.None); if (CommandReceived != null) CommandReceived(this, new WebServerEventArgs(connection, mURI)); } } } catch (Exception e) { //this may be due to a bad IP address Debug.Print(e.Message); } } } }

     

    This function will run all the time in a thread. It’s in an infinite loop which can be break by the cancel variable. First, we need to initialize the Socket. We will use IPv4 with a stream and the TCP protocol. No timeout to receive the request. The, you’ll have to bind this socket to a physical IP address. In our case, we will use all IP address on the port initialized before. Any IP address mean all addresses and in our case only 1 IP address as we do have only 1 Ethernet interface. We are using '”using” to make sure the server Socket will be closed and cleaned properly after usage.

    The way it is working is not too complicated. Remember that we’ve open a Socket named Server, setup it to listen to port 80. This is running in a separate thread in this thread. So in order to analyze the information returned when a connection is accepted (so when a Browser ask for a page), we need to create another Socket pointing to the same Socket, here “using (Socket connection = server.Accept())”. In this case “using” allow the code to clean in the “proper way” when the thread will be finished or then the loop end or when it goes back to the initial loop. It’s thread in thread and if you don’t close things correctly, it can quickly let lots of objects in the memory, objects which will be seen as alive by the garbage collector.

    When there are bytes ready to read with connection.Poll, we just read them. The request is transformed into a string. An http request look like “GET /folder/name.ext?param1=foo&param2=bar HTTP/1.1”. Areal life example looks more like this: "GET /folder/name.ext?param1=foo&param2=bar HTTP/1.1\r\nAccept: text/html, application/xhtml+xml, */*\r\nAccept-Language: fr-FR,fr;q=0.8,en-US;q=0.5,en;q=0.3\r\nUser-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)\r\nAccept-Encoding: gzip, deflate, peerdist\r\nHost: localhost:81\r\nConnection: Keep-Alive\r\nX-P2P-PeerDist: Version=1.1\r\nX-P2P-PeerDistEx: MinContentInformation=1.0, MaxContentInformation=1.0\r\n\r\n"

    For a normal, full web server like IIS or Apache, you’ll analyze all those parameters, and there are lots, see the W3C protocol here. For our usage, the only thing that interest us is the full URL. And it is located between the 2 first spaces. And we will extract the URL, remove the first ‘/’ as I will not use it in the rest of the code.

    Now, the next step is to start answering the request. When someone ask you something, it’s polite to answer Sourire Like for the request, the response need to have couple of header information. And as my usage is extremely simple, I will always consider that it is OK, I’m only delivering text content and that the connection can be closed. By the way, whatever you put there, HTTP is a disconnected protocol, so you should never consider that you are always connected! It’s an error and can drive you to very bad behaviors.

    connection.Send return the first part of the message and then I call an event to tell the creator of the WebServer object that something happened. I send of course the connection object so that the caller will be able to create an HTML page and answer and also the URL so that it can analyze it.

    Last but not least, the try and catch is extremely important. With Sockets a problem can quickly arrive due to a network problem. And I’ve seen it happening on the netduino for no reason. Just capturing the problem and not doing anything makes the web server working for months! Even if you loose the network, the catch will capture the problem and the server will continue to work up to the point the network will work again. The other reason to use it is because of the timeout. If something happen between the client and our webserver, after the timeout, you’ll get in this catch and you’ll start a new socket and the process will go back to something normal. It can happen and happened to me with very long HTML pages I was generating. When I was interrupting the creation and ask for a new page, the socket went into a kind of infinite loop waiting for a request. There should be a smart way to check is something goes well or not but it’s an easy way.

    public delegate void GetRequestHandler(object obj, WebServerEventArgs e);
    public class WebServerEventArgs: EventArgs
    {
        public WebServerEventArgs(Socket mresponse, string mrawURL)
        {
            this.response = mresponse;
            this.rawURL = mrawURL;
        }
        public Socket response { get; protected set;  }
        public string rawURL { get; protected set; }
    
    }
    
    public event GetRequestHandler CommandReceived;
    

    Right after the header is sent back, an event is raised. The arguments are simple here, we do send the Socket object and the URL. If you want to enrich the web server, you can add other elements like the header element rather than sending them right away, the browser requesting the page, the IP address or whatever you want! Again, simple and efficient there.

    Last but not least if you need to stop the Server, you’ll need a function to this and also to clean the code at the end:

    private bool Restart()
    {
        Stop();
        return Start();
    }
    
    public void Stop() { cancel = true; Thread.Sleep(100); serverThread.Suspend(); Debug.Print("Stoped server in thread "); }
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
    
    protected virtual void Dispose(bool disposing)
    {
        if (disposing)
        {
            serverThread = null;
        }
    }
    

    Nothing too complex there, it’s just about pausing the thread (remember, there are other tread attached in it), closing the other thread leaving in the Server object and cleaning everything. I hope it’s the good code to let it clean Sourire But at the end of the day, my only interest is to let this server running all the time. So I don not really care if it will stop correctly!

    Now, to use the server, easy:

    private static WebServer server;
    
    // Start the HTTP Server
    
    WebServer server = new WebServer(80, 10000);
    server.CommandReceived += new WebServer.GetRequestHandler(ProcessClientGetRequest);
    // Start the server.
    server.Start();
    

    Declare a static WebServer if you want it to be unique. Technically, you can have multiple servers running on different port. In my case, no need for this. Then, it’s about creating the object, adding an event and starting the server!

    private static void ProcessClientGetRequest(object obj, WebServer.WebServerEventArgs e)
    

    And you are ready to do some treatment into this function. To return part of the answer, just use e.response.Send as for the header part and you’re done!

    To simplify the process, as it’s a function which you’ll have to call often, I’ve created a function to do this:

    public static string OutPutStream(Socket response, string strResponse)
    {
        byte[] messageBody = Encoding.UTF8.GetBytes(strResponse);
        response.Send(messageBody, 0, messageBody.Length, SocketFlags.None);
        //allow time to physically send the bits
        Thread.Sleep(10);
        return "";
    }
    

    This can be a function you add in your main code or can add in the Web Server code.

    Now you have a fully functional simple Web Server. You can read the previous article on how to handle parameters and analyzing them. The code to manage the parameter is now in the WebServer class.

    I’ll post the code in CodePlex so anyone will be able to use it as a helper.

    Enjoy this code Sourire Again, I’m just a marketing director doing some code. And it’s the code running in my sprinkler management system for the last month without any problem!

  • Laurent Ellerbach

    Using a SPI device with netduino and .NET micro framework

    • 0 Comments

    After playing with I2C, with various IO, I’ve decided to play a bit with SPI Sourire The real project behind using a SPI device is to be able to use it a a multiplexer/demultipler of IO.

    I want to pilot lights for my Lego train. The idea is to have the ability to switch a signal light to green or red. I want to be able to have 16 lights. The netduino board have only 14 digital IO and 6 analogic. And I’m already using some to pilot the Lego train thru infrared. See previous article.

    So I have to find a multiplexing solution. And one is used by most of the netduino fan: SPI. In terms of electronic it’s simple, it is a serial to parallel (and vice versa as you can read also). The component the most used for this is 74HC595. There are couple of basics explanations on the netduino wiki and also good examples as always on Mario Vernari blog.

    Each 74HC595 can have 8 output and they can be chained. Which mean that you can use 2 component to have 16 output. And you basically put them in series. One output of one 74HC595 to the input of the others.

    I spend quite a bit of time to understand how to plug the 74HC595 to the netduino. So here is what to do:

    • Pin 8 is ground
    • Pin 16 is VCC (+5V recommended)
    • Pin 14 to netduino pin D11 (called MOSI) for the first 74HC595 of the chain. For the next one it has to be linked to the Pin 9 of the previous 74HC595
    • Pin 13 to ground (used to activate or not the 8 output)
    • Pin 12 to one output port you’ll choose on the netduino. Let say pin D10. This one is used to select the 74HC595 when you have multiple 74HC595 used in parallel for different purposes (called SS)
    • Pin 11 to netduino pin D13 (called SCLK)
    • Pin 10 to +VSS (this one is used to reset if needed)
    • Pin 15, 1, 2, 3, 4, 5, 6, 7 are the output pin
    • Pin 9 has to be linked to the next 74HC595 of the chain if there are more than 1.

    And that’s it for the basics! Now what I want to do with the output is to be able to light one led green or switch it to red. So I’ll use the output of the 74HC595. When it will be 1, I’ll light the green one, when it will be 0, I’ll light the red one. Here is the hardware to schematic:

    image

    When the signal will be 1, the current will go thru the green light light. And when it will be 0, the inverter will switch to 1 and the red light will light. It just need 3 cables per light and the current can be transported on long distance. The RS resistor can be adapted with a lower resistor to 100 or something like this in order to send a more intense current in the cable as there will be some loss if it is very long. by the way I recommend to read the excellent article on the length of cable and impact on signal noise from Mario.

    Now, in terms of software, It’s not so difficult. In my project, I’m using the SPI for multiple purpose. So I need to use it in a smart way. SPI can be shared and the only thing to do is to change the configuration and write (and read) right after. So I’m using the MultiSPI class from the NETMF Toolbox. It’s an impressive toolbox containing lots of helpers. It goes from hardware to HTTP web client. There is a 74HC595 also but I found it more complex to use that just directly using the MultiSPI class.

    Here the example of class to switch to green (1) or red (0) a specific output of the 2 74HC595:

    public class Signal
    {
        private byte mNumberSignal;
        private bool[] mSignalStatus;
        public const byte NUMBER_SIGNAL_MAX = 16;
        private MultiSPI MySignal;
    
        public Signal(byte NumberOfSignal)
        {
            mNumberSignal = NumberOfSignal;
            if ((mNumberSignal <= 0) && (mNumberSignal > NUMBER_SIGNAL_MAX))
                new Exception("Not correct number of Signals");
            mSignalStatus = new bool[mNumberSignal];
            // open a SPI
            MySignal = new MultiSPI(new SPI.Configuration(
                Pins.GPIO_PIN_D10, // SS-pin
                false,             // SS-pin active state
                0,                 // The setup time for the SS port
                0,                 // The hold time for the SS port
                false,              // The idle state of the clock
                true,             // The sampling clock edge
                1000,              // The SPI clock rate in KHz
                SPI_Devices.SPI1));   // The used SPI bus (refers to a MOSI MISO and SCLK pinset)
    
            //initialise all signals to "false"
            for (byte i = 0; i < mNumberSignal; i++)
                ChangeSignal(i, true);
        }
    
        public byte NumberOfSignals
        { get { return mNumberSignal; } }
    
        public void ChangeSignal(byte NumSignal, bool value)
        {
            if ((NumSignal <= 0) && (NumSignal > mNumberSignal))
                new Exception("Not correct number of Signals");
            //need to convert to select the right Signal
            mSignalStatus[NumSignal] = value;
            // fill the buffer to be sent
            ushort[] mySign = new ushort[1] { 0 };
            for (ushort i = 0; i < mNumberSignal; i++)
                if (mSignalStatus[i])
                    mySign[0] = (ushort)(mySign[0] | (ushort)(1 << i));
            //send the bytes
            MySignal.Write(mySign);
            
    
        }
    
        public bool GetSignal(byte NumSignal)
        {
            if ((NumSignal <= 0) && (NumSignal > mNumberSignal))
                new Exception("Not correct number of Signals");
            return mSignalStatus[NumSignal];
        }
    }
    

    I just need to declare a MultiSPI. The SPI Configuration contains the necessary information to setup the 74HC595. The SS pin is set to D10. So I can use the SPI for other purposed to pilot my Lego infrared modules. And in the Lego module, I’ll have also to declare the SPI as MultiSPI.

    What the MultiSPI class is doing is very simple. It has a static SPI in its private variables. And a SPI.Configuration which is not static.

    /// <summary>Reference to the SPI Device. All MultiSPI 
    devices use the same SPI class from the NETMF, so this reference is static
    </summary> private static SPI _SPIDevice; /// <summary>SPI Configuration. Different for each device, so not a static reference</summary> private SPI.Configuration _Configuration;
    // Store the configuration file for each MultiSPI instance
    Sets the configuration in a local value
    this._Configuration = config; // If no SPI Device exists yet, we create it's first instance if (_SPIDevice == null) { // Creates the SPI Device only 1 time as it is a static! _SPIDevice = new SPI(this._Configuration); }
    public void Write(byte[] WriteBuffer)
    {
        _SPIDevice.Config = this._Configuration;
        _SPIDevice.Write(WriteBuffer);
    }
    

     

    So each time a program want to write in a SPI, the right configuration is selected and the write command is send. It’s working like this to read. The full class is a bit more complex but the main principle is there.

    The rest of my code is very simple and I’m sure I would have been to write it in a better way. I’m just storing the state of a signal and then output all the buffer to the SPI.

    So bottom line, it is very easy to use SPI as a multiplexer/demultipler for IO. That was what I needed. I did not tested yet how to read the data but it should be as simple!

  • Laurent Ellerbach

    Managing my Sprinklers from the Cloud

    • 1 Comments

    I’ve started a project to pilot my sprinklers remotely and make them smart almost 9 months ago. The idea is to be able to remotely open and close my sprinklers but also be able to launch automatic sprinkling cycles.

    After some software and hardware development, I now have a fully working solution:

    WP_000611

    You can see the netduino .NET Microframework board where all the smart is happening. I wrote couple of articles already to show how to implement a web server, launch timers, create calendars, get the date and time. I’ve also wrote couple of articles on hardware.

    Recently, I’ve met with Alain, a Lego fan like me and I show him the way I can control any Lego Power Function element using .NET Microframework. And we discussed about my sprinkler project. I was about to implement the final solution with my existing 9V Gardena electro valves. And he challenged ma on this telling me that I will never know the status of the electro valve if I hacked the Gardena system. And he tell me to go for a 24V DC electro valve. The big advantage is that when you switch on the current, it open the valve, when it’s off, it close. So no unknown state like with the bistable  valve from Gardena. So I bought 3 24V DC electro valve for 22€ each plus couple of cents for the diode (never forget to add a return current diode on a solenoid Sourire).

    WP_000612

    And the display screen with the button you see is a Crouzet automate. It’s a gift from Alain and it’s really cool to have it. For my project, it can be replaced by 3 buttons, 3 leds and 3 transistors but it’s just more cool Sourire The netduino enter in the DC input section and when the voltage is higher than 2V (normal high level is 3.3V), it just switch on an electro valve. So nothing complicated there.

    What has been long is to bring electricity when I wanted this box to be plus networking plus bring cables to all the places where I have electro valves. And as I have in front and rear of the house, I have lots of cables to install nicely in the basement. And just this took 1 full day with a friend who helped me. Thanks Thomas by the way Sourire

    And now from my Windows Phone or a PC or any other device which support HTML, I can open, close and program my sprinklers:

    image

    I’m an happy geek Sourire And as there are always some improvement to do, next step is to add a I2C temperature sensor to get information on the temperature outside and implement automatic rules.

    Stay tune!

Page 1 of 1 (3 items)