Laser Tracking Camera

Mounting your webcam on the motor
I have simply mounted my webcam on a plastic cap. I used a heated nail for poking a hole through the cap's center:

Then, I fixed it on the motor by pushing the motor shaft through the hole:

I taped my webcam on the cap..and that was it.

My setup looks very sloppy, and I'm sure yours will look much better than mine. :) Anyway, let's move on to the fun part!

Finding the laser dot...

Our code will locate the laser dot by finding the brightest pixel in the webcam's field of view. If something in its view is brighter than the laser, the camera will simply turn towards that point instead of the laser dot. :) The program captures images from a webcam, using Andrew Kirillov's motion detection code for image acquisition. Then, it scans the pixels in the bitmaps captured. Based on their respective RGB (Red Green Blue) values, it determines how bright they are. We could use the Bitmap.GetPixel() function for accessing pixels and calculating the brightness of any pixel in a bitmap:

using System.Drawing;

Color c = someBitmap.GetPixel(x,y);

float b = c.GetBrightness();

Wow, that's easy! This code was simple to write, and easy to understand. However, unfortunately, it is VERY slow. If you use this code, it might take several seconds to process a single image becase the GetPixel()/SetPixel() methods are too slow for iterating through bitmaps of any real size. So, in this project, we'll make use of the BitmapData class in GDI+ to access the information we want. The BitmapData only allows us to access the data it stores through a pointer. This means that we'll have to use the unsafe keyword to scope the block of code which accesses the data. Based on an article by Eric Gunnerson, here's a class which will perform very quick unsafe image processing:

   1: public unsafe class UnsafeBitmap

   2: {

   3:     Bitmap bitmap;

   4:  

   5:     int width;

   6:     BitmapData bitmapData = null;

   7:     Byte* pBase = null;

   8:  

   9:     public UnsafeBitmap(Bitmap bitmap)

  10:     {

  11:         this.bitmap = new Bitmap(bitmap);

  12:     }

  13:  

  14:     public UnsafeBitmap(int width, int height)

  15:     {

  16:         this.bitmap = new Bitmap(width, height, PixelFormat.Format24bppRgb);

  17:     }

  18:  

  19:     public void Dispose()

  20:     {

  21:         bitmap.Dispose();

  22:     }

  23:  

  24:     public Bitmap Bitmap

  25:     {

  26:         get

  27:         {

  28:             return (bitmap);

  29:         }

  30:     }

  31:  

  32:     public struct PixelData

  33:     {

  34:         public byte blue;

  35:         public byte green;

  36:         public byte red;

  37:     }

  38:  

  39:     private Point PixelSize

  40:     {

  41:         get

  42:         {

  43:             GraphicsUnit unit = GraphicsUnit.Pixel;

  44:             RectangleF bounds = bitmap.GetBounds(ref unit);

  45:  

  46:             return new Point((int)bounds.Width, (int)bounds.Height);

  47:         }

  48:     }

  49:  

  50:     public void LockBitmap()

  51:     {

  52:         GraphicsUnit unit = GraphicsUnit.Pixel;

  53:         RectangleF boundsF = bitmap.GetBounds(ref unit);

  54:         Rectangle bounds = new Rectangle((int)boundsF.X,

  55:       (int)boundsF.Y,

  56:       (int)boundsF.Width,

  57:       (int)boundsF.Height);

  58:  

  59:         width = (int)boundsF.Width * sizeof(PixelData);

  60:         if (width % 4 != 0)

  61:         {

  62:             width = 4 * (width / 4 + 1);

  63:         }

  64:         bitmapData =

  65:       bitmap.LockBits(bounds, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);

  66:  

  67:         pBase = (Byte*)bitmapData.Scan0.ToPointer();

  68:     }

  69:  

  70:     public PixelData GetPixel(int x, int y)

  71:     {

  72:         PixelData returnValue = *PixelAt(x, y);

  73:         return returnValue;

  74:     }

  75:  

  76:     public void SetPixel(int x, int y, PixelData colour)

  77:     {

  78:         PixelData* pixel = PixelAt(x, y);

  79:         *pixel = colour;

  80:     }

  81:  

  82:     public void UnlockBitmap()

  83:     {

  84:         bitmap.UnlockBits(bitmapData);

  85:         bitmapData = null;

  86:         pBase = null;

  87:     }

  88:     public PixelData* PixelAt(int x, int y)

  89:     {

  90:         return (PixelData*)(pBase + y * width + x * sizeof(PixelData));

  91:     }

  92: }

Be sure to check Eric's article on unsafe image processing.
This class can be used for retrieving the red, green and blue values of any pixel as shown below:

UnsafeBitmap uBitmap = new UnsafeBitmap (original_bitmap);

uBitmap.LockBitmap();

PixelData pixel = uBitmap.GetPixel (x,y);

uBitmap.UnlockBitmap();

You can also alter the red, green and blue values of any pixel in your bitmap:

UnsafeBitmap.PixelData pData = new UnsafeBitmap.PixelData();

pData.red = value;

pData.green = value;

pData.blue = value;

uBitmap.SetPixel (x, y, pData);

To find a laser dot in a bitmap, we have to go through the pixels in the bitmap and calculate each pixel's brightness to find the brightest one. Based on a pixel's RGB values, you can calculate brightness using this formula:

Brightness = (299 * red + 587 * green + 114 * blue) / 1000

This would give a value between 0 to 255.
Here's a sample code to check the brightness of each pixel in a 320 x 240 bitmap:

for (int y = 0; y < 240; y++)

{ 

    for (int x = 0; x < 320; x++) 

    {

        byte red, green, blue;

        red = uBitmap.GetPixel(x, y).red;

        green = uBitmap.GetPixel(x, y).green;

        blue = uBitmap.GetPixel(x, y).blue;

        float brightness = (299 * red + 587 * green + 114 * blue) / 1000;

        if (brightness > certainValue)

            // Do something

    }

}

After our program has the x and y coordinates of the laser dot, it can calculate the number of steps required to turn the webcam towards that point.

Using the Software
Since the program searches for the brightest pixel in the camera's field of view, the lighting conditions of your room can affect its performance. So, adjust the brightness threshold and lighting conditions so that nothing (except the laser) exceeds the brightness threshold.

If your stepper motor doesn't align your webcam with the laser dot properly, try adjusting the pixels-per-step track bar. It tells the program how much your stepper motor moves with each step. My stepper does 40 pixels/step (20 pixels while half stepping). Even though this isn't necessary, you can actually measure how many pixels your stepper moves in one step. Just comment out the calls to the MoveStepper() event in MotionDetector1.cs and run the program. Keep a laser dot fixed somewhere in your webcam's field of view and check its x coordinate in the Output window. After that, move the stepper one step in any direction with your mouse wheel, and see how much its x coordinate changes. Read my previous article for information on how to move a stepper with a mouse wheel, or simply download the code for doing it from here. Well, I still think it's not necessary that you actually measure how much your stepper moves using this tedious technique. Just play with the pixels-per-step track bar until you get things working perfectly.

Now what?

Now that you can track laser, and have a camera actually follow it, here are some interesting ideas to try:

• Wire two stepper motors to your computer. Then, you could make a dual axis, pan/tilt assembly and have your camera follow laser dots in any direction.
• Use two lasers and make an object tracking camera panner, which makes use of laser rangefinding to track the edge of any moving object.

Conclusion
We've reached end of this article. I hope you enjoyed reading it. Use your imagination and ideas to extend this project. If you come up with something interesting, I'd love to hear about it. :) Happy coding.

About the Author

Ashish Derhgawen is an IT student, currently living in New Delhi, India. He has been coding since fourth grade. Some of his other interests are harmonica playing, wildlife and cricket. When he’s not at school, he spends his time working on unusual projects related to robotics, webcams, and electronics besides others. You can reach Ashish through his blog at http://ashishrd.blogspot.com.

Filed under: hardware hacks, robotics