Jeu George's Blogs

Pre-Computing

Pre Computing is a technique normally used to speed up algorithms and by actually computing results of a smaller problem and using interpolation or other techniques to solve a larger problem. You trade-off memory/accuracy for speed here, as you will need to load the pre-computed solutions into memory and/or use interpolation to find solutions to a desired level of accuracy.

A similar approach can be used to count number of set bits in an integer. Basically, you have a pre-computed array of the number of bits in 4,8 or 16 bit size numbers and use this to find the number of bits in an integer. You could also pre-compute for 32 bit sizes but you will need 2³⁴ bytes to hold this data J

static int PreCompute8 [256] =

 {

      0, 1, 1, 2, 1, 2, 2, 3, 1, 2, 2, 3, 2, 3, 3, 4,

      1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,

      1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,

      1, 2, 2, 3, 2, 3, 3, 4, 2, 3, 3, 4, 3, 4, 4, 5,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,

      2, 3, 3, 4, 3, 4, 4, 5, 3, 4, 4, 5, 4, 5, 5, 6,

      3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,

      3, 4, 4, 5, 4, 5, 5, 6, 4, 5, 5, 6, 5, 6, 6, 7,

      4, 5, 5, 6, 5, 6, 6, 7, 5, 6, 6, 7, 6, 7, 7, 8

 };

int BitCountPreCompute (unsigned int u)

{

    // Valid for 32 bit ints only

    return PreCompute8[u&0xffu] +  PreCompute8[(u>>8)&0xffu] +  PreCompute8[(u>>16)&0xffu] +  PreCompute8[(u>>24)&0xffu] ;

}

PreCompute8 holds the number of set bits in for all 8 bit numbers. Using this result, mask out sets of 8 bits in the given integer and index into the PreCompute8 array and compute the count.

Now that we have seen some trivial and  fast approaches to do bit counting. Lets look into one brilliant parallent counting solution.

Parallel  Counting

5.  MIT HAKMEM Count

HAKMEM (Hacks Memo) is a legendary collection of neat mathematical and programming hacks contributed mostly by people at MIT and some elsewhere. This source is from the MIT AI LABS and this brilliant piece of code orginally in assembly was probably conceived in the late 70's.

 int BitCount(unsigned int u)                         
 {
         unsigned int uCount;
    
         uCount = u - ((u >> 1) & 033333333333) - ((u >> 2) & 011111111111);
         return ((uCount + (uCount >> 3)) & 030707070707) % 63;
 }

Lets take a look at the theory behind this idea.

Take a 32bit number n;
n =  a_{31 * 2^{31 + a30 * 230 +.....+ ak * 2k  +....+ a1  * 2 + a0}};

Here ^_a0 through a₃₁ are the values of bits (0 or 1) in a 32 bit number. Since the problem at hand is to count the number of set bits in the number, simply summing up these co-efficients would yeild the solution. (^_a0 + ^_a1 +..+ ^_a31 ).

How do we do this programmatically?

Take the original number n and store in the count variable.
count=n;

Shift the orignal number 1 bit to the right and subtract from the orignal.
count = n - (n >>1);

Now Shift the original number 2 bits to the right and subtract from count;
count = n - (n>>1) - (n>>2);

Keep doing this until you reach the end.
count = n - (n>>1) - (n>>2) - ... -( n>>31);

Let analyze and see what count holds now.
n         = a_{31 * 2^{31 + a30 * 230 +.....+ ak * 2k  +....+ a1  * 2 + a0}};

n >> 1 = a_{31 * 2^{30 + a30 * 229 +.....+ ak * 2k-1  +....+ a1}};
n >> 2 = a_{31 * 2^{29 + a30 * 228 +.....+ ak* 2k-2 +....+ a2}}

;
..
n >> k = a_{31 * 2^{(31-k) + a30 * 2(30-k) +.....+ ak * 2k;}}

..
n>>31 = a₃₁;

You can quickly see that: (Hint: _^2k - 2^k-1  = 2^k-1 ;)
count = n - (n>>1) - (n>>2) - ... -( n>>31) =a₃₁+ _^a30 +..+a₀;
which is what we are looking for;

int BitCount(unsigned int u)
{
       unsigned int uCount=u;
       do
       {
             u=u>>1;
             uCount -= u;  
       }
       while(u);
}

This certainaly is an interesting way to solve this problem. But how do you make this brilliant? Run this in constant time with constant memory!!.
 

 int BitCount(unsigned int u)                         
 {
         unsigned int uCount;
    
         uCount = u - ((u >> 1) & 033333333333) - ((u >> 2) & 011111111111);
         return ((uCount + (uCount >> 3)) & 030707070707) % 63;
 }

For those of you who are still wondering whats going? Basically use the same idea, but instead of looping over the entire number, sum up the number in blocks of 3 (octal) and count them in parallel.

After this statement uCount = n - ((n >> 1) & 033333333333) - ((n >> 2) & 011111111111);
uCount has the sum of bits in each octal block spread out through itself.

So if you can a block of 3 bits

u = a_{2*2^{2 + a1*2+ a0;
u>>1 =}} a_{2^{*2 + a1;
u>>2 =}} a_2^;

u - (u>>1) - (u>>2) is a₂+^_a1+^_a0 which is the sum of bits in each block of 3 bits.

The nexe step is to grab all these and sum them up:

((uCount + (uCount >> 3)) will re-arrange them in blocks of 6 bits and sum them up in such a way the every other block of 3 has the sum of number of set bits in the original number plus the preceding block of 3. The only expection here is the first block of 3.  The idea is not to spill over the bits to adjacent blocks while summing them up. How is that made possible. Well, the maximum number of set bits in a block of 3 is 3, in a block of 6 is 6. and 3 bits can represent upto 7. This way you make sure you dont spill the bits over. To mask out the junk while doing a uCount>>3. Do and AND with  030707070707. THe only expection is the first block as I just mentioned.

What does ((uCount + (uCount >> 3)) & 030707070707) hold now?
Its 2^0 * (2^6 - 1) * sum0 +  2^1 * (2^6 - 1) * sum1 + 2^2 * (2^6 - 1) * sum2  + 2^3 * (2^6 - 1) * sum3 + 2^4 * (2^6 - 1) * sum4 + 2^5 * (2^3 - 1) * sum5
where sum0 is the sum of number of set bits in every block of 6 bits starting from the 'low' position.
What we need is sum0 + sum1 + sum2 + sum3 + sum4 + sum5;
2^6-1 is 63. Clearly a modulo with 63 will get you what you want.

Remember, that this works only with 32 bits numbers, For 64-bit numbers (well I will wait for comments or do this in the next post).

In the previous post we dicussed about some trivial straightforward ways of counting set bits. Here we will discuss some fast ways of doing this.

4.     Pre-Computing Bits:

The fastest ways to solve this problem is through look-up tables. But, like everything else, it comes with a cost. Here, you pay for memory.

  static unsigned int NumBits8Bit[256] ;          

  //Precompute this array of size 256, each element in the array denoting the number of bits in a char (0x00 to 0xff). Then separate divide the number of interest into 4 blocks of 8 bits each by masking the last 8 bits (ANDing it with 0xffU) and right shifting by 8 bits each time. Sum up the count and return

   int BitCount (unsigned int u)

   {

      return NumBits8Bit [u & 0xffU]

          +  NumBits8Bit [(u>> 8) & 0xffU]

          +  NumBits8Bit [(u>>16) & 0xffU]

          +  NumBits8Bit [(u>>24) & 0xffU] ;

   }

You can write several variations to this algorithm, basically changing the amount of bits you want to pre-compute. But, remember that the more bits you pre-compute, the lesser the number of look-ups. In most common cases you can precompute 4,8,16 bits. Doing a 32-bit lookup will take up quite some memory.

In the next post we will look into some parallel counting, constant time without memory and then we will also look into some system defined functionalilty to do this.

 Bits and Pieces

Over the years, I have come across interesting approaches and problems involving bit manipulations. Over this series, I will post a wide range of related solutions and algorithms.

Counting Number of Set Bits

1.     The Trivial Straightforward approach

In this approach, the number is looped through all its bits one by one using a right shift, and checking if the low bit is set, in every iteration by AND’ing with 0x1u

int BitCount(unsigned int u)

{

           unsigned int uCount = 0;

           for(; u; u>>=1)

                uCount += u & 0x1u;   

           return uCount;  

}

This algorithm will iterate through every bit, until there are not more set bits.  Worst case performance occurs when the high bits is set.

2.     Set Bit Iterator

Here, the line u &= u-1 flips the highest set bit in each iteration, until there are no more set bits.

int BitCount (unsigned int u) 

{ 

         unsigned int uCount=0 ;

         for(; u; u&=(u-1))

            uCount++;

         return uCount ;

}

Running time is proportional  to the number set bits, Useful when there are less number of set bits in the number

3.     UnSet Bit Iterator

This is similar to the above method, expect that all bits are toggled before iteration, and uCount is decremented whenever a set bit (originally unset) is encountered.

int BitCount (unsigned int u) 

{ 

        unsigned int uCount= 8 * sizeof(unsigned int); //number of bits in unsigned int

        u ~= (unsigned int) -1 ; // Toggle bits

        for(; u; u&=(u-1))

            uCount--;

        return uCount ;

}

Running time is proportional to the number set bits, Useful when there are less number of unset bits in the number

In the next post we will discuss, fastest ways to solve this problem, and after that we will look into some tricky, nifty and brilliant solutions some of which are constant time, the ones Larry was referring

Color Spaces

The human eye is often able to detect many more colors than digital devices can reproduce. For instance, if you look at a blank, white page of paper, your eye is probably detecting at least 100 distinct shades of white. A white wall can easily have 1500 shades of white.

High-quality digital cameras, scanners, and other image acquisition devices can also detect hundreds of thousands or even millions of colors. Because of the presence of so many detectable colors, imaging professionals have invented models for specifying colors. These models are called color spaces.

The reason these models are referred to as color spaces is that most of them can be mapped into a 2-D, 3-D, or 4-D coordinate system similar to a Cartesian coordinate system. Hence colors can be said to be composed of coordinates in a 2-D, 3-D, or 4-D space. The color components in a color space are also referred to as color channels.

Some color spaces are intended to be independent of any device that is used to produce color images. Some are very device dependent. Both device-dependent and device-independent color spaces are discussed in the following sections:-

• RGB Color Spaces
• HSV Color Spaces
• HLS Color Spaces
• CMY and CMYK Color Spaces
• Device-Dependent Color Spaces
• Device-Independent Color Spaces