Speaking from an assemly level, I would wager...a summation of XORs. Obviously an XOR would be 0 if the same, != 0 if different. Depending on how long the arrays were, you could compare all of the subsequences (so look at it in 8 bit or 16 bit or even 32 bit chunks if the size is a mutliple) using XOR and then add the result to an accumulator. If you had to compare long strings and had a high expectation of difference, you could compare the accumulated sum after each XOR. If the 'strings' were smaller or there was an expectation of equality, you could do all of the XORS and then check the result at the end, the difference being in a higher cost of an additonal compare at each subsequence vs. doing uneeded work. However, assuming P4, the conditional branch would be cheap with branch predication.

C/C++ ?
RtlCompareMemory/RtlEqualMemory/memcmp high-performance platform optimised code ??

C# ? Brute-force and let runtime to optimize your code or force somebody to add "int Array.CompareTo(Array other)" native function.
See http://support.microsoft.com/default.aspx?scid=kb;EN-US;307020 for pretty nice example of byte-by-byte comparation ;-)

It depends on the size of the arrays. If truly large (say multiple megabytes or better), then you get spend some time doing some real CPU optimization. :)

I saw one technique where you compare DWORD size chunks, but rather than advance to the next DWORD it jumps ahead the number of bytes in a cache line (32 for pentiums) and continues the compare. It does this for a block (I think it was 1 mb) and cycles back for the DWORD beyond the initial DWORD for each array.

The reason for this is that you take a hit for reading in a cache line of 32 bytes, but that it continues to load all 32 bytes. Rather than wait for the next 28 bytes to load, you skip ahead and start another cache line going. This makes it the same speed for the first 4 bytes of each cache line, but faster on the followups.

Some researchers in England, I think, wrote code to test this to validate Intels claims of memory access speed on the Pentium 75.

Well, you have to choose the best algorithm that suits the data set in consideration to begin with. If comparisan is a performance bottleneck then there may be alternate ways to achieve the same effect, for e.g. by maintaining a checksum etc.

Once you have chosen the approach, and it still comes down to raw byte comparisan, then you can consider micro optimizations like the one suggested above. Another pentium specific optimization is to exploit the dual pipelines of pentium, i.e. hand code the comparisan routine in assembly so that each pair instructions are executed simultaneously thereby effectively doubling the speed of execution.

PingBack from http://toenailfungusite.info/story.php?id=9514