There's a subtle difference between comparing floating point values with the Equals method and comparing them with the == operator.  (In all the code I show in this post, I use the Double class, however everything I say also applies to the Single class).

When the following code is run, it compiles and produces the results you would expect:

double pi1 = 3.1415926535;
double pi2 = 3.1415926535;

Console.WriteLine("For PI:\n\tDouble.Equals() : {0}\n\tOperator == : {1}",
    Double.Equals(pi1, pi2),
    pi1 == pi2);
For PI:
    Double.Equals() : True
    Operator == : True

Both values of pi compare to be the same regardless of using the Equals method or operator.  However, the following code produces a more interesting result:

double nan1 = Double.NaN;
double nan2 = Double.NaN;
        
Console.WriteLine("For NaN:\n\tDouble.Equals() : {0}\n\tOperator == : {1}",
    Double.Equals(nan1, nan2),
    nan1 == nan2);

When this is run, the results are:

For NaN:
    Double.Equals() : True
    Operator == : False

Why would Double.Equals() produce a result of true, while the == operator produces a result of false?  Well, to start with, lets check out what a Double is.  According to section 7.2 of the ECMA spec (in Partition II):

Class Library Type CIL Type Description
System.Single float32 IEC 60559:1989 32-bit float
System.Double float64 IEC 60559:1989 64-bit float

So what's this IEC 60559:1989 stuff?  It's a standard for the binary format of floating point types.  The C# Standard points out in section 3 that this standard is more commonly known in the United States as ANSI/IEEE Standard 754-1985, or the IEEE Standard for Binary Floating-Point Arithmetic.

According to IEC 60559:1989, two floating point numbers with values of NaN are never equal.  However, according to the specification for the System.Object::Equals method, it's desirable to override this method to provide value equality semantics.  Since System.ValueType provides this functionality through the use of Reflection, the description for Object.Equals specifically says that value types should consider overriding the default ValueType implementation to gain a performance increase.  In fact from looking at the source of System.ValueType::Equals (line 36 of clr\src\BCL\System\ValueType.cs in the SSCLI), there's even a comment from the CLR Perf team to the effect of System.ValueType::Equals not being fast.

So now we have two conflicting ideas of what Equals should mean.  Object::Equals says that the BCL value types should override to provide value equality, and IEC 60559 says that NaN does not equal NaN.  Partition I of the ECMA spec provides resolution for this conflict by making a note about this specific case in section 8.2.5.2.

“Note: Although two floating point NaNs are defined by IEC 60559:1989 to always compare as unequal, the contract for System.Object.Equals, requires that overrides must satisfy the requirements for an equivalence operator.  Therefore, System.Double.Equals and System.Single.Equals return True when comparing two NaNs, while the equality operator returns False in that case, as required by the standard.”

So that's why == and Equals behave differently in the NaN case.  There are a lot of issues to remember when dealing with floating point numbers, and this should be another one.  Always use Equals() if you want NaN to be equal with other NaNs, and always use the == operator if you want NaNs to behave according to the IEEE floating point specification.