If you are using Https, this might fix it.

            ServicePointManager.ServerCertificateValidationCallback = RemoteCertificateValidationCallback;

   public static bool RemoteCertificateValidationCallback(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)

        {

            // do some validation here...

        }

Please refer to http://blogs.msdn.com/suwatch/archive/2007/04/06/x509-and-wcxf-security.aspx for a complete picture.