If you are getting error from the client saying something like this:

Cannot find a token authenticator for the 'System.IdentityModel.Tokens.X509SecurityToken' token type. Tokens of that type cannot be accepted according to current security settings.

and if you are happening to be using this security binding:

SecurityBindingElement element = SecurityBindingElement.CreateMutualCertificateBindingElement(MessageSecurityVersion.WSSecurity10WSTrustFebruary2005WSSecureConversationFebruary2005WSSecurityPolicy11BasicSecurityProfile10);

Then a very likely reason is that your server has included the x509 certificate in the response. The following flag is going to allow this scenario:

((AsymmetricSecurityBindingElement)element).AllowSerializedSigningTokenOnReply = true; 

If the client already has the server's certificate, it typically won't expect it in the response. That is why by default, WCF client is not expecting the server cert in the response.