September, 2009

  • Akash Blogging......

    System.Net.Mail unable to Authenticate against some third party SMTP Servers

    • 2 Comments

    I recently ran into an issue where one of my customer was unable to send mail using System.Net.Mail(.Net 2.0), we always got the "Authentication failed" error.

    We tried sending mail using Microsoft CDO for Windows 2000 Library(Cdosys) and System.Web.Mail and the mails went just fine. if Cdosys works System.Web.Mail(SWM) would normally work because SWM is essentially a wrapper over Cdosys. How can you send mails using Cdosys? Click here

    Why is one API able to send mail and the other fails? We enabled Network Tracing in our .net code and below is what the logs show:

    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : 220 mail.XYZYZYZ
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : XYZXYZZ.com ESMT
    System.Net.Sockets Verbose: 0 : [7240] 00000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 : P Gday mate..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Receive() -> 45#45
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Send()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Send
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 45 48 4C 4F 20 41 4B 41-53 48 42 0D 0A : EHLO AKASHB..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Send() -> 13#13
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Receive()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Receive
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : 250-mail.XYZYZYZ
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : XYZXYZZ.com..250
    System.Net.Sockets Verbose: 0 : [7240] 00000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : -PIPELINING..250
    System.Net.Sockets Verbose: 0 : [7240] 00000030 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : -SIZE 20971520..
    System.Net.Sockets Verbose: 0 : [7240] 00000040 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : 250-VRFY..250-ET
    System.Net.Sockets Verbose: 0 : [7240] 00000050 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : RN..250-AUTH PLA
    System.Net.Sockets Verbose: 0 : [7240] 00000060 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : IN LOGIN..250-EN
    System.Net.Sockets Verbose: 0 : [7240] 00000070 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : HANCEDSTATUSCODE
    System.Net.Sockets Verbose: 0 : [7240] 00000080 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : S..250-8BITMIME.
    System.Net.Sockets Verbose: 0 : [7240] 00000090 : 00 00 00 00 00 00 00 00-00 00 : .250 DSN..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Receive() -> 154#154
    System.Net Verbose: 0 : [7240] SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net Verbose: 0 : [7240] Exiting SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Send()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Send
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : AUTH login Y2222
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : Wwwwwwwwwwwwwwww
    System.Net.Sockets Verbose: 0 : [7240] 00000020 : 00 00 00 00 00 00 00 00 00 00 00 00 00 : mmmmmmmmmmm..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Send() -> 45#45
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Receive()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Receive
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : 334 VXNlcm5hbWU6
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 0D 0A : ..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Receive() -> 18#18
    System.Net Verbose: 0 : [7240] SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net Verbose: 0 : [7240] Exiting SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Send()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Send
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 : xxxxxxxxxxxxxxxx=
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 0D 0A : ..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Send() -> 18#18
    System.Net.Sockets Verbose: 0 : [7240] Socket#32176063::Receive()
    System.Net.Sockets Verbose: 0 : [7240] Data from Socket#32176063::Receive
    System.Net.Sockets Verbose: 0 : [7240] 00000000 : 33 33 34 20 55 47 46 7A-63 33 64 76 63 6D 51 36 : 334 UGFzc3dvcmQ6
    System.Net.Sockets Verbose: 0 : [7240] 00000010 : 0D 0A : ..
    System.Net.Sockets Verbose: 0 : [7240] Exiting Socket#32176063::Receive() -> 18#18
    System.Net Verbose: 0 : [7240] SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net Verbose: 0 : [7240] Exiting SmtpLoginAuthenticationModule#61150033::Authenticate()
    System.Net Error: 0 : [7240] Exception in the SmtpClient#3888474::Send - Authentication failed.

    Note:Data in the logs has been altered to hide confidential information

    To request LOGIN authentication, the client issues the AUTH command with the parameter LOGIN and the user name to be used for authentication, base64-encoded as specified in [RFC4648]. For example, if the client's user name was "Charlie", then the client would initiate AUTH LOGIN as follows (AUTH_LOGIN_COMMAND_USER):

    AUTH LOGIN Q2hhcmxpZQ==<CR><LF>

    If AUTH LOGIN is not supported, then the server should responds with a 504 error Message as specified in [RFC4954] section 4. If AUTH LOGIN is supported on the server, then the server responds with the AUTH_LOGIN_Password_Challenge:

    334 UGFzc3dvcmQ6<CR><LF>

    The client then responds with the password to be used for authentication, base64-encoded as specified in [RFC4648]. For example, if the client's password was "password", then client would respond with the following Login_Password_Response:

    cGFzc3dvcmQ=<CR><LF>

    If the authentication is successful, then the server issues a LOGIN_Succeeded_Response or a LOGIN_Failed_Response, corresponding to a 235 reply for success or a 535 reply for a failure [RFC4954].

    In this case, even though the user name is passed with the AUTH LOGIN the server responds with the AUTH_LOGIN_Username_Challenge again:

    334 VXNlcm5hbWU6<CR><LF>

    Now since the client has already sent the user name, it instead of sending the user name, sends the password. The server then responds with the AUTH_LOGIN_Password_Challenge:

    334 UGFzc3dvcmQ6<CR><LF>

    The client then sends nothing and the Authentication fails! We can clearly see that the SMTP server is not respecting the user name sent with the AUTH LOGIN command. It is optional for the client to send the user name([initial-response]) with the AUTH LOGIN command, if sent the SMTP server should respect it. More details about the SMTP Service Extension for Authentication can be found in the [RFC4954].

    Why did Cdosys and SWM work? Both the API’s do not send the user name along with the AUTH LOGIN and therefore everything works fine.

    Is there a way we can change this behaviour of System.Net.Mail i.e. not send the user name along with the AUTH LOGIN? No.

    What are the alternatives?

    1)Use CDOSYS (Does not send the User Name with the AUTH Login)
    2)Use System.Web.Mail (Does not send the User Name with the AUTH Login)
    3)Contact the SMTP Server owner and have them fix the server.


    Enjoy!

  • Akash Blogging......

    Increasing the number of mails that can be processed(Asynchronously) by a SMTP Agent simultaneously

    • 0 Comments

    In continuation to my previous post, the customer also had a SMTP Agent that was installed on a Edge Server and did context sensitive email analysis for incoming SMTP messages. In this case only 20 email could be processed by the SMTP Agent.

    How were the SMTP messages being sent? The customer had a multithreaded application to send out mail. Fortunately, I was able to reproduce this too.In my case when I sent more than 20 mails using a multithreaded application, I ran into the following exception:

    System.Runtime.InteropServices.COMException (0x80040211): The message could not be sent to the SMTP server. The transport error code was 0x800ccc67. The server
    response was 421 4.3.2 The maximum number of concurrent connections has exceeded a limit, closing transmission channel   at CDO.MessageClass.Send()
       at SendSMTPMails.MassMailer.SendMail() in C:\SendSMTPMails\SendSMTPMails\Program.cs:line 63 Exception caught.

    This is a restrictions on the Receive Connector in Exchange 2007. You will need to configure the receive connector to allow MaxInboundConnectionPerSource to a desired number. The default value is 20.

    You can set the value by running the Set-ReceiveConnector cmdlet.

    Cmdlet:Set-ReceiveConnector "Receive connector Name" -MaxInboundConnectionPerSource 100

    Enjoy!

  • Akash Blogging......

    Increasing the number of mails that can be processed(Asynchronously) by a Routing Agent simultaneously

    • 0 Comments

    I recently ran in an issue where one of my customer was developing a context sensitive email analysis software for MS Exchange 2007. The results of the analysis was used to decide how that email would be dealt with.

    Due to the complexity of the analysis, the estimated processing time per email was about 15 to 30 seconds. To realize an acceptable throughput of emails per minute, they needed to ensure that the email processing was strongly parallelized. Therefore they encapsulated the analysis itself into a separate web service which was being called from a Routing Agent.

    Using the asynchronous programming pattern(http://msdn.microsoft.com/en-us/library/cc720860.aspx), recommended by Microsoft for long I/O waiting times,they encountered a limitation in parallel processing. They noticed that at one time they could only process 6 email at a time. The processing of the seventh email would begin only when one of the mails submitted earlier was done.

    In case of the Hub role this was double throughput compared to the synchronous programming pattern but still nowhere near the required rate of parallel processing. At the existing rate they would only be able to process a maximum of 24 emails per minute. The requirement was to process a minimum of 100 emails per minute.

    Fortunately I was able to reproduce the issue in my lab. Below is the code I used to reproduce the behaviour:

    using System;
    using System.Collections;
    using System.Collections.Generic;
    using System.Diagnostics;
    using System.IO;
    using System.Threading;
    using System.Xml;
    using System.Reflection;
    using Microsoft.Exchange.Data.Mime;
    using Microsoft.Exchange.Data.Transport;
    using Microsoft.Exchange.Data.Transport.Email;
    using Microsoft.Exchange.Data.Transport.Routing;

    namespace AsyncRoutingAgent
    {
    /// <summary>
    /// Agent factory
    /// </summary>
    public class MyRoutingAgentFactory :RoutingAgentFactory
    {
    /// <summary>
    /// Create a new RoutingAgent.
    /// </summary>
    /// <param name="server">Exchange Hub server</param>
    /// <returns>a new agent</returns>
    public override RoutingAgent CreateAgent(SmtpServer server)
    {
    return new MyRoutingAgent();
    }
    }

    public class MyRoutingAgent : RoutingAgent
    {
    private object fileLock = new object();

    public MyRoutingAgent()
    {
    base.OnSubmittedMessage += new SubmittedMessageEventHandler(MyRoutingAgent_OnSubmittedMessage);
    }

    void MyRoutingAgent_OnSubmittedMessage(SubmittedMessageEventSource source, QueuedMessageEventArgs e)
    {
    WriteLog("Entering OnSubmittedMessage");

    // Create an instance of the Web Service via the proxy class

    WriteLog("Creating Instance of Web Service");
    AsyncRoutingAgent.localhost.StockService objWebService = new AsyncRoutingAgent.localhost.StockService();

    // Get the async context so that the server knows that this event should be handled asynchronously.
    // Also save the event source and event args so that they can be accessed from the callback.
    WriteLog("Calling GetAgentAsyncContext");

    AsyncState<SubmittedMessageEventSource, QueuedMessageEventArgs> asyncState =
    new AsyncState<SubmittedMessageEventSource, QueuedMessageEventArgs>(
    source,
    e,
    this.GetAgentAsyncContext());

    WriteLog("Hooking the Event Handler");

    objWebService.GetStockQuoteCompleted += new AsyncRoutingAgent.localhost.GetStockQuoteCompletedEventHandler(objWebService_GetStockQuoteCompleted);

    WriteLog("Calling the Service");

    objWebService.GetStockQuoteAsync("MSFT", 15, asyncState);
    WriteLog("Just before Returning");
    return;
    }

    void objWebService_GetStockQuoteCompleted(object sender, AsyncRoutingAgent.localhost.GetStockQuoteCompletedEventArgs e)
    {
    AsyncState<SubmittedMessageEventSource, QueuedMessageEventArgs> asyncState =
    (AsyncState<SubmittedMessageEventSource, QueuedMessageEventArgs>)e.UserState;

    WriteLog(e.Result);
    // Handle the results of the asynchronous I/O here.
    // The event source and event args are available from the asyncState.
    // Tell the server that this event is complete.
    asyncState.Complete();

    WriteLog("Call Completed");
    }

    public void WriteLog(String Comment)
    {
    lock (fileLock)
    {
    try
    {
    string logDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location) + @"\Log";
    string logFile = logDir + @"\log.txt";

    if (!Directory.Exists(logDir))
    {
    Directory.CreateDirectory(logDir);
    }

    if (!File.Exists(logFile))
    {
    File.CreateText(logFile).Close();
    }

    using (StreamWriter logWriter = File.AppendText(logFile))
    {
    logWriter.Write(DateTime.Now.ToString()+ ":" + Comment+ Environment.NewLine);
    logWriter.Flush();
    }
    }
    catch (System.IO.IOException ex)
    {
    Debug.WriteLine(ex.ToString());
    }
    }

    }

    /// <summary>
    /// This class wraps all the state that has to be saved
    /// for an asynchronous event.
    /// </summary>
    private class AsyncState<SourceType, ArgsType>
    {
    private SourceType source;
    private ArgsType args;
    private AgentAsyncContext asyncContext;

    public AsyncState(
    SourceType source,
    ArgsType args,
    AgentAsyncContext asyncContext)
    {
    this.source = source;
    this.args = args;
    this.asyncContext = asyncContext;
    }

    public SourceType Source
    {
    get { return this.source; }
    }

    public ArgsType Args
    {
    get { return this.args; }
    }

    public void Complete()
    {
    this.asyncContext.Complete();

    this.source = default(SourceType);
    this.args = default(ArgsType);
    this.asyncContext = null;
    }
    }
    }
    }

    The above code generates a log which clearly shows the problem.

    09/04/2009 1:41:25 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:26 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:27 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:28 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:28 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:30 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:40 AM:Call Completed
    09/04/2009 1:41:40 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:42 AM:Call Completed
    09/04/2009 1:41:42 AM:Entering OnSubmittedMessage
    09/04/2009 1:41:42 AM:Call Completed 

    In the logs above you can see that "Entering OnSubmittedMessage" is fired 6 times between 1:41:25 Am - 1:41:30 and the next call to "Entering OnSubmittedMessage" is made at 1:41:40 AM only after one of the call from the Web Service returned.The web service has a delay of 15 seconds to simulated a time consuming service.

    Now to the solution. The Exchange Transport creates multiple instances of the agent if it has multiple messages to process, and the number of instances is limited by the following settings:

    MaxExecutingJobs: Range 1 to 50
    MaxJobThreads: Range 1-25 (Default value 3)

    These setting have to be defined in the edgetransport.exe.config under the appSettings section. In case when the agent is run asynchronously we will only need to alter the value of MaxExecutingJobs and increase its value.

    Add the following lines to the edgetransport.exe.config under the appSettings section. You will need to restart the Microsoft Transport and the EdgeTransport.exe
        <add key="MaxExecutingJobs" value="50"/>
        <add key="MaxJobThreads" value="3"/>

    In case you need more that 50 jobs to be executed at the same time, you will need to run the code on a machine that has more than 1 processors, since the actual MaxExecutingJobs value is calculated by multiplying it with the number of processors your machine has.

    Enjoy!

    Note:In a Routing Agent we should not be doing stuff that takes a long time to execute. We did advise the customer to take a alternative approach to the entire solution.

Page 1 of 1 (3 items)