As we saw in our previous post, 5 basic commands are needed for SMTP. When the receiving mail transfer agent (MTA) receives the message, it inserts additional headers which allow us to trace the message to its source. In the example from the previous post, here would be the headers from the sample message:
From firstname.lastname@example.orgReceived: from mailhost.tzink-is-awesome.com (mailhost.tzink-is-awesome.com [2188.8.131.52]) by mail.tzink.net (8.8.5) for email@example.com with EMSTP id 123456789-0AH for <firstname.lastname@example.org>From: email@example.comTo: firstname.lastname@example.orgDate: Fri, Jun 18, 2007 20:20:20 PSTMessage-ID: <email@example.com>Subject: How's it going?
Let's step through these one by one. The first line is the From address and it is the envelope sender. The envelope sender is generated by the receiving machine and it is generated by the MAIL FROM command from the transmitting machine. Note the lack of a colon in the From header, this distinguishes it from the other From: header later on. The convention is not universal but it is frequent. The envelope headers are generated by the receiving machine while the message headers are created by the transmitting machine.
The next line is a received header. This received header is also an envelope sender because it is generated by the receiving machine.
Received: from mailhost.tzink-is-awesome.com
This piece of mail was received from a machine that calls itself mailhost.tzink-is-awesome.com.
The IP address of the sending machine is 2184.108.40.206. Received headers will always log the sending IP address, and the name of the machine is mailhost.tzink-is-awesome.com. This name is found by doing a reverse DNS lookup of the IP address. In other words, here's what happened:
Not all IP addresses have reverse IP lookups, but when they exist it is easier to implement a weak form of sender authentication. If it didn't exist, the name part would be blank. Another possibility is that the Received: from xxxxx name could be different than the name in the reverse DNS lookup. The next header is the following:
by mail.tzink.net (8.8.5)
The machine that received the message is mail.tzink.net using a program called Sendmail, version 8.8.5.
with EMSTP id 123456789-0AH
The receiving machine assigned the ID number 123456780-0AH to the message. This is used more for mail administrators for checking logs but is also sometimes useful for spam analysts.
This message was addressed to firstname.lastname@example.org. This is the Envelope To, the one that is specified in RCPT TO by the sending machine. It is this address that the message is routed to. Note that this email does not have to be the same as the one in the To: header later on. The envelope sender is not always in a received header, sometimes it is in a header elsewhere in the message.
The next few headers are message headers.
From: email@example.comTo: firstname.lastname@example.orgDate: Fri, Jun 18, 2007 20:20:20 PSTMessage-ID: <email@example.com>Subject: How's it going?
The above headers are created by the transmitting machine. The Message-ID is different than the SMTP ID, it can have the sender's email address embedded in it. Other times there is no intelligible meaning associated with it. This ID is associated with this email message for life.
Note that there are four important routing headers, the Envelope To, the Envelope From, the message To: and the message From:. The Envelope headers are generated by the receiving machine based upon the SMTP commands used by the transmitting machine while the To: and From: headers are extra headers inserted into the body of the message (that often show up in your email client like Thunderbird, Apple Mail or Outlook). The message is routed based on the Envelope headers, not the message headers Also note the absence of a colon in the Envelope headers.
Envelope headers appear differently in different mail servers. Sometimes the envelope sender is specified in the Return-Path header.
It is important to note that my example above is simple. Often times, a message will go through more routing and will have a few more Received: from headers.
Really like this series of posts. Thanks.
> The envelope sender is generated by the receiving
> machine and it is generated by the MAIL FROM command
> from the transmitting machine.
True, but it would be valuable to repeat a comment that you made earlier. The sender's MAIL FROM can specify a forged address. Therefore even though the receiving machine generates the envelope sender line, the receiving machine can still be tricked into putting a forged address into that line. The envelope sender line is no longer useful in tracing spams.
> Received headers will always log the sending IP
> address, and the name of the machine is
Received headers won't always long the sending IP address. More and more ISPs are wising up to the importance of logging that address, but not all do. The next thing to note is that when an ISP is wise enough to log the sending IP address, you have to know how to read the received header. Ignore the asserted name of the machine because spammers will forge it. Ignore the asserted sending IP address in the portion that's also somewhat similar to the envelope sender line because spammers will forge it. You have to recognize your ISP's syntax for the portion of this line where your ISP has added its own log of the sending address, because this is the only part that's accurate. (Exception: If your ISP is a spam supporter then even that part won't be accurate.)
> Often times, a message will go through more routing
> and will have a few more Received: from headers.
Yes, and on very rare occasions those additional Received: lines might still be accurate, but usually they're no longer useful. Spammers forge those too. You still need to include them when reporting a spam to the administrators of the sender's ISP because their administrators might be able to figure out if one more Received: header is accurate, but you can't figure it out yourself. You can only depend on the last Received: header that was added at the top. (Exception: if your ISP is a spam supporter then you can't even depend on that.)
I plan on getting to those points in some of my later posts.
We saw in part 2 of this series that when a receiving email server gets the message, it inserts a Received:
The MAIL FROM and RCPT TO arguments are not "headers" -- they're part of the envelope. The envelope sender may be written into a new header by the receiving MTA, typically Return-Path: but it can often be munged.
This isn't just me being picky -- understanding the difference between the From: header and the envelope sender is critical to understanding the difference between SPF and Sender ID.
Richi, you are correct.
The MAIL FROM is the envelope sender while the RCPT TO is the envelope recipient. I refer to these as envelope headers and sometimes receiving MTA's throw them away, sometimes they put them in the Return-Path. Our own filters put the envelope sender in a field called X-SMTP-MAIL-FROM in the case that we route them to the user's spam quarantine, otherwise we strip them if they go to the user's inbox.
I refer to the From: field in the message headers as the From: header. I differentiate between the message header by always including a colon.
I apologize if I haven't been clear but I definitely aim to distinguish between the two.
I realise I'm a bit late here but I have a query regarding Received headers generated by Exchange.
My Postfix server at home generates headers as you described above, including the envelope recipient (RCPT TO) - this is most usefull when tracking down and stopping spam and other delivery/subscription problems.
But our Exchange server v6.5.7638.1 at work does not add this info to its received headers. Do you have any pointers for how to configure Exchange to do this?
Replying to my own question here: I found an Exchange sink that does the job (almost). It inserts 3 custom headers into the message indicating MAIL FROM, RCPT TO and originating IP address. I found it at http://www.vamsoft.com/tools.asp#smtpenvl
But I'm still interested in a way to get Exchange to generate 'Postfix'-style received headers.
Great! Answers a lot of my questions about email. I read part 1 but only thru Google did I see there was a part 2. Now I notice in Google that there's a "part 32" out there for this article. Where do I find the index or table of contents for all the parts beyond #2?
This is part of a larger series on email authentication:
I am trying to get the header (indicating who the email is going to if you are copying anyone and the subject area)in outlook drafts to show up when I print them to be reviewed by my Manager, please list the steps taken to accomplish this?