SMTP protocol correctness question.

I have been writing SMTP-aware and MIME-aware code recently and have come across something I'd like you're collective opinions about.

What started it all was noticing that Outlook Express mis-decodes faulty quoted-printable. Basically, a quoted-printable "soft line break" is an = followed by a CRLF (in that order). If the LF is missing, OE displays the =, but discards the CR and the next character.

Now, I've recently co-written an IMAP server that talks to a database as it's message store. My colleague wrote the SMTP import script (in Perl). Everything we wrote assumes sane line boundaries. In the meantime, I discovered a warning in the Perl documentation against using a mere \\n for network work, especially under Unix, because line boundaries in text-oriented network protocols are supposed to be \\r\\n or CRLF. In fact, RFC 822 (SMTP) is reasonably clear about this. And so is 2045 and 2046 (MIME).

But what I've discovered is that some programs seem to deliver SMTP with just LF on the line-boundaries. I'm not totally sure, mind, as the only evidence I have to go on is the output from the import script which is run from PostFix's mailbox_command option. Actually, it would be highly ironic if the culprit proved to be OE (sending, that is) because it would mean OE is not compatible with itself! OTOH, most of the offending emails have also passed through PostFix on the sending end and through another install of PostFix here on the receiving end. I rather doubt PostFix would be the culprit here, but it's looking likely. I haven't found any configuration options that might be relevant.

I guess what I'm asking is how prevalent is the missing CR in network protocol implementations? Or is PostFix just doing something a little wierd? (We've got a fix in our import script; it just forces all line boundaries to CRLF.)

Wade.

Post #28,881 by static 2/19/02 12:14:01 AM Reply	SMTP protocol correctness question. I have been writing SMTP-aware and MIME-aware code recently and have come across something I'd like you're collective opinions about. What started it all was noticing that Outlook Express mis-decodes faulty quoted-printable. Basically, a quoted-printable "soft line break" is an = followed by a CRLF (in that order). If the LF is missing, OE displays the =, but discards the CR and the next character. Now, I've recently co-written an IMAP server that talks to a database as it's message store. My colleague wrote the SMTP import script (in Perl). Everything we wrote assumes sane line boundaries. In the meantime, I discovered a warning in the Perl documentation against using a mere \\n for network work, especially under Unix, because line boundaries in text-oriented network protocols are supposed to be \\r\\n or CRLF. In fact, RFC 822 (SMTP) is reasonably clear about this. And so is 2045 and 2046 (MIME). But what I've discovered is that some programs seem to deliver SMTP with just LF on the line-boundaries. I'm not totally sure, mind, as the only evidence I have to go on is the output from the import script which is run from PostFix's mailbox_command option. Actually, it would be highly ironic if the culprit proved to be OE (sending, that is) because it would mean OE is not compatible with itself! OTOH, most of the offending emails have also passed through PostFix on the sending end and through another install of PostFix here on the receiving end. I rather doubt PostFix would be the culprit here, but it's looking likely. I haven't found any configuration options that might be relevant. I guess what I'm asking is how prevalent is the missing CR in network protocol implementations? Or is PostFix just doing something a little wierd? (We've got a fix in our import script; it just forces all line boundaries to CRLF.) Wade. "All around me are nothing but fakes Come with me on the biggest fake of all!"
Post #28,933 by tseliot 2/19/02 12:16:27 PM Reply	I * believe* it's just OE--recent exploit using this IIRC Gimme a while to dig it up. --------------------------------- A stupid despot may constrain his slaves with iron chains; but a true politician binds them even more strongly by the chain of their own ideas;...despair and time eat away the bonds of iron and steel, but they are powerless against the habitual union of ideas, they can only tighten it still more; and on the soft fibres of the brain is founded the unshakable base of the soundest of Empires." Jacques Servan, 1767
Post #28,935 by tseliot 2/19/02 12:30:42 PM Reply	Got it. [link\|http://www.securityfocus.com/archive/1/255910\|[link\|http://www.securityfocus.com/archive/1/255910\|http://www.security...ive/1/255910]] This is similar, anyway. --------------------------------- A stupid despot may constrain his slaves with iron chains; but a true politician binds them even more strongly by the chain of their own ideas;...despair and time eat away the bonds of iron and steel, but they are powerless against the habitual union of ideas, they can only tighten it still more; and on the soft fibres of the brain is founded the unshakable base of the soundest of Empires." Jacques Servan, 1767
Post #29,003 by static 2/19/02 9:07:55 PM Reply	Umm, well, only sorta. That wasn't the problem. AFAIK, OE is rightly mis-parsing incorrectly formatted quoted-printable soft line-breaks in message bodies. From a technical viewpoint, I don't like it's failure mode, but from UI POV, it's actually quite good. No, the concern I had was whether \\n in SMTP instead of \\r\\n (as per the standard) is at all widespread or is it just an aberration in the setup I'm working with? Wade. "All around me are nothing but fakes Come with me on the biggest fake of all!"

Welcome to IWETHEY!