(forw) [sorbs.net #212641] [Webform] SORBS registration systems sends RFC-ignorant mail

[Rick Moen]

I've been aware for a long time that being an anti-spam person can lead
one eventually to be a trigger-happy nut: Regardless of whether Sullivan
is correct about proper uses of the null sender "<>", he is not correct
about (quoted) mail originating at "From: ***@sorbs.net", a
non-deliverable sender -- and _that_ is the main, characteristic sin of
CGI script developers whose scripts generate outgoing mail.

You'll note Sullivan's statement that SORBS also considers MTA Q}callbacks /
callouts an "abusing DDoS tool" regardless and how implemented (e.g., my
system that caches and reuses callout test results), and blocklists the
hosts that use it. Oh well, so be it: Can't make everyone happy.

(linuxmafia.com is not currently blocklisted by anyone. I was just
trying to sign up at the SORBS Web site so I could check SORBS
reporting.)

----- Forwarded message from "SORBS Support (Matthew Sullivan)" <***@support.sorbs.net> -----

From: "SORBS Support (Matthew Sullivan)" <***@support.sorbs.net>
Reply-To: ***@support.sorbs.net
To: ***@linuxmafia.com
Date: Sun, 26 Oct 2008 08:40:03 +1100
Subject: [sorbs.net #212641] [Webform] SORBS registration systems sends RFC-ignorant mail

Gentlemen, I just tried to register user "rickmoen", and duly
submitted the required Web form. Your system then attempted to
send mail with information required to complete registration to my
2008-10-09 12:28:56 1Ko1Br-0007vM-H6 H=anaconda.sorbs.net
[203.15.51.135]:50554
I=[198.144.195.186]:25 F=<> rejected after DATA: Sender callback
verification fa
iled for header From: sender SORBS Registration Server
e is no valid sender in any header line
Envelope-from: <>
P Received: from anaconda.sorbs.net ([203.15.51.135]:50554)
by linuxmafia.com with esmtp (Exim 4.61 #1 (EximConfig
2.0))
id 1Ko1Br-0007vM-H6
P Received: from registration.stealth.sorbs.net (spamhaus.kd1.tisf.net
[64.124.5
2.228])
by anaconda.sorbs.net (Postfix) with ESMTP id C10DE2E072
(EST)
Subject: Registration Confirmation
X-Originating-IP: 64.186.171.234
X-Sent-Via: 64.186.171.234
Date: Fri, 10 Oct 2008 05:28:46 +1000 (EST)
X-Virus-Scanned: ClamAV 0.92/8399/Thu Oct 9 22:27:14 2008 on
anaconda.sorbs.n
et
X-Virus-Status: Clean
Fellahs, if you're going to write a script to send mail, the least you
should do is make it originate that mail from a valid address, and
not from the null sender. The null sender is fine for DSNs, but
not for anything else. Basically, your programmer got lazy. You
basically any theoretically deliverable, valid address.

Try reading the RFCs you might understand them.

The NULL sender (<>) is used for where there is no expected or desired
response. It is not just for DSNs it's also used for mailing list sign
ups where the mail administrator wishes to avoid mailing loops.

Worse, having now whitelisted your domain and thus exempted it from
callbacks, there appears to be no way to convince your registration
system to re-send the confirmation mail: Attempting to get it to
do that fails silently.

Callbacks will result in blocking from the SORBS servers, they are (and
have been proven to be) an abusing DDoS tool.

I'd still like to register user "rickmoen", by the way. Any chance of
doing that, or do I have to invent a second username just to deal
with your broken mail script?

The system will automatically send an email reminder within 7 days of
the original message.

Regards,

M

(The programmer of the script)


----- End forwarded message -----

[Ruben Safir]

Post by Rick Moen
I've been aware for a long time that being an anti-spam person can lead
one eventually to be a trigger-happy nut: Regardless of whether Sullivan
is correct about proper uses of the null sender "<>", he is not correct
non-deliverable sender -- and _that_ is the main, characteristic sin of
CGI script developers whose scripts generate outgoing mail.
You'll note Sullivan's statement that SORBS also considers MTA Q}callbacks /
callouts an "abusing DDoS tool" regardless and how implemented (e.g., my
system that caches and reuses callout test results), and blocklists the
hosts that use it. Oh well, so be it: Can't make everyone happy.
(linuxmafia.com is not currently blocklisted by anyone. I was just
trying to sign up at the SORBS Web site so I could check SORBS
reporting.)

Your mail server automatically is checking to see if the sender is a
legit user?

Ruben

[Rick Moen]

Post by Ruben Safir
Your mail server automatically is checking to see if the sender is a
legit user?

That's right.

I whitelist particular domains that I know to be RFC-challenged.

[Ruben Safir]

Post by Rick Moen

Post by Ruben Safir
Your mail server automatically is checking to see if the sender is a
legit user?

That's right.
I whitelist particular domains that I know to be RFC-challenged.

I used to have a sendmail configuration like that and nearly all ny
Linux friends couldn't send
my mail because they didn't control their DNS (or IP addresses)

Ruben

Post by Rick Moen
_______________________________________________
conspire mailing list
http://linuxmafia.com/mailman/listinfo/conspire

[Rick Moen]

Post by Ruben Safir
I used to have a sendmail configuration like that and nearly all ny
Linux friends couldn't send my mail because they didn't control their
DNS (or IP addresses)

None of the RFCs in question (defining SMTP) require controlling one's
own IP or DNS for compliance. It's a matter of MTA configuration -- and
to my knowledge all common MTAs have defaulted to support for those
standards for quite a long time.

Of course, there are sites (including virtual-hosting sites and such)
that either prevent RFC-compliance or make it non-default. For example,
BALUG's mail-hosting at Dreamhost refused incoming mail to
***@balug.org for several years, and the gentleman (Michael
Hubbard) who arranged that hosting as a Dreamhost customer asserted that
it was impossible to make Dreamhost-hosted domains accept postmaster
mail. It turned out, he was incorrect: He simply hadn't bothered to
configure his Dreamhost account settings to enable that mailbox.

[Ruben Safir]

Post by Rick Moen

Post by Ruben Safir
I used to have a sendmail configuration like that and nearly all ny
Linux friends couldn't send my mail because they didn't control their
DNS (or IP addresses)

But you can't reverse DNS check if the sending machine is behind a
firewall/port forwarding or their dns name myhost.yahoo.com isn't an
official host of yahoo.com

that has been the problem historically.

Ruben

Post by Rick Moen
None of the RFCs in question (defining SMTP) require controlling one's
own IP or DNS for compliance. It's a matter of MTA configuration -- and
to my knowledge all common MTAs have defaulted to support for those
standards for quite a long time.
Of course, there are sites (including virtual-hosting sites and such)
that either prevent RFC-compliance or make it non-default. For example,
BALUG's mail-hosting at Dreamhost refused incoming mail to
Hubbard) who arranged that hosting as a Dreamhost customer asserted that
it was impossible to make Dreamhost-hosted domains accept postmaster
mail. It turned out, he was incorrect: He simply hadn't bothered to
configure his Dreamhost account settings to enable that mailbox.
_______________________________________________
conspire mailing list
http://linuxmafia.com/mailman/listinfo/conspire

--
http://www.mrbrklyn.com - Interesting Stuff
http://www.nylxs.com - Leadership Development in Free Software

So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998

http://fairuse.nylxs.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002

"Yeah - I write Free Software...so SUE ME"

"The tremendous problem we face is that we are becoming sharecroppers to our own cultural heritage -- we need the ability to participate in our own society."

"> I'm an engineer. I choose the best tool for the job, politics be damned.<
You must be a stupid engineer then, because politcs and technology have been attached at the hip since the 1st dynasty in Ancient Egypt. I guess you missed that one."

© Copyright for the Digital Millennium

[Rick Moen]

Post by Ruben Safir
But you can't reverse DNS check if the sending machine is behind a
firewall/port forwarding or their dns name myhost.yahoo.com isn't an
official host of yahoo.com

There _is no RFC requirement_ that MTA hosts have valid reverse DNS.

Some people elect to refuse mail from MTAs whose host lack reverse DNS,
(apparently as some sort of misguided antispam heuristic), but I don't.

[Ruben Safir]

Post by Rick Moen

Post by Ruben Safir
But you can't reverse DNS check if the sending machine is behind a
firewall/port forwarding or their dns name myhost.yahoo.com isn't an
official host of yahoo.com

How are you going to check the user if you can't reach the machine?

Ruben

Post by Rick Moen
There _is no RFC requirement_ that MTA hosts have valid reverse DNS.
Some people elect to refuse mail from MTAs whose host lack reverse DNS,
(apparently as some sort of misguided antispam heuristic), but I don't.
_______________________________________________
conspire mailing list
http://linuxmafia.com/mailman/listinfo/conspire

[Rick Moen]

Post by Ruben Safir
How are you going to check the user if you can't reach the machine?

I think you're making one or more assumption in that question that
isn't correct, which in turn makes it somewhere between difficult and
misleading to attempt to answer it as phrased.

The easiest thing for you to do would probably be to review EximConfig's
documentation, for background on this matter.
http://www.jcdigita.com/eximconfig/

At least part of your problem probably lies in your phrase "check the
user". I suspect you're making some inaccurate assumptions about what
callbacks aim to do that are implicit in your phrase.

[Rick Moen]

Hi, Ruben. Apologies if I was brusque. Your question caught me at a
bad moment, and I didn't have time to elaborate.

Post by Ruben Safir
How are you going to check the user if you can't reach the machine?

So, to elaborate: ;->

My MTA does not aspire, and doesn't need to aspire, to ensure that the
claimed sender of any arriving mail has a UID on his/her local machine.
That's just not relevant to the situation at all. For purposes of
RFC-compliance -- and I'm actually, truth to tell, at this moment at a
small loss to tell you where exactly the RFCs state this requirement --
it suffices for my MTA to investigate whether the claimed sender e-mail
address is deliverable for return mail at the domain's advertised mail
exchangers (MXes). I.e., if your address originates mail, the RFCs
require that the sending mailbox be a valid destination for return mail.

That would be one of several things that can be checked during SMTP time
(during a pending attempt by some remote SMTP host to deliver mail, but
before one's own MTA accepts or declines the mail) as what is termed a
"callout" aka "callback" check. Essentially, _while_ the inbound SMTP
attempt is still ongoing, a separate fork of your MTA initiates a return
SMTP connection to one of the claimed sender domain's publicly
advertised MXes, and announces that it wishes to drop off a mail for the
claimed sender. If the remote MTA says "250" to indicate that the
recipient will accept the return mail, then my own MTA cancels the
partial delivery. My own MTA _also_ caches the successful callout,
so as _not_ to be guilty of the network abuse that the SORBS guy
unreasonably claims is inherent in any use of callouts.

The other common callout items are the existence of a deliverable
"postmaster" and "abuse" mailbox at the claimed sending domain. For
those, I _can_ cite if you need them exactly where the RFCs mandate them
for any domain that sends SMTP.

The point is not to be an RFC zealot, but rather to suss out mail that's
highly likely to be spam/malware -- because spammers/malware authors
typically cannot be bothered to make their ratware deliver mail in an
RFC-compliant manner. Thus, RFC-non-compliance tends to correlate
strongly with spamicity, and makes an excellent antispam heuristic.

And, as I was saying, because reverse DNS is _not_ RFC-mandated, that is
a really _poor_ idea as an antispam heuristic. Yes, I know there are
people who do that, but they're deluded.

[Rick Moen]

I don't really need the RFC quote, but I just want to understand. This
check for the deliverable mailing address can take place during the SMTP
handshake? Or does a separate inquiry to the sending server need to
take place?

It's a separate SMTP connection (back) to an MX of the claimed sending
domain.

So, if the incoming connection is claimed to be from
***@mrbrooklyn.com, my MTA will first try to look up the relevant MX
record (if any):

:r! dig -t mx mrbrooklyn.com +short
[returns null]

SMTP's fallback, absent an explicit MX record, is to use the "A" record:

:r! dig mrbrooklyn.com +short
216.21.239.197

So, my SMTP host will then open a socket on that IP's port 25, do HELO,
and initiate a (partial) e-mail to "***@mrbrooklyn.com". If that is
indicated as an acceptable addresssee ("250 Recipient OK" or something
like that), then my MTA cancels the test message, caches the successful
result, and permits your MTA's pending delivery on the connection the
other way.

Note that my MTA's connection is not necessarily to "the sending
server", just to one of the MXes (mail exchangers) for the claimed
sending domain.

Doesn't that require that the two machines, client and server, be
directly connected? Because most mail I receive seems to be going
through levels of relays.

Again, the callout (callback) check's network socket is not necessarily
to the "server" as in the machine that is currently seeking to drop off
mail: It is to one of the authorised MXes of the claimed sending
domain. That MX should (must) know what's a valid address within its
mail domain and what is not.

[Ruben Safir]

Post by Rick Moen

I don't really need the RFC quote, but I just want to understand. This
check for the deliverable mailing address can take place during the SMTP
handshake? Or does a separate inquiry to the sending server need to
take place?

It's a separate SMTP connection (back) to an MX of the claimed sending
domain.
So, if the incoming connection is claimed to be from
:r! dig -t mx mrbrooklyn.com +short
[returns null]
:r! dig mrbrooklyn.com +short
216.21.239.197
So, my SMTP host will then open a socket on that IP's port 25, do HELO,
indicated as an acceptable addresssee ("250 Recipient OK" or something
like that), then my MTA cancels the test message, caches the successful
result, and permits your MTA's pending delivery on the connection the
other way.
Note that my MTA's connection is not necessarily to "the sending
server", just to one of the MXes (mail exchangers) for the claimed
sending domain.

Doesn't that require that the two machines, client and server, be
directly connected? Because most mail I receive seems to be going
through levels of relays.

Again, the callout (callback) check's network socket is not necessarily
to the "server" as in the machine that is currently seeking to drop off
mail: It is to one of the authorised MXes of the claimed sending
domain. That MX should (must) know what's a valid address within its
mail domain and what is not.

Thanks. I completely understand now. A user though still must have a
record on an authorized MX server. So if one just sticks a GNU Server
on a verizon cunsmer grade FIOS network and sends mail out without
having a record or account on the verizon network, they're
shot...nothing is getting through. If they fix their 'From' line to an
official verizon email account, or have an mx record set up somewhere in
the world for their supposed sending domain, they are good to go.

Ruben

Post by Rick Moen
_______________________________________________
conspire mailing list
http://linuxmafia.com/mailman/listinfo/conspire

[Rick Moen]

Post by Ruben Safir
Thanks. I completely understand now. A user though still must have a
record on an authorized MX server.

Because of this talk of "having a record", I think you're still getting
hung up on some erroneous concept of "checking the user". All that's
being checked is that the domain's MX accepts mail for the claimed
sender address. That's it. If an incoming SMTP connection is
attempting to deliver mail claimed to be from sender "***@example.com",
then my MTA should be able to connect to an MX for domain example.com
and verify that it's willing to accept responses to mailbox
"***@example.com". The MX doesn't need to know anything about user Foo,
and Foo need not have a "record or account" there. All that matters is
that the MX be willing to say "Sure, I'll accept mail addressed to

Post by Ruben Safir
So if one just sticks a GNU Server on a verizon cunsmer grade FIOS
network and sends mail out without having a record or account on the
verizon network, they're shot...nothing is getting through.

You're not getting this. Let's try this from first principles.

The domain-owning user either secures a fixed, static IP from Verizon
Broadband on which his/her SMTP host will live -- case A -- or
negotiates an agreement with Verizon to virtual-host his/her
SMTP-handling domain on Verizon's fixed-IP servers -- case B. (It is
not possible within reason to operate an SMTP host on dynamic IP. There
are people who try. They are deluded.)

In case A, the user points the "A" record for his/her mail-sending
domain (example.com)[1] at the static IP in question, and optionally
creates a matching "MX" record (recommended, by the way). The mail
from "***@example.com" originates from that IP. The host also accepts
return mail, and therefore is the place where callouts verifying the
deliverability of "***@example.com" will check.

In case B, the user points the "A" and optional "MX" records at one of
Verizon's large mail hosts, after verifying with them that they're
willing to forward its outbound mail and accept mail addressed to it.
The user generates on his/her dynamic-IP desktop box the outbound mail
from "***@example.com", and relays it through one of Verizon's big
mail-handling hosts, which is willing to handle it because it originated
on a Verizon IP and is addressed from one of the domains Verizon have in
their MTA allowed-relay table. Because that host is where the MX/A
records point, it's where callouts verifying the deliverability of
"***@example.com" will check -- and it'll say "Sure, I'll accept mail
for '***@example.com'" because that's in its table of mailboxes the
Verizon MTA handles.

If perchance you are not talking about a _domain-owning_ user, then
that's case C: The user generates on his/her dynamic-IP desktop box the
outbound mail from "***@verizon.com", and relays it through one of
Verizon's big mail-handling hosts, which is willing to handle it because
it originated on a Verizon IP and is addressed from domain verizon.com.
Callbacks to verifying the deliverability of ***@verizon.com will come
to one of Verizon's big mail-handling hosts to which one of the MX
entries points for domain verizon.com. It will say "Sure, I'll accept
mail for '***@verizon.com'", becuase that's in its table of mailboxes
the Verizon MTA handles.

Post by Ruben Safir
If they fix their 'From' line to an official verizon email account,
or have an mx record set up somewhere in the world for their supposed
sending domain, they are good to go.

If the domain-owning user does _not_ have an MX or A record set up for
the claimed sending domain, then he/she is by definition sending bogus,
forged SMTP mail, and I definitely do not want my MTA accepting it. You
wouldn't want yours to accept it, either.