Discussion:
Sender Address Verification
(too old to reply)
Wolfgang Hottgenroth
2004-04-15 18:46:46 UTC
Permalink
Hi,


I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.

It is not a kind of c/r verifier but just verifies the sender address
by talking to the bestmx of the sender to find whether the address
itself is valid, just like exim does and postfix supports in the rc of
2.1.

I expect this thing to keep especially queues of backup MX servers a
bit cleaner. Also SPAM with invalid sender addresses would be
rejected, configurable with 4xx or 5xx codes.
Blacklists (only sender addresses with domains in the blacklist are
verified) and whitelists (all sender addresses but those mentioned in
the whitelist are verified) are supported.

It comes as a daemon writing in Python and a m4 FEATURE file to be
included in your sendmail.mc.

I invite you to have a look and to tell me your opinion:

http://www.hottis.de/web/verifier/index.html




Cheers,
Wolfgang
Leif Neland
2004-04-15 21:41:43 UTC
Permalink
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender address
by talking to the bestmx of the sender to find whether the address
itself is valid, just like exim does and postfix supports in the rc of
2.1.
Just like milter-sender you mean?

Leif
Wolfgang Hottgenroth
2004-04-16 08:40:21 UTC
Permalink
Morning!

At Thu, 15 Apr 2004 23:41:43 +0200,
Post by Leif Neland
Post by Wolfgang Hottgenroth
[...]
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
[...]
Just like milter-sender you mean?
Yes, exactly.

But not called through the milter interface but through
the new socket map. And more lightweight as it seems to me.

It looks to me as milter-sender has implemented a great deal of
feature in C code out side of sendmail.

My approach was to have as few as possible external code and implement
features in cf code (delivered in the m4 FEATURE file).

My approach is also to extend the cf code first for extending the
feature set and only if boundaries are touched there extend external
code.



Wolfgang
Nico Kadel-Garcia
2004-04-17 17:02:52 UTC
Permalink
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender address
by talking to the bestmx of the sender to find whether the address
itself is valid, just like exim does and postfix supports in the rc of
2.1.
Ahh. Sounds like another thesis avoidance project by a grad student. Of
course, it would break for every address for which there is no MX record
(such as ***@NOSPAM.hostname.com) and for every SMTP server that does
not allow you to query it this way but dumps such email in a bitbucket or
sends a bounce later (an increasingly common technique to avoid spammers
scanning them for valid addresses) or people who use fake usernames (such as
***@hostname.com), etc., etc.

I recommend that you don't bother, and spend your coding time on a more
useful set of tools (such as helping debug the SPF python code).
Claus Aßmann
2004-04-17 17:42:50 UTC
Permalink
"Wolfgang Hottgenroth" wrote in message
Post by Wolfgang Hottgenroth
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
Ahh. Sounds like another thesis avoidance project by a grad student. Of
course, it would break for every address for which there is no MX record
Why does it "break"? If there is no MX/A/AAAA record then the address
is not valid and hence won't be accepted.
NOSPAM.hostname.com. 3600 A 216.168.41.230
I recommend that you don't bother, and spend your coding time on a more
useful set of tools (such as helping debug the SPF python code).
SPF is broken by design.
--
A: Maybe because some people are too annoyed by top-posting.
Q: Why do I not get an answer to my question(s)?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Per Hedeland
2004-04-17 22:17:32 UTC
Permalink
Post by Nico Kadel-Garcia
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender address
by talking to the bestmx of the sender to find whether the address
itself is valid, just like exim does and postfix supports in the rc of
2.1.
Ahh. Sounds like another thesis avoidance project by a grad student. Of
course, it would break for every address for which there is no MX record
not allow you to query it this way but dumps such email in a bitbucket or
sends a bounce later (an increasingly common technique to avoid spammers
scanning them for valid addresses) or people who use fake usernames (such as
I recommend that you don't bother, and spend your coding time on a more
useful set of tools (such as helping debug the SPF python code).
You have it completely backwards. Sender address verification may not be
able to identify all invalid addresses (but the odds are - contrary to
what you claim - increasing since everyone tries to reject invalid
addresses as early as possible *instead* of bouncing later), and it is
rather costly in terms of resources, but it won't break anything - if
you send mail with an invalid sender address, you can't expect anyone to
transport it for you.

SPF on the other hand breaks all kinds of legitimate e-mail usage - yet
another example of destroying well-established functionality in a
(mostly futile) attempt to "fight" spam.

--Per Hedeland
***@hedeland.org
Nico Kadel-Garcia
2004-04-19 12:28:34 UTC
Permalink
Post by Per Hedeland
Post by Nico Kadel-Garcia
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender
address by talking to the bestmx of the sender to find whether the
address itself is valid, just like exim does and postfix supports
in the rc of
2.1.
Ahh. Sounds like another thesis avoidance project by a grad student.
Of course, it would break for every address for which there is no
server that does not allow you to query it this way but dumps such
email in a bitbucket or sends a bounce later (an increasingly common
technique to avoid spammers scanning them for valid addresses) or
people who use fake usernames (such as
I recommend that you don't bother, and spend your coding time on a
more useful set of tools (such as helping debug the SPF python code).
You have it completely backwards. Sender address verification may not
be able to identify all invalid addresses (but the odds are -
contrary to what you claim - increasing since everyone tries to
reject invalid addresses as early as possible *instead* of bouncing
later), and it is rather costly in terms of resources, but it won't
break anything - if you send mail with an invalid sender address, you
can't expect anyone to transport it for you.
Unfortunately, there's a significant percentage of usable email that
*cannot* be verified this way. This includes the mutilated addresses I
described, offline SMTP servers, servers that are send-only, servers that
refuse to allow off-site verification of email addresses to protect from
spammers, servers that swear *anything* is god in order to create tarpits
for spammers, etc.
Post by Per Hedeland
SPF on the other hand breaks all kinds of legitimate e-mail usage -
yet another example of destroying well-established functionality in a
(mostly futile) attempt to "fight" spam.
SPF didn't break them: they're already broken by the current virus and email
spam problems, especially the forgery of popular domain names and the user's
own domain name to get past blacklists. We had very similar screaming about
"it breaks my usage, oh woe is me" when postmasters started shutting down
open relays.

Review the documentation: most mailing list software deals with it just
fine, and the only things it really seems to break is forwarding, which is
quite unsalvagable in its current form due to the viral and spam loads it
would continue to allow. If we don't use something like SPF, we're going to
wind up stuck with a per-message token authentication system of some sort,
and it'll be *much* harder to manage and is likely to be a huge variety of
computationally expensive and proprietary tools.
Per Hedeland
2004-04-19 22:45:50 UTC
Permalink
Post by Nico Kadel-Garcia
Post by Per Hedeland
Post by Nico Kadel-Garcia
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using the
socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender
address by talking to the bestmx of the sender to find whether the
address itself is valid, just like exim does and postfix supports
in the rc of
2.1.
Ahh. Sounds like another thesis avoidance project by a grad student.
Of course, it would break for every address for which there is no
server that does not allow you to query it this way but dumps such
email in a bitbucket or sends a bounce later (an increasingly common
technique to avoid spammers scanning them for valid addresses) or
people who use fake usernames (such as
I recommend that you don't bother, and spend your coding time on a
more useful set of tools (such as helping debug the SPF python code).
You have it completely backwards. Sender address verification may not
be able to identify all invalid addresses (but the odds are -
contrary to what you claim - increasing since everyone tries to
reject invalid addresses as early as possible *instead* of bouncing
later), and it is rather costly in terms of resources, but it won't
break anything - if you send mail with an invalid sender address, you
can't expect anyone to transport it for you.
Unfortunately, there's a significant percentage of usable email that
*cannot* be verified this way. This includes the mutilated addresses I
described,
Of course mutilated address can be verified - to be invalid, and hence
rejected, as they should be.
Post by Nico Kadel-Garcia
offline SMTP servers, servers that are send-only,
If the MX for a domain points to a send-only server, there are bigger
problems. Please try to make at least a minimal effort to understand
what is being discussed.
Post by Nico Kadel-Garcia
servers that
refuse to allow off-site verification of email addresses to protect from
spammers, servers that swear *anything* is god in order to create tarpits
for spammers, etc.
Yes, as I wrote, the technique can not be guaranteed to reject all
invalid addresses - but the more important part is that it also won't
reject valid ones, unlike SPF.
Post by Nico Kadel-Garcia
Post by Per Hedeland
SPF on the other hand breaks all kinds of legitimate e-mail usage -
yet another example of destroying well-established functionality in a
(mostly futile) attempt to "fight" spam.
SPF didn't break them: they're already broken by the current virus and email
spam problems, especially the forgery of popular domain names
And this is the problematic attitude of a certain class of "spam
fighters" - "Spam breaks e-mail, it's OK to break it even more in trying
to avoid spam".
Post by Nico Kadel-Garcia
and the user's
own domain name to get past blacklists.
Ah, you need SPF so your mail server can check the DNS to find out
whether mail with a sender address in your domain is allowed to come
from some other place than your mail server... - sorry that I didn't see
this huge benefit.:-(
Post by Nico Kadel-Garcia
We had very similar screaming about
"it breaks my usage, oh woe is me" when postmasters started shutting down
open relays.
Unrestricted relaying was useful, and it was a loss when it went away -
but it was not crucial e-mail functionality, mostly just used by gurus
for routing around problematic servers or doing debugging. (Not counting
the cost of everyone having to explicitly configure what previously
"just worked" - this wasn't strictly loss of functionality as long as
the increased admin burden was accepted.)

E-mail forwarding is crucial functionality. Being able to use your
legitimate e-mail address from wherever you want is crucial
functionality.

--Per Hedeland
***@hedeland.org
Nico Kadel-Garcia
2004-04-20 00:48:39 UTC
Permalink
Post by Per Hedeland
Post by Nico Kadel-Garcia
Post by Per Hedeland
Post by Nico Kadel-Garcia
Post by Wolfgang Hottgenroth
Hi,
I've put together a prototype of a Sender Address Verifier using
the socket map option of sendmail 8.13.
It is not a kind of c/r verifier but just verifies the sender
address by talking to the bestmx of the sender to find whether the
address itself is valid, just like exim does and postfix supports
in the rc of
2.1.
Ahh. Sounds like another thesis avoidance project by a grad
student. Of course, it would break for every address for which
there is no
server that does not allow you to query it this way but dumps such
email in a bitbucket or sends a bounce later (an increasingly
common technique to avoid spammers scanning them for valid
addresses) or people who use fake usernames (such as
I recommend that you don't bother, and spend your coding time on a
more useful set of tools (such as helping debug the SPF python code).
You have it completely backwards. Sender address verification may
not be able to identify all invalid addresses (but the odds are -
contrary to what you claim - increasing since everyone tries to
reject invalid addresses as early as possible *instead* of bouncing
later), and it is rather costly in terms of resources, but it won't
break anything - if you send mail with an invalid sender address,
you can't expect anyone to transport it for you.
Unfortunately, there's a significant percentage of usable email that
*cannot* be verified this way. This includes the mutilated addresses
I described,
Of course mutilated address can be verified - to be invalid, and hence
rejected, as they should be.
Post by Nico Kadel-Garcia
offline SMTP servers, servers that are send-only,
If the MX for a domain points to a send-only server, there are bigger
problems. Please try to make at least a minimal effort to understand
what is being discussed.
You first.

Hmm. That's harsh.

There are plenty of SMTP servers in the world that are "send-only" as far as
the outside world is concerned, such as most Linux and Solaris and *BSD
installations. They receive only from localhost: any attempt to reach back
on port 25 to verify addresses is blocked, as they should be, since these
servers are only for outgoing email. Also, there are plenty of sites that
use one server for outgoing and another for incoming SMTP, due to bandwidth
limitations or different requirements for user authentication and filtering
for the different directions.

Demanding that these servers verify all email addresses from them as "valid"
is going to fail, until and unless you get them all to open up their user
verification for all the spammers to use to verify potential email addresses
or unless you can somehow open temporary permissions to any site they send
mail out to. Since mail may get held at various points for significant
periods of time due to connectivity failures, amounting to days for some
configurations, leaving these holes open is just asking for grief. And the
external MX record for a site may be a relay to a local user's machine.
You're going to demand that a site's external incoming SMTP server be
configured to do this fun and games, and risk bouncing lots of valid email?

I don't think so.
Post by Per Hedeland
Post by Nico Kadel-Garcia
servers that
refuse to allow off-site verification of email addresses to protect
from spammers, servers that swear *anything* is god in order to
create tarpits for spammers, etc.
Yes, as I wrote, the technique can not be guaranteed to reject all
invalid addresses - but the more important part is that it also won't
reject valid ones, unlike SPF.
I think I've covered those objections separately.
Post by Per Hedeland
Post by Nico Kadel-Garcia
Post by Per Hedeland
SPF on the other hand breaks all kinds of legitimate e-mail usage -
yet another example of destroying well-established functionality in
a (mostly futile) attempt to "fight" spam.
SPF didn't break them: they're already broken by the current virus
and email spam problems, especially the forgery of popular domain
names
And this is the problematic attitude of a certain class of "spam
fighters" - "Spam breaks e-mail, it's OK to break it even more in
trying to avoid spam".
*Sigh*. When you've got a bleeding arm that's been ripped off by a passing
truck, you tourniquet the hand and learn to use the other hand. You can't go
on pretending that the arm lying on the ground is still usable. And
unfortunately, that's the situation we're facing with these completely
unauthenticated, unverifiable email transmissions. And it's getting worse
every day.
Post by Per Hedeland
Post by Nico Kadel-Garcia
and the user's
own domain name to get past blacklists.
Ah, you need SPF so your mail server can check the DNS to find out
whether mail with a sender address in your domain is allowed to come
from some other place than your mail server... - sorry that I didn't
see this huge benefit.:-(
Hey, don't knock it. Writing the milters by hand to do this is tough, and
fragile, and difficult to maintain. It's a big benefit of the SPF approach.
Post by Per Hedeland
Post by Nico Kadel-Garcia
We had very similar screaming about
"it breaks my usage, oh woe is me" when postmasters started shutting
down open relays.
Unrestricted relaying was useful, and it was a loss when it went away
- but it was not crucial e-mail functionality, mostly just used by
gurus for routing around problematic servers or doing debugging. (Not
counting the cost of everyone having to explicitly configure what
previously "just worked" - this wasn't strictly loss of functionality
as long as the increased admin burden was accepted.)
I'm sorry, but for the last 10 years of my mail handling career it was used
by off-site email users who insisted, *insisted* that their email headers
have their work or academic credential's address. Explaining why this is a
problem in these days of spam has been a long-term problem: it's only this
last year that some of the labs over at MIT and Harvard turned off open
relaying, for example.

It's repeatedly caused a big issue for certain users. These are the same
users who kvetch and moan when they can't simply send the faked "From:"
addresses from their local ISP. And really, they are faked: if I'm using
comcast.net, and I send an email that says "From: ***@mybusiness.com",
that's really not where the email is coming from.

The SPF rules are merely a continuation of blocking open relays, but they
have a much, much finer grade of control.
Post by Per Hedeland
E-mail forwarding is crucial functionality. Being able to use your
legitimate e-mail address from wherever you want is crucial
functionality.
Then you already have plenty of ways to do that. The current unverifiable
mail forwarding whackiness just doesn't happen to be one of them with SPF,
without the additional SRS step. To get such an email accidentally blocked,
you need a whole series of settings to occur.

1: Your sender's email address must be using SPF with the restrictive,
"-all" settings. Few sites are doing that until and unless all the other fun
gets worked out.
2: Your forwarding host must not be using SRS.
3: You must use a ".forward" on that intermediate host where your email
was sent. (I don't believe that an alias has this problem, but I can check.)
4: Your receiving site must use SPF to block external mail that doesn't
match theh "-all" settings of SPF using sites.

These settings are extremely generous, and provide plenty of time for people
to implement the necessary steps. Really, it's designed from the ground up
to be implemented gracefully and allow exactly that time.
Aleksandar Milivojevic
2004-04-20 02:01:45 UTC
Permalink
Post by Nico Kadel-Garcia
It's repeatedly caused a big issue for certain users. These are the same
users who kvetch and moan when they can't simply send the faked "From:"
addresses from their local ISP. And really, they are faked: if I'm using
that's really not where the email is coming from.
The "From:" header, as well as argument of "MAIL FROM" contains the
information on *who* is the sender of email, not from where the email is
comming! This is exactly the same as when sending snail mail. You
write your home address on envelope and in heading, however this doesn't
mean you have to send it from your local post office down the street.
You can send it from Miami downtown, some god forgoten place in middle
of Arizona, or from Namibia.
Claus Aßmann
2004-04-20 02:10:43 UTC
Permalink
Post by Nico Kadel-Garcia
Post by Per Hedeland
Post by Nico Kadel-Garcia
offline SMTP servers, servers that are send-only,
If the MX for a domain points to a send-only server, there are bigger
problems. Please try to make at least a minimal effort to understand
what is being discussed.
You first.
Hmm. That's harsh.
Yeah, esp. since you don't seem to understand the topic.
Post by Nico Kadel-Garcia
There are plenty of SMTP servers in the world that are "send-only" as far as
the outside world is concerned, such as most Linux and Solaris and *BSD
installations. They receive only from localhost: any attempt to reach back
Please _read carefully_ what Per wrote. For your help I quote
the important part again:

"If the MX for a domain points to a send-only server"
^^^^^^^^^^^^^^^

Please understand how MX records and sender verification via
callbacks actually work. Thanks.
--
A: Maybe because some people are too annoyed by top-posting.
Q: Why do I not get an answer to my question(s)?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Jochen Bern
2004-04-20 02:52:08 UTC
Permalink
Post by Nico Kadel-Garcia
Post by Per Hedeland
Post by Nico Kadel-Garcia
servers that are send-only,
If the MX for a domain points to a send-only server, there are bigger
problems.
There are plenty of SMTP servers in the world that are "send-only" as far as
the outside world is concerned
[Groan]

Nico, if you're still referring to Wolfgangs implementation, please note
that he specifically mentioned contacting "the bestmx" for the address
in question in both the OP and his web page.

But anyway ... :
-- Verifying against the same IP that your MTA receives an SMTP
connection from is ***BROKEN*** because you're effectively asking
a spam cannon or zombie whether it's sending spam. A shell script
spitting a couple 2xx messages and run from inetd for TCP port 25
(plus timeout+exit if the client goes away) will provide all the
spam sent a-flyin' with an A-OK stamp.
-- Verifying against the MXes for the address is ***STILL BROKEN***
because this effectively turns your mailserver into an anonymizer/
proxy for address validity probing. Expect lots of happy spammers
to come haunt your MTA.
-- Hooking the test into check_mail, like Wolfgang did, is ***LIKELY
QUITE BROKEN***. It means that at the point where a MAIL FROM is
received, the verifier shoots "back" a series of commands including
*another* MAIL FROM. If the MX uses the same scheme but has different
whitelist/blacklist/From_-for-probe configured, an infinite loop
eating both servers' supply of sockets is going to start. (And I'm
not sure that adding cacheing will really save the concept here.)
-- Finally, the probe, as coded, goes HELO/MAIL FROM/RCPT TO/QUIT; it
would be polite to do an RSET before QUITting.

Regards,
J. Bern
Wolfgang Hottgenroth
2004-04-20 09:08:04 UTC
Permalink
Hi,


At Tue, 20 Apr 2004 04:52:08 +0200,
Post by Jochen Bern
-- Verifying against the MXes for the address is ***STILL BROKEN***
because this effectively turns your mailserver into an anonymizer/
proxy for address validity probing. Expect lots of happy spammers
to come haunt your MTA.
Indeed, that is a problem. Do you have a suggestion?
May be an IP should be blocked for a certain time after a 'sender
address verification' failure.
Post by Jochen Bern
-- Hooking the test into check_mail, like Wolfgang did, is ***LIKELY
QUITE BROKEN***. It means that at the point where a MAIL FROM is
received, the verifier shoots "back" a series of commands including
*another* MAIL FROM. If the MX uses the same scheme but has different
whitelist/blacklist/From_-for-probe configured, an infinite loop
eating both servers' supply of sockets is going to start. (And I'm
not sure that adding cacheing will really save the concept here.)
To avoid this infinite loop I use a sender address which must not be
verified: <>
Wietse wrote in his postfix docu concerning sender address
verification that using this MAILER-DAEMON address could be
problematically since certain MTA will not accept this address. He
uses postmaster@$MYDOMAIN instead. postmaster should also not be
verified. (Unfortunately, my code would verify postmaster. I need to
fix it.)
I don't know which sender address exim uses. I've to try.

Finally IMHO the check_mail ruleset is the right place (assuming this
infinite loop problem is solved) since I want to beat down invalid
addresses as few as possible and definitely don't want to queue the
mail.
Post by Jochen Bern
-- Finally, the probe, as coded, goes HELO/MAIL FROM/RCPT TO/QUIT; it
would be polite to do an RSET before QUITting.
In fact? I was in the assumption that QUIT is polite enough. Says the
RFC or best practice something different? (Actually I don't know,
please help.)



Cheers,
Wolfgang
Jochen Bern
2004-04-20 13:43:57 UTC
Permalink
Post by Wolfgang Hottgenroth
At Tue, 20 Apr 2004 04:52:08 +0200,
Post by Jochen Bern
-- Verifying against the MXes for the address is ***STILL BROKEN***
because this effectively turns your mailserver into an anonymizer/
proxy for address validity probing. Expect lots of happy spammers
to come haunt your MTA.
Indeed, that is a problem. Do you have a suggestion?
May be an IP should be blocked for a certain time after a 'sender
address verification' failure.
Denying access to abusers would be one possibility, and, as far as I can
estimate off the top of my head, the most promising one here. (Note that
in order to get blacklisted, "an IP" first needs to make a TCP
connection to you, which makes leveraging the blacklist mechanism to
launch a DoS attack sufficiently hard to do.)

The second possibility would be to degrade the feedback to the remote
host so as to render it more or less useless for the "proxying attack".
For example, make the status codes used for rejection temporary ones
(4xx's), randomly reject (say) 5% of MAIL FROM:s without ever doing the
background check, and from the checked ones, select another random 5%
that you still accept (but discard later on). Note that the figures need
fiddling to make it "convincing". If not unannounced random changes, to
throw off attackers that try to analyze and adapt.

The third possibility, of course, would be to forget about *bandwidth*
saved and just use the mechanism to reduce *memory* usage (by accepting
the mail and then discarding it). This may sound like a REALLY BAD idea,
but consider the scenario of a spammer running a dictionary attack
against your server. Do you actually save *bandwidth* by rejecting every
single transaction as early as possible? Only if you fail to detect the
attempt and let the spammer finish the dictionary run; if, instead, you
notice it after (say) two hours and block his IP, you'll have lost about
the same amount of bandwidth, only that it has been used for millions of
MAIL FROM/rejects instead of a thousand MAIL/RCPT/DATAs with subsequent
discarding. In order to save bandwidth from square one in *this*
scenario, you would have to teergrube the connection (and, of course,
limit the number of simultaneous connections from the same IP) instead
of serving the spammer a rejection right away.
Post by Wolfgang Hottgenroth
Finally IMHO the check_mail ruleset is the right place (assuming this
infinite loop problem is solved)
Doing so by making assumptions about how the other party will handle
addresses is risky at best, though. I still think that your best bet to
prevent an infinite loop would be short-time cacheing of verification
results. However, since there's nothing that keeps the other party from
using dynamic addresses in *his* verification attempts, to make the
process more traceable or whatever, this would *still* not *guarantee*
that no infinite loop will occur.
Post by Wolfgang Hottgenroth
Post by Jochen Bern
-- Finally, the probe, as coded, goes HELO/MAIL FROM/RCPT TO/QUIT; it
would be polite to do an RSET before QUITting.
In fact? I was in the assumption that QUIT is polite enough. Says the
RFC or best practice something different? (Actually I don't know,
please help.)
I didn't specifically look up an RFC or any other document for this, I
only followed the inherent logic as I understand it. MAIL and RCPT
introduced some state into the TCP connection (by specifying an envelope
and moving the finite state machine to the point where a DATA may
follow). There's a command specifically meant to reset the state, and
that's RSET. Consider it a good_thing_to_do (tm) *just in case* that, in
some distant future, you add connection cacheing to your code; you can
do another MAIL/RCPT(/RSET) after having done a MAIL/RCPT/RSET in the
same connection, but *NOT* after only MAIL/RCPT.

Regards,
J. Bern
John Kelly
2004-04-20 02:36:34 UTC
Permalink
Post by Per Hedeland
E-mail forwarding is crucial functionality
Perhaps it seemed so in the 1970s; but now, there is very little left
from the Internet of that quaint era. In today's Internet, remailing,
with traceability, makes more sense, at least to me.

I popped in with a different question, but I see that leading members
of the sendmail community are still trapped in a 1970's mindset; which
leaves me to wonder how sendmail can ever outgrow its past.

It may be time for me to look at a more modern MTA.
Claus Aßmann
2004-04-20 04:28:10 UTC
Permalink
Post by John Kelly
Perhaps it seemed so in the 1970s; but now, there is very little left
from the Internet of that quaint era. In today's Internet, remailing,
with traceability, makes more sense, at least to me.
I can check very well how my forwarding works and trace it.
Post by John Kelly
I popped in with a different question, but I see that leading members
of the sendmail community are still trapped in a 1970's mindset; which
leaves me to wonder how sendmail can ever outgrow its past.
It may be time for me to look at a more modern MTA.
Yeah, I would suggest to look into postfix.

Then Wietse will tell you that SPF is broken (try to mention it on
the postfix list and you'll get kicked off) and that forwarding
works just fine, thanks.

! Solutions such as SPF do not deprive spammers from falsifying the
! sender address.
!
! And solutions such as SRS actually help spammers by laundering
! their sender address.
!
! Wietse

Good luck looking "at a more modern MTA" if you hope that
the author(s) supports you views about SPF and forwarding.
Strange how MTA authors have similar views about SPF etc...
--
A: Maybe because some people are too annoyed by top-posting.
Q: Why do I not get an answer to my question(s)?
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
Aleksandar Milivojevic
2004-04-20 04:44:19 UTC
Permalink
Post by John Kelly
It may be time for me to look at a more modern MTA.
You can run, but you can't hide.
Andrzej Adam Filip
2004-04-20 06:09:00 UTC
Permalink
Post by John Kelly
Post by Per Hedeland
E-mail forwarding is crucial functionality
Perhaps it seemed so in the 1970s; but now, there is very little left
from the Internet of that quaint era. In today's Internet, remailing,
with traceability, makes more sense, at least to me.
[...]
Let me post my opinion about SPF:
You do not have to be good, it is enough to be the best.
SPF is bad but it seems to be the best proposal to significantly limit
sender faking with real chances of fast implementation.

I hope that SPF implementations will *force* SPF opponents to propose a
better safeguard against sender faking instead of endless complains
"it is bad STOP do nothing STOP" (exaggerated).

SMTP was not designed (long time ago) to protect against abuse (sender
faking, spam) and it must be fixed *somehow*. Replacing SMTP by
*completely redesigned protocol* will take long time.

I prefer SPF like solutions over "small Big Brother" in my mailbox
(filtering based on message context) I am forced to use.

http://spf.pobox.com/ :
[...] Major publishing domains include: AOL.com Altavista.com DynDNS.org

P.S. Life is a limited choice game.
--
Andrzej [en:Andrew] Adam Filip ***@priv.onet.pl ***@xl.wp.pl
http://anfi.homeunix.net/ http://slashdot.org/~anfi
*Random Epigram* :
"All God's children are not beautiful. Most of God's children are, in fact,
barely presentable."
-- Fran Lebowitz
Jochen Bern
2004-04-20 13:04:27 UTC
Permalink
Post by Andrzej Adam Filip
I hope that SPF implementations will *force* SPF opponents to propose a
better safeguard against sender faking instead of endless complains "it
is bad STOP do nothing STOP" (exaggerated).
OK, my $.02 on the matter:

The first question is: Why do spammers use "faked senders" (whatever the
definition of this term in *this* context may be (*)) in the first place?

As we all know, there are things like expendable accounts/addresses,
spam friendly ISPs, registrars that do not thoroughly check contact
information, yadda yadda. If we succeed to enforce "non-faked" sender
addresses, and assumed that the point of faked senders were to just make
it impossible to trace spam back to the spammer, would they have much of
a problem switching to non-faked, but still utterly useless (in terms of
recognizing or backtracing spam) sender addresses? I don't think so.
Getting a technically legitimate address or even domain is neither
costly nor time-consuming nor complicated nor buried under red tape, or
at least not much, and that's unlikely to change.

So, why aren't spammers doing just that right now?

Possibility 1: They're tight-fisted bastards and, when given the choice
between swimming through hot legal water (ID theft, implicitly libelling
the joe-jobbed address, antispam legislation specifically mentioning
falsified contact info, creating zombie computers with virii, you name
it) for free and spending just a couple dollars more, they go for the
first option. In this case, what effect would it have to have the good
guy half of the world stomp out faked senders? It would force them to
switch to option 2. Net effect *so far*: negligible.

Possibility 2: They're not actually interested in disguising as just
SOME sender, they're trying to impersonate a sender or sending domain
that the recipient TRUSTS so as to make filtering more difficult. Now
let's assume that we manage to make that impossible. Will spammers stop
trying to send spam? No, as detailed above, they'll just use different
sender addresses. Net effect: Joe Average Email User will feel more
encouraged to install a whitelist and "be done with it" than ever before.

Is that the future of email that we want to see come true? That everyone
and his MTA takes on the habit of downright IGNORING emails unless the
sender has (say) spelled his sender address in a phone call to have it
whitelisted?

Sorry, not in my books; if I wanted *that*, a much more sound technical
implementation would be to install an FTP server, create write-only
accounts for the "whitelisted" people, tell them userid and password by
the OOB channel we're already assuming to exist, and have them transfer
"emails" in MH format straight to my hard disk. Or go back to UUCP
networking, only with SSH logins instead of modem dialups. Just NOT
SMTP, as it is based on the idea of providing service to whichever
anonymous who can connect to port 25, which is exactly what I'ld be
basically trying to get rid of.

I'm *not at all* ready to take the stance of "if I do not know you
*already*, you seem not to be worth listening to you". And I certainly
don't think that I'm ever going to know the reputation of most people or
entities on the Internet. By logical conclusion, yes, I will continue to
look - with varying degrees of attention, of course - at emails where I
do not have the slightest who the guy sending it may be. What I need in
order to put up a fight against spam in *this* scenario is not simply
that there is some reliable ID attached to email, but also that
obtaining new such IDs all the time will not take place - whether it's
made downright impossible, or just too costly to have spamming stay
profitable.

What technical means does that translate to? Well, I don't think that
that's cast in stone yet. There has been a time where everyone expected
PKIs to bring widely deployed cryptographic signatures and positive ID
(be it Verisign certificates for servers worldwide, SigG conformant
personal certificates in Germany, or whatever), but it didn't work out.
Some hold the belief that the PGP-style web of trust could do such a
thing if given a chance (where the asset that a spammer cannot easily
regenerate would likely be a not-too-new, sufficiently unique listing on
the keyservers). The newest contender is the .mail TLD, making the
subdomains thereunder the asset that's costly to replace. Not to forget
C/R (asset = whitelisted address after responding to the challenge) and
"email stamps" (I forget the official name; the stuff where you have to
solve a cryptographic puzzle before you can move on in the SMTP
transaction, making this very connection the valuable asset). And
probably a good deal more, some using the sender address for IDing
(which would then need to be protected against forgeries), some not. As
long as it works and doesn't chain me to some monopoly, I do not care
which one will prevail. Or which *selection of several*, for that matter.

Regards,
J. Bern

(*) I'm using the SMTP (and NNTP) server of kamp-dsl.de. KAMP is a
*connectivity* provider, and I *could* be a customer of theirs
with no email address technically connected to them *whatsoever*.
Is it "sender faking" when I hand the KAMP SMTP server an email
with my real email address on it, just because I get connectivity
and a POP mailbox from different providers? What if I made a typo
in that address, would THAT be forged, and if not, how on *Earth*
do you expect a technical mechanism to tell *that* from an all-out
forgery?
Aleksandar Milivojevic
2004-04-20 23:06:10 UTC
Permalink
Post by Jochen Bern
What technical means does that translate to? Well, I don't think that
that's cast in stone yet. There has been a time where everyone expected
PKIs to bring widely deployed cryptographic signatures and positive ID
(be it Verisign certificates for servers worldwide, SigG conformant
personal certificates in Germany, or whatever), but it didn't work out.
Some hold the belief that the PGP-style web of trust could do such a
thing if given a chance (where the asset that a spammer cannot easily
regenerate would likely be a not-too-new, sufficiently unique listing on
the keyservers).
PKI's never brought widely deployed signatures simply because the system
was design to generate profit for CAs. For commercial web sites, this
is not problem. $100/year for somebody to certify it is really you is
cheap. For end users, $10/year per email address is one utility bill
too much. And there's also a small problem of why should I trust CA?
Verisign and other CAs fell out of the blue some time ago, and now are
the most trusted entities on Internet. If government can base entire
war on lies, you should expect a commercial entity to be even worse.
Andrzej Adam Filip
2004-04-21 18:10:35 UTC
Permalink
Post by Aleksandar Milivojevic
Post by Jochen Bern
What technical means does that translate to? Well, I don't think that
that's cast in stone yet. There has been a time where everyone
expected PKIs to bring widely deployed cryptographic signatures and
positive ID (be it Verisign certificates for servers worldwide, SigG
conformant personal certificates in Germany, or whatever), but it
didn't work out. Some hold the belief that the PGP-style web of trust
could do such a thing if given a chance (where the asset that a
spammer cannot easily regenerate would likely be a not-too-new,
sufficiently unique listing on the keyservers).
May be the time has came to create "PGP/GPG keys of email address".
It would make automatic signing such keys simple and ease creating warp
of the "web of trust".
AFAIK the key servers do use email addresses provided in the submitted
keys to verify validity of key id.
Post by Aleksandar Milivojevic
PKI's never brought widely deployed signatures simply because the system
was design to generate profit for CAs. For commercial web sites, this
is not problem. $100/year for somebody to certify it is really you is
cheap. For end users, $10/year per email address is one utility bill
too much. And there's also a small problem of why should I trust CA?
Verisign and other CAs fell out of the blue some time ago, and now are
the most trusted entities on Internet. If government can base entire
war on lies, you should expect a commercial entity to be even worse.
good point :-) [ "the winner is always right" as said the looser ]
--
Andrzej [en:Andrew] Adam Filip ***@priv.onet.pl ***@xl.wp.pl
http://anfi.homeunix.net/ http://slashdot.org/~anfi
*Random Epigram* :
The greatest productive force is human selfishness.
-- Robert Heinlein
Jochen Bern
2004-04-21 19:50:37 UTC
Permalink
Post by Andrzej Adam Filip
Post by Jochen Bern
Some hold the belief that the PGP-style web of trust
could do such a thing if given a chance (where the asset that a
spammer cannot easily regenerate would likely be a not-too-new,
sufficiently unique listing on the keyservers).
May be the time has came to create "PGP/GPG keys of email address".
It would make automatic signing such keys simple and ease creating warp
of the "web of trust".
As a matter of fact, key signatures are attached to user IDs, not the
key as a whole. In other words, you could have your keypair carry one
user ID *with* your name and one *without* and let people sign either
only the latter (verification via email) or both (verification via key
signing party or similar).

Not that there were a shortage of people willing to sign your key
without checking *everything* in the user ID. :-C My user IDs typically
read "Jochen P. Bern (DD0KZ) <***@my.email.addresses>" and I've
*NEVER* been asked to provide proof that DD0KZ is in fact my radio
amateur call sign so far. I've had people sign my key without ever
looking into a photo ID (note that I completely omit my middle name in
everyday use). Some participants at a key signing party signed my key
and submitted the result straight to the keyservers (which means that I
can get their signature from there with the email address part never
getting verified). And even most of those who *did* verify some part of
*one* user ID on a key with several IDs failed to restrict their
signature to the user ID(s) that contain the information they did
verify. :-/
Post by Andrzej Adam Filip
AFAIK the key servers do use email addresses provided in the submitted
keys to verify validity of key id.
Not that I remember (latest keypair generated Sep-2002 and submitted to
the keyservers somewhere between that and mid 2003).

Regards,
J. Bern
Andrzej Adam Filip
2004-04-23 08:01:10 UTC
Permalink
[ PGP/GPG key signatures sign particaulr IDs ... ]
Post by Andrzej Adam Filip
AFAIK the key servers do use email addresses provided in the submitted
keys to verify validity of key id.
Not that I remember (latest keypair generated Sep-2002 and submitted to
the keyservers somewhere between that and mid 2003).
has anybody proposed creating "email signing servers" to sign keys after
procedure similar to list subscription ?
[but with submitting some test signed the key]

The server may generate "limited time" signatures e.g. 90 days.

***@domain may sign ***@domain and ...
--
Andrzej [en:Andrew] Adam Filip ***@priv.onet.pl ***@xl.wp.pl
http://anfi.homeunix.net/ http://slashdot.org/~anfi
*Random Epigram* :
Never look a gift horse in the mouth.
-- Saint Jerome
Jochen Bern
2004-04-23 11:01:25 UTC
Permalink
Post by Andrzej Adam Filip
has anybody proposed creating "email signing servers" to sign keys after
procedure similar to list subscription ?
Not that I know ... I'ld probably want to fiddle with the error modes of
the attached MTA if I were to implement such a beast (no queue_warn,
queue_return just a couple hours, no bodies in bounces, maybe bounces
straight to /dev/null in the first place) to prevent signatures to go
*anywhere but the address in the user ID and right NOW* ... and there
should be some rate limiting, as the crypto computations initiated by
incoming requests would likely be able to overwhelm the CPU ...

The worst practical problem that I see on the road is that, in order to
prevent exploitation of some kind, such a server would have to restrict
itself to only the REALLY proper operations - read: Only sign user IDs
that are nothing but a well-formed email address, possibly even reject
pubkeys that have several user IDs - which is clearly over the head of
the average PGP/GPG user ...

Anyone else seeing another show-stopper?
Post by Andrzej Adam Filip
[but with submitting some test signed the key]
I think you're trying to solve a nonexisting problem here. A well-formed
pubkey *is* already self-signed for the very purpose of proving that a
matching secret key is in existence. Or are you thinking of some goon
submitting requests for *someone else's* pubkeys by the keyserverful?
Hmmmm, requiring him to sign a timestamped "order" *would* limit him to
the rate he can create fake keypairs at ...
(No flamewars on the topic of "is transitivity of trust a broken
concept", please. :-} )

Regards,
J. Bern
Andrzej Adam Filip
2004-04-23 14:58:46 UTC
Permalink
Post by Jochen Bern
Post by Andrzej Adam Filip
has anybody proposed creating "email signing servers" to sign keys
after procedure similar to list subscription ?
Not that I know ... I'ld probably want to fiddle with the error modes of
the attached MTA if I were to implement such a beast (no queue_warn,
queue_return just a couple hours, no bodies in bounces, maybe bounces
straight to /dev/null in the first place) to prevent signatures to go
*anywhere but the address in the user ID and right NOW* ... and there
should be some rate limiting, as the crypto computations initiated by
incoming requests would likely be able to overwhelm the CPU ...
I think packages for mail list have developed good techniques to handle
very similar problems already :-)
Post by Jochen Bern
The worst practical problem that I see on the road is that, in order to
prevent exploitation of some kind, such a server would have to restrict
itself to only the REALLY proper operations - read: Only sign user IDs
that are nothing but a well-formed email address, possibly even reject
pubkeys that have several user IDs - which is clearly over the head of
the average PGP/GPG user ...
Anyone else seeing another show-stopper?
I thought about signing *ONLY* keys published at current key servers
e.g. milter rejects "sign request" of keys not published at key servers
(with local "synchronized" key server present).
Post by Jochen Bern
Post by Andrzej Adam Filip
[but with submitting some test signed /by/ the key]
[sorry for missing "by"]
Post by Jochen Bern
I think you're trying to solve a nonexisting problem here. A well-formed
pubkey *is* already self-signed for the very purpose of proving that a
matching secret key is in existence. Or are you thinking of some goon
submitting requests for *someone else's* pubkeys by the keyserverful?
Hmmmm, requiring him to sign a timestamped "order" *would* limit him to
the rate he can create fake keypairs at ...
It is very close to what I have tried to suggest :-)
[ send "timestamps" published by the signing server signed by the key
you with to be signed ]
Post by Jochen Bern
(No flamewars on the topic of "is transitivity of trust a broken
concept", please. :-} )
Automatic "mailing list subscription like" signing of keys is something
different then signing by person according to exactly followed "signing
party" procedures.
But (IMHO) it delivers something valuable anyway.
--
Andrzej [en:Andrew] Adam Filip ***@priv.onet.pl ***@xl.wp.pl
http://anfi.homeunix.net/ http://slashdot.org/~anfi
*Random Epigram* :
When I was little, I went into a pet shop and they asked how big I'd get.
-- Rodney Dangerfield
Wolfgang Hottgenroth
2004-04-19 08:06:46 UTC
Permalink
At Sat, 17 Apr 2004 13:02:52 -0400,
Nico Kadel-Garcia wrote:
<ignore on>
Post by Nico Kadel-Garcia
Ahh. Sounds like another thesis avoidance project by a grad student. Of
<ignore off>
Post by Nico Kadel-Garcia
course, it would break for every address for which there is no MX record
Why shouldn't I break those email addresses? I don't want a serious
mail server to accept mail from addresses which do not accept
bounces. If it would accept that mail and something goes wrong
downstream (recipients mailbox is over quota, suspended due to billing
reasons, simply does not exist ...) my resources (at least my queue
space) would be wasted.
I assume envelope sender addresses to be reachable, otherwise I see no
reason to accept those mail. And for the rare exception, there is the whitelist.
If you want to use a none-reachable sender address no one stops you
from entering it into your aliases file pointing to /dev/null.
Post by Nico Kadel-Garcia
not allow you to query it this way but dumps such email in a bitbucket or
I don't understand this. I won't send any email when probing addresses.
Post by Nico Kadel-Garcia
sends a bounce later (an increasingly common technique to avoid spammers
scanning them for valid addresses) or people who use fake usernames (such as
That is a problem indeed. You mean for instance Yahoo.
Same as above: I assume envelope sender addresses to be reachable. If
you want to profit from using unreachable sender addresses (and no
question I can image reasons to do so) do it on your own costs
(/dev/null in aliases file) and not on the cost of other ones.



Wolfgang
Wolfgang Hottgenroth
2004-04-19 08:07:46 UTC
Permalink
At Sat, 17 Apr 2004 13:02:52 -0400,
Nico Kadel-Garcia wrote:
<ignore on>
Post by Nico Kadel-Garcia
Ahh. Sounds like another thesis avoidance project by a grad student. Of
<ignore off>
Post by Nico Kadel-Garcia
course, it would break for every address for which there is no MX record
Why shouldn't I break those email addresses? I don't want a serious
mail server to accept mail from addresses which do not accept
bounces. If it would accept that mail and something goes wrong
downstream (recipients mailbox is over quota, suspended due to billing
reasons, simply does not exist ...) my resources (at least my queue
space) would be wasted.
I assume envelope sender addresses to be reachable, otherwise I see no
reason to accept those mail. And for the rare exception, there is the whitelist.
If you want to use a none-reachable sender address no one stops you
from entering it into your aliases file pointing to /dev/null.
Post by Nico Kadel-Garcia
not allow you to query it this way but dumps such email in a bitbucket or
I don't understand this. I won't send any email when probing addresses.
Post by Nico Kadel-Garcia
sends a bounce later (an increasingly common technique to avoid spammers
scanning them for valid addresses) or people who use fake usernames (such as
That is a problem indeed. You mean for instance Yahoo.
Same as above: I assume envelope sender addresses to be reachable. If
you want to profit from using unreachable sender addresses (and no
question I can image reasons to do so) do it on your own costs
(/dev/null in aliases file) and not on the cost of other ones.



Wolfgang
Loading...