Mailing List Archive

A couple of comments (I hope I am more awake this time than I have
been for my other posts).

It took me a while to wrap my mind around qmail's approach of doing
things, but once I started treating the qmail core like a part of a
toolbox (just like awk, sed, perl, ps, and other 'tools'), the light
went on.

Qmail is a very specific, small, and effecient tool for delivering
mail. It is not a tool for header rewriting, bang path conversion,
mailing list management, or the like (I am talking about the core
here, not things like qlist).

If you start adding other things into the core (like header
rewriting), then you increase the risk of adding bugs into the core.
If you have a core that is bug-free (or the bug rate is negligibly
small), then you can _reliably_ use that tool as a key internal part
to larger systems, without having to worry about it doing something
unexpected.

If you use the tool as part of the system for rewriting addresses (ie:
write a program wrapper that handles address rewriting for a class of
mail - say outgoing), then you don't have to worry about the address
rewriting part affecting the rest of the systems that use the qmail
'tool' - the changes in the address rewriting part of the system are
isolated to that singe part of the system instead of the core.

Personally, I find myself writing much more robust code when I use
many small tools, each with a distinct function, and glue them
together with perl or some other scripting language. I find the small
bit of overhead this causes to be more than acceptable when weighing
it against the bugs that adding a feature on to a tool (such as the
qmail core) generates.

Perhaps an archive of add-ons (wrappers) that can be "plugged-in" to
qmail to solve standard problems would help.

Nice job Dan.

Brian

In message <19970301104041.11971.qmail@bigred.inka.de>,
Olaf Titz wrote:
> > It'd only be more "resource friendly" for people that actually want to
> > use it. For those that don't, it's unnecessary code bloat.
>
> Okay, let's look at this argument closer for a moment. It is true for
> _every_ imaginable feature a program could have. For people who don't
> use it, it's "code bloat". For people who need it, it is a question of
> efficiency.
>
> Just an example: because qmail can't rewrite headers, my current setup
> forces me to pass each and every message through qmail twice. I call
> this inefficient. (No, that it is still faster than smail does not
> matter for this argument.) Other people might call header rewriting
> code "bloat" just because they don't need them. The latter is the
> canonical approach towards every feature request on this list: "who
> does need that?", "you can do this with a handful of add-ons", etc.
>
> There are a number of features that are often wanted because there is
> a need for them. Implementing them would _not_ constitute code bloat,
> given sufficient need. Or else we wouldn't need MTAs at all. The
> workarounds at the individual sites can be seen as just as bad.
>
> Take this as a plea for less hostility towards feature requests.
> (I'm still missing a possibility for declaring a message remote, in
> contrast to virtualdomains which declares a message local...)
>
> olaf

Brian T. Wightman writes:

> Perhaps an archive of add-ons (wrappers) that can be "plugged-in" to
> qmail to solve standard problems would help.

That's the idea behind http://www.qmail.org. It provides us a place
to distribute these things which are not part of the base
distribution.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

Olaf Titz:
> Just an example: because qmail can't rewrite headers, my current
> setup forces me to pass each and every message through qmail
> twice. I call this inefficient.

I don't mind the "inefficiency". I do mind what it does to logging.

--
Raul

> things, but once I started treating the qmail core like a part of a
> toolbox (just like awk, sed, perl, ps, and other 'tools'), the light

Yes, I see it in a similar light. qmail fits in the Unix approach of
small, combine-able tools.

But nevertheless IMHO some features are missing or improveable.
See it in the following way: the interface between this and other
tools could be better.

> mail. It is not a tool for header rewriting, bang path conversion,
> mailing list management, or the like (I am talking about the core

Bang path conversion is best handled by an external rmail program.
OK on this. You can run into problems with UUCP MTAs that can't handle
RFC822 addresses at all, but that is a more difficult issue anyway.
Similar for X.400 (or even Fidonet :-)

But take another look at header rewriting. Yes, it is possible with an
external tool. But it is just _suboptimal_ and aesthetically
unpleasing to require for this often wanted task that each and every
message pass the MTA twice. Just the possibility that the core
functionality - just the core functionality - _could_ use half the CPU
horsepower that it now uses (disregarding any comparisons to any other
MTA) makes me think there must be room for optimizations. (Same holds
for the beaten-to-death issue of bundling deliveries. Russell insists
on measurements of actual bandwidth taken, but it's trivial to _prove_
that sending a message twice out my network connection requires double
the bandwidth than sending it once, not measurements needed.)

Another problem with header rewriting is that there is not a single
interface for doing it. The standard solution works only for messages
received via SMTP, not for those received via qmail-inject. A wrapper
around qmail-inject would need to duplicate 3/4 of qmail-inject's
functionality. I call this unnecessary duplication of effort.

> Personally, I find myself writing much more robust code when I use
> many small tools, each with a distinct function, and glue them
> together with perl or some other scripting language. I find the small
> bit of overhead this causes to be more than acceptable when weighing

It is not about the small overhead of gluing tools together. (Or else
I would have abandoned C News long ago instead of writing add-ons to
it. ;-) It is about a difficult interface that requires processing
every message twice, i.e. a 100% overhead. That's just too much for
someone who is used to counting assembler cycles in library functions
:-)

olaf

Olaf Titz:
> But take another look at header rewriting. Yes, it is possible with
> an external tool. But it is just _suboptimal_ and aesthetically
> unpleasing to require for this often wanted task that each and every
> message pass the MTA twice.

Why suboptimal? Near as I can tell, the general problem of header
rewriting is turing equivalent.

Of course, it would be nice to have a good set of RFC822 oriented
tools.

> (Same holds for the beaten-to-death issue of bundling
> deliveries. Russell insists on measurements of actual bandwidth
> taken, but it's trivial to _prove_ that sending a message twice out
> my network connection requires double the bandwidth than sending it
> once, not measurements needed.)

You've assumed that bandwidth is the figure of merit. Why? (a)
typical network usage is significantly less than 100%, and even for
high use instances, mail typically constitutes only a small fraction
of the total.

> Another problem with header rewriting is that there is not a single
> interface for doing it.

Virtual hosts?

--
Raul

Olaf Titz <olaf@bigred.inka.de> writes on 4 March 1997 at 15:44:51 +0100

> (Same holds for the beaten-to-death issue of bundling
> deliveries. Russell insists on measurements of actual bandwidth
> taken, but it's trivial to _prove_ that sending a message twice out
> my network connection requires double the bandwidth than sending it
> once, not measurements needed.)

Sure, sending something twice takes more bandwidth than sending it
once. However, sending stuff in parallel is faster than sending stuff
sequentially. And using fully reliable automatic bounce handling with
the owner hack saves list maintainer time. Sending things compressed
takes less bandwidth than sending them normally, too. As nearly
always in any sort of engineering, there isn't an "optimal" product
possible, there are always tradeoffs to be made. Qmail is optimized
to save human time at the expense of some network resources, for
directly-connected systems. This seems to me a perfectly coherent
(and useful) design philosophy.

The fact that qmail delivers even mailing list mail faster (less
elapsed time) than sendmail, with less administrator overhead, using
about the same total network bandwidth, and less CPU, sounds to me
like a winning combination. (If Dan's claims in this area aren't
true, well, nobody has yet performed a measurement that disproves
them.)

> > an external tool. But it is just _suboptimal_ and aesthetically
> > unpleasing to require for this often wanted task that each and every
> > message pass the MTA twice.
> Why suboptimal? Near as I can tell, the general problem of header
> rewriting is turing equivalent.

smail can do it in one pass too, with a remove_header and an
append_header statement, the latter of which contains a lookup
expansion (non-smailers can probably guess what this means - hint:
"lookup" is on dbm files).

Of course the problem is Turing equivalent and of course even
sendmail.cf is not the right approach for, say, a Fidonet gateway. :-)
But there are a lot of less exotic cases that could be handled very
easily.

> You've assumed that bandwidth is the figure of merit. Why? (a)

Because bandwidth costs money. At least here in Germany this is the case.

> > Another problem with header rewriting is that there is not a single
> > interface for doing it.
> Virtual hosts?

No. Virtual hosts only work when you either (a) finally deliver
locally or (b) change the recipient address. It does not work in the
following case: Mail that goes to any host in domain X should have its
headers rewritten and sent to the original, unchanged recipient via
SMTP.

olaf

Olaf Titz writes:
> > You've assumed that bandwidth is the figure of merit. Why? (a)
>
> Because bandwidth costs money. At least here in Germany this is the case.

But the price of bandwidth is dropping precipitiously, and the cost of
people time is growing, as it has been growing for the last five
hundred years. Not reasonable to expect either trend to change over
the lifetime of qmail. Every program involves compromises, and extra
features imposes complexity; complexity that increases the chance of
errors. An MTA is inherently complex. It's reasonable to make
choices to reduce the complexity.

> > > Another problem with header rewriting is that there is not a single
> > > interface for doing it.
> > Virtual hosts?
>
> No. Virtual hosts only work when you either (a) finally deliver
> locally or (b) change the recipient address. It does not work in the
> following case: Mail that goes to any host in domain X should have its
> headers rewritten and sent to the original, unchanged recipient via
> SMTP.

Agreed here. The only way I can see to perform this case using the
existing code is to stuff the outgoing mail into a Maildir and
maildir2smtp it. Problem with doing this is that maildir2smtp's
queuing algorithm is stinky compared to qmail's.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

> But the price of bandwidth is dropping precipitiously, and the cost of
> people time is growing, as it has been growing for the last five
> hundred years. Not reasonable to expect either trend to change over
> the lifetime of qmail. Every program involves compromises, and extra
> features imposes complexity; complexity that increases the chance of
> errors. An MTA is inherently complex. It's reasonable to make
> choices to reduce the complexity.

Perhaps you're unaware that the Internet today is far less robust
and usable than it used to be.

There is more packet loss and seemingly far less effective congestion
control than there used to be.

In general this phenonmenon is blamed on the WWW, and on http clients'
insistence on opening multiple TCP connections, which degrade the
network and where a lost SYN on one connection does not cause a
backoff in the openning of other connections. This is bad for the
network -- it's one of the reasons that HTTP is moving toward
persistent and multiplexable connections.

qmail has the same problems. It may appear to you that you do not have
a bandwidth shortage at your "bottleneck", but openning multiple
connections like this is very bad for the core of the network.

It would be ironic to think that just as we're fixing the Web, mail
transport becomes the next victim.

I suppose it would be interesting to match up numbers data on TCP
connections (if there was any), network packet loss (if it was
meaningful), and qmail deployment (if there was any) and look for
correlations. Unfortunately, I'm only pessimistic about what could be
found.

Scary to think of mail transport learning lessons from the Web, isn't
it?

--jhawk

Olaf Titz:
> > > an external tool. But it is just _suboptimal_ and aesthetically
> > > unpleasing to require for this often wanted task that each and every
> > > message pass the MTA twice.

Raul Miller:
> > Why suboptimal? Near as I can tell, the general problem of header
> > rewriting is turing equivalent.

Olaf Titz:
> smail can do it in one pass too, with a remove_header and an
> append_header statement, the latter of which contains a lookup
> expansion (non-smailers can probably guess what this means - hint:
> "lookup" is on dbm files).

Lookup of what? An smtp address? A host name? A domain?

> Of course the problem is Turing equivalent and of course even
> sendmail.cf is not the right approach for, say, a Fidonet gateway. :-)
> But there are a lot of less exotic cases that could be handled very
> easily.

Sure, but all cases can be handled fairly easily already. The only
significant issue I'm aware of, where improvements could be made, is
logging. [.It's a bit harder to analyze logs when you get two
distinct sets of log entries per message.]

> > You've assumed that bandwidth is the figure of merit. Why? (a)

> Because bandwidth costs money. At least here in Germany this is the case.

Hmm... the proper place to fix that is in the network infrastructure.

However, if bandwidth is a real concern there's a lot more you can do
about it than doubling up some smtp deliveries. Is delving into these
kinds of issues worth your time?

> > > Another problem with header rewriting is that there is not a single
> > > interface for doing it.
> > Virtual hosts?
>
> No. Virtual hosts only work when you either (a) finally deliver
> locally or (b) change the recipient address. It does not work in the
> following case: Mail that goes to any host in domain X should have its
> headers rewritten and sent to the original, unchanged recipient via
> SMTP.

That's ok: deliver it locally to an address which re-writes the
headers then sends the message on its way.

If that's too complex for you, and you have a specific example of what
you want done, I (or someone else here) can show you how to deal with
your example.

--
Raul
To: Olaf Titz <olaf@bigred.inka.de>
cc: djb-qmail@koobera.math.uic.edu
Subject: Re: Features (Re: Rejecting large messages)
In-reply-to: Your message of "Wed, 05 Mar 1997 11:13:17 +0100."
<19970305101322.17352.qmail@bigred.inka.de>
--------
Olaf Titz:
> > > an external tool. But it is just _suboptimal_ and aesthetically
> > > unpleasing to require for this often wanted task that each and every
> > > message pass the MTA twice.

Raul Miller:
> > Why suboptimal? Near as I can tell, the general problem of header
> > rewriting is turing equivalent.

Olaf Titz:
> smail can do it in one pass too, with a remove_header and an
> append_header statement, the latter of which contains a lookup
> expansion (non-smailers can probably guess what this means - hint:
> "lookup" is on dbm files).

Lookup of what? An smtp address? A host name? A domain?

> Of course the problem is Turing equivalent and of course even
> sendmail.cf is not the right approach for, say, a Fidonet gateway. :-)
> But there are a lot of less exotic cases that could be handled very
> easily.

Sure, but all cases can be handled fairly easily already. The only
significant issue I'm aware of, where improvements could be made, is
logging. [.It's a bit harder to analyze logs when you get two
distinct sets of log entries per message.]

> > You've assumed that bandwidth is the figure of merit. Why? (a)

> Because bandwidth costs money. At least here in Germany this is the case.

Hmm... the proper place to fix that is in the network infrastructure.

However, if bandwidth is a real concern there's a lot more you can do
about it than doubling up some smtp deliveries. Is delving into these
kinds of issues worth your time?

> > > Another problem with header rewriting is that there is not a single
> > > interface for doing it.
> > Virtual hosts?
>
> No. Virtual hosts only work when you either (a) finally deliver
> locally or (b) change the recipient address. It does not work in the
> following case: Mail that goes to any host in domain X should have its
> headers rewritten and sent to the original, unchanged recipient via
> SMTP.

That's ok: deliver it locally to an address which re-writes the
headers then sends the message on its way.

If that's too complex for you, and you have a specific example of what
you want done, I (or someone else here) can show you how to deal with
your example.

--
Raul

> > smail can do it in one pass too, with a remove_header and an
> > append_header statement, the latter of which contains a lookup
> > expansion (non-smailers can probably guess what this means - hint:
> > "lookup" is on dbm files).
> Lookup of what? An smtp address? A host name? A domain?

Anything. E.g. user names and addresses:
remove_header=from, append_header="From:
${lookup:sender:dbm{/etc/canonnames}} :$value ($sender_name)"

and the database could look like
olaf Olaf.Titz@bigred.inka.de
etc.

Or it could look up domain names, etc. This is rather flexible, it
works for arbitrary strings and each variable that smail knows about.

> > Because bandwidth costs money. At least here in Germany this is the case.
> Hmm... the proper place to fix that is in the network infrastructure.
> However, if bandwidth is a real concern there's a lot more you can do
> about it than doubling up some smtp deliveries. Is delving into these
> kinds of issues worth your time?

Efficiency is always worth the time. I don't know how much bandwidth
is wasted, globally, by broken DNS servers. Fixing such errors takes
me on the order of half an hour, usually. I have the necessary
knowledge because I need to administer DNS anyway, so the cost is one
half staff hour.

(While speaking of DNS, sendmail which does much too many lookups does
waste resources on an enormous scale too, I think we agree on this
point. Also, programs that always reverse-lookup every address they
encounter for logging also waste bandwidth; if a lookup is needed it
can be done by the log analyzing program. Etc. etc.)

Another example: People put _weeks_ of work into making Web caches run
as perfectly as possible. "Only" to save on bandwidth.

I strongly disagree with "Optimization: Don't do it". This is the
mind-set that has led to Windows 95 sucking up the bigger part of the
processing power of a P100 and needing 24MB for tasks that run
perfectly on my pathetic 386 with 8MB, with a news server in the
background.

> > following case: Mail that goes to any host in domain X should have its
> > headers rewritten and sent to the original, unchanged recipient via
> > SMTP.
> That's ok: deliver it locally to an address which re-writes the
> headers then sends the message on its way.

If this way is the same address as before it will go into the same
virtual domain, i.e. a loop.

> If that's too complex for you, and you have a specific example of what
> you want done, I (or someone else here) can show you how to deal with
> your example.

Real life example: my machine is known as bigred.inka.de and
g212.hadiko.de. The latter is an in-house network. (Yes, it does have
an external MX, but the same situation can arise in in-house networks
with no connection to the outside at all. I have set up such a thing
for a customer too.)
Now I want to have a From address of ...@g212.hadiko.de for mail to
any hosts in the in-house network (i.e. in .hadiko.de) and
...@bigred.inka.de for anything else. My current solution works with
RELAYCLIENT=@fixup, delivery to a rewriting script via a virtual
domain "fixup" (not documented that an address really can have two
"@"s in this process, btw.) This only works because I use MH and MH
sends its mail via SMTP. It wouldn't work for qmail-inject.

Setting any environment variables prior to calling the mail program is
not an option because the sender address to use depends on the
recipient address which is set, of course, by the mail program.

Yes, it can be done with a replacement qmail-inject. It can also be
done with tossing away qmail entirely in favor of smail. ;-)
But I'd really like a solution that is as nice and clean as qmail
itself, not a gross kluge like the one I use now. I also think the
"from broken clients" item from the FAQ falls into this category; it
could be done much more elegantly.

The only thing that is missing is an interface for qmail to plug in an
external filter, nothing more. Is that really too much already?

olaf

On 5 Mar 1997, Russell Nelson wrote:

> > Because bandwidth costs money. At least here in Germany this is the case.
And you guys haven't seen the price-list of Czech Telecom :)

> But the price of bandwidth is dropping precipitiously, and the cost of
> people time is growing, as it has been growing for the last five
> hundred years. Not reasonable to expect either trend to change over
> the lifetime of qmail. Every program involves compromises, and extra
> features imposes complexity; complexity that increases the chance of
> errors. An MTA is inherently complex. It's reasonable to make
> choices to reduce the complexity.

It sounds like "This program is slow because good programmers are too
expensive, just buy better hardware." argument. But consider this: if you
spend $1000 to save $1 every time you use your program, it pays off as
soon as you use it for 1001st time. And MTA is FREQUENTLY used piece of
software, isn't it?

--Pavel Kankovsky aka Peak (troja.mff.cuni.cz network administration)

Howdy jhawk - small world!

>In general this phenonmenon is blamed on the WWW, and on http clients'
>insistence on opening multiple TCP connections, which degrade the
>network and where a lost SYN on one connection does not cause a
>backoff in the openning of other connections. This is bad for the
>network -- it's one of the reasons that HTTP is moving toward
>persistent and multiplexable connections.
>
>qmail has the same problems. It may appear to you that you do not have
>a bandwidth shortage at your "bottleneck", but openning multiple
>connections like this is very bad for the core of the network.

You can throttle down qmail's parallel deliveries if I am not mistaken.
See FAQ 8.1.

Besides - qmail is not the only MTA which is guilty of this: mmdfIIb,
commerical version of listserv product and I believe, sendmail.

>It would be ironic to think that just as we're fixing the Web, mail
>transport becomes the next victim.

We already have this problem.

>Scary to think of mail transport learning lessons from the Web, isn't
>it?

YES!!

Bill

>But take another look at header rewriting. Yes, it is possible with an
>external tool. But it is just _suboptimal_ and aesthetically
>unpleasing to require for this often wanted task that each and every
>message pass the MTA twice.

qmail wasn't designed to do header rewriting. It's flexible enough to
allow it, but it just wasn't high on Dan's list of priorities. qmail
isn't a one-size-fits-all MTA. If header rewriting (or batching or
connection caching or whatever) is the most important feature of an
MTA to you, then you probably won't be happy with qmail. Feel free to
hack on qmail or switch to another MTA, but please don't continue to
raise these issues in this list. Dan is well aware of the desire for
such capabilities, and he will add those he feels are consistent with
the qmail design and worth the effort. No amount of whining for
feature X will cause Dan to add it, so save your breath (and our
ears).

>... Russell insists
>on measurements of actual bandwidth taken, but it's trivial to _prove_
>that sending a message twice out my network connection requires double
>the bandwidth than sending it once, not measurements needed.)

Actually it's Dan, not Russ who demands measurements. Nobody denies
that batching can save bandwidth. The point is that qmail is not
optimized for bandwidth, it's optimized for speed and
functionality. Sacrificing speed or functionality to reduce bandwidth
is simply not consistent with qmail's design.

-Dave

Pavel Kankovsky writes:
> On 5 Mar 1997, Russell Nelson wrote:
>
> > > Because bandwidth costs money. At least here in Germany this is the case.
> And you guys haven't seen the price-list of Czech Telecom :)

So? Host your mailing list in a country with cheap bandwidth. As Dan
keeps saying, there's no point in giving you measures of half-economy.
If you are really intent upon saving money, you will pursue measures
of full economy (which, BTW, do not require changes to qmail).

> It sounds like "This program is slow because good programmers are too
> expensive, just buy better hardware." argument. But consider this: if you
> spend $1000 to save $1 every time you use your program, it pays off as
> soon as you use it for 1001st time. And MTA is FREQUENTLY used piece of
> software, isn't it?

The only time single-rcpt matters is on a mailing list. Individual
messages only ever go to a handful of addresses. VERP requires
single-rcpt. It's VERP that saves the administrator's time, and it
saves that time with every bounce message that is handled
automatically. Single-rcpt costs you something for each message (the
cost of which is going down). Doing without it would also cost you
something for each bounce (the cost of which is going up).

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

On Wed, 5 Mar 1997, Bill Nugent wrote:

> >qmail has the same problems. It may appear to you that you do not have
> >a bandwidth shortage at your "bottleneck", but openning multiple
> >connections like this is very bad for the core of the network.
>
> You can throttle down qmail's parallel deliveries if I am not mistaken.
> See FAQ 8.1.
>
> Besides - qmail is not the only MTA which is guilty of this: mmdfIIb,
> commerical version of listserv product and I believe, sendmail.

Older (vendor, v5) versions of sendmail open parallel connections.
Sendmail v8 uses connection caching (with a configurable number of
simultaneous outgoing open connections). This is extremely useful in
certain special situations (e.g., one mail gateway talking to another
mail gateway) in which a significant number of messages are delivered
to the same host in a short period of time.

Connection caching is doubly desirable if the receiving system is
underpowered, or is running a MTA less robust/efficient than qmail....

------------ Greg Wooledge -------------
------- <wooledge@kellnet.com> -------
--- <http://kellnet.com/wooledge/main.html> ---

>>>>> "Greg" == Greg Wooledge <wooledge@kellnet.com> writes:

Greg> Older (vendor, v5) versions of sendmail open parallel connections.
Greg> Sendmail v8 uses connection caching (with a configurable number of
Greg> simultaneous outgoing open connections). This is extremely useful in
Greg> certain special situations (e.g., one mail gateway talking to another
Greg> mail gateway) in which a significant number of messages are delivered
Greg> to the same host in a short period of time.

This is definitely true in my special situation... I am using qmail to
replace a smap/sendmail configuration (which is does very well) and
thus, *all* remote deliveries are to the same machine.

Greg> Connection caching is doubly desirable if the receiving system is
Greg> underpowered, or is running a MTA less robust/efficient than qmail....

Again, this pretty much describes my case.

--
David Blacka Software Engineer Directory Services/
davidb@rwhois.net Network Solutions, Inc. RWhois Development

> MTA to you, then you probably won't be happy with qmail. Feel free to
> hack on qmail or switch to another MTA, but please don't continue to
> raise these issues in this list. Dan is well aware of the desire for
> such capabilities, and he will add those he feels are consistent with
> the qmail design and worth the effort. No amount of whining for
> feature X will cause Dan to add it, so save your breath (and our
> ears).

This is exactly the kind of attitude that pisses people off and gives
the product a bad name. qmail does deserve to be used by a wider range
of people, but it won't if all inqiries are handled this way.

> Actually it's Dan, not Russ who demands measurements. Nobody denies
> that batching can save bandwidth. The point is that qmail is not
> optimized for bandwidth, it's optimized for speed and
> functionality. Sacrificing speed or functionality to reduce bandwidth
> is simply not consistent with qmail's design.

I still don't think that the _additional functionality_ of a message
filter as I have described is unresonable, and adding it would affect
neither speed nor present functionality.

I know well that connection batching is a completely different issue,
as it doesn't fit anywhere in qmail's one-X-per-message design
(X=process, file, connection, etc.) I just disagree that it's a
non-issue.

olaf

> In general this phenonmenon is blamed on the WWW, and on http clients'
> insistence on opening multiple TCP connections, which degrade the
> network and where a lost SYN on one connection does not cause a
> backoff in the openning of other connections. This is bad for the
>...
> qmail has the same problems. It may appear to you that you do not have

Indeed, some people already have called qmail an "attack tool" for
this feature... (yes, _I_ do set concurrencyremote)

olaf

> So? Host your mailing list in a country with cheap bandwidth. As Dan

Of course, this is what people are doing with their Web servers.
A large number of German companies have Web servers in the US, simply
because they are not cheaper but _much_ cheaper. (Prices for 2Mbps
leased lines differ by a _factor of 10_.)

And this is a never ending discussion topic over here, because of the
following effect: the clients who want to view German Web pages are
mostly located _here_ (no surprise), so this traffic goes mainly over
the Atlantic, the demand for trans-Atlantic bandwidth grows and the
price for it remains high.

The only one who profits from this distorted competition is the
Deutsche Telekom, i.e. the ex-state company who _created_ the
distortion in the first place.

> keeps saying, there's no point in giving you measures of half-economy.
> If you are really intent upon saving money, you will pursue measures
> of full economy (which, BTW, do not require changes to qmail).

Depends on what is full economy. As pointed out above, saving in one
place can cause more cost globally. The only remedy is _design_ for
efficiency.

> The only time single-rcpt matters is on a mailing list. Individual
> messages only ever go to a handful of addresses. VERP requires
> single-rcpt. It's VERP that saves the administrator's time, and it
> saves that time with every bounce message that is handled
> automatically. Single-rcpt costs you something for each message (the

You're right wrt. VERP. This complicates the issue a bit. I'm not
convinced that VERP is _necessary_ for bounce handling, though.

olaf

>> ... Feel free to
>> hack on qmail or switch to another MTA, but please don't continue to
>> raise these issues in this list. Dan is well aware of the desire for
>> such capabilities, and he will add those he feels are consistent with
>> the qmail design and worth the effort. No amount of whining for
>> feature X will cause Dan to add it, so save your breath (and our
>> ears).
>
>This is exactly the kind of attitude that pisses people off and gives
>the product a bad name. qmail does deserve to be used by a wider range
>of people, but it won't if all inqiries are handled this way.

OK, help me understand. What is it exactly that pisses people off?
That Dan doesn't fall all over himself trying to add every requested
``feature'' to qmail? That qmail development isn't democratic? That
qmail isn't the best MTA for every situation?

>I still don't think that the _additional functionality_ of a message
>filter as I have described is unresonable, and adding it would affect
>neither speed nor present functionality.

I don't remember seeing a good description of such a filter. Where
would it go? How would it be configured?

>I know well that connection batching is a completely different issue,
>as it doesn't fit anywhere in qmail's one-X-per-message design
>(X=process, file, connection, etc.) I just disagree that it's a
>non-issue.

Fine, it's an issue. What do you think we should do about it:

1) Bring it up every month or so until Dan gets tired of hearing
about it and adds it to qmail.
2) Realize that it's inconsistent with qmail's design and philosophy
and shouldn't be added to qmail.
3) Realize that Dan's not going to do it, so do it yourself and
distribute the patches.
4) Something else?

The current approach seems to be (1), which I think has essentially no
chance of succeeding and is tiresome to long-time members of the list.

-Dave

Olaf Titz writes:
>
> You're right wrt. VERP. This complicates the issue a bit. I'm not
> convinced that VERP is _necessary_ for bounce handling, though.

Given that a number of gateways ignore RFC821, it'll be decades before
everyone implements RFC1894 (DSN). In the meantime, VERP works, and
reduces the very real cost of mailing list management. Now, if we
could only get people to unsubscribe from the same address they
subscribed to the list, we could *eliminate* the administrative cost
of mailing list management.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

On 6 Mar 1997, Russell Nelson wrote:

> Given that a number of gateways ignore RFC821, it'll be decades before
> everyone implements RFC1894 (DSN). In the meantime, VERP works, and
> reduces the very real cost of mailing list management. Now, if we
> could only get people to unsubscribe from the same address they
> subscribed to the list, we could *eliminate* the administrative cost
> of mailing list management.

I just want to put in my thumbs-up on VERP. I run a 40K user mailing
list, and in the first day of running on qmail with VERP I was able to
remove 2000 dead addresses. With all the bizarre error formats out
there, I always sent errors to /dev/null; with VERP, I can easily take
care of them.

Evan
--
Evan Champion * Director, Network Operations
mailto:evanc@synapse.net * Directeur, Exploitation du reseau
http://www.synapse.net/ * Synapse Internet

Evan Champion writes:
> I just want to put in my thumbs-up on VERP. I run a 40K user mailing
> list, and in the first day of running on qmail with VERP I was able to
> remove 2000 dead addresses.

Hmmm.... Interesting. I wonder if VERP, in conjunction with a bounce
manager (never pass up a chance to mention your own software, even if
it available from http://www.qmail.org for free :), actually *reduces*
the amount of bandwidth because it greatly lowers the cost of
eliminating bad addresses promptly.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)