Mailing List Archive

Evan Champion writes:
> On 23 Feb 1997, Russell Nelson wrote:
>
> > > - RPM and deb packages becoming a standard official distribution format,
> > > right next to the .tar.gz file, to help Qmail spred faster
> > > - stick to FSSTD and kick the binaries/manpages out of /var in the default
> > > installation
> >
> > Kind of Linux-specific, aren't these?
>
> Well, not the kicking binaries/manpages out of /var one. In fact,
> putting binaries/manpages in /var is universally considered very bad
> taste.

Well, as it currently stands, the binaries *are* variable from system
to system. Compiled-in UIDs, remember? Not to defend them, but they
*are* variable.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

i used to think the sendmail forwarding mechanism was going to be a problem
so we counted the number of .forward files to get a feel for the magnitude.
we got a small number (about 7%) so i wasn't so worried but still a little.
then i thought about the tao of qmail and changed ALIAS_EMPTY to do the
right thing given one (or more) of .qmail, .forward, .procmailrc or none of
the above in a /var/mail/$USER environment.

now i must admit that we haven't converted our mailhub to qmail just yet so
i have only used this in small-scale environments but i expect to go
production at the end of the semester.

Ira> - find a nice solution for people used to using .forward

--
paul
pjg@acsu.Buffalo.EDU |public keys at:
| http://urth.acsu.Buffalo.EDU/~pjg/key.html
if the above contains opinions they are mine unless marked otherwise.

Yaroslav Faybishenko writes:
> > > - find a solution to the annoying explosion method (group of mails to the
> > > same host are sent in paralel by separate processes/TCP connections
> > > instead of more efficiant delivery)
> >
> > But it *is* more efficient in terms of the only cost that is
> > increasing -- the time of the people using the computers. Every other
> > cost of email delivery is dropping through the floor -- even in
> > PTT-dominated and third-world countries. CPU, disk, and bandwidth are
> > all getting cheaper. There is no reason for an MTA, being written in
> > the late 90's, to attempt to conserve these resources at the cost of
> > human time.
> >
> > There may be rare circumstances in which bandwidth is more expensive
> > than human time. In that case, it makes sense to batch, compress, and
> > transport the email to a relay point where the bandwidth is cheaper.
> > If bandwidth is truly a concern, then address that concern in an
> > effective manner, rather than taking half measures like aggregating
> > mail to like hosts.
>
> The original poster's argument is that the cost of opening lots
> of TCP connections is quite high. Your argument is that cost of bandwidth
> is getting cheaper, and is not a conern. Comparing cost of _opening_
> lots of TCP connections to cost of bandwidth seems to me like comparing
> apples to oranges.

There are two costs of opening TCP connections -- the network cost and
the CPU/memory cost. The network costs are the SYN/ACK and FIN/ACK
pairs (which use up some bandwidth) and the delay introduced by
slow-start. The extra packets are overwhelmed by the size of the
email message. The CPU and memory costs are completely negligable.
If they aren't, wait a month and they will be.

> For a mailing list like mine, an MTA opening one tcp socket per unique
> MX host would definitely be great.

Why? What is limiting your deliveries? If you haven't measured, it's
too early to optimize.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

On 23 Feb 1997, Russell Nelson wrote:

> > > > - find a solution to the annoying explosion method (group of mails to the
> > > > same host are sent in paralel by separate processes/TCP connections
> > > > instead of more efficiant delivery)
> >
> > The original poster's argument is that the cost of opening lots
> > of TCP connections is quite high. Your argument is that cost of bandwidth
>
> There are two costs of opening TCP connections -- the network cost and
> the CPU/memory cost. The network costs are the SYN/ACK and FIN/ACK
> pairs (which use up some bandwidth) and the delay introduced by

In Israel, our international lines cost overwhelmingly, my tiny 64k costs
like a USA installation of a T1, so my bandwidth problems HURT. my biggest
problem is TCP connections dying because of line overflow. when a 300-400
people mailing list is distributed, and there are 12 messages, or 30 going
to the same host, it will open one connection, shake hands with the target
sendmail once, on the TCP and the application level, and things pass
through faster. if I open 30 connections to it, some will die, wait their
15 minutes and try again, just because they can't even get ACK packets!
that chokes my line and it definitly a useless burdon on my machine,
although it does have enough memory and CPU power.


-------------------------------------------------------------
Ira Abramov <ira@scso.com> Scalable Solutions
SITE Web Presence ("webspace for rent") http://www.site.co.il
Beeper 91826 at 058-212121 FAX (972)2-643-0471
POBox 3600, Jerusalem 91035, Israel Tel (972)2-642-6822
http://www.scso.com/~ira Check out: http://www.linux.org.il

On 23 Feb 1997, Russell Nelson wrote:

: > For a mailing list like mine, an MTA opening one tcp socket per unique
: > MX host would definitely be great.
:
: Why? What is limiting your deliveries? If you haven't measured, it's
: too early to optimize.

I agree that if a list has 40 recipients in the same domain it is
better (time & BW) doing

mail from:<list-owner@foo.com>
rcpt to:<fulano1@the.same.domain>
rcpt to:<fulano2@the.same.domain>
...
rcpt to:<fulano40@the.same.domain>

than send 40 times the same message (one per recipient) to the same
host; and that is too frequent in mailing lists. If you doubt, ask
Debian.

Regards,
--
Roberto Lumbreras Pastor mailto:rover@lander.es
Lander Internet - Spain http://www.lander.es/
Tel +34 1 556.28.83 Fax +34 1 556.30.01

Roberto A. Lumbreras Pastor writes:
> On 23 Feb 1997, Russell Nelson wrote:
>
> : > For a mailing list like mine, an MTA opening one tcp socket per unique
> : > MX host would definitely be great.
> :
> : Why? What is limiting your deliveries? If you haven't measured, it's
> : too early to optimize.
>
> I agree that if a list has 40 recipients in the same domain it is
> better (time & BW) doing

I see no evidence that you have *measured* the amount of user time.
The record is FULL of "obvious" optimizations that do not improve.

Measure, measure, measure. Then optimize. Don't guess or assume.

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

Ira Abramov writes:
> On 23 Feb 1997, Russell Nelson wrote:
>
> > > > > - find a solution to the annoying explosion method (group of mails to the
> > > > > same host are sent in paralel by separate processes/TCP connections
> > > > > instead of more efficiant delivery)
> > >
> > > The original poster's argument is that the cost of opening lots
> > > of TCP connections is quite high. Your argument is that cost of bandwidth
> >
> > There are two costs of opening TCP connections -- the network cost and
> > the CPU/memory cost. The network costs are the SYN/ACK and FIN/ACK
> > pairs (which use up some bandwidth) and the delay introduced by
>
> In Israel, our international lines cost overwhelmingly, my tiny 64k costs
> like a USA installation of a T1, so my bandwidth problems HURT. my biggest
> problem is TCP connections dying because of line overflow. when a 300-400
> people mailing list is distributed, and there are 12 messages, or 30 going
> to the same host, it will open one connection, shake hands with the target
> sendmail once, on the TCP and the application level, and things pass
> through faster. if I open 30 connections to it, some will die, wait their
> 15 minutes and try again, just because they can't even get ACK packets!
> that chokes my line and it definitly a useless burdon on my machine,
> although it does have enough memory and CPU power.

Sounds to me, then, like you ought to find a site that will let you
use them as a relayhost. Since you can find colocation offers for a
couple of hundred dollars a month, someone would likely offer relaying
service for the same price. Maybe even someone on this list?

And, since you've only got a 64Kbps line, you should consider batching
and compressing your mail. As I say, why pursue half measures like
sendmail's aggregated deliveries to the same host, when you're paying
for a relayhost?

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

Greg Wooledge writes:
> On 23 Feb 1997, Russell Nelson wrote:
>
> > > - find a solution to the annoying explosion method (group of mails to the
> > > same host are sent in paralel by separate processes/TCP connections
> > > instead of more efficiant delivery)
>
> > cost of email delivery is dropping through the floor -- even in
> > PTT-dominated and third-world countries. CPU, disk, and bandwidth are
> > all getting cheaper.
>
> Cheaper, yes; finite, yes.

If you permit web surfing from your site, then the bandwidth available
for email is essentially infinite.

> I can think of one very specific case where connection caching is
> nearly mandatory. At a place I used to work, there were several mail
> "servers" which were single-threaded MS-Windows SMTP-MSMail "gateways".
> Only one connection at a time could be made; the remainder were rejected.
> Now obviously this is not qmail's problem, but nevertheless the behavior
> of a non-connection-caching MTA was predictably bad -- during a queue
> run,

Not clear that qmail would perform badly in the same circumstances.
Every message has its own retry time. If 19 out of 20 connection
attempts are rejected, each message will be rescheduled for a small,
but random time in the future. The chance that they will hit again is
small enough.

> qmail doesn't handle DECNet and UUCP mail address formats, but

It can probably be convinced to handle them, just as sendmail can be
convinced to handle them. qmail's advantage is that it works with
ordinary Unix tools, rather than forcing you to learn J. Crufty
extension language.

> >There is no reason for an MTA, being written in
> > the late 90's, to attempt to conserve these resources at the cost of
> > human time.
>
> I think there's always a place for efficiency.

The same argument has been made for coding everything in assembly
language. The question is "what efficiency?" What cost are you
trying to minimize?

--
-russ <nelson@crynwr.com> http://www.crynwr.com/~nelson
Crynwr Software sells network driver support | PGP ok
521 Pleasant Valley Rd. | +1 315 268 1925 voice | Peace, Justice, Freedom:
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | pick two (only mostly true)

On 23 Feb 1997, Russell Nelson wrote:

> Ira Abramov writes:
> > On 23 Feb 1997, Russell Nelson wrote:
> >
> > > > > > - find a solution to the annoying explosion method (group of mails to the
> > > > > > same host are sent in paralel by separate processes/TCP connections
> > > > > > instead of more efficiant delivery)
> > > >
> > > > The original poster's argument is that the cost of opening lots
> > > > of TCP connections is quite high. Your argument is that cost of bandwidth
> > >
> > > There are two costs of opening TCP connections -- the network cost and
> > > the CPU/memory cost. The network costs are the SYN/ACK and FIN/ACK
> > > pairs (which use up some bandwidth) and the delay introduced by
> >
> > In Israel, our international lines cost overwhelmingly, my tiny 64k costs
> > like a USA installation of a T1, so my bandwidth problems HURT. my biggest
> > problem is TCP connections dying because of line overflow. when a 300-400
> > people mailing list is distributed, and there are 12 messages, or 30 going
> > to the same host, it will open one connection, shake hands with the target
> > sendmail once, on the TCP and the application level, and things pass
> > through faster. if I open 30 connections to it, some will die, wait their
> > 15 minutes and try again, just because they can't even get ACK packets!
> > that chokes my line and it definitly a useless burdon on my machine,
> > although it does have enough memory and CPU power.
>
> Sounds to me, then, like you ought to find a site that will let you
> use them as a relayhost. Since you can find colocation offers for a
> couple of hundred dollars a month, someone would likely offer relaying
> service for the same price. Maybe even someone on this list?
>
> And, since you've only got a 64Kbps line, you should consider batching
> and compressing your mail. As I say, why pursue half measures like
> sendmail's aggregated deliveries to the same host, when you're paying
> for a relayhost?
>
Folks, IMHO this is not only a question of money. It's also the question,
how we use our resources. Why to flood the lines with duplicate messages,
only because we can effort it ?

------------------------------------------------------------------------------
Konstantin Eftaxias Tel.: ++49 711 701 756
Eftaxias EDV Beratung Fax.: ++49 711 701 339
Bernhauser Hauptstr. 29 eMail kef@eeb.de
70794 Filderstadt eMail kef@rigel.de
------------------------------------------------------------------------------

> email message. The CPU and memory costs are completely negligable.
> If they aren't, wait a month and they will be.
Uhh? You want to say my machine will get a faster CPU and more memory
spontaneously?! :)

Seriously now: I myself would add two more features to 1.1 wishlist:

1) the ability to limit the size of ANY message passing qmail system. Both
local msgs and msgs received via SMTP. (I've seen too many twinks sending
multimegabyte files via e-mail.)

2) the ability to suspend delivery of certain queued message(s) and
to resume it later or purge the message permanently (With sendmail, I can
do that by renaming files in ../spool/mqueue--nasty trick, indeed, with
qmail, I can do (almost) the same but I have to kill and restart
qmail-send, and this is rather inconvenient)

--Pavel Kankovsky aka Peak (troja.mff.cuni.cz network administration)