Mailing List Archive: event engine library selection

event engine library selection

Mar 14, 2006, 3:03 AM

Post #1 of 10 (2057 views)

DES and I talked a bit about which event library to select for
varnish long time ago and and I belive we have ISC's and Niels
Provos' in the pool.

As far as I can tell from inspection, the ISC code is "industry
strength" where the Provos code is more "engineering sample".

I'm quite worried that the multi-thread support in Provos library
consists of creating events, and then as quickly as possible move
them from the most recent engine to the one where they belong, that
doesn't sound very safe to me.

On the other hand, the Provos engine sounds like it is much more
performance focused than ISC's engine, and that must count heavily
for it as the acceptor/header-collector point is going to see
significant load.

If we decide on the Provos engine, we may have to spend time to fix up his
multi-thread support and hope he will accept the patches.

Comments ?

Poul-Henning

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

event engine library selection [ In reply to ]

Mar 14, 2006, 3:03 AM

Post #2 of 10 (2033 views)

DES and I talked a bit about which event library to select for
varnish long time ago and and I belive we have ISC's and Niels
Provos' in the pool.

As far as I can tell from inspection, the ISC code is "industry
strength" where the Provos code is more "engineering sample".

I'm quite worried that the multi-thread support in Provos library
consists of creating events, and then as quickly as possible move
them from the most recent engine to the one where they belong, that
doesn't sound very safe to me.

On the other hand, the Provos engine sounds like it is much more
performance focused than ISC's engine, and that must count heavily
for it as the acceptor/header-collector point is going to see
significant load.

If we decide on the Provos engine, we may have to spend time to fix up his
multi-thread support and hope he will accept the patches.

Comments ?

Poul-Henning

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

event engine library selection [ In reply to ]

Mar 14, 2006, 4:16 AM

Post #3 of 10 (2041 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> DES and I talked a bit about which event library to select for
> varnish long time ago and and I belive we have ISC's and Niels
> Provos' in the pool.
>
> As far as I can tell from inspection, the ISC code is "industry
> strength" where the Provos code is more "engineering sample".
>
> I'm quite worried that the multi-thread support in Provos library
> consists of creating events, and then as quickly as possible move
> them from the most recent engine to the one where they belong, that
> doesn't sound very safe to me.
>
> On the other hand, the Provos engine sounds like it is much more
> performance focused than ISC's engine, and that must count heavily
> for it as the acceptor/header-collector point is going to see
> significant load.

It also has several features the ISC code lacks, like support for
signals, kqueue, poll etc.

> If we decide on the Provos engine, we may have to spend time to fix up his
> multi-thread support and hope he will accept the patches.

He's a reasonable guy, I don't see any reason why he wouldn't. In any
case, we'll want to keep our own copy of the code in our tree.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

event engine library selection [ In reply to ]

Mar 14, 2006, 4:16 AM

Post #4 of 10 (2034 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> DES and I talked a bit about which event library to select for
> varnish long time ago and and I belive we have ISC's and Niels
> Provos' in the pool.
>
> As far as I can tell from inspection, the ISC code is "industry
> strength" where the Provos code is more "engineering sample".
>
> I'm quite worried that the multi-thread support in Provos library
> consists of creating events, and then as quickly as possible move
> them from the most recent engine to the one where they belong, that
> doesn't sound very safe to me.
>
> On the other hand, the Provos engine sounds like it is much more
> performance focused than ISC's engine, and that must count heavily
> for it as the acceptor/header-collector point is going to see
> significant load.

It also has several features the ISC code lacks, like support for
signals, kqueue, poll etc.

> If we decide on the Provos engine, we may have to spend time to fix up his
> multi-thread support and hope he will accept the patches.

He's a reasonable guy, I don't see any reason why he wouldn't. In any
case, we'll want to keep our own copy of the code in our tree.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

event engine library selection [ In reply to ]

Mar 14, 2006, 4:33 AM

Post #5 of 10 (2040 views)

In message <ujrr755ff9q.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= writes:

>It also has several features the ISC code lacks, like support for
>signals, kqueue, poll etc.

yeah, well, and vice versa, the ISC code has a lot of high-level stuff
that provos code lacks.

>> If we decide on the Provos engine, we may have to spend time to fix up his
>> multi-thread support and hope he will accept the patches.
>
>He's a reasonable guy, I don't see any reason why he wouldn't. In any
>case, we'll want to keep our own copy of the code in our tree.

Lets go with provos code then.

Integrating it in the tree probably takes a bit more auto* foo than I
have, since I guess we need to merge libevent's auto* stuff into our
auto* stuff ?

When I compile libevent on -current I run into weird messages because
<sys/event.h> includes <sys/_lock.h> to cope with various userland
malware. I'm investigating right now what malware that might be.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

event engine library selection [ In reply to ]

Mar 14, 2006, 4:33 AM

Post #6 of 10 (2032 views)

In message <ujrr755ff9q.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= writes:

>It also has several features the ISC code lacks, like support for
>signals, kqueue, poll etc.

yeah, well, and vice versa, the ISC code has a lot of high-level stuff
that provos code lacks.

>> If we decide on the Provos engine, we may have to spend time to fix up his
>> multi-thread support and hope he will accept the patches.
>
>He's a reasonable guy, I don't see any reason why he wouldn't. In any
>case, we'll want to keep our own copy of the code in our tree.

Lets go with provos code then.

Integrating it in the tree probably takes a bit more auto* foo than I
have, since I guess we need to merge libevent's auto* stuff into our
auto* stuff ?

When I compile libevent on -current I run into weird messages because
<sys/event.h> includes <sys/_lock.h> to cope with various userland
malware. I'm investigating right now what malware that might be.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

event engine library selection [ In reply to ]

Mar 14, 2006, 5:07 AM

Post #7 of 10 (2031 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> Lets go with provos code then.
>
> Integrating it in the tree probably takes a bit more auto* foo than I
> have, since I guess we need to merge libevent's auto* stuff into our
> auto* stuff ?

I'll take care of it in a minute.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

event engine library selection [ In reply to ]

Mar 14, 2006, 5:07 AM

Post #8 of 10 (2038 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> Lets go with provos code then.
>
> Integrating it in the tree probably takes a bit more auto* foo than I
> have, since I guess we need to merge libevent's auto* stuff into our
> auto* stuff ?

I'll take care of it in a minute.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

event engine library selection [ In reply to ]

Mar 14, 2006, 7:34 AM

Post #9 of 10 (2032 views)

At 11:03 14.03.2006 +0100, Poul-Henning Kamp wrote:

>DES and I talked a bit about which event library to select for
>varnish long time ago and and I belive we have ISC's and Niels
>Provos' in the pool.
>
>As far as I can tell from inspection, the ISC code is "industry
>strength" where the Provos code is more "engineering sample".
>
>I'm quite worried that the multi-thread support in Provos library
>consists of creating events, and then as quickly as possible move
>them from the most recent engine to the one where they belong, that
>doesn't sound very safe to me.
>
>On the other hand, the Provos engine sounds like it is much more
>performance focused than ISC's engine, and that must count heavily
>for it as the acceptor/header-collector point is going to see
>significant load.
>
>If we decide on the Provos engine, we may have to spend time to fix up his
>multi-thread support and hope he will accept the patches.
>
>Comments ?

My gut feeling would say Provos (speed), but I really am not qualified to
give a guess, never having read any of the code.
If it's not to much work, we could end up with giving a speedy eventlib a
more "industry strength" approach, and that's the best of 2 worlds.

Anders Berg

>Poul-Henning
>
>--
>Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>phk at FreeBSD.ORG | TCP/IP since RFC 956
>FreeBSD committer | BSD since 4.3-tahoe
>Never attribute to malice what can adequately be explained by incompetence.
>_______________________________________________
>varnish-dev mailing list
>varnish-dev at projects.linpro.no
>http://projects.linpro.no/mailman/listinfo/varnish-dev

*****************************************************************
Denne fotnoten bekrefter at denne e-postmeldingen ble
skannet av MailSweeper og funnet fri for virus.
*****************************************************************
This footnote confirms that this email message has been swept by
MailSweeper for the presence of computer viruses.
*****************************************************************

event engine library selection [ In reply to ]

Mar 14, 2006, 7:34 AM

Post #10 of 10 (2040 views)

At 11:03 14.03.2006 +0100, Poul-Henning Kamp wrote:

>DES and I talked a bit about which event library to select for
>varnish long time ago and and I belive we have ISC's and Niels
>Provos' in the pool.
>
>As far as I can tell from inspection, the ISC code is "industry
>strength" where the Provos code is more "engineering sample".
>
>I'm quite worried that the multi-thread support in Provos library
>consists of creating events, and then as quickly as possible move
>them from the most recent engine to the one where they belong, that
>doesn't sound very safe to me.
>
>On the other hand, the Provos engine sounds like it is much more
>performance focused than ISC's engine, and that must count heavily
>for it as the acceptor/header-collector point is going to see
>significant load.
>
>If we decide on the Provos engine, we may have to spend time to fix up his
>multi-thread support and hope he will accept the patches.
>
>Comments ?

My gut feeling would say Provos (speed), but I really am not qualified to
give a guess, never having read any of the code.
If it's not to much work, we could end up with giving a speedy eventlib a
more "industry strength" approach, and that's the best of 2 worlds.

Anders Berg

>Poul-Henning
>
>--
>Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>phk at FreeBSD.ORG | TCP/IP since RFC 956
>FreeBSD committer | BSD since 4.3-tahoe
>Never attribute to malice what can adequately be explained by incompetence.
>_______________________________________________
>varnish-dev mailing list
>varnish-dev at projects.linpro.no
>http://projects.linpro.no/mailman/listinfo/varnish-dev

*****************************************************************
Denne fotnoten bekrefter at denne e-postmeldingen ble
skannet av MailSweeper og funnet fri for virus.
*****************************************************************
This footnote confirms that this email message has been swept by
MailSweeper for the presence of computer viruses.
*****************************************************************