Mailing List Archive: Rewrite of forward.c

Here's forward.c, much simpler and very straightforward:

while( more targets to go )
{
parse to see what command to run
open a pipe to the command
send the header to the pipe
send remaining lines from the database to the pipe
close the pipe

next target
}

It might look like I only picked a different bottleneck than before (here
it's the database, before it was the number of open pipes) but actually
this algorithm is just the database version as before, but with the pipe
openings happening once for each loop rather than all at once and stored
to a list, then going through the list and sending on all the pipes. My
goal was the eliminate the "start everything all at once" resource hog.

The only consumer of forward.c was pipe.c, so there was no trouble paring
it to the bone to match up with the pared down pipe.c. Note that the
function definition has changed. The new pipe.c I posted does not reflect
this change yet, but it's all alpha 1 anyways... I haven't tested
anything, although it looks correct and compiles cleanly ;-)

Aaron

I was explaining this loop to my brother who remarked that reading from
the database every time is a good idea if you'd otherwise need to malloc()
a huge chunk of memory for the message... but if it's just a few hundred
K, then this is a huge waste of time and database.

To address this, I'm going to add a few lines of logic that will allocate
a bit of memory to hold the full message if it's below a certain cut off.
What might be a fair value for this? 1MB? 5MB? Configurable?

Aaron

On Sun, 27 Apr 2003, Aaron Stone wrote:

> Here's forward.c, much simpler and very straightforward:
>
> while( more targets to go )
> {
> parse to see what command to run
> open a pipe to the command
> send the header to the pipe
> send remaining lines from the database to the pipe
> close the pipe
>
> next target
> }
>
> It might look like I only picked a different bottleneck than before (here
> it's the database, before it was the number of open pipes) but actually
> this algorithm is just the database version as before, but with the pipe
> openings happening once for each loop rather than all at once and stored
> to a list, then going through the list and sending on all the pipes. My
> goal was the eliminate the "start everything all at once" resource hog.
>
> The only consumer of forward.c was pipe.c, so there was no trouble paring
> it to the bone to match up with the pared down pipe.c. Note that the
> function definition has changed. The new pipe.c I posted does not reflect
> this change yet, but it's all alpha 1 anyways... I haven't tested
> anything, although it looks correct and compiles cleanly ;-)
>
> Aaron
>

Mailing List Archive

Mailing List Archive

Attached Files: