Mailing List Archive

On Tue, 07 Oct 2008 14:30:48 +0200, Michael Haardt wrote:
> The small patch is experimental and (for now) contained in #ifdefs for
> not disturbing production builds, but allowing to share it with others
> having the same problem. A bunch people certainly require some solution
> by the end of the year, although only one has asked on exim-users already.
>
> Are there any non-political opinions against committing this patch?

The patch doesn't appear to have made it to the list (at least not to the
mail archive or gmane). Can you make that patch available or send it to me
off-list?

Personally, I would think that exim should be able to cope with these kind
of things, but I would prefer to have a generic solution.


Christof

--

http://cmeerw.org sip:cmeerw at cmeerw.org
mailto:cmeerw at cmeerw.org xmpp:cmeerw at cmeerw.org

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

Hi

On Tue, 07 Oct 2008 14:30:48 +0200, Michael Haardt wrote:
> Are there any non-political opinions against committing this patch?

Unfortunately separating the political and technical at this point is
quite difficult, in my opinion, which I'll explain below.

On Fri, 2008-10-10 at 09:36 +0200, Christof Meerwald wrote:
> Personally, I would think that exim should be able to cope with these kind
> of things, but I would prefer to have a generic solution.

Exim can already log far more data than it does by default - the
log_selector config option allows for a bundle of information:

http://www.exim.org/exim-html-current/doc/html/spec_html/ch49.html#SECTlogselector

Given that each member state of the EU can modify the Directive
2006/24/EC to fit their own ends (which in fact the UK govt has done)
it's my belief that providing a fixed format which "fits" is not the job
of the application, but the job of the sysadmin running the application.

In the case of Exim it seems that according to Article 5 of the
directive, the requirements are already fulfilled by the default log
format - this logs:

sending and destination IP
sending envelope email address
all recipients, whether to/cc/bcc or envelope
time
authentication details if applicable
protocols used

I personally do not believe that patching Exim to fulfil the directive
fully is necessary; more pertinently I don't believe it's possible given
the myriad different local interpretations and transpositions applied by
EU member states. On top of that, putting code in like this may actually
breach regulations in other parts of the world (IANAL though).

My reading of the Directive and several interpretation documents leads
me to believe that there is no one format which is necessary, as long as
it's possible to post-process the retained data to locate the
information - which in our case, exigrep is very good at.

I hope others will offer their opinions also, either way. Anyone of a
legal bent might be a useful addition to the thread!

Graeme


--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

On Fri, 10 Oct 2008, Graeme Fowler wrote:

| Given that each member state of the EU can modify the Directive
| 2006/24/EC to fit their own ends (which in fact the UK govt has done)

I'm not totally clear about this aspect. Member states can choose the
period for which data is retained (between 6 months and 2 years).
But is there local scope to change which fields must be retained ?

Article 5 says "Member States shall ensure that the following categories
of data are retained" which I guess means requiring more is an option
but not less.


| it's my belief that providing a fixed format which "fits" is not the job
| of the application, but the job of the sysadmin running the application.

Agreed.

And as you say, it looks like all the info that might be wanted is already
logged by default. Without having seen Michael's patch, I can't comment,
but perhaps he means a log which has LESS info than the default ie. ONLY
that required by law.

No doubt this might also be achieved by suitable post-processing.


--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

* Chris Edwards:

> | Given that each member state of the EU can modify the Directive
> | 2006/24/EC to fit their own ends (which in fact the UK govt has done)
>
> I'm not totally clear about this aspect. Member states can choose the
> period for which data is retained (between 6 months and 2 years).
> But is there local scope to change which fields must be retained ?

Yes. Germany's implementation requires that you log both addresses
(original and new) when you perform address rewriting.

> Article 5 says "Member States shall ensure that the following categories
> of data are retained" which I guess means requiring more is an option
> but not less.

Seems so.

--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

> Given that each member state of the EU can modify the Directive
> 2006/24/EC to fit their own ends (which in fact the UK govt has done)
> it's my belief that providing a fixed format which "fits" is not the job
> of the application, but the job of the sysadmin running the application.

Right now, I am not aware of the national differences, but suggested to
add a log format to select the country, in case there are any. From the
various answers, it sounds like that's indeed required.

> I personally do not believe that patching Exim to fulfil the directive
> fully is necessary; more pertinently I don't believe it's possible given
> the myriad different local interpretations and transpositions applied by
> EU member states. On top of that, putting code in like this may actually
> breach regulations in other parts of the world (IANAL though).

Not a myriad really, just as many as countries in the world that ask
for data retension. ;)

I don't believe the code would violate any regulation, as long as it is
optional to use. After all, Exim can already log lots of things.

> My reading of the Directive and several interpretation documents leads
> me to believe that there is no one format which is necessary, as long as
> it's possible to post-process the retained data to locate the
> information - which in our case, exigrep is very good at.

Several people on the list suggested post-processing the mainlog. To me,
several reasons forbid that:

I am no longer free which log selectors to use. Logging data must
be reasoned by a purpose (if you care about privacy, which I do), and
depending on the purpose and changes of it, I want to log less or more
than required for data retention. Independent logfiles for mainlog and
data retention solve that conflict nicely. Otherwise I had to strip
the mainlog after extracting the data retention information.

The latency requirements of data retention and mainlog may differ, too.
I am not yet aware of a maximum latency allowed by the German law,
but heard telephone switches use 15 minutes, but anything up two 24h
would probably be OK. Should less than 24h be required in the future,
post-processing becomes even more painful and I need to rotate the
mainlog before I really want to.

I don't trust exigrep. Does it handle quoting right? Can I be sure it
does by construction? The mainlog is friendly to humans, and I like that,
but data retention is required by machines and asks for 100% accuracy,
best achieved by simplicity, like setting a logfile option and selecting
the format for Germany.

I did not send a patch to the list yet, because the actual code only adds
the new logfile so far - a trivial patch, and I assumed there would be
opinions against comitting anything in that area.

Michael

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

> Right now, I am not aware of the national differences, but suggested to
> add a log format to select the country, in case there are any. From the
> various answers, it sounds like that's indeed required.
>

> but data retention is required by machines and asks for 100% accuracy,
> best achieved by simplicity, like setting a logfile option and selecting
> the format for Germany.
>
>
Why should exim "in the code" know of the different requirements for all
the countries? That would require a new version of exim each time a
country changes something on their requirement.
IMHO it would be much better to have a "LogFormat" option, which would
simply list the fields required in which order, like
envelope-from;envelope-to;date;.....

That way changing anything is as easy as changing this line in the
configuration. A FAQ-entry could list the requirements currently in
force by different country.

Oliver


--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##