----- Forwarded message from La Monte Henry Piggy Yarroll <piggy@baqaqi.chi.il.us> -----
Date: Fri, 27 Jul 2001 17:17:01 -0500
From: La Monte Henry Piggy Yarroll <piggy@baqaqi.chi.il.us>
To: marc@mbsi.ca
Fri Jul 27 16:01:06 CDT 2001
Netfilter WG
Not an introduction to netfilter and iptables
We assume that you are already familiar with these tools
1. Our current development efforts
1.1 Multiple related connections (or expectations)
IPsniffs your e.g. FTP control and raises an expectation
Currently there is a limitation that you can not have more than one
expectation outstanding.
Not really a problem for FTP, but there are other protocols which can
have three or more oustanding expectations at a time
E.g. IRC You try to establish to a connection to A and then a
connection to B (two expectations), and then the connection for A
comes in.
This will change a lot of the internal structures.
1.2 NFnetlink
Problem: When you look at how iptables communicates with the kernel,
it does it in a clumsy way, inherited from ipchains. It does an
atomic replace of the entire table, even for a single rule.
Problem: There are other places where the user space communicates with
the kernel.
1.2.0 iptable
1.2.1 ipqueue target can have packets sent from any chain to user space and
back
1.2.2 ulog is an extension which is unidirectional packet capture
Solution: Unified mechanism NFnetlink modeled on ?
Chez Shaloist (sp???) appeared from out of the dark and wrote a lot of
code for netfilter. "I have to thank him."
1.3 Stateful failover
Want redundancy in stateful firewalls for failover.
Put in time 4Q2000 1Q2001, but got away from that. There is no
current activity. There is a mailing list with archives.
If you are interested PLEASE contact us.
1.4 Connection tracking
Problem: As soon as you load the connection tracking module it tracks
all connections. It is all-or-nothing. It is nice design, nice
for NAT, nice for a stateful firewall. BUT it isn't necessary that
good because it DOES cost CPU, so you want an exception mechanism.
We discussed this on the list and finally decided on a specific
mechanism.
At the pre-routing hook is connection tracking, and then the mangle
table, and then the NAT? table.
Solution: Move mangle table BEFORE the connection tracking. The only
issue is that you will no longer have state in the pre-routing chain
of the mangle table. This seemed like the most viable solution.
Variant--allow people to configure the order. Probably not a good idea.
Nobody brought up an example.
"Why not add a new table? Tables are trivial to implement." "It adds
user complexity and an additional table traversal--even more
performance impact."
"As the original author of the mangle table I feel that is out of its
orginal charter." "We discussed this on the list--discussion is not
closed, but this seems the best so far."
-----
2.0 What else are we working on?
2.1 Ports to IPv6 are in the work
Configuration options for IPv4 vs options for IPv6 -- there are LOT
more for IPv6.
The IPv6 code was an integrated contribution, but we don't have
anybody who is really maintaining the IPv6 code. We (the four core
maintainers) try to keep it up to date, but there isn't much testing.
Currently we need somebody to really look after the IPv6 code. There
is still a lot missing. This is really very EASY work--you don't
really need to know much about kernel programming to port IPv4
netfilter code to IPv6. Right now netfilter is DEEPLY bound to the
network level, so we right now we still need to use copy/paste pattern
coding...
2.2 iptables save and restore
These are still experimental. It is difficult to debug. Sometimes a
particular rule does not get restored--a single failure can make the
whole transaction fail.
Each match (or target) has its own save/restore function and this code
is very poorly tested.
It is working quite well. You need to do a make experimental, but it
is included in many distrubutions.
If you encounter any problems PLEASE post them to the mailing list.
2.3 patch-o-matic
"Who knows what patch-o-matic wrt netfilter?" [about 6 hands]
patch-o-matic is a shell script which automates patch application. It
checks to make sure that the patch isn't already applied before
applying it.
There are features there which are stable and we can contribute to the
core kernel. But there are features which will probably spend their
entire lives in patch-o-matic.
E.g. experimental mirror target which just mirrors targets back to
you.
How do you use patch-o-matic? You download the iptables package or
anonymous CVS (for the latest). "make patch-o-matic". You get a
printed list of new features--you are asked y/n. If it succeeds, you
will have an extra config option in your kernel.
What kind of things does it have?
- Port-scan detection match
- The IP pool system--a combination of a match and a target. You can
collect IP addresses in a pool which you can then use later for more
matches.
"Is there any reason not to call a list a list?" "It is named 'pool'
because it is not necessarily"
"Is there a restriction on the size of the pool?" "Yes, it depends
upon which pool mechanism you use. There are limitations which I do
not know off hand."
"It might be a way to solve a particular problem I have, but I could
have a VERY LARGE pool." "This is an EXPERIMENTAL feature in
development. You would probably have to add a new pool type."
- statistical match "every 10th packet"
"This could actually be useful in production." "If there are really
are real applications, we CAN submit them to the kernel."
- packet type match
broadcast, multicast, unicast, whatever...
- time match
Match on the receive time on the packets. "I don't think this is the
right way to so this."
- content match
- ip option fields match
- ttl manipulation
"You can increase the ttl..." "boo hiss" "EXPERIMENTAL FEATURES"
- ulog
Replacement for the current log target.
Today we create a string in the kernel and printk the string up to
klogd. Later we have a perl script that parses that and generates
stats.
It just copies the skb and sends it up to user space for further
processing. Normally, you run ulogd which can convert the IP address,
parse stuff, write log file, or put it in a database or whatever.
Interpreters and then output plugins. Interpreters understand packet
parts. Output plugins support different output technologies
(e.g. SQL, syslog, etc...)
[.Have you considered using the interpreters from say tcpdump or
ethereal?]
- tcp window prepping
Contribution from a guy in Hungaria.
On a router you see a TCP packet with a sequence number. You track
the window too and then drop packets which do not fall into the
window.
Guido von Rui (sp???)
- patch for doing SNMP NAT
- patch which does RPC connection tracking
- patch for IRC connection tracking in NAT
- egg bot tracking ????
[We'd be happy to have an SMB connection tracking module.]
- packet drop module
There are places that packets get dropped. This patch lets you
capture these for logging or what-have-you.
- iplimit match
You have a server running inetd. It then starts a service,
e.g. pop3. DOS attack opens thousands of connections causing
thousands of processes. Better solution is to use xinetd or something
of the sort, but if you have proprietary systems etc...
"I had some questions about the way the table is updated. Currently
it is sucked in one big blob." "Yes, I mentioned the NFnetlink
project. There are subportions of NFnetlink for various kinds of
table, e.g. CTnetlink, IPTnetlink."
[lengthy discussion--I got distracted by a battery problem.]
"NFnetlink is a 2.5 extension."
3.0 Future plans
3.1 connection tracking
General framework for connection tracking--not just IP. It should not
be too complicated.
3.2 optimizations
E.g. make iptables more scalable in bigger environments
3.3 example repository
There is good documentation, but not very many examples. We need
examples for common cases. We have created the infrastructure for
that on the home page so that people can submit their examples for
review and inclusion.
3.4 home pages
We called for a volunteer web master. We got an overwhelming number
of people who wanted to do that. We've picked a new web master. The
new wm has submitted a new design, so the web page will change quite a
bit soon.
The CURRENT home page is developer-oriented. Now that we are in 2.4
we need to provide a more user-oriented page.
3.5 multicast support for connection tracking
There are issues.
3.6 bridging firewalls
There is a patch (not part of netfilter). "It is a piece of wire
which loses packets."
It seems now to have reached a stable part, so we can maybe include it
in the official distribution.
3.7 Better support for transparent proxying
"I am not an expert on this. Linux needs some stronger features in
this area."
em-tas-all
DISCUSSION
"I'm with the Linux virtual server project. The packets arrives with
the destination IP address altered--this does not work for us."
"I thought the lvs did all its own NAT, etc..."
[.discussion of 2.2 I didn't follow, It seems it is broken under
2.4...] "If the packets can not be recognized as local, [bad things
happen]."
"I don't see the relation to netfilter. Let's take this offline."
"How are the hooks put in for IPv6?" "The same as IPv4." "What about
fragmentation?" "There isn't any connection tracking for IPv6. We
should not bneed to do any fragmentation since we do not ever change
the size of a packet."
[More discussion about IPv6 interactions.]
--
"Make tests compulsory." "Tests aren't fun." "To do that, we'd have
to impose that on ourselves. We'd have to write tests for
everything." "XP suggests that you can just write tests for things
that you change."
--
[I totally missed this question.] "You would need a transaction
timer mechanism. Did you have a look at CTnetlink? It would bring
you a little bit further. Maybe you should contact Sdfkjdlkjj who
wrote CTnetlink. It seems a very lightweight extension. I don't have
a problem with that."
--
"I want to debug a firewall. I need to see where my packets are
going--which rules they are going through. Is there a tool to do
that?" "I think there was a patch to do this. Look in the archives."
--
----- End forwarded message -----
Date: Fri, 27 Jul 2001 17:17:01 -0500
From: La Monte Henry Piggy Yarroll <piggy@baqaqi.chi.il.us>
To: marc@mbsi.ca
Fri Jul 27 16:01:06 CDT 2001
Netfilter WG
Not an introduction to netfilter and iptables
We assume that you are already familiar with these tools
1. Our current development efforts
1.1 Multiple related connections (or expectations)
IPsniffs your e.g. FTP control and raises an expectation
Currently there is a limitation that you can not have more than one
expectation outstanding.
Not really a problem for FTP, but there are other protocols which can
have three or more oustanding expectations at a time
E.g. IRC You try to establish to a connection to A and then a
connection to B (two expectations), and then the connection for A
comes in.
This will change a lot of the internal structures.
1.2 NFnetlink
Problem: When you look at how iptables communicates with the kernel,
it does it in a clumsy way, inherited from ipchains. It does an
atomic replace of the entire table, even for a single rule.
Problem: There are other places where the user space communicates with
the kernel.
1.2.0 iptable
1.2.1 ipqueue target can have packets sent from any chain to user space and
back
1.2.2 ulog is an extension which is unidirectional packet capture
Solution: Unified mechanism NFnetlink modeled on ?
Chez Shaloist (sp???) appeared from out of the dark and wrote a lot of
code for netfilter. "I have to thank him."
1.3 Stateful failover
Want redundancy in stateful firewalls for failover.
Put in time 4Q2000 1Q2001, but got away from that. There is no
current activity. There is a mailing list with archives.
If you are interested PLEASE contact us.
1.4 Connection tracking
Problem: As soon as you load the connection tracking module it tracks
all connections. It is all-or-nothing. It is nice design, nice
for NAT, nice for a stateful firewall. BUT it isn't necessary that
good because it DOES cost CPU, so you want an exception mechanism.
We discussed this on the list and finally decided on a specific
mechanism.
At the pre-routing hook is connection tracking, and then the mangle
table, and then the NAT? table.
Solution: Move mangle table BEFORE the connection tracking. The only
issue is that you will no longer have state in the pre-routing chain
of the mangle table. This seemed like the most viable solution.
Variant--allow people to configure the order. Probably not a good idea.
Nobody brought up an example.
"Why not add a new table? Tables are trivial to implement." "It adds
user complexity and an additional table traversal--even more
performance impact."
"As the original author of the mangle table I feel that is out of its
orginal charter." "We discussed this on the list--discussion is not
closed, but this seems the best so far."
-----
2.0 What else are we working on?
2.1 Ports to IPv6 are in the work
Configuration options for IPv4 vs options for IPv6 -- there are LOT
more for IPv6.
The IPv6 code was an integrated contribution, but we don't have
anybody who is really maintaining the IPv6 code. We (the four core
maintainers) try to keep it up to date, but there isn't much testing.
Currently we need somebody to really look after the IPv6 code. There
is still a lot missing. This is really very EASY work--you don't
really need to know much about kernel programming to port IPv4
netfilter code to IPv6. Right now netfilter is DEEPLY bound to the
network level, so we right now we still need to use copy/paste pattern
coding...
2.2 iptables save and restore
These are still experimental. It is difficult to debug. Sometimes a
particular rule does not get restored--a single failure can make the
whole transaction fail.
Each match (or target) has its own save/restore function and this code
is very poorly tested.
It is working quite well. You need to do a make experimental, but it
is included in many distrubutions.
If you encounter any problems PLEASE post them to the mailing list.
2.3 patch-o-matic
"Who knows what patch-o-matic wrt netfilter?" [about 6 hands]
patch-o-matic is a shell script which automates patch application. It
checks to make sure that the patch isn't already applied before
applying it.
There are features there which are stable and we can contribute to the
core kernel. But there are features which will probably spend their
entire lives in patch-o-matic.
E.g. experimental mirror target which just mirrors targets back to
you.
How do you use patch-o-matic? You download the iptables package or
anonymous CVS (for the latest). "make patch-o-matic". You get a
printed list of new features--you are asked y/n. If it succeeds, you
will have an extra config option in your kernel.
What kind of things does it have?
- Port-scan detection match
- The IP pool system--a combination of a match and a target. You can
collect IP addresses in a pool which you can then use later for more
matches.
"Is there any reason not to call a list a list?" "It is named 'pool'
because it is not necessarily"
"Is there a restriction on the size of the pool?" "Yes, it depends
upon which pool mechanism you use. There are limitations which I do
not know off hand."
"It might be a way to solve a particular problem I have, but I could
have a VERY LARGE pool." "This is an EXPERIMENTAL feature in
development. You would probably have to add a new pool type."
- statistical match "every 10th packet"
"This could actually be useful in production." "If there are really
are real applications, we CAN submit them to the kernel."
- packet type match
broadcast, multicast, unicast, whatever...
- time match
Match on the receive time on the packets. "I don't think this is the
right way to so this."
- content match
- ip option fields match
- ttl manipulation
"You can increase the ttl..." "boo hiss" "EXPERIMENTAL FEATURES"
- ulog
Replacement for the current log target.
Today we create a string in the kernel and printk the string up to
klogd. Later we have a perl script that parses that and generates
stats.
It just copies the skb and sends it up to user space for further
processing. Normally, you run ulogd which can convert the IP address,
parse stuff, write log file, or put it in a database or whatever.
Interpreters and then output plugins. Interpreters understand packet
parts. Output plugins support different output technologies
(e.g. SQL, syslog, etc...)
[.Have you considered using the interpreters from say tcpdump or
ethereal?]
- tcp window prepping
Contribution from a guy in Hungaria.
On a router you see a TCP packet with a sequence number. You track
the window too and then drop packets which do not fall into the
window.
Guido von Rui (sp???)
- patch for doing SNMP NAT
- patch which does RPC connection tracking
- patch for IRC connection tracking in NAT
- egg bot tracking ????
[We'd be happy to have an SMB connection tracking module.]
- packet drop module
There are places that packets get dropped. This patch lets you
capture these for logging or what-have-you.
- iplimit match
You have a server running inetd. It then starts a service,
e.g. pop3. DOS attack opens thousands of connections causing
thousands of processes. Better solution is to use xinetd or something
of the sort, but if you have proprietary systems etc...
"I had some questions about the way the table is updated. Currently
it is sucked in one big blob." "Yes, I mentioned the NFnetlink
project. There are subportions of NFnetlink for various kinds of
table, e.g. CTnetlink, IPTnetlink."
[lengthy discussion--I got distracted by a battery problem.]
"NFnetlink is a 2.5 extension."
3.0 Future plans
3.1 connection tracking
General framework for connection tracking--not just IP. It should not
be too complicated.
3.2 optimizations
E.g. make iptables more scalable in bigger environments
3.3 example repository
There is good documentation, but not very many examples. We need
examples for common cases. We have created the infrastructure for
that on the home page so that people can submit their examples for
review and inclusion.
3.4 home pages
We called for a volunteer web master. We got an overwhelming number
of people who wanted to do that. We've picked a new web master. The
new wm has submitted a new design, so the web page will change quite a
bit soon.
The CURRENT home page is developer-oriented. Now that we are in 2.4
we need to provide a more user-oriented page.
3.5 multicast support for connection tracking
There are issues.
3.6 bridging firewalls
There is a patch (not part of netfilter). "It is a piece of wire
which loses packets."
It seems now to have reached a stable part, so we can maybe include it
in the official distribution.
3.7 Better support for transparent proxying
"I am not an expert on this. Linux needs some stronger features in
this area."
em-tas-all
DISCUSSION
"I'm with the Linux virtual server project. The packets arrives with
the destination IP address altered--this does not work for us."
"I thought the lvs did all its own NAT, etc..."
[.discussion of 2.2 I didn't follow, It seems it is broken under
2.4...] "If the packets can not be recognized as local, [bad things
happen]."
"I don't see the relation to netfilter. Let's take this offline."
"How are the hooks put in for IPv6?" "The same as IPv4." "What about
fragmentation?" "There isn't any connection tracking for IPv6. We
should not bneed to do any fragmentation since we do not ever change
the size of a packet."
[More discussion about IPv6 interactions.]
--
"Make tests compulsory." "Tests aren't fun." "To do that, we'd have
to impose that on ourselves. We'd have to write tests for
everything." "XP suggests that you can just write tests for things
that you change."
--
[I totally missed this question.] "You would need a transaction
timer mechanism. Did you have a look at CTnetlink? It would bring
you a little bit further. Maybe you should contact Sdfkjdlkjj who
wrote CTnetlink. It seems a very lightweight extension. I don't have
a problem with that."
--
"I want to debug a firewall. I need to see where my packets are
going--which rules they are going through. Is there a tool to do
that?" "I think there was a patch to do this. Look in the archives."
--
----- End forwarded message -----