Mailing List Archive

1 2 3  View All
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 12:05 PM, Plüm, Rüdiger, VF-Group wrote:

>
>
>> -----Original Message-----
>> From: Jim Jagielski [mailto:jim@jaguNET.com]
>> Sent: Mittwoch, 24. August 2011 18:02
>> To: dev@httpd.apache.org
>> Subject: Re: DoS with mod_deflate & range requests
>>
>> Sorting isn't allowed but I get the impression that merging is OK...
>> Roy can confirm...
>
> But merging might require sorting...
>

then we don't do that merge, imo… In other words, we
progress thru the set of ranges and once a range
is merged as far as it can be (due to the next range
not being merge-able with the previous one), we let
it go...
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 12:22 PM, William A. Rowe Jr. wrote:

>
> 0-, 40-50 becomes 0-

> 0-499, 400-599 becomes 0-599

> 1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250

This goes against Roy's recommendation to 416 overlaps… But
I do see that an overlap is specifically noted in an example

Until we are *clear* on what we should be doing, spec-wise, I
think it's unwise to make assumptions…

From the above, I would be more comfortable with

0-, 40-50 ---> 0-
0-499, 400-599 ---> 0-599
1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250, 1051-1100

that it, merge as we can, but never resort...
Re: DoS with mod_deflate & range requests [ In reply to ]
On Wednesday 24 August 2011, Jim Jagielski wrote:
> On Aug 24, 2011, at 12:05 PM, Plüm, Rüdiger, VF-Group wrote:
> >> -----Original Message-----
> >> From: Jim Jagielski [mailto:jim@jaguNET.com]
> >> Sent: Mittwoch, 24. August 2011 18:02
> >> To: dev@httpd.apache.org
> >> Subject: Re: DoS with mod_deflate & range requests
> >>
> >> Sorting isn't allowed but I get the impression that merging is
> >> OK... Roy can confirm...
> >
> > But merging might require sorting...
>
> then we don't do that merge, imo… In other words, we
> progress thru the set of ranges and once a range
> is merged as far as it can be (due to the next range
> not being merge-able with the previous one), we let
> it go...

We could also use a two stage approach: Up to some limit (e.g. 50)
ranges, we return them as the client requested them. Over that limit,
we violate the RFC-SHOULD and sort and merge them.
Re: DoS with mod_deflate & range requests [ In reply to ]
On 24 Aug 2011, at 17:47, Stefan Fritsch wrote:
On Wednesday 24 August 2011, Jim Jagielski wrote:
>> On Aug 24, 2011, at 12:05 PM, Plüm, Rüdiger, VF-Group wrote:
>>>
>>> But merging might require sorting...
>>
>> then we don't do that merge, imo… In other words, we progress thru the set of ranges and once a range is merged as far as it can be (due to the next range not being merge-able with the previous one), we let it go...
>
> We could also use a two stage approach: Up to some limit (e.g. 50) ranges, we return them as the client requested them. Over that limit, we violate the RFC-SHOULD and sort and merge them.

Another option is just to return 200. Servers MAY ignore the Range header. I prefer this because existing clients already handle that case well, and there's no opportunity for a client to exploit this (“malicious” clients that want the whole entity need only request it).

Can anyone see why returning 200 for these complex requests (by ignoring Range / If-Range) is a bad idea?

--
Tim Bannister – isoma@jellybaby.net
Re: DoS with mod_deflate & range requests [ In reply to ]
On Wed, Aug 24, 2011 at 12:42 PM, Jim Jagielski <jim@jagunet.com> wrote:

>
> On Aug 24, 2011, at 12:22 PM, William A. Rowe Jr. wrote:
>
> >
> > 0-, 40-50 becomes 0-
>
> > 0-499, 400-599 becomes 0-599
>
> > 1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250
>
> This goes against Roy's recommendation to 416 overlaps… But
> I do see that an overlap is specifically noted in an example
>

yeah. The very end of section 14.35.1 says an overlap is legal, so I'm
confused.


>
> Until we are *clear* on what we should be doing, spec-wise, I
> think it's unwise to make assumptions…
>

> From the above, I would be more comfortable with
>
> 0-, 40-50 ---> 0-
> 0-499, 400-599 ---> 0-599
> 1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250,
> 1051-1100
>
> that it, merge as we can, but never resort...


how about:

1000-2000,100-200,3000-4000,200-300,1999-3001

?

If we don't return a 416 for that due to overlap, I think the merge should
be;

1000-4000,100-300

If we only merge adjacent ascending ranges, then it seems like an attacker
could just craft a header where the ranges jump around and dodge our fix.

The other small point I wanted to make is that both ends of a range could
overlap previously specified ranges.

Greg
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 2:43 PM, Tim Bannister wrote:

> On 24 Aug 2011, at 17:47, Stefan Fritsch wrote:
> On Wednesday 24 August 2011, Jim Jagielski wrote:
>>> On Aug 24, 2011, at 12:05 PM, Plüm, Rüdiger, VF-Group wrote:
>>>>
>>>> But merging might require sorting...
>>>
>>> then we don't do that merge, imo… In other words, we progress thru the set of ranges and once a range is merged as far as it can be (due to the next range not being merge-able with the previous one), we let it go...
>>
>> We could also use a two stage approach: Up to some limit (e.g. 50) ranges, we return them as the client requested them. Over that limit, we violate the RFC-SHOULD and sort and merge them.
>
> Another option is just to return 200. Servers MAY ignore the Range header. I prefer this because existing clients already handle that case well, and there's no opportunity for a client to exploit this (“malicious” clients that want the whole entity need only request it).
>
> Can anyone see why returning 200 for these complex requests (by ignoring Range / If-Range) is a bad idea?

In what cases would we ignore it? Dependent only on >=X ranges?
It does seem that 14.5 gives us this as an out...
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 3:10 PM, Greg Ames wrote:

>
>
> On Wed, Aug 24, 2011 at 12:42 PM, Jim Jagielski <jim@jagunet.com> wrote:
>
> >From the above, I would be more comfortable with
>
> 0-, 40-50 ---> 0-
> 0-499, 400-599 ---> 0-599
> 1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250, 1051-1100
>
> that it, merge as we can, but never resort...
>
> how about:
>
> 1000-2000,100-200,3000-4000,200-300,1999-3001
>
> ?
>
> If we don't return a 416 for that due to overlap, I think the merge should be;
>
> 1000-4000,100-300

That's what Bill thinks as well, but that almost seems like
a "resorting" to be, such that the 100-200 range (2nd requested)
comes *after* the server sends 3000-4000, which is actually the 3rd
range requested.

>
> If we only merge adjacent ascending ranges, then it seems like an attacker could just craft a header where the ranges jump around and dodge our fix.
>

I think no matter what, we should still have some sort of
upper limit on the number of range-sets we accept… after all,
merge doesn't prevent jumping around ;)
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 11:42 AM, Jim Jagielski wrote:
>
> On Aug 24, 2011, at 12:22 PM, William A. Rowe Jr. wrote:
>
>> 0-, 40-50 becomes 0-
>
>> 0-499, 400-599 becomes 0-599
>
>> 1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250
>
> This goes against Roy's recommendation to 416 overlaps… But
> I do see that an overlap is specifically noted in an example

And... 416 is not identified for this specific purpose, we would need
to go with 400 or fall back on the 200 full-content solution.

> Until we are *clear* on what we should be doing, spec-wise, I
> think it's unwise to make assumptions…
>
> From the above, I would be more comfortable with
>
> 0-, 40-50 ---> 0-
> 0-499, 400-599 ---> 0-599
> 1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250, 1051-1100
>
> that it, merge as we can, but never resort...

We should adamantly refuse to serve bytes 1051-1088 twice, no matter
which scheme we use.
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 3:34 PM, William A. Rowe Jr. wrote:

> On 8/24/2011 11:42 AM, Jim Jagielski wrote:
>>
>> On Aug 24, 2011, at 12:22 PM, William A. Rowe Jr. wrote:
>>
>>> 0-, 40-50 becomes 0-
>>
>>> 0-499, 400-599 becomes 0-599
>>
>>> 1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250
>>
>> This goes against Roy's recommendation to 416 overlaps… But
>> I do see that an overlap is specifically noted in an example
>
> And... 416 is not identified for this specific purpose, we would need
> to go with 400 or fall back on the 200 full-content solution.
>
>> Until we are *clear* on what we should be doing, spec-wise, I
>> think it's unwise to make assumptions…
>>
>> From the above, I would be more comfortable with
>>
>> 0-, 40-50 ---> 0-
>> 0-499, 400-599 ---> 0-599
>> 1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250, 1051-1100
>>
>> that it, merge as we can, but never resort...
>
> We should adamantly refuse to serve bytes 1051-1088 twice, no matter
> which scheme we use.
>

Why? If allowed by the spec, or, at least, not disallowed, then
what is the rationale? After all, the client is expecting it after
it gets bytes 200->250.
Re: DoS with mod_deflate & range requests [ In reply to ]
On 24 Aug 2011, at 20:13, Jim Jagielski wrote:

>> Another option is just to return 200. Servers MAY ignore the Range header. I prefer this because existing clients already handle that case well, and there's no opportunity for a client to exploit this (“malicious” clients that want the whole entity need only request it).
>>
>> Can anyone see why returning 200 for these complex requests (by ignoring Range / If-Range) is a bad idea?
>
> In what cases would we ignore it? Dependent only on >=X ranges?

I don't have any strong opinion about exactly when to ignore Range. From an HTTP client PoV I wouldn't want to get 416 from requesting a satisfiable but complex range (maliciously or otherwise).

Ignoring Range on (ranges >= X) is simple to implement and easy to document, so why not do that?

--
Tim Bannister – isoma@jellybaby.net
Re: DoS with mod_deflate & range requests [ In reply to ]
On Wed, Aug 24, 2011 at 3:19 PM, Jim Jagielski <jim@jagunet.com> wrote:

>
> >
> > If we only merge adjacent ascending ranges, then it seems like an
> attacker could just craft a header where the ranges jump around and dodge
> our fix.
> >
>
> I think no matter what, we should still have some sort of
> upper limit on the number of range-sets we accept… after all,
> merge doesn't prevent jumping around ;)
>
>
The problem I have with the upper limit on the number of range sets is the
use case someone posted for JPEG2000 streaming. That has a lot of range
sets but is completely legit. However, the ranges are in ascending order
and don't overlap. Maybe we could count overlaps and/or non-ascending order
ranges and fall back to 200 + the whole object if it exceeds a limit.

Greg
Re: DoS with mod_deflate & range requests [ In reply to ]
On 24 Aug 2011, at 21:39, Greg Ames wrote:

> On Wed, Aug 24, 2011 at 3:19 PM, Jim Jagielski <jim@jagunet.com> wrote:
>
> >
> > If we only merge adjacent ascending ranges, then it seems like an attacker could just craft a header where the ranges jump around and dodge our fix.
> >
>
> I think no matter what, we should still have some sort of
> upper limit on the number of range-sets we accept… after all,
> merge doesn't prevent jumping around ;)
>
>
> The problem I have with the upper limit on the number of range sets is the use case someone posted for JPEG2000 streaming. That has a lot of range sets but is completely legit. However, the ranges are in ascending order and don't overlap. Maybe we could count overlaps and/or non-ascending order ranges and fall back to 200 + the whole object if it exceeds a limit.

Right - and the other two use cases in the wild are

- PDF readers - which fetch something at the start in RQ 1 and then the index form the end - and then quick looks images for each page and title pages. I've seen them do a second and 3rd request with many 10's of ranges.

- Some of the streaming video (semi/pro) video editors - which fetch a bunch of i-Frames and do clever skip over stuff. Not in the high tens; but 10-15 ranges common.

- Likewise for very clever MXF professional editing equipment - the largest case (yup - it did crash my server) tried to fetch over 2000 ranges :)

So I think we really should endeavor to allow 50 to a few 100 of them. Or at the very least - make it a config option to cut off below 50 or so.

Dw.
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 8:35 AM, Tim Bannister wrote:

> On Tue, Aug 23, 2011, Roy T. Fielding wrote:
>> And the spec says ...
>> When a client requests multiple ranges in one request, the
>> server SHOULD return them in the order that they appeared in the
>> request.
>> My suggestion is to reject any request with overlapping ranges or more than five ranges with a 416, and to send 200 for any request with 4-5 ranges. There is simply no need to support random access in HTTP.
>
> Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197 describe a method for "streaming" JPEG 2000 documents over HTTP, using many more than 5 ranges in a single request.
> A client that knows about any server-side limit could make multiple requests each with a small number of ranges, but discovering that limit will add latency and take more code.

I have no interest in supporting such a use case over HTTP.
Consider how stupid it is to request ranges like their example

Range: bytes=120-168,175-200,205-300,345-346,400-500,555-666,
667-800,900-1000,2500-2567,2890-3056,5678-9000,
10000-12004,12050-12060,15600-15605,17000-17001,
17005-17010,17050-17060,17800-17905,20000-20005

keeping in mind that between each one of those ranges will be
a multipart boundary of approximately 80 bytes! Hence, any
range request that contains gaps of less than 80 bytes should
be considered a denial of service, or at least an idiot programmer
that deserves to be slapped by Apache.

To be clear, I am more than willing to rewrite the part on
Ranges such that the above is explicitly forbidden in HTTP.
I am not sure what the WG would agree to, but I am quite certain
that part of the reason we have an Apache server is to protect
the Internet from idiotic ideas like the above.

....Roy
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 8:55 AM, Plüm, Rüdiger, VF-Group wrote:
> Hm. If I got it right what Roy says above about the spec sorting and merging is
> not an option as we need to stick to the order and number of ranges the client
> requested. But we can deny overlapping with a 416.

We should implement whatever is needed to make the service
secure from this denial of service. If that means changing the
spec, then so be it.

> Or we do a 416 as well if merging would change something.

Or 200 if folks are squeamish about making the developer feel
the pain.

....Roy
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 3:45 PM, Dirk-WIllem van Gulik wrote:
>
> On 24 Aug 2011, at 21:39, Greg Ames wrote:
>
>> On Wed, Aug 24, 2011 at 3:19 PM, Jim Jagielski <jim@jagunet.com
>> <mailto:jim@jagunet.com>> wrote:
>>
>>
>> >
>> > If we only merge adjacent ascending ranges, then it seems like an attacker could
>> just craft a header where the ranges jump around and dodge our fix.
>> >
>>
>> I think no matter what, we should still have some sort of
>> upper limit on the number of range-sets we accept… after all,
>> merge doesn't prevent jumping around ;)
>>
>>
>> The problem I have with the upper limit on the number of range sets is the use case
>> someone posted for JPEG2000 streaming. That has a lot of range sets but is completely
>> legit. However, the ranges are in ascending order and don't overlap. Maybe we could
>> count overlaps and/or non-ascending order ranges and fall back to 200 + the whole object
>> if it exceeds a limit.
>
> Right - and the other two use cases in the wild are
>
> -PDF readers - which fetch something at the start in RQ 1 and then the index form the end
> - and then quick looks images for each page and title pages. I've seen them do a second
> and 3rd request with many 10's of ranges.
>
> -Some of the streaming video (semi/pro) video editors - which fetch a bunch of i-Frames
> and do clever skip over stuff. Not in the high tens; but 10-15 ranges common.
>
> -Likewise for very clever MXF professional editing equipment - the largest case (yup - it
> did crash my server) tried to fetch over 2000 ranges :)
>
> So I think we really should endeavor to allow 50 to a few 100 of them. Or at the very
> least - make it a config option to cut off below 50 or so.

At least, after 256 ranges or so, fall back to a 200 response in lieu of
a 400/416 response.
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 3:12 PM, Jim Jagielski wrote:
>
> On Aug 24, 2011, at 3:34 PM, William A. Rowe Jr. wrote:
>
>> On 8/24/2011 11:42 AM, Jim Jagielski wrote:
>>>
>>> On Aug 24, 2011, at 12:22 PM, William A. Rowe Jr. wrote:
>>>
>>>> 0-, 40-50 becomes 0-
>>>
>>>> 0-499, 400-599 becomes 0-599
>>>
>>>> 1000-1075, 200-250, 1051-1100 becomes 1000-1100, 200-250
>>>
>>> This goes against Roy's recommendation to 416 overlaps… But
>>> I do see that an overlap is specifically noted in an example
>>
>> And... 416 is not identified for this specific purpose, we would need
>> to go with 400 or fall back on the 200 full-content solution.
>>
>>> Until we are *clear* on what we should be doing, spec-wise, I
>>> think it's unwise to make assumptions…
>>>
>>> From the above, I would be more comfortable with
>>>
>>> 0-, 40-50 ---> 0-
>>> 0-499, 400-599 ---> 0-599
>>> 1000-1075, 1025-1088, 200-250, 1051-1100 --> 1000-1088, 200-250, 1051-1100
>>>
>>> that it, merge as we can, but never resort...
>>
>> We should adamantly refuse to serve bytes 1051-1088 twice, no matter
>> which scheme we use.
>>
>
> Why? If allowed by the spec, or, at least, not disallowed, then
> what is the rationale? After all, the client is expecting it after
> it gets bytes 200->250.

The client was malformed, if not malicious. The range 0-,0-,0-,0-
is syntactically valid, but the client doesn't deserve four copies.
Re: DoS with mod_deflate & range requests [ In reply to ]
On Wednesday 24 August 2011, Dirk-WIllem van Gulik wrote:
> > I think no matter what, we should still have some sort of
> > upper limit on the number of range-sets we accept… after all,
> > merge doesn't prevent jumping around ;)
> >
> >
> >
> >
> > The problem I have with the upper limit on the number of range
> > sets is the use case someone posted for JPEG2000
> > streaming. That has a lot of range sets but is completely
> > legit. However, the ranges are in ascending order and don't
> > overlap. Maybe we could count overlaps and/or non-ascending
> > order ranges and fall back to 200 + the whole object if it
> > exceeds a limit.
>
> Right - and the other two use cases in the wild are
>
> - PDF readers - which fetch something at the start in RQ 1
> and then the index form the end - and then quick looks images for
> each page and title pages. I've seen them do a second and 3rd
> request with many 10's of ranges.
>
> - Some of the streaming video (semi/pro) video editors -
> which fetch a bunch of i-Frames and do clever skip over stuff. Not
> in the high tens; but 10-15 ranges common.
>
> - Likewise for very clever MXF professional editing equipment
> - the largest case (yup - it did crash my server) tried to fetch
> over 2000 ranges :)
>
> So I think we really should endeavor to allow 50 to a few 100 of
> them. Or at the very least - make it a config option to cut off
> below 50 or so.

Do you know if those clients send the ranges in order? If they are
sorted, it is easy to check if they are non-overlapping. And in that
case, we could easily allow 1000 ranges.
Re: DoS with mod_deflate & range requests [ In reply to ]
I'm cool w/ that… treat non-ascending ranges as potential hinky
and count those and only allow a certain number of them…

Still not sure if we should count overlaps as bad or not…
that RFC example troubles me:

14.35.1 Byte Ranges
- Several legal but not canonical specifications of the second 500
bytes (byte offsets 500-999, inclusive):
bytes=500-600,601-999
bytes=500-700,601-999

The 2nd seems to imply that one *MUST* merge adjacent overlaps to get the
correct response (500 bytes not 201+399=600bytes)

With all that in mind, I am still of the opinion that any
adjacent overlaps should be merged…

So how about we parse Range and merge all adjacent overlaps
(or merges (200-249,250-999 would merge into 200-999);
We then count how many non-ascends are in that revised set of
ranges and 200 out if it exceeds some config limit. We can also
provide some overall limit on the number of ranges, or at least
the ability to add one (a default of 0 means unlimited)…

Sound OK?

On Aug 24, 2011, at 4:39 PM, Greg Ames wrote:

> On Wed, Aug 24, 2011 at 3:19 PM, Jim Jagielski <jim@jagunet.com> wrote:
>
> >
> > If we only merge adjacent ascending ranges, then it seems like an attacker could just craft a header where the ranges jump around and dodge our fix.
> >
>
> I think no matter what, we should still have some sort of
> upper limit on the number of range-sets we accept… after all,
> merge doesn't prevent jumping around ;)
>
>
> The problem I have with the upper limit on the number of range sets is the use case someone posted for JPEG2000 streaming. That has a lot of range sets but is completely legit. However, the ranges are in ascending order and don't overlap. Maybe we could count overlaps and/or non-ascending order ranges and fall back to 200 + the whole object if it exceeds a limit.
>
> Greg
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 5:00 PM, William A. Rowe Jr. wrote:
>
> At least, after 256 ranges or so, fall back to a 200 response in lieu of
> a 400/416 response.
>


+1 on not sending 416 if we hit some limit and fall back on 200.
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 3:56 PM, Roy T. Fielding wrote:
> On Aug 24, 2011, at 8:35 AM, Tim Bannister wrote:
>
>> On Tue, Aug 23, 2011, Roy T. Fielding wrote:
>>> And the spec says ...
>>> When a client requests multiple ranges in one request, the
>>> server SHOULD return them in the order that they appeared in the
>>> request.
>>> My suggestion is to reject any request with overlapping ranges or more than five ranges with a 416, and to send 200 for any request with 4-5 ranges. There is simply no need to support random access in HTTP.
>>
>> Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197 describe a method for "streaming" JPEG 2000 documents over HTTP, using many more than 5 ranges in a single request.
>> A client that knows about any server-side limit could make multiple requests each with a small number of ranges, but discovering that limit will add latency and take more code.
>
> I have no interest in supporting such a use case over HTTP.
> Consider how stupid it is to request ranges like their example
>
> Range: bytes=120-168,175-200,205-300,345-346,400-500,555-666,
> 667-800,900-1000,2500-2567,2890-3056,5678-9000,
> 10000-12004,12050-12060,15600-15605,17000-17001,
> 17005-17010,17050-17060,17800-17905,20000-20005
>
> keeping in mind that between each one of those ranges will be
> a multipart boundary of approximately 80 bytes! Hence, any
> range request that contains gaps of less than 80 bytes should
> be considered a denial of service, or at least an idiot programmer
> that deserves to be slapped by Apache.
>
> To be clear, I am more than willing to rewrite the part on
> Ranges such that the above is explicitly forbidden in HTTP.
> I am not sure what the WG would agree to, but I am quite certain
> that part of the reason we have an Apache server is to protect
> the Internet from idiotic ideas like the above.

Then if we are opening up the spec for sensible revision, particularly
in the gray areas of what was not answered, insisting that the server
is free to respond to the client with any serialized superset of their
requested ranges [.deliberately ignoring the SHOULD in the section you
had previously quoted] is the right answer. In your pedantic case
above, adjacent ranges < 80 bytes apart would be processed as a
single merged range.

A client insisting on ranges must be prepared to follow the rules
provided to all proxies in that section on range handling, given that
the proxy case is already one user agent case, and the requirements
for proxy handling should certainly be applied in the generic case.

The spec does not actually state that ranges are returned 1:1 in
sequence, and I believe we should liberally read this to protect
the server from abuse. Perhaps we have a threshold number of
ranges which trigger the behavior, or any overlapping (apparently
abusive) range requests would trigger the behavior regardless.
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 4:06 PM, Jim Jagielski wrote:
> I'm cool w/ that… treat non-ascending ranges as potential hinky
> and count those and only allow a certain number of them…
>
> Still not sure if we should count overlaps as bad or not…
> that RFC example troubles me:
>
> 14.35.1 Byte Ranges
> - Several legal but not canonical specifications of the second 500
> bytes (byte offsets 500-999, inclusive):
> bytes=500-600,601-999
> bytes=500-700,601-999
>
> The 2nd seems to imply that one *MUST* merge adjacent overlaps to get the
> correct response (500 bytes not 201+399=600bytes)
>
> With all that in mind, I am still of the opinion that any
> adjacent overlaps should be merged…
>
> So how about we parse Range and merge all adjacent overlaps
> (or merges (200-249,250-999 would merge into 200-999);
> We then count how many non-ascends are in that revised set of
> ranges and 200 out if it exceeds some config limit. We can also
> provide some overall limit on the number of ranges, or at least
> the ability to add one (a default of 0 means unlimited)…
>
> Sound OK?

Yup, sounds good. The only question is non-adjacent overlaps.
Given Roy's pedantic example, I believe we should also start to
dismiss any gap of less than X (80 bytes?) and provide those
bytes as well in the merged range.

Yes, clients may break. They were morons anyways for asking us
to skip a few bytes for them and increase network traffic. Once
the author accommodates the fact that they aren't in control, the
response is semantically accurate.

For that matter, perhaps User-Agent could be used to determine if
we have a backwards-broken client for which we fall into the very
well documented 200 response.
Re: DoS with mod_deflate & range requests [ In reply to ]
On Wed, Aug 24, 2011 at 5:06 PM, Jim Jagielski <jim@jagunet.com> wrote:

> I'm cool w/ that… treat non-ascending ranges as potential hinky
> and count those and only allow a certain number of them…
>
> Still not sure if we should count overlaps as bad or not…
> that RFC example troubles me:
>
> 14.35.1 Byte Ranges
> - Several legal but not canonical specifications of the second 500
> bytes (byte offsets 500-999, inclusive):
> bytes=500-600,601-999
> bytes=500-700,601-999
>
> The 2nd seems to imply that one *MUST* merge adjacent overlaps to get the
> correct response (500 bytes not 201+399=600bytes)
>
> With all that in mind, I am still of the opinion that any
> adjacent overlaps should be merged…
>

So how about we parse Range and merge all adjacent overlaps
> (or merges (200-249,250-999 would merge into 200-999);
> We then count how many non-ascends are in that revised set of
> ranges and 200 out if it exceeds some config limit.


Sounds good to me. Maybe re-define an overlap to include gaps of less than
80 bytes, per Roy's comments, and merge those too.


> We can also
> provide some overall limit on the number of ranges, or at least
> the ability to add one (a default of 0 means unlimited)…
>

sure, but it feels less urgent than the above.

Greg
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 1:56 PM, Roy T. Fielding wrote:
> To be clear, I am more than willing to rewrite the part on
> Ranges such that the above is explicitly forbidden in HTTP.
> I am not sure what the WG would agree to, but I am quite certain
> that part of the reason we have an Apache server is to protect
> the Internet from idiotic ideas like the above.

http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311

....Roy
Re: DoS with mod_deflate & range requests [ In reply to ]
On 8/24/2011 4:54 PM, Roy T. Fielding wrote:
> On Aug 24, 2011, at 1:56 PM, Roy T. Fielding wrote:
>> To be clear, I am more than willing to rewrite the part on
>> Ranges such that the above is explicitly forbidden in HTTP.
>> I am not sure what the WG would agree to, but I am quite certain
>> that part of the reason we have an Apache server is to protect
>> the Internet from idiotic ideas like the above.
>
> http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311

Excellent, thanks. Just curious, isn't this clarification outside of
the remit of 2616bis?
Re: DoS with mod_deflate & range requests [ In reply to ]
On Aug 24, 2011, at 4:39 PM, William A. Rowe Jr. wrote:

> On 8/24/2011 4:54 PM, Roy T. Fielding wrote:
>> On Aug 24, 2011, at 1:56 PM, Roy T. Fielding wrote:
>>> To be clear, I am more than willing to rewrite the part on
>>> Ranges such that the above is explicitly forbidden in HTTP.
>>> I am not sure what the WG would agree to, but I am quite certain
>>> that part of the reason we have an Apache server is to protect
>>> the Internet from idiotic ideas like the above.
>>
>> http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
>
> Excellent, thanks. Just curious, isn't this clarification outside of
> the remit of 2616bis?

Security repairs are never out of scope.

....Roy

1 2 3  View All