Mailing List Archive

On 8/24/2011 6:43 PM, Roy T. Fielding wrote:
> On Aug 24, 2011, at 4:39 PM, William A. Rowe Jr. wrote:
>
>> On 8/24/2011 4:54 PM, Roy T. Fielding wrote:
>>> On Aug 24, 2011, at 1:56 PM, Roy T. Fielding wrote:
>>>> To be clear, I am more than willing to rewrite the part on
>>>> Ranges such that the above is explicitly forbidden in HTTP.
>>>> I am not sure what the WG would agree to, but I am quite certain
>>>> that part of the reason we have an Apache server is to protect
>>>> the Internet from idiotic ideas like the above.
>>>
>>> http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
>>
>> Excellent, thanks. Just curious, isn't this clarification outside of
>> the remit of 2616bis?
>
> Security repairs are never out of scope.

Ack.

So, I suspect the best we can do today, 4 days later, is to implement Roy's
draft [link] as the POC/reference implementation and work with the rest of
the http server community to ensure it is the right solution.

I suggest we publish this as a patch, /not/ as a release, until we find just
a bit more buy-in from the other implementors.

Bill

On Aug 24, 2011, at 4:56 PM, Roy T. Fielding wrote:

> On Aug 24, 2011, at 8:35 AM, Tim Bannister wrote:
>
>> On Tue, Aug 23, 2011, Roy T. Fielding wrote:
>>> And the spec says ...
>>> When a client requests multiple ranges in one request, the
>>> server SHOULD return them in the order that they appeared in the
>>> request.
>>> My suggestion is to reject any request with overlapping ranges or more than five ranges with a 416, and to send 200 for any request with 4-5 ranges. There is simply no need to support random access in HTTP.
>>
>> Deshpande & Zeng in http://dx.doi.org/10.1145/500141.500197 describe a method for "streaming" JPEG 2000 documents over HTTP, using many more than 5 ranges in a single request.
>> A client that knows about any server-side limit could make multiple requests each with a small number of ranges, but discovering that limit will add latency and take more code.
>
> I have no interest in supporting such a use case over HTTP.
> Consider how stupid it is to request ranges like their example
>
> Range: bytes=120-168,175-200,205-300,345-346,400-500,555-666,
> 667-800,900-1000,2500-2567,2890-3056,5678-9000,
> 10000-12004,12050-12060,15600-15605,17000-17001,
> 17005-17010,17050-17060,17800-17905,20000-20005
>
> keeping in mind that between each one of those ranges will be
> a multipart boundary of approximately 80 bytes! Hence, any
> range request that contains gaps of less than 80 bytes should
> be considered a denial of service, or at least an idiot programmer
> that deserves to be slapped by Apache.
>
> To be clear, I am more than willing to rewrite the part on
> Ranges such that the above is explicitly forbidden in HTTP.
> I am not sure what the WG would agree to, but I am quite certain
> that part of the reason we have an Apache server is to protect
> the Internet from idiotic ideas like the above.
>

OK then… we seem to be coalescing into some consensus here…
basically, if the client sends stuff which is brain-dead stupid,
we simply 2000 and send the whole kit-and-kaboodle.

I'd like to propose that we update the byterange filter to perform
the following:

o coalesce all adjacent ranges, whether overlapping or not.
(eg: 200-250,251-300 & 200-250,220-300 both merge to 200-300)
o We count:
> the number of times a gap between ranges is <80bytes
> the number of times we hit a descendent range
(eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as 1)
> the number of ranges total (post ascending merge)
If any >= some config-time limit, we send a 200

This is a start and was chosen simply for ease of implementation…
We can then expand it to be more functional…

Comments?

On Aug 24, 2011, at 5:54 PM, Roy T. Fielding wrote:

> On Aug 24, 2011, at 1:56 PM, Roy T. Fielding wrote:
>> To be clear, I am more than willing to rewrite the part on
>> Ranges such that the above is explicitly forbidden in HTTP.
>> I am not sure what the WG would agree to, but I am quite certain
>> that part of the reason we have an Apache server is to protect
>> the Internet from idiotic ideas like the above.
>
> http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
>

This is why I still love httpd...

On Thursday 25 August 2011, Jim Jagielski wrote:
> OK then… we seem to be coalescing into some consensus here…
> basically, if the client sends stuff which is brain-dead stupid,
> we simply 2000 and send the whole kit-and-kaboodle.
>
> I'd like to propose that we update the byterange filter to perform
> the following:
>
> o coalesce all adjacent ranges, whether overlapping or not.
> (eg: 200-250,251-300 & 200-250,220-300 both merge to 200-300)

This may still confuse a broken client. Maybe we could omit that from
the 2.2 patch for now and only commit to 2.3.

> o We count:
> > the number of times a gap between ranges is <80bytes
> > the number of times we hit a descendent range
> (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
> 1) > the number of ranges total (post ascending merge)
> If any >= some config-time limit, we send a 200
>
> This is a start and was chosen simply for ease of implementation…
> We can then expand it to be more functional…
>
> Comments?

Please also look at the patch at

http://mail-archives.apache.org/mod_mbox/httpd-
dev/201108.mbox/%3C201108250138.49474.sf@sfritsch.de%3E

which greatly reduces the memory needed for the range requests.
BTW, I won't have time to beat that into shape today. If anyone else
has, please go ahead.

> -----Original Message-----
> From: Stefan Fritsch
> Sent: Donnerstag, 25. August 2011 08:21
> To: dev@httpd.apache.org
> Subject: Re: DoS with mod_deflate & range requests
>
> On Thursday 25 August 2011, Jim Jagielski wrote:
> > OK then... we seem to be coalescing into some consensus here...
> > basically, if the client sends stuff which is brain-dead stupid,
> > we simply 2000 and send the whole kit-and-kaboodle.
> >
> > I'd like to propose that we update the byterange filter to perform
> > the following:
> >
> > o coalesce all adjacent ranges, whether overlapping or not.
> > (eg: 200-250,251-300 & 200-250,220-300 both merge to 200-300)
>
> This may still confuse a broken client. Maybe we could omit that from
> the 2.2 patch for now and only commit to 2.3.

Sounds like a plan. Or make it configurable with a default of off in 2.2.x
and on in 2.3.x

>
> > o We count:
> > > the number of times a gap between ranges is <80bytes
> > > the number of times we hit a descendent range
> > (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
> > 1) > the number of ranges total (post ascending merge)
> > If any >= some config-time limit, we send a 200
> >
> > This is a start and was chosen simply for ease of implementation...
> > We can then expand it to be more functional...
> >
> > Comments?


Looks good. Plus we should implement the patch from Stefan below and then we
should be good.

>
> Please also look at the patch at
>
> http://mail-archives.apache.org/mod_mbox/httpd-
> dev/201108.mbox/%3C201108250138.49474.sf@sfritsch.de%3E
>
> which greatly reduces the memory needed for the range requests.
> BTW, I won't have time to beat that into shape today. If anyone else
> has, please go ahead.
>

Regards

Rüdiger

On Aug 25, 2011, at 2:56 AM, Plüm, Rüdiger, VF-Group wrote:

>
>
>> -----Original Message-----
>> From: Stefan Fritsch
>> Sent: Donnerstag, 25. August 2011 08:21
>> To: dev@httpd.apache.org
>> Subject: Re: DoS with mod_deflate & range requests
>>
>> On Thursday 25 August 2011, Jim Jagielski wrote:
>>> OK then... we seem to be coalescing into some consensus here...
>>> basically, if the client sends stuff which is brain-dead stupid,
>>> we simply 2000 and send the whole kit-and-kaboodle.
>>>
>>> I'd like to propose that we update the byterange filter to perform
>>> the following:
>>>
>>> o coalesce all adjacent ranges, whether overlapping or not.
>>> (eg: 200-250,251-300 & 200-250,220-300 both merge to 200-300)
>>
>> This may still confuse a broken client. Maybe we could omit that from
>> the 2.2 patch for now and only commit to 2.3.
>
> Sounds like a plan. Or make it configurable with a default of off in 2.2.x
> and on in 2.3.x
>
>>
>>> o We count:
>>>> the number of times a gap between ranges is <80bytes
>>>> the number of times we hit a descendent range
>>> (eg: 200-1000,2000-3000,1200-1500,4000-5000 would count as
>>> 1) > the number of ranges total (post ascending merge)
>>> If any >= some config-time limit, we send a 200
>>>
>>> This is a start and was chosen simply for ease of implementation...
>>> We can then expand it to be more functional...
>>>
>>> Comments?
>
>
> Looks good. Plus we should implement the patch from Stefan below and then we
> should be good.
>

++1 (see other thread: Fixing Ranges)