Mailing List Archive

Hey guys,
I'm thinking about big flv files in varnishd. Now,many web sites support
video, Typically, these files more than 5M.
I don't konw what's better .. split file in function fetch_straight ?
or do nothing , set up tcp parameters?
I would like to hear anything from you
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-dev/attachments/20080616/d06695d3/attachment-0001.htm

In message <7ab169e90806160307w778fb06ay1eb92b19d6e17ad1 at mail.gmail.com>, "chen
xiaoyong" writes:

>Hey guys,
> I'm thinking about big flv files in varnishd. Now,many web sites support
>video, Typically, these files more than 5M.

There is no intrinsic limit on object size in varnish, but it is not
particular geared towards such use. The major stumbling block is that
Varnish will fetch the entire file from the backend before starting
transmission to the client.

> I don't konw what's better .. split file in function fetch_straight ?
>or do nothing , set up tcp parameters?

I'm not sure I understand what you're trying to say here ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

In message <7ab169e90806160307w778fb06ay1eb92b19d6e17ad1 at mail.gmail.com>, "chen
xiaoyong" writes:

>Hey guys,
> I'm thinking about big flv files in varnishd. Now,many web sites support
>video, Typically, these files more than 5M.

There is no intrinsic limit on object size in varnish, but it is not
particular geared towards such use. The major stumbling block is that
Varnish will fetch the entire file from the backend before starting
transmission to the client.

> I don't konw what's better .. split file in function fetch_straight ?
>or do nothing , set up tcp parameters?

I'm not sure I understand what you're trying to say here ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

On Jun 16, 2008, at 3:15 AM, Poul-Henning Kamp wrote:

> In message
> <7ab169e90806160307w778fb06ay1eb92b19d6e17ad1 at mail.gmail.com>, "chen
> xiaoyong" writes:
>
>> Hey guys,
>> I'm thinking about big flv files in varnishd. Now,many web sites
>> support
>> video, Typically, these files more than 5M.
>
> There is no intrinsic limit on object size in varnish, but it is not
> particular geared towards such use. The major stumbling block is that
> Varnish will fetch the entire file from the backend before starting
> transmission to the client.

Have you considered allowing a configuration for streaming a file to
a client while it's being fetched from the backend server? Upon the
first successful sending of the complete file to the client, it can
then be stored in the cache for subsequent serving. Also, it would be
helpful to be able to limit the number of backend connections that
are streaming in this way so that it's possible to prevent swamping
the origin server when an object is very popular. Perhaps if the
limit is reached, it could reply with a 502 Server Busy reply. In
addition, I think this would be a fantastic addition.

>> I don't konw what's better .. split file in function
>> fetch_straight ?
>> or do nothing , set up tcp parameters?
>
> I'm not sure I understand what you're trying to say here ?

It sounds like he's trying to optimize download performance.

On Jun 16, 2008, at 3:15 AM, Poul-Henning Kamp wrote:

> In message
> <7ab169e90806160307w778fb06ay1eb92b19d6e17ad1 at mail.gmail.com>, "chen
> xiaoyong" writes:
>
>> Hey guys,
>> I'm thinking about big flv files in varnishd. Now,many web sites
>> support
>> video, Typically, these files more than 5M.
>
> There is no intrinsic limit on object size in varnish, but it is not
> particular geared towards such use. The major stumbling block is that
> Varnish will fetch the entire file from the backend before starting
> transmission to the client.

Have you considered allowing a configuration for streaming a file to
a client while it's being fetched from the backend server? Upon the
first successful sending of the complete file to the client, it can
then be stored in the cache for subsequent serving. Also, it would be
helpful to be able to limit the number of backend connections that
are streaming in this way so that it's possible to prevent swamping
the origin server when an object is very popular. Perhaps if the
limit is reached, it could reply with a 502 Server Busy reply. In
addition, I think this would be a fantastic addition.

>> I don't konw what's better .. split file in function
>> fetch_straight ?
>> or do nothing , set up tcp parameters?
>
> I'm not sure I understand what you're trying to say here ?

It sounds like he's trying to optimize download performance.

I'll proudly sponsor it, along with at least one other important
feature I'm seeking. Simply contact me directly and we can make
arrangements.

Thanks,

Adrian

On Jun 16, 2008, at 8:28 AM, Poul-Henning Kamp wrote:

> In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian
> Otto writes
> :
>
>>> There is no intrinsic limit on object size in varnish, but it is not
>>> particular geared towards such use. The major stumbling block is
>>> that
>>> Varnish will fetch the entire file from the backend before starting
>>> transmission to the client.
>>
>> Have you considered allowing a configuration for streaming a file to
>> a client while it's being fetched from the backend server?
>
> Yes.
>
> Do I have a sponsor for it ? No.
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

I'll proudly sponsor it, along with at least one other important
feature I'm seeking. Simply contact me directly and we can make
arrangements.

Thanks,

Adrian

On Jun 16, 2008, at 8:28 AM, Poul-Henning Kamp wrote:

> In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian
> Otto writes
> :
>
>>> There is no intrinsic limit on object size in varnish, but it is not
>>> particular geared towards such use. The major stumbling block is
>>> that
>>> Varnish will fetch the entire file from the backend before starting
>>> transmission to the client.
>>
>> Have you considered allowing a configuration for streaming a file to
>> a client while it's being fetched from the backend server?
>
> Yes.
>
> Do I have a sponsor for it ? No.
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian Otto writes
:

>> There is no intrinsic limit on object size in varnish, but it is not
>> particular geared towards such use. The major stumbling block is that
>> Varnish will fetch the entire file from the backend before starting
>> transmission to the client.
>
>Have you considered allowing a configuration for streaming a file to
>a client while it's being fetched from the backend server?

Yes.

Do I have a sponsor for it ? No.


--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian Otto writes
:

>> There is no intrinsic limit on object size in varnish, but it is not
>> particular geared towards such use. The major stumbling block is that
>> Varnish will fetch the entire file from the backend before starting
>> transmission to the client.
>
>Have you considered allowing a configuration for streaming a file to
>a client while it's being fetched from the backend server?

Yes.

Do I have a sponsor for it ? No.


--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

I might also be interested.

> -----Message d'origine-----
> De : varnish-dev-bounces at projects.linpro.no
> [mailto:varnish-dev-bounces at projects.linpro.no] De la part de
> Adrian Otto
> Envoy? : lundi 16 juin 2008 17:28
> ? : varnish-dev at projects.linpro.no
> Objet : Re: about big flv files
>
> I'll proudly sponsor it, along with at least one other
> important feature I'm seeking. Simply contact me directly and
> we can make arrangements.
>
> Thanks,
>
> Adrian
>
> On Jun 16, 2008, at 8:28 AM, Poul-Henning Kamp wrote:
>
> > In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian
> > Otto writes
> > :
> >
> >>> There is no intrinsic limit on object size in varnish,
> but it is not
> >>> particular geared towards such use. The major stumbling block is
> >>> that Varnish will fetch the entire file from the backend before
> >>> starting transmission to the client.
> >>
> >> Have you considered allowing a configuration for streaming
> a file to
> >> a client while it's being fetched from the backend server?
> >
> > Yes.
> >
> > Do I have a sponsor for it ? No.
> >
> >
> > --
> > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> > phk at FreeBSD.ORG | TCP/IP since RFC 956
> > FreeBSD committer | BSD since 4.3-tahoe
> > Never attribute to malice what can adequately be explained by
> > incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>

I might also be interested.

> -----Message d'origine-----
> De : varnish-dev-bounces at projects.linpro.no
> [mailto:varnish-dev-bounces at projects.linpro.no] De la part de
> Adrian Otto
> Envoy? : lundi 16 juin 2008 17:28
> ? : varnish-dev at projects.linpro.no
> Objet : Re: about big flv files
>
> I'll proudly sponsor it, along with at least one other
> important feature I'm seeking. Simply contact me directly and
> we can make arrangements.
>
> Thanks,
>
> Adrian
>
> On Jun 16, 2008, at 8:28 AM, Poul-Henning Kamp wrote:
>
> > In message <06F9AFC0-24C5-4315-9B22-F7F8EE1913C4 at mosso.com>, Adrian
> > Otto writes
> > :
> >
> >>> There is no intrinsic limit on object size in varnish,
> but it is not
> >>> particular geared towards such use. The major stumbling block is
> >>> that Varnish will fetch the entire file from the backend before
> >>> starting transmission to the client.
> >>
> >> Have you considered allowing a configuration for streaming
> a file to
> >> a client while it's being fetched from the backend server?
> >
> > Yes.
> >
> > Do I have a sponsor for it ? No.
> >
> >
> > --
> > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> > phk at FreeBSD.ORG | TCP/IP since RFC 956
> > FreeBSD committer | BSD since 4.3-tahoe
> > Never attribute to malice what can adequately be explained by
> > incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev
>