Mailing List Archive

Websocket's better support
Hi Varnish dev & users,

Websocket is (still) not dying, and for us, it seems we have to handle more and more this kind of traffic.
It's why I would like to open a discussion and see how we can enhance the websocket support in Varnish.

Currently, the implementation is done thru pipe like this :

cli (Upgrade) -> recv -> pipe <--> backend

So basically, we're putting the ball into backend's hands and nothing else.
It has several disadvantages; one of them is that we cannot interact with the handshake's response which is still in HTTP/1.1.
E.g. adding a set-cookie for stickiness is not possible.
E.g. testing if status code is 101 is not possible

So, first problem, first question: do you think it is possible to allow opening the pipe after vcl_backend_response ?
The flow will be:

cli (Upgrade) -> recv -> pass -> b_fetch -> b_response -> pipe <---> backend

Only this extra step would be a huge improvement of what we can do on websockets connections and will be very beneficial.


Then, a bonus question, cuz it requires probably much more time:
I was wondering if later, websocket protocol can be integrated to the core (e.g. similarly to HTTP2?), to have benefits of metrics, params and logs.
Because, No, websocket messages are not just "bytes thru a pipe" :-)


Thanks for your answers,

--
Jonathan Huot
Thomson Reuters

________________________________

This e-mail is for the sole use of the intended recipient and contains information that may be privileged and/or confidential. If you are not an intended recipient, please notify the sender by return e-mail and delete this e-mail and any attachments. Certain required legal entity disclosures can be accessed on our website.<http://site.thomsonreuters.com/site/disclosures/>

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Websocket's better support [ In reply to ]
On Thu, Apr 20, 2017 at 9:57 AM, <jonathan.huot@thomsonreuters.com> wrote:
> Hi Varnish dev & users,
>
> Websocket is (still) not dying, and for us, it seems we have to handle more and more this kind of traffic.
> It's why I would like to open a discussion and see how we can enhance the websocket support in Varnish.
>
> Currently, the implementation is done thru pipe like this :
>
> cli (Upgrade) -> recv -> pipe <--> backend
>
> So basically, we're putting the ball into backend's hands and nothing else.
> It has several disadvantages; one of them is that we cannot interact with the handshake's response which is still in HTTP/1.1.
> E.g. adding a set-cookie for stickiness is not possible.
> E.g. testing if status code is 101 is not possible
>
> So, first problem, first question: do you think it is possible to allow opening the pipe after vcl_backend_response ?
> The flow will be:
>
> cli (Upgrade) -> recv -> pass -> b_fetch -> b_response -> pipe <---> backend

It's not, but it's not the first time we discussed it:

https://github.com/varnishcache/varnish-cache/wiki/VIP8%3A-No-pipe-in-builtin.vcl-in-V5#discussion

See around 13:41.

You can emulate an "Expect: 100-continue" dance with the backend to at
least make sure an upgrade would be allowed before actually piping:

varnishtest "websocket example"

# Varnish won't let you use 100-continue so we need an ad-hoc solution
# to ask the backend whether websocket is acceptable beforehand.

server s1 {
rxreq
expect req.http.Connection == upgrade
expect req.http.Upgrade == websocket
expect req.http.Expect == 200-ok
txresp

rxreq
expect req.http.Connection == upgrade
expect req.http.Upgrade == websocket
expect req.http.Expect == <undef>
txresp -status 101 \
-hdr "Connection: upgrade" \
-hdr "Upgrade: websocket"

send "pretend we do websocket here"
} -start

varnish v1 -vcl+backend {
sub vcl_recv {
if (req.restarts == 0) {
unset req.http.Pipe;
unset req.http.X-Upgrade;
}
if (req.http.Pipe) {
set req.http.Upgrade = req.http.X-Upgrade;
return (pipe);
}
elsif (req.http.Connection == "upgrade") {
set req.http.X-Upgrade = req.http.Upgrade;
return (pass);
}
}

sub vcl_backend_fetch {
if (bereq.http.X-Upgrade) {
set bereq.http.Connection = "upgrade";
set bereq.http.Upgrade = bereq.http.X-Upgrade;
set bereq.http.Expect = "200-ok";
unset bereq.http.X-Upgrade;
}
}

sub vcl_backend_response {
if (bereq.http.Upgrade && beresp.status == 200) {
set beresp.http.Pipe = bereq.http.Upgrade;
return (deliver);
}
}

sub vcl_deliver {
if (resp.http.Pipe) {
set req.http.Pipe = resp.http.Pipe;
return (restart);
}
}
} -start

client c1 {
txreq -hdr "Connection: upgrade" -hdr "Upgrade: websocket"
rxresp
expect resp.status == 101

# receive the fake websocket traffic
recv 28

# this will fail because there isn't anything left to read
# recv 1
} -run

This example is over-simplified, it only shows that you can act upon
websocket requests. That doesn't solve the fact that you don't have
access to the beresp once you return pipe.

> Only this extra step would be a huge improvement of what we can do on websockets connections and will be very beneficial.
>
>
> Then, a bonus question, cuz it requires probably much more time:
> I was wondering if later, websocket protocol can be integrated to the core (e.g. similarly to HTTP2?), to have benefits of metrics, params and logs.
> Because, No, websocket messages are not just "bytes thru a pipe" :-)

Well, websocket is not HTTP, unlilke... HTTP/2? I don't think it would
fit nicely in VCL, applications do what they want on top of the
session. Unlike HTTP that specifies what happens on the session, we
couldn't make more assumption than bytes passing through.

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
RE: Websocket's better support [ In reply to ]
On Thu, Apr 20, 2017 at 17:18 AM, <dridi@varni.sh> wrote:
> On Thu, Apr 20, 2017 at 9:57 AM, <jonathan.huot@thomsonreuters.com> wrote:
> > Websocket is (still) not dying, and for us, it seems we have to handle more and more this kind of traffic.
> > It's why I would like to open a discussion and see how we can enhance the websocket support in Varnish.
> >
> > Currently, the implementation is done thru pipe like this :
> >
> > cli (Upgrade) -> recv -> pipe <--> backend
> >
> > So basically, we're putting the ball into backend's hands and nothing else.
> > It has several disadvantages; one of them is that we cannot interact with the handshake's response which is still in HTTP/1.1.
> > E.g. adding a set-cookie for stickiness is not possible.
> > E.g. testing if status code is 101 is not possible
> >
> > So, first problem, first question: do you think it is possible to allow opening the pipe after vcl_backend_response ?
> > The flow will be:
> >
> > cli (Upgrade) -> recv -> pass -> b_fetch -> b_response -> pipe <--->
> > backend
>
> It's not, but it's not the first time we discussed it:
> https://github.com/varnishcache/varnish-cache/wiki/VIP8%3A-No-pipe-in-builtin.vcl-in-V5#discussion

Thanks for sharing this Varnish github "wiki", I was not aware of it.
So, if I'm collecting information from VIP6 and VIP8, I can summarize with:

VIP8
13:41 < dridi_> that's because we handle pipe backwards, we should only pipe when there's a 101 switching protocols, after a beresp, not a req
13:42 < phk> dridi_, relevant point there.

VIP6 : (..) it's the backend that announces the protocol switch, so getting a transition from vcl_recv to vcl_pipe makes little sense (..) the pipe transition should belong to the vcl_deliver (..)

It looks like we have a consensus to progress into this direction.
I'm working on an implementation for myself but I'm trying to see how it can have benefit for the most.

First, I'm not going to change any behavior of pipe in vcl_recv.
Then, for Websockets, I think the best place to call pipe is in vcl_deliver.
Then, I will relax Varnish to accept 1xx code (those responses are assumed without Body).
Then, either
we pipe in cnt_transmit after sending back resp to the client, or,
we create a new vcl func vcl_deliver_pipe which is the equivalent of vcl_pipe; however I don't see the benefit since we already have access to req/resp in vcl_deliver.

Then, technical questions arise:
- do we update the director API with a new hook http1deliverpipe, or do we update the current http1pipe to be used by both?
- about stats/logs, do we mix them or do we keep them separated?


Do you think it is an doable plan, useful for the overall community? Or do I continue on a fork without trying to make it merge-able ?

Thanks in advance,
Regards

--
Jonathan Huot
Thomson Reuters
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Websocket's better support [ In reply to ]
> It looks like we have a consensus to progress into this direction.

I wouldn't go as far as saying there's a consensus. If you are
interested in discussing this as a possibility you should attend next
bugwash on IRC (next Monday at 13h if you are in France like I'm
supposing).

I put it on the agenda because of this thread:

https://github.com/varnishcache/varnish-cache/issues/2318

> I'm working on an implementation for myself but I'm trying to see how it can have benefit for the most.
>
> First, I'm not going to change any behavior of pipe in vcl_recv.
> Then, for Websockets, I think the best place to call pipe is in vcl_deliver.
> Then, I will relax Varnish to accept 1xx code (those responses are assumed without Body).
> Then, either
> we pipe in cnt_transmit after sending back resp to the client, or,
> we create a new vcl func vcl_deliver_pipe which is the equivalent of vcl_pipe; however I don't see the benefit since we already have access to req/resp in vcl_deliver.
>
> Then, technical questions arise:
> - do we update the director API with a new hook http1deliverpipe, or do we update the current http1pipe to be used by both?
> - about stats/logs, do we mix them or do we keep them separated?

You should share your intentions during next bugwash if we manage to
cover that point. I expect this bugwash to be a long one (protip:
lunch at noon).

> Do you think it is an doable plan, useful for the overall community? Or do I continue on a fork without trying to make it merge-able ?

I don't reckon doing this on your own would be easy to merge.
Especially for such a breaking change. However it would probably work
better if you took part of the plan from day one. I really have no
idea so join the bugwash instead.

Cheers

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc