Hi.
This is a bit of a different discussion, which is why I am now marking it OT.
I was quite impressed by the numbers you list below (and I still am, nevertheless).
So my first impression was that I am totally incompetent to comment further, because I
have never dealt with such numbers, and I have no idea of the kind of hardware/software
architecture which is needed to deal with that kind of thing.
But then I made some calculations, based on the number of servers which you mentioned
earlier, and that leads - at least on the surface - to a quite different view :
100 servers * 24 cores = 2400 cores
42,368,982 requests per day
On average thus :
= 423,689 per server / day
= 17,653 per core / day
= 735 per core / hour
= 12 per core / minute
= 5 s / request
(and that does not look so exceptional anymore - except for your servers budget)
Of course, I am sure that the kind of averages which I calculate above are very rough,
that the load on your systems is probably not evenly spread, that not all the time of
those 2400 cores is dedicated to this application, and so on.
But as a starting point, it provides at least one observation :
You have 42,368,982 requests per day, of which 5619 fail with a timeout, and that is
0,013% of the total.
And we know that the timeout of a HTTP client is normally in the range of 3-5 minutes.
Yet we see above that, as a very rough average, the mean time for a request is in the
order of 5 seconds.
So at least on the surface, it would look like the requests that fail, take at least
approximately 36 times longer (3 min * 60 = 180 s, divided by 5) than the average.
So are these 0,013% of requests really exceptional in how long they take to complete ?
And if yes, do you know why ?
Do you have any statistics which classify the requests in terms of how long they take ?
Like :
- between 1 and 5 seconds : n1
- between 6 and 15 seconds : n2
...
- more than nnn seconds : nx (subject to client timeout, so error in the log)
On 28.03.2018 13:13, PANG J. wrote:
> As shown below,
>
>
> Last day total requests are 42,368,982, not all are successful, but 42,362,363 are right.
>
> The failed requests are timeout.
>
> Thanks.
>
>
>
> On 2018/3/28 ??? PM 6:37, André Warnier (tomcat) wrote:
>> On 28.03.2018 12:31, PANG J. wrote:
>>> what the client I meant is mobile App.
>>> mobile App gets the result from server via SDK.
>>
>> Ok. But it is very likely that your "mobile app SDK", also has a timeout after it sends
>> a request to a server. Or are you /sure/ that it waits forever ?
>> /Precisely what/ makes you think that it is a server-side timeout ?
>>
>>> in future we may move the computing task into App itself.
>>> But currently they are running on server side.
>>>
>>> thanks.
>>>
>>> On 2018/3/28 ??? PM 6:11, André Warnier (tomcat) wrote:
>>>> I believe that the timeout which Pang J. is mentioning, may be the browser-side timeout,
>>>> which is fixed at the browser level at about 5 minutes or so.
>>>> When a browser sends a request to a server, and it does receive /some/ response within
>>>> the next +-5 minutes, then the browser will drop the connection to the server, and pop
>>>> up a message saying "sorry, the server appears not to respond.."
>>>> In other words, it is not a server timeout, it is a client timeout.
>>>> The only way to avoid this, is to insure that the server sends at least /some/ temporary
>>>> response to the client (*), regularly, so that this browser timeout does not occur.
>>>> Unfortunately, that is a bit more complicated to set up, than just some parameter
>>>> somewhere.
>>>> But there must be plenty of past discussions of this issue already on the www, and
>>>> solution guidelines.
>>
>
This is a bit of a different discussion, which is why I am now marking it OT.
I was quite impressed by the numbers you list below (and I still am, nevertheless).
So my first impression was that I am totally incompetent to comment further, because I
have never dealt with such numbers, and I have no idea of the kind of hardware/software
architecture which is needed to deal with that kind of thing.
But then I made some calculations, based on the number of servers which you mentioned
earlier, and that leads - at least on the surface - to a quite different view :
100 servers * 24 cores = 2400 cores
42,368,982 requests per day
On average thus :
= 423,689 per server / day
= 17,653 per core / day
= 735 per core / hour
= 12 per core / minute
= 5 s / request
(and that does not look so exceptional anymore - except for your servers budget)
Of course, I am sure that the kind of averages which I calculate above are very rough,
that the load on your systems is probably not evenly spread, that not all the time of
those 2400 cores is dedicated to this application, and so on.
But as a starting point, it provides at least one observation :
You have 42,368,982 requests per day, of which 5619 fail with a timeout, and that is
0,013% of the total.
And we know that the timeout of a HTTP client is normally in the range of 3-5 minutes.
Yet we see above that, as a very rough average, the mean time for a request is in the
order of 5 seconds.
So at least on the surface, it would look like the requests that fail, take at least
approximately 36 times longer (3 min * 60 = 180 s, divided by 5) than the average.
So are these 0,013% of requests really exceptional in how long they take to complete ?
And if yes, do you know why ?
Do you have any statistics which classify the requests in terms of how long they take ?
Like :
- between 1 and 5 seconds : n1
- between 6 and 15 seconds : n2
...
- more than nnn seconds : nx (subject to client timeout, so error in the log)
On 28.03.2018 13:13, PANG J. wrote:
> As shown below,
>
>
> Last day total requests are 42,368,982, not all are successful, but 42,362,363 are right.
>
> The failed requests are timeout.
>
> Thanks.
>
>
>
> On 2018/3/28 ??? PM 6:37, André Warnier (tomcat) wrote:
>> On 28.03.2018 12:31, PANG J. wrote:
>>> what the client I meant is mobile App.
>>> mobile App gets the result from server via SDK.
>>
>> Ok. But it is very likely that your "mobile app SDK", also has a timeout after it sends
>> a request to a server. Or are you /sure/ that it waits forever ?
>> /Precisely what/ makes you think that it is a server-side timeout ?
>>
>>> in future we may move the computing task into App itself.
>>> But currently they are running on server side.
>>>
>>> thanks.
>>>
>>> On 2018/3/28 ??? PM 6:11, André Warnier (tomcat) wrote:
>>>> I believe that the timeout which Pang J. is mentioning, may be the browser-side timeout,
>>>> which is fixed at the browser level at about 5 minutes or so.
>>>> When a browser sends a request to a server, and it does receive /some/ response within
>>>> the next +-5 minutes, then the browser will drop the connection to the server, and pop
>>>> up a message saying "sorry, the server appears not to respond.."
>>>> In other words, it is not a server timeout, it is a client timeout.
>>>> The only way to avoid this, is to insure that the server sends at least /some/ temporary
>>>> response to the client (*), regularly, so that this browser timeout does not occur.
>>>> Unfortunately, that is a bit more complicated to set up, than just some parameter
>>>> somewhere.
>>>> But there must be plenty of past discussions of this issue already on the www, and
>>>> solution guidelines.
>>
>
