Mailing List Archive: Replays from Internet

Replays from Internet

Jan 19, 2021, 8:55 AM

Post #1 of 8 (476 views)

Since the beginning of 2021 we have encountered two online orders and
possibly a third, where the customer denies making the order and the
httpd log seems to confirm that.

In each case, the person made an order and a day or more later a
second order was placed for the same item and carrying the same credit
card information. Since everything looked valid and the delay
bypassed our duplicate order check, the order was accepted.

Some background: a customer can connect to our catalogue and move
around untracked for as long as they want until they decide to place
an order. At this point there is only one path to follow to enter
address info, credit card, etc. This ends with a summary of the order
and if they click to proceed, it POST's the server order processor
with the relevant info causing the credit card to be charged and the
order to be entered. In total 3 scripts must be processed in the
correct order.

I scanned for the customer's IP in the httpd access log in each case
and found that when they made the valid order they were on our
catalogue and followed the correct path to place the order, confirming
it as expected.

BUT, and here is what I am having trouble understanding, for the
invalid order ONLY the last request was logged as received by httpd.
It shows the correct source (ie the page that should have resulted in
an order) yet that page does not show in the httpd log as having been
served. In one case, NO other page was served to that customer on
that day ahead of the received order, at least judging from IP
addresses in use.

So what I appear to be seeing is a replay from the Internet which I
find hard to accept as real. Has anyone ever seen this before and if
so what did they do to resolve it? The only other possibility that I
can think of is that their browser cached the page and re-transmitted
it. (a violation of the HTML standard I think for a form page).

The environment is Apache 2.4.25 on Fedora using php-fpm.

Thanks in advance and apologies for the length of this post.

John

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

mrbrklyn at panix

Jan 19, 2021, 9:00 AM

Post #2 of 8 (476 views)

Permalink

this has nothing to do with apache

On Tue, Jan 19, 2021 at 11:55:41AM -0500, John wrote:
> Since the beginning of 2021 we have encountered two online orders and
> possibly a third, where the customer denies making the order and the
> httpd log seems to confirm that.
>
> In each case, the person made an order and a day or more later a
> second order was placed for the same item and carrying the same credit
> card information. Since everything looked valid and the delay
> bypassed our duplicate order check, the order was accepted.
>
> Some background: a customer can connect to our catalogue and move
> around untracked for as long as they want until they decide to place
> an order. At this point there is only one path to follow to enter
> address info, credit card, etc. This ends with a summary of the order
> and if they click to proceed, it POST's the server order processor
> with the relevant info causing the credit card to be charged and the
> order to be entered. In total 3 scripts must be processed in the
> correct order.
>
> I scanned for the customer's IP in the httpd access log in each case
> and found that when they made the valid order they were on our
> catalogue and followed the correct path to place the order, confirming
> it as expected.
>
> BUT, and here is what I am having trouble understanding, for the
> invalid order ONLY the last request was logged as received by httpd.
> It shows the correct source (ie the page that should have resulted in
> an order) yet that page does not show in the httpd log as having been
> served. In one case, NO other page was served to that customer on
> that day ahead of the received order, at least judging from IP
> addresses in use.
>
> So what I appear to be seeing is a replay from the Internet which I
> find hard to accept as real. Has anyone ever seen this before and if
> so what did they do to resolve it? The only other possibility that I
> can think of is that their browser cached the page and re-transmitted
> it. (a violation of the HTML standard I think for a form page).
>
> The environment is Apache 2.4.25 on Fedora using php-fpm.
>
> Thanks in advance and apologies for the length of this post.
>
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

Antony.Stone at apache

Jan 19, 2021, 9:05 AM

Post #3 of 8 (476 views)

Permalink

On Tuesday 19 January 2021 at 18:00:11, Ruben Safir wrote:

> this has nothing to do with apache

I think that's a somewhat harsh way of putting it, but I do agree that since
"that page does not show in the httpd log as having been served" you are
correct, and the problem lies elsewhere. I would suggest looking at any
database logs for transactions made, to see whether that shows where the
duplicate order updates came from.

> On Tue, Jan 19, 2021 at 11:55:41AM -0500, John wrote:
> > Since the beginning of 2021 we have encountered two online orders and
> > possibly a third, where the customer denies making the order and the
> > httpd log seems to confirm that.
> >
> > In each case, the person made an order and a day or more later a
> > second order was placed for the same item and carrying the same credit
> > card information. Since everything looked valid and the delay
> > bypassed our duplicate order check, the order was accepted.
> >
> > Some background: a customer can connect to our catalogue and move
> > around untracked for as long as they want until they decide to place
> > an order. At this point there is only one path to follow to enter
> > address info, credit card, etc. This ends with a summary of the order
> > and if they click to proceed, it POST's the server order processor
> > with the relevant info causing the credit card to be charged and the
> > order to be entered. In total 3 scripts must be processed in the
> > correct order.
> >
> > I scanned for the customer's IP in the httpd access log in each case
> > and found that when they made the valid order they were on our
> > catalogue and followed the correct path to place the order, confirming
> > it as expected.
> >
> > BUT, and here is what I am having trouble understanding, for the
> > invalid order ONLY the last request was logged as received by httpd.
> > It shows the correct source (ie the page that should have resulted in
> > an order) yet that page does not show in the httpd log as having been
> > served. In one case, NO other page was served to that customer on
> > that day ahead of the received order, at least judging from IP
> > addresses in use.
> >
> > So what I appear to be seeing is a replay from the Internet which I
> > find hard to accept as real. Has anyone ever seen this before and if
> > so what did they do to resolve it? The only other possibility that I
> > can think of is that their browser cached the page and re-transmitted
> > it. (a violation of the HTML standard I think for a form page).
> >
> > The environment is Apache 2.4.25 on Fedora using php-fpm.
> >
> > Thanks in advance and apologies for the length of this post.

Regards,

Antony.

--
"Black holes are where God divided by zero."

- Steven Wright

Please reply to the list;
please *don't* CC me.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

yehuda at ymkatz

Jan 19, 2021, 9:06 AM

Post #4 of 8 (476 views)

Permalink

It is likely that the user still has the browser tab open and it refreshes
for some reason, or their browser crashed and reopened, or they use a
session restore tool when they reopen the browser - all of these could
cause an exact duplicate request.
You might want to look into the Post-Redirect-Get pattern.

In a nutshell:
When a POST request completes without any errors, never return any output
except a redirect to a completion page. The browser will make a new GET
request to this page and any POST data will not be included.

This will prevent a form that was submitted successfully from ever being
resubmitted accidentally.

On Tue, Jan 19, 2021 at 12:00 PM John <john.iliffe@iliffe.ca> wrote:

> Since the beginning of 2021 we have encountered two online orders and
> possibly a third, where the customer denies making the order and the
> httpd log seems to confirm that.
>
> In each case, the person made an order and a day or more later a
> second order was placed for the same item and carrying the same credit
> card information. Since everything looked valid and the delay
> bypassed our duplicate order check, the order was accepted.
>
> Some background: a customer can connect to our catalogue and move
> around untracked for as long as they want until they decide to place
> an order. At this point there is only one path to follow to enter
> address info, credit card, etc. This ends with a summary of the order
> and if they click to proceed, it POST's the server order processor
> with the relevant info causing the credit card to be charged and the
> order to be entered. In total 3 scripts must be processed in the
> correct order.
>
> I scanned for the customer's IP in the httpd access log in each case
> and found that when they made the valid order they were on our
> catalogue and followed the correct path to place the order, confirming
> it as expected.
>
> BUT, and here is what I am having trouble understanding, for the
> invalid order ONLY the last request was logged as received by httpd.
> It shows the correct source (ie the page that should have resulted in
> an order) yet that page does not show in the httpd log as having been
> served. In one case, NO other page was served to that customer on
> that day ahead of the received order, at least judging from IP
> addresses in use.
>
> So what I appear to be seeing is a replay from the Internet which I
> find hard to accept as real. Has anyone ever seen this before and if
> so what did they do to resolve it? The only other possibility that I
> can think of is that their browser cached the page and re-transmitted
> it. (a violation of the HTML standard I think for a form page).
>
> The environment is Apache 2.4.25 on Fedora using php-fpm.
>
> Thanks in advance and apologies for the length of this post.
>
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>

Re: Replays from Internet [ In reply to ]

john.iliffe at iliffe

Jan 19, 2021, 9:15 AM

Post #5 of 8 (476 views)

Permalink

Yes, I would agree that it isn't Apache but where else would I find a
group of people with extensive experience with web interfaces.

Our transaction logs and order entry system show these false orders
but that is what I would expect since a perfectly consistent order was
placed online. The problem, so far as I can see, is that the requests
were replayed externally since we first encounter them as a received
packet to httpd. From there our internal systems just processed what
they had.

What is of most interest to me is whether anyone has ever encountered
this sort of situation and if so,how they resolved it.

Regards, and thank you for the prompt reply Antony.

John
==================

On Tue, 2021-01-19 at 18:05 +0100, Antony Stone wrote:
> On Tuesday 19 January 2021 at 18:00:11, Ruben Safir wrote:
>
> > this has nothing to do with apache
>
> I think that's a somewhat harsh way of putting it, but I do agree
> that since
> "that page does not show in the httpd log as having been served" you
> are
> correct, and the problem lies elsewhere. I would suggest looking at
> any
> database logs for transactions made, to see whether that shows where
> the
> duplicate order updates came from.
>
> > On Tue, Jan 19, 2021 at 11:55:41AM -0500, John wrote:
> > > Since the beginning of 2021 we have encountered two online
> orders and
> > > possibly a third, where the customer denies making the order and
> the
> > > httpd log seems to confirm that.
> > >
> > > In each case, the person made an order and a day or more later a
> > > second order was placed for the same item and carrying the same
> credit
> > > card information. Since everything looked valid and the delay
> > > bypassed our duplicate order check, the order was accepted.
> > >
> > > Some background: a customer can connect to our catalogue and
> move
> > > around untracked for as long as they want until they decide to
> place
> > > an order. At this point there is only one path to follow to
> enter
> > > address info, credit card, etc. This ends with a summary of the
> order
> > > and if they click to proceed, it POST's the server order
> processor
> > > with the relevant info causing the credit card to be charged and
> the
> > > order to be entered. In total 3 scripts must be processed in the
> > > correct order.
> > >
> > > I scanned for the customer's IP in the httpd access log in each
> case
> > > and found that when they made the valid order they were on our
> > > catalogue and followed the correct path to place the order,
> confirming
> > > it as expected.
> > >
> > > BUT, and here is what I am having trouble understanding, for the
> > > invalid order ONLY the last request was logged as received by
> httpd.
> > > It shows the correct source (ie the page that should have
> resulted in
> > > an order) yet that page does not show in the httpd log as having
> been
> > > served. In one case, NO other page was served to that customer
> on
> > > that day ahead of the received order, at least judging from IP
> > > addresses in use.
> > >
> > > So what I appear to be seeing is a replay from the Internet
> which I
> > > find hard to accept as real. Has anyone ever seen this before
> and if
> > > so what did they do to resolve it? The only other possibility
> that I
> > > can think of is that their browser cached the page and re-
> transmitted
> > > it. (a violation of the HTML standard I think for a form page).
> > >
> > > The environment is Apache 2.4.25 on Fedora using php-fpm.
> > >
> > > Thanks in advance and apologies for the length of this post.
>
> Regards,
>
>
> Antony.
>
> --
> "Black holes are where God divided by zero."
>
> - Steven Wright
>
> Please reply to
> the list;
> please
> *don't* CC me.
>
> --------------------------------------------------------------------
> -
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

john.iliffe at iliffe

Jan 19, 2021, 9:28 AM

Post #6 of 8 (476 views)

Permalink

Thanks Yehuda, I think you may have caught it.

What happens is that the order entry page returns with an attached pdf
copy of the invoice but this is not a redirect, it is the same page
that they POST'd. The customer then has to click "finished" to get
back to the web site.

So I learned something I didn't know.

Thank you.

John
============================
On Tue, 2021-01-19 at 12:06 -0500, Yehuda Katz wrote:
> It is likely that the user still has the browser tab open and it
> refreshes for some reason, or their browser crashed and reopened, or
> they use a session restore tool when they reopen the browser - all
> of these could cause an exact duplicate request.
> You might want to look into the Post-Redirect-Get pattern.
>
> In a nutshell:
> When a POST request completes without any errors, never return any
> output except a redirect to a completion page. The browser will make
> a new GET request to this page and any POST data will not be
> included.
>
> This will prevent a form that was submitted successfully from ever
> being resubmitted accidentally.
>
> On Tue, Jan 19, 2021 at 12:00 PM John <john.iliffe@iliffe.ca> wrote:
> > Since the beginning of 2021 we have encountered two online orders
> > and
> > possibly a third, where the customer denies making the order and
> > the
> > httpd log seems to confirm that.
> >
> > In each case, the person made an order and a day or more later a
> > second order was placed for the same item and carrying the same
> > credit
> > card information. Since everything looked valid and the delay
> > bypassed our duplicate order check, the order was accepted.
> >
> > Some background: a customer can connect to our catalogue and move
> > around untracked for as long as they want until they decide to
> > place
> > an order. At this point there is only one path to follow to enter
> > address info, credit card, etc. This ends with a summary of the
> > order
> > and if they click to proceed, it POST's the server order processor
> > with the relevant info causing the credit card to be charged and
> > the
> > order to be entered. In total 3 scripts must be processed in the
> > correct order.
> >
> > I scanned for the customer's IP in the httpd access log in each
> > case
> > and found that when they made the valid order they were on our
> > catalogue and followed the correct path to place the order,
> > confirming
> > it as expected.
> >
> > BUT, and here is what I am having trouble understanding, for the
> > invalid order ONLY the last request was logged as received by
> > httpd.
> > It shows the correct source (ie the page that should have resulted
> > in
> > an order) yet that page does not show in the httpd log as having
> > been
> > served. In one case, NO other page was served to that customer on
> > that day ahead of the received order, at least judging from IP
> > addresses in use.
> >
> > So what I appear to be seeing is a replay from the Internet which
> > I
> > find hard to accept as real. Has anyone ever seen this before and
> > if
> > so what did they do to resolve it? The only other possibility
> > that I
> > can think of is that their browser cached the page and re-
> > transmitted
> > it. (a violation of the HTML standard I think for a form page).
> >
> > The environment is Apache 2.4.25 on Fedora using php-fpm.
> >
> > Thanks in advance and apologies for the length of this post.
> >
> > John
> >
> >
> > ------------------------------------------------------------------
> > ---
> > To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> > For additional commands, e-mail: users-help@httpd.apache.org
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

niq at apache

Jan 20, 2021, 1:53 AM

Post #7 of 8 (474 views)

Permalink

> On 19 Jan 2021, at 16:55, John <john.iliffe@iliffe.ca> wrote:
>
> I scanned for the customer's IP

What leads you to suppose the customer's IP was unchanged between
the two orders?

Sorry, there's no sensible explanation: the above is the only comment
I can offer at a technical level. Maybe the customer is messing you
around, or your system is duplicating entries offline. Unless you've
got yourself hopelessly confused and need a good night's sleep and a
day off to clear the head.

--
Nick Kew

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org

Re: Replays from Internet [ In reply to ]

john.iliffe at iliffe

Jan 20, 2021, 7:57 AM

Post #8 of 8 (474 views)

Permalink

Hi Nick:

Yes, I could use a good night's sleep! I'm in the process of
implementing the suggestions from Yehuda and James that I got
yesterday. I had been thinking of something along that line but I
also wanted to know why this turned up twice now after 12 years of
running basically the same functional code. I'm curious!

Re the IP address, well I only have limited information because at the
point where this happens the customer has not completed the order so I
don't have the invoice number which serves as a key. After the order
is processed I have the IP address from whence it came. So, I scanned
the Apache access log on the theory that a single order will come from
one IP address, that is select, enter address info, enter credit card
info, confirm, which is 4 separate scripts, each a form/POST sequence
that gets gathered up as it goes along and the whole data collection
gets passed as a single transaction to the order entry script.

What I found was the first order processed exactly in the sequence
that one would expect: script 1, script 2,...etc. But the replay
only contained a single call to the server, on the order entry script,
but with the correct and complete data and showing the reason for the
call on this script as the previous script in the sequence. This
previous script was never called, or at least not logged by Apache if
it was. So, I figured it had to be a replay of the final call and that
is what I was asking about.

The answers I got seem to cover a probable reason and so that is what
I am currently changing and updating.

Regards, and thank you for taking the time to reply.

John
===================
On Wed, 2021-01-20 at 09:53 +0000, Nick Kew wrote:
> > On 19 Jan 2021, at 16:55, John <john.iliffe@iliffe.ca> wrote:
> >
> > I scanned for the customer's IP
>
> What leads you to suppose the customer's IP was unchanged between
> the two orders?
>
> Sorry, there's no sensible explanation: the above is the only
> comment
> I can offer at a technical level. Maybe the customer is messing you
> around, or your system is duplicating entries offline. Unless
> you've
> got yourself hopelessly confused and need a good night's sleep and a
> day off to clear the head.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org