Mailing List Archive

Hi! Anton from Blunders here.

I will take a look as soon as possible, most likely I will be able to tell
what's going on from server logs. Thank you for reporting - I will put up
better monitoring in the future.

/Anton


On Fri, 17 Mar 2023, 19:49 Marc D'Mello, <marcd2000@gmail.com> wrote:

> Hi all,
>
> I was looking at some of the profiles on Blunders (which is linked from
> the nightly benchmarking site:
> https://home.apache.org/~mikemccand/lucenebench/) and it seems like some
> of the latest Searching profiles are not working. For example:
> https://blunders.io/jfr-demo/searching-2023.03.16.18.02.48/jvm_info. The
> indexing profiles seem to be working fine as far as I can tell, so I wonder
> if this is a problem with how the nightly benchmarks are publishing data to
> the Blunders API.
>
> Thanks,
> Marc
>

I've had a look now - it seems like the .jfr.gz files uploaded for these
search benchmarks are empty (they are well formed gzip files, there is just
no compressed data in them). Creation of these .jfr.gz files happens as
part of the benchmark setup, where I don't (to my knowledge) have access to
any logs.

Mike McCandless is probably the person best suited to dig into this, but
I'm here for any questions and will happily help debug the issue as much as
I can.

Blunders should also do a better job at saying "no data found" instead of
throwing an error as well. I will look into this.

Thank you
Anton

On Fri, 17 Mar 2023 at 20:17, Anton Hägerstrand <anton@blunders.io> wrote:

> Hi! Anton from Blunders here.
>
> I will take a look as soon as possible, most likely I will be able to tell
> what's going on from server logs. Thank you for reporting - I will put up
> better monitoring in the future.
>
> /Anton
>
>
> On Fri, 17 Mar 2023, 19:49 Marc D'Mello, <marcd2000@gmail.com> wrote:
>
>> Hi all,
>>
>> I was looking at some of the profiles on Blunders (which is linked from
>> the nightly benchmarking site:
>> https://home.apache.org/~mikemccand/lucenebench/) and it seems like some
>> of the latest Searching profiles are not working. For example:
>> https://blunders.io/jfr-demo/searching-2023.03.16.18.02.48/jvm_info. The
>> indexing profiles seem to be working fine as far as I can tell, so I wonder
>> if this is a problem with how the nightly benchmarks are publishing data to
>> the Blunders API.
>>
>> Thanks,
>> Marc
>>
>

Hmm, I'll try to figure out why the nightly benchy is uploading such
degenerate JFR zip files!

Mike McCandless

http://blog.mikemccandless.com


On Sat, Mar 18, 2023 at 5:16?AM Anton Hägerstrand <anton@blunders.io> wrote:

> I've had a look now - it seems like the .jfr.gz files uploaded for these
> search benchmarks are empty (they are well formed gzip files, there is just
> no compressed data in them). Creation of these .jfr.gz files happens as
> part of the benchmark setup, where I don't (to my knowledge) have access to
> any logs.
>
> Mike McCandless is probably the person best suited to dig into this, but
> I'm here for any questions and will happily help debug the issue as much as
> I can.
>
> Blunders should also do a better job at saying "no data found" instead of
> throwing an error as well. I will look into this.
>
> Thank you
> Anton
>
> On Fri, 17 Mar 2023 at 20:17, Anton Hägerstrand <anton@blunders.io> wrote:
>
>> Hi! Anton from Blunders here.
>>
>> I will take a look as soon as possible, most likely I will be able to
>> tell what's going on from server logs. Thank you for reporting - I will put
>> up better monitoring in the future.
>>
>> /Anton
>>
>>
>> On Fri, 17 Mar 2023, 19:49 Marc D'Mello, <marcd2000@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I was looking at some of the profiles on Blunders (which is linked from
>>> the nightly benchmarking site:
>>> https://home.apache.org/~mikemccand/lucenebench/) and it seems like
>>> some of the latest Searching profiles are not working. For example:
>>> https://blunders.io/jfr-demo/searching-2023.03.16.18.02.48/jvm_info.
>>> The indexing profiles seem to be working fine as far as I can tell, so I
>>> wonder if this is a problem with how the nightly benchmarks are
>>> publishing data to the Blunders API.
>>>
>>> Thanks,
>>> Marc
>>>
>>

OK I attempted a fix:
https://github.com/mikemccand/luceneutil/commit/2c8ccdf53e93622761a545c1a54377514c338caa

I think this broke at some point when we moved where the JFR files are
written...

Mike McCandless

http://blog.mikemccandless.com


On Tue, Apr 4, 2023 at 10:37?AM Michael McCandless <
lucene@mikemccandless.com> wrote:

> Hmm, I'll try to figure out why the nightly benchy is uploading such
> degenerate JFR zip files!
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Sat, Mar 18, 2023 at 5:16?AM Anton Hägerstrand <anton@blunders.io>
> wrote:
>
>> I've had a look now - it seems like the .jfr.gz files uploaded for these
>> search benchmarks are empty (they are well formed gzip files, there is just
>> no compressed data in them). Creation of these .jfr.gz files happens as
>> part of the benchmark setup, where I don't (to my knowledge) have access to
>> any logs.
>>
>> Mike McCandless is probably the person best suited to dig into this, but
>> I'm here for any questions and will happily help debug the issue as much as
>> I can.
>>
>> Blunders should also do a better job at saying "no data found" instead of
>> throwing an error as well. I will look into this.
>>
>> Thank you
>> Anton
>>
>> On Fri, 17 Mar 2023 at 20:17, Anton Hägerstrand <anton@blunders.io>
>> wrote:
>>
>>> Hi! Anton from Blunders here.
>>>
>>> I will take a look as soon as possible, most likely I will be able to
>>> tell what's going on from server logs. Thank you for reporting - I will put
>>> up better monitoring in the future.
>>>
>>> /Anton
>>>
>>>
>>> On Fri, 17 Mar 2023, 19:49 Marc D'Mello, <marcd2000@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I was looking at some of the profiles on Blunders (which is linked from
>>>> the nightly benchmarking site:
>>>> https://home.apache.org/~mikemccand/lucenebench/) and it seems like
>>>> some of the latest Searching profiles are not working. For example:
>>>> https://blunders.io/jfr-demo/searching-2023.03.16.18.02.48/jvm_info.
>>>> The indexing profiles seem to be working fine as far as I can tell, so I
>>>> wonder if this is a problem with how the nightly benchmarks are
>>>> publishing data to the Blunders API.
>>>>
>>>> Thanks,
>>>> Marc
>>>>
>>>

Thanks Mike - looks promising! I will have a look at the next upload.

Out of curiosity - what is the reason for using constants.NIGHTLY_LOG_DIR
instead of constants.LOGS_DIR, which seems to be what competition.py uses
to write the files? Is it that they are always the same value for the
nightly builds anyway?

thanks,
Anton

On Tue, 4 Apr 2023 at 16:48, Michael McCandless <lucene@mikemccandless.com>
wrote:

> OK I attempted a fix:
> https://github.com/mikemccand/luceneutil/commit/2c8ccdf53e93622761a545c1a54377514c338caa
>
> I think this broke at some point when we moved where the JFR files are
> written...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Apr 4, 2023 at 10:37?AM Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> Hmm, I'll try to figure out why the nightly benchy is uploading such
>> degenerate JFR zip files!
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Sat, Mar 18, 2023 at 5:16?AM Anton Hägerstrand <anton@blunders.io>
>> wrote:
>>
>>> I've had a look now - it seems like the .jfr.gz files uploaded for these
>>> search benchmarks are empty (they are well formed gzip files, there is just
>>> no compressed data in them). Creation of these .jfr.gz files happens as
>>> part of the benchmark setup, where I don't (to my knowledge) have access to
>>> any logs.
>>>
>>> Mike McCandless is probably the person best suited to dig into this, but
>>> I'm here for any questions and will happily help debug the issue as much as
>>> I can.
>>>
>>> Blunders should also do a better job at saying "no data found" instead
>>> of throwing an error as well. I will look into this.
>>>
>>> Thank you
>>> Anton
>>>
>>> On Fri, 17 Mar 2023 at 20:17, Anton Hägerstrand <anton@blunders.io>
>>> wrote:
>>>
>>>> Hi! Anton from Blunders here.
>>>>
>>>> I will take a look as soon as possible, most likely I will be able to
>>>> tell what's going on from server logs. Thank you for reporting - I will put
>>>> up better monitoring in the future.
>>>>
>>>> /Anton
>>>>
>>>>
>>>> On Fri, 17 Mar 2023, 19:49 Marc D'Mello, <marcd2000@gmail.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I was looking at some of the profiles on Blunders (which is linked
>>>>> from the nightly benchmarking site:
>>>>> https://home.apache.org/~mikemccand/lucenebench/) and it seems like
>>>>> some of the latest Searching profiles are not working. For example:
>>>>> https://blunders.io/jfr-demo/searching-2023.03.16.18.02.48/jvm_info.
>>>>> The indexing profiles seem to be working fine as far as I can tell, so I
>>>>> wonder if this is a problem with how the nightly benchmarks are
>>>>> publishing data to the Blunders API.
>>>>>
>>>>> Thanks,
>>>>> Marc
>>>>>
>>>>

On Tue, Apr 4, 2023 at 11:14?AM Anton Hägerstrand <anton@blunders.io> wrote:

> Thanks Mike - looks promising! I will have a look at the next upload.
>

Yay.

Out of curiosity - what is the reason for using constants.NIGHTLY_LOG_DIR
> instead of constants.LOGS_DIR, which seems to be what competition.py uses
> to write the files? Is it that they are always the same value for the
> nightly builds anyway?
>

Hmm good question ;) It really should be constants.LOGS_DIR -- that is
indeed more general. I'll fix. (It is indeed the same value for nightly
benchy).

Mike McCandless

http://blog.mikemccandless.com

>

Actually, I spoke too soon. The NIGHTLY_LOG_DIR is indeed a bit different
-- this is where the nightly benchy writes/reads all past nightly results,
generates charts from, etc.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Apr 4, 2023 at 11:50?AM Michael McCandless <
lucene@mikemccandless.com> wrote:

> On Tue, Apr 4, 2023 at 11:14?AM Anton Hägerstrand <anton@blunders.io>
> wrote:
>
>> Thanks Mike - looks promising! I will have a look at the next upload.
>>
>
> Yay.
>
> Out of curiosity - what is the reason for using constants.NIGHTLY_LOG_DIR
>> instead of constants.LOGS_DIR, which seems to be what competition.py uses
>> to write the files? Is it that they are always the same value for the
>> nightly builds anyway?
>>
>
> Hmm good question ;) It really should be constants.LOGS_DIR -- that is
> indeed more general. I'll fix. (It is indeed the same value for nightly
> benchy).
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>>

Search profiling seems to work again - see e.g.
https://blunders.io/jfr-demo/searching-2023.04.04.18.03.12/jvm_info

Thanks Marc for reporting and Mike for fixing!

/Anton


On Tue, 4 Apr 2023, 17:53 Michael McCandless, <lucene@mikemccandless.com>
wrote:

> Actually, I spoke too soon. The NIGHTLY_LOG_DIR is indeed a bit different
> -- this is where the nightly benchy writes/reads all past nightly results,
> generates charts from, etc.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Tue, Apr 4, 2023 at 11:50?AM Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> On Tue, Apr 4, 2023 at 11:14?AM Anton Hägerstrand <anton@blunders.io>
>> wrote:
>>
>>> Thanks Mike - looks promising! I will have a look at the next upload.
>>>
>>
>> Yay.
>>
>> Out of curiosity - what is the reason for using constants.NIGHTLY_LOG_DIR
>>> instead of constants.LOGS_DIR, which seems to be what competition.py uses
>>> to write the files? Is it that they are always the same value for the
>>> nightly builds anyway?
>>>
>>
>> Hmm good question ;) It really should be constants.LOGS_DIR -- that is
>> indeed more general. I'll fix. (It is indeed the same value for nightly
>> benchy).
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>>

Thank you Anton for bringing closure! And Marc for catching this! Phew :)

Mike McCandless

http://blog.mikemccandless.com


On Thu, Apr 6, 2023 at 5:51?AM Anton Hägerstrand <anton@blunders.io> wrote:

> Search profiling seems to work again - see e.g.
> https://blunders.io/jfr-demo/searching-2023.04.04.18.03.12/jvm_info
>
> Thanks Marc for reporting and Mike for fixing!
>
> /Anton
>
>
> On Tue, 4 Apr 2023, 17:53 Michael McCandless, <lucene@mikemccandless.com>
> wrote:
>
>> Actually, I spoke too soon. The NIGHTLY_LOG_DIR is indeed a bit
>> different -- this is where the nightly benchy writes/reads all past nightly
>> results, generates charts from, etc.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Tue, Apr 4, 2023 at 11:50?AM Michael McCandless <
>> lucene@mikemccandless.com> wrote:
>>
>>> On Tue, Apr 4, 2023 at 11:14?AM Anton Hägerstrand <anton@blunders.io>
>>> wrote:
>>>
>>>> Thanks Mike - looks promising! I will have a look at the next upload.
>>>>
>>>
>>> Yay.
>>>
>>> Out of curiosity - what is the reason for using
>>>> constants.NIGHTLY_LOG_DIR instead of constants.LOGS_DIR, which seems to be
>>>> what competition.py uses to write the files? Is it that they are always the
>>>> same value for the nightly builds anyway?
>>>>
>>>
>>> Hmm good question ;) It really should be constants.LOGS_DIR -- that is
>>> indeed more general. I'll fix. (It is indeed the same value for nightly
>>> benchy).
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>>>