Mailing List Archive

Statistics output limit at 5k objects
Looks like we just went over 5000 volumes in our cluster last night, and
now I am getting this error when collecting some statistics, which we do
via cron, and dump into a database:

pd-dd::*> statistics show -vserver pd -object volume -raw true -counter
read_ops|write_ops|other_ops|read_data|write_data
There are no entries matching your query.

Warning: Failed to get statistics for object volume: The number of
instances for this object exceeds the limit of 5000. Use the "-instance"
parameter to limit the number of instances.

It would appear 5000 is some sort of magic internal number, after which
CDOT will refuse to emit statistics.

Using the -instance parameter does nothting to help the situation, as that
requires specifying the individual volume you are looking for.

I looked around under 'statistics settings' and 'statistics preset' but
don't see any sort of control for this.

Any ideas on how to get around this 5k limit?

9.5p6 FWIW
AW: Statistics output limit at 5k objects [ In reply to ]
Hi Mike,

I do not have a definitive solution for your problem, but depending on the names of your volumes, you could run the output multiple times to get below the limit using the instance parameter (which also accepts wildcards).
If your volumes are called vol0001 – vol5001 f.ex., you could use these five calls to split the queries into 5 queries with max 1000 results:

statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol0*
statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol1*
statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol2*
statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol3*
statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol4*
statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data -instance vol5*

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com<http://www.anexia-it.com/>

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: toasters-bounces@teaparty.net <toasters-bounces@teaparty.net> Im Auftrag von Mike Thompson
Gesendet: Dienstag, 13. August 2019 20:43
An: toasters@teaparty.net Lists <toasters@teaparty.net>
Betreff: Statistics output limit at 5k objects


Looks like we just went over 5000 volumes in our cluster last night, and now I am getting this error when collecting some statistics, which we do via cron, and dump into a database:

pd-dd::*> statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data
There are no entries matching your query.

Warning: Failed to get statistics for object volume: The number of instances for this object exceeds the limit of 5000. Use the "-instance" parameter to limit the number of instances.

It would appear 5000 is some sort of magic internal number, after which CDOT will refuse to emit statistics.

Using the -instance parameter does nothting to help the situation, as that requires specifying the individual volume you are looking for.

I looked around under 'statistics settings' and 'statistics preset' but don't see any sort of control for this.

Any ideas on how to get around this 5k limit?

9.5p6 FWIW
Re: Statistics output limit at 5k objects [ In reply to ]
Yeah I may need to loop through stuff like that, assuming there's no
control hidden somewhere for that limit.

I'll open a case with support on it.

Thanks for the idea Alexander!


On Tue, Aug 13, 2019 at 8:55 PM Alexander Griesser <AGriesser@anexia-it.com>
wrote:

> Hi Mike,
>
>
>
> I do not have a definitive solution for your problem, but depending on the
> names of your volumes, you could run the output multiple times to get below
> the limit using the instance parameter (which also accepts wildcards).
>
> If your volumes are called vol0001 – vol5001 f.ex., you could use these
> five calls to split the queries into 5 queries with max 1000 results:
>
>
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol0*
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol1*
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol2*
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol3*
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol4*
>
> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data -instance vol5*
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> *Von:* toasters-bounces@teaparty.net <toasters-bounces@teaparty.net> *Im
> Auftrag von *Mike Thompson
> *Gesendet:* Dienstag, 13. August 2019 20:43
> *An:* toasters@teaparty.net Lists <toasters@teaparty.net>
> *Betreff:* Statistics output limit at 5k objects
>
>
>
>
>
> Looks like we just went over 5000 volumes in our cluster last night, and
> now I am getting this error when collecting some statistics, which we do
> via cron, and dump into a database:
>
>
>
> pd-dd::*> statistics show -vserver pd -object volume -raw true -counter
> read_ops|write_ops|other_ops|read_data|write_data
> There are no entries matching your query.
>
> Warning: Failed to get statistics for object volume: The number of
> instances for this object exceeds the limit of 5000. Use the "-instance"
> parameter to limit the number of instances.
>
>
>
> It would appear 5000 is some sort of magic internal number, after which
> CDOT will refuse to emit statistics.
>
>
>
> Using the -instance parameter does nothting to help the situation, as that
> requires specifying the individual volume you are looking for.
>
>
>
> I looked around under 'statistics settings' and 'statistics preset' but
> don't see any sort of control for this.
>
>
>
> Any ideas on how to get around this 5k limit?
>
>
>
> 9.5p6 FWIW
>
>
>
>
>
>
>
Re: Statistics output limit at 5k objects [ In reply to ]
I know of the SNMP issues with older cDot and 7-mode versions, but I wonder if using snmp might be a better way here? Especially if you’re doing your own processing to load the data into a dB.

I’m also wondering if there was a way to use more qtrees instead of volumes, but that too opens another can of worms.

And have you looked at oncommand with the performance tool? I suspect it’s not working for you too, since I don’t find it all the useful or easy to drill down into at times.




Sent from my iPhone

> On Aug 13, 2019, at 8:43 PM, Mike Thompson <mike.thompson@gmail.com> wrote:
>
>
> Looks like we just went over 5000 volumes in our cluster last night, and now I am getting this error when collecting some statistics, which we do via cron, and dump into a database:
>
> pd-dd::*> statistics show -vserver pd -object volume -raw true -counter read_ops|write_ops|other_ops|read_data|write_data
> There are no entries matching your query.
>
> Warning: Failed to get statistics for object volume: The number of instances for this object exceeds the limit of 5000. Use the "-instance" parameter to limit the number of instances.
>
> It would appear 5000 is some sort of magic internal number, after which CDOT will refuse to emit statistics.
>
> Using the -instance parameter does nothting to help the situation, as that requires specifying the individual volume you are looking for.
>
> I looked around under 'statistics settings' and 'statistics preset' but don't see any sort of control for this.
>
> Any ideas on how to get around this 5k limit?
>
> 9.5p6 FWIW
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters