Mailing List Archive: Rationale behind different maxDocsPerChunk in Lucene90CompressingStoredFieldsFormat

Rationale behind different maxDocsPerChunk in Lucene90CompressingStoredFieldsFormat

Jul 30, 2021, 1:40 AM

Post #1 of 3 (243 views)

Hello,

What is the rationale behind choosing different maxDocsPerChunk in
Lucene90CompressingStoredFieldsFormat? I found it to be 1024 in case of lz4
while 4096 in case of zlib. Is it an experimented value established from
benchmark or something to do with 60KB or 48KB block size ?

Thanks,
Praveen

Re: Rationale behind different maxDocsPerChunk in Lucene90CompressingStoredFieldsFormat [ In reply to ]

pravs.nishchal at gmail

Aug 9, 2021, 9:14 AM

Post #2 of 3 (240 views)

Permalink

Any thoughts on maxDocsPerChunk <-> chunkSize relation ?

On Fri, Jul 30, 2021 at 2:10 PM Praveen Nishchal <pravs.nishchal@gmail.com>
wrote:

> Hello,
>
> What is the rationale behind choosing different maxDocsPerChunk in
> Lucene90CompressingStoredFieldsFormat? I found it to be 1024 in case of lz4
> while 4096 in case of zlib. Is it an experimented value established from
> benchmark or something to do with 60KB or 48KB block size ?
>
> Thanks,
> Praveen
>

Re: Rationale behind different maxDocsPerChunk in Lucene90CompressingStoredFieldsFormat [ In reply to ]

jpountz at gmail

Aug 10, 2021, 12:35 AM

Post #3 of 3 (240 views)

Permalink

I left a comment on the other thread where you asked a similar question.

On Mon, Aug 9, 2021 at 6:14 PM Praveen Nishchal <pravs.nishchal@gmail.com>
wrote:

> Any thoughts on maxDocsPerChunk <-> chunkSize relation ?
>
> On Fri, Jul 30, 2021 at 2:10 PM Praveen Nishchal <pravs.nishchal@gmail.com>
> wrote:
>
>> Hello,
>>
>> What is the rationale behind choosing different maxDocsPerChunk in
>> Lucene90CompressingStoredFieldsFormat? I found it to be 1024 in case of lz4
>> while 4096 in case of zlib. Is it an experimented value established from
>> benchmark or something to do with 60KB or 48KB block size ?
>>
>> Thanks,
>> Praveen
>>
>

--
Adrien