Mailing List Archive

Combining many field as JSON and made it into a single field - Performance
I tried to combine multiple different fields (Stored Fields) into a JSON object

and added in to lucene document as single field

I get a better performance while reading the field from the index.

though constructing the JSON requires additional time, It still retrieve the data faster

What is the reason for this ?





I done update reading and indexing the data

while reading i convert them back to single fields

from JSON object





No. of Documents
No.of Fields
Run
With JSON & Compression
With JSON & No Compression
Without JSON &Compression
Without JSON & No Compression




Insert
Read
Update
Insert
Read
Update
Insert
Read
Update
Insert
Read
Update

100
10
One
216
35
38
167
55
59
216
23
65
215
25
57

100
10
Two
-
48
93
-
49
102

35
134

25
101

100
10
Three
-
45
80

49
98

37
117

24
106

1000
10
One
404
144
160
447
109
157
503
148
129
482
111
126

1000
10
Two
-
175
268
-
129
259

176
301

135
295

1000
10
Three
-
177
279

142
263

174
299

135
296

10000
100
One
1763
1209
414
2151
1182
357
1574
335
847
1845
321
834

10000
100
Two
-
1352
811
-
1246
643

378
1151

334
1142

10000
100
Three
-
1318
729

1252
701

369
1088

336
1158

100000
1000
One
81021
50172
39480
77690
46138
38554
79289
20623
89653
93558
19357
108458

100000
1000
Two
-
53523
34523

52027
42957

20125
88771

19357
111962

100000
1000
Three
-
53816
33460

51703
43643

20409
98932

19630
110352