Mailing List Archive

Hi Tony,

Have you looked at the faceting demo package [1]?

The faceting documentation that lives there [2] was updated last year
(thanks
@epotyom!) and there haven't been major API changes since.

The `SimpleSortedSetFacetsExample` [3] might be a good starting point for
what
you're trying to do.

If you've already checked these resources, is there a specific question they
didn't help answer?


Stefan


[1]
https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/apache/lucene/demo/facet
[2]
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/package-info.java
[3]
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.java

On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid> wrote:

> Hello,
>
>
>
> Is there a definitive (or at least very good) documentation or video on
> Lucene faceting? I find the existing docs and samples out-dated and
> inaccurate. As such, I'm having trouble getting my mind around how it
> works
> to ensure I index my documents in such a way as to allow faceting. I'm
> trying to avoid the "taxonomy-based" approach.
>
>
>
> Tony
>
>

I have a lot of questions. For example, the javadocs for the "KeywordField" class says: "... doc values for sorting and faceting"

Yet, I don't see a way with 9.8.x Lucene to perform faceting on fields that haven't been specifically indexed for faceting. I clearly need a lesson or two on faceting in Lucene. I don't understand what a "path" is for example. Of course I have tried to work with the examples, but unless I completely rework my indexes, I'm stuck. I thought you could use DocValues somehow to facilitate faceting queries. Anyway, I suppose I'll need to use the source code as my guide and spend a lot of time working with it. I was hoping to find some really good documentation to help me cut to the chase. I appreciate you offering to help answer any questions, seems like I could use an interactive session ????

Thanks!

Tony


-----Original Message-----
From: Stefan Vodita <stefan.vodita@gmail.com>
Sent: Tuesday, November 14, 2023 19:14
To: tony@xfire.io.invalid
Cc: java-user@lucene.apache.org
Subject: Re: Faceting Queries NON-Taxonomy-based

Hi Tony,

Have you looked at the faceting demo package [1]?

The faceting documentation that lives there [2] was updated last year (thanks
@epotyom!) and there haven't been major API changes since.

The `SimpleSortedSetFacetsExample` [3] might be a good starting point for what you're trying to do.

If you've already checked these resources, is there a specific question they didn't help answer?


Stefan


[1]
https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/apache/lucene/demo/facet
[2]
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/package-info.java
[3]
https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.java

On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid> wrote:

> Hello,
>
>
>
> Is there a definitive (or at least very good) documentation or video
> on Lucene faceting? I find the existing docs and samples out-dated
> and inaccurate. As such, I'm having trouble getting my mind around
> how it works to ensure I index my documents in such a way as to allow
> faceting. I'm trying to avoid the "taxonomy-based" approach.
>
>
>
> Tony
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Hi Tony,

I put together a demo for faceting using `KeywordField` [1]. Hopefully that
answers some of your questions.

A path would be a sequence of hierarchical labels that a document belongs
to.
For example, a book could be published on a certain day, of a certain month,
of a certain year, making for a path like `Publish Date/2023/11/16`. When
faceting, you could get counts with respect to each of the labels in the
path
(e.g. counts per year or counts per month of given year).


Stefan


[1] https://github.com/apache/lucene/pull/12817

On Wed, 15 Nov 2023 at 01:01, Tony Schwartz <tony@xfire.io.invalid> wrote:

> I have a lot of questions. For example, the javadocs for the
> "KeywordField" class says: "... doc values for sorting and faceting"
>
> Yet, I don't see a way with 9.8.x Lucene to perform faceting on fields
> that haven't been specifically indexed for faceting. I clearly need a
> lesson or two on faceting in Lucene. I don't understand what a "path" is
> for example. Of course I have tried to work with the examples, but unless
> I completely rework my indexes, I'm stuck. I thought you could use
> DocValues somehow to facilitate faceting queries. Anyway, I suppose I'll
> need to use the source code as my guide and spend a lot of time working
> with it. I was hoping to find some really good documentation to help me
> cut to the chase. I appreciate you offering to help answer any questions,
> seems like I could use an interactive session ????
>
> Thanks!
>
> Tony
>
>
> -----Original Message-----
> From: Stefan Vodita <stefan.vodita@gmail.com>
> Sent: Tuesday, November 14, 2023 19:14
> To: tony@xfire.io.invalid
> Cc: java-user@lucene.apache.org
> Subject: Re: Faceting Queries NON-Taxonomy-based
>
> Hi Tony,
>
> Have you looked at the faceting demo package [1]?
>
> The faceting documentation that lives there [2] was updated last year
> (thanks
> @epotyom!) and there haven't been major API changes since.
>
> The `SimpleSortedSetFacetsExample` [3] might be a good starting point for
> what you're trying to do.
>
> If you've already checked these resources, is there a specific question
> they didn't help answer?
>
>
> Stefan
>
>
> [1]
>
> https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/apache/lucene/demo/facet
> [2]
>
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/package-info.java
> [3]
>
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.java
>
> On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid> wrote:
>
> > Hello,
> >
> >
> >
> > Is there a definitive (or at least very good) documentation or video
> > on Lucene faceting? I find the existing docs and samples out-dated
> > and inaccurate. As such, I'm having trouble getting my mind around
> > how it works to ensure I index my documents in such a way as to allow
> > faceting. I'm trying to avoid the "taxonomy-based" approach.
> >
> >
> >
> > Tony
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Hi Tony-

There are indeed a few different ways faceting can be implemented, which
can be confusing. Can you share a little more about what you're looking to
do with faceting? It sounds like maybe you want to facet on a docvalues
field you already have in your index? If that's the use-case, you might
want to look at `StringValueFacetCounts`. There isn't demo code
specifically for it, but there are some test cases and javadoc. But if you
have some more details on what you're looking to do, myself (or someone
else here) might be able to provide some more pointers.

Cheers,
-Greg

On Thu, Nov 16, 2023 at 9:02?AM Stefan Vodita <stefan.vodita@gmail.com>
wrote:

> Hi Tony,
>
> I put together a demo for faceting using `KeywordField` [1]. Hopefully that
> answers some of your questions.
>
> A path would be a sequence of hierarchical labels that a document belongs
> to.
> For example, a book could be published on a certain day, of a certain
> month,
> of a certain year, making for a path like `Publish Date/2023/11/16`. When
> faceting, you could get counts with respect to each of the labels in the
> path
> (e.g. counts per year or counts per month of given year).
>
>
> Stefan
>
>
> [1] https://github.com/apache/lucene/pull/12817
>
> On Wed, 15 Nov 2023 at 01:01, Tony Schwartz <tony@xfire.io.invalid> wrote:
>
> > I have a lot of questions. For example, the javadocs for the
> > "KeywordField" class says: "... doc values for sorting and faceting"
> >
> > Yet, I don't see a way with 9.8.x Lucene to perform faceting on fields
> > that haven't been specifically indexed for faceting. I clearly need a
> > lesson or two on faceting in Lucene. I don't understand what a "path" is
> > for example. Of course I have tried to work with the examples, but
> unless
> > I completely rework my indexes, I'm stuck. I thought you could use
> > DocValues somehow to facilitate faceting queries. Anyway, I suppose I'll
> > need to use the source code as my guide and spend a lot of time working
> > with it. I was hoping to find some really good documentation to help me
> > cut to the chase. I appreciate you offering to help answer any
> questions,
> > seems like I could use an interactive session ????
> >
> > Thanks!
> >
> > Tony
> >
> >
> > -----Original Message-----
> > From: Stefan Vodita <stefan.vodita@gmail.com>
> > Sent: Tuesday, November 14, 2023 19:14
> > To: tony@xfire.io.invalid
> > Cc: java-user@lucene.apache.org
> > Subject: Re: Faceting Queries NON-Taxonomy-based
> >
> > Hi Tony,
> >
> > Have you looked at the faceting demo package [1]?
> >
> > The faceting documentation that lives there [2] was updated last year
> > (thanks
> > @epotyom!) and there haven't been major API changes since.
> >
> > The `SimpleSortedSetFacetsExample` [3] might be a good starting point for
> > what you're trying to do.
> >
> > If you've already checked these resources, is there a specific question
> > they didn't help answer?
> >
> >
> > Stefan
> >
> >
> > [1]
> >
> >
> https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/apache/lucene/demo/facet
> > [2]
> >
> >
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/package-info.java
> > [3]
> >
> >
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleSortedSetFacetsExample.java
> >
> > On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid>
> wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > > Is there a definitive (or at least very good) documentation or video
> > > on Lucene faceting? I find the existing docs and samples out-dated
> > > and inaccurate. As such, I'm having trouble getting my mind around
> > > how it works to ensure I index my documents in such a way as to allow
> > > faceting. I'm trying to avoid the "taxonomy-based" approach.
> > >
> > >
> > >
> > > Tony
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>

I have been fighting through it, and have had plenty of success now. It's just that it's a bit finnicky and a little confusing. At some point, I may think to create a write-up of my own or a video or something to help others understand how faceting works in Lucene. I'm certainly no expert, but I have been learning a lot about it by debugging and fighting through it. Thank you, and also thank you, Stefan for your responses and for your help. Stefan, I will review your demo as well.

Thank you,

Tony


-----Original Message-----
From: Greg Miller <gsmiller@gmail.com>
Sent: Thursday, November 16, 2023 14:56
To: java-user@lucene.apache.org
Subject: Re: Faceting Queries NON-Taxonomy-based

Hi Tony-

There are indeed a few different ways faceting can be implemented, which can be confusing. Can you share a little more about what you're looking to do with faceting? It sounds like maybe you want to facet on a docvalues field you already have in your index? If that's the use-case, you might want to look at `StringValueFacetCounts`. There isn't demo code specifically for it, but there are some test cases and javadoc. But if you have some more details on what you're looking to do, myself (or someone else here) might be able to provide some more pointers.

Cheers,
-Greg

On Thu, Nov 16, 2023 at 9:02?AM Stefan Vodita <stefan.vodita@gmail.com>
wrote:

> Hi Tony,
>
> I put together a demo for faceting using `KeywordField` [1]. Hopefully
> that answers some of your questions.
>
> A path would be a sequence of hierarchical labels that a document
> belongs to.
> For example, a book could be published on a certain day, of a certain
> month, of a certain year, making for a path like `Publish
> Date/2023/11/16`. When faceting, you could get counts with respect to
> each of the labels in the path (e.g. counts per year or counts per
> month of given year).
>
>
> Stefan
>
>
> [1] https://github.com/apache/lucene/pull/12817
>
> On Wed, 15 Nov 2023 at 01:01, Tony Schwartz <tony@xfire.io.invalid> wrote:
>
> > I have a lot of questions. For example, the javadocs for the
> > "KeywordField" class says: "... doc values for sorting and faceting"
> >
> > Yet, I don't see a way with 9.8.x Lucene to perform faceting on
> > fields that haven't been specifically indexed for faceting. I
> > clearly need a lesson or two on faceting in Lucene. I don't
> > understand what a "path" is for example. Of course I have tried to
> > work with the examples, but
> unless
> > I completely rework my indexes, I'm stuck. I thought you could use
> > DocValues somehow to facilitate faceting queries. Anyway, I suppose
> > I'll need to use the source code as my guide and spend a lot of time
> > working with it. I was hoping to find some really good
> > documentation to help me cut to the chase. I appreciate you
> > offering to help answer any
> questions,
> > seems like I could use an interactive session ????
> >
> > Thanks!
> >
> > Tony
> >
> >
> > -----Original Message-----
> > From: Stefan Vodita <stefan.vodita@gmail.com>
> > Sent: Tuesday, November 14, 2023 19:14
> > To: tony@xfire.io.invalid
> > Cc: java-user@lucene.apache.org
> > Subject: Re: Faceting Queries NON-Taxonomy-based
> >
> > Hi Tony,
> >
> > Have you looked at the faceting demo package [1]?
> >
> > The faceting documentation that lives there [2] was updated last
> > year (thanks
> > @epotyom!) and there haven't been major API changes since.
> >
> > The `SimpleSortedSetFacetsExample` [3] might be a good starting
> > point for what you're trying to do.
> >
> > If you've already checked these resources, is there a specific
> > question they didn't help answer?
> >
> >
> > Stefan
> >
> >
> > [1]
> >
> >
> https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/ap
> ache/lucene/demo/facet
> > [2]
> >
> >
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/ap
> ache/lucene/demo/facet/package-info.java
> > [3]
> >
> >
> https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/ap
> ache/lucene/demo/facet/SimpleSortedSetFacetsExample.java
> >
> > On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid>
> wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > > Is there a definitive (or at least very good) documentation or
> > > video on Lucene faceting? I find the existing docs and samples
> > > out-dated and inaccurate. As such, I'm having trouble getting my
> > > mind around how it works to ensure I index my documents in such a
> > > way as to allow faceting. I'm trying to avoid the "taxonomy-based" approach.
> > >
> > >
> > >
> > > Tony
> > >
> > >
> >
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: java-user-help@lucene.apache.org
> >
> >
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Glad you got it sorted out. If you feel there's some documentation that
could be pushed back into Lucene in some way as well, contributions are
always welcome!

Cheers,
-Greg

On Fri, Nov 17, 2023 at 10:31?AM Tony Schwartz <tony@xfire.io.invalid>
wrote:

> I have been fighting through it, and have had plenty of success now. It's
> just that it's a bit finnicky and a little confusing. At some point, I may
> think to create a write-up of my own or a video or something to help others
> understand how faceting works in Lucene. I'm certainly no expert, but I
> have been learning a lot about it by debugging and fighting through it.
> Thank you, and also thank you, Stefan for your responses and for your
> help. Stefan, I will review your demo as well.
>
> Thank you,
>
> Tony
>
>
> -----Original Message-----
> From: Greg Miller <gsmiller@gmail.com>
> Sent: Thursday, November 16, 2023 14:56
> To: java-user@lucene.apache.org
> Subject: Re: Faceting Queries NON-Taxonomy-based
>
> Hi Tony-
>
> There are indeed a few different ways faceting can be implemented, which
> can be confusing. Can you share a little more about what you're looking to
> do with faceting? It sounds like maybe you want to facet on a docvalues
> field you already have in your index? If that's the use-case, you might
> want to look at `StringValueFacetCounts`. There isn't demo code
> specifically for it, but there are some test cases and javadoc. But if you
> have some more details on what you're looking to do, myself (or someone
> else here) might be able to provide some more pointers.
>
> Cheers,
> -Greg
>
> On Thu, Nov 16, 2023 at 9:02?AM Stefan Vodita <stefan.vodita@gmail.com>
> wrote:
>
> > Hi Tony,
> >
> > I put together a demo for faceting using `KeywordField` [1]. Hopefully
> > that answers some of your questions.
> >
> > A path would be a sequence of hierarchical labels that a document
> > belongs to.
> > For example, a book could be published on a certain day, of a certain
> > month, of a certain year, making for a path like `Publish
> > Date/2023/11/16`. When faceting, you could get counts with respect to
> > each of the labels in the path (e.g. counts per year or counts per
> > month of given year).
> >
> >
> > Stefan
> >
> >
> > [1] https://github.com/apache/lucene/pull/12817
> >
> > On Wed, 15 Nov 2023 at 01:01, Tony Schwartz <tony@xfire.io.invalid>
> wrote:
> >
> > > I have a lot of questions. For example, the javadocs for the
> > > "KeywordField" class says: "... doc values for sorting and faceting"
> > >
> > > Yet, I don't see a way with 9.8.x Lucene to perform faceting on
> > > fields that haven't been specifically indexed for faceting. I
> > > clearly need a lesson or two on faceting in Lucene. I don't
> > > understand what a "path" is for example. Of course I have tried to
> > > work with the examples, but
> > unless
> > > I completely rework my indexes, I'm stuck. I thought you could use
> > > DocValues somehow to facilitate faceting queries. Anyway, I suppose
> > > I'll need to use the source code as my guide and spend a lot of time
> > > working with it. I was hoping to find some really good
> > > documentation to help me cut to the chase. I appreciate you
> > > offering to help answer any
> > questions,
> > > seems like I could use an interactive session ????
> > >
> > > Thanks!
> > >
> > > Tony
> > >
> > >
> > > -----Original Message-----
> > > From: Stefan Vodita <stefan.vodita@gmail.com>
> > > Sent: Tuesday, November 14, 2023 19:14
> > > To: tony@xfire.io.invalid
> > > Cc: java-user@lucene.apache.org
> > > Subject: Re: Faceting Queries NON-Taxonomy-based
> > >
> > > Hi Tony,
> > >
> > > Have you looked at the faceting demo package [1]?
> > >
> > > The faceting documentation that lives there [2] was updated last
> > > year (thanks
> > > @epotyom!) and there haven't been major API changes since.
> > >
> > > The `SimpleSortedSetFacetsExample` [3] might be a good starting
> > > point for what you're trying to do.
> > >
> > > If you've already checked these resources, is there a specific
> > > question they didn't help answer?
> > >
> > >
> > > Stefan
> > >
> > >
> > > [1]
> > >
> > >
> > https://github.com/apache/lucene/tree/main/lucene/demo/src/java/org/ap
> > ache/lucene/demo/facet
> > > [2]
> > >
> > >
> > https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/ap
> > ache/lucene/demo/facet/package-info.java
> > > [3]
> > >
> > >
> > https://github.com/apache/lucene/blob/main/lucene/demo/src/java/org/ap
> > ache/lucene/demo/facet/SimpleSortedSetFacetsExample.java
> > >
> > > On Tue, 14 Nov 2023 at 18:51, Tony Schwartz <tony@xfire.io.invalid>
> > wrote:
> > >
> > > > Hello,
> > > >
> > > >
> > > >
> > > > Is there a definitive (or at least very good) documentation or
> > > > video on Lucene faceting? I find the existing docs and samples
> > > > out-dated and inaccurate. As such, I'm having trouble getting my
> > > > mind around how it works to ensure I index my documents in such a
> > > > way as to allow faceting. I'm trying to avoid the "taxonomy-based"
> approach.
> > > >
> > > >
> > > >
> > > > Tony
> > > >
> > > >
> > >
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>