Mailing List Archive: Fix for vector math precision

Instead of doing double math, new PR to limit vector components to smaller
than 1E17 to prevent overflow: https://github.com/apache/lucene/pull/12373

On Fri, Jun 9, 2023 at 4:38?PM Jonathan Ellis <jbellis@gmail.com> wrote:

> Hi all,
>
> I ran into a bug where the cosine of a large vector taken with itself
> returned NaN. (Cosine of equal vectors should always be 1.) I put
> together a PR to do the internal math of the cosine function with double,
> before returning the result as a float:
> https://github.com/apache/lucene/pull/12281
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>

--
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced