Mailing List Archive: Misconfigured gradle rat inputs?

Misconfigured gradle rat inputs?

May 25, 2021, 10:26 AM

Post #1 of 6 (384 views)

There’s a subject line I never thought I’d type :)

Firstly: can I say how much I appreciate all the work that’s gone into the gradle build? I’ve been doing lots of small PRs for the spans-to-queries work and being able to run checks multiple times in an extremely efficient manner has been a life saver. Massive thanks to Dawid, and also to Robert for all the work on speeding up tests.

I think may have found a bug in the input configuration for our license header checks. Thanks to the new build, I have been running `./gradlew check` before pushing code, but it has let through files with missing headers a few times, which were subsequently caught by the GitHub action running on the PR.

So I tried the following:
- start a new git branch
- run ./gradlew rat -> everything should pass
- edit one of the files to remove the license header
- run ./gradlew rat -> still passes!
- run ./gradlew clean
- run ./gradlew rat -> now I get an error

This looks to me like the fileset that the rat task is looking at is not set up correctly, but I don’t know enough gradle to actually work out what is wrong and what the fix should be.

- A
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

RE: Misconfigured gradle rat inputs? [ In reply to ]

uwe at thetaphi

May 25, 2021, 10:40 AM

Post #2 of 6 (384 views)

Permalink

Hi,

the problem is that the RAT task has only some patterns for filenames as input, but no actual @InputDirectory. If the files change, nothing changes from Gradle's point of view.

The validation/rat-sources.gradle and its inner class "RatTask" must at least declare @InputDirectory with a default value of ${project.projectDir}. This would cause any change to retrigger the task.

To make it more professional, it should declare a FileCollection and apply the patterns, but that's more complicated as this task just wraps the native Ant RAT task.

I can fix this (must be done for Solr and Lucene, both have the problem).

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Alan Woodward <romseygeek@gmail.com>
> Sent: Tuesday, May 25, 2021 7:26 PM
> To: dev@lucene.apache.org
> Subject: Misconfigured gradle rat inputs?
>
> There’s a subject line I never thought I’d type :)
>
> Firstly: can I say how much I appreciate all the work that’s gone into the gradle
> build? I’ve been doing lots of small PRs for the spans-to-queries work and being
> able to run checks multiple times in an extremely efficient manner has been a
> life saver. Massive thanks to Dawid, and also to Robert for all the work on
> speeding up tests.
>
> I think may have found a bug in the input configuration for our license header
> checks. Thanks to the new build, I have been running `./gradlew check` before
> pushing code, but it has let through files with missing headers a few times,
> which were subsequently caught by the GitHub action running on the PR.
>
> So I tried the following:
> - start a new git branch
> - run ./gradlew rat -> everything should pass
> - edit one of the files to remove the license header
> - run ./gradlew rat -> still passes!
> - run ./gradlew clean
> - run ./gradlew rat -> now I get an error
>
> This looks to me like the fileset that the rat task is looking at is not set up
> correctly, but I don’t know enough gradle to actually work out what is wrong
> and what the fix should be.
>
> - A
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Misconfigured gradle rat inputs? [ In reply to ]

romseygeek at gmail

May 25, 2021, 3:04 PM

Post #3 of 6 (384 views)

Permalink

I will defer to the sophisticated gradle policeman on this!

> On 25 May 2021, at 18:40, Uwe Schindler <uwe@thetaphi.de> wrote:
>
> Hi,
>
> the problem is that the RAT task has only some patterns for filenames as input, but no actual @InputDirectory. If the files change, nothing changes from Gradle's point of view.
>
> The validation/rat-sources.gradle and its inner class "RatTask" must at least declare @InputDirectory with a default value of ${project.projectDir}. This would cause any change to retrigger the task.
>
> To make it more professional, it should declare a FileCollection and apply the patterns, but that's more complicated as this task just wraps the native Ant RAT task.
>
> I can fix this (must be done for Solr and Lucene, both have the problem).
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>> -----Original Message-----
>> From: Alan Woodward <romseygeek@gmail.com>
>> Sent: Tuesday, May 25, 2021 7:26 PM
>> To: dev@lucene.apache.org
>> Subject: Misconfigured gradle rat inputs?
>>
>> There’s a subject line I never thought I’d type :)
>>
>> Firstly: can I say how much I appreciate all the work that’s gone into the gradle
>> build? I’ve been doing lots of small PRs for the spans-to-queries work and being
>> able to run checks multiple times in an extremely efficient manner has been a
>> life saver. Massive thanks to Dawid, and also to Robert for all the work on
>> speeding up tests.
>>
>> I think may have found a bug in the input configuration for our license header
>> checks. Thanks to the new build, I have been running `./gradlew check` before
>> pushing code, but it has let through files with missing headers a few times,
>> which were subsequently caught by the GitHub action running on the PR.
>>
>> So I tried the following:
>> - start a new git branch
>> - run ./gradlew rat -> everything should pass
>> - edit one of the files to remove the license header
>> - run ./gradlew rat -> still passes!
>> - run ./gradlew clean
>> - run ./gradlew rat -> now I get an error
>>
>> This looks to me like the fileset that the rat task is looking at is not set up
>> correctly, but I don’t know enough gradle to actually work out what is wrong
>> and what the fix should be.
>>
>> - A
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Misconfigured gradle rat inputs? [ In reply to ]

dawid.weiss at gmail

May 25, 2021, 10:28 PM

Post #4 of 6 (384 views)

Permalink

Yep, looks like missing inputs and the task is skipped. Good catch, Alan. I
can look a this later too, Uwe (just create an issue and assign to me).

On Tue, May 25, 2021 at 7:40 PM Uwe Schindler <uwe@thetaphi.de> wrote:

> Hi,
>
> the problem is that the RAT task has only some patterns for filenames as
> input, but no actual @InputDirectory. If the files change, nothing changes
> from Gradle's point of view.
>
> The validation/rat-sources.gradle and its inner class "RatTask" must at
> least declare @InputDirectory with a default value of
> ${project.projectDir}. This would cause any change to retrigger the task.
>
> To make it more professional, it should declare a FileCollection and apply
> the patterns, but that's more complicated as this task just wraps the
> native Ant RAT task.
>
> I can fix this (must be done for Solr and Lucene, both have the problem).
>
> Uwe
>
> -----
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
> > -----Original Message-----
> > From: Alan Woodward <romseygeek@gmail.com>
> > Sent: Tuesday, May 25, 2021 7:26 PM
> > To: dev@lucene.apache.org
> > Subject: Misconfigured gradle rat inputs?
> >
> > There’s a subject line I never thought I’d type :)
> >
> > Firstly: can I say how much I appreciate all the work that’s gone into
> the gradle
> > build? I’ve been doing lots of small PRs for the spans-to-queries work
> and being
> > able to run checks multiple times in an extremely efficient manner has
> been a
> > life saver. Massive thanks to Dawid, and also to Robert for all the
> work on
> > speeding up tests.
> >
> > I think may have found a bug in the input configuration for our license
> header
> > checks. Thanks to the new build, I have been running `./gradlew check`
> before
> > pushing code, but it has let through files with missing headers a few
> times,
> > which were subsequently caught by the GitHub action running on the PR.
> >
> > So I tried the following:
> > - start a new git branch
> > - run ./gradlew rat -> everything should pass
> > - edit one of the files to remove the license header
> > - run ./gradlew rat -> still passes!
> > - run ./gradlew clean
> > - run ./gradlew rat -> now I get an error
> >
> > This looks to me like the fileset that the rat task is looking at is not
> set up
> > correctly, but I don’t know enough gradle to actually work out what is
> wrong
> > and what the fix should be.
> >
> > - A
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

RE: Misconfigured gradle rat inputs? [ In reply to ]

uwe at thetaphi

May 26, 2021, 2:00 AM

Post #5 of 6 (384 views)

Permalink

Hi,

I tried to fix the problem but gave up because of limited time.

The problem is that this task is global per project and not split into different ones, it’s all mixed together. It works if I add a @InputDirectory with the projectDir, but this leads to strange exceptions, because it also tries to hash files from build directory.

IMHO, the correct way to fix this is:

* Generate a generic RatTask that extends SourceTask (do NOT extend DefaultTask!). This brings a source directory and include/exclude automatically so it’s easiy to configure. All you need is to use the converter when executing the task, that changes a FileCollection to an ANT fileset: this.getSource().addToAntBuilder?(antTaskDeclaration, ”fileset”, FileCollection.AntType.FileSet)
* Create a separate task for each affected sourceset and also one for the base project dir, each with correct includes/excludes

I gave up, as my time was limited and I was not able to quickly split the task into one for each surceset

Uwe

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de

From: Dawid Weiss <dawid.weiss@gmail.com>
Sent: Wednesday, May 26, 2021 7:29 AM
To: Lucene Dev <dev@lucene.apache.org>
Subject: Re: Misconfigured gradle rat inputs?

Yep, looks like missing inputs and the task is skipped. Good catch, Alan. I can look a this later too, Uwe (just create an issue and assign to me).

On Tue, May 25, 2021 at 7:40 PM Uwe Schindler <uwe@thetaphi.de <mailto:uwe@thetaphi.de> > wrote:

Hi,

the problem is that the RAT task has only some patterns for filenames as input, but no actual @InputDirectory. If the files change, nothing changes from Gradle's point of view.

The validation/rat-sources.gradle and its inner class "RatTask" must at least declare @InputDirectory with a default value of ${project.projectDir}. This would cause any change to retrigger the task.

To make it more professional, it should declare a FileCollection and apply the patterns, but that's more complicated as this task just wraps the native Ant RAT task.

I can fix this (must be done for Solr and Lucene, both have the problem).

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de>

> -----Original Message-----
> From: Alan Woodward <romseygeek@gmail.com <mailto:romseygeek@gmail.com> >
> Sent: Tuesday, May 25, 2021 7:26 PM
> To: dev@lucene.apache.org <mailto:dev@lucene.apache.org>
> Subject: Misconfigured gradle rat inputs?
>
> There’s a subject line I never thought I’d type :)
>
> Firstly: can I say how much I appreciate all the work that’s gone into the gradle
> build? I’ve been doing lots of small PRs for the spans-to-queries work and being
> able to run checks multiple times in an extremely efficient manner has been a
> life saver. Massive thanks to Dawid, and also to Robert for all the work on
> speeding up tests.
>
> I think may have found a bug in the input configuration for our license header
> checks. Thanks to the new build, I have been running `./gradlew check` before
> pushing code, but it has let through files with missing headers a few times,
> which were subsequently caught by the GitHub action running on the PR.
>
> So I tried the following:
> - start a new git branch
> - run ./gradlew rat -> everything should pass
> - edit one of the files to remove the license header
> - run ./gradlew rat -> still passes!
> - run ./gradlew clean
> - run ./gradlew rat -> now I get an error
>
> This looks to me like the fileset that the rat task is looking at is not set up
> correctly, but I don’t know enough gradle to actually work out what is wrong
> and what the fix should be.
>
> - A
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>

RE: Misconfigured gradle rat inputs? [ In reply to ]

uwe at thetaphi

May 26, 2021, 2:12 AM

Post #6 of 6 (384 views)

Permalink

I opened:

https://issues.apache.org/jira/browse/LUCENE-9977

and for Solr:

https://issues.apache.org/jira/browse/SOLR-15436

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de

From: Uwe Schindler <uwe@thetaphi.de>
Sent: Wednesday, May 26, 2021 11:01 AM
To: dev@lucene.apache.org
Subject: RE: Misconfigured gradle rat inputs?

Hi,

I tried to fix the problem but gave up because of limited time.

The problem is that this task is global per project and not split into different ones, it’s all mixed together. It works if I add a @InputDirectory with the projectDir, but this leads to strange exceptions, because it also tries to hash files from build directory.

IMHO, the correct way to fix this is:

* Generate a generic RatTask that extends SourceTask (do NOT extend DefaultTask!). This brings a source directory and include/exclude automatically so it’s easiy to configure. All you need is to use the converter when executing the task, that changes a FileCollection to an ANT fileset: this.getSource().addToAntBuilder?(antTaskDeclaration, ”fileset”, FileCollection.AntType.FileSet)
* Create a separate task for each affected sourceset and also one for the base project dir, each with correct includes/excludes

I gave up, as my time was limited and I was not able to quickly split the task into one for each surceset

Uwe

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de>

From: Dawid Weiss <dawid.weiss@gmail.com <mailto:dawid.weiss@gmail.com> >
Sent: Wednesday, May 26, 2021 7:29 AM
To: Lucene Dev <dev@lucene.apache.org <mailto:dev@lucene.apache.org> >
Subject: Re: Misconfigured gradle rat inputs?

Yep, looks like missing inputs and the task is skipped. Good catch, Alan. I can look a this later too, Uwe (just create an issue and assign to me).

On Tue, May 25, 2021 at 7:40 PM Uwe Schindler <uwe@thetaphi.de <mailto:uwe@thetaphi.de> > wrote:

Hi,

the problem is that the RAT task has only some patterns for filenames as input, but no actual @InputDirectory. If the files change, nothing changes from Gradle's point of view.

The validation/rat-sources.gradle and its inner class "RatTask" must at least declare @InputDirectory with a default value of ${project.projectDir}. This would cause any change to retrigger the task.

To make it more professional, it should declare a FileCollection and apply the patterns, but that's more complicated as this task just wraps the native Ant RAT task.

I can fix this (must be done for Solr and Lucene, both have the problem).

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: uwe@thetaphi.de <mailto:uwe@thetaphi.de>

> -----Original Message-----
> From: Alan Woodward <romseygeek@gmail.com <mailto:romseygeek@gmail.com> >
> Sent: Tuesday, May 25, 2021 7:26 PM
> To: dev@lucene.apache.org <mailto:dev@lucene.apache.org>
> Subject: Misconfigured gradle rat inputs?
>
> There’s a subject line I never thought I’d type :)
>
> Firstly: can I say how much I appreciate all the work that’s gone into the gradle
> build? I’ve been doing lots of small PRs for the spans-to-queries work and being
> able to run checks multiple times in an extremely efficient manner has been a
> life saver. Massive thanks to Dawid, and also to Robert for all the work on
> speeding up tests.
>
> I think may have found a bug in the input configuration for our license header
> checks. Thanks to the new build, I have been running `./gradlew check` before
> pushing code, but it has let through files with missing headers a few times,
> which were subsequently caught by the GitHub action running on the PR.
>
> So I tried the following:
> - start a new git branch
> - run ./gradlew rat -> everything should pass
> - edit one of the files to remove the license header
> - run ./gradlew rat -> still passes!
> - run ./gradlew clean
> - run ./gradlew rat -> now I get an error
>
> This looks to me like the fileset that the rat task is looking at is not set up
> correctly, but I don’t know enough gradle to actually work out what is wrong
> and what the fix should be.
>
> - A
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
> For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org <mailto:dev-unsubscribe@lucene.apache.org>
For additional commands, e-mail: dev-help@lucene.apache.org <mailto:dev-help@lucene.apache.org>