Mailing List Archive: Package conflicts trying to install jpegdupes package

Package conflicts trying to install jpegdupes package

skip.montanaro at gmail

Sep 27, 2021, 5:48 AM

Post #1 of 14 (253 views)

I'd like to use the jpegdupes package <https://pypi.org/project/jpegdupes/> but
can't seem to get past compilation issued related to jpegtrans-cffi.
Attempts to install any available versions on my XUbuntu system crap out
with this C compiler error message:

In file included from
jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267:
src/epeg_private.h:71:17: warning: ‘_jpeg_init_source’ declared
‘static’ but never defined [-Wunused-function]
71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo);
| ^~~~~~~~~~~~~~~~~
src/epeg_private.h:72:20: warning: ‘_jpeg_fill_input_buffer’ declared
‘static’ but never defined [-Wunused-function]
72 | METHODDEF(boolean) _jpeg_fill_input_buffer(j_decompress_ptr
cinfo);
| ^~~~~~~~~~~~~~~~~~~~~~~
src/epeg_private.h:73:17: warning: ‘_jpeg_skip_input_data’ declared
‘static’ but never defined [-Wunused-function]
73 | METHODDEF(void) _jpeg_skip_input_data(j_decompress_ptr cinfo,
long num_bytes);
| ^~~~~~~~~~~~~~~~~~~~~
src/epeg_private.h:74:17: warning: ‘_jpeg_term_source’ declared
‘static’ but never defined [-Wunused-function]
74 | METHODDEF(void) _jpeg_term_source(j_decompress_ptr cinfo);
| ^~~~~~~~~~~~~~~~~
src/epeg_private.h:76:17: warning: ‘_jpeg_init_destination’ declared
‘static’ but never defined [-Wunused-function]
76 | METHODDEF(void) _jpeg_init_destination(j_compress_ptr cinfo);
| ^~~~~~~~~~~~~~~~~~~~~~
src/epeg_private.h:77:20: warning: ‘_jpeg_empty_output_buffer’ declared
‘static’ but never defined [-Wunused-function]
77 | METHODDEF(boolean) _jpeg_empty_output_buffer (j_compress_ptr
cinfo);
| ^~~~~~~~~~~~~~~~~~~~~~~~~
src/epeg_private.h:78:17: warning: ‘_jpeg_term_destination’ declared
‘static’ but never defined [-Wunused-function]
78 | METHODDEF(void) _jpeg_term_destination (j_compress_ptr cinfo);
| ^~~~~~~~~~~~~~~~~~~~~~

...
This seems like a rather odd error message for a presumably sort-of-stable
PyPI package. Looking at the version dependencies I see 2.6, 2.7, 3.3 or
PyPy. I thought nmybe I needed to try with Python2... Create a Conda
environment, attempt to install jpegdupes, only to discover it's Python 3
only.

So, I'm kind of stuck. Maybe I need to install Python 3.3 and try that? Any
other ideas?

Thx,

Skip
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 9:41 AM

Post #2 of 14 (253 views)

On Mon, Sep 27, 2021 at 10:49 PM Skip Montanaro
<skip.montanaro@gmail.com> wrote:
>
> I'd like to use the jpegdupes package <https://pypi.org/project/jpegdupes/> but
> can't seem to get past compilation issued related to jpegtrans-cffi.
> Attempts to install any available versions on my XUbuntu system crap out
> with this C compiler error message:
>
> In file included from
> jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267:
> src/epeg_private.h:71:17: warning: ‘_jpeg_init_source’ declared
> ‘static’ but never defined [-Wunused-function]
> 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo);
> | ^~~~~~~~~~~~~~~~~
> src/epeg_private.h:72:20: warning: ‘_jpeg_fill_input_buffer’ declared
> ‘static’ but never defined [-Wunused-function]
> 72 | METHODDEF(boolean) _jpeg_fill_input_buffer(j_decompress_ptr
> cinfo);
> | ^~~~~~~~~~~~~~~~~~~~~~~
> src/epeg_private.h:73:17: warning: ‘_jpeg_skip_input_data’ declared
> ‘static’ but never defined [-Wunused-function]
> 73 | METHODDEF(void) _jpeg_skip_input_data(j_decompress_ptr cinfo,
> long num_bytes);
> | ^~~~~~~~~~~~~~~~~~~~~
> src/epeg_private.h:74:17: warning: ‘_jpeg_term_source’ declared
> ‘static’ but never defined [-Wunused-function]
> 74 | METHODDEF(void) _jpeg_term_source(j_decompress_ptr cinfo);
> | ^~~~~~~~~~~~~~~~~
> src/epeg_private.h:76:17: warning: ‘_jpeg_init_destination’ declared
> ‘static’ but never defined [-Wunused-function]
> 76 | METHODDEF(void) _jpeg_init_destination(j_compress_ptr cinfo);
> | ^~~~~~~~~~~~~~~~~~~~~~
> src/epeg_private.h:77:20: warning: ‘_jpeg_empty_output_buffer’ declared
> ‘static’ but never defined [-Wunused-function]
> 77 | METHODDEF(boolean) _jpeg_empty_output_buffer (j_compress_ptr
> cinfo);
> | ^~~~~~~~~~~~~~~~~~~~~~~~~
> src/epeg_private.h:78:17: warning: ‘_jpeg_term_destination’ declared
> ‘static’ but never defined [-Wunused-function]
> 78 | METHODDEF(void) _jpeg_term_destination (j_compress_ptr cinfo);
> | ^~~~~~~~~~~~~~~~~~~~~~

Those are all warnings. Are there any errors that follow them?

> ...
> This seems like a rather odd error message for a presumably sort-of-stable
> PyPI package. Looking at the version dependencies I see 2.6, 2.7, 3.3 or
> PyPy. I thought nmybe I needed to try with Python2... Create a Conda
> environment, attempt to install jpegdupes, only to discover it's Python 3
> only.
>
> So, I'm kind of stuck. Maybe I need to install Python 3.3 and try that? Any
> other ideas?
>

The package doesn't seem to have had a release since 2018, but the
GitHub repository has had changes as recently as a month ago. You
might have better luck cloning the repo and building that.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

skip.montanaro at gmail

Sep 27, 2021, 10:11 AM

Post #3 of 14 (253 views)

>
> Those are all warnings. Are there any errors that follow them?
>

Maybe I just missed the actual errors, but the compiler exit status was 1,
so there must have been. I'll give it another try.

The package doesn't seem to have had a release since 2018, but the
> GitHub repository has had changes as recently as a month ago. You
> might have better luck cloning the repo and building that.
>

Thanks, good suggestion.

FWIW, I'm trying to whittle down at least 12,000 images to a more
manageable number for the sister of a good friend who recently passed away.
I've got a straightforward dedupe program, but need something which can
compare just the data chunk of JPEGs, ignoring the metadata. This program
apparently does that. Is like to avoid reinventing that wheel.

Skip
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 10:23 AM

Post #4 of 14 (253 views)

On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro <skip.montanaro@gmail.com> wrote:
>>
>> Those are all warnings. Are there any errors that follow them?
>
>
> Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try.

Yeah, one of the annoying things of building large C packages is when
the maintainers don't care about warnings, and then legit errors get
lost in the spam.

>> The package doesn't seem to have had a release since 2018, but the
>> GitHub repository has had changes as recently as a month ago. You
>> might have better luck cloning the repo and building that.
>
>
> Thanks, good suggestion.
>
> FWIW, I'm trying to whittle down at least 12,000 images to a more manageable number for the sister of a good friend who recently passed away. I've got a straightforward dedupe program, but need something which can compare just the data chunk of JPEGs, ignoring the metadata. This program apparently does that. Is like to avoid reinventing that wheel.
>

Makes sense! Though if you're just doing a bitwise comparison of the
data chunks, it shouldn't be TOO hard to reinvent. JFIF is a
reasonably easy format to understand (although the mathematics of
image encoding and decoding are a bit less so).

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

dieter at handshake

Sep 27, 2021, 10:50 AM

Post #5 of 14 (253 views)

Skip Montanaro wrote at 2021-9-27 07:48 -0500:
>I'd like to use the jpegdupes package <https://pypi.org/project/jpegdupes/> but
>can't seem to get past compilation issued related to jpegtrans-cffi.
>Attempts to install any available versions on my XUbuntu system crap out
>with this C compiler error message:
>
> In file included from
>jpegtran/__pycache__/_cffi__xd2d84bdexcdb1023.c:267:
> src/epeg_private.h:71:17: warning: ‘_jpeg_init_source’ declared
>‘static’ but never defined [-Wunused-function]
> 71 | METHODDEF(void) _jpeg_init_source(j_decompress_ptr cinfo);
> | ^~~~~~~~~~~~~~~~~
> ...

You list only warnings (which usually can be ignored).
If the compilation fails, you should see error messages, too.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

2QdxY4RzWzUUiLuE at potatochowder

Sep 27, 2021, 11:03 AM

Post #6 of 14 (253 views)

On 2021-09-28 at 03:23:53 +1000,
Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro <skip.montanaro@gmail.com> wrote:
> >>
> >> Those are all warnings. Are there any errors that follow them?
> >
> >
> > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try.
>
> Yeah, one of the annoying things of building large C packages is when
> the maintainers don't care about warnings, and then legit errors get
> lost in the spam.

It's also possible that the compiler is running in a "warnings are
errors" mode. Hack the Makefile (or equivalent) at your own peril.

One of the annoying things about warnings is that they're not standard,
so squelching them under one compiler makes another compiler (or a
different version of the same compiler) whine. IIRC, a compliant C
compiler can issue a warning for anything it wants, including "this
variable name contains too many vowels" or "this is a computer, and
today may be Tuesday."

In many ways, the engineering pieces of software engineering remain in
their infancy. *sigh*
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 11:16 AM

Post #7 of 14 (253 views)

On Tue, Sep 28, 2021 at 4:04 AM <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>
> On 2021-09-28 at 03:23:53 +1000,
> Chris Angelico <rosuav@gmail.com> wrote:
>
> > On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro <skip.montanaro@gmail.com> wrote:
> > >>
> > >> Those are all warnings. Are there any errors that follow them?
> > >
> > >
> > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try.
> >
> > Yeah, one of the annoying things of building large C packages is when
> > the maintainers don't care about warnings, and then legit errors get
> > lost in the spam.
>
> It's also possible that the compiler is running in a "warnings are
> errors" mode. Hack the Makefile (or equivalent) at your own peril.
>
> One of the annoying things about warnings is that they're not standard,
> so squelching them under one compiler makes another compiler (or a
> different version of the same compiler) whine. IIRC, a compliant C
> compiler can issue a warning for anything it wants, including "this
> variable name contains too many vowels" or "this is a computer, and
> today may be Tuesday."
>
> In many ways, the engineering pieces of software engineering remain in
> their infancy. *sigh*

But that's kinda the point of having different compilers. Warnings
that we consider fairly standard today didn't exist in the past, and
someone had to be the first to add them. If the C language had to
mandate every warning, it'd take forever to make changes.

I don't think that warnings-are-errors mode is a good thing though.
They should remain just as warnings. (And with the most popular
compilers, using the compilation options as specified in their own
makefile, maintainers should ideally try to ensure that there are as
few warnings as possible.)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

2QdxY4RzWzUUiLuE at potatochowder

Sep 27, 2021, 11:44 AM

Post #8 of 14 (253 views)

On 2021-09-28 at 04:16:58 +1000,
Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Sep 28, 2021 at 4:04 AM <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
> >
> > On 2021-09-28 at 03:23:53 +1000,
> > Chris Angelico <rosuav@gmail.com> wrote:
> >
> > > On Tue, Sep 28, 2021 at 3:11 AM Skip Montanaro <skip.montanaro@gmail.com> wrote:
> > > >>
> > > >> Those are all warnings. Are there any errors that follow them?
> > > >
> > > >
> > > > Maybe I just missed the actual errors, but the compiler exit status was 1, so there must have been. I'll give it another try.
> > >
> > > Yeah, one of the annoying things of building large C packages is when
> > > the maintainers don't care about warnings, and then legit errors get
> > > lost in the spam.
> >
> > It's also possible that the compiler is running in a "warnings are
> > errors" mode. Hack the Makefile (or equivalent) at your own peril.
> >
> > One of the annoying things about warnings is that they're not standard,
> > so squelching them under one compiler makes another compiler (or a
> > different version of the same compiler) whine. IIRC, a compliant C
> > compiler can issue a warning for anything it wants, including "this
> > variable name contains too many vowels" or "this is a computer, and
> > today may be Tuesday."
> >
> > In many ways, the engineering pieces of software engineering remain in
> > their infancy. *sigh*
>
> But that's kinda the point of having different compilers. Warnings
> that we consider fairly standard today didn't exist in the past, and
> someone had to be the first to add them. If the C language had to
> mandate every warning, it'd take forever to make changes.

That is indeed one benefit of having different compilers and compilers
that improve over time. It's also a nuisance when the compiler in your
development/unit test environment produces different warnings than
either cross compiler for your target environments (plural).

In Python, it's more likely that MSVC, MinGW, LLVM, and gcc choose
different warnings, and that no collection of settings supresses them
all. Present C projects contain oodles of conditional compilation,
often controlled by the preprocessor; achieving ideal results can be,
well, "tricky." It's not a matter of not caring.

At some point, it comes down to a question of ignoring a warning vs.
cluttering up your code (preprocessor or otherwise) with extra
conditionals. Or being stuck with a warning about unneeded defensive
programming vs. being stuck with a warning about possible arithmetic
overflow on your 32-bit target.

I agree that the standard shouldn't regulate warnings. I'm also saying
that sometimes, it's a pain that it doesn't, and that not every warning
is the package maintainer's fault. :-)

> I don't think that warnings-are-errors mode is a good thing though.
> They should remain just as warnings. (And with the most popular
> compilers, using the compilation options as specified in their own
> makefile, maintainers should ideally try to ensure that there are as
> few warnings as possible.)

Ideally, yes.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 11:49 AM

Post #9 of 14 (253 views)

On Tue, Sep 28, 2021 at 4:46 AM <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
> I agree that the standard shouldn't regulate warnings. I'm also saying
> that sometimes, it's a pain that it doesn't, and that not every warning
> is the package maintainer's fault. :-)

Agreed. For the most part, a proper colorizing compiler can make the
errors stand out among the spam of warnings, but it's hard when people
copy and paste just the text. There's a skill to skimming compiler
output to find what's relevant (for instance, just before an error,
there might be a related warning, which actually is highlighting the
real problem).

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

skip.montanaro at gmail

Sep 27, 2021, 3:08 PM

Post #10 of 14 (253 views)

>
> For the most part, a proper colorizing compiler can make the
> errors stand out among the spam of warnings, but it's hard when people
> copy and paste just the text.
>

In my defense, the entire traceback was red. :-) It's quite possible that
GCC colorized its error/warning messages, but colorizing might have been
suppressed by stderr being fed into a pipe, or by distutils tossing it out.

Skip
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 3:12 PM

Post #11 of 14 (253 views)

On Tue, Sep 28, 2021 at 8:09 AM Skip Montanaro <skip.montanaro@gmail.com> wrote:
>>
>> For the most part, a proper colorizing compiler can make the
>> errors stand out among the spam of warnings, but it's hard when people
>> copy and paste just the text.
>
>
> In my defense, the entire traceback was red. :-) It's quite possible that GCC colorized its error/warning messages, but colorizing might have been suppressed by stderr being fed into a pipe, or by distutils tossing it out.
>

Ah, yeah, that's a nuisance. Honestly, most release systems should be
configured with fewer warnings, but maybe people don't distinguish
between dev and release.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

skip.montanaro at gmail

Sep 27, 2021, 5:40 PM

Post #12 of 14 (253 views)

Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h
file. Thank goodness for the apt-file command. I was able to track
that down to the libturbojpeg0-dev package, install that, and after a
bit more fussing around now have jpegdupes installed.

Thanks for the help,

Skip
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

rosuav at gmail

Sep 27, 2021, 5:44 PM

Post #13 of 14 (253 views)

On Tue, Sep 28, 2021 at 10:40 AM Skip Montanaro
<skip.montanaro@gmail.com> wrote:
>
> Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h
> file. Thank goodness for the apt-file command. I was able to track
> that down to the libturbojpeg0-dev package, install that, and after a
> bit more fussing around now have jpegdupes installed.
>
> Thanks for the help,
>

Sweet! I like stories that have happy endings.

(No compilers were harmed in the making of this.)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Package conflicts trying to install jpegdupes package [ In reply to ]

2QdxY4RzWzUUiLuE at potatochowder

Sep 27, 2021, 6:19 PM

Post #14 of 14 (253 views)

On 2021-09-28 at 10:44:02 +1000,
Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Sep 28, 2021 at 10:40 AM Skip Montanaro
> <skip.montanaro@gmail.com> wrote:
> >
> > Woo hoo! It's installed. The ultimate error was a missing turbojpeg.h
> > file. Thank goodness for the apt-file command. I was able to track
> > that down to the libturbojpeg0-dev package, install that, and after a
> > bit more fussing around now have jpegdupes installed.
> >
> > Thanks for the help,
> >
>
> Sweet! I like stories that have happy endings.
>
> (No compilers were harmed in the making of this.)

Nor compiler authors/vendors/maintainers.
--
https://mail.python.org/mailman/listinfo/python-list