Mailing List Archive

Building ClamAV 0.99 with PCRE support
Hi all,

Are there any recommendations for compilation options on/with pcre?

I've tried several things but can't seem to get the build to work on anything other than OS X 10.11. I never have a problem moving my ClamAV builds between machines, but something's going wrong with PCRE support - even when I build PCRE on the destination machine, I always end up with the following error from clamscan:

LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0: unknown option bit(s) set
LibClamAV Error: cli_pcre_build: failed to build pcre regex
ERROR: Database initialization error: Malformed database


Here's a section of the output with --debug on:

LibClamAV debug: Ignoring signature Email.Trojan-417
LibClamAV debug: main.ndb loaded
LibClamAV debug: main.zmd loaded
LibClamAV debug: main.fp loaded
LibClamAV debug: in cli_tgzload_cleanup()
LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
LibClamAV debug: Using filter for trie 0
LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0: unknown option bit(s) set
LibClamAV Error: cli_pcre_build: failed to build pcre regex
ERROR: Database initialization error: Malformed database
LibClamAV debug: Cleaning up phishcheck
LibClamAV debug: Freeing phishcheck struct
LibClamAV debug: Phishcheck cleaned up

These are the options I'm passing to pcre's configure phase:
./configure --prefix=/usr/local --enable-newline-is-any --enable-utf --enable-unicode-properties --enable-rebuild-chartables --enable-pcre16 --enable-pcre32 --enable-jit

This is what I'm passing to ClamAV's configure phase:
./configure --disable-dependency-tracking --enable-llvm=no --enable-clamdtop --with-user=_clamav --with-group=_clamav --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local

I get the same results regardless of what options I pass to PCRE's configure script. I've also tried pcre-8.37 and pcre-8.38.

Can anyone suggest anything?

Many thanks
Mark

> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com> wrote:
>
> Hi Mark,
>
> Unfortunately, as of right now the only way to get pcre 8.38 is via their
> rc1 candidate (check the pcre-dev mailing list for a tarball).
>
> In practice, the pcre exploit ClamAV warns about (
> http://www.securitytracker.com/id/1032453) relies upon an explicitly
> malicious regex, so you don't have to worry too much unless you're using
> untrusted sigs. Everything should still compile and run just fine, even
> with 8.37.
>
> - Mickey
>
> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com> wrote:
>
>> Hi all,
>>
>> I saw the blog post about v0.99 rc 2 and have downloaded it for testing.
>>
>> It looks like bug 11411 [
>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open, so I
>> decided to download and build PCRE as well.
>>
>> I initially tried the PCRE2 branch but it wasn't recognised by ClamAV's
>> configure script, so I went with the most up-to-date version of PCRE (which
>> is currently 8.37) but now configure outputs the following:
>>
>> configure: WARNING: The installed pcre version may contain a security bug.
>> Please upgrade to 8.38 or later: http://www.pcre.org
>>
>> There is no 8.38 that I can see:
>> https://sourceforge.net/projects/pcre/files/pcre/
>>
>> Are you just assuming that 8.38 will be coming soon to fix the bug, or is
>> there a download somewhere that I'm not seeing?
>>
>> Thanks
>> Mark
>>
>> _______________________________________________
>> http://lurker.clamav.net/list/clamav-devel.html
>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>
>> http://www.clamav.net/contact.html#ml
>>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: Building ClamAV 0.99 with PCRE support [ In reply to ]
It appears that the PCRE library is correctly linking in and ClamAV is
making calls to it. The error message:

LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
unknown option bit(s) set

results directly from a failed compilation of PCRE regex which in this case
is due to an unknown option bit being set.

Looking into it, the options that can be passed to pcre_compile are fairly
common ones; the only real exception is PCRE_NEVER_UTF which was added in
8.33. It's possible that the flag existed on the source machine but not the
destination.Are the PCRE configure options consistent across the source and
all the destination machines?

-Kevin



On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan@gmail.com> wrote:

> Hi all,
>
> Are there any recommendations for compilation options on/with pcre?
>
> I've tried several things but can't seem to get the build to work on
> anything other than OS X 10.11. I never have a problem moving my ClamAV
> builds between machines, but something's going wrong with PCRE support -
> even when I build PCRE on the destination machine, I always end up with the
> following error from clamscan:
>
> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> unknown option bit(s) set
> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> ERROR: Database initialization error: Malformed database
>
>
> Here's a section of the output with --debug on:
>
> LibClamAV debug: Ignoring signature Email.Trojan-417
> LibClamAV debug: main.ndb loaded
> LibClamAV debug: main.zmd loaded
> LibClamAV debug: main.fp loaded
> LibClamAV debug: in cli_tgzload_cleanup()
> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
> LibClamAV debug: Using filter for trie 0
> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> unknown option bit(s) set
> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> ERROR: Database initialization error: Malformed database
> LibClamAV debug: Cleaning up phishcheck
> LibClamAV debug: Freeing phishcheck struct
> LibClamAV debug: Phishcheck cleaned up
>
> These are the options I'm passing to pcre's configure phase:
> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
> --enable-unicode-properties --enable-rebuild-chartables --enable-pcre16
> --enable-pcre32 --enable-jit
>
> This is what I'm passing to ClamAV's configure phase:
> ./configure --disable-dependency-tracking --enable-llvm=no
> --enable-clamdtop --with-user=_clamav --with-group=_clamav
> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
>
> I get the same results regardless of what options I pass to PCRE's
> configure script. I've also tried pcre-8.37 and pcre-8.38.
>
> Can anyone suggest anything?
>
> Many thanks
> Mark
>
> > On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com> wrote:
> >
> > Hi Mark,
> >
> > Unfortunately, as of right now the only way to get pcre 8.38 is via their
> > rc1 candidate (check the pcre-dev mailing list for a tarball).
> >
> > In practice, the pcre exploit ClamAV warns about (
> > http://www.securitytracker.com/id/1032453) relies upon an explicitly
> > malicious regex, so you don't have to worry too much unless you're using
> > untrusted sigs. Everything should still compile and run just fine, even
> > with 8.37.
> >
> > - Mickey
> >
> > On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> I saw the blog post about v0.99 rc 2 and have downloaded it for testing.
> >>
> >> It looks like bug 11411 [
> >> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open, so I
> >> decided to download and build PCRE as well.
> >>
> >> I initially tried the PCRE2 branch but it wasn't recognised by ClamAV's
> >> configure script, so I went with the most up-to-date version of PCRE
> (which
> >> is currently 8.37) but now configure outputs the following:
> >>
> >> configure: WARNING: The installed pcre version may contain a security
> bug.
> >> Please upgrade to 8.38 or later: http://www.pcre.org
> >>
> >> There is no 8.38 that I can see:
> >> https://sourceforge.net/projects/pcre/files/pcre/
> >>
> >> Are you just assuming that 8.38 will be coming soon to fix the bug, or
> is
> >> there a download somewhere that I'm not seeing?
> >>
> >> Thanks
> >> Mark
> >>
> >> _______________________________________________
> >> http://lurker.clamav.net/list/clamav-devel.html
> >> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>
> >> http://www.clamav.net/contact.html#ml
> >>
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: Building ClamAV 0.99 with PCRE support [ In reply to ]
Hi Kevin,

Thanks.

Yes, the configure options are definitely the same. In fact PCRE isn't installed on OS X by default, so I'm compiling it as well and copying all the binaries & libraries etc over to the destination machines at the same time as copying the ClamAV binaries.

Mark

> On 8 Dec 2015, at 6:36 pm, Kevin Lin <klin@sourcefire.com> wrote:
>
> It appears that the PCRE library is correctly linking in and ClamAV is
> making calls to it. The error message:
>
> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> unknown option bit(s) set
>
> results directly from a failed compilation of PCRE regex which in this case
> is due to an unknown option bit being set.
>
> Looking into it, the options that can be passed to pcre_compile are fairly
> common ones; the only real exception is PCRE_NEVER_UTF which was added in
> 8.33. It's possible that the flag existed on the source machine but not the
> destination.Are the PCRE configure options consistent across the source and
> all the destination machines?
>
> -Kevin
>
>
>
> On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan@gmail.com> wrote:
>
>> Hi all,
>>
>> Are there any recommendations for compilation options on/with pcre?
>>
>> I've tried several things but can't seem to get the build to work on
>> anything other than OS X 10.11. I never have a problem moving my ClamAV
>> builds between machines, but something's going wrong with PCRE support -
>> even when I build PCRE on the destination machine, I always end up with the
>> following error from clamscan:
>>
>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
>> unknown option bit(s) set
>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
>> ERROR: Database initialization error: Malformed database
>>
>>
>> Here's a section of the output with --debug on:
>>
>> LibClamAV debug: Ignoring signature Email.Trojan-417
>> LibClamAV debug: main.ndb loaded
>> LibClamAV debug: main.zmd loaded
>> LibClamAV debug: main.fp loaded
>> LibClamAV debug: in cli_tgzload_cleanup()
>> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
>> LibClamAV debug: Using filter for trie 0
>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
>> unknown option bit(s) set
>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
>> ERROR: Database initialization error: Malformed database
>> LibClamAV debug: Cleaning up phishcheck
>> LibClamAV debug: Freeing phishcheck struct
>> LibClamAV debug: Phishcheck cleaned up
>>
>> These are the options I'm passing to pcre's configure phase:
>> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
>> --enable-unicode-properties --enable-rebuild-chartables --enable-pcre16
>> --enable-pcre32 --enable-jit
>>
>> This is what I'm passing to ClamAV's configure phase:
>> ./configure --disable-dependency-tracking --enable-llvm=no
>> --enable-clamdtop --with-user=_clamav --with-group=_clamav
>> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
>>
>> I get the same results regardless of what options I pass to PCRE's
>> configure script. I've also tried pcre-8.37 and pcre-8.38.
>>
>> Can anyone suggest anything?
>>
>> Many thanks
>> Mark
>>
>>> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com> wrote:
>>>
>>> Hi Mark,
>>>
>>> Unfortunately, as of right now the only way to get pcre 8.38 is via their
>>> rc1 candidate (check the pcre-dev mailing list for a tarball).
>>>
>>> In practice, the pcre exploit ClamAV warns about (
>>> http://www.securitytracker.com/id/1032453) relies upon an explicitly
>>> malicious regex, so you don't have to worry too much unless you're using
>>> untrusted sigs. Everything should still compile and run just fine, even
>>> with 8.37.
>>>
>>> - Mickey
>>>
>>> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com>
>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I saw the blog post about v0.99 rc 2 and have downloaded it for testing.
>>>>
>>>> It looks like bug 11411 [
>>>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open, so I
>>>> decided to download and build PCRE as well.
>>>>
>>>> I initially tried the PCRE2 branch but it wasn't recognised by ClamAV's
>>>> configure script, so I went with the most up-to-date version of PCRE
>> (which
>>>> is currently 8.37) but now configure outputs the following:
>>>>
>>>> configure: WARNING: The installed pcre version may contain a security
>> bug.
>>>> Please upgrade to 8.38 or later: http://www.pcre.org
>>>>
>>>> There is no 8.38 that I can see:
>>>> https://sourceforge.net/projects/pcre/files/pcre/
>>>>
>>>> Are you just assuming that 8.38 will be coming soon to fix the bug, or
>> is
>>>> there a download somewhere that I'm not seeing?
>>>>
>>>> Thanks
>>>> Mark
>>>>
>>>> _______________________________________________
>>>> http://lurker.clamav.net/list/clamav-devel.html
>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>>
>>>> http://www.clamav.net/contact.html#ml
>>>>
>>> _______________________________________________
>>> http://lurker.clamav.net/list/clamav-devel.html
>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>
>>> http://www.clamav.net/contact.html#ml
>>
>> _______________________________________________
>> http://lurker.clamav.net/list/clamav-devel.html
>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>
>> http://www.clamav.net/contact.html#ml
>>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: Building ClamAV 0.99 with PCRE support [ In reply to ]
Can I ask you to try this patch and tell me if it fixes the issue? If the
issue persists, please submit the debug log. Thanks.

-Kevin

On Tue, Dec 8, 2015 at 2:00 PM, Mark Allan <markjallan@gmail.com> wrote:

> Hi Kevin,
>
> Thanks.
>
> Yes, the configure options are definitely the same. In fact PCRE isn't
> installed on OS X by default, so I'm compiling it as well and copying all
> the binaries & libraries etc over to the destination machines at the same
> time as copying the ClamAV binaries.
>
> Mark
>
> > On 8 Dec 2015, at 6:36 pm, Kevin Lin <klin@sourcefire.com> wrote:
> >
> > It appears that the PCRE library is correctly linking in and ClamAV is
> > making calls to it. The error message:
> >
> > LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> > unknown option bit(s) set
> >
> > results directly from a failed compilation of PCRE regex which in this
> case
> > is due to an unknown option bit being set.
> >
> > Looking into it, the options that can be passed to pcre_compile are
> fairly
> > common ones; the only real exception is PCRE_NEVER_UTF which was added in
> > 8.33. It's possible that the flag existed on the source machine but not
> the
> > destination.Are the PCRE configure options consistent across the source
> and
> > all the destination machines?
> >
> > -Kevin
> >
> >
> >
> > On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> Are there any recommendations for compilation options on/with pcre?
> >>
> >> I've tried several things but can't seem to get the build to work on
> >> anything other than OS X 10.11. I never have a problem moving my ClamAV
> >> builds between machines, but something's going wrong with PCRE support -
> >> even when I build PCRE on the destination machine, I always end up with
> the
> >> following error from clamscan:
> >>
> >> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >> unknown option bit(s) set
> >> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >> ERROR: Database initialization error: Malformed database
> >>
> >>
> >> Here's a section of the output with --debug on:
> >>
> >> LibClamAV debug: Ignoring signature Email.Trojan-417
> >> LibClamAV debug: main.ndb loaded
> >> LibClamAV debug: main.zmd loaded
> >> LibClamAV debug: main.fp loaded
> >> LibClamAV debug: in cli_tgzload_cleanup()
> >> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
> >> LibClamAV debug: Using filter for trie 0
> >> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >> unknown option bit(s) set
> >> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >> ERROR: Database initialization error: Malformed database
> >> LibClamAV debug: Cleaning up phishcheck
> >> LibClamAV debug: Freeing phishcheck struct
> >> LibClamAV debug: Phishcheck cleaned up
> >>
> >> These are the options I'm passing to pcre's configure phase:
> >> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
> >> --enable-unicode-properties --enable-rebuild-chartables --enable-pcre16
> >> --enable-pcre32 --enable-jit
> >>
> >> This is what I'm passing to ClamAV's configure phase:
> >> ./configure --disable-dependency-tracking --enable-llvm=no
> >> --enable-clamdtop --with-user=_clamav --with-group=_clamav
> >> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
> >>
> >> I get the same results regardless of what options I pass to PCRE's
> >> configure script. I've also tried pcre-8.37 and pcre-8.38.
> >>
> >> Can anyone suggest anything?
> >>
> >> Many thanks
> >> Mark
> >>
> >>> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com> wrote:
> >>>
> >>> Hi Mark,
> >>>
> >>> Unfortunately, as of right now the only way to get pcre 8.38 is via
> their
> >>> rc1 candidate (check the pcre-dev mailing list for a tarball).
> >>>
> >>> In practice, the pcre exploit ClamAV warns about (
> >>> http://www.securitytracker.com/id/1032453) relies upon an explicitly
> >>> malicious regex, so you don't have to worry too much unless you're
> using
> >>> untrusted sigs. Everything should still compile and run just fine, even
> >>> with 8.37.
> >>>
> >>> - Mickey
> >>>
> >>> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com>
> >> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> I saw the blog post about v0.99 rc 2 and have downloaded it for
> testing.
> >>>>
> >>>> It looks like bug 11411 [
> >>>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open,
> so I
> >>>> decided to download and build PCRE as well.
> >>>>
> >>>> I initially tried the PCRE2 branch but it wasn't recognised by
> ClamAV's
> >>>> configure script, so I went with the most up-to-date version of PCRE
> >> (which
> >>>> is currently 8.37) but now configure outputs the following:
> >>>>
> >>>> configure: WARNING: The installed pcre version may contain a security
> >> bug.
> >>>> Please upgrade to 8.38 or later: http://www.pcre.org
> >>>>
> >>>> There is no 8.38 that I can see:
> >>>> https://sourceforge.net/projects/pcre/files/pcre/
> >>>>
> >>>> Are you just assuming that 8.38 will be coming soon to fix the bug, or
> >> is
> >>>> there a download somewhere that I'm not seeing?
> >>>>
> >>>> Thanks
> >>>> Mark
> >>>>
> >>>> _______________________________________________
> >>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>
> >>>> http://www.clamav.net/contact.html#ml
> >>>>
> >>> _______________________________________________
> >>> http://lurker.clamav.net/list/clamav-devel.html
> >>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>
> >>> http://www.clamav.net/contact.html#ml
> >>
> >> _______________________________________________
> >> http://lurker.clamav.net/list/clamav-devel.html
> >> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>
> >> http://www.clamav.net/contact.html#ml
> >>
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
Re: Building ClamAV 0.99 with PCRE support [ In reply to ]
Hi Kevin,

Yes, that's fixed it. Thanks so much.

I still can't fathom why it would work differently on different versions of OS X, but it looks like you've solved the problem I'm seeing for now.

Thanks again
Mark

> On 8 Dec 2015, at 8:21 pm, Kevin Lin <klin@sourcefire.com> wrote:
>
> Can I ask you to try this patch and tell me if it fixes the issue? If the
> issue persists, please submit the debug log. Thanks.
>
> -Kevin
>
> On Tue, Dec 8, 2015 at 2:00 PM, Mark Allan <markjallan@gmail.com> wrote:
>
>> Hi Kevin,
>>
>> Thanks.
>>
>> Yes, the configure options are definitely the same. In fact PCRE isn't
>> installed on OS X by default, so I'm compiling it as well and copying all
>> the binaries & libraries etc over to the destination machines at the same
>> time as copying the ClamAV binaries.
>>
>> Mark
>>
>>> On 8 Dec 2015, at 6:36 pm, Kevin Lin <klin@sourcefire.com> wrote:
>>>
>>> It appears that the PCRE library is correctly linking in and ClamAV is
>>> making calls to it. The error message:
>>>
>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
>>> unknown option bit(s) set
>>>
>>> results directly from a failed compilation of PCRE regex which in this
>> case
>>> is due to an unknown option bit being set.
>>>
>>> Looking into it, the options that can be passed to pcre_compile are
>> fairly
>>> common ones; the only real exception is PCRE_NEVER_UTF which was added in
>>> 8.33. It's possible that the flag existed on the source machine but not
>> the
>>> destination.Are the PCRE configure options consistent across the source
>> and
>>> all the destination machines?
>>>
>>> -Kevin
>>>
>>>
>>>
>>> On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan@gmail.com>
>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Are there any recommendations for compilation options on/with pcre?
>>>>
>>>> I've tried several things but can't seem to get the build to work on
>>>> anything other than OS X 10.11. I never have a problem moving my ClamAV
>>>> builds between machines, but something's going wrong with PCRE support -
>>>> even when I build PCRE on the destination machine, I always end up with
>> the
>>>> following error from clamscan:
>>>>
>>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
>>>> unknown option bit(s) set
>>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
>>>> ERROR: Database initialization error: Malformed database
>>>>
>>>>
>>>> Here's a section of the output with --debug on:
>>>>
>>>> LibClamAV debug: Ignoring signature Email.Trojan-417
>>>> LibClamAV debug: main.ndb loaded
>>>> LibClamAV debug: main.zmd loaded
>>>> LibClamAV debug: main.fp loaded
>>>> LibClamAV debug: in cli_tgzload_cleanup()
>>>> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
>>>> LibClamAV debug: Using filter for trie 0
>>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
>>>> unknown option bit(s) set
>>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
>>>> ERROR: Database initialization error: Malformed database
>>>> LibClamAV debug: Cleaning up phishcheck
>>>> LibClamAV debug: Freeing phishcheck struct
>>>> LibClamAV debug: Phishcheck cleaned up
>>>>
>>>> These are the options I'm passing to pcre's configure phase:
>>>> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
>>>> --enable-unicode-properties --enable-rebuild-chartables --enable-pcre16
>>>> --enable-pcre32 --enable-jit
>>>>
>>>> This is what I'm passing to ClamAV's configure phase:
>>>> ./configure --disable-dependency-tracking --enable-llvm=no
>>>> --enable-clamdtop --with-user=_clamav --with-group=_clamav
>>>> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
>>>>
>>>> I get the same results regardless of what options I pass to PCRE's
>>>> configure script. I've also tried pcre-8.37 and pcre-8.38.
>>>>
>>>> Can anyone suggest anything?
>>>>
>>>> Many thanks
>>>> Mark
>>>>
>>>>> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com> wrote:
>>>>>
>>>>> Hi Mark,
>>>>>
>>>>> Unfortunately, as of right now the only way to get pcre 8.38 is via
>> their
>>>>> rc1 candidate (check the pcre-dev mailing list for a tarball).
>>>>>
>>>>> In practice, the pcre exploit ClamAV warns about (
>>>>> http://www.securitytracker.com/id/1032453) relies upon an explicitly
>>>>> malicious regex, so you don't have to worry too much unless you're
>> using
>>>>> untrusted sigs. Everything should still compile and run just fine, even
>>>>> with 8.37.
>>>>>
>>>>> - Mickey
>>>>>
>>>>> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I saw the blog post about v0.99 rc 2 and have downloaded it for
>> testing.
>>>>>>
>>>>>> It looks like bug 11411 [
>>>>>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open,
>> so I
>>>>>> decided to download and build PCRE as well.
>>>>>>
>>>>>> I initially tried the PCRE2 branch but it wasn't recognised by
>> ClamAV's
>>>>>> configure script, so I went with the most up-to-date version of PCRE
>>>> (which
>>>>>> is currently 8.37) but now configure outputs the following:
>>>>>>
>>>>>> configure: WARNING: The installed pcre version may contain a security
>>>> bug.
>>>>>> Please upgrade to 8.38 or later: http://www.pcre.org
>>>>>>
>>>>>> There is no 8.38 that I can see:
>>>>>> https://sourceforge.net/projects/pcre/files/pcre/
>>>>>>
>>>>>> Are you just assuming that 8.38 will be coming soon to fix the bug, or
>>>> is
>>>>>> there a download somewhere that I'm not seeing?
>>>>>>
>>>>>> Thanks
>>>>>> Mark
>>>>>>
>>>>>> _______________________________________________
>>>>>> http://lurker.clamav.net/list/clamav-devel.html
>>>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>>>>
>>>>>> http://www.clamav.net/contact.html#ml
>>>>>>
>>>>> _______________________________________________
>>>>> http://lurker.clamav.net/list/clamav-devel.html
>>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>>>
>>>>> http://www.clamav.net/contact.html#ml
>>>>
>>>> _______________________________________________
>>>> http://lurker.clamav.net/list/clamav-devel.html
>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>>
>>>> http://www.clamav.net/contact.html#ml
>>>>
>>> _______________________________________________
>>> http://lurker.clamav.net/list/clamav-devel.html
>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>>
>>> http://www.clamav.net/contact.html#ml
>>
>> _______________________________________________
>> http://lurker.clamav.net/list/clamav-devel.html
>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>>
>> http://www.clamav.net/contact.html#ml
>>
> <pcre_flags_try.patch>_______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: Building ClamAV 0.99 with PCRE support [ In reply to ]
In a short explanation, there's a pcre compiler flag we use to suppress UTF
regex patterns which is only available in certain versions of lpcre. The
transfer of binaries across different machines causes this flag to be
dropped by the lpcre binary though I'm not certain why even the direct
importing of the lpcre binary would result in the flag being dropped.

In the future, it might be best to remove the enforcement of the UTF
suppression and leave the issue up to the signature writer.

-Kevin

On Wed, Dec 9, 2015 at 6:19 AM, Mark Allan <markjallan@gmail.com> wrote:

> Hi Kevin,
>
> Yes, that's fixed it. Thanks so much.
>
> I still can't fathom why it would work differently on different versions
> of OS X, but it looks like you've solved the problem I'm seeing for now.
>
> Thanks again
> Mark
>
> > On 8 Dec 2015, at 8:21 pm, Kevin Lin <klin@sourcefire.com> wrote:
> >
> > Can I ask you to try this patch and tell me if it fixes the issue? If the
> > issue persists, please submit the debug log. Thanks.
> >
> > -Kevin
> >
> > On Tue, Dec 8, 2015 at 2:00 PM, Mark Allan <markjallan@gmail.com> wrote:
> >
> >> Hi Kevin,
> >>
> >> Thanks.
> >>
> >> Yes, the configure options are definitely the same. In fact PCRE isn't
> >> installed on OS X by default, so I'm compiling it as well and copying
> all
> >> the binaries & libraries etc over to the destination machines at the
> same
> >> time as copying the ClamAV binaries.
> >>
> >> Mark
> >>
> >>> On 8 Dec 2015, at 6:36 pm, Kevin Lin <klin@sourcefire.com> wrote:
> >>>
> >>> It appears that the PCRE library is correctly linking in and ClamAV is
> >>> making calls to it. The error message:
> >>>
> >>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>> unknown option bit(s) set
> >>>
> >>> results directly from a failed compilation of PCRE regex which in this
> >> case
> >>> is due to an unknown option bit being set.
> >>>
> >>> Looking into it, the options that can be passed to pcre_compile are
> >> fairly
> >>> common ones; the only real exception is PCRE_NEVER_UTF which was added
> in
> >>> 8.33. It's possible that the flag existed on the source machine but not
> >> the
> >>> destination.Are the PCRE configure options consistent across the source
> >> and
> >>> all the destination machines?
> >>>
> >>> -Kevin
> >>>
> >>>
> >>>
> >>> On Tue, Dec 8, 2015 at 12:15 PM, Mark Allan <markjallan@gmail.com>
> >> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> Are there any recommendations for compilation options on/with pcre?
> >>>>
> >>>> I've tried several things but can't seem to get the build to work on
> >>>> anything other than OS X 10.11. I never have a problem moving my
> ClamAV
> >>>> builds between machines, but something's going wrong with PCRE
> support -
> >>>> even when I build PCRE on the destination machine, I always end up
> with
> >> the
> >>>> following error from clamscan:
> >>>>
> >>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>>> unknown option bit(s) set
> >>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >>>> ERROR: Database initialization error: Malformed database
> >>>>
> >>>>
> >>>> Here's a section of the output with --debug on:
> >>>>
> >>>> LibClamAV debug: Ignoring signature Email.Trojan-417
> >>>> LibClamAV debug: main.ndb loaded
> >>>> LibClamAV debug: main.zmd loaded
> >>>> LibClamAV debug: main.fp loaded
> >>>> LibClamAV debug: in cli_tgzload_cleanup()
> >>>> LibClamAV debug: /usr/local/share/clamav/main.cvd loaded
> >>>> LibClamAV debug: Using filter for trie 0
> >>>> LibClamAV Error: cli_pcre_parse: PCRE compilation failed at offset 0:
> >>>> unknown option bit(s) set
> >>>> LibClamAV Error: cli_pcre_build: failed to build pcre regex
> >>>> ERROR: Database initialization error: Malformed database
> >>>> LibClamAV debug: Cleaning up phishcheck
> >>>> LibClamAV debug: Freeing phishcheck struct
> >>>> LibClamAV debug: Phishcheck cleaned up
> >>>>
> >>>> These are the options I'm passing to pcre's configure phase:
> >>>> ./configure --prefix=/usr/local --enable-newline-is-any --enable-utf
> >>>> --enable-unicode-properties --enable-rebuild-chartables
> --enable-pcre16
> >>>> --enable-pcre32 --enable-jit
> >>>>
> >>>> This is what I'm passing to ClamAV's configure phase:
> >>>> ./configure --disable-dependency-tracking --enable-llvm=no
> >>>> --enable-clamdtop --with-user=_clamav --with-group=_clamav
> >>>> --enable-all-jit-targets --with-pcre=/usr/local --prefix=/usr/local
> >>>>
> >>>> I get the same results regardless of what options I pass to PCRE's
> >>>> configure script. I've also tried pcre-8.37 and pcre-8.38.
> >>>>
> >>>> Can anyone suggest anything?
> >>>>
> >>>> Many thanks
> >>>> Mark
> >>>>
> >>>>> On 20 Nov 2015, at 6:01 pm, Mickey Sola <msola@sourcefire.com>
> wrote:
> >>>>>
> >>>>> Hi Mark,
> >>>>>
> >>>>> Unfortunately, as of right now the only way to get pcre 8.38 is via
> >> their
> >>>>> rc1 candidate (check the pcre-dev mailing list for a tarball).
> >>>>>
> >>>>> In practice, the pcre exploit ClamAV warns about (
> >>>>> http://www.securitytracker.com/id/1032453) relies upon an explicitly
> >>>>> malicious regex, so you don't have to worry too much unless you're
> >> using
> >>>>> untrusted sigs. Everything should still compile and run just fine,
> even
> >>>>> with 8.37.
> >>>>>
> >>>>> - Mickey
> >>>>>
> >>>>> On Fri, Nov 20, 2015 at 8:08 AM, Mark Allan <markjallan@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I saw the blog post about v0.99 rc 2 and have downloaded it for
> >> testing.
> >>>>>>
> >>>>>> It looks like bug 11411 [
> >>>>>> https://bugzilla.clamav.net/show_bug.cgi?id=11411 ] is still open,
> >> so I
> >>>>>> decided to download and build PCRE as well.
> >>>>>>
> >>>>>> I initially tried the PCRE2 branch but it wasn't recognised by
> >> ClamAV's
> >>>>>> configure script, so I went with the most up-to-date version of PCRE
> >>>> (which
> >>>>>> is currently 8.37) but now configure outputs the following:
> >>>>>>
> >>>>>> configure: WARNING: The installed pcre version may contain a
> security
> >>>> bug.
> >>>>>> Please upgrade to 8.38 or later: http://www.pcre.org
> >>>>>>
> >>>>>> There is no 8.38 that I can see:
> >>>>>> https://sourceforge.net/projects/pcre/files/pcre/
> >>>>>>
> >>>>>> Are you just assuming that 8.38 will be coming soon to fix the bug,
> or
> >>>> is
> >>>>>> there a download somewhere that I'm not seeing?
> >>>>>>
> >>>>>> Thanks
> >>>>>> Mark
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>>>
> >>>>>> http://www.clamav.net/contact.html#ml
> >>>>>>
> >>>>> _______________________________________________
> >>>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>>
> >>>>> http://www.clamav.net/contact.html#ml
> >>>>
> >>>> _______________________________________________
> >>>> http://lurker.clamav.net/list/clamav-devel.html
> >>>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>>
> >>>> http://www.clamav.net/contact.html#ml
> >>>>
> >>> _______________________________________________
> >>> http://lurker.clamav.net/list/clamav-devel.html
> >>> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>>
> >>> http://www.clamav.net/contact.html#ml
> >>
> >> _______________________________________________
> >> http://lurker.clamav.net/list/clamav-devel.html
> >> Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >>
> >> http://www.clamav.net/contact.html#ml
> >>
> > <pcre_flags_try.patch>_______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml