Mailing List Archive

Long term plans for LLVM bytecode compilation?
Hi there everyone, two quick questions:

1) With 0.100.0 series release (congratulations everyone) and the deprecation of the repackaged LLVM, are we planning on updating the integration so it works with more recent LLVM releases? The 3.6 or lower requirement is going to be increasingly hard to meet when upstream is up to 7.x.

2) With regards to build systems, would there be any appetite for a migration from autotools to CMake? The benefits here would be better tooling integration (since CLion and Visual Studio supports it natively) and making the source code more approachable to new developers. Needless to say, this is not something worth PR’ing if there is no appetite for it.

Thanks!

- Rafael
_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: Long term plans for LLVM bytecode compilation? [ In reply to ]
Hi Rafael,

1:
Current thoughts here are to drop the repackaged LLVM entirely and to deprecate support for system-installed LLVM in favor of the bytecode interpreter. If we put in time to update support for system-installed LLVM, we'd be better served by aiming at the newest releases. Now that said, please note that LLVM's major version number started accelerating only in the last 3 years or so (http://releases.llvm.org/). They have very few minor releases these days.

2:
I am curious about switching from autotools to CMake, though that's a pretty huge task, and I've probably heard as much or more griping about Cmake as I have about autotools. I'd like to spend more time weighing pros and cons.




Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Dec 8, 2018, at 10:36 AM, Rafael Ferreira <raf@uvasoftware.com<mailto:raf@uvasoftware.com>> wrote:

Hi there everyone, two quick questions:

1) With 0.100.0 series release (congratulations everyone) and the deprecation of the repackaged LLVM, are we planning on updating the integration so it works with more recent LLVM releases? The 3.6 or lower requirement is going to be increasingly hard to meet when upstream is up to 7.x.

2) With regards to build systems, would there be any appetite for a migration from autotools to CMake? The benefits here would be better tooling integration (since CLion and Visual Studio supports it natively) and making the source code more approachable to new developers. Needless to say, this is not something worth PR’ing if there is no appetite for it.

Thanks!

- Rafael
_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net<mailto:clamav-devel@lists.clamav.net>
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: Long term plans for LLVM bytecode compilation? [ In reply to ]
Thanks for the reply Micah, are there any performance concerns with the bytecode interpreter? I hear you on cmake, I’m not a big fan of its syntax but it is, at the very least, more static analysis friendly.

> On Dec 8, 2018, at 9:30 AM, Micah Snyder (micasnyd) <micasnyd@cisco.com> wrote:
>
> Hi Rafael,
>
> 1:
> Current thoughts here are to drop the repackaged LLVM entirely and to deprecate support for system-installed LLVM in favor of the bytecode interpreter. If we put in time to update support for system-installed LLVM, we'd be better served by aiming at the newest releases. Now that said, please note that LLVM's major version number started accelerating only in the last 3 years or so (http://releases.llvm.org/). They have very few minor releases these days.
>
> 2:
> I am curious about switching from autotools to CMake, though that's a pretty huge task, and I've probably heard as much or more griping about Cmake as I have about autotools. I'd like to spend more time weighing pros and cons.
>
>
>
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
> On Dec 8, 2018, at 10:36 AM, Rafael Ferreira <raf@uvasoftware.com<mailto:raf@uvasoftware.com>> wrote:
>
> Hi there everyone, two quick questions:
>
> 1) With 0.100.0 series release (congratulations everyone) and the deprecation of the repackaged LLVM, are we planning on updating the integration so it works with more recent LLVM releases? The 3.6 or lower requirement is going to be increasingly hard to meet when upstream is up to 7.x.
>
> 2) With regards to build systems, would there be any appetite for a migration from autotools to CMake? The benefits here would be better tooling integration (since CLion and Visual Studio supports it natively) and making the source code more approachable to new developers. Needless to say, this is not something worth PR’ing if there is no appetite for it.
>
> Thanks!
>
> - Rafael
> _______________________________________________
> clamav-devel mailing list
> clamav-devel@lists.clamav.net<mailto:clamav-devel@lists.clamav.net>
> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel
>
> Please submit your patches to our Bugzilla: http://bugzilla.clamav.net
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> clamav-devel mailing list
> clamav-devel@lists.clamav.net
> http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel
>
> Please submit your patches to our Bugzilla: http://bugzilla.clamav.net
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: Long term plans for LLVM bytecode compilation? [ In reply to ]
Rafael,

Sorry about the [long] delay getting back to you.

Regarding LLVM vs the interpreter:
I wasn't involved in the performance profiling of LLVM/JIT vs the interpreter, as the testing was done a little before I joined, just after the interpreter was made operable again. I was told that the interpreter is a little slower, but due to the size of the bytecode functions, the performance difference between just-in-time bytecode compilation vs bytecode interpretation was insignificant.

I wish I had any sort of numbers to back up that claim, but I'm not aware of any written research materials left behind, and I'm too focused on other efforts at present to try to replicate their work. If anyone takes the initiative on this, bear in mind that bytecode functions are only invoked for certain file types, so any sort of benchmarking will have to take into account each of the file types for which bytecode signatures exist.

Presumably if more processor intensive bytecode signatures are written, the scale would tilt in favor of LLVM/JIT.

Regarding CMake:
We have a lot of custom logic with autoconf/automake to identify dependencies and platform specific features. I think it would be a bit of a nightmare to replicate all of this in Cmake.

That said, if a Cmake expert has the energy to put Cmake build scripts together I’m not opposed to having Cmake support alongside the existing autotools stuff. If eventually the Cmake infrastructure rivals our current autotools stuff, we could swap it out.

Micah


On Dec 8, 2018, at 2:33 PM, Rafael Ferreira <raf@uvasoftware.com> wrote:

Thanks for the reply Micah, are there any performance concerns with the bytecode interpreter? I hear you on cmake, I’m not a big fan of its syntax but it is, at the very least, more static analysis friendly.


_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml