To confirm, I used your test case on an 0.103.0-rc2 build and got:
? bin/clamscan -d mydb.cdb just.rar
/home/micasnyd/.clamav/just.rar: Archived_EXE.UNOFFICIAL FOUND
----------- SCAN SUMMARY -----------
Known viruses: 1
Engine version: 0.103.0-rc2
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 0.005 sec (0 m 0 s)
Start Date: 2020:10:06 16:40:18
End Date: 2020:10:06 16:40:18
From: clamav-users <clamav-users-bounces@lists.clamav.net> On Behalf Of Micah Snyder (micasnyd) via clamav-users
Sent: Tuesday, October 6, 2020 4:35 PM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Micah Snyder (micasnyd) <micasnyd@cisco.com>
Subject: Re: [clamav-users] possible rar issues when files have special characters
Iulian,
That sounds like the same issue to me. Sorry for the frustration! It should be working ok now in 0.103.
Best,
Micah
From: clamav-users <clamav-users-bounces@lists.clamav.net<mailto:clamav-users-bounces@lists.clamav.net>> On Behalf Of iulian stan via clamav-users
Sent: Monday, October 5, 2020 4:33 AM
To: clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
Cc: iulian stan <iulian@sphere.ro<mailto:iulian@sphere.ro>>
Subject: Re: [clamav-users] possible rar issues when files have special characters
Again me,
Ups, now i found this.
Micah, can you confirm it's the same issue that you fixed in 0.103 ?
"
The UnRAR library requires the character-classification locale to be set
to the empty string "" so it will be set according to the environment
variables, as seeen in the rar.cpp example application `main()`.
Without this, extracting RAR archives containing unicode filenames on
non-Windows, non-macOS operating systems may fail
"
---
Best regards,
Iulian
On 2020-10-05 14:13, iulian stan via clamav-users wrote:
Hi all,
I've filled a bug:
https://bugzilla.clamav.net/show_bug.cgi?id=12621 The problem is present on Gentoo(installed via emerge) and Ubuntu 18.04.1 LTS(installed via apt-get). If others can test on other platforms will be great.
Funny thing: on Windows ( clamav-0.102.4-win-x64-portable.zip) it is working ok.
---
Best regards,
Iulian
On 2020-10-04 16:40, Iulian Stan wrote:
Hello,
I didn't had time to investigate too much since is weekend and family will be really unhappy:))
Since the whole investigation was made on the phone i will be brief.
--leave-temps doesn't provide any clue but debug clarifies the problem.
Unfortunately we face a bug(i will also look tomorrow for what is reported already).
Simple put when special characters are set the name of the file(including file extension) is truncated.
With special caracter:
LibClamAV debug: Checking realpath of just.rar
LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
LibClamAV debug: Recognized RAR file
LibClamAV debug: cache_check: 2c04496b1308e6349e3726f91e156235 is negative
LibClamAV debug: in scanrar()
unrar_open: Comments are not present in this archive.
unrar_open: Volume attribute (archive volume): no
unrar_open: Archive comment present: no
unrar_open: Archive lock attribute: no
unrar_open: Solid attribute (solid archive): no
unrar_open: New volume naming scheme ('volname.partN.rar'): yes
unrar_open: Authenticity information present (obsolete): no
unrar_open: Recovery record present: no
unrar_open: Block headers are encrypted: no
unrar_open: First volume (set only by RAR 3.0 and later): no
unrar_open: Opened archive: /home/iulian/viruses/1/just.rar
unrar_peek_file_header: Name: CONSILIERE PLAT
unrar_peek_file_header: Directory?: 0
unrar_peek_file_header: Target Dir: 0
unrar_peek_file_header: RAR Version: 50
unrar_peek_file_header: Packed Size: 5
unrar_peek_file_header: Unpacked Size: 5
LibClamAV debug: RAR: CONSILIERE PLAT, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1
LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLAT:5:5:0:1:1001993670:(nil)
LibClamAV debug: RAR: Extracting file: CONSILIERE PLAT to /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmp
unrar_extract_file: Extracted file to: /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmp
LibClamAV debug: RAR: Extraction complete. Scanning now...
LibClamAV debug: in cli_magic_scandesc (reclevel: 1/16)
LibClamAV debug: Small data (5 bytes)
LibClamAV debug: cli_magic_scandesc: returning 0 at line 4057 (no post, no cache)
unrar_retcode: No more files in archive.
LibClamAV debug: RAR: No more files in archive.
LibClamAV debug: RAR: Exit code: 0
LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0
LibClamAV debug: cli_magic_scandesc: returning 0 at line 3202
LibClamAV debug: cache_add: 2c04496b1308e6349e3726f91e156235 (level 0)
/home/iulian/viruses/1/just.rar: OK
LibClamAV debug: Cleaning up phishcheck
LibClamAV debug: Freeing phishcheck struct
LibClamAV debug: Phishcheck cleaned up
Without special characters:
LibClamAV debug: Checking realpath of anothertest.rar
LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
LibClamAV debug: Recognized RAR file
LibClamAV debug: cache_check: bbe25db3191912601ee2b12860c99627 is negative
LibClamAV debug: in scanrar()
unrar_open: Comments are not present in this archive.
unrar_open: Volume attribute (archive volume): no
unrar_open: Archive comment present: no
unrar_open: Archive lock attribute: no
unrar_open: Solid attribute (solid archive): no
unrar_open: New volume naming scheme ('volname.partN.rar'): yes
unrar_open: Authenticity information present (obsolete): no
unrar_open: Recovery record present: no
unrar_open: Block headers are encrypted: no
unrar_open: First volume (set only by RAR 3.0 and later): no
unrar_open: Opened archive: /home/iulian/viruses/1/anothertest.rar
unrar_peek_file_header: Name: CONSILIERE PLATA_Pdf.exe
unrar_peek_file_header: Directory?: 0
unrar_peek_file_header: Target Dir: 0
unrar_peek_file_header: RAR Version: 50
unrar_peek_file_header: Packed Size: 5
unrar_peek_file_header: Unpacked Size: 5
LibClamAV debug: RAR: CONSILIERE PLATA_Pdf.exe, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1
LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLATA_Pdf.exe:5:5:0:1:1001993670:(nil)
LibClamAV debug: FP SIGNATURE: bbe25db3191912601ee2b12860c99627:95:Archived_EXE.UNOFFICIAL/home/iulian/viruses/1/anothertest.rar: Archived_EXE.UNOFFICIAL FOUND
LibClamAV debug: RAR: Exit code: 1
LibClamAV debug: cli_magic_scandesc: returning 1 at line 3202
LibClamAV debug: Cleaning up phishcheck
LibClamAV debug: Freeing phishcheck struct
LibClamAV debug: Phishcheck cleaned up
Best regads,
Iulian
Sent from my Samsung Galaxy smartphone.
Hello, I didn't had time to investigate too much since is weekend and family will be really unhappy:))Since the whole investigation was made on the phone i will be brief.--leave-temps doesn't provide any clue but debug clarifies the problem.Unfortunately we face a bug(i will also look tomorrow for what is reported already).Simple put when special characters are set the name of the file(including file extension) is truncated.With special caracter:LibClamAV debug: Checking realpath of just.rarLibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)LibClamAV debug: Recognized RAR fileLibClamAV debug: cache_check: 2c04496b1308e6349e3726f91e156235 is negativeLibClamAV debug: in scanrar()unrar_open: Comments are not present in this archive.unrar_open: Volume attribute (archive volume): nounrar_open: Archive comment present: nounrar_open: Archive lock attribute: nounrar_open: Solid attribute (solid archive): nounrar_open: New volume naming scheme ('volname.partN.rar'): yesunrar_open: Authenticity information present (obsolete): nounrar_open: Recovery record present: nounrar_open: Block headers are encrypted: nounrar_open: First volume (set only by RAR 3.0 and later): nounrar_open: Opened archive: /home/iulian/viruses/1/just.rarunrar_peek_file_header: Name: CONSILIERE PLATunrar_peek_file_header: Directory?: 0unrar_peek_file_header: Target Dir: 0unrar_peek_file_header: RAR Version: 50unrar_peek_file_header: Packed Size: 5unrar_peek_file_header: Unpacked Size: 5LibClamAV debug: RAR: CONSILIERE PLAT, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLAT:5:5:0:1:1001993670:(nil)LibClamAV debug: RAR: Extracting file: CONSILIERE PLAT to /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmpunrar_extract_file: Extracted file to: /tmp/just.rar.01e96/clamav-2b546d5049d12d4cfcee3cd6e993f061.tmpLibClamAV debug: RAR: Extraction complete. Scanning now...LibClamAV debug: in cli_magic_scandesc (reclevel: 1/16)LibClamAV debug: Small data (5 bytes)LibClamAV debug: cli_magic_scandesc: returning 0 at line 4057 (no post, no cache)unrar_retcode: No more files in archive.LibClamAV debug: RAR: No more files in archive.LibClamAV debug: RAR: Exit code: 0LibClamAV debug: hashtab: Freeing hashset, elements: 0, capacity: 0LibClamAV debug: cli_magic_scandesc: returning 0 at line 3202LibClamAV debug: cache_add: 2c04496b1308e6349e3726f91e156235 (level 0)/home/iulian/viruses/1/just.rar: OKLibClamAV debug: Cleaning up phishcheckLibClamAV debug: Freeing phishcheck structLibClamAV debug: Phishcheck cleaned upWithout special characters:LibClamAV debug: Checking realpath of anothertest.rarLibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)LibClamAV debug: Recognized RAR fileLibClamAV debug: cache_check: bbe25db3191912601ee2b12860c99627 is negativeLibClamAV debug: in scanrar()unrar_open: Comments are not present in this archive.unrar_open: Volume attribute (archive volume): nounrar_open: Archive comment present: nounrar_open: Archive lock attribute: nounrar_open: Solid attribute (solid archive): nounrar_open: New volume naming scheme ('volname.partN.rar'): yesunrar_open: Authenticity information present (obsolete): nounrar_open: Recovery record present: nounrar_open: Block headers are encrypted: nounrar_open: First volume (set only by RAR 3.0 and later): nounrar_open: Opened archive: /home/iulian/viruses/1/anothertest.rarunrar_peek_file_header: Name: CONSILIERE PLATA_Pdf.exeunrar_peek_file_header: Directory?: 0unrar_peek_file_header: Target Dir: 0unrar_peek_file_header: RAR Version: 50unrar_peek_file_header: Packed Size: 5unrar_peek_file_header: Unpacked Size: 5LibClamAV debug: RAR: CONSILIERE PLATA_Pdf.exe, crc32: 0x3bb935c6, encrypted: 0, compressed: 5, normal: 5, method: 48, ratio: 1LibClamAV debug: CDBNAME:CL_TYPE_RAR:5:CONSILIERE PLATA_Pdf.exe:5:5:0:1:1001993670:(nil)LibClamAV debug: FP SIGNATURE: bbe25db3191912601ee2b12860c99627:95:Archived_EXE.UNOFFICIAL/home/iulian/viruses/1/anothertest.rar: Archived_EXE.UNOFFICIAL FOUNDLibClamAV debug: RAR: Exit code: 1LibClamAV debug: cli_magic_scandesc: returning 1 at line 3202LibClamAV debug: Cleaning up phishcheckLibClamAV debug: Freeing phishcheck structLibClamAV debug: Phishcheck cleaned upBest regads,IulianSent from my Samsung Galaxy smartphone.
-------- Original message --------
From: "G.W. Haywood via clamav-users" <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
Date: 10/4/20 12:27 (GMT+02:00)
To: iulian stan via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
Cc: "G.W. Haywood" <clamav@jubileegroup.co.uk<mailto:clamav@jubileegroup.co.uk>>
Subject: Re: [clamav-users] possible rar issues when files have special characters
Hi there,
On Sun, 4 Oct 2020, iulian stan via clamav-users wrote:
> I know that relying on the file extension is not perfect but i will
> say it is covering most of the threats.
Understood, a pragmatic approach.
> Anyhow my raised question was about: Why .exe is not detected when
> the file inside archive has a special character? This problem is
> manifesting only with RAR. For files which don't have special
> character RAR is behaving as expected.
Good question. Perhaps if you use the --leave-temps option and
inspect the temporary files left after scanning it might shed some
light on the issue. Have you checked the ClamAV Bugzilla issues to
see if there's anything similar mentioned?
Does the same thing also happen if you use clamdscan instead?
Can you simply block all .rar files? I do that for mail, but I don't
generally scan filesystems at all.
--
73,
Ged.
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml -------- Original message --------From: "G.W. Haywood via clamav-users" <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> Date: 10/4/20 12:27 (GMT+02:00) To: iulian stan via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> Cc: "G.W. Haywood" <clamav@jubileegroup.co.uk<mailto:clamav@jubileegroup.co.uk>> Subject: Re: [clamav-users] possible rar issues when files have special
characters Hi there,On Sun, 4 Oct 2020, iulian stan via clamav-users wrote:> I know that relying on the file extension is not perfect but i will> say it is covering most of the threats.Understood, a pragmatic approach.> Anyhow my raised question was about: Why .exe is not detected when> the file inside archive has a special character? This problem is> manifesting only with RAR. For files which don't have special> character RAR is behaving as expected.Good question. Perhaps if you use the --leave-temps option andinspect the temporary files left after scanning it might shed somelight on the issue. Have you checked the ClamAV Bugzilla issues tosee if there's anything similar mentioned?Does the same thing also happen if you use clamdscan instead?Can you simply block all .rar files? I do that for mail, but I don'tgenerally scan filesystems at all.-- 73,Ged._______________________________________________clamav-users mailing listclamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-usersHelp<mailto:listclamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-usersHelp> us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faqhttp://www.clamav.net/contact.html#ml<
https://github.com/vrtadmin/clamav-faqhttp:/www.clamav.net/contact.html#ml>
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml