Mailing List Archive

Upgrade win32 Pthreads
When compiling for x64 using Visual Studio 2013, ClamAV hangs when
scanning a file. The stack trace is below

ntoskrnl.exe!KeSetDmaIoCoherency+0xc06
ntoskrnl.exe!KeWaitForMultipleObjects+0x140f
ntoskrnl.exe!KeWaitForMultipleObjects+0xcb9
ntoskrnl.exe!KeWaitForMutexObject+0x2c0
ntoskrnl.exe!NtWaitForSingleObject+0xb2
ntoskrnl.exe!setjmpex+0x34a3
ntdll.dll!NtWaitForSingleObject+0xa
KernelBase.dll!WaitForSingleObjectEx+0x98
libclamav.dll!ptw32_mcs_flag_wait+0x6d
libclamav.dll!ptw32_mcs_lock_release+0x7e <------ Problem here
libclamav.dll!pthread_once+0xbc
libclamav.dll!cli_logg_setup+0x31
libclamav.dll!scan_common+0x2db
libclamav.dll!cl_scandesc_callback+0x5c
clamscan.exe!scanfile+0x2e7
clamscan.exe!scandirs+0x515
clamscan.exe!scanmanager+0x16cb
clamscan.exe!real_main+0x345
clamscan.exe!main+0x66
clamscan.exe!__tmainCRTStartup+0x19d
clamscan.exe!mainCRTStartup+0xe
kernel32.dll!BaseThreadInitThunk+0x22
ntdll.dll!RtlUserThreadStart+0x34

On x64, ptw32_mcs_lock_release incorrectly calls ptw32_mcs_flag_wait,
which internally creates an event and waits on it. The only way to ever
break out is if another thread signals the event, which could never happen
as clamscan only uses a single thread.

Additionally, it looks like the version of Pthreads included in ClamAV is
riddled with x64 problems, for example, ptw32_mcs_flag_wait stores the
event handle (pointer) in a 32-bit integer.
According to the changelog, many x64 problems have been fixed, including
the one in ptw32_mcs_flag_wait.

Upgrading to the latest version fixes this, and gets ClamAV running great
on x64 Windows compiled with VS 2013.

To upgrade, I simply replaced clamv\win32\3rdparty\pthreads with the files
here
ftp://sourceware.org/pub/pthreads-win32/sources/pthreads-w32-2-9-1-release/
The new pthread.h is missing some defines used in ClamAV, so I just
copied/pasted the old defines into the new pthread.h

// missing defines

/*
* WIN32 C runtime library had been made thread-safe
* without affecting the user interface. Provide
* mappings from the UNIX thread-safe versions to
* the standard C runtime library calls.
* Only provide function mappings for functions that
* actually exist on WIN32.
*/

#if !defined(__MINGW32__)
#define strtok_r( _s, _sep, _lasts ) \
( *(_lasts) = strtok( (_s), (_sep) ) )
#endif /* !__MINGW32__ */

#define asctime_r( _tm, _buf ) \
( strcpy( (_buf), asctime( (_tm) ) ), \
(_buf) )

#define ctime_r( _clock, _buf ) \
( strcpy( (_buf), ctime( (_clock) ) ), \
(_buf) )

/*
* gmtime(tm) and localtime(tm) return 0 if tm represents
* a time prior to 1/1/1970.
*/
#define gmtime_r( _clock, _result ) \
( gmtime( (_clock) ) \
? (*(_result) = *gmtime( (_clock) ), (_result) ) \
: (0) )

#define localtime_r( _clock, _result ) \
( localtime( (_clock) ) \
? (*(_result) = *localtime( (_clock) ), (_result) ) \
: (0) )

#define rand_r( _seed ) \
( _seed == _seed? rand() : rand() )
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: Upgrade win32 Pthreads [ In reply to ]
Thanks for your report. I've opened ticket 11253 at
http://clamav.bugzilla.net for tracking.

Steve

On Thu, Jan 29, 2015 at 9:43 PM, Andy Singer <andy@orbitech.org> wrote:

> When compiling for x64 using Visual Studio 2013, ClamAV hangs when
> scanning a file. The stack trace is below
>
> ntoskrnl.exe!KeSetDmaIoCoherency+0xc06
> ntoskrnl.exe!KeWaitForMultipleObjects+0x140f
> ntoskrnl.exe!KeWaitForMultipleObjects+0xcb9
> ntoskrnl.exe!KeWaitForMutexObject+0x2c0
> ntoskrnl.exe!NtWaitForSingleObject+0xb2
> ntoskrnl.exe!setjmpex+0x34a3
> ntdll.dll!NtWaitForSingleObject+0xa
> KernelBase.dll!WaitForSingleObjectEx+0x98
> libclamav.dll!ptw32_mcs_flag_wait+0x6d
> libclamav.dll!ptw32_mcs_lock_release+0x7e <------ Problem here
> libclamav.dll!pthread_once+0xbc
> libclamav.dll!cli_logg_setup+0x31
> libclamav.dll!scan_common+0x2db
> libclamav.dll!cl_scandesc_callback+0x5c
> clamscan.exe!scanfile+0x2e7
> clamscan.exe!scandirs+0x515
> clamscan.exe!scanmanager+0x16cb
> clamscan.exe!real_main+0x345
> clamscan.exe!main+0x66
> clamscan.exe!__tmainCRTStartup+0x19d
> clamscan.exe!mainCRTStartup+0xe
> kernel32.dll!BaseThreadInitThunk+0x22
> ntdll.dll!RtlUserThreadStart+0x34
>
> On x64, ptw32_mcs_lock_release incorrectly calls ptw32_mcs_flag_wait,
> which internally creates an event and waits on it. The only way to ever
> break out is if another thread signals the event, which could never happen
> as clamscan only uses a single thread.
>
> Additionally, it looks like the version of Pthreads included in ClamAV is
> riddled with x64 problems, for example, ptw32_mcs_flag_wait stores the
> event handle (pointer) in a 32-bit integer.
> According to the changelog, many x64 problems have been fixed, including
> the one in ptw32_mcs_flag_wait.
>
> Upgrading to the latest version fixes this, and gets ClamAV running great
> on x64 Windows compiled with VS 2013.
>
> To upgrade, I simply replaced clamv\win32\3rdparty\pthreads with the files
> here
> ftp://sourceware.org/pub/pthreads-win32/sources/pthreads-w32-2-9-1-release/
> The new pthread.h is missing some defines used in ClamAV, so I just
> copied/pasted the old defines into the new pthread.h
>
> // missing defines
>
> /*
> * WIN32 C runtime library had been made thread-safe
> * without affecting the user interface. Provide
> * mappings from the UNIX thread-safe versions to
> * the standard C runtime library calls.
> * Only provide function mappings for functions that
> * actually exist on WIN32.
> */
>
> #if !defined(__MINGW32__)
> #define strtok_r( _s, _sep, _lasts ) \
> ( *(_lasts) = strtok( (_s), (_sep) ) )
> #endif /* !__MINGW32__ */
>
> #define asctime_r( _tm, _buf ) \
> ( strcpy( (_buf), asctime( (_tm) ) ), \
> (_buf) )
>
> #define ctime_r( _clock, _buf ) \
> ( strcpy( (_buf), ctime( (_clock) ) ), \
> (_buf) )
>
> /*
> * gmtime(tm) and localtime(tm) return 0 if tm represents
> * a time prior to 1/1/1970.
> */
> #define gmtime_r( _clock, _result ) \
> ( gmtime( (_clock) ) \
> ? (*(_result) = *gmtime( (_clock) ), (_result) ) \
> : (0) )
>
> #define localtime_r( _clock, _result ) \
> ( localtime( (_clock) ) \
> ? (*(_result) = *localtime( (_clock) ), (_result) ) \
> : (0) )
>
> #define rand_r( _seed ) \
> ( _seed == _seed? rand() : rand() )
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml