Mailing List Archive

[Bug 1195] sftp corrupts large files on new macbook pro
http://bugzilla.mindrot.org/show_bug.cgi?id=1195

Summary: sftp corrupts large files on new macbook pro
Product: Portable OpenSSH
Version: 4.3p2
Platform: ix86
OS/Version: Mac OS X
Status: NEW
Severity: normal
Priority: P2
Component: sftp
AssignedTo: bitbucket@mindrot.org
ReportedBy: ski@allafrica.com


(originally reported to apple bug reporter, bug # 4529709, but they
punted)

Bug ID #: 4529709
Bug Title: sftp corrupts large files on new macbook pro
-------------------------------------------------------
<GMT27-Apr-2006 20:27:29GMT> Darrin Bodner:
Summary:
Large files (on the order of 1GB) are sometimes (~50%) corrupted when
transferred by using sftp on a new macbookpro (mac os x/intel arch)

Steps to Reproduce:
Enable sshd.
$ sftp localhost
> > get largefile largefile.copy
$ md5 largefile
$ md5 largefile.copy

The results usually differ. Using diff(1) confirms this is not a bug
with md5. Subsequent transfers can have the correct checksum or
different incorrect checksums, suggesting the corruption does not
always happen in the same place. sftp'ing a file from a remote server
running freebsd 6.1 - OpenSSH_4.2p1 FreeBSD-20050903, OpenSSL 0.9.7e-p1
25 Oct 2004 - also gives this problem.

Expected Results:
MD5 sums match, resulting file is not corrupt.

Actual Results:
File is corrupted ~50% of time for a 1GB file. Corruption rate seems to
increase with size.

Regression:
Tested on a new macbook pro with mac os x 10.4.6 with all patches
applied as of 4/27/2006. Transfers from two different freebsd machines
have similar results (different md5 sum, corrupted data).

Notes:
openssh from darwinports's sftp client has the same problem, so there
is likely an upstream bug. strangely, scp does not exhibit this bug
with either the darwinport or apple's built in command.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
http://www.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 1195] sftp corrupts large files on new macbook pro [ In reply to ]
http://bugzilla.mindrot.org/show_bug.cgi?id=1195


ski@allafrica.com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |ski@allafrica.com






------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
http://www.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 1195] sftp corrupts large files on new macbook pro [ In reply to ]
http://bugzilla.mindrot.org/show_bug.cgi?id=1195





------- Comment #1 from ski@allafrica.com 2006-06-09 07:24 -------
Bug ID #: 4529709
Bug Title: sftp corrupts large files on new macbook pro
-------------------------------------------------------
<GMT27-Apr-2006 20:27:29GMT> Darrin Bodner:
Summary:
Large files (on the order of 1GB are sometimes (~50%) corrupted when
transferred by using sftp).

Steps to Reproduce:
Enable sshd.
$ sftp localhost
> > get largefile largefile.copy
$ md5 largefile
$ md5 largefile.copy

The results usually differ. Using diff(1) confirms this is not a bug
with md5. Subsequent transfers can have the correct checksum or
different incorrect checksums, suggesting the corruption does not
always happen in the same place. Grabbing the file from a freebsd 6.1
server (with OpenSSH_4.2p1 FreeBSD-20050903, OpenSSL 0.9.7e-p1 25 Oct
2004) shows the same problem, indicating it is probably not a problem
with the server, but in the client instead.

Expected Results:
MD5 sums match, resulting file is not corrupt.

Actual Results:
File is corrupted ~50% of time for a 1GB file. Corruption rate seems to
increase with size.

Regression:
Tested on a new macbook pro with mac os x 10.4.6 with all patches
applied as of 4/27/2006. Transfers from two different freebsd machines
have similar results (different md5 sum, corrupted data).

Notes:
openssh from darwinports's sftp client has the same problem, so there
is likely an upstream bug. strangely, scp does not exhibit this bug
with either the darwinport or apple's built in command.




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
http://www.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 1195] sftp corrupts large files on new macbook pro [ In reply to ]
http://bugzilla.mindrot.org/show_bug.cgi?id=1195





------- Comment #2 from dtucker@zip.com.au 2006-06-09 20:15 -------
Does OpenSSL's self-test ("make tests") pass on that platform? Also,
does enabling/disabling compression make any difference? Do the files
all end up exactly the same size?

The thing that occurs to me as a possible cause is pipes/socketpairs:
both sftp and scp use them to talk to the underlying ssh process, but
scp uses a small fixed buffer (2kb) and sftp uses a variable sized
buffer that defaults to 32kb. If you repeat your test forcing the sftp
buffer to 2k ("sftp -B 2048") does the corruption still occur?




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
http://www.mindrot.org/mailman/listinfo/openssh-bugs