Syllable Forum Index Syllable
Syllable Forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Transferrer performance issues

 
Post new topic   Reply to topic    Syllable Forum Index -> Bugs
View previous topic :: View next topic  
Author Message
Morph
The Knights of Syllable


Joined: 18 Oct 2007
Posts: 316
Location: Australia

PostPosted: Mon May 24, 2010 4:52 pm    Post subject: Reply with quote

There's also a native GUI ftp app - Transferrer (http://forum.syllable.org/pmwiki/pmwiki.php?n=Projects.FTPClient). I don't know if anyone has tested it with large files, but I'd be happy to hear reports either way!
Back to top
View user's profile Send private message Visit poster's website
aritc02



Joined: 28 Feb 2008
Posts: 214

PostPosted: Mon May 24, 2010 7:57 pm    Post subject: Reply with quote

I have not tested it with large files yet since I have not had a need to, but I have downloaded a few 3 or 4 mb files without any problem and they were audio files to boot. I believe they were mp3 files.
Back to top
View user's profile Send private message
Ed



Joined: 01 May 2010
Posts: 32

PostPosted: Wed May 26, 2010 1:09 pm    Post subject: Reply with quote

Nice FTP app Smile I did some tests with Transferrer connecting to a Pure-ftpd in my LAN
environment:

- I can upload one 700MB iso file correctly. 1.5 MB/s data transfer approx (the terminal
ftp client goes up to 6 MB/s), high cpu usage (cpu 267 MHz). No RAM problems.

- I can download the same iso file ok. 200/300 KB/s data transfer (very slow for Ethernet
100 Mbps, though the terminal ftp achieves close to 600 KB/s as max), low cpu usage,
no RAM problems.

There is only one issue at least with my server using its default config. Transferrer
fails when I try to download one file after login the server (I must upload first one
file and then I can download files in the same session without errors). This happens
because as soon as the file request is done, the client starts to send 'tcp zero window' packets
on the data channel so the server side stops the transfer keeping alive the connection.

As I said, when I start uploading files everything (upload and download) works ok.
I tested downloading files from some Internet public servers and only one of them gave
me the same error, so I will keep using Transferrer for Internet downloads.

I observed another minor issue: It lost connection sometimes when I'm browsing the
server directory tree.
Back to top
View user's profile Send private message
Morph
The Knights of Syllable


Joined: 18 Oct 2007
Posts: 316
Location: Australia

PostPosted: Wed May 26, 2010 4:30 pm    Post subject: Reply with quote

Thanks for the report! High cpu usage is partly due to the extra debugging info that gets logged to the console, and due to the inefficient progress window updating. I don't know why the transfer rates are slower than the cli ftp app - Transferrer uses libcurl internally to do all the networking. If you get a chance, could you compare the rates between Transferrer and the cli curl util? Thanks! If curl is also slow compared to cli ftp then we can blame libcurl.

I don't know about the tcp zero window packets - that's a low-level tcp thing, isn't it, so possibly a bug in our tcp stack? Otherwise maybe another libcurl issue. Hopefully some of these libcurl issues will automatically disappear in a future version of libcurl Smile
Back to top
View user's profile Send private message Visit poster's website
Ed



Joined: 01 May 2010
Posts: 32

PostPosted: Thu May 27, 2010 9:07 am    Post subject: Reply with quote

Yea, I see those 'tcp zero window' packets in the traffic captured by Wireshark running
in the server machine. It's about TCP flow control.

Copy/paste (wiki.wireshark.org/TCP_Analyze_Sequence_Numbers):
"TCP ZeroWindow - Occurs when a receiver advertises a receive window size of zero. This
effectively tells the sender to stop sending because the receiver's buffer is full.
Indicates a resource issue on the receiver, as the application is not retrieving data
from the TCP buffer in a timely manner."

That resource issue could be an excesive cpu usage at the beginning of the first file
request in one stablished session. The tcp/ip stack has no cicles to handle the data
packets as fast as they are arriving, so it ask the server for decrease the window
size until reach zero which occurs quite fast. The cpu usage continues at 100% until I
close Transferrer (I can quit it in the right way, I don't need kill the process).
The tcp stack works as expected in this case.

The first test with 'curl' downloading one pdf file (2MB) was ok. 457.23KB/sec and low
cpu usage.

I'm going to do more test later with larger files and compare the performance between
curl and Transferrer (and cli ftp) taking the accurate data rate given by Pure-ftpd in
its logs. I'll try to isolate that error also and I'll post the result then.
Back to top
View user's profile Send private message
Kaj
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 2201
Location: Friesland

PostPosted: Thu May 27, 2010 9:40 am    Post subject: Reply with quote

I split off this thread from the virtual memory topic.
Back to top
View user's profile Send private message Visit poster's website
Morph
The Knights of Syllable


Joined: 18 Oct 2007
Posts: 316
Location: Australia

PostPosted: Thu May 27, 2010 5:40 pm    Post subject: Reply with quote

I suspect that it might be a glitch with libcurl. Sounds like libcurl requests the file, and the server starts sending it, but then libcurl forgets that it should be collecting the data from the kernel. Libcurl isn't really intended for interactive ftp, so Transferrer sometimes has to jump through hoops to get it working.
Back to top
View user's profile Send private message Visit poster's website
Ed



Joined: 01 May 2010
Posts: 32

PostPosted: Thu May 27, 2010 7:19 pm    Post subject: Reply with quote

Tests results with one file of 36MB.

Transferrer:
Download => 225.94KB/sec
Upload => 1557.34KB/sec
another Upload test => 1703.88KB/sec

Curl:
Download => 434.29KB/sec
Upload => 5196.08KB/sec

Ftp cli:
Download => 963.42KB/sec
Upload => 5282.28KB/sec

About the Transferrer gui error which is the cause of the 100% cpu and the 'tcp zero
window' packets:.

Maybe is a libcurl error as you said. Take in mind that this only occurs downloading.
Uploads (which is read and send data) works well. The curl command works ok downloading
because it is not interactive.

Thanks Smile
Back to top
View user's profile Send private message
Kaj
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 2201
Location: Friesland

PostPosted: Fri May 28, 2010 9:34 am    Post subject: Reply with quote

Looks like several problems combined. Something must be not quite right in our network stack, because these download numbers are bad. Then clearly Transferrer has a problem of its own that slows everything down. But cURL also seems to have an extra problem with downloading, being that much slower than INetUtils.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Syllable Forum Index -> Bugs All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group