Me Too - SFTP Closed by Server

Get help with using the PHP Secure Communications Library.

Moderator: Nuxius

Forum rules
The purpose of this forum is to provide support for phpseclib, a pure PHP SSH / SFTP / RSA library.

Posts by new users are held in a moderation queue and are not publicly visible until the post is approved.

Me Too - SFTP Closed by Server

Postby rjking » Thu Jul 11, 2013 2:47 am

Hello TerraFrost!

First off, *amazing* project you have created here! I am having essentially the identical problem to http://www.frostjedi.com/phpbb3/viewtopic.php?f=46&t=167732 - Which is the SFTP PUT operation intermittently fails with connection being closed by server.

What's interesting to me is that it always works on my dev machine:
* Windows 7 / PHP v5.3.5 / Apache v2.2.17

but fails in my production environment:
* Ubuntu 10.04 / PHP v5.3.2 / Apache v2.2.14

The process is going to the *same* destination SFTP server ("SSH-2.0-3.2.0 SSH Secure Shell Windows NT Server"), uploading the same file (4.7MB)! I have installed mcrypt in both environments (before installing, I was getting a Tx rate of ~10kb/s, now ~2Mb/s!).

The upload fails at different points in the upload, with the following php errors thrown:
* Connection closed by server (SSH2.php)
* Expected SSH_FXP_STATUS (SFTP.php)
* Connection closed prematurely (SSH2.php)

I have logging enabled, the only thing that stood out to me was the last entry in the SSH log:

Code: Select all
<- NET_SSH2_MSG_DISCONNECT (since last: 0.0001, network: 0s)
00000000  00:00:00:02:00:00:00:26:57:69:6e:64:6f:77:20:6f  .......&Window o
00000010  76:65:72:66:6c:6f:77:20:72:65:63:65:69:76:65:64  verflow received
00000020  20:63:68:61:6e:6e:65:6c:20:64:61:74:61:2e:00:00   channel data...
00000030  00:02:65:6e                                      ..en


I started with your 0.3.5 release, then after reading posts here, tried the following:
* Modified the "$packet_size = 0x4000;" to 0x2000 (in the 3 places you mentioned) - problem persisted
* Checked out the latest GIT code - problem persisted
* Made the "0x2000" changes to the latest GIT code - problem persisted

Any ideas here? Any help would be greatly appreciated!
rjking
Traveler
 
Posts: 5
Joined: Thu Jul 11, 2013 12:53 am

Re: Me Too - SFTP Closed by Server

Postby TerraFrost » Sun Jul 14, 2013 9:50 am

I apologize for the delay.

Anyway I just made a commit to Git that might fix this issue for you. If you could let me know that'd be great. Thanks!
TerraFrost
Legendary Guard
 
Posts: 12357
Joined: Wed Dec 04, 2002 6:37 am

Re: Me Too - SFTP Closed by Server

Postby rjking » Tue Jul 16, 2013 12:38 am

Hello-

I updated (overwrote) my files with your latest from GIT (I did not re-do any of the previous manual changes that I had made), and now it is incredibly slow. I didn't get through the first transfer of a 1.2MB file, but after quite some time it had only sent ~200KB. Should I make any of the previous manual changes?

Also, as an update to my first post, I did also make the "$i == 50" to "$i == 10" change in SFTP.php, which seemed to reduce the odds of failure. It also slightly slowed down the transfer of the file from ~18s to ~35s, not sure if the speed of transfer has anything to do with the error I'm seeing, or if it's purely coincidental.

Thanks for your help!
rjking
Traveler
 
Posts: 5
Joined: Thu Jul 11, 2013 12:53 am

Re: Me Too - SFTP Closed by Server

Postby TerraFrost » Tue Jul 16, 2013 12:57 am

My guess: the slow down is caused by phpseclib either getting stuck in a loop or waiting for a packet that's never coming.

If you could post the full logs instead of picking and choosing what parts of them you think are relevant that'd be great.

Alternatively, if you could give me access to the SFTP, itself, that would make things even easier for me.

Thanks!
TerraFrost
Legendary Guard
 
Posts: 12357
Joined: Wed Dec 04, 2002 6:37 am

Re: Me Too - SFTP Closed by Server

Postby rjking » Tue Jul 16, 2013 9:13 pm

Hello again!

I have attached both logs of my last transfer (SSH & SFTP). It completed successfully, but hit a slowdown after the first 203,150 bytes transferred. It raced to the 200k, then crawled. In the interest of getting a file uploaded in a reasonable time, I modified my script to generate a 204kB file.

The data is being transferred to the California DOJ, and contains criminal record information. For this reason, I can't allow you to access the server without first getting approval from them (connections also require usage of password-protected Keypairs). If you end up needing this access, let me know and I'll put in a request. I also modified my script to remove all sensitive information.

Thanks so much for your help!
rjking
Traveler
 
Posts: 5
Joined: Thu Jul 11, 2013 12:53 am

Re: Me Too - SFTP Closed by Server

Postby TerraFrost » Thu Jul 18, 2013 5:11 am

From your logs:

Code: Select all
<- NET_SSH2_MSG_CHANNEL_OPEN_CONFIRMATION (since last: 0.0002, network: 0s)
00000000  00:00:00:02:00:00:00:00:00:00:27:10:00:00:40:00  ..........'...@.

Breaking that down...

recipient channel: 00:00:00:02
sender channel: 00:00:00:00
initial window size: 00:00:27:10
maximum packet size: 00:00:40:00

Basically, I think you have the same problem as this guy. ie. window size smaller than the max packet size.

Anyway, I think the latest commit fixes his problem and hopefully it'll fix yours as well.

If you could let me know that'd be great.

Thanks!
TerraFrost
Legendary Guard
 
Posts: 12357
Joined: Wed Dec 04, 2002 6:37 am

Re: Me Too - SFTP Closed by Server

Postby rjking » Thu Jul 18, 2013 7:09 pm

Hello-

I've tried again with your latest code, sending file failed on all 6 attempts I made.

The behavior is still the same, first 203,150 bytes very quickly upload, then it goes into a 60-sec stall cycle.

I get two php errors each time the transfer errors out:
* Connection closed by server (SSH2.php)
* Expected SSH_FXP_STATUS (SFTP.php)

Logs are attached... Thanks!
rjking
Traveler
 
Posts: 5
Joined: Thu Jul 11, 2013 12:53 am

Re: Me Too - SFTP Closed by Server

Postby TerraFrost » Sat Jul 20, 2013 5:29 pm

Can you replace your _send_channel_packet() method in SSH2.php with this one?:

Code: Select all
    /**
     * Sends channel data
     *
     * Spans multiple SSH_MSG_CHANNEL_DATAs if appropriate
     *
     * @param Integer $client_channel
     * @param String $data
     * @return Boolean
     * @access private
     */
    function _send_channel_packet($client_channel, $data)
    {
        // The maximum amount of data allowed is determined by the maximum
        // packet size for the channel, and the current window size, whichever
        // is smaller.
        $max_size = min(
            $this->packet_size_client_to_server[$client_channel],
            $this->window_size_client_to_server[$client_channel]
        ) - 4;
        while (strlen($data) > $max_size) {
            $packet = pack('CN2a*',
                NET_SSH2_MSG_CHANNEL_DATA,
                $this->server_channels[$client_channel],
                $max_size,
                $this->_string_shift($data, $max_size)
            );

            $this->window_size_client_to_server[$client_channel]-= $max_size + 4;

            if (!$this->_send_binary_packet($packet)) {
                return false;
            }

            if ($max_size == $this->window_size_client_to_server[$client_channel] - 4) {
                $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
                // using an invalid channel will let the buffers be built up for the valid channels
                $this->_get_channel_packet(-1);
                $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
                $max_size = min(
                    $this->packet_size_client_to_server[$client_channel],
                    $this->window_size_client_to_server[$client_channel]
                ) - 4;
            }
        }

        if (strlen($data) >= $this->window_size_client_to_server[$client_channel] - 4) {
            $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
            $this->_get_channel_packet(-1);
            $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
        }

        $this->window_size_client_to_server[$client_channel]-= strlen($data) + 4;

        return $this->_send_binary_packet(pack('CN2a*',
            NET_SSH2_MSG_CHANNEL_DATA,
            $this->server_channels[$client_channel],
            strlen($data),
            $data));
    }

...and let me know if that works for you?

My reason for thinking that might fix the problem is this:

Code: Select all
-> NET_SSH2_MSG_CHANNEL_DATA (since last: 0.002, network: 0.0001s)
00000000  00:00:00:00:ff:ff:ff:fc:00:00:07:d4:06:00:00:00  ................

The first four bytes represent the channel. The second four represent the length. And since the window size is 0x1000 the length shouldn't be more than that. But 0xFFFFFFFC is -4 in base-2. Looking at the relevant code it seems like it could be an issue, too, so... I changed it!

Thanks!
TerraFrost
Legendary Guard
 
Posts: 12357
Joined: Wed Dec 04, 2002 6:37 am

Re: Me Too - SFTP Closed by Server

Postby rjking » Mon Jul 22, 2013 5:03 pm

Wow - that certainly changed things! The full 4.5MB upload completed in 52s - I ran it twice without a single error - Thank-you so much!

Will this change persist for any future releases of your library, or will I need to re-apply anything?
rjking
Traveler
 
Posts: 5
Joined: Thu Jul 11, 2013 12:53 am

Re: Me Too - SFTP Closed by Server

Postby TerraFrost » Mon Jul 22, 2013 6:18 pm

I expect it'll persist. Was kinda hoping that the other person having window size issues would be able to test it out but it's possible that since their last issue has been resolved that they've kinda moved on and aren't really paying attention to this forum anymore. Hard to say. I guess we'll see lol.

Anyway glad to hear that it's working for you! :D
TerraFrost
Legendary Guard
 
Posts: 12357
Joined: Wed Dec 04, 2002 6:37 am


Return to phpseclib support

Who is online

Users browsing this forum: No registered users and 1 guest

cron