Page 1 of 1

Me Too - SFTP Closed by Server

PostPosted: Thu Jul 11, 2013 2:47 am
by rjking
Hello TerraFrost!

First off, *amazing* project you have created here! I am having essentially the identical problem to http://www.frostjedi.com/phpbb3/viewtopic.php?f=46&t=167732 - Which is the SFTP PUT operation intermittently fails with connection being closed by server.

What's interesting to me is that it always works on my dev machine:
* Windows 7 / PHP v5.3.5 / Apache v2.2.17

but fails in my production environment:
* Ubuntu 10.04 / PHP v5.3.2 / Apache v2.2.14

The process is going to the *same* destination SFTP server ("SSH-2.0-3.2.0 SSH Secure Shell Windows NT Server"), uploading the same file (4.7MB)! I have installed mcrypt in both environments (before installing, I was getting a Tx rate of ~10kb/s, now ~2Mb/s!).

The upload fails at different points in the upload, with the following php errors thrown:
* Connection closed by server (SSH2.php)
* Expected SSH_FXP_STATUS (SFTP.php)
* Connection closed prematurely (SSH2.php)

I have logging enabled, the only thing that stood out to me was the last entry in the SSH log:

Code: Select all
<- NET_SSH2_MSG_DISCONNECT (since last: 0.0001, network: 0s)
00000000  00:00:00:02:00:00:00:26:57:69:6e:64:6f:77:20:6f  .......&Window o
00000010  76:65:72:66:6c:6f:77:20:72:65:63:65:69:76:65:64  verflow received
00000020  20:63:68:61:6e:6e:65:6c:20:64:61:74:61:2e:00:00   channel data...
00000030  00:02:65:6e                                      ..en


I started with your 0.3.5 release, then after reading posts here, tried the following:
* Modified the "$packet_size = 0x4000;" to 0x2000 (in the 3 places you mentioned) - problem persisted
* Checked out the latest GIT code - problem persisted
* Made the "0x2000" changes to the latest GIT code - problem persisted

Any ideas here? Any help would be greatly appreciated!

Re: Me Too - SFTP Closed by Server

PostPosted: Sun Jul 14, 2013 9:50 am
by TerraFrost
I apologize for the delay.

Anyway I just made a commit to Git that might fix this issue for you. If you could let me know that'd be great. Thanks!

Re: Me Too - SFTP Closed by Server

PostPosted: Tue Jul 16, 2013 12:38 am
by rjking
Hello-

I updated (overwrote) my files with your latest from GIT (I did not re-do any of the previous manual changes that I had made), and now it is incredibly slow. I didn't get through the first transfer of a 1.2MB file, but after quite some time it had only sent ~200KB. Should I make any of the previous manual changes?

Also, as an update to my first post, I did also make the "$i == 50" to "$i == 10" change in SFTP.php, which seemed to reduce the odds of failure. It also slightly slowed down the transfer of the file from ~18s to ~35s, not sure if the speed of transfer has anything to do with the error I'm seeing, or if it's purely coincidental.

Thanks for your help!

Re: Me Too - SFTP Closed by Server

PostPosted: Tue Jul 16, 2013 12:57 am
by TerraFrost
My guess: the slow down is caused by phpseclib either getting stuck in a loop or waiting for a packet that's never coming.

If you could post the full logs instead of picking and choosing what parts of them you think are relevant that'd be great.

Alternatively, if you could give me access to the SFTP, itself, that would make things even easier for me.

Thanks!

Re: Me Too - SFTP Closed by Server

PostPosted: Tue Jul 16, 2013 9:13 pm
by rjking
Hello again!

I have attached both logs of my last transfer (SSH & SFTP). It completed successfully, but hit a slowdown after the first 203,150 bytes transferred. It raced to the 200k, then crawled. In the interest of getting a file uploaded in a reasonable time, I modified my script to generate a 204kB file.

The data is being transferred to the California DOJ, and contains criminal record information. For this reason, I can't allow you to access the server without first getting approval from them (connections also require usage of password-protected Keypairs). If you end up needing this access, let me know and I'll put in a request. I also modified my script to remove all sensitive information.

Thanks so much for your help!

Re: Me Too - SFTP Closed by Server

PostPosted: Thu Jul 18, 2013 5:11 am
by TerraFrost
From your logs:

Code: Select all
<- NET_SSH2_MSG_CHANNEL_OPEN_CONFIRMATION (since last: 0.0002, network: 0s)
00000000  00:00:00:02:00:00:00:00:00:00:27:10:00:00:40:00  ..........'...@.

Breaking that down...

recipient channel: 00:00:00:02
sender channel: 00:00:00:00
initial window size: 00:00:27:10
maximum packet size: 00:00:40:00

Basically, I think you have the same problem as this guy. ie. window size smaller than the max packet size.

Anyway, I think the latest commit fixes his problem and hopefully it'll fix yours as well.

If you could let me know that'd be great.

Thanks!

Re: Me Too - SFTP Closed by Server

PostPosted: Thu Jul 18, 2013 7:09 pm
by rjking
Hello-

I've tried again with your latest code, sending file failed on all 6 attempts I made.

The behavior is still the same, first 203,150 bytes very quickly upload, then it goes into a 60-sec stall cycle.

I get two php errors each time the transfer errors out:
* Connection closed by server (SSH2.php)
* Expected SSH_FXP_STATUS (SFTP.php)

Logs are attached... Thanks!

Re: Me Too - SFTP Closed by Server

PostPosted: Sat Jul 20, 2013 5:29 pm
by TerraFrost
Can you replace your _send_channel_packet() method in SSH2.php with this one?:

Code: Select all
    /**
     * Sends channel data
     *
     * Spans multiple SSH_MSG_CHANNEL_DATAs if appropriate
     *
     * @param Integer $client_channel
     * @param String $data
     * @return Boolean
     * @access private
     */
    function _send_channel_packet($client_channel, $data)
    {
        // The maximum amount of data allowed is determined by the maximum
        // packet size for the channel, and the current window size, whichever
        // is smaller.
        $max_size = min(
            $this->packet_size_client_to_server[$client_channel],
            $this->window_size_client_to_server[$client_channel]
        ) - 4;
        while (strlen($data) > $max_size) {
            $packet = pack('CN2a*',
                NET_SSH2_MSG_CHANNEL_DATA,
                $this->server_channels[$client_channel],
                $max_size,
                $this->_string_shift($data, $max_size)
            );

            $this->window_size_client_to_server[$client_channel]-= $max_size + 4;

            if (!$this->_send_binary_packet($packet)) {
                return false;
            }

            if ($max_size == $this->window_size_client_to_server[$client_channel] - 4) {
                $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
                // using an invalid channel will let the buffers be built up for the valid channels
                $this->_get_channel_packet(-1);
                $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
                $max_size = min(
                    $this->packet_size_client_to_server[$client_channel],
                    $this->window_size_client_to_server[$client_channel]
                ) - 4;
            }
        }

        if (strlen($data) >= $this->window_size_client_to_server[$client_channel] - 4) {
            $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
            $this->_get_channel_packet(-1);
            $this->bitmap^= NET_SSH2_MASK_WINDOW_ADJUST;
        }

        $this->window_size_client_to_server[$client_channel]-= strlen($data) + 4;

        return $this->_send_binary_packet(pack('CN2a*',
            NET_SSH2_MSG_CHANNEL_DATA,
            $this->server_channels[$client_channel],
            strlen($data),
            $data));
    }

...and let me know if that works for you?

My reason for thinking that might fix the problem is this:

Code: Select all
-> NET_SSH2_MSG_CHANNEL_DATA (since last: 0.002, network: 0.0001s)
00000000  00:00:00:00:ff:ff:ff:fc:00:00:07:d4:06:00:00:00  ................

The first four bytes represent the channel. The second four represent the length. And since the window size is 0x1000 the length shouldn't be more than that. But 0xFFFFFFFC is -4 in base-2. Looking at the relevant code it seems like it could be an issue, too, so... I changed it!

Thanks!

Re: Me Too - SFTP Closed by Server

PostPosted: Mon Jul 22, 2013 5:03 pm
by rjking
Wow - that certainly changed things! The full 4.5MB upload completed in 52s - I ran it twice without a single error - Thank-you so much!

Will this change persist for any future releases of your library, or will I need to re-apply anything?

Re: Me Too - SFTP Closed by Server

PostPosted: Mon Jul 22, 2013 6:18 pm
by TerraFrost
I expect it'll persist. Was kinda hoping that the other person having window size issues would be able to test it out but it's possible that since their last issue has been resolved that they've kinda moved on and aren't really paying attention to this forum anymore. Hard to say. I guess we'll see lol.

Anyway glad to hear that it's working for you! :D