FTP over SSH
My first post. I am not a unix system admin. I have been searching a lot for solutions to this problem, but have not found commands to do this. So I would really appreciate some detailed with help with this.
In a nutshell : I want to do FTP over SSH. Cant use SFTP or SCP.
Details : I have Machine 1 and Machine 2. Both Redhat Linux. I have an application that runs on Machine 1. From here, this application ftp-s files to and from Machine 2. The application doesnt support SFTP, and dont want to go thru that route of code changes to the application to do SFTP. But we still want secure data transfer between Machine 1 and Machine 2.
So we want to try SSH, tunneling, port forwarding to send FTP traffic over SSH.
The command we want to try (on macine1) is as below
ssh -f -N os_user#machine2 -L12345:machine2:21
In the above, kindly replace # with the "at" sign. Ths software in this forum is not letting me type that symbol. It thinks it is a URL (heaven knows why), and says I cant post URLs unless I have 15 or more posts. Oh well ..
If this works, the ftp client on Machine1 can simply connect to Machine1 itself on port 12345. The SSH will take care of encrypting and sending the information to Machine 2. (hopefully).
Of course there are caveats and complications. Ftp is more than port 21, so various things need to be done for above to work. Right now I cant get it to work. Any thoughts on this approach ? Is this a bad idea ?
Any replies most appreciated. Note : I cannot use SFTP, SCP etc because I would have to change the application (code) for that.
Thanks in advance !
- Noob-hat (like Red-hat, for newbies :) )
It's helpful that they're both RedHat machines. If you can get the ssh to allow a user on machine 1 to log into machine2 using key-based authentication you've won the main battle. You can forward ftp ports, it should work, but there are other options.
Once the ssh connection works, i.e. once you get to the point where you can do 'ssh user@machine' and it gives you a log-in prompt using your own key pair you can then build on that.
I'd recommend you take a look at 'sshfs' (it should be available in your package manager). It uses the fuse (filesystem in user space) extensions to mount a directory using ssh, and once mounted you can use it like any ordinary drive, i.e. you can just use 'cp ...' to copy files in bash.
Your incantation to make this work would look something like this:
Then you can simply copy the files:
# mkdir ~/remotedir
# sshfs machine2:/path/to/filesystem/destination ~/remotedir
and unmount using :
# cp /path/to/files/* ~/remotedir/
Of course, this is Linux so you will have lots of other options - once ssh is working, scp should work without using fuse to mount the filesystems:
# fusermount -u ~/remotedir
Setting up your ssh keys can be a bit daunting for new users - but the internet has some pretty good resources that can explain this far better than I can in a short post on here. You might like to take a look at this good beginners guide to setting up ssh, and especially your encryption keys. And we mustn't forget that RedHat do provide extensive documentation on ssh (and just about everything else in their OS) but it can be a bit daunting for inexperienced users.
# scp /path/to/files/* user@machine2:/path/to/filesystem/destination
NB: the limit on posting URLs is a bit inconvenient for those who've just joined - we're sorry about that, but it's done that way because we get lots of spam-bots, and this thwarts them pretty well. You can post links normally when you've made 15 posts (not including coffee lounge posts) - and if you're joining in with the community, it's surprising how quickly you do that.
Thanks for the detailed response, Roxoff. I did not completely follow all of that, I guess there is much to learn regarding ssh.
You wrote : > once you get to the point where you can ...and it gives you a log-in prompt using your own key pair ...
Not sure what you meant by 'and it gives you a login prompt *using your own key pair*' ...Here is what my session transcript looks like when I ssh from machine1 to machine2. It is giving me a login prompt and letting me log in, not sure if it is using "my own key pair".
again, please replace # with the 'at' sign below.
[user1#machine1]--> uname -a
Linux machine1 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
[user1#machine1]--> ssh user2#machine2
user2#machine2's password: [I typed my password here]
Last login: Thu Apr 5 00:32:05 2012 from 22.214.171.124
[user2#machine2]--> uname -a
Linux machine2 2.6.18-128.el5 #1 SMP Wed Dec 17 11:41:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
also, I dont have sshfs on my system. will talk to my system admin.
A bit of googling got me to :
www dot laubenheimer dot net slash ssh-keys dot shtml
which explains SSH key authentication. I suppose this is kind of what I would need to do to "win the main battle" as you say above, yeah ?
Yes, that's right. Getting SSH to log in and give you a command prompt is the first step. Once that's going you can make it do all kinds of things. The more you read about this, the better you'll understand it. Take a look at the beginners guide link I posted, it should help quite a bit too.
Thanks a lot Roxoff ! That worked like a charm, when I tried. I was now able to "mount" machine1:/dir1 to "point to" machine2:/dir2. Now user1 can create / change / delete files on machine1:/dir1 and it automatically appears on machine2:/dir2, just like NFS mounted directories.
But after reading the Wikipedia article on SSHFS, I have some fears. Is there a possiblity of losing data integrity ? The wikipedia line "applications may respond in ways that are unpredictable or misleading" is where I am getting concerned.
I am not aware of how SSHFS is implemented internally. I dont think it is doing an ftp everytime I make a change to the file on machine1:/dir1. Yet I can see the changes immediately on machine2:/dir2.
My application writes to a file named x.temp on machine1:/dir1. When it is done, my application renames it to x.dat. The "consumer" application on machine2 only picks up *.dat files from machine2:/dir2 so at least I am reasonably sure the consumer application wont pick up an incomplete file. However, what does a "rename" entail, especially over SSHFS ? If my primary application is halfway thru the renaming (especially over the network, with encryption and all that), is there a chance that the "consumer" application "sees" the renamed file before it is renamed ? Sorry if this sounds stupid, but just had to get that out of my mind, because I dont follow how all the changes on machine1:/dir1 are getting reflected immediately on machine2:/dir2