fuseftp: Implementing userspace filesystems with Perl

Update 2011: Well, I haven’t done anything with Perl in the last couple of years, but to update this Page: FuseFTP is now up on Github, you can fork it there.

Well, as I mentioned earlier, 2.6.14 has this nifty new FUSE feature, allowing the implementation of a file system in userspace. Unfortunately, there was no implementation of a FTP filesystem. Fortunately, there are Perl bindings for FUSE🙂.

One can implement a userspace filesystem using the whole power of Perl, and that makes it quite easy. I implemented a basic ftp filesystem this morning, which uses Fuse.pm and Net::FTP to allow read only access to ftp servers. It works quite well and was really easy, so this might be the beginning of a couple of nice userspace filesystems written in Perl. I could actually create MegasusFS….

BTW, you can download fuseftp from my Github Account.

Usage is quite easy, you can simply:

marcus@hal2000:~: mkdir kernel
marcus@hal2000:~: fuseftp kernel/ ftp.kernel.org &
Successfully logged into ftp.kernel.org
[1] 7150
marcus@hal2000:~: cd kernel/
marcus@hal2000:~/kernel: ls
for_mirrors_only  lost+found  pub  usr  welcome.msg
marcus@hal2000:~/kernel: cd ..
marcus@hal2000:~: fusermount -u kernel/
[1]+  Done                    fuseftp kernel/ ftp.kernel.org

70 thoughts on “fuseftp: Implementing userspace filesystems with Perl

  1. Hi. I have the following problem:
    I need to connect to an ftp host using a ftp-proxy like this:
    [LAN][FTP Proxy][FTP Host]

    So I tried to set the ftp_proxy environment variable. But I just don’t get it running😦 Filezilla, my FTP Client is running by using “USER RemoteID@RemoteHost”. Anyone can help?

    Regards

    Philipp

    Like

  2. I’m having problems creating new dirs with fuseftp. “mkdir aha” always responds something like “cannot create directory aha: File not found”. Actually, in German:

    mkdir: kann Verzeichnis „aha“ nicht anlegen: Datei oder Verzeichnis nicht gefunden

    And the command “ls” shows this funny line:

    ?——— ? ? ? ? ? aha

    However, after unmounting and remounting the filesystem, the directory shows up correctly and is usable.

    I’m running Debian unstable. Any ideas?

    Regards,
    Bernd

    Like

  3. Hi,
    how can I mount as a Normal-User?

    chmod 4755 fusermount
    doesn’t work.

    I’m running SuSE 9.2.

    Regards,
    Daniel

    Like

  4. File not found problems…
    i can connect fine, but listing gives me:

    ?——— ? ? ? ? ? #FILENAME
    ?——— ? ? ? ? ? #FILENAME
    ?——— ? ? ? ? ? #FILENAME
    ?——— ? ? ? ? ? #FILENAME
    and copying gives me:

    Trying to flush not open file web4/test
    Trying to close not open file web4/test
    cp: cannot create regular file `/mnt/backup/test’: No such file or directory

    I connect using:
    /usr/bin/fuseftp /mnt/backup/ USER:PASS@192.168.1.2:web4

    fuseftp 0.8 AND fuse-2.5.3 with all perl modules installed…
    perl v5.8.5
    .. any ideas ?

    Regards,
    Glenn

    Like

  5. ** Fuseftp and SAMBA **

    I use fuseftp to mount a ftp server directory in FC5. The mounted directory is exported to windows PC via SAMBA.

    Initially, it works and after, say several minutes, the mounted directory becomes empty in FC5 as well as in Windows.

    I have to umount and mount again.

    I have run fuseftp in debug but the log remains nothing special. Below is a partial debug log:

    ========
    unique: 682, error: 0 (Success), outsize: 16
    unique: 683, opcode: GETATTR (3), nodeid: 1, insize: 40
    unique: 683, error: 0 (Success), outsize: 112
    unique: 684, opcode: OPENDIR (27), nodeid: 1, insize: 48
    unique: 684, error: 0 (Success), outsize: 32
    unique: 685, opcode: RELEASEDIR (29), nodeid: 1, insize: 56
    unique: 685, error: 0 (Success), outsize: 16
    unique: 686, opcode: GETATTR (3), nodeid: 1, insize: 40
    unique: 686, error: 0 (Success), outsize: 112
    ========

    Pls advise.

    Like

  6. Well, I guess that you’re running in the still unresolved reconnection problem at the moment. Fuseftp (or more specific, the underlying Net::FTP) times out after a while without Fuseftp knowing that the connection is lost. This need to be fixed, but I’m quite out of time at the moment.

    Like

  7. Hey Marcus,

    I’ve been trying to get rsync to work with fuseftp mounted target. I haven’t had a lot of luck, but in the process I’ve found a few bugs in version 0.8.

    Mkdir/rmdir bug.

    The ftp_makedir and ftp_rmdir functions don’t properly clear dir_seen for the directory the new directory was made/removed in. You also need to delete the attr_cache for the newly made/deleted directory. If you do “mkdir foo” you get “mkdir: cannot create directory `foo’: No such file or directory”, and when you do an ls -l you get:

    ?——— ? ? ? ? ? foo

    Rename bug.

    The ftp_rename function doesn’t properly clear dir_seen, for the source and target directories. It also doesn’t clear the type_cache and attr_cache for old and new files. This bug appears when you “mv foo bar” and then do an “ls -l” in the directory.

    total 0
    ?——— ? ? ? ? ? bar

    After the move you can also do “ls -l foo” and still get cached information about the nonexistant file.

    O_RDWR bug.

    If a file is opened for O_RDWR the code does not handle this case, and opens it for read only. I haven’t completely fixed this problem so you can both read and write the file, I’ve just hacked around it so O_RDWR is a synonym for O_WRONLY. This helps me getting rsync to work.

    Here is my diff:

    176a177
    >
    273,286c274,275
    my ($dir) = $newname =~ m|(.+)/.+|;
    > delete $dir_seen{$dir} if $dir;
    321c310
    if ($flags & (O_WRONLY)) {
    471,476c460
    delete $dir_seen{$dir};
    490,495c474
    delete $dir_seen{$dir};

    Like

  8. For some reason my diff got mangled in my last post, trying again.

    176a177
    >
    273,286c274,275
    my ($dir) = $newname =~ m|(.+)/.+|;
    > delete $dir_seen{$dir} if $dir;
    321c310
    if ($flags & (O_WRONLY)) {
    471,476c460
    delete $dir_seen{$dir};
    490,495c474
    delete $dir_seen{$dir};

    Like

  9. Here’s another bug.

    Let’s say you open a file for reading, the code calls $ftp->retr and stashes the handle away for later read calls. Before you read/close the file you attempt to stat another file on the filesystem. This hangs. The problem is ftp_getattr gets called and a $ftp->cwd gets executed. This will hang because the FTP server is thining it’s sending you a file, and won’t accept any more commands. At least that’s why I think it doesn’t work.

    Mike

    Like

  10. Hi,

    I mounted a remote ftp target with
    fuseftp –passive –cache=memory ./mountpoint/ user:pass@host:/location

    During cp-ing away a 5MB file into a local filesystem I got a bunch of such errors:

    substr outside of string at /usr/bin/fuseftp line 381.
    substr outside of string at /usr/bin/fuseftp line 381.
    substr outside of string at /usr/bin/fuseftp line 381.

    The copy of the file was corrupted afterwards. It still had the same filesize tough…

    Like

  11. hi, i am having problems w/ ver 0.8. here is the console log:

    $ fuseftp ftp-remote psikutas
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into psikutas
    Backgrounding…
    jose@darkstar:~$ Day too small – -25569 > -24856
    Sec too small – -25569

    Like

  12. I’m using FC6 and I’ve installed the version 0.8. Here, passive connections not working, and Net::FTP use passive mode as default. I’ve changed fuseftp from:

    my $ftp;
    if ($arg_opts{‘–passive’}) {
    $ftp = new Net::FTP($host, Passive => 1, Port => $port);
    } else {
    $ftp = new Net::FTP($host, Port => $port);
    }

    to

    my $ftp;
    if ($arg_opts{‘–passive’}) {
    $ftp = new Net::FTP($host, Passive => 1, Port => $port);
    } else {
    $ftp = new Net::FTP($host, Passive => 0, Port => $port);
    }

    And it works.

    Like

  13. Hi Marcus,

    thanks for that great program!🙂
    For lightweight use it works great for me, but when I do backups with the dar utility (http://dar.linux.free.fr) which are around 50GB in size, then fuseftp:
    1) eats 620MB of memory during operation (I did NOT specify –cache=memory)
    2) errors out after some minutes with this message:

    substr outside of string at /usr/local/bin/fuseftp line 378.
    /bin/ls: cannot access /mnt/ftpbackup/dar_vmail-20070105*: No such file or directory

    Line 378 is the last line of ftp_read().
    If I can be of any help to fix this problem I’ll be glad to provide more infos / do testing.

    Note that with curlftpfs it works fine, but I’d like to get your nice utility working.

    Mark

    Like

  14. I installed fuseftp for backup files to a FTP Server. A ~200Mbyte file takes hours. Speed to the FTP Server is about ~5Mbyte/sec.

    I think it is not made to transfer hundreds of Mbytes or even Gbates of data…

    Like

  15. I have installed Fuse on Ubuntu 6.10. I create a folder and connect to the FTP server like so:

    mkdir mysite/
    sudo fuseftp mysite/ username:password@hostaddress

    I get this result:

    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into hostaddress
    Backgrounding…

    However if I do ls -l I get the following:

    ?——— ? ? ? ? ? mysite

    Can anyone help?

    Like

  16. Hi Marcus,

    I would like to implement fuseftp in my LiveCD Nekdesk.

    Therefore I want to write a mounting utility with kde-kommander.
    I would like to know the errorlevels fuseftp produces, that I will be able to depend the neccessary dialogs on.

    Regards, Manfred

    Like

  17. Hallo …

    I install on the Linux RedHat Enterprice 3 the fuse rpm “fuse-2.6.5-1.el3.rf”

    To padlock I am install the fuseftp-o.8.tar.gz.

    After the Command fuseftp cam the Error Messages:

    #########

    [root@OMS_Linux7 fuseftp-0.8]# fuseftp
    Can’t locate Fuse.pm in @INC (@INC contains: /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 /usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.0 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.0 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.0/i386-linux-thread-multi /usr/lib/perl5/5.8.0 .) at /usr/bin/fuseftp line 44.
    BEGIN failed–compilation aborted at /usr/bin/fuseftp line 44.

    #############

    You can help me ….

    The best regards vor Germany

    Hans

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s