fuseftp: Implementing userspace filesystems with Perl

Update 2011: Well, I haven’t done anything with Perl in the last couple of years, but to update this Page: FuseFTP is now up on Github, you can fork it there.

Well, as I mentioned earlier, 2.6.14 has this nifty new FUSE feature, allowing the implementation of a file system in userspace. Unfortunately, there was no implementation of a FTP filesystem. Fortunately, there are Perl bindings for FUSE :-).

One can implement a userspace filesystem using the whole power of Perl, and that makes it quite easy. I implemented a basic ftp filesystem this morning, which uses Fuse.pm and Net::FTP to allow read only access to ftp servers. It works quite well and was really easy, so this might be the beginning of a couple of nice userspace filesystems written in Perl. I could actually create MegasusFS….

BTW, you can download fuseftp from my Github Account.

Usage is quite easy, you can simply:

marcus@hal2000:~: mkdir kernel
marcus@hal2000:~: fuseftp kernel/ ftp.kernel.org &
Successfully logged into ftp.kernel.org
[1] 7150
marcus@hal2000:~: cd kernel/
marcus@hal2000:~/kernel: ls
for_mirrors_only  lost+found  pub  usr  welcome.msg
marcus@hal2000:~/kernel: cd ..
marcus@hal2000:~: fusermount -u kernel/
[1]+  Done                    fuseftp kernel/ ftp.kernel.org

70 thoughts on “fuseftp: Implementing userspace filesystems with Perl

  1. Is there any way to use fuseftp with an account that has ‘@’ in the account name?

    Net::FTP seems to break the string at the first @ character, so it interprets us@er:pass@host as username ‘us’ and host ‘er’, and gives the following error message:

    Can’t connect to er: Net::FTP: Bad hostname ‘er’

    Like

  2. You are missing support for passive connections, I hacked this small thing into your code (could be done nicer, but it’s 7am here and i need to get into my bed soon):
    ————————————————
    #make the connection
    my $ftp;
    if ($arg_opts{“–passive”}) {
    $ftp = new Net::FTP($host, Passive => 1) or
    die “Can’t connect to $host: $@n”;
    } else {
    $ftp = new Net::FTP($host) or
    die “Can’t connect to $host: $@n”;
    }
    ————————————————

    Of course the help need to be updated after this, too…but it is outdated in every case (hint: “fusesmb”).

    Nice work btw. πŸ˜‰

    Like

  3. Thanks, applied, it is in 0.4. Good catch about the fusesmb, the fun thing is that I never actually had a look at fusesmb’s source, so this just came from my brain confusing the one three letter abbriviation with another.

    Like

  4. Thanks for this. It’s working nicely here – I’m in the process of packaging it for Arch Linux.

    I’m getting an error from one particular server, debug output as follows:

    $ fuseftp –debug heanet http://ftp.heanet.ie &
    [1] 24532
    $ Successfully logged into http://ftp.heanet.ie
    unique: 1, opcode: INIT (26), nodeid: 0, insize: 48
    INIT: 7.2
    INIT: 7.2
    unique: 1, error: 0 (Success), outsize: 24

    $ ls heanet
    unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 40
    unique: 2, error: 0 (Success), outsize: 112
    ls: heanet: Input/output error
    $ fusermount -u heanet
    unique: 3, opcode: FORGET (2), nodeid: 1, insize: 48
    FORGET 1/1
    [1]+ Done fuseftp –debug heanet http://ftp.heanet.ie

    I can lftp to the server, so I know it’s working. I’ve also tried it with –passive, but the result is the same.

    Like

  5. I have another patch witch is to simple to create a patch. Add these lines after ‘print “Successfully logged into $hostn”;’:
    unless (DEBUG || $arg_opts{‘–foreground’})
    {
    fork() and exit(0);
    }

    Now fuseftp will put itself into the background. Works fine here…
    Of course the option “–foreground” must be added in the docu, too.

    And btw.: Seems like this blog translates “–” (“- -“, two minus) into “-” (one minus)….

    Like

  6. it would be nice to be able to ftp to a server listening on a nonstandard port (i.e. not port 21); is that already possible?

    Like

  7. AltLinux Sisyphus

    AS: Perl FUSE tests shows one error, so this is not clear test.

    1) i wish there’d be a way to use generic mount command.
    Perhaps in the mountpoind directory there can be some dot-file with more info for fuse and fuse filesystems?

    2) what about access grants ? I made folder for local anonymous ftp server as /mnt/temp.ostnet.ru (root:root, chmod 1+rw)
    When i make fuseftp i got all dirs in /mnt/temp.ostnet.ru belonging to me (bdv:bdv) but with only grants of rx, i could not make subdirectory there or upload files 😦

    3) fuse FtpFS is imho to be able to log its messages to SysLog or kernel log perhaps.

    4) FtpFS is to handle filenames locales.
    NEwest FTP RFC claims it is UTF8 used for naming files, but… i saw no such modern ftp servers nor clients still.
    In russia, usually FTP server is patched to accept/list filenames in windows-1251 charset fir dumb Win32 clients and more sofisticated unix clients (lftp, gFTP, etc) usually have option to use it instead of their unix-box locale. http://www.freesource.info/wiki/Lokalizacija/NATSPECDescription

    Like

  8. As usual, people will start to use a small hack for the weirdest things – and here I go:

    I have this small dedicated server that comes with 40GB disk and 40GB ftp-accessible backup space.

    I can use fuseftp to mount it and copy over locally created tarballs with backups and even run things like tar -czf /mnt/ftp-backup/lala.tar.gz /var.

    but I have something else in mind πŸ˜‰

    at first, is there a way to keep the mount active (i.e. sending NOOP for at least 30 minutes?), second
    I’m trying to turn that backup space into a filestorage for my backup utility, but am getting weird errors as if some things like stat, etc. won’t work.

    A small example…
    Connecting to Storage daemon File at xxxx:9103 …
    Sending label command for Volume “lala” Slot 0 …
    dev.c:111 Unable to stat device /mnt/ftp-backup/bacula: ERR=Permission denied

    Device “FileStorage” requested by DIR could not be opened or does not exist.
    3999 Device “FileStorage” not found or could not be opened.
    […]

    Well, I’m running this as root, and I can’t tell why this permission denied is coming up. Do You have any hint for me?

    Florian

    Like

  9. I upgraded fuse to 2.5.0 today, and fuseftp no longer works. Here’s the error message:

    $ fuseftp my_mnt/ http://ftp.myisp.com
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into http://ftp.myisp.com
    Backgrounding…
    $ /usr/bin/perl: relocation error: /usr/lib/perl5/site_perl/5.8.7/i686-linux-thread-multi/auto/Fuse/Fuse.so: symbol fuse_mount, version FUSE_2.2 not defined in file libfuse.so.2 with link time reference

    Downgrading to 2.4.2 solves it for the moment.

    Like

  10. Try rebuilding the Fuse.pm Perl Module, it might not be compatible with the new Fuse API without recompilation or fixing, but this would be out of my hands.

    Like

  11. Thanks for the suggestion. I rebuilt fuse.pm, and it looked like that was the fix – the error message was not displayed. However, the server was not mounted, either, even though the “Successfully logged etc” and “Backgrounding” messages were shown. I tried again with –debug and it reported a segfault. No other details were given.

    I’ll report it upstream.

    Like

  12. When running with Fuse.2.4.2-2 I get the following error.

    /usr/bin/perl: symbol lookup error: /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi/auto/Fuse/Fuse.so: undefined symbol: fuse_mount_compat22

    Any clues what I have done wrong?

    Ta

    Like

  13. Mmm, looks as if the module has problems finding some dependencies but otherwise compiled fine. Try forcing the install and then see if it works.

    Like

  14. I have just installed fuseftp and am delighted with it. However it will not run from /etc/fstab. Here is my entry:
    fuseftp#username:password@hostname /mnt/KFT fuse user,rw 0 0

    and here is the error if I try to mount it:

    mount /mnt/KFT
    Use of uninitialized value in join or string at /usr/lib/perl5/5.8.6/File/Spec/Unix.pm line 73.
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Couldn’t make FTP connection to /mnt/KFT on port 21: Net::FTP: Bad hostname ‘/mnt/KFT’!
    [mellor@Pier-1~]$

    Like

  15. Well, the mechanism fstab uses to call mount seems to be incompatible with fuseftps arguments, it gives the mountpoint as hostname. Maybe try rearanging the arguments in fstab.

    Like

  16. Please maybe someone can help me..
    Tried to do the example with kernel.org:

    dau@nerd1:~$ fuseftp kernel/ http://ftp.kernel.org
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into http://ftp.kernel.org
    Backgrounding…
    dau@nerd1:~$ fusermount: failed to open /dev/fuse: Is a directory
    could not mount fuse filesystem! at /usr/local/lib/perl/5.8.8/Fuse.pm line 106.

    Then i tried to rebuild Fuse.pm but the make test results:

    nerd1:~/.cpan/build/Fuse-0.06# make test
    PERL_DL_NONLAZY=1 /usr/bin/perl “-Iblib/lib” “-Iblib/arch” test.pl
    test/s/mount…..ok 1/3
    # Failed test ‘mount succeeded’
    # in test/s/mount.t at line 20.
    # Looks like you failed 1 test of 3.
    test/s/mount…..dubious
    Test returned status 1 (wstat 256, 0x100)
    DIED. FAILED test 3
    Failed 1/3 tests, 66.67% okay
    test/chmod…….not properly mounted
    Compilation failed in require at test/chmod.t line 2.
    BEGIN failed–compilation aborted at test/chmod.t line 2.
    test/chmod…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/chown…….not properly mounted
    Compilation failed in require at test/chown.t line 2.
    BEGIN failed–compilation aborted at test/chown.t line 2.
    test/chown…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/getattr…..not properly mounted
    Compilation failed in require at test/getattr.t line 2.
    BEGIN failed–compilation aborted at test/getattr.t line 2.
    test/getattr…..dubious
    Test returned status 2 (wstat 512, 0x200)
    test/getdir……not properly mounted
    Compilation failed in require at test/getdir.t line 2.
    BEGIN failed–compilation aborted at test/getdir.t line 2.
    test/getdir……dubious
    Test returned status 2 (wstat 512, 0x200)
    test/link……..not properly mounted
    Compilation failed in require at test/link.t line 2.
    BEGIN failed–compilation aborted at test/link.t line 2.
    test/link……..dubious
    Test returned status 2 (wstat 512, 0x200)
    test/mkdir…….not properly mounted
    Compilation failed in require at test/mkdir.t line 2.
    BEGIN failed–compilation aborted at test/mkdir.t line 2.
    test/mkdir…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/mknod…….not properly mounted
    Compilation failed in require at test/mknod.t line 2.
    BEGIN failed–compilation aborted at test/mknod.t line 2.
    test/mknod…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/open……..not properly mounted
    Compilation failed in require at test/open.t line 2.
    BEGIN failed–compilation aborted at test/open.t line 2.
    test/open……..dubious
    Test returned status 2 (wstat 512, 0x200)
    test/read……..not properly mounted
    Compilation failed in require at test/read.t line 2.
    BEGIN failed–compilation aborted at test/read.t line 2.
    test/read……..dubious
    Test returned status 2 (wstat 512, 0x200)
    test/readlink….not properly mounted
    Compilation failed in require at test/readlink.t line 2.
    BEGIN failed–compilation aborted at test/readlink.t line 2.
    test/readlink….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/rename……not properly mounted
    Compilation failed in require at test/rename.t line 2.
    BEGIN failed–compilation aborted at test/rename.t line 2.
    test/rename……dubious
    Test returned status 2 (wstat 512, 0x200)
    test/rmdir…….not properly mounted
    Compilation failed in require at test/rmdir.t line 2.
    BEGIN failed–compilation aborted at test/rmdir.t line 2.
    test/rmdir…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/statfs……not properly mounted
    Compilation failed in require at test/statfs.t line 2.
    BEGIN failed–compilation aborted at test/statfs.t line 2.
    test/statfs……dubious
    Test returned status 2 (wstat 512, 0x200)
    test/symlink…..not properly mounted
    Compilation failed in require at test/symlink.t line 2.
    BEGIN failed–compilation aborted at test/symlink.t line 2.
    test/symlink…..dubious
    Test returned status 2 (wstat 512, 0x200)
    test/truncate….not properly mounted
    Compilation failed in require at test/truncate.t line 2.
    BEGIN failed–compilation aborted at test/truncate.t line 2.
    test/truncate….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/unlink……not properly mounted
    Compilation failed in require at test/unlink.t line 2.
    BEGIN failed–compilation aborted at test/unlink.t line 2.
    test/unlink……dubious
    Test returned status 2 (wstat 512, 0x200)
    test/utime…….not properly mounted
    Compilation failed in require at test/utime.t line 2.
    BEGIN failed–compilation aborted at test/utime.t line 2.
    test/utime…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/write…….not properly mounted
    Compilation failed in require at test/write.t line 2.
    BEGIN failed–compilation aborted at test/write.t line 2.
    test/write…….dubious
    Test returned status 2 (wstat 512, 0x200)
    test/s/umount….fusermount: entry for /tmp/fusemnt-root not found in /etc/mtab

    # Failed test ‘unmount’
    # in test/s/umount.t at line 8.
    # Looks like you failed 1 test of 1.
    test/s/umount….dubious
    Test returned status 1 (wstat 256, 0x100)
    DIED. FAILED test 1
    Failed 1/1 tests, 0.00% okay
    Failed Test Stat Wstat Total Fail Failed List of Failed
    ——————————————————————————-
    test/chmod.t 2 512 ?? ?? % ??
    test/chown.t 2 512 ?? ?? % ??
    test/getattr.t 2 512 ?? ?? % ??
    test/getdir.t 2 512 ?? ?? % ??
    test/link.t 2 512 ?? ?? % ??
    test/mkdir.t 2 512 ?? ?? % ??
    test/mknod.t 2 512 ?? ?? % ??
    test/open.t 2 512 ?? ?? % ??
    test/read.t 2 512 ?? ?? % ??
    test/readlink.t 2 512 ?? ?? % ??
    test/rename.t 2 512 ?? ?? % ??
    test/rmdir.t 2 512 ?? ?? % ??
    test/s/mount.t 1 256 3 1 33.33% 3
    test/s/umount.t 1 256 1 1 100.00% 1
    test/statfs.t 2 512 ?? ?? % ??
    test/symlink.t 2 512 ?? ?? % ??
    test/truncate.t 2 512 ?? ?? % ??
    test/unlink.t 2 512 ?? ?? % ??
    test/utime.t 2 512 ?? ?? % ??
    test/write.t 2 512 ?? ?? % ??
    Failed 20/20 test scripts, 0.00% okay. 2/4 subtests failed, 50.00% okay.
    make: *** [test_dynamic] Fehler 1

    Like

  17. Remark:
    The /dev/fuse directory was manually created by root…
    If i delete it the user got this:

    dau@nerd1:~$ fusermount: failed to open /dev/fuse: No such file or directory
    could not mount fuse filesystem! at /usr/local/lib/perl/5.8.8/Fuse.pm line 106.

    Like

  18. I’ve just tried out fuseftp. And unfortunately, I found out that lufs, although not longer supported, works much better. Fuseftp can’t cope with directories with names like .svn (anything beginning with .) and generally, is not good at writing data. So, for now i stick with lufs. But I will be watching you, it would be nicer to use software that is still supervised than abandonware πŸ˜‰

    Like

  19. Hi there.
    I’ve tried to install fuseftp though difficult but have got a little success.
    However, fuseftp seems to not work correctly.
    When I mount my local ftp as a directory, I can see all the filenames but having problem reading the contents of the files.

    Like I did mount my mp3 folder into Linux and used mpg123 trying to play one, but just keep getting this message:

    “substr outside of string at /usr/bin/fuseftp line 381.”

    and it keeps repeat and took the whole screen.
    I’m using fuseftp 0.8 and complied libfuse-perl by myself.

    Like

  20. Hi,

    I have a buggy ftp that refuses connection the first time fuseftp tries to connect. It would be nice if fuseftp would have a retry counter, allowing you to retry automatically. Addiotionally I have a lot of ftp servers around that don’t accept e.g. more than 10 users. woulb be nice if fuseftp could handle this.

    btw: I got those ugly FUSE cpan errors too, simply use cpan
    cpan> install Fuse
    cpan> look Fuse
    > make install
    > exit

    That’s it.

    Like

  21. There is a trouble on some FTP servers, when ftp_getdir returns only list of files, not directories. I think, that’s because some servers doesn’t process correctly NLST command. Maybe, it would better not to use Net::FTP->ls()(that uses NLST command), use Net::FTP->dir() instead and parse it’s output?

    Like

  22. I’m having trouble using fstab or the “mount” command to use fuseftp. If you look into mount.fuse, you’ll see that fuse binaries are supposed to follow the following syntax:
    ${FSTYPE} ${MOUNTPATH} ${MOUNTPOINT} ${OPTIONS}

    However, fuseftp expects
    fuseftp OPTIONS MOUNTPOINT MOUNTPATH

    I changed mount.fuse as a workaround, but really the order in which fuseftp gets it’s parameters should be changed πŸ™‚

    Like

  23. In other news:
    –options=allow_others doesn’t work. I think I’m using a wrong version of Fuse… I have version 0.8 of Fuse.pm, is that alright?

    # fuseftp –options=allow_others /mnt/boing/ user@host
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into host
    Backgrounding…
    Use of an invalid mountopt argument at /usr/bin/fuseftp line 588
    #

    Like

  24. hey there,

    just used fuseftp to access a ftp server within my network here. mounting, reading, deleting and stuff works just fine but the performance is somehow dissapointing.

    i use 100mbit here which allows me roughly 9mb/sec using ncftp. if i try to fill the same host via fuseftp i hardly get 200kb/sec.

    is there something i missed? or is this a common problem?

    thanks in advance,
    toskala

    Like

  25. I use gentoo and gave fuseftp a try as the ftp-filesystem using lufis was unusable, hanging nearly all of the time.
    But with fuseftp i get the following error:

    fuseftp kernel/ http://ftp.kernel.org
    fuseftp 0.8 – 2005 (c) by Marcus Thiesen
    Successfully logged into http://ftp.kernel.org
    Backgrounding…
    perl: fuse_opt.c:67: fuse_opt_insert_arg: Assertion `pos argc’ failed.

    😦

    Like

  26. I can’t seen to find Cache/File.pm anywhere in my Fedora Core 4 box. The best cpan comes up with is Cache/FileCache.pm. Did Cache change? A quick substitution of FileCache for File in Makefile.PL and fuseftp doesn’t work, if they did change the name of the module, they also changed the API.

    Like

  27. Maybe this is a strange configuration option somewhere on your system? It somehow does look as if you don’t have fuse at all in your system.

    Like

  28. fuseftp is not really abandonware but more like write once, works, ignore. The thing is that I don’t really have the time at the moment to maintain this stuff…. anyone want to fork?

    Like

  29. Well, actually fuseftp uses both, because it’s hard to find out what is a driectory and what not… I’d like to get an example of this server to see what is the exact difference between it’s output and the regular one.

    Like

  30. The problem here is that the way command line arguments for FUSE application are more or less standard. Maybe one could implement different roles for the script to handle the command line different depending on what $0 says it is called… but time is something that is really rare.

    Like

  31. This is a typo I’ll never get rid off… the problem is that the name is “allow_other” and this is specified by fuse…. the documentation is wrong at some place.

    Like

  32. Well, it shouldn’t be dead slow, but still were running through perl code and we’re doing a lot of caching but still we sometimes need to issue a plentitude of commands (for example a simple ‘ls’ isn’t that simple at all). Maybe that is where the preformance goes. I didn’t experience it to be really slow, but it is still a non local emulated access, which is always slower as a direct access to your local harddrive.

    Like

Comments are closed.