Looking for a way to have a portable filesystem (or mounting without root)

[dkulchenko]

I have a free software project I'm working on that provides portable versions of Linux applications capable of being carried around on removable media, with settings and documents traveling along.

While developing the portable launcher, I fell into a problem: FAT32 partitions do not support symbolic links. This becomes a problem because 99% of Linux programs make use of symlinks for libraries, shared files, and whatnot. Therefore, I can solve the problem by duplicating directories where the symlinks would be, but this is extremely inefficient, and becomes a space problem on USB drives.

In my search for solutions, I thought of having a filesystem in a file that mounts into a temporary directory, from which the program then executes. The problem with this solution is that to mount a file, you require root privileges, and my target user base is users that will be using the apps on random computers, users that will not have root privileges on those computers.

My question is: Is it possible to have a read-write filesystem contained in a single file, and be able to mount it without root privileges? Maybe there is an alternative to mounting that accomplishes the same thing. Any ideas?

NOTE: Because of the situation, the users should not require root access on the host computers that they will be running apps on. It is safe to assume that they will have root access on the computers from which they will be downloading the apps from.

ANOTHER NOTE: I can't just use ext3 for the drive, because the portable apps will be for public use, and almost all removable media is either FAT or FAT32, neither of which support symlinks.

[WuNMOjTIQanXWBcH]

Question:
I request clarification. From your description of your problem I can only derive 2 scenarios, both of which donʼt make too much sense to me, although I try to provide their implicitely associated questions as best as I currently can. If my understanding of them is erroneous, donʼt bother to read the associated answers.

1. 
   Scenario 1:
   From what I understand, when the time comes that you want to run an application, a minimal GNU/Linux system, which is resident to the USB drive, is already running. Because the application is not installed on this partition (w.l.o.g. I assume itʼs only one partition) I deduce this is the FAT32 partition you were describing (if not: why not install there directly? Wasnʼt it read/write-able?). I donʼt know if this is sanely possible at all¹, but into this system you try to mount a mini-image that contains the desired application with all its dependencies, like a chroot-environment.
   
   Answer:
   If so, then that would certainly be possible: you could have those image files stored in a fixed location, let a boot startup script, that possess root privileges, use «losetup» to associate those image files to loop devices and state in «/etc/fstab» that those devices are user-mountable (via the «user» option).
   
   Warning:
   This is dangerous for a number of different reasons:
   
   • Installing a GNU/Linux system on a FAT partition. The loss of disk space (no symlinks) and performance (linked list FS) must be substantial.
     
   • Giving the user the ability to mount, even though only certain ones, loop devices. If a user somehow succeeds in replacing one of these image files, then you could have just given him/her the root password directly as he will be able to replace them with an image file where thereʼs a suid root application on that grants any caller root priliveges. Even worse, as the images seem to be read/write-able, the user only needs to get access to the images on the USB drive once at a computer with root access and [s]he can directly place those binaries on there; as practicly everyone will have root access to a GNU/Linux system sometime in his/her life, this is equivalent to placing the root password in a README text file, that is placed in every folder on the computer.
     
   • Lots of kittens will die.
     
   
   
   
2. 
   Scenario 2:
   Thereʼs a GNU/Linux system or equivalent (VMWare, Windows with Linux emulator, etc.) already running and you want to “import” an environment resident to the USB drive.
   
   Answer:
   That oneʼs easy: nobody uses FAT32 as his/her “/” partition. However you cannot expect to mount image files on these systems without root privileges naturally. In fact, Iʼd rather not count on that. But you donʼt need this as you can simply put these environments directly onto a partition on the USB drive (like a portable home directory) and mount them into any system, provided that the applications on the USB drive donʼt have any nasty precompiled directory dependencies. If a user canʼt even mount these partitions, then [s]he cannot use the USB drive at all.

ANOTHER NOTE: I can't just use ext3 for the drive, because the portable apps will be for public use, and almost all removable media is either FAT or FAT32, neither of which support symlinks.

I do not understand this statement. In my opinion removable media contains the filesystems the user who partitions and formats the drive wants. The only drives Iʼve ever seen which carried predetermined filesystems where readonly media.

Footnotes:

1. I just did a
   

   $> find /usr/lib/ -type l | wc -l
   1329

   on a server system. This does not bode well.

[dkulchenko]

Thanks for the detailed answer, WuNMOjTIQanXWBcH! Scenario 2 best applies to me. I would already have a running Linux system, but you slightly misunderstood the mini-image thing. My images would not "import" into the environment. I simply want them to mount into a temporary directory, like "/home/user/.mount", the program ran from there, then unmounted when done. My dir structure of the mini-image is not /usr, /bin, it's completely different, and my program takes care of that. I simply need to mount a mini-image from the drive into a folder, execute my program, and unmount when done.

I assume that the drive is already mounted, because most desktop Linux systems auto-mount, and my target user base is mostly end-users.

[WuNMOjTIQanXWBcH]

Scenario 2 best applies to me. I would already have a running Linux system, but you slightly misunderstood the mini-image thing. My images would not "import" into the environment. I simply want them to mount into a temporary directory, like "/home/user/.mount", the program ran from there, then unmounted when done. My dir structure of the mini-image is not /usr, /bin, it's completely different, and my program takes care of that. I simply need to mount a mini-image from the drive into a folder, execute my program, and unmount when done.

I assume that the drive is already mounted […]

Then the same answer as to scenario 2 applies here:

However you cannot expect to mount image files on these systems without root privileges naturally. In fact, Iʼd rather not count on that.

The neccessary rules to allow non-root users to mount loop devices had to be setup in the host system. Because you donʼt have the neccessary privileges, this wonʼt be possible (and still be a very bad idea for the aforementioned security reasons).

I still do not understand what prevents you from using a filesystem that supports symbolic links. This way there would be no need to put these applications into image files and you could simply put them directly onto the USB drive in a reasonable directory tree and simply start the applications in question from there. Thereʼd be no need to mount anything (except the USB driveʼs main partition, of course).

[dkulchenko]

But this translates to 2 solutions:

1. Have the user backup all their data, format their drive as ext3/ext2, then restore all their data. This also means that the drive is now not readable by Windows anymore.

2. Have the user convert their drive to NTFS on a Windows box (which would mean retaining Windows support), but it would require that the user install ntfs-3g on each computer they use so that they can read-write to the NTFS drive, and my situation does not allow me to impose the user to install anything like that on their host computers.

I don't think I can really impose either one of these 2 onto the user. Maybe there is another solution other than these 2 you can think of?

All this is, of course, because 99% of USB drives that you buy come preformatted with FAT32, and 5% of endusers reformat their USB drive to something else.

[WuNMOjTIQanXWBcH]

1. Have the user backup all their data, format their drive as ext3/ext2, then restore all their data. This also means that the drive is now not readable by Windows anymore.
[…]
All this is, of course, because 99% of USB drives that you buy come preformatted with FAT32, and 5% of endusers reformat their USB drive to something else.

Now I can at least partially understand why you want to use FAT32 as the USB driveʼs partitionʼs filesystem. However a few questions are still open.

What do you mean by

1. […] This also means that the drive is now not readable by Windows anymore.

Is there any impending reason why you cannot have 2 partitions on the USB drive: one data partition (FAT32) and one for the GNU/Linux applications?

As for

All this is, of course, because 99% of USB drives that you buy come preformatted with FAT32, and 5% of endusers reformat their USB drive to something else.

Do you really think that is a valid reason? At one time a user has to setup this USB drive anyway. At exactly that point [s]he could be provided with the instructions neccessary to partition the USB drive properly (or let an assisting utility do the work). As an analogy: A minority (< 10%) of the common Windoze or GNU/Linux userbase actually knows how boot sectors or MBRs work (this is a subjective guess), yet they use them every time they install an OS or boot from CD. This knowledge is abstracted as a simple click (“burn CD” in Windoze) or CLI command (i.e. “cdrecord” in GNU/Linux systems).

Maybe there is another solution other than these 2 you can think of?

Not at the moment.

[dkulchenko]

Is there any impending reason why you cannot have 2 partitions on the USB drive: one data partition (FAT32) and one for the GNU/Linux applications?

Hmm... That's an interesting idea. So, at first run, I could have the user log in as root, then my script would resize the FAT32 partition, create a new ext3 partition in the remaining space, then put the programs onto there? Interesting... There would be a few size concerns (I would have to check for amount of space free on drive, and prompt user of the size they want the new Linux partition to be), but that would be doable.