linux operating system from scratch ?

[sam111]

What I have done
On a blank usb thumb drive
format/partitioned it using ext3 file system
download , unzip , compiled the linux 3.0 kernel.
downloaded and installed grub on the usb
copied the kernel to the usb
set the configure menu.lst for grub to boot the kernel I compiled

So far the usb had a /boot/grub folder that contained the menu.lst , grub files and the kernel.

Then I tried booting the kernel
I get no init found try passing init = bootarg

Question 1
Don't know what I have to do to solve this or what is causing this?
Question 2
My many objective is to do everything from scratch using the least amount of stuff to have a workable os. So I am wondering what additional files,folders, or packages I have to install to this usb. I know I needed a bootloader /boot grub , the kernel , maybe coreutils, maybe a bash shell.
Is their anything else? can I do with out dev , proc , etc , usr , root , home ,...etc. I know it is wise to have these but I doubt these folders are necessary for the linux os to function?


currently when I boot up the kernel I get to a prompt
(mkinitrd)

[hazel]

That's a fascinating project! I don't know the minimum you need but you certainly do need /proc, /dev and /etc. /proc and /dev are empty directories; they'll be populated by the kernel. You need to install init as well as bash and you need an /etc/inittab file. I think you also need udev with a 2.6 kernel. You also must have glibc in /lib because practically all programs use it.

I hope some other people are prepared to contribute to this because I am really interested to see what would be the minimum.

[atreyu]

Hazel is dead-on with the things you are missing. Just a note that you can check out Busybox (and uClibc) to provide some of these things. If your goal is to build a tiny distribution, Busybox is the way to go - as long as you don't mind non-fully featured commands (i.e., bastardized versions of find, grep, etc.).

[sam111]

hazel, you had mentioned I need /dev /proc ,...etc
Why doesn't the linux os/kernel create them if they don't exist.
Since my understanding is /proc holds the processes in memory on disk each folder is the pid killing a pid kills the contains of that folder.
/proc is just a short way of viewing a process in memory's contents
/dev is for device files holds all the character and block devices

/media , /mnt , /cdrom are just mounting folders I don't think you need them you could just tell mftab to mount in a different folder your devices.

/root , /home these are just users directories not sure if they need to be created or can you create just one directory with a different name then root or home. Like everybodiesHome or does those directories have to be their because the kernel has them hardcoded???

for /bin , /sbin can I just not create those and just create everybodiesBin or some name for both. What sbin and bin are , are just folders that contain programs/scripts that the user or system admin use. (usually alot of programs in these folders are created by coreutils package) Just curious if anybody knows if the folders can be renamed or does the kernel look for those specific named folders?

/etc , /usr , /var ,/opt , /lib , selinux , tmp , lost+found simlary question for these directory is their away to rename these directories and tell the kernel the renamed directories are the equivalent of /etc , /usr , .... Or does the structure of the linux /kernel have to have these folders.

Last question
forgetting about the structure of folders if I had all the folders created then installed busybox (or some bash shell) , coreutils (core commands for the shell), init and maybe glibc would I be all set. And beable to boot it with out getting kernel painic or (mkinitrd) prompt showing up? (i.e a command shell prompt that I can execute a simple program like date , time , cat ,...etc)

Why glibc isn't that just a c/c++ library that would only be needed if somebody downloads /installs a program or compiles something that uses these functions. If it is necessary then when installing busybox ot coreutils won't that be a dependancy?

init what is that for? I have a file called initrd that the compiled source gave me can I use this and add it to grub to fix this problem?

[atreyu]

The kernel will populate /proc, but it needs to exist first. Same with /sys. I think udev will create most things for you, but I've had issues if I didn't already have at least a few of the basic char devices there already.

I think you are confusing the Linux kernel with a proper userland suite of tools (i.e. GNU tools or Busybox alternatives, etc.). The kernel is just an inb/t layer that communicates between you and the hardware. You need the filesystem structure and userland tools ("ls", "df", etc.) to interact with the kernel/your hardware.

You need at least a skeleton filesystem in tact once your kernel is finished booting, so that init (which the kernel calls) can take over and do stuff (like run scripts in /etc/init.d/ and process /etc/inittab and so forth).

At the least you should have something like:

/etc/inittab
/etc/rc.d/rc.sysinit
/proc
/sys
/dev (along with some key char devices)
/bin/sh (and assoc. /lib files)
/sbin/init (and assoc. /lib and config files)
/sbin/mingetty (and assoc. /lib and config files)
/sbin/udevd (and assoc. /lib and config files)
/lib/modules/($uname -r)
/boot/vmlinuz-($uname -r)
/boot/initrd-$(uname -r).img
/boot/grub/* (and assoc. /lib files)

This is just an example of some of the essential things. There are surely things I'm missing, but you get the idea.

As for /bin, /sbin/, etc., again, the kernel does not use these paths. But your initscripts (the things that start processes and get you to a login prompt) will likely be requiring some additional commands (df, ps, etc.) in order to complete successfully. You can modify the paths of these commands directly in the initscripts, and install them to the alternate paths, but why would you?

[sam111]

/etc/inittab
/etc/rc.d/rc.sysinit
/proc
/sys
/dev (along with some key char devices)
/bin/sh (and assoc. /lib files)
/sbin/init (and assoc. /lib and config files)
/sbin/mingetty (and assoc. /lib and config files)
/sbin/udevd (and assoc. /lib and config files)
/lib/modules/($uname -r)
/boot/vmlinuz-($uname -r)
/boot/initrd-$(uname -r).img
/boot/grub/* (and assoc. /lib files)

Ok , do you know of what packages I need to get for this info?
2 Main questions
When I sudo apt-get install <package> is their away to tell it to install on a external source like a thumb drive instead of on my working system? ( i.e not touch the main os I am working on)
Also same question I was wondered when making the kernel after it is compiled/linked and you have the kernel. Typing "make install" I believe copies the kernel to the /boot directory and updates the grub menu.lst. Is their away to tell "make install" to place/install it on an external device like the thumb drive (i.e not installing the kernel in my main linux os )
Or is it just make then go to the kernel and manually copy it with cp or drag and drop then update the menu.lst manually.

Those are really my two main problems write now.
After I resolve them I can just start download a ton of different software and installing it on the thumb drive (not effecting my main os) andseeing what works with what then pull stuff out ,...etc etc

[atreyu]

No, I don't know of an automatic way to install deb packages to an alternate directory. I'm not so sure this is what you want to do anyway. You probably want full control over exactly what gets installed to your new OS.

If I were you, and I wanted to make an OS on some external drive that is binary compatible with another one, I'd get the packages installed on that other one first, then copy over only the portions of the package I need to make it work, to the external drive.

Case in point, for the "arch" tool, I'd just copy over /bin/arch and any libs it was linked to at compile time. You can get that with:

ldd /bin/arch

In this case, I'd copy over these files:

/lib/tls/libc.so.6
/lib/ld-linux.so.2

(and related symlinks)

EDIT: another easy thing to do is just uncompress the .deb files, they already have the proper file structure embedded in them. I think you would do something like:

ar x package.deb
tar zxf data.tar.gz

[sam111]

Ok , I will do it that way.

Curious though when I apt-get install something is that just downloading the deb package (and dependencies), decompressing them and copying the files to the directories.

Or is their more things it is doing? Like updating files, running programs ,...etc?

Or is apt-get (front end for dpkg) , ...etc package managers just doing the equivalent of downloading a tar file , untaring it, and copying the files where they need to.

apt 0.8.13.2ubuntu4.1 for i386 compiled on Jul 7 2011 18:30:45
Usage: apt-get [options] command
apt-get [options] install|remove pkg1 [pkg2 ...]
apt-get [options] source pkg1 [pkg2 ...]

apt-get is a simple command line interface for downloading and
installing packages. The most frequently used commands are update
and install.

Commands:
update - Retrieve new lists of packages
upgrade - Perform an upgrade
install - Install new packages (pkg is libc6 not libc6.deb)
remove - Remove packages
autoremove - Remove automatically all unused packages
purge - Remove packages and config files
source - Download source archives
build-dep - Configure build-dependencies for source packages
dist-upgrade - Distribution upgrade, see apt-get(
dselect-upgrade - Follow dselect selections
clean - Erase downloaded archive files
autoclean - Erase old downloaded archive files
check - Verify that there are no broken dependencies
markauto - Mark the given packages as automatically installed
unmarkauto - Mark the given packages as manually installed
changelog - Download and display the changelog for the given package
download - Download the binary package into the current directory

Options:
-h This help text.
-q Loggable output - no progress indicator
-qq No output except for errors
-d Download only - do NOT install or unpack archives
-s No-act. Perform ordering simulation
-y Assume Yes to all queries and do not prompt
-f Attempt to correct a system with broken dependencies in place
-m Attempt to continue if archives are unlocatable
-u Show a list of upgraded packages as well
-b Build the source package after fetching it
-V Show verbose version numbers
-c=? Read this configuration file
-o=? Set an arbitrary configuration option, eg -o dir::cache=/tmp
See the apt-get(, sources.list(5) and apt.conf(5) manual
pages for more information and options.
This APT has Super Cow Powers.

Because if it is just the latter then I would agree in just installing temporarily on the main os then copying the files and folders it made to the new os.

Any package manager experts out their?
I am not sure what a .deb file structure is? Is it just a list of dependencies needed and the equivalent compressed executable files (equivalent to a tar file format)

[atreyu]

If dpkg is like rpm (as apt-get is to yum), then it is doing more than just copying the files. It is also probably running pre and post-install scripts, possibly backing or modifying program config files, and updating the package database.

The deb structure is pretty basic - see here. The data.tar.gz file has all the program files to be installed, bundled in the filesystem hierarchy that is to be used for installation.

[sam111]

I will look into the dpkg , rpm package formats thank you for the link thats just what I needed.

I guess what I am mainly wondering now though, is why do we need apt-get , or yum if we have dpkg and rpm?
Correct me if I am wrong but the only differences of useing (apt-get,yum) to (dpkg,rpm) is that it is downloading the .deb packages from the internet. dpkg and rpm assume that the (.deb , .rpm files) are already located on your machine for install.

You could get the same affect as apt-get or yum by just using wget or curl and downloading the .deb or .rpm files then installing them using dpkg or rpm once local. Correct me if I am wrong or missing any functionality.