Wednesday, July 8, 2009

Call for system library requests

I've been recenly using an uclibc toolchain and polishing a minimal rootfs (not the quick hack I released with the first working kernel). This setup won't need an ext2/ext3 partition, just a couple of files you'll have to place in your existing miniSD FAT parition.

This, however, comes at a price: the root filesystem is readonly (romfs... we have plenty of space, no need to lose microseconds decompressing cramfs or squashfs). This means you cannot install anything in the root filesystem, and will have to use the FAT partition. This is OK for almost everything but for system libraries, because they use symbolic links which cannot be created on a FAT filesystem. There are ways to achieve this (comments on this later), but none is simple and certainly not for the first release.

So I intend to release what might become the reference rootfs and development kit, and as such, I would like it to include all the required libraries for applications. So far it includes most of the general purpose libraries that buildroot allows to install plus SDL and derivatives.

I would like to hear from you, devs that are porting apps to dingux: what libraries are your applications using?.

And finally, some right-off-of-the-top-of-my-head ideas on a way to allow addition of system libraries to a romfs based root filesystem:
  • Allow expansion of the rootfs vía filesystem overlays: unionfs (heavyweigth but high functionality) or mini_fo (lightweight but limited functionality).
  • Mount /usr/local/lib as a tmpfs (i.e. ram disk) and on startup create there symbolic links to extra libraries in the FAT partition. Each extra library would have to come with a special file that would define the symbolic links to be created.
Any other ideas?

27 comments:

  1. Just an fyi on the libs and buildroot. Most of them build fine but there are a few issues you should keep in mind. For sdl_gfx and sdl_sound I added those to the package directory for buildroot so they are selectable, eventually I will make a patch for buildroot and upload this so it can be added to the buildroot svn. Second, for the ogg support you should be using tremor since thats the fixed point version, but buildroot has issues when installing it. It looks like it installs it into the rootfs but not into the toolchain itself, or the other way around cant remmember. Either way you should check and make sure that tremor installed correctly along with the headers because I had to copy those in manually because they werent done correctly in the buildroot process.

    ReplyDelete
  2. Also a good test to use is the waternet source code. I was given this by joyrider, he doesnt want the actual source to be leaked out since its hacky, but it uses most of the sdl libs and vorbis and tremor. So it tries to link in most of the libs that people use so its a good test to see if the toolchain has everything setup properly. I can ask him if he can send you a link to the sourcecode or if I can send you the link

    ReplyDelete
  3. I'm not a programmer but I think it would be good to set up java applications to run on the dingoo. Since many phone apps would probably work well because of similar resources. I want to see mobile Gmaps or something similar on there with the ability to download maps through usb.

    ReplyDelete
  4. You could add opkg support so any additional library can be installed by user.

    See http://wiki.openmoko.org/wiki/Opkg for more info.

    I don't think we need any special files defining symlinks: ldconfig does it automaticaly in some way, you could use the same approach.

    Do not use unionfs, there is aufs. While unionfs is in process of fixing OOPSes, aufs is gaining performance and new features.

    Most games need music, so it will be nice to have an MP3-decoder(libmad), MIDI softsynth ((lib)timidity), tracker format support (libmikmod, libmodplug), libogg and libvorbis.

    ReplyDelete
  5. Another possibility is to loop-mount 'package' files, add /PKG/PKGNAME/lib to ld_library_path and /PKG/PKGNAME/bin to the path. No need for symlinks. Or the symlinks can be inside the package file.
    Then for 'data' files, they will be located for instance in /PKG/PKGNAME/data.
    We still need to mount a tmpfs in /PKG.

    I think that squashfs or cramfs can actually speed up application launch times, because less data is read from the SD, and the Dingoo has quite a lot of power. But maybe it will enlarge the kernel and / or use more ram.

    When you release the 'reference' files, please explain how to get them, how to compile the toolchain and so on, in order for developers to be able to replicate your setup.

    About libraries I'd like to see included:
    zlib, libjpeg, libpng

    ReplyDelete
  6. I'm not really sure, but, shouldn't it be possible to do a loopback of a filesystem image using EXT3 rather than using overlays or whatever? It wouldn't have to be read-only since it wouldn't be ROMFS or similar and I think there doesn't have to be any overhead since it would just write the data directly to the image as-is without anything actually moving around in the host filesystem (eg FAT16 or FAT32.) If this were possible, it would also solve a problem in making this easier for Windows users. A generic filesystem image could be simply distributed to get people started and then they could add their own stuff to it later. (Later, once the linux distro is more complete and you can do more things like file transfers one would never even need to boot into linux on the PC.) That would certainly make it easier for less experienced users and every step of the process on the PC could be done in Windows for those less experienced users.

    I've seen one live distro (Finnix) use an approach similar to this, though it does still use UnionFS (I think with SquashFS for the main filesystem and UnionFS for the writable overlay if I remember correctly.) On startup it has to search the PC for the medium, find the filesystem image, and then it mounts that image and switches the root to it. However, Finnix is designed to be run from a CDR/RW, so doesn't expect to find a writable medium. I think if the host filesystem is writable you could possibly get away with a loopback?

    I've heard in particular of some people doing this sort of thing for running a desktop linux distro (such as Mandriva or Ubuntu, though I forget any specific examples) on a MS filesystem (I've even heard of it used with NTFS.) Of course, I understand that the Dingoo linux distro must be kept as minimal as possible and I'm not sure if it's capable of doing the same things in this respect. I think in particular this guide shows how to do this sort of thing:
    http://www.faqs.org/docs/Linux-mini/Loopback-Root-FS.html#s3
    Though you can probably do it a little more cleanly than that since it's ok to define a more rigid standard here.

    PS. Is it supposed to work with FAT32 now? I see people talking about it, but when I tried a FAT32 partition it would simply freeze whenever trying to load the kernel+initrd image. When I started over with a smaller FAT16 partition it worked fine. This is kind of a problem when you have an 8GB SD card though and want to dedicate just a bit of that to linux, using the rest for videos and such. Especially since most things won't let you create a FAT16 partition larger than 2GB. Does it have some special requirements for the FAT32 cluster size or something perhaps? Every time I try a FAT32 partition the bootloader freezes when trying to load linux (though it will boot the original OS fine btw) and when I switch to FAT16 it works fine.

    ReplyDelete
  7. Xbox Linux distributions were able to use loopback filesystems because the harddrive is required to be formatted a certain way or else the system doesn't start. It worked fine otherwise. How hard would that be to implement?

    ReplyDelete
  8. The whole SDL family (that is SDL_mixer, SDL_net, SDL_image, SDL_gfx, SDL_ttf, SDL_sound and so on), vorbis, vorbisidec, allegro (port of this one would be great), SGE.. that's all i can think of now.
    And generally check what's included in the current uclibc rootfs/toolchain.

    ReplyDelete
  9. @Evan

    The whole SDL family will be included. I had already noticed the tremor issue and will try to fix it manually or by modifying buildroot.

    For waternet, just compile it for your desktop, run ldd against it and send me the output. That should be enough to know which libraries it's using.

    @Derk

    I noticed that buildroot has an option to build java VM and classes, but didn't work out of the box, so I put it aside. Will eventually try to get it to compile sometime.

    @kmeaw

    Could you please specify the advantages of opkg against, for example, ipkg ? (I mention the latter because that's the one included in buildroot).

    Thanks for the aufs suggestion. If I decide to go with the overlay filesystem approach I'll use it instead of unionfs. I'm a bit reluctant though to include a whole new filesystem due to the overhead, so I'll go for it only as a last resort.

    @cyberic

    Creating the links on startup on a tmpfs looks like a simpler approach. Note also that threr is a limited number of loop devices.

    Regarding squashfs/cramfs, you're right. I'll perform some tests to make sure the bottleneck is not in the miniSD reading.

    All the libraries you mention are in buildroot and I enabled their compilation.

    @Nazo

    Using ext2/ext3 for an image file sitting in a FAT paritition poses some problems:

    - Since all writes have to be ultimately performed by the FAT code, the journaling advantages of ext3 are actually useless.

    - ext2/ext3 is HUGE in kernel. Removing it alltogether is VERY tempting.

    There is minix which is very lightweigth, but it has a 64MB partition size limit.

    Using romfs has the advantage of being extremely lightweight (in kernel code and memory requirements) and providing read-only protection to the system core: rememeber, the A320 can crash or be powered off at any time by the user. Nothing will be more reliable than a read-only root filesystem.

    @Ryan

    My currently working romfs-on-FAT scheme already uses loopfs. This is how it works:

    1- The kernel includes an ultratiny initramfs which:
    2- Mounts the first (FAT) partition of the miniSD on /boot.
    3- Sets up a loopback device on /boot/rootfs.
    4- Mounts the loopback device on /mnt/root.
    5- Moves the /boot mount to /mnt/root/boot.
    6- Pivots root to /mnt/root and releases the initramfs.






    @All

    I like the "only /bin" approach more than the "/bin and /usr/bin" which buildroot produces. Is there a simple way to achieve this in buildroot?.

    ReplyDelete
  10. I already posted a library list in another thread, but I'll repeat it here where it's on-topic ;)

    SDL and co are already on your list, that's good.

    For image decoding, libpng and libjpeg. zlib is useful and required by libpng anyway.

    For font rendering FreeType, SDL_ttf and a good standard font (DejaVu? Droid?) would be very handy. Note that you can slim down FreeType a lot by removing support for everything except TTF. If you'd like an example configuration, please mail me.

    A small XML parser would be useful too. Maybe Expat or a minimal build of libxml2. For easy porting, both would be better.

    For embedded scripting, Tcl and Lua are popular. openMSX (which I'm porting) uses Tcl; I can always link to it statically though if there aren't many other packages that need it.

    ReplyDelete
  11. @booboo
    Hi booboo
    Are you still working on the frontend or are you wainting to finish the new system library ?

    ReplyDelete
  12. @booboo

    ipkg development has stopped since May 2006. Latest commit to opkg source tree was about 5 days ago. ipkg still has some memory leaks, which made my iPAQ with 32M run out of free memory. opkg works faster with a database of about 1000 packages on Openmoko Freerunner, than ipkg.

    ReplyDelete
  13. @kmeaw: ipkg are still used in angstrom, or openembedded, for instance. This is only a package format, that you can extract with 'ar -x', or install on a writable filesystem.

    Opkg (and ipkg as well) are not designed to be mounted, but to be extracted.

    ReplyDelete
  14. Some important libs (all of them are audio decoding libs) for me would be: libmpg123 (compiled with the generic integer decoder), Tremor (integer vorbis decoder), libFLAC, libmpcdec (Musepack decoder, also important to enable integer decoding here. It is not the default) and mikmod.

    ReplyDelete
  15. booboo, the java vm issue is actually a really easy fix, its a one line change of code. I had to do it once before because whoever maintained the buildroot build of javavm has long since stopped. It is basically just an uninitialized variable problem. If I remember correctly it was some file descriptor and after reading through the code it was fine to set it to 0 or -1 at declaration, dont remember which. So you should just be able to go to where it is and make that one change and I think it builds from that point on.

    ReplyDelete
  16. @Evan

    Can you send a patch for the javaVM compile problem?

    ReplyDelete
  17. I'd like to have uClibc++. I'm currently using the old uClibc toolchain and libstdc++ eats up 776k.

    I've decided to set up redmine and separate svn for the daemon (wanted a redmine install anyway). It's done and I'll make it public in the next couple of days.

    ReplyDelete
  18. I would say Qt Embedded (if kernel supports local domain sockets). Qt Embedded is great for writing all kinds of applications. For example, I wrote a Dingux file broweser similar to the one in the original firmware in about half an hour using QtE. Development was done using the QtCreator IDE on Linux desktop and then recompiled for Dingux by just switching target platform...

    ReplyDelete
  19. @Master: I would be interested in looking at Qt embedded. Please release some software/notes after the 'base files' have been released!

    ReplyDelete
  20. @booboo: sure I can make a patch, probably wont be able to get around to it till early next week though.

    ReplyDelete
  21. smpeg would be nice. :)
    libbzip2 could be useful too.

    Other nice things to have:
    mingetty so we can autologin as root and start any application from inittab easily.
    nano to edit configuration files on the fly. (I can't stand vi >_>)

    ReplyDelete
  22. Getting rid of /usr/bin sounds a bit drastic... why do you want to do that?!

    ReplyDelete
  23. @Ed

    Makes things easier when adding /opt/whatever/bin to PATH in the toolchain. But in the host I don't like to have all the stuff in /bin.

    ReplyDelete
  24. Is there any place specific you want me to upload that patch to for the classpath?

    ReplyDelete
  25. Also are you using buildroot off of the svn, or latest official? I am going to submit patches soon for the sdl_gfx and sdl_sound, not sure how long they take to go in, but if your on svn you should get them when available. If not I can probably upload patches for those as well

    ReplyDelete
  26. My bug report for buildroot about tremor not being installed into the toolchain just got moved to fixed. So the latest version of buildroot in git should have that problem fixed so you wont have to do it by hand.

    ReplyDelete
  27. I would also like to vote for aufs2. I have used it in many projects and am quite impressed.

    ReplyDelete