Sunday, July 5, 2009

Implementing system controls

I will comment how I have though that the system controls could be implemented, as always, in order to get some feedback both from final users and developers.

I think that the most important limitation of the A320 is its 32MB of RAM, and that is why I think we need a"only one foreground application running" approach for dingux. These would be the sequence of events after rootfs is mounted by the kernel:
  1. /sbin/init is executed
  2. /sbin/init executes the initialization scripts, launching any required daemons.
  3. /sbin/init executes the main system menu application in "respawn" mode. At this point we have the daemons, the init process and the main menu system application running.
  4. When the main system menu application wants to launch an external application, it does it by using the exec call. This means that the process is replaced in memory. At this point we have the daemons, the init process and the external application running.
  5. When the external application finishes, the init process notices and respawns the main system menu application.
This way, we avoid having some MB of memory locked by the main system menu while it's not actually running. This is an approach completely different to the used in desktop systems, where the window manager and desktop applications are "always there" retaining control of the system.

This approach poses a problem: who takes care of the system controls (volume, LCD brightness and such) ?. It cannot be the main menu system application because then we would not be able to change settings while running an emulator or other application.

I've come up with an scheme that I think would do the job:
  • Modify the keyboard kernel driver to appear as two different input devices. All the special key combinations (power+whatever) would be generated by the second input device, and there would be a special mode entered by one of these combinations in which normal keycodes would also be sent to the second input device.
  • Modify the framebuffer kernel driver to appear as two different framebuffer devices. The second framebuffer would be shown (only while it is in open) as a traslucent overlay over the first framebuffer. The traslucency would be done in software and thus would use quite a chunk of CPU power, but this will be ok in the usage scenarios I will describe.
  • Make a system daemon process that will be listening to the event queue of the second input device, and will open and use the second framebuffer whenever it wants to show an overlay screen.
Usage cases:
  • The user presses the "volume up" key combination (power+up), the system daemon opens the second framebuffer device and shows a "volume up" splash screen for a short period.
  • The user presses the "menu" key combination (power+select), the system daemon opens the second framebuffer device and shows a menu where you can terminate the current running application (which would be identified by looking at who is using the first framebuffer), use a virtual keyboard to send keystrokes to the current running application (i.e. generated by the first input device) etc.
The rationale of the CPU intensive traslucency not being important is as follows: if you are using special key combinations to change the volume, you won't mind if the currently running application slows down a little for a short time. If you are using the virtual keyboard, the currently running application will be stopped in an input box dialog, so you won't notice the slowdown.

You may be thinking that the system daemon would be an application "stuck in memory" which defeats the initial purpose of saving memory. The system daemon will have to be implemented as very optimized code in terms of memory usage code, but most importantly, it will have only a reduced and limited set of feattures, while the system menu application can grow as necessary, including lots of eyecandy, an audio player (instead of an external application), etc.

Someone is already working in suck a daemon and my mission is to provide the required kernel space functionality. As usual, I kindly request your comments and suggestions. In particular, I'm learning about some kernel subsystems on the go, so if you have more experience in input and/or framebuffer devices, I'll be specially glad to hear you opinion on the feasibility of this whole approach.

UPDATE: I did a quick check and it seems that all input events from all input devices reach the controlling tty (makes sense because if you plug two keyboards to a linux box you can type from any of them), which means having two input devices is not the right way to divert certain input events to different processes. An easy way to achieve similar functionality would be to use a dedicated char device only known to the system control daemon, from which it would get special keystrokes and "kidnapped" normal keystrokes in menu mode. This would also solve an issue I didn't notice when writing this entry: if we implement a virtual keyboard in userspace, how do we "inject" the resulting virtual keystrokes back in the input subsystem?.

Well, I guess I could just use special keycodes for all operations to be handled by the system control daemon and assume that the current running application will just ignore them. However, I don't think "assuming" is good practice in general...

7 comments:

  1. AFAIR userspace processes can open a /dev/input device and send the EVIOCGRAB ioctl to prevent that keyboard events are sent to the foreground process.

    ReplyDelete
  2. You can implement these special keypresses as ACPI events like it is done on laptops for Fn+something keys like brightness, volume, etc. The daemon will subscribe to acpid events of acpid or even could (if the memory is an issue and running acpid is a bad idea) read /proc/acpi/events directly.

    I don't think we need a separate framebuffer device to draw an overlay. Upon a special event the control daemon can switch VT to a free one, copy old framebuffer contents into a temporary buffer, draw overlay over it. When everything is done and user wants his app back, daemon should restore framebuffer contents and switch the VT back.

    This approach will also solve the problem of handling non-special keys while in overlay mode - if the VT is switched, they would not be delivered into the "foreground" application.

    ReplyDelete
  3. Using the "uevent" driver, it's possible to inject input events. However, I do not know exactly at which point they are injected; you'd have to be careful to not create a loop where the system control daemon would process the events it injected just before.

    ReplyDelete
  4. @kmeaw: I like your ideas very much. The overlay however is required for the "volume up/down" splash. As far as I understand booboo he wants the splash to display while the application continues to run.

    I had another idea: If we gonna have a menu that can be opened at any time while an application is running, then we could as well include the possibility for the running application to add its own options (like button remapping/frameskip/etc). This would be a huge usability feature IMO

    ReplyDelete
  5. @Bastian: I like your idea! Moreover i would like to see a few of useful embedded apps able to run in "background mode", just like the music player or a simple text viewer, as it happens with the original firmaware. Many users reported as "very useful" the ability of reading files during emulation. Personally i'd like being able of reading ebooks while listenig to music or radio, as i can do with the original fw.

    ReplyDelete
  6. Maybe a stupid idea, but could you not have a device like a framebuffer, but it keeps a buffer only of active pixels along with their x y location... if the buffer is empty nothing is drawn, but if it has contents (e.g. you drew a volume control image) they are copied into the correct places in the real framebuffer by the driver? That way you don't have to copy the rest of the normal framebuffer contents backwards and forwards.

    Also, I have been wondering - anyone got any idea if the USB-host pins are available anywhere on the dingoo PCB?

    ReplyDelete
  7. @booboo: I'd like to move the daemon to a public repository soon. Can I
    do this on your google code project? I think having a centralized
    development infrastructure could be beneficial.

    It still needs a little bit of cleanup and the removal of some
    obsolete sdl checks from the autoconf stuff, I also need to hook
    up a button to bring it up.

    All the modules still need quiet a lot of work, but there's
    infrastructure.

    I also had to (at least partially re-)learn some stuff, since I'm
    a toolkit-guy nowadays and haven't done much low-level linux-stuff
    yet. So my progress wasn't as quick as it could have been.

    I still want to work on it, but maybe I can attract others by
    making it public. I just don't nearly have the time to hack
    on it full-time.

    @kmeaw:vt switching sounds good, will this work?

    @mth: the uevent stuff is exactly what i want to do for the virtual
    keyboard / mouse, but haven't found the time to try yet.

    @batman52: The daemon could do that (running background apps) using
    the (not very thoroughly tested) app launcher. also some UI for it would
    have to be implemented, unless you can live with button mappings
    (which already work, still need one thing to be done). hm, they are also
    quicker, if you haven't got that many programs it should be enough.

    Drawing animated background + translucent UI would require a lot of CPU
    cycles, indeed. I always had drawing a static snapshot of the background
    in mind, because nobody wants a sluggish background app AND menu system.
    The background process would be stopped and restarted by the daemon.
    You could just skip frames when updating the FB to reduce the CPU load,
    but that wouldn't achieve much in the end. Battery lifetime also should
    play a role.

    For the rather static bars of the "basic display mode" of the daemon
    you could also just read out the values every few ms and update the
    corresponding FB regions. Doing that when only their values change
    would even further reduce the CPU usage, but it's only visible for
    short periods of time, so I don't know if it's worth the effort.

    If you want to overlay it with, say, black you could have a pre-blended
    version of the FB and blend with the active UI elements

    I think the menu system will look just fine being opaque. special themes
    could be implemented, however. This could be done by implementing some
    mechanism to retrieve the FB content as it was before a vt switch
    from the daemon. I think at that point we could also provide a c library
    (input mapping, screenshots, whatever comes up) which communicates
    with the daemon when neccessary for community-created programs.

    Here's what I have:
    - a deamon structurized in modules
    - modules less than more working:
    input,mouse(-emu),battery,(app)launcher,screen,sound
    - config file parsing and button mapping
    - sdl independence (draws directly to framebuffer)

    I also have some code in some test apps which I'll merge when
    I find the time.

    ReplyDelete