Thursday, October 22, 2009

New system/toolchain

Quick post. I just uploaded the latest system files and toolchain.

Changes are:
  • Compiled for MIPS32. Previous toolchain release was wrongly compiled again for MIPS I.
  • The whole system (libraries and executables) have been compiled with -O3.
  • Fixed MP3/OGG integer decoding in SDL_mixer.
Now for something completely different: if you are having trouble getting dingux up and running, try the following as suggested by some users:
  • If you can't get the dual-boot installer to start, that is, you see nothing on the A320 screen after successfully running the two usbtool commands, try to unplug then the USB cable. No idea at all what's the problem and how unplugging the cable fixes it, though (I would need such an A320 and attach a serial console).
  • If, after installing the dual-boot, you cannot get dingux running, try to do a complete FAT32 format of your card (with ot without partition, dingux now supports both), and then copy zImage and rootfs before anything else. There might be some problem in u-boot-1.1.6 preventing it from reading FAT32 files in some circumstances, though I tried to fix it by backporting code from the latest u-boot.

Monday, September 28, 2009

Progress on x760+ / buildroot published

First and foremost: sorry for being so hard to get in touch with lately. We've been in defcon 1 for the last week at work preparing a demo for the current project (OMAP3530 based) which is crucial to the future of the company, so I had reduced my daily routine to sleep-eat-work-sleep. My inbox is exploding.

Sweetlimre commented on having a writable home directory (see interview). If I recall correctly, I wrote about this already. The main application executed by busybox's init process is responsible for this. At this stage in the boot process no HOME environment variable has been set. I could easily modify busybox's init to set it to "/usr/local/home", but I felt it's better to stick to a vanilla busybox as much as possible. Oh, wait: maybe if (as I recommended) the main menu application is using exec() to launch emus/apps things are not so simple. Need some input from devs here, and if the only solution is to modify busybox's init, so be it.

Regarding the modified buildroot I'm using, as he requested it's now available for download at the google code project page. I hadn't published it before just because I could not find time to fix all the dirty hacks I used, which resulted in a partially manual build. I'll be glad to add anything developers suggest. Just send me the buildroot recipe. I would really like to stay away from OpenEmbedded. I have to use it for the OMAP3530 and IMHO it is way overengineered.

Some good news on the Ingenic front: the Qi-Hardware guys have managed to get Ingenic to release their kernel development trees DAILY. This means immediate access to any useful fix they make. There are 2.6.24.3 and 2.6.27 branches, and I'm working on porting the A320 support code to the later, mostly to see if it helps somehow with the USB/DMA and SD/MMC standing bugs. Note that in the case of embedded devices like the A320 I don't think that newer kernel is necessarily better.

Regarding the Gemei x760+ things are going slower than expected. I dumped the hardware initialization .DL from NAND flash and disassembled it, only to discover I was looking in the wrong place. Let me explain:

When the JZ4740 boots a piece of code called the IPL (Initial Program Loader) is executed from ROM. Depending on the state of some pins this code either enters USB boot mode or boots from NOR or NAND flash. In the A320 we can only choose to enter USB boot mode or boot from NAND. The IPL only supports 512 and 2048 page size NAND, so despite the fact that the NAND chip in the A320 has a 4096 page size it is handled by the IPL as if it was 2048.

The IPL reads the four first pages (8KB total) of NAND into the instruction cache (because the SDRAM is not yet available). This is called the SPL (secondary program loader) and its purpose is to do a basic hardware initialization, most notably making the SDRAM available, and load the system loader from NAND. The A320 SPL also handles the NAND as if it was 2048.

In the original firmware the SDL is stored in the first 8KB of the first NAND block (0x00000-0x3FFFF). It loads the system loader from 0x50000-0xBFFFF. Before loading the operating system the A320 system loader does something interesting: it loads from 0x40000-0x4FFFF a piece of code that I call the hardware initialization DL. It is a dynamic linked object code chunk that does board-specific hardware initialization: GPIO, LCD, etc. This is the interesting stuff. In both the older and newer A320 the LCD initialization code was reverse engineered from this DL.

However, the x760+ LCD initialization seems to be mostly done by the system loader itself. It stil loads the DL and uses it for some GPIO initialization that is also related to the LCD controller, but not much more. The DL does contain a large LCD register initialization code routine, but it is unused (and I lost a lot of time reverse engineering it).

Since the system loader code is much larger (~260KB) than the DL code (~10KB), it's gonna take some time to reverse engineer the LCD handling code. Note also that I had already reverse engineered the A320 DL code and that helped a lot, but the system loader is unexplored territory.

Wednesday, September 16, 2009

Testing new dual-boot installer

I've just released a new dual-boot installer in the hope that it fixes the "flash write error" problem some users have experienced. The error is caused by one or more bad pages in the first block of NAND. This is normal and there is a mechanism that was supposed so handle it but wasn't working. As I don't have bad pages in this first block in my two A320s, I can't test it myself, so, until someone reports that it is working for him, consider it beta.

UPDATE 1: so far three success reports. Yeah, looks like this bug is squashed!.

UPDATE 2: confirmed. No more flash write errors.

Thursday, September 10, 2009

Merged latest Ingenic patch

No big news. Ingenic released a new kernel patch, which mostly adds support for a new development board. There are though some minor changes in common parts of the assembly code in the core, so, just in case they had fixed some bug I applied the patch (their ChangeLog doesn't say anything, though).

I tested the two major standing problems (USB failure when using DMA and card corruption when using 4-bit bus mode) and they're both still there. Regarding this last issue, I recently had my card corrupted again, so there is a chance that setting the card in 1-bit bus mode only makes the bug much less likely to happen.

Some other in-no-particular-order news:
  • Fixed the network mask problem (changed from 255.0.0.0 to 255.255.255.252).
  • Rebuilt the whole toolchain and rootfs as MIPS32r1 architecture (was MIPS I). This should result in faster, smaller code. Ingenic doesn't even mention in their documentation that the CPU is MIPS, much less say which MIPS flavor it is. However, Vladimir Silyaev noticed that the Ingenic kernel default configuration is for MIPS32r1, so the toolchain should be too).
  • The guys at Qi-Hardware sent me a prototype of their Ben NanoNote device. A post on this soon. They also got a newer patch from Ingenic (kernel 2.6.27), but I'm yet to find time to examine it for fixes that might solve the USB DMA or the card corruption problems (not optimistic thought, because any important fixes should have gone out in their latest "official" 2.6.24.3 patch).
  • Some users have reported flash write failures during dual-boot installation. The original SPL restore feature works fine, though. It seems that they have bad pages in the first flash block (which is normal in NAND flash) and the flash write tool doesn't handle them properly. This should be easy to fix, but will take some time because I haven't got an A320 with bad pages.

Tuesday, September 8, 2009

Good news on Gemei x760+


Today I installed the serial console connector on the Gemei x760+ and had a chance to examine the hardware a bit better. I would say that except for the buttons and the LCD the hardware is exactly equal. Same CPU, same SDRAM, same FM chip, same TV-out chip, etc.

I tried to boot dingux straight to SDRAM vía USB boot mode and everything worked, except that the kernel paniced because I had no SD/MMC card inserted and thus no rootfs in place. I could see the LCD backlight blinking, so the same GPIO pin is used to control it.

As I mentioned in the previous post test point 1 (TP1) is the console TX signal, and I confirmed that test point 2 (TP2) is the RX signal.

So, we should have dingux runing on the x760+ in a few weeks. All that's needed is map the GPIO pins (if different from A320) and disassemble the hardware initialization code in the unbricker tool to get the LCD initialization code.

I would say the donated money has been put to good use :-). Thank you all again.

Offtopic UPDATE 1: I see a problem coming: the lack of start and select keys. On one side, you know that the dingus kernel uses power+start+whatever for special functions (volume, brightness, reboot, etc). On the other side, developers have been porting applications and using the very convenient start and select keys. I would like to hear suggestions on how do deal with that. Besides up/down/left/right and the four "action" buttons, the x760+ has only power and reset buttons.

Offtopic UPDATE 2: I was wrong in the previous post: the x760+ indeed has a power button (not slider), and does charge through USB.

Friday, September 4, 2009

Gemei x760+ (UPDATED)


As mentioned in the preceeding post, the Gemei x760+ arrived from DealExtreme yesterday. Fridays use to be a bit frenzy at work but a while ago I could finally take a break and dissect it.

At a glance:
  • Very sturdy feel.
  • Shorter but wider and thicker than the A320.
  • Smaller battery than A320 (you'll see the internals later), should mean shorter battery life.
  • I'm not a gamer myself, but the gaming controls of the A320 are much better. In the x760+ you touch the rubber pad, while the A320 has hard plastic caps/cross over it.
  • No on/off switch.
  • Large SD/MMC card slot, maybe more convenient than miniSD in the A320.
  • Seems not to charge from USB, which is certainly quite inconvenient. Has a separate power port, which is +5V like USB, so I just can't understand this design decission.
It came uncharged, so I can't at the moment comment on the LCD quality. Size is obviously the same.

Here are a couple of pictures so you can compare sizes:



Splitting it open was a bit hard. You must remove the four tiny screws from the sides and then pry it open carefully. The plastic snaps that hold it together fit extremelly well, so I would say it is almost impossible to open it without breaking at least a couple of them. If you want to do it anyway, I'll soon make available the full set of pictures where you see where they're located before opening. That should increase your chances. I haven't put it back together yet, so I don't know how the couple of broken snaps affect the body integrity, but I bet it's almost unnoticeable.


In the following picture you can see the circuit board with better detail. The test point 1 is highlighted, and it is the serial console transmit signal (3.3V). I had a hard time finding it because as opposed to all the other test pads, this one has solder on it and this does not have the typical golden appearance:


Once you remove the internal screws, you gain access to the other side of the board where the LCD is located. Note that as opposed to the A320 where the LCD is soldered, here the LCD has a connector and thus can be easily removed:


This should be the LCD model (some clues on it too in the hidded diagnostics screen of the firmware). Haven't even googled for the LCD model yet:


Finally, once the LCD is removed, this is what you see on the back of the PCB:


The highlighted test point 2 is my best suspect for the serial console receive signal, but this is yet to be confirmed.

For the curious, this is the serial console output seen on the serial console transmit signal during a normal boot:

NAND Booting...ECD755B6..
loader size = 0x00050CA0
.00000119:1..
OK
NAND Loading...
get ccpmp_config ok!!!
ccpmp_config.firmware_name = GM760P.HXF. ccpmp_config.update_key = 123, ccpmp_config.lcm.width = 320, ccpmp_config.lcm.height = 240.
loader normal mode...
Creating ftl device...
id:EC D7 55 B6 78
id:00 00 00 00 00
id:00 00 00 00 00
id:00 00 00 00 00
OK.
usb_connect = 1
into lcd_init.
into rgb_lcd_init.
into rgb_user_init.
into rgb_lcd_mode_init.
out rgb_lcd_mode_init.
rgb_user_init ok!!!
out rgb_lcd_init.
Start decode...
OK 153601.
out lcd_init.
get_lcd_brightness -- value = 3.
00000525:1.00000535:1.000003C4:1.len is 0x 500000
os_len = 0x 26c3a0. checksum = 0x0b3a8155.
0000243C:1.ret = 1
Run image...

c_main enter------!!
kseg init OK!
new loader, system config ok!
intc init OK!
intc lib OK!

the os is start

UPDATE: as Douglas points out in the comments, the x760+ does charge through USB and does have a power button with functionality similar to the A320 power slider.

Thursday, September 3, 2009

Gemei x760+ is here

Ordered from trustworthy DealExtreme, just arrived. Can't believe it after such an oddisey with MP4nation.

Review, pictures and disassembly tomorrow.

BTW, DealExtreme just added this to their catalog, and added this a few days ago too.

First is from JXD, second is from Benss. No hardware info whatsoever, but with 8GB of internal flash, 4.3" LCD and integrated cámera, they both offer a good bang for the buck. Let's wait and see if details on the hardware emerge in the next weeks...

Tuesday, August 18, 2009

Pyrinees


Lost in Pyrinees for a few days with the family. Trees, cows, bulls, horses and not much broadband. Let's hope the x760+ is waiting at home when I'm back.

Saturday, August 15, 2009

MP4nation still sucking big time (UPDATED)

This is copy of an email I just sent to admin@mp4nation.net. I also opened another ticket.

The x760+ was clearly marked as "on stock" when I placed the order (otherwise I would have placed it somewhere else), but ok, I can accept the x760+ was actually backordered. But customer support is just pulling my leg: every and each time I've opened a ticket they just reply that the item will ship in a couple of days.

This. Sucks. Big. Time. Stay away from MP4nation.

Hope at least I get the money back. To those who donated the money to get dingux running on the x760+, all I can say is that this is out of my control. Sorry.

------------------------

1- I placed order #18466 on jun 25 (almost two months ago).

2- After 20 days I opened ticket #736735, and this was the reply:

"I am sorry for the delay and the inconvenience caused, the item (or items) you had ordered became backordered. However it is now back in stock and your order is to be processed and sent within the next 2 working days."

3- After 30 days I opened ticket #955719 asking for immediate shipment or immediate refund, and this was the reply:

"Your order will be sent by EMS on Friday to you, it should arrive by Tuesday to you. We are sorry for the delay."

So far, the order is still "processing". Every time I contact customer support they reply that my order will ship in a couple of days. At this point, after two months, I just want the money back.

UPDATE: MP4nation has finally refunded the payment. Ordering from dealextreme. I was a bit pissed off by them selling an item not in stock while clearly marked as "in stock", but what is truly inacceptable are lies about restock and immediate shipping. I could have got the money back one month ago.

Thursday, August 13, 2009

New release of dualboot and system

Get it all from here. You are urged to update at least zImage/rootfs to fix the data corruption on SD/MMC write.

Dualboot now supports partitionless cards (those in which filesystem starts at sector 0). Besides that, the only change is that now u-boot cannot be stopped (which was useful anyway only for those with a serial console installed). Noise in the RX line when undriven was causing garbage chars to be read as input and u-boot stopped boot and waited (forever) for further commands. This is most likely relevant only for people like me with a serial console installed, and even if you are one of those, you may have not noticed it, since it only happens if the RX pin is undriven and the USB cable is connected (noise probably comes from the USB circuitry).

If you have already a working system (which means your card is partitioned) and have never experienced spurious lockups during dingux boot, probably you don't need to update your dualboot, but I would advise to do so anyway.

The kernel has some minor fixes (likely unnecessary, just happened to notice them while hunting the miniSD data corruption on write bug) and the miniSD interface configuation set to 1-bit. Consider this a workaround, because it seems to fix the data corruption but kills throughput. You better have a slow interface than one that can kill your filesystem, no matter how low is the probability.

Regarding the rootfs, the changes are:
  • Updated initramfs to allow to specify several alternate "boot=" and"root=" arguments. This is required to support both partitionless and normal cards.
  • Added option "utf8" in initramfs when mounting boot partition, so foreign language characters are properly handled (thanks Rookie1).
  • Changed MMC controller to 1-BIT mode. Clobbers performance but data corruption on writes seems to be fixed. Workaround until the real problem is identified and fixed.
  • Added patch set for timidity in SDL_mixer (/etc/timidity.cfg and /usr/share/midi/instruments/*).
  • Upgraded to busybox-1.14.3.
REMEMBER: ethernet-over-USB and SD/MMC are as of this release suboptimally configured as a workaround for known bugs. I am working on these issues, please be patient.

Developers are encouraged to read README-A320.

And now for something completely different, I'm in touch with the Qi Hardware team. They are a group of former OpenMoko employees now working on both open hardware and software devices. Their first device is also based on an Ingenic SoC and they are making great efforts to improve kernel support. Most important is that they are in touch with Ingenic and are trying to get them to open more documentation and code (yesterday a newer 2.6.27.x kernel was released!). This is very promising and we will be working together.

Thursday, August 6, 2009

Workaround for showstopper card write corruption

I've been lately throwing some serious time on this issue. It is a slow process because each trial-and-error iteration takes about 5 minutes (must write hundreds of MB to get the corruption to show up). I'm checking the SD/MMC code line by line with the aid of the leaked JZ4740 manuals (which are far far far from being clear), comparing the driver kernel code to the uCOS-II code also released by Ingenic (I've got the feeling it is a bit more up to date) and so on. I've fixed the DMA round robin prioriry bug, which was of course not the cause of the corruption, fixed some other apparent bugs in the jz_mmc.c code, and I'm testing everything that comes to my mind, including slowing down the SD/MMC clock. But it is a slow process, please be patient. There's also the possibility that I can have access to further Ingenic documentation (will comment on this in another post) which could help on this issue.

Well, the good news is that one of the things I tried worked: setting the SD/MMC interface in 1-bit mode (instead of 4-bit) seems to fix the corruption. The downside is that the throughput goes down the drain (yeah, you guessed it, one fourth of throughput of the 4-bit mode). I was about to say that this pretty much rules out a DMA related problem, but I'm not sure that's 100% true since lower throughput also means lower stress on the DMA subsystem which could avoid the glitch or corner case causing the corruption. Anyway, the main suspect is still the SD/MMC interface, either the silicon or the A320 hardware. My bet so far is that the code is not properly detecting and handling an error condition that arises during writes.

I'll be preparing today a new release of dual-boot and system packages. Besides the SD/MMC interface in 1-bit mode, it will include some other improvements like support for partitionless cards.

Friday, July 31, 2009

SHOWSTOPPER

I have minor good news: both the network over USB and FTP server work fine.

And here are the bad news: there is a data corruption problem when writing to the miniSD in linux. Some users had reported filesystem corruption but seemed to be a minority and I suspected could be a compatibilidy problem in a certain type/brand of card.

This is a fscking showstopper and has become right now #1 in the priority list. The only workaround as of now is to NOT WRITE AT ALL to the miniSD. I know this means states cannot be saved (brightness, volume, game states, etc) since there is right now no safe place to store them.

I will dedicate all efforts to identify the problem and fix it as soon as possible, but this is likely to take some time since the error is not deterministic nor predictable.

If you want to check it out yourself, run in your A320 the following commands:

cd /boot
dd if=/dev/urandom bs=1M count=100 | tee test1.bin | md5sum
md5sum test1.bin
cp test1.bin test2.bin
md5sum test2.bin

The three calculated MD5 sums should be equal (the first one is calculated from the pseudorandom data as it comes out of /dev/urandom, the second one is calculated from the same data stored in file test1.bin, and the third is caculated from file test2.bin which is a copy of test1.bin.

The corruption seems to happen on writing. Reading is ok, which you can test yourself by calculating the MD5 sum of a large file many times. All will yield the same value, whereas whenever you copy the data to a new file the MD5 sum will change.

NOTE that this is not filesystem related, i.e. happens in the FAT and ext2/ext3 releases.

Timidity patch set

Robert2098 cleverly pointed out that a patch set is necessary for MIDI music in some games (those that use SDL_mixer, actually). Since SDL_mixer is part of the rootfs, I think that the patch set must be included too.

However, when I was just about to use the set found at the SDL_mixer page, I noticed that there is also the freepat project, which is the patch set included in my Ubuntu 9.04 desktop.

So, a question for those knowledgeable in the matter: which patch set should I include? (the one available in the SDL_mixer page is about 18M while freepat is about 33M... smaller is better given the limited amount of memory in the A320).

On a side note, I added support for partitionless cards. Not sure if it's really a good idea to allow two different card layouts. Will go out in the next release (which will include dual-boot and system, since changes in both were required). Bastian linked to a patch for u-boot so I thought it would be relatively easy to fix, but turned out not to work, but at that time I was already into it and ended up with my own working fix. Had to modify also the initramfs embedded in the kernel because it must now try two different devices as boot partition: mmcblk0 (partitionless cards) and mmcblk0p1 (partitioned cards).

Thursday, July 30, 2009

WARNING: FTP server may be broken (UPDATED)

Right after I got ethernet working in PIO mode, I made some transfers using FTP and just noticed that large files get corrupted. I cannot tell for sure that the problem is in FTP but it is the main suspect as transfer of same file using netcat resulted in no corruption.

I'll fix it ASAP, meanwhile, stick to using the original firmware to transfer files.

UPDATE: good news is that network and FTP server are both ok, very bad news is that corruption happens when writing to miniSD. See this post.

Ethernet over USB problem fixed (workaround, actually)

I've been digging into the USB ethernet problem. As I announced, I proceeded by elimination: backported all fixes (which turned out to be minor) to the ethernet gadget from recent kernels in order to see if the problem was there, and it wasn't (as expected anyway, just had to narrow the search). In the process I backported also fixes to the serial gadget, just for fun :-).

So the problem is in the Ingenic USB device code (driver/usb/gadget/jz4740_udc.c). After some serious testing I've found out that it is a DMA related problem. As I've said countless times I'm learning on the go and only know the basics of DMA, so I'll need some serious time to trace down and squash this bug.

Meanwhile, the driver has an option to disable DMA and use PIO (programmed I/O). It's slower (16Mbps versus 40Mbps in my setup) but seems to work consistently.

I've uploaded a new release with just this change, so ethernet is actually usable to transfer files and debug applications. Get it here. The rootfs has not changed, so you just need to copy the appropriate zImage for your LCD type.

Now, on a totally different matter, Bastian has pointed out one possible cause of dingux system installation not working for some people: you need a card with partitions. Some cards come with a filesystem starting right on the first block without a partition table. Your OS of choice deals with it but u-boot (that you flashed in your A320 when you installed dual-boot and who is in charge of loading zImage from the miniSD) seems to get confused if he can't find a partition table.

So, make sure you card is formatted FAT32 and has a partition table.

Wednesday, July 29, 2009

MP4nation sucks

I placed an order for a Gemei x760+ more than a month ago. When the order was placed the x760+ was shown as in stock (otherwise I would have sourced it from somewhere else). I opened a ticked ten days ago and they answered that it would be shipping in a couple of days. My order still shows as "processing" and I've opened another ticket asking for immediate shipment or refund.

I'm sorry for those that donated to purchase the x760+ and get Dingux support for it, but this is way out of my control. If I end up getting and refund, can you suggest a trustworthy place to order?.

Tuesday, July 28, 2009

(Yet another) release of system and toolchain

As usual, get them from here.

Summary of changes for system:
  • Enabled joystick support in SDL but forcing the dummy driver.
  • Minor fix in automatic subdirectory creation in /usr/local.
Summary of changes for toolchain:
  • Properly installed vorbis integer decoding library (Tremor) (hope so, untested).
  • Enabled joystick support in SDL but forcing the dummy driver.
Enabling SDL joystick support allows easier porting of applications that won't compile unless it is enabled (but they'll work perfectly if there is actually no joystick). Forcing the dummy joystick driver code minimizes the "fat" (=unecessary and unused code) added to the library.

Monday, July 27, 2009

Baffling dual-boot bug

There's one dual-boot bug that keeps me utterly baffled. I'm not sure if this is happening to others beside me because it involves the serial console connection. I'll describe it here with the dual purpose of confirming that it happens to others and requesting suggestions:

1- Install dual boot.
2- Plug the USB cable (either connected to a PC or just to the wall charger).
3- Reset the device while holding SELECT pressed.

The SPL dingux splash screen appears but it does not boot dingux, just hangs there. If you perform the same procedure but without the USB cable connected dual boot works file.

So what's so baffling and why didn't I notice this before?

It boots fine if the serial console cable is connected. And that's the cable one needs to see what's going on during the boot process. So if I connect it the problem dissapears, and if I don't connect it I can't see what's going on.

Question is: does this happen too to those that haven't installed the serial console?.

It may be that the SPL code fails to load u-boot, or that the u-boot code fails to load zImage. One could think that maybe the u-boot code is receiving some stray input from the unconnected serial console and stopping to wait for more input, but then, why does this happen only when the USB power is plugged?.

Looks like the only way to investigate this is to modify the SPL and u-boot sources to output debug stuff to screen.

UPDATE 1: just don't know why I didn't think of it before. I just connected the serial console AFTER the boot hangs and confirmed that u-boot has received some garbage input and is sitting there like a stupid waiting for further instructions. I'm not sure how the USB power applied is causing this garbage in the serial input when it's undriven, but that is definitely what's happening. The fix should be dead easy: modify u-boot to completely ignore any input.

UPDATE 2: fixed, should go in the next dual-boot release. Unless people reports this is causing problems, I'll wait until I make some other pending changes into the dual boot to make the release.

Sunday, July 26, 2009

On screen tearing

The final display device, your LCD, works at a fixed frequency which results in a certain refresh rate. Think of it as the screen being "painted" a lot of times per seconds. The "painting" of the screen is an standalone process that is performed by a chip on the LCD itself from its own memory. Think of what would happen if you change the memory contents while the screen is being painted. The only way to do this is to have some kind of syncronization signal from the LCD chip. That way, you wait until the right moment and update the memory contens while they are not being used.

Now think that things are even more complex: actually there are at least three processes and three memory buffers involved: an application normally has an internal buffer where it writes a whole screen before sending it to display. Then the framebuffer driver in the linux kernel has another buffer where contents sent by the application are stored, and finally there is the already mentioned LCD internal buffer. Transfers from one buffer to the next must all be synchronized somehow to the screen painting in order to avoid the tearing effect.

Now for the bad news: I'm yet to confirm it, but the A320 hardware design is... well, let's say it's not the best I've seen. I believe there is no way to read the "repaint" status of the LCD or get a synchronization signal which would allow the linux kernel framebuffer driver to transfer contents to the LCD memory in a way that would not cause some tearing. The LCD controller has registers that expose this information, but they cannot be read because the R/W signal is just not connected to the display. The CPU has a dedicated R/W signal for the LCD but the designers ignored it and just left the LCD stuck in write mode. You can write the registers but you cannot read them. This is also the cause of the impossibility of autodetecting the LCD type: again, both known LCD types have specific registers to identify the model, but we just can't read them !!!. There are some other blatant design mistakes for which they had to implement a workaround, but they are not related to tearing.

It is a shame: LCD synchronization is perfectly possible but they seem to have taken all steps to make it impossible. There might be a very twisted way to somewhat achieve synchronization, but I'm yet to investigate it.

Anyway, you may be wondering: why do some games/emulators show tearing in the original firmware and not in dingux or just the opposite?. It has to do with frequency mixing. The math behind it is far from simple, but it can be summarized this way: when you mix two frequencies you obtain both the sum and the substraction. If the framebuffer driver is refreshing the LCD at 70Hz and the LCD is refreshing the pixels at 50Hz you get both 20Hz and 120Hz. In this case you won't probably see any tearing because both frequencies are high enough, but now think what would happen if an application happens to refresh the framebuffer at 55Hz. You get 5Hz and 105Hz, and you bet the 5Hz are pretty visible.

Now think that as I said earlier, there are at least three processes/buffers involved and that each one works at its own frequency.

We can program the LCD refresh rate. We can set it higher or lower, it doesn't matter. What matters is its relation to the refresh rate of the application that is using the screen at a given time, and that, of course, changes from one application to another.

I'll keep on investigating about the whole LCD subsystem and find a way to mitigate the tearing effect, but bear in mind that with current hardware it might be impossible. Note also that as a last resort the LCD refresh rate could be changed in a per-application basis to minimize the tearing in each case.

System and toolchain updated

As usual, get it from the google code page download section (still uploading at the time of writing this post, so maybe you'll have to wait a bit, my upstream here is tiny).

ChangeLog (for the system):
  • Backported from kernel 2.6.24.6 fix for usb ethernet gadget. Now Windows RNDIS driver works (but ethernet communication is still broken and will fail a few seconds after starting and high bandwidth transfer via FTP).
  • Patched SDL so SDL_NOMOUSE=1 is no longer necessary.
  • Updated to busybox-1.14.2 + current patches.
  • Added symlink /etc/TZ ---> /usr/local/etc/timezone
  • Automatic creation of default subdirectories in /usr/local.
  • Automatic creation of default /usr/local/etc/timezone.
The only differences in the toolchain is that the I changed the owner to root before packing (the first release was owned by my user) and that the included SDL library also no longer needs SDL_NOMOUSE, but since this is only a problem when running in the A320 itself, if your application is dynamically linked you probably don't need to download the new toolchain.

I intended to fix the HOME=/ problem. Actually that default value is set by init, so the only way to change it was changing the init code which I felt was not the right thing to do. So the workaround of using a shell script as /usr/local/sbin/main and setting HOME=/usr/local/home there is still necessary. If you want to save some memory, remember to use "exec" to execute your menu application, because if you just call your executable you'll leave a whole shell sitting in memory and doing nothing but waiting for your process to exit.

Thursday, July 23, 2009

Battery charge/discharge graphs

batman52 took the trouble of sampling the battery voltage both during the discharge and charge cycles. This is what it looks like:


This should help setting the battery status thresholds. It has also helped identifying the GPIO pin used to sense the battery charge status provided by the not-yet-identified power management integrated circuit. Though this information is yet to be verified, I've updated the GPIO mapping tables.

You probably know already that you can read the battery voltage from /proc/jz/battery (in millivols). You can also read the GPIO pin status from gpio[0-3]_pxpin files in the same directory. Do not mess around with the other gpio[0-3]_* files since you could damage your hardware.

Tuesday, July 21, 2009

USB ethernet is broken

As many of you have reported, the USB ethernet gadget device didnot work with Windows. This is due to a kernel bug which was fixed in 2.6.24.6 (thanks to mofive for pointing this out). I've backported the trivial fix and now the A320 is detected as network device and the drivers seem to work with Windows at a first glance.

However, something else is badly broken: if you start a file transfer (from the A320 to the PC to keep the flash write delay out of the measurement) you'll see a high sustained throutput for a few seconds (about 40Mbit/s), and then the link crashes. In linux the link recovers after a delay, in windows the crash is permanent until you unplug and plug again the USB cable.

(note: iperf is available in the rootfs, you can use it to test transfer without using FTP)

The bug may be in the ethernet gadget code itself or in the JZ4740 USB driver code. I've traced the evolution of the ethernet gadget code through all subsequent kernel versions and there seem to be some fixes I could use. I intend to first try to backport these fixes and see it this solves the problem. If it does not, I'll then look to the JZ4740 USB driver code.

I will be releasing a new rootfs/toolchain tomorrow which will include the mentioned USB ethernet gadget fix to work with windows (but remember, it is broken anyway) and a modified SDL library that no longer requires "export SDL_NOMOUSE=1".

Monday, July 20, 2009

System installation pack and toolchain released

Big news. Grab both from the google code download section here.

The installation pack includes the kernel images and a root filesystem in ROMFS format. You just need to place both in the root directory of the first (and most likely only) FAT partition of your miniSD card. No need for ext2/ext3 any longer, which should make Windoze users quite happy.

Another big improvement is console and file transfer. As soon as linux boots your PC will detect a new network card and it will be assigned a network address (by a DHCP server in your A320). Telnet to 10.1.0.2 to get console access, FTP to 10.1.0.2 to transfer files.

Enabling networking in the kernel adds enlarges it noticeably, but this is compensated by the removal of ext2/ext3, which is not longer needed and is quite heavy too.

Dingux will try to run the file "local/sbin/main" in your miniSD on startup, and if it exits it will respawn it. Get your menu application of choice, rename it as "main" and place it in "local/sbin" to get it running on startup.

To system menu application developers, I'd advice to use the exec call to launch other applications. The launched application will replace yours in memory (so the menu won't be sitting silently in the background waiting for the application to end and using precious RAM), and when it exits your menu will be launched again.

I've tried to include as many libraries as possible in the root filesystem. If you need some library not included, link it statically as a temporary solution and email me: I'll try to include it in the next release.

You should notice an important boot speed improvement. The initialization sequence has been optimized in many ways, but the post noticeable is the active monitoring of the miniSD detection. Now the system waits just as long as needed until the miniSD block devices are created, while before there was a fixed delay of one second (the rootdelay kernel parameter was used and the minimum value you can set is one second).

UPDATE: note that existing applications do not work on this new root filesystem. They will have to be recompiled using the new toolchain. I bet developers will catch up quickly.

14528 problems (7 errors, 14521 warnings)

Yeah. Don't know what I'm doning wrong, but I'm leaving java (classpath) out of the rootfs for now. Ezelkov1 kindly provided a patch to get classpatch compiling but it does not seem to work for me, and I just don't have the time to search through 14528 "problems".

Wednesday, July 15, 2009

New A320 arrived

Today I went back home and picked up the package from Hong Kong. It's the A320 I purchased (with your generous donations, thanks againg), and it's got, as expected, the newer ILI9331 LCD controller.

I'll be dissecting it and adding the serial console connector tomorrow. As I did with the first A320, I'll shot a lot of photos and post a report.

For you guys waiting for the gemei x760+ support, I am sorry to report that I've checked with MP4nation and my order is still "processing". Hasn't even been shipped!!!. I just opened a ticket, let's see how is their customer support.

UPDATE: got reply from MP4nation:

I am sorry for the delay and the inconvenience caused, the item (or items) you had ordered became backordered. However it is now back in stock and your order is to be processed and sent within the next 2 working days.

Again we are sorry for the inconvenience caused.


So, assuming they finally ship it in a couple of days, and knowing how snail mail works here, I don't expect to be able to start working on the Gemei x760+ until first or second week of august. Sorry.

Sunday, July 12, 2009

WIP

Work in progress... miscellaneous updates in no particular order:

My neighbour told me that something has (finally!) arrived from Hong Kong. I may be the newer LCD A320 or the Gemei x760+. I'm living in a different place this month (you know: wife and kids, holidays, beach...), and won't be able to pick it up until next week.

I've just fixed the ethernet over usb. Next kernel/rootfs release will communicate only via ethernet: telnet for console (developers) and FTP for file transfer. This is how it works:
  1. A320 boots, sets up the usb0 interface (192.168.110.2) and launches DHCP, telnet and FTP daemons. The default gateway is set to 192.168.110.1.
  2. PC detects a new USB ethernet adapter plugged and gets an address via DHCP. It is assigned 192.168.110.1 by the DHCP server in the A320.
  3. Since the PC is the default gateway, if you configure routing you can get internet access from the A320.
I think I have now a quite complete uclibc based toolchain and rootfs. I'm still working in a FAT-only rootfs, which should be ready for release sometime next week.

I've included all the SDL stuff except SGE, because it doesn't use autotools and the included makefile is not cross-compile ready. I could not compile Allegro either: looks like it's too bound to i386 architectures. If you can port it, I'll be glad to include it in the next kernel/rootfs release.

Wednesday, July 8, 2009

Call for system library requests

I've been recenly using an uclibc toolchain and polishing a minimal rootfs (not the quick hack I released with the first working kernel). This setup won't need an ext2/ext3 partition, just a couple of files you'll have to place in your existing miniSD FAT parition.

This, however, comes at a price: the root filesystem is readonly (romfs... we have plenty of space, no need to lose microseconds decompressing cramfs or squashfs). This means you cannot install anything in the root filesystem, and will have to use the FAT partition. This is OK for almost everything but for system libraries, because they use symbolic links which cannot be created on a FAT filesystem. There are ways to achieve this (comments on this later), but none is simple and certainly not for the first release.

So I intend to release what might become the reference rootfs and development kit, and as such, I would like it to include all the required libraries for applications. So far it includes most of the general purpose libraries that buildroot allows to install plus SDL and derivatives.

I would like to hear from you, devs that are porting apps to dingux: what libraries are your applications using?.

And finally, some right-off-of-the-top-of-my-head ideas on a way to allow addition of system libraries to a romfs based root filesystem:
  • Allow expansion of the rootfs vía filesystem overlays: unionfs (heavyweigth but high functionality) or mini_fo (lightweight but limited functionality).
  • Mount /usr/local/lib as a tmpfs (i.e. ram disk) and on startup create there symbolic links to extra libraries in the FAT partition. Each extra library would have to come with a special file that would define the symbolic links to be created.
Any other ideas?

Tuesday, July 7, 2009

New kernel binaries

I just compiled and uploaded the latest kernel for ILI9325 and ILI9331 LCD controllers (choose yours).

These are the changes:
  • Battery reading implemented.
  • LCD backlight working.
  • Fixed kernel oops when USB cable unplugged.
The later is a long standing bug. It turns out the USB device controller was calling a non-existent suspend function when the cable was unplugged. One hour chase, one second fix.

On a side note, thanks for your valuable comments and suggestions in the previous post.

Sunday, July 5, 2009

Implementing system controls

I will comment how I have though that the system controls could be implemented, as always, in order to get some feedback both from final users and developers.

I think that the most important limitation of the A320 is its 32MB of RAM, and that is why I think we need a"only one foreground application running" approach for dingux. These would be the sequence of events after rootfs is mounted by the kernel:
  1. /sbin/init is executed
  2. /sbin/init executes the initialization scripts, launching any required daemons.
  3. /sbin/init executes the main system menu application in "respawn" mode. At this point we have the daemons, the init process and the main menu system application running.
  4. When the main system menu application wants to launch an external application, it does it by using the exec call. This means that the process is replaced in memory. At this point we have the daemons, the init process and the external application running.
  5. When the external application finishes, the init process notices and respawns the main system menu application.
This way, we avoid having some MB of memory locked by the main system menu while it's not actually running. This is an approach completely different to the used in desktop systems, where the window manager and desktop applications are "always there" retaining control of the system.

This approach poses a problem: who takes care of the system controls (volume, LCD brightness and such) ?. It cannot be the main menu system application because then we would not be able to change settings while running an emulator or other application.

I've come up with an scheme that I think would do the job:
  • Modify the keyboard kernel driver to appear as two different input devices. All the special key combinations (power+whatever) would be generated by the second input device, and there would be a special mode entered by one of these combinations in which normal keycodes would also be sent to the second input device.
  • Modify the framebuffer kernel driver to appear as two different framebuffer devices. The second framebuffer would be shown (only while it is in open) as a traslucent overlay over the first framebuffer. The traslucency would be done in software and thus would use quite a chunk of CPU power, but this will be ok in the usage scenarios I will describe.
  • Make a system daemon process that will be listening to the event queue of the second input device, and will open and use the second framebuffer whenever it wants to show an overlay screen.
Usage cases:
  • The user presses the "volume up" key combination (power+up), the system daemon opens the second framebuffer device and shows a "volume up" splash screen for a short period.
  • The user presses the "menu" key combination (power+select), the system daemon opens the second framebuffer device and shows a menu where you can terminate the current running application (which would be identified by looking at who is using the first framebuffer), use a virtual keyboard to send keystrokes to the current running application (i.e. generated by the first input device) etc.
The rationale of the CPU intensive traslucency not being important is as follows: if you are using special key combinations to change the volume, you won't mind if the currently running application slows down a little for a short time. If you are using the virtual keyboard, the currently running application will be stopped in an input box dialog, so you won't notice the slowdown.

You may be thinking that the system daemon would be an application "stuck in memory" which defeats the initial purpose of saving memory. The system daemon will have to be implemented as very optimized code in terms of memory usage code, but most importantly, it will have only a reduced and limited set of feattures, while the system menu application can grow as necessary, including lots of eyecandy, an audio player (instead of an external application), etc.

Someone is already working in suck a daemon and my mission is to provide the required kernel space functionality. As usual, I kindly request your comments and suggestions. In particular, I'm learning about some kernel subsystems on the go, so if you have more experience in input and/or framebuffer devices, I'll be specially glad to hear you opinion on the feasibility of this whole approach.

UPDATE: I did a quick check and it seems that all input events from all input devices reach the controlling tty (makes sense because if you plug two keyboards to a linux box you can type from any of them), which means having two input devices is not the right way to divert certain input events to different processes. An easy way to achieve similar functionality would be to use a dedicated char device only known to the system control daemon, from which it would get special keystrokes and "kidnapped" normal keystrokes in menu mode. This would also solve an issue I didn't notice when writing this entry: if we implement a virtual keyboard in userspace, how do we "inject" the resulting virtual keystrokes back in the input subsystem?.

Well, I guess I could just use special keycodes for all operations to be handled by the system control daemon and assume that the current running application will just ignore them. However, I don't think "assuming" is good practice in general...

Battery reading and LCD backlight working

Some updates to the kernel (will try to find some time tomorrow to compile and release new binaries):

I just got battery reading working (thanks tor...@pltn.org). You can get the voltage in millivolts by reading from /proc/jz/battery. At the moment, and until we have some charge/discharge voltage curves, we cannot relate voltage level to battery charge status.

I know that most (if not all) of you are using the serial console over the USB cable to access your A320, and since the kernel panics when unpluging the cable, it is not possible to trace the discharge curve unless one builds an special USB cable without the +5V line. So in the next days I'll work on fixing the cable unplug bug, and then ask for help on tracing the battery charge/discharge curves.

I also got the LCD backlight working. The FBIOSETBACKLIGH ioctl command now works, but I also implemented a procfs interface and since it doesn't require kernel headers (from the userspace developer's point of view), I would strongly advice to use it instead of ioctls.

Just read/write a value between 0 and 100 to /proc/jz/lcd_backlight. As a refinement I might add a linearization table (the apparent brightness increase is much larger from 0 to 10 than from 90 to 100).

Finally, a couple of notes about the keyboard mapping:
  1. I replaced KEY_KEYBOARD by KEY_MENU (power+select). This keycode is intended to bring up an overlay screen which might include a virtual keyboard, but which will certainly include more functions (volume, brightness, process control, etc). So makes a little more sense to use MENU than KEYBOARD:
  2. I added the KEY_EXIT keycode (power+left shoulder). It is intended to kill the "foreground" application, that is, the one using the framebuffer.
Note: when I say "intended" I mean that an userspace daemon will be needed that will capture those keycodes, bring up the overlay screen, etc.

Thursday, July 2, 2009

Look ma, net, no wires

batman52 made an interesting comment in the previous post: WiFi.

Indeed, much to my surprise there are miniSDIO WiFi cards on the market. The JZ4740 docs says that the SD/MMC controller supports SDIO, so in theory it should be possible to add WiFi to the A320.

Of course this is a long shot and there are much more important things yet to get working, but just imagine the possibilities... I just have to give it a try. I think it is worth investing some of the donated money in purchasing one of these cards. What do you think?.

Request for comments on file transfer

As some of you have already noticed, all the networking stuff is disabled in my binary kernel image releases. This is because the A320 is not a networked device and as such in principle it doesnot make much sense to waste precious memory. Note that module loading is also disabled for the same reason (static hardware, reduces kernel memory usage). I'll later explain why all this is relevant.

File transfer is not yet enabled in the released kernels. The reasons for this are:
  • The storage device gadget driver (which would be the "standard way") is not tested yet.
  • The storage device gadget driver accesses the storage as raw block devices, and thus requires exclusive access to them. In other words: the filesystem is mounted by the host PC. This is why you can see the ext2/ext3 partition from a linux PC despite the fact that the original firmware does not support it.
  • Having a serial console is a MUST for application developers. This is provided by the serial gadget driver, which is enabled now (and has a hideous bug that crashes the kernel when unplugging the USB cable).
  • You can't use more than one gadget device simultaneously in the 2.6.24.3 kernel. Though I think this is solved in recent kernels by implementing composite gadget devices, backporting it to the 2.6.24.3 kernel is a huge task, and even if that is achieved, the composite device we would need (serial + storage) is not implemented yet even in the latest kernel.
To summarize: we need the serial console and it is not usable together with the storage gadget function. The obvious way to solve this is backporting composite gadget device support from recent kernels and coding a new serial + storage device.

I can think of several ther alternative ways to get what we need:
  1. Using a serial file transfer protocol over the serial gadget. This is rather inconvenient and definitely not for the end user. I know next to nothing about OBEX, but I'm given to understand that it can work also over a serial line. Maybe its usage would be simpler that a classic protocols for the end user.
  2. Module loading/unloading of the serial and storage gadget devices. This requires enabling module loading (which enlarges the kernel), is cumbersome to implement, and probably requires fixing first the above mentioned unplug bug.
  3. Establishing a network connection, either by using the ethernet gadget device or by using SLIP over the serial gadget (the first would be easier). Once the network connection is established you can telnet to get console access and you can transfer files via FTP or SCP. This requires enabling the kernel and user space networking stuff, and would probably be a bit inconvenient for the end user, since it would require to configure the gadget network card.
My guess is that backporting + coding a new composite USB device is the way to go. It's a fairly large task and requires good knowledge of the kernel USB stuff (which I don't have). It would take time, but I guess this is acceptable since right now we have a working way to transfer files.

All your comments and suggestions are welcome.

Wednesday, July 1, 2009

Porting a menu frontend

I got a little time to check out frontend menu systems for the A320. As always, the way to go is reuse existing open source software, and I came up with two candidates: gmenu2x and gp2xmb. If you know some other, please let me know.

You may recall that one of the first quick ports I did when I got linux running on the A320 was gmenu2x. I did that using the huge glibc and other stuff from the Ingenic toolchain, but this time I wanted to do it the right way: using an uclibc toolchain.

I quickly went over OpenEmbedded and settled with buildroot. It's the tool by the uclibc guys and has a nice kernel like configure menu to which I'm used. I succesfully built the toolchain, and I recommend using buildroot for compiling just the toolchain and some basic libraries. For bigger or specialized libraries (like SDL), I recommend configuring and building manually using the just built toolchain. I'll be testing this toolchain in the following days and release a new uclibc based rootfs.

These are the porting results:
  • gmenu2x: unable to get the latest stable version working in linux nor in the A320. The svn code compiled and sort of worked both in linux and in the A320 but despite the icons seems to be loaded they're not shown on screen. Didn't go any further because I do not intend to debug the gmenu2x code.
  • gp2xmb: got the svn code working almos right from the start, both in linux and in the A320. The sound is choppy in both cases, which leads me to believe it's not a problem of the kernel sound driver (reminder: still using the buggy and ugly OSS driver).
Personally I prefer gp2xmb. There's quite a lot of work to be done to adapt to the A320 (plus the sound problem), but the code seems quite clean and well structured. Check out the video:

Note: it should be only the second half, but for some reasong I could not edit it without breaking further the audio sync (in which you can notice it's choppy), sorry. You see how I reset dingux using POWER+START+SELECT, boot into the original firmware, power off, power on and get a bit too late to select dingux thus booting again into original firmware, etc. Notice that in the latest dual-boot release you can press SELECT to boot dingux anytime while the dingux splash screen is show, that is, you actually need not to power on while pressing SELECT.

Sunday, June 28, 2009

Some clarifications

  1. The zImages in the dual-boot installer packages are only for flashing. Do not use them as your main kernel, i.e. do not place them in the miniSD. Get your zImage separately from the downloads section on the google code project page.
  2. If your screen flashes, it means kernel panic. At the moment I have notice of this happening only in three scenarios: when you have done something wrong and the kernel cannot mount the ext2/ext3 root filesystem on the second partition of the miniSD, when it can mount it but there's no /sbin/init (or can't execute it), or when you unplug the USB cable in linux. The later is a bug in the serial USB gadget that is yet to be fixed.

Saturday, June 27, 2009

Yet another dual-boot release: serious spurious bug fixed

You can download it here.

I didn't noticed this bug because as you know I modified my A320 and attached a serial port converter. Today I was playing around with it and noticed the problem.

It turns out that the Ingenic provided function to initialize the UART disables the pull-up resistors of the TX and RX pins. That means that the RX pin is left floating if not connected to an external circuit (which is the case of ALL of you). U-Boot listens to the serial port for commands, and if it detects a break condition or some other input, it stops the boot process and waits for further commands, which never arrive. The fix is as simple as leaving the pull-ups enabled.

I am sorry for not detecting this earlier. Please upgrade your dual boot.

Also, I made a couple of "aestetic" changes as suggested by someone: no more "white screens" while loading. The "white screen" glitch is caused by the LCD initialization code both in the original firmware system loader and in the linux framebuffer driver. Normally you should not see this white screen because the LCD backlight should be off. However the dual boot SPL shows the splash screen and leaves the LCD backlight on.

The original firmware case is solved by switching the backlight off just before jumping into the system loader. The system loader quickly initializes the LCD, so you'll see a black screen only for a split second.

However, in linux I cannot do that because it takes significantly longer for the framebuffer driver to kick in, so it's better to leave the LCD on. But here, as opposed to the original firmware, we have the sources. The solution has been just to always switch the backlight off before performing the LCD initialization in the framebuffer driver.

LCD type autodetection not possible

Quick post: I'm almost 100% sure that LCD type autodetection is not and will never be possible. The /RD signal of the LCD is just not connected, so despite the fact that the LCD has a specific register to identify the model, it cannot be read.

Makes sense: maintaining two or more firmwares is definitely something to avoid, so if the makers of the A320 are doing it, it's because that's the only way.

Please, note that they use a trick to somewhat alleviate the problem: the LCD initialization code is stored in a special place in the NAND flash which is never touched when performing normal firmware updates. However, those of you that have used the unbricking tool have noticed that when restoring from scratch, one has to specify the LCD model by selecting the appropriate .DL file.

So, for the time being, there will be two releases, one for each LCD type. We cannot use the trick that the original firmware does because we cannot use the NAND flash to store settings. However, if (when) linux becomes a firmware replacement, we'll go that way.

Friday, June 26, 2009

New dual-boot release: fixed slowdown in original firmware

As usual, get it here.

I know some people is more than happy with what is actually a side effect of the slower pixel clock: no more screen tearing in certain emulators/games. However, I feel the priority is making dual-boot completely transparent to original firmware. And most importantly, the original firmware is unacceptably slow for some. For those that do need the slow pixel clock "untearing" effect, someone will soon release an "LCD underclocking" application (.app).

Regarding "tearing", it's a visible artifact resulting from frequency mixing. The two frequencies involved are the LCD refresh frequency (set by the LCD pixel clock) and the application screen refresh frequency (commonly called FPS or frames per second). When you multiply two frequencies you obtain two other frequencies: the sum and the substraction. The sum is usually too high to be noticed (65Hz+60Hz = 125Hz), but the substraction is a whole different story (65Hz-60Hz = 5Hz, which is very visible). This is the same effect you can see when you use a camcorder to record an old tube TV.

The only way to get rid of it is retrace syncronization, that is, to force the application to refresh screen at the same rate than the LCD is refreshed. I haven't studied very well how the LCD works and is connected, but I suspect that there is no easy way to synchronize the applications and the LCD. It may not be impossible, but it is certainly going to be difficult.

What you can be sure of is that the original firmware makes no efforts to synchronize, and that is not going to change anytime soon. Our best bet regarding this issue is definitely linux.

Finally one personal note: this has been so fast because I had plenty of time. I've been sick at home for the last three days: I had a very bad muscle contracture on tuesday evening and have had to stay in bed for the last two days. Today I'm better but can't stay sit or standing for very long. Fortunately with help from my wife I found a way to comfortably use my laptop while laying on my stomach. It's quite funny but works. Don't be surprised if you don't see any updates for the next days since I have lots of work (the one that pays the bills) pending.

Gotcha !!! (LCD pixel clock problem fixed)

This is what I've done: I built a program with the SDK that runs on the original firmware and that writes all the registers and stuff to a log file. I booted the original firmware with and without dual boot and compared the log files.

And man, this "bug" is so exotic I have to explain it (warning: your head may explode):

This is a summary of events related to clocks and LCD that happen from reset:
  1. The SPL initializes the PLL and all system clocks.
  2. The system loader initializes the LCD and shows the Dingoo Digital splash screen.
  3. Either the system loader or the ccpmp.bin program that it loads initialize again the system clocks.
The initialization of the system clocks has one funny detail: you program the PLL for a certain frequency (say 336MHz) and you configure it to output either that frequency or half that frequency. The different peripheals (including the LCD) generate their clocks by dividing the PLL output.

This is what happens in the original firmware:
  1. The SPL code initializes system clocks: PLL 336MHz, PLL output 336/2=168MHz.
  2. The system loader initializes the LCD clock generator. It programs the divider to obtain 16.8MHz. It accounts for the "divide by two" configuration and thus the required divider for the LCD is 10 (168MHz / 10 = 16.8MHz).
  3. The system loader (or ccpmp.bin) initializes again the system clocks. As the configuration of the system clocks is the same set in step (1), nothing changes, in particular the LCD clock.
This is what happens with dual boot:
  1. The dual boot SPL initializes system clocks: PLL 336MHz, PLL output 336MHz (not divided by two !!!).
  2. The system loader initializes the LCD clock generator. It programs the divider to obtain 16.8MHz. As the PLL output is not divided by two, the required divider for the LCD is 21 (336MHz / 21 = 16MHz, which is the best possible approximation to 16.8MHz).
  3. The system loader (or ccpmp.bin) initializes again the system clocks, but now, something changes: the PLL output is programmed to be divided by two. As the LCD clock divider remains unchanged, the net result is that the LCD clock is divided by two, resulting in 8MHz.
As you see, this is actually a bug in the original firmware !!!. Either you must initialize the LCD after the system clocks or you must recalculate the LCD clock after a change in the system clocks configuration.

IMPORTANT: something similar was happening with the USB clock. That might explain the effect of plugging the USB cable that some users notices, though I can't think of what is actually happening. Also, I can't explain why some users hardly notice the slowdown while others say it's unbearable.

Not sure if this was the only thing that needed to be fixed. I'm still investigating in the GPIO differences I noticed.

I have already fixed it and plan to repackage it all again and make another release. However, there's an interesting dilemma here: some users are extremely happy because the reduced LCD pixel clock seems to fix tearing in some games/emulators. I all for fixing it because:
  • I don't have time to play. I'm too busy coding :-)
  • The whole idea is to leave the original firmware unchanged.
  • It is not ok for some users. We need something that works for everybody.
There is an easy solution: just as there is an application to overclock the A320, it is very easy to program an application to change the LCD pixel clock. I'm a bit swamped so I beg your pardon but I won't program such application. If someone wants to, here's the code you need (from this file):

/* Timing setting */
__cpm_stop_lcd();

pclk = 16800000; /* Pixclk */

pll_div = (REG_CPM_CPCCR & CPM_CPCCR_PCS); /* clock source,0:pllout/2 1: pllout */
pll_div = pll_div ? 1 : 2 ;

val = (__cpm_get_pllout() / pll_div) / pclk;
val--;
if (val > 0x1ff)
val = 0x1ff;

__cpm_set_pixdiv(val);

REG_CPM_CPCCR |= CPM_CPCCR_CE ; /* Update divide */

__cpm_start_lcd();

Another suspect: DMA starvation (updated: not really)

I've been thinking about the slowness problem and did some tests. I have another suspect: DMA starvation.

I think that as I already mentioned in the previous post, the original firmware doesn't reconfigure the LCD stuff if it is already configured, which means that the LCD stays configured as I do in the dual bool SPL code.

And in the SPL code, I set the pixel clock to 16MHz. It is quite high and has the advantage of a high refresh rate which is why some display artifacts in the emulators seem to be gone.

The downside is that it consumes a large amount of DMA bandwidth and the rest of functions that need DMA, like NAND flash / miniSD access, get starved.

I've verified that in linux when I set the LCD pixel clock to 24MHz the LCD works fine buy the kernel can't read from the miniSD. I suppose something similar is happening in the original firmware. The guy that ported linux to the onda vx747 also mentioned this problem (DMA starvation) and he implemented DMA round robin priority, but I'm not sure if that's the way to go because I'm sure we don't want the LCD display to be affected by disk access (either NAND of miniSD). He also implemented some optimizations in the framebuffer driver that I still have lo give a look at.

I have two ways of fixing the problem:
  1. Deinitialize the LCD just before jumping to the original firmware system loader. I don't know how to do this in an effective way because I don't know exactly how the original firmware detects that the LCD has already been initialized.
  2. Find out the LCD pixel clock that the original firmware is using and use it in the SPL. This is done by programming an application to run on the original firmware that reads the configuration registers and saves them to a log file.
I'll give a try to both approaches.

UPDATE: I've read all the internal registers both with and without dual boot. As suspected, the LCD pixel clock is different... but it is actually lower with dual boot !!!. That trashes my DMA starvation theory. There are some other differences (in particular some GPIO registers) that I'm investigating. Stay tuned.

Dual boot pixclk = 8 MHz
Original FW pixclk = 16.8 MHz

Uninstall capability added to dual-boot installer

As before, get it from the google code project page here.

This is a quick release for those experiencing problems. Doing a full firmware restore takes time and will erase your files, so I though it would be relatively easy to add uninstall capability to the dual-boot installer. The instructions to boot the installer are the same, but the installation menu now allows you to choose between flashing dual-boot or original firmware.

IMPORTANT:
  • The problem of original firmware running slow is not fixed.
  • The restoration is done by flashing the original 8K of SPL code. But this is the SPL taken from my A320 after restoration of original firmware v1.1. I'm almost sure this code is the same for all A320s out there, but obviously I cannot test it. Please report success (or failure).
Regarding the slowness problem, I can't reproduce it in my A320. However, I have an hypotesis: people have reported that despite the slowness of the original firmware, emulators run ok and LCD tearing and other artifacts are less evident or have vanished. The dual boot SPL initializes the LCD and shows a splash screen. The original SPL does not do it and maybe the original firmware detects that the LCD is already initialized and doesn't do it again, leaving the initial configuration, in particular the pixel clock. I used a 16MHz pixel clock which results in a high refresh rate which could be alleviating LCD artifacts during emulation, but for some reason the menu application chokes on that.

I'm going to send modified versions of the dual-boot SPL to people that has reported the slowness problem. If the LCD initialization is the problem, I can skip it when booting into the original firmware.

Please be patient, keep in mind that I cannot reproduce the problem in my A320 and that makes things difficult. The newer A320 with the ILI9331 LCD is on its way from DX, but I bet it won't be here before a week or so.

Thursday, June 25, 2009

Slow???

Some people have reported that the original firmware (at least the menu) runs slower. I would like to get as much information as possible about this problem, so if you have noticed something, send me an email ("About Me" ---> "View my complete profile") with a description as complete as possible: description of the problem, firmware version, everything in the "About" screen and the hidden system info screen, etc. Even a video showing the problem would help.

If you want to remove the dual-boot, you can do it by doing a full firmware restore. Google for the unbricking tools. Remember that you'll lose all your stored files, so backup them before restoring.

I just can't think of a feasible reason for the slowness: the dual boot SPL mimics the hardware initialization of the original firmware up to the last operation, most notably the CPU and peripheal clocks. But I might have missed something...

So, please report and document as completely as possible the problem and I'll look in to it as time permits.

Dual boot installer released

Get it from the google code project download section here.

The README file explains it all. You can flash dual boot in your A320 both from Windows and Linux. Note that the installer will give you the ability to select from original firmware on internal flash or linux on miniSD. The later means the dual boot code will look for a zImage kernel file in the first partition of the miniSD and launch it. That's all.

IMPORTANT: I'm almost 100% sure the ILI9331 variant works fine, but could not test it since the newer A320 (paid by your generous donations) hasn't arrived yet. Postal service sucks big time here. If you have any troubles, please let me know.

Some notes for rootfs developers:
  • The flashed U-Boot environment can't be changed. It just loads a zImage from miniSD and that's all.
  • If you want to pass special command line parameters to the kernel, you must embed them in the zImage (see CONFIG_CMDLINE). That means you need to recompile the kernel.
  • If you want an initramfs, you'll also have to embed it into the kernel (see CONFIG_INITRAMFS_SOURCE).
I guess that U-Boot environment could be read from miniSD, but I don't see the point at the moment and didn't bother to implement. Will do if the need arises.

This is how the installer works: the instructions let you boot a zImage. It's just the kernel with three particularities:
  • The console font size is set to 8x16 (instead of the tiny one).
  • The NAND flash support is enabled and forced to 2K page size (required to properly write the SPL area which is the first eraseblock).
  • There is an embedded initramfs which contains libc, busybox, mtd-tools, dialog, and a script that executes on startup, shows a disclaimer and does the job.
Using Linux to do the actual flashing has the advantage of having a single installer. The instructions for Windows and Linux differ because they use different tools to boot the kernel with embedded initramfs, but once the kernel is running the flashing (or any other task that I might perform) is independent of the PC operating system.

UPDATE 1: yes, the logo is shown even if you are booting into the original firmware. I though it was nice to show it as sort of saying "this A320 is modded to boot linux". However, it can be easily changed if enough people prefer it not shown.

UPDATE 2: please please please read the keyboard related section of this document. It explains the current key map and special key combinations in linux, in particular how to reboot the machine. I've noticed that the immediate reboot key combination (POWER+START+SELECT) is a bit inconvenient if you are rebootint into the original firmware, because you must be quick releasing SELECT or you boot again into linux. This wasn't a problem when boot selection was being done by U-Boot because it takes a little time to load, but now selection is done right in the SPL, in fact the SELECT key state reading is the first thing done. I'll be glad to hear your comments and suggestions on this issue.

UPDATE 3: some clarification: the dual-boot installer lets you boot linux withou having to use a PC. But you had to have linux installed already in your miniSD as described in the QuickStart guide in the wiki section of the google code project page. I guess it's been a bit confusing that I've released a kernel image at the same time than the dual boot installer. You still need to install the root filesystem in a second ext2/ext3 partition.

UPDATE 4: yes, formatting and installing the root filesystem from windows is not yet solved, but it is not difficult and someone (maybe me) will do it soon. All we need is to place the rootfs file in the FAT partition and embed a initramfs into the zImage that will first mount the FAT partition and then the rootfs file as a loopback device.

Wednesday, June 24, 2009

Names, names, names

Thank you all for your suggestions. Some thoughts, in no particular order:

  • Someone suggested "booboo linux". While I feel praised, I don't think it's fair. Yes, I've put quite a bunch of work on this, but remember that I did all on top of the linux code base and the Ingenic patches. I you think of it, my contribution is comparatively tiny. I'm standing on the shoulders of giants.
  • Not any name will do. The domain must be available. I've put the name in the dual-boot splash screen and I think it's a great way to promote the project, but for that to be effective it must lead to a web page. I love "lingoo", but is not available. Chances of a short name like "dix", being available are also close to none (and yes, being a non native english speaker it was not immediately obvious to me what it sounds like).
  • What about "dingux" ?. The domain is available.

Again, your comments are highly welcome.

UPDATE: changed to DINGUX. Some don't like how it sounds, but seemed a good blend of dingoo and linux and I want to focus on coding. The domain is now www.dingux.com, please update your links.

Tuesday, June 23, 2009

Quick update

Sorry for the slow updates. I've had to deal with lots of unexpected trouble during testing of dual-boot, plus school finished recently and thus my two daughters require more attention.

I appreciate your support very much and I know you're all holding your breath for the dual-boot functionality, so, please excuse me for the delay.

This is what's been done:
  • Dual boot code has been moved from U-Boot to the SPL: removes tiny delay introduced when booting the original firmware.
  • LCD support implemented both in SPL and U-Boot (both ILI9325 and ILI9331). The LCD now goes live immediately on bootup, which is great when loading linux because otherwise you would see a black LCD for a couple of seconds until the kernel framebuffer driver kicks in, and it is a bit confusing.
  • Implemented access to the SPL area (first eraseblock of NAND) in the linux kernel. This allows flashing the dual-boot binaries from linux.
  • Streamlined the flashing of the dual-boot binaries from linux.
This is what needs to be done:
  • Add a simple dialog that shows a disclaimer and asks for confirmation before flashing the dual-boot binaries.
That is very easy and I just need a couple of hours to do it, so I can assure there will be a release tomorrow or the day past tomorrow.

One final note: someone posted a comment pointing out that lingoox might sound a bit offensive in some contexts. If this is a problem we're on time to change it... so please let me know.

Tuesday, June 16, 2009

You made it! (again)... ordering a Gemei x760+

Thank you all your support, encouragement, and donations.

There's now enough donated money so I'm ordering a Gemei x760+ and will start working on it as soon as I get my hands on it.

My A320 so far has given me maybe a hundred hours of fun, but isn't it weird only two of them were actually listening to music, playing videos or playing games?.

Sunday, June 14, 2009

Dual boot video

Still preparing the dual boot package release. Meanwhile, here's a short video of dual boot working. The final version lits the LCD as soon as you switch it on, but shows nothing on it until the kernel framebuffer driver takes control. It's not nice but is better than having a dark screen (which felt like something was not working).

One note on boot time: the current boot process involves u-boot and a one second delay before mounting the root filesystem. u-boot is necessary because we are booting from miniSD and the delay is needed to let the kernel hotplug system detect the miniSD. Eventually, when linux replaces the original firmware (don't hold your breath), those delays will dissapear.

UPDATE: yes, I said I'd be releasing binaries this weeked, but though I've dedicated quite a lot of hours, there were many issues to address. Please be patient.

Keyboard driver rewritten

Dual boot required a working way to reboot linux and get back to the original firmware, so I set out to implement that functionality into the keyboard driver, and noticed I should be using the input-polldev code rather than the original approach, so I ended up rewriting the driver (and learning in the process quite a lot about the linux input layer).

The normal keyboard map hasn't changed (mimics the GP2X), but now there are special combinations that use POWER and START. I've tried to come up with a clever set of combinations, but I'm always thinking I might be missing something important, so I kindly ask for your comments and suggestions. Please have a look at the keyboard section of the README-A320 file (you can also checkout the kernel code, compile and test it, at least until I release a new kernel binary).

Wednesday, June 10, 2009

Dual boot working !!!

Just a quick post to tell that dual boot is working now.

I'm not making a release right now, but I will most likely this weekend. The reason is that this is something people will be actually flashing in their consoles, so before making a release, I need to:
  • Test some corner cases.
  • Clean up u-boot code and commit it to google code.
  • Write detailed instructions for end users.
For the tech-oriented, this is how it works:
  • Only the first NAND block is modified. NAND is treated as if it was 2KB page size (it is actually 4K), because that is how the ROM IPL treats it.
  • u-boot SPL is placed in pages 0-3 (0x00000000-0x00001FFF).
  • u-boot is placed in pages 32-127 (0x00010000-0x0003FFFF).
  • u-boot has been modified to support A320 hardware and to understand the non-standard way in which error correction data is stored in the area where the original firmware system loader lives (pages 128 and up).
  • If SELECT key is not pressed, u-boot loads the original firmware system loader from NAND offset 0x40000, size 0x80000, into DRAM address 0x80E00000. Then jumps to 0x80E10004.
  • If SELECT key is pressed, u-boot loads linux from the miniSD.

UPDATE: for those that just joined, and answering user comments:
  • Yes, this means you can boot and use Linux on your A320 without a PC.
  • Yes, you still need some menu application. I don't provide that since I have limited time and want to focus on the kernel, but someone will (there are plenty of very skilled programmers turning their attention to the A320).
  • Hardware support is still limited to video, sound and keyboard, that is, just what you need to play emulators (can you spell mame?). Now that dual boot is working I'll focus on improving hardware support.
  • Lack of full hardware support means linux won't replace the original firmware anytime soon (but will eventually, that's my goal). So we have to get along with the original firmware and that means linux must run from the miniSD. Altering the internat NAND flash was necessary though in order to tap into the boot process and launch linux from the miniSD.
  • Work on the x760+ will start as soon as I get my hands on it. There's almost enough donated money (thank you very much again for you support) as to purchase both a newer LCD A320 and a x760+. I'm still trying to purchase the newer LCD A320 from someone who can guarantee I'll get one of the newer models, and regarding the x760+... can you suggest where to buy it?.

Monday, June 8, 2009

Dual boot documentation updated

Added what I found out so far from disassembly of the original firmware SPL, and some more thoughts on how to implement dual boot. Check it out here.

UPDATE 1: I hit an unexpected roadblock. I had u-boot working already on the A320, but the OOB format in the NAND area where the original firmware system loader is stored is rather unusual, if not completely nonstandard. That means I must study the SPL code a bit more to understand the format and modify u-boot to handle it (u-boot decides the OOB format on his own depending on the flash chip type).

UPDATE 2: main obstacle overcome. Now I know how ECC data is stored in the OOB area where the system loader is stored. Since it is non-standard I still have to modify u-boot to handle it. It's going to take a bit longer because u-boot is a bit "too smart" and autodetects NAND type, which means I have to either override it or implement special commands to manually force the NAND configuration.

UPDATE 3: done. Dual boot is working. I just need to clean up u-boot code a bit and prepare a binary and easy install instructions for end users.

New kernel binaries

I've commited some stuff to the repository:
  • Added README-A320.
  • Added support for ILI9331 LCD controller found in newer A320.
  • Make IPU memory reservation optional (and default disabled). Saves 4MB of memory and anyway it's too soon to start working on mplayer for dingoo-linux. If you really need it, just enable it (see README-A320).
  • Default enable CPU frequency scaling. JUST ENABLE IT, untested and anyway you can only REDUCE the CPU speed. Will start working on make this usable and be able to squeeze until the last MHz out of the A320.
And also have placed a couple of kernel binaries in google code downloads section. For now, you must choose the one suitable for your LCD type. When I get my hands on one of the newer A320 models I'll try to implement autodetection.

Still using OSS. It's buggy and uses a crazy amount of memory, but so far it works if you stick to the S16 sample format.

I'm making some progress on dual boot. At least it's now clear to me the only way to go is to completely reverse engineer the original firmware SPL and patch it to load u-boot from a different NAND location if SELECT is pressed. The approach of chainloading the original SPL has proven to ve a dead-end so far.

UPDATE: after too much lost sleep, I've finished disasemblying and understanding how the original firmware SPL works and I think I know now where and how to place things to properly tap into the original firmware boot process. Since I got u-boot working weeks ago, we should have dual boot working shortly.

Friday, June 5, 2009

Newer A320 with ILI9331 LCD controller working !!!

Looks like the first report was a false alarm. Later reports say it works. Not too bad for a code released after zero testing...

So that's it. Linux works for users of newer A320s. I won't cancel the purchase of one of the newer A320 units though (thanks again for donations!), since I still need it for testing the framebuffer driver optimization I'm working on. Next machine to acquire is a Gemei x760+.

Please, if you make a donation, tell me what system you own or are interested in being supported, so I can prioritize use of the donated money.

NOTE: if you own one of the newer A320 with the ILI9331 LCD controller, please contact me. I might need to get someone to perform quick tests of kernels to enable LCD model selection/detection, at least until I get myself one newer A320.

Wednesday, June 3, 2009

Back to ALSA

Ingenic seems to provide both OSS and ALSA support in their kernel. I went OSS (which is deprecated) instead of ALSA just because I could get no sound output with the later while the former worked out of the box. As a side effect, OSS is lightweight compared to ALSA, which is an advantage in systems with limited RAM, as is the case of the A320.

I've been trying to fix some issues in the OSS driver and today I gave up. The code is the biggest POS I've seen in my whole life. I would write it from scratch if I had the JZ4740 manual, but for some reason beyond me Ingenic hasn't released it, so instead of learing how the hardware works and then write the code I have to understand how the hardware works from studying the code. No way with such a POS.

So I'll start working in getting sound output from ALSA, whose code seems to have an acceptable quality. It's certainly a pity losing the lightweight advantage of OSS. The libasound library that comes with the Ingenic toolchain is about 1MB size (stripped!, for god's sake, what's in there?). I'm not knowledgeable on ALSA internals, so any suggestions on how to get it thinner would be welcome.

UPDATE: confirmed... something is REALLY wrong with the Ingenic OSS driver. It takes nothing short of 3.5MB of memory !!!. Makes no sense (and the used buffers are much smaller).

Testing support for newer A320 with ILI9331 LCD controller

I finished disassemblying the .DL hardware initialization code of the newer A320 with ILI9331 LCD controller and modified the code in the kernel framebuffer driver. Since I don't have one of the newer A320 (soon to be fixed, thanks again for your donations) there's no way I can test it, so I need help here. Download the zImage (mediafire, I don't want to upload untested stuff to google code), try it and let me know if the framebuffer works.

If it works, I'll do some minor modifications in the kernel code to allow selecction of LCD initialization sequence and commit it to the subversion repository on google code. I'll be releasing two kernel images until I get one of the newer A320 units so I can work on autodetecting the LCD type.

UPDATE 1 (WRONG): at least one user has reported that the new driver doesn't work. It might be a silly detail but I'm blind and there's absolutely no way I can go any further until I get one of the newer A320. There's enough money in the donation pool (thanks again!) so I'll be ordering it in short (I'm trying to contact CraigX to see if he can make sure of shipping a newer model if I purchase it from him). Please be patient.

UPDATE 2: looks like that was a false alarm. It works. Read this entry.