Syllable Forum Index Syllable
Syllable Forums
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

(eeepc) usb gets SD ready AFTER boot-shell gives up

 
Post new topic   Reply to topic    Syllable Forum Index -> Bugs
View previous topic :: View next topic  
Author Message
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Fri Oct 16, 2009 4:45 am    Post subject: (eeepc) usb gets SD ready AFTER boot-shell gives up Reply with quote

I have a device on '/dev/disk/scsi/hda/0' (or, at least, i THINK it is, reasoning to follow,) which is an 8GB SD card. I understand that USB booting is somewhat under fire recently, (or particularly installing to USB,) but I installed to the USB device using Linux:

[code:1]qemu -hda /dev/sdf -cdrom SyllableDesktop-0.6.7-20091011.i586.iso -boot d -m 384[/code:1]

Which worked flawlessly. Then I synced, put the SD in my EeePC, and grub started up as usual. At having boot issues, the first thing I did was try booting it with Qemu, and it worked perfectly, so I went back to fiddling with my Eee.

I had to enable ACPI and disable the EHCI module in Grub, making my menu display:

[code:1]kernel /system/kernel.so root=/dev/disk/scsi/hda/0
module /system/config/kernel.cfg
module /system/drivers/dev/bus/acpi
module /system/drivers/dev/bus/pci
module /system/drivers/dev/bus/usb
module /system/drivers/dev/bus/scsi
module /system/drivers/dev/hcd/usb_ohci
module /system/drivers/dev/hcd/usb_uhci
module /system/drivers/dev/disk/usb
module /system/drivers/fs/afs[/code:1]

I've tried variations, such as specifying the 'root (0,0)', which has the same result, and disabling ACPI, SMP, and both. The USB doesn't appear to initialise without ACPI enabled.

By reading the kernel_init function in init.c I noticed the option of specifying root=@boot, so I tried that, but it just scrolls one line over and over (since it's in a 'while' loop..):

[code:1]0:init::init(-1) : Checking under /dev/disk/scsi[/code:1]

(likely since it isn't a CD, which that function requires it to be..)

But I noticed an 'init_devices' call being done AFTER attempting to mount the boot disk in /syllable/system/sys/kernel/kernel/init.c at line 460:

[code:1] 433 init_devices_boot(); /* Initialize devices manager */
434 init_elf_loader();
435
436 init_block_cache();
437
438 printk( "Mount boot FS: %s %s %s\n", g_zBootDev, g_zBootFS, g_zBootFSArgs );
439 mkdir( "/boot", 0 );
440
441 if ( strcmp( g_zBootDev, "@boot" ) == 0 )
442 {
443 if ( find_boot_dev() < 0 )
444 {
445 printk( "Unable to find boot device\n" );
446 }
447 else
448 printk( "Found boot device\n" );
449 }
450 else if ( sys_mount( g_zBootDev, "/boot", g_zBootFS, 0, g_zBootFSArgs ) < 0 )
451 {
452 printk( "Error: failed to mount boot file system.\n" );
453 }
454
455 g_bRootFSMounted = true;
456 protect_dos_mem();
457
458 sti();
459
460 init_devices(); /* Load busmanagers */[/code:1]

Unfortunately I don't know the specific model of my EeePC other that "2G Surf", the first ones that sold in north america when the line was introduced.

Now that I have the printk_max flag in my arsenal, I'll try to tune in on the output from just before the boot-shell failure and prepend that by editing this post.

The output is (i'm typing this while looking at the screen, so please excuse formatting deviation or typos). Also, I'm writing these in reverse (latest line printed, then the one above, then above that, etcetera), since I'm stepping back with printk_max.

... lots of PCI IRQ transforms, PCI interrupt link enablings, etc., if this is important, please say so and I'll go back further and type them out verbatim....
[code:1]0:init::init(-1) : PCI: Device 0x8086:0x2659 IRQ transform 7 -> 7
0:init::init(-1) : PCI: Device 0x8086:0x265a IRQ transform 10 -> 10
0:init::init(-1) : PCI: Device 0x8086:0x265b IRQ transform 5 -> 5
0:init::init(-1) : PCI: Device 0x8086:0x265c IRQ transform 3 -> 3
0:init::init(-1) : PCI: Device 0x8086:0x2448 has no interrupt pin
0:init::init(-1) : PCI: Device 0x8086:0x2641 has no interrupt pin
0:init::init(-1) : PCI: Device 0x8086:0x2653 IRQ transform 0 -> 7
0:init::init(-1) : PCI: Device 0x8086:0x266a IRQ transform 0 -> 7
0:init::init(-1) : USB: Busmanager registered
0:init::init(-1) : USB: USB HUB driver registered
0:init::init(-1) : SCSI: Busmanager initialized
0:init::init(-1) : No USB OHCI controller found
0:init::init(-1) : Disabling device /boot/system/drivers/dev/hcd/usb_ohci on bus pci
0:init::init(-1) : Error: device /boot/system/drivers/dev/hcd/usb_ohci failed to initialize itself
0:init::init(-1) : USB UHCI at I/O 0xe400
0:init::init(-1) : USB: Bus 0 registered
0:init::init(-1) : IRQ 3 enabled
0:init::init(-1) : USB: New device strings: Mfr=0, Product=2, SerialNumber=1
0:init::init(-1) : Product: USB UHCI Root Hub
0:init::init(-1) : SerialNumber: e400
0:init::init(-1) : 2 ports detected
0:init::init(-1) : USB: Device claimed by USB HUB driver[/code:1]
.. it finds 4 in total, taking IRQ 3, 7, 10, and 5, at 0xe400, 0xe480, 0xe800, and 0xe880..
... so instead of typing it all out, take my word for it. I can come back to this if needed by setting my printk_max to 285....
[code:1]0:init::init(-1) : 2 ports detected
0:init::init(-1) : USB: Device claimed by USB HUB driver
0:init::init(-1) : USB UHCI at I/O 0xe880, IRQ 5
0:init::init(-1) : USB: Bus 3 registered
0:init::init(-1) : IRQ 5 enabled
0:init::init(-1) : USB: New device strings: Mfr=0, Product=2, SerialNumber=1
0:init::init(-1) : Product: USB UHCI Root Hub
0:init::init(-1) : SerialNumber: e880
0:init::init(-1) : 2 ports detected
0:init::init(-1) : USB: Device claimed by USB HUB driver
0:init::init(-1) : USB: USB disk driver registered
0:init::init(-1) : USB disk driver loaded
0:init::init(-1) : Mount boot FS: /dev/disk/scsi/hda/0 afs
0:init::init(-1) : initialize_fs called in afs
0:init::init(-1) : Mount afs on '/dev/disk/scsi/hda/0' flags 00000000
0:init::init(-1) : Error: afs_mount() failed to open device /dev/disk/scsi/hda/0
0:init::init(-1) : Error: failed to mount boot file system.
0:init::init(-1) : Error: iterate_directory() failed to open directory /dev/bus (-2)
0:init::init(-1) : Error: iterate_directory() failed to open directory /dev/hcd (-2)
0:init::init(-1) : Volume 5 (0x00161820) added
0:init::init(-1) : Error: iterate_directory() failed to open directory /boot/system/drivers/net/if (-2)
0:init::init(-1) : Start init host
0:boot-shell::boot-shell(-1) : Starting init...
0:boot-shell::boot-shell(-1) : Failed to load boot-shell
0:kernel::usb_hub_thread : USB New device strings: Mfr=1, Product=2, SerialNumber=4
0:kernel::usb_hub_thread : Manufacturer: ENE
0:kernel::usb_hub_thread : Product: UB6225
0:kernel::usb_hub_thread : SerialNumber: 146030377350
0:kernel::usb_hub_thread : USB disk connected
0:kernel::usb_hub_thread : ENE UB6225 detected
0:kernel::usb_hub_thread : Transport: Bulk
0:kernel::usb_hub_thread : Protocol: SCSI
0:kernel::usb_hub_thread : SCSI: Scanning for devices on host USB disk
0:kernel::usb_hub_thread : Scanning device 0:0:0
0:kernel::usb_hub_thread : SCSI: Vendor: USB2.0
0:kernel::usb_hub_thread : SCSI: Model: CardReader SD0
0:kernel::usb_hub_thread : SCSI: Revision: 0100
0:kernel::usb_hub_thread : SCSI: Type: Direct-Access
0:kernel::usb_hub_thread : SCSI: ANSI SCSI revision: 02
0:kernel::usb_hub_thread : SCSI: Removable device
0:kernel::usb_hub_thread : Decode partition table for hda
0:kernel::usb_hub_thread : Partition 0 : 32256 -> 8192378879 2a (8192346624)
0:kernel::usb_hub_thread : Scanning device 0:0:1
0:kernel::usb_hub_thread : USB: Device claimed by USB disk driver[/code:1]
Back to top
View user's profile Send private message
Vanders
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 849

PostPosted: Fri Oct 16, 2009 6:21 am    Post subject: Reply with quote

Interesting & thorough analysis, thanks. I'll look into the init.c issue (there are a couple of init() functions, including init_device_boot(), so we may need to fiddle around there). I'll see if we can try it for the next build.
Back to top
View user's profile Send private message Send e-mail
Kaj
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 1987
Location: Friesland

PostPosted: Fri Oct 16, 2009 10:09 am    Post subject: Reply with quote

Wow, ecellent report! I just ran into the root=@boot limitation, too. It would be good if that wouldn't be limited to CD drives.
Back to top
View user's profile Send private message Visit poster's website
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Mon Oct 19, 2009 5:47 am    Post subject: Reply with quote

I'm trying out a build with a minor change to the root=@boot routine (find_boot_dev, which doesn't really find the boot device at all)

In fact, I suggest renaming this stanza to @find, @auto, or @search, instead of @boot.

Despite this 'helping' the USB SD card to boot, it doesn't address the primary issue of boot failing prior to the USB device registering.

Probably the best way to do this would be forcing init to wait for the USB bus to complete scanning, if one exists at all.

I can only assume by the point of failure any USB roots are registered, so the next question (since I know nothing about how USB registration works), is there a way to induce a scan of the root hub(s) for devices, and is there a static point where the USB stack can state that all devices currently attached to the bus are identified, activated, initialised, and ready for use (if compatible)?

Can you probe for devices on a USB bus similar to IDE probing? *fingers crossed*

As a stopgap solution, I'm trying a similar 'snooze' while(false) loop in init.c around line 455 (sys_mount call) to keep trying to mount the specified device.

With any luck, the USB thread will wake up in time for it to work. I'll keep you apprised and provide a working patch.
Back to top
View user's profile Send private message
Kaj
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 1987
Location: Friesland

PostPosted: Mon Oct 19, 2009 6:12 am    Post subject: Reply with quote

Great work! root=@search sounds good to me. Vanders can probably tell you more about USB behaviour. Eventually, we need a kernel messaging system to handle such time dependencies.
Back to top
View user's profile Send private message Visit poster's website
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Mon Oct 19, 2009 6:30 am    Post subject: Reply with quote

[quote]Eventually, we need a kernel messaging system to handle such time dependencies.[/quote]

I was thinking the same thing, but this is a whole new can of worms. How would such a system be implemented in an extensible fashion, for example?
Back to top
View user's profile Send private message
Vanders
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 849

PostPosted: Mon Oct 19, 2009 8:40 am    Post subject: Reply with quote

[quote="doneill"]I was thinking the same thing, but this is a whole new can of worms. How would such a system be implemented in an extensible fashion, for example?[/quote]

That's fairly simple: we'll be using the same "packed" Message objects that are already used by libsyllable (but re-written in C, not C++!), so you can attach and detach data in a free-form manner. It's something I definitely want to get done before we release 0.6.7, and I'll probably be doing it right after I've finished what I'm working on now.

Your loop in sys_mount() sounds a lot like the MNTF_SLOW flag that I added for old CD-ROM drives, but then removed a little while back as it was no longer needed! If you check the CVS history you should see the commit when I removed it.

Have you tried re-arranging the init_*() functions as we discussed above? That may help alleviate the race during initialisation.

[quote="doneill"]Can you probe for devices on a USB bus similar to IDE probing?[/quote]

That's a good question, and I'm not sure myself. The way it works currently is that the HCD reacts to interrupts from the controller, which will tell it if the port status has changed (attached or detached). It'll gather some device info from the port and pass it to the bus manager, which will then ask each USB driver in turn if it recognises the device that has been attached. I'm not sure what happens when the HCD driver is loaded initially though: one assumes it must scan the root hub for any devices which are already attached?
Back to top
View user's profile Send private message Send e-mail
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Mon Oct 19, 2009 1:14 pm    Post subject: Reply with quote

My hypothesis is that the HCD driver activates, the attached root hub and child hub(s) activate, then the devices activate. Once 'on', they power up their little chips and check in to HQ (the bus).

Sure, it's a simplistic idea, but provided that, the devices I've tried indicate an off status according to their indicator LEDs. With this info I have no clue if they're actually on and talking with the USB bus or simply playing dead until the bus says otherwise.

In any case, since an invalid mount point would otherwise fail and lock the kernel up, looping the attempt doesn't seem like a bad idea, even as a final design decision... I mean, either it sits their failed and doing nothing or it loops and boots. Win-win! Laughing

When Syllable gets a fancy boot screen like Haiku down the road, you could have it display a 'searching for filesystem' status, or somesuch? Smile
Back to top
View user's profile Send private message
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Tue Oct 20, 2009 4:10 am    Post subject: Reply with quote

Okay, here's my working patch so far:

http://riverroadcable.com/~doneill/init.c-boot_retry_and_@search.patch

Works fine on Qemu.. tried on an EeePC using a root FS on an SD card in the SDcard slot and it found the kernel, boot_shell made it through, but then I had an init() segfault. *shrug*

It could be something obvious I did.. not sure, I'll keep fiddling.

(Hm, booting the normal way crashes too... seems to be detaching the USB device after mounting during boot sequence for some reason.. now "Schedule called while g_sSchedSpinLock.sl_nNest == 1"..? This seems like a deeper issue than my init.c patch?)

Seems to work fine if I don't use the "USB" boot option in Grub, particularly, omitting:

module /boot/drivers/dev/bus/usb
module /boot/drivers/dev/hcd/usb_ohci
module /boot/drivers/dev/hcd/usb_uhci
module /boot/drivers/dev/disk/usb

works... strange.
Back to top
View user's profile Send private message
Vanders
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 849

PostPosted: Tue Oct 20, 2009 6:25 am    Post subject: Reply with quote

[quote="doneill"]Okay, here's my working patch so far:

http://riverroadcable.com/~doneill/init.c-boot_retry_and_@search.patch
[/quote]

Looks good so far, although be careful of the coding style, particular variable names. It isn't major but you might want to read [url=http://development.syllable.org/documentation/coding-style.html]if you get a chance[/url]

[quote="doneill"]Works fine on Qemu.. tried on an EeePC using a root FS on an SD card in the SDcard slot and it found the kernel, boot_shell made it through, but then I had an init() segfault. *shrug*

Schedule called while g_sSchedSpinLock.sl_nNest == 1"..? This seems like a deeper issue than my init.c patch?[/quote]

Ouch, that's a tricky one: it could be caused by a lot of things, and it's not going to be easy to track down. You might want to try using the printk_max= kernel argument to see if there is anything more obvious before that output begins.
Back to top
View user's profile Send private message Send e-mail
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Tue Oct 20, 2009 1:07 pm    Post subject: Reply with quote

I haven't had any luck pinning down the source yet, the crash seems to change points of failure too which makes me think i should look at the last common elements of each crashed function in the backtraces instead.

Would that make sense?
Back to top
View user's profile Send private message
Vanders
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 849

PostPosted: Tue Oct 20, 2009 4:43 pm    Post subject: Reply with quote

[quote="doneill"]I haven't had any luck pinning down the source yet, the crash seems to change points of failure too which makes me think i should look at the last common elements of each crashed function in the backtraces instead.

Would that make sense?[/quote]

Yes. You may actually be some way up the stack where the actual functions happens: if you're smashing the stack for example, the trace after that point may be meaningless.

It's also incredibly easy to get into all sorts of problems once you're in the kernel, because the stack can change in all sorts of ways that you don't get in userspace.
Back to top
View user's profile Send private message Send e-mail
doneill



Joined: 16 Oct 2009
Posts: 40

PostPosted: Thu Oct 22, 2009 10:08 am    Post subject: Reply with quote

Well, you've seen my patch, aside from I think one iterator and a handle, I have nothing on the stack at all. Other than what is in the patch I've made no changes to the kernel source, but I've noticed the recompiled kernel extensions (disk/usb, hcd, etc.) don't work, but the original ones do. I think there's something funky with my compiling itself.

That said, it still boots fine in Qemu, only fails on the Eeepc. Interesting side-note, Haiku is stable in Qemu, but the exact same installation is unstable on my Eeepc. I see a trend..
Back to top
View user's profile Send private message
Vanders
The Knights of Syllable


Joined: 14 Sep 2007
Posts: 849

PostPosted: Thu Oct 22, 2009 10:20 am    Post subject: Reply with quote

[quote="doneill"]I've noticed the recompiled kernel extensions (disk/usb, hcd, etc.) don't work, but the original ones do. I think there's something funky with my compiling itself.[/quote]

That's possible. How are you compiling your kernel? It's usually best to use Builder (http://syllable.cvs.sourceforge.net/*checkout*/syllable/syllable/system/apps/utils/Builder), which will setup the build environment etc. correctly. You should also make sure you have the latest kernel headers from CVS if you are compiling from CVS sources.
Back to top
View user's profile Send private message Send e-mail
Rasmus09



Joined: 11 Apr 2012
Posts: 1

PostPosted: Wed Apr 11, 2012 3:28 am    Post subject: Reply with quote

This is an great report! I have the same thing with root=@search too. I really like it.
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Syllable Forum Index -> Bugs All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group