Improve IDE Controller - #1325
Conversation
Merged all functionally relevant changes, omitted:
1. Left out almost all of the "dbg_log()" changes, they're like 90% of the original PR and make the essential changes in the diffs a bit hard to read, these can be added back in at a later point in time.
2. Also left out the "wants_cdrom" option and implemented its desired effect with "cdrom: { ejected: true }" as was suggested.
3. The test "tests/api/cdrom-insert-eject.js" is also left out, can be added later back in.
This patch should behave like the last changeset from JoeOsborn in 2023.
Implement a single PCI IDE device in Compatibility Mode for up to 4 drives. Changes to ide.js: - added new exported class IDEPCIAdapter, the root class for up to 4 drives - introduced new config scheme for IDEPCIAdapter (see comment about adapter_config) - moved PCI-device related members from IDEDevice to IDEPCIAdapter - rewrote constructor of IDEDevice (no longer exported) - fixed (?) write-access to ata_port registers 1-6 (see comment below) - added missing declarations for BAR2 and BAR3 in pci_bars[] - made definition of the PCI configuration space a bit more more verbose - renamed "master_port" to "bus_master_port" to avoid confusion - added a few comments here and there Regarding write-access to ata_port registers 1-6: It is not clear why these 6 functions simultaneously modify master and slave attributes instead of just modifying this.current_interface (as is the case everywhere else in the code). This patch changes that to use this.current_interface, experimental until there is some explanation. Also changed cpu.js to now use IDEPCIAdapter instead of IDEDevice. TODO: 1. clear up the matter around write-access to ata_port registers 1-6 2. in ide.js, a single-device channel needs to be better supported, currently the code reports a Hard-Disk device of size 0 for an unconnected slave drive, and there must be a better way to signal this case. 3. in cpu.js, this.devices.hda and this.devices.cdrom point to a IDEDevice, they should point to an IDEInterface for this.devices.hdb to make sense. This needs to be better understood and fixed for slave drives to work. 4. getting some very early output under SeaBIOS, but the screen clears so quickly that I cannot read it and it's also not printed to the serial console
…ased access to IDEDevice and IDEInterface objects
- immplemented Command and Status register requirements from the standard, ignored all others - rearranged IDE register definitions, added a descriptive comment to each (standard lingo) - fixed the EXECUTE DEVICE DIAGNOSTIC (90h) ATA command
In v86, the name "device" is already used in the context of PCI, and additionally, in the ATA/ATAPI standard the term "device" is used for disk drives. The name used for this entity (IDEDevice) in the ATA/ATAPI standard is "channel", and since "channel" is not otherwise used in ide.js it seems right to rename IDEDevice to IDEChannel.
Merged and improved on the work from JoeOsborn in respect to formatting in dbg_log() output. Used register names from the standard for better readability of the output, and added a few missing logs.
|
@copy, in order to continue comparing the code in I'd like to align the standard ATA register names with their corresponding member variables. Here the related names, (a) as in the ATA/ATAPI-6 standard, (b) as they currently are in Before doing so I'd like to ask if that's ok with you, maybe there are good reasons for the member names as they are that I just don't know about. I will also start to replace some of the many hard-coded literal register assignments with named constants. Too many magic numbers. |
Renamed members that carry register values to better align with the actual wording in the ATA/ATAPI-6 standard. Other changes in this commit include: - added a bunch of named constants for ATA registers and their bits - improved PCI configuration space setup - renamed class IDEPCIAdapter to IDEController
|
Aside of lots of code cleanup, there are two issues left that will need some more time.
Other than that, it works very well. Example: A few weeks ago, FreeDOS 1.4 was released, and I could not create an installation from their live CD onto a hard-disk under v86. With this PR, it works perfectly smooth and is a real joy. Both issues are not critical, but I find them serious enough to continue working on them. Regarding the first issue, currently you can boot with or without a CD inserted, and you can insert a CD later, works fine. However, you cannot eject it, and you cannot replace it, once you've inserted a CD that's it until you restart the VM. I have to learn about these mechanics in ATAPI, that means in SCSI, and that will need a bit of time. The second issue is about configurations where only a single disk (instead of two) is connected to a channel (IDE controllers support two channels for a total of 4 disks). Physically, a Y-shaped cable was connected to an IDE channel and two disks could be connected to that cable, and disks had jumpers which had to be configured mutually exclusive to tell them apart (they were called "master" and "slave" disks, back in the days). Now, the ATA/ATAPI-6 standard says that in this case, the master disk is supposed to answer as a proxy for requests aimed at the slave disk in case it doesn't exist, with very special rules. I'm still working on the meaning of this, I've implemented it but it fatally breaks Boch's ATAPI detection, but that might be due to ATAPI bugs in v86. I removed that implementation for now, but kept it for a future point in time. I suppose this was specified this way to distinguish the single-disk case from the no-disk case. I think the next thing to do is to get a better understanding of ATAPI, and specifically SCSI. Does anyone happen to have any tips for reading material about SCSI? |
|
I'm pretty sure now that the subset of SCSI commands used with ATAPI in v86 is specified in SCSI Multimedia Commands – 3 (MMC-3), on the surface it matches the code in v86 (the implemented command set is documented in the standard). Its release date also almost matches the one of the ATA/ATAPI-6 standard.
|
In the original code, status register bit DSC (Drive Seek Complete, 0x10) is always set when completing without error, but ATA/ATAPI-6 specifies this bit to be command dependent. This PR now sets the status register exactly like it was before, just using named constants instead of magic numbers. Also replaced most of the remaining magic number assignments to the ATA status register.
In the context of ATAPI there's only a single linked document in ide.js: https://www.t10.org/ftp/x3t9.2/document.87/87-106r0.txt. This link points to "PROPOSAL FOR CD-ROM IN SCSI-2" from 1987 (!) and its content matches the current code in v86, which the SCSI-3 documents did not at all (only the command codes matched, but otherwise SCSI-3 is very different from SCSI-2).
The SCSI Status field ("Status Key" and "Additional Status Code" aka. ASC) was incorrectly passed from the SCSI layer to the ATA register layer, and ATAPI command "TEST UNIT READY" was not implemented correctly.
This led to Linux not properly detecting when no CD was inserted into the CD-ROM device, which led to error output in "hwinfo" like (it attempted to read a disk that's no inserted):
> root@debian-iscsi:~# hwinfo --cdrom
> > block.5.1: /dev/sr0 cache[ 78.371349] I/O error, dev sr0, sector 64 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
> [ 78.412349] I/O error, dev sr0, sector 64 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
> [ 78.412349] Buffer I/O error on dev sr0, logical block 16, async page read
> [ 78.448365] I/O error, dev sr0, sector 68 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
> [ 78.448369] Buffer I/O error on dev sr0, logical block 17, async page read 08: SCSI 100.0: 10602 CD-ROM
This commit fixes that problem by now properly returning SCSI Sense and Additional Sense responses to the ATAPI host.
|
Understood and fixed a rather important bug that sat between the ATA/ATAPI and the SCSI layer. This fix makes the CD-ROM device to work without errors under Linux now! Debian bash log about CD-ROM with comments# NOTE: booted with ejected CD-ROM disk
Debian GNU/Linux 12 debian-iscsi tty1
debian-iscsi login: root
Password:
# run hwinfo. NOTE: no more errors with "hwinfo" as before here, and see "Drive status"
root@debian-iscsi:~# hwinfo --cdrom
08: SCSI 100.0: 10602 CD-ROM
[Created at block.259]
Unique ID: KD9E.cJjDrst78LF
Parent ID: BUZT.bgG9RSCjJU8
SysFS ID: /class/block/sr0
SysFS BusID: 1:0:0:0
SysFS Device Link: /devices/pci0000:00/0000:00:1f.0/ata2/host1/target1:0:0/1:0:0:0
Hardware Class: cdrom
Model: "SONY CD-ROM CDU-1000"
Vendor: "SONY"
Device: "CD-ROM CDU-1000"
Revision: "1.1a"
Driver: "ata_piix", "sr"
Driver Modules: "ata_piix", "sr_mod"
Device File: /dev/sr0 (/dev/sg1)
Device Files: /dev/sr0, /dev/disk/by-path/pci-0000:00:1f.0-ata-2, /dev/cdrom,/dev/disk/by-path/pci-0000:00:1f.0-ata-2.0, /dev/disk/by-diskseq/2
Device Number: block 11:0 (char 21:1)
Drive status: no medium
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #4 (IDE interface)
Drive Speed: 1
# attempt to mount CD-ROM with still ejected disk (NOTE: error message is correct)
root@debian-iscsi:~# mount -t iso9660 -o ro /dev/sr0 /mnt/
mount: /mnt: no medium found on /dev/sr0.
dmesg(1) may have more information after failed mount system call.
# [external: load image "FD14LIVE.iso" into v86 cdrom device via v86 API]
root@debian-iscsi:~# mount -t iso9660 -o ro /dev/sr0 /mnt/
root@debian-iscsi:~# ls /mnt/
CHANGES.LOG devel fdconfig.sys freedos isolinux net setup.bat
COMMAND.COM fdauto.bat FDOS-x86 games KERNEL.SYS packages
# ok, unmount CD-ROM
root@debian-iscsi:~# umount /mnt
# [external: unload image from v86 cdrom device via v86 API]
# attempt to mount CD-ROM with ejected disk to check the disk is really gone
root@debian-iscsi:~# mount -t iso9660 -o ro /dev/sr0 /mnt/
mount: /mnt: no medium found on /dev/sr0.
dmesg(1) may have more information after failed mount system call.
# [external: load image "dsl-4.11.rc2.iso" into v86 cdrom device via v86 API]
root@debian-iscsi:~# mount -t iso9660 -o ro /dev/sr0 /mnt
root@debian-iscsi:~# ls /mnt/
boot index.html KNOPPIX
# run hwinfo with inserted disk
root@debian-iscsi:~# hwinfo --cdrom
08: SCSI 100.0: 10602 CD-ROM
[Created at block.259]
Unique ID: KD9E.cJjDrst78LF
Parent ID: BUZT.bgG9RSCjJU8
SysFS ID: /class/block/sr0
SysFS BusID: 1:0:0:0
SysFS Device Link: /devices/pci0000:00/0000:00:1f.0/ata2/host1/target1:0:0/1:0:0:0
Hardware Class: cdrom
Model: "SONY CD-ROM CDU-1000"
Vendor: "SONY"
Device: "CD-ROM CDU-1000"
Revision: "1.1a"
Driver: "ata_piix", "sr"
Driver Modules: "ata_piix", "sr_mod"
Device File: /dev/sr0 (/dev/sg1)
Device Files: /dev/sr0, /dev/disk/by-path/pci-0000:00:1f.0-ata-2.0, /dev/disk/by-path/pci-0000:00:1f.0-ata-2, /dev/cdrom, /dev/disk/by-diskseq/15, /dev/disk/by-label/KNOPPIX, /dev/disk/by-uuid/2012-09-26-14-40-23-00
Device Number: block 11:0 (char 21:1)
Config Status: cfg=new, avail=yes, need=no, active=unknown
Attached to: #4 (IDE interface)
Drive Speed: 1
Volume ID: "KNOPPIX"
Application: "MKISOFS ISO 9660/HFS FILESYSTEM BUILDER & CDRECORD CD-R/DVD CREATOR (C) 1993 E.YOUNGDALE (C) 1997 J.PEARSON/J.SCHILLING"
Creation date: "2012092614402300"
El Torito info: platform 0, bootable
Boot Catalog: at sector 0x0026
Media: none starting at sector 0x0027
Load: 2048 bytesFreeDOS on the other hand still doesn't notice when I eject the Compact Disk, but that's not unexpected.
|
|
In the ATA/ATAPI-8 spec I stumbled upon this sentence, well possible that I overlooked it in earlier revisions:
I checked and found it missing in our code, added it, and now the CD-ROM device runs flawlessly under Windows 95:
Note that unlike earlier:
Eject/reinsert under FreeDOS is unfortunately still scuffed, though. |
The minor change from yesterday changed Win95's behaviour such that it now issues an ATAPI command ("PAUSE") that wasn't implemented yet which led to Win95 crashing in v86 Debug mode. Added that command and now it works again.
A few ATAPI commands did not return an error (CD Not Ready condition) with ejected medium as specified in MMC-3 (https://www.t10.org/ftp/t10/document.97/97-108r0.pdf). Adding these to our subset of ATAPI commands improved the FreeDOS 1.4 situation around eject/insert a bit (the problem really is eject, insert always works given the medium is ejected). Waiting around 20-30 sec after eject seems to work reliably.
Adds support to eject a CD from within Windows 95 through the context menu of the CD icon. Before this commit eject was only supported externally through the v86 API.
|
Ejecting a disk from within the guest OS works well across different OSes:
When ejecting from outside using the v86 API it works well under Windows 95, and under FreeDOS 1.4 you have to wait 15-20 seconds to let it "click". |
|
I setup my local v86 with a second hard-disk SeaBIOS 1.16.2 SeaBIOS detects all three drives fine, from its debug output: Guest OS: Debian 12 Works using a 26M blank file for root@debian:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 2G 0 disk
└─sda1 8:1 0 2G 0 part /
sdb 8:16 0 26M 0 disk
sr0 11:0 1 1024M 0 rom
root@debian:~# cfdisk /dev/sdb
root@debian:~# mkfs.ext4 -L mysdb1 /dev/sdb1
root@debian:~# mount /dev/sdb1 /mnt
root@debian:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 2G 0 disk
└─sda1 8:1 0 2G 0 part /
sdb 8:16 0 26M 0 disk
└─sdb1 8:17 0 24M 0 part /mnt
sr0 11:0 1 1024M 0 rom
root@debian:~# ls /mnt
lost+foundGuest OS: FreeDOS 1.4 Works using the same 26M blank file for Guest OS: Windows 95 Works using a preformatted 8M image file (FAT-16, labelled MYDISK, non-bootable) for
Detail: Linux commands used to create the 8M FAT-16 image file for Windows 95# create 8M empty file
dd if=/dev/zero of=empty_8M_FAT16.img bs=1M count=8
# use partition TYPE "FAT16<32M", do not mark bootable
sudo cfdisk empty_8M_FAT16.img
# format using FAT
sudo losetup -f empty_8M_FAT16.img
sudo partx -a /dev/loop0
sudo mkfs.fat -n MYDISK /dev/loop0p1
sudo partx -d /dev/loop0
sudo losetup -d /dev/loop0So this works really well :) |
There was an overlap in the PCI space register declarations of the Floppy and the IDE controller. Since the conflicting registers were used exclusively by the two controllers this was fine, but 9front prints out this boot warning message:
ioalloc: 3f0 - 3f5 floppy: clashes with: 3f4 - 3f5 PCI.0.31.0
Note that "PCI.0.31.0" is our IDE controller (with pci_id 0x1f).
Problem: IDE actually uses only register address 0x3f6, but declared BAR1 for address range 0x3f4-0x3f7 which overlaps with the Floppy controller (the same with BAR3 and its address range of 0x374-0x377, but there was no overlap with other devices here).
Fix:
- changed BAR1 base address from 0x3f4 to 0x3f6 and its size from 4 to 1
- changed BAR3 base address from 0x374 to 0x376 and its size from 4 to 1
This commit fixes that problem and the 9front boot message from above.
|
I made another attempt to make the 9front ISO image boot and succeeded this time, but it's just so weird. Here's a screenshot of the 9front desktop:
Aside from enabling ACPI and using SeaBIOS in Release mode you need to mount the 9front ISO9660 image on You can download the zst-compressed image here: 9front-7781.38dcaeaa222c.386.iso.zst and run it using the v86 web UI, just make sure to enable ACPI. Boot procedureYou'll be prompted for 5 boot parameters during the boot process, you can accept their given defaults by simply pressing ENTER each time. The first 4 boot parameters ( Next, 9front will switch to the graphical desktop and prompt for the 5th parameter (at the bottom), press ENTER to accept the mouse port default of
Wait a few seconds and you should see the normal 9front desktop. ISO image on
|
The 9front ISO starts with System Area that contains MBR bootloader, just like in raw hard drive images:
According to the El Torito specs (https://fossies.org/linux/libisofs/doc/boot_sectors.txt, see lines 157 - 167), the boot media emulation is disabled:
(i.e. if it is to simulate real hard disk, the byte So it reads sectors directly from a CD (and from a HDD too?) with ISO9660 filesystem.
That DOS (FAT) partition on |
|
@copy: I think we're good, I don't see this as WIP any more (meaning I don't see any serious issues left). |
Agreed. The remaining puzzle piece was making this compatible with old state images, which I pushed in #1345 |
|
Merged in #1345, thanks a lot! |
|
@SuperMaxusa: I think we should just go ahead and close these 6 issues, they can be reopened anytime in case. I'm usually a bit hesitant to close other people's issues, that's why I haven't done it on my own yet. @copy: Ok? |
4 participants
This PR improves on PR #901 "Fix booting from cd while hda present, booting from hda while cd present", please see my comment there and also issue #1323 "ATA/ATAPI PCI adapter" for more details if you like.
SeaBIOS now properly boots with both a HDA and a CD-ROM device configured, the CD-ROM may or may not be empty. Tested under FreeDOS 1.3, FreeDOS 1.4, Debian 12, FreeBSD 12, Windows 3.1, Windows 95, Syllable-0.6.7-1 and more. A CD-ROM disk can be inserted after booting with an empty CD drive, and ejecting a disk mostly works.
Boch's BIOS shows the same problems with or without this PR, when booting FreeDOS fails to find or activate the NE2K network card, and Debian fails to find any IDE drive (neither HDA nor CD-ROM). However, it does boot with this dual IDE configuration (HD and CD) which it doesn't without this PR.
Technically, this implementation should support any combination of up to 4 ATA-HDD and/or ATAPI-CD-ROM devices (tested with three devices
hda,hdbandcdrom), though this would cause some serious headache in the web UI setup.Major fixes and improvements in this PR:
This code attempts to implement a sufficient subset of the latest revisions of the IDE-related standards (that is: ATA/ATAPI-8 and MMC-3). However, some deprecated commands from older revisions are also supported for guest OSes that make still use of them.
References that this IDE code is based on:
Other related reading material: