PCIe is broken when EFI booting

[jmontleon]

If I EFI boot with load mmc 0:5 $fdt_addr_r dtb/spacemit/m1-x_milkv-jupiter.dtb; load mmc 0:6 $kernel_addr_r EFI/fedora/grubriscv64.efi; bootefi $kernel_addr_r $fdt_addr_r

I get the following:

[    2.428750] k1x-dwc-pcie ca400000.pcie: has no power on gpio.
[    2.436770] k1x-dwc-pcie ca400000.pcie: host bridge /soc/pcie@ca400000 ranges:
[    2.443972] k1x-dwc-pcie ca400000.pcie:       IO 0x009f002000..0x009f101fff -> 0x009f002000
[    2.452290] k1x-dwc-pcie ca400000.pcie:      MEM 0x0090000000..0x009effffff -> 0x0090000000
[    2.460607] k1x-dwc-pcie ca400000.pcie: resource collision: [mem 0x90000000-0x9effffff] conflicts with Reserved [mem 0x80000000-0xffffffff]
[    2.473103] k1x-dwc-pcie ca400000.pcie: failed to initialize host
[    2.479210] k1x-dwc-pcie ca400000.pcie: probe with driver k1x-dwc-pcie failed with error -12
[    2.488328] k1x-dwc-pcie ca800000.pcie: has no power on gpio.
[    2.496278] k1x-dwc-pcie ca800000.pcie: host bridge /soc/pcie@ca800000 ranges:
[    2.503487] k1x-dwc-pcie ca800000.pcie:       IO 0x00b7002000..0x00b7101fff -> 0x00b7002000
[    2.511818] k1x-dwc-pcie ca800000.pcie:      MEM 0x00a0000000..0x00afffffff -> 0x00a0000000
[    2.520153] k1x-dwc-pcie ca800000.pcie:      MEM 0x00b0000000..0x00b6ffffff -> 0x00b0000000
[    2.528449] k1x-dwc-pcie ca800000.pcie: resource collision: [mem 0xa0000000-0xafffffff pref] conflicts with Reserved [mem 0x80000000-0xffffffff]
[    2.541375] k1x-dwc-pcie ca800000.pcie: failed to initialize host
[    2.547486] k1x-dwc-pcie ca800000.pcie: probe with driver k1x-dwc-pcie failed with error -12

If I comment out this reserved area PCIe works when EFI booting:
https://github.com/spacemit-com/linux-k1x/blob/k1/arch/riscv/boot/dts/spacemit/k1-x.dtsi#L425-L429

		area_reserved@80000000 {
			/* dram area is 0~2GB, and 4GB~, the 2GB~4GB is io area */
			reg = <0x0 0x80000000 0x0 0x40800000>;
			no-map;
		};

This looks a lot like what is described in this commit 233515a

I've tried reusable instead of no-map which changes the error slightly but still does not work, adding compatible for k1x dwc pci, and some other things, but so far only removing the entire reserved area entry stops the problem.

Do you have any insight into what might be going wrong here and the proper way to fix it?

[jmontleon]

My possibly wrong understanding from doing some reading is that reserved-memory with no-map ends up with the EfiReservedMemoryType and becomes unusable when booting via EFI.

[yaneti]

Looking over at other dts-es in the linux tree I don't see any other example where the bar space for a pcie controller is marked as reserved-memory. I think just removing both area_reserved@ stanzas is the way to go here.

Any potential fallout after that might need some additional device tree node attributes, and not a blanket reserved

[kevin-zhm]

Has been fixed, 9a3d175