Device Tree revisited

[notro]

I have a proposal for a way to get Device Tree support and move to ARCH_BCM2835.

Steps

• Enable the Common Clock framework (COMMON_CLK). This also works without DT.
  Replaces arch/arm/mach-bcm2708/clock.c which is removed
• Add bootloader option to set ATAG_MEM to usable memory only, and fill chosen/bootargs with the non-DT (extended) ATAG_CMDLINE.
  I expand on this further down.
• Add Device Tree IRQ mapping to arch/arm/mach-bcm2708/armctrl.c
  GPIO interrupts is mapped to a virtual bank 3 since we can't use irq-bcm2835 (at least I couldn't get it to work).
• Add Device Tree support to arch/arm/mach-bcm2708/bcm2708.c
• Add minimal Device Tree files
  bcm2708.dtsi
  bcm2708-rpi-b.dts
• Enable Device Tree support (OF_USE)

At this point the kernel boots with Device Tree.
Platform devices can now be added through Device Tree.

Next steps

• Add pinctrl driver (modified version of pinctrl-bcm2835)
  It provides DT support to the gpio_chip added in bcm2708_gpio
• Add DT support to i2c-bcm2708 and spi-bcm2708
  Move SPI/I2C devices to DT
  It is also possible to enable i2c-bcm2835 and spi-bcm2835 as well. Testing can begin and maybe we can move to them sooner than later.

At this point SPI and I2C devices can also be added, and pins/GPIOs can be configured (in/out, alt0 etc.) in DT.

Move drivers to DT
Now the rest of the drivers can get DT support and the devices be moved from bcm2708.c to DT
(I have done most of these already in patches lying around)

At this point all devices are added through Device Tree.

Normal ATAG_CMDLINE

Now the normal ATAG_CMDLINE can be used for chosen/bootargs.

Normal chosen/bootargs
Now config.txt can be used for chosen/bootargs if preferred.

Start using dmaengine
Currently most of the drivers manipulate the DMA registers directly.
To remedy this, bcm2708-dmaengine/bcm2835-dma has to get support for slave_sg (I believe this is what is needed).
Then bcm2708_fb, sdhci-bcm2708 and others can start using dmaengine.

Mailbox driver
There is work going on upstream for this (Lubomir Rintel and others).
The Mailbox API isn't settled yet (latest patch version).
Drivers would need to start using this.

FIQ
We need to have FIQ support in ARCH_BCM2835.
I don't know if it is already supported in irq-bcm2835.
Is shortcut the same as FIQ?

At this point, switching to ARCH_BCM2835 should (hopefully) be close at hand.
Some of these steps can of course happen in parallel.

rpi-update
Maybe rpi-update could handle config.txt for Device Tree kernels.
If a .dtb is present
add DT options and use the .dtb
else
remove DT options if present.

VideoCore Bootloader in DT mode
Current behaviour and proposed changes (through a new config option or something similar if needed):

config.txt

device_tree=bcm2708-rpi-b.dtb
device_tree_address=0x100
disable_commandline_tags=1

ATAG_MEM
This is set to the amount of installed memory, not the part that's available to the CPU.
This requires a /memreserve/ node in DT to keep Linux from using memory it doesn't have.
The value of this node depends on the GPU/CPU memory split value in config.txt.
Changing this to available memory, removes the need for /memreserve/. This is how ARCH_BCM2835 expects it, and U-boot delivers it.

ATAG_CMDLINE

chosen/bootargs
This is the contents of cmdline.txt.
Changing this to the extended commandline used in non-DT mode, means no drivers have to be changed when moving to DT.

One caveat with the VC bootloader is that chosen/bootargs has to be prefilled with 1024 characters (COMMAND_LINE_SIZE), since the bootloader isn't equipped to allocate more space in the Device Tree.
The bootloader must have this ability to be able to boot vanilla Linux and pass in cmdline.txt.
libfdt is in use in many projects to provide Device Tree manipulation abilities (Linux, U-boot and others).

Example
I have a working example using a modified U-Boot to get the desired bootloader behaviour.
It sort of covers the steps up to SPI/I2C (it was made before this text was written).

Install

sudo REPO_URI=https://github.com/notro/rpi-firmware BRANCH=dt rpi-update

Patches used in this example:

• U-boot
• Linux (here DT support is controlled with CONFIG_BCM2708_DT)

Issue history

• Need help with Device Tree 2013-09-21
• Device tree population 2012-05-17

[notro]

Device Tree Overlays (2014-05-28) support in mainline is getting closer:

The following patchset introduces Device Tree overlays, a method
of dynamically altering the kernel's live Device Tree.

Comment by Grant Likely on the latest patch version:

Good changes. This version is considerably better than the past one.
I've still got comments, but I'm a lot more comfortable with the
approach.

When this is in, the BeagleBone capemanager patchset will shrink in size/width and maybe we can use it in some form on the DT enabled Pi.

[popcornmix]

Okay, I'm interested in this plan.

If you want to submit any kernel PRs that bring us closer to supporting DT, and don't break things for non-DT users, then go ahead.

I should be able to provide a test start.elf that fills in the DT as desired.
If you could produce a dtb file and kernel image that prints out the device tree fields, and what you would like the fields to contain (i.e. what you get from uboot), that would make my life easier.

[notro]

I wrote ATAG_CMDLINE a couple of places where it should have been chosen/bootargs, sorry about that. I have fixed it.

Bootloader
This is the bootloader behaviour I need (enable with disable_commandline_tags=2 if you prefer to keep the current behaviour as an option):

• ATAG_MEM = Memory available to the CPU (default: 512 total - 64 GPU = 448 CPU)
• DT chosen/bootargs = extended command line (dmachans etc. + cmdline.txt)

No other bootloader changes are needed at this time.
We'll deal with Device Tree needs for drivers when we get to that.

Kernel config
CONFIG_BCM2708_DT will be used to enable Device Tree

Pull Requests
How about this for starters:
PR for Common Clock Framework
PR with individual commits for: armctrl.c, bcm2708.c, dts files and config options (or should I split this into more PR's?)
PR for new pinctrl driver
PR for spi-bcm2708
PR for i2c-bcm2708

Should I make PR's against rpi-3.15.y?

[popcornmix]

Yes, I think PR against rpi-3.15.y makes sense.
The PR's mentioned sound okay (no need to split more finely)
I'll have a go at updated firmware tomorrow.

[notro]

Thanks.
You can test the firmware with these if you want:

• https://github.com/notro/rpi-firmware/blob/dt/bcm2708-rpi-b.dtb
• https://github.com/notro/rpi-firmware/blob/dt/zImage

Or I can do the testing if the change is trivial.

[notro]

The clocks are added in a different way with the Common Clock Framework, so I'm reworking that.
Can I remove the CONFIG_ARCH_BCM2708_CHIPIT #ifdef's?
There's no such config option, so even if it was added manually to .config, make oldconfig would remove it.

Ref: https://github.com/raspberrypi/linux/blob/rpi-3.15.y/arch/arm/mach-bcm2708/bcm2708.c#L208

New clock setup: https://gist.github.com/notro/b3e5ef5e9a51bb21e4ac

# cat /sys/kernel/debug/clk/clk_summary
   clock                        enable_cnt  prepare_cnt  rate        accuracy
---------------------------------------------------------------------------------
 sdhost_clk                     1           1            250000000  0
 osc_clk                        0           0            500000000  0
 uart_clk                       3           4            3000000    0

[popcornmix]

CONFIG_ARCH_BCM2708_CHIPIT hasn't been used for years, so feel free to remove it.

[popcornmix]

I've added to config.txt

device_tree=bcm2708-rpi-b.dtb
device_tree_address=0x100
disable_commandline_tags=1
kernel=zImage

and put the linked zImage and bcm2708-rpi-b.dtb in /boot.

Would you expect to get any console output or anything on framebuffer? I get nothing.

[ghollingworth]

Just to note, I've got an intern working on creating suitable drivers that can be upstreamed in such a way that we get here by starting with the upstream kernel and moving our support across rather than add more hackery to support upstream facilities....

Currently he's working on a new mmc driver for the bcm2835 architecture which doesn't then require sdhci hackery to upstream

Gordon

[notro]

Just to note, I've got an intern working on creating suitable drivers that can be upstreamed in such a way that we get here by starting with the upstream kernel and moving our support across rather than add more hackery to support upstream facilities....

Do you have a timeline for that and does he/she have previous kernel development experience?
Is there anything in this plan you regard as hackery (except the pinctrl driver)?

Currently he's working on a new mmc driver for the bcm2835 architecture which doesn't then require sdhci hackery to upstream

This is good news. My brief testing of that driver (sdhci-bcm2835.c) was disappointing.

[popcornmix]

@notro
Can you confirm if what I asked in #627 (comment) should boot?

[notro]

Can you confirm if what I asked in #627 (comment) should boot?

I'm sorry, I missed that. Sometimes I don't get e-mail alerts in this thread when new messages comes in.

I don't have a HDMI monitor, so I tried it (with U-boot) on my sons monitor, and it worked fine.

Back on my desk with serial, I added the config.txt changes and booted. I get output, but it crashes before the framebuffer driver is loaded.
I thought it was because of missing commandline options, but adding those to cmdline.txt didn't help.

This is what I get

[    1.269194] Unable to handle kernel NULL pointer dereference at virtual address 00000026

Instead of

[    1.297300] bcm2708_dma: DMA manager at f2007000

I have to investigate this further.

Full console output: https://gist.github.com/notro/376d63def4bab8c1c8b3

[notro]

This is the ATAG_MEM issue.
The VC bootloader sets ATAG_MEM=512MB, whereas U-boot sets ATAG_MEM=448MB
The kernel crashes because it also uses the GPU only memory.

VC

[    0.000000] Memory: 504404K/524288K available (4508K kernel code, 260K rwdata, 1448K rodata, 152K init, 702K bss, 19884K reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)

U-boot

[    0.000000] Memory: 439380K/458752K available (4508K kernel code, 260K rwdata, 1448K rodata, 152K init, 702K bss, 19372K reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     lowmem  : 0xc0000000 - 0xdc000000   ( 448 MB)

[popcornmix]

Okay I was missing "kernel_address=0x8000". Now I get some boot messages (including the Memory line one) and the panic.
I'll see if I can get the same numbers as u-boot.

[notro]

This seems to be the way U-boot gets the RAM size:

#define BCM2835_MBOX_TAG_GET_ARM_MEMORY         0x00010005

BCM2835_MBOX_INIT_TAG(&msg->get_arm_mem, GET_ARM_MEMORY);

Ref: http://git.denx.de/?p=u-boot/u-boot-arm.git;a=blob;f=board/raspberrypi/rpi_b/rpi_b.c;h=f33fae91700cadf344905ae0f31198ee38be6c53;hb=HEAD#l44