Add support for Rockchip rk3588

[edtubbs]

Added support for the RK3588 to enable NanoPC-T6, based on the ROCK4 platform. NanoPC-T6 will also be added to the manifest and build repositories:
https://github.com/edtubbs/manifest/tree/nanopc-t6
https://github.com/edtubbs/build/tree/nanopc-t6

[jforissier]

Hi @edtubbs, thanks for the patch. Please add you Signed-off-by: tag to the commit description, as well as:

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>

[jforissier]

One small thing perhaps: better use "Add support for Rockchip rk3388" as a subject and mention NanoPC-T6 in the commit description. Thanks!

[macromorgan]

Unrelated question, but do you have the registers for doing this on the rk3568 as well? It looks like some of them (the easy ones) can be pulled from the TRM, but there are others I'm curious about. Also, will you be adding the HUC stuff as well? That might come in handy if we need to program the RPMB of an eMMC.

Thank you.

[DaniilKl]

I see rk3588 has two firewalls: FIREWALL_DDR with base address 0xFE030000 and FIREWALL_SYSMEM with base address 0xFE038000 have you been able to find out the difference? I guess the FIREWALL_DDR hold MMU registers, and FIREWALL_SYSMEM holds SMMU registers, but I cannot confirm this.

[DaniilKl]

How have you been able to find out how to set MMU configuration for this SoC? I cannot find any exact MMU register mappings in SoC's TRM. Actually, this file seems like a platform_rk3399.c copy.

[DualTachyon]

Many different hardware blocks have their own MMUs. Most are the same type, but not always. These tend to be something different than the ARM SMMU type (these also exist, protecting PCIE and PHP blocks). They are documented in the TRMs in each block's respective chapter, although it is mostly in the register sections.

[github-actions]

This pull request has been marked as a stale pull request because it has been open (more than) 30 days with no activity. Remove the stale label or add a comment, otherwise this pull request will automatically be closed in 5 days. Note, that you can always re-open a closed issue at any time.

[NickDarvey]

Any chance of this getting merged? Is it awaiting responses to @DaniilKl's questions?

[jforissier]

Any chance of this getting merged? Is it awaiting responses to @DaniilKl's questions?

Yes. I'd like to make sure the config is correct, and if the file is indeed the same as rk3399 then it should be renamed not duplicated.

[edtubbs]

Thanks for the feedback! I referenced TF-A values in rk3588_def.h for FIREWALL_DDR and FIREWALL_SYSMEM. FIREWALL_DDR controls DDR access and FIREWALL_SYSMEM is internal/system memory. Both are also used in RK3588's u-boot initialization (arch_cpu_init).

The platform_rk3588.c file began as a copy of platform_rk3399.c, with plans to expand it for TRNG and HUK. Open to renaming if needed.

[DaniilKl]

FIREWALL_DDR controls DDR access

@edtubbs, if it does so, why you are using SGRF registers instead of FIREWALL_DDR to configure memory protection for OP-TEE OS in platform_secure_ddr_region?

[edtubbs]

FIREWALL_DDR designates access by individual masters, but SGRF assigns entire DDR regions to the secure world (e.g., CFG_TZDRAM_START + CFG_TZDRAM_SIZE). We could use FIREWALL_DDR to enable more master-specific access control in the future.

[edtubbs]

Here’s the debug log confirming platform_secure_ddr_region protection:

D/TC:0 0 platform_secure_ddr_region:35 protecting region 1: 0x8400000-0xa400000
D/TC:0 0 do_init_calls:19 service_initcall level 2 check_ta_store()
D/TC:0 0 check_ta_store:449 TA store: "early TA"
D/TC:0 0 check_ta_store:449 TA store: "Secure Storage TA"
D/TC:0 0 check_ta_store:449 TA store: "REE"
D/TC:0 0 do_init_calls:19 service_initcall level 2 early_ta_init()
D/TC:0 0 early_ta_init:56 Early TA f04a0fe7-1f5d-4b9b-abf7-619b85b4ce8c size 49724 (compressed, uncompressed 113448)
D/TC:0 0 do_init_calls:19 service_initcall level 2 verify_pseudo_tas_conformance()
D/TC:0 0 do_init_calls:19 service_initcall level 3 tee_fs_init_key_manager()
D/TC:0 0 do_init_calls:19 driver_initcall level 1 probe_dt_drivers()
D/TC:0 0 do_init_calls:19 driver_initcall level 2 init_multi_core_panic_handler()
D/TC:0 0 do_init_calls:19 driver_initcall level 2 mobj_init()
D/TC:0 0 do_init_calls:19 driver_initcall level 2 default_mobj_init()
D/TC:0 0 do_init_calls:19 driver_initcall level 2 ftmn_boot_tests()
D/TC:0 0 ftmn_boot_tests:198 Calling simple_call()
D/TC:0 0 ftmn_boot_tests:198 Return from simple_call()
D/TC:0 0 ftmn_boot_tests:199 Calling two_level_call()
D/TC:0 0 ftmn_boot_tests:199 Return from two_level_call()
D/TC:0 0 ftmn_boot_tests:200 Calling chained_calls()
D/TC:0 0 ftmn_boot_tests:200 Return from chained_calls()
D/TC:0 0 ftmn_boot_tests:202 *************************************************
D/TC:0 0 ftmn_boot_tests:203 **************  Tests complete  *****************
D/TC:0 0 ftmn_boot_tests:204 *************************************************
D/TC:0 0 do_init_calls:19 driver_initcall level 3 gic_set_primary_done()
D/TC:0 0 do_init_calls:19 driver_initcall level 3 release_probe_lists()
D/TC:0 0 do_init_calls:19 finalcall level 1 release_external_dt()
I/TC: Primary CPU switching to normal world boot

[DaniilKl]

FIREWALL_DDR designates access by individual masters, but SGRF assigns entire DDR regions to the secure world

What is the source of this information, could you reference it? I tried to verify this using RK3588 publicly available TRMs, but was unable to do that. The TRMs miss information for all FIREWALL_* memory regions, and there is no explicit information that any of the SYS_GRF registers configure MMU.

Here’s the debug log confirming platform_secure_ddr_region protection:

This is not a confirmation of the correct configuration of protection of the memory regions. platform_secure_ddr_region:35 protecting region 1: 0x8400000-0xa400000 is only a print function that prints some info., therefore it is not possible to state the correctness of the regions protection configuration.

Actually, does OP-TEE have any tests for confirming the correctness of the regions protection configuration a posteriori?

[edtubbs]

U-Boot configures component access (SDMMC, eMMC, crypto_ns, FSPI) by master to DDR regions using FIREWALL_DDR_BASE. This approach is also used in other Rockchip processors, such as the RK3506 and RK3562. TF-A uses SGRF to set DDR access (TF-A SGRF reference) in the sgrf_init function (sgrf_init function).

The platform_secure_ddr_region function writes to the register file, but it may also need to configure the region via FIREWALL_DDR_BASE (ddr_fw_rgn_config). I’ll work on testing DDR access from the normal world to verify protections.

[DualTachyon]

You can find the meaning of the FIREWALL_xxx blocks here: https://github.com/DualTachyon/rk3588-svd/tree/main/def/security/firewall

[edtubbs]

I've updated the patch to map and enable the DDR firewall, following the TF-A approach (reference). Verified protections by reading near region boundaries in U-Boot:

=> md 0x083ffff0 4
083ffff0: fffff7bf fffffff7 fffffdfd ffffffff    ................
=> md 0x083ffff0 5
083ffff0: fffff7bf fffffff7 fffffdfd ffffffff    ................
08400000:"Synchronous Abort" handler, esr 0x96000010

=> md 0x0a400000 10
0a400000: ff4fffff fffffdff efffffdf 7ffffdfb    ..O.............
0a400010: ffffffff 7fbffeff ff5fdfdf feff7fff    .........._.....
0a400020: fffff5ff df7fffbd bfbffff7 fffffe6f    ............o...
0a400030: f7bbeffe 5fbffffb 7ff7ff7f ffffdbdf    ......._........
=> md 0x0a3fffff 10
0a3fffff:"Synchronous Abort" handler, esr 0x96000010