concurrent ta

[neil1899]

hi,all
when i run xtest 1013, in order to test concurrent: when i use the commit of:
80c558e
it will cause my test failed，l->curr_thread == -1 failed
because on my platform such as sun-xi, i don't config GENERIC_BOOT.

and when i run my test.sh to test the concurrent stability, it failed. log is

• XTEST_TEE_1013 Test concurency with concurrent TA
  o XTEST_TEE_1013.1 Busy loop with 3 parallel threads
  Max concurrency 1
  XTEST_TEE_1013.1 OK
  o XTEST_TEE_1013.2 Hashing with 3 parallel threads
  [39440.278095@1] misc opteearmtz00: Can't find shm for 000000000710d000
  [39440.278843@1] BUG: failure at /mnt/fileroot/wei.nie/android-l/vendor/amlogic/tdk-1.0.1/linuxdriver/core/tee_supp_com.c:221/tee_supp_write()!
  [39440.291347@1] Kernel panic - not syncing: BUG!
  [39440.295713@1] CPU: 1 PID: 3065 Comm: tee-supplicant Tainted: P O 3.14.29-gfd3994a-dirty Update Notice.md #1
  [39440.304765@1] Call trace:
  [39440.307363@1] [] dump_backtrace+0x0/0x144
  [39440.312874@1] [] show_stack+0x1c/0x28
  [39440.318050@1] [] dump_stack+0x74/0xb8
  [39440.323223@1] [] panic+0xe8/0x20c
  [39440.328060@1] [] tee_supp_write+0x1d8/0x218 [optee]
  [39440.334438@1] [] vfs_write+0xac/0x198
  [39440.339611@1] [] SyS_write+0x50/0xb0
  [39440.344700@0] CPU0: stopping
  [39440.347547@0] CPU: 0 PID: 0 Comm: swapper/0 Tainted: P O 3.14.29-gfd3994a-dirty Update Notice.md #1
  [39440.355910@0] Call trace:

thanks in advance!

[jenswi-linaro]

The l->curr_thread == -1 error is usually due to some function expecting to be called from a thread, but instead called from for instance the temp stack.

Best way forward is probably to switch to GENERIC_BOOT for sun-xi, patches are welcome!

As for the kernel panic, I don't know.

[neil1899]

hi，jenswi：
TEE test application started with device [opteearmtz00]
#####[ 47.551245@1] misc opteearmtz00: Can't find shm for 0000000007114000
[ 47.557492@1] BUG: failure at /mnt/fileroot/wei.nie/android-l/vendor/amlogic/tdk-1.0.1/linuxdriver/core/tee_supp_com.c:221/tee_supp_write()!
[ 47.569911@1] Kernel panic - not syncing: BUG!
in the function of tee_supp_write
BUG_ON(!shm); the shm is null.

when i run my script
while [ 1 ]
do
xtest 1013
done
like this.

[pascal-brand38]

I have the same kind of issue. I am investigating it.

[pascal-brand38]

Could you check with #594

[pascal-brand38]

#594 has been abandonned. Please test OP-TEE/optee_linuxdriver#37 instead

[neil1899]

thanks so much! i have test it, it's ok now.

[neil1899]

sorry，
use my script to test the patch, it's failed.
but it's so long time.

TEE test application started with device [opteearmtz00]
######################################################

XTEST_TEE_TEST

##########[ 7735.919348@0] Unable to handle kernel NULL pointer dereference at virtual address 00000130
[ 7735.928280@1] pgd = ffffffc01c658000
[ 7735.930207@1] [00000130] *pgd=0000000000000000
[ 7735.934527@1] Internal error: Oops: 96000045 [#1] PREEMPT SMP
[ 7735.940216@1] Modules linked in: optee_armtz(O) optee(O) dwc_otg mali(O) aml_nftl_dev(PO) [last unloaded: optee]
[ 7735.950306@1] CPU: 1 PID: 27859 Comm: tee_xtest Tainted: P O 3.14.29-gfd3994a-dirty #1
[ 7735.959017@1] task: ffffffc01cb35000 ti: ffffffc0201cc000 task.ti: ffffffc0201cc000
[ 7735.966613@1] PC is at mutex_unlock+0x18/0x58
[ 7735.970924@1] LR is at tee_shm_free_from_rpc+0x44/0x88 [optee]
[ 7735.976696@1] pc : [] lr : [] pstate: 60000145
[ 7735.984198@1] sp : ffffffc0201cfa50
[ 7735.987648@1] x29: ffffffc0201cfa50 x28: 0000000000000000
[ 7735.993082@1] x27: ffffff8002087000 x26: 00000000ffff5701
[ 7735.998516@1] x25: 0000000032000003 x24: ffffff800208800c
[ 7736.003949@1] x23: 0000000000000008 x22: ffffffc0201cfb38
[ 7736.009383@1] x21: ffffffc0221cd018 x20: ffffffc013d6fa18
[ 7736.014817@1] x19: 0000000000000118 x18: 0000000000000000
[ 7736.020250@1] x17: 0000000000000000 x16: 0000000000000000
[ 7736.025685@1] x15: 0000000000000000 x14: 0000000000000000
[ 7736.031118@1] x13: 0000000000000000 x12: 0000000000000000
[ 7736.036552@1] x11: 0000000000000000 x10: 0000000000000000
[ 7736.041986@1] x9 : 0000000000000000 x8 : 0000000000000000
[ 7736.047419@1] x7 : 0000000000000000 x6 : ffffffc02257a000
[ 7736.052853@1] x5 : ffffffc02257a000 x4 : 0000000000000140
[ 7736.058287@1] x3 : 0000000000000001 x2 : 0000000038307000
[ 7736.063720@1] x1 : 0000000038307000 x0 : ffffffbffc0502ec

[pascal-brand38]

OP-TEE/optee_linuxdriver#38 may solve it. Could you have a look and test?

[neil1899]

yep， i have test it. thanks so much!

[neil1899]

hi，@pascal-brand-st-dev
It crashs either, it only runs 12 hours. but if i don't apply support concurrent ta patch and don't run 1013(test concurrent case), the optee_test runs 72 hours and don't crash.

the log is ：

• XTEST_TEE_1012 Test Single Instance Multi Session features with SIMS TA
  o XTEST_TEE_1012.1 Single Instance Multi Session
  [55917.915968@0] Unable to handle kernel NULL pointer dereference at virtual address 00000000
  [55917.918774@0] pgd = ffffffc01b963000
  [55917.922400@0] [00000000] *pgd=0000000000000000
  [55917.926902@1] Internal error: Oops: 96000005 [Update Notice.md #1] PREEMPT SMP
  [55917.932214@1] Modules linked in: dwc_otg aml_thermal(O) optee_armtz(O) optee(O) mali(O) aml_nftl_dev(PO)
  [55917.941616@1] CPU: 1 PID: 5150 Comm: tee_xtest Tainted: P W O 3.14.29-ge81712d GitHub usage documentation #5
  [55917.949720@1] task: ffffffc013fbe000 ti: ffffffc01b578000 task.ti: ffffffc01b578000
  [55917.957315@1] PC is at tee_session_close_and_destroy+0xc4/0xe4 [optee]
  [55917.963779@1] LR is at tee_session_close_and_destroy+0xc0/0xe4 [optee]
  [55917.970247@1] pc : [] lr : [] pstate: a0000145
  [55917.977748@1] sp : ffffffc01b57bdd0
  [55917.981198@1] x29: ffffffc01b57bdd0 x28: ffffffc01b578000
  [55917.986632@1] x27: ffffffc00108d000 x26: 0000000000000006
  [55917.992066@1] x25: 0000000000000180 x24: ffffffc01b97db10
  [55917.997499@1] x23: ffffffc034c2a3e0 x22: 0000000000000000
  [55918.002933@1] x21: ffffffc00eeb5218 x20: ffffffc021d8d818
  [55918.008367@1] x19: ffffffc01b577e58 x18: 0000000000000000
  [55918.013801@1] x17: 0000000000000000 x16: 0000000000000000
  [55918.019234@1] x15: 0000000000000000 x14: 0000000000000000
  [55918.024668@1] x13: 0000000000000000 x12: 0000000000000000
  [55918.030102@1] x11: 0000000000000000 x10: 0000000000000000
  [55918.035536@1] x9 : 0000000000000000 x8 : 0000000000000000
  [55918.040969@1] x7 : 0000000000000000 x6 : ffffffc01b577b00
  [55918.046403@1] x5 : ffffffc01b577b00 x4 : 0000000000000140
  [55918.051837@1] x3 : 0000000000000001 x2 : ffffffc021d8d8f0
  [55918.057270@1] x1 : 0000000000000000 x0 : 0000000000000000

[neil1899]

hi @pascal-brand-st-dev
could you reproduct the crash problem?

[pascal-brand38]

No, I cannot reproduce it.
I have run for 16 hours, without issues:

while [ 1 ]; do xtest 1013; done

[pascal-brand38]

@neil1899 Could you check if you can still reproduce the issue when applying OP-TEE/optee_linuxdriver#39 ?

[neil1899]

ok, i will apply optee_linuxdriver#37 & 38 & 39, then i do the test.

and before i apply optee_linuxdriver#37, 38. i run for 12 hours, with issues:
while [ 1 ]; do xtest; done(ont only run 1013)

[neil1899]

hi, @pascal-brand-st-dev
I think it's ok now.

thanks so much!

[neil1899]

Hi, @pascal-brand-st-dev
I have a bad news:
when i update optee os to c042fbe this commit.

you konw, the performance of optee enhances highly, optee test runs so quickly. when it runs 48 hours,
i repeate the issue again, before that, i run 72 hours ok.

the log is
log.txt

[neil1899]

i found out a commit OP-TEE/optee_client@dfb74af
i have tried it.