Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Booting OPTEE on iMX7 #2513

Closed
tomajmich opened this issue Aug 30, 2018 · 21 comments
Closed

Booting OPTEE on iMX7 #2513

tomajmich opened this issue Aug 30, 2018 · 21 comments

Comments

@tomajmich
Copy link

Hi Guys,
I am trying to boot OPTEE and U-Boot on the Warp7 with iMX7 following the descriptions here:
https://github.com/bryanodonoghue/arm-trusted-firmware/blob/11650a714f8c7ed9dd9401648ee9fb4e9206ae24/docs/plat/warp7.rst

It seems that I get to the smc call in reset_primary in core/arch/arm/kernel/generic_entry_a32.S, but nothing else happens after that. This looks like the smc handler in ATF is missing or simply does not pass the entry point to BL33(U-Boot). Could you give me a hint on what to look next, what tests to perform or what could be the possible root cause?
The logs I am getting currently are as follows:

VERBOSE: 	OPTEE      0x9e000000-0xa0000000
VERBOSE: 	ATF/BL2    0x9df00000-0x9e000000
VERBOSE: 	SHRAM      0x9deff000-0x9df00000
VERBOSE: 	FIP        0x80000000-0x80100000
VERBOSE: 	DTB        0x83000000-0x83100000
VERBOSE: 	UBOOT/BL33 0x87800000-0x87900000
NOTICE:  BL2: v1.5(debug):v1.5-730-g01b4eb7-dirty
NOTICE:  BL2: Built : 09:39:36, Aug 30 2018
INFO:    BL2: Loading image id 4
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=4 at address 0x9e000000
INFO:    Image id=4 loaded: 0x9e000000 - 0x9e00001c
INFO:    OPTEE ep=0x9e000000
INFO:    OPTEE header info:
INFO:          magic=0x4554504f
INFO:          version=0x2
INFO:          arch=0x0
INFO:          flags=0x0
INFO:          nb_images=0x1
INFO:    BL2: Loading image id 23
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=23 at address 0x83000000
INFO:    Image id=23 loaded: 0x83000000 - 0x83009396
INFO:    BL2: Loading image id 21
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=21 at address 0x9e000000
INFO:    Image id=21 loaded: 0x9e000000 - 0x9e042400
INFO:    BL2: Skip loading image id 22
INFO:    BL2: Loading image id 5
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=5 at address 0x87800000
INFO:    Image id=5 loaded: 0x87800000 - 0x87865398
NOTICE:  BL2: Booting BL32
INFO:    Entry point address = 0x9e000000
INFO:    SPSR = 0x1d3
VERBOSE: Argument #0 = 0x9df0641c
VERBOSE: Argument #1 = 0x0
VERBOSE: Argument #2 = 0x83000000
VERBOSE: Argument #3 = 0x0
E/TC:0 console_init:120 TMIC init console
E/TC:0 mydebug:125 TMIC HERE
D/TC:0 add_phys_mem:526 TEE_SHMEM_START type NSEC_SHM 0x9fe00000 size 0x00200000
D/TC:0 add_phys_mem:526 TA_RAM_START type TA_RAM 0x9e100000 size 0x01d00000
D/TC:0 add_phys_mem:526 VCORE_UNPG_RW_PA type TEE_RAM_RW 0x9e042000 size 0x000be000
D/TC:0 add_phys_mem:526 VCORE_UNPG_RX_PA type TEE_RAM_RX 0x9e000000 size 0x00042000
D/TC:0 add_phys_mem:526 ROUNDDOWN(IRAM_S_BASE, CORE_MMU_DEVICE_SIZE) type TEE_COHERENT 0x00100000 size 0x00100000
D/TC:0 add_phys_mem:526 ROUNDDOWN(IRAM_BASE, CORE_MMU_DEVICE_SIZE) type TEE_COHERENT 0x00900000 size 0x00100000
D/TC:0 add_phys_mem:526 AIPS3_BASE type IO_SEC 0x30800000 size 0x00400000
D/TC:0 add_phys_mem:526 AIPS2_BASE type IO_SEC 0x30400000 size 0x00400000
D/TC:0 add_phys_mem:526 AIPS1_BASE type IO_SEC 0x30000000 size 0x00400000
D/TC:0 add_phys_mem:526 ANATOP_BASE type IO_SEC 0x30300000 size 0x00200000
D/TC:0 add_phys_mem:539 Physical mem map overlaps 0x30300000
D/TC:0 add_phys_mem:526 GIC_BASE type IO_SEC 0x31000000 size 0x00100000
D/TC:0 add_phys_mem:526 CONSOLE_UART_BASE type IO_NSEC 0x30800000 size 0x00200000
D/TC:0 verify_special_mem_areas:464 No NSEC DDR memory area defined
D/TC:0 add_va_space:565 type RES_VASPACE size 0x00a00000
D/TC:0 add_va_space:565 type SHM_VASPACE size 0x02000000
D/TC:0 dump_mmap_table:698 type IO_SEC       va 0x98600000..0x989fffff pa 0x30800000..0x30bfffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC       va 0x98a00000..0x98dfffff pa 0x30400000..0x307fffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC       va 0x98e00000..0x991fffff pa 0x30000000..0x303fffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_NSEC      va 0x99200000..0x993fffff pa 0x30800000..0x309fffff size 0x00200000 (pgdir)
D/TC:0 dump_mmap_table:698 type NSEC_SHM     va 0x99400000..0x995fffff pa 0x9fe00000..0x9fffffff size 0x00200000 (pgdir)
D/TC:0 dump_mmap_table:698 type TA_RAM       va 0x99600000..0x9b2fffff pa 0x9e100000..0x9fdfffff size 0x01d00000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_COHERENT va 0x9b300000..0x9b3fffff pa 0x00900000..0x009fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_COHERENT va 0x9b400000..0x9b4fffff pa 0x00100000..0x001fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type RES_VASPACE  va 0x9b500000..0x9befffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC       va 0x9bf00000..0x9bffffff pa 0x31000000..0x310fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type SHM_VASPACE  va 0x9c000000..0x9dffffff pa 0x00000000..0x01ffffff size 0x02000000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_RAM_RX   va 0x9e000000..0x9e041fff pa 0x9e000000..0x9e041fff size 0x00042000 (smallpg)
D/TC:0 dump_mmap_table:698 type TEE_RAM_RW   va 0x9e042000..0x9e0fffff pa 0x9e042000..0x9e0fffff size 0x000be000 (smallpg)
D/TC:0 core_mmu_alloc_l2:238 L2 table used: 1/4
I/TC:  
D/TC:0 init_canaries:164 #Stack canaries for stack_tmp[0] with top at 0x9e06e7f8
D/TC:0 init_canaries:164 watch *0x9e06e7fc
D/TC:0 init_canaries:165 #Stack canaries for stack_abt[0] with top at 0x9e06f038
D/TC:0 init_canaries:165 watch *0x9e06f03c
D/TC:0 init_canaries:167 #Stack canaries for stack_thread[0] with top at 0x9e071078
D/TC:0 init_canaries:167 watch *0x9e07107c
D/TC:0 init_canaries:167 #Stack canaries for stack_thread[1] with top at 0x9e0730b8
D/TC:0 init_canaries:167 watch *0x9e0730bc
E/TC:0 init_primary_helper:907 update complete
I/TC:  OP-TEE version: 3.2.0-dev #25 Thu Aug 30 07:14:46 UTC 2018 arm
D/TC:0 tee_ta_register_ta_store:534 Registering TA store: 'REE' (priority 10)
D/TC:0 tee_ta_register_ta_store:534 Registering TA store: 'Secure Storage TA' (priority 9)
D/TC:0 mobj_mapped_shm_init:559 Shared memory address range: 9c000000, 9e000000
E/TC:0 plat_rng_init:354 Warning: seeding RNG with zeroes
D/TC:0 plat_rng_init:355 TMIC D plat_rng_init
E/TC:0 plat_rng_init:356 TMIC E plat_rng_init
D/TC:0 imx_wdog_base:125 path: /soc/aips-bus@30000000/wdog@30280000
I/TC:  Initialized
D/TC:0 init_primary_helper:918 Primary CPU switching to normal world boot
E/TC:0 generic_boot_init_primary:967 init primary helper
@jbech-linaro
Copy link
Contributor

or simply does not pass the entry point to BL33(U-Boot)

There are a couple of E/... lines there, have you added them? If not, then it might be that OP-TEE in the end hasn't been fully initialized for some reason. Otherwise it looks like OP-TEE has been initialized. So in that case I'd check what is going on when TF-A is about do the handover, just as you've mentioned already (@OP-TEE/plat-imx might be in a better position to support here).

@tomajmich
Copy link
Author

tomajmich commented Aug 30, 2018

Thanks @jbech-linaro for the comment. Actually the only E line that comes originally from OPTEE is:

E/TC:0 plat_rng_init:354 Warning: seeding RNG with zeroes

the rest are my debug messages.

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

Hi @tomajmich.
Looking at the log - I'd wonder if there is something wrong with the version of u-boot you built ?

Here's how I make u-boot

make clean; make mrproper; make warp7_bl33_defconfig;make u-boot.imx arch=ARM CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- -j 5 KBUILD_VERBOSE=1

Here's how I make ATF

make realclean
rm -rf ${TEMP}
make CROSS_COMPILE=${CROSS_COMPILE} PLAT=warp7 ARCH=aarch32 ARM_ARCH_MAJOR=7 ARM_CORTEX_A7=yes DEBUG=1 LOG_LEVEL=50 V=1 AARCH32_SP=optee PLAT_WARP7_UART=1 all
make fiptool
tools/fiptool/fiptool create --tos-fw fiptool_images/tee-header_v2.bin --tos-fw-extra1 fiptool_images/tee-pager_v2.bin --tos-fw-extra2 fiptool_images/tee-pageable_v2.bin --nt-fw fiptool_images/u-boot.bin --hw-config fiptool_images/imx7s-warp.dtb warp7.fip
mkdir ${TEMP}
~/Development/mbl-u-boot/tools/mkimage -n u-boot.cfgout.warp7 -T imximage -e 0x9df00000 -d ./build/warp7/debug/bl2.bin ./build/warp7/debug/bl2.bin.imx > ${TEMP}/${BL2_IMX}.log

The load addresses look consistent with my own boot log so I'm guessing that its the hand-off to u-boot that has gone sideways.

Bootlog
_VERBOSE:        OPTEE      0x9e000000-0xa0000000
VERBOSE:        ATF/BL2    0x9df00000-0x9e000000
VERBOSE:        SHRAM      0x9deff000-0x9df00000
VERBOSE:        FIP        0x80000000-0x80100000
VERBOSE:        DTB        0x83000000-0x83100000
VERBOSE:        UBOOT/BL33 0x87800000-0x87900000
NOTICE:  BL2: v1.5(release):ac2ad596
NOTICE:  BL2: Built : 00:54:25, Aug 28 2018
INFO:    BL2: Loading image id 4
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=4 at address 0x9e000000
INFO:    Image id=4 loaded: 0x9e000000 - 0x9e00001c
INFO:    OPTEE ep=0x9e000000
INFO:    OPTEE header info:
INFO:          magic=0x4554504f
INFO:          version=0x2
INFO:          arch=0x0
INFO:          flags=0x0
INFO:          nb_images=0x1
INFO:    BL2: Loading image id 23
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=23 at address 0x83000000
INFO:    Image id=23 loaded: 0x83000000 - 0x83006b4e
INFO:    BL2: Loading image id 21
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=21 at address 0x9e000000
INFO:    Image id=21 loaded: 0x9e000000 - 0x9e040180
INFO:    BL2: Skip loading image id 22
INFO:    BL2: Loading image id 5
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO:    Loading image id=5 at address 0x87800000
INFO:    Image id=5 loaded: 0x87800000 - 0x878666f0
NOTICE:  BL2: Booting BL32
INFO:    Entry point address = 0x9e000000
INFO:    SPSR = 0x1d3
VERBOSE: Argument #0 = 0x9df0641c
VERBOSE: Argument #1 = 0x0
VERBOSE: Argument #2 = 0x83000000
VERBOSE: Argument #3 = 0x0


U-Boot 2018.09-rc2+fslc+g3bdb88cb25 (Aug 28 2018 - 00:21:23 +0000)

CPU:   Freescale i.MX7S rev1.2 800 MHz (running at 792 MHz)
CPU:   Extended Commercial temperature grade (-20C to 105C) at 48C
Reset cause: POR
Board: WARP7 in secure mode OPTEE DRAM 0x9e000000-0xa0000000
I2C:   ready
DRAM:  480 MiB
PMIC: PFUZE3000 DEV_ID=0x30 REV_ID=0x11
MMC:   FSL_SDHC: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment

In:    serial
Out:   serial
Err:   serial
SEC0: RNG instantiated
Net:   usb_ether
Warning: usb_ether (eth0) using random MAC address - d2:4f:8a:1a:7c:8c

Hit any key to stop autoboot:  0 
switch to partitions #0, OK
mmc0(part 0) is current device
switch to partitions #0, OK
mmc0(part 0) is current device
1213 bytes read in 6 ms (197.3 KiB/s)
Running bootscript from mmc ...
Executing script at 80800000
11168896 bytes read in 163 ms (65.3 MiB/s)
Booting secure Linux from mmc ...
Kernel image @ 0x80800000 [ 0x000000 - 0xaa6c80 ]
Flattened Device Tree blob at 83000000
   Booting using the fdt blob at 0x83000000
   Using Device Tree in place at 83000000, end 83009cef_

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.37-fslc+g8f2c1f84d929 (oe-user@oe-host) (gcc version 8.2.0 (GCC)) 1 SMP Tue Aug 28 00:11:57 UTC 2018

@tomajmich
Copy link
Author

Hi @bryanodonoghue, thanks for looking into the logs.
Seems that how I was building it doesn't really differ from what you have.

The script I use for generating the ATF and fip:

cd u-boot
make clean
make mrproper
make warp7_bl33_defconfig
make u-boot.imx ARCH=arm CROSS_COMPILE=/opt/Tools/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- -j12 KBUILD_VERBOSE=1
cd ..

#TF-A
cd arm-trusted-firmware
make realclean
rm -rf temp
make PLAT=warp7 CROSS_COMPILE=/opt/Tools/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- ARCH=aarch32 ARM_ARCH_MAJOR=7 ARM_CORTEX_A7=yes AARCH32_SP=optee LOG_LEVEL=50 PLAT_WARP_UART=1 DEBUG=1 V=1 all -j12
make fiptool
mkdir temp
../u-boot/tools/mkimage -n ../u-boot/u-boot.cfgout -T imximage -e 0x9df00000 -d ./build/warp7/debug/bl2.bin ../bl2.bin.imx > temp/bl2_imx.log
cd ..

#OP-TEE
cd optee_os
make PLATFORM=imx PLATFORM_FLAVOR=mx7swarp7 ARCH=arm CROSS_COMPILE=/opt/Tools/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- CFG_PAGEABLE_ADDR=0 CFG_DT_ADDR=0x83000000 CFG_TEE_CORE_NB_CORE=1 CFG_TRACE_LEVEL=4 -j12
cd ..

#fiptool_images
rm fiptool_images/*
cp u-boot/u-boot.bin fiptool_images/
cp optee_os/out/arm-plat-imx/core/tee-header_v2.bin fiptool_images/
cp optee_os/out/arm-plat-imx/core/tee-pager_v2.bin fiptool_images/
cp optee_os/out/arm-plat-imx/core/tee-pageable_v2.bin fiptool_images/
cp linux-fslc/arch/arm/boot/dts/imx7s-warp.dtb fiptool_images/
arm-trusted-firmware/tools/fiptool/fiptool create --tos-fw fiptool_images/tee-header_v2.bin --tos-fw-extra1 fiptool_images/tee-pager_v2.bin --tos-fw-extra2 fiptool_images/tee-pageable_v2.bin --nt-fw fiptool_images/u-boot.bin --hw-config fiptool_images/imx7s-warp.dtb warp7.fip

Now, when I try to boot the only difference I see in the log is this line:

E/TC:0 plat_rng_init:354 Warning: seeding RNG with zeroes

which doesn't show up in your log. The message says "warning" but the message itself is an error("E/TC"). So maybe there is something wrong with the RNG initialization.
Would it be possible for you to send me a bl2.bin.imx and fip so I can exclude any hardware issue?

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

OK.
Let's try this OPTEE version

SRCREV="0ab9388c0d553a6bb5ae04e41b38ba40cf0474bf"
SRC_URI="git://git.linaro.org/landing-teams/working/mbl/optee_os.git"

If that works then there's a bug in tip-of-tree we need to fix.

@tomajmich
Copy link
Author

This is what I get when trying to build optee:

make: *** No rule to make target 'core/arch/arm/include/mm/generic_ram_layout.h', needed by 'out/arm-plat-imx/core/include/generated/.asm-defines.s'.  Stop.
make: *** Waiting for unfinished jobs....
  CHK     out/arm-plat-imx/conf.mk

@tomajmich
Copy link
Author

Hmm, I removed the whole directory, did a fresh clone and this time it built correctly.
I don't see the RNG error anymore, but I don't see U-Boot either...

@tomajmich
Copy link
Author

I've done the same fresh clone thing with master and the "seeding RNG with zeroes" is there, so this seems like a regression.

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

I just built again from source - it works.
Let's just do it step by step.

uboot bleeding edge + bl33 patches

git.linaro.org/landing-teams/working/mbl/u-boot.git (fetch)
git checkout -b atf-build 37f18beff1cf12b4800b2b4bdf9eeeb30bcc5214
make clean; make mrproper; make warp7_bl33_defconfig;make u-boot.imx arch=ARM CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- -j 5 KBUILD_VERBOSE=1
Remember the defconfig needs to be warp7_bl33_defconfig using the stock u-boot will break

optee 3.1.0

commit 0ab9388 (HEAD -> linaro-master, tag: 3.1.0, linaro/master, optee-3.1.0)
git checkout -b atf-build 0ab9388c0d553a6bb5ae04e41b38ba40cf0474bf
make CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- PLATFORM=imx PLATFORM_FLAVOR=mx7swarp7 ARCH=arm CFG_PAGEABLE_ADDR=0 CFG_DT_ADDR=0x83000000 DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 -j 8

ATF

github.com:bryanodonoghue/arm-trusted-firmware.git
git checkout -b atf-build 01b4eb7481c9027b4a3c22475f20b5929cc166b7

copy u-boot to a staging dir
cp ../../mbl-u-boot/u-boot.bin fiptool_images/u-boot.bin

copy tee to same staging dir
cp /run/media/deckard/73c0aaf6-fc7a-4539-8265-f562d8de8e18/development/linaro/mbl-optee_os/out/arm-plat-imx/core/*tee* fiptool_images

Build unified ATF + FIP
make CROSS_COMPILE=${CROSS_COMPILE} PLAT=warp7 ARCH=aarch32 ARM_ARCH_MAJOR=7 ARM_CORTEX_A7=yes DEBUG=1 LOG_LEVEL=50 V=1 AARCH32_SP=optee PLAT_WARP7_UART=1 all
make fiptool
tools/fiptool/fiptool create --tos-fw fiptool_images/tee-header_v2.bin --tos-fw-extra1 fiptool_images/tee-pager_v2.bin --tos-fw-extra2 fiptool_images/tee-pageable_v2.bin --nt-fw fiptool_images/u-boot.bin --hw-config fiptool_images/imx7s-warp.dtb warp7.fip

Generate IMX header

~/Development/mbl-u-boot/tools/mkimage -n u-boot.cfgout.warp7 -T imximage -e 0x9df00000 -d ./build/warp7/debug/bl2.bin ./build/warp7/debug/bl2.bin.imx > ${TEMP}/${BL2_IMX}.log
Apologies I omitted this step earlier.

Burn to board

./flash.sh /dev/disk/by-id/usb-Linux_UMS_disk_0_WaRP7-0xf42400d3000000d4-0\:0 flashing bl2.bin.imx-signed to /dev/disk/by-id/usb-Linux_UMS_disk_0_WaRP7-0xf42400d3000000d4-0:0

Here's my flash.sh

#!/bin/bash
echo "flashing bl2.bin.imx-signed to $1"
sudo dd if=bl2.bin.imx-signed of=$1 bs=512 seek=2 conv=notrunc
echo "flashing warp7.fip to $1"
sudo dd if=./warp7.fip of=$1 bs=512 seek=2048 conv=notrunc
sudo umount $1*

Output:
VERBOSE: OPTEE 0x9e000000-0xa0000000
VERBOSE: ATF/BL2 0x9df00000-0x9e000000
VERBOSE: SHRAM 0x9deff000-0x9df00000
VERBOSE: FIP 0x80000000-0x80100000
VERBOSE: DTB 0x83000000-0x83100000
VERBOSE: UBOOT/BL33 0x87800000-0x87900000
NOTICE: BL2: v1.5(debug):v1.5-730-g01b4eb748
NOTICE: BL2: Built : 11:24:36, Aug 30 2018
INFO: BL2: Loading image id 4
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO: Loading image id=4 at address 0x9e000000
INFO: Image id=4 loaded: 0x9e000000 - 0x9e00001c
INFO: OPTEE ep=0x9e000000
INFO: OPTEE header info:
INFO: magic=0x4554504f
INFO: version=0x2
INFO: arch=0x0
INFO: flags=0x0
INFO: nb_images=0x1
INFO: BL2: Loading image id 23
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO: Loading image id=23 at address 0x83000000
INFO: Image id=23 loaded: 0x83000000 - 0x83006b4e
INFO: BL2: Loading image id 21
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO: Loading image id=21 at address 0x9e000000
INFO: Image id=21 loaded: 0x9e000000 - 0x9e042180
INFO: BL2: Skip loading image id 22
INFO: BL2: Loading image id 5
VERBOSE: FIP header looks OK.
VERBOSE: Using FIP
INFO: Loading image id=5 at address 0x87800000
INFO: Image id=5 loaded: 0x87800000 - 0x878668e8
NOTICE: BL2: Booting BL32
INFO: Entry point address = 0x9e000000
INFO: SPSR = 0x1d3
VERBOSE: Argument #0 = 0x9df0641c
VERBOSE: Argument #1 = 0x0
VERBOSE: Argument #2 = 0x83000000
VERBOSE: Argument #3 = 0x0
D/TC:0 add_phys_mem:526 CFG_SHMEM_START type NSEC_SHM 0x9fe00000 size 0x00200000
D/TC:0 add_phys_mem:526 CFG_TA_RAM_START type TA_RAM 0x9e100000 size 0x01d00000
D/TC:0 add_phys_mem:526 VCORE_UNPG_RW_PA type TEE_RAM_RW 0x9e040000 size 0x000c0000
D/TC:0 add_phys_mem:526 VCORE_UNPG_RX_PA type TEE_RAM_RX 0x9e000000 size 0x00040000
D/TC:0 add_phys_mem:526 ROUNDDOWN(IRAM_S_BASE, CORE_MMU_DEVICE_SIZE) type TEE_COHERENT 0x00100000 size 0x00100000
D/TC:0 add_phys_mem:526 ROUNDDOWN(IRAM_BASE, CORE_MMU_DEVICE_SIZE) type TEE_COHERENT 0x00900000 size 0x00100000
D/TC:0 add_phys_mem:526 AIPS3_BASE type IO_SEC 0x30800000 size 0x00400000
D/TC:0 add_phys_mem:526 AIPS2_BASE type IO_SEC 0x30400000 size 0x00400000
D/TC:0 add_phys_mem:526 AIPS1_BASE type IO_SEC 0x30000000 size 0x00400000
D/TC:0 add_phys_mem:526 ANATOP_BASE type IO_SEC 0x30300000 size 0x00200000
D/TC:0 add_phys_mem:539 Physical mem map overlaps 0x30300000
D/TC:0 add_phys_mem:526 GIC_BASE type IO_SEC 0x31000000 size 0x00100000
D/TC:0 add_phys_mem:526 CONSOLE_UART_BASE type IO_NSEC 0x30800000 size 0x00200000
D/TC:0 verify_special_mem_areas:464 No NSEC DDR memory area defined
D/TC:0 add_va_space:565 type RES_VASPACE size 0x00a00000
D/TC:0 add_va_space:565 type SHM_VASPACE size 0x02000000
D/TC:0 dump_mmap_table:698 type IO_SEC va 0x98600000..0x989fffff pa 0x30800000..0x30bfffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC va 0x98a00000..0x98dfffff pa 0x30400000..0x307fffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC va 0x98e00000..0x991fffff pa 0x30000000..0x303fffff size 0x00400000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_NSEC va 0x99200000..0x993fffff pa 0x30800000..0x309fffff size 0x00200000 (pgdir)
D/TC:0 dump_mmap_table:698 type NSEC_SHM va 0x99400000..0x995fffff pa 0x9fe00000..0x9fffffff size 0x00200000 (pgdir)
D/TC:0 dump_mmap_table:698 type TA_RAM va 0x99600000..0x9b2fffff pa 0x9e100000..0x9fdfffff size 0x01d00000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_COHERENT va 0x9b300000..0x9b3fffff pa 0x00900000..0x009fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_COHERENT va 0x9b400000..0x9b4fffff pa 0x00100000..0x001fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type RES_VASPACE va 0x9b500000..0x9befffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir)
D/TC:0 dump_mmap_table:698 type IO_SEC va 0x9bf00000..0x9bffffff pa 0x31000000..0x310fffff size 0x00100000 (pgdir)
D/TC:0 dump_mmap_table:698 type SHM_VASPACE va 0x9c000000..0x9dffffff pa 0x00000000..0x01ffffff size 0x02000000 (pgdir)
D/TC:0 dump_mmap_table:698 type TEE_RAM_RX va 0x9e000000..0x9e03ffff pa 0x9e000000..0x9e03ffff size 0x00040000 (smallpg)
D/TC:0 dump_mmap_table:698 type TEE_RAM_RW va 0x9e040000..0x9e0fffff pa 0x9e040000..0x9e0fffff size 0x000c0000 (smallpg)
D/TC:0 core_mmu_alloc_l2:261 L2 table used: 1/4
I/TC:
D/TC:0 init_canaries:185 #Stack canaries for stack_tmp[0] with top at 0x9e06e7f8
D/TC:0 init_canaries:185 watch *0x9e06e7fc
D/TC:0 init_canaries:186 #Stack canaries for stack_abt[0] with top at 0x9e06f038
D/TC:0 init_canaries:186 watch *0x9e06f03c
D/TC:0 init_canaries:188 #Stack canaries for stack_thread[0] with top at 0x9e071078
D/TC:0 init_canaries:188 watch *0x9e07107c
D/TC:0 init_canaries:188 #Stack canaries for stack_thread[1] with top at 0x9e0730b8
D/TC:0 init_canaries:188 watch *0x9e0730bc
I/TC: OP-TEE version: 3.1.0-dev #1 Thu Aug 30 10:15:50 UTC 2018 arm
D/TC:0 tee_ta_register_ta_store:604 Registering TA store: 'REE' (priority 10)
D/TC:0 tee_ta_register_ta_store:604 Registering TA store: 'Secure Storage TA' (priority 9)
D/TC:0 mobj_mapped_shm_init:559 Shared memory address range: 9c000000, 9e000000
D/TC:0 imx_wdog_init:122 path: /soc/aips-bus@30000000/wdog@30280000
I/TC: Initialized
D/TC:0 init_primary_helper:885 Primary CPU switching to normal world boot

U-Boot 2018.09-rc2-00077-g3bdb88cb25 (Aug 24 2018 - 18:28:54 +0100)

CPU: Freescale i.MX7S rev1.2 800 MHz (running at 792 MHz)
CPU: Extended Commercial temperature grade (-20C to 105C) at 48C
Reset cause: POR
Board: WARP7 in secure mode OPTEE DRAM 0x9e000000-0xa0000000
I2C: ready
DRAM: 480 MiB
PMIC: PFUZE3000 DEV_ID=0x30 REV_ID=0x11
MMC: FSL_SDHC: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment

In: serial
Out: serial
Err: serial
SEC0: RNG instantiated
Net: usb_ether
Warning: usb_ether (eth0) using random MAC address - 42:4d:60:e0:fc:60

Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0(part 0) is current device
switch to partitions #0, OK
mmc0(part 0) is current device
1213 bytes read in 7 ms (168.9 KiB/s)
Running bootscript from mmc ...

Executing script at 80800000

11168896 bytes read in 163 ms (65.3 MiB/s)
Booting secure Linux from mmc ...
Kernel image @ 0x80800000 [ 0x000000 - 0xaa6c80 ]

Flattened Device Tree blob at 83000000

Booting using the fdt blob at 0x83000000
Using Device Tree in place at 83000000, end 83009cef

Starting kernel ...

[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.37-fslc+g8f2c1f84d929 (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Tue Aug 28 00:11:57 UTC 2018
[ 0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[ 0.000000] OF: fdt: Machine model: Warp i.MX7 Board

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

I'd double and then triple check that version of u-boot you are using, it looks like the fault to me.

@tomajmich
Copy link
Author

I am sure U-Boot and ATF are compiled from the SHA you used. I really don't know what is different here.
The DTB I use is from https://github.com/WaRP7/linux-fslc.git branch: 4.1-1.0.x-imx
I don't flash the bl2.bin.imx signed, is the signing step necessary for debugging?
Again, would it be possible to send me a working bl2 and fip?

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

Signing is not a necessary step no.
I guess the DTB must be the issue - we are working with kernel 4.14+.
Here's a set of binaries for you : https://drive.google.com/drive/folders/1O806P6um4ErPfj0hhggW2LFkk5KJ5dzx?usp=sharing

For reference this is our kernel

SRCREV = "8f2c1f84d9292abf8c865db64aff952d0c7494f5"
KBUILD_DEFCONFIG_imx7s-warp-mbl ?= "warp7_mbl_defconfig"
FILESEXTRAPATHS_prepend:="${THISDIR}/files:"
SRC_URI = "git://git.linaro.org/landing-teams/working/mbl/linux.git;

And thanks, I need to update the ATF documentation to call out a recent kernel version... I'll also look into that stock 4.1 from NXP ... the DTB that goes with it probably is the blockage

@tomajmich
Copy link
Author

Uff, the binaries you sent do work on my board, so this must be the DTB thing.
Thanks a lot @bryanodonoghue !!!

@0xB0D
Copy link
Contributor

0xB0D commented Aug 30, 2018

np

@tomajmich
Copy link
Author

Hi @bryanodonoghue,
just wanted to let you know that the issue might not necessarily be related to the DTB.
As an experiment I used all the files that I tried initially following the original instructions and the only file I used from the binaries that you shared on gdrive was the tee_pager_v2.bin and guess what, it worked.
Any idea why building optee with:

commit 0ab9388 (HEAD -> linaro-master, tag: 3.1.0, linaro/master, optee-3.1.0)
git checkout -b atf-build 0ab9388c0d553a6bb5ae04e41b38ba40cf0474bf
make CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- PLATFORM=imx PLATFORM_FLAVOR=mx7swarp7 ARCH=arm CFG_PAGEABLE_ADDR=0 CFG_DT_ADDR=0x83000000 DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 -j 8

would give me a different tee_pager_v2.bin than the one you shared?

@0xB0D
Copy link
Contributor

0xB0D commented Sep 3, 2018

You have
make CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- PLATFORM=imx PLATFORM_FLAVOR=mx7swarp7 ARCH=arm CFG_PAGEABLE_ADDR=0 CFG_DT_ADDR=0x83000000 DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 -j 8

I have
make CROSS_COMPILE=/opt/linaro/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf- PLATFORM=imx PLATFORM_FLAVOR=mx7swarp7 ARCH=arm CFG_PAGEABLE_ADDR=0 CFG_DT_ADDR=0x83000000 DEBUG=y CFG_TEE_CORE_LOG_LEVEL=4 -j 8

The only thing that should change is the string on the build date.

@tomajmich
Copy link
Author

Hi @bryanodonoghue ,
sorry for bugging you, but there is really something strange going on.
I used a clean ubuntu docker image and built optee from the exact commit you did(0ab9388) and used the following toolchain:(https://releases.linaro.org/components/toolchain/binaries/7.2-2017.11/arm-linux-gnueabihf/).
I compared the tee.dmp files from the build on my host and docker and didn't find any difference, but when I compare it with your tee.dmp I do see some differences.
I have uploaded to gdrive the optee_os I built to prove that I am using the correct repository and commit:
https://drive.google.com/open?id=1NLYXR-G9uy_3hLxHRa3aW9bpxR5RLYac
I am really curious what is making the difference and keeps me from compiling a correct tee.elf. If it's not related to the toolchain then maybe to some environment differences on the host (but then again, building on a clean ubuntu docker didn't work either)...
I would appreciate any suggestions...

@0xB0D
Copy link
Contributor

0xB0D commented Sep 4, 2018

Its just occured to me that a colleague of mine changed the NS_ENTRY point and I have this uncommited change in my reference tree

git diff core/arch/arm/plat-imx/conf.mk
diff --git a/core/arch/arm/plat-imx/conf.mk b/core/arch/arm/plat-imx/conf.mk
index 5c9f04df..eaaabee5 100644
--- a/core/arch/arm/plat-imx/conf.mk
+++ b/core/arch/arm/plat-imx/conf.mk
@@ -43,7 +43,7 @@ endif
ifneq (,$(filter $(PLATFORM_FLAVOR),mx7swarp7))
CFG_DDR_SIZE ?= 0x20000000
CFG_DT ?= y
-CFG_NS_ENTRY_ADDR ?= 0x80800000
+CFG_NS_ENTRY_ADDR ?= 0x87800000
CFG_PSCI_ARM32 ?= y
CFG_TEE_CORE_NB_CORE ?= 1
endif

I'll make a PR for this change since it shouldn't affect any other project ...

There's no way you'll be making a TEE that can hand-over to bl33 u-boot without this change.

@0xB0D
Copy link
Contributor

0xB0D commented Sep 4, 2018

as a matter of fact in the interregnum I'll update the ATF instructions to point to a tree in the landing team repo with this change pushed...

Thanks for spotting this @tomajmich

@0xB0D
Copy link
Contributor

0xB0D commented Sep 4, 2018

BTW the reason the NS_ENTRY address has to change is that the bootflow

BootROM -> ATF -> OPTEE -> u-boot -> Kernel
has a conflict with the original version
BootROM -> u-boot -> OPTEE -> Kernel

But either way yeah - the NS_ENTRY_ADDR is certainly wrong in the image you've built :(

@tomajmich
Copy link
Author

Yes, that was it!!!
Thanks a lot!
I think it's time to close the subject.
Regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants