Mind Chasers Inc.
Mind Chasers Inc.

Methods to Accelerate Linux Kernel Development using Yocto Project and the NXP MCIMX8M-EVK

A discussion of techniques to accelerate development with the Linux Kernel and related when working with the Yocto Project and an NXP i.MX EVK system.

Overview

We recently began working with the NXP MCIMX8M-EVK evaluation kit for the i.MX 8MQuad processor. It's certainly an amazing system, and there is plenty of documentation. However, we found some of the Yocto Linux related documentation stale and confusing, so we are posting this article regarding our steps to accelerate building, booting, and debugging the Linux kernel, device tree, and root filesystem.

If you're new to Yocto or the MCIMX8M-EVK, you may want to first review Build, Install, and Develop from Source with Yocto Linux and the NXP MCIMX8M-EVK.

Listed below are some important data points for working with the NXP MCIMX8M-EVK:

  • i.MX 8MQuad is powered by a quad core 1.5 GHz ARM CORTEX-A53 with an auxiliary ARM M4
  • We are working with the NXP release "L4.14.98-2.0.0_ga". This was developed using Yocto Project 2.5 (Sumo).
  • Sumo was released April 2018. Yocto Project is currently developing a 3.0 release.
  • The very important meta-fsl-bsp-release Git repo is hosted at https://source.codeaurora.org/external/imx/meta-fsl-bsp-release. This is referred to as the "i.MX release layer" and "i.MX BSP release layer".
  • The Yocto Machine (for conf files): imx8mqevk
  • There is an EVKB and EVK. We are using the EVKB, and this has an upgrade to the WIFI over the EVK. The PCB design files currently posted on nxp.com are not for our EVKB.
EVK Board
NXP MCIMX8M-EVK

In this article, we discuss the following steps in order:

  1. Create a toolchain for use outside the Yocto Build Environment
  2. Build the NXP linux-imx kernel using the external SDK / toolchain
  3. Use U-boot to load the kernel, device tree, and rootfs using NFS
  4. Create and share kernel patches via email to speed collaboration

Create a toolchain for building the Linux kernel outside the Yocto project

This section assumes that you have already successfully built the NXP BSP using the Yocto Project. If you haven't, please review the steps detailed in Build, Install, and Develop from Source with Yocto Linux and the NXP MCIMX8M-EVK.

Build the GNU toolchain / SDK for the NXP BSP as shown below. Note that our build machine is an Ubuntu Linux 18.04 Intel PC.

$ cd /opt/nxp/
$ source sources/poky/oe-init-build-env bld-wayland/

$ bitbake fsl-image-qt5-validation-imx -c populate_sdk

$ ls /opt/nxp/bld-wayland/tmp/work
aarch64-mx8m-poky-linux  all-poky-linux        sdk-provides-dummy-nativesdk-pokysdk-linux  x86_64-linux
aarch64-poky-linux       imx8mqevk-poky-linux  sdk-provides-dummy-target-poky-linux        x86_64-nativesdk-pokysdk-linux

$ cd /opt/nxp/bld-wayland/tmp/deploy/sdk
$ ls
...
fsl-imx-wayland-glibc-x86_64-fsl-image-qt5-validation-imx-aarch64-toolchain-4.14-sumo.sh
...

./fsl*.sh

NXP i.MX Release Distro SDK installer version 4.14-sumo
=======================================================
Enter target directory for SDK (default: /opt/fsl-imx-wayland/4.14-sumo): 
You are about to install the SDK to "/opt/fsl-imx-wayland/4.14-sumo". Proceed[Y/n]? Y
Extracting SDK...............................................................................................done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
 $ . /opt/fsl-imx-wayland/4.14-sumo/environment-setup-aarch64-poky-linux

Since we're going to be using this toolchain primarily to build the Linux Kernel, let's make the following change to the environment script:

- export LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
+ export LDFLAGS="-O1 --hash-style=gnu --as-needed"

Now, let's source our toolchain environment, clone the linux-imx kernel outside our Yocto tree, and give our toolchain a try:

cd /build
$ source /opt/fsl-imx-wayland/4.14-sumo/environment-setup-aarch64-poky-linux

$ echo $CROSS_COMPILE
aarch64-poky-linux-

$ echo $ARCH
arm64

$ aarch64-poky-linux-gcc --version
...
aarch64-poky-linux-gcc (GCC) 7.3.0

$ git clone https://source.codeaurora.org/external/imx/linux-imx.git
Cloning into 'linux-imx'...

$ cd linux-imx
$ git checkout 5d6cbeaf
...
HEAD is now at 5d6cbeafb80c MLK-21447: ASoC: fsl_rpmsg_i2s: underrun in m4 for msg delayed

$ make V=1 LOADADDR=20008000 O=build_evk/ defconfig
...
# configuration written to .config
#
make[1]: Leaving directory '/build/linux-imx/build_evk'

$ find . -name '.config'
./build_evk/.config

Note above that our build output is being written to /build/linux-imx/build_evk

Build the linux-imx kernel

Now that we have our toolchain and repo set up, let's build the linux-imx kernel and modules:

$ cd /build/linux-imx

# build the kernel
$ make V=1 LOADADDR=20008000 O=build_evk/

# compile the kernel modules
$ make V=1 LOADADDR=20008000 O=build_evk/ modules

# install the modules
$ make V=1 LOADADDR=20008000 O=build_evk/ modules_install INSTALL_MOD_PATH=modules

$ ls build_evk/vmlinux
build_evk/vmlinux

$ ls build_evk/arch/arm64/boot/dts/freescale/fsl-imx8mq-evk.dtb
build_evk/arch/arm64/boot/dts/freescale/fsl-imx8mq-evk.dtb

$ ls build_evk/modules/lib/modules/
4.14.98-05983-g5d6cbeafb80c

Before we move on, let's create a Cscope project to help with debugging and introspection:

$ make V=1 LOADADDR=20008000 O=build_evk/ cscope

$ cd build_evk
$ cscope -d

Your terminal should become a Cscope command window. Try typing in "start_kernel" into "Find this global definition". Cscope is fantastic for a quick lookup on where structs and the like are defined.

Use U-boot to perform an NFS boot

This section assumes you have connected your NXP EVKB to your (Linux) PC via the Type-B USB connector. On Ubuntu Linux, we like kermit.

If you connect after your EVK has already booted Linux, you'll get a login prompt as shown below. Log in and then reboot. Note that you'll want to stop U-boot from booting again, so be ready to hit a key when you see the U-boot shell prompt.

NXP i.MX Release Distro 4.14-sumo imx8mqevk ttymxc0

imx8mqevk login: 

$ reboot
...
...
...

U-Boot 2018.03-imx_v2018.03_4.14.98_2.0.0_ga+g87a19df5e4 (Oct 19 2019 - 21:39:32 +0000)

CPU:   Freescale i.MX8MQ rev2.1 1500 MHz (running at 1000 MHz)
CPU:   Commercial temperature grade (0C to 95C) at 43C
Reset cause: POR
Model: Freescale i.MX8MQ EVK
DRAM:  3 GiB
TCPC:  Vendor ID [0x1fc9], Product ID [0x5110], Addr [I2C0 0x50]
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
Loading Environment from MMC... OK
No panel detected: default to HDMI
Display: HDMI (1280x720)
In:    serial
Out:   serial
Err:   serial

 BuildInfo:
  - ATF 1cb68fa
  - U-Boot 2018.03-imx_v2018.03_4.14.98_2.0.0_ga+g87a19df5e4

switch to partitions #0, OK
mmc1 is current device
flash target is MMC:1
Net:   eth0: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot:  0 

Do yourself a favor and increase bootdelay and save it:

> setenv bootdelay 10 
> saveenv
Saving Environment to MMC... Writing to MMC(1)... OK

Let's take a look at our U-Boot environment:

u-boot=> printenv
baudrate=115200
boot_fdt=try
bootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if run loadimage; then run mmcboot; else run netboot; fi; fi; else booti ${loadaddr} - ${fdt_addr}; fi
bootcmd_mfg=run mfgtool_args;if iminfo ${initrd_addr}; then if test ${tee} = yes; then bootm ${tee_addr} ${initrd_addr} ${fdt_addr}; else booti ${loadaddr} ${initrd_addr} ${fdt_addr}; fi; else echo "Run fastboot ..."; fastboot 0; fi;
bootdelay=2
bootscript=echo Running bootscript from mmc ...; source
console=ttymxc0,115200 earlycon=ec_imx6q,0x30860000,115200
emmc_dev=0
ethaddr=00:04:9f:06:12:fc
ethprime=FEC
fastboot_dev=mmc1
fdt_addr=0x43000000
fdt_file=fsl-imx8mq-evk.dtb
fdt_high=0xffffffffffffffff
fdtcontroladdr=fc8fee60
image=Image
initrd_addr=0x43800000
initrd_high=0xffffffffffffffff
jh_clk=
jh_mmcboot=setenv fdt_file fsl-imx8mq-evk-root.dtb; setenv jh_clk clk_ignore_unused; if run loadimage; then run mmcboot; else run jh_netboot; fi;
jh_netboot=setenv fdt_file fsl-imx8mq-evk-root.dtb; setenv jh_clk clk_ignore_unused; run netboot;
kboot=booti
loadaddr=0x40480000
loadbootscript=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${script};
loadfdt=fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr} ${fdt_file}
loadimage=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${image}
mfgtool_args=setenv bootargs console=${console},${baudrate} rdinit=/linuxrc clk_ignore_unused
mmcargs=setenv bootargs ${jh_clk} console=${console} root=${mmcroot}
mmcautodetect=yes
mmcboot=echo Booting from mmc ...; run mmcargs; if test ${boot_fdt} = yes || test ${boot_fdt} = try; then if run loadfdt; then booti ${loadaddr} - ${fdt_addr}; else echo WARN: Cannot load the DT; fi; else echo wait for boot; fi;
mmcdev=1
mmcpart=1
mmcroot=/dev/mmcblk1p2 rootwait rw
netargs=setenv bootargs ${jh_clk} console=${console} root=/dev/nfs ip=dhcp nfsroot=${serverip}:${nfsroot},v3,tcp
netboot=echo Booting from net ...; run netargs;  if test ${ip_dyn} = yes; then setenv get_cmd dhcp; else setenv get_cmd tftp; fi; ${get_cmd} ${loadaddr} ${image}; if test ${boot_fdt} = yes || test ${boot_fdt} = try; then if ${get_cmd} ${fdt_addr} ${fdt_file}; then booti ${loadaddr} - ${fdt_addr}; else echo WARN: Cannot load the DT; fi; else booti; fi;
script=boot.scr
sd_dev=1
soc_type=imx8mq

Some necessary env vars we'll need for NFS boot are shown below. Note that we're not going to make use of DHCP. Instead, we know and use the static IP address for our EVK board. serverip is our TFTP server that we'll set up below.

u-boot=> setenv serverip <NFS server IP addr>
u-boot=> setenv ipaddr <target IP addr>
u-boot=> saveenv

Before we set up our own env variables for nfs_boot, let's take a closer look at what NXP configured by default:

And if we wanted to use the default NXP configuration to perform a default boot, then we would use boot, which is basically the same as 'run bootcmd':

> printenv bootcmd
bootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if run loadimage; then run mmcboot; else run netboot; fi; fi; else booti ${loadaddr} - ${fdt_addr}; fi

loadimage: load the Linux kernel image into RAM:

u-boot=> fatload mmc 1:1 0x40480000 Image
23007744 bytes read in 989 ms (22.2 MiB/s)

loadfdt: load the Device Tree into RAM:

u-boot=> fatload mmc 1:1 0x43000000 fsl-imx8mq-evk.dtb
44727 bytes read in 16 ms (2.7 MiB/s)

mmcargs: set console and root:

setenv bootargs "console=ttymxc0,115200 earlycon=ec_imx6q,0x30860000,115200 root=/dev/mmcblk1p2 rootwait rw"

Boot the Kernel with booti:

booti 0x40480000 - 0x43000000

We now know enough to create our nfs_boot env variable:

u-boot=> setenv nfs_boot "tftp 0x40000000 evk/bsp/nfs.img; source 0x40000000"
u-boot=> saveenv
Saving Environment to MMC... Writing to MMC(1)... OK

nfs_boot will perform a tftp of our "nfs.img" script from our TFTP server. Once the script is loaded into RAM, the script will run it using the source command.

Note that our nfs_boot requires that you set up a TFTP server and configure your Linux box for NFS. If you don't know how to do this on Ubuntu, then refer to the references at the bottom of this page.

We show below our TFTP script nfs.txt and how we build our our nfs.img from nfs.txt:

$ cd /build/tftp/evk/bsp
$ more nfs.txt 
setenv bootargs root=/dev/nfs nfsroot=192.168.3.75:/build/rootfs/evk/rootfs,v4,tcp ip=192.168.3.99:192.168.3.75:192.168.3.1:255.255.255.0:i
mx8mqevk:eth0:off console=ttymxc0,115200 earlycon=ec_imx6q,0x30860000,115200 
tftp 0x40480000 evk/bsp/Image
tftp 0x43000000 evk/bsp/fsl-imx8mq-evk.dtb
booti 0x40480000 - 0x43000000

$ mkimage -T script -C none -n "uboot load script" -d nfs.txt nfs.img

Note that mkimage is available on Ubuntu in the "u-boot-tools" package:

$ sudo apt install u-boot-tools

As you can see in the script above, we're booting our kernel, device tree, and rootfs over the network. Some details on our nfs.txt script:

  • 192.168.3.75: Our TFTP/NFS server, which is also our Ubuntu 18.04 build machine
  • 192.168.3.99: Our NXP EVK
  • 192.168.3.1: Our Ethernet gateway
  • /build/rootfs/evk/rootfs: local rootfs folder copied from Yocto Project build

We copy Image and fsl-imx8mq-evk.dtb from our build_evk folder to the location specified in our script under our local TFTP root folder. However, our rootfs is reused from our Yocto Project build of the NXP BSP. We copy the entire rootfs folder to a location that can be mounted via NFS and change the owernship to root as shown below:

$ cd /opt/nxp/bld-wayland/tmp/work/imx8mqevk-poky-linux/fsl-image-qt5-validation-imx/1.0-r0
$ sudo cp -r rootfs <nfs rootfs folder>
$ cd <nfs rootfs folder>
$ sudo chown -R root:root *

Once these steps are complete, we can perform an NFS boot as follows:

> run nfs_boot
Using ethernet@30be0000 device
TFTP from server 192.168.3.75; our IP address is 192.168.3.99
Filename 'evk/bsp/nfs.img'.
Load address: 0x40000000
Loading: #
	 63.5 KiB/s
done
Bytes transferred = 393 (189 hex)
## Executing script at 40000000
Using ethernet@30be0000 device
TFTP from server 192.168.3.75; our IP address is 192.168.3.99
Filename 'evk/bsp/Image'.
Load address: 0x40480000
Loading: #################################################################
	 #################################################################
	 ...
	 #######
	 4.2 MiB/s
done
Bytes transferred = 23327232 (163f200 hex)
Using ethernet@30be0000 device
TFTP from server 192.168.3.75; our IP address is 192.168.3.99
Filename 'evk/bsp/fsl-imx8mq-evk.dtb'.
Load address: 0x43000000
Loading: #########
	 418.9 KiB/s
done
Bytes transferred = 44646 (ae66 hex)
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
   Using Device Tree in place at 0000000043000000, end 000000004300de65

Starting kernel ...

Create and share patches via email

Below we show modifying the linux-imx kernel, creating a patch, and emailing it using Git tools:

# add a printk to mx6s_capture.c and diff it
$ git diff drivers/media/platform/mxc/capture/mx6s_capture.c
..
@@ -2018,6 +2021,8 @@ static int mx6s_csi_probe(struct platform_device *pdev)
 
        csi_dev->vdev = vdev;
 
+       printk("CSI Bridge base regs=0x%lx\n",csi_dev->regbase);
+
        video_set_drvdata(csi_dev->vdev, csi_dev);
        mutex_lock(&csi_dev->lock);

# create a dev branch for custom patches
$ git checkout -b development
M	drivers/media/platform/mxc/capture/mx6s_capture.c

$ git add drivers/media/platform/mxc/capture/mx6s_capture.c

$ git commit -m 'add a printk to probe'

$ git format-patch -1
0001-add-a-printk-to-probe.patch

$ git send-email --to <recipient> --suppress-cc=all 0001-add-a-printk-to-probe.patch

On the receiving side, the recipient can save the patch to the linux-imx folder and apply as shown:

$ git am < 0001-add-a-printk-to-probe.patch
Applying: add a printk to probe

Summary

In this article, we have shown various ways that we accelerate development with the NXP linux-imx kernel. These tips also apply in general to any embedded Linux kernel.

It should be noted that we can modify the linux-imx kernel source and rebuild & reboot in typically less than five minutes on a quad-core Intel I7, 16GB RAM, and a magnetic drive, which is a modest build machine by today's standards.

We'll continue to update this article as we continue building our expertise with the NXP MCIMX8M-EVK.

References and Resources

Acronyms and Terms

  • OCRAM: On-Chip RAM Memory Controller
  • OP-TEE: Open Portable Trusted Execution Environment
  • SPL: Second Program Loader
  • SRC: i.MX8 System Reset Controller
  • TCM: Tightly-Coupled-Memory
  • TZ: Trust Zone

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please follow us on Twitter and join our mailing list.

Related articles on this site:

share
subscribe to mailing list:

Please help us improve this article by adding your comment or question:

your email address will be kept private
authenticate with a 3rd party for enhanced features, such as image upload
previous month
next month
Su
Mo
Tu
Wd
Th
Fr
Sa
loading