Mind Chasers Inc.
Mind Chasers Inc.

Methods to Accelerate Linux Kernel Development using Yocto and the NXP MCIMX8M-EVK L5.4.24_2.1.0 BSP

Techniques to accelerate development with the Linux Kernel and related when working with the Yocto Project and an NXP i.MX EVK system.


The NXP MCIMX8M-EVK evaluation kit for the i.MX 8MQuad processor is certainly an amazing system, and there is plenty of documentation. However, we found some of the Yocto Linux related documentation stale and confusing, so we are posting this article regarding our steps to accelerate building, booting, and debugging the Linux kernel, device tree, and root file system.

If you're just getting started with your EVK, then you may want to also review part 1 of this series: Build, Install, and Develop with NXP's Latest Yocto BSP for the MCIMX8M-EVK using the Linux 5.4 Kernel .

Listed below are some of the data points that we may need to know when building the linux-imx kernel outside of the Yocto build system:

  • The NXP i.MX 8MQuad is powered by a quad core 1.5 GHz ARM CORTEX-A53 with an auxiliary ARM M4
  • We are working with the NXP release "L5.4.24_2.1.0". This was developed using the Yocto Project Zeus (3.0) branch / release.
  • Zeus 3.0 was released October 2019.
  • Our Yocto Machine: imx8mqevk
  • Our Yocto distro: "fsl-imx-wayland"
  • There is an EVKB and EVK. We are using the EVKB, and this has an upgrade to the WIFI over the EVK and a later revision of the i.MX8 SoC.
EVK Board

In this article, we discuss the following steps in order:

  1. Create a toolchain for use outside the Yocto Build Environment
  2. Build the NXP linux-imx kernel using the external SDK / toolchain
  3. Use U-boot to load the kernel, device tree, and rootfs using TFTP and NFS
  4. Create and share kernel patches via email to speed collaboration

Create a toolchain for building the Linux kernel outside the Yocto project

This section assumes that you have already successfully built the NXP BSP using the Yocto Project. If you haven't, please review the steps detailed in Build, Install, and Develop with NXP's Latest Yocto BSP for the MCIMX8M-EVK using the Linux 5.4 Kernel.

Build the GNU toolchain / SDK for the NXP BSP as shown below. Note that our build machine is an Ubuntu Linux 18.04 Intel PC.

$ cd /build/imx8/
$ source sources/poky/oe-init-build-env bld-wayland/

$ bitbake imx-image-multimedia -c populate_sdk

$ ls /build/imx8/bld-wayland/tmp/work
aarch64-mx8m-poky-linux  all-poky-linux        sdk-provides-dummy-nativesdk-pokysdk-linux  x86_64-linux
aarch64-poky-linux       imx8mqevk-poky-linux  sdk-provides-dummy-target-poky-linux        x86_64-nativesdk-pokysdk-linux

$ cd /build/imx8/bld-wayland/tmp/deploy/sdk
$ ls *.sh


$ ./fsl*.sh

NXP i.MX Release Distro SDK installer version 5.4-zeus
Enter target directory for SDK (default: /opt/fsl-imx-wayland/5.4-zeus): 
You are about to install the SDK to "/opt/fsl-imx-wayland/5.4-zeus". Proceed [Y/n]? y
Extracting SDK.........done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.
 $ . /opt/fsl-imx-wayland/5.4-zeus/environment-setup-aarch64-poky-linux

Since we're going to be using this toolchain primarily to build the Linux Kernel, let's make the following change to the environment script:

- export LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now"
+ export LDFLAGS="-O1 --hash-style=gnu --as-needed -fstack-protector-strong -z,relro,-z,now"

Now, let's source our toolchain environment, clone the linux-imx kernel outside our Yocto tree, and give our toolchain a try:

cd /build
$ source /opt/fsl-imx-wayland/5.4-zeus/environment-setup-aarch64-poky-linux 


$ echo $ARCH

~$ aarch64-poky-linux-gcc --version
aarch64-poky-linux-gcc (GCC) 9.2.0

Build the linux-imx Kernel

Before we clone the linux-imx kernel, lets make sure we know the source repo and commit we want. It's our goal to build the same kernel as that was built by the NXP Yocto BSP.

$ cd /<BSP build path>/tmp/work/imx8mqevk-poky-linux/linux-imx/5.4-r0/git

$ git log

commit babac008e5cf168abca1a85bda2e8071ca27a5c0 (HEAD -> imx_5.4.24_2.1.0, tag: rel_imx_5.4.24_2.1.0, origin/imx_5.4.24_2.1.0)
Author: Peng Fan <peng.fan@nxp.com>
Date:   Sat May 30 15:03:46 2020 +0800

    MLK-24224-2 arm64: imx_v8_defconfig: select xen watchdog

$ more .git/config
[remote "origin"]
	url = https://source.codeaurora.org/external/imx/linux-imx.git

It may be helpful to note that the following two Ubuntu development packages will be required when building the kernel:

$ sudo apt install libelf-dev libyaml-dev

Now we're ready to clone and build the NXP linux-imx kernel outside the Yocto Project.

$ cd /build
$ git clone https://source.codeaurora.org/external/imx/linux-imx.git
Cloning into 'linux-imx'...

$ cd linux-imx
$ git checkout babac00
HEAD is now at babac008e5cf MLK-24224-2 arm64: imx_v8_defconfig: select xen watchdog

$ cp arch/arm64/configs/imx_v8_defconfig build_evk/.config

$ make  V=1 LOADADDR=40480000 O=build_evk olddefconfig

$ more build_evk/.config 
# Automatically generated file; DO NOT EDIT.
# Linux/x86 5.4.24 Kernel Configuration

Note above that our build output is being written to /build/linux-imx/build_evk

Build the linux-imx kernel

Now that we have our toolchain & build environment configured, let's build the linux-imx kernel and modules:

# build the kernel
$ make V=1 LOADADDR=40480000 O=build_evk/
  aarch64-poky-linux-ld -r  -EL  -maarch64elf  --build-id  -T ../scripts/module-common.lds -T ../arch/arm64/kernel/module.lds -o sound/usb/snd-usbmidi-lib.ko sound/usb/snd-usbmidi-lib.o sound/usb/snd-usbmidi-lib.mod.o;  true
sh ../scripts/modules-check.sh
make -f ../scripts/Makefile.build obj=arch/arm64/boot arch/arm64/boot/Image
  aarch64-poky-linux-objcopy  -O binary -R .note -R .note.gnu.build-id -R .comment -S vmlinux arch/arm64/boot/Image
make -f ../scripts/Makefile.build obj=arch/arm64/boot arch/arm64/boot/Image.gz
  cat arch/arm64/boot/Image | gzip -n -f -9 > arch/arm64/boot/Image.gz
make[1]: Leaving directory '/build/linux-imx/build_evk'

# compile the kernel modules
$ make V=1 LOADADDR=40480000 O=build_evk/ modules
make -f ../scripts/Makefile.modpost
  sed 's/ko$/o/' modules.order | scripts/mod/modpost -m  -o ./Module.symvers        -s -T - vmlinux
make -f ../scripts/Makefile.modfinal
sh ../scripts/modules-check.sh
make[1]: Leaving directory '/build/linux-imx/build_evk'

# install the modules
$ make V=1 LOADADDR=40480000 O=build_evk/ modules_install INSTALL_MOD_PATH=modules
  sh ../scripts/depmod.sh /sbin/depmod 5.4.24-00060-gbabac008e5cf
make[1]: Leaving directory '/build/linux-imx/build_evk'

$ ls build_evk/vmlinux

$ find build_evk/ -name 'imx8mq-evk.dtb'

$ ls build_evk/modules/lib/modules/

Before we move on, let's create a Cscope project to help with debugging and introspection:

$ make V=1 LOADADDR=40480000 O=build_evk/ cscope

$ cd build_evk
$ cscope -d

Your terminal should become a Cscope command window. Try typing in "start_kernel" into "Find this global definition". Cscope is fantastic for a quick lookup on where structs and the like are defined.

Use U-boot to perform an NFS boot

This section assumes you have connected your NXP EVKB to your (Linux) PC via the Type-B USB connector. On Ubuntu Linux, we like kermit.

If you connect after your EVK has already booted Linux, you'll get a login prompt as shown below. Log in and then reboot. Note that you'll want to stop U-boot from booting again, so be ready to hit a key when you see the U-boot shell prompt. Note that we're using the version of U-Boot included in the NXP BSP and installed on our SD Card.

NXP i.MX Release Distro 5.4-zeus imx8mqevk ttymxc0

imx8mqevk login: 

$ reboot

U-Boot 2020.04-5.4.24-2.1.0+g4979a99482 (Aug 08 2020 - 03:58:09 +0000)

CPU:   i.MX8MQ rev2.1 1500 MHz (running at 1000 MHz)
CPU:   Commercial temperature grade (0C to 95C) at 43C
Reset cause: POR
Model: NXP i.MX8MQ EVK
DRAM:  3 GiB
TCPC:  Vendor ID [0x1fc9], Product ID [0x5110], Addr [I2C0 0x50]
Loading Environment from MMC... Run CMD11 1.8V switch
*** Warning - bad CRC, using default environment

[*]-Video Link 0imx8m_hdmi_probe
 (1280 x 720)
	[0] display-controller@32e00000, video
	[1] hdmi@32c00000, display
In:    serial
Out:   serial
Err:   serial

  - ATF b0a00f2
  - U-Boot 2020.04-5.4.24-2.1.0+g4979a99482

Run CMD11 1.8V switch
switch to partitions #0, OK
mmc1 is current device
flash target is MMC:1
Run CMD11 1.8V switch
Warning: ethernet@30be0000 using MAC address from ROM
eth0: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot:  0 

Do yourself a favor and increase bootdelay and save it:

> setenv bootdelay 10 
> saveenv
Saving Environment to MMC... Writing to MMC(1)... OK

Let's take a look at our default U-Boot environment:

u-boot=> printenv
bootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if run loadimage; then run mmcboot; else run netboot; fi; fi; else booti ${loadaddr} - ${fdt_addr}; fi
bootcmd_mfg=run mfgtool_args;if iminfo ${initrd_addr}; then if test ${tee} = yes; then bootm ${tee_addr} ${initrd_addr} ${fdt_addr}; else booti ${loadaddr} ${initrd_addr} ${fdt_addr}; fi; else echo "Run fastboot ..."; fastboot 0; fi;
bootscript=echo Running bootscript from mmc ...; source
jh_mmcboot=setenv fdt_file imx8mq-evk-root.dtb; setenv jh_clk clk_ignore_unused; if run loadimage; then run mmcboot; else run jh_netboot; fi; 
jh_netboot=setenv fdt_file imx8mq-evk-root.dtb; setenv jh_clk clk_ignore_unused; run netboot; 
loadbootscript=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${script};
loadfdt=fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr} ${fdt_file}
loadimage=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${image}
mfgtool_args=setenv bootargs console=${console},${baudrate} rdinit=/linuxrc clk_ignore_unused 
mmcargs=setenv bootargs ${jh_clk} console=${console} root=${mmcroot}
mmcboot=echo Booting from mmc ...; run mmcargs; if test ${boot_fdt} = yes || test ${boot_fdt} = try; then if run loadfdt; then booti ${loadaddr} - ${fdt_addr}; else echo WARN: Cannot load the DT; fi; else echo wait for boot; fi;
mmcroot=/dev/mmcblk1p2 rootwait rw
netargs=setenv bootargs ${jh_clk} console=${console} root=/dev/nfs ip=dhcp nfsroot=${serverip}:${nfsroot},v3,tcp
netboot=echo Booting from net ...; run netargs;  if test ${ip_dyn} = yes; then setenv get_cmd dhcp; else setenv get_cmd tftp; fi; ${get_cmd} ${loadaddr} ${image}; if test ${boot_fdt} = yes || test ${boot_fdt} = try; then if ${get_cmd} ${fdt_addr} ${fdt_file}; then booti ${loadaddr} - ${fdt_addr}; else echo WARN: Cannot load the DT; fi; else booti; fi;

Some necessary env vars we'll need for NFS boot are shown below. Note that we're not going to make use of DHCP. Instead, we know and use the static IP address for our EVK board. serverip is our TFTP server that we'll set up below.

u-boot=> setenv serverip <NFS server IP addr>
u-boot=> setenv ipaddr <target IP addr>
u-boot=> saveenv

Before we set up our own env variables for nfs_boot, let's take a closer look at what NXP configured by default for SD Card / MMC boot:

And if we wanted to use the default NXP configuration to perform a default boot, then we would use boot, which is basically the same as 'run bootcmd':

u-boot=> printenv bootcmd
bootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if run loadimage; then run mmcboot; else run netboot; fi; fi; else booti ${loadaddr} - ${fdt_addr}; fibootcmd=mmc dev ${mmcdev}; if mmc rescan; then if run loadbootscript; then run bootscript; else if run loadimage; then run mmcboot; else run netboot; fi; fi; else booti ${loadaddr} - ${fdt_addr}; fi

loadimage: load the Linux kernel image into RAM:

u-boot=> fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${image}    
28021248 bytes read in 333 ms (80.2 MiB/s)

loadfdt: load the Device Tree into RAM:

u-boot=> fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr} ${fdt_file}
43912 bytes read in 15 ms (2.8 MiB/s)

u-boot=> echo ${fdt_file}

bootargs: set console and root:

bootargs=console=ttymxc0,115200 root=/dev/mmcblk1p2 rootwait rw

Boot the Kernel with booti:

booti 0x40480000 - 0x43000000

We now know enough to create our nfs_boot env variable:

u-boot=> setenv nfs_boot "tftp 0x40000000 imx8/bsp/imx8.img; source 0x40000000"
u-boot=> saveenv
Saving Environment to MMC... Writing to MMC(1)... OK

nfs_boot will perform a tftp of our "imx8.img" script from our TFTP server. Once the script is loaded into RAM, the script will run it using the source command.

Note that our nfs_boot requires that you set up a TFTP server and configure your Linux box for NFS. If you don't know how to do this on Ubuntu, then refer to the references at the bottom of this page.

We show below our TFTP script imx8.txt and how we build imx8.img from imx8.txt:

$ cd /build/tftp/imx8/bsp/
$ more imx8.txt
setenv bootargs root=/dev/nfs nfsroot=,v4,tcp ip=
mx8mqevk:eth0:off console=ttymxc0,115200 earlycon=ec_imx6q,0x30860000,115200 
tftp 0x40480000 imx8/bsp/Image
tftp 0x43000000 imx8/bsp/imx8mq-evk.dtb
booti 0x40480000 - 0x43000000

$ mkimage -T script -C none -n "uboot load script" -d nfs.txt nfs.img

Note that mkimage is available on Ubuntu in the "u-boot-tools" package:

$ sudo apt install u-boot-tools

As you can see in the script above, we're booting our kernel, device tree, and rootfs over the network. Some details on our nfs.txt script:

  • Our TFTP/NFS server, which is also our Ubuntu 18.04 build machine
  • Our NXP EVK
  • Our Ethernet gateway
  • /build/rootfs/imx8/rootfs: local rootfs folder copied from Yocto Project build

We copy Image and imx8mq-evk.dtb from our build_evk folder to the location specified in our script under our local TFTP root folder. However, our rootfs is reused from our Yocto Project build of the NXP BSP. We copy the entire rootfs folder to a location that can be mounted via NFS and change the owernship to root as shown below:

$ cd /build/imx8/bld-wayland/tmp/work/imx8mqevk-poky-linux/imx-image-multimedia/1.0-r0
$ sudo cp -r rootfs <nfs rootfs folder>
$ cd <nfs rootfs folder>
$ sudo chown -R root:root *

Once these steps are complete, we can perform an NFS boot as follows:

> run nfs_boot
Using ethernet@30be0000 device
TFTP from server; our IP address is
Filename 'imx8/bsp/nfs.img'.
Load address: 0x40000000
Loading: #
	 63.5 KiB/s
Bytes transferred = 393 (189 hex)
## Executing script at 40000000
Using ethernet@30be0000 device
TFTP from server; our IP address is
Filename 'imx8/bsp/Image'.
Load address: 0x40480000
Loading: #################################################################
	 4.2 MiB/s
Bytes transferred = 23327232 (163f200 hex)
Using ethernet@30be0000 device
TFTP from server; our IP address is
Filename 'imx8/bsp/imx8mq-evk.dtb'.
Load address: 0x43000000
Loading: #########
	 418.9 KiB/s
Bytes transferred = 44646 (ae66 hex)
## Flattened Device Tree blob at 43000000
   Booting using the fdt blob at 0x43000000
   Using Device Tree in place at 0000000043000000, end 000000004300de65

Starting kernel ...

Create and share patches via email

Below we show modifying the linux-imx kernel, creating a patch, and emailing it using Git tools:

# add a printk to mx6s_capture.c and diff it
$ git diff drivers/media/platform/mxc/capture/mx6s_capture.c
@@ -2018,6 +2021,8 @@ static int mx6s_csi_probe(struct platform_device *pdev)
        csi_dev->vdev = vdev;
+       printk("CSI Bridge base regs=0x%lx\n",csi_dev->regbase);
        video_set_drvdata(csi_dev->vdev, csi_dev);

# create a dev branch for custom patches
$ git checkout -b development
M	drivers/media/platform/mxc/capture/mx6s_capture.c

$ git add drivers/media/platform/mxc/capture/mx6s_capture.c

$ git commit -m 'add a printk to probe'

$ git format-patch -1

$ git send-email --to <recipient> --suppress-cc=all 0001-add-a-printk-to-probe.patch

On the receiving side, the recipient can save the patch to the linux-imx folder and apply as shown:

$ git am < 0001-add-a-printk-to-probe.patch
Applying: add a printk to probe


In this article, we have shown various ways that we accelerate development with the NXP linux-imx kernel. These tips also apply in general to any embedded Linux kernel.

It should be noted that we can modify the linux-imx kernel source and rebuild & reboot in typically less than five minutes on a quad-core Intel I7, 16GB RAM, and a magnetic drive, which is a modest build machine by today's standards.

This article is a work in progress, so please submit any errors or questions in the comment blox below.

References and Resources

Acronyms and Terms

  • OCRAM: On-Chip RAM Memory Controller
  • OP-TEE: Open Portable Trusted Execution Environment
  • SPL: Second Program Loader
  • SRC: i.MX8 System Reset Controller
  • TCM: Tightly-Coupled-Memory
  • TZ: Trust Zone

Didn't find an answer to your question? Post your issue below or in our new FORUM, and we'll try our best to help you find a solution.

And please note that we update our site daily with new content related to our open source approach to network security and system design. If you would like to be notified about these changes, then please follow us on Twitter and join our mailing list.

Related articles on this site:

subscribe to mailing list:

Please help us improve this article by adding your comment or question:

your email address will be kept private
authenticate with a 3rd party for enhanced features, such as image upload
previous month
next month