Tutorial: Building a Yocto Image for NVIDIA Jetson

Introduction:

Yocto is a framework that provides tools and processes for creating custom Linux images for embedded devices.

It empowers developers with the flexibility to build Linux systems tailored to their needs while also offering production-grade quality, enhanced security, and long-term support.

For these reasons, Yocto has become a popular choice among robotics engineers using NVIDIA Jetson devices to create custom images for their applications. However, Yocto's flexibility comes at a cost—it has a steep learning curve. Setting up an image can take hours (not to mention the lengthy build times), which means you might spend several days preparing your image before even having the opportunity to test it on your robot.

I encountered this challenge myself, so I created this tutorial to help other engineers build with Yocto faster.

In this guide, we’ll create a base Yocto image for the Jetson that includes ROS 2 and OpenCV, useful tools for whatever robotics applications you build.

This tutorial is designed for beginner to intermediate Yocto users who want to go from 0 →1, starting from scratch, to successfully flashing a system image onto their Jetson device.

By the end of this tutorial, you’ll understand how Yocto works, how to build a Yocto image, and how to customize it further to suit your specific application needs.

Let's dive in!

Yocto Basics:

This tutorial assumes you have a basic understanding of working with a Jetson and other embedded systems.

If you’re new to Yocto, don’t worry—we’ll provide a quick rundown to help you get started. Check out this Miru blog for a deeper dive into Yocto’s core concepts and benefits.

Here's some basic information to get you started:

1. BitBake

   • The build engine of Yocto. It processes recipes (instructions) to generate packages, kernels, and complete system images.

2. Recipes (.bb Files)

   • These are the core instructions that tell Yocto how to build a specific piece of software. Recipes include details like dependencies, source code locations, and build configurations.

3. Layers

   • Layers are modular collections of recipes, configurations, and other metadata. You can think of them as "building blocks" that can be stacked to customize your image. For example, the meta-tegra layer adds support for NVIDIA Jetson devices.

4. Machine Configuration

   • Yocto supports many hardware platforms, and you specify your target machine using a configuration file. This ensures the system image is optimized for your device (in our case, Jetson).

5. Poky

   • Yocto includes a reference distribution called Poky, which is often used as a starting point. Poky bundles essential tools and metadata for building a basic Linux system.

6. Build Directory (Build Environment)

   • This is where Yocto does its work. It stores temporary files, build artifacts, and configuration details. You’ll spend most of your time here when using Yocto.

If any of these pieces are still confusing, don’t worry. As we go through the tutorial, things will make more sense.

Requirements:

Hardware:

• x86 or ARM64 machine running Ubuntu 22.04 (recommended) or 20.04. You’ll need at least 50 GB of free space (but more is highly recommended, as Yocto builds are substantial).

• NVIDIA Jetson device

• SD card for flashing the Jetson

Dependencies:

Before starting, ensure you’ve installed the required dependencies. Run the following command on your host:

sudo apt update
sudo apt install -y gawk wget git diffstat unzip texinfo build-essential chrpath \
socat cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping \
python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint xterm curl lz4 \
qemu-user-static

Preparing Jetson Files:

First, we’ll need to download the Jetson BSP (Board Support Package) and the Root File System (RFS) for Yocto to use in the build.

What Are BSP and RFS, and Why Do We Need Them?

• BSP (Board Support Package):

  • A BSP is a collection of drivers, config files, and tools that ensure Yocto can produce a system image optimized for your specific Jetson device.

• RFS (Root File System):

  • The RFS is the base directory structure of the Linux OS. It contains libraries, binaries, and utilities that allow the OS to boot and function properly.

I’m using NVIDIA Jetson Linux 36.4, compatible with Jetpack 6.2.

You can download the BSP and RFS directly from this link.

Once downloaded, extract the tarballs into your project directory:

cd ~/Documents/yocto-jetson-ros-tutorial
mkdir bsp
cd bsp


tar -xvpf ~/Downloads/jetson_BSP.tbz2 -C ~/Documents/yocto-jetson-ros-tutorial/bsp/
sudo tar -xvpf ~/Downloads/jetson_sample_root_filesystem.tbz2 -C ~/Documents/yocto-jetson-ros-tutorial/bsp/Linux_for_Tegra/rootfs

You can also choose to download additional tools like CUDA or OTA Tools. While this tutorial skips these steps, make sure to extract any additional components into your /bsp directory using the same method.

Once the BSP and RFS are extracted into your Yocto project, we must apply the binaries.

sudo ~/Documents/yocto-jetson-ros-tutorial/bsp/Linux_for_Tegra/apply_binaries.sh

Yocto Build Preparation:

We’re ready to download the Yocto-specific files needed to build our image. These files include the layers that BitBake will use in its recipes to build the final system image.

First, navigate into your project directory:

cd ~/Documents/yocto-jetson-ros-tutorial

Next, we need to clone the Yocto Poky repository. Poky is the base layer for all Yocto projects and includes BitBake, the Yocto build engine, and other core layers.

I’ll be using the Yocto Scarthgap release, but feel free to use any version you prefer:

git clone --single-branch -b scarthgap git://git.yoctoproject.org/poky.git

Navigate to the root of your project directory and create a /sources directory. This is where we’ll clone all the additional Yocto layers needed for our build.

Layers to clone:

• meta-tegra is the Yocto layer for NVIDIA Jetson

• meta-ros is the Yocto layer for ROS and ROS2

• meta-openembedded is a layer that provides a range of tools and libraries like OpenCV, Python, and OpenSSL/SSH

Remember, layers are modular collections of recipes and config files. They combine to create the final Yocto image. As in our case, layers often focus on specific functionality or hardware.

cd ~/Documents/yocto-jetson-ros-tutorial
mkdir sources
cd sources

git clone --single-branch -b scarthgap https://github.com/OE4T/meta-tegra.git
git clone --single-branch -b scarthgap https://github.com/ros/meta-ros.git
git clone --single-branch -b scarthgap https://github.com/openembedded/meta-openembedded.git

With all the required layers cloned, we’re ready to initialize the build environment and customize our image.

cd ~/Documents/yocto-jetson-ros-tutorial/poky
source oe-init-build-env

Editing Configurations for Build:

Since we’ve added custom layers to the Yocto build, we need to update the build configuration files so that Yocto can use these layers during the build process.

First, let’s edit bblayers.conf. This file tells Yocto which layers to include in the build.

nano ~/Documents/yocto-jetson-ros-tutorial/poky/build/conf/bblayers.conf

Your original bblayers.conf file should look something like this, without the references to the meta-tegra, meta-ros, or meta-openembedded layers.

POKY_BBLAYERS_CONF_VERSION = "2"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta-poky \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta-yocto-bsp \
  "

Add the new layers to the file.

POKY_BBLAYERS_CONF_VERSION = "2"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta-poky \
  /home/miru/Documents/yocto-jetson-ros-tutorial/poky/meta-yocto-bsp \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-tegra \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-ros/meta-ros-common \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-ros/meta-ros2 \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-ros/meta-ros2-humble \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-openembedded/meta-oe \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-openembedded/meta-python \
  /home/miru/Documents/yocto-jetson-ros-tutorial/sources/meta-openembedded/meta-networking \
  "

The local.conf file customizes the build environment by specifying details like the target machine, included packages, and additional features.

If you need to figure out the name of your target machine, the following command will help. My target machine is a jetson-orin-nano-devkit.

ls ~/Documents/yocto-jetson-ros-tutorial/sources/meta-tegra/conf/machine

Now open the local.conf file for editing.

Feel free to follow the Yocto documentation to customize your build environment.

I’m keeping it simple by adding ROS2 Humble and OpenCV. I’ve also changed my PACKAGE_CLASSES from .rpm to .deb. I've also included IMAGE_FSTYPES, which allows us to flash our target device with an SD card.

nano ~/Documents/yocto-jetson-ros-tutorial/poky/build/conf/local.conf

Update the file with these changes:

• Set the target machine:

  MACHINE = "jetson-orin-nano-devkit"

• Switch the package manager to .deb (optional):

  PACKAGE_CLASSES = "package_deb"

• Add ROS 2 Humble and OpenCV to the build:

  IMAGE_INSTALL_append = " ros-humble opencv"

• Specify image file types for flashing: Adding the following lines lets you flash an SD card for Jetson.

  IMAGE_CLASSES += "image_types_tegra"
  IMAGE_FSTYPES = "tegraflash tar.gz"

Potential Fixes:

A few errors caused the build to fail on my setup. I’m sharing them with you in case you encounter any similar issues.

You can either fix them proactively or as problems arise in your build. If doing the latter, remember to clean and rebuild the individual packages each time you make a fix.

Feel free to skip these if none apply to you!

CMake (meta-ros layer):

If you’re using ROS2 Humble, there’s a problem with the build process of ament-cmake-core-native. This happened because the PYTHONPATH environment variable was not correctly set, meaning that Python could not locate the ament_package module. We’ll need to fix the issue manually.

You can also see this GitHub issue to reference the problem and the fix.

Navigate to this directory and edit the ros_opt_prefix.bbclass file.

We will need to change this line.

export PYTHONPATH:prepend = "${STAGING_DIR_NATIVE}${PYTHON_SITEPACKAGES_DIR}:”

This is the fixed version.

export PYTHONPATH = "${STAGING_DIR_NATIVE}${PYTHON_SITEPACKAGES_DIR}:”

Patch Resolution (meta-ros layer)

During the build, the patch 0001-CMakeLists.txt-prevent-building-zstd-with-ExternalPr.patch failed to be applied to the do_patch task. The patch was designed to modify the CMakeLists.txt file for the zstd-vendor package, but it’s likely outdated.

The patch was referenced in the .bbappend file for the zstd-vendor package.

Find the line where the patch was referenced.

SRC_URI += "file://0001-CMakeLists.txt-prevent-building-zstd-with-ExternalPr.patch"

And disable it by commenting it out.

# SRC_URI += "file://0001-CMakeLists.txt-prevent-building-zstd-with-ExternalPr.patch"

hwloc (meta-openembedded layer)

The hwloc package failed the do_package_qa task because the libhwloc subpackage depended on libcudart.so.12, which was not listed as a runtime dependency.

So, we edited the recipe.

~/Documents/yocto-jetson-ros-tutorial/sources/meta-ros/meta-ros2-humble/recipes-bbappends/rosbag2/zstd-vendor_0.15.12-1.bbappend

And we added this line to the recipe, adding the missing dependency.

RDEPENDS:libhwloc += "cuda-cudart"

Start Yocto Build!

Now that we’ve set everything up, we have one more step before we start the build. We want to see all the layers in our image for a last check. This is also helpful for determining whether your build environment is active.

cd ~/Documents/yocto-jetson-ros-tutorial/poky/build
bitbake-layers show-layers

If everything looks good, the only thing left to do is start the build. Yocto offers a range of options for the type of image you want to build. Choose the one that suits your needs.

bitbake -v core-image-base

Beware, Yocto builds take a LONG time. Now’s a great time to do something else—like walk your dog or wash your car. You won’t be missing out on much!

Flashing your device

Once your Yocto image is successfully built, we can flash it onto our target device.

meta-tegra does not support flashing our target Jetson device with an SD card image, so we can’t use flashing software like Balena Etcher.

Instead, we’ll flash it ourselves using the terminal.

We need to locate the ext4 file to ensure it was successfully generated. For me, the file name was core-image-base-jetson-orin-nano-devkit.rootfs.tegraflash.tar.gz.

ls -lh ~/Documents/yocto-jetson-ros-tutorial/build/tmp/deploy/images/<MACHINE>

Extract the .tegraflash.tar.gz file into a temporary directory.

mkdir ~/yocto-image
cd ~/yocto-image
tar -xvf core-image-base-<MACHINE>

Now, prepare the SD card. Insert the SD card into your computer and identify the device.

lsblk

Once you identify your device (mine is /dev/sdb), flash the SD card with the script provided by the tarball.

sudo ./dosdcard.sh /dev/sdb -y

Before we boot, we need to configure boot settings. Mount the SD card and edit the boot config file.

sudo mount /dev/sdb1 /mnt
sudo nano /mnt/boot/extlinux/extlinux.conf

Update the APPEND line to point to the right root device.

APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait ${bootargs}

Now unmount the SD card.

sudo umount /mnt

Now, insert your SD card into your Jetson device and boot up. You should see the NVIDIA logo, followed by kernel boot messages.

If this is your first time booting with Yocto (or you encounter boot issues), you may need to update your SPI flash. Go here for more instructions on how to do that. Depending on your device, you may need to follow Jetson documentation to put your device into recovery mode.

Congrats—you’ve successfully used Yocto to flash your Jetson device with a custom Linux image.

Happy building!