ZCU104 board¶

The ZCU104 board enables testing DDR4 SO-DIMM modules. It features a Zynq UltraScale+ MPSoC device consisting of a Processing System (PS) with quad-core ARM Cortex-A53 and programmable logic (PL).

On the ZCU104 board, the Ethernet PHY is connected to PS instead of PL. For this reason, it is necessary to route the Ethernet/EtherBone traffic as follows :PC <-> PS <-> PL. A simple EtherBone server is implemented for this purpose (the source code can be found in the firmware/zcu104/etherbone/ directory).

The following instructions show how to set up the board for the first time.

For FPGA design documentation for this board, refer to the Digital design chapter.

Board configuration¶

To make the ZCU104 boot from an SD card, it is necessary to ensure proper switch configuration. The mode switch (SW6) consisting of 4 switches is located near the FMC LPC Connector (J5) (the same side of the board as USB, HDMI, Ethernet). For details, refer to the ZCU104 Evaluation Board User Guide (UG1267). To use an SD card, configure the switches as follows:

Preparing SD card¶

For a basic, simple setup, get the pre-built SD card image zcu104.img from github releases and load it to a microSD card. To load it to the SD card, insert the card into your PC card slot and find the device name. lsblk can be used to check for available devices. An example output looks as follows:

$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 931.5G  0 disk
└─sda1        8:1    0 931.5G  0 part /data
sdb           8:16   1  14.8G  0 disk
├─sdb1        8:17   1   128M  0 part /run/media/BOOT
└─sdb2        8:18   1   128M  0 part /run/media/rootfs
nvme0n1     259:0    0 476.9G  0 disk
├─nvme0n1p1 259:1    0   512M  0 part /boot
└─nvme0n1p2 259:2    0 476.4G  0 part /

In the output above, the SD card is sdb with two partitions sdb1 and sdb2.

Warning

Make sure to select the proper device name to avoid damaging the hard drive in your system! Assure the device SIZE matches the capacity of your microSD card and compare the outputs of the lsblk command with and without the SD card inserted.

Make sure to unmount all partitions on the card before loading the image. For example, assuming the SD card is /dev/sdb (device is without a number), use sudo umount /dev/sdb1 /dev/sdb2 to unmount its partitions.

To load the image, use the following command, replacing <DEVICE> according to the output of lsblk (in the example above it would be /dev/sdb):

sudo dd status=progress oflag=sync bs=4M if=zcu104.img of=<DEVICE>

Now, the microSD card should be ready to use. If it is loaded successfully, you will be able to mount the two partitions (BOOT and rootfs) on your PC and browse the files. First, check whether your system auto-mounted the partitions. If not, you can use:

sudo mkdir -p /mnt/boot /mnt/rootfs
sudo mount /dev/sdb1 /mnt/boot
sudo mount /dev/sdb2 /mnt/rootfs

Loading the bitstream¶

Instead of loading the bitstream through the JTAG interface, copy it to the microSD card BOOT partition (FAT32). The bitstream will be then loaded by the bootloader during system startup.

The prebuilt card image comes with a sample bitstream, but in order to use the provided rowhammer Python scripts, you need to create a fresh bitstream.

Copy it to the BOOT partition (FAT32) of the microSD card. Make sure it is named zcu104.bit.

When the SD card is ready, insert it into the microSD card slot on your ZCU104 board and power on the board.

Onboard LEDs are the first indication that the bitstream is loaded successfully. When the board is powered up, the LED will be red and then turn green if the bitstream is loaded successfully. The ZCU104 bitstream will also make the four LEDs near the user buttons turn on and off in a circular pattern. The serial console over USB can be used to further check if the board is working correctly.

ZCU104 microUSB¶

ZCU104 has a microUSB port connected to the FTDI chip. It provides 4 channels connected as follows:

Channel A is configured to support the JTAG chain.
Channel B implements UART0 MIO18/19 connections.
Channel C implements UART1 MIO20/21 connections.
Channel D implements an UART2 PL-side bank 28 4-wire interface.

The channels should show up as subsequent /dev/ttyUSBx devices (0-3 if no others were present). Channel B is connected to the console in the PS Linux system.

To log in to the board, connect the microUSB cable to the PC and find Channel B among the ttyUSB devices in your system. If only ttyUSB0 through ttyUSB3 are visible, then Channel B will be ttyUSB1.

Open the serial console using e.g. picocom or minicom (you may need to install one). With picocom, use the following command (may require sudo):

picocom -b 115200 /dev/ttyUSB1

Press enter. When you see the following prompt:

Welcome to Buildroot
buildroot login:

Use root as login and leave password empty. You can set up a password if needed.

Network setup¶

Connect the ZCU104 board to your local network (or directly to a PC) using an Ethernet cable.

The board uses a static IP address - 192.168.100.50 by default. If it does not conflict with your local network configuration, you can skip this section. You can find the default configuration here.

To verify connectivity, use ping 192.168.100.50. You should see data being transmitted, e.g.:

$ ping 192.168.100.50
PING 192.168.100.50 (192.168.100.50) 56(84) bytes of data.
64 bytes from 192.168.100.50: icmp_seq=1 ttl=64 time=0.332 ms
64 bytes from 192.168.100.50: icmp_seq=2 ttl=64 time=0.072 ms
64 bytes from 192.168.100.50: icmp_seq=3 ttl=64 time=0.081 ms

Modifying the network configuration¶

If you need to modify the configuration, edit the /etc/network/interfaces file. The Linux rootfs on the SD card is fairly minimal and vi is the only editor available. You can also mount the card on your PC and edit the file.

After changing the configuration, reboot the board (type reboot in the serial console) and test if you can ping it with ping <NEW_IP_ADDRESS>.

SSH access¶

These instructions are optional but can be useful for more convenient updates of the bitstream (no need to remove the SD card from the slot on ZCU104).

Note

SSH on the board is configured to allow passwordless root access for simplicity but if that poses a security risk, modify /etc/ssh/sshd_config according to your needs and add a password for root.

You can log in over SSH using (replace the IP address if you modified board network configuration):

ssh root@192.168.100.50

To access the boot partition, first mount it with:

mount /dev/mmcblk0p1 /boot

This can be automated by adding the following entry in /etc/fstab:

/dev/mmcblk0p1 /boot            vfat    rw              0       2

When the boot partition is mounted, you can use scp to load the new bitstream, e.g.

scp build/zcu104/gateware/zcu104.bit root@192.168.100.50:/boot/zcu104.bit

Then use the reboot command to restart the board.

Controlling the board¶

When the setup has been finished the board can be controlled as any other board. Make sure to use export TARGET=zcu104 before using the scripts (and export IP_ADDRESS=... if you modified the network configuration).

ZCU104 SD card image¶

The easiest way is to use the prebuilt SD card image. It is also possible to build the image from source if needed.

The SD card image consists of a boot partition and a rootfs. Currently, only rootfs is built using buildroot. The boot partition contents have to be built manually.

Bootloaders & kernel¶

Currently, we are using Xilinx FSBL, but it should be possible to use U-Boot SPL (link1, link2).

FSBL and PMU firmware can be built with following the steps:

Create a project from the Vivado example project “Base Zynq UltraScale+ MPSoC” for ZCU104 eval board. Open the PS IP configurator and add the following:

PS-PL Interfaces -> AXI HPM0 FPD (32-bit), AXI HPM1 FPD (32-bit)
disable Carrier Detect in Memory Interfaces -> SD -> SD 0

The following script can be used to generate FSBL, PMU firmware and Device Tree:

#!/usr/bin/tclsh

set hwdsgn  [open_hw_design PATH/TO/Base_Zynq_MPSoC_wrapper.hdf]

generate_app -hw $hwdsgn -os standalone -proc psu_cortexa53_0 -app zynqmp_fsbl -compile -sw fsbl -dir ./fsbl/
generate_app -hw $hwdsgn -os standalone -proc psu_pmu_0 -app zynqmp_pmufw -compile -sw pmufw -dir ./pmufw

set_repo_path PATH/TO/device-tree-xlnx
create_sw_design device-tree -os device_tree -proc psu_cortexa53_0
generate_target -dir dts

close_hw_design [current_hw_design]

The Device Tree generated by Vivado is missing the ethernet-phy node. Modify pcw.dtsi as follows:

&gem3 {
    phy-mode = "rgmii-id";
    status = "okay";
    xlnx,ptp-enet-clock = <0x0>;
    phy0: phy@c {
        reg = <0xc>;
        ti,rx-internal-delay = <0x8>;
        ti,tx-internal-delay = <0xa>;
        ti,fifo-depth = <0x1>;
        ti,rxctrl-strap-worka;
    };
};

Then generate the Device Tree Blob in the dts directory:

gcc -I include -I . -E -nostdinc -undef -D__DTS__ -x assembler-with-cpp -o system.dts system-top.dts
dtc -I dts -O dtb -o system.dtb system.dts

Build the rest of the required components:

ARM trusted firmware: (https://github.com/Xilinx/arm-trusted-firmware.git) e6eea88b14aaf456c49f9c7e6747584224648cb9 (tag: xlnx_rebase_v2.2)
U-Boot: (https://github.com/Xilinx/u-boot-xlnx.git) d8fc4b3b70bccf1577dab69f6ddfd4ada9a93bac (tag: xilinx-v2018.3)
Linux kernel: (https://github.com/Xilinx/linux-xlnx.git) 22b71b41620dac13c69267d2b7898ebfb14c954e (tag: xlnx_rebase_v5.4_2020.1)

Note

It may be necessary to apply the patches from firmware/zcu104/buildroot/board/zynqmp/patches when building U-Boot/Linux.

When building U-Boot, make sure to update its configuration (u-boot-xlnx/.config) with the following options:

CONFIG_USE_BOOTARGS=y
CONFIG_BOOTARGS="earlycon clk_ignore_unused console=ttyPS0,115200 root=/dev/mmcblk0p2 rootwait rw earlyprintk rootfstype=ext4"
CONFIG_USE_BOOTCOMMAND=y
CONFIG_BOOTCOMMAND="load mmc 0:1 0x2000000 zcu104.bit; fpga load 0 0x2000000 $filesize; load mmc 0:1 0x2000000 system.dtb; load mmc 0:1 0x3000000 Image; booti 0x3000000 - 0x2000000"

These configure U-Boot to load the bitstream from the SD card and then start the system. When unfolding CONFIG_BOOTCOMMAND, we can see:

load mmc 0:1 0x2000000 zcu104.bit
fpga load 0 0x2000000 $filesize
load mmc 0:1 0x2000000 system.dtb
load mmc 0:1 0x3000000 Image
booti 0x3000000 - 0x2000000

Example of building ARM Trusted firmware:

make distclean
make -j`nproc` PLAT=zynqmp RESET_TO_BL31=1

Example of building U-Boot:

make -j`nproc` distclean
make xilinx_zynqmp_zcu104_revC_defconfig
# now modify .config directly or using `make menuconfig` as described earlier
make -j`nproc`

Example of building Linux:

make -j`nproc` ARCH=arm64 distclean
make ARCH=arm64 xilinx_zynqmp_defconfig
# optional `make menuconfig`
make -j`nproc` ARCH=arm64 dtbs
make -j`nproc` ARCH=arm64

Then download zynq-mkbootimage and prepare the following boot.bif file:

image:
{
    [fsbl_config] a53_x64
    [bootloader] fsbl.elf
    [pmufw_image] pmufw.elf
    [, destination_cpu=a53-0, exception_level=el-2] bl31.elf
    [, destination_cpu=a53-0, exception_level=el-2] u-boot.elf
}

Open a terminal and make sure that the filepaths specified in boot.bif are correct. Then use``mkbootimage –zynqmp boot.bif boot.bin`` to create the boot.bin file.

Root filesystem¶

Download buildroot:

git clone git://git.buildroot.net/buildroot
git checkout 2020.08.2

Note

As of time of writing git checkout f45925a951318e9e53bead80b363e004301adc6f was required to avoid fakeroot errors when building.

Then prepare configuration using external sources and build everything:

make BR2_EXTERNAL=/PATH/TO/REPO/rowhammer-tester/firmware/zcu104/buildroot zynqmp_zcu104_defconfig
make -j`nproc`

Flashing SD card¶

You can use fdisk to directly partition the SD card /dev/xxx or use the provided genimage configuration to create an SD card image that can be then directly copied to the SD card. The second method is usually more convenient.

Formatting SD card manually¶

Use fdisk or other tool to partition the SD card. The recommended partitioning scheme is as follows:

Partition 1, FAT32, 128M
Partition 2, ext4, 128M

Then create the filesystems:

sudo mkfs.fat -F 32 -n BOOT /dev/OUR_SD_CARD_PARTITION_1
sudo mkfs.ext4 -L rootfs /dev/OUR_SD_CARD_PARTITION_2

Write the rootfs:

sudo dd status=progress oflag=sync bs=4M if=/PATH/TO/BUILDROOT/output/images/rootfs.ext4 of=/dev/OUR_SD_CARD_PARTITION_2

Mount the boot partition and copy the boot files and kernel image created earlier as well as the ZCU104 bitstream:

cp boot.bin /MOUNT/POINT/BOOT/
cp /PATH/TO/rowhammer-tester/build/zcu104/gateware/zcu104.bit /MOUNT/POINT/BOOT/
cp /PATH/TO/linux-xlnx/arch/arm64/boot/Image /MOUNT/POINT/BOOT/
cp /PATH/TO/linux-xlnx/arch/arm64/boot/dts/xilinx/zynqmp-zcu104-revA.dtb /MOUNT/POINT/BOOT/system.dtb

Note: make sure to name the device tree blob system.dtb for the U-Boot to be able to find it (as shown in above commands).

Using genimage¶

The ZCU104 buildroot configuration will also build the genimage tool for the host system by default. The image configuration is described in the firmware/zcu104/image.cfg file. A script named firmware/zcu104/genimage.sh is also provided for convenience. Run it without arguments to get help. Then run it, providing correct paths to all the required files, to generate the zcu104.img file.

The image can be then copied to the SD card device (not partition! so e.g. /dev/sdb, not /dev/sdb1) using dd:

sudo dd status=progress oflag=sync bs=4M if=/PATH/TO/zcu104.img of=/dev/OUR_SD_CARD

Last update: 2024-12-18