RockPro64: booting OS from SATA, installing Arch ARM

If U-boot problems with RockPro64 are fixed then most of this cumbersome setup will become unnecessary and this article will get 5 times shorter. Right now it might be easier to use this fork, author claims SATA support (I didn't try it). I chose upstream U-boot to send bugs to developers.

Currently the master branch of U-boot has 2 problems with RockPro64 - SATA doesn't start and the board is not booting since commit 3ae64582, hangs after loading the kernel. Wrote to developers about the second issue, there seems to be some interest. If it's fixed I'll report SATA issue next. There is a patch with PCIe code from OpenBSD, but the description is not encouraging.

So right now booting upstream U-boot is rather tricky:

• U-boot is on SD card
• There is a small flash drive in USB2 port with ext4 filesystem to store /extlinux/extlinux.conf, kernel and initrd
• Other partitions (/, swap, /home etc) are on SATA devices

Commands below are for Arch or Parabola on amd64. There are cross-compilation packages in Arch repos - let's install them:

# pacman -S arm-none-eabi-gcc aarch64-linux-gnu-gcc dtc

Download latest ATF release (currently arm_cca_v0.3), remove binary files and build:

curl -O https://codeload.github.com/ARM-software/arm-trusted-firmware/tar.gz/refs/tags/arm_cca_v0.3
tar -xzf arm_cca_v0.3
cd arm-trusted-firmware-*
find . -name '*.bin' -exec rm -vf '{}' \;
make realclean
make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rk3399
cd ..

Now we can build U-boot 2020.07, to boot from SD we need u-boot-rockchip.bin:

curl -O https://ftp.denx.de/pub/u-boot/u-boot-2020.07.tar.bz2
tar -xjf u-boot-2020.07.tar.bz2
cd u-boot-2020.07/
export BL31=../arm-trusted-firmware-*/build/rk3399/release/bl31/bl31.elf
export CROSS_COMPILE=aarch64-linux-gnu- CC=aarch64-linux-gnu-gcc
make mrproper
make rockpro64-rk3399_defconfig
make

A convenient distro to get started is Armbian. Download Bullseye from here, write it to SD card (/dev/sde here, needs to be 4Gb or larger):

unxz -dc Armbian_21.08.1_Rockpro64_bullseye_current_5.10.60.img.xz > /dev/sde

Boot RockPro64 with it, download ArchLinux ARM "Generic" root tarball from here. Create a 20Gb partition on SATA device, mount it to /mnt, unpack Arch there:

tar -xzf ArchLinuxARM-aarch64-latest.tar.gz -C /mnt

I wrote the following steps from memory, please contact me if something doesn't work, I'll try to fix it. Mount the filesystem on USB flash drive to /mnt_usb, move boot files from SATA to USB so that U-boot can see them:

mkdir /mnt_usb/boot_arch
cp -r /mnt/boot/* /mnt_usb/boot_arch/
lsblk --fs -oMOUNTPOINTS,UUID|awk '/^\/mnt_usb\s/ {print $2}' # UUID_USB, see below

Add entries to /etc/fstab: 1) to mount the filesystem on USB and 2) to bind the boot_arch directory on USB filesystem to /boot so that pacman uses it during kernel upgrades:

UUID=<UUID_USB> /bootUSB ext4 defaults,noatime 0 1
/bootUSB/boot_arch /boot    none    bind 0 0

Add Arch boot entry to extlinux/extlinux.conf on USB filesystem using PARTUUID of SATA partition (get it with lsblk -o+PARTUUID):

timeout 60
menu title Muh boot options
default Arch_Linux_SSD

label Arch_Linux_SSD
  kernel /boot_arch/Image
  append root=PARTUUID=111111-1111-111 nowatchdog console=ttyS0,115200n8 console=tty1 rootwait rw earlyprintk LANG=en_US.UTF-8
  fdtdir /boot_arch/dtbs
  initrd /boot_arch/initramfs-linux.img

Write U-boot to SD card, if you have a spare one (any size) - use it so you will still have a working one with Armbian. In this example SD card is /dev/sde, check it on your PC with lsblk:

dd if=/dev/zero of=/dev/sde bs=4096 count=2600
dd if=/path/u-boot-git/u-boot-rockchip.bin of=/dev/sde seek=64
sync

Insert it into RockPro64, power on - if all went well you'll see U-boot 2020.07 and Arch will boot from SSD. Add pacman Arch ARM keys as described here.

Download linux-libre-firmware from Parabola repo, install it instead of linux-firmware:

RemoteFileSigLevel = Never # in /etc/pacman.conf
pacman -U https://www.parabola.nu/packages/libre/x86_64/parabola-keyring/download
RemoteFileSigLevel = Required DatabaseOptional # in /etc/pacman.conf
pacman -U linux-libre-firmware-1:1.4-1-any.pkg.tar.xz

I added -mtune=cortex-a72.cortex-a53 to CFLAGS in /etc/makepkg.conf so GCC optimizes makepkg-built packages for this CPU architecture.

What works:

• Bluetooth (headphones)
• 1080p video playback
• WiFi 802.11n Access Point
• Sharing Internet connection via 1 Gbit/s cable (crossover)
• Battle for Wesnoth: it's a big project and it's not in Arch ARM repos, so I decided to build it, see if it'll work on aarch64. Works like a charm: asp export wesnoth ; cd wesnoth ; sed -i 's/^arch=.*/arch=(aarch64)/' PKGBUILD ; makepkg, profit. Thanks to Wesnoth developers for portable code. Made a thread about it on Arch ARM, if I have time will make a github PR to add Wesnoth to Community repo, it looks simple enough
• All the usual software I use on a home PC

Next on the list - linux-libre. I'll have to either build it for Arch/aarch64 or switch to a distro where it's already packaged. This will take some time.