Had to downgrade gcc to version 11 on Arch ARM (otherwise ATF won't build),
then (using git repos):
arm-trusted-firmware $ git checkout v2.7.0
arm-trusted-firmware $ find . -name '*.bin' -exec rm -vf '{}' \;
arm-trusted-firmware $ make PLAT=rk3399 bl31
arm-trusted-firmware $ cd ../u-boot
u-boot $ git checkout v2022.04
u-boot $ make mrproper && make rockpro64-rk3399_defconfig
u-boot $ make BL31=../arm-trusted-firmware/build/rk3399/release/bl31/bl31.elf
Create a GPT partition on SD card to hold boot files:
/boot1GiBSD/
├── boot_arch
│ ├── dtbs/
│ ├── Image
│ └── initramfs-linux.img
└─── extlinux
└── extlinux.conf
Write U-boot image to the same SD card (it will not break GPT) - directions in
doc/README.rockchip in U-boot source tree, "Booting from an SD card on
RK3399", Option 3. Works fine.
SPI flash doesn't work though, sf probe complains:
unrecognized JEDEC id bytes: ff, ff, ff
FreeBSD
- Download
SD Card Image for armv7/GENERICSD from
here, burn to SD card as usual
- This image does not contain a U-boot capable of booting Cubieboard1. I
couldn't figure out how to download FreeBSD packages without having FreeBSD
installed, but U-boot image for Cubieboard1 is available in Parabola
repos,
after unpacking write it to the same SD card with
dd
if=u-boot-sunxi-with-spl.bin conv=notrunc,sync of=/dev/sd_card_device bs=1k
seek=8. More info here
- Boot, go through installation wizard, reboot, log in. Out of the box, FreeBSD
takes 2.3Gb of space
$ uname -srm
FreeBSD 13.1-RELEASE arm
FreeBSD spoofed the MAC address for Cubieboard's builtin NIC for some reason.
Overall it works well - boots quickly, the usual shell work feels
snappy.
NetBSD
- Download
armv7.img.gz right from the homepage,
burn to SD card as usual
- Add U-boot, the command is slightly different from FreeBSD:
dd
if=u-boot-sunxi-with-spl.bin conv=sync of=/dev/sd_card_device bs=1k seek=8
(source)
- Download .tgz "sets" from
here,
place them on a
FAT partition on a USB stick
- Insert both SD card and USB stick, boot. Mount the partition on USB stick.
An adventure in and of itself - the name of the block device is different
across
commands
(
/dev/rsd0d, sd0 (not /dev/sd0), /dev/rsd0e and I don't remember what
I used for mounting) - see the link for details
- Run
sysinst command, choose "Local directory", set the "Binary sets dir" to
mounted USB partition dir. Clear all the other text fields, otherwise it will
fail with some cryptic error. Choose "(Re-)Install additional sets" and
"Minimal install"
- Reboot, log in. Out of the box, NetBSD takes 2.2Gb of space
$ uname -srm
NetBSD 9.2 evbarm
NetBSD is significantly slower than FreeBSD (I/O operations, installing
packages, ssh logins etc). By default, the hostname is the one received from
DHCP server. NetBSD used the native MAC address of the builtin NIC, just
like Debian.
OpenBSD
I followed the installation manual to get it onto the SD card and it sent the
board into a reboot loop with "DRAM init failure" error. I searched for it on
the web, didn't find any obvious solutions/workarounds, not really interested
in digging into it at the moment.
I want to use this board as a CI target, the BSD + armv7hf combination
should be exotic enough to catch portability issues.
I wanted to self-host a server to sync my contacts across devices. One that uses
an open protocol (CardDAV) and easy to self-host.
Radicale was very easy to set up on Arch, but
davx5 client (Android) couldn't sync the changes.
Xandikos works flawlessly in combination with nginx. The
package is in Arch repos but required some assembly:
# (as root)
mkdir /var/lib/xandikos /etc/xandikos
chown xandikos:xandikos /var/lib/xandikos
useradd -U -s /usr/bin/nologin xandikos
htpasswd -c /etc/xandikos/htpasswd usr
/etc/systemd/system/xandikos.service, ugly hack here because Xandikos can't
use an existing socket:
[Unit]
Description=Xandikos CalDAV/CardDAV server
After=network.target
[Install]
WantedBy=multi-user.target
[Service]
RuntimeDirectory=xandikos
RuntimeDirectoryMode=0770
User=xandikos
Group=http
ExecStart=/usr/bin/xandikos \
-d /var/lib/xandikos \
--current-user-principal=/usr \
-l /run/xandikos/socket
ExecStartPost=/usr/bin/sh -c 'sleep 2; chmod g+w /run/xandikos/socket'
Restart=on-failure
KillSignal=SIGQUIT
Type=simple
/etc/nginx/sites-available/xandikos:
upstream xandikos {
server unix:/run/xandikos/socket; # nginx will need write permissions here
}
server {
server_name home-dav;
# Service discovery, see RFC 6764
location = /.well-known/caldav {
return 307 $scheme://$host/user/calendars;
}
location = /.well-known/carddav {
return 307 $scheme://$host/user/contacts;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://xandikos;
auth_basic "Login required";
auth_basic_user_file /etc/xandikos/htpasswd;
}
listen 192.168.2.1:8099;
}
Add this line to nftables ruleset
to allow sync on LAN only:
tcp dport 8099 ip saddr { 192.168.2.0/24 } ip daddr 192.168.2.1 accept comment "Accept connections to xandikos behind nginx"
To sync to local directories I use
vdirsyncer, on Android - davx5 (in
F-Droid repos).
If U-boot problems with RockPro64 are fixed then most of this cumbersome setup
will become unnecessary and this article will get 5 times shorter. Right now it
might be easier to use this fork, author
claims SATA support (I didn't try it). I chose upstream U-boot to send bugs to
developers.
Currently the master branch of U-boot has 2 problems with RockPro64 - SATA
doesn't start and the board is not booting since commit 3ae64582,
hangs after loading the kernel. Wrote to developers about the
second issue, there seems to be some interest. If it's fixed I'll report SATA
issue next. There is a
patch with PCIe code from OpenBSD,
but the description is not encouraging.
So right now booting upstream U-boot is rather tricky:
- U-boot is on SD card
- There is a small flash drive in USB2 port with ext4 filesystem to store
/extlinux/extlinux.conf, kernel and initrd
- Other partitions (
/, swap, /home etc) are on SATA devices
Commands below are for Arch or Parabola on amd64. There are cross-compilation
packages in Arch repos - let's install them:
# pacman -S arm-none-eabi-gcc aarch64-linux-gnu-gcc dtc
Download latest ATF release (currently arm_cca_v0.3), remove binary files and
build:
curl -O https://codeload.github.com/ARM-software/arm-trusted-firmware/tar.gz/refs/tags/arm_cca_v0.3
tar -xzf arm_cca_v0.3
cd arm-trusted-firmware-*
find . -name '*.bin' -exec rm -vf '{}' \;
make realclean
make CROSS_COMPILE=aarch64-linux-gnu- PLAT=rk3399
cd ..
Now we can build U-boot 2020.07, to boot from SD we need u-boot-rockchip.bin:
curl -O https://ftp.denx.de/pub/u-boot/u-boot-2020.07.tar.bz2
tar -xjf u-boot-2020.07.tar.bz2
cd u-boot-2020.07/
export BL31=../arm-trusted-firmware-*/build/rk3399/release/bl31/bl31.elf
export CROSS_COMPILE=aarch64-linux-gnu- CC=aarch64-linux-gnu-gcc
make mrproper
make rockpro64-rk3399_defconfig
make
A convenient distro to get started is Armbian. Download Bullseye from
here, write it to SD card
(/dev/sde here, needs to be 4Gb or larger):
unxz -dc Armbian_21.08.1_Rockpro64_bullseye_current_5.10.60.img.xz > /dev/sde
Boot RockPro64 with it, download ArchLinux ARM "Generic" root tarball from
here. Create a 20Gb
partition on SATA device, mount it to /mnt, unpack Arch there:
tar -xzf ArchLinuxARM-aarch64-latest.tar.gz -C /mnt
I wrote the following steps from memory, please contact me if something doesn't
work, I'll try to fix it. Mount the filesystem on USB flash drive to /mnt_usb,
move boot files from SATA to USB so that U-boot can see them:
mkdir /mnt_usb/boot_arch
cp -r /mnt/boot/* /mnt_usb/boot_arch/
lsblk --fs -oMOUNTPOINTS,UUID|awk '/^\/mnt_usb\s/ {print $2}' # UUID_USB, see below
Add entries to /etc/fstab: 1) to mount the filesystem on USB and 2) to bind
the boot_arch directory on USB filesystem to /boot so that pacman uses it
during kernel upgrades:
UUID=<UUID_USB> /bootUSB ext4 defaults,noatime 0 1
/bootUSB/boot_arch /boot none bind 0 0
Add Arch boot entry to extlinux/extlinux.conf on USB filesystem using PARTUUID
of SATA partition (get it with lsblk -o+PARTUUID):
timeout 60
menu title Muh boot options
default Arch_Linux_SSD
label Arch_Linux_SSD
kernel /boot_arch/Image
append root=PARTUUID=111111-1111-111 nowatchdog console=ttyS0,115200n8 console=tty1 rootwait rw earlyprintk LANG=en_US.UTF-8
fdtdir /boot_arch/dtbs
initrd /boot_arch/initramfs-linux.img
Write U-boot to SD card, if you have a spare one (any size) - use it so you will
still have a working one with Armbian. In this example SD card is /dev/sde,
check it on your PC with lsblk:
dd if=/dev/zero of=/dev/sde bs=4096 count=2600
dd if=/path/u-boot-git/u-boot-rockchip.bin of=/dev/sde seek=64
sync
Insert it into RockPro64, power on - if all went well you'll see U-boot 2020.07
and Arch will boot from SSD. Add pacman Arch ARM keys as described here.
Download linux-libre-firmware from Parabola repo, install
it instead of linux-firmware:
RemoteFileSigLevel = Never # in /etc/pacman.conf
pacman -U https://www.parabola.nu/packages/libre/x86_64/parabola-keyring/download
RemoteFileSigLevel = Required DatabaseOptional # in /etc/pacman.conf
pacman -U linux-libre-firmware-1:1.4-1-any.pkg.tar.xz
I added -mtune=cortex-a72.cortex-a53 to CFLAGS in /etc/makepkg.conf so
GCC optimizes makepkg-built packages for this CPU architecture.
What works:
- Bluetooth (headphones)
- 1080p video playback
- WiFi 802.11n Access Point
- Sharing Internet connection via 1 Gbit/s cable (crossover)
- Battle for Wesnoth: it's a big project and it's
not in Arch ARM repos, so I decided to build it, see if it'll work on aarch64.
Works like a charm:
asp export wesnoth ; cd wesnoth ; sed -i
's/^arch=.*/arch=(aarch64)/' PKGBUILD ; makepkg, profit. Thanks to Wesnoth
developers for portable code. Made a thread about it on Arch ARM, if I have
time will make a github PR to add Wesnoth to Community repo, it looks simple
enough
- All the usual software I use on a home PC
Next on the list -
linux-libre. I'll have to
either build it for Arch/aarch64 or switch to a distro where it's already
packaged. This will take some time.
I've been using a libreboot-ed ThinkPad T500, but USB
system started to show its age - only one port works, and it's a random one on
each boot.
Started to look for a replacement with these requirements:
- Supported by Libre/Free Software - linux-libre, upstream U-boot without
vendor blobs
- Ability to install OS to SATA/NVMe drive. SD/eMMC/USB are too
slow/unreliable
- Sharing Internet connection via twisted pair and WiFi
- Working Bluetooth
- Upstream software: linux-libre, U-boot. I don't want to install forks of
outdated versions.
Main candidates:
- Blackbird, POWER9
CPU. Has a BMC, all firmware sources are available. Firmware for built-in
NIC is in development, described here. Drawbacks are price and the fact
that you can't order it (logistical problems due to covid, according to
Talospace)
- MNT Reform. ARM64 laptop, no vendor blobs except
16KiB one to initialize RAM. Shipping next year
- RockPro64 - RK3399 SBC, 4G RAM,
good-enough CPU (2xA72 + 4xA53), PCIe x4, 4xUSB, 1xRJ-45 (1 Gbit/s). Gentoo
approves
- Gigabyte GA-G41M-ES2L for
libreboot treatment
- HoneyComb LX2
aarch64, 16 A72 cores, a full-fledged motherboard. More expensive than
RockPro64 (it has much more functionality). I couldn't find if it requires
vendor blobs, and it's not in U-boot repo (by name)
I chose RockPro64. Pine64 sells a SATA card for it but it uses an ASM chip.
Judging by forum threads and my experience ASM chips are not reliable - my
mPCIe card on ASM1061 works with HDD but not SSD for some reason. Found this
card - 4 ports, Marvell
88SE9215
chip, supported by Linux. Bluetooth and WiFi adapters that do not require
vendor blobs were found on H-node (an excellent
resource).
Resulting BOM:
- SATA controller PCE4SAT-M02 (PCIe)
- Bluetooth controller Asus BT400 (USB)
- WiFi D-Link DWA-126 (USB) - Access Point mode
- Ethernet D-Link Dub-E100 100Mbit/s (USB) - for ISP connection
- 2 USB hubs
- Chipset cooler Zalman ZM-NB32J - fits, here is an old review
- Aluminum box 155x85x120 mm
- Regular PC PSU (I already had one)
DIY power connector Molex -> DC barrel:
Bolted a small aluminum plate to SATA card to hold the cables:
More photos:
Next up: installing U-boot and Arch Linux ARM.