Big, Statically Linked Programs
In a previous article, a Rust program was generated using Cargo:
hello-rust.
$ cd $HOME/src/hello-rust
$ stat -c "%s" target/arm-buildroot-linux-gnueabihf/release/hello-rust
218732
The size of the binary file generated is 213 KB. Stripped, it will go down to
115 KB. This is indeed a big "Hello World"! Due to the architecture of the
runtime, all I/O handling code is included in any statically linked binary,
which is the default for rustc (running strings on the file ends up with a
scary result).
If multiple Rust programs should be included in the target root filesystem of an
embedded system, this would be a terrible waste of space.
It is possible to get a smaller file by adding -C prefer-dynamic to the
rustc command line. But for the program to run properly, the cross-compiled
versions of shared libraries with the missing symbols will have to be copied in
the target root file system.
These libraries were built along with the Rust compiler and are available in
$HOME/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib/.
Generating Dynamically Linked Programs with Cargo
It is possible to generate dynamically linked programs with Cargo. But it
requires a feature only available in version 0.10, which is not available yet.
Build Cargo from Master Branch
Go to the Buildroot base directory and grab the current version of Cargo (master
branch):
git clone --recursive https://github.com/rust-lang/cargo \
$HOME/build/demo-rust/qemu/arm/build/host-cargo-lastest
Configure, build and install:
export PATH=$HOME/build/demo-rust/qemu/arm/host/usr/bin:$PATH
export PKG_CONFIG=$HOME/build/demo-rust/qemu/arm/host/usr/bin/pkgconf
export LIBRARY_PATH=$HOME/build/demo-rust/qemu/arm/host/usr/lib
pushd $HOME/build/demo-rust/qemu/arm/build/host-cargo-latest
./configure --prefix=$HOME/build/demo-rust/qemu/arm/host/usr \
--localstatedir=$HOME/build/demo-rust/qemu/arm/host/var/lib \
--sysconfdir=$HOME/build/demo-rust/qemu/arm/host/etc
make
make install
popd
Build Test Program
Then, create a new version of the Cargo configuration file for the "hello-world"
crate:
cd $HOME/src/hello-rust
cat <<EOF > .cargo/config
[target.arm-buildroot-linux-gnueabihf]
linker = "arm-buildroot-linux-gnueabihf-gcc"
[build]
rustflags = ["-C", "prefer-dynamic"]
EOF
Note the last two lines, where Cargo is instructed to pass an additional flag to
the Rust compiler when building the project. The new flag is quite explicit.
Tell the Rust compiler where the target specification can be found:
export RUST_TARGET_PATH=$HOME/build/demo-rust/qemu/arm/host/etc/rustc
Now, build the crate:
cargo build \
--target=arm-buildroot-linux-gnueabihf \
--release
Once the build is finished, check the size of the generated binary:
$ stat -c "%s" target/arm-buildroot-linux-gnueabihf/release/hello-rust
6880
The binary file has drastically shrinked! Once stripped, it will shrink
to 3680 bytes. Copy it into the target root filesystem:
cp target/arm-buildroot-linux-gnueabihf/release/hello-rust \
$HOME/build/demo-rust/qemu/arm/target/usr/bin
Copying the Missing Shared Libraries
Usually, ldd is used to list the dependencies of a dynamically linked
program. Unfortunately, this tool is not available in the Buildroot environment.
But it is possible to use the cross-compiled version of ld.so, the
dynamic linker/loader, to achieve the same goal.
On the target, the dynamic loader is
HOME/build/demo-rust/qemu/arm/target/lib/ld-2.22.so. As it is only
executable on an ARM machine, QEMU userspace emulator will be used to run it.
$ qemu-arm -r 4.5.0 -E LD_TRACE_LOADED_OBJECTS=1 -E LD_VERBOSE=1 \
> $HOME/build/demo-rust/qemu/arm/target/lib/ld-2.22.so \
> target/arm-buildroot-linux-gnueabihf/release/hello-rust
libstd-ca1c970e.so => not found
libgcc_s.so.1 => not found
libc.so.6 => not found
Version information:
target/arm-buildroot-linux-gnueabihf/release/hello-rust:
libgcc_s.so.1 (GCC_3.5) => not found
libc.so.6 (GLIBC_2.4) => not found
Note
QEMU userspace emulator for ARM is invoked with -r 4.5.0, which is
the Linux version used, to avoid the error "FATAL: kernel too old".
As shown, the program depends on the C runtime (libc.so.6), the GCC
low-level runtime library
(libgcc_s.so.1) and libstd-ca1c970e.so, the Rust standard library.
The dependencies are listed, but as the shared libraries have not been found,
the inspection is incomplete. Here is the result when passing the location of
the libraries:
$ qemu-arm -r 4.5.0 -E LD_TRACE_LOADED_OBJECTS=1 -E LD_VERBOSE=1 \
> $HOME/build/demo-rust/qemu/arm/target/lib/ld-2.22.so \
> --library-path $HOME/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib:$HOME/build/demo-rust/qemu/arm/target/lib \
> target/arm-buildroot-linux-gnueabihf/release/hello-rust
libstd-ca1c970e.so => /home/stewie/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib/libstd-ca1c970e.so (0xf64be000)
libgcc_s.so.1 => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1 (0xf6492000)
libc.so.6 => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6 (0xf6356000)
libdl.so.2 => /home/stewie/build/demo-rust/qemu/arm/target/lib/libdl.so.2 (0xf6342000)
libpthread.so.0 => /home/stewie/build/demo-rust/qemu/arm/target/lib/libpthread.so.0 (0xf6319000)
libm.so.6 => /home/stewie/build/demo-rust/qemu/arm/target/lib/libm.so.6 (0xf629f000)
/lib/ld-linux-armhf.so.3 => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so (0xf6fcf000)
Version information:
target/arm-buildroot-linux-gnueabihf/release/hello-rust:
libgcc_s.so.1 (GCC_3.5) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
/home/stewie/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib/libstd-ca1c970e.so:
libdl.so.2 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libdl.so.2
libgcc_s.so.1 (GCC_4.3.0) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libgcc_s.so.1 (GLIBC_2.0) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.0) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libgcc_s.so.1 (GCC_4.0.0) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.3.1) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libgcc_s.so.1 (GCC_3.5) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1
libm.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libm.so.6
libpthread.so.0 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libpthread.so.0
libc.so.6 (GLIBC_2.17) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
/home/stewie/build/demo-rust/qemu/arm/target/lib/libgcc_s.so.1:
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
/home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6:
ld-linux-armhf.so.3 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so
ld-linux-armhf.so.3 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so
/home/stewie/build/demo-rust/qemu/arm/target/lib/libdl.so.2:
ld-linux-armhf.so.3 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so
libc.so.6 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
/home/stewie/build/demo-rust/qemu/arm/target/lib/libpthread.so.0:
ld-linux-armhf.so.3 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so
ld-linux-armhf.so.3 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/ld-2.22.so
libc.so.6 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
/home/stewie/build/demo-rust/qemu/arm/target/lib/libm.so.6:
libc.so.6 (GLIBC_PRIVATE) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
libc.so.6 (GLIBC_2.4) => /home/stewie/build/demo-rust/qemu/arm/target/lib/libc.so.6
That is quite an exhaustive list! But it shows that only libstd-ca1c970e.so
is required, as all the other dependencies are already in the target root
filesystem.
To copy it, execute:
cp $HOME/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib/libstd-ca1c970e.so \
$HOME/build/demo-rust/qemu/arm/target/usr/lib
chmod a+wx $HOME/build/demo-rust/qemu/arm/target/usr/lib/libstd-ca1c970e.so
As this library has been originally installed in
$HOME/build/demo-rust/qemu/arm/host/usr/lib, the runtime search path in the ELF file is incorrect. To check
and modify it, build patchelf:
make O=$HOME/build/demo-rust/qemu/arm host-patchelf
The current value of rpath is:
$ patchelf --print-rpath $HOME/build/demo-rust/qemu/arm/target/usr/lib/libstd-ca1c970e.so
/home/stewie/build/demo-rust/qemu/arm/host/usr/lib/rustlib/arm-buildroot-linux-gnueabihf/lib
To set it to an empty value, execute:
patchelf --set-rpath '' $HOME/build/demo-rust/qemu/arm/target/usr/lib/libstd-ca1c970e.so
Run Test Program from System
Go back to the Buildroot directory. Before rebuilding the system image, generate
the cache of the loader (this operation is not automagically performed by
Buildroot):
qemu-arm -r 4.5.0 $HOME/build/demo-rust/qemu/arm/staging/sbin/ldconfig -v \
-r $HOME/build/demo-rust/qemu/arm/target
Then, rebuilt the system image:
make O=$HOME/build/demo-rust/qemu/arm
Now, start the system with QEMU:
qemu-system-arm \
-M vexpress-a9 \
-m 256 \
-kernel $HOME/build/demo-rust/qemu/arm/images/zImage \
-dtb $HOME/build/demo-rust/qemu/arm/images/vexpress-v2p-ca9.dtb \
-drive file=$HOME/build/demo-rust/qemu/arm/images/rootfs.ext2,if=sd,format=raw \
-append "console=ttyAMA0,115200 root=/dev/mmcblk0" \
-serial stdio \
-net nic,model=lan9118 \
-net user
Log as "root" (no password) and execute the test program:
Welcome to Buildroot
buildroot login: root
# hello-rust
Hello World!
Excellent!
Note
When using QEMU from Debian Jessie (version 2.1.2), the test program
can not be executed because the loader fails to read
libstd-ca1c970e.so. Using strace shows that the kernel
attempts to read beyond the limits of the filesystem, which is an EXT2
image on an emulated SD card. dmesg also shows complaints about the
SD card having a bad geometry.
This is a QEMU bug, which does not seem to align the size of the
filesystem (8192 KB) correctly. Setting the "exact size in blocks" to
16384 in the "Filesystem images" menu of Buildroot and regenerating
the image solves the problem.