After recently receiving the shipment for my Onion Omega2 Kickstarter reward, I did as any other software developer might do: I started figuring out what it would take to get software running on it. Onion’s Omega2 documentation has information about installing and using Python, but while this is powerful and aids product adoption, limitations of developing directly on the device soon appear. Limited disk space, limited RAM, and limited CPU speeds will hinder development and builing of most compiled languages. To me, this sounds like a great opportunity to learn how to cross-compile applications, allowing for development and building of applications in my normal development environment. I’ve been tinkering with Rust recently, so it became my language of choice for this exercise.
tl;dr It works.
— Shane Logsdon (@shanelogsdon)
January 11, 2017
Not having cross-compiled applications before, I did some research into what it takes to cross-compile:
- Know your target triple
- Have your application code
- Have a build toolchain from your target available on your host (build) system
- Use the target toolchain to build you application
The target triple (or triplet) is an identifier that represents three pieces of information, architecture, vendor, and operating system, and will typically follow the form:
I wanted to build something in Rust that was more than a simple “Hello World” application that wrote to the console, so I looked to Rocket to build a simple web application server. Let’s take a look at the application code to see what we’re working with.
Scaffold the project
Do a quick build to pull our dependencies down:
rocket_codegen requires some Rust nightly features at the moment, so lets use
rustup to override our current Rust toolchain:
That time should do it if you’re using a nightly release for the first time, but if your’ve already had a nightly installed, you may run into this issue:
We’re told that our installed version of Rust nightly is too old, and we need to install a newer one. Luckily, it’s a couple of quick commands to fix:
Once our initial build completes, we’ll want to update our application code in
src/main.rs to leverage Rocket:
We can then build again and test our application (Rocket listens on
http://localhost:8000/ by default). At this point we have a working application for our host system, which in my case has the triple
Since we now have a working application, we need to figure out how to get our application cross-compiled. Some googling resulted in some useful information specifically for Rust.
rust-cross has some excellent information on this process, but since I didn’t even know what the Omega2’s architecture was, I figured I better find out. I booted up my Omega2+ and
ssh‘d into it:
That told me enough to start my search for the reuired build chain. At this point, I went to
rustup to see what architecture’s it supported.
rustupnot only manages Rust stable, beta, and nightly installations but also manages Rust toolchains for all the architectures Rust supports!
rustup is showing 6
mips-related targets. We’ve narrowed it down some, but we still don’t know the exact one we require or if Rust/
rustup even support it. I took to looking through the community forums searching for
mips and began to see others looking to do some cross-compilation of code. Across a few separate thread, I put together some information:
- The Omega2’s use the MediaTek MT7688 SoC (system on chip) which include a MIPS® 24KEc™ CPU
- The Omega2 OS is based on the LEDE Project, a fork of the OS behind OpenWrt
- OpenWrt/LEDE have SDKs for building the OS firmware images which include the build toolchain
Eventually, I found a few forum threads with references to WereCatf’s repository, a GitHub fork of the LEDE Project’s SDK with the necessary changes to add the Omega2 and Omega2+ build DTS (device tree source) configurations add a few other fixes. With the SDK, we have everything we need to build our application for the Omega2, but now the SDK needs to be built since we only have the source and nothing specific for the Omega2.
Luckily, the build process for the LEDE SDK is the same as the OpenWrt SDK, and at least for MacOS, the build requirements are the same. I’ve included OpenWrt’s instructions for MacOS 10.11 here, but other versions and OS’s can be found on their documentation site.
Install Xcode or at least Xcode command line tools from the MacOSX App Store
Add duplicates repository to homebrew for grep formulae:
brew tap homebrew/dupes
Install additional formulae:
brew install coreutils findutils gawk gnu-getopt gnu-tar grep wget quilt xz
gnu-getoptis keg-only, so force linking it:
brew ln gnu-getopt --force
To get rid of “date illegal option” you can add to your
.bash_profile(wasn’t required for me):
OS X by default comes with a case-insensitive filesystem. OpenWrt won’t build on that. As a workaround, create a (Sparse) case-sensitive disk-image that you then mount in Finder and use as build directory:
hdiutil create -size 20g -type SPARSE -fs "Case-sensitive HFS+" -volname OpenWrt OpenWrt.sparseimagehdiutil attach OpenWrt.sparseimage
Change to your newly created and mounted disk image:
Now proceed normally (
If, like me, you have no idea how to “proceed normally”, let me fill you in. We’re going to obtain the source, configure it for our needs, and build it.
This one’s going to be quick and simple using
Since OpenWrt/LEDE can be used on multiple architectures, we need to configure the SDK to be compatible with the Omega2. There are a few ways to do this, but we’ll use
make menuconfig here for a
ncurses-based configuration process.
Tip: The menus use
downkeys to move between options,
rightto move between commands for a given screen (located at the bottom),
enterto select a command (usually “Select”, “Exit”, and “Save”), and
spaceto enable/select an option.
The three items we need to set (with desired values) are:
- Target System:
MediaTek Ralink MIPS
MT7688 based boards
- Target Profile:
Note: I also enabled the
Build the LEDE SDKand
Package the LEDE-based Toolchainoptions, but I have no idea if this affects the end result. They sounded important/useful. Having those enabled allowed for me to use the toolchain later, but I didn’t have the desire to go back to check if it was necessary.
Don’t forget to save the configuration or else the SDK will build with its defaults.
Building the SDK’s toolchain is another easy and simple process, but it takes some time to complete.
Let your system do its thing for a while, and do something enjoyable. You can also wait, wait, wait, wait. The good news to take away here is that this only needs to be done once per architecture for your build environment, so if you only use this SDK for the Omega2, it will only need to be built again if you want the build toolchain on another system Docker, etc. Eventually, it should finish, leaving your toolchain within the SDK directory:
mips targets (pasted below), we may be able to choose one finally:
Our toolchain seems to be for the
mipsel architecture and is compatible with
libc compatible library for compiling statically-linked applications, so the
mipsel-unknown-linux-musl Rust toolchain could work for us. Attempting to run
cargo compile at this point will result in a big wall of text and the following error:
This is due to my host system’s linker (
/usr/bin/cc) being used during the build but being incompatible with the
mipsel architecture. Being completely new to cross-compilation, I had no idea how to use the correct build toolchain. Luckily, Rust ecosystem developers love documentation, and Cargo’s documentation includes a page on configuration that gave me a hint in the
target.$triple.linker configuration key:
Adding that to my
Cargo.toml file … didn’t help. Turns out that target configuration options are ignored in a project’s
Cargo.toml and need to be in a
.cargo/config (also covered by the Cargo documentation page on configuration). The resulting directory structure with the added
cargo build again bears some results:
It’s built, but does it run? Let’s ship it over to the Omega2+ to test:
That uploaded the application’s release binary to the root user’s
$HOME directory and can be ran with
cd /root && ./rocket_testing:
— Shane Logsdon (@shanelogsdon)
January 11, 2017
I don’t believe this is perfect, but it will get the majority of applications compiled for the Omega2. I’ve already ran into an issue when using Diesel and Postgres in a project, but I feel like it only needs some tweaking to get it going. This post will be updated once I figure that bit out.
The Omega2 isn’t the only build target available for Rust, as shown by
rustup target list, and is accompanied by
arm (e.g. Raspberry Pi Zero),
armv7 (e.g. Raspberry Pi 2 Model B), and
wasm32 (WebAssembly, currently available in Chrome Canary and Firefox Nightly). Cross-compilation could allow you to target all these platforms with the same code base, useful if you’re building an Internet of Things
botnet application and want to use multiple device types, or it could allow you to ship compiled binaries for your customers’s various production environments using a single build environment configuration.