Rust and Tinyverse

Why we are building on Rust

The Tinyverse is a family of projects from HotG which brings machine learning into the mainstream.

Our flagship project is Rune, a tool for packaging up machine learning pipelines and allowing them to be deployed anywhere, be it the server, a mobile device, or the IoT.

Here in the Tinyverse, we love Rust. The language itself is a perfect fit for what we do, we've been able to save thousands of developer hours by leveraging existing libraries, and the online community has been nothing but supportive. If you are just as in love with Rust as we are, we are looking for you!

Ferris the Rust Mascot <3

It's just been an all-around nice experience - a trait we would like to bring to the Tinyverse community.

The Ecosystem

A core component of Rune is the Machine Learning pipeline. Pipelines are a series of data transformations implemented as either Rust code (“Procedure Blocks”, proc blocks for short) or Machine Learning models, which will take your inputs and yield the output you desire.

One of Rune's major themes is lowering the barrier to entry for machine learning, but this is only possible when the platform can provide the tools and abstractions you need.

The phrase “batteries included” comes to mind here.

You can see this when looking at the wider Rust ecosystem. While still a youngster compared to older cousins like pip and npm, crates.io already has most of the tools you will ever need and to achieve something similar, we’ll need a couple of things…

Library Support

There should be a large range of libraries that people can draw upon. This is because the range of possible applications for Machine Learning is diverse, with developers needing to draw on a large variety of algorithms to fit their data into the right format before it can be passed to a Machine Learning model.

Typical proc blocks wrap existing algorithms (e.g., the Fast Fourier Transform) and provide APIs consumed by other steps in the pipeline. The implementation isn’t necessarily difficult but requires grunt work to make all various algorithms work in Rune. As we explore more use-cases, we are building an ecosystem of proc-blocks that can be reused by any other Rune developer.

If you are someone passionate about Machine Learning or know a thing or two about the industry, give Rune a try and let us know how you go. Or even better, pop over to our Discord server and discuss it with other like-minded people!

Equally as importantly, it needs to be easy to find these libraries. There’s no point in having access to tens of thousands of libraries if you can’t find what you need, after all.

This is a lot easier because it can be solved by developing better tools or central registries. It’s still an area of active research, but we’ll keep you posted.

Portability

One of the goals for Rune is to be portable and executable everywhere. That means our code needs to be able to run on a wide range of environments, varying from x86 servers, mobile platforms like iOS and Android, single board computers like the Raspberry Pi or Jetson Nano, or bare metal devices.

Cross-compilation across devices is an integral part of portability. Rust's cross-compilation story is second to none and serves us very well in providing portability.

Our relationship with cross-compilation is two-fold.

On one hand, a Rune itself is just a Rust project that has been compiled to WebAssembly. On the other hand, at the most abstract, a Rune is a pure function that takes inputs and generates outputs, with little interaction with the outside world.

All we needed to do was add a #![no_std] attribute to the generated code and make sure all dependencies compile without the standard library and everything fell into place.

Furthermore, the library which takes a Rune and lets you run it inside your application (the Rune “runtime”) needs to compile to machine code on the target platform.

We use wasmer as our WebAssembly VM and have had nothing but good experiences with it. When it came to compiling the runtime library for Android all we needed to do was set a couple of environment variables so the cc crate could compile some setjmp/longjmp shims, then cargo build --target aarch64-linux-android Just Worked.

We want the users of Rune to have a similar experience. We’ll take on the complexity so our downstream users don’t have to!

User-Friendly Tooling

At some point, every single Rustacean who has ever written a piece of code has hit a compiler error. Regardless of your experience levels, typos and type errors are inevitable.

Something that sets the Rust compiler apart from almost every other compiler is how hard it tries to tell the user what went wrong and how.

A lot of this boils down to understanding errors that are commonly encountered by new users and putting in the effort to guide them to the correct solution. Crates like codespan-reporting are very important to this process because they let you track the location of an element in the source code and provide a way to print diagnostics that point to those elements or provide suggestions.

We have an integration test suite designed specifically with this usability in mind. It borrows heavily from the Rust Compiler Testing Framework, using various example files and makes sure rune build the command to generates the desired output.

This test suite is in continuous development so if you spot an error message that is less than ideal, create a ticket on GitHub or message us on Discord.

Escape Hatches

All successful projects will eventually reach a point where their users want to do something the original authors could never have contemplated.

You can see this in the Rust standard library in the form of std::os::unix::io::AsRawFd or various methods for working with raw pointers and unchecked operations. The standard library authors understood that sometimes developers will need to side-step their abstractions, typically because it is required by something else (e.g. interacting with other OS APIs) or to give the developer more control (e.g. performance).

With Rune, we understand this and have designed everything with this in mind. We want to make easy things easy and hard things possible.

At its core, a Rune is just a Rust project which uses Rune’s libraries to encode an ML pipeline in a way that the outside world can use. The rune build command is just a shortcut for taking in a declarative specification of a Rune and generating the corresponding Rust code.

By default, these generated projects are saved to your cache directory (~/.cache/runes/ on Linux). That means if you want to write your own Rune from scratch it’s easy to look at an existing one, copy it to its own repo, and add your own modifications from there.

Give it a try. You may find that writing a Rune from scratch isn’t as hard as you think.

Conclusions

Computers are one of the most complex inventions humans have come up with and arguably the best compliment you can give a piece of software is that It Just Works.

There are a large number of factors that contribute to the It Just Works experience, and here at HotG, we are dedicated to applying them to each and every aspect of the Tinyverse.