Updates from the Tinyverse

March 2022 Edition

Mar 31, 2022

Hi Tinyversians!

It has been a busy month and the HOTG team has been plugging away across our platform. Let’s jump straight into it :D

Platform Updates

React Native

First, we have finally added support for React Native! React Native is a popular application development framework that uses a bridge over JS to call native modules such as librunecoral. Behind the scenes we had to work hard on memory copy over the bridge. However, with some elbow grease we were able to crack this problem and now have an initial version working. Have a look at an example app in action here.

Repo with example: https://github.com/hotg-ai/react-native-rune

Runtime: Rust first, iOS build & Tensorflow 3.8 Support

The Librunecoral (librc) project is our native bindings across various platforms and provides a consistent runtime for Rune to be executed on. Until this release librc was leveraging an older version of our runtime implementation built in C/C++. In this release we have moved over our entire runtime to Rust with bindings to Wasm3. This is a big deal as it means the Rust runtime from linux embedded, iOS, Android and more is less fragmented. We have had a Rust runtime implementation for several weeks now but were blocked by our ability to build on iOS.

As of 3 days ago our engineering team were finally able to get bazel and iOS builds working!

The cherry on top is that in the process of upgrading bazel builds we were able to add Tensorflow 3.8 support!

Rune ABI and WIT

Investing in better interface definitions

With more implementations with partners and developer we learned that our initial implementation of our Rune interface has begun to show limitations. Each iteration of the ABI required us to go an update a variety of runtime implementation across the ecosystem. Needless to say this really hamstrung our ability to iterate and release new ABI improvements. One solution to this are declarative interfaces that can be used to auto generate code bindings to the various runtimes. WIT bindgen is early stage project from the byte code alliance. Using WIT we are able to provide clear interface definitions for all of our Rune proc blocks. By making our interfaces declarative we can now begin work on updating our ABIs to support more features rapidly!

An example of how we use WIT is to provide tensor metadata.

/// Information about a tensor.
resource tensor-metadata {
    /// Create a new named tensor.
    static new: function(name: string) -> tensor-metadata
    /// A human-friendly description of the tensor.
    ///
    /// The text may include markdown.
    set-description: function(description: string)
    /// Add a hint that provides the runtime with contextual information about
    /// this node.
    add-hint: function(hint: tensor-hint)
}

This interface is then fulfilled with in our proc-blocks in rust as

let output = TensorMetadata::new("labels");
            output.set_description("The corresponding labels.");
            let hint = supported_shapes(
                &[ElementType::Utf8],
                Dimensions::Fixed(&[0]),
            );
            output.add_hint(&hint);

You can see the entire Rune interface defined in github.

What’s next

We plan on redesigning more of our ABI using WIT to iterate on ways to do graph-context sharing which will allow for better error reporting and ragged tensors. If you are curious about our work here please come chat with us on discord.

Let's chat about WIT

Hammer Studio Forge

Security, UX, Diagramming Engine and Documentation

We have been meeting many developers since our Feb8th launch and rapidly releasing improvements. Our release notes are published on studio show details on various improvements so far.

One specific update that I am particularly excited about is our diagramming engine.

On Forge Studio we have completely overhauled our rendering engine so that we can make designing pipelines less painful. The new diagram rendering engine provides better interaction support and eventing. Better eventing allows us to finally have visual validation

This allows ML pipeline designers to rapidly see issues in their transformations and tensors.

Documentation Improvements

Over on the documentation side we have put a ton of new articles from examples tutorials and more. One of our community’s favorite is building custom computer vision models for the edge.

TinyMLOps - Paper submission

One of the paper we authored with the Dr. Pieter Simoens from Ghent University has now been accepted for the Parallel AI and Systems for the Edge workshop (paise 2022) https://www.paise.org.

It is titled: TinyMLOps: Operational Challenges for Widespread Edge AI Adoption and you can read it all here: https://arxiv.org/abs/2203.10923

In a continuously fragmenting and distributed world, ensuring having a strategy for the Edge becomes critical. There are inherent production level challenges for the Edge such as:

repeatability
deployments
security
IP protection
orchestration

Give the paper a read for a broad understanding of the challenges, and feel free to reach out to us if you want to discuss more :).

Forge App of the Month!

Game SDK for Mobile Web

If you made it this far you get to see an easter egg :D As you may have heard we have been getting into the unicorn business with Laguna Games :) We are putting together an SDK that will let you use Rune with Unity games :D

My best,

Kartik (f0rodo)

Share tinyVerse

HOT-G Socials:

Twitter @hotg_ai and @hammer_otg | LinkedIn | Discord

tinyVerse