Foundations of Our Edge Computing Platform
Guiding principles behind Hammer Forge
My cofounder wrote a compelling piece earlier this month on building for the edge and beyond. It chronicles an honest journey we have undertaken from our experience building for the cloud, deploying real world AI in domains outside of ad-tech, and into the world of edge computing. This post highlights the hacks we have had to deal with to make things work in production.
Engineers, Product Managers, CTOs, and CSOs coming from the world of cloud computing are used to a plethora of tools to make their life less complicated, but many of these are missing for running production edge computing.
Six foundation pieces
When we started building HOT-G, we laid out a set of six key foundational principles that a new stack of edge computing must include. We’ve talked about them in our past posts independently, and in this post, I want to bring it all home to provide a clear and succinct vision of how we build things.
Developer experience (DX) centricity
Repeatability to reduce developer overhead
Reliability for builds and deployments
Monitoring and Observability as a first-class citizen
Audit-ability with open source
Security out of the box
#1 Developer experience (DX) centricity
What I mean by developer experience is the sum total of how developers interface with their tools, end-to-end, day-in and day-out.
- Jean Yang from a16z post Developer Experience (DX)
There are many tools available for building machine learning models. Building for edge devices, albeit harder, is getting a lot of attention from some very good companies.
Developer experience extends beyond training. It encompasses collaboration during testing, deploying, and all aspects of the control plane - what we call the whole “Orchestration on the edge”
Our first foray into learning and getting feedback on our DX mindset was in our first hackathon - Hack the North.
We have come a long way since that hackathon and have incorporated those learnings into our SaaS low-code environment called Hammer Forge which is launching shortly very soon!
Forge Studio - our low-code environment:
#2 Repeatability to reduce developer overhead
Repeatability for us means two things:
Isolation of builds from host environments
Minimizing developer effort to target multiple platforms
We are addressing both these items using our open-source containerization technology called Rune for edge computing.
Containerization makes the builds portable, which means developers dev/test environments are not dramatically different from production deployment environment.
Additionally, Rune also provides pipeline capabilities on the edge to perform MLOps, which means the effort to retarget different devices becomes manageable configurations.
We have discussed the origins of container technology being inspired by the cloud computing world below.
In addition, we described our take on portable edge computing:
#3 Reliability for builds and deployments
The hallmark of a good production orchestration system includes repeatable builds, traceable deployments, and automated workflows.
The traditional workhorse for this in cloud computing world is the Continuous Integration / Continuous Delivery CI/CD system. Edge computing adds additional layers of complexity because sometimes the triggers are not based on code changes, but updates to models, or edge pipelines.
Hammer Forge (our SaaS) platform comes bundled with Foundry* - a CI / CD system for setting up repeatable and continuous deployment for edge computing.
We have briefly touched on this in one of our recent posts:
*Note: Foundry is an enterprise cloud native product available for enterprise customers
#4 Monitoring and Observability as a first-class citizen
Building and deploying models to run on edge is just the beginning. Models are workloads that run on devices placed in a specific environment. There are constant changes to the environment which affect the model performance. For example, an embedded computer vision ML model could have been trained to recognize objects in an environment, but the device could be placed where there is poor lighting, causing the models to predict incorrectly.
Models on the edge need to be monitored and observed continuously - just like production cloud workloads.
We articulated our thoughts on Monitoring and Observability here.
Hammer Forge is launching with monitoring features, and we are adding observability features later this year to help identify model and data drifts.
#5 Audit-ability with open source
Workloads that run on the edge should have verified software, and the best way to do this is by having open-source technology. Our container technology, Rune is completely open source. All the SDKs that help integrate into the devices are open source.
The tools we use for cloud are mostly open source because it allows us to audit and leverage the community as needed. Edge computing should be no different.
Our ethos for open source comes deeply from our developer culture, coming from the Rust community.
We are building our community around open collaboration. The fragmentation of devices means it is impossible for HOT-G to cover every type of device and platform out there. We will strongly rely on the community coming together and rallying around Rune to extend to several platforms in the future.
If you would love to contribute, join our discord!
#6 Security out of the box
The final guiding principle is something that is often an afterthought in most products - Security.
Many enterprises collect unique data that can be used to train models which then end up becoming intellectual property (IP). Deploying these models to edge devices means opening to several types of attacks, such as:
Direct model theft
GAN attacks to reverse data from model
A whole bunch of attacks on the machine learning models including identifying personally identifiable info (PII).
Having built and deployed ML for sensitive information such as healthcare, security, and legal teams can kill products from launching if these are not addressed.
Security is a deep topic with several layers of attack vectors and merits its own post which we will be publishing in a couple of weeks. So, stay tuned.
While it is impossible to address/thwart every type of attack vector, the important thing to do is have tools and systems that help identify them.
Hammer Forge is launching with a few key security features which we will talk about in this next post.
Building this stack of interoperable tools takes time, effort, and resources. We have been diligent in building them with early adopters and champions from our community, students, and pilots to optimize our learning. This year we are ready to commercialize and continue adding missing pieces of our edge computing stack quickly.
If you love our content, consider supporting our team by subscribing and sharing TinyVerse! We cannot wait for you to see what we have in store in the pipeline this year!
Don’t forget to join the Hammer Forge’s Waitlist - we will be onboarding selected customers soon!
HOT-G Socials:
Twitter @hotg_ai and @hammer_otg | LinkedIn | Discord
 | A guest post by
|