The Open Systems Assembly Protocol (OSAP!) is a new approach for the generation and integration of modular, interoperable hardware systems, and a love letter to computer plumbing and hardware heterogeneity.
In practice, it’s a small message passing layer that provides naming, routing and flowcontrol services to modular hardware and software using source-routed packets. This allows us to quickly build distributed networks of micro-controllers and computers that work together across heterogeneous link layers, using a common graph representation to describe large systems in one gestalt.
Since configuring big graphs of hardware and software can get messy, OSAP includes network traversal routines and a discoverable graph API for intelligent systems assembly algorithms or graphical UIs for humans.
OSAP collapses hardware and software modules and connects them all across heterogeneous networking link layers. Any resource can become a graph vertex, and programs are made by transmitting data through the graph using source routed packets.
Graph structures are discoverable and configurable using an MVC (model view controller) scheme, meaning that remote resources can be reconfigured from i.e. a UI, or some high level systems config script.
OSAP is polyglot, and currently available for JS in the browser or node.js, as well as embedded systems in CPP. It is arduino-friendly.
It’s also scalable, and really, truly distributed. We can make big messy graphs with as many link layers as are possible to forward packets through: eventually we just run out of space in a packet within which to describe a route.
Last note: OSAP isn’t just about embedded systems; systems always include a little bit of script: JS or PY. This means we can describe whole software stacks in dataflow terms using OSAP;
OSAP’s basic APIs allow for the generation and operation of
endpoints, which are like network data objects, as well as
virtualBusses which are software wrappers for data link layers. Module authors use this API to encapsulate their projects as OSAP bundles.
Endpoints can be written to, or read from. They keep a list of routes (to other vertices) over which they try to publish new data whenever they are written to. We build applications by configuring endpoint routes; this can be done in static configurations using the OSAP API, and also using a graphical tool that visualizes and modifies graphs in real time.
I put OSAP to use in the Clank! project, where it provides naming, routing and flow control for a littany of machine control modules. I’m limited in my ability to publish firmwares, circuits, and controllers from either project, but check the blog for dev notes.
OSAP is like an onion; I am making an intentional effort to build it up in layers, each successive layer being a useful system in its own right. This also lets us shed high-level layers when we put OSAP in simple systems, like firmwares (which normally won’t need multisegment endpoints, or to render UIs, for example) or even FPGAs (which are a likely future path for high speed OSAP routers, but would only need to orchestrate layer-zero stuff).
I have also written a list of big open challenges for OSAP, if you want to sink your teeth into something fun with me.
The transport layer, together with link layers, is responsible for ferrying packets from vertex to vertex under flow-control conditions. It’s functioning, but deserves some tidying up on packet structures and it uses more RAM than is perhaps necessary.
The virtual port and virtual bus structures are solid, and it’s not too difficult to add new ones. Here’s a list of tested link layers:
- UART (with Arduino Serial objects)
- USB Serial (likewise ^)
- Websocket Client / Websocket Server (in JS)
- the UART-Clocked Bus (embedded)
In the works:
- I2C multi-host, with
- Manchester-Encoded, Isolated, FPGA-Driven super-sweet-tiny-PHY
Software endpoints can be easily declared and used (written to and read-from), but they are currently untyped, and multisegment transmission is in the works but not optimal.
The configuration and discovery layer for endpoints (which enables the graph rendering and editing in real-time) is real patchwork at the moment.
I’ve just built the first OSAP UI this week (it’s April 21, 2022), and I’m no UI expert. I’m also more focused on realtime control than on rapid reconfiguration at the moment. Do you love writing UIs? Designing UIs? Do you want to enable people to build big messy dataflow systems without writing code? Get in touch!
In the past I’ve developed dataflow systems where we could remotely instantiate new code modules (squidworks). OSAP started largely as a re-write of those systems, so I know how to do all of these tricks (and plan on it) but I am building lower layers with a little more intention this run around.
OSAP is a kind of strange new way to build systems, so it needs a little more documentation. I know that this website is sparse-ish, if there’s something you’re looking for that you think should be here, but you can’t find it, please please send me a note and let me know what it is.
OSAP wants to make truly modular Open Source Hardware contributions possible. At the moment, it can be difficult to build complex applications using lots of different OSHW projects: the “systems integration” step is the most challenging.
Rather than building OSHW systems from collections of breadboards, dev boards, and arduino sketches that are glued together with unknown pieces of end-user code, we could release network-ready modules that use OSAP (or something similar) as a common message passing layer, then use graph configurations to integrate those modules into larger systems.