Rv32im + ecall as the lowest-level ISA/API

General Research
1

Preface

We want to be able to run (parts of) Anoma programs in/on many base environments:

  • on existing blockchain VMs, such as the EVM and SVM
  • on zkVMs, such as RISC0
  • in the browser environment (best application distribution vector)
  • on regular desktop operating systems
  • on mobile operating systems (in the future)

At the moment, we do not have a single standardized low-level ISA/API that is supported in all of these environments (which are themselves quite heterogeneous). Instead, we have:

  • With Galileo/APv1: “backend” application code in Solidity and Rust → RISC0 (resource logics) (runs on EVM/RISC0), “frontend” application code in a mix of Javascript and Rust → WASM (runs in the browser), “service” application code in a mix of Rust and Elixir (runs on a Linux server).
  • With the local domain + controller software: Elixir (runs on desktop only).
  • Old work on Anockma (Nock) and Cairo (another zkVM).

If we had a single standardized ISA/API which could be supported in all of these environments, we could greatly reduce the variety of different components needed to write an end-to-end Anoma application. This is closely related to the project of a low-level API, although note that here I’m talking about an even lower-level API (i.e. the RM and other application abstractions would be built on top of this layer).

Of these environments, by far the most performance-constrained are existing blockchain VMs, zkVMs, and the browser. I do not think that we need the most performant option to start – we can get away with something that isn’t too awful and optimize particular applications if they turn out to be popular. If we can find an option that works for those three, making it work on desktop and mobile operating systems should not be a problem.

Proposal

I propose that we use rv32im + ecalls (RISC-V “syscall” interface) as a standardized “low-level ISA”, where, more specifically:

  • The basic rv32im instruction set provides 32-bit integer computation, memory, and control flow
  • We define a set of custom ecalls (syscalls), including:
    • System calls which provide optimized cryptographic operations (e.g. BLS12-381 elliptic curve operations) (similar to what RISC0 does)
    • System calls which provide local persistent storage operations (read/write/scan)
    • System calls which provide network operations (send message to pub/sub topic or node)
    • System calls which provide distributed storage operations (scry)

All environments would need to support the basic rv32im instruction set, but they could support different subsets of system calls (and implement them differently), and authors/users of programs (or the compilers / other software acting on their behalf) will need to keep this in mind.

For example, both the SVM and EVM support bn254 elliptic curve operations as “precompiles”, but only the EVM supports BLS12-381 operations, so our BLS12-381 syscalls would not be supported by the SVM “Anoma low-level runtime”.

Further abstractions such as the resource machine, object systems, Anoma-level ideas, AVM ideas, etc. would be built on top of this base level.

In general, it appears that (with sufficient usage of ecall for key operations) this is feasible:

  • rv32im is supported by RISC0 and other zkVMs
  • rv32im can be compiled to the EVM and SVM with moderate overhead
  • rv32im can be run in the browser
  • I’m sure that we can manage desktop and mobile (although I haven’t investigated in detail).
  • We can (relatively) easily compile a simple Scheme to rv32im, and efforts are underway to compile a subset of BEAM.

The abstraction layers would then look roughly as follows:

new-abstraction-stack
new-abstraction-stack1562Ă—1223 45 KB

I think that this proposal is concordant with the ideas in Thoughts on a low-level Anoma API and A Roadmap for AL, and welcome feedback.

3 Likes
2

Exactly what sorts of syscalls would we need? So far, I can think of the following:

  • Optimized cryptographic operations: elliptic curve operations, hash functions, etc. These could all be implemented in rv32im directly and are just more efficient as specially optimized operations (this is very similar to jets in Nock). Each environment would support a subset of these (probably all of them for browser/desktop/mobile, limited for zkVM/EVM/SVM).
  • Local storage operations: read from, write to, and scan “local” storage (persistent memory).
    • Implemented by localStorage or similar in the browser enviroment.
    • Implemented by persistent storage on EVM/SVM environments.
    • Not supported by zkVM environment.
    • Implemented by persistent DB storage on desktop/mobile.
  • Network communication: send message to Anoma network address, send message to pub/sub topic (maybe these can be combined, maybe not).
    • Implemented by relay from the browser environment.
    • Implemented by relay from EVM/SVM environments (log an event which a listening node will pick up and relay the message).
    • Implemented by direct network communications on desktop/mobile.
    • Not supported by zkVM environment.
  • Distributed storage operations: scry distributed storage (with scry-properties).
    • Implemented by the zkVM/browser/desktop/mobile runtimes (this is the most complicated syscall).
    • Not supported by EVM/SVM (except for maybe very simple scries).
2 Likes
3

Thanks for this! It is an interesting proposition. I have two questions:

  • How do you envision the ecalls would practically exist in relation to RV32IM here- for example, to what extent will this introduce uncertainty regarding which programs are safe from a ZK perspective? Also, when you say some of the cryptographic operations can be implemented in RV32IM, is that to say that they do not require, for example, floating points? I believe there’s an entire RISC-V extension for cryptographic modules, but I’m not expert.

  • Also, could we make explicit what is being left remaining out from RISC-V if we’re re-introducing this set of syscalls into RV32IM, maybe regularly used extensions or syscalls? I can’t recall all of the various extensions off the top of my head.

1 Like
4

Why do these require to be sys calls it seems premature to consider them as such, likely but ontop of some other calls in AL.

For example nock doesn’t formally define 12, it’s a meta operation implemented in a self reflective compiler. I’m not saying we ought not to do it, but rather note the capability of the system and we might find better ways to achieve it than fix it into an instruction set. How our scry works is an interface and would likely be bootstrapped with different implementations meaning a more primitive capability would be needed rather than something powerful

2 Likes
5

Good point (also discussed with @l4e21), they don’t need to be, although we’ll need to think about how scry would work in different environments.

6

Here, we are mainly interested in the smallest steps of computation[1], in particular in post-ordering execution.

For post-ordering execution, the most important aspect that is already covered above is the sending of messages over the network. Now, I wonder @isheff, in the context of this post on a low-level API for controllers: would we want to separate out the writes to blob storage into a separate category of load/store operations or would they be subsumed by the above mentioned operations on

?

  1. Typically, I would add in a single thread, but that may be clear from the general context. ↩︎

7

There are already many projects as you pointed out that support RV32IM , interpreters in the zkVM, the browsers (although why not WASM, which can be compiled to riscv), and others. That’s good, indeed. Also, looking for a standard for Anoma lowest-level ISA based on an already standard + extension smells good, also reminds me as you linked one we research on Cairo ISA and its algebraic RISC.

On syscalls for cryptographic operations, skimming some websites, this seems that it’s better to have precompiles, otherwise is expensive? not expert either, but it should be declarative and connect to the next point (as it’s backend specific).

Treating scry as a syscall is a bad design decision. It goes against the layering I propose in the AVM. I strongly agree with mariari:

In the current AVM design, scry is a declarative query—the runtime decides how to implement it (distributed hash table, content-addressed storage, etc.). Reducing it to a key-value syscall loses this flexibility.

“what level of abstraction should the “standard API” target?” Tobias’ comment above is perfect.

so, I suggest, perhaps something like High-level (AVM) -> AL (acting as os-lang) -> (zkVM/browser target? This way, we preserve the AVM’s semantic richness + type safety :stuck_out_tongue: while enabling compilation to RV32IM + precompiles for zkVM deployment (on demand).

Do you have any snippet that demostrates how to actually target rv32im+ecall??

2 Likes
8

Yes, agreed, hence the idea here for flexible ecalls/syscalls.

Makes sense :+1:

I think in broad strokes the layering (from high-level to low-level) of:

  • Typesystem(s), which is/are built on top of:
  • Anoma-level, which compiles to:
  • ALLIR (rv32im+ecall/precompiles)

makes sense. While I think that the research and development work was useful, I no longer think that we should use the term or concept “AVM”, it’s just too vague (this one’s on me historically).

1 Like
9

Well, we shouldn’t call a type system built in/ontop AL higher level than AL. It is after all just a normal predicate in the language that can be invoked, meaning that it is itself just some AL code.

Thus the lower levels would be whatever ALM (AL machine code, probably some kind of WAM like machinery, not designed nor thought of concretely yet) code AL generates and goes ontop of rv32im+precompiles if we wish to go in the rv32im direction.

10

I think it is important to explicate on the matter of AL and how live systems tend to be built, as I think it will explain how AL can be extended and how it ought to be built.

Namely the core of a live system has to be simple (See some future writing I have on this point) as we have to account for how the system can change under us, thus having a lot of orthogonal features at a core part of the system gives us that many more cases to handle in a correct way. Doing this poorly for some features of the system means that those parts of the system can’t be used in a live way making the livness of the system moot (might as well boot it).

However this brings up another problem, if handling a lot of orthogonal features is rough, how do we manage to get high level computation then? Trying to add the features ontop one by one brings us to the same problem, each of those features have to worry about how it interacts with the system!!! Thus the only approach that works well is by keeping a simple core that can extend itself upwards!

We can see this approach work successfully in practice in the following examples:

  1. The implementation of Self paper covers just how simple a prototypical object oreinted model is. There is almost nothing to the implementation and yet you get self which is a more dynamic version of smalltalk (the paper uses self and not smalltalk to show how they can optimize in a harder environment than smalltalk!).
  2. The Simple Reflective Kernel Paper shows how you can make an entire meta reflective object system in less than 32 methods. This model is more akin to how smalltalk actually works at a protocol level, and shows what barebones bootstrapping can give you. This paper answers the question on how to answer infinite recursion questions for a consistent environment. On the power scale, you can derive things like Abstract clases that are baked into java in 3 lines of code!
  3. Forth is the simplest family of programming languages that can be bootstrapped in raw assembly very easily (see my video on Jones Forth). It ends up feeling a lot higher level than C because it is so consistent. Things like lexical variables can be dervied. Forth’s computation model is a lot simpler than the object models and thus has some issues scaling with certain techniques (though Factor shows that with the right abstractions ontop it actually is as high level as Smalltalk or Lisp!).
  4. Common Lisp. Although it has 26 special forms (think functions with special behaviour you typically can’t derive within the standard langauge) and a standard over 1000 pages, it is designed in such a way very deep extensions can be had. A lot of the standard are just functions that are built ontop (no different from having a large standard library. And it remains extensible as those 26 special forms are known upfront and we can reduce every term to either a function or a special form. This allows users to write their own tools like a codewalker.
    • A lot of the reason why the language is extensible is that each set of abstractions (layers if you will) are built and reified inside the language and build reflective APIs that the force the language developer to develop the feature as if they were users. Two good books on the subject can be found here:
  5. Prolog. Prolog is built on a complicated WAM machine, but after this level the language then composes all of itself traditionally. Handling the top level and loading of programs well.

This list is non exhaustive, as languages like APL and Erlang also fit this philosophy of design. In fact it is hard to find languages which can be developed and deployed interactively that don’t work this way. If we extend our definition of liveness to just system execution and not the language with it then systems like the JVM with Java and C under Unix are examples since they can be operated live (the JVM can dynamically load Java, C has DLL files, etc).

Therefore since we wish to make a truly live system in Anoma, AL must be written in a way that allows us to successfully achieve this, which means the following:

  1. The language of the system is AL, most everything in the system will be written in it (Everything for Unix at the end of the day goes back to C. for the Erlang virtual machine Erlang sets the stage and languages like Elixir follow how the system works).
  2. AL will have to compile to something, this will be AL Machine (or ALM for short) code. This will probably look like the WAM, but details are not clear yet
  3. ALM can most likely be compiled to something like rv32im+ecall/precompiles (per this thread).
  4. These 3 are the only major layers, there are not foreign systems below or above it like the AVM, as if we wish to make a truly live system it has to be holisitc in design. Adding layers which do not naturally evolve within the layers creates friction points that will lead to a loss of livness in design. The compiler may for optimization reasons may create additional layers in practice but that is an implementation detail (Sea of Nodes for ALM!?!?).
    • It’s the system language of Anoma, thus it’s a low level language! The reflective nature of the abstractions means that we can hack the system at a very low level!
    • It’s reflective thus it’s a high level language! We can have layers of abstraction without losing power as the layers of abstraction are designed in the school of thought as Genera/Common Lisp meaning that if we want special behaviour we have to us our own mechanisms to achieve that. (the language/system designers are not special).
  5. Since AL is reflective, we can make things like type checkers as functions. We can design pluggable types.
    • Elixir’s new type system is interesting as it’ll introduce a rather nice static type checker to Elixir, meaning there are 2 type systems can employ in Elixir. If one is interested i can talk about nuances of where this does and does not fail to be live.
  6. The system around AL will be the local domain, this is the programming environment (also led by @l4e21) around AL. From this things like controllers will be written (currently in Elixir in our Elixir local domain).
4 Likes
11

All of this makes sense to me, and that the core should be small and extendable, with different layers or capabilities. Since I’ve heard your ideas about AL for a few years now, and after the last HHHs, what I’d like to see next are clear design write‑ups and a full implementation that can be tested locally—rather than descriptions of what “will” be included or what you “want” or more plans rather than actions. See the need:

So, while those plans are possible, bringing AL to the table would always raise concerns for me, including how our limited manpower at Anoma and across the other language projects might conflict and where we would end up.

That being said, I’d expect the final outcome of this forum post to be in a better format, similar to my AVM shares: specs, assumptions, implementation details, limitations, and examples—something tangible, an MVP. I think it’s doable, considering you can fork a system like Pharo, add backends, and so on, at least based on a few sessions with Pharo, GT, and Smalltalk while studying its object system for the AVM.

That’s what I like about Chris’s proposal: building on a well-known standard, not on will-projects, so that the outcome can leverage all the existing ecosystem tomorrow.

Anyway, I believe you’ve already made some progress on an Elixir prototype, though I’m not sure whether it’s finished.

Really, take this as a friendly nudge to get AL shipped, if you really believe in it.

3 Likes
12

The problem of man power is an interesting one. If we wish to realize a custom controller then we are in essence taking on the role of implementing an operating system. And in doing so all this work is needed anyways. Doing proper design for the system we want means that we have to do less work overall.

Well, proposals are a bit weaker than wills, as will means that is the direction we are taking and going in, whereas simply planning to implement something standard does mean we still have to do the labour of getting to that point. Thankfully the rv32im work has been planned with AL in mind. In particular the AL track of work has been planned in two phases previously:

  1. Get something useful for the organization quickly
  2. Get important research we wish to get done, done so we can work on the long term system.

We’ve made some good progress on both fronts.

On the short term front, we have been doing compiler work from elixir to rv32im. This is useful for deploying programs to Anoma in Elixir. Since we plan to write AL in Elixir and write to data structures in Elixir, we can put both our compiled programs and the compiler online. Meaning that this is a meaningful step towards proper Anoma deployment. Another point is that we’ve been implementing APIs and data flows that will be present in AL and isn’t currently common within Elixir practice, meaning that one “can program in the future AL Today” (as @ray puts it).

This can be seen here:

On the long term front a lot of ideation has been happening and ideas have been fleshed out thanks to @l4e21. Work can be seen on this front in a few areas:

There are of course more in the works as there is an essay about livness and event sourcing in the works.

What is nice is that there has been a shift, previously I was told those on engineering can only work on AL after a year and a half, and since engineering is the department with the technical expertise for such a project it’s mostly stayed ideating. However since priorities have shifted we can actually work on the operating system language of anoma, which has been long overdue.

As an organization I do believe we can ship some good versions of AL.

4 Likes