[AnomaBeta] Exploration of browser-side risc0 / ARM

Applications
1

Exploration of browser-side risc0 / ARM

The core of this exploration is to determine whether we can run functionality from the shielded Anoma Resource Machine in browser via WebAssembly. As arm-risc0 is built on risc0-zkvm, it presents some challenges when it comes to compiling ARM to wasm.

Building with Rust

Ultimately, we are asking whether risc0’s RISC V VM will run successfully in WebAssembly, and whether there are any runtime issues we need to be concerned with, but first, it must compile.

The first step is to see which features from arm-risc0 can be compiled to a wasm32-unknown-unknown target in Rust (this is the minimal WebAssembly target that is supported by modern browsers, and is the target which allows binding to JS with wasm-bindgen). In my testing, I was able to build every feature except transaction (which builds upon the functionality of the prove feature in risc0), assuming I’m not enabling the nif feature (Rustler) as I’m not interested in Elixir bindings:

  • logic_circuit
  • compliance_circuit
  • fast_prover
  • composite_prover
  • groth16_prover

These features will at least build successfully to wasm32-unknown-unknown. However, we need the prove feature of risc0-zkvm, which presents the challenge of compiling a target that supports libc and std. This requires that our WebAssembly is built to a target that supports standard machine I/O. There are two main targets that provide this, which are Emscripten and WASI. While Emscripten can work with a Rust toolchain, it is a bit dated and bloated (also, the emcc compiler does not work well with the dependencies of risc0-zkvm).

Using Wasmer SDK via the @wasmer/sdk JS package, we get WASI support in the browser. Wasmer provides a WASI runtime (which is itself a *.wasm binary with JS bindings) in which we can execute our WASI/WASIX binaries. Rust provides the following wasm32 targets for WASI (this is not an exhaustive list):

Rust supports these targets assuming you have wasi-sdk (this provides clang, wasi-libc, the wasm-ld driver for linking, etc.) installed with an environment variable pointing to it, e.g., WASI_SDK_PATH=/opt/wasi-sdk. In my testing, I also had to set CC, CPP, and CXX variables to get all dependencies using the correct compiler

Challenges

The risc0-zkvm repo includes many C and C++ dependencies which, while (in theory) should be supported by the wasi-sdk toolchain, can be problematic depending on their handling of __wasm32__ architecture detection. In some cases, any wasm32 target will be treated as no-std, which assumes a wasm32-unknown-unknown target but is unnecessary for wasm32-wasip1. This can (and does) lead to compilation failure. Many Rust crates are simple wrappers around C/C++ libraries to provide bindings, and we are at the mercy of the original libraries’ handling of wasm32, assuming they handle it at all. However, we should be able to support anything that isn’t specific to some particular hardware, i.e., anything that is supported by WASI (@wasmer/sdk even gives us access to threads in browser by way of launching WebWorkers).

Some problematic crates required by risc0-zkvm include:

Mishandling (or ignoring) __wasm32__ detection leads to fatal errors. Some are very simple to fix, which would only require that we publish a repo with a fix to override as a patch in Cargo.toml, or otherwise publish a fix upstream wherever this dependency originates. A simple work-around (read: hack) in the case of nvtx is to update /nvtx-sys/export.c to include a definition for THREAD_ID when on __wasm32__ architecture:

/* Threading for WASM */
#if defined(__wasm32__)
#include <sys/syscall.h>
#define THREAD_ID 1
#endif

GPU hardware-specific code should never be built for wasm32 in the first place, or for any target that indicates proving should be done in CPU. Ideally this should be fixed in risc0-zkvm so this issue can be avoided. In theory, a CPU-only feature-set will work with a WASI target.

In another case, the ring crate (providing ring_core headers) also fails to compile. In build.rs, the clang detection call below leads to a case where RING_CORE_NOSTDLIBINC will is defined even though I am using clang, which in turn leads to a fatal assert.h include not found error:

if (target.arch == WASM32)
        || (target.os == "linux" && target.env == "musl" && target.arch != X86_64)
    {
        // TODO: Expand this to non-clang compilers in 0.17.0 if practical.
        if !compiler.is_like_clang() {
            let _ = c.flag("-nostdlibinc");
            let _ = c.define("RING_CORE_NOSTDLIBINC", "1");
        }
    }

Note that I did originally experiment with using gcc and g++, which did successfully build the project, but that was only because it was not setting the architecture properly (Linux x86 instead of wasm32), which of course led to errors during linking. Therefore, we must get it to build with the wasi-sdk C/C++ toolchain (clang). A long-term goal would be to get these fixes merged upstream to avoid patching, but I think patching is acceptable while we explore ARM features in browser.

Differences between WebAssembly targets

Using Wasmer SDK (@wasmer/sdk) for wasm32-wasip1 over generating JS with wasm-bindgen for wasm32-unknown-unknown introduces an additional challenge of how we approach WebAssembly functionality.

wasm32-unknown-unknown

With wasm32-unknown-unknown, we get the most minimal and efficient WebAssembly binary, though without the feature-set of the WASI standard. wasm-bindgen and wasm-pack allow us to publish our Rust crate as a library providing functions which can be directly imported into TypeScript/JavaScript. This was the approach taken with Namada SDK, and it is indeed very nice to work with.

wasm32-wasip1

With WASI (specifically with Wasmer SDK), we have to think a bit differently about how we interact with the WASM-runtime, as the resulting *.wasm (WASI/WASIX) executable is a program itself. While it may be possible to bind to public Rust functions, that will require more investigation and isn’t yet obvious (it becomes problematic as we’re not dealing with wasm32-unknown-unknown where the WASM can be instantiated directly in browser).

This isn’t necessarily a problem. A “wasi” app can be thought of like a command-line program, where it accepts STDIN and provides STDOUT and STDERR. The input and output to the runtime can be simple strings or serialized complex data types, and can be set-up to accept flags and even sub-commands. The WASI-runtime (Wasmer) is loaded once into memory, and serves the function of executing WASI binaries on demand, and manages launching worker threads for these, spawning workers for threaded-applications, and passing data in and out of the *.wasm. What would be very interesting to explore is how we can build a long-running service in WASI with message handling and event-dispatching.

I concocted a simple end-to-end example of invoking a WASI *.wasm using @wasmer/sdk in the anoma-toolkit repo. With the Wasmer JS SDK initialized in the app, I essentially load my custom *.wasm with:

// @ts-ignore
import wasmerWasmUrl from "./wasm/wasmer.wasm?url";

// Wasmer Wasm Initializer for Vite projects
export default async function init(): Promise<WebAssembly.Module> {
  return WebAssembly.compileStreaming(fetch(wasmerWasmUrl));
}

And then I invoke sdk.runWasix() with the initialized WebAssembly module using the SDK:

const module = await init();
const instance = await sdk.runWasix(module, {});

if (instance.stdin) {
  const stdin = instance.stdin.getWriter();
  const encoder = new TextEncoder();

  const encodedInput = encoder.encode("Test input");

  await stdin.write(encodedInput);
  await stdin.close();

  // Wait for `main()` to complete
  const result = await instance.wait();
  if (result.ok) {
    return result.stdout; // String response from wasm
  }
}

This invokes the following main() function from Rust:

fn main() -> std::io::Result<()> {
    let mut user_input = String::new();
    io::stdin().read_to_string(&mut user_input)?;

    let output = format!("From wasm -> {}", &user_input);
    io::stdout().write_all(output.as_bytes())?;

    Ok(())
}

I’m following this up with an implementation of the simple_counter example from arm-risc0 as the first end-to-end demo with ARM in browser. This of course requires first fixing the compilation errors described above, but will be a decent representation of what is involved in deploying Anoma ARM apps in browser. A better end-to-end test would be the kudo_application, which should be much closer to a real-world proof-of-concept.

ARM

For our purposes with the Anoma Resource Machine with risc0-zkvm, which approach we use for any given WASM binary will be determined by what that particular binary needs to support. If we need some functionality in WASM that doesn’t require the prove feature of risc0-zkvm (a part of the transaction feature of arm-risc0), then we can compile to wasm32-unknown-unknown and generate bindings with wasm-bindgen. For functionality of the transaction feature in arm-risc0, our only hope is to build to wasm32-wasip1. The Wasm binary will be specific to a particular Anoma app, or a particular library of functionality (via wasm-bindgen). We won’t have a monolithic Wasm binary as is the case with Namada SDK.

Ultimately, we want to simply include Anoma ARM apps as dependencies (Git repos) to Rust crates that build to the wasm32-wasip1 target, whose sole-responsibility will be providing I/O to various functions. We can also define schemas for serializing/deserializing any complex type that needs to be represented in the front-end. Any functionality we can use on the front-end that does not require transaction features can simply be built with wasm-bindgen and imported directly.

Further exploration will look at how to best engineer the Rust apps that will be loaded into the browser for ease of use in the @anoma/toolkit.

Resources

Wasmer
Wasi
Rust
3 Likes
2

Thanks a lot for the write up @justin (and of course for the fantastic exploration work!).

I specifically want to address this part just to have asked the obvious question - @cwgoes @ArtemG do we need the prove function? I sense that if (for whatever reason) we would not need it, this would make Justin’s life much easier.

As a separate question, @cwgoes are you okay with moving this to a public category? I’d love to forward this to our colleagues at Risc0 and ask them for feedback.