You can learn more about the language itself or how to install it on the Rue documentation. There’s also a playground website where you can try some examples right in your browser.

What is Rue?

Around 3 years ago when I was getting into Chialisp, I yearned for an alternative that was more similar to modern programming languages I was used to (such as TypeScript, or more recently Rust). The strange Lisp syntax and lack of typing was frustrating to me and was harder to mentally model (though I’ve gotten more used to it by now). And usually when I get frustrated about things, I take it into my own hands to do something about it rather than complain.

So around the same time I initially started working on Rue Lang, a new programming language that compiles to CLVM. I actually had a fair bit of experience writing recursive descent parsers prior to working on Chia, so that part was natural to me. And I’d always been interested in programming language design, but never was able to write anything other than simple interpreters before.

Since I had to juggle Rue with my full time job at Chia Network Inc and other projects such as Sage Wallet, progress was slow (especially when I was procrastinating on hard to solve design challenges or compiler bugs). The compiler went through several complete rewrites from scratch due to large scale architectural issues that I learned from along the way.

However, I’ve recently had some time to work on the latest version of the compiler and I’ve gotten it across the finish line (at least for an initial release). I’m sure it won’t be perfect, but just like when I initially launched Sage, it marks a point of no return so to speak.

What is Rue solving?

One of the most common things I hear from people who are interested in developing on Chia is that it’s difficult to understand Chialisp. I believe that Rue will allow people to get past some of their initial hesitation to get involved, and focus more on the Coin Set Model rather than the language itself while learning. This combined with improved tooling will create a better developer experience for beginners.

Additionally, there’s an opportunity to add new compiler optimizations that reduce the size of the CLVM byte code for puzzles. The results are already promising, and there is room to improve. New versions of the compiler can introduce additional optimizations without breaking puzzles that still use older compiler versions.

And finally, as we start to introduce more complicated decentralized applications on the Chia blockchain, having a more capable compiler will become more important. Being able to easily audit the code, leverage powerful language server protocol features, and efficiently write many puzzles with the same codebase will increase productivity and safety.

What’s the catch?

Rue is very new, and should be considered on the “bleeding edge” at the moment. While its test suite aims to regression test common edge cases and many full example puzzles, there is bound to be bugs as with any new software. Chialisp on the other hand, has had years to be battle tested by many primitives that are deployed in production.

The plan is to incrementally roll out some puzzles using Rue on-chain, and to ensure everything is working as intended:

1. They will be fully tested in both the Rue compiler codebase and the driver code to ensure the puzzles work as intended

2. New tests will be continually added in the compiler to build confidence

3. CLVM byte code for these puzzles will also be manually inspected before deployment to ensure nothing was overlooked

You can never be too careful.

What’s next?

I will keep iterating on the Rue compiler, and roll out new features such as:

1. Editor completions

2. Goto definition

3. Type information on hover

4. Importing multiple files (modules)

Please feel free to give Rue a try and let me know if you have any feedback on the Sage Wallet Discord!

CLVM Cost

The cost of a coin spend is measured by a unit called “cost”. This doesn’t necessarily directly correlate to XCH paid as a transaction fee, however they are related.

The following things have a CLVM cost:

1. 12,000 per byte stored in the blockchain database for a program

2. 1,200,000 per AGG_SIG condition

3. 1,800,000 per CREATE_COIN condition

4. Various costs for allocating memory and using operators at runtime

The thing that tends to add up the most dramatically is the 12,000 cost per byte. So, this is what Chia puzzle developers and the compiler optimize the most heavily.

There is a maximum of 11,000,000,000 (11 billion) cost per block. If your program uses 11,000,000 (11 million) cost, you can spend 1,000 of them per block. Thus, there’s a direct relationship between cost and throughput.

The mempool allows for transactions that are at most half the size of a block, and a total space of approximately 10 times the size of a block. There isn’t a minimum fee unless the mempool is full, but transactions are prioritized for block inclusion based on the highest fee per cost ratio.

You can read more about the mempool on the official Chia docs.

Constant Folding

The simplest of optimizations is constant folding, wherein you simplify expressions at compile time. This isn’t particularly interesting, but it allows for nested calls to concat, sha256, and various operators to be simplified as much as possible to reduce the amount of bytes requires to express the intended value.

Importantly, the Chialisp compiler decides to interpret calls to sha256 with known values and insert the resulting hash instead of the call to the operator. This results in 32 bytes (approximately 384,000 CLVM cost) being added to the program, which is in most cases quite inefficient. Rue instead elects to preserve the operator call unless you explicitly use sha256_inline.

Let Bindings

Because classic Chialisp doesn’t have a way to declare and reuse bindings, it’s common to see code like this:

(mod (first second)
    (defun stager (first second computed_third computed_fourth)
        ...
    )

    (stager first second (+ first second) (- second first))
)

This is essentially achieving the same thing - you are calculating a value, and binding it to a parameter in a function. However, notice that first and second are being passed around unchanged - you have to manually pass values in from one function to the next just to prevent reuse of calculated values. This adds unnecessary runtime and byte cost in the construction of the parameters to stager.

In Rue, you can simply declare bindings like so:

fn main(first: Int, second: Int) -> Int {
    let computed_third = first + second;
    let computed_fourth = second - first;
    ...
}

This is both more readable, and more efficient. Instead of passing the individual values through to another function, it will group the bindings and prepend them to the rest of the environment. This can sometimes result in a bit larger CLVM paths (which are how references to values in the environment work), but it’s more efficient both in terms of runtime cost and byte cost, so it’s well worth it.

Binary Trees

All credit for the idea of this optimization goes to (Twitter).

In CLVM, the environment to a program is a binary tree. However, we often use it as a glorified list - both for function calls, and things like captures and bindings. This throws away a lot of the cost savings you could get from organizing things in a tree structure.

As a simple example, as far as I can tell it’s not even possible in Chialisp to use the dot operator at the end of the argument list of a function to indicate that you don’t want it to be nil terminated. This adds an additional call to the cons operator for every function call, and increases the CLVM paths unnecessarily. Rue has the spread operator to allow you to do this without needing to write CLVM directly.

However, as a result of a discussion with , we figured out how this could be optimized further. Instead of simply prepending binding groups to the environment, you can organize the symbols into an optimized binary tree (more frequently used symbols come first) and cons it with the rest of the environment.

This provided some cost savings, but after generalizing it to function captures and function parameters, the costs of Rue programs went dramatically down. Every single function call, including functions defined in the standard library such as tree_hash, requires less bytes to include in the generated CLVM and has a lower runtime cost.

The caveat with this optimization is that functions which are converted into values (closures) lose symbol information, and the compiler can’t figure out how to organize the binary tree anymore (or if it even should). Thus, this optimization is disabled for all function values (ie if you don’t call a function directly by its name). Additionally, there will be an extern keyword which can be used to disable this functionality.

The main function could benefit from this optimization as well, but it’s currently disabled to preserve the layout of the solution for now. This will likely be configurable in the future.

Verification Statements

In CLVM, there are four similar operators called bls_pairing_identity, bls_verify, secp256k1_verify, and secp256r1_verify. These take in a list of arguments (which can’t be determined at runtime currently) and either return nil if the verification is successful, or raise an error if not. I call these, and any function with a similar pattern, a “verification statement”, since the return value is meaningless.

In Chialisp, you would have to awkwardly figure out where to put these in your program, since the return value will be nil. This often means manually constructing lists and inserting these verification opcodes in the place of the nil terminator. In Rue, this is done for you so you don’t have to think about it.

If you use an expression as a statement (ending it with a semicolon), the compiler will include it in the generated CLVM in the best way it can, with a few optimizations to improve on CLVM cost.

Firstly, if there are enough instances of nil in the current scope of the generated LIR, it will insert each verification statement in place of one of them. Otherwise, or if any of the verification statements may return a non-nil value according to the type system, it will insert all of them in an all operator in the first nil slot. And finally, if there were no nil values to substitute, it will wrap the rest of the expression in a cons pair, insert the verifications as the first value, and then unwrap the rest value. This has the effect of running the verification statements while discarding the value (which means any assertions will raise an error if they fail as expected).

The Results

I’ve been experimenting with porting common Chialisp puzzles to Rue, both to test the compiler’s correctness as I make changes, and to incrementally optimize the generated CLVM.

Here are a few of the latest comparisons of the number of bytes (less than 100% is better):

1. CAT2 is 93.5% of the size

2. Partial offers are 95.9% of the size

3. Royalty split is 101% of the size (more work to be done here)

For the CAT example, this could mean that if you were to spend 100 CAT coins in Chialisp, you could instead spend 107 in Rue with the same CLVM cost.

Conclusion

When designing Rue, my main concern was that puzzles written in it wouldn’t be efficient enough to be used in production, and nobody would adopt the language (even if the syntax and type system were nice to use) as a result of this. However, I’ve been pleasantly surprised by the optimizations I’ve been able to implement into the compiler to bring it close to, and in many cases exceed, the Chialisp compiler.

I’ll keep working on optimizing the language (even after the first release is out). But, it’s also important to write more and more unit tests to ensure program correctness isn’t left behind in the pursuit of efficiency. Thanks for reading!

Ramblings of Rigidity

Rue and Improved

Rue Lang is a new programming language I’ve been working on for quite a while. It’s based on Rust, and aims to provide an alternative to Chialisp for writing puzzles for smart coins on the Chia blockchain…

Read more

8 months ago · 1 like · Brandon Haggstrom

Type Systems

It’s important to first understand what the goal of a type system is. The Wikipedia article goes into a lot of detail, but essentially it boils down to preventing compilation of programs that use data in the wrong way. For example, if you try to assign an integer to a function that expects a boolean, it will fail to compile. This prevents a large array of logic errors that could lead a program to either crash or have an unintended result.

Many languages employ dynamic type checking, wherein the values have type information associated with them at runtime. While interpreting the program, the runtime will check the type and raise an error if there is a mismatch. However, static type checking enforces these checks at compile time, before the program is ever run. This can be useful for catching bugs earlier on, improving maintainability, and optimizing compiled programs.

For these reasons, and because it makes code easier to audit, Rue is a strictly typed programming language. Types are required for all values, whether they are inferred or explicitly specified. Bypassing the type checker or overriding the type of a value is intentionally cumbersome to discourage doing so, since it reduces the safety of the program.

However, making a language statically typed will be frustrating unless the type system is powerful enough to sufficiently express the intention of complex programs with types. Every time the compiler isn’t smart enough to understand what you’re trying to do, and gives you a type error that you know is unnecessary, it takes you out of the flow of writing code and forces you to find workarounds.

CLVM Types

There are only two fundamental types in CLVM (the bytecode that Rue compiles down into):

1. Atom - an arbitrary sequence of bytes

2. Pair - two values (each of which can either be an atom or pair)

Thus, the first two Rue types are born: Atom and (A, B).

You can nest these types:

(Atom, (Atom, Atom))

Atoms

Often atoms have specific semantics or constraints that you’d like to encode in the type system.

There are various subtypes that are provided:

1. Bytes - a sequence of bytes

2. Bytes32 - a sequence of exactly 32 bytes (useful for SHA-256 hashes)

3. PublicKey - a BLS-12381 public key

4. Int - an arbitrary precision signed integer

5. Bool - a union of true or false

6. nil - an empty atom

As seen with booleans and nil, it’s possible to use specific values as types. For example, you can use 42 as a type that represents that value. Anything else, be it an arbitrary int or a different value, will not be assignable to that type.

Unions

Now that you can get specific about whether a type is an atom, a pair, or a specific value, it’s important to be able to represent union types. A union represents more than one type - where A | B means it matches either A or B.

Type Aliases

A type alias is like a variable for types. It lets you hide the complexity behind a reusable name.

type Owner = PublicKey | nil;

This can also be used to create recursive data types (the Wikipedia article linked actually represents the equivalent of the following example in Lisp):

type List<T> = nil | (T, List<T>);

This means that a list can either be nil, or one element followed by zero or more other elements (another List). Even though List is a built-in type in the language, this is the exact way that it’s implemented (and you can write the type yourself if you wanted to).

Recursive types were the biggest challenge to implement in the type system. Every operation that interacts with types, including comparison, type checking, subtraction, and replacement, must account for the fact that you might end up infinitely checking the same type over and over. Solutions in the Rue compiler include maximum recursion depths, a stack of “visited” comparisons, and a cache of references to results that are yet to be determined. I’m very excited that I was able to preserve such a powerful aspect of the type system and fight through all of the problems I faced when designing it.

Another example of a recursive type is Any (which represents, well, any CLVM value):

type Any = Atom | (Any, Any);

Structs

Now that you can represent any CLVM structure in Rue, it’s important to be able to name the fields in a list.

This is where structs come in:

struct Coin {
  parent_coin_id: Bytes32,
  puzzle_hash: Bytes32,
  amount: Int,
}

This type has the same structure as (Bytes32, (Bytes32, (Int, nil))), but it allows you to reference the fields by name (such as coin.amount).

Structs also have the unique property that they are semantically different from one another. If you define two identical structs named Coin1 and Coin2, you cannot assign from one to the other (even though they have the same structure), unless you cast them explicitly using the as operator.

A struct can have default values, which makes it easier to define types such as conditions:

struct ReserveFee {
  opcode = 52,
  amount: Int,
}

Now you can construct this with ReserveFee { amount: 42 } rather than needing to specify the opcode as well.

Type Guards

Well, now we have a bunch of types that we can use to represent the structure and semantics of CLVM values accurately. But we’re missing a way to check what type a value is at runtime. This is where type guards come in.

A type guard is a combination of two things:

1. Type check - which generates code that checks the type of a value at runtime

2. Type replacement - which modifies the type of a symbol or field at compile time

As a concise example:

fn count(list: List<Int>) -> Int {
  // We’ve reached the end of the list, so there’s no length.
  if list is nil {
    return 0;
  }

  // Now we know that list is (Int, List<Int>) rather than nil
  1 + count(list.rest)
}

When you use the expression list is nil you are creating a type guard - it will output the CLVM (not (l list)), which checks that the value is an atom (nil in this case). Inside of the if branch, the type of list is nil. However, on the outside, the type is (Int, List<T>) thanks to subtraction! This means that we’ve narrowed the type down by checking the value at runtime.

This feature is the most important aspect of the type system, and also what took the most effort to get right. It’s optimized such that it won’t output unnecessary checks (if the value can never be a type, there’s no point in checking for it).

Subtraction

Subtraction is instrumental in making sure that inverting a type guard for else branches works.

Fundamentally, these are the steps to implement subtraction:

1. Recursively extract the original type into a list of union variants (along with their corresponding semantic types, if any)

2. Remove any union variants that are assignable to the right hand side

3. Repackage and group the union variants into their corresponding semantic types

Essentially this means that any types that were a subset of the if branch are filtered out of the else branch (which matches the behavior at runtime).

Replacement

Type guarding doesn’t just work on entire symbols - if you check the type of a field in a symbol whose type is a struct, the symbol’s type will be replaced with one wherein that field is subtracted in the else branch.

This involves recursively walking the tree of pairs and substituting based on the path of the field that you type guarded. If there are multiple type guards on separate fields, they will be aggregated together in the final type when referenced later on in the branch.

Function Types

Function types are a bit special - technically a function could be any CLVM value, but not every CLVM value could be a function. And there’s really no way to determine whether a value is a certain function type at compile time. Thus, it’s possible to type guard on a function type, but not check whether a value is a function.

Although, a function type is always assignable to itself or any function type with a superset of its argument and return types.

Conclusion

We can build fully functional Chia puzzles by putting all these pieces together (pun intended). The hope is that you get the same level of flexibility that Chialisp provides, with the benefits of static typing at compile time.

In future blog posts, I’ll cover how the dependency graph, HIR lowerer, and optimizer work. In the meantime, I’ll keep working on the compiler, and getting it closer to a new alpha release. The rewrite has been quite the project, but it’s nearing initial completion.

Thanks for reading!

When I first started working on Rue a couple years ago, I knew it would be a time consuming project. But what I didn’t prepare for was the sheer amount of time I’d spend writing and rewriting the same code over and over again. I’ve come about as close as you can get to going insane without actually going insane (at least, not yet).

So was it worth it? Well, I’ll let you be the judge of that.

What is Chialisp?

It’s important to start with where we are today. All puzzles on the Chia blockchain are currently written using a language named Chialisp. It is, as the name suggests, based on Lisp.

Chialisp is pure - given the same input, a program will always provide the same output. Programs do not rely on any external state, and in fact are themselves stateless (there are no variables that can be mutated).

It’s also a functional language, which means that functions aren’t given special treatment and are just values that can be passed around like any other. The combination of these two traits makes it straightforward to audit the logic of the program if you can understand the syntax.

What is CLVM?

CLVM (the Chialisp Virtual Machine) is a low level bytecode language. You can think of it like the Chia equivalent of assembly language, or similar to the EVM on Ethereum. The bytecode gets executed on a fully sandboxed virtual machine when you spend a coin.

Every value in CLVM is either a pair of two values or an atom (a sequence of bytes). This is enough to represent anything, including lists and operators. However, these constructs are extremely low level and hard to read without having access to the pre-compilation source code. Additionally, the number of available operators is quite limited compared to what you may expect.

The idea is that Chialisp compiles down into CLVM, rather than us writing the CLVM bytecode itself. However, Chialisp is missing several abstractions we come to expect from programming languages:

1. It does not support lexical scoping, where you can declare bindings in multiple nested scopes and reference them from anywhere within. You can only reference constants or parameters of the current function (not parameters of the outer module).

2. It does not have a type checker. All values are either a pair or an atom (like in CLVM) and there’s no way to enforce at compile time that the correct value is passed into a function.

3. There’s no standard library, aside from a few helper macros like list and if.

4. Classic Chialisp lacks the ability to declare variable bindings, though the less commonly used modern compiler does support them.

5. The syntax is often hard to understand and is missing niceties like statements. This reduces readability, and therefore auditability.

And thus we end up writing a lot of CLVM opcodes directly in our Chialisp programs, which defeats much of the point of a higher level language.

Introducing Rue

Rue is a high level programming language that compiles to CLVM just like Chialisp does. The compiler might sound like a simple translation between Rue and CLVM. But that couldn’t be further from the truth. Instead, it requires several stages to convert the syntax into something that can be interpreted by the CLVM runtime.

Lexer

The lexer is responsible for breaking the source code into small pieces called “tokens”. The reason for doing this is primarily to make it easier to parse one token at a time efficiently rather than needing to backtrack when you reach a character you don’t expect. It also improves the parser’s error messages considerably.

This means turning code like this:

3 + 2

Into the following tokens: Int, Whitespace, Int

Parser

Once we’ve produced a list of tokens, the parser will convert these into “parse tokens”, which are slightly simplified versions of the lexer tokens. Things like missing end quotes are turned into parser errors at this stage.

Then, a hand written grammar is applied to the parse tokens recursively to convert them into a lossless parse tree. All whitespace is preserved in the document, to make it possible to reconstruct the source code later (which is useful for language servers and formatters, for example).

Instead of a linear list of tokens, we now have a tree of syntax nodes for language constructs like functions, statements, and expressions.

AST

The parse tree is useful, but it’s loosely typed and lacking concrete structure. It’s possible for subtrees to be missing due to parsing errors. We need an abstraction on top of it that provides a simpler interface to work with for the compiler.

This is where the Abstract Syntax Tree (AST) comes in. We can lazily walk the parse tree and find relevant nodes, using AST types with methods such as expr() and args().

Thankfully it’s pretty easy to convert from the parse tree to the AST, since they’re both in a tree structure. This is the final step before the actual compilation happens.

HIR

The next step is to compile from the AST to the HIR (high-level intermediate representation). This is when the bulk of the work of the compiler happens:

1. We declare types

2. We declare symbols that can reference declared types

3. We compile types that can reference other types

4. We compile symbols and check the resolved types

5. We compile expressions into HIR nodes

It’s done in this order to ensure that the order of items (functions and constants) in the program doesn’t matter. You can reference an item from another regardless of the order (even recursively) as long as they’re both in scope. However, because of this, every item must have a well defined return type and can’t rely on inference (because the body might not have been resolved by the time the type is needed).

The type system is still a work in progress, so I won’t dive too much into it. But it contains types like Bytes, Bytes32, Int, and so on. Everything must be either explicitly typed or have the type inferred from its value. This reduces the number of runtime errors you run into while debugging programs written in Rue.

LIR

There used to be a MIR phase between the HIR and LIR, but that was removed for simplicity. The LIR (low-level intermediate representation) is a single step above CLVM and represents all of the operators that you can compile code into. There aren’t many abstractions at this level and it’s perfect for performing final optimizations before generating the final CLVM bytecode.

The most challenging aspect of converting between the HIR and LIR is the environment. Rue has a concept of lexical scoping, if statements, and immutable variable bindings. On the other hand, CLVM only understands the “environment”, which is a non-hierarchical tree of values that are available to the program contained within.

If you want to add a binding to the environment, you have to effectively curry it into the rest of the program. This means taking the program (which is a function) and creating a new one with some of its arguments pre-applied. However, when you do this, you have to also shift to the right the environment path for all other symbols referenced inside the program (since the new symbols are inserted on the left).

An if statement or assert statement gets translated into an if expression with the rest of the program as one of its branches. These constructs are just syntactic sugar for expressions.

When you lower a function into the LIR, you need to make sure that any symbols it references that are in an enclosing scope (captures) get explicitly passed in when the function is called. Additionally, if you try to use a function as a value, it must first be turned into a closure that has its captured arguments curried in at runtime.

This translation step is the hardest to get right, and now that I’ve figured it out I don’t plan on touching it more than I have to (since it works). The last time I built the Rue compiler, this roadblock was the reason I stopped working on it for about 8 months. The HIR => MIR conversion I had at the time was fundamentally broken and I didn’t know how to fix it.

So what now?

Now that I’ve solved the HIR => LIR conversion process, I can focus on the type system and adding language features like I was before. This is the exciting part since I have a clear picture of how the language should work (it will be slightly different than previous designs).

I expect to have a usable and well tested implementation of the Rue compiler in the coming month(s). Until then, stay tuned!

I want to give a huge thanks to Don Kackman for all of the help with the project and for making it possible to get this release across the finish line.

You can download binaries for MacOS, Windows, and Linux on GitHub or download the app for desktop, Android, or iOS on the Sage website (although the mobile versions need to go through their respective approval processes first).

The biggest two changes are architectural, but a lot of features and bug fixes have been slipped in along the way.

Action System

We’ve fully transitioned from manually constructing transactions to using the new Action System introduced in the Wallet SDK. The action system (not to be confused with Yakuhito’s new Action Layer which itself will power many awesome primitives) is a major simplification to how coins can be spent on the Chia blockchain. If you’re interested in a high level explanation of how actions work, I’ve attached my presentation (in fact, the first formal presentation I’ve done before) from Chia Toronto.

Database Redesign

The wallet’s entire database schema has been rewritten from the ground up. There were a lot of motivators for this change:

1. Since the database had been designed alongside the wallet, a lot of outdated or less informed decisions went into it. This led to unnecessary complexity, bloat, and tech debt (such as having to copy data to multiple redundant places in the database).

2. It was hard to add new features to the wallet, such as non-custody p2 puzzles (including clawbacks) and revocable CATs. Adding option contracts to such a database schema (which we plan to do later) would have been a nightmare.

3. A lot of information that is shared between each type of asset (XCH, CATs, NFTs, DIDs) was instead sporadically stored in various places. We’ve now consolidated it into a single assets table, for the most part.

4. Because natural keys (coin ids, puzzle hashes, etc) were the primary key in the database, queries with lots of joins became slow. This has been fixed by using surrogate keys (auto incrementing numeric ids) instead.

Revocable CATs

A new Revocable CAT primitive has been introduced, and will first be used by Permuto Capital. This is an add-on to the CAT standard that enables the issuer to revoke coins if needed. The S-1 filing details why and when this might happen, but in summary it’s due to legal compliance reasons. Almost all CATs that are not securities will continue to not be revocable (ie only the current holder has the right to spend the coin).

The Chia Cloud Wallet supports Revocable CATs already, so Sage now supports the standard as well for feature parity. However, due to the potential security risks of someone sending you a revocable CAT and misleading you into thinking it’s not revocable, there are various warnings and safeguards in place to make sure users are aware of which type of CAT they are looking at.

Clawback v2

A preliminary implementation of Clawback v2 has been implemented in Sage. This might not be the clawback standard you are used to from the Chia reference wallet. Essentially, rather than the sender being able to claw back forever, they have a fixed time at which the ability to claw back expires (the same instant at which the recipient can spend the coins). This also means that the recipient no longer has to “claim” the coin by sending it to themselves.

While this new clawback standard can provide a better user experience, it comes with a couple drawbacks:

1. If you forget to claw back in time before the expiration, you will not be able to. Make sure you set your clawback expiration time far enough into the future that you have enough time to react (and don’t forget)!

2. If you send a clawback coin to a recipient that doesn’t support the Clawback v2 standard (ie any wallet other than Sage at the time of writing this, but exchange wallets are known to not support clawbacks), you will have until the expiration to claw the funds back, since they won’t be able to receive it naturally. If you forget to do this, however, there is an additional way to spend the coins to “push through” the coin to the receiver. So the coins aren’t lost even if the receiver doesn’t support Clawback v2, it’s just more work to get it through to them.

The original clawback standard remains unsupported at this time, however it may be implemented in the future depending on demand for it.

Resyncing

We all wish it weren’t the case, but unfortunately the wallet will sometimes fall out of sync with the current state of your wallet on the blockchain. Although improvements are constantly being made to Sage to reduce the need for a resync, it’s still required from time to time if something goes wrong.

However, there are a few major improvements to resyncing in this release:

1. The default behavior for resyncing is now to preserve coin and asset data. This means that it will typically only take a few seconds rather than waiting for a long time for all of your coins to trickle back in. Instead of re-indexing everything from scratch it will only check for coins that are missing or out of date. However, you can still decide to delete additional data while resyncing if that’s not sufficient.

2. There’s a new “Delta Sync” option. The default behavior in Sage is to only look for and sync coins that are after the previous block height that you were synced to. However, if things happen in the wrong order and you end up with some coins missing (due to a race condition), you may need to resync. If you turn off Delta Sync, the wallet will check for coins it owns every time it’s started. This comes with a slight performance tradeoff, so it’s up to you whether you want to turn it off or not.

RPC API

There is still no formal documentation for the RPC API, however I’d like to point out that there have been some breaking changes as a result of the refactors made in this release. Please check the Sage RPC crate for more details if you rely on these API endpoints. This API is not yet stable, so please use it with caution and pay attention to updates.

Other Features

There have been various other additions in this release, including:

1. Support for viewing offers with chia-offers.com short URLs

2. CSV export and offer sharing on mobile

3. You can now mint multiple editions of a single NFT at once

4. Offers work with fast forward (ie offer involving vaults)

Bug Fixes

The following bugs have been fixed:

1. You will now get an error in the UI if a transaction fails. Additionally, bugs relating to unconfirmed coins being left behind or coins you should own mistakenly disappearing until a resync should now be resolved.

2. NFT collections will now no longer be displayed if you don’t own any NFTs in them anymore.

3. The app would crash about every other time it was opened on iOS. This should be entirely resolved now, but please report any further crashes you run into.

4. Some UI errors could lead to blank screens. Known issues have been resolved, but if anything new comes up, please let us know.

Conclusion

Now that all of this has been finished, I feel a lot better about being able to add new features in a shorter amount of time like we used to. You can expect more frequent Sage releases going forward.

The next feature that is yet to be released is Option Contracts. I was originally planning on including it in this release, but due to the amount of other changes being made and the complexity of the feature, I decided to punt on it for now so that it could be ironed out more.

Let us know on Discord if you have any feature requests or find any bugs with the latest release!

The problem

When you create a new wallet, you are generating a 12 or 24 word seed phrase and (hopefully) backing it up somewhere safe in case you lose it. Then, it gets imported into the app to be used to find coins and sign transactions. There are various potential pitfalls with this approach.

First of all, if you lose the seed phrase, there’s no way to sign transactions to spend your coins, so they’re effectively lost forever. As such, it’s important to back up the key in a secure place where only you can find it.

However, that introduces the second problem, which is that if anyone ever gets access your seed phrase they could immediately spend all of your coins without restriction. For example, if someone compromises your device or finds a backup you have saved somewhere of your mnemonic, you could lose everything.

For storing relatively small amounts of assets that you use on a regular basis, the risks associated with a hot wallet like this can be acceptable. However, it’s not sufficient if you need to custody a lot of funds or have multiple people that can spend them.

Introducing vaults

A vault is what’s known as a “singleton”. Like an NFT, it will always create a single new instance of itself when spent. You can use a vault to solve the problems described previously.

Let’s break down how that works:

Normally, when you spend a coin you need to sign a message with your private key. This means the key needed to spend the coin is encoded within the coin itself. With a vault, you are essentially extracting the custody out of the coin and into a standalone singleton. The vault can “authorize” the spend of the other coins.

This means that you don’t need to spend all of the coins to change the way you custody them. For example, if you decide you want to change which key you’re keeping the coins in, you can simply update the vault and they will all be implicitly “owned” by the new key.

However, in this scenario you are still using a single key to control the coins, which isn’t much better than before. Fortunately, vaults are very flexible and there are many different puzzle pieces you can fit together to make a vault more secure.

Secp256k1 and secp256r1

Standard transactions on Chia use what is known as a BLS key. These have very useful properties, such as the ability to aggregate multiple signatures together to save space. However, BLS keys are not as widely supported as other cryptographic signature schemes such as SECP.

K1 is commonly used on other blockchains such as Bitcoin and Ethereum. So, you could in theory achieve a higher level of interoperability with other chains by using this curve inside of a vault.

However, the biggest advancement in security on Chia involves the R1 curve, which enables you to use isolated hardware to sign transactions on certain devices.

Here is a couple examples that use R1:

• Apple’s Secure Enclave, which is embedded silicon in modern iPhones that allows you to store keys that can only be used by unlocking the phone. Additionally, it shouldn’t be possible to extract keys from the Secure Enclave, which means you can be confident that only you have access to them.

• Passkeys (which can be used with the WebAuthn standard) allow you to securely store a key in your browser, device, or password manager. Signing is performed either on-device or by external FIDO security keys (e.g. Yubikey).

The Chia Signer app created by Chia Network Inc allows you to use your iPhone as a vault signer on the Cloud Wallet.

Multisig

With a vault, you can use an “M of N” multisig instead of a single key. This is implemented with merkle trees as an optimization in some cases so that not every spend path needs to be revealed on-chain if they aren’t being used. Here’s a few examples:

• You could allow 3 people to have access to a vault with a 1 of 3.

• With a 2 of 2, you can require two keys to always sign off on transactions.

• A 3 of 5 would allow the majority to agree on whether to do a transaction without needing unanimous approval.

This is pretty powerful since it allows you to decentralize the control a bit and prevent a single person from being able to spend all of the assets. It can help solve the “wrench attack” where a single person can be threatened into performing a transaction, since you require multiple people to sign off on it.

This gets a little into the weeds, but to expand on this further, you could in theory nest a multisig inside of another. For instance, if you want to require 3 of 4 people to do a transaction, but there’s a also separate pair of 2 that can do it instead. This would be a 1 of 2 with a 3 of 4 and a 2 of 2 inside of it.

Delayed recovery

With this advanced multisig setup, it’s more important than ever to make sure you don’t lose access to your funds even if you lose your phone, need to switch to a new one, or can no longer use a passkey.

With vaults, you can create a special “recovery key” that can be used to change the custody configuration of the vault if all else fails. However, you don’t want to give full control over the vault to this recovery key, especially if you’re using a multisig. Imagine if you have a 3 of 5 vault but one person uses the recovery key to take control.

To solve this, we use what’s known as “delayed recovery”. When you initiate a recovery of a vault, you can’t authorize spends for any of the coins. A time lock period for recovery (which is configurable beforehand) is kicked off. Using a watchtower service, each signer would get a notification that recovery has been initiated for the vault. They can then decide to cancel it (and remove the recovery key) or let it happen, until the time lock expires. Importantly, the new vault configuration has already been decided at the time of recovery being initiated, so you have confidence it will do what it says it’s going to do.

It’s worth noting that the recovery key is similar to the custody key in that you can configure it however you want. It could be a BLS key, passkey, or even a multisig.

In conclusion

With vaults on Chia, the doors are wide open to a future of better custody of digital assets. And combined with peer-to-peer offers, we’re able to have fully trustless and decentralized finance without sacrificing security.

Stay tuned for a more advanced deep dive into what you can do with vaults and how they interact with offers!

You can visit the Rue Docs and GitHub repository for more info, but keep in mind that the language is still very unfinished, as we’ll get into later. Keep reading to learn more!

CLVM

First, let’s start off with a bit of background. The Chia blockchain has an on-chain programming environment called CLVM (the Chialisp Virtual Machine), which provides a set of low level opcodes which control the conditions under which a coin can be spent. I won’t go too into the weeds here, but will give a summary.

For example, take this program in Chialisp:

You can think of a puzzle as a function that takes in some arguments (referred to as the solution) and returns a list of conditions. In this case, the AGG_SIG_ME condition (which roughly means aggregate signature for my coin) ensures that the public key has signed off on the conditions passed into the solution.

This compiles to the following CLVM bytecode, which is what is actually used on-chain:

It’s nearly impossible to figure out what’s going on without the original source code. So that’s why we have the higher level Chialisp language with a compiler to generate this. We sort of get the best of both worlds, it’s simple and efficient to execute when compiled, and readable by us humans before compilation.

The Problem

That was a pretty simple example of Chialisp, but it can get much more complicated.

For instance, this is the simplest example that illustrates my point that I could find, from the CAT puzzle:

Lists in CLVM are made up of cons-pairs. In this case, a lineage_proof is structured like this (where f means first and r means rest, the two halves of the pair):

1. (f lineage_proof) = parent_parent_coin_id

2. (f (r lineage_proof)) = parent_inner_puzzle_hash

3. (f (r (r lineage_proof))) = parent_amount

So to know what this code is doing you need to know what lineage_proof is first.

Imagine if you could write code like this instead:

Sure, it’s more verbose, but with this you have the following benefits:

1. The syntax is more familiar if you’re coming from a language like JavaScript or Rust.

2. Everything is strictly typed at compile time, so you can catch errors during development instead of at runtime.

3. Types like LineageProof are more clearly defined, so you can refer to the fields by name instead of having to manually index them.

Enter, Rue Lang

The code above is written in a new language called Rue, which compiles directly to CLVM, just like Chialisp does. My primary goal with the language is to lower the barrier to entry to create smart coins on the Chia blockchain. I believe Rue has the potential to greatly simplify the development process, and make it less difficult to write secure code.

Like Chialisp, the language is “purely functional”, which means:

1. Once you declare a variable, it cannot be modified. I like to think of it more as binding a value to a name rather than making a variable, since they can’t vary.

2. Functions themselves are values, so they can be passed around like any other. This is known as “higher order functions”.

3. There are “no side effects”. If you run a program multiple times with the same input, it will always produce the same output.

However, there are a few key differences:

1. As of writing, Rue has a more powerful type system (in fact, by default Chialisp is completely untyped).

2. It has a built-in standard library of common functionality you’ll need, like conditions and tree hashing.

3. You can export and import symbols from other modules. This could pave the way for making a package manager for Chia.

4. Generally, the syntax and scoping makes code simpler and reduces the number of surprises. That said, modern Chialisp has gotten a lot better in this regard.

5. There’s a proper language server that has type errors, compiler warnings, syntax highlighting, and in the future features like Go To Definition.

The code produced by Rue and Chialisp are different, and currently it is not a design goal to make the outputs completely compatible with each other. However, the same program written in both languages should behave the same.

So, where are we at?

I got pretty far with implementing the compiler, and you can view the current codebase in the GitHub repo.

However, I ran into some issues when testing more complicated programs that illuminated underlying problems with the compiler’s code. The type system also still needs to be improved a bit. I haven’t completely decided whether to rewrite the compiler from the ground up or just redesign certain parts of it incrementally. But I have learned so much since I started the project that it’s tempting to start over (especially since I can’t easily resolve the current problems).

As you may know, I work at Chia Network Inc. In addition to that I also maintain Sage Wallet and the Chia Wallet SDK in my spare time, which pretty much takes up most of the free time I have. Because of this, I haven’t been able to dedicate as much time as I would like to working on Rue Lang, and it’s been left in this sort of broken state of which I can’t really recommend anyone to use it in production.

What can you do to help?

If you’ve read this far and are thinking “what is he on about”, that pretty much sums up my concern. It’s going to be difficult to find someone with the knowledge about or interest to learn all of the following:

1. Chialisp

2. The low level CLVM runtime

3. Rust programming

4. Compiler design

If you do and would like to help with the compiler directly, please reach out.

That said, if I’m being realistic I need to just find more time to work on it so that it can get done. Currently, much of my time is spent on Sage Wallet and the Wallet SDK, both of which are more approachable projects for people to contribute to. The more others can help out with the development of those projects, the more free time I have to work on Rue (and I greatly appreciate those who have already contributed).

I’m open to other ideas on how we can make this new language happen, if you have them. Open source development can be time consuming, but I believe it’s all possible, eventually.

Thanks for reading!

Here are some of the projects you can expect to hear more about:

Chia Wallet SDK

A collection of libraries for building apps (especially wallets) on Chia, including driver code for Chialisp puzzles, signing utilities, a simulator framework, and support for making and taking offer files.

I’d say this is the most widely useful project for the developer community, and it’s quickly becoming a foundational part of various ecosystem projects.

Sage Wallet

A performant and capable light wallet written using the Chia Wallet SDK. It connects to random peers on the network, syncs XCH, CATs, NFTs, and DIDs from them, and provides a good user experience for transacting with them.

There’s a lot of functionality that still needs to be added to Sage. In fact, there are 50 open issues at the time of writing this, mostly feature requests or UX improvements, but some of them are issues that need to be solved.

Rue Lang

Although still little more than a rough draft of a language, Rue is poised to be a promising alternative to Chialisp with a syntax that will feel more at home for most developers. It provides some niceties like compiler enforced typing, modules (although not fully implemented yet), compile time type checking enforce through runtime code generation, and a standard library.

I’ll be writing more about Rue and my plans for it in the next post.

Puzzle Schema

This is a project that hasn’t had much development yet since it relies on Rue Lang being in a more stable state. Essentially the idea is to make it possible to describe the driver code (which controls how coins are spent) in the same language as the coin’s puzzle is written in. These schemas could then be shared around and imported by wallets to enable them to support arbitrary custom coins.

This project, once completed, will improve ecosystem adoption of new primitives or combinations of primitives, by making the logic around spending coins more modular.

Thanks for reading! Subscribe for free to receive new posts and support my work.