Add backends generator for compiler/simulator plugin description

[rjiejie]

User want to use SAIL to develop or describe new/custom instruction, and let it create
compiler toolchain or simulator plugin, e.g. LLVM, QEMU.
It can help to speed up development cycle.

Compiler/Simulator toolchain from Alibaba Damo support Json format description of instruction.
it uses json as input and create all toolchain plugins, user can use new/custom instruction in their
compiler or simulator.

I want to add backends generator for this Json spec,
any suggestions ?

Toolchain: https://www.xrvm.cn/soft-tools/tools/TAC
XCE (Plugin tools): XuanTieCustomExtensionV1.0.0.pdf (https://www.xrvm.cn/community/download?versionId=4569139451887947776)

[rjiejie]

Hi Alasdair Armstrong (@Alasdair) and Brian Campbell (@bacam),

I would like to follow up on the backend proposal in this issue and clarify
its intended scope.

We would like to develop a structured JSON backend for Sail and, subject to
your guidance and review, contribute it to the main Sail repository. The goal
is to expose selected information from Sail models—such as instruction
encodings, operands, assembly mappings, and architectural semantics—in a
machine-readable form for downstream tooling.

XCE would be the initial consumer and validation use case, but the backend
itself would remain general-purpose and avoid embedding XCE-specific behavior
wherever possible.

For the initial contribution, we propose to define a minimal, versioned schema,
generate deterministic JSON for a representative subset of Sail, diagnose
unsupported constructs explicitly, and add both golden-output and downstream
validation tests. Support could then be extended incrementally.

Before proceeding further, we would appreciate your guidance on two points:

1. Would you be supportive, in principle, of accepting such a general-purpose
   JSON backend into the main Sail repository?

2. If so, would you prefer that we first submit a short design proposal
   describing the schema, initial scope, and choice of Sail representation, or
   open a draft PR for review against a concrete implementation?

Even a brief indication of the preferred direction would help us avoid
unnecessary rework.

If another maintainer is more appropriate for this decision, please feel free
to redirect us.

Thank you for your time and guidance.

[rjiejie]

Hi @Alasdair and @wxrdnx,

I would like to follow up on this proposal and clarify its relationship with
the JSON backend in #1582.

Our goal is closely aligned with #1582: to convert Sail's internal
representation into a cross-language JSON representation that can be consumed
without depending on OCaml.

The additional requirement in our use case is that the JSON should contain
sufficient structured information to be consumed directly by compiler,
disassembler, and simulator generation tools. This includes, for example,
instruction encodings, operands, assembly syntax, and relevant semantic
information available from the Sail model.

XCE is the initial consumer used to validate this workflow, but the underlying
goal is a reusable, machine-readable representation rather than an
XCE-specific integration in the Sail core.

Would it be preferable to contribute these requirements and use cases to
#1582, or to develop them as a separate but related backend in the main Sail
repository?

If helpful, I can provide a minimal JSON example and the corresponding Sail
instruction to make the required representation concrete.

Thank you.

[wxrdnx]

Hi @rjiejie:

Wow, thanks for the follow up.

The goal of #1582 was fairly simple: serialize Sail's internal AST into JSON so that tools written in languages other than OCaml can consume it. That's all. It exposes data close to Sail's internal representation so that the internal AST can be consumed by JSON parsers written in other languages.

In fact, this proposal comes from a more specific academic research use case. I was working on RISC-V binary analysis, and the JSON representation was intended to expose the information needed by my C++-based BA tool. So the JSON data is probably not "sufficiently structured" in a way that can be easily consumed directly by compiler, disassembler, ... It is merely a direct translation of OCaml's internal AST into JSON. It does not include instruction encodings, assembly syntax, ...

For some additional context, my original intention was simply to maintain a fork myself. However, due to some extremely unfortunate circumstances, I stopped working on that project, and I completely forgot that this PR even existed .__.

That being said, I think our ideas are similar, and I believe there is some overlap between our goals. @Alasdair, what do you think?

[Alasdair]

I would be fine with this kind of json backend, but it does need to use the json library we already use (yojson) rather than just producing json by interpolating strings. It should probably also be set up as a parameterised module that has a custom annotation -> json function for the type annotations in the syntax tree, so people using libsail as an OCaml library can customise the output.

There's also some questions as to how we represent tagged unions with positional arguments, this is like 90% of the syntax tree and these are the two things that json supports extremely poorly, so it's worth careful consideration. Rust's serde-json library has a few options for this when serialising Rust tagged unions (enums).

[rjiejie]

Thank you @wxrdnx and @Alasdair. This clarifies both the intended scope and the
main design constraints.

I will start with a fresh implementation that produces Yojson.Safe.t values
throughout, with the serializer exposed as a parameterized module so that
libsail users can provide their own annotation-to-JSON conversion.

Before covering the full AST, I will prepare a small design prototype covering
representative nullary, unary, and multi-argument constructors. I will compare
the possible tagged-union encodings, including Serde's established enum
representations, with particular attention to schema stability and ease of use
from non-OCaml tools.

The initial draft will include the proposed JSON shapes and golden tests, so
that the representation can be reviewed before it becomes a larger
implementation. I will use #1582 as a useful reference for AST coverage, while
designing the yojson-based serialization layer from the outset.

Thank you for the guidance.