Introduction

Diplomat is a framework and tool for generating bindings to Rust libraries from an extensible set of languages.

Diplomat is for unidirectional bindings: it's for when foreign code wishes to call into a Rust library, but not vice versa. If you're looking for bidirectional bindings, tools like cxx are a good bet.

Diplomat is a proc macro paired with a tool. The proc macro is capable of generating an extern "C" binding layer around tagged Rust code, while the tool is able to generate corresponding C, C++, JS, or <insert language here> that philosophically matches the API on the Rust side. This means that methods in Rust map to "methods" in the target language, Result in Rust map to tagged unions in C++ and exceptions in Javascript, etc. These all work through the generated C API, however they preserve higher level API features which C cannot express.

A note on the design

You can read the full design doc here.

Diplomat does not do cross-crate global analysis, it restricts its view to specially tagged modules, and only generates bindings based on information found in those modules. This means that changing some struct in some dependency will not magically affect your generated C++/JS/etc APIs; all such change can only come from deliberate change to these tagged modules. This also means that Diplomat can cleanly define a subset of Rust used for declaring the exported API without impacting the flavor of Rust used in dependencies. One can imagine #[diplomat::bridge] blocks to almost be a DSL for bridging between your Rust APIs and a more general API shape that can be translated cleanly across languages.

Diplomat is designed such that it should not be a large amount of effort to write new language targets for Diplomat.

Setup

To install the diplomat CLI tool, run

$ cargo install diplomat-tool

You can then add diplomat as a dependency to your project like so:

diplomat = "0.5.0"
diplomat-runtime = "0.5.0"

It is recommended to create a separate crate for the FFI interface. Diplomat will only read the contents of specially tagged modules so it is possible to mix Diplomat code with normal Rust code, but it is prefereable to minimize this since proc macros can make debugging hard.

User Guide

Setup

To install the diplomat CLI tool, run

$ cargo install diplomat-tool

You can then add diplomat as a dependency to your project like so:

diplomat = "0.5.0"
diplomat-runtime = "0.5.0"

It is recommended to create a separate crate for the FFI interface. Diplomat will only read the contents of specially tagged modules so it is possible to mix Diplomat code with normal Rust code, but it is prefereable to minimize this since proc macros can make debugging hard.

Basics

When using Diplomat, you'll need to define Rust modules that contain the Rust APIs you want to expose. You can do this by using the diplomat::bridge macro:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    pub struct MyFFIStruct {
        pub a: i32,
        pub b: bool,
    }
    
    impl MyFFIStruct {
        pub fn create() -> MyFFIStruct {
            MyFFIStruct {
                a: 42,
                b: true
            }
        }

        pub fn do_a_thing(self) {
            println!("doing thing {:?}", self.b);
        }
    }
}
}

This is a simple struct with public fields; which is easier to reason about in an introductory example. Most APIs exposed via Diplomat will be via "opaque types", to be covered in the chapter on opaque types.

Every type declared within a diplomat::bridge module along with all methods in its associated impl will be exposed over FFI. For example, the above code will generate the following extern API:

#![allow(unused)]
fn main() {
#[no_mangle]
extern "C" fn MyFFIStruct_create() -> MyFFIStruct {
    MyFFIStruct::create()
}

#[no_mangle]
extern "C" fn MyFFIStruct_do_a_thing(this: &MyFFIStruct) {
    this.do_a_thing()
}
}

We can then generate the bindings for this API using the diplomat-tool CLI.

C++

For example, if we want to generate C++ bindings, we can create a folder `cpp/`` and generate bindings in it by running:

$ diplomat-tool cpp cpp/

This will generate the following struct in MyFFIStruct.hpp, along with some boilerplate:

struct MyFFIStruct {
 public:
  int32_t a;
  bool b;
  static MyFFIStruct create();
  void do_a_thing();
};

If we want to generate Sphinx documentation to cpp-docs, we can run with that as an additional parameter:

$ diplomat-tool cpp cpp/ --docs cpp-docs/

WASM

For WASM JS/TypeScript bindings, you can use the following options, with similarly named directories:

$ diplomat-tool js js/ --docs js/docs/

This will generate JS that has a MyFFIStruct class, with a static create() method, a do_a_thing() method, and getters for the fields. This JS will require there to be a wasm.mjs file that loads in the built wasm file (See issue #80 for improving this), which you can base off of this file.

C

While low-level C headers are generated in the process of running diplomat-tool cpp, you can also generate just the C headers with

$ diplomat-tool c c/

Note that Diplomat's C mode generates direct bindings to the lower level extern "C" API, and is not idiomatic C code. It is recommended that one build a higher level API around the C API (perhaps by writing a plugin) if C bindings are desired.

Types

Diplomat only supports a small set of types that can be passed over FFI. These are:

• Builtins:
  • All integers, as well as bool and char
  • &[T] where T is an integer, bool, or char
  • &str (string slices)
  • DiplomatWriteable for returning strings
  • Result<T, E> in return values
  • Option<T> of opaque types
  • () as a Result Ok/Error type, or as a return value
• Custom types
  • Custom opaque types (passed as references or via Box<T>)
  • Custom structs and C-like enums

More types can be supported in the future (We have issues for iterators and callbacks)

The main distinction to keep track of is between "opaque types" and "structs": opaque types are for when you want to wrap a Rust object that has its own semantics, whereas "structs" are for when you want to transparently pass around multiple values at once (usually when you want to make an options struct as an argument, or return multiple values at once).

Opaque Types

In the vast majority of cases, we'd like to expose Rust types over FFI "opaquely", that is, the FFI code does not know anything about the contents of these types, rather it wants to do things with the type.

By default, Diplomat will not let you expose fields of types other than the allowed types over FFI. The following code will trigger a resolution error when running diplomat-tool:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    pub struct MyFFIType {
        pub a: i32,
        pub b: Vec<String>, // or "SomeTypeDefinedElsewhere"
    }
    
    impl MyFFIType {
        pub fn create() -> MyFFIType {
            todo!()
        }
    }
}
}

Of course, if Diplomat is to be able to usefully expose Rust APIs without requiring everything be defined within Diplomat's bridge blocks, there has to be some way to include them in this the API.

For this in Diplomat we declare opaque types, which can only exist behind pointers. Such types can contain whatever they want, but they can never be passed over the stack through FFI, and the other side cannot peek into them in ways other than calling explicitly defined methods.

For example, say we have the following type:

#![allow(unused)]
fn main() {
struct MyCollection {
    name: String,
    items: Vec<String>,
}

impl MyCollection {
    fn new(name: String) -> Self {
        Self {
            name, items: vec![]
        }
    }

    fn push(&mut self, s: String) {
        self.items.push(s)
    }

    fn dump(&self) {
        println!("Collection {} with items {:?}", self.name, self.items);
    }
}
}

To expose it over FFI, we'd do something like:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    // import this from wherever, does not need
    // to be the same crate
    use super::MyCollection as RustCollection;

    #[diplomat::opaque]
    pub struct MyCollection(RustCollection);

    impl MyCollection {
        pub fn create(s: &str) -> Box<MyCollection> {
            Box::new(MyCollection(RustCollection::new(s.into())))
        }

        pub fn push(&mut self, s: &str) {
            self.0.push(s.into())
        }

        pub fn dump(&self) {
            self.0.dump()
        }
    }
}
}

This will generate code exposing create(), push(), and dump() over FFI, as well as glue to ensure the destructor is called. However this will not expose any way to get at the RustCollection.

For example, the generated C++ looks something like

class MyCollection {
 public:
  static MyCollection create(const std::string_view s);
  void push(const std::string_view s);
  void dump();
  // snip
 private:
  // just a pointer with a custom destructor
  std::unique_ptr<capi::MyCollection, MyCollectionDeleter> inner;
};

When exposing your library over FFI, most of the main types will probably end up being "opaque".

Boxes are return-only

Box<T> can only be returned, not accepted as a parameter. This is because in garbage collected languages it is not possible to know if we are the unique owner when converting back to Rust. There are some techniques we could use to add such functionality, see #317

Structs and enums

Diplomat allows for exposing basic structs and enums over FFI. Typically these should be used as inputs and outputs to other methods, rather than having methods of their own, however it is possible to give them methods which capture self by-value.

Structs are most commonly found when making an options type for a method, or when doing multiple return values.

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    use my_thingy::MyThingy;

    // just exists so we can get methods
    #[diplomat::opaque]
    pub struct Thingy(MyThingy);

    pub struct ThingySettings {
        pub a: bool,
        pub b: u8,
        pub speed: SpeedSetting,
    }

    #[diplomat::enum_convert(my_thingy::SpeedSetting)]
    pub enum SpeedSetting {
        Fast, Medium, Slow
    }

    #[diplomat::enum_convert(my_thingy::ThingyStatus)]
    pub enum ThingyStatus {
        Good,
        Bad
    }

    impl Thingy {
        pub fn create(settings: ThingySettings) -> Box<Thingy> {
            // Convert our FFI type to whatever internal settings type was needed
            let settings = my_thingy::ThingySettings {
                a: settings.a,
                b: settings.b,
                speed: settings.speed.into()
            };
            Box::new(Thingy::new(settings))
        }

        pub fn get_status(&self) -> ThingyStatus {
            self.0.get_status().into()
        }
    }
}
}

Enums exposed via Diplomat must be simple C-like enums. Structs may only contain fields which are allowed.

In C++ the structs are translated to simple structs and the enums become simple enum classes. In JS the structs become objects with fields, and the enums are exposed as strings that get converted at the boundary.

diplomat::enum_convert

Diplomat can autogenerate Into impls to an enum from your library using #[diplomat::enum_convert]:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    // ...

    #[diplomat::enum_convert(my_thingy::SpeedSetting)]
    enum SpeedSetting {
        Fast, Medium, Slow
    }

    // ...

}
}

In case the enum is #[non_exhaustive], you may need to supply a needs_wildcard argument, like so: #[diplomat::enum_convert(my_library::SpeedSetting)].

Structs containing boxes

By default, structs cannot contain output-only types like Box<T>. This can be opted in to by using #[diplomat::out]

#![allow(unused)]
fn main() {
mod ffi {
    use my_thingy::MyThingy;

    #[diplomat::opaque]
    pub struct Thingy(=MyThingy);

    #[diplomat::out]
    pub struct ThingyAndExtraStuff {
        pub thingy: Box<Thingy>,
        pub stuff: u32
    }

    impl Thingy {
        pub fn create() -> ThingyAndExtraStuff {
            let thingy = Box::new(Thingy(MyThingy::new()));
            let stuff = 42;
            ThingyAndExtraStuff {
                thingy, stuff
            }
        }
    }

}
}

Option types

Option types in Diplomat are relatively straightforward, you simply use Option<T> and it turns into the idiomatic equivalent over FFI.

Option<T> currently only works when wrapping reference types (Box<OpaqueType> and &OpaqueType).

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    // just exists so we can get methods
    #[diplomat::opaque]
    pub struct Thingy;

    impl Thingy {
        fn maybe_create() -> Option<Box<Thingy>> {
            Some(Box::new(Thingy))
        }
    }
}
}

In C++ this will return a std::option<Thingy>, and in JS it will return a potentially-null object.

Result types

Result types are returned by using Result<T, E> (or DiplomatResult<T, E>).

For example, let's say we wish to define a fallible constructor:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    #[diplomat::opaque]
    struct Thingy(u8);

    impl Thingy {
        pub fn try_create(string: &str) -> Result<Box<Thingy>, ()> {
            let parsed: Result<u8, ()> = string.parse().map_err(|_| ());
            parsed.map(Thingy).map(Box::new)
        }
    }
}
}

On the C++ side, this will generate a method on Thingy with the signature

  static diplomat::result<Thingy, std::monostate> try_create(const std::string_view string);

diplomat::result is a type that can be found in the generated diplomat_runtime.hpp file. The most basic APIs are .is_ok() and .is_err(), returning bools, and .ok() and .err() returning std::options. There are further APIs for constructing and manipulating these that can be found in the header file.

On the JS side it will continue to return the Thingy class but it will throw the error (as an empty object in this case) in case of an error.

Writeables

Most languages have their own type to handle strings. To avoid unnecessary allocations, Diplomat supports DiplomatWriteable, a type with a Write implementation which can be used to write to appropriate string types on the other side.

For example, if we want to have methods that philosophically return a String or a Result<String>, we can do the following:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    use diplomat_runtime::DiplomatWriteable;
    use std::fmt::Write;

    #[diplomat::opaque]
    #[derive(Debug)]
    pub struct Thingy(u8);

    impl Thingy {
        pub fn debug_output(&self, writeable: &mut DiplomatWriteable) {
            write!(writeable, "{:?}", self);
        }

        pub fn maybe_get_string(&self, writeable: &mut DiplomatWriteable) -> Result<(), ()> {
            write!(writeable, "integer is {}", self.0).map_err(|_| ())
        }
    }
}
}

On the JS side these will get converted to APIs that return strings (maybe_get_string will potentially throw in the case of an error, as is usual with DiplomatResult)

In C++ multiple APIs are generated:

  std::string debug_output();
  diplomat::result<std::string, std::monostate> maybe_get_string();
// and
  template<typename W> void debug_output_to_writeable(W& writeable);
  template<typename W> diplomat::result<std::monostate, std::monostate> maybe_get_string_to_writeable(W& writeable);

Essentially, versions of the API returning std::string are generated, where the write!() operation will end up writing directly to the std::string with no additional intermediate Rust String allocations.

WriteableTrait

The template versions work on any type that is hooked into WriteableTrait, allowing . Types can be hooked into WriteableTrait as follows:

template<> struct WriteableTrait<MyStringType> {
  static inline capi::DiplomatWriteable Construct(MyStringType& t) {
    // ...
  }
}

This requires constructing a DiplomatWriteable from the custom string type, which is documented in more detail in the source. Essentially, it involves constructing an ad-hoc virtual dispatch object for the type.

Documentation

Some Diplomat backends support --docs, which will generate additional documentation from your Markdown doc comments

$ diplomat-tool cpp cpp/ --docs cpp-docs/

The C++ and JS backends generate Sphinx docs. If using TypeScript, the definition files will automatically come with tsdoc-compatible doc comments.

A limited amount of intra-doc-links are supported: it is possible to link to custom types (but not methods or variants) using [`FooBar`] syntax, like Rust.

Furthermore, you can use #[diplomat::rust_link(path::to::rust::type, Struct)] to autogenerate links for to published docs, which typically show up as a "For more information see <link>" at the bottom of the docs for the given item. Since Diplomat cannot do resolution on other crates, it relies on the rust_link annotation to provide the kind of Rust item or doc page being linked to. An additional compact parameter can be passed in case you wish to provide multiple rust_links that are to be collapsed into a single "For more information see 1, 2, 3" line.

Put together, this might look something like the following:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {
    use my_thingy::MyThingy;

    /// A Thingy
    #[diplomat::rust_link(my_thingy::MyThingy, Struct)]
    #[diplomat::opaque]
    pub struct Thingy(MyThingy);

    #[diplomat::enum_convert(my_thingy::SpeedSetting)]
    #[diplomat::rust_link(my_thingy::SpeedSetting, Enum)]
    pub enum SpeedSetting {
        Fast, Medium, Slow
    }

    #[diplomat::enum_convert(my_thingy::ThingyStatus)]
    #[diplomat::rust_link(my_thingy::ThingyStatus, Enum)]
    pub enum ThingyStatus {
        Good,
        Bad
    }

    impl Thingy {
        /// Make a [`MyThingy`]!
        #[diplomat::rust_link(my_thingy::MyThingy::new, FnInStruct)]
        pub fn create(speed: SpeedSetting) -> Box<Thingy> {
            Box::new(Thingy(Thingy::new(speed.into())))
        }

        /// Get the status
        #[diplomat::rust_link(my_thingy::MyThingy::get_status, FnInStruct)]
        pub fn get_status(&self) -> ThingyStatus {
            self.0.get_status().into()
        }
    }
}
}

The full list of item kinds recognized by rust_link is:

• Struct
• StructField
• Enum
• EnumVariant
• EnumVariantField
• Trait
• FnInStruct
• FnInEnum
• FnInTrait
• DefaultFnInTrait
• Fn
• Mod
• Constant
• AssociatedConstantInEnum
• AssociatedConstantInTrait
• AssociatedConstantInStruct
• Macro
• AssociatedTypeInEnum
• AssociatedTypeInTrait
• AssociatedTypeInStruct
• Typedef

Lifetimes

Diplomat is able to safely handle methods that do complex borrowing, as long as the lifetimes are fully specified (i.e., not elided).

In general, Diplomat attempts to follow language norms when it comes to patterns akin to borrowing. In C++, for example, the norm is to document the behavior of a method, which is what Diplomat does. However, in JS, the norm is to not have use-after-frees, which Diplomat attempts to achieve by stashing extra references to borrowed-from objects.

For example, let's take this iterator API:

#![allow(unused)]
fn main() {
#[diplomat::bridge]
mod ffi {

    #[diplomat::opaque]
    pub struct MyFancyVec(Vec<u32>); // not very fancy

    pub struct MyFancyIterator<'a>(std::slice::Iter<'a, u32>);


    impl MyFancyVec {
        pub fn new(count: usize) -> Box<MyFancyVec> {
            // make a random vector, this is an example
            let vec = (5..(count + 5)).collect();
            Box::new(MyFancyVec(vec));
        }

        pub fn iter<'a>(&'a self) -> Box<MyFancyIterator<'a>> {
            Box::new(MyFancyIterator(self.0.iter()))
        }
    }

    impl<'a> MyFancyIterator<'a> {
        fn next(&mut self) -> u32 {
            // We don't support Option of primitives in Diplomat currently
            // Just default to 0. We could use a struct to represent this instead
            // but it's not relevant to the example
            self.0.next().copied().unwrap_or(0)
        }
    }
}
}

It's crucial the return type of MyFancyVec::iter() is not held on to longer than the MyFancyVec it came from.

In C++, this will produce documentation that looks like the following:

Lifetimes: `self` must live at least as long as the output.

On the other hand, in JS, the JS object wrapping MyFancyIterator will internally stash a reference to the MyFancyVec, which will be noticed by the GC, keeping the vector alive as long as needed.

This also works with non-opaque structs; you can have a function that takes in a struct where one field has a lifetime, and returns a different struct with a similar property, and Diplomat will document or GC-link the appropriate fields.

Backend Developer Guide

This is yet to be fleshed out. In general, if trying to write a backend, please use the Diplomat HIR ("higher level IR"). This is similar to a syntax tree but is far easier to work with, with paths being pre-resolved and a bunch of invalid states being unrepresentable.

It's obtained from a TypeContext, which is itself constructed from an Env, from an ast::File, which can be constructed from a syn module covering the entire crate.

Currently Diplomat has c2 and cpp2 backends that use the HIR. The other backends still use the AST, but we hope to move off of that. We recommend you look at the c2 and cpp2 backends of diplomat-tool to understand how to implement your own backend.

You can write a new backend as a standalone library, or as a module under tool. The Diplomat team is happy to accept new modules but may not necessarily commit to keeping them working when Diplomat changes. We promise to notify you if such a module breaks, and will always try to fix things when it's a minor change.