Facebook is embracing Rust, one of the most loved and fastest-growing programming languages available today. In addition to bringing new talent to its Rust team, Facebook has announced that it is officially joining the nonprofit Rust Foundation. Alongside fellow members including Mozilla (the creators of Rust), AWS, Microsoft, and Google, Facebook will be working to sustain and grow the language’s open source ecosystem.
For developers, Rust offers the performance of older languages like C++ with a heavier focus on code safety. Today, there are hundreds of developers at Facebook writing millions of lines of Rust code. And while it’s clear that Facebook is increasingly invested in the future of the language, it’s important to understand how we grew to this point.
2016–2017: Early use in source control
Our oldest Rust codebase dates to 2016, when the rate of source code changes in Facebook’s monorepo started to encroach on the maximum commit rate that the Mercurial source control management tool could keep up with. In response to this, Facebook’s Source Control team launched a rewrite project called Mononoke with the goal of increasing Mercurial’s commit rate by some additional orders of magnitude to serve Facebook’s thousands of developers and automated processes.
Developing Mononoke in C++ was the obvious choice at first. At the time, Facebook’s back-end codebase was very C++ heavy, meaning Mononoke would have been implemented in C++ by default. But the Source Control team needed to consider the reliability needs of the source control back end. When corruption or downtime can potentially bring services to a halt, reliability is a top priority. That’s why the team chose to go with Rust over C++.
Jeremy Fitzhardinge, a software engineer in production confidence at Facebook, describes this experience in a talk at RustConf 2019.
Mononoke served as a great test bed because it was naturally fairly isolated from other Facebook systems. As long as Mononoke could use the Mercurial protocol to speak with client services and the Thrift protocol to communicate with some storage systems, choosing Rust wouldn’t affect anything outside of the Source Control team’s work.
The Source Control team was willing and able to support itself in any Rust-specific tooling and infrastructure. This played out well, and Mononoke has been the production back end for our monorepo since 2019, successfully scaling over the years.
2017–2019: The adoption curve
With Mononoke as evidence that it was viable and lived up to its claims, over time, other projects considered and adopted Rust as well. At first, these were typically developer tooling projects that didn’t need to integrate with the broader service infrastructure, or small services/daemons that could do their work with just a few handwritten wrappers around some C++ client libraries.
Many of the engineers at Facebook who adopted Rust came from Python and Javascript backgrounds. They appreciated Rust’s combination of high performance with compile-time error detection. As more success stories, such as performance improvements at two to four orders of magnitude, circulated within the company, interest grew in using Rust for back-end service code and exploring its use in mobile apps as well.
2019–2020: Some dedicated support for Rust
From 2017 through 2019, the Source Control team doubled as the unofficial Rust support team within Facebook. But by 2019, the number of Rust developers at Facebook had grown exponentially, to over 100.
As one significant example of that growth, Rust is the leading language in the development of the Diem (formerly Libra) blockchain, which is overseen by the independent Diem Association. Facebook, through its digital wallet Novi, is a member of the Diem Association. The Diem blockchain is primarily written in Rust, covering 94 percent of the open source codebase. In addition, Move, a new, secure programming language to be used for the blockchain, was developed with Rust.
Given this growth in need, the part-time assistance of the Source Control team wasn’t enough to support the number of teams standing to benefit.
We created a small Rust developer experience team, which dedicates its attention to tooling and integration challenges, such as building the mechanism for using the language’s open source package registry ecosystem in production non-Cargo builds. The team formed a central point of contact for Rust developers across the company to unblock use cases, prioritize short-term developer experience concerns, improve core libraries, and generally set the fledgling Rust codebase up for success as it passed the million-line mark.
Onward (2021 and beyond)
At the end of 2020, we re-upped our commitment by launching a Rust team in our Programming Languages organization, the same org responsible for Facebook’s C++ standards work and toolchains.
This team has a longer-term focus beyond immediate developer experience priorities in the context of our codebase. In particular, all contributions to grow and sustain Rust as a healthy and productive language and community are now in scope.
In the short term, this new team is focused on four areas:
- Supporting internal users from a language and toolchain perspective: This includes toolchain rollouts, cross-cutting migrations, code review/audit, best practices, and acting as a point of contact for language and toolchain questions.
- Contributing positively in the community outside Facebook: The team performs code reviews of the standard library and compiler and provides developer resources toward priorities of the Rust community.
- Easy and safe interoperability of Rust with C++: We have a vast amount of C++ code that is necessary to communicate with the back-end systems that services are built on. We need developers to be able to use these libraries safely and easily without sacrificing the benefits that Rust offers. Inversely, if we want to see Rust components integrated with our larger C++ binaries, we need smart runtime interoperability in asynchronous code. Facebook’s servers are highly distributed and heavily threaded. Rust tasks need to play nicely on a C++ threadpool and safely share synchronization primitives and I/O resources. We have already done a lot of work in the C++ space improving asynchrony, having supported and quickly adopted C++20’s coroutines. Bringing Rust into the game will be an extension of that and will build on the excellent work already happening in the Rust asynchronous library stack.
- Actively supporting and engaging with the Rust Foundation: Facebook has been committed to the Rust community and expanding its development with Rust since 2016. We are taking that commitment even further by joining the Rust Foundation as a Platinum member, where we hope to help continue to drive the advancement and adoption in a proactive and impactful way.
Rust’s journey within Facebook is far from finished. This team is small, but it will grow over time as support needs scale. We are excited and optimistic about the trajectory of Rust within Facebook and across the industry. Our plans for work inside Facebook, open source contributions, and more community-oriented work will develop over the course of 2021.
