Build Systems for Multi-Language Monorepo

Updated Sep 03, 2021#system#tooling#monorepo

Most real-world programs use some kind of build system to manage complexities of compiling multiple source files for multiple configurations (debug vs. release), multiple platforms (x86, x64, ARM, and so on), custom build steps, and even multiple executables that must be compiled in a certain order.

Build systems are critical to creating a repeatable, reproducible and consistent build process. This becomes important as the company grows in size and more developers are added to the team and as more products are added to the product line.

Basically you’ll use these systems to orchestrate a combination of following tasks: fetch dependecies, compile source files, build executables, run tests, bundle resources, perform static analysis, or generate documentation.

Build systems are categorized into imperative (tells the systems what to do) or declarative (declares what we would like to have as the result).

If you’re only managing almost pure multi-package JavaScript/TypeScript monorepos then JavaScript monorepo tools are probably good enough on daily basis. Or build tools for specific languages like C++ (make, cmake, premake), Java (ant+ivy, maven), C# (msbuild).

Following build systems are very powerful but very complicated and steep to learn, great for enterprise-level multi-language monorepos (a codebase containing multiple projects using multiple programming languages and frameworks) but seem a bit overkill for small teams or individuals.

Name GitHub Written In From Build Languages
Bazel 17.3k Java, Python Google Starlark
Gradle 12.3k Groovy, Java Gradle Inc Groovy or Kotlin DSL
Buck 8k Java Facebook Starlark
Please 1.7k Go Thought Machine Subset of Python
Pants 1.6k Rust Twitter Python

Bazel

Bazel is an open-source build system originated from Google, uses a human-readable high-level build language, supports projects in multiple languages, builds outputs for multiple platforms, supports large codebases across multiple repositories and large numbers of users.

Speed up your builds and tests — Bazel only rebuilds what is necessary. With advanced local and distributed caching, optimized dependency analysis and parallel execution, you get fast and incremental builds.

One tool, multiple languages — Build and test Java, C++, Android, iOS, Go and a wide variety of other language platforms. Bazel runs on Windows, macOS, and Linux.

Scalable — Bazel helps you scale your organization, codebase and Continuous Integration system. It handles codebases of any size, in multiple repositories or a huge monorepo.

Extensible to your needs — Easily add support for new languages and platforms with Bazel’s familiar extension language. Share and re-use language rules written by the growing Bazel community.

Buck

Buck is a build system developed and used by Facebook. It encourages the creation of small, reusable modules consisting of code and resources, and supports a variety of languages on many platforms.

Speed up your builds — Buck builds independent artifacts in parallel to take advantage of multiple cores on your machine. Further, it reduces incremental build times by keeping track of unchanged modules so that the minimal set of modules is rebuilt.

Add reproducibility to your builds — Buck only uses the declared inputs, which means everybody gets the same results.

Get correct incremental builds — Buck looks at the contents of your inputs, not their timestamps to figure out what needs to be built. As a result, incremental builds should always be correct, so there’s no need to perform a clean build.

Understand your dependencies — With buck query, you can better understand your dependencies and what is required to build your product. Integrate with your IDE. With buck project, your project can be better understood by your IDE, making you and your team more productive.

Please

Please is a cross-platform (Linux, macOS, FreeBSD) cross-language (Go, Python, Java, JavaScript) build system with an emphasis on high performance, extensibility and correctness. It is designed to support interleaving custom build commands as first-class citizens in the system.

Build isolation — Build steps are executed in their own tightly controlled hermetic environment. With access only to files and env variables that have been made explicitly available, builds have never been more correct.

Blazing fast — Built from the ground up with incrementallity in mind, Please only builds what it needs. Combined with task parallelism and distributed caching, this makes for an amazingly performant build system at scale.

Feels amazing to use — Build and test any target, all with the same familiar CLI! Please can even manage and download your toolchain. Define aliases for common tasks leveraging Please auto-completions for the ultimate developer experience.

For any technology — With a powerful framework for automating every part of your build process, get started quickly with the built-in definitions, then expand on them crafting a truly holistic build process.

Gradle

Gradle is an open-source build tool that is designed to be flexible enough to build almost any type of software.

JVM foundation — Gradle runs on the JVM and you must have a Java Development Kit (JDK) installed to use it. Note that Gradle isn’t limited to building just JVM projects, and it even comes packaged with support for building native projects.

Build anything — Write in Java, C++, Python or your language of choice. Package for deployment on any platform. Go monorepo or multi-repo. And rely on Gradle’s unparalleled versatility to build it all.

Automate everything — Use Gradle’s rich API and mature ecosystem of plugins and integrations to get ambitious about automation. Model, integrate and systematize the delivery of your software from end to end.

Deliver faster — Scale out development with elegant, blazing-fast builds. From compile avoidance to advanced caching and beyond, we pursue performance relentlessly so your team can deliver continuously.

The most notable restriction with Gradle is that dependency management currently only supports Maven- and Ivy-compatible repositories and the filesystem.

Pants

Pants is a fast, scalable build system. Pants supports Java, Scala, Python, C/C++, Go, Javascript/Node, Thrift, Protobuf and Android code. Adding support for other languages, frameworks and code generators is straightforward.

Some noteworthy features include explicit dependency modeling, fine-grained invalidation, shared result caching, concurrent execution, remote execution, unified interface for multiple tools and languages, extensibility and customizability via a plugin API.