It’s not rocket science ¶

In 2014, Graydon Hoare, the creator of the Rust programming language, wrote a blog post entitled, “The Not Rocket Science Rule Of Software Engineering.” In the post, he desribes a CI system his team cobbled together circa 2001. The purpose of the system was to enforce one simple rule:

The Not Rocket Science Rule Of Software Engineering:

automatically maintain a repository of code that always passes all the tests

Lurking under that placid description is a simple concept, but it’s challenging to execute.

Why is this so hard ¶

Most CI systems only guarantee that the tests pass in your branch. But that’s different than saying your tests will pass when your branch merges.

One reason for this mismatch may be integration friction: your branch diverged from the main branch over time.

It’s unsafe to merge changes in parallel in a conventional CI system. Semantic conflicts between patches can break the build post-merge. Imagine renaming a method in one branch while merging a patch that depends on that method in another.

The queue ¶

Adopting a rebase-and-merge policy is a possible solution — where you rebase and merge all patches one at a time.

But processing a queue in serial is slow and won’t scale up as you add more developers.

Speculative future state ¶

By enqueuing patches to rebase-and-merge, you know what the codebase will look like after each patch is merged. That work can be done before you test and before you merge.

At the end of last year, GitHub introduced the Merge Queue, a mirror of GitLab’s (premium-tier only) Merge Trains feature to do exactly this.

You can put your merge requests into a queue to be merged. Each merge request in the queue is combined with the merge requests in front of it. It then runs its tests as if it had already merged.

We can run the tests for all patches in the queue in parallel. Waiting to merge each patch until all the patches in front of it pass their tests.

If a patch fails, we remove it and re-run the tests for all the patches behind it in the queue—it’s not rocket science!

This is how the Merge Queue and the Merge Train work today.

Multiply by 𝑛 ¶

One of my former colleagues was fond of saying:

The only numbers that matter in computer science are 0, 1, and 𝑛.

The “not rocket science” problem gets more complicated when a production build is composed of 𝑛 repos.

At my work, we’ve yet to embrace the monorepo trend 😬. The result is a release to production composed of roughly 200 different repositories – some of which depend on one another in an ever-spidering graph of maddening dependencies.

A merge queue of merge queues ¶

In 2012, the Openstack Infrastructure team started work on Zuul. What’s unique about Zuul is that it can assure that a build works, even when it’s composed of several repositories.

Zuul maintains what it calls “dependent pipelines.” Dependent pipelines can ensure merge requests across multiple repos get tested and merged together as if they were part of a single Merge Queue – a merge queue of merge queues.

Today it’s a decade after development began on Zuul — more than two decades after Graydon Hoare built his CI system. And the most prominent and best-funded software forges in the history of humanity are pretty gosh darn close to catching up.