Studies from the trenches

An IEEE study titled “Code Reviewing in the Trenches” by researchers at Microsoft found code authors aren’t getting timely reviews, and code reviewers aren’t given enough time to do the reviews they’re assigned. This is a problem.

In 2013, Peter Rigby and Christian Bird—both empirical software engineering researchers—found similar median review times among projects as varied as Google’s Chromium and Microsoft’s Office suite. Compared with the outdated industrial process of software inspection, their study’s modern code review practices were faster, easier, and less formal. Software inspection took 10 days to review a change; projects in their study took one day to review a change.

In 2018, Google compared Rigby and Bird’s findings with review data from inside their company. Among their findings, they discovered reviews inside Google were considerably faster than the reviews from Rigby and Bird’s study—the median time to review code inside Google was about an hour.

Organizations enable timely code review.

Why is code review at Google fast? Google’s code reviews follow the recommendations in Microsoft’s “From the Trenches” study, which says: “how an organization sets the stage for reviewing activities and how it supports and values code reviewing is critical to the success of code reviews.”

• Training: At Google, they offer readability certifications to developers. Developers with python certifications review python and developers with javascript certifications review javascript.
• Ownership: Clear code ownership makes it easier to find the right reviewer. At Google, the OWNERS file serves this role.
• Metrics: Count and celebrate the time spent reviewing. The metrics from both the Google study and Rigby and Bird are a great place to start.
• Time: We need to set aside time for reviews.
• Culture: As noted in the book Accelerate, John Shook, in recounting his experience introducing LEAN manufacturing at Toyota, says the best way to change culture is to focus on behavior. Enforcing good code review behavior makes code review culture better.

The research cited above studies massive, distributed software projects. Startups and independent developers may find these organizational practices don’t apply—and they’re probably right.