blog - Tyler Cipriani

To see posts by date, check out the archives

GitHub has always been a great code host. But GitHub’s code review system was an afterthought. Even now, Pull Requests still lag behind.

Oh yeah, there’s pull requests now

– GitHub blog, Sat, 23 Feb 2008

When GitHub launched, it had no code review.

Ten years later, when Microsoft acquired GitHub for $7.5 Billion, GitHub’s Pull Request model—“GitHub flow”—had become the default way to collaborate via Git.

But the Pull Request was never designed. It emerged. Though not from careful consideration of the needs of developers or maintainers.

GitHub swallowed software by making it easy to host code. Code review was an afterthought.

First-generation Pull Requests

Git has built-in pull requests—git request-pull. The Linux kernel has used them since 2005. But GitHub never used request-pull.

According to Linus Torvalds—Git’s creator—GitHub “decided to replace it with their own totally inferior version.”

When the Pull Request debuted in 2008 it worked like this:

Create a fork and click “Pull Request.”
Send a message to someone¹ with a link to your fork, asking them to merge it.

But while git request-pull generated a message template including a diff stat and changelog, GitHub hamstrung Pull Requests.

GitHub provided only a small, empty <textarea>—Pull Requests were little more than unstructured emails to other GitHub users.

And Pull Requests still lacked any way to see changes via the web.

“Code Review = Discussion + Code”?

It took two years for GitHub to show the git diff between two repos on GitHub.

In 2010, “cross repository compare view” coupled with an unthreaded comments section and became Pull Requests 2.0².

GitHub Pull Requests circa 2010. This is from the official documentation on GitHub.

Of course, the code and the comments were still on two different pages. It took another year before you could comment in the code.

Inline code comments

In 2011, rtomayko made the first inline comment on a change, writing, in full: “+1”.

Inline code review was far from a revelation. Guido van Rossum’s Mondrian—his 20% project at Google—had inline code comments by 2006. And there was an open-source version of Mondrian on the public web by 2008.

The Linux Kernel (using git format-patch) had code comments since 2005.

GitHub’s code review is still behind.

In 2008, GitHub’s developers created a new kind of code review.

But key features were missing. GitHub slowly tacked on these features:

2010: Compare view
2011: Ignore whitespace changes
2016: “Approve”, “Request changes”, and “Start a review”
2017: Required review
2019: Multiline comments
2023: Merge queues

Now, it’s 2024. And here’s a biased list of what I think is still missing:

Commit review – Ability to comment on the commit message.
Your turn – Like Gerrit’s attention sets – Microsoft recently did a study on a system called Nudge which was a similar idea, it yielded good results, reducing review completion time 60%.
User-defined review labels – “Approve”/“Request changes” is so limited—instead of using a complicated system of tags to mark changes ready for design approval, technical writing approval, style approval, and code approval—let repo owners figure out what approvals make sense for their project.
Hide bot comments – Allow me to hide bot comments so I can see the human comments.
Push to pull – Push to a special remote to create a pull request using my commit: git push origin <branch>:refs/pull-request/<target-branch>.
Review in notes – Annotate commits with metadata in a special git note refs/notes/review.
Stacked diffs – Just come on. You have infinite money.

And at this point I made Gerrit, but with more branches.

“Someone” was a person chosen by you from a checklist of the people who had also forked this repository at some point.↩︎
“Code Review = Discussion + Code.” was the headline of a blog post GitHub wrote circa 2010 introducing Pull Requests 2.0 ↩︎

GIT - the stupid content tracker

– Linus Torvalds, Initial revision of “git”, the information manager from hell

After years of code review with stacked diffs¹, I’ve been using GitLab merge requests at work.

Merge requests frustrated me until helpful folks pointed me toward GerritLab, a small Python tool for making stacked merge requests in GitLab—exactly what I was looking for.

But to talk to GitLab, GerritLab required a cleartext token in my ~/.gitconfig. I wanted to stow my token in a password vault, so I crafted a change for GerritLab that used gitcredentials(7).

Like most git features, git credentials are obscure, byzantine, and incredibly useful. It works like this:

import subprocess, json

INPUT = """\
protocol=https
host=example.com
username=thcipriani

"""

git_credentials_fill = subprocess.run(
    ["git", "credential", "fill"],
    input=INPUT,
    text=True,
    stdout=subprocess.PIPE,
)

git_credentials = {
    key: value for line in git_credentials_fill.stdout.splitlines()
    if '=' in line
    for key, value in [line.split('=', 1)]
}

print(json.dumps(git_credentials, indent=4))

Which looks like this when you run it:

$ ./example-git-creds.py
Password for 'https://thcipriani@example.com':
{
    "protocol": "https",
    "host": "example.com",
    "username": "thcipriani",
    "password": "hunter2"
}

The magic here is the shell command git credentials fill, which:

Accepts a protocol, username, and host on standard input.
Delegates to a “git credential helper” (git-credential-libsecret in my case). A credential helper is an executable that retrieves passwords from the OS or another program that provides secure storage.
My git credential helper checks for credentials matching https://thcipriani@example.com and finds none.
Since my credential helper comes up empty, git prompts me for my password.
Git sends <key>=<value>\n pairs to standard output for each of the keys protocol, host, username, and password.

To stow the password for later, I can use git credential approve.

subprocess.run(
    ["git", "credential", "approve"],
    input=git_credentials_fill.stdout,
    text=True
)

If I do that, the next time I run the script, git finds the password without prompting:

$ ./example-git-creds.py
{
    "protocol": "https",
    "host": "example.com",
    "username": "thcipriani",
    "password": "hunter2"
}

Git credential’s purpose

The problem git credentials solve is this:

With git over ssh, you use your keys.
With git over https, you type a password. Over and over and over.

Beleaguered git maintainers solved this dilemma with the credential storage system—git credentials.

With the right configuration, git will stop asking for your password when you push to an https remote.

Instead, git credentials retrieve and send auth info to remotes.

The maze of options

My mind initially refused to learn git credentials due to its twisty little maze of terms that all sound alike:

git credential fill: how you invoke a user’s configured git credential helper
git credential approve: how you save git credentials (if this is supported by the user’s git credential helper)
git credential.helper: the git config that points to a script that poops out usernames and passwords. These helper scripts are often named git-credential-<something>.
git-credential-cache: a specific, built-in git credential helper that caches credentials in memory for a while.
git-credential-store: STOP. DON’T TOUCH. This is a specific, built-in git credential helper that stores credentials in cleartext in your home directory. Whomp whomp.
git-credential-manager: a specific and confusingly named git credential helper from Microsoft®. If you’re on Linux or Mac, feel free to ignore it.

But once I mapped the terms, I only needed to pick a git credential helper.

Configuring good credential helpers

The built-in git-credential-store is a bad credential helper—it saves your passwords in cleartext in ~/.git-credentials.²

If you’re on a Mac, you’re in luck³—one command points git credentials to your keychain:

git config --global credential.helper osxkeychain

Third-party developers have contributed helpers for popular password stores:

1Password
pass: the standard Unix password manager
OAuth
Git’s documentation contains a list of credential-helpers, too

Meanwhile, Linux and Windows have standard options. Git’s source repo includes helpers for these options in the contrib directory.

On Linux, you can use libsecret. Here’s how I configured it on Debian:

sudo apt install libsecret-1-0 libsecret-1-dev
cd /usr/share/doc/git/contrib/credential/libsecret/
sudo make
sudo mv git-credential-libsecret /usr/local/bin/
git config --global credential.helper libsecret

On Windows, you can use the confusingly named git credential manager. I have no idea how to do this, and I refuse to learn.

Now, if you clone a repo over https, you can push over https without pain⁴. Plus, now you have a handy password library for shell scripts:

#!/usr/bin/env bash

input="\
protocol=https
host=example.com
user=thcipriani

"
eval "$(echo "$input" | git credential fill)"

echo "The password is: $password"

“stacked diffs” or “stacked pull-requests”—there’s no universal term.↩︎
git-credential-store is not a git credential helper of honor. No highly-esteemed passwords should be stored with it. This message is a warning about danger. The danger is still present, in your time, as it was in ours.↩︎
I think. I only have Linux computers to test this on, sorry ;_;↩︎
Or the config option pushInsteadOf, which is what I actually do.↩︎

Humans do no operate on hexadecimal symbols effectively […] there are exceptions.

– Dan Kaminsky

When SSH added ASCII art fingerprints (AKA, randomart), the author credited a talk by Dan Kaminsky.

As a refresher, randomart looks like this:

$ ssh-keygen -lv -f ~/.ssh/id_ed25519.pub
256 SHA256:XrvNnhQuG1ObprgdtPiqIGXUAsHT71SKh9/WAcAKoS0 thcipriani@foo.bar (ED25519)
+--[ED25519 256]--+
| .++ ...         |
| o+.... o        |
|E .oo=.o .       |
| . .+.=   .      |
|    o= .S.o.o    |
|   o  o.o+.= +   |
|  . .  .o B *    |
|   . .   + & .   |
|      ..+o*.=    |
+----[SHA256]-----+

Ben Cox describes the algorithm for generating random art on his blog. Here’s a slo-mo version of the algorithm in action:

ASCII art ssh fingerprints slo-mo algorithm

But in Dan’s talk, he never mentions anything about ASCII art.

Instead, his talk was about exploiting our brain’s hardware acceleration to make it easier for us to recognize SSH fingerprints.

The talk is worth watching, but I’ll attempt a summary.

What’s the problem?

We’ll never memorize SHA256:XrvNnhQuG1ObprgdtPiqIGXUAsHT71SKh9/WAcAKoS0—hexadecimal and base64 were built to encode large amounts of information rather than be easy to remember.

But that’s ok for SSH keys because there are different kinds of memory:

Rejection: I’ve never seen that before!
Recognition: I know it’s that one—not the other one.
Recollection: rote recall, like a phone number or address.

For SSH you’ll use recognition—do you recognize this key? Of course, SSH keys are still a problem because our working memory is too small to recognize such long strings of letters and numbers.

Hacks abound to shore up our paltry working memory—what Dan called “brain hardware acceleration.”

Randomart attempts to tap into our hardware acceleration for pattern recognition—the visiuo-spacial sketchpad, where we store pictures.

Dan’s idea tapped into a different aspect of hardware acceleration, one often cited by memory competition champions: chunking.

Memory chunking and sha256

The web service what3words maps every three cubic meters (3m²) on Earth to three words.

The White House’s Oval Office is ///curve.empty.buzz.

Three words encode the same information as latitude and longitude—38.89, -77.03—chunking the information to be small enough to fit in our working memory.

The mapping of locations to words uses a list of 40 thousand common English words, so each word encodes 15.29 bits of information—45.9 bits of information, identifying 64 trillion unique places.

Meanwhile sha256 is 256 bits of information: ~116 quindecillion unique combinations.

                                                                64000000000000 # 64 trillion (what3words)
115792089237316195423570985008687907853269984665640564039457584007913129639936 # 116 (ish) quindecillion (sha256)

For SHA256, we need more than three words or a dictionary larger than 40,000 words.

Dan’s insight was we can identify SSH fingerprints using pairs of human names—couples.

The math works like this¹:

131,072 first names: 17 bits per name (×2)
524,288 last names: 19 bits per name
2,048 cities: 11 bits per city
17+17+19+11 = 64 bits

With 64 bits per couple, you could uniquely identify 116 quindecillion items with four couples.

Turning this:

$ ssh foo.bar
The authenticity of host 'foo.bar' can't be established.
ED25519 key fingerprint is SHA256:XrvNnhQuG1ObprgdtPiqIGXUAsHT71SKh9/WAcAKoS0.
Are you sure you want to continue connecting
(yes/no/[fingerprint])?

Into this²:

$ ssh foo.bar
The authenticity of host 'foo.bar' can't be established.
SHA256:XrvNnhQuG1ObprgdtPiqIGXUAsHT71SKh9/WAcAKoS0
Key Data:
    Svasse and Tainen Jesudasson from Fort Wayne, Indiana, United States
    Illma and Sibeth Primack from Itārsi, Madhya Pradesh, India
    Maarja and Nisim Balyeat from Mukilteo, Washington, United States
    Hsu-Heng and Rasim Haozi from Manali, Tamil Nadu, India
Are you sure you want to continue connecting
(yes/no/[fingerprint])?

With enough exposure, building recognition for these names and places should be possible—at least more possible than memorizing host keys.

I’ve modified this from the original talk, in 2006 we were using md5 fingerprints of 160-bits. Now we’re using 256-bit fingerprints, so we needed to encode even more information, but the idea still works.↩︎
A (very) rough code implementation is on my github.↩︎

Remote companies have to work harder at everything.

The effort goes beyond “remote-friendly”—you need remote culture.

But once you have a remote culture, it’s hard to imagine going back. After nine years of working remotely, the only thing I miss about working in person is seeing people’s messy desks.

Why desks matter

Loneliness is a problem for remote workers—video chats are a terrible substitute for happy hour.

Plus, in person, you get to see people’s desks—it’s fun—it’s how you get to know people.

And I know other people think it’s fun, too: we remoties share our pictures of our workspaces all the time. Everyone should share their workspaces (here’s mine circa 2016).

My desk

This is my messy office as of today. (No cleaning and no judgments 🥹 allowed when sharing your workspace.)

Some things of note in this picture in no particular order:

The desk and stuff on it
- Ikea Markus chair and Trotten standing desk
- My laptop—Framework 13 AMD
- Vortex Race 3 Keyboard with Cherry MX clear switches
- Logitech ERGO rollerball mouse
- Ugmonk analog – the most straightforward productivity system I’ve found.
- Field notes expedition edition – “I’m not writing it down to remember it later, I’m writing it down to remember it now.”
- Endless recorder notebook – this notebook is slowly supplanting my Leuchtturm1917.
- Micron 08 pens
- Elgato wave mic arm with a cheapo Amazon Basics microphone. When you spend all day in meetings, it’s important to sound good.
- Nikon D610 with 24–70mm f/2.8 lens for video calls
  - The camera sits atop a Manfrotto 244 Variable Friction Magic Arm attached to the desk via Manfrotto super clamp – see how many magic arms you spot on the International Space Station (answer: bunches)
- The small red things in the foreground are USB data blockers—USB adapters with the data lines removed. I usually keep these in my travel bag.
- The white cube is an ODISTAR Desktop Vacuum Cleaner.
- dretec timer, for when I have the will to pomodoro.
- Post-Its®. So many Post-Its.
On the shelf
- Perplexus maze ball held in place by a third-hand kit for soldering
- My trusty ThinkPad X230
- Lots of old notebooks and thank you cards
- On the top shelf are some posters and essential reminders: “DON’T BECOME BITTER AND JADED” and “DON’T LET ADULTHOOD CORRUPT YOU.”
My overflowing homelab rack
- Startech 6U wall mount
- Mikrotik hEX RB750Gr3 5-port Ethernet Gigabit Router
- Ubiquiti edgeswitch 16
- Synology DS212—yes, from 2012(!), it’s still kicking thanks to 2×6TB Western Digital Red NAS HDDs.
- Gigabyte mini PC – 16GB RAM, 1TB NVMe, balancing precariously atop the Synology. The PC runs a handful of VMs: PiHole, HomeAssistant, Grafana, and Prometheus.
- 2×Raspberry Pi 3B+s doing not much of anything
- CyberPower CPS1215RMS Rackmount Surge Protector
- $6 clamp work light with a $60 hue bulb bouncing light off a Neewer 43 in. reflector to light my video calls
On the floor
- Home Depot homer bucket, storing my Weaver arborist throw weight and line which is crucial for my job as an engineering manager.
- A trash can with a lot of dryer sheets
- An old dog bed

None of these are affiliate links since no one would want to be affiliated with this mess.

2017 solar eclipse—obscuration 93.8% (view original)

In 2017, I opted to skip the crowds and the drive and settle for a 94% solar eclipse. I fully regret that decision.

Weather permitting, I’ll be photographing the full solar eclipse from the path of totality next Monday. While I’ve amassed a ton of gear, the main resource I’ve dumped into this project is time—time planning, practicing, and hacking.

After investing all that time, here’s my plan.

Why

I’m never going to produce an eclipse photo comparable to the work of Miloslav Druckmüller—so why bother with photography at all?

Photography is my hobby, and what’s a hobby without a challenge? Sure, the siren song of cool gear is part of it—I do love gear—but it also takes planning, hacking, and editing skills to create a great picture.

I got to spend time rooting around inside libgphoto2, breaking out the soldering iron to jury-rig a custom ESP32-based release cable, and practicing every move I’ll make on eclipse day. For me, this is fun.

Your mileage may vary.

Gear

Here is my gear checklist for this year:

Imaging:

The lens and teleconverter give me an effective focal length of 560mm. The 1.4µm pixel pitch of the A7rIII means I’ll cover 1.7 arc-seconds per pixel at my focal length—right in the sweet spot (below two, above one).

Here’s what a cropped image of the solar disk looks like on this setup:

You can compare this photo to an image taken yesterday by the Solar & Heliospheric Observatory (SOHO) spacecraft—both show sunspots AR3617 and AR3619. I’m gratified that a picture from my backyard shows the same details as one from outer space.

Other essentials

M5StickC ESP32-PICO – I’m using this as an intervalometer. I’ve programmed it with the Alpha Fairy firmware and will control it using pyserial. I soldered together a release cable that I’ll use to control the shutter.
iOptron SkyGuider Pro – This is an tracking mount. It cancels out the motion of the Earth so the sun appears in the same spot in the frame (if I polar align right).
Thousand Oaks 77mm threaded RF-film solar filter – for use pre-totality. I bought this for the 2017 eclipse. This prevents the sun from destroying my lens and camera.
Manfrotto XUME 77mm lens adapter – secures the solar filter to my lens via a magnet, letting me pull it off instantly rather than fiddling with unscrewing a filter.

Software

Planning – PhotoPills helped me plan where to be and where to look.
Polar alignment – I’ll use SkEye for daytime polar alignment, pointing my tracking mount downwards towards σ-Octantis, which will align my tracker with the motion of the Earth.
Weather/Clouds – VentuSky shows real-time GOES satellite imagery and lets you browse weather prediction models. SkippySky shows real-time total cloud cover, seeing, and transparency (and also shows the eclipse path). Clear Dark Sky is the classic astronomer’s forecast tool.

My plan

I’m pegging all my images to f/8, ISO 100, and only adjusting shutter speed.

I’ll be bracketing five photos, each three stops of light away from each other (±3EV). I’ll adjust my base shutter speed twice.

Pre-totality, I’ll do 1/100 second shutter speed. This should cover me for full-disk images, right up through Baily’s beads:

1/100”
1/800”
1/13”
1/6400”
0.6”

After totality, I’ll move the shutter speed to 1/15 of a second. Fast enough to ensure I get at least one clear shot and slow enough to get earthshine if all goes to plan with my polar alignment:

1/15”
1/125”
1/2”
1/1000”
4”

If it all falls apart? Well. I’ll be in Austin—a great place to while away the time longing for clear skies.