To see posts by date, check out the archives

Eternal shell history 🐢
Tyler Cipriani Posted
Tar, XKCD 1168 by Randall Monroe Licensed: CC-by-NC 2.5

I’ve never been able to trust ~/.bash_history. Regardless of my configuration, I was forever searching for that one gnarly jq filter that had somehow disappeared. Then, I found a better way.

Over the past eight years, I’ve hoarded ¾ million lines of bash history:

$ wc -l < ~/.muh_history
763075

This accounts for every shell command I’ve run since 2016—all saved to a 102MB file: ~/.muh_history.

$ ls -lh ~/.muh_history
-rw------- 1 thcipriani thcipriani 102M Feb 27 21:23 /home/thcipriani/.muh_history

It’s a perfect, searchable archive of my work.

hist a command to search ~/.muh_history

I’ve come to rely on ~/.muh_history to take notes for me while I focus on solving problems. I trust this system, because it’s so simple.

🛟 Simple history via prompt command

To save my history, I exploit the bash PROMPT_COMMAND variable.

Bash will execute anything you assign to PROMPT_COMMAND before showing your prompt:

$ export PROMPT_COMMAND='fortune && echo 🥳'
Q:  Do you know what the death rate around here is?
A:  One per person.
🥳
$

You can use this variable to write the output of history 1 to a file:

$ export PROMPT_COMMAND='history 1 >> ~/.muh-history.test'
$ fortune
Caution: breathing may be hazardous to your health.
$ cat ~/.muh-history.test
6851  2024-02-25 18:44:30-0700 export PROMPT_COMMAND='history 1 >> ~/.test-history'
6852  2024-02-25 18:44:35-0700 fortune

I started doing this in 20151, but the more metadata I added—bash process ID, current directory, user—the more powerful my history became.

Today, my history file looks like this:

$ tail -1 ~/.muh-history
847008 /home/thcipriani thcipriani 2024-02-27T15:52:16-0700 echo 'Weeee!' | figlet | lolcat

Now, I can crawl these bits of metadata to look at my history in interesting ways. For example:

  • Trace the history of a single shell session.
  • Show commands within a time range.
  • Search for commands run in the current directory.

🧐 Problems and tradeoffs of ~/.muh_history

My eternal shell history has been working well for eight years. But every solution has tradeoffs and problems:

  • One machine – This works on one machine. If you need history saved across more than one machine, Atuin seems like what you want (though I’ve never tried it).
  • Local security – This stores any command you run, including those with woopsied passwords. The HISTCONTROL=ignorespace setting helps—it makes history ignore commands that start with a space.
  • Bash only – I use the default Linux shell for most operating systems: bash. Nicer shells can do the same things as PROMPT_COMMAND and history (e.g., zsh’s preexec, fish’s fish_preexec).
  • Too simplisticMcFly and Atuin seem like robust, active alternatives. And both use PROMPT_COMMAND under the hood. Maybe I’d start there if I were starting today, but ~/.muh_history pre-dates both projects.
  • Remote security – It would be Bad™ to install this on a remote machine unless you’re the admin of that machine. Never cross a sysadmin.

⚠️ Flailing at bash’s built-in history

@michaelhoffman.bsky.social, Wed, 02 Sep 2015

Goals of ~/.muh_history:

  1. Eternal – I want the option of keeping history forever.
  2. History builtin commands – “↑” and ctrl-r should work in the default way.
  3. History builtin depth – I want a few weeks of history available to ctrl-r.
  4. Instant startup – Minimize lag when starting new bash sessions.
  5. No logrotate – Avoid logrotate, systemd timers, and cronjobs to manage history—nothing to troubleshoot or maintain.

And the hacks I’ve seen to juice bash’s built-in ~/.bash_history fail at least one of these:

  • HIST(FILE)SIZE=<huge number>2 – On startup, bash loads a HISTSIZE amount of HISTFILE into memory—this could cause lag during bash startup unless you logrotate regularly. And there’s a risk of losing commands if/when your sessions crash.
  • unset HIST(FILE)SIZE/HIS(FILE)SIZE=-1 – This should make HIST(FILE)SIZE infinite, so the same caveats apply as using a huge number. Plus, this has a history of failing in some instances.
  • HISTFILE=~/.history/$(date -I) – For the first bash session you start in the morning ctrl-r will give you nothing.
  • PROMPT_COMMAND='history -a && history -r' – Before showing your prompt, write your in-memory history to HISTFILE and re-read it into memory. This mixes up the history of different sessions, so hitting “↑” may show history from a different session.

Other pitfalls:

  1. Write on exit – when you exit a session, bash dumps HISTSIZE lines of your command history into ~/.bash_history. If your session crashes: all gone. This makes HIST(FILE)SIZE unappealing to me.
  2. Append vs. overwrite – if your shell initialization files skip setting histappend, bash will overwrite ~/.bash_history rather than append your session history on exit.
  3. Small defaults – bash stores your 500 most recent commands in your session history and 500 commands from old sessions in ~/.bash_history.

I’ve set some sensible defaults and ignored clever ideas. Here are my settings:

# append to history vs. overwrite
shopt -s histappend
# ignore commands starting with space; ignore duplicates
HISTCONTROL=ignoredups:ignorespace
# Up the history in memory: 500 → 10,000
HISTSIZE=10000
# Up the history on disk: 500 → 20,000
HISTFILESIZE=20000
# RFC 3339 format; e.g., 2024-02-27T15:52:16-0700
HISTTIMEFORMAT='%FT%T%z '

  1. When I read the wonderful, shorter version of this article: Andy Teijelo Pérez’s Bash eternal history↩︎

  2. I scraped of GitHub for HISTSIZE=. It’s all over the place. Max: 10,000,000,000,000,000; 25 percentile: 1,000; 75th percentile: 100,000; median: 10,000. I set mine at the median. Works fine.↩︎

Framework DIY 13 AMD review ⚙️
Tyler Cipriani Posted
Framework Laptop DIY 13 AMD with ethernet dongle and screwdriver

The laptop industry is a tragedy.

Meanwhile, Framework built something different—repairable, Linux-ready laptops that respect users. Framework is a rare company worth your support.1

So, this month, I bought their 13.5″, AMD-powered laptop—the Framework 13 AMD.

But I’ve seen mixed reviews from other Linux users. I like the Framework ethos—I hope I like their laptops, too.

🛠️ DIY Hardware and assembly

Framework offers two editions of its laptops:

  1. Pre-built – Fully assembled, complete with a useless (to me) Windows™ install.
  2. DIY – Do it yourself (DIY). Some assembly required—BYO-OS.

I opted for the DIY edition—a misnomer, given assembly took five minutes.

Framework DIY—little assembly required (NVMe, RAM, keyboard, bezel: that’s it)
  • CPU 8-core/16-thread 5.1Gz AMD (AMD Ryzen™ 7 7840U)
    • AMD Radeon 780M integrated GPU (works fine with amdgpu driver)
    • Ryzen AI Neural Processing Unit (NPU) (AMD released an xdna driver last week. I have yet to try it.)
  • 64 GB RAM – DDR5-5600
  • 2TB NVMe
  • 13.5” (diagonal) matte (🥳) screen

This is a powerful machine.

🏋️ Weight

Framework 13 weighing in at 1,323Kg, about the same as the 2011 Macbook Air

A notebook that weighs more than a kilo is simply not a good thing

Linus Torvalds

The Framework weighs more than a kilo.

Fully assembled (stickers and all), my new laptop tips the scales at 1,323g.

It’s 100g lighter than my x220 but 100g heavier than my partner’s M2 Macbook Air.

The Framework weighs as much as the 2011 Macbook Air—a sure sign innovation has stopped in this space.

🔌 Ports/dongles

Ports on the ThinkPad vs Framework—sadly absent: PCMCIA card slot

I’m torn.

I can arrange my laptop’s USB, power, and ethernet ports however I want them.

And folks in the Framework community are cooking up new ideas.

But these are dongles. Brilliant dongles, but dongles nonetheless—I have to tote them on my travels and keep track of them all.

A dongle by any other name…

Now, I need a little pouch for my adorable dongles.

🪫 Battery life

Folks flagged short battery life as a problem for these machines—especially the Intel version. Is that true for the AMD version?

To test this, I simulated some strenuous web surfing—clicking Wikipedia links faster than is humanly possible.2

Results:

Time to battery empty Brightness Delay between page clicks Avg CPU Percentage Avg Watts
02:20:21 100% 0s 13.8% 23.4
02:55:07 0% 0s 13.8% 19.8
12:57:57 😅 0% 10s 1.1% 4.1

As long as I’m not slamming through every page of Wikipedia, the battery would get me through most work days.

During the workday, I use between 5 and 10 watts.3 While that might not give me 13 hours, it beats my ThinkPad X220’s 1.5-hour battery life.

Linux setup

Linux veterans relayed painful experiences running their OS on older Framework models.

But my experience was (mostly) jank-free.

Ubuntu 22.04

I installed Ubuntu first, since it’s the sole Linux distribution Framework’s website listed as “Stable.”

And Ubuntu 22.04 ran flawlessly.

Chalk this up to the detailed Framework Ubuntu setup guide, with its giant gob of copy-pasta commands—much laudable, painstaking effort has gone into making this experience perfect.

Debian Bookworm

I perused the Debian Wiki’s Framework pages and the Debian Install Guide as references to install Debian Bookworm.

Audio, wifi, bluetooth, touchpad, webcam, and every button worked out of the box.

Then I closed the lid, but nothing happened. Sleep failed.

Problems with s2idle on AMD machines are common. Problems are so common that Freedesktop cobbled together a script with cute emojis to help troubleshoot: amd_s2idle.py.

Framework user forums pointed me to the firmware-amd-graphics Debian package bug 1053856.

After firmware fiddling and an hour+ tweaking Xmonad for the high-dpi (2256x1504) display: all’s well.

I hate computers, but this one is pretty good.

@FramworkPuter, 2024-01-19

What I like:

  • Repairable – I hoard a closet of old ThinkPads because I know they’ll end up at the dump otherwise.
  • Hardware camera/mic switch + RFKill – Hardware switches beat camera covers any day. And a laptop that respects its users’ privacy is lovely.
  • Reference designs – While it’s not open hardware, Framework releases reference designs under a Creative Commons license.
  • Matte screen – Why are shiny screens an option? Who wants that?

What I dislike:

  • Keyboard – It’s mushy. Plus, the button under [/?] is [←], which is breaking my brain. I’m used to it being right CTRL (which I use as AltGr).
  • Brightness – Even at 0% brightness, the screen is too bright. There’s probably something I can do here.
  • HDMI requires back slots – HDMI expansion card plugged into the front left expansion slot failed. Moving to one of the back slots works.
  • 3:2 aspect ratio – Why? It’s an outre choice. I’m having a bad time mirroring to 16:9 displays. Plus, horizontal screen space is great for tiling window managers.
  • Trackpad – I still like buttons. The trackpad is good, but I’m a luddite—ThinkPads spoil you.
  • Keyboard backlight – Speaking of ThinkPads, why have we abandoned the ThinkLight?

🏛️ Verdict

In a barren industry where planned obsolescence is the norm, Framework produces nice hardware for a fair price.

The Framework AMD 13 is a powerful, modern laptop capable of running Linux. And all the buttons seem to do what they’re supposed to.

I look forward to the day when I can Ship-of-Theseus the guts of this beast to get an even beefier boxen. It sure beats throwing it on the pile of ThinkPads collecting dust in my closet.


EDIT 2024-01-31T13:53:30-07:00: Before, this article referred to the Framework AMD 13 as the “13th generation.” Commentors pointed out that that was incorrect. The 13th generation Framework laptops refer to the 13th generation of the Intel CPUs, not the Framework hardware.


  1. Man, I hope this comment ages well.↩︎

  2. I scripted the “Getting to Philosophy” Wikipedia game for the top 400 Wikipedia Articles of 2023↩︎

  3. Unless I do something silly like attend a zoom meeting.↩︎

Treats from git's contrib tools🍭
Tyler Cipriani Posted

The intention is to keep interesting tools around git here, maybe even experimental ones

Junio C Hamano, git/contrib/README

Git’s source repo includes a “contrib” directory containing tools that extend git.

But these tools are hidden from most users. And they require extra steps to install. So they’re less well-known than they should be.

These are the tools I’ve found useful.

diff-highlight

diff-highlight makes git diff easier to read, making subtle changes stand out.

Standard git diff (top) vs. diff-highlight (bottom)

Git diff-highlight’s author described it as “a simple and stupid script for highlighting differing parts of lines in a unified diff.”

Try it out with:

git log -p --color | /usr/share/doc/git/contrib/diff-highlight/diff-highlight

If you like what you see, make it your pager with this oneliner:

git config --global core.pager '/usr/share/doc/git/contrib/diff-highlight/diff-highlight | less'

git-prompt.sh

git-prompt.sh makes information from git status accessible at a glance.

For the basics, it’s easy1:

  1. Source git-prompt.sh from your shell init file.
  2. Add the magic incantation somewhere in your prompt (e.g., PS1='\u@\h $(__git_ps1 " (%s)")\$ ').
  3. Tweak with environment variables until you get a prompt that works for you.

Here’s my config:

GIT_PS1_SHOWUNTRACKEDFILES=1
GIT_PS1_SHOWDIRTYSTATE=1
GIT_PS1_SHOWUPSTREAM="auto verbose"
. ~/bin/git-prompt.sh
PS1='\u@\h $(__git_ps1 " (%s)")\$ '

Inside a repo on main in a clean worktree, this shows:

me@💻 (main|u=)$

If you make a big mess, the prompt makes it impossible to miss:

me@💻  (main *+%|u+1-1)$
         │   │││ │ │ │
         │   │││ │ │ └── count: changes behind upstream (-1)
         │   │││ │ └──── count: changes ahead of upstream (+1)
         │   │││ └────── indicator: upstream set (u)
         │   ││└──────── indicator: untracked file (%)
         │   │└───────── indicator: staged, uncommitted change indicator (+)
         │   └────────── indicator: unstaged change (*)
         └────────────── info: current branch name

A git-aware shell prompt solves a multitude of woes. My prompt has saved me from countless sticky git situations.

subtree

git submodule allows you to include other git repos within your repo. But anyone who’s used it can tell you: it gets confusing fast.

git subtree eases git submodule pain.

Both submodule and subtree let you add a library as a subfolder of your project.

But subtree integrates the history of the library into your main project. And you can get it back out as a standalone library later.

This lets you branch and tag a project along with its libraries—a critical limitation to git submodule.

For example:

git clone https://github.com/thcipriani/my-parent && cd my-parent
git subtree add --prefix=mylib https://github.com/thcipriani/my-library

Adds my-library to the my-parent project in a folder called mylib—the two projects’ history gets merged. From there, it works like a monorepo.

After I’ve made a series of changes inside my-parent, including changes to mylib, I can extract the history of mylib with:

git subtree split --prefix=mylib
0b64183b7a0a27ad1f466d5cac61cbfefd1e598e
git push https://github.com/thcipriani/my-library 0b64183b7a0a27ad1f466d5cac61cbfefd1e598e:master

Or, in one step:

git subtree push --prefix=mylib https://github.com/thcipriani/my-library main

More

  • git worktree: is an example of a contrib script that’s made the jump to a git built-in command. Maybe, in time, other commands in this list will, too.
  • git-jump: opens your editor to interesting bits of code. For example, git jump diff opens vim to the first diff hunk.
  • mw-to-git: lets you read and edit MediaWiki wikis (like Wikipedia) as if they were a local git repo (cf: maintaining userscripts with git)
  • pre-auto-gc-battery: a hook that prevents git’s automatic garbage collection if you’re on a laptop using battery power

And there are even more weird gems to unearth. Take a look!


  1. The script contains detail usage instuctions, too↩︎

Racy git magic
Tyler Cipriani Posted

Exploting a long-standing git bug for my own amusement.

And I think there is one known race: the index mtime itself is not race-free.

– Linus Torvalds, Re:git bugs, 2008

A well-known race condition skulks through git’s plumbing.

And I can demo it via a git magic trick 🪄1

$ tree -L 1 -a .
.
├── file
└── .git

$ cat file
okbye
$ git status
On branch main
Changes to be committed:
        new file:   file

$ git ls-files --modified  # No output. A clean working directory.
$ git commit --message="The file sez $(cat file)"
$ git log --oneline HEAD -1
600fcac (HEAD -> main) The file sez okbye
$ git status
On branch main
nothing to commit, working tree clean

Nothing up my sleeves:

  • The git repo had one staged file, file, containing okbye—nothing else
  • I committed it
  • And the commit message is The file sez okbye

Now, the big reveal:

$ cat file
okbye
$ git show HEAD:file
hello
$ git status
nothing to commit, working tree clean

Boom. ✨Magic✨

Git is clueless that the wrong file is in your work tree.

Even git restore has no effect:

$ git restore file
$ cat file
okbye

Git maintainers know this sleight-of-hand well—dubbing it the “Racy Git Problem” circa 2006.

But it can still generate heisenbugs in unexpected places.

What is the racy git problem?

The two biggest problems in computer science are:

  1. cache invalidation,
  2. naming things, and
  3. off-by-one

Racy git is a cache invalidation problem.

Git speeds operations by stowing two bits of data about each file in your work tree:

  1. The file size
  2. The last time the file was modified—its mtime

So, if you tweak a file, without changing the mtime or the size—how would git know?

Before 2006, git was oblivious.

Now, it’s wised up—if the file looks unchanged (via size and mtime), then git performs another check. If file mtime >= the mtime of the index file (.git/index), then git rebuilds the index.

Thus, the core of my lame magic trick:

#!/usr/bin/env bash
echo hello > file

# Stage the file for commit
git update-index --add file

# Send .git/index's mtime INTO THE FUTURE!!!1!
touch --date='1 second' .git/index

# Now modify "file" behind git's back. So. Sneaky.
echo okbye > file

# CAVEAT OF DOOoooM: this has to happen within a single second to work---yeah. git's good. :D

How this happens in the real world

This problem can still happen in the real world.

I stumbled onto this while working with git fat (a forerunner of GitHub’s “Git Large File Storage”—git-lfs) in 2017.

At that time, we had a tool that pulled git code onto hundreds of servers. And every so often, a server would fail to fetch large files.

The basic steps were:

  • ssh into 100s of servers in parallel
  • git clone <repo>
  • git fat pull

git fat pull rsync’d large binary files, mostly jars, into the working directory.

Frustratingly, when this failed, I could jump on the server and rerun git fat pull, which worked every time.

This was the racy git problem in disguise.

See, git fat uses git filters (just like git-lfs). Filters are scripts that git runs automagically at checkout or commit time.

But git is smart—it only triggers filters when it has to—when your working directory differs from its index.

So, when we ran git clone within the same second as git fat pull, the files on disk were the same as in the index—so, git never triggered the filter.

Why did it work when I ran it manually? Because git fat was smart, too—it touch’d files on disk, which invalidated the index which caused git to run the filter command.

And so the racy git problem persists. It plagues every system that relies on the git filters2 to. this. day.


  1. This will only seem like magic if you can distiguish normal git operations from magic. For nerds only.↩︎

  2. This means almost all large file storage mechanisms in git. git annex’s “indirect mode” eschewed git’s smudge and clean filters. But I see both “direct” and “indirect” mode are now deprecated. I’m unclear how this works today ¯\_(ツ)_/¯.↩︎

Daylight confusion week
Tyler Cipriani Posted
Days since last timezone issue: -1 (via @accousticmirror)

It’s Daylight Confusion Week.

So, my meetings with Europeans shift by an hour—either forward or backward, depending on who made the meeting.

Yes, this is hard to think about. Here’s a chart:

23 Oct 30 Oct 😵 06 Nov
San Francisco, CA 📌 08:00 08:00 08:00
Paris, France 17:00 16:00 (-1hr) 17:00

See the problem?

  1. 🇪🇺 ← -1:00: Meetings for most of Europe are an hour earlier if the meeting is pinned to the US.
  2. 🇺🇸 → +1:00: Meetings for most of North America are an hour later if the meeting is pinned to Europe.

Next week, it’ll all be back to normal, at least until March 2024.

As a software engineer, my first instinct is to ask how we can fix this. But that’s the best part: we can’t.

Why this happens

tl;dr: daylight savings complicates timezone math.

In much of Europe, Daylight Savings Time ends today.

But North America will neglect their clocks for another week.

So, this week, Central Europe falls back one hour—making them one hour closer to North American timezones than usual. This happens two weeks a year—once in March and now.

But between Central Europe, North America, and places in the Southern Hemisphere—we’re sometimes 2-hours out of sync.

And for places without daylight savings time, this one-hour shift sticks around for half a year then shifts back.

It’s a mess.

OK, so pin meetings to UTC

Coordinated Universal Time (UTC) is lovely. But it won’t fix for Daylight Confusion Time.

We coordinate meetings using UTC. As in: “Hey, wanna meet Mondays at 15:00 UTC?”

But we book meetings in our local time zones.

If we used UTC instead, Daylight Confusion Week would expand to the entirety of Daylight Savings Time. Meetings would shift an hour in March and roll back in November.

There’s no “base timezone” that keeps the weekly meeting at the same local time for everyone all year long.

23 Oct 30 Oct 😵 06 Nov
UTC 15:00 15:00 16:00 (+1hr)
San Francisco, CA 📌 08:00 08:00 08:00
Paris, France 17:00 16:00 (-1hr) 17:00

No, really, there’s got to be a way to fix this

In order of preference, this is how I think we could fix this:

  • Stop having recurring meetings
  • Ban daylight savings everywhere forever
  • Coordinate our daylight savings times
  • Continue to suffer for a couple weeks a year

So far, we’ve made our choice. We’ve opted to suffer.