🛟 Simple history via prompt command

To save my history, I exploit the bash PROMPT_COMMAND variable.

Bash will execute anything you assign to PROMPT_COMMAND before showing your prompt:

$ export PROMPT_COMMAND='fortune && echo 🥳'
Q:  Do you know what the death rate around here is?
A:  One per person.
🥳
$

You can use this variable to write the output of history 1 to a file:

$ export PROMPT_COMMAND='history 1 >> ~/.muh-history.test'
$ fortune
Caution: breathing may be hazardous to your health.
$ cat ~/.muh-history.test
6851  2024-02-25 18:44:30-0700 export PROMPT_COMMAND='history 1 >> ~/.test-history'
6852  2024-02-25 18:44:35-0700 fortune

I started doing this in 2015¹, but the more metadata I added—bash process ID, current directory, user—the more powerful my history became.

Today, my history file looks like this:

$ tail -1 ~/.muh-history
847008 /home/thcipriani thcipriani 2024-02-27T15:52:16-0700 echo 'Weeee!' | figlet | lolcat

Now, I can crawl these bits of metadata to look at my history in interesting ways. For example:

• Trace the history of a single shell session.
• Show commands within a time range.
• Search for commands run in the current directory.

🧐 Problems and tradeoffs of ~/.muh_history

My eternal shell history has been working well for eight years. But every solution has tradeoffs and problems:

• One machine – This works on one machine. If you need history saved across more than one machine, Atuin seems like what you want (though I’ve never tried it).
• Local security – This stores any command you run, including those with woopsied passwords. The HISTCONTROL=ignorespace setting helps—it makes history ignore commands that start with a space.
• Bash only – I use the default Linux shell for most operating systems: bash. Nicer shells can do the same things as PROMPT_COMMAND and history (e.g., zsh’s preexec, fish’s fish_preexec).
• Too simplistic – McFly and Atuin seem like robust, active alternatives. And both use PROMPT_COMMAND under the hood. Maybe I’d start there if I were starting today, but ~/.muh_history pre-dates both projects.
• Remote security – It would be Bad™ to install this on a remote machine unless you’re the admin of that machine. Never cross a sysadmin.

⚠️ Flailing at bash’s built-in history

Goals of ~/.muh_history:

1. Eternal – I want the option of keeping history forever.
2. History builtin commands – “↑” and ctrl-r should work in the default way.
3. History builtin depth – I want a few weeks of history available to ctrl-r.
4. Instant startup – Minimize lag when starting new bash sessions.
5. No logrotate – Avoid logrotate, systemd timers, and cronjobs to manage history—nothing to troubleshoot or maintain.

And the hacks I’ve seen to juice bash’s built-in ~/.bash_history fail at least one of these:

• HIST(FILE)SIZE=<huge number>² – On startup, bash loads a HISTSIZE amount of HISTFILE into memory—this could cause lag during bash startup unless you logrotate regularly. And there’s a risk of losing commands if/when your sessions crash.
• unset HIST(FILE)SIZE/HIS(FILE)SIZE=-1 – This should make HIST(FILE)SIZE infinite, so the same caveats apply as using a huge number. Plus, this has a history of failing in some instances.
• HISTFILE=~/.history/$(date -I) – For the first bash session you start in the morning ctrl-r will give you nothing.
• PROMPT_COMMAND='history -a && history -r' – Before showing your prompt, write your in-memory history to HISTFILE and re-read it into memory. This mixes up the history of different sessions, so hitting “↑” may show history from a different session.

Other pitfalls:

1. Write on exit – when you exit a session, bash dumps HISTSIZE lines of your command history into ~/.bash_history. If your session crashes: all gone. This makes HIST(FILE)SIZE unappealing to me.
2. Append vs. overwrite – if your shell initialization files skip setting histappend, bash will overwrite ~/.bash_history rather than append your session history on exit.
3. Small defaults – bash stores your 500 most recent commands in your session history and 500 commands from old sessions in ~/.bash_history.

I’ve set some sensible defaults and ignored clever ideas. Here are my settings:

# append to history vs. overwrite
shopt -s histappend
# ignore commands starting with space; ignore duplicates
HISTCONTROL=ignoredups:ignorespace
# Up the history in memory: 500 → 10,000
HISTSIZE=10000
# Up the history on disk: 500 → 20,000
HISTFILESIZE=20000
# RFC 3339 format; e.g., 2024-02-27T15:52:16-0700
HISTTIMEFORMAT='%FT%T%z '