{
    "version": "https://jsonfeed.org/version/1",
    "title": "Tyler Cipriani: blog",
    "home_page_url": "https://tylercipriani.com/blog/",
    "feed_url": "https://tylercipriani.com/blog/index.json",
    "items": [
        {

    "id": "https://tylercipriani.com/blog/2026/04/24/on-the-software-supply-chain-doom-spiral/",

    "title": "GitHub Actions and consequences",
    "url": "https://tylercipriani.com/blog/2026/04/24/on-the-software-supply-chain-doom-spiral/",

    "author": {
        "name": "Tyler Cipriani"
    },


    "tags": [

     "computing"

    ],

    "date_published": "2026-04-24T20:54:04Z",
    "date_modified": "2026-04-26T19:29:35Z",


    "content_html": "<style>.title {text-wrap:balance;} #content > p:first-child {text-wrap:balance;}</style>\n<p>Hackers are pwning packages at an exhausting clip.</p>\n<p>In late February, a hackerbot AI<a href=\"https://tylercipriani.com/blog/#fn1\" class=\"footnote-ref\"\nid=\"fnref1\" role=\"doc-noteref\"><sup>1</sup></a> yoinked the release key\nfor a single project. Within a month, fifty-ish other projects had cred\nstealers. Each infected repo swiped credentials for the next.</p>\n<p>This spate of supply-chain hacks started from a well-known GitHub\nActions trap. A trap that AI can exploit or push us into.</p>\n<section id=\"github-actions-are-a-trap\" class=\"level2\">\n<h2>GitHub Actions are a trap</h2>\n<p>Trivy is an open-source security scanner. But if you used Trivy in\nlate March, you had a bad time.</p>\n<p>On March 19th, hackers pushed a version of Trivy that tried to\nsmuggle secrets from anywhere it ran. Trivy cited a “misconfiguration”\nin their continuous integration (CI) system, GitHub Actions.</p>\n<p>But the exploit was less a misconfiguration and more a GitHub Actions\ntrap.</p>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/thumbs/6f/f8ccb8fabb7987a2a2f8ae17e45d4f/large.jpg\"\nalt=\"Admiral Ackbar warning about the trap in GitHub Actions\" />\n<figcaption aria-hidden=\"true\">Admiral Ackbar warning about the trap in\nGitHub Actions</figcaption>\n</figure>\n<p>Here’s a simplified version of how Trivy got pwnd<a href=\"https://tylercipriani.com/blog/#fn2\"\nclass=\"footnote-ref\" id=\"fnref2\"\nrole=\"doc-noteref\"><sup>2</sup></a>:</p>\n<div class=\"sourceCode\" id=\"cb1\"><pre\nclass=\"sourceCode yaml\"><code class=\"sourceCode yaml\"><span id=\"cb1-1\"><a href=\"https://tylercipriani.com/blog/#cb1-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"co\"># INSECURE. DO NOT USE.</span></span>\n<span id=\"cb1-2\"><a href=\"https://tylercipriani.com/blog/#cb1-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"fu\">on</span><span class=\"kw\">:</span></span>\n<span id=\"cb1-3\"><a href=\"https://tylercipriani.com/blog/#cb1-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">  pull_request_target</span></span>\n<span id=\"cb1-4\"><a href=\"https://tylercipriani.com/blog/#cb1-4\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb1-5\"><a href=\"https://tylercipriani.com/blog/#cb1-5\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"fu\">jobs</span><span class=\"kw\">:</span></span>\n<span id=\"cb1-6\"><a href=\"https://tylercipriani.com/blog/#cb1-6\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">  </span><span class=\"fu\">check</span><span class=\"kw\">:</span></span>\n<span id=\"cb1-7\"><a href=\"https://tylercipriani.com/blog/#cb1-7\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">    </span><span class=\"fu\">steps</span><span class=\"kw\">:</span></span>\n<span id=\"cb1-8\"><a href=\"https://tylercipriani.com/blog/#cb1-8\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">      </span><span class=\"kw\">-</span><span class=\"at\"> </span><span class=\"fu\">uses</span><span class=\"kw\">:</span><span class=\"at\"> action/checkout@deadbeefdeadbeefdeadbeefdeadbeefdeadbeef</span></span>\n<span id=\"cb1-9\"><a href=\"https://tylercipriani.com/blog/#cb1-9\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">        </span><span class=\"fu\">with</span><span class=\"kw\">:</span></span>\n<span id=\"cb1-10\"><a href=\"https://tylercipriani.com/blog/#cb1-10\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">          </span><span class=\"fu\">ref</span><span class=\"kw\">:</span><span class=\"at\"> refs/pull/${{ github.event.pull_request.number }}/merge</span></span>\n<span id=\"cb1-11\"><a href=\"https://tylercipriani.com/blog/#cb1-11\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">      </span><span class=\"kw\">-</span><span class=\"at\"> </span><span class=\"fu\">uses</span><span class=\"kw\">:</span><span class=\"at\"> ./.github/actions/setup-go</span></span>\n<span id=\"cb1-12\"><a href=\"https://tylercipriani.com/blog/#cb1-12\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"at\">      </span><span class=\"kw\">-</span><span class=\"at\"> </span><span class=\"fu\">uses</span><span class=\"kw\">:</span><span class=\"at\"> some/go-static-analysis@c0ffeec0ffeec0ffeec0ffeec0ffeec0ffeec0ff</span></span></code></pre></div>\n<p>At first glance, this code looks fine:</p>\n<ul>\n<li>No secrets referenced.</li>\n<li>Third-party actions pinned to an immutable hash.</li>\n<li>Check out a pull request. Perform some static analysis.</li>\n</ul>\n<p>But this code is a verbatim antipattern from a 2021 GitHub blog post\ntitled “<a\nhref=\"https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/\">preventing\npwn requests</a>”:</p>\n<blockquote>\n<p>if the <code>pull_request_target</code> workflow only […] runs\nuntrusted code but doesn’t reference any secrets, is it still\nvulnerable?</p>\n<p>Yes it is</p>\n<p>– <a\nhref=\"https://securitylab.github.com/resources/github-actions-preventing-pwn-requests/\">GitHub\nSecurity Lab</a></p>\n</blockquote>\n<p>The problem is <code>pull_request_target</code>:</p>\n<ul>\n<li><code>pull_request_target</code> – plunks a nice, juicy\n<code>GITHUB_TOKEN</code> into the environment.</li>\n<li><code>actions/checkout</code> – takes an optional parameter\n<code>persist-credentials</code>, which removes secrets if set to\n<code>false</code>. But the default for the parameter is\n<code>true</code>.</li>\n</ul>\n<p>Setting the <code>persist-credentials</code> parameter to\n<code>false</code> has been an open issue in GitHub Actions <a\nhref=\"https://github.com/actions/checkout/issues/485\">since\n2021</a>.</p>\n</section>\n<section id=\"your-home-is-a-crime-scene\" class=\"level2\">\n<h2>Your <code>$HOME</code> is a crime scene</h2>\n<p>Once hackers had Trivy’s keys, they published a new version of Trivy\nto steal more keys.</p>\n<p>LiteLLM used Trivy in their CI. The same CI they used to publish code\nto PyPI, the Python software registry. When LiteLLM’s CI ran the\ncompromised Trivy, hackers nabbed their publishing key.</p>\n<p>And on March 24th, when Callum McMahon fired up his IDE, his MacBook\nfroze. And that’s how he discovered <a\nhref=\"https://futuresearch.ai/blog/no-prompt-injection-required/\">the\nLiteLLM hijack</a>.</p>\n<p>McMahon’s MacBook was flailing at bad code that hackers snuck into\nLiteLLM. And the bad code trying to steal credentials:</p>\n<ul>\n<li><code>~/.netrc</code></li>\n<li><code>~/.aws/credentials</code></li>\n<li><code>~/.config/gcloud</code></li>\n<li><code>~/.config/gh</code></li>\n<li><code>~/.azure</code></li>\n<li><code>~/.docker/config.json</code></li>\n<li><code>~/.npmrc</code></li>\n<li><code>~/.git-credentials</code></li>\n<li><code>~/.kube/</code></li>\n</ul>\n<p>Files that are typically strewn around <code>$HOME</code>\ndirectories, full of tokens and keys, often unencrypted.</p>\n</section>\n<section id=\"ai-and-the-supply-chain-doom-spiral\" class=\"level2\">\n<h2>AI and the supply chain doom spiral</h2>\n<p>We’ve dealt with problems like unencrypted credentials, unpinned\ndependencies, and CI footguns forever.</p>\n<p>But AI has accelerated <em>everything</em>, including repeating\nsecurity mistakes.</p>\n<p>On the day of the Trivy compromise, I asked Claude, “how do I scan\ndocker registry images for security vulnerabilities?”</p>\n<p>The reply, in part:</p>\n<pre><code>CI/CD Integration Example (GitHub Actions with Trivy)\n\n    - name: Scan image for vulnerabilities\n      uses: aquasecurity/trivy-action@master</code></pre>\n<p>Broken in two ways:</p>\n<ol type=\"1\">\n<li>Unpinned references – <code>master</code> is a reference that\nchanges all the time. If hackers zombify the repo, I’d be the first\nvictim.</li>\n<li>Active vulnerability – No mention whatsoever of the <a\nhref=\"https://nvd.nist.gov/vuln/detail/CVE-2026-33634\">CVE</a> posted\nthat day. I never asked, so Claude never checked.</li>\n</ol>\n<p>Meanwhile, Vercel’s CEO has attributed his company’s recent data\nbreach to a hacker that was “<a\nhref=\"https://nitter.net/rauchg/status/2045995362499076169\">accelerated\nby AI</a>.” And Anthropic’s latest hype tour includes <a\nhref=\"https://www.theguardian.com/technology/2026/apr/10/us-summoned-bank-bosses-to-discuss-cyber-risks-posed-by-anthropic-latest-ai-model\">briefing\nthe US Federal Reserve Chair</a> about vulnerabilities unearthed by\ntheir frontier model.</p>\n<p>Bad guys with LLMs get superpowers. Good guys with LLMs fall prey to\nmid-2010’s CI problems.</p>\n<p>And the same tool that can root out <a\nhref=\"https://red.anthropic.com/2026/mythos-preview/#ftnt_ref4\">27-year-old\nsecurity problems in OpenBSD</a>, will still tell you to pin your GitHub\nactions to <code>@master</code>.</p>\n</section>\n<section class=\"footnotes footnotes-end-of-document\"\nrole=\"doc-endnotes\">\n<hr />\n<ol>\n<li id=\"fn1\" role=\"doc-endnote\"><p>Or somone calling themselves\n<code>hackerbot-claw</code>, at any rate.<a href=\"https://tylercipriani.com/blog/#fnref1\"\nclass=\"footnote-back\" role=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn2\" role=\"doc-endnote\"><p>My GitHub Actions example is a\nsimpler verison of the action removed in <a\nhref=\"https://github.com/aquasecurity/trivy/pull/10259\">aquasecurity/trivy\n#10259</a>.<a href=\"https://tylercipriani.com/blog/#fnref2\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n</ol>\n</section>\n"

},
{

    "id": "https://tylercipriani.com/blog/2025/08/15/git-lfs/",

    "title": "The future of large files in Git is Git",
    "url": "https://tylercipriani.com/blog/2025/08/15/git-lfs/",

    "author": {
        "name": "Tyler Cipriani"
    },


    "tags": [

     "computing"

    ],

    "date_published": "2025-08-15T19:55:33Z",
    "date_modified": "2025-08-15T20:05:00Z",


    "content_html": "<style>.title {text-wrap:balance;} #content > p:first-child {text-wrap:balance;}</style>\n<p>If Git had a nemesis, it’d be large files.</p>\n<p>Large files bloat Git’s storage, slow down <code>git clone</code>,\nand wreak havoc on Git forges.</p>\n<p>In 2015, GitHub released Git LFS—a Git extension that hacked around\nproblems with large files. But Git LFS added new complications and\nstorage costs.</p>\n<p>Meanwhile, the Git project has been quietly working on large files.\nAnd while LFS ain’t dead yet, the latest Git release shows the path\ntowards a future where LFS is, finally, obsolete.</p>\n<section\nid=\"what-you-can-do-today-replace-git-lfs-with-git-partial-clone\"\nclass=\"level2\">\n<h2>What you can do today: replace Git LFS with Git partial clone</h2>\n<p>Git LFS works by storing large files outside your repo.</p>\n<p>When you clone a project via LFS, you get the repo’s history and\nsmall files, but skip large files. Instead, Git LFS downloads only the\nlarge files you need for your working copy.</p>\n<p>In 2017, the Git project introduced <strong>partial clones</strong>\nthat provide the same benefits as Git LFS:</p>\n<blockquote>\n<p>Partial clone allows us to avoid downloading [large binary assets]\n<em>in advance</em> during clone and fetch operations and thereby reduce\ndownload times and disk usage.</p>\n<p>– Partial Clone Design Notes, <a\nhref=\"https://git-scm.com/docs/partial-clone\">git-scm.com</a></p>\n</blockquote>\n<p>Git’s partial clone and LFS both make for:</p>\n<ol type=\"1\">\n<li><strong>Small checkouts</strong> – On clone, you get the latest copy\nof big files instead of <strong>every</strong> copy.</li>\n<li><strong>Fast clones</strong> – Because you avoid downloading large\nfiles, each clone is fast.</li>\n<li><strong>Quick setup</strong> – Unlike shallow clones, you get the\nentire history of the project—you can get to work right away.</li>\n</ol>\n<p><strong>What is a partial clone?</strong></p>\n<p>A Git partial clone is a clone with a <code>--filter</code>.</p>\n<p>For example, to avoid downloading files bigger than 100KB, you’d\nuse:</p>\n<pre><code>git clone --filter=&#39;blobs:size=100k&#39; &lt;repo&gt;</code></pre>\n<p>Later, Git will lazily download any files over 100KB you need for\nyour checkout.</p>\n<p>By default, if I <code>git clone</code> a repo with many revisions of\na noisome 25 MB PNG file, then cloning is slow and the checkout is\nobnoxiously large:</p>\n<div class=\"sourceCode\" id=\"cb2\"><pre\nclass=\"sourceCode bash\"><code class=\"sourceCode bash\"><span id=\"cb2-1\"><a href=\"https://tylercipriani.com/blog/#cb2-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> time git clone https://github.com/thcipriani/noise-over-git</span>\n<span id=\"cb2-2\"><a href=\"https://tylercipriani.com/blog/#cb2-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">Cloning</span> into <span class=\"st\">&#39;/tmp/noise-over-git&#39;</span>...</span>\n<span id=\"cb2-3\"><a href=\"https://tylercipriani.com/blog/#cb2-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">...</span></span>\n<span id=\"cb2-4\"><a href=\"https://tylercipriani.com/blog/#cb2-4\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">Receiving</span> objects: 100% <span class=\"er\">(</span><span class=\"ex\">153/153</span><span class=\"kw\">)</span><span class=\"ex\">,</span> 1.19 GiB</span>\n<span id=\"cb2-5\"><a href=\"https://tylercipriani.com/blog/#cb2-5\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb2-6\"><a href=\"https://tylercipriani.com/blog/#cb2-6\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">real</span>    3m49.052s</span></code></pre></div>\n<p>Almost four minutes to check out a single 25MB file!</p>\n<div class=\"sourceCode\" id=\"cb3\"><pre\nclass=\"sourceCode bash\"><code class=\"sourceCode bash\"><span id=\"cb3-1\"><a href=\"https://tylercipriani.com/blog/#cb3-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> du <span class=\"at\">--max-depth</span><span class=\"op\">=</span>0 <span class=\"at\">--human-readable</span> noise-over-git/.</span>\n<span id=\"cb3-2\"><a href=\"https://tylercipriani.com/blog/#cb3-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">1.3G</span>    noise-over-git/.</span>\n<span id=\"cb3-3\"><a href=\"https://tylercipriani.com/blog/#cb3-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> ^ 🤬</span></code></pre></div>\n<p>And 50 revisions of that single 25MB file eat 1.3GB of space.</p>\n<p>But a partial clone side-steps these problems:</p>\n<div class=\"sourceCode\" id=\"cb4\"><pre\nclass=\"sourceCode bash\"><code class=\"sourceCode bash\"><span id=\"cb4-1\"><a href=\"https://tylercipriani.com/blog/#cb4-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> git config <span class=\"at\">--global</span> alias.pclone <span class=\"st\">&#39;clone --filter=blob:limit=100k&#39;</span></span>\n<span id=\"cb4-2\"><a href=\"https://tylercipriani.com/blog/#cb4-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> time git pclone https://github.com/thcipriani/noise-over-git</span>\n<span id=\"cb4-3\"><a href=\"https://tylercipriani.com/blog/#cb4-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">Cloning</span> into <span class=\"st\">&#39;/tmp/noise-over-git&#39;</span>...</span>\n<span id=\"cb4-4\"><a href=\"https://tylercipriani.com/blog/#cb4-4\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">...</span></span>\n<span id=\"cb4-5\"><a href=\"https://tylercipriani.com/blog/#cb4-5\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">Receiving</span> objects: 100% <span class=\"er\">(</span><span class=\"ex\">1/1</span><span class=\"kw\">)</span><span class=\"ex\">,</span> 24.03 MiB</span>\n<span id=\"cb4-6\"><a href=\"https://tylercipriani.com/blog/#cb4-6\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb4-7\"><a href=\"https://tylercipriani.com/blog/#cb4-7\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">real</span>    0m6.132s</span>\n<span id=\"cb4-8\"><a href=\"https://tylercipriani.com/blog/#cb4-8\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> du <span class=\"at\">--max-depth</span><span class=\"op\">=</span>0 <span class=\"at\">--human-readable</span> noise-over-git/.</span>\n<span id=\"cb4-9\"><a href=\"https://tylercipriani.com/blog/#cb4-9\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">49M</span>     noise-over-git/</span>\n<span id=\"cb4-10\"><a href=\"https://tylercipriani.com/blog/#cb4-10\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> ^ 😻 <span class=\"er\">(</span><span class=\"ex\">the</span> same size as a git lfs checkout<span class=\"kw\">)</span></span></code></pre></div>\n<p>My filter made cloning 97% faster (3m 49s → 6s), and it reduced my\ncheckout size by 96% (1.3GB → 49M)!</p>\n<p>But there are still some caveats here.</p>\n<p>If you run a command that needs data you filtered out, Git will need\nto make a trip to the server to get it. So, commands like\n<code>git diff</code>, <code>git blame</code>, and\n<code>git checkout</code> will require a trip to your Git host to\nrun.</p>\n<p>But, for large files, this is the same behavior as Git LFS.</p>\n<p>Plus, I can’t remember the last time I ran <code>git blame</code> on\na PNG 🙃.</p>\n</section>\n<section id=\"why-go-to-the-trouble-whats-wrong-with-git-lfs\"\nclass=\"level2\">\n<h2>Why go to the trouble? What’s wrong with Git LFS?</h2>\n<p>Git LFS foists Git’s problems with large files onto users.</p>\n<p>And the problems are significant:</p>\n<ul>\n<li><strong>🖕 High vendor lock-in</strong> – When GitHub wrote Git LFS,\nthe other large file systems—Git Fat, Git Annex, and Git Media—were\nagnostic about the server-side. But GitHub locked users to their\nproprietary server implementation and charged folks to use it.<a\nhref=\"https://tylercipriani.com/blog/#fn1\" class=\"footnote-ref\" id=\"fnref1\"\nrole=\"doc-noteref\"><sup>1</sup></a></li>\n<li><strong>💸 Costly</strong> – GitHub won because it let users host\nrepositories for free. But Git LFS started as a paid product. Nowadays,\nthere’s a free tier, but you’re dependent on the whims of GitHub to set\npricing. Today, a 50GB repo on GitHub will cost $40/year for storage. In\ncontrast, storing 50GB on Amazon’s S3 standard storage is $13/year.</li>\n<li><strong>😰 Hard to undo</strong> – Once you’ve moved to Git LFS,\nit’s impossible to undo the move without rewriting history.</li>\n<li><strong>🌀 Ongoing set-up costs</strong> – All your collaborators\nneed to install Git LFS. Without Git LFS installed, your collaborators\nwill get confusing, metadata-filled text files instead of the large\nfiles they expect.</li>\n</ul>\n</section>\n<section id=\"the-future-git-large-object-promisors\" class=\"level2\">\n<h2>The future: Git large object promisors</h2>\n<p>Large files create problems for Git forges, too.</p>\n<p>GitHub and GitLab put limits on file size<a href=\"https://tylercipriani.com/blog/#fn2\"\nclass=\"footnote-ref\" id=\"fnref2\" role=\"doc-noteref\"><sup>2</sup></a>\nbecause big files cost more money to host. Git LFS keeps server-side\ncosts low by offloading large files to CDNs.</p>\n<p>But the Git project has a new solution.</p>\n<p>Earlier this year, Git merged a new feature: <strong>large object\npromisers</strong>. Large object promisors aim to provide the same\nserver-side benefits as LFS, minus the hassle to users.</p>\n<blockquote>\n<p>This effort aims to especially improve things on the server side, and\nespecially for large blobs that are already compressed in a binary\nformat.</p>\n<p>This effort aims to provide an alternative to Git LFS</p>\n<p>– Large Object Promisors, <a\nhref=\"https://git-scm.com/docs/large-object-promisors\">git-scm.com</a></p>\n</blockquote>\n<p><strong>What is a large object promisor?</strong></p>\n<p>Large object promisors are special Git remotes that only house large\nfiles.</p>\n<p>In the bright, shiny future, large object promisors will work like\nthis:</p>\n<ol type=\"1\">\n<li>You push a large file to your Git host.</li>\n<li>In the background, your Git host offloads that large file to a large\nobject promisor.</li>\n<li>When you clone, the Git host tells your Git client about the\npromisor.</li>\n<li>Your client will clone from the Git host, and automagically nab\nlarge files from the promisor remote.</li>\n</ol>\n<p>But we’re still a ways off from that bright, shiny future.</p>\n<p>Git large object promisors are still a work in progress. Pieces of\nlarge object promisors merged to Git in <a\nhref=\"https://lore.kernel.org/git/xmqqfrjfilc8.fsf@gitster.g/\">March of\n2025</a>. But there’s <a\nhref=\"https://gitlab.com/groups/gitlab-org/-/epics/9094\">more to do</a>\nand <a href=\"https://gitlab.com/groups/gitlab-org/-/epics/15972\">open\nquestions</a> yet to answer.</p>\n<p>And so, for today, you’re stuck with Git LFS for giant files. But\nonce large object promisors see broad adoption, maybe GitHub will let\nyou push files bigger than 100MB.</p>\n</section>\n<section id=\"the-future-of-large-files-in-git-is-git.\" class=\"level2\">\n<h2>The future of large files in Git is Git.</h2>\n<p>The Git project is thinking hard about large files, so you don’t have\nto.</p>\n<p>Today, we’re stuck with Git LFS.</p>\n<p>But soon, the only obstacle for large files in Git will be your\nhalf-remembered, ominous hunch that it’s a bad idea to stow your MP3\nlibrary in Git.</p>\n<hr style=\"margin-top: 3em;\" />\n<div style=\"margin: 1em auto;\">\n<p>Edited by <a href=\"https://refactoringenglish.com/\">Refactoring\nEnglish</a></p>\n</div>\n</section>\n<section class=\"footnotes footnotes-end-of-document\"\nrole=\"doc-endnotes\">\n<hr />\n<ol>\n<li id=\"fn1\" role=\"doc-endnote\"><p>Later, other Git forges made their <a\nhref=\"https://about.gitlab.com/blog/towards-a-production-quality-open-source-git-lfs-server/\">own\nLFS servers</a>. Today, you can push to multiple Git forges or use an\nLFS transfer agent, but all this makes set up harder for contributors.\nYou’re pretty much locked-in unless you put in extra effort to get\nunlocked.<a href=\"https://tylercipriani.com/blog/#fnref1\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn2\" role=\"doc-endnote\"><p>File size limits: <a\nhref=\"https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github\">100MB\nfor GitHub</a>, <a\nhref=\"https://docs.gitlab.com/user/gitlab_com/#account-and-limit-settings\">100MB\nfor GitLab.com</a><a href=\"https://tylercipriani.com/blog/#fnref2\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n</ol>\n</section>\n"

},
{

    "id": "https://tylercipriani.com/blog/2025/05/21/git-commits/",

    "title": "Per-project git commit templates",
    "url": "https://tylercipriani.com/blog/2025/05/21/git-commits/",

    "author": {
        "name": "Tyler Cipriani"
    },


    "tags": [

     "computing"

    ],

    "date_published": "2025-05-21T19:22:45Z",
    "date_modified": "2025-06-02T23:05:20Z",


    "content_html": "<blockquote>\n<p>People should try to compare the quality of the kernel git logs with\nsome other projects, and cry themselves to sleep.</p>\n<p>– Linus Torvalds</p>\n</blockquote>\n<p>I’ll never remember your project’s commit guidelines.</p>\n<p>Every project insists on something different:</p>\n<ul>\n<li><a\nhref=\"https://www.conventionalcommits.org/en/v1.0.0/\">Conventional\ncommits</a></li>\n<li><a\nhref=\"https://zeromq.org/how-to-contribute/#write-good-commit-messages\">Problem/Solution\nformat</a></li>\n<li><a href=\"https://gitmoji.dev/\">Gitmoji</a></li>\n<li>The twisty maze of <a\nhref=\"https://docs.kernel.org/process/submitting-patches.html#when-to-use-acked-by-cc-and-co-developed-by\">trailers\nin the Linux Kernel</a></li>\n</ul>\n<p>But <a\nhref=\"https://git-scm.com/docs/git-config#Documentation/git-config.txt-codecommittemplatecode\">git\ncommit templates</a> help. Commit templates provide a scaffold for\ncommit messages, offering documentation where you need it: inside the\neditor where you’re writing your commit message.</p>\n<section id=\"what-is-a-git-commit-template\" class=\"level2\">\n<h2>What is a git commit template?</h2>\n<p>When you type <code>git commit</code>, git pops open your text\neditor<a href=\"https://tylercipriani.com/blog/#fn1\" class=\"footnote-ref\" id=\"fnref1\"\nrole=\"doc-noteref\"><sup>1</sup></a>. Git can pre-fill your editor with a\ncommit template—it’s like a form you fill out.</p>\n<p>Creating a commit template is simple.</p>\n<ul>\n<li>Create a plaintext file – mine lives at\n<code>~/.config/git/message.txt</code></li>\n<li>Tell git to use it:</li>\n</ul>\n<div class=\"sourceCode\" id=\"cb1\"><pre\nclass=\"sourceCode bash\"><code class=\"sourceCode bash\"><span id=\"cb1-1\"><a href=\"https://tylercipriani.com/blog/#cb1-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"fu\">git</span> config <span class=\"at\">--global</span> <span class=\"dt\">\\</span></span>\n<span id=\"cb1-2\"><a href=\"https://tylercipriani.com/blog/#cb1-2\" aria-hidden=\"true\" tabindex=\"-1\"></a>    commit.template <span class=\"st\">&#39;~/.config/git/message.txt&#39;</span></span></code></pre></div>\n<p><a\nhref=\"https://gist.github.com/thcipriani/5e79a11b69b66a493249bac1439bfe02\">My\ndefault template</a> packs everything I know about writing a commit.</p>\n</section>\n<section id=\"project-specific-templates-with-includeif\" class=\"level2\">\n<h2>Project-specific templates with <code>IncludeIf</code></h2>\n<p>The real magic of commit templates is you can have different\ntemplates for each project.</p>\n<p>Different projects can use different templates with git’s\n<code>includeIf</code> configuration setting.<a href=\"https://tylercipriani.com/blog/#fn2\"\nclass=\"footnote-ref\" id=\"fnref2\" role=\"doc-noteref\"><sup>2</sup></a></p>\n<p>Large projects, such as <a\nhref=\"https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/SubmittingPatches?id=4e8a2372f9255a1464ef488ed925455f53fbdaa1\">the\nLinux kernel</a>, <a\nhref=\"https://git-scm.com/docs/SubmittingPatches#describe-changes\">git</a>,\nand <a\nhref=\"https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines\">MediaWiki</a>,\nhave their own commit guidelines.</p>\n<p>For Wikimedia work, I stow git repos in\n<code>~/Projects/Wikimedia</code> and at the bottom of my global git\nconfig (<code>~/.config/git/config</code>) I have:</p>\n<pre><code>[includeIf &quot;gitdir:~/Projects/Wikimedia/**&quot;]\n    path = ~/.config/git/config.wikimedia</code></pre>\n<p>In <code>config.wikimedia</code>, I point to my Wikimedia-specific\ncommit template. I also override other git config settings like my\n<code>user.email</code> or <code>core.hooksPath</code>.</p>\n</section>\n<section id=\"an-example-my-global-template\" class=\"level2\">\n<h2>An example: my global template</h2>\n<p>My default commit template contains three sections:</p>\n<ol type=\"1\">\n<li>Subject – 50 characters or less, capitalized, no end\npunctuation.</li>\n<li>Body – Wrap at 72 characters with a blank line separating it from\nthe subject.</li>\n<li>Trailers – Standard formats with a blank line separating them from\nthe body.</li>\n</ol>\n<p>In each section, I added pointers for both format<a href=\"https://tylercipriani.com/blog/#fn3\"\nclass=\"footnote-ref\" id=\"fnref3\" role=\"doc-noteref\"><sup>3</sup></a> and\ncontent.</p>\n<p>For the header, the guidance is quick:</p>\n<pre><code>\n# 50ch. wide ----------------------------- SUBJECT\n#                                                |\n#     &quot;If applied, this commit will...&quot;          |\n#                                                |\n#     Change / Add / Fix                         |\n#     Remove / Update / Document                 |\n#                                                |\n# ------- ↓ LEAVE BLANK LINE ↓ ---------- /SUBJECT\n</code></pre>\n<p>For the body, I remind myself to answer basic questions:</p>\n<pre><code># 72ch. wide ------------------------------------------------------ BODY\n#                                                                      |\n#     - Why should this change be made?                                |\n#       - What problem are you solving?                                |\n#       - Why this solution?                                           |\n#     - What&#39;s wrong with the current code?                            |\n#     - Are there other ways to do it?                                 |\n#     - How can the reviewer confirm it works?                         |\n#                                                                      |</code></pre>\n<p>And that’s it, except for git trailers.</p>\n</section>\n<section id=\"the-twisty-maze-of-git-trailers\" class=\"level2\">\n<h2>The twisty maze of git trailers</h2>\n<p>My template has a section for trailers used by the projects I work\non.</p>\n<pre><code>#     TRAILERS                                                         |\n#     --------                                                         |\n#     (optional) Uncomment as needed.                                  |\n#     Leave a blank line before the trailers.                          |\n#                                                                      |\n# Bug: #xxxx\n# Acked-by: Example User &lt;user@example.com&gt;\n# Cc: Example User &lt;user@example.com&gt;\n# Co-Authored-by: Example User &lt;user@example.com&gt;\n# Requested-by: Example User &lt;user@example.com&gt;\n# Reported-by: Example User &lt;user@example.com&gt;\n# Reviewed-by: Example User &lt;user@example.com&gt;\n# Suggested-by: Example User &lt;user@example.com&gt;\n# Tested-by: Example User &lt;user@example.com&gt;\n# Thanks: Example User &lt;user@example.com&gt;</code></pre>\n<p>These trailers serve as useful breadcrumbs of documentation. Git can\nparse them using standard commands.</p>\n<p>For example, if I wanted a tab-separated list of commits and their\nrelated tasks, I could find <code>Bug</code> trailers using\n<code>git log</code>:</p>\n<div class=\"sourceCode\" id=\"cb6\"><pre\nclass=\"sourceCode bash\"><code class=\"sourceCode bash\"><span id=\"cb6-1\"><a href=\"https://tylercipriani.com/blog/#cb6-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> TAB=%x09</span>\n<span id=\"cb6-2\"><a href=\"https://tylercipriani.com/blog/#cb6-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> BUG_TRAILER=<span class=\"st\">&#39;%(trailers:key=Bug,valueonly=true,separator=%x2C )&#39;</span></span>\n<span id=\"cb6-3\"><a href=\"https://tylercipriani.com/blog/#cb6-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> SHORT_HASH=%h</span>\n<span id=\"cb6-4\"><a href=\"https://tylercipriani.com/blog/#cb6-4\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> SUBJ=%s</span>\n<span id=\"cb6-5\"><a href=\"https://tylercipriani.com/blog/#cb6-5\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> FORMAT=<span class=\"st\">&quot;</span><span class=\"va\">${SHORT_HASH}${TAB}${BUG_TRAILER}${TAB}${SUBJ}</span><span class=\"st\">&quot;</span></span>\n<span id=\"cb6-6\"><a href=\"https://tylercipriani.com/blog/#cb6-6\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">$</span> git log <span class=\"at\">--topo-order</span> <span class=\"at\">--no-merges</span> <span class=\"dt\">\\</span></span>\n<span id=\"cb6-7\"><a href=\"https://tylercipriani.com/blog/#cb6-7\" aria-hidden=\"true\" tabindex=\"-1\"></a>      <span class=\"at\">--format</span><span class=\"op\">=</span><span class=\"st\">&quot;</span><span class=\"va\">$FORMAT</span><span class=\"st\">&quot;</span></span>\n<span id=\"cb6-8\"><a href=\"https://tylercipriani.com/blog/#cb6-8\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">d2b09deb12f</span>     T359762 Rewrite Kurdish <span class=\"er\">(</span><span class=\"ex\">ku</span><span class=\"kw\">)</span> <span class=\"ex\">Latin</span> to Arabic converter</span>\n<span id=\"cb6-9\"><a href=\"https://tylercipriani.com/blog/#cb6-9\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">28123a6a262</span>     T332865 tests: Remove non-static fallback in HookRunnerTestBase</span>\n<span id=\"cb6-10\"><a href=\"https://tylercipriani.com/blog/#cb6-10\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">4e919a307a4</span>     T328919 tests: Remove unused argument from data provider in PageUpdaterTest</span>\n<span id=\"cb6-11\"><a href=\"https://tylercipriani.com/blog/#cb6-11\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">bedd0f685f9</span>             objectcache: Improve <span class=\"kw\">`</span><span class=\"fu\">RESTBagOStuff::handleError()</span><span class=\"kw\">`</span></span>\n<span id=\"cb6-12\"><a href=\"https://tylercipriani.com/blog/#cb6-12\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ex\">2182a0c4490</span>     T393219 tests: Remove two data provider in RestStructureTest</span></code></pre></div>\n</section>\n<section id=\"stop-remembering-commit-message-guidelines\" class=\"level2\">\n<h2>Stop remembering commit message guidelines</h2>\n<p>Git commit templates free your brain from remembering what to write,\nallowing you to focus on the story you need to tell.</p>\n<p>Save your brain for what it’s good at.</p>\n</section>\n<section class=\"footnotes footnotes-end-of-document\"\nrole=\"doc-endnotes\">\n<hr />\n<ol>\n<li id=\"fn1\" role=\"doc-endnote\"><p>Starting with\n<code>core.editor</code> in your git config, <code>$VISUAL</code> or\n<code>$EDITOR</code> in your shell, finally falling back to\n<code>vi</code>.<a href=\"https://tylercipriani.com/blog/#fnref1\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn2\" role=\"doc-endnote\"><p>You could also set it inside a repo’s\n<code>.git/config</code>, <code>includeIf</code> is useful if you have\nmultiple repos with the same standards under one directory.<a\nhref=\"https://tylercipriani.com/blog/#fnref2\" class=\"footnote-back\" role=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn3\" role=\"doc-endnote\"><p>All cribbed from <a\nhref=\"https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html\">Tim\nPope</a><a href=\"https://tylercipriani.com/blog/#fnref3\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n</ol>\n</section>\n"

},
{

    "id": "https://tylercipriani.com/blog/2025/03/05/boox-go-10-3-review/",

    "title": "Boox Go 10.3, two months in",
    "url": "https://tylercipriani.com/blog/2025/03/05/boox-go-10-3-review/",

    "author": {
        "name": "Tyler Cipriani"
    },


    "tags": [

     "computing"

    ],

    "date_published": "2025-03-05T03:41:36Z",
    "date_modified": "2025-03-13T23:11:42Z",


    "content_html": "<blockquote>\n<p>[The] Linux kernel uses GPLv2, and if you distribute GPLv2 code, you\nhave to provide a copy of the source (and modifications) <em>once\nsomeone asks for it</em>. And now I’m asking nicely for you to do so\n🙂</p>\n<p>– Joga, <a\nhref=\"https://archive.is/HP2Qk\">bbs.onyx-international.com</a></p>\n</blockquote>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/thumbs/01/0c363d3da2755ee421a393d908b697/large.jpg\"\nalt=\"Boox in split screen, typewriter mode\" />\n<figcaption aria-hidden=\"true\">Boox in split screen, typewriter\nmode</figcaption>\n</figure>\n<p>In January, I bought a Boox Go 10.3—a 10.3-inch, 300-ppi, e-ink\nAndroid tablet.</p>\n<p>After two months, I use the Boox daily—it’s replaced my planner,\nnotebook, countless PDF print-offs, and the good parts of my phone.</p>\n<p>But Boox’s parent company, Onyx, is sketchy.</p>\n<p>I’m conflicted. The Boox Go is a beautiful, capable tablet that I use\nevery day, but I recommend avoiding as long as Onyx continues to\ndisregard the rights of its users.</p>\n<section id=\"how-im-using-my-boox\" class=\"level2\">\n<h2>How I’m using my Boox</h2>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/thumbs/f3/63eee65c63696d95c13156cd9f67ab/large.jpg\"\nalt=\"My e-ink floor desk\" />\n<figcaption aria-hidden=\"true\">My e-ink floor desk</figcaption>\n</figure>\n<p>Each morning, I plop down in front of my <a\nhref=\"https://www.amazon.com/MagicHold-Rotating-Height-Adjusting-13-15-6/dp/B01MG1EWPQ?th=1\">MagicHold</a>\nlaptop stand and journal on my Boox with Obsidian.</p>\n<p>I use Syncthing to back up my planner and sync my <a\nhref=\"https://www.zotero.org/\">Zotero</a> library between my Boox and\nlaptop.</p>\n<p>In the evening, I review my <a\nhref=\"https://mirzakhani.io/products/zen-planner-2025\">PDF planner</a>\nand plot for tomorrow.</p>\n<p>I use these apps:</p>\n<ul>\n<li><strong>Obsidian</strong> – a markdown editor that syncs between all\nmy devices with no fuss for $8/mo.</li>\n<li><strong>Syncthing</strong> – I love Syncthing—it’s an encrypted,\ncontinuous file sync-er without a centralized server.</li>\n<li><strong>Meditation apps</strong><a href=\"https://tylercipriani.com/blog/#fn1\" class=\"footnote-ref\"\nid=\"fnref1\" role=\"doc-noteref\"><sup>1</sup></a> – Guided meditation away\nfrom the blue light glow of my phone or computer is better.</li>\n</ul>\n<p>Before buying the Boox, I considered a reMarkable.</p>\n<p>The <a\nhref=\"https://remarkable.com/store/remarkable-paper/pro\">reMarkable\nPaper Pro</a> has a beautiful color screen with a frontlight, a nice\npen, and a “<a\nhref=\"https://remarkable.com/store/remarkable-2/type-folio\">type\nfolio</a>,” plus it’s <a\nhref=\"https://www.calmtech.institute/products\">certified by the Calm\nTech Institute</a>.</p>\n<p>But the reMarkable is a distraction-free e-ink tablet. Meanwhile, I\nneed <em>distraction-lite</em>.</p>\n</section>\n<section id=\"what-i-like\" class=\"level2\">\n<h2>What I like</h2>\n<ul>\n<li><strong>Calm(ish) technology</strong> – The Boox is an intentional\ndevice. Browsing the internet, reading emails, and watching videos is\nhard, but that’s good.</li>\n<li><strong>Apps</strong> – Google Play works out of the box. I can\ninstall F-Droid and change my launcher without difficulty.</li>\n<li><strong>Split screen</strong> – The built-in launcher has a split\nscreen feature. I use it to open a PDF side-by-side with a notes\ndoc.</li>\n<li><strong>Reading</strong> – The screen is a 300ppi Carta 1200, making\ntext crisp and clear.</li>\n</ul>\n</section>\n<section id=\"what-i-dislike\" class=\"level2\">\n<h2>What I dislike</h2>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/thumbs/10/733dfc26554fb3363483cf306f2d5a/large.jpg\"\nalt=\"I filmed myself typing at 240fps, each frame is 4.17ms. Boox’s typing latency is between 150ms and 275ms at the fastest refresh rate inside Obsidian.\" />\n<figcaption aria-hidden=\"true\">I filmed myself typing at 240fps, each\nframe is 4.17ms. Boox’s typing latency is between 150ms and 275ms at the\nfastest refresh rate inside Obsidian.</figcaption>\n</figure>\n<ul>\n<li><strong>Typing</strong> – Typing latency is noticeable.\n<ul>\n<li>At Boox’s highest refresh rate, after hitting a key, text takes\nbetween 150ms to 275ms to appear.</li>\n<li>I can still type, though it’s distracting at times.</li>\n</ul></li>\n</ul>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/thumbs/67/b6fd836c59e5ecd6aa429d5fda4ab6/large.jpg\"\nalt=\"The horror of the default pen\" />\n<figcaption aria-hidden=\"true\">The horror of the default\npen</figcaption>\n</figure>\n<ul>\n<li><strong>Accessories</strong>\n<ul>\n<li><strong>Pen</strong> – The default pen looks like a child’s\nwhiteboard marker and feels cheap. I replaced it with the <a\nhref=\"https://www.amazon.com/Amazon-Kindle-Scribe-Premium-Improved/dp/B0D2FBN1W8/\">Kindle\nScribe Premium pen</a>, and the writing experience is vastly\nimproved.</li>\n<li><strong>Cover</strong> – It’s impossible to find a nice cover. I’m\nusing a $15 cover that I’m encasing in stickers.</li>\n</ul></li>\n<li><strong>Tool switching</strong> – Swapping between apps is slow and\nclunky. I blame Android and the current limitations of e-ink more than\nBoox.</li>\n<li><strong>No frontlight</strong> – The Boox’s lack of frontlight\nprevents me from reading more with it. I knew this when I bought my\nBoox, but devices with frontlights seem to make other compromises.</li>\n</ul>\n<p><strong>Onyx</strong></p>\n<p>The Chinese company behind Boox, Onyx International, Inc., runs the\nservers where the Boox routes telemetry. I block this traffic with <a\nhref=\"https://pi-hole.net/\">Pi-Hole</a><a href=\"https://tylercipriani.com/blog/#fn2\"\nclass=\"footnote-ref\" id=\"fnref2\"\nrole=\"doc-noteref\"><sup>2</sup></a>.</p>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/2025-03-04_boox-telemetry.png\"\nalt=\"pihole-ing whatever telemetry Boox collects\" />\n<figcaption aria-hidden=\"true\">pihole-ing whatever telemetry Boox\ncollects</figcaption>\n</figure>\n<p>I inspected this traffic via <a href=\"https://mitmproxy.org/\">Mitm\nproxy</a>—most traffic was benign, though I never opted into sending any\ntelemetry (nor am I logged in to a Boox account). But it’s also an\nAndroid device, so it’s feeding telemetry into Google’s gaping maw,\ntoo.</p>\n<p>Worse, <a href=\"https://archive.is/HP2Qk\">Onyx is flouting the terms\nof the GNU Public License</a>, declining to release Linux kernel\nmodifications to users. This is anathema to me—<a\nhref=\"https://www.gnu.org/licenses/gpl-violation.en.html\">GPL\nviolations</a> are tantamount to theft.</p>\n<p>Onyx’s disregard for user rights makes me regret buying the Boox.</p>\n</section>\n<section id=\"verdict\" class=\"level2\">\n<h2>Verdict</h2>\n<p>I’ll continue to use the Boox and feel bad about it. I hope my\ndigging in this post will help the next person.</p>\n<p>Unfortunately, the e-ink tablet market is too niche to support the\nkind of solarpunk future I’d always imagined.</p>\n<p>But there’s an opportunity for an open, Linux-based tablet to\ndominate e-ink. Linux is playing catch-up on phones with PostmarketOS.\nMeanwhile, the best e-ink tablets have to offer are old, unupdateable\nversions of Android, like the OS on the Boox.</p>\n<p>In the future, I’d love to pay a license- and privacy-respecting\ncompany for beautiful, calm technology and recommend their product to\neveryone. But today is not the future.</p>\n</section>\n<section class=\"footnotes footnotes-end-of-document\"\nrole=\"doc-endnotes\">\n<hr />\n<ol>\n<li id=\"fn1\" role=\"doc-endnote\"><p>I go back and forth between “Waking\nUp” and “Calm”<a href=\"https://tylercipriani.com/blog/#fnref1\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn2\" role=\"doc-endnote\"><p>Using <a\nhref=\"https://github.com/JordanEJ/Onyx-Boox-Blocklist\">github.com/JordanEJ/Onyx-Boox-Blocklist</a><a\nhref=\"https://tylercipriani.com/blog/#fnref2\" class=\"footnote-back\" role=\"doc-backlink\">↩︎</a></p></li>\n</ol>\n</section>\n"

},
{

    "id": "https://tylercipriani.com/blog/2024/10/24/plain-text-accounting/",

    "title": "Eventually consistent plain text accounting",
    "url": "https://tylercipriani.com/blog/2024/10/24/plain-text-accounting/",

    "author": {
        "name": "Tyler Cipriani"
    },


    "date_published": "2024-10-24T00:37:44Z",
    "date_modified": "2024-11-13T02:16:50Z",


    "content_html": "<style>.title { text-wrap: balance }</style>\n<figure>\n<img src=\"https://photos.tylercipriani.com/2024-11-08_expenses.png\"\nalt=\"Spending for October, generated by piping hledger → R\" />\n<figcaption aria-hidden=\"true\">Spending for October, generated by piping\nhledger → R</figcaption>\n</figure>\n<p>Over the past six months, I’ve tracked my money with <a\nhref=\"https://hledger.org/\"><code>hledger</code></a>—a plain text\ndouble-entry accounting system written in Haskell. It’s been\nsurprisingly painless.</p>\n<p>My previous attempts to pick up <em>real</em> accounting tools\nfloundered. Hosted tools are privacy nightmares, and my stint with <a\nhref=\"https://gnucash.org/\">GnuCash</a> didn’t last.</p>\n<p>But after stumbling on Dmitry Astapov’s “<a\nhref=\"https://github.com/adept/full-fledged-hledger/wiki/\">Full-fledged\n<code>hledger</code></a>” wiki<a href=\"https://tylercipriani.com/blog/#fn1\" class=\"footnote-ref\"\nid=\"fnref1\" role=\"doc-noteref\"><sup>1</sup></a>, it\nclicked—<strong>eventually consistent</strong> accounting. Instead of\nmodeling your money all at once, take it one hacking session at a\ntime.</p>\n<blockquote>\n<p>It should be easy to work towards eventual consistency. […] I should\nbe able to [add financial records] bit by little bit, leaving things\nhalf-done, and picking them up later with little (mental) effort.</p>\n<p>– Dmitry Astapov, Full-Fledged Hledger</p>\n</blockquote>\n<section id=\"principles-of-my-system\" class=\"level2\">\n<h2>Principles of my system</h2>\n<p>I’ve cobbled together a system based on these principles:</p>\n<ul>\n<li><strong>Avoid manual entry</strong> – Avoid typing in each\ntransaction. Instead, rely on CSVs from the bank.</li>\n<li><strong>CSVs as truth</strong> – CSVs are the only things that\nmatter. Everything else can be blown away and rebuilt anytime.</li>\n<li><strong>Embrace version control</strong> – Keep everything under\nversion control in Git for easy comparison and safe\nexperimentation.</li>\n</ul>\n</section>\n<section id=\"learn-hledger-in-five-minutes\" class=\"level2\">\n<h2>Learn <code>hledger</code> in five minutes</h2>\n<p><code>hledger</code> concepts are heady, but its use is simple. I\ndivide the core concepts into two categories:</p>\n<ul>\n<li>Stuff <code>hledger</code> cares about:\n<ul>\n<li><strong>Transactions</strong> – how <code>hledger</code> moves money\nbetween accounts.</li>\n<li><strong>Journal files</strong> – files full of transactions</li>\n</ul></li>\n<li>Stuff I care about:\n<ul>\n<li><strong>Rules files</strong> – how I set up accounts, import CSVs,\nand move money between accounts.</li>\n<li><strong>Reports</strong> – help me see where my money is going and\nif I messed up my rules.</li>\n</ul></li>\n</ul>\n<p><strong>Transactions</strong> move money between accounts:</p>\n<pre><code>2024-01-01 Payday\n    income:work      $-100.00\n    assets:checking   $100.00</code></pre>\n<p>This transaction shows that on Jan 1, 2024, money moved from\n<code>income:work</code> into <code>assets:checking</code>—Payday.</p>\n<p>The sum of each transaction should be $0. Money comes from somewhere,\nand the same amount goes somewhere else—double-entry accounting. This is\npowerful technology—it makes mistakes impossible to ignore.</p>\n<p><strong>Journal files</strong> are text files containing one or more\ntransactions:</p>\n<pre><code>2024-01-01 Payday\n    income:work              $-100.00\n    assets:checking           $100.00\n2024-01-02 QUANSHENG UVK5\n    assets:checking          $-29.34\n    expenses:fun:radio        $29.34</code></pre>\n<p><strong>Rules files</strong> transform CSVs into journal files via\nregex matching.</p>\n<p>Here’s a CSV from my bank:</p>\n<pre><code>Transaction Date,Description,Category,Type,Amount,Memo\n09/01/2024,DEPOSIT Paycheck,Payment,Payment,1000.00,\n09/04/2024,PizzaPals Pizza,Food &amp; Drink,Sale,-42.31,\n09/03/2024,Amazon.com*XXXXXXXXY,Shopping,Sale,-35.56,\n09/03/2024,OBSIDIAN.MD,Shopping,Sale,-10.00,\n09/02/2024,Amazon web services,Personal,Sale,-17.89,</code></pre>\n<p>And here’s a <code>checking.rules</code> to transform that CSV into a\njournal file so I can use it with <code>hledger</code>:</p>\n<pre><code># checking.rules\n# --------------\n# Map CSV fields → hledger fields[0]\nfields date,description,category,type,amount,memo,_\n# `account1`: the account for the whole CSV.[1]\naccount1    assets:checking\naccount2    expenses:unknown\nskip 1\n\ndate-format %m/%d/%Y\ncurrency $\n\nif %type Payment\n    account2 income:unknown\nif %category Food &amp; Drink\n    account2 expenses:food:dining\n\n# [0]: &lt;https://hledger.org/hledger.html#field-names&gt;\n# [1]: &lt;https://hledger.org/hledger.html#account-field&gt;</code></pre>\n<p>With these two files (<code>checking.rules</code> and\n<code>2024-09_checking.csv</code>), I can make the CSV into a\njournal:</p>\n<pre><code>$ &gt; 2024-09_checking.journal \\\n    hledger print \\\n    --rules-file checking.rules \\\n    -f 2024-09_checking.csv\n$ head 2024-09_checking.journal\n2024-09-01 DEPOSIT Paycheck\n    assets:checking        $1000.00\n    income:unknown        $-1000.00\n\n2024-09-02 Amazon web services\n    assets:checking          $-17.89\n    expenses:unknown          $17.89</code></pre>\n<p><strong>Reports</strong> are interesting ways to view transactions\nbetween accounts.</p>\n<p>There are registers, balance sheets, and income statements:</p>\n<pre><code>$ hledger incomestatement \\\n    --depth=2 \\\n    --file=2024-09_bank.journal\n\nRevenues:\n               $1000.00 income:unknown\n-----------------------\n               $1000.00\n\n\nExpenses:\n                 $42.31 expenses:food\n                 $63.45 expenses:unknown\n-----------------------\n                $105.76\n-----------------------\nNet:            $894.24</code></pre>\n<p>At the beginning of September, I spent <code>$105.76</code> and made\n<code>$1000</code>, leaving me with <code>$894.24</code>.</p>\n<p>But a good chunk is going to the default expense account,\n<code>expenses:unknown</code>. I can use the\n<code>hleger aregister</code> to see what those transactions are:</p>\n<pre><code>$ hledger areg expenses:unknown \\\n    --file=2024-09_checking.journal \\\n    -O csv | \\\n  csvcut -c description,change | \\\n  csvlook\n| description              | change |\n| ------------------------ | ------ |\n| OBSIDIAN.MD              |  10.00 |\n| Amazon web services      |  17.89 |\n| Amazon.com*XXXXXXXXY     |  35.56 |\nl</code></pre>\n<p>Then, I can add some more rules to my\n<code>checking.rules</code>:</p>\n<pre><code>if OBSIDIAN.MD\n    account2 expenses:personal:subscriptions\nif Amazon web services\n    account2 expenses:personal:web:hosting\nif Amazon.com\n    account2 expenses:personal:shopping:amazon</code></pre>\n<p>Now, I can reprocess my data to get a better picture of my\nspending:</p>\n<pre><code>$ &gt; 2024-09_bank.journal \\\n    hledger print \\\n    --rules-file bank.rules \\\n    -f 2024-09_bank.csv\n$ hledger bal expenses \\\n    --depth=3 \\\n    --percent \\\n    -f 2024-09_checking2.journal\n              30.0 %  expenses:food:dining\n              33.6 %  expenses:personal:shopping\n               9.5 %  expenses:personal:subscriptions\n              16.9 %  expenses:personal:web\n--------------------\n             100.0 %</code></pre>\n<p>For the Amazon.com purchase, I lumped it into the\n<code>expenses:personal:shopping</code> account. But I could dig\ndeeper—download <a\nhref=\"https://www.amazon.com/hz/privacy-central/data-requests/preview.html\">my\norder history from Amazon</a> and categorize that spending.</p>\n<p>This is the power of working bit-by-bit—the data guides you to the\nnext, deeper rabbit hole.</p>\n</section>\n<section id=\"goals-and-non-goals\" class=\"level2\">\n<h2>Goals and non-goals</h2>\n<p>Why am I doing this? For years, I maintained a monthly spreadsheet of\naccount balances. I had a balance sheet. But I still had questions.</p>\n<figure>\n<img\nsrc=\"https://photos.tylercipriani.com/2024-11-04_hledger-gnuplot-avg.png\"\nalt=\"Spending over six months, generated by piping hledger → gnuplot\" />\n<figcaption aria-hidden=\"true\">Spending over six months, generated by\npiping hledger → gnuplot</figcaption>\n</figure>\n<p>Before diving into accounting software, these were my goals:</p>\n<ul>\n<li><strong>Granular understanding of my spending</strong> – The big\none. This is where my monthly spreadsheet fell short. I knew I had money\nin the bank—I kept my monthly balance sheet. I budgeted up-front the %\nof my income I was saving. But I had no idea where my <em>other\nmoney</em> was going.</li>\n<li><strong>Data privacy</strong> – I’m unwilling to hand the keys to my\naccounts to YNAB or Mint.</li>\n<li><strong>Increased value over time</strong> – The more time I put in,\nthe more value I want to get out—this is what you get from professional\ntools built for nerds. While I wished for low-effort setup, I wanted the\ntool to be able to grow to more uses over time.</li>\n</ul>\n<p><strong>Non-goals</strong>—these are the parts I never cared\nabout:</p>\n<ul>\n<li><strong>Investment tracking</strong> – For now, I left this out of\nscope. Between monthly balances in my spreadsheet and online investing\ntools’ ability to drill down, I was fine.<a href=\"https://tylercipriani.com/blog/#fn2\"\nclass=\"footnote-ref\" id=\"fnref2\"\nrole=\"doc-noteref\"><sup>2</sup></a></li>\n<li><strong>Taxes</strong> – Folks smarter than me help me understand my\nyearly taxes.<a href=\"https://tylercipriani.com/blog/#fn3\" class=\"footnote-ref\" id=\"fnref3\"\nrole=\"doc-noteref\"><sup>3</sup></a></li>\n<li><strong>Shared system</strong> – I may want to share reports from\nthis system, but no one will have to work in it except me.</li>\n<li><strong>Cash</strong> – Cash transactions are unimportant to me. I\nwithdraw money from the ATM sometimes. It evaporates.</li>\n</ul>\n<p><code>hledger</code> can track all these things. My setup is flexible\nenough to support them someday. But that’s unimportant to me right\nnow.</p>\n</section>\n<section id=\"monthly-maintenance\" class=\"level2\">\n<h2>Monthly maintenance</h2>\n<p>I spend about an hour a month checking in on my money Which frees me\nto spend time making fancy charts—an activity I <a\nhref=\"https://gist.github.com/thcipriani/9cd0a0e1e6d42fe27483d0a84b7de33d\">perversely\nenjoy</a>.</p>\n<figure>\n<img\nsrc=\"http://photos.tylercipriani.com/2024-11-12_hledger-gnuplot-cashflow.png\"\nalt=\"Income vs. Expense, generated by piping hledger → gnuplot\" />\n<figcaption aria-hidden=\"true\">Income vs. Expense, generated by piping\nhledger → gnuplot</figcaption>\n</figure>\n<p>Here’s my setup:</p>\n<pre><code>$ tree ~/Documents/ledger\n.\n├── export\n│   ├── 2024-balance-sheet.txt\n│   └── 2024-income-statement.txt\n├── import\n│   ├── in\n│   │   ├── amazon\n│   │   │   └── order-history.csv\n│   │   ├── credit\n│   │   │   ├── 2024-01-01_2024-02-01.csv\n│   │   │   ├── ...\n│   │   │   └── 2024-10-01_2024-11-01.csv\n│   │   └── debit\n│   │       ├── 2024-01-01_2024-02-01.csv\n│   │       ├── ...\n│   │       └── 2024-10-01_2024-11-01.csv\n│   └── journal\n│       ├── amazon\n│       │   └── order-history.journal\n│       ├── credit\n│       │   ├── 2024-01-01_2024-02-01.journal\n│       │   ├── ...\n│       │   └── 2024-10-01_2024-11-01.journal\n│       └── debit\n│           ├── 2024-01-01_2024-02-01.journal\n│           ├── ...\n│           └── 2024-10-01_2024-11-01.journal\n├── rules\n│   ├── amazon\n│   │   └── journal.rules\n│   ├── credit\n│   │   └── journal.rules\n│   ├── debit\n│   │   └── journal.rules\n│   └── common.rules\n├── 2024.journal\n├── Makefile\n└── README</code></pre>\n<p>Process:</p>\n<ol type=\"1\">\n<li><strong>Import</strong> – download a CSV for the month from each\naccount and plop it into\n<code>import/in/&lt;account&gt;/&lt;dates&gt;.csv</code></li>\n<li><strong>Make</strong> – run <code>make</code></li>\n<li><strong>Squint</strong> – Look at <code>git diff</code>; if it looks\ngood, <code>git add . &amp;&amp; git commit -m \"💸\"</code> otherwise\nreview <code>hledger areg</code> to see details.</li>\n</ol>\n<p>The <code>Makefile</code> generates everything under\n<code>import/journal</code>:</p>\n<ul>\n<li>journal files from my CSVs using their corresponding rules.</li>\n<li>reports in the <code>export</code> folder</li>\n</ul>\n<p>I include all the journal files in the <code>2024.journal</code> with\nthe line: <code>include ./import/journal/*/*.journal</code></p>\n<p>Here’s the <code>Makefile</code>:</p>\n<div class=\"sourceCode\" id=\"cb11\"><pre\nclass=\"sourceCode Makefile\"><code class=\"sourceCode makefile\"><span id=\"cb11-1\"><a href=\"https://tylercipriani.com/blog/#cb11-1\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dt\">SHELL </span><span class=\"ch\">:=</span><span class=\"st\"> /bin/bash</span></span>\n<span id=\"cb11-2\"><a href=\"https://tylercipriani.com/blog/#cb11-2\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dt\">RAW_CSV </span><span class=\"ch\">=</span><span class=\"st\"> </span><span class=\"ch\">$(</span><span class=\"kw\">wildcard</span><span class=\"st\"> import/in/**/*.csv</span><span class=\"ch\">)</span></span>\n<span id=\"cb11-3\"><a href=\"https://tylercipriani.com/blog/#cb11-3\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dt\">JOURNALS </span><span class=\"ch\">=</span><span class=\"st\"> </span><span class=\"ch\">$(</span><span class=\"kw\">foreach</span><span class=\"st\"> file</span><span class=\"kw\">,</span><span class=\"ch\">$(</span><span class=\"dt\">RAW_CSV</span><span class=\"ch\">)</span><span class=\"kw\">,</span><span class=\"ch\">$(</span><span class=\"kw\">subst</span><span class=\"st\"> /in/</span><span class=\"kw\">,</span><span class=\"st\">/journal/</span><span class=\"kw\">,</span><span class=\"ch\">$(</span><span class=\"kw\">patsubst</span><span class=\"st\"> %.csv</span><span class=\"kw\">,</span><span class=\"st\">%.journal</span><span class=\"kw\">,</span><span class=\"ch\">$(</span><span class=\"dt\">file</span><span class=\"ch\">))))</span></span>\n<span id=\"cb11-4\"><a href=\"https://tylercipriani.com/blog/#cb11-4\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb11-5\"><a href=\"https://tylercipriani.com/blog/#cb11-5\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ot\">.PHONY:</span><span class=\"dt\"> all</span></span>\n<span id=\"cb11-6\"><a href=\"https://tylercipriani.com/blog/#cb11-6\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dv\">all:</span><span class=\"dt\"> </span><span class=\"ch\">$(</span><span class=\"dt\">JOURNALS</span><span class=\"ch\">)</span></span>\n<span id=\"cb11-7\"><a href=\"https://tylercipriani.com/blog/#cb11-7\" aria-hidden=\"true\" tabindex=\"-1\"></a>    hledger is -f 2024.journal &gt; export/2024-income-statement.txt</span>\n<span id=\"cb11-8\"><a href=\"https://tylercipriani.com/blog/#cb11-8\" aria-hidden=\"true\" tabindex=\"-1\"></a>    hledger bs -f 2024.journal &gt; export/2024-balance-sheet.txt</span>\n<span id=\"cb11-9\"><a href=\"https://tylercipriani.com/blog/#cb11-9\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb11-10\"><a href=\"https://tylercipriani.com/blog/#cb11-10\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"ot\">.PHONY</span> <span class=\"er\">clean</span></span>\n<span id=\"cb11-11\"><a href=\"https://tylercipriani.com/blog/#cb11-11\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dv\">clean:</span></span>\n<span id=\"cb11-12\"><a href=\"https://tylercipriani.com/blog/#cb11-12\" aria-hidden=\"true\" tabindex=\"-1\"></a>        rm -rf import/journal/**/*.journal</span>\n<span id=\"cb11-13\"><a href=\"https://tylercipriani.com/blog/#cb11-13\" aria-hidden=\"true\" tabindex=\"-1\"></a></span>\n<span id=\"cb11-14\"><a href=\"https://tylercipriani.com/blog/#cb11-14\" aria-hidden=\"true\" tabindex=\"-1\"></a><span class=\"dv\">import/journal/%.journal:</span><span class=\"dt\"> import/in/%.csv</span></span>\n<span id=\"cb11-15\"><a href=\"https://tylercipriani.com/blog/#cb11-15\" aria-hidden=\"true\" tabindex=\"-1\"></a>    <span class=\"ch\">@</span><span class=\"fu\">echo </span><span class=\"st\">&quot;Processing csv </span><span class=\"ch\">$&lt;</span><span class=\"st\"> to </span><span class=\"ch\">$@</span><span class=\"st\">&quot;</span></span>\n<span id=\"cb11-16\"><a href=\"https://tylercipriani.com/blog/#cb11-16\" aria-hidden=\"true\" tabindex=\"-1\"></a>    <span class=\"ch\">@</span><span class=\"fu\">echo </span><span class=\"st\">&quot;---&quot;</span></span>\n<span id=\"cb11-17\"><a href=\"https://tylercipriani.com/blog/#cb11-17\" aria-hidden=\"true\" tabindex=\"-1\"></a>    <span class=\"ch\">@</span><span class=\"fu\">mkdir -p </span><span class=\"ch\">$(</span><span class=\"kw\">shell</span><span class=\"st\"> dirname </span><span class=\"ch\">$@)</span></span>\n<span id=\"cb11-18\"><a href=\"https://tylercipriani.com/blog/#cb11-18\" aria-hidden=\"true\" tabindex=\"-1\"></a>    <span class=\"ch\">@</span><span class=\"fu\">hledger print --rules-file rules/</span><span class=\"ch\">$(</span><span class=\"kw\">shell</span><span class=\"st\"> basename </span><span class=\"ch\">$$</span><span class=\"st\">(dirname </span><span class=\"ch\">$&lt;)</span><span class=\"fu\">)/journal.rules -f </span><span class=\"st\">&quot;</span><span class=\"ch\">$&lt;</span><span class=\"st\">&quot;</span><span class=\"fu\"> &gt; </span><span class=\"st\">&quot;</span><span class=\"ch\">$@</span><span class=\"st\">&quot;</span></span></code></pre></div>\n<p>If I find anything amiss (e.g., if my balances are different than\nwhat the bank tells me), I look at <code>hleger areg</code>. I may tweak\nmy rules or my CSVs and then I run\n<code>make clean &amp;&amp; make</code> and try again.</p>\n<p>Simple, plain text accounting made simple.</p>\n<p>And if I ever want to dig deeper, <code>hledger</code>’s <a\nhref=\"https://hledger.org/1.40/hledger.html\">docs</a> have more to\nteach. But for now, the balance of effort vs. reward is perfect.</p>\n</section>\n<section class=\"footnotes footnotes-end-of-document\"\nrole=\"doc-endnotes\">\n<hr />\n<ol>\n<li id=\"fn1\" role=\"doc-endnote\"><p>while reading a blog post from <a\nhref=\"https://jmtd.net/log/hledger_1yr/\">Jonathan Dowland</a><a\nhref=\"https://tylercipriani.com/blog/#fnref1\" class=\"footnote-back\" role=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn2\" role=\"doc-endnote\"><p>Note, this is covered by <a\nhref=\"https://github.com/adept/full-fledged-hledger/wiki/Investments-easy-approach\">full-fledged\nhledger – Investements</a><a href=\"https://tylercipriani.com/blog/#fnref2\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n<li id=\"fn3\" role=\"doc-endnote\"><p>Also covered in <a\nhref=\"https://github.com/adept/full-fledged-hledger/wiki/Tax-returns\">full-fledged\nhledger – Tax returns</a><a href=\"https://tylercipriani.com/blog/#fnref3\" class=\"footnote-back\"\nrole=\"doc-backlink\">↩︎</a></p></li>\n</ol>\n</section>\n"

}

    ]
}
