To see posts by date, check out the archives

CI Files in Their Own Namespace
Tyler Cipriani Posted

I recently had a random idea to fix a problem that evidently only bothers me. It has become de rigueur to litter every project with a random smattering of top-level dotfiles. I’m thinking of the .travis.yaml-type files that are regarded as a necessary evil to perform some kind of testing or deployment in a particular repo.

I have designed several systems that use this trope. Yes, I feel appropriately ashamed.

My problem is simple: dotfiles clutter my work tree. Given all of the myriad problems with the current state of software “engineering” this particular issue may seem a relatively minor one; however, if this issue is minor, a minor fix may well be all that’s needed.

The idea I had recently is to use a ref namespace in git for these files.

Something like:

$ git checkout --orphan CI
$ git rm -rf .
$ mkdir travis
$ cat > travis/config.yaml
language: ruby
rvm:
 - 2.2
 - jruby

$ git commit -a -m 'Add travis config for testing'
$ git push origin refs/heads/CI:refs/meta/ci

In GitHub, using the example above, refs/meta/ci doesn’t appear in the branch dropdown in the web UI, but the ref is stored on the server and accessible by people fetching the repo.

In CI (or any tooling hoping to use these configuration files) this namespace can be easily accessed via:

$ git fetch origin refs/meta/ci:refs/meta/ci
$ git show refs/meta/ci:travis/config.yaml
language: ruby
rvm:
 - 2.2
 - jruby

A whole refs/meta/ci namespace with folders for each service seems so much nicer to me.

There are a few downsides I can think of to this method. Firstly, it’s git-specific; however, I am sure the concept of a branch is not entirely foreign to most version control systems and it could be supported to some degree. In practical terms, supporting git seems to be good enough for most tools.

Another downside is opacity. That is, CI files are less discoverable using this method; however, if this style of configuration file were to become the de facto standard then much tooling could be written to surface interfaces for these files and their namespaces.

Halloween Nerd Projects 👻
Tyler Cipriani Posted

Last year I got 229 trick-or-treaters at my house.

Candy, yo!
Candy, yo!

The previous year it was 194. The year before that it was 176. The year before that I lived in a duplex that was roughly two blocks from my current house; that year I didn’t think to track how many trick-or-treaters showed up at my door – it was less than 10.

The only logical conclusion we can draw from the above data is that our current house is imbued with dark Halloween voodoo. We’ve made every attempt to decorate accordingly. The first year it was spider webs and a black light. For every subsequent year we’ve tried to out-do our previous selves.

It begins

In order to create an authentic movie-experience at home I own a projector (enjoy with flavacol for extra authenticity). In 2015 I discovered AtmosFearFX which is a company that produces content meant to be projected onto walls or out of windows or onto curtains. For our front-room I project a ghostly apparition onto a sheer curtain so that it roughly looks like it’s walking around inside. This has been a big hit with trick-or-treaters, so much so that I bought a(n extremely crappy) mini projector that I would never be able to use for movies, but – none-the-less – works great for projecting weird zombie hands groping for freedom on the window of our porch.

This was the same year I began incorporating microcontrollers to trigger effects. I built a project with a Particle core attached to a reed switch on my front door. The Particle board sent a web request to my hue light bulb on the porch (scream sound warning):

Halloween 2015: Controlling Hue bulbs with a Particle and Reed Switch

Now

Fast-forward a few years and my setup has become even more elaborate. Sometime around 2016 I began fooling around with pneumatic pop-ups. After a few DuckDuckGos I stumbled on FrightProps and subsequently began spending too much money there.

My big project this year is a pneumatic pop-up that uses an air-compressor. This is controlled by a motion-sensing Raspberry Pi.

The key component is the solenoid:

3-way solenoid with 1/4" ports
3-way solenoid with 1/4" ports

This controls the 30PSI (or so) coming from my air compressor to a pneumatic cylinder. The solenoid is controlled via 12V. This is more than my Raspberry Pi is capable of pushing out over GPIO. Fortunately, I have an Adafruit MotorHAT. The MotorHAT has its own 12V power supply and can be controlled by a Raspberry Pi. I also used a PIR sensor to detect motion and trigger my popup.

Raspberry Pi with MotorHAT
Raspberry Pi with MotorHAT

The pop-up dummy itself is made out of 1/2" PVC tubing, zip ties, a skeleton I bought at Home Depot, and love.

PVC and love
PVC and love

I control the whole contraption with some python code running on the Pi. I’ve wrapped the Adafruit MotorHAT code in (what I think is) a nicer interface for this project:

Which lets me test-drive the setup pretty easily via ipython:

>>> from scary import Pneumatic
>>> motor_output = 4
>>> popup = Pneumatic()
>>> popup.up()
>>> popup.is_up
True
>>> popup.down()
Halloween 2018: Controlling a pneumatic with python
Git Advice
Tyler Cipriani Posted

Git can be confusing. It’s more confusing fumbling with git under pressure. As a release engineer fumbling with git under pressure is a appreciable chunk of my job; as such, I’ve learned a trick or two.

Recently I added some general git advice to our shared wiki page for deployers. I am reproducing this advice here for posterity. The goal of this advice is to ensure that deployers are seeing all the information they need to make smart decisions about the current state of a git repository. There are times when this advice has allowed me to figure out a problem with a git repository simply by cd-ing into its worktree.

  1. Use a git-aware prompt. The git-prompt.sh script that is included in git’s contrib tree is my preferred prompt. There are instructions for use in comments at the beginning of the file. One simple way to use it (on Debian machines anyway) is to add the following to your shell initialization file: {.bash} GIT_PS1_SHOWUNTRACKEDFILES=1 GIT_PS1_SHOWDIRTYSTATE=1 GIT_PS1_SHOWUPSTREAM="auto verbose" . /etc/bash_completion.d/git-prompt PS1='[\u@\h \W$(__git_ps1 " (%s)")]\$ '

  2. Set status.submoduleSummary. By default submodules have limited visibility in git status which makes it easy to miss a git submodule update step. After adding status.submoduleSummary to your ~/.gitconfig, git will show you a short summary of submodule changes in the output of git status. Set it by executing: {.shell-session} you@computer:~$ git config --global status.submoduleSummary true

Reverse Polish Notation, Lambdas, and Currying in Python
Tyler Cipriani Posted

I just finished reading The Python Corner’s post “Lambdas and Functions in Python.” The post acts as an introduction to the use of functions as first-class objects in python. The demo code is the implementation of a “Reverse Polish Notation” calculator.

I had never heard of reverse polish notation(RPN) before this post. The short explanation of RPN is available on Wikipedia:

In reverse Polish notation, the operators follow their operands; for instance, to add 3 and 4, one would write 3 4 + rather than 3 + 4.

In that blog post RPN is implemented as a stack of operands and after an operator is pushed onto the stack, the compute() method is called which triggers the evaluation of the lambda specified by the operator. Like this:

The point of the post is to show a python dict using operands as keys with lambdas as values; this demonstrates that lambdas are functions and functions are first-class objects. This allows the compute method of the RPNEngine class to look up a lambda in a dict, and pop() off the stack using the signature function of the inspect module to determine how many arguments are needed for a particular lambda. From there, lambda evaluation is handed off to helper functions named, for instance, compute_operation_with_two_operands and compute_operation_with_one_operand

Currying

One other functional concept that could have helped the example code is that of currying. Currying involves changing a function with multiple arity into a series of evaluations of multiple functions each with an arity of 1.

This is a fancy way to say:

By turning the compute_operation_with_n_operands-type functions into curried functions, the code gets much cleaner. That is, instead of a switch like:

You can implement a curried function using a callable python object and do something like:

func = self.catalog[operation]

while not func.resolved:
    func(self.pop())

This gets rid of the clunky compute_operation_with_n_operands functions. Here is the full code for a solution using currying:

#!/usr/bin/env python3
"""
Engine class for RPN Calculator
"""

import math

from functools import partial
from inspect import signature


class Curry(object):
    """
    Curry a callable

    Given a callable, returns a an object that can be used like a curried
    callable.

    >>> c1 = Curry(lambda x, y: x + y)
    >>> c2 = Curry(lambda x, y: x + y)
    >>> c1(2, 2) == c2(2)(2)
    True

    :func: callable
    """
    def __init__(self, func):
        self.func = func
        self.argc = len(signature(self.func).parameters)
        self.resolved = False
        self.answer = None

    def __call__(self, *args):
        if len(args) == self.argc:
            self.answer = self.func(*args)
            self.resolved = True

        for arg in args:
            self.func = partial(self.func, arg)
            self.argc = len(signature(self.func).parameters)

        return self


class RPNEngine(object):
    """
    Reverse Polish Notation (RPN) Engine

    A RPN calculator
    >>> rpn = RPNEngine()
    >>> rpn.push(2)
    >>> rpn.push(2)
    >>> rpn.compute('+') == 4
    True
    >>> rpn.compute('AC')
    >>> rpn.push(2)
    >>> rpn.compute('^2') == 4
    True
    """
    def __init__(self):
        self.stack = []
        self.functions = self._get_functions()

    def _get_functions(self):
        return {
            '+': Curry(lambda x, y: x + y),
            '-': Curry(lambda x, y: x - y),
            '*': Curry(lambda x, y: x * y),
            '/': Curry(lambda x, y: x / y),
            '^2': Curry(lambda x: x * x),
            "SQRT": Curry(lambda x: math.sqrt(x)),
            "C": Curry(lambda: self.stack.pop()),
            "AC": Curry(lambda: self.stack.clear()),
        }

    def push(self, item):
        self.stack.append(item)

    def pop(self):
        try:
            return self.stack.pop()
        except IndexError:
            pass

    def compute(self, operation):
        func = self.functions.get(operation)

        if not func:
            raise BaseException('%s not a valid function' % operation)

        if len(self.stack) < func.argc:
            raise BaseException(
                '%s requires %d operands, %d given' % (
                    operation,
                    func.argc,
                    len(self.stack)
                )
            )

        if func.argc == 0:
            func()

        while not func.resolved:
            func(self.pop())

        return func.answer

Reading the final code in the Python Corner post made me me really itchy to implement the solution I posted here.

Bloop! Audible Error Logs
Tyler Cipriani Posted

As a frequent deployer of software, I’m usually watching several different flavors of system monitoring all at the same time. A few logstash dashboards in a few browser windows, a few grafana dashboards, a few different logs being tailed in a few different tmux windows.

If it all goes wrong during a deployment – if the error log suddenly explodes – I want to know. Even if I’m looking at another metric, or at IRC, or at my email – I want to know about log explosion immediately.

So I made a thing that plays a little noise whenever a new line is recieved on stdin. I call it Bloop!

And I’ve been using it to tail logs like:

ssh logsever -- tail -f /var/log/important/error.log | bloop -s

So now my stomach drops when I hear, “bloop…bloop…blobloopboopbloop”.

I’m sure this thing has been build a million times because it’s a braindead-simple idea, but I couldn’t find the right words to search for it so I made new one. Now all I need is an EKG monitor so I can see how much this bloop thing makes my heart rate match the overall error rate.