To see posts by date, check out the archives

Git Advice
Tyler Cipriani Posted

Git can be confusing. It’s more confusing fumbling with git under pressure. As a release engineer fumbling with git under pressure is a appreciable chunk of my job; as such, I’ve learned a trick or two.

Recently I added some general git advice to our shared wiki page for deployers. I am reproducing this advice here for posterity. The goal of this advice is to ensure that deployers are seeing all the information they need to make smart decisions about the current state of a git repository. There are times when this advice has allowed me to figure out a problem with a git repository simply by cd-ing into its worktree.

  1. Use a git-aware prompt. The git-prompt.sh script that is included in git’s contrib tree is my preferred prompt. There are instructions for use in comments at the beginning of the file. One simple way to use it (on Debian machines anyway) is to add the following to your shell initialization file:

    GIT_PS1_SHOWUNTRACKEDFILES=1
    GIT_PS1_SHOWDIRTYSTATE=1
    GIT_PS1_SHOWUPSTREAM="auto verbose"
    . /etc/bash_completion.d/git-prompt
    PS1='[\u@\h \W$(__git_ps1 " (%s)")]\$ '
  2. Set status.submoduleSummary. By default submodules have limited visibility in git status which makes it easy to miss a git submodule update step. After adding status.submoduleSummary to your ~/.gitconfig, git will show you a short summary of submodule changes in the output of git status. Set it by executing:

    you@computer:~$ git config --global status.submoduleSummary true
Reverse Polish Notation, Lambdas, and Currying in Python
Tyler Cipriani Posted

I just finished reading The Python Corner’s post “Lambdas and Functions in Python.” The post acts as an introduction to the use of functions as first-class objects in python. The demo code is the implementation of a “Reverse Polish Notation” calculator.

I had never heard of reverse polish notation(RPN) before this post. The short explanation of RPN is available on Wikipedia:

In reverse Polish notation, the operators follow their operands; for instance, to add 3 and 4, one would write 3 4 + rather than 3 + 4.

In that blog post RPN is implemented as a stack of operands and after an operator is pushed onto the stack, the compute() method is called which triggers the evaluation of the lambda specified by the operator. Like this:

rpn = rpn_engine()
rpn.push(2)
rpn.push(2)
print(rpn.compute('+'))  # Prints 4

The point of the post is to show a python dict using operands as keys with lambdas as values; this demonstrates that lambdas are functions and functions are first-class objects. This allows the compute method of the RPNEngine class to look up a lambda in a dict, and pop() off the stack using the signature function of the inspect module to determine how many arguments are needed for a particular lambda. From there, lambda evaluation is handed off to helper functions named, for instance, compute_operation_with_two_operands and compute_operation_with_one_operand

Currying

One other functional concept that could have helped the example code is that of currying. Currying involves changing a function with multiple arity into a series of evaluations of multiple functions each with an arity of 1.

This is a fancy way to say:

add = lambda x, y: x + y
add_curried = lambda x: lambda y: x + y

assert(add(2, 2) == add_curried(2)(2))

By turning the compute_operation_with_n_operands-type functions into curried functions, the code gets much cleaner. That is, instead of a switch like:

if number_of_operands == 2:
    self.compute_operation_with_two_operands(self.catalog[operation])

if number_of_operands == 1:
    self.compute_operation_with_one_operand(self.catalog[operation])

You can implement a curried function using a callable python object and do something like:

func = self.catalog[operation]

while not func.resolved:
    func(self.pop())

This gets rid of the clunky compute_operation_with_n_operands functions. Here is the full code for a solution using currying:

#!/usr/bin/env python3
"""
Engine class for RPN Calculator
"""

import math

from functools import partial
from inspect import signature


class Curry(object):
    """
    Curry a callable

    Given a callable, returns a an object that can be used like a curried
    callable.

    >>> c1 = Curry(lambda x, y: x + y)
    >>> c2 = Curry(lambda x, y: x + y)
    >>> c1(2, 2) == c2(2)(2)
    True

    :func: callable
    """
    def __init__(self, func):
        self.func = func
        self.argc = len(signature(self.func).parameters)
        self.resolved = False
        self.answer = None

    def __call__(self, *args):
        if len(args) == self.argc:
            self.answer = self.func(*args)
            self.resolved = True

        for arg in args:
            self.func = partial(self.func, arg)
            self.argc = len(signature(self.func).parameters)

        return self


class RPNEngine(object):
    """
    Reverse Polish Notation (RPN) Engine

    A RPN calculator
    >>> rpn = RPNEngine()
    >>> rpn.push(2)
    >>> rpn.push(2)
    >>> rpn.compute('+') == 4
    True
    >>> rpn.compute('AC')
    >>> rpn.push(2)
    >>> rpn.compute('^2') == 4
    True
    """
    def __init__(self):
        self.stack = []
        self.functions = self._get_functions()

    def _get_functions(self):
        return {
            '+': Curry(lambda x, y: x + y),
            '-': Curry(lambda x, y: x - y),
            '*': Curry(lambda x, y: x * y),
            '/': Curry(lambda x, y: x / y),
            '^2': Curry(lambda x: x * x),
            "SQRT": Curry(lambda x: math.sqrt(x)),
            "C": Curry(lambda: self.stack.pop()),
            "AC": Curry(lambda: self.stack.clear()),
        }

    def push(self, item):
        self.stack.append(item)

    def pop(self):
        try:
            return self.stack.pop()
        except IndexError:
            pass

    def compute(self, operation):
        func = self.functions.get(operation)

        if not func:
            raise BaseException('%s not a valid function' % operation)

        if len(self.stack) < func.argc:
            raise BaseException(
                '%s requires %d operands, %d given' % (
                    operation,
                    func.argc,
                    len(self.stack)
                )
            )

        if func.argc == 0:
            func()

        while not func.resolved:
            func(self.pop())

        return func.answer

Reading the final code in the Python Corner post made me me really itchy to implement the solution I posted here.

Bloop! Audible Error Logs
Tyler Cipriani Posted

As a frequent deployer of software, I’m usually watching several different flavors of system monitoring all at the same time. A few logstash dashboards in a few browser windows, a few grafana dashboards, a few different logs being tailed in a few different tmux windows.

If it all goes wrong during a deployment – if the error log suddenly explodes – I want to know. Even if I’m looking at another metric, or at IRC, or at my email – I want to know about log explosion immediately.

So I made a thing that plays a little noise whenever a new line is recieved on stdin. I call it Bloop!

And I’ve been using it to tail logs like:

ssh logsever -- tail -f /var/log/important/error.log | bloop -s

So now my stomach drops when I hear, “bloop…bloop…blobloopboopbloop”.

I’m sure this thing has been build a million times because it’s a braindead-simple idea, but I couldn’t find the right words to search for it so I made new one. Now all I need is an EKG monitor so I can see how much this bloop thing makes my heart rate match the overall error rate.

Using AWK like grep with envvars
Tyler Cipriani Posted

This is a little tip that was impossible for me to DuckDuckGo (that’s right, I use DuckDuckGo like a verb). Like everyone who lives the daily joy (and, frequently, deep frustration) of using the command line to accomplish day-to-day tasks I am a fairly deft manipulator of strings. Often, I turn to AWK to achieve powerful results quickly.

Using AWK like grep

There is usually no reason to use grep with AWK. AWK lets you use /[whatever]/ to search for lines containing [whatever]. You can also use /! [whatever]/ to search for the opposite of [whatever]. After you’ve found the line you want to act upon, you can manipulate it further using the powerful functionality of AWK.

Today, however, I wanted to use AWK to find the rotation of my current display using xrandr. The name of current display device was stored in the environment variable $DISPLAY_DEVICE (and in this instance was LVDS-1). I could not figure out, nor could I find the right words to search for, how to do this.

I tried:

xrandr -q | awk -v device="$DISPLAY_DEVICE" '/device/ {print}'

I tried

xrandr -q | awk '/'"$DISPLAY_DEVICE"'/ {print}'

Finally, I figured it out! This is something I thought I should document for others using the same search terms (and for myself in a month):

The Magic

awk -v device="$DISPLAY_DEVICE" '$0 ~ device {print}'

To get the rotation of a device from xrandr if you know the device name it’s:

xrandr -q | awk -v device='LVDS-1' '$0 ~ device {gsub("([()]| primary)", ""); print $4}'

<3 AWK so darn much.

2017 by the numbers
Tyler Cipriani Posted

These are some meaningless numbers.

I felt pretty powerless against the rising tide of horribleness that seemingly permeated every aspect of 2017 – that is not captured by these numbers.