Bloop! Audible Error Logs
Tyler Cipriani Posted

As a frequent deployer of software, I’m usually watching several different flavors of system monitoring all at the same time. A few logstash dashboards in a few browser windows, a few grafana dashboards, a few different logs being tailed in a few different tmux windows.

If it all goes wrong during a deployment – if the error log suddenly explodes – I want to know. Even if I’m looking at another metric, or at IRC, or at my email – I want to know about log explosion immediately.

So I made a thing that plays a little noise whenever a new line is recieved on stdin. I call it Bloop!

And I’ve been using it to tail logs like:

ssh logsever -- tail -f /var/log/important/error.log | bloop -s

So now my stomach drops when I hear, “bloop…bloop…blobloopboopbloop”.

I’m sure this thing has been build a million times because it’s a braindead-simple idea, but I couldn’t find the right words to search for it so I made new one. Now all I need is an EKG monitor so I can see how much this bloop thing makes my heart rate match the overall error rate.

30
Posted

Conway’s Favicon

Posted

by

Tyler Cipriani

30
Posted
Bash Completion
Tyler Cipriani Posted

Basics

Most Debian installed bash completions live under /usr/share/bash_completion/completions

# the first argument ($1) is the name of the command whose  arguments are being completed
# the second argument ($2) is the word being completed,
# and the third argument ($3) is the word preceding the word  being  completed  on the current command line
# In the context of this function the following variables are defined:
# COMP_LINE, COMP_POINT, COMP_KEY, and COMP_TYPE
# as well as COMP_WORDS and COMP_CWORD
#  It must put the possible completions in the COMPREPLY array
# See bash(1) '^  Programmable Completion' for more information
_some_function() {
  local cur cmd
  cur=${COMP_WORDS[$COMP_CWORD]}
  cmd=( "${COMP_WORDS[@]}" )
}
complete -F _some_function command

complete options

Some complete options you want:

  • -o bashdefault - if no completions are found do the bash default thing
  • -o default - readline completions if both the complete function and bash expansions fail
  • -o nospace - don’t append a space at the end of matches (useful if you’re doing directory stuffs)
  • -S or -P - a prefix or suffix that is added at the end of a completion generated by the function passed to complete -F

Example

# Many of the ideas presented in this script were stolen
# in-part or wholesale from
# <https://github.com/git/git/blob/master/contrib/completion/git-completion.bash>
__scap_subcommands=
__scap_get_subcommands() {
    if [ -n "$__scap_subcommands" ]; then
        return
    fi

    __scap_subcommands=$(scap --_autocomplete)
}

_scap() {
    local cur cmd=() sub rep

    cur=${COMP_WORDS[$COMP_CWORD]}
    cmd=( "${COMP_WORDS[@]}" )

    if (( COMP_CWORD == 1 )); then
        __scap_get_subcommands
        rep=$( compgen -W "$__scap_subcommands" -- "$cur" )
        COMPREPLY=( $rep )
        return
    fi

    # limit the command to autocomplete to the first 3 words
    if (( COMP_CWORD >= 2 )); then
        # don't complete any sub-subcommands, only options
        if [[ -n "$cur" && "${cur:0:1}" != '-' ]]; then
            COMPREPLY=()
            return
        fi
        cmd=( "${COMP_WORDS[@]:0:3}" )
    fi

    # replace the last word in the command with '--_autocomplete'
    cmd[ $(( ${#cmd[@]} - 1 )) ]='--_autocomplete'

    rep=$( compgen -W "$( ${cmd[@]} )" -- "$cur" )

    COMPREPLY=( $rep )
}

# By default append nospace except when completion comes from _scap
complete -S' ' -o bashdefault -o default -o nospace -F _scap scap
27
Posted
Cucumber tests getting started
Tyler Cipriani Posted

https://gorails.com/setup/ubuntu/14.04

Check setup

  • ruby -v
  • gem list selenium
  • gem install selenium-webdriver
  • Firefox
  • IRB

Ruby install

 curl -fsSL https://gist.github.com/mislav/055441129184a1512bb5.txt | \
  rbenv install --patch 2.2.3
~ 6m 50s
 rbenv global 2.2.2
~
 ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
~
 echo "gem: --no-ri --no-rdoc" > ~/.gemrc
~

Selenium webdriver

require "selenium-webdriver" https://code.google.com/p/selenium/wiki/RubyBindings

driver.class http://www.rubydoc.info/gems/selenium-webdriver/Selenium/WebDriver/Driver

Core tests

[mw-core]/tests/browser/features/create_account.feature

Schenario Outline can be passed variable

Scenario: [English text] # Just a comment, nothing is parsed
Given [this thing] # Sets up the test
When [other thing] # Action that is part of the test
And
But

Bundler

cd [mwcore]/tests/browser
bundle install
bundle exec cucumber features/create_account.feature:14

Important

export BROWSER=firefox
export MEDIAWIKI_ENVIRONMENT=mw-vagrant-host
License: Creative Commons Attribution-ShareAlike License
26
Posted
Bitlebee/IRC info
Tyler Cipriani Posted

Helpful Posts

- pthree.org post

  • Datko's book has this same kinda stuff.

Restart bitlbee without restarting Weechat

sudo service bitlbee stop
/disconnect localhost
sudo service bitlbee start
/connect localhost/6667
/msg &bitlbee identify [password]

Setup OTR

First had to install the bitlbee-plugin-otr plugin:

sudo apt-get install bitlbee-plugin-otr

In the &bitlebee window

account list
> @root 0 (gtalk): jabber, thcipriani@gmail.com (connected)
> @root 1 (twitter): twitter, thcipriani (connected)

Will return a list of accounts that are currently setup in bitlbee.

To generate a key for the 0th account—gmail in my case:

otr keygen 0
License: Creative Commons Attribution-ShareAlike License
VHS to MP4 on Linux
Tyler Cipriani Posted

I recently ordered a set of videos that I remember from a few years ago that did not make the jump to DVD, unfortunately.

I do have a VCR; however, I never have it hooked up because…why would I?

The solution here: convert my VHS tapes into H.264/MP3 mp4-contained files.

Now the question is: how?

Hardware

I managed to grab an EasyCap D60 Recording device from Amazon.

This device is supported inside the linux kernel (from version 3.18 forward…maybe?)

Once I plugged in this device, it was working:

lsusb

This bad boy:

Bus 1 Device 016: ID 1b71:3002 Fushicai USBTV007 Video Grabber [EasyCAP]

I checked out:

ls /dev | grep -i video

And I noticed a new video device video1. Easy.

Capture Software

I used VLC to caputre raw input.

  1. MediaOpen Caputre Device
  2. Video Device Name/dev/video1
  3. Audio Device Namehw:2,0
  4. Play pulldown menu → Convert
  5. Dump Raw Input
  6. Destination File/home/tyler/Videos
  7. Start
  8. Hit play on the VCR
  9. Hit the Rec. button in VLC

The auto-named avi file in ~/Videos was FUCKING HUGE.

ls -lh ~/Videos | grep -i avi

Conversion

I found a blog where a person does this. I have a vauge memory about doing this at UpSync, so I'll give it a shot: 2-pass mp4 conversion.

Let's see what happens!

ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p -preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 1 -f mp4 -y /dev/null
ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p \
  -preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 2 ~/Videos/out.mp4

The settings above created an mp4 that could be played via x264 on a RaspberryPi 3.

License: Creative Commons Attribution-ShareAlike License
26
Posted

Alt-Numpad for Linux

Posted

by

Tyler Cipriani

26
Posted

(ab)Using Git Notes

Posted

by

Tyler Cipriani

25
Posted

To see posts by date, check out the archives

The 14KB in the TCP Initial Window
Tyler Cipriani Posted

Today I went to coffee with a buddy of mine from $DAY_JOB - 2. During coffee we were discussing all the folk theories of “fast” websites. I made some off-the-cuff remark that I didn’t really understand – that you should try to squeeze the most important bits of your website into the first X bytes (X being a value I didn’t remember) for an opaque reason that has to do with TCP congestion algorithms that I’ve never understood.

We spent a good chunk of time looking at Wireshark captures of an HTTP request. It got muddled and we got off-track because we were looking at an HTTPS request and we started talking about TLS1.2, CLIENT-HELLO, and the TLS handshake in general.

I eventually found the source for what I was yammering about.

Due to how TCP estimates the capacity of a connection (i.e. TCP Slow Start), a new TCP connection cannot immediately use the full available bandwidth between the client and the server. Because of this, the server can send up to 10 TCP packets on a new connection (~14KB) in first roundtrip, and then it must wait for client to acknowledge this data before it can grow its congestion window and proceed to deliver more data.

After I got home and ate some ice cream, I did some more reading trying to understand this stuff. I think a lot of what was confusing about the Wireshark stuff we were looking at is it also had TLS1.2 mixed in there. Requesting a plain-old HTTP, port 80 site made looking at packets a bit more sane.

It seems like the size of a TCP packet is negotiated during the SYN, SYN-ACK phase of the TCP three-way handshake. In the Options section of the TCP segment (for a segment that has the SYN-flag set in the TCP segment header) a sender or receiver may indicate a Max Segment Size (MSS). An MSS is derived from a machine’s Maximum Transmission Unit (MTU). In the case of my machine the MTU is set to 1500. I found this via the ip command:

tyler@magneto:~$ ip -4 link show dev wlp3s0
2: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000

The MSS is derived from the MTU and is (at most?) the MTU minus the 40 bytes used for the TCP header and the IP header, so my machine sent the MSS size of 1460 to the server in the Options section of the SYN TCP segment that it initially sent my webserver.

Client MSS

So the data portion of a TCP segment sent in this particular connection should never exceed 1460 bytes.

While we were looking at Wireshark packet captures we kept noticing a TCP segment size of 1486 in the “length” column with a TCP segment data length of 1432. I noticed that my webserver sent an MSS size of 1432, which I suppose became the limiting factor in how much data could be sent per segment (rather than the 1460 MSS sent by my client – the browser):

Server MSS

Each TCP segment had a header-size of 20 bytes, so that leaves 34 unexplained bytes which (I guess) were part of the IP header.

So why try to squeeze a website into ~14KB? Seems like you should be trying to squeeze it into 1432 bytes.

I found my answer in IETF RFC 6928 – a proposal to increase the TCP Initial Window to 10 segments. The Initial Congestion Window (cwnd) is the maximum number of bytes that a server can send without receiving any acknowledgement that the bytes were received via a client ACK packet. This maximum number of bytes to send is calculated by multiplying some number by MSS. As of RFC 6928 some number is equal to 10 (initially).

I checked the value of cwnd on some established connections on my server:

tyler@magneto:~$ sudo ss -ti
    ...cwnd:10...

And it is indeed set to 10.

This means that my server can send cwnd * MSS bytes of information before the client has to acknowledge that anything has been receieved. So, in the case of the Wireshark connection I recorded, my webserver can send 1432 * 10 bytes, or 14320 bytes:

tyler@magneto:~$ units
Currency exchange rates from www.timegenie.com on 2016-06-22
202926 units, 109 prefixes, 88 nonlinear units

You have: 14320 bytes
You want: kilobytes
        * 14.32
        / 0.069832402

OR around 14KB of data can be sent to a browser before another round-trip from the client to the webserver has to happen.

So, overall, a nice, relaxing coffee and catch-up session with an old coworker.

25
Posted
GNU Autotools
Tyler Cipriani Posted

configure.ac

  • Written in m4sh (m4 + shell)
  • Program initialization: AC_INIT macro—needs name, version, maintainer
  • Initialize automake: AM_INIT_AUTOMAKE
  • Make sure that C-compiler exists: AC_PROG_CC
  • Build Makefile from Makefile.in, replaces @PACKAGE_VERSION@-type variables: AC_CONFIG_FILES([Makefile])
  • Output the script: AC_OUTPUT
AC_INIT([hello-world], [0.1], [tyler@tylercipriani.com])

# Initialize automake
AM_INIT_AUTOMAKE

# Ensure C-compiler exists
AC_PROG_CC

# Specify what files are to be created
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

Makefile.am

  • The ./configure script (created by configure.ac), expects to find Makefile.in
  • Makefile.in is long and dumb (just like ./configure), so we make a Makefile.am
  • automake generates Makefile.in
  • foreign AUTOMAKEOPTIONS tells that the layout is "non standard"
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = hello-world
hello-world_SOURCES = hello-world.c

Put it all together:

  • create m4 environment: aclocal
  • autoconf does: configure.acconfigure
  • automake --add-missing does: Makefile.amMakefile.in
  • autoreconf --install: autoreconf runs autoconf, autoheader, aclocal, automake, libtoolize, and autopoint
License: Creative Commons Attribution-ShareAlike License
Ikiwiki things
Tyler Cipriani Posted

Install

Mostly it was pretty easy

tar xvzf ikiwiki_3.20160121.tar.gz
cd ikiwiki_3.20160121
sudo PERL5LIB=`pwd` PERL_MM_USE_DEFAULT=1 perl -MCPAN -e 'CPAN::Shell->install("Bundle::IkiWiki")'
sudo PERL5LIB=`pwd` PERL_MM_USE_DEFAULT=1 perl -MCPAN -e 'CPAN::Shell->install("Bundle::IkiWiki::Extras")'
make
sudo make install

OpenID Setup

Ikiwiki requires an OpenID setup.

You can login to Ikiwiki with OpenID, and I think that's neat!

Since OpenID is somewhat dead, I decided to setup a personal SimpleID.

I put this all on the analytics server.

First I had to create a DNS record for openid.tylercipriani.com so letsencrypt would work.

Then I made a new letsencrypt cert:

/usr/local/src/letsencrypt/letsencrypt-auto certonly --standalone -d openid.tylercipriani.com

Then I grabbed and installed openid

cd /usr/local/src
curl -sLO http://downloads.sourceforge.net/simpleid/simpleid-1.0.1.tar.gz
tar xvzf simpleid-1.0.1.tar.gz
mkdir /srv/www/openid.tylercipriani.com
mv simpleid/{cache,identities,store,www} /srv/www/openid.tylercipriani.com
mv /srv/www/openid.tylercipriani.com/{config.php.dist,config.php}

Edited the config.php file, changed root webserver and the pretty url thing

I also made a virtualserver: openid.tylercipriani.com

Identity file: thcipriani.identity

identity="https://tylercipriani.com/"
password="blah:sha1:salt"
administrator=1
[user_info]
...
[sreg]
...

Blog setup

The things that I didn't understand about Ikiwiki:

  1. htmlscrubber removes all your scripts
  2. Some changes to the setup file (like git_wrapper updates) require you to run ikiwiki --changesetup [setupfile]
  3. I am running a staging wiki on my laptop, a wiki on my webserver, and then pushing the compiled files to s3: WAT‽
License: Creative Commons Attribution-ShareAlike License
Mutt
Tyler Cipriani Posted

Contacts

Install goobook:

sudo pip2 install -U goobook

I ran into some problems in arch, had to upgrade/reinstall a few deps:

(ノ^ヮ^)ノ*:・゚✧ sudo pip2 install -U pyasn1
(ノ^ヮ^)ノ*:・゚✧ sudo pip2 install -U cryptography

goobook then needs to authenticate with gmail:

goobook authenticate

Once you go through the oauth song and dance you should be able to run a query

(ノ^ヮ^)ノ*:・゚✧ goobook query tyler

thcipriani@gmail.com    Tyler Cipriani  other groups: "System Group: Friends"

From there, just need to add a bit of magic to the ~/.muttrc file

set query_command="goobook query %s"
macro index,pager a \
    "<pipe-message>goobook add<return>" "add sender to google contacts"
bind editor <Tab> complete-query

Viewing URLs

Install urlview: ncurses program parses urls out of text.

Create a $HOME/urlview file:

COMMAND xdg-open %s &

Bind \,u in the editor:

macro pager ,u "|urlview<enter>" "call urlview to open links"

Searching Mail

Searching works really well via notmuch

Afer install, setup via

notmuch setup

After setup, it's just a matter of calling

notmuch new

to index the things

License: Creative Commons Attribution-ShareAlike License
24
Posted
AWS Cloudfront SSL Cert
Tyler Cipriani Posted

Setup an Comodo positive SSL cert on AWS Cloudfront

  1. Buy Comodo positive SSL cert via namecheap

  2. Inspect last years cert:

    openssl req -noout -text -in tylercipriani_com.csr
  3. Generate private key and signing request

  • You can generate a new key as a separate step, or as part of the req command

  • To generate a new key manually use:

    openssl genrsa -out /path/to/output.key 2048
    mkdir -p ~/Projects/Certs/com.tylercipriani/2015-11-24
    cd ~/Projects/Certs/com.tylercipriani/2015-11-24
  • To generate a new key as part of a signing reuest use:

    openssl req -sha256 -new -newkey rsa:2048 -nodes -keyout /path/to/output.key -out /path/to/output.csr
         ensures output key is not encrypted ────┘
  • Answer lots of questions

    Country Name (2 letter code) [XX]:US
    State or Province Name (full name) []:Colorado
    Locality Name (eg, city) [Default City]:Longmont
    Organization Name (eg, company) [Default Company Ltd]:Tyler Cipriani
    Organizational Unit Name (eg, section) []:.
    Common Name (eg, your name or your server's hostname) []:tylercipriani.com
    Email Address []:spam@tylercipriani.com
    A challenge password []:
    • What is a challenge password: https://tools.ietf.org/html/rfc2985#page-16
  1. Should now have a tylercipriani_com.csr and an tylercipriani_com.key

  2. Do the namecheap needful:

    • Login to namecheap and activate your certificate
    • xsel -p < tylercipriani_com.csr then paste in the “Enter CSR” field
    • Choose, “Apache, Nginx, or Other” as the server type
    • Verify the domain, click “Next”
    • Verify CSR info, click “Next”
    • Select “Email-based” Domain Control Validation (DCV) method
    • Company contacts page, “NA” as company name, my address in address, tyler@tylercipriani.com in admin email
  3. You’ll get an email that asks you to enter a validation into a Comodo site, do that

  4. You’ll get an email with tylercipriani_com.zip

  5. aws configure to ensure that your awscli is setup

  6. Use awscli to upload

    awscli iam upload-server-certificate \
      --server-certificate-name tylercipriani_com_2015-11-24 \
      --certificate-body file://tylercipriani_com.crt \
      --private-key file://tylercipriani_com.key \
      --certificate-chain file://tylercipriani_com.ca-bundle \
      --path /cloudfront/
  7. Login to your aws dashboard, click cloudfront, go do Distribution Settings, click edit, find the new ssl key name in the dropdown and click, “Yes, edit”

  8. Should work…

23
Posted

Updates around here

Posted

by

Tyler Cipriani

21
Posted

To see posts by date, check out the archives

2016
Posted

Book Review: CODE by Charles Petzold

Posted

by

Tyler Cipriani

Coreboot on the ThinkPad X220 with a Raspberry Pi

Posted

by

Tyler Cipriani

My Remote Desk

Posted

by

Tyler Cipriani

Book Review: The Tombs of Atuan by Ursula K Le Guin

Posted

by

Tyler Cipriani

Git Annex Metadata Filtered Views

Posted

by

Tyler Cipriani

The 14KB in the TCP Initial Window

Posted

by

Tyler Cipriani

Git-Config Colors and Include

Posted

by

Tyler Cipriani

Why I Will Continue to Use Linode

Posted

by

Tyler Cipriani

What is Code Review?

Posted

by

Tyler Cipriani

ImageMagick to Extract a PDF Page

Posted

by

Tyler Cipriani

2015
Posted

GPG Things

Posted

by

Tyler Cipriani

AWS Cloudfront SSL Cert

Posted

by

Tyler Cipriani

Asus debian jessie install

Posted

by

Tyler Cipriani

Cucumber tests getting started

Posted

by

Tyler Cipriani

Puppet, apt-get-update, source-lists, and Dumbness

Posted

by

Tyler Cipriani

Git annex notes

Posted

by

Tyler Cipriani

Toward a More Useful X Keyboard

Posted

by

Tyler Cipriani

2014
Posted

Conway’s Favicon

Posted

by

Tyler Cipriani

Cross-Browser JavaScript Scrollbar Detection

Posted

by

Tyler Cipriani

Development Environments with Vagrant, Docker, and Supervisord

Posted

by

Tyler Cipriani

Create a Baller/Useful MOTD with ANSI Art

Posted

by

Tyler Cipriani

Replacing Jekyll with Pandoc and a Makefile

Posted

by

Tyler Cipriani

2012
Posted

My ZSH (and bash) prompt

Posted

by

Tyler Cipriani

Integrating git with your existing website

Posted

by

Tyler Cipriani

Using Amazon Product Advertising API to Grab Album Art

Posted

by

Tyler Cipriani

CSS display inline-block Extra Margin/Space

Posted

by

Tyler Cipriani

Website Contrast Ratio in Practice

Posted

by

Tyler Cipriani

Alt-Numpad for Linux

Posted

by

Tyler Cipriani

20
Posted
Books Read (so far) in 2016
Tyler Cipriani Posted

The goal here is to hit 2 books a month, so 24 for the year. I’m on track as of now. Perilous.

  1. Stranger in a Strange Land by Robert A. Heinlein
  2. Black Hole by Charles Burns
  3. Fun Home by Alison Bechdel
  4. The Girl on the Train by Paula Hawkins
  5. Starship Troopers by Robert A. Heinlein
  6. Ready Player One by Ernest Cline
  7. The Circle by Dave Eggers
  8. Notorious RBG by Irin Carmon and Shana Knizhnik
  9. The Three-Body Problem by Cixin Liu
  10. A Wizard of Earthsea by Ursula K. Le Guin
  11. Stoner by John Williams
  12. Prost! The Story of German Beer by Horst D. Dornbusch
  13. Bock by Darryl Richman
  14. Do Androids Dream of Electric Sheep by Philip K. Dick
  15. Room by Emma Donoghue
  16. Kitchen Confidential: adventures in the culinary underbelly by Anthony Bourdain
  17. H is for Hawk by Helen Macdonald
Library Thing(s)
Tyler Cipriani Posted
Earlier this year, I stumbled across Library Thing and almost immediately decided that I wanted to join—which is a rare thing for me concerning online services. So I did what any normal person would do and I bought a barcode scanner and wrote some python to build my catalog. The python script is just a loop that shows a prompt and waits for an ISBN, t to input a tag, q to quit. Meanwhile it's generating a csv file that can be uploaded to LibraryThing. The fruits of this labor were/are my (and Blazey's) Library Thing Catalog
18
Posted

To see posts by date, check out the archives

15
Posted
13
Posted
13
Posted
Git annex notes
Tyler Cipriani Posted

Setup New repo

  1. init git:

    git init
  2. init git annex:

    git annex init local
  3. Add S3 Remote named public-s3

    git annex initremote public-s3 type=S3 encryption=none bucket=tyler.zone chunk=0
  4. Add file to git annex

    git annex add [big-file]
  5. copy current directory to public-s3 remote

    git annex copy --to public-s3
  6. remove local copy—will only remove if already in remote—nothing lost

    git annex drop [file]
  7. Commit

    git -a -m 'initial commit'

Setup NFS as remote using Rsync over ssh

  1. Add Rsync as unencrypted remote

    git annex initremote nas type=rsync rsyncurl=nfs.blazeyandltyler.com:/volume1/homes/tyler/PicturesAnnex encryption=none
  2. Add file to nas

    git annex copy --to=nas

Pull down photos

eval (~/.amazonrc)
git annex enableremote public-s3
git annex get [whatever]

Make available to the public (after public s3 setup):

git annex addurl --file [filename] "http://tyler.zone/$(git annex lookupkey [filename])"

List remotes and remote info

Show remotes

git annex enableremote

Show remote info

git annex info [remote-name]
git annex info tylercipriani-raw

Finding info about files

Find a file:

git annex whereis [filename]

Adding a file to http://tyler.zone

eval $(~/.amazonrc)
git annex copy --to=photo-site [file]
git annex addurl --file [file] "http://tyler.zone/$(git annex lookupkey [filename])"

Sharing via Amazon s3

Setup

Make amazon bucket with policy:

{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "AllowPublicRead",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::[bucket-name]/*"
    }
  ]
}

Put creds out into the environment

eval (~/.amazonrc)
cd into repo:
cd /home/tyler/Projects/Timelapse
License: Creative Commons Attribution-ShareAlike License
12
Posted
GPG Things
Tyler Cipriani Posted

Get your fingerprint:

gpg --list-secret-keys --fingerprint

Get someone elses key

Using email

gpg2 --search-keys EMAIL

Using keyid

gpg2 --recv-key KEYID

Sign keys

gpg2 --recv-key KEYID
gpg2 --sign-key KEYID
gpg2 --send-key KEYID

Photos

View someone's photo

gpg --edit-key KEYID showphoto

Add a photo

Suggested image sizes are 240x288 pixels (GnuPG) or 120x144 pixels (PGP) to make a JPEG of 4K-6K in size. A 1.2 h:w ratio w = h / (144 / 120)

gpg --edit-key KEYID addphoto

Keysigning party etherpad

To Sign keys

gpg2 --recv-key KEYID
gpg2 --sign-key KEYID
gpg2 --send-key KEYID

Script to quickly sign all keys

 for k in $(grep pub ksp-wmf-20160108.txt | awk '{print $2}' | awk -F '/' '{print $2}'); do
    echo "Receiving $k"
    gpg2 --recv-key $k
    echo "Signing key $k"
    gpg2 --sign-key $k
    echo "Sending key $k"
    gpg2 --send-key $k
done

IF YOU NEED A KEY, GET HELP!!

Resources

PGP/GPG Intro

OpenPGP Best practices

WMF Keysigning

https://people.wikimedia.org/~faidon/ksp-wmf-20160108.txt

SHA256 Checksum: 52F2 CF39   6A54 6D56   7F55 7138   7264 11BA
                    AEE3 F34E   8880 681C   9A67 75D1   3BBC 74DC              [ ]
                    52F2CF39 6A546D56 7F557138 726411BA AEE3F34E 8880681C 9A6775D1 3BBC74DC

                    RIPEMD160 Checksum: BB44 91B6 0A4D 5865 2105    2A4C 19FB 11AD 8BCC C3C3    [ ]
                    BB44 91B6 0A4D 5865 2105  2A4C 19FB 11AD 8BCC C3C3

Mutt problems

"can't query passphrase in batch mode"

Uncomment, use agent in ~/.gnupg/gpg.conf

Pass problems

No Secret Key

Problem

gpg: decryption failed: No secret key

Solution

  1. install pinentry-curses
  2. ~/.gnupg/gpg-agent.conf
pinentry-program /usr/bin/pinentry
  1. gpg-connect-agent reloadagent /bye
License: Creative Commons Attribution-ShareAlike License
11
Posted

Conway’s Favicon

Posted

by

Tyler Cipriani

11
Posted
AWS Cloudfront SSL Cert
Tyler Cipriani Posted

Setup an Comodo positive SSL cert on AWS Cloudfront

  1. Buy Comodo positive SSL cert via namecheap

  2. Inspect last years cert:

    openssl req -noout -text -in tylercipriani_com.csr
  3. Generate private key and signing request

  • You can generate a new key as a separate step, or as part of the req command

  • To generate a new key manually use:

    openssl genrsa -out /path/to/output.key 2048
    mkdir -p ~/Projects/Certs/com.tylercipriani/2015-11-24
    cd ~/Projects/Certs/com.tylercipriani/2015-11-24
  • To generate a new key as part of a signing reuest use:

    openssl req -sha256 -new -newkey rsa:2048 -nodes -keyout /path/to/output.key -out /path/to/output.csr
         ensures output key is not encrypted ────┘
  • Answer lots of questions

    Country Name (2 letter code) [XX]:US
    State or Province Name (full name) []:Colorado
    Locality Name (eg, city) [Default City]:Longmont
    Organization Name (eg, company) [Default Company Ltd]:Tyler Cipriani
    Organizational Unit Name (eg, section) []:.
    Common Name (eg, your name or your server's hostname) []:tylercipriani.com
    Email Address []:spam@tylercipriani.com
    A challenge password []:
    • What is a challenge password: https://tools.ietf.org/html/rfc2985#page-16
  1. Should now have a tylercipriani_com.csr and an tylercipriani_com.key

  2. Do the namecheap needful:

    • Login to namecheap and activate your certificate
    • xsel -p < tylercipriani_com.csr then paste in the “Enter CSR” field
    • Choose, “Apache, Nginx, or Other” as the server type
    • Verify the domain, click “Next”
    • Verify CSR info, click “Next”
    • Select “Email-based” Domain Control Validation (DCV) method
    • Company contacts page, “NA” as company name, my address in address, tyler@tylercipriani.com in admin email
  3. You’ll get an email that asks you to enter a validation into a Comodo site, do that

  4. You’ll get an email with tylercipriani_com.zip

  5. aws configure to ensure that your awscli is setup

  6. Use awscli to upload

    awscli iam upload-server-certificate \
      --server-certificate-name tylercipriani_com_2015-11-24 \
      --certificate-body file://tylercipriani_com.crt \
      --private-key file://tylercipriani_com.key \
      --certificate-chain file://tylercipriani_com.ca-bundle \
      --path /cloudfront/
  7. Login to your aws dashboard, click cloudfront, go do Distribution Settings, click edit, find the new ssl key name in the dropdown and click, “Yes, edit”

  8. Should work…

10
Posted

Dotfile Boilerplate

Posted

by

Tyler Cipriani

09
Posted

Git Annex Metadata Filtered Views

Posted

by

Tyler Cipriani

The 14KB in the TCP Initial Window

Posted

by

Tyler Cipriani

Git-Config Colors and Include

Posted

by

Tyler Cipriani

Why I Will Continue to Use Linode

Posted

by

Tyler Cipriani

What is Code Review?

Posted

by

Tyler Cipriani

ImageMagick to Extract a PDF Page

Posted

by

Tyler Cipriani

Git Revision Colon Slash

Posted

by

Tyler Cipriani

Delete Local Git Commits

Posted

by

Tyler Cipriani

sensible-editor

Posted

by

Tyler Cipriani

GPG Security things

Posted

by

Tyler Cipriani

08
Posted

Dotfile Boilerplate

Posted

by

Tyler Cipriani

08
Posted
Asus debian jessie install
Tyler Cipriani Posted
  1. Reboot, hitting F2 over-and-over

  2. Turn off Fastboot and Secureboot in UEFI

  3. Download Non-free firmware

  4. Extract firmware from deb

    ar xv firmware-iwlwifi.deb tar xvf data.tar.xz

  5. Copy to ext4 formatted sd-card

    mkfs.ext4 /dev/[sd-card] mount -t ext4 /dev/[sd-card] /mnt/sd-card cp -r lib/firmware/* /mnt/sd-card

  6. Insert Install Media

  7. Setup installer to recognize dmraid

  8. Detect network hardware

    • Ctrl-Alt F2
    • mkdir /lib/firmware
    • blkid (to list mounts)
    • mount -t ext4 /dev/sdd1 /lib/firmware
    • modprobe ahci
    • Ctrl-Alt F1
  9. Manual partition

    https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot

  10. Run live disk, chroot, fuck around

    http://forums.debian.net/viewtopic.php?f=17&t=116036

Troubleshooting:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711147

  • Show root partition:

    cat /proc/cmdline
  • initramfs modules

    /etc/initramfs-tools/modules update-initramfs –k all –u

  • Large xterm:

    xterm -fn 10x20

  • chroot

    boot ubuntu live sudo -i cryptsetup luksOpen /dev/mapper/isw_cejababahfd_ASUS_03 cryptroot lvm vgscan lvm vgchange -ay mount /dev/mapper/taskmaster–vg-root /mnt mount /dev/mapper/isw_cejababahfd_ASUS_02 /mnt/boot mount -o bind /sys /mnt/sys mount -t proc /proc /mnt/proc mount -o bind /dev /mnt/dev chroot /mnt

  • chroot & install dmraid

    chroot apt-get install dmraid initramfs modules

  • break raid:

http://ubuntuforums.org/showthread.php?t=2193133&page=2&p=12913689#post12913689

For realz Steps!

  1. Download Ubuntu 14.04 Live OS and Debian 8.1.0 netinst

  2. Copy to a Ubuntu USB Drive:

    • use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
    • dd bs=4M ~/Downloads/ubuntu-14.04.2-desktop-amd64.iso of=/dev/[usbdrive-without-number]
  3. Download Non-free firmware

  4. Extract firmware from deb

    • ar xv firmware-iwlwifi.deb
    • tar xvf data.tar.xz
  5. Copy to ext4 formatted sd-card

    • mkfs.ext4 /dev/[sd-card]
    • mount -t ext4 /dev/[sd-card] /mnt/sd-card
    • cp -r lib/firmware/* /mnt/sd-card
  6. Boot Ubuntu from usb/change UEFI opts

    • Insert Ubuntu USB drive in Asus, Boot while holding F2
    • Boot → Fastboot [Disabled]
    • Security → Secure Boot Menu → Secure Boot Control [Disabled]
    • Advanced → SATA Configuration → SATA Mode Selection [AHCI]
    • Boot → Boot Option Priorites (hit + until USB drive is at the top)
    • F10 (save and exit)
  7. Try Ubuntu without installing

  8. Enable network, start terminal (Start → uxterm → xterm -fn 10x20)

  9. sudo gparted, delete all partitions on /dev/sda /dev/sdb

  10. Create partition table on /dev/sdb:

    • Select /dev/sdb
    • Device → Create Partition Table…
  11. Shutdown, remove usb

  12. Copy Debian netinst to USB Drive:

    • use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
    • sudo dd bs=4M if=Downloads/debian-8.0.0-amd64-netinst.iso of=[usbdrive-without-number]
  13. Detect network hardware

    • Ctrl-Alt F2
    • mkdir /lib/firmware
    • blkid (to list mounts)
    • mount -t ext4 /dev/sdd1 /lib/firmware
    • modprobe ahci
    • Ctrl-Alt F1
  14. Partition Disks

    • Manual
      1. sda & sbd delete existing partitions
      2. sda1 256MB EFI System partition
      3. sda2 512MB mountat /boot ext4
      4. sda3 REMAINDER physical volume for raid
      5. sdb1 ALL physical volument for raid
      6. configure software raid, raid0 [x]sda3 [x]sdb1
      7. raid0 disk1 use as physical volume for encryption
      8. configure encrypted volumes [x]/dev/md0
      9. encrypted-disk 1: use as physical volume for lvm
      10. configure lvm volume group
      11. 20 GB / ; 4 GB swap ; remainder /home
  15. Post xmonad-install disable GDM login:

06
Posted
Asus debian jessie install
Tyler Cipriani Posted
  1. Reboot, hitting F2 over-and-over

  2. Turn off Fastboot and Secureboot in UEFI

  3. Download Non-free firmware

  4. Extract firmware from deb

    ar xv firmware-iwlwifi.deb tar xvf data.tar.xz

  5. Copy to ext4 formatted sd-card

    mkfs.ext4 /dev/[sd-card] mount -t ext4 /dev/[sd-card] /mnt/sd-card cp -r lib/firmware/* /mnt/sd-card

  6. Insert Install Media

  7. Setup installer to recognize dmraid

  8. Detect network hardware

    • Ctrl-Alt F2
    • mkdir /lib/firmware
    • blkid (to list mounts)
    • mount -t ext4 /dev/sdd1 /lib/firmware
    • modprobe ahci
    • Ctrl-Alt F1
  9. Manual partition

    https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot

  10. Run live disk, chroot, fuck around

    http://forums.debian.net/viewtopic.php?f=17&t=116036

Troubleshooting:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711147

  • Show root partition:

    cat /proc/cmdline
  • initramfs modules

    /etc/initramfs-tools/modules update-initramfs –k all –u

  • Large xterm:

    xterm -fn 10x20

  • chroot

    boot ubuntu live sudo -i cryptsetup luksOpen /dev/mapper/isw_cejababahfd_ASUS_03 cryptroot lvm vgscan lvm vgchange -ay mount /dev/mapper/taskmaster–vg-root /mnt mount /dev/mapper/isw_cejababahfd_ASUS_02 /mnt/boot mount -o bind /sys /mnt/sys mount -t proc /proc /mnt/proc mount -o bind /dev /mnt/dev chroot /mnt

  • chroot & install dmraid

    chroot apt-get install dmraid initramfs modules

  • break raid:

http://ubuntuforums.org/showthread.php?t=2193133&page=2&p=12913689#post12913689

For realz Steps!

  1. Download Ubuntu 14.04 Live OS and Debian 8.1.0 netinst

  2. Copy to a Ubuntu USB Drive:

    • use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
    • dd bs=4M ~/Downloads/ubuntu-14.04.2-desktop-amd64.iso of=/dev/[usbdrive-without-number]
  3. Download Non-free firmware

  4. Extract firmware from deb

    • ar xv firmware-iwlwifi.deb
    • tar xvf data.tar.xz
  5. Copy to ext4 formatted sd-card

    • mkfs.ext4 /dev/[sd-card]
    • mount -t ext4 /dev/[sd-card] /mnt/sd-card
    • cp -r lib/firmware/* /mnt/sd-card
  6. Boot Ubuntu from usb/change UEFI opts

    • Insert Ubuntu USB drive in Asus, Boot while holding F2
    • Boot → Fastboot [Disabled]
    • Security → Secure Boot Menu → Secure Boot Control [Disabled]
    • Advanced → SATA Configuration → SATA Mode Selection [AHCI]
    • Boot → Boot Option Priorites (hit + until USB drive is at the top)
    • F10 (save and exit)
  7. Try Ubuntu without installing

  8. Enable network, start terminal (Start → uxterm → xterm -fn 10x20)

  9. sudo gparted, delete all partitions on /dev/sda /dev/sdb

  10. Create partition table on /dev/sdb:

    • Select /dev/sdb
    • Device → Create Partition Table…
  11. Shutdown, remove usb

  12. Copy Debian netinst to USB Drive:

    • use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
    • sudo dd bs=4M if=Downloads/debian-8.0.0-amd64-netinst.iso of=[usbdrive-without-number]
  13. Detect network hardware

    • Ctrl-Alt F2
    • mkdir /lib/firmware
    • blkid (to list mounts)
    • mount -t ext4 /dev/sdd1 /lib/firmware
    • modprobe ahci
    • Ctrl-Alt F1
  14. Partition Disks

    • Manual
      1. sda & sbd delete existing partitions
      2. sda1 256MB EFI System partition
      3. sda2 512MB mountat /boot ext4
      4. sda3 REMAINDER physical volume for raid
      5. sdb1 ALL physical volument for raid
      6. configure software raid, raid0 [x]sda3 [x]sdb1
      7. raid0 disk1 use as physical volume for encryption
      8. configure encrypted volumes [x]/dev/md0
      9. encrypted-disk 1: use as physical volume for lvm
      10. configure lvm volume group
      11. 20 GB / ; 4 GB swap ; remainder /home
  15. Post xmonad-install disable GDM login:

05
Posted
GNU Autotools
Tyler Cipriani Posted

configure.ac

  • Written in m4sh (m4 + shell)
  • Program initialization: AC_INIT macro—needs name, version, maintainer
  • Initialize automake: AM_INIT_AUTOMAKE
  • Make sure that C-compiler exists: AC_PROG_CC
  • Build Makefile from Makefile.in, replaces @PACKAGE_VERSION@-type variables: AC_CONFIG_FILES([Makefile])
  • Output the script: AC_OUTPUT
AC_INIT([hello-world], [0.1], [tyler@tylercipriani.com])

# Initialize automake
AM_INIT_AUTOMAKE

# Ensure C-compiler exists
AC_PROG_CC

# Specify what files are to be created
AC_CONFIG_FILES([Makefile])
AC_OUTPUT

Makefile.am

  • The ./configure script (created by configure.ac), expects to find Makefile.in
  • Makefile.in is long and dumb (just like ./configure), so we make a Makefile.am
  • automake generates Makefile.in
  • foreign AUTOMAKEOPTIONS tells that the layout is "non standard"
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = hello-world
hello-world_SOURCES = hello-world.c

Put it all together:

  • create m4 environment: aclocal
  • autoconf does: configure.acconfigure
  • automake --add-missing does: Makefile.amMakefile.in
  • autoreconf --install: autoreconf runs autoconf, autoheader, aclocal, automake, libtoolize, and autopoint
License: Creative Commons Attribution-ShareAlike License
Ikiwiki things
Tyler Cipriani Posted

Install

Mostly it was pretty easy

tar xvzf ikiwiki_3.20160121.tar.gz
cd ikiwiki_3.20160121
sudo PERL5LIB=`pwd` PERL_MM_USE_DEFAULT=1 perl -MCPAN -e 'CPAN::Shell->install("Bundle::IkiWiki")'
sudo PERL5LIB=`pwd` PERL_MM_USE_DEFAULT=1 perl -MCPAN -e 'CPAN::Shell->install("Bundle::IkiWiki::Extras")'
make
sudo make install

OpenID Setup

Ikiwiki requires an OpenID setup.

You can login to Ikiwiki with OpenID, and I think that's neat!

Since OpenID is somewhat dead, I decided to setup a personal SimpleID.

I put this all on the analytics server.

First I had to create a DNS record for openid.tylercipriani.com so letsencrypt would work.

Then I made a new letsencrypt cert:

/usr/local/src/letsencrypt/letsencrypt-auto certonly --standalone -d openid.tylercipriani.com

Then I grabbed and installed openid

cd /usr/local/src
curl -sLO http://downloads.sourceforge.net/simpleid/simpleid-1.0.1.tar.gz
tar xvzf simpleid-1.0.1.tar.gz
mkdir /srv/www/openid.tylercipriani.com
mv simpleid/{cache,identities,store,www} /srv/www/openid.tylercipriani.com
mv /srv/www/openid.tylercipriani.com/{config.php.dist,config.php}

Edited the config.php file, changed root webserver and the pretty url thing

I also made a virtualserver: openid.tylercipriani.com

Identity file: thcipriani.identity

identity="https://tylercipriani.com/"
password="blah:sha1:salt"
administrator=1
[user_info]
...
[sreg]
...

Blog setup

The things that I didn't understand about Ikiwiki:

  1. htmlscrubber removes all your scripts
  2. Some changes to the setup file (like git_wrapper updates) require you to run ikiwiki --changesetup [setupfile]
  3. I am running a staging wiki on my laptop, a wiki on my webserver, and then pushing the compiled files to s3: WAT‽
License: Creative Commons Attribution-ShareAlike License
Mutt
Tyler Cipriani Posted

Contacts

Install goobook:

sudo pip2 install -U goobook

I ran into some problems in arch, had to upgrade/reinstall a few deps:

(ノ^ヮ^)ノ*:・゚✧ sudo pip2 install -U pyasn1
(ノ^ヮ^)ノ*:・゚✧ sudo pip2 install -U cryptography

goobook then needs to authenticate with gmail:

goobook authenticate

Once you go through the oauth song and dance you should be able to run a query

(ノ^ヮ^)ノ*:・゚✧ goobook query tyler

thcipriani@gmail.com    Tyler Cipriani  other groups: "System Group: Friends"

From there, just need to add a bit of magic to the ~/.muttrc file

set query_command="goobook query %s"
macro index,pager a \
    "<pipe-message>goobook add<return>" "add sender to google contacts"
bind editor <Tab> complete-query

Viewing URLs

Install urlview: ncurses program parses urls out of text.

Create a $HOME/urlview file:

COMMAND xdg-open %s &

Bind \,u in the editor:

macro pager ,u "|urlview<enter>" "call urlview to open links"

Searching Mail

Searching works really well via notmuch

Afer install, setup via

notmuch setup

After setup, it's just a matter of calling

notmuch new

to index the things

License: Creative Commons Attribution-ShareAlike License
04
Posted
Bitlebee/IRC info
Tyler Cipriani Posted

Helpful Posts

- pthree.org post

  • Datko's book has this same kinda stuff.

Restart bitlbee without restarting Weechat

sudo service bitlbee stop
/disconnect localhost
sudo service bitlbee start
/connect localhost/6667
/msg &bitlbee identify [password]

Setup OTR

First had to install the bitlbee-plugin-otr plugin:

sudo apt-get install bitlbee-plugin-otr

In the &bitlebee window

account list
> @root 0 (gtalk): jabber, thcipriani@gmail.com (connected)
> @root 1 (twitter): twitter, thcipriani (connected)

Will return a list of accounts that are currently setup in bitlbee.

To generate a key for the 0th account—gmail in my case:

otr keygen 0
License: Creative Commons Attribution-ShareAlike License
VHS to MP4 on Linux
Tyler Cipriani Posted

I recently ordered a set of videos that I remember from a few years ago that did not make the jump to DVD, unfortunately.

I do have a VCR; however, I never have it hooked up because…why would I?

The solution here: convert my VHS tapes into H.264/MP3 mp4-contained files.

Now the question is: how?

Hardware

I managed to grab an EasyCap D60 Recording device from Amazon.

This device is supported inside the linux kernel (from version 3.18 forward…maybe?)

Once I plugged in this device, it was working:

lsusb

This bad boy:

Bus 1 Device 016: ID 1b71:3002 Fushicai USBTV007 Video Grabber [EasyCAP]

I checked out:

ls /dev | grep -i video

And I noticed a new video device video1. Easy.

Capture Software

I used VLC to caputre raw input.

  1. MediaOpen Caputre Device
  2. Video Device Name/dev/video1
  3. Audio Device Namehw:2,0
  4. Play pulldown menu → Convert
  5. Dump Raw Input
  6. Destination File/home/tyler/Videos
  7. Start
  8. Hit play on the VCR
  9. Hit the Rec. button in VLC

The auto-named avi file in ~/Videos was FUCKING HUGE.

ls -lh ~/Videos | grep -i avi

Conversion

I found a blog where a person does this. I have a vauge memory about doing this at UpSync, so I'll give it a shot: 2-pass mp4 conversion.

Let's see what happens!

ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p -preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 1 -f mp4 -y /dev/null
ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p \
  -preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 2 ~/Videos/out.mp4

The settings above created an mp4 that could be played via x264 on a RaspberryPi 3.

License: Creative Commons Attribution-ShareAlike License
Beaglebone Black Setup on Debian 8
Tyler Cipriani Posted

Flash eMMC with latest Debian image

  1. Download Debian image [whatever].img.xz

  2. Insert microSD card and make sure that it is unmounted

    • lsblk
    • fdisk -l
  3. Copy the img to the microSD card

    sudo dd bs=4 if=bone-debian-7.8-lxde-4gb-armhf-2015-03-01-4gb.img of=/dev/mmcblk0 conv=fsync

    This step will take a lot of time.

  4. Mount the microSD card and make sure it flashes to the beaglebone:

    sudo mount /dev/mmcblk0p1 /mnt/sd-card
    vim /mnt/sd-card/boot/uEnv.txt

    Uncomment the line:

    cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh
  5. Hookup the USB-to-serial cable

    • Connect Black Wire to Pin 1 (closest to 5v barrel)
    • Connect Green Wire to Pin 4 (2 pins from pin 1)
    • Connect White Wire to Pin 5
  6. Connect to USB, check output of dmesg for which /dev/ the serial connection is on

  7. Connect to serial connection via screen

    screen /dev/ttyUSB0 115200
  8. put sdcard in BBB and power up, hold the S2 button, power up

  9. Wait, the status lights will flash in a cylon pattern, you can watch the eMMC flash progress via screen

  10. All User LEDs should be solid on completion

More info is available on the BeagleBoneBlack Wiki

Initial setup

Connect over SSH

To connect via ssh: plugin beaglebone to computer via SSH

ssh -l root 192.168.7.2

Wifi Setup

  1. Plugin the wifi adapter

  2. Reboot (unplug it and plug it back in)

  3. Generate your pre-shared key using wpa_passphrase (see Debian Wiki)

  4. vim /etc/network/interfaces

    auto wlan0
    iface wlan0 inet dhcp
        wpa-ssid "network-name"
        wpa-psk "network-password"
  5. ifdown wlan0; ifup wlan0

  6. use ip -o addr show to confirm that you have an ip address

Update debian

Use the Debian Upgrade Script to update debian:

#!/usr/bin/env bash
# Debian auto-upgrade script
# https://debian-handbook.info/browse/stable/sect.automatic-upgrades.html

# kill all cached creds
sudo -k

# ask for new creds
sudo -v

export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
yes '' | sudo apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

Basic security

Make a root password

  • Install pwgen: sudo apt-get install pwgen
  • Generate a root password: pwgen -Bsy 16 1
  • Store that password in your password store—you'll never remember it
  • Login to beaglebone via ssh, run passwd

Add a privileged non-root user

  • Generate a non-privileged user password: pwgen -Bsy 16 1

  • Save password in password store

  • Add the user

    sudo groupadd sudo-user
    sudo groupadd ssh-user
    useradd tyler
    mkdir /home/tyler
    mkdir /home/tyler/.ssh
    touch /home/tyler/.ssh/authorized_keys
    chown -R tyler:tyler /home/tyler
    chmod 700 /home/tyler/.ssh
    chmod 600 /home/tyler/.ssh/authorized_keys
    usermod -a -G sudo-user tyler
    usermod -a -G ssh-user tyler
    usermod --shell /bin/bash tyler
    passwd tyler
  • give that user sudo privileges

    • EDITOR=vim visudo -f /etc/sudoers.d/sudo-user
    • Add the line: %sudo-user ALL=(ALL) NOPASSWD:ALL
  • Add your laptop's key to user's authorized_keys

    # This should happen from your local machine: laptop/desktop/whatever
    cat ~/.ssh/id_rsa.pub | ssh -l tyler 192.168.7.2 'mkdir -p .ssh && cat >> ~/.ssh/authorized_keys'

Remove demo user

  • userdel -fr debian

Lockdown ssh

  • Generate better hostkeys

    cd /etc/ssh
    rm ssh_host_*key*
    ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key < /dev/null
  • Modfiy /etc/ssh/sshd_config to make it like below:

    Ciphers aes256-ctr,aes192-ctr,aes128-ctr
    KexAlgorithms diffie-hellman-group-exchange-sha256
    MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160
    Protocol 2
    HostKey /etc/ssh/ssh_host_rsa_key
    PubkeyAuthentication yes
    PermitRootLogin no
    PasswordAuthentication no
    AllowGroups ssh-user
  • Restart SSH service ssh restart

  • Open a new terminal window and make sure you can still login (you may need to delete and reaccept hostkeys)

Fun Stuff

  • Change ssh banner:

    sudo apt-get install figlet
    awk '$1 !~ /default/' /etc/issue.net > ~/issue.net && sudo mv ~/issue.net /etc/issue.net
    sudo sh -c 'figlet BeagleBone >> /etc/issue.net'
  • BeagleBone pin-out

Things to remember

thank god for 1wire temp sensor blog posts:

License: Creative Commons Attribution-ShareAlike License
neovim on debian
Tyler Cipriani Posted

Installing Neovim

  1. Install Prerequisites

    sudo apt-get -f install libtool libtool-bin autoconf automake cmake g++ pkg-config unzip libmsgpack-dev libuv-dev libluajit-5.1-dev
  2. Clone repo

    git clone https://github.com/neovim/neovim.git
  3. Make install

    cd neovim
    make
    sudo make install

Setup Neovim

ln -s ~/.vim $XDG_CONFIG_HOME/nvim
ln -s ~/.vimrc $XDG_CONFIG_HOME/nvim/init.vim
License: Creative Commons Attribution-ShareAlike License
02
Posted
GPG Things
Tyler Cipriani Posted

Get your fingerprint:

gpg --list-secret-keys --fingerprint

Get someone elses key

Using email

gpg2 --search-keys EMAIL

Using keyid

gpg2 --recv-key KEYID

Sign keys

gpg2 --recv-key KEYID
gpg2 --sign-key KEYID
gpg2 --send-key KEYID

Photos

View someone's photo

gpg --edit-key KEYID showphoto

Add a photo

Suggested image sizes are 240x288 pixels (GnuPG) or 120x144 pixels (PGP) to make a JPEG of 4K-6K in size. A 1.2 h:w ratio w = h / (144 / 120)

gpg --edit-key KEYID addphoto

Keysigning party etherpad

To Sign keys

gpg2 --recv-key KEYID
gpg2 --sign-key KEYID
gpg2 --send-key KEYID

Script to quickly sign all keys

 for k in $(grep pub ksp-wmf-20160108.txt | awk '{print $2}' | awk -F '/' '{print $2}'); do
    echo "Receiving $k"
    gpg2 --recv-key $k
    echo "Signing key $k"
    gpg2 --sign-key $k
    echo "Sending key $k"
    gpg2 --send-key $k
done

IF YOU NEED A KEY, GET HELP!!

Resources

PGP/GPG Intro

OpenPGP Best practices

WMF Keysigning

https://people.wikimedia.org/~faidon/ksp-wmf-20160108.txt

SHA256 Checksum: 52F2 CF39   6A54 6D56   7F55 7138   7264 11BA
                    AEE3 F34E   8880 681C   9A67 75D1   3BBC 74DC              [ ]
                    52F2CF39 6A546D56 7F557138 726411BA AEE3F34E 8880681C 9A6775D1 3BBC74DC

                    RIPEMD160 Checksum: BB44 91B6 0A4D 5865 2105    2A4C 19FB 11AD 8BCC C3C3    [ ]
                    BB44 91B6 0A4D 5865 2105  2A4C 19FB 11AD 8BCC C3C3

Mutt problems

"can't query passphrase in batch mode"

Uncomment, use agent in ~/.gnupg/gpg.conf

Pass problems

No Secret Key

Problem

gpg: decryption failed: No secret key

Solution

  1. install pinentry-curses
  2. ~/.gnupg/gpg-agent.conf
pinentry-program /usr/bin/pinentry
  1. gpg-connect-agent reloadagent /bye
License: Creative Commons Attribution-ShareAlike License
01
Posted
Beaglebone Black Setup on Debian 8
Tyler Cipriani Posted

Flash eMMC with latest Debian image

  1. Download Debian image [whatever].img.xz

  2. Insert microSD card and make sure that it is unmounted

    • lsblk
    • fdisk -l
  3. Copy the img to the microSD card

    sudo dd bs=4 if=bone-debian-7.8-lxde-4gb-armhf-2015-03-01-4gb.img of=/dev/mmcblk0 conv=fsync

    This step will take a lot of time.

  4. Mount the microSD card and make sure it flashes to the beaglebone:

    sudo mount /dev/mmcblk0p1 /mnt/sd-card
    vim /mnt/sd-card/boot/uEnv.txt

    Uncomment the line:

    cmdline=init=/opt/scripts/tools/eMMC/init-eMMC-flasher-v3.sh
  5. Hookup the USB-to-serial cable

    • Connect Black Wire to Pin 1 (closest to 5v barrel)
    • Connect Green Wire to Pin 4 (2 pins from pin 1)
    • Connect White Wire to Pin 5
  6. Connect to USB, check output of dmesg for which /dev/ the serial connection is on

  7. Connect to serial connection via screen

    screen /dev/ttyUSB0 115200
  8. put sdcard in BBB and power up, hold the S2 button, power up

  9. Wait, the status lights will flash in a cylon pattern, you can watch the eMMC flash progress via screen

  10. All User LEDs should be solid on completion

More info is available on the BeagleBoneBlack Wiki

Initial setup

Connect over SSH

To connect via ssh: plugin beaglebone to computer via SSH

ssh -l root 192.168.7.2

Wifi Setup

  1. Plugin the wifi adapter

  2. Reboot (unplug it and plug it back in)

  3. Generate your pre-shared key using wpa_passphrase (see Debian Wiki)

  4. vim /etc/network/interfaces

    auto wlan0
    iface wlan0 inet dhcp
        wpa-ssid "network-name"
        wpa-psk "network-password"
  5. ifdown wlan0; ifup wlan0

  6. use ip -o addr show to confirm that you have an ip address

Update debian

Use the Debian Upgrade Script to update debian:

#!/usr/bin/env bash
# Debian auto-upgrade script
# https://debian-handbook.info/browse/stable/sect.automatic-upgrades.html

# kill all cached creds
sudo -k

# ask for new creds
sudo -v

export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
yes '' | sudo apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade

Basic security

Make a root password

  • Install pwgen: sudo apt-get install pwgen
  • Generate a root password: pwgen -Bsy 16 1
  • Store that password in your password store—you'll never remember it
  • Login to beaglebone via ssh, run passwd

Add a privileged non-root user

  • Generate a non-privileged user password: pwgen -Bsy 16 1

  • Save password in password store

  • Add the user

    sudo groupadd sudo-user
    sudo groupadd ssh-user
    useradd tyler
    mkdir /home/tyler
    mkdir /home/tyler/.ssh
    touch /home/tyler/.ssh/authorized_keys
    chown -R tyler:tyler /home/tyler
    chmod 700 /home/tyler/.ssh
    chmod 600 /home/tyler/.ssh/authorized_keys
    usermod -a -G sudo-user tyler
    usermod -a -G ssh-user tyler
    usermod --shell /bin/bash tyler
    passwd tyler
  • give that user sudo privileges

    • EDITOR=vim visudo -f /etc/sudoers.d/sudo-user
    • Add the line: %sudo-user ALL=(ALL) NOPASSWD:ALL
  • Add your laptop's key to user's authorized_keys

    # This should happen from your local machine: laptop/desktop/whatever
    cat ~/.ssh/id_rsa.pub | ssh -l tyler 192.168.7.2 'mkdir -p .ssh && cat >> ~/.ssh/authorized_keys'

Remove demo user

  • userdel -fr debian

Lockdown ssh

  • Generate better hostkeys

    cd /etc/ssh
    rm ssh_host_*key*
    ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key < /dev/null
  • Modfiy /etc/ssh/sshd_config to make it like below:

    Ciphers aes256-ctr,aes192-ctr,aes128-ctr
    KexAlgorithms diffie-hellman-group-exchange-sha256
    MACs hmac-sha2-512,hmac-sha2-256,hmac-ripemd160
    Protocol 2
    HostKey /etc/ssh/ssh_host_rsa_key
    PubkeyAuthentication yes
    PermitRootLogin no
    PasswordAuthentication no
    AllowGroups ssh-user
  • Restart SSH service ssh restart

  • Open a new terminal window and make sure you can still login (you may need to delete and reaccept hostkeys)

Fun Stuff

  • Change ssh banner:

    sudo apt-get install figlet
    awk '$1 !~ /default/' /etc/issue.net > ~/issue.net && sudo mv ~/issue.net /etc/issue.net
    sudo sh -c 'figlet BeagleBone >> /etc/issue.net'
  • BeagleBone pin-out

Things to remember

thank god for 1wire temp sensor blog posts:

License: Creative Commons Attribution-ShareAlike License
neovim on debian
Tyler Cipriani Posted

Installing Neovim

  1. Install Prerequisites

    sudo apt-get -f install libtool libtool-bin autoconf automake cmake g++ pkg-config unzip libmsgpack-dev libuv-dev libluajit-5.1-dev
  2. Clone repo

    git clone https://github.com/neovim/neovim.git
  3. Make install

    cd neovim
    make
    sudo make install

Setup Neovim

ln -s ~/.vim $XDG_CONFIG_HOME/nvim
ln -s ~/.vimrc $XDG_CONFIG_HOME/nvim/init.vim
License: Creative Commons Attribution-ShareAlike License
Reverse Polish Notation, Lambdas, and Currying in Python
Tyler Cipriani Posted

I just finished reading The Python Corner’s post “Lambdas and Functions in Python.” The post acts as an introduction to the use of functions as first-class objects in python. The demo code is the implementation of a “Reverse Polish Notation” calculator.

I had never heard of reverse polish notation(RPN) before this post. The short explanation of RPN is available on Wikipedia:

In reverse Polish notation, the operators follow their operands; for instance, to add 3 and 4, one would write 3 4 + rather than 3 + 4.

In that blog post RPN is implemented as a stack of operands and after an operator is pushed onto the stack, the compute() method is called which triggers the evaluation of the lambda specified by the operator. Like this:

rpn = rpn_engine()
rpn.push(2)
rpn.push(2)
print(rpn.compute('+'))  # Prints 4

The point of the post is to show a python dict using operands as keys with lambdas as values; this demonstrates that lambdas are functions and functions are first-class objects. This allows the compute method of the RPNEngine class to look up a lambda in a dict, and pop() off the stack using the signature function of the inspect module to determine how many arguments are needed for a particular lambda. From there, lambda evaluation is handed off to helper functions named, for instance, compute_operation_with_two_operands and compute_operation_with_one_operand

Currying

One other functional concept that could have helped the example code is that of currying. Currying involves changing a function with multiple arity into a series of evaluations of multiple functions each with an arity of 1.

This is a fancy way to say:

add = lambda x, y: x + y
add_curried = lambda x: lambda y: x + y

assert(add(2, 2) == add_curried(2)(2))

By turning the compute_operation_with_n_operands-type functions into curried functions, the code gets much cleaner. That is, instead of a switch like:

if number_of_operands == 2:
    self.compute_operation_with_two_operands(self.catalog[operation])

if number_of_operands == 1:
    self.compute_operation_with_one_operand(self.catalog[operation])

You can implement a curried function using a callable python object and do something like:

func = self.catalog[operation]

while not func.resolved:
    func(self.pop())

This gets rid of the clunky compute_operation_with_n_operands functions. Here is the full code for a solution using currying:

#!/usr/bin/env python3
"""
Engine class for RPN Calculator
"""

import math

from functools import partial
from inspect import signature


class Curry(object):
    """
    Curry a callable

    Given a callable, returns a an object that can be used like a curried
    callable.

    >>> c1 = Curry(lambda x, y: x + y)
    >>> c2 = Curry(lambda x, y: x + y)
    >>> c1(2, 2) == c2(2)(2)
    True

    :func: callable
    """
    def __init__(self, func):
        self.func = func
        self.argc = len(signature(self.func).parameters)
        self.resolved = False
        self.answer = None

    def __call__(self, *args):
        if len(args) == self.argc:
            self.answer = self.func(*args)
            self.resolved = True

        for arg in args:
            self.func = partial(self.func, arg)
            self.argc = len(signature(self.func).parameters)

        return self


class RPNEngine(object):
    """
    Reverse Polish Notation (RPN) Engine

    A RPN calculator
    >>> rpn = RPNEngine()
    >>> rpn.push(2)
    >>> rpn.push(2)
    >>> rpn.compute('+') == 4
    True
    >>> rpn.compute('AC')
    >>> rpn.push(2)
    >>> rpn.compute('^2') == 4
    True
    """
    def __init__(self):
        self.stack = []
        self.functions = self._get_functions()

    def _get_functions(self):
        return {
            '+': Curry(lambda x, y: x + y),
            '-': Curry(lambda x, y: x - y),
            '*': Curry(lambda x, y: x * y),
            '/': Curry(lambda x, y: x / y),
            '^2': Curry(lambda x: x * x),
            "SQRT": Curry(lambda x: math.sqrt(x)),
            "C": Curry(lambda: self.stack.pop()),
            "AC": Curry(lambda: self.stack.clear()),
        }

    def push(self, item):
        self.stack.append(item)

    def pop(self):
        try:
            return self.stack.pop()
        except IndexError:
            pass

    def compute(self, operation):
        func = self.functions.get(operation)

        if not func:
            raise BaseException('%s not a valid function' % operation)

        if len(self.stack) < func.argc:
            raise BaseException(
                '%s requires %d operands, %d given' % (
                    operation,
                    func.argc,
                    len(self.stack)
                )
            )

        if func.argc == 0:
            func()

        while not func.resolved:
            func(self.pop())

        return func.answer

Reading the final code in the Python Corner post made me me really itchy to implement the solution I posted here.

Jun 2018
S M T W T F S