As a frequent deployer of software, I’m usually watching several different flavors of system monitoring all at the same time. A few logstash dashboards in a few browser windows, a few grafana dashboards, a few different logs being tailed in a few different tmux windows. If it all goes wrong during a deployment – if the error log suddenly explodes – I want to know. Even if I’m looking at another metric, or at IRC, or at my email – I want to know about log explosion immediately. So I made a thing that plays a little noise whenever a new line is recieved on stdin. I call it Bloop! And I’ve been using it to tail logs like: So now my stomach drops when I hear, “bloop…bloop…blobloopboopbloop”. I’m sure this thing has been build a million times because it’s a braindead-simple idea, but I couldn’t find the right words to search for it so I made new one. Now all I need is an EKG monitor so I can see how much this bloop thing makes my heart rate match the overall error rate. Most Debian installed bash completions live under Some complete options you want: https://gorails.com/setup/ubuntu/14.04 require "selenium-webdriver" https://code.google.com/p/selenium/wiki/RubyBindings driver.class http://www.rubydoc.info/gems/selenium-webdriver/Selenium/WebDriver/Driver First had to install the In the Will return a list of accounts that are currently setup in bitlbee. To generate a key for the 0th account—gmail in my case: I recently ordered a set of videos that I remember from a few years ago that did not make the jump to DVD, unfortunately. I do have a VCR; however, I never have it hooked up because…why would I? The solution here: convert my VHS tapes into H.264/MP3 mp4-contained files. Now the question is: how? I managed to grab an EasyCap D60 Recording device from Amazon. This device is supported inside the linux kernel (from version 3.18 forward…maybe?) Once I plugged in this device, it was working: This bad boy: I checked out: And I noticed a new video device I used VLC to caputre raw input. The auto-named avi file in I found a blog where a person does this. I have a vauge memory about doing this at UpSync, so I'll give it a shot: 2-pass mp4 conversion. Let's see what happens! The settings above created an mp4 that could be played via x264 on a RaspberryPi 3. To see posts by date, check out the archives Today I went to coffee with a buddy of mine from We spent a good chunk of time looking at Wireshark captures of an HTTP request. It got muddled and we got off-track because we were looking at an HTTPS request and we started talking about TLS1.2, CLIENT-HELLO, and the TLS handshake in general. I eventually found the source for what I was yammering about. Due to how TCP estimates the capacity of a connection (i.e. TCP Slow Start), a new TCP connection cannot immediately use the full available bandwidth between the client and the server. Because of this, the server can send up to 10 TCP packets on a new connection (~14KB) in first roundtrip, and then it must wait for client to acknowledge this data before it can grow its congestion window and proceed to deliver more data. After I got home and ate some ice cream, I did some more reading trying to understand this stuff. I think a lot of what was confusing about the Wireshark stuff we were looking at is it also had TLS1.2 mixed in there. Requesting a plain-old HTTP, port 80 site made looking at packets a bit more sane. It seems like the size of a TCP packet is negotiated during the The MSS is derived from the MTU and is (at most?) the MTU minus the 40 bytes used for the TCP header and the IP header, so my machine sent the MSS size of 1460 to the server in the So the data portion of a TCP segment sent in this particular connection should never exceed 1460 bytes. While we were looking at Wireshark packet captures we kept noticing a TCP segment size of 1486 in the “length” column with a TCP segment data length of 1432. I noticed that my webserver sent an MSS size of 1432, which I suppose became the limiting factor in how much data could be sent per segment (rather than the 1460 MSS sent by my client – the browser): Each TCP segment had a header-size of 20 bytes, so that leaves 34 unexplained bytes which (I guess) were part of the IP header. So why try to squeeze a website into ~14KB? Seems like you should be trying to squeeze it into 1432 bytes. I found my answer in IETF RFC 6928 – a proposal to increase the TCP Initial Window to 10 segments. The Initial Congestion Window ( I checked the value of And it is indeed set to 10. This means that my server can send OR around 14KB of data can be sent to a browser before another round-trip from the client to the webserver has to happen. So, overall, a nice, relaxing coffee and catch-up session with an old coworker.ssh logsever -- tail -f /var/log/important/error.log | bloop -sLinks ¶
Basics ¶
/usr/share/bash_completion/completions# the first argument ($1) is the name of the command whose arguments are being completed
# the second argument ($2) is the word being completed,
# and the third argument ($3) is the word preceding the word being completed on the current command line
# In the context of this function the following variables are defined:
# COMP_LINE, COMP_POINT, COMP_KEY, and COMP_TYPE
# as well as COMP_WORDS and COMP_CWORD
# It must put the possible completions in the COMPREPLY array
# See bash(1) '^ Programmable Completion' for more information
_some_function() {
local cur cmd
cur=${COMP_WORDS[$COMP_CWORD]}
cmd=( "${COMP_WORDS[@]}" )
}
complete -F _some_function commandcomplete options ¶
-o bashdefault - if no completions are found do the bash default thing-o default - readline completions if both the complete function and bash expansions fail-o nospace - don’t append a space at the end of matches (useful if you’re doing directory stuffs)-S or -P - a prefix or suffix that is added at the end of a completion generated by the function passed to complete -FExample ¶
# Many of the ideas presented in this script were stolen
# in-part or wholesale from
# <https://github.com/git/git/blob/master/contrib/completion/git-completion.bash>
__scap_subcommands=
__scap_get_subcommands() {
if [ -n "$__scap_subcommands" ]; then
return
fi
__scap_subcommands=$(scap --_autocomplete)
}
_scap() {
local cur cmd=() sub rep
cur=${COMP_WORDS[$COMP_CWORD]}
cmd=( "${COMP_WORDS[@]}" )
if (( COMP_CWORD == 1 )); then
__scap_get_subcommands
rep=$( compgen -W "$__scap_subcommands" -- "$cur" )
COMPREPLY=( $rep )
return
fi
# limit the command to autocomplete to the first 3 words
if (( COMP_CWORD >= 2 )); then
# don't complete any sub-subcommands, only options
if [[ -n "$cur" && "${cur:0:1}" != '-' ]]; then
COMPREPLY=()
return
fi
cmd=( "${COMP_WORDS[@]:0:3}" )
fi
# replace the last word in the command with '--_autocomplete'
cmd[ $(( ${#cmd[@]} - 1 )) ]='--_autocomplete'
rep=$( compgen -W "$( ${cmd[@]} )" -- "$cur" )
COMPREPLY=( $rep )
}
# By default append nospace except when completion comes from _scap
complete -S' ' -o bashdefault -o default -o nospace -F _scap scapCheck setup ¶
Ruby install ¶
Selenium webdriver ¶
Core tests ¶
[mw-core]/tests/browser/features/create_account.featureBundler ¶
Important ¶
Helpful Posts ¶
Restart bitlbee without restarting Weechat ¶
Setup OTR ¶
bitlbee-plugin-otr plugin:&bitlebee windowaccount list
> @root 0 (gtalk): jabber, thcipriani@gmail.com (connected)
> @root 1 (twitter): twitter, thcipriani (connected)
otr keygen 0
Hardware ¶
Bus
1
Device
016:
ID
1b71:3002
Fushicai
USBTV007
Video
Grabber
[EasyCAP]
video1. Easy.Capture Software ¶
Media → Open Caputre DeviceVideo Device Name → /dev/video1Audio Device Name → hw:2,0Play pulldown menu → ConvertDump Raw InputDestination File → /home/tyler/VideosStart~/Videos was FUCKING HUGE.Conversion ¶
ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p \
-preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 2 ~/Videos/out.mp4$DAY_JOB - 2. During coffee we were discussing all the folk theories of “fast” websites. I made some off-the-cuff remark that I didn’t really understand – that you should try to squeeze the most important bits of your website into the first X bytes (X being a value I didn’t remember) for an opaque reason that has to do with TCP congestion algorithms that I’ve never understood.
SYN, SYN-ACK phase of the TCP three-way handshake. In the Options section of the TCP segment (for a segment that has the SYN-flag set in the TCP segment header) a sender or receiver may indicate a Max Segment Size (MSS). An MSS is derived from a machine’s Maximum Transmission Unit (MTU). In the case of my machine the MTU is set to 1500. I found this via the ip command:tyler@magneto:~$ ip -4 link show dev wlp3s0
2: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000Options section of the SYN TCP segment that it initially sent my webserver.
cwnd) is the maximum number of bytes that a server can send without receiving any acknowledgement that the bytes were received via a client ACK packet. This maximum number of bytes to send is calculated by multiplying some number by MSS. As of RFC 6928 some number is equal to 10 (initially).cwnd on some established connections on my server:cwnd * MSS bytes of information before the client has to acknowledge that anything has been receieved. So, in the case of the Wireshark connection I recorded, my webserver can send 1432 * 10 bytes, or 14320 bytes:tyler@magneto:~$ units
Currency exchange rates from www.timegenie.com on 2016-06-22
202926 units, 109 prefixes, 88 nonlinear units
You have: 14320 bytes
You want: kilobytes
* 14.32
/ 0.069832402
Links ¶
configure.ac ¶
- Written in m4sh (m4 + shell)
- Program initialization:
AC_INITmacro—needs name, version, maintainer - Initialize automake:
AM_INIT_AUTOMAKE - Make sure that C-compiler exists:
AC_PROG_CC - Build
MakefilefromMakefile.in, replaces@PACKAGE_VERSION@-type variables:AC_CONFIG_FILES([Makefile]) - Output the script:
AC_OUTPUT
Makefile.am ¶
- The
./configurescript (created byconfigure.ac), expects to findMakefile.in Makefile.inis long and dumb (just like./configure), so we make aMakefile.amautomakegeneratesMakefile.inforeignAUTOMAKEOPTIONS tells that the layout is "non standard"
Put it all together: ¶
- create m4 environment:
aclocal autoconfdoes:configure.ac→configureautomake --add-missingdoes:Makefile.am→Makefile.inautoreconf --install: autoreconf runs autoconf, autoheader, aclocal, automake, libtoolize, and autopoint
Install ¶
Mostly it was pretty easy
- Grabbed the tar from debian
- Followed install instructions from Ikiwiki's install page
OpenID Setup ¶
Ikiwiki requires an OpenID setup.
You can login to Ikiwiki with OpenID, and I think that's neat!
Since OpenID is somewhat dead, I decided to setup a personal SimpleID.
I put this all on the analytics server.
First I had to create a DNS record for openid.tylercipriani.com so letsencrypt would work.
Then I made a new letsencrypt cert:
Then I grabbed and installed openid
cd /usr/local/src
curl -sLO http://downloads.sourceforge.net/simpleid/simpleid-1.0.1.tar.gz
tar xvzf simpleid-1.0.1.tar.gz
mkdir /srv/www/openid.tylercipriani.com
mv simpleid/{cache,identities,store,www} /srv/www/openid.tylercipriani.com
mv /srv/www/openid.tylercipriani.com/{config.php.dist,config.php}Edited the config.php file, changed root webserver and the pretty url thing
I also made a virtualserver: openid.tylercipriani.com
Identity file: thcipriani.identity
Blog setup ¶
The things that I didn't understand about Ikiwiki:
- htmlscrubber removes all your scripts
- Some changes to the setup file (like
git_wrapperupdates) require you to runikiwiki --changesetup [setupfile] - I am running a staging wiki on my laptop, a wiki on my webserver, and then pushing the compiled files to s3: WAT‽
Contacts ¶
Install goobook:
I ran into some problems in arch, had to upgrade/reinstall a few deps:
goobook then needs to authenticate with gmail:
Once you go through the oauth song and dance you should be able to run a query
(ﾉ^ヮ^)ﾉ*:・ﾟ✧ goobook query tyler
thcipriani@gmail.com Tyler Cipriani other groups: "System Group: Friends"From there, just need to add a bit of magic to the ~/.muttrc file
Viewing URLs ¶
Install urlview: ncurses program parses urls out of text.
Create a $HOME/urlview file:
Bind \,u in the editor:
Searching Mail ¶
Searching works really well via notmuch
Afer install, setup via
After setup, it's just a matter of calling
to index the things
Setup an Comodo positive SSL cert on AWS Cloudfront
- Buy Comodo positive SSL cert via namecheap
Inspect last years cert:
Generate private key and signing request
- You can generate a new key as a separate step, or as part of the
reqcommand To generate a new key manually use:
To generate a new key as part of a signing reuest use:
Answer lots of questions
Country Name (2 letter code) [XX]:US State or Province Name (full name) []:Colorado Locality Name (eg, city) [Default City]:Longmont Organization Name (eg, company) [Default Company Ltd]:Tyler Cipriani Organizational Unit Name (eg, section) []:. Common Name (eg, your name or your server's hostname) []:tylercipriani.com Email Address []:spam@tylercipriani.com A challenge password []:- What is a challenge password: https://tools.ietf.org/html/rfc2985#page-16
- Should now have a
tylercipriani_com.csrand antylercipriani_com.key - Do the namecheap needful:
- Login to namecheap and activate your certificate
xsel -p < tylercipriani_com.csrthen paste in the “Enter CSR” field- Choose, “Apache, Nginx, or Other” as the server type
- Verify the domain, click “Next”
- Verify CSR info, click “Next”
- Select “Email-based” Domain Control Validation (DCV) method
- Company contacts page, “NA” as company name, my address in address, tyler@tylercipriani.com in admin email
- You’ll get an email that asks you to enter a validation into a Comodo site, do that
- You’ll get an email with
tylercipriani_com.zip aws configureto ensure that your awscli is setupUse awscli to upload
- Login to your aws dashboard, click cloudfront, go do Distribution Settings, click edit, find the new ssl key name in the dropdown and click, “Yes, edit”
Should work…
To see posts by date, check out the archives
The goal here is to hit 2 books a month, so 24 for the year. I’m on track as of now. Perilous.
- Stranger in a Strange Land by Robert A. Heinlein
- Black Hole by Charles Burns
- Fun Home by Alison Bechdel
- The Girl on the Train by Paula Hawkins
- Starship Troopers by Robert A. Heinlein
- Ready Player One by Ernest Cline
- The Circle by Dave Eggers
- Notorious RBG by Irin Carmon and Shana Knizhnik
- The Three-Body Problem by Cixin Liu
- A Wizard of Earthsea by Ursula K. Le Guin
- Stoner by John Williams
- Prost! The Story of German Beer by Horst D. Dornbusch
- Bock by Darryl Richman
- Do Androids Dream of Electric Sheep by Philip K. Dick
- Room by Emma Donoghue
- Kitchen Confidential: adventures in the culinary underbelly by Anthony Bourdain
- H is for Hawk by Helen Macdonald
t to input a
tag, q to quit. Meanwhile it's generating a csv file that
can be uploaded to LibraryThing.
The fruits of this labor were/are my (and Blazey's) Library Thing Catalog
To see posts by date, check out the archives
Setup New repo ¶
init git:
init git annex:
Add S3 Remote named public-s3
Add file to git annex
copy current directory to public-s3 remote
remove local copy—will only remove if already in remote—nothing lost
Commit
Setup NFS as remote using Rsync over ssh ¶
Add file to nas
Pull down photos ¶
Make available to the public (after public s3 setup): ¶
List remotes and remote info ¶
Show remotes ¶
Show remote info ¶
Finding info about files ¶
Find a file:
Adding a file to http://tyler.zone
Links ¶
Setup ¶
Make amazon bucket with policy:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::[bucket-name]/*"
}
]
}Put creds out into the environment
Get your fingerprint: ¶
Get someone elses key ¶
Using email ¶
Using keyid ¶
Sign keys ¶
Photos ¶
View someone's photo ¶
Add a photo ¶
Suggested image sizes are 240x288 pixels (GnuPG) or 120x144 pixels (PGP) to make a JPEG of 4K-6K in size. A 1.2 h:w ratio w = h / (144 / 120)
Keysigning party etherpad ¶
To Sign keys ¶
Script to quickly sign all keys ¶
IF YOU NEED A KEY, GET HELP!! ¶
Resources ¶
PGP/GPG Intro
- https://ssd.eff.org/en/module/introduction-public-key-cryptography-and-pgp
- https://ssd.eff.org/en/module/how-use-pgp-mac-os-x
OpenPGP Best practices
WMF Keysigning ¶
Mutt problems ¶
"can't query passphrase in batch mode" ¶
Uncomment, use agent in ~/.gnupg/gpg.conf
Pass problems ¶
No Secret Key ¶
Problem ¶
Solution ¶
- install
pinentry-curses ~/.gnupg/gpg-agent.conf
pinentry-program /usr/bin/pinentry
gpg-connect-agent reloadagent /bye
Setup an Comodo positive SSL cert on AWS Cloudfront
- Buy Comodo positive SSL cert via namecheap
Inspect last years cert:
Generate private key and signing request
- You can generate a new key as a separate step, or as part of the
reqcommand To generate a new key manually use:
To generate a new key as part of a signing reuest use:
Answer lots of questions
Country Name (2 letter code) [XX]:US State or Province Name (full name) []:Colorado Locality Name (eg, city) [Default City]:Longmont Organization Name (eg, company) [Default Company Ltd]:Tyler Cipriani Organizational Unit Name (eg, section) []:. Common Name (eg, your name or your server's hostname) []:tylercipriani.com Email Address []:spam@tylercipriani.com A challenge password []:- What is a challenge password: https://tools.ietf.org/html/rfc2985#page-16
- Should now have a
tylercipriani_com.csrand antylercipriani_com.key - Do the namecheap needful:
- Login to namecheap and activate your certificate
xsel -p < tylercipriani_com.csrthen paste in the “Enter CSR” field- Choose, “Apache, Nginx, or Other” as the server type
- Verify the domain, click “Next”
- Verify CSR info, click “Next”
- Select “Email-based” Domain Control Validation (DCV) method
- Company contacts page, “NA” as company name, my address in address, tyler@tylercipriani.com in admin email
- You’ll get an email that asks you to enter a validation into a Comodo site, do that
- You’ll get an email with
tylercipriani_com.zip aws configureto ensure that your awscli is setupUse awscli to upload
- Login to your aws dashboard, click cloudfront, go do Distribution Settings, click edit, find the new ssl key name in the dropdown and click, “Yes, edit”
Should work…
Reboot, hitting F2 over-and-over
Turn off Fastboot and Secureboot in UEFI
Download Non-free firmware
Extract firmware from deb
ar xv firmware-iwlwifi.deb tar xvf data.tar.xz
Copy to ext4 formatted sd-card
mkfs.ext4 /dev/[sd-card] mount -t ext4 /dev/[sd-card] /mnt/sd-card cp -r lib/firmware/* /mnt/sd-card
Insert Install Media
Setup installer to recognize dmraid
- SataRaid DebianInstaller
- Hit ‘E’ with ‘Install’ selected
- append ‘dmraid=true’ to eol
- F10 to save-and-exit
Detect network hardware
- Ctrl-Alt F2
- mkdir /lib/firmware
- blkid (to list mounts)
- mount -t ext4 /dev/sdd1 /lib/firmware
- modprobe ahci
- Ctrl-Alt F1
Manual partition
https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot
Run live disk, chroot, fuck around
http://forums.debian.net/viewtopic.php?f=17&t=116036
Troubleshooting:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711147
Show root partition:
cat /proc/cmdlineinitramfs modules
/etc/initramfs-tools/modules update-initramfs –k all –u
Large xterm:
xterm -fn 10x20
chroot
boot ubuntu live sudo -i cryptsetup luksOpen /dev/mapper/isw_cejababahfd_ASUS_03 cryptroot lvm vgscan lvm vgchange -ay mount /dev/mapper/taskmaster–vg-root /mnt mount /dev/mapper/isw_cejababahfd_ASUS_02 /mnt/boot mount -o bind /sys /mnt/sys mount -t proc /proc /mnt/proc mount -o bind /dev /mnt/dev chroot /mnt
chroot & install dmraid
chroot apt-get install dmraid initramfs modules
break raid:
http://ubuntuforums.org/showthread.php?t=2193133&page=2&p=12913689#post12913689
For realz Steps! ¶
- Download Ubuntu 14.04 Live OS and Debian 8.1.0 netinst
Copy to a Ubuntu USB Drive:
- use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
dd bs=4M ~/Downloads/ubuntu-14.04.2-desktop-amd64.iso of=/dev/[usbdrive-without-number]
Download Non-free firmware
Extract firmware from deb
ar xv firmware-iwlwifi.debtar xvf data.tar.xz
Copy to ext4 formatted sd-card
mkfs.ext4 /dev/[sd-card]mount -t ext4 /dev/[sd-card] /mnt/sd-cardcp -r lib/firmware/* /mnt/sd-card
Boot Ubuntu from usb/change UEFI opts
- Insert Ubuntu USB drive in Asus, Boot while holding F2
- Boot → Fastboot [Disabled]
- Security → Secure Boot Menu → Secure Boot Control [Disabled]
- Advanced → SATA Configuration → SATA Mode Selection [AHCI]
- Boot → Boot Option Priorites (hit + until USB drive is at the top)
- F10 (save and exit)
- Try Ubuntu without installing
- Enable network, start terminal (Start → uxterm → xterm -fn 10x20)
- sudo gparted, delete all partitions on /dev/sda /dev/sdb
Create partition table on /dev/sdb:
- Select /dev/sdb
- Device → Create Partition Table…
- Shutdown, remove usb
Copy Debian netinst to USB Drive:
- use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
sudo dd bs=4M if=Downloads/debian-8.0.0-amd64-netinst.iso of=[usbdrive-without-number]
Detect network hardware
- Ctrl-Alt F2
- mkdir /lib/firmware
- blkid (to list mounts)
- mount -t ext4 /dev/sdd1 /lib/firmware
- modprobe ahci
- Ctrl-Alt F1
Partition Disks
- Manual
- sda & sbd delete existing partitions
- sda1 256MB EFI System partition
- sda2 512MB mountat /boot ext4
- sda3 REMAINDER physical volume for raid
- sdb1 ALL physical volument for raid
- configure software raid, raid0 [x]sda3 [x]sdb1
- raid0 disk1 use as physical volume for encryption
- configure encrypted volumes [x]/dev/md0
- encrypted-disk 1: use as physical volume for lvm
- configure lvm volume group
- 20 GB / ; 4 GB swap ; remainder /home
- Manual
Post xmonad-install disable GDM login:
Reboot, hitting F2 over-and-over
Turn off Fastboot and Secureboot in UEFI
Download Non-free firmware
Extract firmware from deb
ar xv firmware-iwlwifi.deb tar xvf data.tar.xz
Copy to ext4 formatted sd-card
mkfs.ext4 /dev/[sd-card] mount -t ext4 /dev/[sd-card] /mnt/sd-card cp -r lib/firmware/* /mnt/sd-card
Insert Install Media
Setup installer to recognize dmraid
- SataRaid DebianInstaller
- Hit ‘E’ with ‘Install’ selected
- append ‘dmraid=true’ to eol
- F10 to save-and-exit
Detect network hardware
- Ctrl-Alt F2
- mkdir /lib/firmware
- blkid (to list mounts)
- mount -t ext4 /dev/sdd1 /lib/firmware
- modprobe ahci
- Ctrl-Alt F1
Manual partition
https://wiki.debian.org/DebianInstaller/SoftwareRaidRoot
Run live disk, chroot, fuck around
http://forums.debian.net/viewtopic.php?f=17&t=116036
Troubleshooting:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=711147
Show root partition:
cat /proc/cmdlineinitramfs modules
/etc/initramfs-tools/modules update-initramfs –k all –u
Large xterm:
xterm -fn 10x20
chroot
boot ubuntu live sudo -i cryptsetup luksOpen /dev/mapper/isw_cejababahfd_ASUS_03 cryptroot lvm vgscan lvm vgchange -ay mount /dev/mapper/taskmaster–vg-root /mnt mount /dev/mapper/isw_cejababahfd_ASUS_02 /mnt/boot mount -o bind /sys /mnt/sys mount -t proc /proc /mnt/proc mount -o bind /dev /mnt/dev chroot /mnt
chroot & install dmraid
chroot apt-get install dmraid initramfs modules
break raid:
http://ubuntuforums.org/showthread.php?t=2193133&page=2&p=12913689#post12913689
For realz Steps! ¶
- Download Ubuntu 14.04 Live OS and Debian 8.1.0 netinst
Copy to a Ubuntu USB Drive:
- use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
dd bs=4M ~/Downloads/ubuntu-14.04.2-desktop-amd64.iso of=/dev/[usbdrive-without-number]
Download Non-free firmware
Extract firmware from deb
ar xv firmware-iwlwifi.debtar xvf data.tar.xz
Copy to ext4 formatted sd-card
mkfs.ext4 /dev/[sd-card]mount -t ext4 /dev/[sd-card] /mnt/sd-cardcp -r lib/firmware/* /mnt/sd-card
Boot Ubuntu from usb/change UEFI opts
- Insert Ubuntu USB drive in Asus, Boot while holding F2
- Boot → Fastboot [Disabled]
- Security → Secure Boot Menu → Secure Boot Control [Disabled]
- Advanced → SATA Configuration → SATA Mode Selection [AHCI]
- Boot → Boot Option Priorites (hit + until USB drive is at the top)
- F10 (save and exit)
- Try Ubuntu without installing
- Enable network, start terminal (Start → uxterm → xterm -fn 10x20)
- sudo gparted, delete all partitions on /dev/sda /dev/sdb
Create partition table on /dev/sdb:
- Select /dev/sdb
- Device → Create Partition Table…
- Shutdown, remove usb
Copy Debian netinst to USB Drive:
- use lsblk/blkid to find USB drive name without number, e.g. /dev/sdb NOT /dev/sdb1
sudo dd bs=4M if=Downloads/debian-8.0.0-amd64-netinst.iso of=[usbdrive-without-number]
Detect network hardware
- Ctrl-Alt F2
- mkdir /lib/firmware
- blkid (to list mounts)
- mount -t ext4 /dev/sdd1 /lib/firmware
- modprobe ahci
- Ctrl-Alt F1
Partition Disks
- Manual
- sda & sbd delete existing partitions
- sda1 256MB EFI System partition
- sda2 512MB mountat /boot ext4
- sda3 REMAINDER physical volume for raid
- sdb1 ALL physical volument for raid
- configure software raid, raid0 [x]sda3 [x]sdb1
- raid0 disk1 use as physical volume for encryption
- configure encrypted volumes [x]/dev/md0
- encrypted-disk 1: use as physical volume for lvm
- configure lvm volume group
- 20 GB / ; 4 GB swap ; remainder /home
- Manual
Post xmonad-install disable GDM login:
Links ¶
configure.ac ¶
- Written in m4sh (m4 + shell)
- Program initialization:
AC_INITmacro—needs name, version, maintainer - Initialize automake:
AM_INIT_AUTOMAKE - Make sure that C-compiler exists:
AC_PROG_CC - Build
MakefilefromMakefile.in, replaces@PACKAGE_VERSION@-type variables:AC_CONFIG_FILES([Makefile]) - Output the script:
AC_OUTPUT
Makefile.am ¶
- The
./configurescript (created byconfigure.ac), expects to findMakefile.in Makefile.inis long and dumb (just like./configure), so we make aMakefile.amautomakegeneratesMakefile.inforeignAUTOMAKEOPTIONS tells that the layout is "non standard"
Put it all together: ¶
- create m4 environment:
aclocal autoconfdoes:configure.ac→configureautomake --add-missingdoes:Makefile.am→Makefile.inautoreconf --install: autoreconf runs autoconf, autoheader, aclocal, automake, libtoolize, and autopoint
Install ¶
Mostly it was pretty easy
- Grabbed the tar from debian
- Followed install instructions from Ikiwiki's install page
OpenID Setup ¶
Ikiwiki requires an OpenID setup.
You can login to Ikiwiki with OpenID, and I think that's neat!
Since OpenID is somewhat dead, I decided to setup a personal SimpleID.
I put this all on the analytics server.
First I had to create a DNS record for openid.tylercipriani.com so letsencrypt would work.
Then I made a new letsencrypt cert:
Then I grabbed and installed openid
cd /usr/local/src
curl -sLO http://downloads.sourceforge.net/simpleid/simpleid-1.0.1.tar.gz
tar xvzf simpleid-1.0.1.tar.gz
mkdir /srv/www/openid.tylercipriani.com
mv simpleid/{cache,identities,store,www} /srv/www/openid.tylercipriani.com
mv /srv/www/openid.tylercipriani.com/{config.php.dist,config.php}Edited the config.php file, changed root webserver and the pretty url thing
I also made a virtualserver: openid.tylercipriani.com
Identity file: thcipriani.identity
Blog setup ¶
The things that I didn't understand about Ikiwiki:
- htmlscrubber removes all your scripts
- Some changes to the setup file (like
git_wrapperupdates) require you to runikiwiki --changesetup [setupfile] - I am running a staging wiki on my laptop, a wiki on my webserver, and then pushing the compiled files to s3: WAT‽
Contacts ¶
Install goobook:
I ran into some problems in arch, had to upgrade/reinstall a few deps:
goobook then needs to authenticate with gmail:
Once you go through the oauth song and dance you should be able to run a query
(ﾉ^ヮ^)ﾉ*:・ﾟ✧ goobook query tyler
thcipriani@gmail.com Tyler Cipriani other groups: "System Group: Friends"From there, just need to add a bit of magic to the ~/.muttrc file
Viewing URLs ¶
Install urlview: ncurses program parses urls out of text.
Create a $HOME/urlview file:
Bind \,u in the editor:
Searching Mail ¶
Searching works really well via notmuch
Afer install, setup via
After setup, it's just a matter of calling
to index the things
Helpful Posts ¶
- Datko's book has this same kinda stuff.
Restart bitlbee without restarting Weechat ¶
Setup OTR ¶
First had to install the bitlbee-plugin-otr plugin:
In the &bitlebee window
account list
> @root 0 (gtalk): jabber, thcipriani@gmail.com (connected)
> @root 1 (twitter): twitter, thcipriani (connected)
Will return a list of accounts that are currently setup in bitlbee.
To generate a key for the 0th account—gmail in my case:
otr keygen 0
I recently ordered a set of videos that I remember from a few years ago that did not make the jump to DVD, unfortunately.
I do have a VCR; however, I never have it hooked up because…why would I?
The solution here: convert my VHS tapes into H.264/MP3 mp4-contained files.
Now the question is: how?
Hardware ¶
I managed to grab an EasyCap D60 Recording device from Amazon.
This device is supported inside the linux kernel (from version 3.18 forward…maybe?)
Once I plugged in this device, it was working:
This bad boy:
| Bus | 1 | Device | 016: | ID | 1b71:3002 | Fushicai | USBTV007 | Video | Grabber | [EasyCAP] |
I checked out:
And I noticed a new video device video1. Easy.
Capture Software ¶
I used VLC to caputre raw input.
Media→Open Caputre DeviceVideo Device Name→/dev/video1Audio Device Name→hw:2,0Playpulldown menu →ConvertDump Raw InputDestination File→/home/tyler/VideosStart- Hit play on the VCR
- Hit the Rec. button in VLC
The auto-named avi file in ~/Videos was FUCKING HUGE.
Conversion ¶
I found a blog where a person does this. I have a vauge memory about doing this at UpSync, so I'll give it a shot: 2-pass mp4 conversion.
Let's see what happens!
ffmpeg -i ~/Videos/vlc-record-2016-04-22-14h47m57s-Streaming-.avi -c:v libx264 -pix_fmt yuv420p \
-preset slow -threads 0 -b:v 825k -strict -2 -c:a aac -b:a 96k -pass 2 ~/Videos/out.mp4The settings above created an mp4 that could be played via x264 on a RaspberryPi 3.
Flash eMMC with latest Debian image ¶
- Download Debian image [whatever].img.xz
- Insert microSD card and make sure that it is unmounted
lsblkfdisk -l
Copy the img to the microSD card
This step will take a lot of time.
Mount the microSD card and make sure it flashes to the beaglebone:
Uncomment the line:
- Hookup the USB-to-serial cable
- Connect Black Wire to Pin 1 (closest to 5v barrel)
- Connect Green Wire to Pin 4 (2 pins from pin 1)
- Connect White Wire to Pin 5
- Connect to USB, check output of
dmesgfor which/dev/the serial connection is on Connect to serial connection via screen
- put sdcard in BBB and power up, hold the S2 button, power up
- Wait, the status lights will flash in a cylon pattern, you can watch the eMMC flash progress via
screen All User LEDs should be solid on completion
More info is available on the BeagleBoneBlack Wiki
Initial setup ¶
Connect over SSH ¶
To connect via ssh: plugin beaglebone to computer via SSH
Wifi Setup ¶
- Plugin the wifi adapter
- Reboot (unplug it and plug it back in)
- Generate your pre-shared key using
wpa_passphrase(see Debian Wiki) vim
/etc/network/interfacesifdown wlan0; ifup wlan0- use
ip -o addr showto confirm that you have an ip address
Update debian ¶
Use the Debian Upgrade Script to update debian:
#!/usr/bin/env bash
# Debian auto-upgrade script
# https://debian-handbook.info/browse/stable/sect.automatic-upgrades.html
# kill all cached creds
sudo -k
# ask for new creds
sudo -v
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
yes '' | sudo apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgradeBasic security ¶
Make a root password ¶
- Install
pwgen:sudo apt-get install pwgen - Generate a root password:
pwgen -Bsy 16 1 - Store that password in your password store—you'll never remember it
- Login to beaglebone via ssh, run
passwd
Add a privileged non-root user ¶
- Generate a non-privileged user password:
pwgen -Bsy 16 1 - Save password in password store
Add the user
sudo groupadd sudo-user sudo groupadd ssh-user useradd tyler mkdir /home/tyler mkdir /home/tyler/.ssh touch /home/tyler/.ssh/authorized_keys chown -R tyler:tyler /home/tyler chmod 700 /home/tyler/.ssh chmod 600 /home/tyler/.ssh/authorized_keys usermod -a -G sudo-user tyler usermod -a -G ssh-user tyler usermod --shell /bin/bash tyler passwd tyler- give that user sudo privileges
EDITOR=vim visudo -f /etc/sudoers.d/sudo-user- Add the line:
%sudo-user ALL=(ALL) NOPASSWD:ALL
Add your laptop's key to user's
authorized_keys
Remove demo user ¶
userdel -fr debian
Lockdown ssh ¶
Generate better hostkeys
Modfiy
/etc/ssh/sshd_configto make it like below:- Restart SSH
service ssh restart - Open a new terminal window and make sure you can still login (you may need to delete and reaccept hostkeys)
Fun Stuff ¶
Change ssh banner:
- BeagleBone pin-out
Things to remember ¶
thank god for 1wire temp sensor blog posts:
- http://interactingobjects.com/ds18b20-temperature-sensor-on-a-beaglebone-black-running-ubuntu/
http://klaus.ede.hih.au.dk/index.php/BBB_and_OneWire
iwconfig when can't find wlan0 interface (could be wlan1)
Installing Neovim ¶
Clone repo
Setup Neovim ¶
Get your fingerprint: ¶
Get someone elses key ¶
Using email ¶
Using keyid ¶
Sign keys ¶
Photos ¶
View someone's photo ¶
Add a photo ¶
Suggested image sizes are 240x288 pixels (GnuPG) or 120x144 pixels (PGP) to make a JPEG of 4K-6K in size. A 1.2 h:w ratio w = h / (144 / 120)
Keysigning party etherpad ¶
To Sign keys ¶
Script to quickly sign all keys ¶
IF YOU NEED A KEY, GET HELP!! ¶
Resources ¶
PGP/GPG Intro
- https://ssd.eff.org/en/module/introduction-public-key-cryptography-and-pgp
- https://ssd.eff.org/en/module/how-use-pgp-mac-os-x
OpenPGP Best practices
WMF Keysigning ¶
Mutt problems ¶
"can't query passphrase in batch mode" ¶
Uncomment, use agent in ~/.gnupg/gpg.conf
Pass problems ¶
No Secret Key ¶
Problem ¶
Solution ¶
- install
pinentry-curses ~/.gnupg/gpg-agent.conf
pinentry-program /usr/bin/pinentry
gpg-connect-agent reloadagent /bye
Flash eMMC with latest Debian image ¶
- Download Debian image [whatever].img.xz
- Insert microSD card and make sure that it is unmounted
lsblkfdisk -l
Copy the img to the microSD card
This step will take a lot of time.
Mount the microSD card and make sure it flashes to the beaglebone:
Uncomment the line:
- Hookup the USB-to-serial cable
- Connect Black Wire to Pin 1 (closest to 5v barrel)
- Connect Green Wire to Pin 4 (2 pins from pin 1)
- Connect White Wire to Pin 5
- Connect to USB, check output of
dmesgfor which/dev/the serial connection is on Connect to serial connection via screen
- put sdcard in BBB and power up, hold the S2 button, power up
- Wait, the status lights will flash in a cylon pattern, you can watch the eMMC flash progress via
screen All User LEDs should be solid on completion
More info is available on the BeagleBoneBlack Wiki
Initial setup ¶
Connect over SSH ¶
To connect via ssh: plugin beaglebone to computer via SSH
Wifi Setup ¶
- Plugin the wifi adapter
- Reboot (unplug it and plug it back in)
- Generate your pre-shared key using
wpa_passphrase(see Debian Wiki) vim
/etc/network/interfacesifdown wlan0; ifup wlan0- use
ip -o addr showto confirm that you have an ip address
Update debian ¶
Use the Debian Upgrade Script to update debian:
#!/usr/bin/env bash
# Debian auto-upgrade script
# https://debian-handbook.info/browse/stable/sect.automatic-upgrades.html
# kill all cached creds
sudo -k
# ask for new creds
sudo -v
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
yes '' | sudo apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgradeBasic security ¶
Make a root password ¶
- Install
pwgen:sudo apt-get install pwgen - Generate a root password:
pwgen -Bsy 16 1 - Store that password in your password store—you'll never remember it
- Login to beaglebone via ssh, run
passwd
Add a privileged non-root user ¶
- Generate a non-privileged user password:
pwgen -Bsy 16 1 - Save password in password store
Add the user
sudo groupadd sudo-user sudo groupadd ssh-user useradd tyler mkdir /home/tyler mkdir /home/tyler/.ssh touch /home/tyler/.ssh/authorized_keys chown -R tyler:tyler /home/tyler chmod 700 /home/tyler/.ssh chmod 600 /home/tyler/.ssh/authorized_keys usermod -a -G sudo-user tyler usermod -a -G ssh-user tyler usermod --shell /bin/bash tyler passwd tyler- give that user sudo privileges
EDITOR=vim visudo -f /etc/sudoers.d/sudo-user- Add the line:
%sudo-user ALL=(ALL) NOPASSWD:ALL
Add your laptop's key to user's
authorized_keys
Remove demo user ¶
userdel -fr debian
Lockdown ssh ¶
Generate better hostkeys
Modfiy
/etc/ssh/sshd_configto make it like below:- Restart SSH
service ssh restart - Open a new terminal window and make sure you can still login (you may need to delete and reaccept hostkeys)
Fun Stuff ¶
Change ssh banner:
- BeagleBone pin-out
Things to remember ¶
thank god for 1wire temp sensor blog posts:
- http://interactingobjects.com/ds18b20-temperature-sensor-on-a-beaglebone-black-running-ubuntu/
http://klaus.ede.hih.au.dk/index.php/BBB_and_OneWire
iwconfig when can't find wlan0 interface (could be wlan1)
Installing Neovim ¶
Clone repo
Setup Neovim ¶
I just finished reading The Python Corner’s post “Lambdas and Functions in Python.” The post acts as an introduction to the use of functions as first-class objects in python. The demo code is the implementation of a “Reverse Polish Notation” calculator.
I had never heard of reverse polish notation(RPN) before this post. The short explanation of RPN is available on Wikipedia:
In reverse Polish notation, the operators follow their operands; for instance, to add 3 and 4, one would write 3 4 + rather than 3 + 4.
In that blog post RPN is implemented as a stack of operands and after an operator is pushed onto the stack, the compute() method is called which triggers the evaluation of the lambda specified by the operator. Like this:
The point of the post is to show a python dict using operands as keys with lambdas as values; this demonstrates that lambdas are functions and functions are first-class objects. This allows the compute method of the RPNEngine class to look up a lambda in a dict, and pop() off the stack using the signature function of the inspect module to determine how many arguments are needed for a particular lambda. From there, lambda evaluation is handed off to helper functions named, for instance, compute_operation_with_two_operands and compute_operation_with_one_operand
Currying ¶
One other functional concept that could have helped the example code is that of currying. Currying involves changing a function with multiple arity into a series of evaluations of multiple functions each with an arity of 1.
This is a fancy way to say:
add = lambda x, y: x + y
add_curried = lambda x: lambda y: x + y
assert(add(2, 2) == add_curried(2)(2))By turning the compute_operation_with_n_operands-type functions into curried functions, the code gets much cleaner. That is, instead of a switch like:
if number_of_operands == 2:
self.compute_operation_with_two_operands(self.catalog[operation])
if number_of_operands == 1:
self.compute_operation_with_one_operand(self.catalog[operation])You can implement a curried function using a callable python object and do something like:
func = self.catalog[operation]
while not func.resolved:
func(self.pop())This gets rid of the clunky compute_operation_with_n_operands functions. Here is the full code for a solution using currying:
#!/usr/bin/env python3
"""
Engine class for RPN Calculator
"""
import math
from functools import partial
from inspect import signature
class Curry(object):
"""
Curry a callable
Given a callable, returns a an object that can be used like a curried
callable.
>>> c1 = Curry(lambda x, y: x + y)
>>> c2 = Curry(lambda x, y: x + y)
>>> c1(2, 2) == c2(2)(2)
True
:func: callable
"""
def __init__(self, func):
self.func = func
self.argc = len(signature(self.func).parameters)
self.resolved = False
self.answer = None
def __call__(self, *args):
if len(args) == self.argc:
self.answer = self.func(*args)
self.resolved = True
for arg in args:
self.func = partial(self.func, arg)
self.argc = len(signature(self.func).parameters)
return self
class RPNEngine(object):
"""
Reverse Polish Notation (RPN) Engine
A RPN calculator
>>> rpn = RPNEngine()
>>> rpn.push(2)
>>> rpn.push(2)
>>> rpn.compute('+') == 4
True
>>> rpn.compute('AC')
>>> rpn.push(2)
>>> rpn.compute('^2') == 4
True
"""
def __init__(self):
self.stack = []
self.functions = self._get_functions()
def _get_functions(self):
return {
'+': Curry(lambda x, y: x + y),
'-': Curry(lambda x, y: x - y),
'*': Curry(lambda x, y: x * y),
'/': Curry(lambda x, y: x / y),
'^2': Curry(lambda x: x * x),
"SQRT": Curry(lambda x: math.sqrt(x)),
"C": Curry(lambda: self.stack.pop()),
"AC": Curry(lambda: self.stack.clear()),
}
def push(self, item):
self.stack.append(item)
def pop(self):
try:
return self.stack.pop()
except IndexError:
pass
def compute(self, operation):
func = self.functions.get(operation)
if not func:
raise BaseException('%s not a valid function' % operation)
if len(self.stack) < func.argc:
raise BaseException(
'%s requires %d operands, %d given' % (
operation,
func.argc,
len(self.stack)
)
)
if func.argc == 0:
func()
while not func.resolved:
func(self.pop())
return func.answerReading the final code in the Python Corner post made me me really itchy to implement the solution I posted here.