EDIT: 2016-08-19 I got an email a very long time ago that I meant to post about on ye old weblog from a fellow named Tom Duck who built a CMS based around some of the ideas presented here. It’s called Bassclef and it looks pretty amazing and folks should check it out. I used to use del.icio.us to keep track of links, then it went away. After del.icio.us shutdown, I used a ton of uniquely awful services to keep track of links. Eventually, I came around to the idea that all I needed was a series of markdown files and github: BOOM! Public link repositiory—just like del.icio.us back in the day. After a while I started thinking, I could make these files a lot more presentable if I did some jekyll-ifying and served them out on github. My and then I got tylercipriani.com/linuxtips; neat. I ran into some problems with that approach along the way. Mostly problems with git and the separate branches and the order in which I’d commit and pull and whatever, it was/is a headache. I started thinking about Pandoc. Pandoc is this haskell library that makes miracles of text happen. Got an org-mode file and need TeX? Done. Got a markdown slideshow that needs to become a beamer slide show? OK, sure. Fuck Beamer, how about markdown slides → Reveal.js slides? You bet your sweet sensual bologna. then add it to your path in your Alright, so I’ve got tons of markdown files, fairly structured, with bunches of links and I need html5. I’ll create a Running So now that I’m outputting html, I still need to be able to: Pandoc is able to do all of these things—easy-peasy-lemon-squeezy. First, to establish a layout, let’s copy the default html5 layout file for Pandoc: I’ll make some small tweaks to that file, keep the variables I need, ditch the variables I don’t need. Here is the html5 layout file I came up with: Next, I need to figure out how to include a css stylesheet. A quick search for Updated Finally, I need to be able to include a unique A bit more digging in the online docs, however, nets better solution: YAML Metadata blocks—just like jekyll! So, for each markdown file in my repo, I’ll add a block to the top that looks like this: Alright, so now that I have the basics of Pandoc down, I need to whip my First thing is I want to convert ALL of my markdown files to html, not just the Running This I’m a big fan of pre-processors, so the css/main.css file (which doesn’t actually exist as of yet) should be converted from Now running Also, it’s always nice to have a I’d also like to be able to preview before commiting this file by typing Finally, speaking of commiting this file, let’s make Now when I update my So here’s the result and the final EDIT: 2017-01-08 Igor Rzegocki has a project that uses the same technique outlined here to make some incredible ready-made Street FIghter MotDs – it is really cool, and if you are interested in abusing ansi escape sequences to make MotDs you should check it out :) Everyone universally agrees that most Message of the Days (MOTDs) are stupid and suck. By the end of reading this post, your mind grapes should be swollen with the knowledge of how to make an MOTD that isn’t stupid and, some would say, doesn’t suck. This should have you covered: I use Util-Say to create motd messages. I started out using img2xterm, but I’ve found I get better results with Util-Say. You can also try On CentOS and Debian, I usually just throw the ponysay file directly into On Ubuntu Boxes (since they use update-motd(1)), I do pretty much the same thing except I just make a bash script in There are likely better articles on creating a useful MOTD, (here’s one that looks kinda cool) but there are exactly none-better articles on creating MOTDs that are so flossy! I’ve used Vagrant a fair amount in my day, and it’s great. I enjoy being able to spin-up toy linux environments to test out ideas. I tend to use the Chef provisioner with Vagrant to build-out a local environment that matches my server fairly closely. I’ve been thinking about Chef a lot lately. Often is the time, in moments of profound frustration, that I’ve had the thought that Chef is nothing more than a useless, leaky abstraction that separates me from something I know fairly well—Linux. This thought is usually fleeting: Chef provides many needed abstractions that are, ultimately, much easier to grok than the underlying Linux system. Further, Chef allows you to keep a(n ostensibly) well-tested system under version control. I’ve come to the realization that my problem with Chef is not really a problem with Chef, but a problem with Linux itself. Linux system administration is difficult because Linux commands are non-deterministic and rely heavily on system state (e.g., installed software, permissions, network settings and availability). Maintaining a bare-metal, long-running server non-interactively using Chef sucks because any hand-tinkering via ssh is going to fuck with the “state” of the system—creating different results for subsequent chef-client runs. This system state adjustment may or may not be reflected in the chef repository (which double sucks). I started to think about Docker. I feel Docker addresses the problem of program state better than other currently available solutions (although, Nix is looking pretty promising as well). While Docker is still a Linux system—and, ipso facto, state-dependant—it’s also ephemeral and therefore, by not persisting changes to state, Docker has created a previously unavailable (on bare metal hardware), lightweight workaround to the problem of system state. As is my wont, I decided today to play a bit with Docker on Vagrant and, lo-and-below, I found that the newest version of Vagrant (1.6.2, as of May 26th) can actually use docker as a provider, that is, as an alternative to VirtualBox. Using Docker as a provider means that you can run a fully-independent development enviroment, on your host machine without the overhead of VirtualBox. Neat. “Imma setup a local development environment for Ubuntu 14.04, nginx and php-fpm using Vagrant, Supervisord and Docker,” says I. To keep my project directory nice and tidy, I’ve separated-out most of the files needed by the Docker provider into a The The Since docker handles so much of what was previously handled by Vagrant provisioner, the In mine, I’ve got: Most of the work of provisioning a container is handled by Docker and the Dockerfile. In fact, if you were only ever going to run this container on a Linux machine, I don’t think that Vagrant adds any needed functionality to the This This config also starts supervisor with the With all of our configuration in place there isn’t much left to do aside from running the vagrant instance and allowing docker to create our container. With that, you now have a container running nginx and php-fpm that is sharing a folder with you at This process is really simple AND super lightweight. I’ve been running my docker/vagrant instance for about 45 minutes alongside chrome and tmux/xterm without any noticeable jankyness on a notoriously janky laptop.
Previously Jekyllfied ¶
linuxtips repo is just a dumb jekyll repo. Esentially all linuxtips is is just a big README file. So, for that repo, I created a gh-pages branch with a _config.yml and a _layout directory and popped in a Makefile:INDEX = ${CURDIR}/index.md
$(INDEX):
@ git show origin/master:README.md > $@
@ perl -i -pe 'print "---\nlayout: default\ntitle: Linux Tips\n---\n\n" if $$. == 1;' $@Pandoc ¶
Imma install Pandoc… ¶
Debian? ¶
.${SHELL}rc file:OSX? ¶
Imma Use Pandoc… ¶
Makefile proof-of-concept for this:make takes my README.md and makes this:<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="generator" content="pandoc">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
<title></title>
<style type="text/css">code{white-space: pre;}</style>
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body>
<h1 id="tyler-ciprianis-bookmarks">Tyler Cipriani's Bookmarks</h1>
<p>In an effort to <em>not</em> have 100+ tabs open…</p>
<ul>
<li><a href="http://www.flickr.com/photos/tylercipriani/">My Photography</a></li>
<li><a href="Design.md">Design</a></li>
<li><a href="Development.md">Development</a></li>
<li><a href="Business.md">Business</a></li>
<li><a href="Fun.md">Fun</a></li>
</ul>
</body>
</html>Title/Layout/CSS ¶
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>$if(title-prefix)$$title-prefix$ - $endif$$pagetitle$</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="http://fonts.googleapis.com/css?family=Open+Sans:800">
$for(css)$
<link rel="stylesheet" href="$css$">
$endfor$
</head>
<body>
<!--[if lt IE 9]>
<script src="http://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
<main class="container">
$body$
</main>
</body>
</html>css in pandoc(1) turns up the --css flag which enables you to link to a css stylesheet.Makefile:index.html: README.md
pandoc -s --template "_layout" --css "css/main.css" -f markdown -t html5 -o "$@" "$<"<title> tag string for each page. Again, a search through pandoc(1) for variable yields results; using the -V flag enables you to set template variables using -V [KEY]=[val].$pagetitle$ is the variable I defined in my _layout.html5 that I’m now passing as a template to Pandoc.Makefile fanciness ¶
Makefile into shape.README.md. So howzabout I add a wildcard target for all the html files? Also, the whole point of this is to make a github pages site, so let’s add that to the Makefile tooREPO := $(shell git config --get remote.origin.url)
GHPAGES = gh-pages
all: $(GHPAGES) $(addprefix $(GHPAGES)/, $(addsuffix .html, $(basename $(wildcard *.md))))
$(GHPAGES):
git clone "$(REPO)" "$(GHPAGES)"
(cd $(GHPAGES) && git checkout $(GHPAGES)) || (cd $(GHPAGES) && git checkout --orphan $(GHPAGES) && git rm -rf .)
$(GHPAGES)/%.html: %.md
pandoc -s --template "_layout" -c "css/main.css" -f markdown -t html5 -o "$@" "$<"make at this point should checkout your current git repository to a subdirectory called gh-pages (which should be added to .gitignore on master).Makefile first tries to checkout an existing gh-pages branch, otherwise it creates a new orphan branch for gh-pages. After that, we glob the current directory for any file name *.md and run it through pandoc, placing the result in gh-pages/[whatever].htmlNiceities ¶
less. The easiest way to do that: add a package.json with less as a dependency.npm install should create a new node_modules directory (which should be added to .gitignore on master). Now we need to add a lessc step to our Makefile.LESSC = node_modules/less/bin/lessc
LESSFILE = less/main.less
CSSDIR = $(GHPAGES)/css
CSSFILE = $(CSSDIR)/main.css
$(CSSFILE): $(CSSDIR) $(LESSFILE)
$(LESSC) "$(LESSFILE)" "$(CSSFILE)"
$(CSSDIR):
mkdir -p "$(CSSDIR)"clean target in any Makefilemake servecommit a target, too.commit:
cd $(GHPAGES) && \
git add . && \
git commit --edit --message="Publish @$$(date)"
cd $(GHPAGES) && \
git push origin $(GHPAGES)links repo’s markdown files I issue a simple series of commands: make checks-out my gh-pages branch and builds the html and css files, make serve lets me look at the output html at localhost:8000, and, finally, make commit pushes those changes live.MakefileREPO := $(shell git config --get remote.origin.url)
GHPAGES = gh-pages
LESSC = node_modules/less/bin/lessc
LESSFILE = less/main.less
CSSDIR = $(GHPAGES)/css
CSSFILE = $(CSSDIR)/main.css
all: init clean $(GHPAGES) $(CSSFILE) $(addprefix $(GHPAGES)/, $(addsuffix .html, $(basename $(wildcard *.md))))
$(GHPAGES)/%.html: %.md
pandoc -s --template "_layout" -c "css/main.css" -f markdown -t html5 -o "$@" "$<"
$(CSSFILE): $(CSSDIR) $(LESSFILE)
$(LESSC) "$(LESSFILE)" "$(CSSFILE)"
$(CSSDIR):
mkdir -p "$(CSSDIR)"
$(GHPAGES):
@echo $(REPO)
git clone "$(REPO)" "$(GHPAGES)"
@echo "Donezo"
(cd $(GHPAGES) && git checkout $(GHPAGES)) || (cd $(GHPAGES) && git checkout --orphan $(GHPAGES) && git rm -rf .)
init:
@command -v pandoc > /dev/null 2>&1 || (echo 'pandoc not found http://johnmacfarlane.net/pandoc/installing.html' && exit 1)
@[ -x $(LESSC) ] || npm install
serve:
cd $(GHPAGES) && python -m SimpleHTTPServer
clean:
rm -rf $(GHPAGES)
commit:
cd $(GHPAGES) && \
git add . && \
git commit --edit --message="Publish @$$(date)"
cd $(GHPAGES) && \
git push origin $(GHPAGES)
.PHONY: init clean commit serve
Prerequisites ¶
Creating the Util-say file ¶
$ git clone https://github.com/maandree/util-say
$ cd util-say
$ ./img2ponysay -- yourimg.png > yourimg.txt./img2ponysay -2 -- youimg.png > yourimg.txt but I’ve never had good results with thatMOTD-ifying ¶
/etc/motd and maybe add on some other useful info:$ sudo cat yourimg.txt > /etc/motd
$ sudo figlet "$(hostname)" >> /etc/motd
$ sudo printf "Public IP: $(dig +short myip.opendns.com @resolver1.opendns.com)\n" >> /etc/motd/etc/update-motd.d/25-baller-motd#!/usr/bin/env bash
cat yourimg.txt
figlet "$(hostname)"
printf "Public IP: $(dig +short myip.opendns.com @resolver1.opendns.com)\n"
command -v fortune &> /dev/null && fortuneMy Ever-Evolving Rant About DevOps ¶
Why Docker Curtails My Rage ¶
Project Layout ¶
Docker folder. This results in the directory structure below.├── Docker
│ ├── Dockerfile
│ ├── nginx
│ │ └── default
│ ├── php-fpm
│ │ └── php-fpm.conf
│ └── supervisor
│ └── supervisord.conf
├── Vagrantfile
└── www
└── index.phpDockerfile is used to build the main docker machine and the subfolders in the Docker directory contain configuration used in the Dockerfile.www folder is my fake php project folder.VagrantFile ¶
Vagrantfile for a Docker-backed Vagrant instance is pretty sparse.Dockerfile ¶
docker.io cli. In terms of portability, however, Vagrant is, at this time, a necessary evil to run docker on OSX and Windows.FROM ubuntu:latest
MAINTAINER Tyler Cipriani, tyler@tylercipriani.com
# Download and install php, nginx, and supervisor, hey, just linux for a change!
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:nginx/stable
RUN apt-get update
RUN apt-get -y dist-upgrade
RUN apt-get install -y php5-fpm nginx supervisor
# Setup config files
RUN echo "daemon off;" >> /etc/nginx/nginx.conf
ADD ./nginx/default /etc/nginx/sites-enabled/default
ADD ./supervisor/supervisord.conf /etc/supervisor/supervisord.conf
ADD ./php-fpm/php-fpm.conf /etc/php5/fpm/php-fpm.conf
# Shared volume
RUN mkdir -p /var/www
VOLUME ["/var/www"]
# Default command for container, start supervisor
CMD ["supervisord", "--nodaemon"]
USER root
# Expose port 80 of the container
EXPOSE 80Dockerfile takes care of building a docker container from the latest Ubuntu image (14.04 as of May 26th, 2014). Running this code installs:
--nodaemon flag by default. Docker can run a container running a non-daemonized program as a daemon (much like supervisor can run non-daemonized programs as daemons). Supervisor is used as a way of running both nginx and php-fpm as non-daemonized programs. It is also noteworthy that the dockerfile creates and/or modifies configuration files for php-fpm and nginx to make sure they both run in non-daemon mode.nginx/defaultserver {
listen 80 default_server;
root /var/www;
index index.php index.html;
# pass the PHP scripts to FastCGI server
location ~ \.php$ {
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
}php-fpm/php-fpm.conf[global]
pid = /var/run/php5-fpm.pid
error_log = /var/log/php5-fpm.log
daemonize = no
include=/etc/php5/fpm/pool.d/*.confsupervisor/supervisord.conf[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0700 ; sockef file mode (default 0700)
[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket
[program:php5-fpm]
command=/usr/sbin/php5-fpm -c /etc/php5/fpm
stdout_events_enabled=true
stderr_events_enabled=true
[program:nginx]
command=/usr/sbin/nginx
stdout_events_enabled=true
stderr_events_enabled=trueJam Time ¶
$ sudo docker pull ubuntu # to grab the latest Ubuntu image, Vagrant should probably do this but doesn't
$ sudo vagrant up --provider=docker --debug # use debug if you don't want to sit waiting with no info for a long time on the first run/var/www. Navigating to http://localhost:8080/index.php should show you the contents of your ./www/index.php file.
Posted
Posted
Posted