OS Boostrap Script for Python/Node Application

Today the EC2 machine that I have been using to run my blog application for the past couple years expires, so I had to move to a new instance. Unfortunately, while I had done a great job of automating the setup for my blog, I had not done a good job of automating the bootstrapping of the operating systems that I use. I think it is really important that system initialization steps be automated, so you can easily deploy additional nodes, rebuild exact systems from scratch, and quickly recover from a disaster.

To make things easier, I wrote a bootstrapping script for the two operating systems that I use (OSX and Ubuntu), which sets up the software I need to develop in Python and Node.js, and start the initialization script that is part of my blog repo. This article will introduce my bootstrapping script.

How do it…

The canonical reference will always be at https://gist.github.com/mattsnider/5485500. Here is what the script looked like at the time of this writing:

#!/bin/bash
# expects python is installed with OS distribution

# single line command for execution
# wget -O - <RAW_URL> | bash

# determine environment
if hash apt-get 2>/dev/null; then
    echo "Bootstrapping UBUNTU"
    UBUNTU=true
    # update apt-get and install a C compiler
    sudo apt-get -y update && sudo apt-get -y upgrade
    sudo apt-get -y install gcc
else
    echo "Bootstrapping OSX"
    # on OSX we use brew as the application repo
    ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
    brew doctor
    OSX=true
fi

# install python, when necessary
if [ $UBUNTU ]; then
    sudo apt-get -y install python-dev python-setuptools
fi

# setup pip and virtualenv
sudo easy_install pip
sudo pip install virtualenv
sudo pip install virtualenvwrapper
mkdir $HOME/.virtualenvs

# install git
if [ $UBUNTU ]; then
    sudo apt-get -y install git
fi
if [ $OSX ]; then
    brew install git
fi

# install node and npm
if [ $UBUNTU ]; then
    sudo apt-get install node npm
fi
if [ $OSX ]; then
    brew install node npm
fi

To run the script as a single command, grab the raw URL from the gist and execute:

wget -O - <RAW_URL> | bash
# eg. wget -O - https://gist.github.com/mattsnider/5485500/raw/ec8b38022145dac41b1ef4ad5dd262cc845b4ee0/bootstrap.sh | bash

How it works…

The script is broken up logically and comments atop each section describe what the section is doing. First, the code determines the operating system by checking if the program apt-get exists. For either OS a temporary variable is created to be used by later if statements. When the OS is Ubuntu, it updates apt-get and ensures that a C compiler is installed. When the OS is OSX, it installs brew which will be used as the application packaging tool for installing new software.

The next step is to ensure python and python-setuptools are installed, so we have python and can use easy_install to install pip. This is kept logically separate from the first step, in-case I decide to support additional operating systems in the future.

The third step installs pip, virtualenv and virtualenvwrapper, and ensures that the required ~/.virtualenv exists. These tools are a must have for any python developer.

Next the script installs git for repository management and lastly it installs node and npm for development in Node.js.

At this point the system is ready to run whatever initialization scripts you have for your application. For my blog, I need to manually clone the git repo, create a virtual environment, and install fabric in that environment. Then I have a fabric script that automates the rest of the installation.

Context Binding Functions

I realized this week that I have never dedicated an article to context (scope) binding functions in JavaScript and thought I would remedy that today. JavaScript context binding functions are functions that guarantee the execution context of a callback function, allowing the developer to control what this is set to.

How do it…

We’ll look at two types of binding functions. The first, seamlessly wraps an inner callback function with a provided context, so ...

Multi Tab/Window Messaging in JavaScript

I recently implemented a JavaScript only communication system for sending messages between windows/tabs (just referred to as window for the remainder of this document) for Ariba.com. We have a very heavy-handed approach that requires identifying each window and queueing messages in the order that are sent, no matter the originating window. And the final kicker is that it needs to work on some older browsers. We settled on a shared communication channel using ...

Crowd-Sourced Computing on the Web - Proof of Concept

Years ago services like [email protected] introduced us to crowd-sourcing computing cycles in order to analyze big data. Unfortunately, these programs require that you install some software on your computer and keep it running. Lately, I’ve been noodling over applying this same concept to the web, and I thought I’d share my thoughts. When somebody is viewing a webpage, most of the time the browser is relatively idle, and probably their machine as well. ...

Hash-Hack for Cross-Domain IFrame Communication

I recently needed a simple, legacy browser (*cough* IE 6) compatible, solution for sending cross-domain communication between an iframe and its parent window. Fortunately, the URL protocol allows changing the hash part of the URL without causing the page to refresh, and the same is true when updating the URL of an iframe. This means that you can use a hash-hack to communicate between the frames. If you do not need to support < IE ...

Simple JavaScript Requirement System for Asynchronous Scripts

I recently went through the chore of making sure that all the scripts on this site are loaded asynchronously using JavaScript Deferment, as shown in Django Template Tags for JavaScript Deferment. This ended up causing problems with my JavaScript, because most of my scripts depend on blog.js, which is now loaded asynchronously. I decided to write a simple require system that other scripts could use to ensure that blog.js is already loaded, and today ...

Python-Style Decorators in JavaScript

Python-style decorators (or Java annotations) are a useful feature that is not natively available in JavaScript. This article describes a technique for apply Python-style decorators to JavaScript functions. We’ll call them annotations, so not to confuse them with the decorator pattern.

How do it…

To illustrate what an annotation will be, lets try a simple example:
 function alertFoo() { alert('foo'); } alertFoo = annotate(alertFoo).by(alertBar); 
We annotate the alertFoo function ...

Develop Locally Using Git, Instead of Your Company's Crappy Version Control

At my new company (Ariba), I am forced to use a beastly version control system (Perforce) and it was really cramping my style. I couldn’t easily branch or check in small incremental changes (even locally), so I was going days to weeks without checking in, waiting until my code was production ready. This meant if I made a mistake and had to roll back, or wanted to maintain multiple version of a file, I ...

Shell Sort

Continuing our sorting algorithm discussion, today’s article will cover a technique known as Shell Sort. Shell sort is a quadratic (O(n2)) in-place comparison sort algorithm developed by Donald Shell in 1959 that is basically Insertion Sort with a gap between comparing indexes. The assumption is that comparing farther apart elements can move some out-of-place elements into position faster.

How do it…

The first thing we need is an algorithm to ...

Encrypting Settings Files

When I was developing for Votizen, I always felt uneasy that we had our settings files with many important passwords and auth keys stored unencrypted on a third-party service. Sure, SSL was used for fetching and committing code, and only a few SSH public keys were allowed access to our repository, but it is one more opportunity for your site to be hacked. I recently read John Resig’s post Keeping Passwords in Source ...