Using PhantomCSS for Regression Testing Your CSS

I finally had a chance to play with PhantomCSS, a CSS regression testing tool that compares screenshots to see if CSS changes have affected the design, and wanted to share my experience with you. We are using it at Ariba to validate that I do not break any designs as I migrate us from two giant, unmanageable CSS files into smaller, modularize and better written lessCSS files.

PhantomCSS uses CasperJS, built on PhantomJS as a testing framework with screenshot capabilities, and ResembleJS to compare images.

It was surprisingly easy to get it to work when I used all the sub-components packaged with the download, but incredible difficult when I tried to use the sub-components I already had installed via npm.

Getting ready

The easiest way to get started is to clone the project and cd into it:

git clone https://github.com/Huddle/PhantomCSS.git
cd PhantomCSS

The project contains a version of CasperJS, resembleJS, and PhantomJS. However, the included PhantomJS only works on windows. For installation on other systems, check out the PhantomJS Installation Guide (version >=1.2 was required at the time of this writing). Except for PhantomJS, you should make sure you are using the included software versions (if you don’t change anything, then you should be using the included packages). I initially tried to use the CasperJS that I had previously installed from NPM and wasted a lot of time trying to get it PhantomCSS working with the latest version of CasperJS, when the included package works right away.

How do it…

To ensure everything is working run the test applications twice:

phantomjs demo/testsuite.js

You should see the following the first time:

Must be your first time?
Some screenshots have been generated in the directory ./screenshots
This is your 'baseline', check the images manually. If they're wrong, delete the images.
The next time you run these tests, new screenshots will be taken.  These screenshots will be compared to the original.
If they are different, PhantomCSS will report a failure.

THE END.

And this the second:

PhantomCSS found: 4 tests.
None of them failed. Which is good right?
If you want to make them fail, go change some CSS - weirdo.

The test application creates a simple nodeJS webserver and sends a couple of requests against it. The screenshots will be in the screenshots directory. If you want to see what happens when a regression is detected, change a style or visible text in demo/coffeemachine.html and rerun testsuite.js.

Now, to get started with your own test. Create a new testing file:

touch demo/mytest.js

And put the following code in the file (trimmed from testsuite.js):

/*
	Initialise CasperJs
*/

phantom.casperPath = 'CasperJs';
phantom.injectJs(phantom.casperPath + '/bin/bootstrap.js');
phantom.injectJs('jquery.js');

var casper = require('casper').create({
	viewportSize: {
		width: 1027,
		height: 800
	}
});

/*
	Require and initialise PhantomCSS module
*/

var phantomcss = require('./phantomcss.js');

phantomcss.init({
	screenshotRoot: './screenshots',
	failedComparisonsRoot: './failures'
});

/*
	The test scenario
*/

var url = 'http://www.mattsnider.com'; // replace with your URL

casper.
	start(url).
	// screenshot the initial page load
	then(function() {
		phantomcss.screenshot('<SELECTOR_TO_SCREENSHOT>', '<LABEL_SCREENSHOT>');
	}).
	then(function() {
		// do something
	}).
	// second screenshot
	then(function() {
		phantomcss.screenshot('<SELECTOR_TO_SCREENSHOT>', '<LABEL_SCREENSHOT>');
	});

/*
	End tests and compare screenshots
*/

casper.
	then(function now_check_the_screenshots() {
		phantomcss.compareAll();
	}).
	run(function end_it() {
		console.log('\nTHE END.');
		phantom.exit(phantomcss.getExitStatus());
	});

Change the casper test as necessary and run it against your site:

phantomjs demo/mytest.js

How it works…

Most of the navigation and testing work is handled by CasperJS, so most likely any you want to do/test is documented here. That said, at the time of this writing PhantomCSS was still on Casper 1.0.2, while the documentation was for version 1.1.0 (I couldn’t find an active version of 1.0.2 documentation, if you find one, please let me know in the comments). The simple example we build (mytest.js) just opens a URL, takes a screenshot, does something that you specify, and takes another screenshot. From this starting point you should be able to begin building more comprehensive tests.

The first time PhantomCSS runs it will perform all specified operations and take a series of screenshots (as specified by calls to phantomcss.screenshot('', '');), but the phantomcss.compareAll(); will print out a message saying that there is nothing to compare. You have just captured screenshots of the unchanged, baseline version of your site. Now go make changes to the CSS of your site and next time PhantomCSS runs, it will create new screenshots and compare them to the original. Should the screenshots be different, then PhantomCSS will error and tell you what steps failed.

The boilerplate at the top of mytest.js is used to find and hook all the component libraries into PhantomJS, before we create a webpage using CasperJS. Next PhantomCSS is initialized, specifying the screenshot and failure directories. There are a lot of other configuration properties that may be specified, but these two are the most important for simple tests. CasperJS is invoked by calling .start(url). CasperJS uses the promise pattern, so you can chain function calls (a shown in the example above), or you can in call casper. individually (as shown by the end section).

There’s more…

Lastly, here are some pointers and gotchas with CasperJS that might help you:

  • Use this.echo("your string"); from inside of a callback function to print something to the terminal (or you may also use require("utils").dump("your string");), but console.log won’t print to the terminal.
  • Use this.echo(this.getHTML(selector, true)) when you aren’t sure that the correct element was found by the selector.
  • Inside the this.evaluate(callback) and this.thenEvaluate(callback) callback functions, you can run JavaScript against the page as you normally would (such as document.getElementById(id)), but you won’t have access to the CasperJS helper functions.
  • Attach listeners to CasperJS if you want to print the messages and alerts that are triggered by the HTML page you are evaluating (see code below).

function printMsg(msg) {
    casper.echo(msg);
}
casper.on("remote.alert", printMsg);
casper.on("remote.message", printMsg);

OS Boostrap Script for Python/Node Application

Today the EC2 machine that I have been using to run my blog application for the past couple years expires, so I had to move to a new instance. Unfortunately, while I had done a great job of automating the setup for my blog, I had not done a good job of automating the bootstrapping of the operating systems that I use. I think it is really important that system initialization steps be automated, so ...

Diablo 3 Database Tool

Lately, in my spare time, I have been playing Diablo 3. As my characters become more powerful, I spend more and more time scrounging the auction house looking for gear, and wondering how I stack up against other players, characters, and builds. While the game itself is well thought out, the auction house, player rewards, and community aspects seem to be tacked on at the last minute without as much forethought, and unfortunately there is ...

A Python JSON Client for the LinkedIn API

The LinkedIn API is fairly robust and well documented, but is lacking a good JSON-based Python API for interacting with it. I recently opened-sourced LinkedIn-API-JSON-Client to fill this gap. It currently implements all the user profile related API calls, and is used in production by Votizen.com. This is a simple tutorial for how you can use it for your application as well.

Getting ready

You will need Python 2.x running in a ...

Using Sphinx to Easily Manage Engineering Documents

As an engineering organization grows, eventually there is a need to document more than just code comments. When this happens, there are many solutions for handling documentation, and most of them are equally bad. Previously, at Votizen.com we used a wiki, but it was difficult to organize, search, and links/documents rot. Recently, we chose to ditch the wiki, converting our documents to reStructuredText (.rst), and instead use the tool Sphinx to compile our documents ...

SyntaxHighlighter

Part of this blogs new design will include a code syntax highlighter, which will make reading code snippets and copying them for personal use easier. The syntax highlighter is a JavaScript solution that will covert specially formatted pre tags into clean documentation. It is the same tool used by YUI throughout their documentation and is public available at Syntax Highlighter Project. Unfortunately, the latest version of the system (1.5) is not well documented. Most ...

Google Page Speed

Google recently released their internal page speed analysis tool. On the surface this tool appears a lot like ySlow, but looking deeper into the analysis reveals many more ways to improve the performance of a website beyond the ySlow suggestions.

Using YUI Doc to Document JavaScript Files

To start, I need to say that the YUI Documentation tool is, by far, the most difficult to use tool developed by the YAHOO team. Getting YUIDoc working will take some time and patience. For starters, Python must be installed with several addons. Then once Python is working, your JavaScript files need to be completely reorganized, as each module must be in a separate directory. And lastly, if you augment an object in a ...

FireBug Console Emulator

FireBug is the most powerful debugging tool available to web application developers. There are a myriad of well exposed features, such as break points, stack traces, HTML editing, network monitoring, etc. However, there is another powerful feature that is often overlooked; FireBug adds an object, console, to the JavaScript window, which allows the developer to log various actions in their code. But like the rest of FireBug, this feature is only available when ...

Node Tree Visualization

Originally, this post was intended to be a response to sams question about an efficient way to store node trees. While, he found the answer to his question, I wanted to compare the searching efficiency for a few different methods for storing node trees. However, I was having too much fun working on the node tree visualization algorithm, so I want to share it today, and I will get to the tree node searching ...