This Thursday, May 26 2011, I will be speaking on a Continous Deployment panel for Wealthfront & IMVU in downtown Palo Alto. In preparation for the talk, I thought I'd start a series explaining how easy it is to do continous deployment in Python/Django. Unfortunately, writing this first article took me two weeks, so the rest of the articles will be released after the panel. Over the next couple weeks, we will be discussing the following:

• Starting Your First Django Project
• Testing and Django
• Mock and Coverage
• Using Fabric for Painless Scripting
• Using Celery to handle Asynchronous Processes
• Deployment/Monitoring Strategies

Today's article will discuss setting up Django and focus on tips for building apps. There is a lot to cover, and I won’t be going into much depth. Please consult the Django site, if you get stuck (links included at end and throughout article).

Getting ready

To familiarize yourself with how to start a Python project, see Getting Started with Python Development.

If you followed that article, then open the terminal and enter your projects context:

start_myproject

Next install the version of Django you want:

pip install Django==1.3

Currently, Django only supports Python 2.x, so if you are using Python 3.x, you'll have to install 2.x and configure your system to use it.

How to do it…

Once Django is installed, you can use the following to start a new project (mysite):

django-admin.py startproject mysite

You will now see the directory mysite. At this point it is a good idea to put your project’s directory under some kind of source control.

There will be the following files in the mysite directory:

mysite/
    __init__.py
    manage.py
    settings.py
    urls.py

You now have a working Django project. It doesn't do anything, but it works. The follow command starts the Django development app server on port 8000:

python manage.py runserver

To start a Python shell with the Django context (useful for debugging), use:

python manage.py shell

Next, you'll need to install some kind of backend for your models, such as MySQL:

pip install MySQL-python==1.2.3

The settings.py file is where you configure your entire Django project (see the Django documentation for more info). At the very least you'll be wanting to change the DATABASES and INSTALLED_APPS variables:

DATABASES = {
	'default': {
		'NAME': 'DB_NAME',
		'ENGINE': 'django.db.backends.mysql',
		'USER': 'DB_USER',
		'PASSWORD': 'DB_PASSWORD',
		'HOST': 'DB_HOST',
	}
}
…
INSTALLED_APPS = [
	'django.contrib.admin', # Django admin app
	'django.contrib.admindocs', # Easy doc gerneration tool
	'django.contrib.auth', # Django managed authentication
	'django.contrib.contenttypes', 
	'django.contrib.humanize', # Extra template tags and filters
	'django.contrib.messages', # Django managed sessions
	'django.contrib.redirects', # Easy view redirection
	'django.contrib.sessions', # Django managed sessions
	'django.contrib.sitemaps', # Django sitemap generation app
	'django.contrib.staticfiles', # server static files locally
	'myapp1', # the app we create in this article
#	'south', # 3rd party app used for DB migrations
]

Django developers recommend that you build features as discrete self contained apps, living inside of your project. Each of the django.contrib entries in the INSTALLED_APPS of the previous example represents a self contain app that does something. To start your own app type:

python manage.py startapp myapp1

This will create the following files:

myapp1/
    __init__.py
    models.py
    tests.py
    views.py

I recommend also creating the following files:

myapp1/
    templates/
        base.html
        home.html
    template_tags/
    base.py
    context_processors.py
    forms.py
    listeners.py
    managers.py
    middleware.py
    urls.py

All of these files will be briefly explained below.

myapp1/__init__.py

The __init__.py is used by Python to indicate that this directory should behave like a Python module. Beginners shouldn't touch this file, as it can cause circular dependencies and other nastiness, except to document what the app does and its requirements.

myapp1/models.py

Models are the mechanism used by Django to communicate with your database. Here is a simple user profile object:

from datetime import datetime
from django.db import models
from django.contrib.auth.models import User
from myapp1.managers import UserProfileManager

class UserProfile(models.Model):
	user = models.ForeignKey(User)
	bio = models.TextField(blank=True, help_text="The bio entered by the user.")
	dob = models.DateField(null=True, blank=True)
	count_login = models.PositiveIntegerField(default=1, help_text="The number of times this user has logged in")
	create_ts = models.DateTimeField(default=datetime.now)

	objects = UserProfileManager() # defined in managers.py

	@properties
	def full_name(self):
		return self.user.get_full_name()

	def __unicode__(self):
		return self.get_full_name()

Here, we extend the Django models.Model class and then define model fields. This example shows a few common fields and a very simple model. For more details on models and available fields, see The Django Book Chapter 5 - Models & Chapter 10 - Advanced Models.

By including the django.contrib.auth app, as shown in settings.py, we can leverage Django's built in authentication system, which includes a User object, known as auth user. An auth user is any entity capable of logging into the site. The UserProfile model class references the User as a foreign key, which we can query against after login.

The objects property of models is used to query the database. The most common functions are get, which returns a single entry or throws an exception, and filter, which returns a query set:

try:
	user_profile = UserProfile.objects.get(user=user)
except UserProfile.DoesNotExist:
	pass
	
# profiles for users logging in more than 2 times
user_profiles = UserProfile.objects.filter(count_login__gt=2)

The command line manage.py tool, from the root of your project, can be used to sync the DB with your model changes. The syncdb command will create new tables and modify old ones and the sqlall will output the SQL that would be generated for a clean syncdb. For safer schema and data migrations, I recommend using the South migration tool.

python manage.py syncdb myapp1
python manage.py sqlall myapp1

This example also shows @properties, which are python functions that can be called like properties (instance.full_name instead of instance.full_name()). If you want to setup predefined queries, then override the objects property, as shown. Lastly, overriding the __unicode__ function tells Python how this object should print when converted to a string.

myapp1/tests.py

Tests should all be stored in the tests.py file. Here is an example function and view test:

from django.test import TestCase
from mattsniderdotcom.common.test.client import TestClient
class MyApp1TestCase(TestCase):
	"""A testcase that evaluates function stuff"""
	fixtures = ['users', 'user_profiles']
	
	def setUp(self):
		# setup each test here
		pass
		
	def tearDown(self):
		# setup each test here
		pass
		
	def test_mytest(self):
		# all tests must start with keyword 'test'
		self.assertTrue(1)
		self.assertFalse(0)
		self.assertEquals(0, 1)

class MyApp1ViewsTestCase(TestCase):
	"""A testcase that evaluates views and urls"""
	
	def setUp(self):
		self.client = TestClient()
		# self.client.login_user(user_obj)
		
	def tearDown(self):
		# setup each test here
		pass
		
	def test_mytest(self):
		response = self.client.get('myurl')
		self.assertContains(response.content, 'some expected string')
		post_data = {'t1': 1, 't2': 2,}
		response = self.client.post('myurl', post_data)
		self.assertContains(response.content, 'some expected string')
		self.assertRedirects(response, 'expected_redirect_url', status_code=302)

The TestCase object comes with a bunch of different assertions available; the most common ones are shown in this example. Generally, there are two different types of tests, those that test functions and those that test views. View testing should use the TestClient object to fake URL requests. Fixtures are loaded and unloaded between each test and make fake data available to the tests. To execute an app's test, run:

python manage.py test myapp1

or for all tests, including those from installed Django and 3rd-party apps:

python manage.py test

To create a fixture:

python manage.py dumpdata myapp1 > myapp1/fixtures/myapp1.json

For more information on fixtures and testing, see Testing Django Applications.

myapp1/views.py

All request handlers belong in the views.py file. Although, the file name indicates these are views, functions defined here are more like controllers than views, instead the templates are your actual views. Here are a couple of examples views:

from django.core.urlresolvers import reverse
from django.http import HttpResponse
from django.shortcuts import render_to_response, get_object_or_404, redirect
from django.template import RequestContext

def hello_world(request):
    return HttpResponse('hello world')
    
def user_home(request, user_id, template_name="home.html"):
	if 'POST' = request.method:
		return redirect('myapp1:home', kwargs={'user_id': user_id})
	user = get_object_or_404(User, id=user_id)
	user_profile = UserProfile.objects.get(user=user)
	
	template_context = {
		'css': 'home.css',
		'name': name
	}
	
	return render_to_response(template_name, template_context, context_instance=RequestContext(request))
    
def do_something(request, template_name="home.html"):
	# how to test if the user is logged in
	if request.user.is_authenticated():
		name = user.get_full_name()
	else:
		raise Http404
		
	template_context = {
		'css': 'home.css',
		'name': name
	}
	return render_to_response(template_name, template_context, context_instance=RequestContext(request))

View functions should accept at least the request object as the first argument and return a response. However, they can be passed any number of additional arguments, depending on how the URLs are defined. Views can return strings directly, as shown in the hello world example, but more often it is better to render a template. If the URLs define a variable, such as the user_id, then use the get_object_or_404 function to automatically raise 404 error, when the item is not found. The render_to_response function is the easiest way to have Django find your template and pass in the necessary context. You can also easily redirect to other URLs defined in any of your urls.py, by using returning the redirect function.

For additional information, see The Django Book Chapter 3 - Views & URLConfs, Chapter 8 - Advanced Views & URLConfs, and Chapter 11 - Generic Views.

myapp1/urls.py

To hook in your views, you will need to define some urls. First define urls in your app (myapp1/urls.py):

from django.conf.urls.defaults import *

urlpatterns = patterns('profile.views',
	url(r'^hello_world/$', 'hello_world', name='hello_world'),
	url(r'^home/(?P\d+)/$', 'user_home', name='home'),
	url(r'^home_alt/$', 'do_something', name='home_alt'),
)

Then setup a link to your app's urls from the root urls.py:

from django.conf.urls.defaults import *

from django.contrib import admin
from django.conf import settings
from django.contrib.staticfiles.urls import staticfiles_urlpatterns
admin.autodiscover()

urlpatterns = patterns('',
	url(r'^myapp1/', include('myapp1.urls', namespace='myapp1')),
	(r'^admin/doc/', include('django.contrib.admindocs.urls')),
	(r'^admin/', include(admin.site.urls)),
)

if settings.DEBUG:
	urlpatterns += staticfiles_urlpatterns()

# for handling 404 and 500, you'll need to define these somewhere
#handler404 = 'views.page_not_found'
#handler500 = 'views.server_error'

The urlpatterns in the urls.py will be processed and added to the available URLs for the project. This is a tuple so you can easily concatenate additional declarations. The pattern function accepts a list of tuple definitions or url instances. The first argument or each URL definition is the regex of the URL or relative regex for apps, the second is the view or name of the view, or the urls of a given app. For a detailed explanation, see Chapter 3 - Views & URLConf.

This example will create three URLs for myapp1: /myapp1/hello_world/, /myapp1/home/USER_ID/, /myapp1/home_alt/. The URLs names have been namespace to myapp1, so if you do a reverse lookup, such as done by the redirect function, then you will need to preface with the name with the namespace, myapp1:home (namespacing prevents conflicts between apps). The other definitions setup the admin app urls and built in support for local static files.

myapp1/templates/

Django templates can be any kind of file, but are usually HTML or XML. The templates directory of all apps is automatically parsed when the runserver is called. The Django templating system will search and replace template tags {% ... %} and in the template files. Additionally, templates can extend each other using the block template tag or you can import fragments from other templates.

This base.html template would be the core HTML template used by all other HTML templates in this app:

<html>
<head><title>{% block title %}default title{% endblock %}</title></head>
<body>
<div id="header">Site Header</div>
<div id="body">{% block body %}{% endblock %}</div>
<div id="footer">Site Header</div>
</body>
</html>

While the home.html shows how to extend the base.html:

{% extends "base.html" %}
{% load humanize %}
{% block body %}
<h2>Hello World {{ some_num|intcomma }}</h2>
{% include "template_name.html" %}
{% endblock %}

These templates examples also show how to use the humanize template tag and importing a fragment, template_name.html; although the fragment is not shown here. If some_num were passed into the template context, then the humanize intcomma filter would add commas every third digit. For more information, see Built-in Template Tags and Filters and Humanize.

myapp1/template_tags.py

The templatetags directory is for writing template tags and filters specific to the app. The Django template parser is very robust and dynamic, letting developers add tags as necessary. Template tags is a big topic, which is covered in the article Custom Template Tags and Filters.

This is a simple template tag which will print its return value in the template:

from django import template
register = template.Library()
@register.simple_tag
def math(value, operator, modifier):
    """
    Allows numbers to be simply modified in views.
    """
    if '+' == operator:
        return value + modifier
    elif '-' == operator:
        return value - modifier
    elif '*' == operator:
        return value * modifier
    elif '/' == operator:
        return value / modifier
    else:
        raise Exception('Unsupported operator=' % operator)

This is a template tag which will pass its return as the template context to the specified template for rendering:

from django import template
register = template.Library()
@register.inclusion_tag('common_button.html')
def button(content, button_class='button'):
    return {
            'class': button_class,
            'content': content,
    }

myapp1/base.py

The base.py file is a place to put most of the app logic. Keeping logic out of the views and other files, tends to make the code more DRY and reusable. This is not a Django magic file, but is a common practice used by Django engineers.

myapp1/context_processors.py

Django allows developers to write context processor functions that modify every context object passed into the templates. Place these files in context_processors.py. For more information, see The Django Book Chapter 9 - Advanced Templating.

myapp1/forms.py

Put app's forms in forms.py. Form objects are declared like models, but are used to evaluate requests. Forms are also a large topic, so for more details read Chapter 07 - Forms.

Here is a simple authentication form:

from django.contrib.auth.models import User
from django.contrib.auth import authenticate
from django import forms
from django.utils.translation import ugettext_lazy as _

class AuthenticationForm(forms.Form):
    username = forms.CharField(label=_("Username"), max_length=30)
    password = forms.CharField(label=_("Password"), widget=forms.PasswordInput)

    def __init__(self, request=None, *args, **kwargs):
        self.request = request
        self.user_cache = None
        super(AuthenticationForm, self).__init__(*args, **kwargs)

    def clean(self):
        username = self.cleaned_data.get('username')
        password = self.cleaned_data.get('password')

        if username and password:
            self.user_cache = authenticate(username=username, password=password)
            if self.user_cache is None:
                raise forms.ValidationError(_("Please enter a correct username and password. Note that both fields are case-sensitive."))
            elif not self.user_cache.is_active:
                raise forms.ValidationError(_("This account is inactive."))
        return self.cleaned_data

    def get_user_id(self):
        if self.user_cache:
            return self.user_cache.id
        return None

    def get_user(self):
        return self.user_cache

And here is the view that uses the form:

def signup(request, authentication_form=AuthenticationForm, template_name='accounts_signup.html'):
	if 'POST' == request.method:
		form = authentication_form(request.POST)

		if form.is_valid():
			form.save()
			return redirect('profile:home')
	else:
		form = authentication_form()

	return render_to_response(template_name, {
		'form': form,
	}, context_instance=RequestContext(request))

myapp1/listeners.py

Django has a powerful event system called signals that fire at various times during a model lifecycle. I suggest putting any signals you setup in the listeners.py. For more details, see Signals.

myapp1/managers.py

For common model queries, use model managers and organize them in the managers.py file. Here is an example manager that creates a shortcut for fetching UserProfile by the User object:

class UserProfileManager(models.Manager):
	def get_by_user(self, user):
		return self.get(user=user)
		
UserProfile.objects.get_by_user(user=user)

myapp1/middleware.py

The middleware.py can be used to override requests passed into and responses returned by views. Please read, The Django Book Chapter 17 - Middleware before attempting to write or change middleware.

There's more…

There is an insane amount of information that could be covered. I tried to curate it so this article would be useful without having too much or too little information. Keep in mind that, the Django team provides excellent documentation, which you can find in The Django Book or the Django Docs.