This Thursday, May 26 2011, I will be speaking on a Continous Deployment panel for Wealthfront & IMVU in downtown Palo Alto. In preparation for the talk, I thought I'd start a series explaining how easy it is to do continous deployment in Python/Django. Unfortunately, writing this first article took me two weeks, so the rest of the articles will be released after the panel. Over the next couple weeks, we will be discussing the following:
- Starting Your First Django Project
- Testing and Django
- Mock and Coverage
- Using Fabric for Painless Scripting
- Using Celery to handle Asynchronous Processes
- Deployment/Monitoring Strategies
Today's article will discuss setting up Django and focus on tips for building apps. There is a lot to cover, and I won’t be going into much depth. Please consult the Django site, if you get stuck (links included at end and throughout article).
Getting ready
To familiarize yourself with how to start a Python project, see Getting Started with Python Development.
If you followed that article, then open the terminal and enter your projects context:
start_myproject
Next install the version of Django you want:
pip install Django==1.3
Currently, Django only supports Python 2.x, so if you are using Python 3.x, you'll have to install 2.x and configure your system to use it.
How to do it…
Once Django is installed, you can use the following to start a new project (mysite
):
django-admin.py startproject mysite
You will now see the directory mysite
. At this point it is a good idea to put your project’s directory under some kind of source control.
There will be the following files in the mysite directory:
mysite/ __init__.py manage.py settings.py urls.py
You now have a working Django project. It doesn't do anything, but it works. The follow command starts the Django development app server on port 8000:
python manage.py runserver
To start a Python shell with the Django context (useful for debugging), use:
python manage.py shell
Next, you'll need to install some kind of backend for your models, such as MySQL:
pip install MySQL-python==1.2.3
The settings.py
file is where you configure your entire Django project (see the Django documentation for more info). At the very least you'll be wanting to change the DATABASES
and INSTALLED_APPS
variables:
DATABASES = { 'default': { 'NAME': 'DB_NAME', 'ENGINE': 'django.db.backends.mysql', 'USER': 'DB_USER', 'PASSWORD': 'DB_PASSWORD', 'HOST': 'DB_HOST', } } … INSTALLED_APPS = [ 'django.contrib.admin', # Django admin app 'django.contrib.admindocs', # Easy doc gerneration tool 'django.contrib.auth', # Django managed authentication 'django.contrib.contenttypes', 'django.contrib.humanize', # Extra template tags and filters 'django.contrib.messages', # Django managed sessions 'django.contrib.redirects', # Easy view redirection 'django.contrib.sessions', # Django managed sessions 'django.contrib.sitemaps', # Django sitemap generation app 'django.contrib.staticfiles', # server static files locally 'myapp1', # the app we create in this article # 'south', # 3rd party app used for DB migrations ]
Django developers recommend that you build features as discrete self contained apps, living inside of your project. Each of the django.contrib
entries in the INSTALLED_APPS
of the previous example represents a self contain app that does something. To start your own app type:
python manage.py startapp myapp1
This will create the following files:
myapp1/
__init__.py
models.py
tests.py
views.py
I recommend also creating the following files:
myapp1/
templates/
base.html
home.html
template_tags/
base.py
context_processors.py
forms.py
listeners.py
managers.py
middleware.py
urls.py
All of these files will be briefly explained below.
myapp1/__init__.py
The __init__.py
is used by Python to indicate that this directory should behave like a Python module. Beginners shouldn't touch this file, as it can cause circular dependencies and other nastiness, except to document what the app does and its requirements.
myapp1/models.py
Models are the mechanism used by Django to communicate with your database. Here is a simple user profile object:
from datetime import datetime from django.db import models from django.contrib.auth.models import User from myapp1.managers import UserProfileManager class UserProfile(models.Model): user = models.ForeignKey(User) bio = models.TextField(blank=True, help_text="The bio entered by the user.") dob = models.DateField(null=True, blank=True) count_login = models.PositiveIntegerField(default=1, help_text="The number of times this user has logged in") create_ts = models.DateTimeField(default=datetime.now) objects = UserProfileManager() # defined in managers.py @properties def full_name(self): return self.user.get_full_name() def __unicode__(self): return self.get_full_name()
Here, we extend the Django
models.Model
class and then define model fields. This example shows a few common fields and a very simple model. For more details on models and available fields, see The Django Book Chapter 5 - Models & Chapter 10 - Advanced Models. By including the django.contrib.auth
app, as shown in settings.py
, we can leverage Django's built in authentication system, which includes a User object, known as auth user. An auth user is any entity capable of logging into the site. The UserProfile model class references the User as a foreign key, which we can query against after login.
The objects
property of models is used to query the database. The most common functions are get
, which returns a single entry or throws an exception, and filter
, which returns a query set:
try: user_profile = UserProfile.objects.get(user=user) except UserProfile.DoesNotExist: pass # profiles for users logging in more than 2 times user_profiles = UserProfile.objects.filter(count_login__gt=2)
The command line manage.py
tool, from the root of your project, can be used to sync the DB with your model changes. The syncdb
command will create new tables and modify old ones and the sqlall
will output the SQL that would be generated for a clean syncdb. For safer schema and data migrations, I recommend using the South migration tool.
python manage.py syncdb myapp1 python manage.py sqlall myapp1
This example also shows @properties
, which are python functions that can be called like properties (instance.full_name
instead of instance.full_name()
). If you want to setup predefined queries, then override the objects
property, as shown. Lastly, overriding the __unicode__
function tells Python how this object should print when converted to a string.
myapp1/tests.py
Tests should all be stored in the tests.py
file. Here is an example function and view test:
from django.test import TestCase from mattsniderdotcom.common.test.client import TestClient class MyApp1TestCase(TestCase): """A testcase that evaluates function stuff""" fixtures = ['users', 'user_profiles'] def setUp(self): # setup each test here pass def tearDown(self): # setup each test here pass def test_mytest(self): # all tests must start with keyword 'test' self.assertTrue(1) self.assertFalse(0) self.assertEquals(0, 1) class MyApp1ViewsTestCase(TestCase): """A testcase that evaluates views and urls""" def setUp(self): self.client = TestClient() # self.client.login_user(user_obj) def tearDown(self): # setup each test here pass def test_mytest(self): response = self.client.get('myurl') self.assertContains(response.content, 'some expected string') post_data = {'t1': 1, 't2': 2,} response = self.client.post('myurl', post_data) self.assertContains(response.content, 'some expected string') self.assertRedirects(response, 'expected_redirect_url', status_code=302)
The TestCase object comes with a bunch of different assertions available; the most common ones are shown in this example. Generally, there are two different types of tests, those that test functions and those that test views. View testing should use the TestClient object to fake URL requests. Fixtures are loaded and unloaded between each test and make fake data available to the tests. To execute an app's test, run:
python manage.py test myapp1
or for all tests, including those from installed Django and 3rd-party apps:
python manage.py test
To create a fixture:
python manage.py dumpdata myapp1 > myapp1/fixtures/myapp1.json
For more information on fixtures and testing, see Testing Django Applications.
myapp1/views.py
All request handlers belong in the
views.pyfile. Although, the file name indicates these are views, functions defined here are more like controllers than views, instead the templates are your actual views. Here are a couple of examples views:
from django.core.urlresolvers import reverse from django.http import HttpResponse from django.shortcuts import render_to_response, get_object_or_404, redirect from django.template import RequestContext def hello_world(request): return HttpResponse('hello world') def user_home(request, user_id, template_name="home.html"): if 'POST' = request.method: return redirect('myapp1:home', kwargs={'user_id': user_id}) user = get_object_or_404(User, id=user_id) user_profile = UserProfile.objects.get(user=user) template_context = { 'css': 'home.css', 'name': name } return render_to_response(template_name, template_context, context_instance=RequestContext(request)) def do_something(request, template_name="home.html"): # how to test if the user is logged in if request.user.is_authenticated(): name = user.get_full_name() else: raise Http404 template_context = { 'css': 'home.css', 'name': name } return render_to_response(template_name, template_context, context_instance=RequestContext(request))
View functions should accept at least the
request
object as the first argument and return a response. However, they can be passed any number of additional arguments, depending on how the URLs are defined. Views can return strings directly, as shown in the hello worldexample, but more often it is better to render a template. If the URLs define a variable, such as the
user_id
, then use the get_object_or_404
function to automatically raise 404 error, when the item is not found. The render_to_response
function is the easiest way to have Django find your template and pass in the necessary context. You can also easily redirect to other URLs defined in any of your urls.py, by using returning the
redirect
function.For additional information, see The Django Book Chapter 3 - Views & URLConfs, Chapter 8 - Advanced Views & URLConfs, and Chapter 11 - Generic Views.
myapp1/urls.py
To hook in your views, you will need to define some urls. First define urls in your app (myapp1/urls.py
):
from django.conf.urls.defaults import * urlpatterns = patterns('profile.views', url(r'^hello_world/$', 'hello_world', name='hello_world'), url(r'^home/(?P\d+)/$', 'user_home', name='home'), url(r'^home_alt/$', 'do_something', name='home_alt'), )
Then setup a link to your app's urls from the root urls.py
:
from django.conf.urls.defaults import * from django.contrib import admin from django.conf import settings from django.contrib.staticfiles.urls import staticfiles_urlpatterns admin.autodiscover() urlpatterns = patterns('', url(r'^myapp1/', include('myapp1.urls', namespace='myapp1')), (r'^admin/doc/', include('django.contrib.admindocs.urls')), (r'^admin/', include(admin.site.urls)), ) if settings.DEBUG: urlpatterns += staticfiles_urlpatterns() # for handling 404 and 500, you'll need to define these somewhere #handler404 = 'views.page_not_found' #handler500 = 'views.server_error'
The urlpatterns in the urls.py
will be processed and added to the available URLs for the project. This is a tuple so you can easily concatenate additional declarations. The pattern
function accepts a list of tuple definitions or url
instances. The first argument or each URL definition is the regex of the URL or relative regex for apps, the second is the view or name of the view, or the urls of a given app. For a detailed explanation, see Chapter 3 - Views & URLConf.
This example will create three URLs for myapp1
: /myapp1/hello_world/
, /myapp1/home/USER_ID/
, /myapp1/home_alt/
. The URLs names have been namespace to myapp1
, so if you do a reverse lookup, such as done by the redirect
function, then you will need to preface with the name with the namespace, myapp1:home
(namespacing prevents conflicts between apps). The other definitions setup the admin app urls and built in support for local static files.
myapp1/templates/
Django templates can be any kind of file, but are usually HTML or XML. The templates
directory of all apps is automatically parsed when the runserver
is called. The Django templating system will search and replace template tags {% ... %}
and in the template files. Additionally, templates can extend each other using the block
template tag or you can import fragments from other templates.
This base.html
template would be the core HTML template used by all other HTML templates in this app:
<html> <head><title>{% block title %}default title{% endblock %}</title></head> <body> <div id="header">Site Header</div> <div id="body">{% block body %}{% endblock %}</div> <div id="footer">Site Header</div> </body> </html> While thehome.htmlshows how to extend thebase.html: {% extends "base.html" %} {% load humanize %} {% block body %} <h2>Hello World {{ some_num|intcomma }}</h2> {% include "template_name.html" %} {% endblock %}
These templates examples also show how to use the humanize template tag and importing a fragment, template_name.html
; although the fragment is not shown here. If some_num
were passed into the template context, then the humanize intcomma filter would add commas every third digit. For more information, see Built-in Template Tags and Filters and Humanize.
myapp1/template_tags.py
The templatetags
directory is for writing template tags and filters specific to the app. The Django template parser is very robust and dynamic, letting developers add tags as necessary. Template tags is a big topic, which is covered in the article Custom Template Tags and Filters.
This is a simple template tag which will print its return value in the template:
from django import template register = template.Library() @register.simple_tag def math(value, operator, modifier): """ Allows numbers to be simply modified in views. """ if '+' == operator: return value + modifier elif '-' == operator: return value - modifier elif '*' == operator: return value * modifier elif '/' == operator: return value / modifier else: raise Exception('Unsupported operator=' % operator)
This is a template tag which will pass its return as the template context to the specified template for rendering:
from django import template register = template.Library() @register.inclusion_tag('common_button.html') def button(content, button_class='button'): return { 'class': button_class, 'content': content, }
myapp1/base.py
The base.py
file is a place to put most of the app logic. Keeping logic out of the views and other files, tends to make the code more DRY and reusable. This is not a Django magic file, but is a common practice used by Django engineers.
myapp1/context_processors.py
Django allows developers to write context processor functions that modify every context object passed into the templates. Place these files in context_processors.py
. For more information, see The Django Book Chapter 9 - Advanced Templating.
myapp1/forms.py
Put app's forms in forms.py
. Form objects are declared like models, but are used to evaluate requests. Forms are also a large topic, so for more details read Chapter 07 - Forms.
Here is a simple authentication form:
from django.contrib.auth.models import User from django.contrib.auth import authenticate from django import forms from django.utils.translation import ugettext_lazy as _ class AuthenticationForm(forms.Form): username = forms.CharField(label=_("Username"), max_length=30) password = forms.CharField(label=_("Password"), widget=forms.PasswordInput) def __init__(self, request=None, *args, **kwargs): self.request = request self.user_cache = None super(AuthenticationForm, self).__init__(*args, **kwargs) def clean(self): username = self.cleaned_data.get('username') password = self.cleaned_data.get('password') if username and password: self.user_cache = authenticate(username=username, password=password) if self.user_cache is None: raise forms.ValidationError(_("Please enter a correct username and password. Note that both fields are case-sensitive.")) elif not self.user_cache.is_active: raise forms.ValidationError(_("This account is inactive.")) return self.cleaned_data def get_user_id(self): if self.user_cache: return self.user_cache.id return None def get_user(self): return self.user_cache
And here is the view that uses the form:
def signup(request, authentication_form=AuthenticationForm, template_name='accounts_signup.html'): if 'POST' == request.method: form = authentication_form(request.POST) if form.is_valid(): form.save() return redirect('profile:home') else: form = authentication_form() return render_to_response(template_name, { 'form': form, }, context_instance=RequestContext(request))
myapp1/listeners.py
Django has a powerful event system called signals that fire at various times during a model lifecycle. I suggest putting any signals you setup in the listeners.py
. For more details, see Signals.
myapp1/managers.py
For common model queries, use model managers and organize them in the managers.py
file. Here is an example manager that creates a shortcut for fetching UserProfile by the User object:
class UserProfileManager(models.Manager): def get_by_user(self, user): return self.get(user=user) UserProfile.objects.get_by_user(user=user)
myapp1/middleware.py
The middleware.py
can be used to override requests passed into and responses returned by views. Please read, The Django Book Chapter 17 - Middleware before attempting to write or change middleware.
There's more…
There is an insane amount of information that could be covered. I tried to curate it so this article would be useful without having too much or too little information. Keep in mind that, the Django team provides excellent documentation, which you can find in The Django Book or the Django Docs.