Getting Things Done, Django-style: Part 2
Update: I no longer have this project in my Subversion repository but have archived the project which you can download here. Note that this was written against Django 0.96 and a number of things have changed such as validators and manipulators (some assembly required).
In the first part of this series I discussed what I intended to do: develop a web-based application for managing my todo list, following the Getting Things Done (GTD) approach. This follow-on article jumps into how I'm doing it, covering version 1 of the application.
Aside: I know its more au courant to use a 0.x version for an early cut of an application, but dammit, I like integers. So this is version 1 - bugs, warts and all.
For you kids following along at home, you can grab the code from subversion with the following command:
svn co http://davidavraamides.net/svn/da
That will grab my entire site, which is really just the todo app and the software for this blog. I didn't separate these out into separate subversion projects so its a little messy.
URL Design
Thinking about the URL structure of your web application is a good starting point that helps ferret out the design of the app. I knew I wanted to view all my tasks organized by project or context, and also to show just a filtered view of tasks for a specific project or context. Obviously, its useful to have a view of completed tasks, too.
I also wanted to be able to enter new tasks and edit or delete existing ones from the main task list pages. So each of my main URLs supports an optional part that identifies the task under edit when applicable.
This led to the following first cut for my urls.py file:
urlpatterns = patterns( '', # project/<project_id>/[/task/<task_id>]/ (r'^project/(?P<project_id>\d+)(/task/(?P<task_id>\d+))?/$', 'danet.todo.views.project'), # context/<context_id>/[/task/<task_id>]/ (r'^context/(?P<context_id>\d+)(/task/(?P<task_id>\d+))?/$', 'danet.todo.views.context'), # project[/task/<task_id>]/ (r'^project(/task/(?P<task_id>\d+))?/$', 'danet.todo.views.projects'), # context[/task/<task_id>]/ (r'^context(/task/(?P<task_id>\d+))?/$', 'danet.todo.views.contexts'), # done/ (r'^done/$', 'danet.todo.views.done'), # del/<task_id>/ (r'^del/(?P<task_id>\d+)/$', 'danet.todo.views.delete'), # chk/<task_id>/ (r'^chk/(?P<task_id>\d+)/$', 'danet.todo.views.check'), # (default) (r'^$', 'django.views.generic.simple.redirect_to', {'url':'/todo/project/'}), )
Note that the application prefix, /todo/, is handled in the project urls.py file which includes this application level one. The urls.py file above looks a little more complicated than it really is. There are really five primary URLs covered in the first five entries above. They handle, respectively:
- Tasks for a specific project (
project/2/), with an optional task id if one is being edited (project/2/task/7/). - Tasks for a specific context (
context/3/) and optional task id. - Tasks for all projects (
project/) - Tasks for all contexts (
context/) - Completed tasks (
done/)
The next one, del/<task_id>/, is the target of a POST and is used for deleting a task. Likewise, the subsequent pattern, chk/<task_id>/, is used for "checking" a task, i.e. toggling the completed status of the task.
The final pattern, the default location for the site, simply redirects to the view of all tasks organized by project.
Models
There are four models in the application, although only one, Task is interesting. The other three (Project, Context and Priority) are simply properties of a task. So lets start with the simple ones.
class Context(models.Model): name = models.CharField(maxlength=40, unique=True) def __str__(self): return self.name def get_absolute_url(self): return '/todo/context/%d' % self.id class Meta: ordering = ['name'] class Admin: pass
A Context represents the, well, context that you are doing your tasks in, such as coding, email, errands or office. At this point we only need a name for it. The get_absolute_url method refers to the URL pattern which shows a list of tasks for just this specific context.
class Project(models.Model): name = models.CharField(maxlength=40, unique=True) def __str__(self): return self.name def get_absolute_url(self): return '/todo/project/%d' % self.id class Meta: ordering = ['name'] class Admin: pass
The Project model is essentially the same, which begs the question of why I didn't just use a common Group model for both cases, or even use a CharField in Task with a choice list.
The real reason is: overengineering planning for change. I didn't want to use text fields directly in a Task for a number of reasons:
- Typos could result in two versions of what was intended to be the same project or context (i.e. "office" and "ofice").
- Its difficult to rename them later from inside the application (if I later decided that "office" should be "work").
- It would be hard to factor these values out into a separate model later if I needed to, once I had data in the database.
And I wanted the freedom to evolve the Project and Context models independently. I may never do that, but I wanted the the option.
Priority is also very similar to the other two models, but it has a couple of extra fields: order specifies a sort ordering for this field. Its fine to sort context and project alphabetically, but priority should be sorted by importance. I've also added a color field in anticipation that I'll probably want to display different priorities in different colors. Note: this may lead to a solution that uses inline styles rather than a central CSS style sheet for the effect, but I think that's OK in this case.
class Priority(models.Model): name = models.CharField(maxlength=40, unique=True) order = models.IntegerField() color = models.CharField(maxlength=40, null=True, blank=True) def __str__(self): return self.name class Meta: ordering = ['order'] class Admin: pass
A couple other points worthy of note for these three models are:
- They all made the
namefield unique rather than setting it as a primary key. This is just good database design, IMO, as the names could change, which a primary key should never do. - They all specify their ordering so they will display consistently in a
selectfield. - All three are enabled in Django's
adminapplication. At this point I don't plan to manage these lists in the todo application itself so theadminwill provide that feature. That will probably hold true forever for both contexts and priorities, but the project list is a different story. I may want the ability to define new projects "on the fly" and if so, I'll add that feature later and manage them outside of the admin.
If the models were U2, you'd be saying "OK, I've seen Larry, Adam and The Edge, but where's Bono?" Or more to the point, what's a todo application without tasks?
The Task model actually has some meat on its bones. It obviously needs a description (i.e. "plug in my A/C adapter soon!") and has foreign keys to the three models above. A task can also have an optional due date, and notes of arbitrary length. Finally, a task keeps track of when it was created and when it was completed.
class Task(models.Model): description = models.CharField(maxlength=255) context = models.ForeignKey(Context) project = models.ForeignKey(Project) priority = models.ForeignKey(Priority) due = models.DateField(null=True, blank=True) notes = models.TextField(null=True, blank=True) created = models.DateTimeField(auto_now_add=True) completed = models.DateTimeField(null=True, blank=True) def __str__(self): return self.description
Unlike the other models, a task is not available in the admin as there is no need. You'll notice that while some fields are optional (due and notes), context, project and priority are required. That seems to go against the idea of making it easy to enter tasks.
I went back and forth on this and made a compromise: while they are required, they all default to reasonable values in the UI ('none' for context and project and 'normal' for priority). Thus, you can still enter a task easily without specifically setting these fields and they essentially remain unset.
The reason for this approach was due to some problematic behavior in Django with ForeignKey fields that are nullable. Django doesn't know how to do outer joins when it joins related tables (via the select_related() or extra(tables=...) methods). So if any of these fields are null, you drop rows in your query.
Views
You'll remember from the URLs discussion that there are four views that are very similar. While each view has its own function in views.py, they all delegate the heavy lifting to a common, more generic helper function _tasks.
def projects(request, task_id=None): return _tasks(request, 'todo/projects.html', 'project', None, task_id) def project(request, project_id, task_id=None): project = Project.objects.get(pk=project_id) return _tasks(request, 'todo/project.html', 'project', project, task_id) def contexts(request, task_id=None): return _tasks(request, 'todo/contexts.html', 'context', None, task_id) def context(request, context_id, task_id=None): context = Context.objects.get(pk=context_id) return _tasks(request, 'todo/context.html', 'context', context, task_id)
The plural forms (projects and contexts) call _tasks passing the name of the grouping ('project' or 'context') and the optional task_id which will have been extracted from the URL, if specified.
The singular views (project and context) lookup the corresponding grouping object and pass that along to _tasks as well.
The _tasks function starts out by finding all opens tasks or tasks that were completed today. (I like to see recently completed tasks. It gives me a sense of accomplishment.) If a specific context or project was specified, that object will have been looked up and passed in as the group parameter and is used to filter the list of tasks.
def _tasks(request, template, group_name, group, task_id=None): today = datetime.date.today() # get all open tasks, or ones completed today tasks = Task.objects.filter(Q(completed__isnull=True) | \ Q(completed__gte=today)) print tasks # filter by grouping, if specified if group: tasks = tasks.filter(**{ group_name + '__id':group.id}) tasks = _modify_task_query(tasks, group_name) # get the appropriate manipulator task = None if task_id: try: manipulator = Task.ChangeManipulator(task_id) except Task.DoesNotExist: raise Http404 task = manipulator.original_object else: manipulator = Task.AddManipulator() # since this is called from various forms of the url, we # need to preserve the same base url for links, redirects, etc base_url = re.sub(r'/task/\d+', '', request.path) # the usual form processing logic if request.POST: new_data = request.POST.copy() errors = manipulator.get_validation_errors(new_data) if not errors: manipulator.do_html2python(new_data) new_task = manipulator.save(new_data) return HttpResponseRedirect(base_url) else: errors = new_data = {} if task: new_data = task.__dict__ else: new_data = { 'context_id': 1, 'project_id': 1, 'priority_id': 1 } form = forms.FormWrapper(manipulator, new_data, errors) form_data = {'form': form, 'tasks': tasks, 'base_url': base_url, 'group': group } return render_to_response(template, form_data)
Then we call a helper function, _modify_task_query, to add some computed columns that are used to provide the preferred sort order in a consistent manner across all views of tasks. Then you'll see a familiar code pattern for using either an AddManipulator or a ChangeManipulator for creating or editing a task. There are a couple of things to call out in this section of the code:
-
We want to return back to the same page that we were called from but the URL may have the extra
/task/<id>/piece on it so we strip it off before using it for the post-save redirection. We can't use an absolute URL here because we can call this function from many different URLs. -
In the
elseclause (when this is not a POST), we set the defaults values for theselectboxes. I use some initial SQL files to pre-load these fields with known data and load the data in a way where I know that the first record in each table is the default value.
The done function is slightly different from the others. It does't use the _tasks helper. Instead it just queries the completed tasks in a reverse date/time order before it calls the template.
The delete and check functions are similar to one another. They lookup the specified task object and modify it appropriately. The check function simply toggles the completed date field between the current date/time and null to indicate if the task is complete or not. Both views are called with a return URL query so they know where to redirect after the database udpate.
Templates
There are five primary templates that are used to render the views previously discussed. The "main" view for the todo app, projects.html, is shown below.
{% extends "todo/base.html" %}
{% block content %}
<h1>Tasks by Project</h1>
{% load tags %}
{% include "todo/task_form.html" %}
{% if tasks %}
{% regroup tasks by project as grouped %}
{% for group in grouped %}
<h2>
<a href="{{ group.list.0.project.get_absolute_url }}">{{ group.grouper }}</a>
</h2>
<ul>
{% for task in group.list %}
<li>{% show_task task %}</li>
{% endfor %}
</ul>
{% endfor %}
{% endif %}
{% endblock %}
Remember, at this point I haven't put any attention on styling the application so the template is rather simple, consisting of a title header, task form, and a sequence of unordered lists to display the tasks grouped by project.
The templates extend a common base template, todo/base.html. This is also fairly simple at this point, specifying a section menu made from a few anchor tags, followed by the content block (extended by the "real" templates), and finally a footer is added to show some basic performance stats using a middleware component I previously wrote.
<html> <head> <title>ToDo</title> </head> <body> <a href="/todo/project/">Projects</a> | <a href="/todo/context/">Contexts</a> | <a href="/todo/done/">Completed</a> {% block content %}{% endblock %} <div id="footer"> <!-- STATS: Total: %(totTime).2f Python: %(pyTime).2f DB: %(dbTime).2f Queries: %(queries)d --> </div> </body> </html>
The projects.html template also displays the form used to create new tasks and edit existing ones by including another template, task_form.html:
<form action="." method="post"> <p> <label for="id_desc">Task:</label> {{ form.description }} {% if form.description.errors %}*** {{ form.description.errors|join:", " }}{% endif %} </p> <p> <label for="id_context">Context:</label> {{ form.context }} {% if form.context.errors %}*** {{ form.context.errors|join:", " }}{% endif %} </p> <p> <label for="id_project">Project:</label> {{ form.project }} {% if form.project.errors %}*** {{ form.project.errors|join:", " }}{% endif %} </p> <p> <label for="id_priority">Priority:</label> {{ form.priority }} {% if form.priority.errors %}*** {{ form.priority.errors|join:", " }}{% endif %} </p> <p> <label for="id_due">Due:</label> {{ form.due }} {% if form.due.errors %}*** {{ form.due.errors|join:", " }}{% endif %} </p> <p> <label for="id_notes">Notes:</label> {{ form.notes }} {% if form.notes.errors %}*** {{ form.notes.errors|join:", " }}{% endif %} </p> <input type="Submit"/> </form>
This is a standard Django form based on the FormWrapper class. It just displays the editable fields of a Task model and optionally displays errors associated with a field when the form fails validation. Its not much to look at yet, but the key point to note is that by pulling this out into a separate template, its easy to reuse and keep consistent throughout the application. That will be quite helpful when I do start to focus on the look and feel of the application.
projects.html includes another template, but its done in a different way. I use a custom template tag show_task to render a task in HTML using the template task.html. The custom tag is an inclusion tag which allows you to pass arguments to it and then have it include another template snippet that references those arguments. This is just what I need for drawing a task in a functional way.
The inclusion tag is trivial. It registers itself as an inclusion tag referencing the template todo/tasks.html and simply places the task argument in a dict. That's all inclusion tags really need to do: setup a dictionary with the data for a template.
register = template.Library() @register.inclusion_tag('todo/task.html') def show_task(task): return {'task': task }
The template then just uses the data in the dictionary ("task") to render the actual task, doing a lot of conditional drawing based on which fields are set.
{% if task.is_completed %}
X - ({{ task.completed|date:"m/d/y h:iA"|lower }})
{% endif %}
{{ task.description }}
{% if task.project %}={{ task.project }}{% endif %}
{% if task.context %}@{{ task.context }}{% endif %}
{% if task.priority %}!{{ task.priority }}{% endif %}
{{ task.due|date:"m/d/y" }}
{% if task.is_completed %}
<a href="javascript:window.location =
'/todo/chk/{{ task.id }}?r='+window.location.href;false;">Undo</a>
{% else %}
<a href="{{ base_url }}task/{{ task.id }}/">Edit</a>
<a href="javascript:window.location =
'/todo/del/{{ task.id }}?r='+window.location.href;false;">Del</a>
<a href="javascript:window.location =
'/todo/chk/{{ task.id }}?r='+window.location.href;false;">Chk</a>
{% endif %}
The contexts.html is so similar to projects.html that its easier to show the diffs:
$ diff projects.html contexts.html 3c3 < <h1>Tasks by Project</h1> --- > <h1>Tasks by Context</h1> 8c8 < {% regroup tasks by project as grouped %} --- > {% regroup tasks by context as grouped %} 12c12 < <a href="{{ group.list.0.project.get_absolute_url }}"> {{ group.grouper }}</a> --- > <a href="{{ group.list.0.context.get_absolute_url }}"> {{ group.grouper }}</a>
They differ only in their title header, the field they use for grouping tasks, and the field they use for the URL to jump to a specific project or context.
The template that shows a single project is similar, but a little simpler in that it doesn't require the regroup loop:
{% extends "todo/base.html" %}
{% load tags %}
{% block content %}
<h1>Tasks for {{ group.name }} Project</h1>
{% include "todo/task_form.html" %}
{% if tasks %}
<ul>
{% for task in tasks %}
<li>{% show_task task %}</li>
{% endfor %}
</ul>
{% endif %}
{% endblock %}
The single context template, context.html differs from project.html in only one place: the title <h1> element. Note that with little effort these two pairs of templates could be unified into just one of each type: for all tasks and for one group (project or context). I decided at this point to keep it more explicit as I'm not sure if I will want them to stay so similar. And they are so simple right now its not much effort to keep them in sync.
SQL Initial Data
Django has a little-known feature where it can execute some SQL scripts whenever it builds the SQL table for a model. You just create a file called <model>.sql and put it in a folder called sql/ under your app. The feature is often used to load initial data into a table, but can be used for other purposes like adding indices. I created four SQL scripts: 3 for the "property" models: Project, Context and Priority, and one for Tasks. The property scripts are to load the tables with the starting set of data I want to use so I don't have to do it each time I rebuild the database. Here is project.sql which inserts the standard set of projects I'm currently using. The only thing to note here is that I make sure to insert "none", the default project, first. This ensures that it will have id 1 which is handy (in my _tasks view helper, I set the default ids to "1").
insert todo_project (name) values ('none'), ('blog'), ('data'), ('documentation'), ('family'), ('finances'), ('holdings'), ('house'), ('life'), ('random'), ('risk'), ('todo') ;
The others are quite similar, except that priority.sql also inserts the ordering field I talked about.
insert todo_priority (name, `order`) values ('normal', 2), ('high', 1), ('low', 3);
The SQL script for Tasks is different. I use it as a way to migrate my tasks from one version to the next (when I need to rebuild the database). I have a simple script to backup my todo_tasks table in a format that can be reloaded after a database rebuild:
mysqldump -u root -p -n -c -t -r sql\task.sql web todo_task
This produces a fairly robust format where if I only add new columns, or even change columns to a compatible type, my data will just migrate over. If I do something more complicated, I will have to tweak the file.
Wrap Up
Now I have a simple todo application that actually works. Its quite ugly at this point so in the next part I'll put some lipstick on this pig! In part 3 I'll add styling, some icons and better layout of the screen to make the application look nicer. I'll also add a print view for printing a copy of my list - a poor man's "offline" mode. In fact, I'll enter these into my todo list right now...
3 Comments