Justomatic

It's pretty much effortless 
Filed under

python

 

Setting Up A Django Development Virtual Environment on OS X

This is a simple script I currently use to set up a Python virtualenv for Django development. It just sets up a new virtualenv and checks out the latest revision of Django from their web site. Maybe someone else will find it useful. Improvements are definitely welcome!

#! /bin/bash
virtualenv env
svn co http://code.djangoproject.com/svn/django/trunk/django env/lib/python2.5/site-packages/django
cp env/lib/python2.5/site-packages/django/bin/django-admin.py django-admin.py

I save the script in /usr/local/bin as "djangovirtualenv.sh" and whenever I want to setup a new environment I do this:

justin$ cd ~/projects
justin$ mkdir NewProject
justin$ cd NewProject
justin$ djangovirtualenv.sh
justin$ source env/bin/activate
justin$ python djangoadmin.py startproject newproject

You can get virtualenv from PyPi

Filed under  //   django   python   shell   virtualenv  

Origin Story

Every superhero needs an origin story.

Occasionally people ask me about the origins of my nickname BDFL (Benevolent Dictator For Life). At some point, Wikipedia claimed it was from a Monty Python skit, which is patently false, although it has sometimes been called a Pythonesque title. I recently trawled through an old mailbox of mine, and found a message from 1995 that pinpoints the origin exactly. I'm including the entire message here, to end any doubts that the term originated in the Python community.

I now find myself contemplating the existence of a Bizarro GvR.

Filed under  //   gvr   programming   python  

Because You're Smarter Than XML

While developing Atom Syndication support for the new site I started to get fed up (ha!) with the standard methods of XML generation in Python. Obtuse and verbose, the number of lines of code it took to generate a simple feed grew to an order of magnitude greater then the number of lines it took to serve that same feed. I wanted something a little less error prone and a little more convenient than string manipulation, but I didn't need any sort of validation or schema support or any of the other things that make the standard XML libraries "smart". The end result was StupidXML, a genuinely "stupid" XML generation library. I've put the code up on Google Code in case someone else will find it useful, let me know if you find any bugs!

Filed under  //   development   oss   python   xml  

Google App Engine And The Value Of What It Isn't

Since the announcement monday of Googles App Engine service I've heard little but complaints (apart from a few informed assesments) from the internet at large. Now this doesn't surprise me and I'm sure on some level the App Engine teams is grateful for the attention, negative or otherwise. However, I think this reaction sheds some light on how many web developers have become so entrenched in their own development "comfort zone" that they can't see the value in exploring beyond the boundaries of their own languages and frameworks.

Google Hasn't Taken Anything From You

From some of the reactions I've read this week you'd think that by choosing Python as the initial language for App Engine Google had somehow rendered all other development environments obsolete (and ran over everyone's dog in the process). There also seems to be a number of developers lamenting how hard it will be to "port" their existing applications to this new environment as if that had been mandated by some higher power. Why can't we just get along look at this from the perspective that whatever options you had for web application development on Sunday, you now have one more. Whether you choose to use it or not.

You Got Your Framework In My Framework

Another issue that's rubbing people the wrong way is the simple fact that either their framework of choice isn't supported (Pylons, Turbogears, CherryPy) or isn't fully supported (Django). I use, and respect, these frameworks but I personally think it was a mistake to "support" Django in App Engine. Whatever they gained by playing to one of the largest Python web developer communities, I think they lost more by giving people the impression that App Engine is just another mod_python to bootstrap your framework to. App Engine IS an application framework, maybe not a full stack, but enough that there are as many parts of Django that are either redundant or incompatible as there are parts that are functional.

You Can Choose Your Own Colors Even When You Have To Color Within The Lines

I've spent a decent part of this week evaluating the App Engine SDK and having a blast building my own framework to fill in the holes in Googles service. As so much functionality is available from the SDK, it's been easy to assemble the few pieces I need and wrap them up in a package that I think will be flexible enough to handle pretty much any project I can throw at it. Based loosely on the Pylons framework, A simple WSGI "front controller", Routes, WSGI based app controllers and a few convenience utilities was all it took to get a fairly robust system up and running. As an added bonus, the entire framework (apart from external libraries) is spread over only 3 files.

We're All Still Working Out The "Kinks"

My little framework's not perfect yet, and probably never will be, but neither is App Engine. I still need a good template engine, and a web form generation library for the datastore models would be nice. But even without these "niceties" it's a very productive environment considering it probably took me less then 6 hours to build. I've be denied certain choices I've become reliant on (PIL, Mako), but with quality of the initial APIs and Googles repeated insistence that the framework is nowhere near feature complete leave me at least hopeful for replacements in leu of fixes.

Filed under  //   django   google   python   wsgi  

App Engine Revisited

So I still haven't gotten my App Engine invite (sigh) but I've been working with the SDK and I've got a skeleton framework up and running with WebOb and Routes for controller dispatch. I've also put together a simple blogging application. I find the system limited, but limited in a way I think will inspire creativity among those who like to push the boundaries they've been given.

The included WSGI framework is admittedly simple, and I'm personally not that fond of the parts of Django they've managed to shoehorn into the environment so I think I'll be rolling my own framework for all future projects. A sort of Pylons lite, if you will. Routes, Mako (if possible *crosses fingers*, I have "issues" with Django templates) and Googles data store/ORM. I'm still debating the necessity of a session framework like Beaker if I'm going to be using the Users API but it would be nice to have.

Anyway, I hope to get an invite soon (hello? anybody?) but at least I'll be ready when I do. Hopefully this blog will soon be served up from AppEngine!

Filed under  //   google   python  

Google Announces Google App Engine

The highlights for now:

  • Python! *Squeeee*
  • No C Extensions (boo! understandable, but still boo.)
  • WSGI!
  • Django!
  • Google Mail, Apps and Accounts integration!
  • BigTable™
  • FREE!
  • Did I mention Python and WSGI?!

I've got to go change my pants now, but I'll post my extended thoughts later. I'm furious I didn't hear about this sooner and get one of the initial invites. WTF internets?!

Filed under  //   google   python  

A Pure Python WSGI Server

I'm currently evaluating Python WSGI HTTP servers for a new project. Right now, it seems my best options are the Paste server and the Cherrypy server as the twisted implementation is still in development and a little rough around the edges from what I've heard.

If you've had any experience with these libraries, or would like to suggest a solution that's not listed here, please drop me a comment or an email.

Filed under  //   python   wsgi  

SQLite As A Production Database (No, Really)

There's a certain stigma surrounding Sqlite. It seems that many developers will happily use it to prototype a project, but wouldn't think of deploying on anything less than Postgres (or MySQL, if you swing that way). I say this partially because I used to feel this way myself. I've run more low traffic web apps off of a Postgres server than I'd care to admit now that I've discovered the inherent convenience and manageability of a file-based SQLite database.

With SQLite an entire python web application fits within a single directory on the disk. This makes backup, testing and maintenance easier than you can likely imagine. SQLite also requires next to no setup, and most likely, is already running on your server.

Recently a user came across a bug in one of my web applications. In order to reproduce the bug, I didn't have to set up a test database and populate it with dummy data. Instead I merely copied the actual production database file along with the application files and within minutes had an identical copy of the application, data and all, running on my development machine.

There are trade-offs with SQLite and the developers make this very clear. Concurrent writes are an issue (I'd wager that actually very few web sites depend upon a large number of concurrent database writes) and the types system can throw traditional database developers for a twist. All the same, as the development community comes to grips with the fact that scripting languages can solve real problems as well or better (when you factor in time-to-market) than compiled languages, I think projects such as SQLite and CouchDB will finally motivate developers to begin looking for data persistence solutions in places other than monolithic relational database servers.

"SQLite isn't trying to replace Oracle, it's trying to replace 'fopen'" - D. Richard Hipp, creator of SQLite

- FLOSS Weekly Podcast with D. Richard Hipp, the creator of SQLite.

Filed under  //   databases   django   python   sqlite  

Objects With Shared State In Python

Back when I was hacking on PHP and rolling my own webapp frameworks ("in the day" as it were) I'd often use the singleton pattern for creating a app "registry" where global data and variables could be stored and accessed by objects anywhere within the framework. Since moving most of my web development projects to Python a couple years ago I haven't really had any use for such global, shared state objects. That is, until I received this issue ticket for Photologue.

Some background...

Photologue provides a way to define "Photo Sizes" within your database that all your uploaded images can be resized to automatically. When a photo is requested in a certain size, the system loads the appropriate size description and resizes the image as specified (if the sized image has not already been cached). This gives the programmer/designer/site-admin a lot of freedom when developing apps and pages as any number of photo sizes can be defined at any time without changing any code or restarting the server. This was also causing the Django's ORM to make an unreasonably large amount of queries when a page full of images was loaded as each image was querying for the photo size object as it was loaded.

The fix was twofold. First, when a Photo object is initialized a function is called that adds a number of convenience functions for accessing the defined photo sizes. For instance if you define a "thumbnail" photo size a method named "get_thumbnail_url" would be added to the instance and would return the url of the photo sized to the "thumbnail" specification. These functions were originally added by "curry"ing existing functions on the model ("get_SIZE_url, get_SIZE_path, etc.) and the names of all found photo sizes. When called these function would use the name supplied to load the photo size from the database. The first fix was simply to pass the actual photo size object it self to these functions, eliminating the need to load them later:

for photosize in sizes:
    setattr(self, 'get_%s_size' % photosize.name, curry(self._get_SIZE_size, photosize=photosize))
    setattr(self, 'get_%s_url' % photosize.name, curry(self._get_SIZE_url, photosize=photosize))
    setattr(self, 'get_%s_path' % photosize.name, curry(self._get_SIZE_path, photosize=photosize))

Which brings us to the the title of this post...

I needed a way to load the photo size models once and then let any and all photos access these sizes without having to load them again. A global variable would work (as suggested by the original issue submitter) but seems kludgey and one thing I love about Python is being able to solve problem in a way that is as elegant as it is functional. So I did a little research and came across a pattern for creating classes with shared state. That is, all separate instances created from this class will maintain the same state wherever they exist within your program. When one instance is modified, all instances reflect the change. For example here's the class used to cache photo sizes within Photologue:

class PhotoSizeCache(object):
    __state = {"sizes": {}}
    def __init__(self):
        self.__dict__ = self.__state

Now, when a photo needs to access the list of photo sizes it simple instantiates an instance of PhotoSizeCache, which automatically is assigned the class's global state, and checks to see if the "sizes" dictionary has a length. If not it loads the full list of sizes and stores them within the cache for other any other objects to find. The final result was a drastic reduction in the number of hits made on the database (three down from over two thousand on a test page loading around seven hundred images). It's still global state (me bad?) but I think it has a certain beauty that's only made possible by Python's dynamic nature.

Filed under  //   django   howto   patterns   photologue   python