When I solve any problem through programming I always think about the generic problem I’m solving and when its sufficiently complex I try to break it down into a number of smaller, repeatable problems that have similar characteristics and can leverage much common code.

I’m currently trying to implement my my current big framework on top of Django. It’s an interesting project – it does set things up to be nice and easy. However I have concerns about how it decomposes problems.

Basically its metaphor for subdividing a web-site is to decompose it into a number of applications. An application is something that, in general, will have the following properties:

1. A URL schema (nominally defined in a urls.py)
2. A set of models that describe a set of Active Records that map onto an SQL database schema.
3. A set of views. These are basically functions that handle web requests are specific URLs.
4. A set of templates. Using Django’s template language.

Now, this is a fine set of things to provide. However I feel there are a number of problems with Django’s approach for anything that is not just a simple “helloworld” style project. Even as a novice with Web Development I ran into limitations pretty quickly (though I should point out that I have about 10 years experience with other software domains – specifically embedded C++ development tools).

The limitations that I see are as follows:

1. Lots of Magic.
   My feeling is that there is a lot of magic that Django does behind the scenes to register classes via Meta-Classes and automagically do  lots of stuff for you. It is not always clear what this stuff is and how it might interact with other non-Django code.  Its a opaque magic. I feel to some extent it violates Python’s “better to be explicit than implicit” philosophy.
2. Very Web 1.0. So much of the standard infrastructure is based around a very old stylee web where you view-edit-submit changes. And this process takes you through a series of different web pages. This is quite frustrating.
3. SQL need only apply. All the model classes are all based around the central notion of an SQL database. Whilst this is good for small projects if you need any kind of scalability then its not so good.
   Interestingly in a talk at Google I/O Guido van Rossum talks about this limitation.  It basically means you can’t use any of the “automated” Admin style functionality that Django provides.
4. Implicit Dependencies.
   The dependency structure of Django Applications is implicit in the Python code that comprises your application. To discover all the dependencies of your app you will have to trawl though all your code to figure out what they are.
5. Fuzzy Deployment Boundaries. I feel that the boundaries between development and deployment are purposefully made fuzzy via the django-admin and management tools (manage.py) that you’re supposed to use when using Django.
6. Untestable. I don’t feel that Django Apps are very testable. If we move beyond the Web 1.0 then any reasonably sophisticated application will have a bunch of JavaScript, template files and a set of Django files to provide the Python based functionality. Not to mention default CSS, images and other static content as necessary. I think the conventions for storing templates and such like along with the app are not very strong with Django.

So its getting late.

The reason I wanted to get to this phase is that I think there is a better alternative approach to doing this.

Some thinking:

• We re-use the idea as used in Trac for having Extension points and clear “kernel” of functionality to make explicit some of the Magic that the framework is doing behind the scenes. This is a common design pattern and is especially useful when we need to be able to extend a system dynamically. (obviously really, given the name).
• We strengthen the concept of an application and turn it more into something that is self-contained and can be built, tested and deployed on its own.
• We provide clear separation between build, composition and deployment phases of a application creation. These are analogous to what would be Compilation, Linking and Loading in C or C++.
• We provide a framework for implementing Web 2.0 style applications. My current feeling is that this should be based around Panels at both client and server ends. At the client, panels will be presented to users that show data and allow that data to be edited. Meanwhile, server side a “panel API” will provide support for the panel – including doing things like validating form data and sending errors back to the client, help with scrolling through large data sets, in-line tabular editing, AJAX based auto-complete support etc.
• AJAX functionality provided by MooTools.
• Testability – there should be in-built support for doing client-side unit tests of JavaScript functionality, of server-side API functionality as well as the integration of both into a coherent whole.

I will follow this up later this week with more information about what I’ve protyped and how I intend to proceed.  But the crux is that ultimately I don’t feel that the Django “application” is  particularly useful abstraction in breaking down a sophisticated web site. Lets hope I can come up with something better!

I’ve basically been ill for the most part. I attended a friends stag do in London on the weekend of the 13th – ended up getting absolutely trashed and lost my jumper in Spearming Rhinos – classy!

After we left another club I ended up detaching from my friends and walking back to Paddington to catch the train back home. Needless to say Autumn is here and Summer is a truly distant memory. I froze my arse off waiting for the train.

On Sunday I think my lousy feeling was 100% hangover but on Monday when I woke up I felt like crap again. I attributed this to a prolonged hangover but by the end of the day realised I was actually coming down with a bad cold or a light flu.

So felt like crap for the rest of the week. Only just about started to feel normal again on Thursday and Friday, by which time I had to go down and visit an old friend from University.

I was concerned about being ill and around his daughter and pregnant wife and was re-assured it wasn’t going to be a problem as long as it wasn’t swine flu. I needn’t have worried. Through nursery, which as far as I can tell are just big snot exchanges, kids are basically just giant petri dishes/medical centrifuges where new forms of disease are evolved. I left my friends on the Sunday feeling worse than when I came! Late sunday was a complete write off as was most of Monday.

Anyway, its early in the morning on Tuesday now and at last I’m feeling much better – although I have a bit of a lingering cough.

Should be in reasonably form for my friend, Antony, who is coming tomorrow to help me with the planning for my big project. We’ll see how it goes. Hopefully I won’t be ill too many more times this winter. Must get work done!

Why do I think that will be the case? Well I have a cool name for it obviously! The other blog names I had I could never quite get behind, but I’m happy with funksoulcoder. It is going to become the basis of my entire online presence.

My username is thus fusoco – thats funk soul coder get it? You’re obviously not that dumb and it not really such a hard concept to grasp.

Anyway, I’m normally pretty reluctant to expose myself (in a figurative, show people my work kind of a way, not hang around in a bush late at night waiting to scare people…) so I thought I’d give the 100% anonymous thing a try and see what impact that has.

The hidden story there is that my reluctance to put my ideas out there in public is a big cause of procrastination. As soon as I know something that I’ve written is going ot be seen by even 1 person I tend to put off completion of that thing so as to avoid potential criticism. Yes, its a bit pathetic, and not really very “Web 2.0″but work with me here…I shall overcome!

So, what will I discuss then, I don’t know – pretty much anything I think, but probably related to politics, computer science, programming and thoughts on the future – I’d love to be a futurologist! Someone who just makes shit up about the future – what a job! T