Showing posts with label working-notes. Show all posts
Showing posts with label working-notes. Show all posts

Wednesday, February 22, 2012

Getting started with django, heroku, and mongolab

Long post here -- I just spent a couple hours going from zero to mongo with django and heroku.  Here are my working notes.

For this test, I started from a pure vanilla django project:
django-admin startproject testproj

I made a single edit to settings.py, adding "gunicorn" to the list of INSTALLED_APPS.

For setting up django on heroku, I basically followed the description given in these very good notes by Ken Cochrane.  I've done this before and there were no surprises, so I'll skip the stuff about setting up heroku and git.  I did nothing special except install gunicorn.

Here's my Procfile:
web: python testproj/manage.py collectstatic --noinput; python testproj/manage.py run_gunicorn -b "0.0.0.0:$PORT"

This is overkill at the outset--we don't have any static files, so collecting them doesn't do anything.

And my requirements.txt:
Django==1.3.1
gunicorn==0.13.4
psycopg2==2.4.4
simplejson==2.3.2

That got me to the famous django opening screen, live on heroku:
It worked!
Congratulations on your first Django-powered page.

Just to emphasize that I've done nothing special so far, here's the tree for the git repo I'm using.  (I've suppressed .pyc and *~ files.)

    .
    ├── Procfile
    ├── requirements.txt
    └── testproj
        ├── __init__.py
        ├── manage.py
        ├── settings.py
        └── urls.py

(Come to think of it, there should probably be a .gitignore file in there as well.)

Let's pick up from the point.

I added the mongolab starter tier (free up to 240 MB) to the heroku app.  I did this from the heroku console, because I wanted to see what options were there.  (Oddly, none of the dedicated tiers show up there.)  In the future, I'll probably just use the command line:
heroku addons:add mongolab:starter

Next, I followed the instructions on heroku's documentation, and grabbed the URI:
heroku config | grep MONGOLAB_URI

The MONGOLAB_URI is in the following format:
mongodb://username:password@host:port/database

At this point, heroku's documents stopped being much help, because they don't cover python.  So I switched over here instead.  I wanted to understand all the steps, so I refrained from copy-pasting.

I installed a few supporting libraries
pip install pymongo django-mongodb-engine djangotoolbox

And added the appropriate lines to requirements.txt
pymongo==2.1.1
django-mongodb-engine==0.4.0
djangotoolbox==0.9.2

I then configured the database in settings.py.
DATABASES = {
    'default': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'heroku_app1234567',
        'USER': 'heroku_app1234567',
        'PASSWORD': 'abcdefghijklmnopqrstuvwxyz',
        'HOST': 'ds031117.mongolab.com',
        'PORT': '31117',
    }
}

Note: With versions of django-mongodb-engine over 0.2, ENGINE should be 'django_mongodb_engine', not 'django_mongodb_engine'.  Get this wrong and you'll see something like:
django.core.exceptions.ImproperlyConfigured:
'django_mongodb_engine.mongodb' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
    'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: No module named mongodb.base

I never did anything about the "current bug" that Dennis mentions.  Apparently it's been patched, whatever it was.

Quick check: so far, running locally (./manage.py runsersver) and pushing to heroku both work.  The next step is to add a model or two.  Time to fix that:
django-admin startapp onebutton
Here's models.py:
from django.db import models

class ButtonClick( models.Model ):
    click_time = models.DateTimeField( auto_now=True )
    animal = models.CharField( max_length=200 )
I was going to build a stupid-simple app with a single button to "catch" animals from a random list and store them in the DB, but it's getting late, so let's just jump to the proof of concept.

Going straight to the django shell on my computer...
$python manage.py shell
>>> from testproj.onebutton.models import ButtonClick as BC
>>> c1 = BC( animal="Leopard" )
>>> c1.save()
>>>
It works!  Open up the mongolab console (through the add-ons tab in heroku) and the database shows a single record in onebutton_buttonclick collection.

We still haven't written views and templates, and (more importantly) validated that the heroku app can talk to the DB over at mongolab, but I'm going to call this good enough for now.  Mission accomplished.

Thursday, February 16, 2012

Working notes on cloud-based MongoDB with python

I've been thinking about getting into mongoDB for a good while.  I'm looking for a platform that works, scales, and integrates with python with a minimum of hassle.  Cheap would be nice too.

Tonight, Google and I sat down to do some nuts and bold research.  Here are my notes.  Have anything to add?


PS - Based on what I found, I'm thinking Heroku + MongoLab + PyMongo + Django is probably the best way to get my feet wet, since I'm already comfortable with django and heroku. 

I'll be trying this in the near future -- will let you know how it goes.



Cloud hosts for mongoDB:
  •     MongoLab
  •     MongoHQ
  •     MongoMachine -- bought by MongoHQ

Reviews here say MongoLab > MongoHQ w.r.t customer service
    http://www.quora.com/Heroku/How-would-I-use-the-mongolab-add-on-with-python

python ORMs for mongo
  •     mongonaut
  •     mongoengine
  •     mongokit/django-mongokit
  •     pymongo (simple wrapper, no ORM)
  •     ming
  •     django-mongodb
  •     django-nonrel

Strong recc for mongoengine > mongokit, esp for django developers.
    http://www.quora.com/MongoDB/Whats-the-best-MongoDB-ORM-for-Python

Says mongoEngine is faster than mongoKit
    http://www.peterbe.com/plog/mongoengine-vs.-django-mongokit

Slides also argue for mongoEngine
    http://www.peterbe.com/plog/using-mongodb-in-your-django-app/django-mongodb-html5-slides/html5.html

Says that pyMongo > mongoEngine
    http://stackoverflow.com/questions/2740837/which-python-api-should-be-used-with-mongo-db-and-django

mongoNaut is clearly not mature -- off the island!
    http://readthedocs.org/docs/django-mongonaut/en/latest/index.html

Instructions for setting up django and mongo, if that's your thing
    http://dennisgurnick.com/2010/07/06/bootstrapping-a-django-with-mongo-project/

PyMongo documentation
    http://api.mongodb.org/python/1.7/faq.html

MongoLab's example of integration, plus a small amount of stackoverflow chatter about it.
    https://github.com/mongolab/mongodb-driver-examples/blob/master/python/pymongo_simple_example.py
    http://stackoverflow.com/questions/8859532/how-can-i-use-the-mongolab-add-on-to-heroku-from-python


Decision reached!
    Heroku + MongoLab + PyMongo

Tuesday, December 6, 2011

Fixing rabbitmq after switching servers

I ran into some backend headaches with django, celery, and rabbitmq over the weekend. The EC2 instance I was using crashed, so I had to switch servers. When I did, rabbitmq-server, which the broker behind celery, which is the task queue behind django -- broke.  The broker broke.  And considering that there are still parts of django I'm wrapping my head around, chasing down a system error three levels deep was quiet the pain.

Happily, I got some very good help from the rabbitmq-discuss list:

The database RabbitMQ uses is bound to the machine's hostname, so if you copied the database dir to another machine, it won't work.  If this is the case, you have to set up a machine with the same hostname as before and transfer any outstanding messages to the new machine.  If there's nothing important in rabbit, you could just clear everything by removing the RabbitMQ files in /var/lib/rabbitmq.

I deleted everything in /var/lib/rabbitmq/mnesia/rabbit/ and it started up without trouble.  Hooray!

I also got a warning about using outdated software:

1.7.2 is a *very* old version of the broker.  You should really upgrade, as a lot of new features and a lot of bugfixes have gone in since then.

Now, as best I can recall, I originally installed rabbitmq with apt-get, running in Ubuntu 10.04. To fix this, I got the newest .deb from the rabbitMQ site and installed it.  Here are the steps:

wget http://www.rabbitmq.com/releases/rabbitmq-server/v2.7.0/rabbitmq-server_2.7.0-1_all.deb
sudo apt-get install erlang-nox
sudo apt-get -f install
sudo apt-get install erlang-nox
sudo dpkg -i rabbitmq-server_2.7.0-1_all.deb

Very happy to be out of the woods on this one. I'm not a sysadmin and I don't aspire to be.

Saturday, December 3, 2011

Getting started in git

With some coaxing from my brother, I finally entered the world of git today. Working notes.

Nice how-to for webfaction
http://www.nitinh.com/2011/02/github-setup-on-webfaction-how-to/

I had this problem...
http://stackoverflow.com/questions/2702731/git-fails-when-pushing-commit-to-github

Getting started on github
http://help.github.com/linux-set-up-git/

The inner workings of the distributed system are still a bit of a mystery, but I gather I'm not the only one in that boat.  For now, it works: I can take changes in development and push/pull them to my production server.  Progress!

dev$ git commit -a
dev$ git push origin
prd$ git git pull origin master

Thursday, December 1, 2011

Production mode in django: lxml, celery, the works.

Firing up production mode in django.  These refs were helpful.

LXML
http://answers.bitnami.org/questions/1765/use-lxml-with-djangostack

Celery
http://django-celery.readthedocs.org/en/latest/getting-started/first-steps-with-django.html