Thoughts on computation, social science, and lifehacking
from an up-and-coming data scientist.
Tuesday, November 29, 2011
Configuring django with wsgi
#Getting the latest version of R on an older version of ubuntu:
http://cran.r-project.org/bin/linux/ubuntu/README
#Using PIP and VirtualEnv with django
http://www.saltycrane.com/blog/2009/05/notes-using-pip-and-virtualenv-django/
sudo apt-get install python-setuptools python-dev build-essential
pip install numpy
sudo pip install numpy
sudo pip install scipy
sudo pip install rpy2
I never quite cracked this one. It seems that by default, pip doesn't work with the bitnami djangostack. I just used easy_install, with good results.
#Configuring WSGI
http://bitnami.org/forums/forums/djangostack/topics/getting-started
http://serverfault.com/questions/327380/beginner-installing-first-app-on-ec2-bitnami-stack
Neither of these worked perfectly for me, but they got me to the point where I could figure it out.
#Serving static files with WSGI
https://docs.djangoproject.com/en/dev/howto/static-files/
https://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/
https://docs.djangoproject.com/en/dev/howto/deployment/modwsg/#serving-files
Trust the django docs to have a good built-in explanation.
Monday, November 28, 2011
CSAAW presentation: "How to plan a heist: Challenges, models, and tactics for researching information flow"
Title:
How to plan a heist: Challenges, models, and tactics for researching information flow
Abstract:
By definition, all social systems involve information flow: complex stimuli that connect individuals and coordinate action. However, like executing a heist, measuring and making inferences about these flows is a difficult, thorny problem. This presentation will describe these challenges (hidden networks, subtle signals), introduce mathematical tools for analyzing them (Pearl's graphical causal models, Shannon's mutual information), and discuss tactics for moving forward (experiments, causal and behavioral aggregation, and action space mining). These principles may also be useful for breaking into banks, pilfering paintings, and conning mob bosses.
Friday, November 25, 2011
using mechanical turk for research
Perspectives on Psychological Science has a short piece on using Amazon’s Mechanical Turk as a subject pool: “Amazon’s Mechanical Turk: A New Source for Inexpensive, Yet High-Quality, Data?”
As Google Scholar shows, Mechanical Turk is being used in lots of clever ways.
Mechanical Turk has been called a digital sweatshop. Here are two perspectives – an Economic Letters piece: “The condition of the Turking class: Are online employers fair and honest?” And, a piece calling for intervention: “Working the crowd: Employment and labor law in the crowdsourcing industry.”
Here’s the Mechanical Turk page. Here are some research-related tasks that you can get paid for.
I've done a lot of work with mturk. It's great for large-scale repetitive tasks, but expect to spend a fair amount of effort setting up and doing quality control. One of the really interesting things about the Google Scholar links is the number of different journals and fields using mturk for research.