Design patterns for data-centric software

I wrote a few days ago about software design patterns, including the thought that we're going to discover new patterns for data-centric software.  Let me unpack that concept.

First, by data-centric software, I don't mean software intended for data analysis (e.g. R, excel, or google charts).  I mean any software that collects and/or responds to data in the course of doing whatever else it does.

Web analytics are a great example of this.  The primary purpose of a web page is to serve content.  But at the same time, it's easy to track pageviews and traffic.  Compared to an untracked web site, a site instrumented with google analytics is more data-centric, because it's generating data in the background.

As I read it, the original design patterns are intended mainly to minimize long-term development costs.  The key question is "How should code be structured to make it easy to read, debug, maintain, extend, etc?"  It's all about saving developers' time in the long run.*

Five years after the original set of design patterns was popularized, another book was published, focusing on design patterns for distributed software.  This time, the key questions expanded to include bandwidth and concurrency: "How should we structure code to make the best use out of distributed computing resources?"

I think we're due for another expansion, because data-centric code introduces another optimization target: useful information.**  Just as the list of patterns expanded to deal with networking and multiprocessing, it will expand again as data processing and analytics become integral to software design.***

Off the top of my head, here's a quick list of data-centric patterns.

• A/B testing
• Funnel analysis
• Recommender systems (very broad category!)
• Top hits (most visited, emailed, etc.)
• Automatic bug reports
• Likes, +1s, Retweets

This list isn't complete, and it's clear that best practice is still evolving.  For example, A/B testing has been industry-standard for a long time, but I recently read a good argument that a multi-armed bandit algorithm is better than A/B testing, because it gathers all the same information, plus integrating that feedback directly into the site design.  It's a very natural extension and improvement over an older data-centric design pattern.  I'm sure that many other such improvements are possible.

Anyway, I think it's still too early to try to write a comprehensive list.  But I'd still like to expand this list to cover as many cases as possible.  What else belongs here?

*A few of the patterns address things like limited memory and processing power, but they're the exceptions.

** Defining useful opens up a whole new can of worms, which I won't get into here.

***This relates back to the concept that I've written about before: software design for analytics.

Software design patterns

Following a tip from an experienced software developer, I've been reading up on software design patterns: flyweights, factories, facades, etc.  These are general patterns for object-oriented programming that show up again and again.  The original canon included 23 patterns; that list has since expanded to include patterns for networking and multiprocessing.

These design patterns remind me of Go proverbs -- high-level heuristics for better strategy, sometimes contradictory.  Knowing them can be extremely helpful, but it's no guarantee that you can deploy them correctly. (Here's a good list of common go proverbs.)

Anyway, reading the original Design Patterns book, I've had three main reactions:

1. Data-centric software development is going to discover its own list of software design patterns.

2. There are patterns for research design, just like there are patterns for software design.

3. I already know most of the software patterns -- yay!*

Since I just can't sleep tonight, I figured I'd queue up a few blog posts talking about the first two.  Look for those in a couple days.

*Given my very ad hoc background in software design, I've been pleasantly surprised to find that most of the software design patterns are already familiar.  For example, python is already very good with iterators and decorators.  And working with web frameworks has taught me a lot about factories.  And many of the others are much less important in python because objects are dynamically typed.  Anyway, it's nice to discover that I've picked a lot of this up by osmosis.  (Pat self on the back.)

More about not using netlogo

I got a fair amount of pushback on my last post about NetLogo.  The basic gist was that yes, logo is an antiquated programming language, but no, that's no reason to write off the Netlogo platform as a whole.

Here's a considered response from Rainer, who studies epidemiology and public health at U Michigan.:

Regarding your Netlogo sympathies ... I totally understand where you are coming from ... Python is my favorite programming language, I also know Java well. I build ABMs in Netlogo and RePast, as well as in Python.

It took me a while to get into NetLogo and I have to agree it is a horrible "programming language". One of the major advantages of NetLogo however is that it takes care of a lot of overhead (graphical output, controls, parameter sweeps, R integration). NetLogo 5 now has some form of functional programming, list comprehension, dictionaries and other features that I haven't fully explored.

I never thought I would be a NetLogo advocate but it does have its place in the world of simulation.

These are fair, very practical points.  They are basically the same reasons we continue to use the QWERTY keyboard, even though Dvorak is probably a little faster and a lot less painful.

The difference is that keyboards are at the end of the adoption cycle, and ABMs are at the beginning.  With modeling software, there's still time to change and avoid decades of deadweight legacy loss.  Since NetLogo is primarily used as a pedagogical tool, it seems a shame that we are forcing new students to invest in a dead-end language.

With all that in mind, I've decided to become a bit of a gadfly with respect to Netlogo.   More on this subject tomorrow...  In the meantime, don't use it!

Software design for analytics: A manifesto in alpha

If you take the lean startup ideas of quick iteration, learning, and hypothesis checking seriously, then it makes sense to build your software in a way that lends itself to doing analytics.  Lately, I've been doing a lot of both (software development and analytics), so I've been thinking about how to help them play nice together.

Seems to me that MVP/MVC (or even MVVM, if you're into that kind of thing) are good at the following:

• User experience
• Database performance
• Debugging

However, when doing analytics, my needs are different. I need to extract data from the system in a way that lets me answer useful questions. Instead of thinking about performance, scalability, etc., I'm thinking about independent and dependent variables, categories and units of analysis, and causal inference.  The "system requirements" for this kind of thing are very different.

Having worked with (and built) several different systems at this point, I've realized that some designs make analytics easier, and some make them much, much harder. And nothing in general-purpose guidelines for good software design guarantees good design for analytics.

Since so much of what I do is analytics, I'd like to ferret out some best practices for that kind of development.  I don't have any settled ideas yet, but I thought I'd put some observations on paper.

Some general ideas:

1. Merges are a pain point. When doing analytics, I spend a large fraction of my time merging and converting data. Seems like there ought to be some good practices/tools to take away some of the pain.
2. Visualization is also a pain point, but I'm less optimistic about fixing it.  There's a lot of art to good visualization.
3. Units of analysis might be a good place to begin/focus thinking.  They tend to change less often than variables, and many design issues for archiving, merging, reporting, and hypothesis testing focus on units of analysis.
4. The most important unit of analysis is probably the user, because most leap-of-faith assumptions center on users and markets, and because people are just plain complicated.  In some situations (e.g. B2B), the unit of analysis might be a group or organization, but even then, users are going to play an important role.
5. Make it easy to keep track of where the data come from!  Any time you change the
6. From a statistical perspective, we probably want to assume independence all over the place for simplicity -- but be aware that that's what we're doing!  For instance, it might make sense to assume that sessions are independent, even though they're actually linked across users.
7. User segmentation seems like an underexploited area.  That is, most UI optimization is done using A-B testing, which optimizes for the "average user."  But in many cases, it could be very useful to try to segment the population into sub-populations, and figure out how their needs are different.  This won't work when we only have short interactions with anonymous users.  But if we have some history or background data (e.g. FB graph info), if could be a very powerful tool.
8. Corollary: grab user data whenever it's cheap.

I'll close with questions.  What else belongs in this list?  Are there other people who are thinking about similar issues?  What process and technical solutions could help?  NoSQL and functional programming come to mind, but I haven't thought through the details.

Resources on NLP sentence diagramming

Here are some notes from a recent search for resources on automatic sentence diagramming. I was looking for code/software to diagram sentences automatically, ideally in python.

Vocab
Also, as far as I can tell, linguists, grammarians, and English majors call sentence diagrams "Reed-Kellogg" diagrams. NLP and computer science types call the diagrams "parse trees" or "concrete syntax trees," and produce them (usually) using probabalistic context free grammars (PCFGs).
 
Search result
If you want code that just works, the Stanford Parser looks like the place to start. It's in java, but I'm sure that won't be too much of a problem, because you can call it from the command line.

Python's NLTK might also work, but there are lots of different parsers, and it's not clear whether and how much training they require.

There are various other software packages out there -- some of them online -- but I doubt they'd support much volume, and batching would be a pain.





Background
http://en.wikipedia.org/wiki/Sentence_diagram
http://en.wikipedia.org/wiki/Parse_tree
http://en.wikipedia.org/wiki/Context-free_grammar

NLTK
http://nltk.googlecode.com/svn/trunk/doc/book/ch08.html
http://www.cs.bgu.ac.il/~elhadad/nlp11/nltk-pcfg.html
http://www.ibm.com/developerworks/linux/library/l-cpnltk/index.html

Stanford Parser
http://nlp.stanford.edu/downloads/lex-parser.shtml
http://nlp.stanford.edu:8080/parser/index.jsp
http://projects.csail.mit.edu/spatial/Stanford_Parser

Other Software & Code
http://www.sendraw.ucf.edu/
http://1aiway.com/nlp4net/services/enparser/reedkelloggtree.aspx
http://faculty.washington.edu/dillon/GramResources/

Funny blog posts about politics and grammar, or the lack thereof
http://boingboing.net/2009/02/17/how-obamas-sentences.html
http://www.slate.com/articles/life/the_good_word/2008/10/diagramming_sarah.html