Definitions for data science

Since I'm rebooting this blog, this seems like a good moment to lay out a framework for data science.  I'll tackle definitions now, and process next time.

Pinning down scope and definitions is important for data science, because the field is growing rapidly, with a sense that the sky is the limit.  Without priorities and a grasp of what data science isn't, we run the risk of overreaching, wasting our time, and leaving everyone disappointed.  I won't claim that my definition is the only definition, or even the best definition.  But it works for me, and it has some virtues worth discussing.

Essentially, I think of data science as "answering questions with data," or more precisely, "providing empirical answers to well-posed questions." By empirical, I mean "based on information that all participants can observe in common."  By well-posed, I mean "admitting a definitive answer": once we see the right answer, we can all agree that it's the right one. In the language of formal logic, a well-posed question is one that admits a deductively valid conclusion.  So, data = empirical, science = questions.

The main difference between my definition and most of the others floating around (e.g. here) is that I focus on the goal of data science (answering questions), not the tools or methods for getting there (e.g. data munging, predictive analytics, writing mapReduce queries).

I find that defining data science by goals instead of tools adds clarity, for two reasons.  First, goals usually provide a more defined boundary than tools.  Almost none of the tools of data science are unique to data science. Software engineers do lots of "hacking"; forecasters do lots of statistical modeling; DB admins use plenty of NoSQL.  None of these things on its own provides a bright line for determining who is a data scientist or not, so we have to take a fuzzy average over lots of categories, and end up with a large gray area of jobs that are "kind of" data science. In contrast, it's usually pretty clear if your goal is answering questions (a.k.a. "providing insight," "running analytics," "informing decisions") or not.

Second, focusing on goals lets us differentiate approaches by effectiveness. Without a clear understanding of the job of data science, it's impossible to tell the difference between professionals who choose the right tools to get the job done, and bandwaggoners who are just playing with every shiny new toy.  Since the bandwagoning has started already, I think we'll be well served to differentiate between effective data scientists and the tools they use.

Analogy: I'm in the hospital for an appendectomy as I write this, going under the knife in a few hours.  I find it much more comforting to think of the doctors in terms of goals ("People who help you regain their health") than tools and methods ("People who cut holes in you with scalpels and wires").  Similarly, I'd be much happier hiring a data scientist who is good at answering questions, than one who is good with mongoDB, or Bayesian models.  Having the tools is necessary but not sufficient to accomplish the goals.

With those ideas on the table, here are some comparisons I'd like to explore in the future:

• How is data science different from "big data"?
• How is data science different from statistics?
• How is data science different from data analysis?
• How is data science different from science in general?
• How is data science different from software engineering?

What do you think?  Discuss.

Authorship of Ron Paul newsletters -- statistical text analysis and Bayesianism

In the midst of all the hullabaloo about Ron Paul's decades-old racists newsletters, and his denial that he "never read that stuff," I ran across an interesting attempt to adding some data to the discussion.  Filed under "pointless data exercises" and "politics," blogger Peter Larson has used text analysis to compare his blog, Ron Paul's recent speeches, and the original newsletters.  He calls his results a smoking gun, with a question mark tacked on, and argues that Ron Paul wrote most of the original newsletters.

Before I go on, let me say that this is a really cool application.  Instead of the he-said-she-said debate that's running in the media, this piece brings some actual data to bear on the conversation.  Bravo!

That said, now I'm going to harp on statistics and inference.  The problem with Larson's analysis is that he never addresses the question, "If Ron Paul didn't write the newsletters, who did?"  Without answering that question, and putting some probabilities behind it, it's going to be very hard for text analysis to prove the issue one way or another. (Larson admits this on his blog.) Right now, his analysis proves that between himself and Ron Paul, Paul is much more likely to have written the letters.  Not exactly a smoking gun.


That said, it's interesting data.  For the record, I'm mostly convinced.  In my mind, the statistics are flawed, but they still lend some weight against Paul.  Oddly enough, Larson's finding that several of the letters were probably *not* written by Ron Paul was particularly persuasive.  It feels human and messy, the way I would expect this kind of thing to be.



Final thoughts, mainly intended for my statistically minded friends: yes, this is an unabashedly Bayesian perspective.  I'm demanding priors, which probably can't be specified to anyone's satisfaction.

IMHO, the frequentist approach has even deeper problems.  From a frequentist perspective (which is where Larson's original, and deeply confusing p-values come from.) we use Paul's recent speeches and writings to estimate some parameters of his current text-generating process.  We then compare the newsletters to that process and estimate the probability that the older text was generated by the same process.

Problem: we *know* the old text was not generated by the same process.  It was written (allegedly) by a younger Ron Paul, on different topics, speaking into a different political climate.  Without a broader framework, it's impossible to determine whether the differences are important.  The Bayesian approach provides a direct way of assessing that framework.  The frequentist approach doesn't -- at least not that I can see, without jumping through a lot of hoops -- and in the meantime, it obscures the test that's actually being conducted.