Nifty tools for playing with words

Here are a bunch of sites I use to play with words -- whether brainstorming or trying to accomplish something specific with text analysis.

A rhyming dictionary. Helpfully splits up the word list by syllables, so you can finish that sonnet you've been working on.


Here's a nifty little site for generating portmanteaus (word splices):  http://www.werdmerge.com/


http://www.leandomainsearch.com: generates themed domain names, and checks to make sure they're unclaimed by URL squatters.

Online lorem generator.  Here's the same thing in python.

Markov text generation: http://www.beetleinabox.com/markov.html.

Permute words and letters.  This seems less useful to me... It gives all the combinations, not just the ones that make some kind of sense.

Lavarand used to do random haikus and corporate memos, but it looks like they've broken down.

Google ngrams on AWS public data sets.  These are combinations of words that commonly co-occur in English.

Yes, yes.  And then there's wordle.  Too pretty for the rest of us.

What else belong on this list?

For future reference: upgrading to R 2.14 on Ubuntu 11.10

Today, I needed to upgrade from R 2.13 to R 2.14 on ubuntu 11.10, because I wanted to install the "tm" package.

It's easy once you know how -- you just need to get your debian sources.list pointed to the right place. For future ref, here are my notes on how this feat is accomplished.


Instructions for upgrading
http://cran.r-project.org/bin/linux/ubuntu/

List of CRAN mirrors
http://cran.r-project.org/mirrors.html

My favorite mirror:
http://cran.mtu.edu/

The line I added to /etc/apt/sources.list:

deb http://cran.mtu.edu/bin/linux/ubuntu oneiric/

From there, it's just a simple

   sudo apt-get update
   sudo apt-get install r-base
   sudo apt-get install r-base-dev

snowCrawl bug fixes and easier downloading

I've made a few small fixes to the snowCrawl python library. (See the updates page for details.)

The most important change is that I've created zipped and tar-gzipped versions of the repository for download. Now you can get snowcrawl even if you don't speak subversion. See the downloads page for one-click downloads.

Python html cleaner

A friend asked for help cleaning html files today --- getting rid of scripts, styles, tags, and whatnot.

Here's a quick 14-line python script to do the job. It takes all the html files in the ./docs directory and writes them out as clean text to the ./output directory.

Note the lxml dependency!

from lxml.html import clean
import glob, re

cleaner = clean.Cleaner( style=True, scripts=True, comments=True, safe_attrs_only=True )

filenames = glob.glob('docs/*')
for f in filenames:
    print '='*80
    text = file(f,'r').read()
    text = cleaner.clean_html( text )    #Remove scripts, styles, etc.
    text = re.sub('<.*?>', '', text )    #Remove html tags
    text = re.sub('\s+', ' ', text )    #Remove whitespace
    print text
    file( 'output/'+f.split('/')[-1].split('.')[0]+'.txt', 'w').write( text )

Ruby on rails on EC2 in 20 minutes

I wound up with an extra 20 minutes the other day, so I looked to see how hard it would be to set up a rails server on an EC2 instance.

Answer: very not hard.

I haven't done much with rails in the past (One week of code with my brother looking over my shoulder.)

I grabbed the RubyStack 2.3-0 Dev (Ubuntu 10.04) AMI from http://bitnami.org/stack/rubystackrails.

Once it initialized, I ssh'ed in and ran:

rails new blog
cd blog

I followed the directions here to edit the gemfile, then ran:

rake db:create
rails server

Tada! Instant rails server.


PS - Yes, yes I know about heroku.  In this case, building the app as an EC2 AMI is an essential part of the project. I want non-programmers to be able to clone the instance, and sharing a public AMI makes things pretty easy.

JSON editors

I've been doing a lot of work with json lately, and realized that it would be handy to have a program to edit and validate json syntax.

Turns out I'm not the first person to see this need.  Here are two nice, in-browser json editors.  Nifty!

• http://braincast.nl/samples/jsoneditor/
• http://jsoneditor.appspot.com/

RegExr

Here's a great little site that lets you test regular expressions. The splash page says:

Welcome to RegExr 0.3b, an intuitive tool for learning, writing, and testing Regular Expressions. Key features include:

• real time results: shows results as you type
• code hinting: roll over your expression to see info on specific elements
• detailed results: roll over a match to see details & view group info below
• built in regex guide: double click entries to insert them into your expression
• online & desktop: regexr.com or download the desktop version for Mac, Windows, or Linux
• save your expressions: My Saved expressions are saved locally
• search Community expressions and add your own
• create Share Links to send your expressions to co-workers or link to them on Twitter or your blog [ex. http://RegExr.com?2rjl6]

Built by gskinner.com with Flex 3 [adobe.com/go/flex] and Spelling Plus Library for text highlighting [gskinner.com/products/spl].