Tuesday, 17 March 2015

neo4j and Social Media Scraping for twitter, Stack Exchange and Github

I've been experimenting for the last couple of days with tools to help me identify experts and their comments in particular technical areas. I'm hoping to combine data from twitter, stack exchange and git-hub in a way that allows me to look for people who have
  • tweeted about a topic
  • posted answers on StackExchange or
  • published relevant code on GitHub
I'm also interested in how they interact.

There's a relevant demonstration using clojure and Ruby described on the neo4j blog, and I am using that as a basis for my Python development.

If you're not familiar with neo4j, it's a graph datase. Visit their site to find out what that term means if it's new to you. What matters is that it's a great way to store and quesry social replationships and social meida activity.

I'll report on progress, but one of my jobs for tomorrow will be to see how usable my code is on the Raspberry Pi 2.