Wednesday, June 13, 2012

Who you follow but he or she does not follow you back on Twitter

This tutorial primarily targets DIY inclined nonprogrammers or very lazy programmers. In order to do professional interaction with Twitter one would use their API which is quite well documented and development portal is available here https://dev.twitter.com/
For DIY approach we are happy to pull HTML, page source and work with it. I use Firefox and add-on Firebug by Joe Hewitt. If you do not have Firebug between your add-ons - install it.  If you are using Google Chrome Cedric created similar tutorial here http://www.cedricve.me/2012/04/10/howto-scrape-your-twitter-followers-with-perl/ though code and goal is somewhat different.
Now login into your Twitter account and expand all your followers to the bottom of the page. If we save page now there will be no required data, it is loaded via Java script how we scrolling and expanding list. To see data we invoke Firebug via context menu, right click on the page and select "Inspect Element with Firebug".


In search box (right from context menu on the picture) we type "stream-items-id" and hit enter. Selecting whole "div" we right click and copy inner HTML. Paste that in your favorite text editor and save, people using Windows should use Notepad++. Do the same for list following. I saved followers into file called fm and following into ft, if you like different naming scheme, please rename files in code accordingly.
In directory where fm and ft are saved create Perl script with following content:


when you run it, terminal or command line or from IDE it should produce output like this:

Queen_Europe - 407347022
FaulkesSouth - 102351359
lcogt - 80797776
glenn_hughes - 15565190
FaulkesNorth - 102350867
Rogozin - 36980694
wikileaks - 16589206
Nigel_Farage - 19017675
StoppINDECT – 169043724

those are screen names and user ID pairs.