[Air-L] Advice on ripping a twitter feed

[Fenwick Mckelvey]

Hi,
The Infoscape Lab has some experience tracking twitter feeds that I
would like to share. First, twitter feeds are RSS feeds. As a result,
a number of tools exists to archive an analysis content, like the
Coding Analysis Toolkit. We use a RSS aggregator called Gregarius
(http://gregarius.net/)  to collect Twitter feeds and save them in a
database. Importantly, Twitter moves faster than blogs. Most RSS
aggregators  collect on an hourly or daily basis. We have had to
manually refresh our aggregator  every 10 minutes to catch the flow of
tweets in busy periods. Second, we have used hashtags as a way to
sample discussion in Twitter. During our recent Canadian election, we
tracked hashtags related to a leadership debate. Hashtags can be
tricky because they change rapidly and seem to naturally emerge, so we
also relied on a basket of users as well.

You can see our Twitter coverage of the Canadian debates here:
http://www.cbc.ca/news/canadavotes/campaign2/ormiston/2008/10/debate_hangover.html.
We are pretty happy with this time-sensitive sample of Twitter because
it captured how people flock to the site during important moments like
the debates. If any one has any more questions about our perspective
or method on Twitter, I'd be happy to help.

All the best,
Fen

On Thu, Apr 2, 2009 at 6:50 PM, Stuart Shulman <stuart.shulman at gmail.com> wrote:
> Very cool. Thanks Andrew!
> This is a great time to send feedback about BAT. The purpose of the Blog
> Analysis Toolkit is to establish a socially-constructed repository of blog
> posts that are archived and accessible for research purposes. There are
> about 250 BAT users at the present moment archiving about 200 blogs.The
> posts are formatted in one of two ways to allow coding at the document or
> paragraph level using another free software system, the Coding Analysis
> Toolkit <http://cat.ucsur.pitt.edu/> (CAT). Once you join the system you
> have access to all the archived posts and you can add new blogs to the
> archiving process.
>
> We have just started a new programmer to improve the platform, which is a
> free by-product of ongoing NSF-funded research. We want to increase its
> functionality and usability, so AoIR members are strongly encouraged to let
> us know what you want BAT to do in the future. We face challenges doing some
> simple things, like getting the comments and the archives. If you know how,
> perhaps how join the BAT development team.
> The quick-start BAT tutorial is online at:
>
> http://www.screencast.com/t/OcRziCMg
>
> ~Stu
>
>
>
> On Thu, Apr 2, 2009 at 6:10 PM, Andrew Long
> <ALong at infoscience.otago.ac.nz>wrote:
>
>> Incidentally, I have tried the Blog Analysis toolkit (see blelow) and it
>> works fine.
>> Grab the RSS feed from the right-hand side of the Twitter website and
>> set this as
>> the blog URL.
>>
>>
>
> --
> Dr. Stuart W. Shulman
> Assistant Professor
> Department of Political Science
> University of Massachusetts Amherst
> 200 Hicks Way
> Amherst, MA 01003
>
> http://people.umass.edu/stu/
> stu at polsci.umass.edu
> 413-545-5375
>
> Editor, Journal of Information Technology and Politics
> http://www.jitp.net
>
> Director, QDAP-UMass
> http://www.umass.edu/qdap/
>
> Associate Director, National Center for Digital Government
> http://www.umass.edu/digitalcenter/
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>



-- 
Fenwick McKelvey
PhD Student in Communication and Culture
Ryerson / York Universities

Research Associate
Infoscape Research Lab
http://www.infoscapelab.ca

Research Associate
VideoCom Research Initiative
http://videocom.knet.ca