[Air-L] Twitter Scraper

[dominique.a.salas]

Thank you all so much. I'm excited to research these options!


Dominique Salas, MFA PhD candidate in Latin American Studies, Tulane University dominiqueasalas.tumblr.com 
-------- Original message --------From: Stuart Shulman <stuart.shulman at gmail.com> Date: 9/26/16  6:57 AM  (GMT-06:00) To: Karine Nahon <karine at ekarine.org> Cc: Maurice Vergeer <m.vergeer at maw.ru.nl>, dominique salas <dominique.a.salas at gmail.com>, air-l <air-l at listserv.aoir.org> Subject: Re: [Air-L] Twitter Scraper 
Karine is right, but you can get every Tweet if you are able to generate funding to employ a service like Gnip:
http://support.gnip.com/apis/powertrack2.0/

The new Gnip PowerTrack has many cool expanded capabilities, including longer rules, emoji search and cashtags. With a student account on DiscoverText ($24/month), the Twitter data price is $3/10,000 Tweets day forward and it includes access to the PowerTrack 2.0. You can do a lot of great exploratory research for free using the 30-day trial. The same volume pricing is available for historical data via Sifter, plus a fee of $20/day searched. 
In terms of viral Tweets, the automated duplicate detection and near duplicate clustering presents a road map of RTs and MTs, as well as a unique sampling method when coding data or training classifiers.
DiscoverText & Sifter Explained:
https://vimeo.com/124029796

https://vimeo.com/126214352
~Stu
Stu Shulman
Amherst Regional High School, Coach
MA Olympic Development Program, Assistant Coach








On Mon, Sep 26, 2016 at 1:49 AM, Karine Nahon <karine at ekarine.org> wrote:
Dominique,

Note that harvesting data in case of viral information is more complicated because of limitations (how much information can you basically mine) which exist in different APIs.

Karine





Karine Nahon/Author of Going Viral/Best Information Science Book Award and Outstanding Academic Title/eKarine.org

Associate Professor/Interdisciplinary Center (IDC) / University of Washington



On 26/9/16, 08:11, "Air-L on behalf of Maurice Vergeer" <air-l-bounces at listserv.aoir.org on behalf of m.vergeer at maw.ru.nl> wrote:



    Dear Dominique,



    please look at this site (http://socialmediadata.wikidot.com/) for an

    extended list of social media tools. Many are stand alone applications and

    some are packages within another software environment. Some focus on

    tweets, while others on networks. Depending what you need one or the other

    might serve your need.

    Because your project seems issue related, a tool using the search API for

    hashtag sampling seems most appropriate. I use yourtwapperkeeper

    (standalone on a linux machine) and streamR in R (windows Mac or Linux).

    The benefit of using a package in R is that R you can use R subsequently

    for further analysis. But it's a steep learning curve, but definitely pays

    off in the long run.



    HTH

    Maurice



    On Mon, Sep 26, 2016 at 1:37 AM, dominique salas <

    dominique.a.salas at gmail.com> wrote:



    > Hello all,

    >

    > I’m new to the listserv but look forward to learning in this great

    > community. I was recommended to join the listserv to ask what might be the

    > most efficient Twitter scraper/bot set-up as of late. I am trying to track

    > circulated and augmented arguments online, but since many are viral and

    > also are picked up by various news networks, getting a dataset is

    > elementary and crucial.

    >

    > I look forward to hearing back, even if you anticipate problems or issues

    > I might run into.

    >

    >

    > Dominique Salas, MFA

    > PhD candidate in Latin American Studies, Tulane University

    > _______________________________________________

    > The Air-L at listserv.aoir.org mailing list

    > is provided by the Association of Internet Researchers http://aoir.org

    > Subscribe, change options or unsubscribe at:

    > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org

    >

    > Join the Association of Internet Researchers:

    > http://www.aoir.org/









    --

    ________________________________________________

    Maurice Vergeer

    To contact me, see http://mauricevergeer.nl/node/5

    To see my publications, see http://mauricevergeer.nl/node/1

    ________________________________________________

    _______________________________________________

    The Air-L at listserv.aoir.org mailing list

    is provided by the Association of Internet Researchers http://aoir.org

    Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org



    Join the Association of Internet Researchers:

    http://www.aoir.org/



_______________________________________________

The Air-L at listserv.aoir.org mailing list

is provided by the Association of Internet Researchers http://aoir.org

Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org



Join the Association of Internet Researchers:

http://www.aoir.org/