[Air-L] Downloading Twitter feeds

Wed Apr 19 15:08:55 PDT 2023

It works with Twitter Developer credentials with NodeXL's import function
for social network analysis.

On Wed, Apr 19, 2023 at 4:40 PM Marc Gallofré Ocaña via Air-L <
air-l at listserv.aoir.org> wrote:

> I can confirm that the Academic API v2 for Twitter is working, at least
> outside US.
>
> Use a python library like Tweepy and a code like the following you can get
> a stream of data from tweets in US. You will need to create a project in
> the Dev Dashboard and add your previous Apps to the project. The project
> should have been created for research purposes and the app too:
>
> import tweepy
> import operator
>
> class MyStream2(tweepy.StreamingClient):
>
>     def on_tweet(self, tweet):
>         print(tweet.text)
>
>
> streaming_client = MyStream2(BEARER_TOKEN)
> res = streaming_client.get_rules()
> if res.data:
>     rules = list(map(operator.attrgetter("id"),res.data))
>     streaming_client.delete_rules(rules)
> streaming_client.add_rules(tweepy.StreamRule("place_country:US"))
> streaming_client.filter()
>
> Marc
>
> -----Original Message-----
> From: Air-L <air-l-bounces at listserv.aoir.org> On Behalf Of Stuart Shulman
> via Air-L
> Sent: Tuesday, April 18, 2023 12:46 PM
> To: M.E.Sciubba <mesciubba at gmail.com>
> Cc: air-l at listserv.aoir.org
> Subject: Re: [Air-L] Downloading Twitter feeds
>
> The answer is complex, uncertain, and shifting. It is more than just a
> technical issue of "Where can I get data now?" There are fundamental legal,
> jurisdictional, market, equity, and political questions unresolved.
> Resolution seems unlikely in the short term. As a political scientist
> interested in election interference, I do not anticipate a positive
> resolution before the 2024 U.S. election. I would love to be wrong on that
> prediction. I have heard there are a few firms that retain agreements to
> market real time and historical Twitter data. You might ask your librarian
> about access to Crimson Hexagon, Meltwater, or other similar services with
> long term agreements Mr. Musk has not broken, yet. Nobody on the AIR-L list
> has confirmed whether the academic API is shut down. Is it? I don't have
> those credentials to check.
>
> As I have written previously, there are massive troves of Tweets in every
> computer science department and many social science departments.
> Any one of the raw JSON Tweet archives could be loaded into a free account
> on DiscoverText and then shared for teaching or research. Talk to the data
> folks on your campus and across your discipline about what is extant. Do
> not look for spreadsheets; find the raw JSON.
> Spreadsheets are significantly degraded historical objects that cannot be
> considered accurate representations of Tweets. Tweets live in the Twitter
> display and die in spreadsheets. I remain befuddled by the Python/R nexus.
> Those spreadsheets of Tweets simply are not Twitter.
> Any qualitative researcher knows what I mean. Meaning is produced in the
> interaction of the display elements. Meaning is destroyed, diluted, and
> bent out of shape in spreadsheets.
>
> The loss of real time access to new data is a problem and dangerous for
> democratic systems. This does not change the fact that so much data has
> been gathered and stored, like so many old newspapers. For example, here is
> a list of the Top 50 datasets in my DiscoverText account that any academic
> on this list or anywhere in the world can access via the "peer network"
> that enables collaboration, annotation, measurement of inter-rater
> reliability, and the creation of gold standard training sets for
> machine-learning via a graphical user interface that features the Twitter
> display.
>
>
> https://docs.google.com/spreadsheets/d/1j-Y40WmwFIX8pidfAcxgAt5JB3A3UyB2psh_ULQdynA/edit?usp=sharing
>
> "The Case Against Spreadsheets as a Primary Twitter Research Tool"
> https://vimeo.com/526218014
> Question: When is a Tweet not a Tweet? Answer: When it appears in a
> spreadsheet, or is deleted, or the account has been suspended.
>
>
>
> On Tue, Apr 18, 2023 at 5:57 AM M.E.Sciubba via Air-L <
> air-l at listserv.aoir.org> wrote:
>
> > Hi Rotem and All,
> >
> > I am also looking for solutions to this. I hope somebody has an idea
> > of how to keep doing research on Twitter.
> >
> > Eleonora
> > *--*
> >
> > Dr. Maria Eleonora Sciubba
> > (she/her)
> > (2022): “Adesso m’incazzo!”: Swearwords as resources for managing
> > negative emotions in interaction
> > <https://mediazioni.unibo.it/article/view/15263>
> > (2021)
> > https://rolsi.net/2021/06/02/guest-blog-em-ca-for-racial-justice/
> >
> > *TSHD - Grant Design & Writing *
> > *Department of Culture Studies - Senior Researcher* Editor, Internship
> > Organizational Supervisor *Diggit Magazine*
> > <https://www.diggitmagazine.com/> Tilburg University
> > Twitter: @LolaSciubba
> >
> >
> >
> >
> >
> > *Be green. Keep it on the screen.*
> >
> >
> > Il giorno mar 18 apr 2023 alle ore 11:15 Rotem Perach via Air-L <
> > air-l at listserv.aoir.org> ha scritto:
> >
> > > Hi All,
> > >
> > > Can anyone recommend a way to download the feeds (historical tweets
> > > and
> > > re-tweets) of Twitter users? Is scraping for example using Data
> > > Miner a good idea?
> > > With the changes to Twitter's API rules, API-based websites such as
> > > Vicinitas.io have stopped working (or will stop soon), and I'm
> > > looking
> > for
> > > other approaches.
> > >
> > > Thank you for your help.
> > >
> > > Kind regards,
> > >
> > >
> > > Rotem
> > >
> > > -----------
> > >
> > > Dr Rotem Perach (he/him)
> > > Senior Research Fellow
> > > School of Social Sciences
> > > University of Westminster
> > >
> > > ORCID<https://orcid.org/0000-0002-8647-4367> | Twitter<
> > > https://twitter.com/DrRotemPerach/>
> > >
> > >
> > > The University of Westminster is a charity and a company limited by
> > > guarantee. Registration number: 977818 England. Registered Office:
> > > 309 Regent Street, London W1B 2HW.
> > >
> > > This message and its attachments are private and confidential. If
> > > you
> > have
> > > received this message in error, please notify the sender and remove
> > > it
> > and
> > > its attachments from your system.
> > > _______________________________________________
> > > The Air-L at listserv.aoir.org mailing list is provided by the
> > > Association of Internet Researchers http://aoir.org Subscribe,
> > > change options or unsubscribe at:
> > > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> > >
> > > Join the Association of Internet Researchers:
> > > http://www.aoir.org/
> > >
> > _______________________________________________
> > The Air-L at listserv.aoir.org mailing list is provided by the
> > Association of Internet Researchers http://aoir.org Subscribe, change
> > options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> >
> > Join the Association of Internet Researchers:
> > http://www.aoir.org/
> >
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list is provided by the Association
> of Internet Researchers http://aoir.org Subscribe, change options or
> unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>