[Air-L] Downloading Twitter feeds

Sat Apr 22 10:50:50 PDT 2023

Hi Rose, I am very interested in how you solve that puzzle.

I have been working lately with WhatsApp groups that are publicly available
and anonymizing and performing aggregate analysis have gotten me through
ethics approval. If you are going to perform more qualitative research it
might be very advisable that users are aware of it and consent, which might
prove very complicated.

Let me know how this moves forward.

Best wishes
Marcelo

Em sex., 21 de abr. de 2023 às 09:32, Butler, Rose via Air-L <
air-l at listserv.aoir.org> escreveu:

> Hello
> I have an ethical question about scraping data from FB (that is already in
> the public domain)
>
> I am preparing a project for ethical clearance. The research focuses on
> two community FB groups of 3000 and 7000 members respectively and of which
> I am a member of both.
> As part of the research methods I will automate the collection of data for
> analysis (partly on spreadsheets), all of this would be possible manually
> if I had the time – automation allows me to collect a lot of data quickly –
> it is not extending access.
>
> If this data is anonymised and not published would that be enough to
> protect the privacy of members within the group?
> I wondered if anyone had experience of doing this and what additional
> safeguards they needed to implement for ethical clearance. The problem here
> is that a closed group is semi-private, individuals may feel like it is
> private even if it spans the whole community (this time a small town).
>
> Thanks
>
>
>
> From: Air-L <air-l-bounces at listserv.aoir.org> on behalf of Stuart Shulman
> via Air-L <air-l at listserv.aoir.org>
> Date: Tuesday, 18 April 2023 at 12:10
> To: M.E.Sciubba <mesciubba at gmail.com>
> Cc: air-l at listserv.aoir.org <air-l at listserv.aoir.org>
> Subject: Re: [Air-L] Downloading Twitter feeds
> CAUTION: This message was sent from outside the University, purportedly
> from air-l-bounces at listserv.aoir.org .
>
> Please check the sender is legitimate before responding. Please treat any
> links or attachments with care -  do not follow or open them unless you are
> sure they are genuine.
>
>
>
>
> The answer is complex, uncertain, and shifting. It is more than just a
> technical issue of "Where can I get data now?" There are fundamental
> legal, jurisdictional, market, equity, and political questions
> unresolved. Resolution seems unlikely in the short term. As a
> political scientist interested in election interference, I do not
> anticipate a positive resolution before the 2024 U.S. election. I
> would love to be wrong on that prediction. I have heard there are a
> few firms that retain agreements to market real time and historical
> Twitter data. You might ask your librarian about access to Crimson
> Hexagon, Meltwater, or other similar services with long term
> agreements Mr. Musk has not broken, yet. Nobody on the AIR-L list has
> confirmed whether the academic API is shut down. Is it? I don't have
> those credentials to check.
>
> As I have written previously, there are massive troves of Tweets in
> every computer science department and many social science departments.
> Any one of the raw JSON Tweet archives could be loaded into a free
> account on DiscoverText and then shared for teaching or research. Talk
> to the data folks on your campus and across your discipline about what
> is extant. Do not look for spreadsheets; find the raw JSON.
> Spreadsheets are significantly degraded historical objects that cannot
> be considered accurate representations of Tweets. Tweets live in the
> Twitter display and die in spreadsheets. I remain befuddled by the
> Python/R nexus. Those spreadsheets of Tweets simply are not Twitter.
> Any qualitative researcher knows what I mean. Meaning is produced in
> the interaction of the display elements. Meaning is destroyed,
> diluted, and bent out of shape in spreadsheets.
>
> The loss of real time access to new data is a problem and dangerous
> for democratic systems. This does not change the fact that so much
> data has been gathered and stored, like so many old newspapers. For
> example, here is a list of the Top 50 datasets in my DiscoverText
> account that any academic on this list or anywhere in the world can
> access via the "peer network" that enables collaboration, annotation,
> measurement of inter-rater reliability, and the creation of gold
> standard training sets for machine-learning via a graphical user
> interface that features the Twitter display.
>
>
> https://docs.google.com/spreadsheets/d/1j-Y40WmwFIX8pidfAcxgAt5JB3A3UyB2psh_ULQdynA/edit?usp=sharing
>
> "The Case Against Spreadsheets as a Primary Twitter Research Tool"
> https://vimeo.com/526218014
> Question: When is a Tweet not a Tweet? Answer: When it appears in a
> spreadsheet, or is deleted, or the account has been suspended.
>
>
>
> On Tue, Apr 18, 2023 at 5:57 AM M.E.Sciubba via Air-L <
> air-l at listserv.aoir.org> wrote:
>
> > Hi Rotem and All,
> >
> > I am also looking for solutions to this. I hope somebody has an idea of
> how
> > to keep doing research on Twitter.
> >
> > Eleonora
> > *--*
> >
> > Dr. Maria Eleonora Sciubba
> > (she/her)
> > (2022): “Adesso m’incazzo!”: Swearwords as resources for managing
> negative
> > emotions in interaction <https://mediazioni.unibo.it/article/view/15263>
> > (2021) https://rolsi.net/2021/06/02/guest-blog-em-ca-for-racial-justice/
> >
> > *TSHD - Grant Design & Writing *
> > *Department of Culture Studies - Senior Researcher*
> > Editor, Internship Organizational Supervisor *Diggit Magazine*
> > <https://www.diggitmagazine.com/>
> > Tilburg University
> > Twitter: @LolaSciubba
> >
> >
> >
> >
> >
> > *Be green. Keep it on the screen.*
> >
> >
> > Il giorno mar 18 apr 2023 alle ore 11:15 Rotem Perach via Air-L <
> > air-l at listserv.aoir.org> ha scritto:
> >
> > > Hi All,
> > >
> > > Can anyone recommend a way to download the feeds (historical tweets and
> > > re-tweets) of Twitter users? Is scraping for example using Data Miner a
> > > good idea?
> > > With the changes to Twitter's API rules, API-based websites such as
> > > Vicinitas.io have stopped working (or will stop soon), and I'm looking
> > for
> > > other approaches.
> > >
> > > Thank you for your help.
> > >
> > > Kind regards,
> > >
> > >
> > > Rotem
> > >
> > > -----------
> > >
> > > Dr Rotem Perach (he/him)
> > > Senior Research Fellow
> > > School of Social Sciences
> > > University of Westminster
> > >
> > > ORCID<https://orcid.org/0000-0002-8647-4367> | Twitter<
> > > https://twitter.com/DrRotemPerach/><
> https://twitter.com/DrRotemPerach/%3e>
> > >
> > >
> > > The University of Westminster is a charity and a company limited by
> > > guarantee. Registration number: 977818 England. Registered Office: 309
> > > Regent Street, London W1B 2HW.
> > >
> > > This message and its attachments are private and confidential. If you
> > have
> > > received this message in error, please notify the sender and remove it
> > and
> > > its attachments from your system.
> > > _______________________________________________
> > > The Air-L at listserv.aoir.org mailing list
> > > is provided by the Association of Internet Researchers http://aoir.org
> > > Subscribe, change options or unsubscribe at:
> > > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> > >
> > > Join the Association of Internet Researchers:
> > > http://www.aoir.org/
> > >
> > _______________________________________________
> > The Air-L at listserv.aoir.org mailing list
> > is provided by the Association of Internet Researchers http://aoir.org
> > Subscribe, change options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> >
> > Join the Association of Internet Researchers:
> > http://www.aoir.org/
> >
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>