[Air-L] Downloading Twitter feeds

Sun Apr 23 06:49:58 PDT 2023

Good point Floriana, but again: are a private company's terms of service a
legal boundary, sufficient to prevent researchers to deal with issues of
public interest. If you ask me, no. But I know  many IRBs follow the terms
of service, which is a very lazy shortcut, IMHO, to solve complex
ethical issues.

I would very much like to know what others think, though.

Best
Marcelo

Em sáb., 22 de abr. de 2023 às 14:11, Grasso, Floriana <
floriana at liverpool.ac.uk> escreveu:

> Hi Rose
>
> I suspect you will not be able to collect *any* data from Facebook, no
> matter what ethic approval you have, as their terms of service
> <https://www.facebook.com/legal/terms> say (in 3.2.3):
>
> You may not access or collect data from our Products using automated means
> (without our prior permission) or attempt to access data that you do not
> have permission to access. We also reserve all of our rights against text
> and data mining.
>
>
> which basically says you cannot scrape content, but I would be interested
> to know if the legal team at your institution thinks otherwise?
>
> Floriana
>
> ------------------------------
> *From:* Air-L <air-l-bounces at listserv.aoir.org> on behalf of Marcelo
> Santos via Air-L <air-l at listserv.aoir.org>
> *Sent:* 22 April 2023 18:50
> *To:* Butler, Rose <R.Butler at shu.ac.uk>
> *Cc:* air-l at listserv.aoir.org <air-l at listserv.aoir.org>
> *Subject:* Re: [Air-L] Downloading Twitter feeds
>
> Hi Rose, I am very interested in how you solve that puzzle.
>
> I have been working lately with WhatsApp groups that are publicly available
> and anonymizing and performing aggregate analysis have gotten me through
> ethics approval. If you are going to perform more qualitative research it
> might be very advisable that users are aware of it and consent, which might
> prove very complicated.
>
> Let me know how this moves forward.
>
> Best wishes
> Marcelo
>
> Em sex., 21 de abr. de 2023 às 09:32, Butler, Rose via Air-L <
> air-l at listserv.aoir.org> escreveu:
>
> > Hello
> > I have an ethical question about scraping data from FB (that is already
> in
> > the public domain)
> >
> > I am preparing a project for ethical clearance. The research focuses on
> > two community FB groups of 3000 and 7000 members respectively and of
> which
> > I am a member of both.
> > As part of the research methods I will automate the collection of data
> for
> > analysis (partly on spreadsheets), all of this would be possible manually
> > if I had the time – automation allows me to collect a lot of data
> quickly –
> > it is not extending access.
> >
> > If this data is anonymised and not published would that be enough to
> > protect the privacy of members within the group?
> > I wondered if anyone had experience of doing this and what additional
> > safeguards they needed to implement for ethical clearance. The problem
> here
> > is that a closed group is semi-private, individuals may feel like it is
> > private even if it spans the whole community (this time a small town).
> >
> > Thanks
> >
> >
> >
> > From: Air-L <air-l-bounces at listserv.aoir.org> on behalf of Stuart
> Shulman
> > via Air-L <air-l at listserv.aoir.org>
> > Date: Tuesday, 18 April 2023 at 12:10
> > To: M.E.Sciubba <mesciubba at gmail.com>
> > Cc: air-l at listserv.aoir.org <air-l at listserv.aoir.org>
> > Subject: Re: [Air-L] Downloading Twitter feeds
> > CAUTION: This message was sent from outside the University, purportedly
> > from air-l-bounces at listserv.aoir.org .
> >
> > Please check the sender is legitimate before responding. Please treat any
> > links or attachments with care -  do not follow or open them unless you
> are
> > sure they are genuine.
> >
> >
> >
> >
> > The answer is complex, uncertain, and shifting. It is more than just a
> > technical issue of "Where can I get data now?" There are fundamental
> > legal, jurisdictional, market, equity, and political questions
> > unresolved. Resolution seems unlikely in the short term. As a
> > political scientist interested in election interference, I do not
> > anticipate a positive resolution before the 2024 U.S. election. I
> > would love to be wrong on that prediction. I have heard there are a
> > few firms that retain agreements to market real time and historical
> > Twitter data. You might ask your librarian about access to Crimson
> > Hexagon, Meltwater, or other similar services with long term
> > agreements Mr. Musk has not broken, yet. Nobody on the AIR-L list has
> > confirmed whether the academic API is shut down. Is it? I don't have
> > those credentials to check.
> >
> > As I have written previously, there are massive troves of Tweets in
> > every computer science department and many social science departments.
> > Any one of the raw JSON Tweet archives could be loaded into a free
> > account on DiscoverText and then shared for teaching or research. Talk
> > to the data folks on your campus and across your discipline about what
> > is extant. Do not look for spreadsheets; find the raw JSON.
> > Spreadsheets are significantly degraded historical objects that cannot
> > be considered accurate representations of Tweets. Tweets live in the
> > Twitter display and die in spreadsheets. I remain befuddled by the
> > Python/R nexus. Those spreadsheets of Tweets simply are not Twitter.
> > Any qualitative researcher knows what I mean. Meaning is produced in
> > the interaction of the display elements. Meaning is destroyed,
> > diluted, and bent out of shape in spreadsheets.
> >
> > The loss of real time access to new data is a problem and dangerous
> > for democratic systems. This does not change the fact that so much
> > data has been gathered and stored, like so many old newspapers. For
> > example, here is a list of the Top 50 datasets in my DiscoverText
> > account that any academic on this list or anywhere in the world can
> > access via the "peer network" that enables collaboration, annotation,
> > measurement of inter-rater reliability, and the creation of gold
> > standard training sets for machine-learning via a graphical user
> > interface that features the Twitter display.
> >
> >
> >
> https://docs.google.com/spreadsheets/d/1j-Y40WmwFIX8pidfAcxgAt5JB3A3UyB2psh_ULQdynA/edit?usp=sharing
> >
> > "The Case Against Spreadsheets as a Primary Twitter Research Tool"
> > https://vimeo.com/526218014
> > Question: When is a Tweet not a Tweet? Answer: When it appears in a
> > spreadsheet, or is deleted, or the account has been suspended.
> >
> >
> >
> > On Tue, Apr 18, 2023 at 5:57 AM M.E.Sciubba via Air-L <
> > air-l at listserv.aoir.org> wrote:
> >
> > > Hi Rotem and All,
> > >
> > > I am also looking for solutions to this. I hope somebody has an idea of
> > how
> > > to keep doing research on Twitter.
> > >
> > > Eleonora
> > > *--*
> > >
> > > Dr. Maria Eleonora Sciubba
> > > (she/her)
> > > (2022): “Adesso m’incazzo!”: Swearwords as resources for managing
> > negative
> > > emotions in interaction <
> https://mediazioni.unibo.it/article/view/15263>
> > > (2021)
> https://rolsi.net/2021/06/02/guest-blog-em-ca-for-racial-justice/
> > >
> > > *TSHD - Grant Design & Writing *
> > > *Department of Culture Studies - Senior Researcher*
> > > Editor, Internship Organizational Supervisor *Diggit Magazine*
> > > <https://www.diggitmagazine.com/>
> > > Tilburg University
> > > Twitter: @LolaSciubba
> > >
> > >
> > >
> > >
> > >
> > > *Be green. Keep it on the screen.*
> > >
> > >
> > > Il giorno mar 18 apr 2023 alle ore 11:15 Rotem Perach via Air-L <
> > > air-l at listserv.aoir.org> ha scritto:
> > >
> > > > Hi All,
> > > >
> > > > Can anyone recommend a way to download the feeds (historical tweets
> and
> > > > re-tweets) of Twitter users? Is scraping for example using Data
> Miner a
> > > > good idea?
> > > > With the changes to Twitter's API rules, API-based websites such as
> > > > Vicinitas.io have stopped working (or will stop soon), and I'm
> looking
> > > for
> > > > other approaches.
> > > >
> > > > Thank you for your help.
> > > >
> > > > Kind regards,
> > > >
> > > >
> > > > Rotem
> > > >
> > > > -----------
> > > >
> > > > Dr Rotem Perach (he/him)
> > > > Senior Research Fellow
> > > > School of Social Sciences
> > > > University of Westminster
> > > >
> > > > ORCID<https://orcid.org/0000-0002-8647-4367> | Twitter<
> > > > https://twitter.com/DrRotemPerach/><
> > https://twitter.com/DrRotemPerach/%3e>
> <https://twitter.com/DrRotemPerach/%3e%3E>
> > > >
> > > >
> > > > The University of Westminster is a charity and a company limited by
> > > > guarantee. Registration number: 977818 England. Registered Office:
> 309
> > > > Regent Street, London W1B 2HW.
> > > >
> > > > This message and its attachments are private and confidential. If you
> > > have
> > > > received this message in error, please notify the sender and remove
> it
> > > and
> > > > its attachments from your system.
> > > > _______________________________________________
> > > > The Air-L at listserv.aoir.org mailing list
> > > > is provided by the Association of Internet Researchers
> http://aoir.org
> > > > Subscribe, change options or unsubscribe at:
> > > > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> > > >
> > > > Join the Association of Internet Researchers:
> > > > http://www.aoir.org/
> > > >
> > > _______________________________________________
> > > The Air-L at listserv.aoir.org mailing list
> > > is provided by the Association of Internet Researchers http://aoir.org
> > > Subscribe, change options or unsubscribe at:
> > > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> > >
> > > Join the Association of Internet Researchers:
> > > http://www.aoir.org/
> > >
> > _______________________________________________
> > The Air-L at listserv.aoir.org mailing list
> > is provided by the Association of Internet Researchers http://aoir.org
> > Subscribe, change options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> >
> > Join the Association of Internet Researchers:
> > http://www.aoir.org/
> > _______________________________________________
> > The Air-L at listserv.aoir.org mailing list
> > is provided by the Association of Internet Researchers http://aoir.org
> > Subscribe, change options or unsubscribe at:
> > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> >
> > Join the Association of Internet Researchers:
> > http://www.aoir.org/
> >
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>