[Air-L] Tool for collecting Instagram images/websites/data?

Mon Sep 19 00:10:16 PDT 2016

Hi Rainer,

When collecting data for my Harkive project (which collects on the #harkive hashtag across various platforms, including Instagram, Twitter, etc), I recently started using Zapier, which I found solved a lot of time-consuming issues. 

The main advantage for my purposes was that it significantly reduced data cleaning/organising post collection. As I’m sure you are aware, the API of each service uses slightly different terms for common elements of the data available. EG: On Twitter the text content of a Tweet is contained with the <text> element, whereas on Tumblr the text written by a user is in the <body> element. Not only that, but date/time stamps often appear in different formats. What Zapier allowed me to do was collect from the various APIs, reformat to common data formats, and then write all data to specific columns in a central GoogleDocs Spreadsheet - so, all commonly formatted date/time stamps appeared in a single column, all usernames in another, and so on, regardless of the service they originated from. I was also able to create a new entry in each row that labelled each entry as originating from Twitter, Tumblr, Instagram, etc. This was all in an attempt to have my data rendered as ‘Tidy’, in the Hadley Wickham sense, automatically. 

The disadvantage of Zapier is that there is a charge. I was collecting over a short period of time so I was able to keep this cost quite low. If you are collecting a lot of data over a sustained period of time, you may find it prohibitively expensive. 

Let me know if you would like to know more - I am happy to share my rough workflow notes with you, or can show you in Berlin during the conference.

Kind regards
Craig

> On 18 Sep 2016, at 10:24, Maurice Vergeer <m.vergeer at maw.ru.nl> wrote:
> 
> maybe the R package instaR:
> https://github.com/pablobarbera/instaR/blob/master/examples.R
> For longterm applications you could learn web scraping with R package rvest
> (there ar more options for web scraping)
> The quick-and-dirty method is search a tag on Instagram with a browser,
> scroll down multiple times to load as much pictures as you need, then save
> the resulting Instagram page in a new specific folder, which then contains
> all pictures with that  specific tag.
> 
> hope that helps
> 
> 
> 
> On Sat, Sep 17, 2016 at 5:27 PM, Rainer Hillrichs <hillrichs at uni-mannheim.de
>> wrote:
> 
>> Dear all,
>> 
>> I searched on the list and on the web but couldn't find anything: I'm
>> looging for a tool that collects Instagram images, websites, and data
>> associated with a specific tag. Basically, I want to type in a tag and end
>> up with a folder full of images, websites, and a table with data (e.g. user
>> name, date posted, URL, other tags). I already suspect that is a lot to ask
>> for ;-) Even a simpler tool would be a good start! As long as I don't have
>> to to end up saving individual images, websites, and typiing/copying stuff
>> into a table.
>> 
>> Suggestions very much appreciated!
>> Rainer
>> 
>> 
>> --
>> Dr. Rainer Hillrichs
>> Universität Mannheim
>> https://uni-mannheim.academia.edu/RainerHillrichs
>> _______________________________________________
>> The Air-L at listserv.aoir.org mailing list
>> is provided by the Association of Internet Researchers http://aoir.org
>> Subscribe, change options or unsubscribe at: http://listserv.aoir.org/
>> listinfo.cgi/air-l-aoir.org
>> 
>> Join the Association of Internet Researchers:
>> http://www.aoir.org/
> 
> 
> 
> 
> -- 
> ________________________________________________
> Maurice Vergeer
> To contact me, see http://mauricevergeer.nl/node/5
> To see my publications, see http://mauricevergeer.nl/node/1
> ________________________________________________
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
> 
> Join the Association of Internet Researchers:
> http://www.aoir.org/