[Air-L] Submitting R/Python scraper to Facebook App Review

[Stuart Shulman]

All,

We had all but given up on the Facebook API, even though when we connected
what became DiscoverText back in 2009, it was a very robust source of
diverse post & comment data, including via free text search. Over time, it
got to the point where even many public or formerly public pages were
delivering no data at all over the API. Many goose eggs. Post 2014 API
changes, we just assumed it was more or less useless for research.

However, I have a few odd facts to report. Roughly 10% of all data flowing
into our platform still comes from Facebook. I have been scratching my head
about this for a while, since I can barely imagine which pages are giving
up this data and we have done no work to find out what the story is. Mostly
we tell people Twitter wants to be studied and Facebook does not.

Second, based on a client request, I just tried yesterday to get the
comments from the Mary Kay Facebook page, anticipating few or none. Much to
my surprise, we are at 193,000+ items and counting. I have not personally
had this kind of success getting FB data since before the 2014 changes. Of
course, I have rarely even tried it, because of all the searches and pages
that return no data. This now has me wondering more about the specific role
of the Page Owners and their settings.

I'm not an engineer. However, there are clearly page-specific factors that
are impacting the ability of folks to access Facebook data for research
purposes, irrespective of whether you are using R, Python, or a commercial
application. Researchers may need to actively work with Facebook page
owners to get better data access. Another idea is to send a delegation to
Facebook to talk about this. I know there is something happening under the
auspices of Gary King's work, but I have not seen the Call to Action yet.

A decade ago you could get a meeting at Facebook. Is that still true? I'd
like to be a fly on that wall.

~Stu

Stu ShulmanMA Olympic Development Program (ODP), Assistant Coach
NEFC-West 2008 Boys, Head Coach





On Tue, Jul 24, 2018 at 10:07 PM Astvansh, Vivek <astvansh at iu.edu> wrote:

> Justin:
>
> Thank you for asking.
> I am in a similar boat; my RA is trying to write a new FB scraper. I
> submitted my application last evening and it was a pain. Going by your
> experience, it is likely that FB will decline my application as well. I
> will let you my fortune favors me. If you are able to get your R scraper to
> work, please consider emailing me.
>
> Good luck!
>
> -----Original Message-----
> From: Air-L <air-l-bounces at listserv.aoir.org> On Behalf Of Justin Ho
> Sent: Tuesday, July 24, 2018 4:04 PM
> To: air-l at listserv.aoir.org
> Subject: [Air-L] Submitting R/Python scraper to Facebook App Review
>
> Hi everyone,
>
> I am a PhD student and I have to collect Facebook data from time to time
> for my PhD project, I have been using R and it worked just fine. However,
> after the recent changes in Facebook policy, personal accounts can no
> longer generate short-lived token, to obtain an access token, we must first
> create an App, and to enable the App to scrape Facebook data, we must
> submit an App Review for "Page Public Content Access". I tried submitting
> my R codes and a screencast of me using the codes, but it was rejected on
> the basis that:
>
> Your screencast doesn't show how the use of this permission directly
> > improves the user experience in your app. Unfortunately, we also
> > weren't able to determine this from testing your app manually.
>
> All permissions data must be visibly used within your app. We do not accept
> > permission requests for data that you may decide to use later.
>
>
> From above feedback, it seems the page public content access is only
> available for built-in Facebook App (like NetVizz), instead of standalone
> R/Python programs. I am wondering if anyone has successfully passed the
> review? Could you share your experience of how to do it?
>
> Thanks a lot!
>
> Best regards,
> Justin
> _______________________________________________
> The Air-L at listserv.aoir.org mailing list
> is provided by the Association of Internet Researchers http://aoir.org
> Subscribe, change options or unsubscribe at:
> http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>
> Join the Association of Internet Researchers:
> http://www.aoir.org/
>