[Air-L] emotion detection machines - round 2

[Charles M. Ess]

Thanks, both -
excellent question / point, Annette.  Here are the research questions, 
slightly anonymized:

RQ1. Is there an emotion bias on [a major newspaper's] Facebook page?
a) What is the distributions of emotions in news postings on the 
newspaper's Facebook page?
b) What is the proportion of each emotion [of five emotions] in each 
news posting?
c) What is the proportion of news with different emotions?
d) What are the top two dominant emotions and their patterns of 
distribution in each news post?

RQ2. How do user’s emotions influence news’ sharing and diffusion? Or is 
there a relationship between user’s emotions and the scales of news’ 
sharing and diffusion?
a) What is the relationship between the dominant emotion in a news post 
and the numbers of engagement (the number of comments + the number of 
sharing + the number of emoticons)
b) How does the average number of engagement differ when different 
emotions come to the first position?

RQ3. Is there a relationship between the emotions of the news post on 
the newspaper's Facebook site and the emotions conveyed by commentators? 
Or What kind of emotion has a stronger agenda-setting effect on the 
newspaper's Facebook site?

So, RQs 2 and 3 drive the need for collecting the comments and analyzing 
their emotional contents vis-a-vis the emotional contents of a given 
news post.

The larger curiosity is to determine what emotions drive / discourage 
engagement.

Hope this helps - again, many thanks and looking forward to further 
comments and suggestions.

best,
- c.



On 25/10/2019 17:13, Annette Markham wrote:
> I agree that the issue is likely larger than simply the accuracy of Senpy.
> 
> Perhaps you already discussed this, but what is the research question in this case? Is it necessary to analyze the comments and if so, why is a total dataset analysis required? If we can better understand the goal of the analysis, we can better assess potential ways to accomplish it. In the previous discussion, the research design seemed rather method or even tool driven, which might not be the optimal way to frame the design, since the tool is flawed.
> 
> FWIW, I'd be happy to have this larger discussion,
> 
> annette
> 
> 
> On 10/25/19, 16:23, "Air-L on behalf of Necip Enes GENGEÇ" <air-l-bounces at listserv.aoir.org on behalf of necipenesgengec at gmail.com> wrote:
> 
>      Hi Charles,
>      
>      I think it's a complete research problem by itself that you are struggling
>      with. I didn't used Senpy before but it's a predictor at the end of the day
>      and it'll only be close to real conclusion. Incase you want to reduce the
>      manual work, I suggest running the analysis with multiple solutions eg.
>      with Meltwater, Senpy etc. and drive a conclusion for the manual check from
>      a combined result.
>      
>      Cheers,
>      Necip
>      
>      25 Eki 2019 Cum 16:14 tarihinde Charles M. Ess <c.m.ess at media.uio.no> şunu
>      yazdı:
>      
>      > Dear AoIRists,
>      >
>      > First of all, many thanks again to everyone who contributed suggestions
>      > and critical comments regarding emotion detection machines.
>      >
>      > My student settled on Senpy, despite important criticisms and
>      > limitations, and is now in a second phase of analysis.
>      >
>      > The student is collecting emotional analyses of both primary posts and
>      > comments.  Primary posts range around 800 - comment posts are ca. 7000.
>      > The student went through the primary posts, and found that Senpy misread
>      > a post ca. 20% of the time, e.g., Senpy's analysis of a post might be
>      > "sadness," but when read in context, the emotional response was clearly
>      > happiness.
>      >
>      > Query: what is there to do, if anything, with Senpy's analysis of the
>      > 7000+ comment posts?
>      > It is not possible for the student to do the same process of manually
>      > cleaning the data.
>      > So: does the student just take up the results as they are, assuming that
>      > there will likely be a 20% error rate and simply accept that as a limit
>      > to the method / analysis?
>      > Or: ???
>      > I can't think of a good way forward here (no surprise: my PhD was on
>      > Kant ...) - so I'm hoping very likely many AoIRists will have one or
>      > more good solutions or suggestions.
>      >
>      > Many thanks in advance, and all best,
>      > - charles
>      >
>      >
>      > --
>      > Professor in Media Studies
>      > Department of Media and Communication
>      > University of Oslo
>      > <http://www.hf.uio.no/imk/english/people/aca/charlees/index.html>
>      >
>      > Postboks 1093
>      > Blindern 0317
>      > Oslo, Norway
>      > c.m.ess at media.uio.no
>      > _______________________________________________
>      > The Air-L at listserv.aoir.org mailing list
>      > is provided by the Association of Internet Researchers http://aoir.org
>      > Subscribe, change options or unsubscribe at:
>      > http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>      >
>      > Join the Association of Internet Researchers:
>      > http://www.aoir.org/
>      >
>      _______________________________________________
>      The Air-L at listserv.aoir.org mailing list
>      is provided by the Association of Internet Researchers http://aoir.org
>      Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
>      
>      Join the Association of Internet Researchers:
>      http://www.aoir.org/
> 
> 

-- 
Professor in Media Studies
Department of Media and Communication
University of Oslo
<http://www.hf.uio.no/imk/english/people/aca/charlees/index.html>

Postboks 1093
Blindern 0317
Oslo, Norway
c.m.ess at media.uio.no