ao3commentoftheday:

longlivefeedback:

ao3commentoftheday:

BIG, HUGE thanks to @rollychan for crunching the numbers and making the charts. This data is based on 4,639 respondents to the kudos poll.

 What do kudos mean?

Respondents were able to select all meanings that apply to them. 

According to people who self-identify as readers:

• 1872 mean “This fic was awesome! I loved it!”
• 966 mean “I recommend this story!”
• 1494 mean “Thank you so much for sharing your story!”
• 1238 mean “Good job”
• 103 mean “It was okay, but not good enough for me to comment”
• 571 mean “I finished the entire story”

According to people who self-identify as writers:

• 12 mean “This fic was awesome! I loved it!”
• 8 mean “I recommend this story!”
• 13 mean “Thank you so much for sharing your story!”
• 17 mean “Good job”
• 11 mean “It was okay, but not good enough for me to comment”
• 10 mean “I finished the entire story”

According to people who self-identify as both readers and writers:

• 922 mean “This fic was awesome! I loved it!”
• 496 mean “I recommend this story!”
• 790 mean “Thank you so much for sharing your story!”
• 724 mean “Good job”
• 155 mean “It was okay, but not good enough for me to comment”
• 324 mean “I finished the entire story”

The poll currently has 10,160 respondents. I am going to stop accepting responses at this point and link these results. I hope you find this as interesting as I do! I learned a thing!

These results are very interesting, and so I decided to run a couple of tests to determine whether these differences are statistically significant – that is, what is the probability that these numbers look like they’re indicating differences between readers and authors, but it’s actually due to random chance? 

To do this, I used the socsstatistics’ Chi Square calculator to perform a test for independence. Because there were so few people who identified themselves as only writers, I only looked at the “Readers” and the “Reader-Writers” numbers. 

For each option (this fic is awesome, good job, I finished the story, etc), I reformatted the answers into yes/no questions using the total number for each category, provided by @rollychan. 

As an example: These figures include a total of 4,369 respondents, and 2,073 people identified themselves as only readers. Out of these, 1,872 people selected “This fic was awesome, I loved it!” as an answer, and 201 did not. Thus, we have 1,872 “yes” responses and 201 “no” responses to the question “Do you use the kudos option to mean ‘this fic was awesome, I loved it’?” 

Because we have a large sample size, I’m setting the alpha threshold at p < 0.05. That is, there has to be less than a 5% probability that these results happened due to chance, and a 95% probability that these results mean there is an actual difference between the answer from Readers (2073 total) and Reader-Writers (2514 total). 

Results

• “This fic is awesome, I loved it!” – there is a difference (p < 0.01)
• “I recommend this story” – there is a difference (p < 0.01)
• “Thank you for sharing your story” – there is a difference (p < 0.01)
• “Good job” – there is a difference (p < 0.01)
• “It was okay, but not good enough for me to comment” – these data do not show a significant difference (p = 0.08) 
• “I finished this story” – there is a difference (p < 0.01)

So, interesting! There has been some discussion as to whether authors attribute less positive meanings to kudos than people who are only readers, and the answer to that appears to be a firm yes. 

However, a lot of focus has been put on the “it was okay, but not good enough for me to comment” meaning on kudos (which, I would like to point out that only around 5% of Readers and 6% of Reader-Writers selected “yes” as a response). Is there a significant difference in the responses of these two sets of kudos-givers? With these data, we find that the answer is no. The p value is quite close to the threshold, but nowhere near the results for the other questions. So, while it may be indicating a trend, it would be interesting to see if repeating this test with the full set of 10k+ answers would give us something different!

– Rose 

I love me some stats. And I love even more people who do the stats work for me 🙂 This is FANTASTIC. Thank you so much!