So how do you measure ROI of online communities?

Tweet

The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success.

The workshops that I’ve run with clients at FreshNetworks only prove that every online community has a different set of aims and so needs to be measured in different ways. However, a few basic principals exist that show the possible ways in which you can measure ROI.

First of all it’s useful to distinguish between actual measures of ROI and measures of the health of the community. As people who build and manage online communities, at FreshNetworks we are interested in metrics such as number of people who log-on each day, proportion who make contributions, average amount of time spent on the community and other such measures. These help us to benchmark the community against others that we have run and understand how each community is developing. It’s also useful for us to be able to build a set of metrics across community types so that we can better understand consumer behaviour in them. In most cases however, such measures are not (just) what we use to measure ROI.

A client may want to own the debate in a certain area. It may want to work with its customers to improve or develop its product in an online research community. It may be looking to engage a new audience demographic. It may want to amplify the word of mouth for its services. There are many reasons why people may set up online communities and understanding these on a very granular level is the first stage to effective measurement of ROI.

Then it’s just a matter of work out which factors will help you to understand how you are performing against these aims. There are really two types of data you can measure in a community

  1. Qualitative data - this could mean measuring the quality of conversations about the brand, the response to the brand and competitors, the increase in actual conversations and dialogue about the brand, the extent to which these conversations are positive, the quality of ideas generated. Whatever you’re trying to achieve there will be qualitative measures that help you to measure this. Of course the problem with qualitative data is exactly that - it’s qualitative. Although this is the nature of online communities - they’re a space for conversations and discussions. So we often find that establishing detailed qualitative measures of success can be the most revealing and the most informative.
  2. Quantitative data - to some extent this is much easier to measure, the trick is measuring the right thing and not measuring too much. From Google Analytics to more bespoke packages there are ways of quantifying a member’s interactions with the site and building a set of measures from this. It may be about reaching more people (possibly of a certain type), or about engaging the same people for longer. Quantitative measures can be good for understanding how people interact with the site.

This is really the fundamental difference between these two types of data and these two ways of measuring. Qualitative data helps us to understand the quality and usefulness of the contributions to the online community. Quantitative data, on the other hand, shows us how people have interacted with the site.

Most are important to measure and different clients have different aims from their online community - this means that different baskets of measures are needed. Sometimes you’re more interested in the quality of debate and so want to include a greater number of qualitative measures; sometimes you’re more interested in the reach and penetration of the community and so want to include more quantitative measures.

Whatever you measure it is that first question that’s important: what are you trying to acheive with this online community; how will it help your business? Answering that is where it all begins and one of the most critical stages in planning a successful online community strategy.

  • Social Media Measurement Camp August 08
  • Where Does Social Media Sit in a Firm? Probably Many Places
  • Great Article on Measuring Social Media ROI
  • Social Media + Measurement + ROI = it’s all about CRM
  • Measuring the Value of Your Blog: Reflections Over the Last Year
  • Communities = more than the sum of their social media parts
  • ROI for Social Media Engagement: B-B Lead Generation
  • How to Measure Social Media ROI for Business
  • Building your Community Strategy
  • Online Media Measurement Framework
  •  
  •  
  •  
  •  
  •  


For social media agency support get in touch or follow us on Twitter.

14 Comments

  1. Bill Johnston, Forum One Networks:

    Great post Matt -

    I published highlights from our Online Community ROI and Reporting study on the Online Community Report blog.
    http://www.onlinecommunityreport.com/archives/363-Online-Community-ROI-Models-and-Reporting.html

    This has a good list with examples of the types of qualitative and quantitative data that community managers are tracking and reporting on.

    Bill J

  2. links for 2008-08-06 [delicious.com] | Daan Jansonius:

    [...] FreshNetworks Blog » Blog Archive » So how do you measure ROI of online communities? (tags: roi metrics measurement marketing communities community) [...]

  3. Simon:

    Hi Matt - very interesting topic, and one without a simple answer. Obviously, ROI metrics depend on objectives (whether purchase, intent, involvement or awareness) but I was wondering whether you have ever tried quantifying resultant chatter in terms of PR value? In your opinion, would this be useful or misleading?
    Best
    Simon

  4. FreshNetworks Blog » Blog Archive » KPIs and Metrics - more on online community measurement:

    [...] FreshNetworks « So how do you measure ROI of online communities? [...]

  5. Matt Rhodes:

    Bill,

    Thanks for the comment and the pointer to the report - a really useful source of info on what others are doing!

    Matt

  6. Matt Rhodes:

    Simon,

    Good question. There’s probably a few things here.

    For a start there is the direct PR that results from the community. We’ve done a bit of work on this and are actually running a community at the moment where some measures of success are going to be very much about generating PR on the issues (probably more importantly than the brand). Measuring this can be easy (you know what PR you have actively driven and can often tell from the coverage itself what comes from the online community). Or you might just decide that the aim is to increase the total volume of discussions and coverage on an issue - a less hard measure and one that’s less easy to tie directly to the community, although you could infer the influence.

    Then there is probably the PR value of the conversations themselves. There are a number of attempts to give a value in PR terms of the conversations on online communities. These usually work on the basis of media buy value, mainly as this is a figure most Marketing Executives are comfortable with. It’s true that online comments are more ‘trusted’ than print for some people and the reach can often be better (ie more specific and tailored) and so I do like some of these attempts. We’ve been working on various measures at FreshNetworks and are currently testing these. When we have any more concrete thoughts on this I’ll post them here.

    Matt

  7. Links do Dia: 08.08.08 « Dissonância Cognitiva:

    [...] So how do you measure ROI of online communities? - FreshNetworks Blog The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success. [...]

  8. Simon:

    Thanks for the answer Matt - I look forward to the findings

  9. FreshNetworks Blog » Blog Archive » Building the Web 2.0 enterprise:

    [...] on this blog about measurement and ROI in online communities and in social media (see posts here, here and here) and it seems that this is the biggest barrier that firms need support with. Perhaps as [...]

  10. LAB75 » Blog Archive » FreshNetworks: So How Do You Measure ROI of Online Communities? Turn Active Sharing: about Active Sharing wolleffm (0) My Videos Favorites Subscriptions Contacts more… Account ▼ QuickList (0) Help Sign Out Worldw:

    [...] The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success. (more) [...]

  11. FreshNetworks Blog » Blog Archive » Does Google have the answer to measuring ROI in social media?:

    [...] So How Do You Measure ROI of Online Communities? [...]

  12. Is it possible to measure ROI in social media?:

    [...] Matt Rhodes wrote two interesting posts about the topic that made me think about this one. So how do you measure ROI of online communities? Does Google have the answer to measuring ROI in social [...]

  13. FreshNetworks Blog » Blog Archive » Five things that probably won’t happen in social media in 2009:

    [...] spoken a lot in 2008 about measuring ROI in online communities and across social media. It’s something that has been discussed at every conference [...]

  14. Bookmarks for December 21st through December 28th | DavePress:

    [...] FreshNetworks Blog » Blog Archive » So how do you measure ROI of online communities? - "The answer to this question is simple and complex. The simple answer is that you make sure you establish a set of criteria when you launch the community and then judge success against these. The complex answer come with some of the ways in which you may judge success." [...]