May 25, 2010
PEJ New Media study: Good social media research, questionable claims on blogs and news
This week, Pew’s Project on Excellence in Journalism published a new study: New Media, Old Media: How blogs and social media agendas relate and differ from the traditional press. This intriguing study covers several bases, including a comparison of which types of news stories get shared most across blogs and social media (particularly YouTube and Twitter). The report’s second paragraph illustrates the key strengths and weaknesses of this research effort: “While most original reporting still comes from traditional journalists, technology makes it increasingly possible for the actions of citizens to influence a story’s total impact.”
If you’re reading, citing, or reporting on this study, it helps to understand some key context about how this research was done, and what kind of news it does and does not cover…
By Amy Gahran
PEJ’s research into how citizen actions can influence a news story’s impact looks pretty solid, and is quite interesting. Some highlights:
- Peer sharing is a key primary news source. “44% of online news users get news at least a few times a week through e-mails, automatic updates or posts from social networking sites.”
- People share different types of news through different channels. “Of the 29 weeks that we tracked all three social platforms, blogs, Twitter and YouTube shared the same top story just once. That was the week of June 15-19, 2009, when the protests that followed the Iranian elections led on all three.”
- Social media attention is fleeting. “On blogs, 53% of the lead stories in a given week stay on the list no more than three days. On Twitter that is true of 72% of lead stories, and more than half (52%) are on the list for just 24 hours.
Blogs and news: PEJ’s questionable data sources.
The “Blogosphere” section of the PEJ report includes this assertion:
“Despite the unconventional agenda of bloggers, traditional media still provides the vast majority of their information. More than 99% of the stories linked to came from legacy outlets like newspapers and broadcast networks. American legacy outlets made up 75% of all items. Web-only sites, on the other hand, made up less than 1% of the links in the blogosphere.” (Emphasis added.)
This finding has led to some headlines such as Blogs depend on traditional media (The Australian), New Media Loves to Link to Old Media—Almost Exclusively (TheWrap)
This made me wonder: Which blogs was PEJ checking for this study? The report’s methodology section explains:
”...To study new and social media, PEJ wanted to be able to include as wide a range of outlets as possible. For unlike the traditional press, blogs and social media pages reach into the millions and change daily as new ones emerge and other dissolve. In exploring various options, we saw value combining the work of some sites that specialize in tracking these outlets continuously with our own coding scheme and analytics.“Two prominent Web tracking sites, Technorati and Icerocket, monitor millions of blogs and pieces of social media, using the links to articles embedded on these sites as a proxy for determining what these subjects are. The website Tweetmeme uses a similar method to monitor the popular links on the social networking site Twitter.
“Each of these sites offers lists of the most linked-to news stories based on the number of blogs, tweets, or other pages that link to them. PEJ does not determine what constitutes a ‘news’ story (as opposed to some other topic), but rather relies on the classifications used by each of the tracking sites.
“A PEJ staff member manually captured the lists from each site every weekday between 9 and 10 am ET. From those lists, the top five linked to articles were captured for further analysis by PEJ staff.
“Through July 3, 2009, PEJ captured information about blogs from both Technorati and Icerocket. However, the relevant component of Technorati’s site stopped working in early July and has been down ever since. Therefore, the 26 NMI reports beginning the week of July 6-10 only included blog data from Icerocket.”
A close look at PEJ’s chosen resources for blog data reveal some significant potential flaws.
Technorati’s “What’s Popular” section is no longer available on that blog aggregator’s site. The most recent version of that page available from the Internet Archive shows that the “What’s Popular” section once featured “news stories people are talking about right now, ordered by new links to news sites in the last 48 hours.” Those references to “news stories” and “news sites,” and the nature of the links listed there, indicate that in this section Technorati focused specifically on blog posts that discussed or expanded upon content produced by mainstream news.
Similarly, Icerocket says its Top News Stories section showcases “Top stories posted in the blogosphere, measured by new links to Official News Sources in the last 48 hours.” In other words, this is a list of links to stories published by mainstream news outlets that are getting the most links from recent blog posts.
Icerocket’s Top News Stories differs from its Top Blog Posts section. Top Blog Posts lists “Top stories being discussed in the blogosphere right now.” When you scan this list of links, you’ll see some posts from mainstream news organizations (like Boston.com and the Wall St. Journal’s All Things D tech news site). But you’ll also see posts from sites like Google’s Adsense blog, or The Anchoress blog on First Things (a site published by the Institute on Religion and Public Life), or the Clean Techies blog.
If PEJ’s goal was to gauge the prevalence of original news reporting in the blogosphere, it might have done better to include Icerocket’s Top Blog Posts in its research base, rather than Top News Stories. Even better, they might have gathered data from Google Blogs.
PEJ apparently chose to count blog links coming mainly from a preselected portion of the blogosphere that focuses mainly on what mainstream news orgs are talking about. Given that context, it’s not surprising that they found that 99% of the outbound links from those blogs led to traditional news stories.
...But it’s probably a stretch for PEJ to make the blanket claim, based on this data, that bloggers (in general, not just those blogs in particular) rely on traditional media for the vast majority of their information. Or that “most original reporting still comes from traditional journalists.”
Both of those assertions may or may not be true; PEJ’s research simply is not sufficient to support them.
The Disconnect: What is “news”?
I suspect that part of the problem here is terminology and convention, rather than substance.
We’ve all been immersed in a culture where, for more than a century, when most people said “the news” they really meant “content produced by news organizations.” That’s an easily discernible category—but it’s circular reasoning to basically say, “news is whatever news organizations do.”
For instance, Schneier on Security (an independent blog by globally recognized security expert Bruce Schneier), frequently features quite significant original news and analysis. But it’s not generally called a “news blog.”
Resources such as Icerocket’s Top News Stories which focus on the ripple effects of mainstream media are completely appropriate if that’s what you’re trying to measure. But if you’re trying to gauge instances of original reporting across the blogosphere or social media, Icerocket’s Top Blog Posts or Google Blogs might be a more appropriate data resource.
There are all kinds of news out there, coming from all kinds of places. Many of the sources from which journalists gathered news are now publishing their own news and analysis directly. They’re not waiting to be quoted by a news organization. They’re being found and shared directly, via blogs and social media.
PEJ’s research on the impact of social media on the dissemination of news is valuable, and I recommend reading it. Just be aware that people share many kinds of news, big and small, for many reasons. Linking and sharing is clearly a fast-growing channel for news discovery, and news ventures should understand and capitalize on that mechanism.
But also, recognize that “the news” is becoming less and less defined by what news organizations produce. Traditional news orgs are still a big part of the news picture—but other news sources matter too, and they’re growing. Increasingly, news is becoming more about the “long tail” than the “top story.”
Comments (4) • Permalink • Tell-a-Friend
Tags: research, social media, pew project for excellence in journalism, blogs, reporting
Comments
link-bags | link-plastic | link-clothes:It was very useful for me. Keep sharing such ideas in the future as well. This was actually what I was looking for, and I am glad to came here! Thanks for sharing the such information with us.Thanks for the nice blog.
By coach handbag, 05/27/10 at 2:00 am
link-bags | link-plastic | link-bags:It was very useful for me. Keep sharing such ideas in the future as well. This was actually what I was looking for, and I am glad to came here! Thanks for sharing the such information with us.Thanks for the nice blog.
By coach handbag, 05/27/10 at 2:02 am
Good post, and I think you have a great critique here. As PEJ acknowledges in the study, measuring something as huge and diverse as “Twitter” and “the blogosphere” is difficult, and it really helps to have as much input as possible before and after in thinking these things through.
The only thing I balk at is terminology like “PEJ’s questionable data sources” in big bold print that makes it appear as though the methodology is deeply flawed. I see this over and over again lately. Somebody comes out with a study, and the method is solid (I have PhD in journalism, so not totally pulling that out of my butt), but both sides immediately exaggerate the claims to fit their own preconceptions. I don’t know how you could read that study and come out with a headline like “traditional media still rules!” as you pointed out, but I also don’t think that it’s fair to cry foul on methodology because the claims don’t match what is starting to become dogma. I’m firmly in the camp that believes blogs and social media ARE contributing original content and reporting and traditional types that look down their noses at such drive me mad, but at the same time, I don’t want to become TOO utopian about the whole thing and overstate the case without hard evidence, either.
Again, it’s not that I don’t think the critique is valid and useful and I’m not trying to slam it, but it just worries me a little when an important study gets ripped because they made some difficult but necessary methodological choices.
Carrie Brown aka @brizzyc
University of Memphis
By brizzyc, 06/18/10 at 11:56 am
Social media research is no an easy thing to do so I respect those companies which are actually doing that. First of all thanks for the great article of yours here. I have enjoyed reading it for sure. Of course there are some thing where I can’t agree with you but overall it is almost perfect. I hope to see more such amazing articles from you in the nearest future too. Thanks!
Regards, Steven from diamond watches
By Steven88, 01/24/11 at 1:48 am
Page 1 of 1 pages
Commenting is not available in this section entry.
Exploring innovation, transformation and leadership in a new ecosystem of news, by journalist and change advocate 
RSS Feed