Nieman Journalism Lab |
- Real-life trending topics: Behavio unlocks your smartphone’s senses
- Android sensory data, streaming breaking news, and better real-time filters: The winners of the 2012 Knight News Challenge
- There’s no such thing as an objective filter: Why designing algorithms that tell us the news is hard
- In Australia, cutbacks and an ownership shift point toward fewer points of view
| Real-life trending topics: Behavio unlocks your smartphone’s senses Posted: 18 Jun 2012 12:30 PM PDT
“When you leave the house, the three things you usually take with you are your keys, your wallet, and your phone,” says Nadav Aharony. One of those three things is a “sensing and processing machine,” he says. (Hint: not your keys.) The smartphone in your pocket knows where you are, who you’re with, how fast you’re moving, whether you’re standing up or sitting down. What if we could refine all of this invisible metadata into meaningful stories? Aharony and his colleagues, Cody Sumter and Alan Gardner — all recent or soon-to-be recent MIT Media Lab grads — are building an open-source platform that simplifies this kind of mobile data collection. They want to enable a new generation of apps that can detect social and behavioral trends in communities. Now, with a $355,000 Knight News Challenge grant, the trio will turn this three-year side project into a full-time concern called Behavio. The guys are all quitting their jobs or giving up job offers to do this. Their next job is to figure out how this software could work for journalists.
Our devices could power early-warning systems for disease outbreaks or provide life-saving data during disasters. Think of it as “trending topics for a community,” Sumter told me. For example, “an apartment complex full of people stopped showing up to work…If you had some third-party news sources, people were able to donate their own data, this would have sent up a massive red flag.” Although smartphones are equipped with dozens of sensors — for light, sound, gravity, motion, proximity, pressure, even magnetic fields — “it’s very hard today to tap into these capabilities,” Aharony said. “When the big organizations do this, it’s usually closed-source…so small developers end up doing everything from scratch.” Storing data securely, sending it to a server, and optimizing for battery life are big technical challenges. Funf (“fun framework”) is the framework the team created to solve these problems for developers. If you have zero programming experience but want to play, they created Funf in a Box, a wizard that lets you pick from all kinds of options and then deposits a fully baked sample app into your Dropbox account. (Funf only works with Android devices right now, as Google’s devices are much more accommodating to tinkerers than Apple’s.) The team mocked up a sample interface that superimposes “smart” metadata on a photo. You can see the user was walking at three miles per hour, he was outside, he had been walking for about a mile yet, he was with three friends, it was 56 degrees and partly cloudy, and the ambient noise level was moderate.
Alex “Sandy” Pentland, the director of MIT’s Human Dynamics Group and Aharony’s adviser, has been doing this kind of work with controlled groups for a few years. Pentland has tracked the spread of political opinions, fads, and influenza through communities. Pentland was able to predict which subjects were coming down with the flu before they reported any symptoms (PDF). A few days before participants reported having sore throats — before they felt sick — motion sensors captured less activity during the day. Participants who reported fever sent fewer late-night texts and had fewer face-to-face interactions, as determined by Bluetooth sensors picking up other Bluetooth sensors nearby. These changes in behavior are virtually undetectable to us, but computers are good at detecting deviations from routine. The always-on devices in our pockets can silently capture it all. This new breed of apps raises all kinds of privacy concerns, of course. But “in open-source, you can actually show what you’re collecting,” Aharony said. They hope to avoid privacy scares, such as we’ve seen from the likes of Apple and Path, through open source’s transparency. “We are embedding privacy detection. The framework doesn’t save any data,” he said. Say a developer writes an app that captures which phone numbers a person dials most frequently. Each number is hashed with a one-way encoding algorithm. So (617) 555-1234 is stored as Behavio fits with the Knight Foundation’s theme for this iteration of News Challenge, networks. The new, nimbler News Challenge is more pragmatic, preferring people who are builders more than dreamers. Aharony, Sumter, and Gardner have a head start, with working code and about 500 developers already using the framework. Gardner said Behavio will focus on building the framework more than individual apps. “We’re sort of enabling this ecosystem,” he said. “We want a kid in college to build an app in a year he’d never be able to build on his own.” |
| Posted: 18 Jun 2012 10:47 AM PDT At the 2012 MIT-Knight Civic Media Conference (livestream here), the winners of the latest round of the Knight News Challenge have just been announced. This is the first go-round since the News Challenge’s reboot earlier this year, which aimed to both quicken the pace and give applicants a focus. Instead of one competition a year, Knight now runs three; instead of a (relatively) open call for applications, it now asks for ideas tied to a broad theme — in this case, networks.
One item of note: For the first time, Knight isn’t sharing the funding amounts for two News Challenge winners. That’s because they’re being funded through the foundation’s Knight Enterprise Fund, which operates as an early-stage venture fund for for-profit ventures, rather than through straight grants. (We wrote recently about Knight’s similar investment in Umbel.) So today’s funding total is somewhere north of $1.37 million. The four grantees for whom we do know the amounts were all between $320,000 and $360,000. Without further ado, here are today’s six winners. We’ll have fuller profiles of each over the next couple of days. And meanwhile, the second News Challenge of the year is well underway. The theme this time is data, and time’s running short: The deadline to enter is noon Eastern on June 21 — that’s Thursday. (Even if you don’t plan on entering, check out the description just to see use of the word “datasexuals.”) The topic of the third round this year hasn’t been chosen yet, Knight CEO Alberto Ibargüen just said from the stage. BehavioAward: $355,000
Peepol.tvAward: $360,000
Recovers.orgAward: $340,000
Signalnoi.seAward: undisclosed
Tor ProjectAward: $320,000
WatchupAward: undisclosed
|
| There’s no such thing as an objective filter: Why designing algorithms that tell us the news is hard Posted: 18 Jun 2012 07:45 AM PDT
We are all immersed in an incomprehensible abundance of available information, and we can only read or watch or consume some meaningless fraction of it. What we see, and what we don’t see, is heavily mediated by information filtering algorithms: Google web search, the Facebook news feed, personalization and recommendation engines of all kinds. Filtering algorithms shape our knowledge and our society. They are now a permanent part of what it means to perceive the world. So it’s important to design them well, to make good choices here. What makes a “good” filtering algorithm? There’s no easy answer. How can there be? The question of who should see what when is not like the question of how far the moon is from the Earth, or what the mayor ate for breakfast today, or whether Elvis is still alive. Those questions have simple answers that are either right or wrong. But choosing who sees what information is not like that at all: There’s no one answer, just many possible visions of the type of world we’d like to live in.
Yet we still need answers. Otherwise there’s no way to build the algorithms we must have, and no way to critique them. Different disciplines have different approaches to this problem. At the risk of caricature, let’s say there are two broad camps here. The “technologists” are engineers, computer scientists, people with training in quantitative fields. They are the people likely to be directly responsible for building our filtering systems. The “humanists” are editors, curators, writers, sociologists, humanities scholars. They spend their life on the handcrafted work of deciding that this, and not that, deserves our attention — or examining the consequences of such choices. These two cultures need each other, but they don’t seem to speak the same language. Technologists have long used two numbers to measure how well a search engine works. “Precision” counts how many of the returned items were actually relevant to a user’s query, while “recall” measures how many relevant items the search should have returned, but didn’t. Together, these two percentages — false positives and false negatives — give a clear indication of search engine performance. You can make a change to the algorithm and see if the numbers go up or down, and by how much. Recommendation engines are another type of filtering system, and in practice each optimizes for some numerical value. A retailer like Amazon might measure how much each customer ends up buying, while Netflix is more concerned with choosing movies that you’re going to like (as measured by the ratings you assign). News personalization systems often use the number of clicks on your customized headlines as a proxy for success. These sorts of measures are crucial for engineering work, but of course they miss much. Search engine performance metrics only work if you, the human, already have a clear idea of what counts as “relevant” for a given query. Dollars and clicks are useful metrics, but they squeeze rich social interactions into a limited economic mindset. Humanists tend to think about things like whether the user “needs” the information, or whether it might challenge them, inspire them, or teach them something new. Or they might be concerned about filter bubbles, the idea that personalized information filters will end up telling us only what we want to hear, fragmenting us into smaller and smaller factions that never really talk to one another. Other people wonder about serendipity in online information systems, imagining ways to help us discover things we might never have looked for. All of these people ask difficult questions about how our media shape our societies, and what’s good or bad about that. But how can we actually know if any particular system is producing “filter bubbles” or not? And given the choice between two algorithms that might be quite similar, how should we pick one over the other? Descriptions of humanistic qualities are usually far too vague to be turned into code. Public Insight Network co-founder Andrew Haeg has suggested that a good filtering algorithm should align with Maslow’s hierarchy of human needs. Sounds great, except that I have no idea how to translate that into an algorithm, or how I would test my code against the real world to see if it satisfied those needs. As sociologist Stuart Hall famously put it:
Technologists often have a refreshing pragmatism here: Facebook ran a controlled experiment with 250 million users to see what effect removing something from your news feed has on whether you or not you eventually see it. Google continually tests search algorithm changes by asking thousands of users whether they like the results in Column A or Column B better. But this kind of engineering is no substitute for envisioning truly new ways of organizing information; decades ago, Horst Rittel and Melvin Webber explained why “optimization” will not solve societal problems, inventing the term “wicked problem” in the process. Conversely, humanists are often unwilling or unable to grapple with the dirty technical details of how algorithms are built, and what is actually possible. It took a computer programmer, Seth Finkelstein, to explain how algorithmic constraints create the sociological effects of Google’s web search. Nice work, and there’s an argument to be made that anyone contemplating filtering algorithms should understand the engineering involved. But we’re also going to need simple ways to explain all of this to non-specialists. Many people accused Twitter of censorship when #occupywallstreet failed to make the trending topics list despite simultaneous protests in dozens of cities. In fact there was no censorship, just a quirk of the trend-finding algorithm — an algorithm that “claims to know the mind of the public.” Moreover, it’s not just about code. All filtering algorithms operate on human input, and some are much more reliant on humans than others. Web search engines look at the links between pages, links that were put there by people. Collaborative filtering algorithms based on “likes” or voting (à la Reddit, Digg, Slashdot, etc.) solicit direct feedback. The result of any filtering algorithm depends on a complex interaction between the code that implements it and the people who use it. The same algorithm in different social contexts, or administered by different people, can produce wildly different results. We have to look at culture and code together. So filtering algorithm design is one of those wildly interdisciplinary problems. The challenge is to imagine systems that:
That’s very hard. It requires a rare type of cross-domain thinking, because we don’t yet really know how to combine the pragmatic demands of technology with the social aspirations of the humanities. But it’s also an exciting time to be working in digital journalism, where these two cultures meet every day. Algorithmic image by Anders Hoff used under a Creative Commons license. |
| In Australia, cutbacks and an ownership shift point toward fewer points of view Posted: 18 Jun 2012 06:34 AM PDT
Editor’s Note: The United States may have led the way (if one may call it leadership) in the disruption of its newspaper industry, but it’s hardly alone. The news industry in Australia today got the latest bad news: 1,900 jobs lost at Fairfax Media, owner of the Sydney Morning Herald and The Age in Melbourne. The two newsrooms will be merged, with a net loss of about 150 positions; the papers will switch to a tabloid format and put up online paywalls. And, to top it off, mining magnate Gina Rinehart — the richest woman in the world — is buying up shares of the company and seeking editorial control. (Rinehart has funded climate change skeptics and is strongly opposed to mining taxes, as one might imagine.) In a country the size of Australia with its substantial ownership concentration in its media — 11 of its 12 newspapers in capital cities are owned by either Fairfax or News Corp. — having alternate voices is critical. Last year, we wrote about a new site called The Conversation which is attempting to provide that. Here, reprinted from The Conversation, is Andrew Jaspan — former editor-in-chief of The Age — on what the changes at Fairfax could mean to Australian media. The next two weeks will be defining moments for Australia. It’s when Fairfax is likely to morph into Gina-fax. On Tuesday Gina Rinehart, the world’s richest woman, is expected to confirm that she has acquired up to 19.9 percent of Fairfax. The current Board, led by ex-Woolworths and now Walmart director Roger Corbett, is expected to raise the white flag in their efforts to ward off Rinehart’s bid for control. Rinehart is believed to want two or three seats on the board, and control of the Fairfax’s editorial positioning. And what she wants she can afford to buy.
Running in parallel, Fairfax will announce this week one of the most radical restructuring of its metropolitan mastheads, The Age and Sydney Morning Herald. From July 1 the two papers will be nationalized, that is, converted into one newsroom across both titles. There will be some local differences to allow the content to be rebranded for the Melbourne and Sydney audiences, but two voices in our shallow pool of diversity will become one. And Fairfax will reduce its editorial workforce on the two papers by around 25 percent from roughly 800 to 600. In tandem, Kim Williams, the chief executive of News Ltd., is expected to announce the most radical restructuring of the entire News Ltd. workforce with a reduction of up to 1,500 staff. This perfect storm has been brewing for some time. The decline and implosion of the media was seen as a European or American disease that Australia would avoid, much like the GFC. The seeds of Fairfax’s destruction were born in the mid 1990s when it failed to fully engage, understand and act on the disruptive threats of the internet. The story of Fairfax’s decline is one of managerial failure. The company has been run by senior executives and boards with no direct experience running a media company. Instead, leaders at Fairfax have been property developers, management consultants, accountants, and rugby players. Those people did not have the experience or understanding of a people-media business to steer the ship into safe waters. Instead they allowed Fairfax to remain at sea while competitors savaged the business. One by one Fairfax was stripped of its classified advertising “rivers of gold”. The jobs went to Seek.com.au, Cars to Carsales.co.au, homes to Realestate.com.au. And shorn of those easy revenues the only way Fairfax CEOs could “stay in the game” was to cut costs faster than revenues fell (all the while pocketing eye-watering salaries and bonuses). Instead of having the foresight to embrace and invest in the digital age by bringing together mastheads to work collegiately, Fairfax leadership instead chose to separate the online team from the print team and run them as two distinct businesses, with “Fairfax Digital” competing for advertising revenues with the so-called “Fairfax Publishing”. In 2007, I was asked to lead a team of three senior executives to visit the most progressive newspaper/media companies in the US and UK and report back to the then CEO, David Kirk. We went to the Wall Street Journal, New York Times, USA Today, Washington Post, The London Telegraph, The Financial Times and The Guardian. We reported back to Kirk that every one of these had brought together “print” and “digital” into one resource. That is one editorial team, one advertising team and one back office. Kirk flatly opposed doing the same on the grounds the two businesses were both very profitable. And he wanted to keep it that way. Five years later, with the company’s market value slashed from A$7 billion to just over A$1 billion, this integration will finally be imposed next month. And for the first time in living memory the change will be led by a former journalist and senior editor, the CEO, Greg Hywood, along with the advice of consultants Bain & Co. (Mitt Romney’s crew). But it’s too late to save the Fairfax we know. The share price has collapsed from A$5 to A$0.60 or less because no one in the market believes there is a coherent strategy for the company. And that has left the company weak and defenceless to predators such as Rinehart. Staff, meanwhile, have been living in denial. Though finally last week the penny dropped among the editorial staff that Gina’s tilt at Fairfax will happen. That has led to great despondency, and many rightly concerned about their future. And of course, once in, she is in control, and they will be told if they don’t like it, they can ship out. What does this all mean? Rinehart is not an investor in Fairfax to earn a return like the rest of the company’s long-suffering institutional investors. She is making her play to change the climate of opinion in Australia. Back in 2010 she and her fellow mining barons spent A$22 million to get rid of Kevin Rudd’s proposed mining tax. And so successful was the campaign that they got rid of Rudd and saved themselves an estimated A$20 billion in taxes. Rinehart’s appointment of Australia’s leading climate change sceptic, Ian Plimer, as an advisor to her mining companies is simply a taste of what’s to come. As one senior Fairfax editor remarked, expect this kind of front page once Rinehart gets control: “Exclusive: Climate Change is a Hoax.” Rinehart aims to change the terms of debate in Australia for good. Her fellow Channel 10 director, “Hungry Jack” Cowin, the burger man, will likely join Rinehart on the board of Fairfax. Cowin has already made clear that the Fairfax Board has every right to set the editorial tone of the papers. And that Andrew Bolt, who already has the Bolt Report show on Channel 10, would be welcome at a Rinehart dominated Fairfax to “balance the message that’s being communicated to the community”. With such a program, Rinehart and Co. may well tell staff and readers that if they don’t like it they can go elsewhere. The problem in Australia is where to? The media is in crisis elsewhere in the West, but usually there is a choice, somewhere else to go to get a job or to get your news and commentary. Right now if you live in Hobart, Adelaide, Perth, Darwin or Brisbane you have no choice, just the one paper. In Melbourne and Sydney, there was choice. Readers who, like Rinehart, prefer the editorial tone and message of The Australian, with its line on mining tax little different to that run by BHP, will be spoilt for choice. And scepticism towards climate change will now be shared by all three quality mastheads. Those with different views will have limited options. Is this the modern, open, progressive, democratic, tolerant, knowledge-based, clever country we aspire to be? Or are we seeing the same rise of the oligarch as in Russia where the resource-rich billionaires also dominate the media? Or Italy, where Silvio Berlusconi owned the majority of the TV stations and newspapers and imposed his right-wing agenda, and ultimately won control of the country as Prime Minister? This is an important moment for all those who cherish democratic and pluralistic debate and a freedom to information that is factual and reliable to inform decision-making. Given that both the Fairfax and News Ltd. papers are “interested parties” in the outcome, you will be hard pressed to get a full and dispassionate account of the next few weeks’ momentous events. That is what The Conversation will aim to provide. We will be leading a debate over the next few weeks, and keeping tabs on the media developments. We hope you will engage with us through your comments and suggestions for the coverage you would like to see us run. It’s important to have your say while the matter is live, rather than bleat about it afterwards. Andrew Jaspan is editor of The Conversation and the former editor-in-chief of The Age. 
1794 map of Australia by Samuel Dunn. |
With the split into three, there are fewer winners to be announced today than in past KNC cycles. But my unscientific observation was that the general quality of applications was up; the 
This article was originally published at 
| You are subscribed to email updates from Nieman Journalism Lab To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google Inc., 20 West Kinzie, Chicago IL USA 60610 | |