Making cities smarter How citizens' collective intelligence can guide better decision making

Cities have the capability of providing something for everybody, only because, and only when, they are created by everybody.—Jane Jacobs, The Death and Life of Great American Cities¹

The wisdom of the crowd

Imagine that you are an urban planner charged with putting walkways in a newly built city park. How would you go about deciding where to put the paths?

Several approaches come to mind: You could survey likely park users. You could copy “best practices” from other parks. Or you could build sophisticated computer simulations to project future foot traffic.

Or you could take a different approach: What if you just open the park and observe the paths in the dirt naturally created by foot traffic? By using a process of discovery rather than design, you could leverage the built-in knowledge and preferences of real park users to find an optimal walkway solution.

Prior to the construction of Freedom Tower in New York City, the great urban theorist Jane Jacobs suggested that the Ground Zero site scrap its existing street grid: “I was at a school in Connecticut where the architects watched paths that the children made in the snow all winter, and then when spring came, they made those the gravel paths across the green. Why not do the same thing here?”²

Jane Jacobs is perhaps the pre-eminent urban philosopher of modern times, and her modest parable exemplifies an idea common to her theories of urban design—namely that the best designs respect the wishes of actual city dwellers. Jacobs takes a dim view of the grand designs of central planners, and once urged her audience “to respect—in the deepest sense—strips of chaos that have a weird wisdom of their own.”³

Most discussions of smart cities focus on infrastructure: Big data and information technology being used to better manage urban assets such as public transit, wastewater systems, roads, and so on. The term smart typically denotes physical assets connected to the Internet of Things via sensor technology, generating streams of valuable data: smart parking meters, smart streetlights, smart water use.

Both literally and in a broader figurative sense, connected devices help “make the trains run on time” and can yield greener, more efficient cities. But truly smart cities have to encompass more than just infrastructure and city services. A truly smart city leverages technology to elicit the wisdom of its citizens. A smart city won’t have a population that is any more intelligent than a traditional city, but it will enable smarter decisions—by city planners, by individual citizens, and by groups.

Bottom-up vs. top-down

As important as connected devices and assets are, focusing exclusively on the hardware of smart cities leads to an incomplete vision. A large part of the promise of data science and Internet technology is that they enable not just smart things but smarter decisions. Three interrelated forces are at play.

The data science revolution: Thanks to mobile phones, connected cars, and wearable self-tracking devices, our lives are increasingly digitally captured. In essence, we are leaving digital footprints everywhere we go just like the children in the snow. This means that fine-grained data about our preferences, behaviors, and knowledge can be analyzed to create recommendations that enable better decisions.

Behavioral economics: The big data revolution coexists with the “behavioral nudge” revolution ushered in by the pioneering work in psychology, behavioral economics, and choice architecture of such figures as Daniel Kahneman, Amos Tversky, Richard Thaler, and Cass Sunstein. Data science and behavioral science complement one another. Big data points toward a solution but often needs help coming up with a recommendation on how to act. A key insight of choice architecture is that minor, often inexpensive tweaks to choice environments can yield outsized effects on people’s actions. Behavioral design thinking, particularly when combined with data science and digital technology, offers policymakers a powerful set of tools for achieving cities where people make smarter decisions.

Tapping into the wisdom of the crowd through technology: The Internet and mobile devices are enabling new forms of mass collaboration. Digitally connected citizens are the ultimate “network of sensors” that enables local information to get to decision makers in a timely fashion. In addition, recent methodological breakthroughs in survey design enable policymakers to better tap into the wisdom of crowds.

Common to each of these themes is the aspiration to base decisions on detailed local knowledge and choice-preserving decision environments, and—when possible—to achieve order from the bottom up: outcomes that reflect the voluntary choices of individuals rather than outcomes imposed by the top-down decisions of planners.

This doesn’t mean planners and their plans aren’t important—creating frameworks that tap into the wishes of individuals isn’t easy. These themes are exemplified in Jacobs’s story about children spontaneously and collaboratively laying down “desire paths” in the snow. The wise planner bases her decision on the emergent order resulting from the aggregated knowledge and preferences of the crowd. The smart city doesn’t decree smart outcomes, but rather it creates a platform for better decision making.

When Jacobs urges city planners to respect the “strips of chaos” that emerge in an urban environment, she is echoing economist Friedrich Hayek’s insights about markets. Hayek explained how the uncontrolled actions of individuals allowed predictable, efficient, and human-serving outcomes to emerge in the absence of central planning. Hayek’s “spontaneous order” of the marketplace—coupled with modern theories of crowdsourcing, prediction markets, and collective intelligence—all tell the same story: Establishing mechanisms to tap into the choices of individuals, especially when they are equipped with accurate information about trade-offs and alternatives, will usually produce solutions superior to the well-intentioned designs of central planners, be they urban planners or economic planners.

In the new age of data science, it is possible to achieve a 21st-century version of Jacobs’s vision.

The predictive polis: Big data and predictive analytics

For decades, the New York City Department of Buildings focused on complaints when deciding which properties to inspect for unsafe conditions and structural hazards. This made sense.

The problem was, in 2011, the city received almost 25,000 complaints about just one type of problem, illegal conversions, and had only 200 inspectors to cover the workload. These illegal conversions, in which landlords would divide apartments into smaller units to accommodate more people than the apartment was zoned for, were disasters waiting to happen. Dozens of people might occupy a space intended for five, generating issues in terms of fire safety, crime, and public health.

In response, the Mayor’s Office of Data Analytics, a crew of scientifically minded problem solvers led by Michael Flowers, was able to radically improve inspection efficiency by using predictive analytics. “When we prioritize the complaint list, we’re reducing our time to respond to the most dangerous places, in effect reducing the number of days that residents are living at risk,” Flowers says.⁴

Crucially, Flowers’s data science geek squad didn’t create a fancy algorithm sitting in isolation at their desks at City Hall—they built their predictive data model with the help of building inspectors who’d been in the field for years. “The field inspectors were like, ‘Yeah, I know which places are dumps in this neighborhood, because I’ve been working this beat for so long,’” Flowers says. “So we injected that employee experience into the data and fed that into a risk filter.” The result was a triaged list of properties for inspection.

The results were staggering. Previously only 13 percent of complaints had ended up requiring vacate orders. After Flowers’s team filtered out the number of false positives, the share of complaints leading to vacate orders escalated to 70 percent.⁵ “We didn’t reengineer anything,” Flowers says.⁶

In this case, the city didn’t gather any data it didn’t have before. It simply started making better decisions by using modern methods of data analysis. The results were impressive. The city found, for instance, that improved building inspections lowered risks for firefighters, since fires in illegal conversions were 15 times more likely than other fires to result in injury or death for firefighters.⁷ “It’s very, very clear that if the buildings department doesn’t do its job, it’s felt downstream by the fire department,” he explains.⁸ Thanks in no small part to this analytics-driven approach, in June 2015, New York City experienced zero fire deaths for the first time since 1916.⁹

Citizens help cities make better decisions

Increasingly, smart cities are making their data publicly available, creating the possibility for open-sourcing some of the data analytics. This means that citizen data scientists, both amateurs and professionals, can access open-sourced data and deliver unexpected insights.

Consider one experiment in Boston, where the city’s Data Portal¹⁰ hosts data sets available for public use on restaurant food safety inspections and other aspects of city life.¹¹ As in many other cities, health inspectors had long selected restaurants randomly, spending too much time at clean, rule-abiding sites and too little at restaurants needing more scrutiny.

To better target the restaurants in need of attention, Boston’s City Hall partnered with both Yelp and economists from Harvard Business School to leverage the information in Yelp users’ restaurant reviews, repurposing those reviews as raw data powering predictive algorithms.¹² In 2015, the city and its partners sponsored an open competition and made available to some 700-plus contestants both Boston’s restaurant inspection data and Yelp’s restaurant reviews dating back to 2006. The goal was to develop an algorithm that would predict health violations and enable city officials to better target restaurants for inspections.¹³

The participants analyzed the text of the reviews, including common words and phrases,¹⁴ and the Harvard economists evaluated the submissions against the city’s actual inspections covering 364 restaurants conducted in the six weeks following the competition.¹⁵ The verdict: Using the winning algorithm would have made inspectors 30 to 50 percent more productive in finding violations.¹⁶

The short version is this: City data + smart citizens = better city decisions.

Behavior change by design: The power of choice architecture

A city gets smarter only if all these data and analysis result in better decisions and outcomes for residents. That’s where behavioral insights—the science of choice architecture and behavioral “nudges”—comes in. We view behavioral science and data science as natural complements: Predictive models can be used to flag the cases in most need of attention; behavioral insights provide the tools for prompting the desired behavior change.¹⁷ Beyond this, using behavioral insights creatively is a powerful way of employing people-centric design to make cities smarter.

In San Francisco, Harvard University economist Mike Luca worked with Yelp to put information into the hands of diners.¹⁸ For every restaurant rated in the bottom 5 percent for hygiene by the city, a “consumer alert” warning on the Yelp app itself would give would-be diners important information precisely when it was most useful—at decision-making time. Compare this with traditional inspection reports—sometimes displayed in restaurants’ back hallways or buried in a report somewhere on a city official’s desk.¹⁹ Putting this sort of public information into citizens’ hands through technology can “nudge” them into making better, or at least more fully informed, decisions.

The behavioral insights movement was sparked by Richard Thaler and Cass Sunstein’s influential book Nudge, which introduced the concept of choice architecture to a wide audience.²⁰ While some observers criticize choice architecture as a form of manipulative social engineering, we see it instead as a way of thoughtfully providing information to help people better navigate their environments.

Nudge was, in fact, inspired by Don Norman’s classic The Design of Everyday Things, which argues that everyday devices should be designed based on users’ psychology rather than engineers’ preferences.²¹ For example, if the great majority of users can’t figure out how to operate a mobile phone, the fault must clearly be with the design of the device, not with the users. If we can design better gadgets by accounting for human psychology, why not do the same for public policies and choice environments?

Until a decade ago, Lake Shore Drive had one of Chicago’s most dangerous stretches of road. To reduce the number of accidents near the Oak Street curve, the city created a visual illusion by painting a sequence of white lines on the pavement, each shorter than the previous one, on the approach to the dangerous curve. The succession of shrinking lines gives drivers the feeling that they are speeding up, prompting them to slow down and take the curve at a safer speed. City traffic engineers reported 36 percent fewer crashes in the six months following the lines’ introduction.²²

However, cities communicate with residents in other ways than just on the road. Many of the notices that administrators distribute can be incomprehensible, strewn with jargon and seemingly written by—and for—lawyers. Philadelphia demonstrates another way. Aiming to reduce delinquency in city tax payments—in 2010, nearly 10 percent of city property taxes went unpaid—Philadelphia has been rewriting letters sent to delinquent taxpayers,²³ collaborating with academics to test different communications strategies.²⁴ The city found that appealing to civic duty, for example, by citing specific public services the property taxes provide, encouraged more people to pay the bill. Its most significant impact are on residents with relatively low levels of tax debt ($0 to $300).²⁵

New Mexico also used data-driven “nudge” communications to tackle a thorny problem: claimants fudging the truth to boost their unemployment insurance payments. Officials at the New Mexico Department of Workforce Solutions recognized that many fraudulent claims were the result of small fibs, not serious scams. So rather than taking the traditional (and expensive) approach of criminal enforcement, they employed a mix of data science and behavioral economics to nudge claimants toward greater honesty.²⁶

Each week, claimants must certify that they are looking for work and document all earnings. When the system spots an answer that doesn’t fit the usual pattern or range, it triggers a pop-up message emphasizing the importance of providing correct information. Administrators tested a dozen different messages, and because claimants must certify each week, New Mexico quickly learned which messages were most effective.²⁷ In the year after the smarter system went live, fraudulent payments fell by half; unrecovered overpayments of all stripes have been reduced by almost 75 percent, saving the state almost $7 million.²⁸

So far we’ve explored how data science and behavioral “nudge” science can enable smarter individual-level decisions on the part of both city officials and citizens. It turns out that the cloud technologies connecting our devices—and us—to the Internet of Things enable smarter crowd decisions as well.

Achieving collective intelligence: The urban ballet

Collective intelligence is nothing new. Groups of people, including families, armies, and business teams, have always exhibited varying degrees of collective intelligence. Collective intelligence is manifested even in groups of animals: Swarms of insects and flocks of birds are capable of aggregating the information gleaned from individual group members to find food and nesting places.

Indeed, Jacobs’s example of children creating desire paths in the snow also exemplifies collective intelligence: A kind of macro-level order emerging from individuals harmoniously going about their independent business. The idea is also implicit in Jacobs’s most famous passage, from the end of her 1963 book The Death and Life of Great American Cities, in which she compares the beautiful order that emerges from the complex, dynamic, and undirected actions of city dwellers to a “sidewalk ballet”:

This order is all composed of movement and change, and although it is life, not art, we may fancifully call it the art form of the city and liken it to the dance . . . an intricate ballet in which the individual dancers and ensembles all have distinctive parts which miraculously reinforce each other and compose an orderly whole.²⁹

We believe that Jacobs, had she not died in 2006, would be tantalized by the possibilities the Internet and cloud computing technology are creating for achieving novel forms of collective intelligence.

Thomas Malone, the founder of the MIT Collective Intelligence Center, points out that Internet technology enables new forms of collective intelligence that were impossible only a few decades ago. Wikipedia is one well-known example: a highly refined—quite literally encyclopedic—product that is produced by thousands of dispersed individuals operating with minimal central control. It is a powerful form of collective intelligence—but still one driven by manual actions.

The automated world of the Internet of Things (IoT)—allows us to take Malone’s point still further.³⁰ Citizens equipped with mobile phones capable of capturing, transmitting, and receiving information form a digital sidewalk ballet, contributing localized bits of knowledge, ideas, and opinions that lead to smarter decisions.

A classic example is the navigation app Waze, which helps drivers find the most efficient routes in cities around the world. By actively sharing reports through the app or simply keeping it open while driving, users map out traffic in real time, pinpointing congestion, speed traps, accidents, and other hazards. Drivers just open the app and enter their destination, and the system taps into this knowledge from the crowd, guiding every user toward the best possible route.³¹ The bidirectional information flows enabled by cloud computing and mobile technology allow drivers to self-organize in a way that was impossible even a few years ago. Like birds in a flock, IoT-connected cars and drivers now routinely exhibit a high degree of collective intelligence.

Citizens as sensors

The technology’s true power is its ability to facilitate the process of cocreation. The city of Boston partners with Waze to use driver data to reduce traffic congestion: Waze forwards data to the city’s traffic engineers, who blend them with their own data gathered from sensors and cameras to make real-time adjustments to traffic signals. As with any successful collaboration, both partners benefit. “We also provide information back to Waze on changes that we’ve made in real time, but also any known changes that we have coming up that are planned,” says Gina Fiandaca, the Boston Transportation Department commissioner.³² This helps Waze and its users in the Boston area—some 400,000 individuals—plan trips based on the most up-to-date information possible.

Another way the citizen-as-sensor concept has made Boston smarter is with the Street Bump app.³³ Using sensors in phones to map bumpy roads, the app empowers drivers to report potholes themselves. As a side benefit, useful insights have emerged—for instance, unexpectedly, users reported sunken manhole covers four times more often than potholes. Armed with this knowledge, the city worked with utility companies to fix 1,250 of the worst covers.³⁴

Buenos Aires, the capital of Argentina, uses the citizens-as-sensors approach to respond to complaints or suggestions. The city has developed a mobile app and established a call center to register citizen complaints. The city analysts also collect troves of text on multiple social media platforms to analyze citizens’ complaints. All three channels are integrated with a geographic information system to know the exact location of complaints. Based on location and type of complaint, the city government assigns a vendor to fix the complaint, which is then verified by staff members from the audit department to close the complaint. The swift resolution of complaints has led to a record increase in the citizen satisfaction rate and slashed the average time to resolve a complaint from 600 days in 2011 to 42 days in 2015.

Internet technology not only enables the harvesting of data from connected citizens—it offers novel ways of harvesting ideas and opinions to achieve a literal form of “the wisdom of crowds.” Wiki surveys, which evolve based on the input of responders, are a prime example.

Wiki surveys are adaptable, in that they elicit new possible survey answers from survey takers.³⁵ As a result, it is possible, and indeed quite common, for “answers” that hadn’t occurred to the survey designers to land at the top of the list. New York City launched its first wiki survey in October 2010 in conjunction with a series of community meetings while putting together PlaNYC 2030, a citywide sustainability plan. City officials asked a question: “What do you think is a better idea for creating a greener, greater New York City?” They seeded the survey with 25 possible answers. Over about four months, 1,436 respondents contributed 31,893 responses and 464 new ideas—including 8 of the top 10 scoring ideas. Only 2 of the top 10 were among the original seed ideas.³⁶ Jacobs would have been pleased.

As more data get shared, collective intelligence can empower everyone, from city officials to individual citizens, from civic groups to universities. Cities can boost the opportunities for such collective intelligence by using IoT technology to collect more data and making more public data available in usable form.

The SmartSantander project in the Spanish city of Santander involves 20,000 sensors that measure traffic flow, parking spaces, noise, pollution, temperature, moisture levels, and other metrics. The city has saved money by directing services where sensors say they are needed—for example, dimming streetlights on empty streets. Parking sensors have helped inform drivers where spots are available, and the SmartSantanderRA app even lets a user point her smartphone at, say, a civic monument to learn about it—such as what that guy did to deserve a statue or which performance will play tonight in the nearby concert hall. The app can turn users’ smartphones into sensors, helping citizens play a dual role in the SmartSantander project: as testers and as extensions of its capacity.

Jacobs’s ladder: Smart citizens, smart cities

The promise of the smart city should be about more than WiFi hotspots and talking trash cans. Without question, optimizing physical infrastructure to deliver smart mobility, smart city services, and smart energy is part of the smart city story. But to be worthy of the name, a smart city should also be using technology to promote better decision making.

Smarter decisions come about by using data science, behavioral science, and digital technology to empower better, less centralized, and more empirically informed decision making. Installing sensors that collect data for optimizing the performance of physical devices is part of what it takes to achieve the smart city. Going the last mile involves using technology to tap into the city’s greatest asset: its citizens.

This means shifting the focus away from top-down plans for optimizing physical infrastructure and making sure to include people in the smart city model. After all, the ultimate goal is to make cities a better place to live—not just a more efficient deliverer of infrastructure. People-centric design thinking promotes an emergent order and collective intelligence—the sidewalk ballet—and is central to Jane Jacobs’s view of cities. Jacobs wrote, “There is no logic that can be superimposed on the city; people make it, and it is to them, not buildings, that we must fit our plans.”

Thanks to technology, this vision has never been more attainable.