The term “Big Data” has become ubiquitous, yet adoption of predictive techniques has a long way to go. Data is abundant, is growing exponentially, and is essential to many industries and nonprofits today. Society has not, however, fully embraced the remarkable benefits data-based prediction can contribute to global prosperity and resiliency in times of duress. It is imperative that private- and public-sector organizations work together to construct ways to safely and productively utilize data for the benefit of communities around the world.

We have moved past the age of Big Data and have entered the age of prediction. The accuracy of prediction grows with the amount of data you can access, meaning that the exponentially increasing availability of data is expanding our predictive power on a daily basis. Commercial applications, such as predicting retail sales or behaviors of individual consumers, are common. How can this predictive power be applied for social good?

The world needs to awaken to the potential of harnessing the power of prediction and take action now if we are to drive future prosperity and benefits for humanity.

Today, we have tools to predict more accurately not just natural phenomena like the weather but also subtle shifts in human behavior that can lead to crises such as pandemics, starvation, and conflict. Medicine is being transformed through access to data from genomic sequencing, medical imaging, and clinical trials. New data sources, when combined with today’s algorithms, allow us to detect patterns that were previously invisible and to form insights that can transform how we make decisions. Seemingly unrelated data sources can serve as meaningful proxies, which allow for predictions that may otherwise be undiscovered.

We believe that increasing the application of these methods to social and humanitarian issues must be the next step. The United Nations, with this in mind, launched Global Pulse in 2009 after recognizing the need for actionable information to protect vulnerable communities. The group now has a large network of innovation partners with which it works to experiment with new data sources and scale up proven solutions.

In Mexico, Global Pulse worked with debit card transaction data to measure the impact of a hurricane on coastal communities and track their rates of recovery. In Uganda, expenditures on mobile phone credit have been used to provide highly accurate estimates of poverty, as well as data on the movements of mobile subscribers, to predict the spread of measles. Off the coast of Libya, deep learning algorithms have been applied to data on the movement of vessels to detect migrant rescues and predict where refugees are likely to come ashore.

WorldQuant has undertaken similar efforts to utilize its ability to make predictions for public good in areas such as health care through the establishment of the WorldQuant Initiative for Quantitative Prediction with Weill Cornell Medicine. Many potentially game-changing predictions depend on access not only to the predictive abilities of companies like WorldQuant but also to data that is owned by the private sector. There is an incredible opportunity to start using the power of prediction to benefit society and contribute to global prosperity. Yet today, these opportunities remain largely unrealized. There are still few examples of smart, prediction-driven public services that one can point to. One of the main obstacles is the difficulty in obtaining access to timely data.

The model for unlocking the social value of these insights is global data philanthropy. The goal should be for companies to share their data in ways that protect both privacy and corporate competitiveness, while allowing researchers to extract predictive insights for global development, crisis prevention, and peace. These partnerships, examples of which include mobile operators, banks, retail chains, and social media platforms, are based on sound due diligence processes, data protection and privacy guidelines, and codes of ethics around data sharing, access, and responsible use. 

Data partnerships between private and public bodies are the key to building global prosperity and resiliency.

We need new global architectures of data partnerships. Achieving scale and sustainability won’t be possible through purely philanthropic approaches. We have already seen the mobile industry launch the first initiative on using Big Data for disaster response and public health emergencies, where using data to save lives is its own incentive. There are myriad other opportunities to serve the public good through solid business cases for public-sector markets; knowing how people move through a city can improve transportation systems, assess the dynamics of tourism, determine the locations for new schools or hospitals, and predict the heath impacts of pollution. 

It is important that we consider the cost of missing opportunities to use data for public good. This can support a wide range of public services and contribute to societal resiliency. The world needs to awaken to the potential of harnessing the power of prediction and take action now if we are to drive future prosperity and benefits for humanity.