Big Data

Others Working on This

Amy Bach, Jae Um, Michael Sander, Damien Riehl, and Sarah Sutherland

On this Page

Data collected and analyzed at scale can reveal information from which organizations can draw conclusions and make well-informed decisions. (Big data and business intelligence are related but distinct fields.)

Big data implies huge datasets (terabytes of data) with relatively low information density. In other words, you’ve got just a few points of data about millions or billions or trillions of things.

Humans have been straining to gather, save, and analyze more and more data throughout recorded history (at least since recorded history is technically data that was gathered, saved, and analyzed).

It’s fun to stop and consider some of the surprising social developments that motivated people to better wrangle data: the US census, proliferation of credit cards, longitudinal academic research, calculations to measure astrological phenomena or predict market trends, just to name a few. Of course, we now associate BIG DATA with BIG TECH companies; and for good reason. But corporations and academics were crunching the numbers way before Google made it cool.

In recent decades, key technological advances have made it so much faster and cheaper to share, gather, and save data, and so many companies have found unprecedented value in analyzing that data, that it often feels urgent to gather and save data long before we know what to do with it. Well resourced companies do this the most. But, the tools for accessing data (big and small) are only getting cheaper and more user friendly.

Crime Data

One of the clearest intersections of data and justice is crime data. It is a very complex topic. There is no standardization or uniformity in how criminal systems work across the country, there is a severe lack of transparency and efficiency in government sectors, and they are evolving, paralyzing, and conflicting political dynamics tied up with crime and how people respond to it. Due to these factors there is often very little data available on some very important subjects related to crime. This paragraph only skims the surface of some vast, complex, and highly controversial issues that countless criminologists have devoted their careers to. We hope that you will study that work closely.

Innovators have already recognized this problem and taken steps to follow it. Measures of Justice is a standout organization, founded by Amy Bach, that aims to gather, clean, and share crime data from across the country. They also push for more government agencies to collect and release data in standardized formats to enable more people to learn from it.

Published on January 6th, 2022. Last updated on January 18th, 2022, by Sam Glover.