Unstructured data — everything from social media posts and sensor data to email, images and web logs — is growing at an unprecedented pace. Here are just a few mind-blowing statistics: Twitter sees about 175 million tweets each day and has more than 465 million accounts. 571 new websites are created every minute of every day. And the world creates 2.5 quintillion bytes of data per day from unstructured data sources like sensors, social media posts and digital photos. Clearly, unstructured data is growing exponentially, and government is no exception.
What does that mean? “Unstructured” means just that — the elements within the data have no structure. For example, even a simple blog post has many elements embedded in it — the date and time it was posted, the content, embedded links, author, etc. That makes searching and analysis much more difficult than for structured data, like transactions.
Unstructured data is so valuable, however, that organizations are finding ways to extract meaning out of it — meaning that can translate into connections and patterns that would otherwise be missed. Depending on the mission, those missed connections could result in real missed opportunities. In most cases, unstructured data is best analyzed with cutting-edge analytics tools; with these tools, agencies can make inroads in reducing fraud, preventing crime, ferreting out waste, and even confirming acts of terror.