Skip to content

Digital Impact was created by the Digital Civil Society Lab at Stanford PACS and was managed until 2024. It is no longer being updated.

You Knew It Was Coming: Big Data

DataPicBut we’ll let C.S. Lewis start this conversation, not the social sector or the data scientist: “Even in literature and art, no man who bothers about originality will ever be original: whereas if you simply try to tell the truth (without caring twopence how often it has been told before) you will, nine times out of ten, become original without ever having noticed it.” That said, let’s try our hand at Big Data amidst the fast-growing volume of (seemingly redundant) content on the topic, aiming for a few truths that will uncover how we can get a handle on the tools and approaches that will help us as we help people. [First things first, let’s stop capitalizing it.]

There’s an unfunny irony in the definition of big data.

The challenges we face with “capture, curation, storage, search, sharing, transfer, analysis, and visualization,” source could easily refer to just learning about big data itself, long before getting to what we can or should be doing about it. The steepness of that learning curve can cause some to tuck heads in the sand and others to wait and watch for a “perfect” social sector response.

For others, it’s a cause to celebrate that “big data will cure all ills.” These folks face an equally loud chorus of warnings that “big data will wreck the world.” And, as with any new phenomenon, there is a significant number of people and organizations just going about the business of trying to make big data work. These are the people practicing intelligent iterations: making investments, conducting experiments, succeeding and failing in an effort to learn and to use information to advance their missions.

Given these dynamics, we’d like to use this month to create a discussion that weaves in and out of the thinking and doing in big data, relates directly to the social sector, and generates a dialogue that can survive as a meaningful reference point. (Bits and bytes are now immortal with big data, right? May as well make ’em good.)

The content you can expect this month

Curation of Existing Content: There’s been a significant amount published on big data for the social sector already. This month, most of our content will be content that’s already been posted within the space – with new perspectives added by you and other commentators. We’ve chosen the three sub-themes (below) as a point of entry to the topic

Original Content: We will offer original comment from a few individuals this month on what big data means for the social sector: Jerry Nichols – Senior Director, Worldwide Performance Management at SAP on “Big Data For The Consumer’s Journey.” Previous contributor, Sunand Menon, will provide an overview of the nature of Big Data and how we can incorporate it into the social sector. Plus, we’ll hear from a couple of surprise guests from outside of the typical data conversation.

Let’s frame it with three sub-themes

Understanding Big Data: There is a lot to discover in the practice of big data. For example: What does big data look like when applied to international development? Does big data help us to understand which questions to ask, or provide us a more rigorous way of answering those questions?

Small Data: What are we doing with the data we have now? Where should we focus our relatively limited resources, in light of greater availability of large data sets?

Open Data: The intersection of “open” and “big,”as more data is opened up to organizations and the public. For example, will open data truly mean a greater demand for high quality data?

From you, we’ll ask for a kind of participation that doesn’t walk around any of this too lightly. Jump into the comments section after each post. Frustrations, disagreements, new ideas, old ideas!, tinkerers, contrarians and evangelists are welcome. Trust that we’re willing to engage a thorough discussion. We need words, ideas, and action all mashed-up and top-of-mind to get it right. Join in.