Issac Asimov, Hari Seldon, and the Invention of Big Data

Google, Amazon, Verizon, the NSA, and others are currently compiling an “Encyclopedia Galactica” of metadata for the same reason Seldon did in Asimov’s books… to influence decisions that people make. Having data about where people go, how long they stay there, who they call and how long they talk, and hundreds of other metrics, Big Data analysts seek patterns in previous behaviors to influence future behaviors of individuals.

In the process of packing books for a move, I came across some old science fiction by my one of my favorite writers, Issac Asimov. He was a brilliant thinker and scientist, an award winning science fiction and non-fiction writer, and, I have come to realize, the true inventor of Big Data and its use. Foundation, the first book written in a seven book sci-fi series (the book actually became the third book in the series, as subsequent prequel books were published)  was originally published in 1951, long before the technology that allows us to gather the vast amounts of electronic data we have today was even conceived of.

In his “Foundation” series, Asimov writes about a pivotal character, Hari Seldon, a brilliant scientist and statistician that developed the science of what he called “psychohistory” and what we call “Big Data Analysis” (he invented the word psychohistory, among others. The word is now used by the mental health industry, but has a different meaning). In the Foundation series, Seldon is able to make predictions on the behaviors of populations based on enormous pools of data, and manipulate these populations by influencing events at certain locations in space and time.  The agents of influence through which he works across the millennia are the writers of the “Encyclopedia Galactica”, the compilation of all the information in the known universe.

Google, Amazon, Verizon, the NSA, and others are currently compiling an “Encyclopedia Galactica” of metadata for the same reason Seldon did in Asimov’s books… to influence decisions that people make. Having data about where people go, how long they stay there, who they call and how long they talk, and hundreds of other metrics, Big Data analysts seek patterns in previous behaviors to  influence future behaviors of individuals. The difference was that Seldon’s vision was to “short-circuit” a galactic dark ages that would last for thirty thousand years and reduce it to a mere two thousand, whereas businesses are using the data to influence buying decisions, and the NSA uses it to stop terrorists (at least that what they tell us – a more noble purpose, to be sure).

One can only hope that, since the ‘Genie’ of Big Data has been let out of the bottle, that it can and will be used for more noble purposes than mere profit. If we are to be influenced in our decisions, perhaps we can be influenced to develop healthy habits, reduce our carbon footprint, slow and reverse population growth to sustainable levels, and a whole host of other behaviors that benefit humankind and the planet as a whole. Odds are, however it won’t be used that way, unless profit can be made… or maybe big data can find a way to influence that too.