As humanity gathered lots of data, it is not surprising that there is much talk about “big data”. As a consequence, most government agencies think that it is mandatory for them to design technologies to make progress in the field of big data science. Despite the progress in the field of big data, it is still not clear what the term “big data” exactly refers to.
According to Wikipedia, big data means “data sets so large and complex that they become awkward to work with using on-hand database management tools”. This is not a helpful definition as apart from replacing “big” with “large”, the definition suggests that data are too big or large only in relation to our current computational power. Some other definitions include large sets of data extracted from web transactions, email, video streams, or all other types of digital sources available.
Such definitions may give the impression that there is no real trouble with “big data” being a loosely defined term given the fact that computers cannot deal with so much data efficiently. Yet, this is exactly the core of the issue. From an epistemological point of view, there is too much big data, which can be technologically solved by discovering more techniques to be able to manage big data.
It may be useful to take into account that the term ‘big data’ arose only after the emergence of buzz expressions such as “information overload”. This is not to say that we should complain about the exponential growth of data, as data is a great asset and an amazing resource to exploit. As we cannot be forced to digest every available byte, the fact that we are becoming data-richer by the day cannot be the main issue.
Since the problem lies not in the wealth of data that is becoming available, clearly the solution cannot be merely to develop more and better techniques as they only would exacerbate the issue by generating more data.
The real epistemological issue is small patterns given the fact that companies can easily recognize where the new patterns can generate real value through means of their gigantic databases and how to further exploit that data generating new wealth.
Small patterns play a big role in today’s open market of ideas as they represent the new frontier of competition ranging from science to governance. It can also be risky in some cases such as the case of the US retail company Target, which tries to predict the pregnancy rate within a household based on some products, sent a mistaken coupon to a family where a teenager had not informed her parents about her new status.
On the other hand, it should also be taken into account that what makes the small patterns gain importance is its proper accumulation. Both the existence and the lack of big data entail signals. As the information scholar, Shannon, made the analogy, the famous detective Sherlock Holmes was able to resolve a difficult case due to the silence of his dog. As Shannon states, if big data are not “barking” when they should, something is going on.
In a similar vein, merely hoarding data will not solve the issue of coping with big data. Since 2000, there has been more and more data gathered within the infosphere which shifted the issue from what to record to what to delete.
As Plato asserts, ultimately, the game will be won by those who “know how to ask and answer questions” (Plato, Cratylus, 390c). In the realm of the infosphere, those who know which data may be useful in order to exploit their valuable patterns may eventually win the game.
As big data continues to grow, a feasible method of coping with it would be to know what to seek. At the moment, conveying such epistemological skills are usually seen within the realm of analytical thinking skills. Yet, to the extent that how much of our well-being depends on it a philosophical inquiry may be required to further delve into its methods. While the art of philosophy might have something to learn, it may also have something to teach. Wouldn’t Plato agree as well?
“Indeed, the worst of living creatures in the sight of God are the deaf and dumb who do not use reason.” (Qur’an, 8:22)