IJIMAI 2016 - Special Issue on Big Data and AI - Vol. 3 Issue 6

Digital information has redefined the way in which both public and private organizations are faced with the use of data to improve decision making. The importance of Big Data lies in the huge amount of data generated every day, especially following the emergence of online social networks (Facebook, Twitter, Google Plus, etc.) and the exponential growth of devices such as smartphones, smartwatches and other wearables, sensor networks, etc. as well as the possibility of taking into account increasingly updated and more varied information for decision making.

With proper Big Data analysis we can spot trends, get models from historical data for predicting future events or extract patterns from user behaviour, and thus be able to tailor services to the needs of users in a better way.

Using Big Data is becoming widespread among organizations of all kinds. It is a fact that large companies, start-ups, government agencies and non-governmental organizations are gradually being encouraged to use microdata generated by digital devices to operate more efficiently. When these microdata are aggregated they turn into massive amounts of data which require specialized tools and skills to be managed.

The challenge organizations are facing is that the increasing amount of data is too large or too unstructured to be managed and analysed with traditional methods. Think of the data derived from the sequence of clicks from the Web, social media content - tweets, blogs, Facebook wall postings (Facebook alone accounts for more than 1 billion active users generating social interaction content. Google processes on average over 53 thousand search queries per second, making it over 4.6 billion in a single day) - or radio frequency identification systems, which generate up to a thousand times more data than conventional barcode systems (12 million RFID tags – used to capture data and track movement of objects in physical world – had been sold in by 2011. By 2021, it is estimated that that number will have risen to 209 billion. Walmart manages more than 1 million customer transactions per hour). In the World 10.000 payment card transactions are recorded every second. The amount of data transferred over mobile networks increased by 81% to 1.5 Exabyte per month between 2012 and 2014. More than 5 billion people make phone calls, send text messages and surf the Internet with mobile phones. Every day they send 340 million tweets (4.000 per second!). To date they’ve generated 2.5 trillion bytes of data. However, very little of this information is in the form of rows and columns of traditional databases.