The “real” reason we have big data today

    By: Mr. Adrian Bridgwater on Jun 23, 2012

    We are seeing an increasing focus on data integration both in the cloud and at the so-called big data level every day now -- (not that ‘cloud’ and ‘big data’ are necessarily two different places anyway of course) -- and this is in itself creating a subset of new solutions focused on helping the flow of data both to and from big data stores such as Hadoop and others.

    The argument here is that developers are now tasked with integrating big data sets into and out of environments where devices (and the applications they are running) may be working with what can only logically be called “little data” by comparison.

    The REAL WORLD challenge

    Programmers and database administrators will now need to find the agility to combine (or “integrate” if you prefer) and process (or “manage” if you prefer) data from all their operations within the new highly scalable data stores that we now interface with online.

    Industry comments suggest that big data stores such as Hadoop may now need visual data integration tooling – and that this may now be particularly the case given the wider need to “execute increasingly complex workloads” against massive amounts of data at high speed.

    So as demand for powerful big data analytic platforms appears to be coming at us quickly right now it should be no surprise that Sybase hosts analyst white papers on this type of topic here:

    Big data is ALL THE RAGE!

    As Bloor analyst Philip Howard notes, “big data is all the rage.”

    But Howard asks, while we will address the question of what big data is, the real question (he says) is how it differs from what we might consider to be the so-called “traditional world” of analytics and data warehousing that we knew perhaps as recently as just a couple of years ago.

    The REAL reason WHY we have big data

    “Historically, data warehouses catered for data that was originally transactional,” argues Bloor’s Howard.

    “Then... as companies started to want to analyse other forms of data, such as clickstream data, these were converted into relational format (they are not intrinsically relational) so that they could be stored and analysed with a conventional warehouse. However, as the world has become increasingly instrumented with sensors, monitors and other devices, the need to store such data has ballooned.”




    Released: June 23, 2012, 12:14 pm | Updated: March 22, 2014, 6:41 pm
    Keywords: Opinion News | Hadoop




    Copyright © 2014 ISUG-TECH. All Rights Reserved
    All material, files, logos and trademarks within this site are copyright their respective organizations

    Terms of Service - Privacy Policy - Contact the Help Desk