2015 will be a breakthrough year for big data. While 2014 validated that big data is here to stay and thrive, 2015 will witness an eclectic mix of innovation and market strategies driven by both the vendors as well as customers. In the text below, I have discussed the top 10 trends big data analytics will witness this year.
1. In-Memory databases will gain prominence
Real-time analytics is gaining in prominence and need. It is also the segment where the real money lies in the future. And the truth is that Hadoop, despite its abilities to processing humongous data sets isn’t actually tailor-made for real-time operational analytics. The answer to this is in-line analytics with in-memory databases. Some examples include DB2, Gemfire, Aerospike, SAP’s HANA and Oracle’s Databases In-Memory. It is not that these databases are very new to the market. What is making them attractive is a combination of drop in prices and emerging market demand. Fast dropping prices of memory devices has made the cost-benefit analysis on performance vs. cost of data storage in DRAM positive. In 2015, I expect that the market for in-memory database will heat up, both in terms of product improvements among key rivals and new players foraying into the market.
2. More Sensor driven data will drive Datafication
Nobody can deny that IoT (Internet of Things) is going to be a major trend in 2015. To create meaningful user experience, stakeholders will require Datafication and that can be achieved only when large connected IoT networks will employ device-to-device level data analytics. The required device-to-device data creation and collection is enabled by sensor-to-sensor level interactions. While it is not expected that sensor-to-sensor level data collection will outperform transactional data, but in 2015 I expect sensor driven data to gain more interest and a gradual market increase. New companies foraying into the business might try to focus their R&D on it to gain initial niche in the market.
3. Big data analytics as a data security tool will gain traction
Collected data, with novelties in collection methods, storage, transfer and content, is likely to outgrow the capabilities of the traditional security software systems sooner than expected. For many a firms, data assets are gradually increasing in stature as one of the most important competitive edges in the market. I expect SORs to gain more attention from stakeholders. Interestingly, firms and experts have started to look at big data analytics as a powerful tool that can help protect the data security infrastructure by identifying signals of persistent threat attacks. In 2015, we should be able to hear more market noises in this area.
4. Market landscape to continue to remain dynamic
Some new players will continue to fail, partly for approaching the market with nascent model or solutions that didn’t succeed and partly for failing to compete with the larger players in the same segments. Large customers seeking end-to-end enterprise solutions will continue to help the valuation of the market. Customers will continue to negotiate with vendors for solutions that require minimal in-house data scientist pool. We might see a few top companies try and go public and that should help the overall market sentiment, driving some degree of consolidation among geographically and functionally diversified smaller players. Also, look out for those big failure case studies in the year as well.
5. Business continuity and disaster recovery will gain prominence
With increasing focus on real-data systems, business stakeholders will ask more questions on how efficient data systems are, how strong the data recovery mechanisms are and how confident are vendors of ensuring business continuity of processes relying on data systems. I expect that in 2015, there will be targeted focus from vendors in selling business continuity as an integral part of the service and product portfolios.
6. Data governance will enter the board rooms as an agenda
Particularly for large companies, with more and more functions creating and accessing data systems, it is imperative that data governance practices are laid out to control the resulting chaos. Realizing this, CTOs/CIOs are likely to drive the agenda for bringing up comprehensive data governance practices. In 2015, this trend could go viral among large companies managing massive data systems.
7. Deep Learning will become the new buzzword among players
By employing neural networks, deep learning helps finding patterns in massive unstructured data of varied quality sets without manual programming for specific functions. A machine learning derivation, deep learning’s potential are yet to be majorly tested among real-life business situations. However, as with everything else in big data industry so far, the word will see traction among vendors and we can expect something innovative to hit the market in the second half of 2015.
8. Distributed Analytics will gain R&D attention
Distributed analytics model collects and analyses data in multiple places (HTAP systems, Hadoop/Spark sensors, BI systems, data warehouses) compared to the traditional layer-cake best-practice model. Arguably, the resulting model is more efficient and real-time. In 2015, enterprise architects are likely to spend considerable effort in developing the data interaction infrastructure for building these analysis hubs.
9. Hadoop vendors will continue to improvise
Vendors such as Pig, Hive, Hortonworks and Cloudera that have built their offerings on Hadoop and making it more manageable, will continue to improvise and differentiate their offerings. From a Pureplay strategy perspective, it is important that these vendors are not seen as mere Hadoop implementers. In 2015, these companies will face an ever increasing threat from next-gen technologies such as Apache Spark, also built on Hadoop. Whoever wins the race, the year will see Hadoop as a name gradually getting dwarfed by the new customized offerings in the market.
10. Big Data as a Service (BDaaS) will witness an early promising period
Google’s announcement of upcoming public access to Dataflow (internal big data service) last year has given life to big data as a service model. The service has merit as it bridges the storing and processing gap. Most likely, any BDaaS solution will have a PaaS, SaaS or IaaS layer. Other players that will gain momentum in 2015 will be Amazon Web Service’s Elastic MapReduce (EMR), Altiscale, Qubole and Ersatz Labs.