(Ernesto-Sevilla/Shutterstock)
The world is awash in data, and the pace of data generation is increasing. You don’t need a crystal ball to tell you that. But what will the new year bring in the field of big data analytics? We leave that to our panel of ad-hoc experts.
Data warehouses have been popular, especially in the cloud. But in 2023, expect more customers to move away from traditional data warehouses to real time data storage, says Jay Upchurch, an executive VP and CIO at SAS.
“In 2023, we will continue to see movement away from traditional data warehousing to storage options that support analyzing and reacting to data in real time. Organizations will lean into processing data as it becomes available and storing it in a user-friendly format for reporting purposes (whether that’s as a denormalized file in a data lake or in a key-value NoSQL database like DynamoDB). Whether a manufacturer monitoring streaming IoT data from machinery, or a retailer monitoring ecommerce traffic, being able to identify trends in real time will help avoid costly mistakes and capitalize on opportunities when they present themselves.”
In the old days, enterprises sought a single version of the truth. But in the new big data world, enlightened organizations realize that it’s not always that simple, according to Alan Jacobson, the CDAO for Alteryx.
“Often, there can be multiple right answers to a question depending on how one defines the parameters around that question. What’s often more important than a single version of truth, or one right answer, is the ability to communicate the context of the question – the ‘why.’ Based on that, companies can pull data in ways to understand and drive business results. We’ll see more companies become more analytically mature by asking better questions, recognizing the nuances in finding answers, and discovering their own insights instead of relying on the single version of truth. In the end, they will drive a culture of analytics. “
(ramcreations/Shutterstock)
The new generation of cloud data warehouses have grown quickly. But when users see the analytic technology that comes next, the cloud DWs will be relegated to niche technology status, predicts Tomer Shiran, CPO and co-founder of Dremio.
“In 2023 Snowflake will become more of a niche technology. With Snowflake’s costs increasing on average 71% year over year, based on their earnings report, customers are getting to a point where they can no longer afford to continue that kind of exponential increase in costs. Because of this, customers are going to be much more cautious about what they put in there, and will put up walls of approvals and rules regarding who’s allowed to use and access what…The demand to make data accessible and to become data driven is still there, and data’s still growing very fast. But, customers need systems that are able to do that at scale, and customers need them to be cost efficient. The industry is moving towards those types of systems.”
Dan Spurling, the SVP of product engineering at Teradata, comes bearing two gifts: one a prediction on digital twins, the other on data reduction.
“I believe there will be advances in the ML/AI evolution tied to digital twins or simulations; moving beyond just sensors that predict machine failure or buying propensities, and moving into predictions of economic markets, food production, population health, etc.,” he says. On data reduction “There is an exponentially increasing amount of data, but I believe we will see rise of solutions that deduce the meaningful bits of data from the overall mass of data collected, or even reduce the footprint of data using new technologies beyond current classic data storage techniques.”
Most of the data in the world is of the unstructured variety. Yet most analytics database are relational and are designed to crunch tabular data. What gives? For Frank Liu, the director of operations at Zilliz, the answer is clear: the world needs vector databases to unleash the value trapped in unstructured data.
“As businesses embrace the AI era and attempt to make full use of its benefits in production, there occurs a significant spike in the volume of unstructured data taking all sorts of forms that need to be made sense of. To cope with these challenges in extracting tangible value from unstructured data, vector databases–a new type of database management technology purpose-built for unstructured data processing–is on the rise and will take hold in years to come.”
(sutadimages/Shutterstock)
The slipshod nature of big data management has doomed many an advanced analytic or AI project to an early death. Data management tool vendors have responded with a cornucopia of tools, including data catalogs and other creations, but the lack of cohesion among these setse will cause them to lose traction in 2023, predicts Rex Ahlstrom, the CTO of Syniti (formerly BackOffice Associates).
“While each of these components is valuable on its own, they are far more potent together. Therefore, specialized technologies will not be sufficient to address the expanding problems in data management. Even the categories and standards that analysts use to classify market leaders are undergoing rapid change. To further emphasize the requirements and linked nature of these solutions, words like augmented, suggested and discovery are being used in front of traditional categories like data quality, integration and master data management. In 2023, it’s likely that established leaders will lose ground to innovators.”
In 2023, the shift from function-specific business models to data-centric ones will accelerate, predicts Eliud Polanco, the president of Fluree.
“For the past 20 years, business IT investments were focused on increasing productivity at the function level….We’ve reached a peak threshold of function-optimized productivity, and the new arena for competitive differentiation is out-smarting the competition, versus out-executing them. This requires putting data in the center and having all business functions be able to securely collaborate and leverage data coming from across all other functions. In this data-centric model, the data is the product, and the functions come to the data rather than the other way around.”
The data mesh concept will continue to grow in 2023. But if it wasn’t for misinformation about it, data meshes would be growing faster, says Jens Graupmann, SVP of product and innovation at Exasol.
(Oleksii Lishchyshyn/Shutterstock)
“In 2023, we anticipate even greater pressure on organizations to move faster and build resilient, agile data architectures that will push data teams towards data mesh implementations. However, despite the growing enthusiasm around data mesh, we do expect roadblocks due to misinformation. In order to move forward, misinformation needs to be eradicated so that data mesh can be successfully adopted at scale. For example, despite being marketed as such, you cannot buy a data mesh — it is not a technology. There is also still much discussion and confusion about how to prevent data meshes from exacerbating data silos, and whether or not data mesh and data fabric are actually the same thing. To overcome these challenges and move beyond any debates or uncertainties, companies must take responsibility for educating themselves to strengthen their understanding of what a data mesh is and how it can optimize their data management strategy.”
Our recommendation is to take a look at this prediction by Rakesh Jayaprakash, head of product management at Zoho.
“AI and ML models have been crucial in highlighting underlying correlations in data which is not obvious to human interpretation usually. In the next two to three years, these models will further evolve to suggest corrective action based on the analysis. Actionable insights will be accompanied by recommendations towards possible actions. Such recommendation engines will be highly vertical-specific and use case-specific to begin with before becoming vertical agnostic.”
Lloyd Adams, President at SAP North America, sees a low-code future for AI.
“Artificial intelligence will increasingly enable software development processes that are more proactively guided and written by other software. This will allow business users to create new applications using text prompts with the assistance of the application development tools. While this prospect may cause professional developers to feel anxious, the shift promises to create new opportunities within IT, rather than eliminate old ones. Software developers will become adept at enabling this evolution by learning how to provide the right prompts to an AI tool to generate the code that a no-code application developer will need. Also, generally, at a fundamental level AI, AR+VR and simulation software are going to rule. To support this necessary backbone, trends in improving compute network and storage are going to take an exponential leap in the next 3-5 years. So, tech changes will be driven at compute, storage and network level!”
Related Item:
Unleash the 2023 Big Data Predictions!