Making sure that your data is organized is top of the list in the recently released AWS Blueprint Executive Blueprint for Enterprise AI Transformation and highlights the necessity of revising the strategy for data for the future. “As large foundation models and language models are made available through out-of-the-box applications, the difference is not based on the actual model but rather on the design, content and access to the data behind this model, ” the blueprint’s creators say.
This implies opening up data from a range of newly discovered or underutilized sources. “Unlike traditional systems which rely mostly on structured information, genAI demands comprehensive access to all types of data – including multimodal and unstructured formats like audio, video text, code and more and access in real-time.”
Along with the necessity to control data comes the requirement to change the architecture and establish solid control in place. This is the issue, but. Just 26 per cent of Chief Data Officers are convinced their data-related capabilities will be able to enable new revenue streams based on AI, according to a survey by the IBM Institute for Business Value. They have a difficult time using their data to support AI. The biggest data obstacles they have to overcome are accessibility, integrity, completeness, as well as accuracy and consistency.
It’s interesting to note that the relationship that exists between AI and data flow in both ways. Not only is having the appropriate amount of data available at the right moment crucial to AI, but AI will also make sure that the data is utilised to its maximum potential.
As of now, “agentic AI is the technology that has the greatest impact on data management in the present,” said Ashwin Patil, the principal, data engineering and analytics practice manager of Deloitte Consulting. “Most companies handle huge amounts of structured and unstructured information using GenAI. Agentic AI greatly enhances or automates an earlier manual procedure of analysing data, carrying out quality checks, creating business rules and connecting data across different applications.”
LLMs are able to be integrated into a data infrastructure, according to Jim Liddle, AI entrepreneur and former chief innovation officer for data analytics as well as AI in Nasuni. “This surpasses the conventional AI assistants that are built on top of traditional file systems. Instead, it creates an understanding layer of semantics which fundamentally changes the way in which the data that is not structured can be stored, accessed, classified, and then taken action on.”
So, “platforms don’t simply sync files; they’ll be able to interpret the context of content, information and patterns of usage,” Liddle predicted. “With accessibility to every data using an unifying namespace, companies can access dormant information that was previously stored in file shares or archives. This is a change from passive storage for files to business-aware, intelligent systems for data.”
Additionally, there is a significant benefit for the people who manage the data: “The effect is primarily on those who are responsible for putting the information together and analysing what this data is, and coordinating the movement of that data,” said Aron Semle, the chief technology officer of HighByte.
For instance, LLM capabilities are being integrated into projects that manage data to assist users in answering questions such as ‘what’s the connection between these two sets of data and assist in creating the SQL search.”
As a result, “LLMs aid experts in speeding up the process of discovering how to deliver, transfer, as well as troubleshoot the data between various data management platforms while lowering the threshold for entry for those with less experience,” Semle added. “One benefit we’re seeing with LLMs is that they’re providing data to a wider population, which can give users greater access to their information, and allow them to make more effective use of it.”
So using an AI-first data strategy can pay off in numerous ways. The AWS authors suggest five steps for properly integrating data to create an AI-focused system.
Perform a thorough data analysis: Create usage cases that make use of the low-hanging fruit, and quickly prove that you are successful: “For example, reducing the handling time of service calls by 30 per cent,” the authors suggest. “Unify all relevant data into an efficient, secure storage system.
solution and put in place appropriate guardrails as soon as possible.”
Modernise your data structure: “This is done by breaking down silos as well as
The definition of the data product owners of key business units. Establish common
Governance structures for governance.”
Develop internal capabilities: Train Data teams in rapid engineering,
vector databases, and accountable AI in the process of training team members across the
Organisation is focused on AI basics and responsible usage to maximise adoption.”
Be sure that there’s a person within the loop: A person in the loop, in conjunction with the utilisation of LLMs to give feedback, could be used to “continuously examine and improve the quality of data and improve the performance of models.”
Track the progress made: “Track business outcomes, operational performance indicators, and trust and data metrics like retrieval precision rate, the factual consistency score and the number of active daily users.”