According to Alation’s State of Data Culture Report, 87% of employees attribute poor data quality to why most organizations fail to adopt AI meaningfully. Based on a 2020 study by McKinsey, high-quality data is crucial for digital transformations to propel an organization, past competitors.
As machine learning algorithms coding frameworks evolve rapidly, it’s safe to say the scarcest resource in AI is high-quality data at scale. High-quality data is the bottleneck.
Despite several findings on the importance of data in the AI industry, more than 90% of research papers in the AI domain are still model-centric. According to Andrew Ng, this is due to the difficulty in creating large datasets that can become generally recognized standards.
The fact is that the current threshold that Machine Learning has reached could only be breached by improving both the quality and quantity of data.
Thus the data-centric movement was born. The movement represents the recent transition from focusing on modeling to the underlying data used to train and evaluate models.
As a next step in the movement, today we’re excited to announce the Data-Centric AI Community — a place to discuss data quality for data science.
Photo by Papaioannou Kostas on Unsplash
Data-Centric AI is the approach to AI development that considers the training dataset as the centerpiece of the solution instead of the model.
Let’s take a step back and understand the hype on data-centric AI. Coined by Andrew Ng, data-centric AI emphasized the importance of focusing on data quality over algorithms and models. Further, deeplearning.ai and Landing AI announced the first-ever data-centric competition. Not only did it create awareness but also inverted the traditional competitions and asked to improve a dataset given a fixed model.
Finally, in 2021, the Data-Centric AI workshop was conducted to cultivate the DCAI community into a vibrant interdisciplinary field that tackles practical data problems. Several companies have adopted the approach and produced results. According to Landing AI, Some improvements from the adoption of a data-centric approach include:
With all the proven benefits in the industry, launching the DCAI community aims to complete the missing piece in the data-centric AI movement.
While the data-centric approach is still evolving and can span over various stages of a machine learning lifecycle, we’ve identified the most significant pain points amidst the data scientists and aim to focus on those in the DCAI Community.
We call them the 3 pillars of the DCAI community:
In addition, we have (and continue to) collate all the useful open-source tools, and resources on the 3 pillars, at the Awesome-Data-Centric-AI GitHub repository.
Data-Centric AI Community website
“What can I expect?” — I hear you ask.
At the Data-Centric AI Community, we believe that together, we can actively change the paradigm towards better data. We want to bring together experts from the industry and foster meaningful conversations.
Expect a regular calendar of events and content creation that will help you understand this approach better and allow you to become a data-centric ai evangelist. As we partner with experts in the industry, you will get the much-needed guidance directly from those who have already done what you plan to do.
Accelerating AI with improved data is at the core of what we do, and this open-source community is yet another step towards our meaningful journey. We invite you to be part of it — together, the possibilities are endless.
Fabiana Clemente is CDO at YData. Accelerating AI with improved data. YData provides the first data development platform for Data Science teams. 195
Coined by Andrew Ng in 2021, the concept of “Data-Centric AI” has taken both academia and industry by storm. It has given rise to hundreds of research publications, fostered the creation of special tracks and colloquiums in the most...
Machine Learning models are excellent tools to analyze large data sets and can have incredible accuracy in challenging tasks, from face recognition to credit scoring. Unfortunately, these models are not entirely free from bias and can...
Since Andrew Ng coined the term in 2021, the number of companies that identify themselves as providing data-centric AI tools has exploded. From synthetic data to data monitoring, companies all over the machine learning workflow have jumped...
