Скачать с ютуб Data, Metadata, GenAI (IEEE Seminar, the University of Auckland) в хорошем качестве

Data, Metadata, GenAI (IEEE Seminar, the University of Auckland)

Data is the lifeblood of business and economy. It is also the unsung hero behind AI & ML. ChatGPT was trained on 300 billion words that span across Wikipedia, books, and the general web. It’s this proprietary stash of data that gave ChatGPT human-like intelligence and enables it to stand out from the sea of Large Language Models (LLM) released in the past 12 months. Managing this “big data” used to be a major technical challenge. Fortunately, advancements in data infrastructure, such as Spark, Ray, Kafka, and Presto, in the past decade have enabled companies and individuals to store, process, and query astronomical amounts of data at a scale. However, the ease of data creation has led to a different kind of challenge—Data Governance, namely the ability to trust the myriad of raw and derived data that are poorly documented. Many companies have turned to metadata—data about data, as the solution to this problem that involves technology, people, and processes. In this talk, we’ll share our experience of solving various Data Governance challenges at LinkedIn and Metaphor using metadata, and how we leverage Generative AI to supercharge the process.