Скачать с ютуб Apache XTable (incubating): Interoperability among lakehouse table formats в хорошем качестве

Apache XTable (incubating): Interoperability among lakehouse table formats

Apache Hudi, Iceberg, and Delta Lake have emerged as leading open-source projects, providing decoupled storage with powerful primitives for transaction and metadata layers, commonly referred to as table formats, in cloud storage. When data is written to a distributed file system, these three formats are not significantly different. They all offer a table abstraction over a set of files, including a schema, commit history, partitions, and column stats. When engineers and organizations must choose a table format, they face a challenging decision. Each project has its rich set of features that may cater to different use cases. So, the question arises: do we really need to choose? Enter Apache XTable—an open-source project that provides omni-directional interoperability between lakehouse table formats. XTable doesn't introduce a new or separate format but offers abstractions and tools for translating table format metadata. This allows you to write data in any format of your choice and convert the source table format to one or more targets that can be consumed by the compute engine of your choosing. This presentation will showcase how XTable addresses the challenging problem of selecting a specific table format and the growing need for interoperability in today's lakehouse workloads.