Mar 29, 2023
Data Lineage Patterns with Tomas Kratky - AgileData #39
In this episode of the Agile Data N’ Info Podcast, host Shane Gibson sits down with Tomas Kratky, founder of Manta and a software engineer with over two decades of experience helping large enterprises master data lineage.
In this episode, the conversation explores the intricate world of data lineage and the shift toward active metadata. Tomas explains how lineage should move beyond just simple traceability or the mere collection of data to focus on “design lineage,” giving organizations full visibility into all possible system dependencies to enable safe and agile change management. By focusing on integrating and activating metadata, companies can embed insights directly into users’ daily workflows rather than relying on massive, often empty, centralized data catalogs.
The discussion also dives deep into the complexities of mapping metadata across diverse enterprise architectures, clarifying the immense challenge of parsing everything from custom SQL grammar to Excel sheets and legacy SAS macros. Tomas breaks down how automated, detailed lineage can unlock powerful use cases, such as proactively alerting users of data quality drops or significantly reducing compute costs by identifying and decommissioning unused data pipelines.
Whether you are curious about why lineage is often relegated to a feature rather than a standalone category, how the industry’s obsession with data self-service overlooks the “silver service” needs of everyday business users, or want to learn about the ongoing challenges facing open metadata sharing standards, this episode is packed with valuable insights for data professionals.
You can get in touch with Tomas on LinkedIn
If you want want to read a summary generated with GenAI, head over to:
https://agiledata.substack.com/i/207557902/google-notebooklm-briefing
If you want to download the transcript for the podcast, head over to:
https://agiledata.substack.com/i/207557902/transcript
Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/
Read more on the Agile Data Way of Working over at https://AgileDataGuides.com/
If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.
Subscribe: Apple Podcast | Spotify | Google Podcast | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser | Deezer | Podcast Addict |
No comments yet. Be the first to say something!