Monday Morning Data Chat

Latest Episodes:

#32 - Metadata-First Architecture (uncut version) May 24, 2021

Show notes

Joe joins us from the beach (please excuse the choppy audio on Joe's phone) to chat with Matt about metadata driven architecture. We discuss Raghu Murthy's fantastic talk at the Utah Data Engineering Meetup about metadata-first architecture, orchestration, and why metadata is going to play a central role in data engineering over the next several years.

Raghu's talk at the Utah Data Engineering Meetup: https://www.youtube.com/watch?v=Q2v1xO_ADns

Metadata-first architecture (Article at Datacoral): https://www.datacoral.com/blog/how-to-use-metadata-to-future-proof-your-data-stack/

#dataengineering #datascience #data

---------------------------------

TERNARY DATA

We are Matt and Joe, and we’re "recovering data scientists". Together, we run a data architecture company called Ternary Data. Ternary Data is not your typical data consultancy. Get no-nonsense, no BS data engineering strategy, coaching, and advice. Trusted by great companies, both huge and small.

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#31 - Let's Talk Data Engineering - Live Q&A on LinkedIn Live & YouTube May 17, 2021

Show notes

Let's talk data engineering! We did a live Monday Morning Data Chat, and answered listener questions in a live Q&A on LinkedIn Live and YouTube.

#dataengineering #datascience #data

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#30 - The State Of Data Engineering in 2021 (article review) May 10, 2021

Show notes

This week, we talk about this article from LakeFs on the State of Data Engineering in 2021. We give our candid take, and much more. Enjoy!

LakeFS article

Matt Turck's 2020 data landscape slide

#dataengineering #datascience #data

------------------------------------

Timestamps

Intro - 0:00
Proliferation of tools - 2:00
Data lakes and data warehouses - 7:30
Metadata and metastores - 9:30
Data science and analytics usability - 16:15
Organizational metadata - 18:05
Observations and predictions - 19:30
Tool consolidation in the data industry - 25:50
The rapidly evolving data space - 35:30

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#29 - Cloud Data Warehouse Anti-Patterns May 03, 2021

Show notes

Cloud data warehouses such as Snowflake, BigQuery, and Redshift are hugely popular. And rightfully so - they work great for modern data needs. However, people often make costly mistakes when trying to use them.

This week, we discuss some very common anti-patterns - and solutions for better usage - with cloud data warehouses. We cover general CDW concerns, and best practices for loading and querying data. Enjoy!

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#28 - The Open Data Ecosystem Apr 26, 2021

Show notes

This week, we discuss two articles related related to the Open Data Ecosystem. We give our thoughts on why interoperability is the killer feature of the new open data movement, some challenges with the open data ecosystem, and why it's important to always focus on your end users.

Articles discussed:

What is the Open Data Ecosystem and Why it’s Here To Stay

Choosing Open Wisely

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#27 - Should You Go To Graduate School For A Data Career? Apr 19, 2021

Show notes

This week, we answer a popular question - "Should I go to grad school to advance my data career?" We discuss the big considerations of graduate school, the pros and cons, potential programs for data science and data engineering, and whether bootcamps are a viable alternative to graduate school.

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#26 - Data Engineering Portfolio Projects Apr 13, 2021

Show notes

This week we discuss a VERY popular question we're often asked - "How do I create a data engineering project portfolio?"

We cover some ideas for what to include in your project portfolio, as well as some suggestions for making it visible to potential hiring managers and recruiters.

---------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg

#25 Is The Cloud A Fad? + Listener Q&A (LinkedIn Live) Mar 30, 2021

Show notes

This week we discuss a VERY popular question we're often asked - "How do I create a data engineering project portfolio?".

We cover some ideas for what to include in your project portfolio, as well as some suggestions for making it visible to potential hiring managers and recruiters.

This is a live Q&A on LinkedIn Live.

Note: Apologies in advance for Joe's mic, which cut out during the first part of the episode.

-------------------------------------

TERNARY DATA

Ternary Data Site - https://ternarydata.com

LinkedIn - https://www.linkedin.com/company/ternary-data/

YouTube - https://www.youtube.com/channel/UC3H60XHMp6BrUzR5eUZDyZg