AWS re:Invent 2018

Latest Episodes:

ANT309: Build and Govern Your Data Lakes with AWS Glue Dec 01, 2018

Show notes

As data volumes grow and customers store more data on AWS, they often have valuable data that is not easily discoverable and available for analytics. Learn how AWS Glue makes it easy to build and manage enterprise-grade data lakes on Amazon S3. AWS Glue can ingest data from variety of sources into your data lake, clean it, transform it, and automatically register it in the AWS Glue Data Catalog, making data readily available for analytics. Learn how you can set appropriate security policies in the Data Catalog and make data available for a variety of use cases, such as run ad-hoc analytics in Amazon Athena, run queries across your data warehouse and data lake with Amazon Redshift Spectrum, run big data analysis in Amazon EMR, and build machine learning models with Amazon SageMaker and AWS Glue. Additionally, Robinhood will share how they were able to move from a world of data silos to building a robust, petabyte scale data lake on Amazon S3 with AWS Glue. Robinhood is one of the fastest-growing brokerages, serving over five million users with an easy to use investment platform that offers commission-free trading of equities, ETFs, options, and cryptocurrencies. Learn about the design paradigms and tradeoffs that Robinhood made to achieve a cost effective and performant data lake that unifies all data access, analytics, and machine learning use cases.

ANT310: Architecting for Real-Time Insights with Amazon Kinesis Dec 01, 2018

Show notes

Amazon Kinesis makes it easy to speed up the time it takes for you to get valuable, real-time insights from your streaming data. In this session, we walk through the most popular applications that customers implement using Amazon Kinesis, including streaming extract-transform-load, continuous metric generation, and responsive analytics. Our customer Autodesk joins us to describe how they created real-time metrics generation and analytics using Amazon Kinesis and Amazon Elasticsearch Service. They walk us through their architecture and the best practices they learned in building and deploying their real-time analytics solution.

ANT311: NFL & Forwood Safety Deploy Analytics at Scale w/ Amazon QuickSight Dec 01, 2018

Show notes

Enabling interactive data and analytics for thousands of users can be expensive and challenging-from having to forecast usage, provisioning and managing servers, to securing data, governing access, and ensuring auditability. In this session, learn how Amazon QuickSight's serverless architecture and pay-per-session pricing enabled the National Football League (NFL) and Forwood Safety to roll out interactive dashboards to hundreds and thousands of users. Understand how the NFL utilizes embedded Amazon QuickSight dashboards to provide clubs, broadcasters, and internal users with Next Gen Stats data collected from games. Also, learn about Forwood's journey to enabling dashboards for thousands of Rio Tinto users worldwide, utilizing Amazon QuickSight readers, federated single sign-on, dynamic defaults, email reports, and more.

ANT312: Hadoop/Spark to Amazon EMR, Architect It for Security & Governance Dec 01, 2018

Show notes

Customers are migrating their analytics, data processing (ETL), and data science workloads running on Apache Hadoop/Spark to AWS in order to save costs, increase availability, and improve performance. In this session, AWS customers Airbnb and Guardian Life discuss how they migrated their workload to Amazon EMR. This session focuses on key motivations to move to the cloud. It details key architectural changes and the benefits of migrating Hadoop/Spark workloads to the cloud.

ANT316: Effective Data Lakes: Challenges and Design Patterns Dec 01, 2018

Show notes

Data lakes are emerging as the most common architecture built in data-driven organizations today. A data lake enables you to store unstructured, semi-structured, or fully-structured raw data as well as processed data for different types of analytics-from dashboards and visualizations to big data processing, real-time analytics, and machine learning. Well-designed data lakes ensure that organizations get the most business value from their data assets. In this session, you learn about the common challenges and patterns for designing an effective data lake on the AWS Cloud, with wisdom distilled from various customer implementations. We walk through patterns to solve data lake challenges, like real-time ingestion, choosing a partitioning strategy, file compaction techniques, database replication to your data lake, handling mutable data, machine learning integration, security patterns, and more.

ANT322: High Performance Data Streaming with Amazon Kinesis: Best Practices Dec 01, 2018

Show notes

Amazon Kinesis makes it easy to collect, process, and analyze real-time, streaming data so you can get timely insights and react quickly to new information. In this session, we dive deep into best practices for Kinesis Data Streams and Kinesis Data Firehose to get the most performance out of your data streaming applications. Comcast uses Amazon Kinesis Data Streams to build a Streaming Data Platform that centralizes data exchanges. It is foundational to the way our data analysts and data scientists derive real-time insights from the data. In the second part of this talk, Comcast zooms into how to properly scale a Kinesis stream. We first list the factors to consider to avoid scaling issues with standard Kinesis stream consumption, and then we see how the new fan-out feature changes these scaling considerations.

ANT323: Build Your Own Log Analytics Solutions on AWS Dec 01, 2018

Show notes

With Amazon Elasticsearch Service's simplicity comes a multitude of opportunity to use it as a back end for real-time application and infrastructure monitoring. With this wealth of opportunities comes sprawl - developers in your organization are deploying Amazon Elasticsearch Service for many different workloads and many different purposes. Should you centralize into one Amazon Elasticsearch Service domain? What are the tradeoffs in scale and cost? How do you control access to the data and dashboards? How do you structure your indexes - single tenant or multi-tenant? In this session, we'll explore whether, when, and how to centralize logging across your organization to minimize cost and maximize value and learn how Autodesk has built a unified log analytics solution using Amazon Elasticsearch Service.

ANT324: Amazon Athena: What's New and How SendGrid Innovates Dec 01, 2018

Show notes

Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run. In this session, we live demo exciting new capabilities the team have been heads down building. SendGrid, a leader in trusted email delivery, discusses how they used Athena to reinvent a popular feature of their platform.

ANT327: Best Practices to Secure Data Lake on AWS Dec 01, 2018

Show notes

As customers are looking to build Data lakes to AWS, managing security, catalog and data quality becomes a challenge. Once data is put on Amazon S3, there are multiple processing engines to access it. This could be either through a SQL interface, programmatic, or using API. Customers require federated access to their data with strong controls around Authentication, Authorization, Encryption, and Audit. In this session, we explore the major AWS analytics services and platforms that customers can use to access data in the data Lake and provide best practices on securing them.

ANT328: How Instacart's Catalog Flourished While Hyper-Growing Dec 01, 2018

Show notes

In this session, we learn how Instacart reimagined its catalog data processing pipeline to utilize Snowflake, the data warehouse built for the cloud. Instacart grew from a hand-entered catalog to one that processes billions of data points daily. Keeping pace with customer demand prompted Instacart to take an entirely new approach to addressing the unique challenges of grocery catalog curation. Through Snowflake's unique architecture, which separates compute from storage, Instacart has increased their ability to quickly scale while improving the accuracy, traceability, and quality of their reporting. In turn, better information leads to offering more customized grocery catalog options that delight their customers. This session is brought to you by AWS partner, Snowflake Computing.

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected

Our TOPPODCAST Picks

Follow Us

Stay Connected

AWS re:Invent 2018

Related Podcasts

1

2

3

4

5

6