Technology
**AWS Bites**

AWS Bites is the show where we answer questions about AWS! This show is brought to you be Eoin Shanaghy and Luciano Mammino, certified AWS experts.

Latest Episodes:

65. Solving SQS and Lambda concurrency problems Jan 26, 2023

Show notes

In this episode of the AWS Bites Podcast, we dive into the serverless pattern of using AWS Lambda together with SQS. We explain the basics of both Lambda and SQS for those who may not be familiar with them. We talk about how we use Lambda, a Function as a Service offering in AWS, to write our own functions and have AWS run them in response to certain events. And we also discuss SQS, a scalable and managed queuing system available on AWS, which we use to offload work to background workers.

We delve into how the two services work together through the use of "Event Source Mapping" in Lambda, which polls our SQS queue and makes synchronous Lambda invocation requests when messages are available. We also mention how this feature provides us with the ability to control batch size and window, as well as specify filters to save execution time and cost. But we also share one of the limitations we faced when using SQS and Lambda together which was the lack of control over concurrency and the potential for excessive throttling.

But recently, AWS has released a new feature called "SQS maximum concurrency support" which allows us to specify a maximum number of invocations for an Event Source Mapping. This solves the problem of excessive throttling and eliminates the need to use reserved concurrency. It also allows for more control over concurrency when using multiple Event Source Mappings with the same function. We explain how this new feature has improved our workflow and made it much more efficient.

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem, an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

AWS Lambda
Amazon SQS
Series of blog posts by Zac Charles covering the original problem and the solution in detail
Official AWS blog post with the announcement of the maximum concurrency feature
Our previous episode on SQS
Our video-series on AWS event services

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter:

#AWS #rust #lambda

64. How do you write Lambda Functions in Rust? Jan 20, 2023

Show notes

Are you curious about using Rust to write AWS Lambda functions?

In this episode of AWS BItes, we will be discussing the pros and cons of using Rust for serverless applications. With Rust, you'll be able to take advantage of its fast performance and memory efficiency. Plus, its programming model makes it easy to write safe and correct code. However, Rust is not a native runtime for Lambda, but rather a library that implements a custom runtime built and maintained by AWS. This custom runtime is built on top of the Tokio async runtime and even has a built-in middleware engine, which allows for easy hook-in of reusable logic and building your own middleware.

But what if you're new to Rust? Don't worry, we'll also be walking you through the steps on how to write your first Lambda in Rust. From cargo-lambda to the serverless framework plugin for Rust, we'll be sharing different alternatives for building and deploying your Rust-based Lambda functions.

So join us on this journey as we explore the exciting world of Rust and Lambda.

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem, an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

Rust 1.0 original announcement: https://blog.rust-lang.org/2015/05/15/Rust-1.0.html
The Rust programming language home page: https://www.rust-lang.org/
Firecracker runtime: https://firecracker-microvm.github.io
Luciano's Twitch profile: https://twitch.tv/loige
AWS Rust runtime library: https://github.com/awslabs/aws-lambda-rust-runtime
Tokio, Rust async runtime: https://tokio.rs/
Example of how to enable the tracing middleware in Lambda: https://github.com/awslabs/aws-lambda-rust-runtime/blob/99dba6447253ac87cf3cefeb2ba130b50514f9df/examples/http-tower-trace/src/main.rs#L4
Rustup tool to install the Rust toolchain: https://rustup.rs/- Reference article on how to write a Lambda in Rust using cargo-rust: https://blog.scanner.dev/getting-started-with-serverless-rust-in-aws-lambda/
Cargo-lambda, a cargo extension that helps with writing, running, testing, and deploying lambdas written in Rust: https://www.cargo-lambda.info
Serverless framework plugin for Rust: https://www.serverless.com/plugins/serverless-rust
Eoin's article on Container Image Support in AWS Lambda: https://dev.to/eoinsha/container-image-support-in-aws-lambda-deep-dive-2keh- AWS SDK for Rust: https://github.com/awslabs/aws-sdk-rust
Coding challenges to learn rust: https://exercism.org/

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter:

#AWS #rust #lambda

63. How to automate transcripts with Amazon Transcribe and OpenAI Whisper Jan 13, 2023

Show notes

We built a Step Function that allows us to generate high-quality transcripts for AWS Bites podcast!

After evaluating different approaches and technologies we ended up using Amazon transcribe and OpenAI whisper. They both have their pros and cons but combined together they gave us everything we were looking for with quite a good degree of accuracy!

In this episode, we describe our use case, our research, and how eventually we did go about productionizing our final solution.

If you run a podcast and you would like to do something similar, we have open source our solution. It's called PodWhisperer and you can find it on GitHub: https://github.com/fourTheorem/podwhisperer .

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem, an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

PodWhisperer on GitHub: https://github.com/fourTheorem/podwhisperer/
Amazon Transcribe: https://aws.amazon.com/transcribe
OpenAI Whisper announcement blog post: https://openai.com/blog/whisper/

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter:

#AWS #speechtotext #openai

62. Top 3 re:Invent 2022 announcements Dec 09, 2022

Show notes

re:Invent 2022, the biggest AWS conference of the year is just over and there were tons of interesting announcements: many new features and some interesting new AWS products!

But we are not going to bother you with yet another walkthrough of all of them.

In this episode of AWS Bites podcast we just discuss our top 3 announcements and explained what we liked and what could have made them even better! We will talk about EventBridge Pipes, Step Functions Distributed Map, and Application Composer.

What are your favorite announcements? Let us know on Twitter or in the comments!

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem, an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

Previous Episode on EventBridge: https://awsbites.com/23-what-s-the-big-deal-with-eventbridge/
Previous episode on cost: https://awsbites.com/61-how-do-i-control-aws-cost/
EventBridge Pipes Official Announcement: https://aws.amazon.com/blogs/aws/new-create-point-to-point-integrations-between-event-producers-and-consumers-with-amazon-eventbridge-pipes
Step Functions Distributed Map Official Announcement: https://aws.amazon.com/blogs/aws/step-functions-distributed-map-a-serverless-solution-for-large-scale-parallel-data-processing/
Application Composer Official Announcement: https://aws.amazon.com/blogs/compute/visualize-and-create-your-serverless-workloads-with-aws-application-composer/
SNS Payload Message Filtering: https://aws.amazon.com/blogs/compute/visualize-and-create-your-serverless-workloads-with-aws-application-composer/
Verified Access: https://aws.amazon.com/about-aws/whats-new/2022/11/aws-verified-access-preview/
CloudWatch Cross-Account Observability: https://aws.amazon.com/blogs/aws/new-amazon-cloudwatch-cross-account-observability/
Official AWS Blog post with Top AWS re:Invent 2022 announcements: https://aws.amazon.com/blogs/aws/top-announcements-of-aws-reinvent-2022/

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter:

#AWS #reinvent #announcements

61. How do I control AWS costs? Dec 02, 2022

Show notes

Let's face it: when it comes to AWS, cost is one of the scariest topics!

Why? Mostly because the underlying model can get very complex.

There are too many variables and ultimately it's just hard to predict how much is a given workload going to cost you on AWS. Are going to be bankrupted by this unpredictable cost? Probably not!

In this episode, we share some suggestions and tools on how to approach cost when going to AWS. It's not a simple topic, but it's something you need to embrace, learn and get confident with. With a bit of effort, cost will not be so scary anymore and you'll be able to take advantage of all the awesome services and features of AWS without being so worried about cost!

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem, an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

Amazon FREE Tier: https://aws.amazon.com/free/free-tier-faqs/
AWS Activate Program: https://aws.amazon.com/activate/
FourTheorem on MAP (Migration Acceleration Program): https://fourtheorem.com/solutions/migration-acceleration/
What are the benefits of tags (past episode): https://awsbites.com/32-what-are-the-benefits-of-tags/
AWS Horror stories (past episode): https://awsbites.com/8-what-are-your-aws-horror-stories/
AWS Well Architected Framework cost optimisation pillar https://docs.aws.amazon.com/wellarchitected/latest/cost-optimization-pillar/welcome.htm
Hands-on labs on cost calculation: https://www.wellarchitectedlabs.com/cost/
AWS Pricing calculator: https://calculator.aws

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

#AWS #cost #expense

60. What is AWS Lambda? Nov 25, 2022

Show notes

AWS Lambda is one of the most famous AWS services these days. If you are just starting with your cloud journey you might be confused about what Lambda actually is, what are the limitations, and when you should be using it or not.

In this episode, we provide a beginner-friendly introduction to Lambda and summarise everything there’s to know about it: when to use it and when not, differences with containers, the pricing model, limitations, and integrations.

By the end of this episode, we will also chime in with some of our opinions and share whether we believe that Lambda is the future of cloud computing or not!

💰 SPONSORS 💰

AWS Bites is sponsored by fourTheorem (https://fourtheorem.com/). fourTheorem is an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

The original announcement of AWS Lambda from 2014: https://aws.amazon.com/about-aws/whats-new/2014/11/13/introducing-aws-lambda/
Using Lambda for High-Performance Computing (AWS Blog Post): https://aws.amazon.com/blogs/hpc/a-serverless-architecture-for-high-performance-financial-modelling/
Lambda vs Fargate vs EC2 pricing (article by Eoin Shanaghy): https://fourtheorem.com/why-aws-lambda-pricing-has-to-change-for-the-enterprise/
Our previous episode of what serverless v2 will look like: https://awsbites.com/59-what-will-serverless-2-0-look-like/
AWS Lambda tiered pricing announcement by AWS: https://aws.amazon.com/blogs/compute/introducing-tiered-pricing-for-aws-lambda/
Understanding AWS Lambda Scaling and Throughput (Video by Marcia Villalba & Julian Wood): https://www.youtube.com/watch?v=wenqZqNOVZw
Keet by Holepunch (P2P messaging app): https://keet.io/
SocketSupply (a company building a P2P platform): https://socketsupply.co/

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on Twitter: - https://twitter.com/eoins - https://twitter.com/loige

#AWS #serverless #lambda

59. What will Serverless 2.0 look like? Nov 18, 2022

Show notes

The definition of serverless has already changed in the few years since it first emerged. There have been many success stories using serverless - in startups and the enterprise. But what comes next?

In this episode, we will clarify our definition of Serverless, what are the main challenges with it today, and speculate on what we believe will come next! By the end of this episode, you’ll have heard our thoughts and predictions on what Serverless 2.0 will look like. We will also reveal who we think will be the main challenger to AWS for domination of serverless as it goes mainstream!

AWS Bites is sponsored by fourTheorem. fourTheorem is an AWS Consulting Partner offering training, cloud migration, and modern application architecture.

In this episode, we mentioned the following resources:

The SLIC Watch project for automated observability best practices
Vercel
Serverless cloud
SST (Serverless Stack)
Darklang
FaunaDB
Cloud Firestore
Cockroach DB serverless
Digital Ocean
Our series of live coding streams where we build a serverless WeTransfer clone on AWS from scratch

You can listen to AWS Bites wherever you get your podcasts:

Apple Podcasts: https://podcasts.apple.com/us/podcast/aws-bites/id1585489017
Spotify: https://open.spotify.com/show/3Lh7PzqBFV6yt5WsTAmO5q
Google: https://podcasts.google.com/feed/aHR0cHM6Ly9hbmNob3IuZm0vcy82YTMzMTJhMC9wb2RjYXN0L3Jzcw==
Breaker: https://www.breaker.audio/aws-bites
RSS: https://anchor.fm/s/6a3312a0/podcast/rss

Do you have any AWS questions you would like us to address?
Leave a comment here or connect with us on Twitter:
- https://twitter.com/eoins
- https://twitter.com/loige

#AWS #serverless #future

58. What can kitties teach us about AWS? Nov 11, 2022

Show notes

Building actual projects is a great way to consolidate our understanding of AWS. In this episode, we present 4 different project ideas to explore services and concepts in the space of web application development, machine learning, and data science.

Ok, you are probably wondering where kitties come into the equation here. Every one of these 4 project ideas involves kitties! 🐱

We can learn stuff and have some fun too!

In this episode, we mentioned the following resources:

- Our sponsor, fourTheorem: https://fourtheorem.com/

- The Cat Detector workshop https://github.com/fourTheorem/workshops/tree/master/cat-detector

- AI as a Service (book): https://www.manning.com/books/ai-as-a-service

- Kaggle dataset cat vs dog: https://www.kaggle.com/datasets/shaunthesheep/microsoft-catsvsdogs-dataset

- The best website in the world: https://http.cat

- The true meaning of the 418 HTTP status code: https://http.cat/418

- The cat breeds dataset on Kaggle: https://www.kaggle.com/datasets/ma7555/cat-breeds-dataset

- Our series of live coding streams where we build a serverless WeTransfer clone on AWS from scratch: https://www.youtube.com/playlist?list=PLAWXFhe0N1vI1_z-06EzJ22pz95_gBrId

You can listen to AWS Bites wherever you get your podcasts: