AWS Bites

153. LLM Inference with Bedrock

If you’re curious about building with LLMs, but you want to skip the hype and learn what it takes to ship something reliable in production, this episode is for you.We share our real-world experience building AI-powered apps and the gotchas you hit after the demo: tokens and cost, quotas and throttling, IAM and access friction, marketplace subscriptions, and structured outputs that do not break your JSON parser.We focus on Amazon Bedrock as AWS’s managed inference layer: how to get started with the current access model, how to choose models, how pricing works, and what to watch for in production.We also go deep on structured outputs: constrained decoding, schema design that improves output quality, and how to avoid “grammar compilation timed out”. In this episode, we mentioned the following resources: * fourTheorem: Bedrock structured outputs guide https://fourtheorem.com/amazon-bedrock-structured-outputs/ [https://fourtheorem.com/amazon-bedrock-structured-outputs/] * Amazon Bedrock https://aws.amazon.com/bedrock/ [https://aws.amazon.com/bedrock/] * Bedrock docs https://docs.aws.amazon.com/bedrock/latest/userguide/ [https://docs.aws.amazon.com/bedrock/latest/userguide/] * Bedrock pricing https://aws.amazon.com/bedrock/pricing/ [https://aws.amazon.com/bedrock/pricing/] * Structured outputs https://docs.aws.amazon.com/bedrock/latest/userguide/structured-outputs.html [https://docs.aws.amazon.com/bedrock/latest/userguide/structured-outputs.html] * Cross-region inference https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html [https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html] * Quotas https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html [https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html] * Throttling help https://repost.aws/knowledge-center/bedrock-throttling-error [https://repost.aws/knowledge-center/bedrock-throttling-error] * Prompt caching https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html [https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html] * Troubleshooting error codes https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html [https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html] Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn: - ⁠https://twitter.com/eoins⁠ [https://twitter.com/eoins] | ⁠https://bsky.app/profile/eoin.sh⁠ [https://bsky.app/profile/eoin.sh] | ⁠https://www.linkedin.com/in/eoins/⁠ [https://www.linkedin.com/in/eoins/] - ⁠https://twitter.com/loige⁠ [https://twitter.com/loige] | ⁠https://bsky.app/profile/loige.co⁠ [https://bsky.app/profile/loige.co] | ⁠https://www.linkedin.com/in/lucianomammino/ [https://www.linkedin.com/in/lucianomammino/]

152. Exploring Lambda Durable Functions

AWS Lambda is fantastic for small, stateless code on demand. But when your “function” starts looking like a workflow (retries, backoff, long waits, human approvals, callbacks), classic Lambda patterns can feel like a fight: 15-minute max runtime, no built-in state, and orchestration glue everywhere (Step Functions, queues, schedules, and state you did not want to own). In this episode of AWS Bites, Eoin and Luciano explore AWS Lambda Durable Functions, announced at re:Invent 2025. It’s still Lambda (same runtimes and scaling), but with durable execution superpowers: named steps, automatic checkpointing, and the ability to suspend and resume from a safe point without redoing completed work. We unpack the replay/resume model under the hood, when this approach shines, and the gotchas (determinism, idempotency, replay-aware logging, debugging resumed runs). To make it real, we share how we rebuilt PodWhisperer v2 using Durable Functions to orchestrate a GPU-powered WhisperX pipeline, LLM refinement, speaker naming, and caption generation. In this episode, we mentioned the following resources: * AWS announcement blog post: https://aws.amazon.com/blogs/aws/build-multi-step-applications-and-ai-workflows-with-aws-lambda-durable-functions/ [https://aws.amazon.com/blogs/aws/build-multi-step-applications-and-ai-workflows-with-aws-lambda-durable-functions/] * Durable Functions best practices: https://docs.aws.amazon.com/lambda/latest/dg/durable-best-practices.html [https://docs.aws.amazon.com/lambda/latest/dg/durable-best-practices.html] * The replay model deep dive (Dev.to): https://dev.to/aws/the-replay-model-how-aws-lambda-durable-functions-actually-work-2a79 [https://dev.to/aws/the-replay-model-how-aws-lambda-durable-functions-actually-work-2a79] * Build workflows that last (Dev.to): https://dev.to/aws/aws-lambda-durable-functions-build-workflows-that-last-3ac7 [https://dev.to/aws/aws-lambda-durable-functions-build-workflows-that-last-3ac7] * Testing Durable Functions in TypeScript (Dev.to): https://dev.to/aws/testing-aws-lambda-durable-functions-in-typescript-5bj2 [https://dev.to/aws/testing-aws-lambda-durable-functions-in-typescript-5bj2] * Developing Durable Functions with AWS SAM (Dev.to): https://dev.to/aws/developing-aws-lambda-durable-functions-with-aws-sam-ga9 [https://dev.to/aws/developing-aws-lambda-durable-functions-with-aws-sam-ga9] * Hands-on notes: https://www.andmore.dev/blog/lambda_durable_functions/ [https://www.andmore.dev/blog/lambda_durable_functions/] * PodWhisperer (open source): https://github.com/fourTheorem/podwhisperer/ [https://github.com/fourTheorem/podwhisperer/] * WhisperX: https://github.com/m-bain/whisperX [https://github.com/m-bain/whisperX] Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn: - https://twitter.com/eoins [https://twitter.com/eoins] | https://bsky.app/profile/eoin.sh [https://bsky.app/profile/eoin.sh] | https://www.linkedin.com/in/eoins/ [https://www.linkedin.com/in/eoins/] - https://twitter.com/loige [https://twitter.com/loige] | https://bsky.app/profile/loige.co [https://bsky.app/profile/loige.co] | https://www.linkedin.com/in/lucianomammino/ [https://www.linkedin.com/in/lucianomammino/]

151. EC2 ❤️ Lambda - Lambda Managed Instances

AWS just made Lambda… less serverless. Lambda Managed Instances (Lambda MI) brings managed EC2 capacity into Lambda, and it changes the rules: environments stay warm, a single environment can handle multiple concurrent invocations, and scaling becomes proactive and asynchronous instead of “spin up on demand when traffic hits.”In this episode of AWS Bites, Eoin and Luciano break down what Lambda MI unlocks (and what it costs): fewer traditional cold starts, but a new world of capacity planning, headroom, and potential throttling during fast spikes. We compare it to Default Lambda, explain how the new scaling signals work, and what “ACTIVE” really means when publishing can take minutes on a new capacity provider.To make it real, we built a video-processing playground: an API, a CPU-heavy processor, and a Step Functions workflow that scales up before work and back down after. We share the practical lessons, the rough edges (regions, runtimes, mandatory VPC, minimum 2 GB + 1 vCPU, concurrency pitfalls), and the pricing reality: requests + EC2 cost + a 15% management fee. In this episode, we mentioned the following resources: * Lambda Managed Instances official docs: https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html [https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances.html] * Our example repo (video processing playground): https://github.com/fourTheorem/lambda-mi [https://github.com/fourTheorem/lambda-mi] * Concurrency mental model reference (Vercel Fluid Compute): https://vercel.com/fluid [https://vercel.com/fluid] * Lambda MI Node.js runtime best practices (concurrency considerations): https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-nodejs-runtime.html [https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-nodejs-runtime.html] Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn: - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/eoins⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbWlYNEdBeDBCSU56N3U1M0o1NDF0dS1sbkdRd3xBQ3Jtc0tra2RBajhnX0lBQ0dRVjZWMFZtNUdiX0JKVnZiVEdHZFBIcmtiSWpIU0Nvelc1UnliZDNZZ1hrS19tUUdtOHVwUTFzeTVrSkRuN2dwZDBKUXJlcmc5enY5YkJoUndoNEhIYVlsLUJKMmZRdUh2bVR4bw&q=https%3A%2F%2Ftwitter.com%2Feoins&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/eoin.sh⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/eoin.sh] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/eoins/⁠⁠⁠⁠ [https://www.linkedin.com/in/eoins/] ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.linkedin.com/in/eoins/]- ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/loige⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa3d0X2t5cUtreE02S2xuV1ZMTWNNMGdkamdqQXxBQ3Jtc0ttbDhQN2JicGhXOGxVX2Zqa1FLVVFKNERkY2VzUThBU3JONmdaM1dwWkVxRG5NQjdYeHU5TUxCWmxKSmZVclZLUXBSb2JGdkZUTnQzSEdpTUd1RDhGT3Y1LTR0TU5YUVZQbUNYOFJxRi1ZZTkxcGhkdw&q=https%3A%2F%2Ftwitter.com%2Floige&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/loige.co⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/loige.co] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/lucianomammino/ [https://www.linkedin.com/in/lucianomammino/]

150. Exploring All-New ECS Managed Instances (MI) Mode

Love AWS Fargate, but occasionally hit the “I need more control” wall (GPUs, storage, network bandwidth, instance sizing)? In this episode of AWS Bites, Eoin and Luciano put the brand-new Amazon ECS Managed Instances (ECS MI) under the microscope as the “middle path” between Fargate simplicity and ECS on EC2 flexibility. We unpack what ECS MI actually is and where it fits in the ECS spectrum, especially how it changes the way you think about clusters and capacity providers. From there we get practical: we talk through the pricing model (EC2 pricing with an additional ECS MI fee that can be a bit counterintuitive if you rely heavily on Reserved Instances or Savings Plans), and we share what it feels like to finally get GPU support in an experience that’s much closer to Fargate than to “full EC2 fleet management”. To make it real, we walk through what we built: a GPU-enabled worker that transcribes podcast audio using OpenAI Whisper, including the end-to-end setup in CDK (roles, capacity provider wiring, task definitions, and service configuration). Along the way we call out the rough edges we ran into, like configuration options that look like they might enable Spot-style behavior, and the operational realities you should expect, such as tasks taking roughly 3–4 minutes to start when ECS needs to provision fresh capacity. We close by mapping out the workloads where ECS MI shines (queue-driven GPU jobs, HPC-ish compute, tighter storage/network control) and the scenarios where it’s probably the wrong choice, like when you need custom AMIs, SSH access, or stricter isolation guarantees. In this episode, we mentioned the following resources: * Amazon ECS Managed Instances: ⁠https://aws.amazon.com/ecs/managed-instances/⁠ [https://aws.amazon.com/ecs/managed-instances/] * ECS Managed Instances documentation: ⁠https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ManagedInstances.html⁠ [https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ManagedInstances.html] * Amazon Bottlerocket (what it is): ⁠https://aws.amazon.com/bottlerocket/⁠ [https://aws.amazon.com/bottlerocket/] * Our CDK ECS MI template: ⁠https://github.com/fourTheorem/cdk-ecs-mi-template⁠ [https://github.com/fourTheorem/cdk-ecs-mi-template] * Ep 42. How do you containerise and run your API with Fargate?: ⁠https://awsbites.com/42-how-do-you-containerise-and-run-your-api-with-fargate/⁠ [https://awsbites.com/42-how-do-you-containerise-and-run-your-api-with-fargate/] * Ep 72. How do you save cost with ECS?: ⁠https://awsbites.com/72-how-do-you-save-cost-with-ecs/⁠ [https://awsbites.com/72-how-do-you-save-cost-with-ecs/] * Ep 10. Lambda or Fargate for containers?: ⁠https://awsbites.com/10-lambda-or-fargate-for-containers/⁠ [https://awsbites.com/10-lambda-or-fargate-for-containers/] * Ep 38. How do you choose the right compute service on AWS?: ⁠https://awsbites.com/38-how-do-you-choose-the-right-compute-service-on-aws/⁠ [https://awsbites.com/38-how-do-you-choose-the-right-compute-service-on-aws/] * Ep 143. Is App Runner better than Fargate?: ⁠https://awsbites.com/143-is-app-runner-better-than-fargate/⁠ [https://awsbites.com/143-is-app-runner-better-than-fargate/] Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn: - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/eoins⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbWlYNEdBeDBCSU56N3U1M0o1NDF0dS1sbkdRd3xBQ3Jtc0tra2RBajhnX0lBQ0dRVjZWMFZtNUdiX0JKVnZiVEdHZFBIcmtiSWpIU0Nvelc1UnliZDNZZ1hrS19tUUdtOHVwUTFzeTVrSkRuN2dwZDBKUXJlcmc5enY5YkJoUndoNEhIYVlsLUJKMmZRdUh2bVR4bw&q=https%3A%2F%2Ftwitter.com%2Feoins&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/eoin.sh⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/eoin.sh] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/eoins/⁠⁠⁠ [https://www.linkedin.com/in/eoins/] ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.linkedin.com/in/eoins/]- ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/loige⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa3d0X2t5cUtreE02S2xuV1ZMTWNNMGdkamdqQXxBQ3Jtc0ttbDhQN2JicGhXOGxVX2Zqa1FLVVFKNERkY2VzUThBU3JONmdaM1dwWkVxRG5NQjdYeHU5TUxCWmxKSmZVclZLUXBSb2JGdkZUTnQzSEdpTUd1RDhGT3Y1LTR0TU5YUVZQbUNYOFJxRi1ZZTkxcGhkdw&q=https%3A%2F%2Ftwitter.com%2Floige&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/loige.co⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/loige.co] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/lucianomammino/ [https://www.linkedin.com/in/lucianomammino/]

149. Headless CMS on AWS

We dive deep into Strapi, an open-source headless CMS that's changing how we approach content management and CRUD applications. In this episode, we explore how Strapi separates content from presentation, allowing marketing teams to manage content independently while developers maintain full control over the frontend experience. We discuss the powerful features that caught our attention, including the visual content type builder, dynamic zones, components, and the innovative blocks editor that stores content as JSON rather than HTML. We also cover practical aspects like local development workflows, data synchronization between environments, and deployment strategies on AWS. While we highlight some rough edges around documentation and minor bugs, we share our overall positive experience and provide insights into when Strapi might be the right choice for your next project. In this episode, we mentioned the following resources: * Strapi CMS: https://strapi.io/ [https://strapi.io/] * Strapi on GitHub: https://github.com/strapi/strapi [https://github.com/strapi/strapi] * Strapi Docs: https://docs.strapi.io/ [https://docs.strapi.io/] * Strapi S3 Provider: https://www.npmjs.com/package/@strapi/provider-upload-aws-s3 [https://www.npmjs.com/package/@strapi/provider-upload-aws-s3] * Strapi installation and pre-requisites: https://docs.strapi.io/cms/installation/cli [https://docs.strapi.io/cms/installation/cli] * The React Block Content renderer: https://github.com/strapi/blocks-react-renderer [https://github.com/strapi/blocks-react-renderer] * Contentful: https://www.contentful.com/ [https://www.contentful.com/] * Storyblok: https://www.storyblok.com/ [https://www.storyblok.com/] * Sanity: https://www.sanity.io/ [https://www.sanity.io/] * HyGraph: https://hygraph.com/ [https://hygraph.com/] * Byline CMS: https://bylinecms.app/ [https://bylinecms.app/] Do you have any AWS questions you would like us to address? Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn: - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/eoins⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbWlYNEdBeDBCSU56N3U1M0o1NDF0dS1sbkdRd3xBQ3Jtc0tra2RBajhnX0lBQ0dRVjZWMFZtNUdiX0JKVnZiVEdHZFBIcmtiSWpIU0Nvelc1UnliZDNZZ1hrS19tUUdtOHVwUTFzeTVrSkRuN2dwZDBKUXJlcmc5enY5YkJoUndoNEhIYVlsLUJKMmZRdUh2bVR4bw&q=https%3A%2F%2Ftwitter.com%2Feoins&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/eoin.sh⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/eoin.sh] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/eoins/⁠⁠ [https://www.linkedin.com/in/eoins/] ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.linkedin.com/in/eoins/]- ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://twitter.com/loige⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa3d0X2t5cUtreE02S2xuV1ZMTWNNMGdkamdqQXxBQ3Jtc0ttbDhQN2JicGhXOGxVX2Zqa1FLVVFKNERkY2VzUThBU3JONmdaM1dwWkVxRG5NQjdYeHU5TUxCWmxKSmZVclZLUXBSb2JGdkZUTnQzSEdpTUd1RDhGT3Y1LTR0TU5YUVZQbUNYOFJxRi1ZZTkxcGhkdw&q=https%3A%2F%2Ftwitter.com%2Floige&v=lyMHJW-3l9k] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠https://bsky.app/profile/loige.co⁠⁠⁠⁠⁠⁠⁠⁠⁠ [https://bsky.app/profile/loige.co] | ⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.linkedin.com/in/lucianomammino/ [https://www.linkedin.com/in/lucianomammino/]