Building on AWS, Part 1: Presigned Uploads

I’m starting a senior engineering role (woo! 🎉) at a company that’s heavy on AWS. Lambda, API Gateway, Step Functions, SQS, the works. I’ve heard the term “serverless” tossed around here and there but never really got to work with it, so I’m diving head-first into AWS by trying to build a miniature of what I feel I’d be working with.

My background is Python/Django, SvelteKit, Docker, PostgreSQL. I’ve done the simple stuff: setting up EC2 instances, the basics you need to get a Django app running. I even got my hands on EKS since we ran our app in Docker and Kubernetes at my previous company, though that was mostly on OVH (a French cloud provider), and there wasn’t much pipeline work for me at the time. So my personal AWS rating would probably be a 3.5/10, and I’m hoping to get to at least a 6 or 7 by the end of this series.

Instead of just reading docs, I decided to build a simplified meeting notes pipeline. Audio file goes in, transcription and AI summary comes out, user gets notified. The idea is to build each piece manually through the AWS Console first to understand what it does, then tear it down and rebuild with CloudFormation (AWS’s Infrastructure as Code tool).

This post covers Session 1: getting an API endpoint that generates a presigned S3 upload URL.

The goal

A client sends a POST request with a filename. The server returns a presigned URL that the client can use to upload a file directly to S3. No file goes through the server itself. This is a common pattern for handling large file uploads, and it’s how the product handles meeting recordings.

The pieces needed:

• An S3 bucket to receive the files
• A Lambda function that generates the presigned URL
• An IAM role that gives the Lambda permission to write to the bucket
• An API Gateway that exposes the Lambda as an HTTP endpoint

Writing the Lambda handler

Lambda functions in Python are just regular functions with a specific signature: lambda_handler(event, context). AWS invokes this function whenever a request comes in.

def lambda_handler(event, context):
    # event is a dict with the HTTP request details
    # context has runtime info (timeout, memory, etc.)
    ...

The event dict contains things like httpMethod, body, headers, basically everything about the incoming HTTP request. You parse the body, do your thing, and return a dict with statusCode, headers, and body.

The actual logic is straightforward. Parse the filename from the request body, generate a presigned URL using boto3 (AWS’s Python SDK), and return it.

Before deploying anything, I tested the handler locally by just calling the function from a test script:

from src.presign.handler import lambda_handler

event = {
    "httpMethod": "POST",
    "body": '{"filename": "test.webm"}'
}
response = lambda_handler(event, None)
print(response)

Since boto3 uses your local AWS credentials, this actually hits real AWS services. The presigned URL it generates works against your real bucket. I confirmed this with a quick curl:

curl -X PUT -T some-file.webm "PRESIGNED_URL_HERE"

File showed up in the bucket. Handler works.

IAM roles: the permission layer

This part was new to me. In the VPS world, your server process has whatever permissions the OS user has. In AWS, every service needs an explicit IAM role that says exactly what it’s allowed to do.

A role has two parts:

1. Trust policy (AssumeRolePolicyDocument): who can use this role. In our case, we’re saying “Lambda functions can assume this role.”
2. Permission policies: what the role can do. We’re granting s3:PutObject on our specific bucket, and CloudWatch logging permissions.

The trust policy confused me at first. When you create a role in the console, you pick “AWS Service > Lambda” from a dropdown and move on. Behind the scenes, that generates a JSON document saying lambda.amazonaws.com is allowed to call sts:AssumeRole. It uses service principal identifiers like lambda.amazonaws.com rather than CloudFormation resource types like AWS::Lambda::Function because IAM is its own system that predates CloudFormation. The identifiers need to work everywhere, not just in templates.

The permission policy uses a principle called “least privilege.” Instead of giving the Lambda broad S3 access, you scope it to exactly one action (s3:PutObject) on exactly one resource (arn:aws:s3:::your-bucket-name/*). The /* means objects inside the bucket, not the bucket itself. This matters a lot for a compliance-focused product.

API Gateway: the HTTP layer

API Gateway sits in front of your Lambda and gives it an HTTP endpoint. You define resources (URL paths) and methods (GET, POST, etc.), and point them at your Lambda.

I was genuinely in awe that something like this exists. It’s like running a restaurant where you don’t have to worry about the front of house, the seats and tables, or the waiters. AWS handles all of that for you, and you just make sure the kitchen is able to respond to requests properly.

In the console, this is a few clicks. Create a REST API, add a /presign resource, add a POST method, select your Lambda, deploy to a stage. You get a URL like:

https://abc123xyz.execute-api.ap-southeast-1.amazonaws.com/dev/presign

One thing I didn’t expect: API Gateway needs its own permission to invoke your Lambda. In the console, this happens silently when you select your Lambda as the integration target. It adds a resource-based policy to the Lambda saying “this API Gateway can call you.” I only discovered this mattered when I later tried to set it up in CloudFormation and it was not automatic.

I tested by deliberately leaving out that permission. The API returned a 500, and CloudWatch showed nothing. That’s because the Lambda never executed. API Gateway tried to invoke it, AWS said “no permission,” and API Gateway returned a generic 500. The Lambda was never involved, so there was nothing to log. Once I added the permission, everything worked.

Tearing it down and rebuilding with CloudFormation

This is where the real learning happened.

CloudFormation lets you define all your AWS resources in a YAML file and deploy them as a single stack. You describe the desired end state, and AWS figures out what to create and in what order.

What felt like 5 clicks in the console turned into a template with six resources, each with explicit references connecting them:

Resources:
  UploadBucket:            # S3 bucket
  LambdaRole:              # IAM role with trust + permission policies
  PresignFunction:         # Lambda function
  MeetingNotesApi:         # API Gateway (just the container)
  PresignResource:         # The /presign path
  PresignPostMethod:       # POST method pointing to Lambda
  ApiDeployment:           # Deploy to dev stage
  ApiLambdaPermission:     # Let API Gateway invoke Lambda

Every implicit connection from the console becomes explicit. The Lambda references the role via !GetAtt LambdaRole.Arn. The API method references the Lambda via its ARN. The deployment depends on the method existing first. The Lambda permission references both the API and the function.

The deployment uses a two-step process:

# Package: zips Lambda code, uploads to S3, rewrites template
aws cloudformation package 
  --template-file template.yaml 
  --s3-bucket your-deploy-bucket 
  --output-template-file packaged.yaml

# Deploy: creates or updates the stack
aws cloudformation deploy 
  --template-file packaged.yaml 
  --stack-name meeting-notes-pipeline 
  --capabilities CAPABILITY_NAMED_IAM

The package step is necessary because CloudFormation can’t read local file paths. You write Code: ./src/presign/ in your template, and package zips that directory, uploads it to S3, and rewrites the reference to point at the uploaded zip. You deploy packaged.yaml, not template.yaml.

CAPABILITY_NAMED_IAM is a flag you pass to acknowledge that your template creates IAM resources with explicit names. AWS makes you opt into this because named IAM resources could overwrite existing ones.

I hit a few errors along the way. My IAM policy structure was wrong: I used Actions instead of Action, Resources instead of Resource. I forgot the AssumeRolePolicyDocument entirely. My first deploy failed silently because the changeset validation caught the errors before anything was created. Running aws cloudformation describe-change-set showed the actual problem.

After fixing everything, the stack deployed, and I could curl the new endpoint and upload files just like before, but now the entire infrastructure is defined in one version-controlled file.

Why not just use a VPS?

I asked myself this a lot during the process. A FastAPI app on a VPS covers most use cases perfectly fine, and is way simpler to reason about. You SSH in, tail logs, restart the process. Done.

The serverless approach makes more sense when you consider two things:

1. Ops burden at a small team. With Lambda, nobody has to be on-call for the infrastructure. No capacity planning, no OS patching, no scaling decisions. For a startup with a small engineering team, that’s not about saving money. It’s about not spending an engineer babysitting servers.

2. Event-driven workloads. A meeting gets uploaded, five things happen in sequence, then nothing until the next upload. Running a server 24/7 for that is like keeping a taxi idling in your driveway.

Most production systems use both patterns. The API layer might run as a traditional service, while the processing pipeline is serverless.

What I learned

The console teaches you what, CloudFormation teaches you how things connect. Clicking through the console is great for understanding individual services. Writing CloudFormation forces you to understand every implicit connection the console hides. You can’t skip anything.

AWS loves explicit permissions. Coming from the VPS world where your process just has access to everything, the IAM model feels verbose. But for a compliance product where you need to prove that each service only has the minimum access it needs, it makes a lot of sense.

Test locally first. There’s AWS SAM (Serverless Application Model) which can spin up a Lambda-like environment through Docker, but it felt like too much overhead for what I needed. Your Lambda handler is just a Python function. Call it from a script, verify the output, then deploy. The feedback loop is way faster than deploying and checking CloudWatch.

Errors in CloudFormation are cryptic but traceable. When a deploy fails, describe-stack-events and describe-change-set are your tools. The error messages aren’t great, but they at least point you to the right resource.

What’s next

Session 2 is where it gets event-driven. Right now a file lands in S3 and sits there. Next, I’m wiring up S3 event notifications to push to an SQS queue, a Lambda to consume from SQS, and DynamoDB to track job status. That’s where it starts to feel like a real async pipeline.