DIY Lambda ZIP Pipeline vs. a Managed ZIP Service

"We'll just write a Lambda that zips the files" is the most natural sentence in this problem space — the pieces all exist, and a prototype works in an afternoon. The real comparison isn't prototype vs. service, though; it's the finished pipeline vs. the service. This page lists what finished actually means, what it costs, and the cases where building it yourself is still the right call.

What the finished DIY pipeline includes

The afternoon prototype is step 2 of about eight:

1. Job orchestration — exports outgrow a synchronous request fast, so you add async invocation or a queue, a job record, and a status endpoint.
2. Streaming ZIP assembly — archiver piping S3 reads into a multipart upload, keeping memory flat. The prototype.
3. ZIP64 correctness — archives past 4 GB or 65k entries, tested against the unzip tools your users actually run.
4. Partial-failure policy — one deleted object out of 5,000: fail everything, or skip and report? Either way, code and UX.
5. Retries and resume-on-crash — a 12-minute build that dies at minute 11 (deploy, OOM, S3 hiccup) restarts from zero unless you checkpoint.
6. The 15-minute wall — Lambda's cap doesn't move, so big exports need chunking across invocations (now it's a distributed system) or a different compute home.
7. Delivery — an expiring signed link with Range support, which means storing the artifact, lifecycle rules to delete it, and paying its S3 egress on every download.
8. Observability — the export that "never arrived" needs logs that answer why in minutes.

None of these are exotic. They're a few weeks of engineering plus permanent ownership — pager, updates, and the growth of everyone's exports pointed at your pipeline.

The running-cost comparison

The line that surprises teams is delivery bandwidth: a DIY pipeline that stores its artifact on S3 pays ~$0.09/GB every time the user clicks download. The managed path's economics are the inverse — sources read once, repeat downloads free.

When DIY is the right call

Honestly, often:

• Exports are small and bounded — invoices under 100 MB will never meet the 15-minute wall, and a streaming Lambda finishing in 40 seconds needs no help.
• Vendor constraints are real — compliance regimes where file bytes may not transit a third party end the discussion.
• You already run the machinery — a team with queues, job tracking, and artifact storage in place is adding a feature, not a pipeline.
• Export volume is huge and steady — at sufficient scale, metered service pricing loses to owned infrastructure you can amortize.

When managed is the right call

• Export sizes are unbounded or growing — user-generated content always is.
• The feature list above reads like someone's next quarter, and that someone has a product to build.
• Delivery quality matters: progress, resume, links that work in an email — table stakes from a service, real work to build.
• You want the fetch-and-zip out of your blast radius: the Lambda that only signs URLs (the pattern) can't take your API down.

FAQ

Isn't a managed service just someone else's Lambda?

Functionally it's the finished version of the list above — checkpointed builds, ZIP64, split output, expiring resumable delivery — priced as a utility. Whether that's worth the dependency is the whole decision.

Can I start DIY and switch later?

Yes, and cheaply: both designs put a URL list at the center. The switch is replacing your archive step with one POST — the listing, auth, and product code all survive.

What breaks first if I stay DIY and grow?

The 15-minute wall or delivery, in some order. Chunked multi-invocation builds are where the complexity multiplies (the timeout page maps the escalation), and per-download egress is where the bill does.

Is Fargate/ECS the middle ground?

It removes the 15-minute cap and keeps the work in your VPC — a reasonable home for big DIY builds. Everything else on the list (ZIP64, retries, delivery, cleanup) transfers unchanged; you've moved the compute, not the ownership.