Black Friday-Ready E-Commerce Platform on AWS with Surge Autoscaling

In a nutshell

Picture a stadium concert where the doors open at 8:00 PM sharp. If you wait until the crowd is already pressing against the gates to start hiring ticket-takers, you get a crush and a stampede. Two ideas save the night. First, you staff up before the doors open, because you already know when they open — that is pre-scaling (and its automated cousin, predictive scaling). Second, when the crowd is bigger than the gates can pass, you form an orderly line and admit people at a steady rate instead of letting everyone shove at once — that is queue-based load leveling and a waiting room. Almost everything in this lesson is one of those two moves applied to a different part of an online store.

“Surge autoscaling” is the discipline of surviving a traffic spike that arrives faster than any reactive system can respond to it. On a normal day a shop might handle a few thousand requests per second; the instant a Black Friday doorbuster goes live it can jump to over a hundred thousand in under two minutes. That is not a gentle ramp you can chase — it is a vertical wall. This lesson shows the architecture and the specific AWS scaling primitives that let a store treat that wall as a planned event rather than an emergency: scaling ahead of demand, decoupling the slow write path from the fast customer click, and degrading on purpose by shedding the least valuable traffic first.

Level: Advanced · Time: ~45 min

Prerequisites. You will get the most from this if you have already met a few building blocks: EC2 Auto Scaling (how a fleet grows and shrinks), ECS on Fargate (running containers without managing servers), and SQS with dead-letter queues (a durable message buffer). If those are new, skim them first; this lesson assembles them into one peak-ready system.

After this you will be able to:

• Explain why reactive CPU autoscaling structurally fails against a vertical traffic wall, and name the five scaling primitives that fix it.
• Choose between target-tracking, step, scheduled, and predictive scaling — and know when to pre-warm with warm pools instead.
• Absorb a spike two ways: the container way (Fargate + queue) and the serverless way (Lambda reserved/provisioned concurrency + API Gateway throttling + SQS).
• Scale the data tier deliberately: DynamoDB on-demand vs. provisioned, the hot-partition limit, Aurora Serverless v2 vs. read replicas, and RDS Proxy for connection storms.
• Design graceful degradation — a waiting room and load-shedding — and prepare a known event with Service Quota pre-raises, load tests, and a game day.

A mid-market fashion retailer — call it the kind of brand that does ₹1,800 crore a year online and another ₹1,400 crore in stores — gets one sentence from its CEO in the September board meeting: “Last Black Friday the site fell over for ninety minutes at 8 PM and we will not let that happen again.” The post-mortem from the prior year is brutal and specific. At the stroke of the doorbuster launch, traffic went from a steady 3,000 requests per second to roughly 120,000 in under two minutes. The autoscaler — reacting to CPU metrics on a five-minute lag — was still spinning up the previous surge’s capacity when the database connection pool saturated, checkout started throwing 500s, and the load balancer dutifully kept routing customers into a dying fleet. The brand lost an estimated ₹22 crore in that ninety-minute window and, worse, a chunk of trust. This article is the reference architecture for the rebuild: a Black Friday-ready storefront on AWS that treats a 40x surge not as an emergency but as a planned-for Tuesday.

The pressures in retail at peak are unforgiving and they all arrive at once. Spikiness is the defining trait — not a gentle ramp but a near-vertical wall the instant a 50%-off banner goes live, and reactive autoscaling is structurally too slow for a wall. Revenue per second is so high that even a 99.9% availability target leaks real money; the brand thinks in dollars-per-minute-of-downtime, not nines. The database is the choke point — stateless web tiers scale linearly and cheaply, but the writes (inventory decrements, order inserts) hit a relational store that does not, and that asymmetry is where peak architectures live or die. And the customer’s patience is zero — a shopper who sees a spinner at checkout abandons the cart and tells everyone. The architecture below answers each of these with a specific, named mechanism rather than “add more servers and hope.”

Why the naive scaling story fails

Three obvious fixes get proposed every year, and each fails predictably at 40x.

“Just turn the autoscaler up.” Target-tracking on CPU is reactive — it observes a breach, waits out a cooldown, launches tasks, and waits for them to pass health checks. On a vertical traffic wall, the fleet is perpetually chasing a number it passed minutes ago. You cannot out-react a doorbuster.

“Just over-provision for peak and leave it running.” Sizing the steady fleet for the 8 PM spike means paying for 40x capacity 360 days a year to use it for two hours, which finance vetoes the moment they see the bill, and which still does not protect the database tier that does not scale horizontally on writes.

“Just make checkout synchronous and fast.” Coupling the customer’s checkout click directly to an inventory write and a payment call means every downstream slowness — a payment gateway hiccup, a hot inventory row — becomes the customer’s spinner and the customer’s abandoned cart. The write path will be slower than the read path at peak; the only question is whether the customer waits for it.

The real architecture does three things instead: it scales ahead of demand with predictive scaling and pre-warming, it decouples the write path so the customer’s order is captured in milliseconds and settled asynchronously, and it degrades deliberately — shedding the least valuable load first — rather than collapsing uniformly. Those three ideas drive every component choice that follows.

The scaling toolbox: five ways to add capacity

Before the architecture makes sense, you need the vocabulary of how capacity actually appears. “Autoscaling” is not one thing — it is a family of policies with very different reaction speeds, and picking the wrong one is exactly how the retailer lost ninety minutes. Here are the five levers, ordered from slowest-to-react to earliest-to-act, with the peak-day job each one is good for.

1. Target tracking — the thermostat (reactive)

Target tracking is the default and the one everyone reaches for first. You pick a metric and a target value — “keep average CPU at 50%” or “keep 1,000 requests per target” — and AWS adds or removes capacity to hold the number there, exactly like a thermostat holding a room temperature. It is simple, self-correcting, and correct for gentle changes. Its weakness is baked into the definition: it can only act after the metric has already moved. On a vertical wall the metric is at 100% before the first new task passes a health check, so target tracking spends the whole spike chasing a number it passed minutes ago.

The subtle, expensive mistake is tracking the wrong metric. CPU is a lagging proxy for a request-driven web tier — utilisation climbs only after requests are already queuing inside the app. The metric that leads demand is the request rate itself. For an ALB-fronted fleet that is ALBRequestCountPerTarget; for a queue-drain fleet it is the backlog. “Scale on the metric that leads demand, not the one that lags it” is the single most important tuning lesson on this page, and it is why the order-workers in this design track queue depth rather than CPU.

2. Step scaling — bigger breach, bigger jump

Step scaling reacts to a CloudWatch alarm but lets you define stepped adjustments keyed to how badly the metric is breached: add 2 tasks if CPU is 60–70%, add 6 if it is 70–85%, add 20 above 85%. That non-linear response is genuinely useful on a spike — a small breach gets a small correction, a big breach gets a big one — but it is still reactive, still fired by an alarm that needs a couple of evaluation periods to trip. Use it as a fast, aggressive top-up layered on other policies, not as your only defence.

3. Scheduled scaling — set the floor by the clock

Scheduled scaling is deterministic and, for a known event, the most reliable lever there is. You tell Auto Scaling: at 19:45, set MinCapacity to 400. There is no metric, no alarm, no lag — the floor simply rises before the doorbuster timestamp, so the capacity is already there when the wall arrives. This is the automated form of “staff the gates before the doors open.” Every planned peak (Black Friday, a flash sale, a ticket on-sale, a product launch) should be wrapped in scheduled actions that raise the minimums well ahead of the event and lower them well after.

4. Predictive scaling — the ML forecast (proactive)

Predictive scaling uses machine learning on up to 14 days of history to forecast demand 48 hours ahead and provisions capacity before the forecasted ramp, re-evaluating hourly. It is excellent for the recurring shape of demand — the daily morning rush, the weekly weekend bump — because that shape repeats and the model learns it. Two caveats matter on peak day. First, it needs at least 24 hours of history and it predicts the usual, so the very first Black Friday you run is precisely the day history cannot predict — pair predictive with scheduled for the anomaly. Second, run it in forecast-only mode first to sanity-check the forecast against reality before letting it act, and consider MaxCapacityBreachBehavior so a forecast above your max can still be honoured on the day.

5. Warm pools and pre-warming — capacity you can turn on instantly

Every policy above assumes new capacity appears quickly. For Fargate it roughly does — a task is a container pull and start, tens of seconds. For EC2-backed fleets it does not: a cold instance must boot, join the cluster, pull images, and pass health checks — minutes you do not have on a wall. A warm pool solves this by keeping pre-initialised instances in a Stopped, Hibernated, or Running state alongside the Auto Scaling group; on scale-out, ASG pulls a ready instance from the pool and skips the boot-and-bootstrap, cutting time-to-service dramatically. Stopped is cheapest (you pay only for EBS), Hibernated resumes with RAM intact, Running is instant but full-price. An instance-reuse policy returns instances to the warm pool on scale-in instead of terminating them, so you are not re-paying the warm-up on every oscillation. The companion mechanism is the lifecycle hook: it pauses an instance at Pending:Wait (launch) or Terminating:Wait (shutdown) so you can run setup or a graceful connection-drain before the instance is declared in-service or destroyed — the hook holds for a heartbeat timeout (default 1 hour, up to 48) until your automation calls CompleteLifecycleAction. Warm pools, lifecycle hooks, and instance refresh are covered in depth in the EC2 Auto Scaling warm pools lesson; this storefront runs on Fargate specifically to sidestep most of that complexity, but the mental model — have the capacity warm before the wall — is identical.

The ALB warm-up problem — the load balancer scales too

A trap teams hit at 40x: the Application Load Balancer itself scales its capacity units gradually over minutes, so a genuinely instantaneous flood can outrun the ALB before it has grown. Three mitigations, in order of preference: front the ALB with CloudFront (as this architecture does) so the edge absorbs the initial burst and smooths what reaches the ALB; ramp synthetic traffic during your load test so the ALB is already scaled when real traffic hits; and for extreme, truly instantaneous spikes prefer a Network Load Balancer, which handles sudden volume without a pre-warm because it scales differently. The one-line rule: the balancer is a component that scales too, so warm it like everything else.

A worked example — how much to pre-warm

Suppose steady state is 3,000 requests/second and the doorbuster forecast is 120,000 RPS. A single Fargate storefront task comfortably serves ~400 RPS at your latency budget (measure this in the load test — do not guess). Naive math says peak needs 120,000 ÷ 400 = 300 tasks. But you never let target tracking start from the steady floor and climb 40x live — by the time it reacts you have lost the event. Instead:

1. Scheduled action raises MinCapacity from the steady ~10 tasks to a pre-warmed floor of, say, 240 tasks (80% of forecast) at 19:45, fifteen minutes before the 20:00 drop.
2. Predictive scaling contributes the recurring-shape portion of the ramp where history is meaningful.
3. Target tracking on ALBRequestCountPerTarget (target ~350 to leave headroom under the 400 ceiling) handles only the residual 20% and any forecast error — a manageable job because it is no longer chasing the whole wall.
4. A short scale-out cooldown / instance warmup (30–60s) lets the fleet add the residual quickly, while a long scale-in cooldown (several minutes) stops it from dropping tasks during a brief lull and then scrambling to re-add them.

That layering — scheduled floor + predictive shape + target-tracking residual — is the whole trick. No single policy survives a 40x wall; the combination does.

Architecture overview

The platform separates cleanly into two paths that must be reasoned about independently: a read-heavy browse path that serves the catalogue, product pages, and search to the flood of shoppers, and a write-critical commerce path that captures carts and orders and must never lose a customer’s intent even while it sheds load. Most peak failures come from treating these as one system; keeping them distinct is the first design move.

The defining property of the whole topology is this: the customer’s request is answered as close to the edge as possible and as far from the database as possible. Most Black Friday traffic is browsing the same few hundred hot products, and the architecture is built so the database barely notices that crowd.

Browse path, following the request:

1. A shopper hits Akamai at the very edge for TLS termination, global anycast, bot mitigation, and — critically at peak — WAF and rate-based bot defenses that strip out the credential-stuffing and scalper-bot traffic which can be 30-40% of “demand” during a hyped drop. Akamai also absorbs a large share of static and cacheable content so it never reaches AWS.
2. Behind Akamai sits Amazon CloudFront as the AWS-native CDN and the front door to origin. CloudFront caches product images, CSS/JS bundles, and — using cache policies tuned for peak — even semi-dynamic catalogue fragments, so a hot product page is served from a POP without touching compute.
3. Cache misses reach an Application Load Balancer fronting the storefront tier running on Amazon ECS on Fargate. Fargate is chosen deliberately over EC2-backed ECS: there are no nodes to pre-warm or patch, task launch is fast, and at peak the only scaling dimension is task count, which is exactly the simplicity you want at 2 AM on Black Friday.
4. The storefront’s read queries for catalogue, pricing, and inventory-display hit Amazon ElastiCache for Redis first. The cache is the load-bearing wall of the browse path: hot products, category listings, and price lookups are served from memory, and only a genuine miss falls through to the database.
5. Misses land on Amazon Aurora (PostgreSQL-compatible) — but on the reader endpoint, against a fleet of read replicas, never the writer. Browse traffic is read-only by definition and is isolated onto replicas so it can never contend with the order-write path.

Commerce path, the part that must not break:

1. The shopping cart lives in Amazon DynamoDB, not in a relational table. A cart is a per-session, key-value, write-heavy object with no need for joins, and DynamoDB with on-demand capacity (or pre-provisioned with auto-scaling for a predictable peak) absorbs millions of cart mutations at single-digit-millisecond latency without a connection pool to exhaust. This single choice removes the cart from the database’s blast radius entirely.
2. When the shopper clicks Place Order, the request does not synchronously write the order and decrement inventory. Instead the order-capture service validates the cart, reserves payment authorization, writes an “order accepted” record, and publishes the order onto an Amazon SQS queue — then returns success to the customer in milliseconds. The customer’s intent is now durably captured; the slow work happens behind the queue.
3. A pool of order-processing workers (ECS Fargate services, scaled on queue depth) consume from SQS, perform the transactional work against Aurora’s writer — insert the order, decrement inventory in a controlled way, trigger fulfillment — and handle retries. SQS is the shock absorber: it lets the customer-facing tier accept orders far faster than the database can durably commit them, smoothing a vertical spike into a steady drain the writer can sustain.
4. Poison messages and repeated failures route to an SQS dead-letter queue for inspection rather than blocking the line, and an idempotency key on each order makes the at-least-once delivery safe to retry without double-charging or double-decrementing.

Identity, secrets, and security wrap both paths. Customer auth is handled by the storefront’s own consumer identity provider; the internal operators, fulfillment staff, and the on-call engineers authenticate through Okta as the workforce IdP (federated to Microsoft Entra ID where the retailer’s Microsoft 365 estate requires native Azure RBAC for back-office tools), so every human touching the admin console or the deploy pipeline carries an SSO identity with MFA and conditional access. Application secrets — the payment-gateway API keys, the third-party tax-service token, database credentials for the workers — are issued by HashiCorp Vault with short-lived dynamic database credentials, so a leaked credential expires on its own and no long-lived password sits in a task definition or environment variable. (That discipline is non-negotiable here: this team has been burned by credentials in source control before and treats Vault-issued, auto-rotating secrets as the only acceptable pattern.)

Component breakdown

A few of these choices carry the weight of the design and deserve the why.

Why DynamoDB for the cart and SQS in front of orders. These two choices, together, are what take the database off the critical path of a spike. A relational cart table means every “add to cart” is a write to the same store that commits orders, and at 40x that contention alone can topple the writer. Moving the cart to DynamoDB makes it a horizontally-scaling, schemaless, latency-flat store that simply does not have a connection pool to exhaust. And putting SQS between order capture and order commit decouples the rate at which customers place orders from the rate at which the database can durably commit them — the customer gets an instant “order received,” and the writer drains the queue at its own sustainable pace. The retailer would rather tell a customer “your order is confirmed and processing” in 80 milliseconds than make them watch a spinner for eight seconds while a payment gateway is slow.

Why predictive scaling, not just target tracking. Target-tracking reacts; predictive scaling anticipates. AWS Auto Scaling’s predictive policy learns the daily and weekly demand shape and provisions ahead of a forecasted ramp. But Black Friday is precisely the day history does not predict, so predictive scaling is paired with a scheduled scaling action that pre-warms the Fargate fleet (and pre-provisions DynamoDB and Aurora replicas) to a known floor before the doorbuster timestamp, with target-tracking layered on top to handle the residual. The lesson from the failed year is blunt: on a vertical wall you must already have the capacity when the wall arrives, because there is no time to react after it does.

Why Aurora split reader/writer with RDS Proxy. Reads and writes have opposite scaling stories, so they get opposite treatment. Browse reads fan out across auto-scaling Aurora read replicas behind the reader endpoint and never touch the writer. The single writer is the scarcest resource in the whole system, so it is protected on every side: SQS rate-limits what reaches it, RDS Proxy pools and multiplexes connections so a stampede of workers cannot exhaust max_connections, and it is sized with deliberate headroom because you cannot horizontally add writers mid-event.

Serverless-first: absorbing a surge with Lambda, API Gateway, and SQS

This architecture runs the storefront on Fargate, but Fargate-plus-a-queue is not the only way to meet a wall — the other major pattern is serverless-first, where compute scales to the request without a fleet to pre-warm at all. Understanding it matters even here, because this design already borrows its central idea (queue-based load leveling with SQS) and because the order-capture and order-processing tiers are natural Lambda candidates. Here is the serverless surge toolkit and where its sharp edges are.

Lambda concurrency — three dials you must know

Lambda scales by running more concurrent execution environments, but “it scales automatically” hides three dials that decide whether it survives peak:

• Account concurrency limit — a soft, per-region ceiling on total simultaneous executions, 1,000 by default. At 120,000 RPS with a 100 ms function, you need 120,000 × 0.1 = 12,000 concurrent executions (by Little’s Law: concurrency = requests/sec × avg duration in seconds). The default 1,000 throttles you into the ground on day one — raising this quota is a mandatory pre-event step, not an afterthought.
• Burst / scaling rate — since late 2023, each function scales up by up to 1,000 concurrent executions every 10 seconds, independently per function, up to your account limit. That is fast, but it is still a rate: from a cold floor you cannot reach 12,000 instantly, so for a vertical wall you pre-warm with the next dial.
• Reserved vs. provisioned concurrency — reserved concurrency carves a guaranteed slice of the account pool for one function (it both caps that function and protects the others from it; set it to 0 as an emergency kill switch). Provisioned concurrency pre-initialises N execution environments so there is no cold start for the first N concurrent requests — you pay for it whether used or not, and you scale it ahead of the event with Application Auto Scaling on a schedule, exactly like the Fargate scheduled floor. For a known 20:00 drop, ramp provisioned concurrency up at 19:45 and back down at 22:00. Cold-start mitigation (provisioned concurrency vs. SnapStart, and how to right-size it) is covered in the Lambda cold-starts lesson.

API Gateway — throttle on purpose, not by accident

If Lambda sits behind API Gateway, the gateway is your first throttle and your first line of defence. REST APIs default to an account-level 10,000 requests/second steady-state with a 5,000 burst (a token bucket), per region — another soft quota to raise before peak. Beyond the account limit you get finer control that is useful at peak, not just restrictive:

• Per-method / per-stage throttling caps a single expensive route (say, search) so it cannot starve checkout.
• Usage plans + API keys apply per-client rate, burst, and daily quota — the right tool for partner/affiliate traffic that must not consume capacity the buy flow needs.
• When a caller exceeds a limit, API Gateway returns HTTP 429 Too Many Requests. Design clients to back off and retry with jitter on a 429 rather than hammering — an un-jittered retry storm turns a throttle into an outage.

Deliberate throttling is load-shedding at the front door: it is far better to cleanly 429 a fraction of low-value traffic than to let everything through and collapse the whole API.

SQS — queue-based load leveling, examined

SQS is the shock absorber this architecture leans on, and it is worth understanding why it works. A standard queue offers effectively unlimited throughput with at-least-once delivery and best-effort ordering; it decouples the rate at which producers (order-capture) enqueue from the rate at which consumers (order-workers) drain. That decoupling is the entire point: the customer-facing tier can accept orders far faster than Aurora can durably commit them, and the queue holds the difference instead of the customer holding a spinner. Details that bite at peak:

• In-flight limit — a standard queue allows roughly 120,000 in-flight (received-but-not-deleted) messages. If workers receive faster than they delete (or visibility_timeout is too long), you can hit this ceiling and see OverLimit errors — a signal your consumers are too slow, not that the queue is broken.
• Visibility timeout must exceed worker p99 processing time, or a still-processing message becomes visible again and gets processed twice. The existing design sets it to 90s deliberately.
• DLQ + maxReceiveCount quarantine poison messages after N failed attempts so one bad order does not block the line; long polling (ReceiveMessageWaitTimeSeconds up to 20) cuts empty receives and cost. FIFO vs. standard, DLQ redrive, and poison-message handling are covered in the SQS/SNS fan-out and DLQ lesson.
• FIFO caveat — if you genuinely need strict ordering, FIFO queues are limited to 300 messages/second (3,000 with batching), which is often too slow for peak order capture. Prefer standard + idempotency keys and reconstruct ordering downstream, exactly as this architecture does.

When serverless-first wins outright: spiky, unpredictable, or event-driven workloads where you would otherwise pay for idle Fargate tasks, and teams that want to offload the fleet entirely. When Fargate-plus-queue wins: steady baselines with predictable peaks, long-running connections, or workloads whose per-request cost is lower on always-warm containers than on per-invocation Lambda. This retailer runs a large, steady storefront with a known peak — Fargate for the base, the serverless idea (SQS leveling) for the surge.

Scaling the data tier under a surge

The web tier scales with money; the data tier scales with architecture, and this is where peak designs live or die. Stateless compute fans out linearly, but a database has finite connections, hot rows, and — for relational engines — a single writer. Below is how each data service in this design behaves under a wall and how to prepare it.

DynamoDB — instant scale, with two sharp edges

DynamoDB is the cart store precisely because it has no connection pool to exhaust and scales horizontally on its own. But “serverless and infinite” hides two edges.

On-demand vs. provisioned + auto-scaling. On-demand mode charges per request and adapts capacity automatically — but only up to twice the previous peak a table has seen, and it needs about 30 minutes to accommodate a jump beyond that. So a table that idles at 500 writes/second cannot instantly absorb 40,000 the moment the drop lands; it will throttle while it ramps. Two fixes for a known event: pre-warm the table by driving traffic up gradually in your load test so its “previous peak” is already high, or use provisioned capacity with Application Auto Scaling and a scheduled higher minimum set before the event (auto-scaling alone reacts on CloudWatch with minutes of lag — the scheduled floor is what makes it peak-safe, mirroring the compute story). Newer accounts can also set a maximum throughput on on-demand tables to cap runaway cost during an attack or bug.

The hot-partition limit. DynamoDB spreads data across partitions by the partition key, and each physical partition tops out at 3,000 read units and 1,000 write units per second — a hard ceiling no amount of table-level capacity overrides. If your partition key has low cardinality (say, order_status with three values) all traffic funnels onto a few partitions and throttles at the per-partition limit while the table sits far below its provisioned total. Adaptive capacity mitigates this automatically — it shifts capacity toward hot partitions and isolates frequently accessed items — but it cannot break the per-partition ceiling for a single hot key. The cart escapes this because its key is the session ID: millions of distinct high-cardinality keys spread evenly across partitions. The lesson generalises: choose a partition key with high cardinality and even access, or write-shard a hot key (append a suffix 0..N and scatter-gather on read). A “trending product inventory counter” is the classic hot-key trap in retail — keep counters like that in ElastiCache or shard them, never on one DynamoDB key.

Aurora — Serverless v2 or read replicas, and RDS Proxy for the connection storm

The single Aurora writer is the scarcest resource in the whole system; you protect it from three directions.

Read scale is easy. Aurora supports up to 15 read replicas behind a reader endpoint that load-balances browse queries, with an auto-scaling policy that adds replicas on CPU or connection targets. Browse traffic — read-only by definition — is isolated entirely onto replicas so it can never contend with order writes.

Write scale is the hard part. You cannot add writers mid-event, so you have two preparation strategies. Aurora Serverless v2 scales a single instance’s capacity in fine-grained Aurora Capacity Units (0.5 ACU steps, roughly 2 GiB of memory each, from 0.5 up to 256 ACU) in seconds — excellent for spiky, hard-to-forecast load because it grows and shrinks with the workload without a manual resize. Provisioned Aurora with a deliberately oversized, high-headroom writer instance is the alternative when the peak is well-understood and you want predictable, flat performance. Either way, the writer is finite — which is why SQS rate-limits what reaches it.

The connection storm. A surge does not only bring queries; it brings connections. Hundreds of order-workers (or thousands of Lambda invocations) each opening a database connection can blow past max_connections and topple the writer even when CPU is fine — connection exhaustion, not compute, is a top cause of peak database failure. RDS Proxy sits between the workers and Aurora, pooling and multiplexing connections so a stampede of clients shares a small, stable set of backend connections; it also holds connections open across an Aurora failover to cut failover time and supports IAM authentication so no password sits in the worker. Connection pooling, failover behaviour, and IAM auth are covered in the RDS Proxy lesson. The rule: anything with volatile, high-fan-out, short-lived connections in front of RDS/Aurora wants RDS Proxy.

ElastiCache — the load-bearing wall of the read path

ElastiCache for Redis is what keeps the browse flood off Aurora entirely. Two peak roles: read offload (hot products, category listings, price lookups served from memory so only a genuine miss falls through to a replica) and session/state store (cart-adjacent session data, rate-limit counters, and those trending-product counters that must never touch a DynamoDB hot key). Scale it with cluster mode (sharding across nodes for more memory and throughput) plus read replicas per shard and Multi-AZ for failover. Tune TTLs to the volatility of the data — long TTLs on stable catalogue entries, deliberately short TTLs on stock counts so “only 2 left” is not badly stale. The higher the cache hit ratio, the smaller and cheaper the Aurora reader fleet behind it — cache hit ratio is a cost lever as much as a latency one.

The one-paragraph mental model

Reads scale with replicas and cache; you buy read headroom with money. Writes scale with architecture — you keep write pressure away from the single writer with a cart on DynamoDB, orders behind SQS, connections pooled by RDS Proxy, and hot counters in Redis. Every data-tier decision in this design is an answer to the same question: how do we make sure the writer barely notices the wall?

Implementation guidance

Provision with Terraform; gate the IaC with Wiz Code. The whole estate — VPC, subnets across three Availability Zones, ECS services, Aurora cluster, ElastiCache, DynamoDB tables, SQS queues, scaling policies — is declared in Terraform with remote state. Every Terraform change goes through a pull request that Wiz Code scans pre-merge for misconfigurations (a public S3 bucket, an over-permissive security group, an unencrypted queue) so an insecure change is caught in review, not in the running account. Hosts that are not containers — the bastion/admin tier, the virtual network appliances — are configured with Ansible for repeatable, auditable state.

A minimal Terraform shape for the order-buffer queue and its dead-letter companion communicates the intent — durable capture, safe retries:

resource "aws_sqs_queue" "orders_dlq" {
  name                      = "bf-orders-dlq"
  message_retention_seconds = 1209600 # 14 days to inspect poison messages
}

resource "aws_sqs_queue" "orders" {
  name                       = "bf-orders"
  visibility_timeout_seconds = 90      # > worker p99 processing time
  receive_wait_time_seconds  = 20      # long polling, fewer empty receives
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.orders_dlq.arn
    maxReceiveCount     = 5            # retry, then quarantine to DLQ
  })
}

And the scaling shape that matters most — workers tracking queue depth, not CPU, so the order drain rate follows the backlog:

resource "aws_appautoscaling_policy" "workers_on_queue" {
  name               = "scale-workers-by-backlog"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.order_workers.resource_id
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"

  target_tracking_scaling_policy_configuration {
    target_value       = 1000 # target backlog-per-task; tune to drain SLA
    customized_metric_specification {
      metric_name = "ApproximateNumberOfMessagesVisible"
      namespace   = "AWS/SQS"
      statistic   = "Average"
    }
    scale_in_cooldown  = 120
    scale_out_cooldown = 30   # scale out fast, scale in slow
  }
}

Deploy via GitOps, freeze before the event. Application builds and tests run in Jenkins or GitHub Actions (authenticating to AWS through OIDC so there is no long-lived access key to leak), producing immutable container images. Argo CD then reconciles the desired state declared in Git onto the ECS/EKS estate, so what is running is always exactly what is in the repository and a rollback is a Git revert. Crucially, a ServiceNow change freeze locks the production estate in the days before Black Friday — the most reliable peak system is the one nobody has touched in a week — with only pre-approved, rehearsed changes allowed through the gate.

Kill the static secrets; federate the humans. Order workers and storefront tasks obtain database credentials as short-lived dynamic secrets from HashiCorp Vault’s database engine, leased for minutes and auto-rotated, so a compromised task cannot yield a durable credential. Payment-gateway and tax-service API keys live in Vault too, never in a task definition. Human access to the admin console, the deploy pipeline, and the AWS account is federated through Okta with MFA and conditional access, brokered to Microsoft Entra ID for the back-office tools that live in the Microsoft estate — so every privileged action carries a real, audited workforce identity.

Enterprise considerations

The load-shedding strategy — degrade on purpose. This is the heart of a peak architecture and the part most teams skip. When demand genuinely exceeds capacity, the system must shed the least valuable load first instead of failing uniformly. The retailer encodes a priority order: protect checkout and payment above all, then add-to-cart, then browse, and shed in reverse. Concretely: a virtual waiting room at the Akamai/CloudFront edge admits shoppers into the buy flow at a controlled rate and politely queues the overflow (“you’re in line, ~2 minutes”) rather than letting them all stampede checkout and crash it for everyone. Non-essential features — personalized recommendations, the live “X people viewing” widget, wishlist sync — are toggled off by feature flags the moment latency budgets tighten, freeing capacity for the buy path. CloudFront serves a cached, slightly stale catalogue if Aurora readers are saturated. The principle: a customer who waits two minutes in a tidy queue and then checks out successfully is a sale; a customer who hits a 500 at checkout is a loss and a tweet. Graceful degradation converts the former into the latter’s place.

Security & Zero Trust. Customer traffic is scrubbed at the edge by Akamai’s WAF and bot defenses — at peak, bot and scalper traffic is not noise, it is a material fraction of “demand” and shedding it early protects real capacity. Virtual network appliances (an HA pair of next-gen firewall/WAF instances across AZs) inspect north-south traffic on the admin and partner-integration plane that CloudFront does not front. Wiz runs continuous CSPM and attack-path analysis across the AWS account, alerting on any drift to public exposure or an over-broad IAM role, while Wiz Code shifts that left into the Terraform PR. CrowdStrike Falcon sensors provide runtime threat detection on the Fargate workloads and the admin EC2 hosts, feeding the retailer’s SOC. Any guardrail breach or SLO violation auto-raises a ServiceNow incident, so security and reliability events become tracked tickets rather than log lines lost in the war-room scroll. IAM follows least privilege per service, and OIDC federation means the CI/CD pipeline holds no static AWS keys at all.

Cost optimization. Peak architectures are an exercise in paying for elasticity, not for a permanent peak.

Datadog meters cost-relevant signals — task count over time, DynamoDB consumed capacity, NAT and data-transfer egress — so the post-peak review can show finance exactly what the two-hour surge cost and prove the elastic model beats the over-provisioned one.

Scalability — where each tier tops out. The storefront Fargate tier scales near-linearly on task count and is rarely the ceiling. ElastiCache scales with cluster-mode shards and replicas. Aurora readers scale out to fifteen replicas, so the browse path has enormous headroom; the writer is the real ceiling, which is exactly why SQS, RDS Proxy, and a deliberately oversized writer instance exist — and why the very write-heaviest workloads (the cart) were moved off Aurora to DynamoDB entirely. DynamoDB and SQS are effectively unbounded at this scale. The honest summary: you scale the read path with money and the write path with architecture, and a peak design lives or dies on how much write pressure it can keep away from the single writer.

Reliability & DR (RTO/RPO). Everything spans three Availability Zones: Fargate tasks, Aurora (writer plus replicas with automatic failover, typically under 30 seconds), ElastiCache with Multi-AZ, and the regionally-redundant DynamoDB and SQS. For the order path the durability guarantee is concrete: once SQS has accepted an order message, the order is not lost even if every worker dies — they restart and resume draining. A pragmatic target for the commerce path: RTO under 5 minutes, RPO near zero, because the queue and Aurora’s continuous backup mean accepted orders survive a tier failure. For a full regional event, the catalogue and assets are already global at the CDN, and a warm cross-region Aurora replica plus DynamoDB global tables make a region failover a rehearsed runbook rather than an improvisation. The single most important reliability practice is the game-day: the team load-tests the full stack to above projected peak weeks in advance, deliberately triggers the load-shedding and waiting-room paths, and fixes what breaks while it is cheap to break.

Observability — the war room. The order path is traced end to end in Datadog (Dynatrace is the alternative the team evaluated) with APM: one trace covering edge → storefront → order-capture → SQS → worker → Aurora-commit, so a latency or error regression is attributable to a specific hop. The dashboards that matter during the event are business signals, not just infrastructure: orders per second accepted vs. committed, SQS queue depth and age of oldest message (the canary for whether the writer is keeping up), checkout success rate, p95 add-to-cart and place-order latency, cache hit ratio, and Aurora writer CPU and connection count. Synthetic checks hammer the checkout flow continuously, and SLO monitors page the on-call and open a ServiceNow incident the moment checkout success dips. The growing gap between “orders accepted” and “orders committed” is the single number the war room watches — it is the early warning that the queue is filling faster than it drains, with minutes of runway to act before customers feel it.

The training and runbook angle. A peak event is as much an operations rehearsal as an engineering one. The retailer runs its seasonal fulfillment staff, store associates, and support agents through structured readiness courses on its Moodle LMS — the peak runbook, the escalation tree, how the waiting room looks to a customer so support can answer “am I stuck?” calls calmly — so the humans are as pre-warmed as the Fargate fleet. The on-call engineering rotation drills the same runbook against the game-day environment until the load-shedding toggles and failover steps are muscle memory.

Explicit tradeoffs

Accept these or do not build it. Decoupling orders behind SQS buys survivability at the cost of eventual consistency: the customer is told “order received” before it is durably committed and inventory is decremented, which means you must handle the rare case where an accepted order cannot be fulfilled (oversold stock) with a graceful, automated apology-and-refund path — and you must make every step idempotent so at-least-once delivery never double-charges. Predictive and scheduled scaling demand that you know your event calendar and rehearse; they reward planning and punish the surprise spike you forgot to pre-warm for. The load-shedding and waiting-room machinery is real engineering you build and test — a waiting room nobody has rehearsed is a liability, not a safety net. And the full multi-AZ, predictive-scaled, queue-buffered estate is genuine operational complexity that a small flash-sale site does not need and should not carry.

The alternatives, and when they win. If your peaks are modest and gentle, plain target-tracking autoscaling on Fargate with a healthy Aurora is enough — skip the queue and the waiting room. If you are a smaller team that wants to offload undifferentiated heavy lifting, a managed commerce platform (a hosted storefront, or AWS’s serverless-first patterns end to end) trades control for less to operate at peak. If your write volume is genuinely extreme and relational guarantees are negotiable, going fully event-sourced with DynamoDB and Lambda for the entire order lifecycle removes the Aurora writer ceiling altogether — at the cost of a harder consistency and reporting story. This architecture is the right destination when the stakes are a 40x spike, millions in revenue per hour, and a CEO who has already lived through the outage once.

The shape of the win

For the retailer, the payoff is not “the site stayed up.” It is that at 8:00:00 PM on Black Friday the doorbuster goes live, 120,000 requests per second hit an edge and a fleet that were already provisioned for it, the browse flood is answered from Akamai, CloudFront, and Redis without the database breaking a sweat, every Place Order click is captured in DynamoDB and SQS in under a hundred milliseconds and confirmed instantly, the writer drains the queue at its own steady pace, and when a brief overflow does occur a few thousand shoppers wait politely in a queue for ninety seconds and then check out successfully — instead of ninety minutes of 500s and a ₹22 crore hole. Everything upstream — the predictive pre-warming, the read/write split, the cart on DynamoDB, the orders behind SQS, the Vault-issued credentials, the Wiz and CrowdStrike coverage, the Datadog war-room dashboard, the rehearsed load-shedding — exists so that the single most important number of the year, orders successfully placed in the first ten minutes, goes up instead of to zero. Start narrower if your peak is gentler, but for a brand whose whole year rides on one night, this is where it has to land.

Going deeper

The sections above give you the working architecture. This one is for the engineer who has to make it survive contact with a real 40x event — the internals, quotas, and failure modes that separate a design that looks right on a slide from one that holds at 20:00:00.

Service Quotas — the outage you cause yourself

The most avoidable peak outage is hitting a soft AWS quota you never raised. Every scaling story above has a ceiling that defaults far below Black Friday needs, and some quota increases require human review that takes days — you cannot request them at 19:55. Build a pre-event checklist and request increases a week ahead:

Run Service Quotas (or AWS Trusted Advisor’s limit checks) and diff current usage against the forecast for every service on the path. The quota you forget is the one you page on.

Predictive scaling internals and its blind spot

Predictive scaling trains on up to 14 days of CloudWatch history, produces a 48-hour forecast recomputed hourly, and provisions to meet it ahead of the ramp. Understand two behaviours. First, it models the recurring signal — so an unprecedented event (your first Black Friday, a viral moment) is outside its training distribution and it will under-provision; this is exactly why scheduled actions exist as the deterministic backstop. Second, by default it will not exceed the group’s MaxCapacity; if your forecast legitimately exceeds max, set MaxCapacityBreachBehavior to IncreaseMaxCapacity with a buffer, or the policy silently caps you. Always start in forecast-only mode and eyeball the predicted-vs-actual chart for a cycle or two before letting it act on production.

The backlog-per-task refinement

The worker scaling policy in the implementation section targets the absolute ApproximateNumberOfMessagesVisible, which is simple and works, but it couples the target to fleet size — 1,000 visible messages means something different behind 10 workers than behind 200. The production-grade refinement is the AWS backlog-per-task pattern: publish a CloudWatch metric-math value of queue_depth ÷ running_task_count and target that. Derive the target from your SLA:

acceptableBacklogPerTask = acceptableLatencySeconds ÷ avgProcessingSecondsPerMessage

If you promise orders commit within 60s and each takes 0.3s to process, the target is 60 ÷ 0.3 = 200 messages per task. Now the policy scales to hold drain latency constant regardless of fleet size — which is what you actually promised the business. Keep the fast-out / slow-in cooldowns from the original snippet so a momentary dip does not shed workers mid-drain.

Idempotency is not optional — it is load-bearing

At-least-once delivery plus retries plus a DLQ redrive means every order message can be delivered more than once — a worker can crash after committing to Aurora but before deleting the SQS message, and the next worker will re-process it. Without idempotency that is a double-charge and a double inventory decrement: a customer-trust incident born at peak. Make the commit idempotent with a conditional write keyed on the order’s idempotency key — for example a DynamoDB PutItem with ConditionExpression = attribute_not_exists(order_id), or a unique constraint in Aurora — so the second attempt is a no-op, not a second charge. Idempotency is what makes “at-least-once” safe; treat it as a hard requirement of the queue pattern, not a nice-to-have.

Failure modes to rehearse

• Thundering-herd retries. A brief blip makes thousands of clients retry simultaneously, and the synchronized retry is a second, self-inflicted spike. Mandate exponential backoff with jitter on every client and SDK call; un-jittered retries convert a hiccup into an outage.
• Cache stampede. A hot key expires and thousands of concurrent misses hit Aurora at once. Mitigate with request coalescing (single-flight), slightly randomised TTLs, and serving stale-while-revalidate.
• Scale-in during a lull. A momentary dip mid-event triggers aggressive scale-in, then the crowd returns and you re-warm from cold. Long scale-in cooldowns and high scheduled minimums for the event window prevent the oscillation.
• Metric blind spot. You scaled on CPU, but the bottleneck was database connections; the fleet grew while the writer died. Always scale on the metric that is the constraint — request rate or backlog — and alarm separately on the constraint you are not scaling on (writer connections, in-flight messages).
• Spot reclamation mid-drain. Fargate Spot workers can be reclaimed with a ~2-minute warning (a SIGTERM and task-state change). Trap it, stop receiving new messages, finish in-flight work, and let SQS redeliver anything unfinished — the queue makes Spot safe here precisely because the DLQ and visibility timeout tolerate an interrupted worker.

Cost at the spike, without over-provisioning the year

The whole point of elasticity is paying for 40x only for the hours it exists. Cover the steady baseline with Compute Savings Plans (a one- or three-year commitment on a baseline dollar/hour, ~30–50% off), burst the peak on on-demand Fargate, and run the interruptible order-drain workers on Fargate Spot where the DLQ and visibility timeout make reclamation safe. Let DynamoDB on-demand track real cart traffic so you carry no idle write capacity off-season. Meter task-count-over-time, DynamoDB consumed capacity, and NAT/data-transfer egress so the post-peak finance review can prove the elastic model beat the always-on-40x bill — that evidence is what keeps finance from vetoing the design next year.

Practice challenges

Work these in order — they escalate from reading the architecture to designing under real constraints. Try each before opening the solution.

1. (Beginner) Name the failure. Steady traffic is 3,000 RPS; a doorbuster takes it to 120,000 RPS in 90 seconds. The team has target-tracking autoscaling on average CPU at 50%, a five-minute alarm period, and a five-minute cooldown. In one sentence, why does checkout still fall over?

<details> <summary>Solution</summary>

Target tracking on CPU is reactive and CPU is a lagging metric — by the time CPU crosses 50% and the alarm trips (minutes later), the wall has already arrived and the fleet is chasing a number it passed minutes ago, so requests pile up faster than tasks launch. Why: you cannot out-react a vertical wall; you must have the capacity before it hits (scheduled/predictive pre-scaling) and scale on a leading metric (request rate), not CPU. </details>

2. (Beginner) Pick the primitive. Match each situation to the best scaling lever: (a) a doorbuster at a known 20:00 timestamp, (b) the recurring daily morning rush, © a small residual wobble around a well-provisioned floor, (d) an EC2-backed fleet whose instances take four minutes to boot.

<details> <summary>Solution</summary>

(a) Scheduled scaling — deterministic floor raised before the known time. (b) Predictive scaling — ML learns the recurring shape. © Target tracking — a thermostat for gentle residual changes. (d) Warm pools — pre-initialised instances that skip the four-minute boot on scale-out. Why: each lever is tuned to a different demand shape and reaction speed; peak day uses them layered, not one alone. </details>

3. (Intermediate) Size the concurrency. Your order-capture is a Lambda behind API Gateway. Peak is 120,000 requests/second and the function averages 80 ms. How many concurrent executions do you need, and what two things must you change from defaults before the event?

<details> <summary>Solution</summary>

Concurrency = requests/sec × avg duration in seconds = 120,000 × 0.08 = 9,600 concurrent executions. Before the event: (1) raise the account concurrency quota (default 1,000 → at least ~12,000 with headroom) via Service Quotas, days ahead; (2) set provisioned concurrency and ramp it up on a schedule before 20:00 so the first several thousand requests do not pay a cold start. Why: Little’s Law sets the floor, the default quota throttles you long before it, and provisioned concurrency removes cold-start latency at the wall. </details>

4. (Intermediate) Fix the hot partition. A “units sold” counter for the single trending doorbuster product lives in a DynamoDB item keyed product_id. At peak it throttles hard even though the table’s provisioned capacity is nowhere near exhausted. Diagnose and fix.

<details> <summary>Solution</summary>

All writes hit one partition key, so they funnel onto one physical partition capped at 1,000 write units/second regardless of the table total — the classic hot-partition/hot-key limit that adaptive capacity cannot break for a single key. Fixes: write-shard the counter (product_id#0 … product_id#N, scatter writes across N suffixes, sum on read), or move the fast-moving counter to ElastiCache (INCR) and reconcile to DynamoDB asynchronously. Why: DynamoDB scales by spreading load across many keys; a single hot key defeats that by definition. </details>

5. (Advanced) Design the shed order. Demand genuinely exceeds capacity by 20% for ten minutes. You cannot serve everyone. Write the load-shedding priority order and name one concrete mechanism for each tier you protect or drop.

<details> <summary>Solution</summary>

Protect in this order and shed in reverse: checkout/payment > add-to-cart > browse > non-essential features. Mechanisms: a virtual waiting room at the edge (Akamai/CloudFront) admits shoppers into the buy flow at a controlled rate and queues the overflow (“~2 min”); feature flags switch off recommendations, “X people viewing,” and wishlist sync to free capacity; CloudFront serves a slightly stale cached catalogue if Aurora readers saturate; API Gateway per-method throttling caps search so it cannot starve checkout. Why: a customer who waits two minutes then checks out is a sale; one who gets a 500 at checkout is a loss and a tweet — degrade the cheap stuff to protect the revenue path. </details>

6. (Advanced) Make the queue safe. Order-workers read from SQS, commit to Aurora, then delete the message. A worker crashes after the Aurora commit but before the delete. What happens on redelivery, and what one property must the commit have to make this safe? Sketch the guard.

<details> <summary>Solution</summary>

After the visibility timeout the message becomes visible again and another worker re-processes it — a double order insert and inventory decrement — because SQS is at-least-once. The commit must be idempotent: guard it with a conditional write on the idempotency key, e.g. DynamoDB PutItem with ConditionExpression = "attribute_not_exists(order_id)", or a unique constraint in Aurora, so the second attempt is a no-op. Why: at-least-once delivery guarantees duplicates will happen at peak; idempotency is the property that makes retries and DLQ redrive safe rather than a double-charge incident. </details>

Common beginner mistakes

These are the misconceptions that survive right up until the first real peak, when they become incidents. Each is a wrong mental model, not just a wrong setting.

“Autoscaling means I’m ready for any spike.” The misconception is that any autoscaling policy handles any growth. Reactive target tracking handles a gentle ramp; it structurally cannot handle a vertical wall, because it only acts after the metric has already breached. The right model: on a known wall you pre-provision with scheduled/predictive scaling before the event, and reactive policies only mop up the residual. Autoscaling is a portfolio of policies, not a single switch.

“Scale on CPU — it’s the obvious health metric.” CPU is a lagging indicator for a request-driven workload: utilisation only rises after requests are already queuing inside the process. By the time CPU tells you to scale, customers are already waiting. Scale on the metric that leads demand — ALBRequestCountPerTarget for a web fleet, backlog-per-task for a queue-drain fleet — and reserve CPU for a safety alarm, not the primary scaling signal.

“Over-provisioning for peak is the safe choice.” Running 40x capacity all year “to be safe” is safe only until finance sees the bill and deletes the line item — and it still doesn’t protect the single Aurora writer, which does not scale horizontally on writes no matter how many web tasks you run. The right model: pay for elasticity (scheduled + predictive + Spot), and solve the write tier with architecture (queue, RDS Proxy, cart off DynamoDB), not with a bigger permanent fleet.

“The database will scale with the web tier.” Beginners assume the whole stack scales uniformly. Stateless compute scales linearly and cheaply; the relational writer does not. Under a surge the writer fails first — from connection exhaustion long before CPU — while the web tier looks healthy. The right model: reason about the read path and the write path separately, and design specifically to keep write pressure away from the single writer.

“On-demand DynamoDB means infinite instant capacity.” On-demand adapts, but only up to ~2x a table’s previous peak, and a single low-cardinality (hot) key throttles at the hard per-partition limit (3,000 read / 1,000 write units) no matter how much table capacity you provision. The right model: choose high-cardinality partition keys, write-shard hot keys, and pre-warm or pre-provision tables for a known event.

“SQS decoupling means I don’t have to think about duplicates.” A queue buys survivability, but standard SQS is at-least-once — retries and DLQ redrive guarantee some messages arrive twice. Assuming exactly-once turns a normal redelivery into a double-charge. The right model: idempotency (a conditional write on the order key) is a required part of the queue pattern, not an optional extra.

“A waiting room and load-shedding are things I’ll turn on if needed.” Untested degradation is a liability, not a safety net — a waiting room nobody has rehearsed fails in a novel way exactly when you need it. The right model: build the feature flags and the waiting room in advance and exercise them in a game day, so the shed path is muscle memory before the crowd arrives.

Glossary

Target-tracking scaling — a policy that holds a chosen metric at a target value (like a thermostat), adding or removing capacity to keep it there. Reactive: acts only after the metric moves.

Step scaling — a policy that makes stepped capacity adjustments sized to how badly a CloudWatch alarm is breached (small breach, small add; large breach, large add).

Scheduled scaling — capacity changes triggered by the clock (e.g., raise the minimum at 19:45), with no metric or alarm — the deterministic way to pre-provision for a known event.

Predictive scaling — an ML policy that forecasts demand from historical CloudWatch metrics and provisions ahead of the forecasted ramp; great for recurring patterns, blind to unprecedented spikes.

Warm pool — a set of pre-initialised EC2 instances (Stopped, Hibernated, or Running) held beside an Auto Scaling group so scale-out skips boot-and-bootstrap time.

Lifecycle hook — a pause point at instance launch (Pending:Wait) or termination (Terminating:Wait) that lets automation run setup or graceful drain before the instance is in-service or destroyed.

Little’s Law — concurrency = arrival rate × average service time; the formula that turns “120,000 RPS at 80 ms” into “9,600 concurrent executions.”

Reserved concurrency (Lambda) — a guaranteed, capped slice of the account concurrency pool dedicated to one function (also isolates other functions from it; set to 0 as a kill switch).

Provisioned concurrency (Lambda) — pre-initialised execution environments that eliminate cold starts for the first N concurrent requests; paid-for and scaled ahead of an event.

API Gateway throttling — request-rate limits (account default 10,000 RPS / 5,000 burst for REST APIs, plus per-method limits and usage plans) that return HTTP 429 when exceeded; deliberate throttling is front-door load-shedding.

Queue-based load leveling — placing a queue (SQS) between a fast producer and a slower consumer so the producer’s spike is absorbed and the consumer drains at its own sustainable rate.

Visibility timeout — the window during which a received SQS message is hidden from other consumers; must exceed worker p99 processing time or a message is processed twice.

Dead-letter queue (DLQ) — a companion queue that captures messages which fail maxReceiveCount times, quarantining poison messages so they don’t block the line.

Idempotency key — a unique token per operation that lets a repeated (at-least-once) delivery be safely ignored via a conditional write, preventing double-charges and double-decrements.

On-demand vs. provisioned (DynamoDB) — on-demand charges per request and auto-adapts (up to ~2x prior peak); provisioned sets fixed RCU/WCU that Application Auto Scaling adjusts on a utilisation target.

Hot partition — a physical DynamoDB partition receiving disproportionate traffic; capped at 3,000 read / 1,000 write units per second regardless of table capacity — the ceiling a single hot key hits.

Adaptive capacity (DynamoDB) — the automatic mechanism that shifts capacity toward hot partitions and isolates frequently accessed items; helps skew but cannot break the per-partition limit for one key.

Aurora Capacity Unit (ACU) — the fine-grained scaling unit of Aurora Serverless v2 (≈2 GiB memory each, 0.5 to 256 ACU), adjusted in seconds to track the workload.

Read replica / reader endpoint (Aurora) — up to 15 read-only copies behind a load-balancing endpoint that offload browse queries from the single writer.

RDS Proxy — a managed connection pool in front of RDS/Aurora that multiplexes many client connections onto few backend connections, surviving a connection storm and speeding failover.

Connection storm — a surge of simultaneous new database connections (from many workers or Lambda invocations) that can exhaust max_connections and topple a writer even at low CPU.

Load-shedding — deliberately dropping the least valuable traffic (recommendations, browse) to protect the most valuable (checkout) when demand exceeds capacity, instead of failing uniformly.

Virtual waiting room — an edge mechanism that admits shoppers into the buy flow at a controlled rate and politely queues the overflow, converting a stampede into an orderly line.

Service Quotas — AWS’s per-account, per-region soft limits (Lambda concurrency, Fargate/EC2 vCPU, API Gateway RPS, etc.); many need days of lead time to raise before a peak.

Game day — a rehearsed load test above projected peak that deliberately triggers scaling, failover, and load-shedding paths so failures are found and fixed while they are cheap.