Adds retry support to the Amazon.Lambda.DurableExecution

[GarrettBeatty]

#2216

What

Adds retry support to Amazon.Lambda.DurableExecution on top of #2360. A step that throws can now be retried with configurable backoff and jitter. The Lambda suspends between attempts and is re-invoked by the service when the retry timer fires, so compute is not billed during the wait.

Public API:

Type	Purpose
IRetryStrategy	Decides whether a failed step should retry, with what delay.
RetryDecision	Output of IRetryStrategy.ShouldRetry — ShouldRetry flag plus Delay.
RetryStrategy	Static factory: Default, Transient, None, Exponential(...), FromDelegate(...).
JitterStrategy	None / Half / Full for exponential backoff.
StepSemantics	AtLeastOncePerRetry (default) / AtMostOncePerRetry.
StepConfig.RetryStrategy, StepConfig.Semantics	Per-step retry configuration.

How

When a step throws, StepOperation.HandleStepFailureAsync calls IRetryStrategy.ShouldRetry(ex, attemptNumber). If the strategy says retry, the SDK writes a RETRY checkpoint with NextAttemptDelaySeconds and suspends — RunAsync returns Pending. The service holds the execution until the delay elapses, then re-invokes us. On replay, StepOperation.ReplayAsync sees the PENDING status and either re-suspends (timer not yet up) or re-executes the step with an incremented attempt counter.

 Invocation N                 Service                Invocation N+1
 ────────────                 ───────                ──────────────
 step throws
   │
   ▼
 ShouldRetry(ex, n)? ── no ──► FAIL checkpoint ──►  done
   │ yes
   ▼
 RETRY checkpoint   ─────────► schedule timer
 (NextAttemptDelay)            (delay seconds)
   │
   ▼
 SuspendAndAwait
 returns Pending    ─────────► ...wait...
                               timer fires ──────► ReplayAsync
                                                     │
                                                     ▼
                                                   PENDING + elapsed?
                                                     │
                                                     ▼
                                                   re-execute step
                                                   (attempt n+1)

AtMostOncePerRetry semantics handle non-idempotent steps (charging a card, sending an email). The SDK writes a START checkpoint and blocks until the batcher flushes it before user code runs. If Lambda crashes between user code and the SUCCEED flush, replay sees STARTED with no terminal record and routes the attempt through HandleStepFailureAsync instead of re-executing — the side effect runs at most once per attempt.

ExponentialRetryStrategy supports max attempts, initial/max delay, backoff rate, jitter, and exception filtering by type or message regex. Built-in factories: Default (6 attempts, 5s/60s, 2× backoff, full jitter), Transient (3 attempts, 1s/5s, half jitter), None. RetryStrategy.FromDelegate(...) covers arbitrary policies.

Testing

21 new unit tests in Amazon.Lambda.DurableExecution.Tests (130 total, up from 109 in #2360):

• RetryStrategyTests (14) — exponential backoff math, jitter, max-attempt exhaustion, exception-type and message-pattern filtering, delegate strategies.
• DurableContextTests retry block (6) — checkpoint-and-suspend on retry, fail-without-strategy, retry exhaustion, future/past PENDING replay, AtMostOnce start-flush ordering, STARTED replay routing through the retry handler.

Integration tests in Amazon.Lambda.DurableExecution.IntegrationTests — RetrySucceeds and RetryExhausts run end-to-end against the durable-execution service.

Out of scope (follow-up PRs)

• MapAsync / ParallelAsync / RunInChildContextAsync / WaitForConditionAsync
• CallbackAsync, InvokeAsync
• DefaultJsonCheckpointSerializer
• DurableLogger replay-suppression (currently NullLogger)
• Annotations source-generator integration / [DurableExecution] attribute
• DurableTestRunner / Amazon.Lambda.DurableExecution.Testing package
• dotnet new lambda.DurableFunction blueprint

[Copilot]

Pull request overview

Builds on PR #2360 to add retry support to the Amazon.Lambda.DurableExecution SDK. Failed steps can now be retried with configurable backoff and jitter via service-mediated retries (the SDK checkpoints a RETRY operation and suspends the Lambda so the user is not billed during backoff). Adds at-most-once semantics for non-idempotent steps via a synchronously-flushed START checkpoint that allows crash detection on replay.

Changes:

• New public retry API: IRetryStrategy, RetryDecision, RetryStrategy factories (Default/Transient/None/Exponential/FromDelegate), JitterStrategy, StepSemantics, and StepConfig.RetryStrategy/StepConfig.Semantics.
• StepOperation adds PENDING (retry-timer) and STARTED (AtMostOnce crash-recovery) replay arms, a HandleStepFailureAsync decision tree, and START-checkpoint emission (sync for AtMostOnce, fire-and-forget for AtLeastOnce).
• 21 new unit tests plus integration-test updates asserting StepStarted events and richer history logging.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
Config/IRetryStrategy.cs	New strategy interface + RetryDecision struct
Config/RetryStrategy.cs	ExponentialRetryStrategy, DelegateRetryStrategy, JitterStrategy, StepSemantics, factories
Config/StepConfig.cs	Adds RetryStrategy and Semantics properties
Internal/StepOperation.cs	PENDING/STARTED replay arms, retry decision tree, START-checkpoint emission
Internal/TerminationManager.cs	Adds RetryScheduled termination reason
Internal/CheckpointBatcher.cs	Doc-only update describing fire-and-forget semantics
Tests/RetryStrategyTests.cs	14 unit tests for exponential math/jitter/filters/delegate
Tests/DurableContextTests.cs	6 retry/AtMostOnce/Pending replay tests
Tests/DurableFunctionTests.cs	Updated to assert START + SUCCEED + WAIT-START flat sequence
IntegrationTests/*.cs	Add StepStarted-event assertions; richer history dump in DurableFunctionDeployment

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[GarrettBeatty]

now that we are emitting START steps (which are needed for retries) we are asserting them in the IT tests

[GarrettBeatty]

in previous PR only SUCCEEDED or FAILED mattered. But now for replays, we need to keep track of how many times the function was executed, which is done via the number of STARTED steps.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

[GarrettBeatty]

unknown status i think we should error - need to check

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.