Add ParallelAsync for concurrent branch execution (DOTNET-8662)#2375
Draft
GarrettBeatty wants to merge 1 commit into
Draft
Add ParallelAsync for concurrent branch execution (DOTNET-8662)#2375GarrettBeatty wants to merge 1 commit into
GarrettBeatty wants to merge 1 commit into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
|
|
||
| COPY bin/publish/ ${LAMBDA_TASK_ROOT} | ||
|
|
||
| ENTRYPOINT ["/var/task/bootstrap"] |
GarrettBeatty
force-pushed
the
gcbeatty/durable-parallel
branch
from
19c0128 to
fa13eef
Compare
GarrettBeatty
force-pushed
the
gcbeatty/durable-wave0
branch
from
464c591 to
d308c3b
Compare
GarrettBeatty
force-pushed
the
gcbeatty/durable-parallel
branch
from
fa13eef to
b7a06b4
Compare
GarrettBeatty
force-pushed
the
gcbeatty/durable-wave0
branch
from
d308c3b to
be4c3ad
Compare
Adds parallel branch execution to the .NET Durable Execution SDK.
ParallelAsync runs N branches concurrently with configurable concurrency
limits and completion policies, returning an IBatchResult<T> with
per-branch status and error information.
Per-branch checkpoint payloads are serialized via the ILambdaSerializer
registered on ILambdaContext.Serializer (typically configured through
LambdaBootstrapBuilder.Create(handler, serializer)), matching the
StepAsync / RunInChildContextAsync pattern. There are no separate
reflection / AOT-safe overload pairs: the AOT story is determined
entirely by which serializer the user registers with the runtime.
Public surface:
- IDurableContext.ParallelAsync<T> (2 overloads: Func[] vs
DurableBranch<T>[])
- DurableBranch<T> record (Name + Func)
- ParallelConfig (MaxConcurrency, CompletionConfig, NestingType)
- CompletionConfig with factories AllSuccessful() / FirstSuccessful() /
AllCompleted(); ToleratedFailureCount / ToleratedFailurePercentage
(validated 0.0-1.0)
- IBatchResult<T> with All / Succeeded / Failed / Started accessors,
GetResults, GetErrors, ThrowIfError, HasFailure, CompletionReason,
count properties
- IBatchItem<T> with Index, Name, Status, Result, Error
- BatchItemStatus { Succeeded, Failed, Started }
- CompletionReason { AllCompleted, MinSuccessfulReached,
FailureToleranceExceeded }
- NestingType (Nested default; Flat throws NotSupportedException - reserved)
- ParallelException (carries IBatchResult; future-subclassable)
Internal:
- ParallelOperation<T> orchestrator dispatches branches with optional
semaphore-bounded concurrency. Each branch runs as a
ChildContextOperation<T> with deterministic ID via
OperationIdGenerator.CreateChild.
- Branch failures aggregated as IBatchItem<T> entries; orchestrator
throws ParallelException only when CompletionConfig signals
FailureToleranceExceeded.
- Parent CONTEXT checkpoint records summary (CompletionReason +
per-branch index/name/status); branch results live on per-branch
CONTEXT checkpoints.
- ExecutionState now thread-safe (lock around reads/writes of
_operations, _visitedOperations, _isReplaying). Required for
concurrent branch replay; affects all operations but no regressions.
- ParallelOperation awaits Task.WhenAll(inFlight) before disposing
the semaphore so cancellation/exception during dispatch lets
in-flight branches settle cleanly.
- Reuses OperationSubTypes.Parallel / OperationSubTypes.ParallelBranch
from Wave 0.
Adds 31 unit tests + 6 integration tests covering CompletionConfig
matrix, MaxConcurrency, FirstSuccessful short-circuit, replay
determinism, mixed-status replay, cancellation, and concurrency
stress.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GarrettBeatty
force-pushed
the
gcbeatty/durable-parallel
branch
from
b7a06b4 to
08b2095
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#2216
Summary
Adds parallel branch execution to the .NET Durable Execution SDK.
ParallelAsyncruns N branches concurrently with configurable concurrency limits and completion policies, returning anIBatchResult<T>with per-branch status and error information.Per-branch checkpoint payloads are serialized via the
ILambdaSerializerregistered onILambdaContext.Serializer(typically configured throughLambdaBootstrapBuilder.Create(handler, serializer)), matching the StepAsync / RunInChildContextAsync pattern. There are no separate reflection / AOT-safe overload pairs: the AOT story is determined entirely by which serializer the user registers with the runtime (e.g.,SourceGeneratorLambdaJsonSerializer<TContext>for AOT scenarios).Stacked on top of #2372 (Wave 0 cross-cutting types).
Fixes DOTNET-8662.
The shared
IBatchResult<T>family added here will be reused by MapAsync (Wave 2).Public surface
IDurableContext.ParallelAsync<T>(2 overloads:Func[]vsDurableBranch<T>[])DurableBranch<T>record (Name + Func)ParallelConfig(MaxConcurrency, CompletionConfig, NestingType)CompletionConfigwith factoriesAllSuccessful()/FirstSuccessful()/AllCompleted();ToleratedFailureCount/ToleratedFailurePercentage(validated 0.0-1.0)IBatchResult<T>withAll/Succeeded/Failed/Startedaccessors,GetResults,GetErrors,ThrowIfError,HasFailure,CompletionReason, count propertiesIBatchItem<T>withIndex,Name,Status,Result,ErrorBatchItemStatus { Succeeded, Failed, Started }CompletionReason { AllCompleted, MinSuccessfulReached, FailureToleranceExceeded }NestingType(Nested default; Flat throwsNotSupportedException- reserved for a follow-up)ParallelException(carriesIBatchResult; future-subclassable)Internal
ParallelOperation<T>orchestrator dispatches branches with optional semaphore-bounded concurrency. Each branch runs as aChildContextOperation<T>with a deterministic ID viaOperationIdGenerator.CreateChild.IBatchItem<T>entries; orchestrator throwsParallelExceptiononly whenCompletionConfigsignalsFailureToleranceExceeded.ExecutionStatenow thread-safe (lock around reads/writes of_operations,_visitedOperations,_isReplaying). Required for concurrent branch replay; affects all operations but no regressions.ParallelOperationawaitsTask.WhenAll(inFlight)before disposing the semaphore so cancellation/exception during dispatch lets in-flight branches settle cleanly.OperationSubTypes.Parallel/OperationSubTypes.ParallelBranchfrom Wave 0.Test plan
Generated with Claude Code