Table of Contents

Struct FanOutQueryCommand<T>

Namespace
DynaSharp
Assembly
DynaSharp.dll

A Query of several partitions at once, merged by sort key, the read of write shards and time buckets: orders.Query(shards, SortKeyCondition.Ge(from)).Descending().ItemsAsync(ct). Chain the options and end with ItemsAsync(CancellationToken), PageAsync(int, ContinuationToken, CancellationToken) or CountAsync(CancellationToken) in one expression.

public readonly ref struct FanOutQueryCommand<T> : IFilterable<FanOutQueryCommand<T>>, IRetryable<FanOutQueryCommand<T>> where T : IDynamoDbDocument<T>

Type Parameters

T

How the items are read.

Implements
Inherited Members

Remarks

It is one query per partition. A fan-out costs at least one read per partition, and each page of each partition is one call, with its own span, retries and metrics, and its own capacity. It is not a transaction and not a snapshot: partitions are read at different times. The default of a read is eventually consistent, see Consistent().

Items merge in the order DynamoDB gives within a partition: strings and binary values by their bytes, so that Z comes before a, and numbers by value, at their full 38 digits. Items with equal sort keys come by the position of their partition in the list, and Descending() reverses both. A table or index without a sort key has only ties, so its partitions come one after the other.

A partition is read when the merge needs it. To give the first item it needs the first page of every partition, because any of them could hold what comes first. After that it reads the next page of a partition only when that partition's turn comes, so the partitions that have much to give are the ones that are read, and leaving the enumeration early reads nothing more. At most one page per partition is in memory, which is what makes Limit(int) worth setting when there are many partitions.

As with any read of an entity with a discriminator, the filter holds it, see QueryCommand<T>. A filter that passes little makes a partition follow its pages until it passes something, or until the key it has read up to sorts after what the other partitions offer. See GetCommand<T> for what a command is and what it cannot do.

Methods

Consistent()

Asks for a strongly consistent read of each partition, which sees every write that succeeded before that read started and costs twice the capacity. Partitions are still read at different times.

public FanOutQueryCommand<T> Consistent()

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Remarks

A global secondary index is always eventually consistent, so this throws on one that is known to be global, which is an index opened through its entity or one with another partition key than the table, and DynamoDB answers with a ValidationException for any other.

Exceptions

InvalidOperationException

The index is known to be global.

CountAsync(CancellationToken)

Counts the items of all partitions with Select: COUNT, so no item travels. It reads every partition to its end, up to MaxConcurrency(int) at once, at the capacity of reading them all, as a count of a filtered read costs the items scanned. With no partitions it sends nothing and gives 0.

public ValueTask<int> CountAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancels the read.

Returns

ValueTask<int>

The number of items that pass the filter. Await it once.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered, or it has a projection, which is no use for a count and which DynamoDB rejects.

ObjectDisposedException

The client is disposed.

ValidationException

DynamoDB rejects a request; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

OverflowException

There are more items than an int holds.

Descending()

Reads every partition downwards and merges from the top, so the newest of all partitions come first. Items with equal sort keys come by falling position, so the result is the exact reverse of the ascending one.

public FanOutQueryCommand<T> Descending()

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Filter(ExpressionHandler)

Adds a filter in the grammar of DynamoDB, for what the conditions of handles do not cover: .Filter($"size({Attr("tags")}) < {10}"). DynamoDB applies it after reading, so capacity is charged for every item read, not for those returned.

public FanOutQueryCommand<T> Filter(ExpressionHandler condition)

Parameters

condition ExpressionHandler

The condition.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Remarks

An interpolated Attr stands for an attribute name and any other interpolated value for a value. A page can come back empty. A filter cannot name a key of the table or index that is read; that is what the key condition is for. A second filter means both must hold.

Filter<TCondition>(TCondition)

Adds a filter. DynamoDB applies it after reading, so it shortens the result, not the read: capacity is charged for every item scanned, and a page can come back empty. A second filter means both must hold.

public FanOutQueryCommand<T> Filter<TCondition>(TCondition condition) where TCondition : ICondition

Parameters

condition TCondition

The condition.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Type Parameters

TCondition

The type of the condition.

ItemsAsync(CancellationToken)

Reads the merged items as they come, following pages that a filter left empty. Nothing is sent until the enumeration starts, a request goes out only when the merge cannot go on without it, and between two items no request is in flight, so breaking out early saves everything that was not read yet and leaves nothing behind. With no partitions there is no item and no request.

public IAsyncEnumerable<T> ItemsAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancels the read, as does WithCancellation on the result. It is looked at per item.

Returns

IAsyncEnumerable<T>

The items, which can be enumerated once; a second enumeration throws InvalidOperationException, so build the command again to read again.

Remarks

The first failure stops the requests in flight and surfaces once, from MoveNextAsync, as the exception it is. Cancelling stops them too. Items that were merged before that have been given out, and there is no way to continue: for that, read in pages with PageAsync(int, ContinuationToken, CancellationToken).

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.

Limit(int)

Limits the items DynamoDB reads per page of each partition, before the filter. It does not limit the result, and it is not the size of a merged page, which PageAsync(int, ContinuationToken, CancellationToken) takes. A cap on items is .ItemsAsync(ct).Take(n), which stops when it has them. A small limit reads less ahead of what the merge gives out, and costs more calls.

public FanOutQueryCommand<T> Limit(int items)

Parameters

items int

The most items per page of a partition.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

items is below 1.

MaxAttempts(int)

Sets the number of attempts of this call including the first, in place of MaxAttempts. A retry after a lost response repeats a write that may have been applied, so 1 makes an ADD count at most once, and a lost response then reports a failure after the write was applied.

public FanOutQueryCommand<T> MaxAttempts(int attempts)

Parameters

attempts int

The number of attempts.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

attempts is below 1.

MaxConcurrency(int)

Sets how many requests may be in flight at once, 8 unless set. It takes effect where several partitions are needed at once, above all at the start of a read and in CountAsync(CancellationToken). It is the way to keep a fan-out over many partitions from using up the throughput of a table.

public FanOutQueryCommand<T> MaxConcurrency(int requests)

Parameters

requests int

The most requests at once.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

requests is below 1.

OnMappingFailure(Action<DynamoDbMappingException>)

Skips an item that cannot be read as T and hands its DynamoDbMappingException to onFailure, where the read would otherwise end with it. Ending the read stays the default. See OnMappingFailure(Action<DynamoDbMappingException>) for what the exception holds and when the callback runs.

public FanOutQueryCommand<T> OnMappingFailure(Action<DynamoDbMappingException> onFailure)

Parameters

onFailure Action<DynamoDbMappingException>

What to do with the failure of a skipped item.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Remarks

The pages of several partitions arrive at once, so the callback can run on several threads at once. A skipped item has no place in the merge and is in neither Items nor Count, which counts what the page gives. Next continues after the last item the caller got from each partition, so a skipped item behind that item can be read again by the next page, and the callback has it again. Limit(int) counts it, Take(n) does not, and CountAsync(CancellationToken) counts it. An item without the keys the merge orders by is skipped the same way.

Exceptions

ArgumentNullException

onFailure is null.

PageAsync(int, ContinuationToken, CancellationToken)

Reads one page of the merge, for paging that a client drives across requests: build the same query each time, with the same partitions in the same order and the same direction, and pass the position of the page before. It reads on through empty pages until the page is full or the partitions are done, so it can make many calls, and only a page that holds fewer items than pageSize is sure to be the last.

public ValueTask<Page<T>> PageAsync(int pageSize, ContinuationToken after = default, CancellationToken cancellationToken = default)

Parameters

pageSize int

The items of the page.

after ContinuationToken

The position to continue from, or default for the first page. It must come from the same query. A ContinuationToken? passes as after ?? default.

cancellationToken CancellationToken

Cancels the read.

Returns

ValueTask<Page<T>>

The page. Await it once.

Remarks

Next holds one entry per partition: the key of the last item the caller got from it and the key of the item that comes next, or the fact that it is done. What a partition had read beyond the page is read again when its turn comes, so no item is lost or repeated. Until then the second key lets the merge pass it by, so a partition that sorts later, a time bucket for example, costs no read for the pages before it, while a partition that sorts in between is read on the page that reaches it. Its text grows with the number of partitions, by two keys each, and like every token it shows the key values. A client with TokenSigner signs it over the partition keys of the query as well, see ContinuationToken. If the table changes between pages, the merge goes by the keys of the token, so an item written before the next key can come later than its place.

Unless Limit(int) says otherwise, each partition is read in pages of pageSize, because it cannot give more to this page, and the capacity of reading on would be wasted. ScannedCount and, with ReturnCapacity(ReturnConsumedCapacity), ConsumedCapacity count what all its calls read.

Exceptions

ArgumentOutOfRangeException

pageSize is below 1.

FormatException

after is not a position of a fan-out, or of one with this number of partitions and this direction, the same as Parse(string?) for text that is no token, or it does not fit what TokenSigner says, as for a plain query. It comes from a client, so this is what bad input from there looks like, when the awaited task ends.

InvalidOperationException

The command was already sent, extended or rendered.

ObjectDisposedException

The client is disposed.

ValidationException

DynamoDB rejects a request, for example a position of another query; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbMappingException

An item cannot be read as T.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

PageAsync(int, CancellationToken)

Reads the first page, as PageAsync(int, ContinuationToken, CancellationToken) does at the default position, for a caller that has a token and no position yet.

public ValueTask<Page<T>> PageAsync(int pageSize, CancellationToken cancellationToken)

Parameters

pageSize int

The items of the page.

cancellationToken CancellationToken

Cancels the read. It has no default, because PageAsync(pageSize) belongs to the overload with the position.

Returns

ValueTask<Page<T>>

The page. Await it once.

Exceptions

ArgumentOutOfRangeException

pageSize is below 1.

InvalidOperationException

The command was already sent, extended or rendered.

ObjectDisposedException

The client is disposed.

ValidationException

DynamoDB rejects a request; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbMappingException

An item cannot be read as T.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

Project(params ReadOnlySpan<Attr>)

Reads only some attributes, given as handles: Project(Order.Attrs.Status, Order.Attrs.Total). See Project(params ReadOnlySpan<string>).

public FanOutQueryCommand<T> Project(params ReadOnlySpan<Attr> attributes)

Parameters

attributes ReadOnlySpan<Attr>

The attributes. A second call adds to the first.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Exceptions

ArgumentException

A path is not valid.

Project(params ReadOnlySpan<string>)

Reads only some attributes. It saves bandwidth, not capacity: DynamoDB charges for whole items. The merge needs the keys of an item, so they join the projection on their own, also for Item, where they show up in the result, and a read of an entity needs its discriminator as well. A global secondary index gives only what it projects.

public FanOutQueryCommand<T> Project(params ReadOnlySpan<string> paths)

Parameters

paths ReadOnlySpan<string>

The names or document paths. Reserved words need no care, and a second call adds to the first.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

Exceptions

ArgumentException

A path is not valid.

ReturnCapacity(ReturnConsumedCapacity)

Asks for the consumed capacity of each call, for the span of that call and the capacity metric. PageAsync(int, ContinuationToken, CancellationToken) also adds up its calls in ConsumedCapacity: units add, details add by the name of the index, and a number that a response leaves out counts as zero. That includes pages whose items did not make it into the result, so it is what the page cost, and a read again on the next page is charged again. The other ways of sending it have no result to put it on.

public FanOutQueryCommand<T> ReturnCapacity(ReturnConsumedCapacity detail = ReturnConsumedCapacity.Total)

Parameters

detail ReturnConsumedCapacity

How much detail to return.

Returns

FanOutQueryCommand<T>

The command, which replaces this one.

ToRequestJson()

Renders the body of the first page of the first partition, as DynamoDB JSON, without sending it; the other partitions differ only in :pk. It ends the command and is meant for tests and debugging. The text contains the key and the values. With no partitions, :pk has no value.

public string ToRequestJson()

Returns

string

The request body.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.