Table of Contents

Struct ScanCommand<T>

Namespace
DynaSharp
Assembly
DynaSharp.dll

A Scan under construction: every item of a table or index, in no useful order, at the cost of reading them all. Chain the options and end with ItemsAsync(CancellationToken), PagesAsync(CancellationToken), PageAsync(ContinuationToken, CancellationToken) or CountAsync(CancellationToken) in one expression.

public readonly ref struct ScanCommand<T> : IFilterable<ScanCommand<T>>, IRetryable<ScanCommand<T>> where T : IDynamoDbDocument<T>

Type Parameters

T

How the items are read.

Implements
Inherited Members

Examples

var holders = customers.Scan().Filter(Customer.Attrs.Email.Eq("alice@example.com")).ItemsAsync(ct);

Remarks

A scan is what a query is without its key condition, and QueryCommand<T> says how pages and discriminators work. It charges capacity for the whole table whatever the filter leaves, so a scan that runs often is better as a query of an index.

To go faster, Parallel(int) reads several segments at once and merges them. Segment(int, int) reads one segment, for workers that each own one.

Methods

Consistent()

Asks for a strongly consistent read, which sees every write that succeeded before it started and costs twice the capacity. A read is eventually consistent otherwise. A global secondary index is always eventually consistent, so this throws on one that is known to be global, which is an index opened through its entity or one with another partition key than the table, and DynamoDB answers with a ValidationException for any other.

public ScanCommand<T> Consistent()

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

InvalidOperationException

The index is known to be global.

CountAsync(CancellationToken)

Counts the items of all pages with Select: COUNT, so no item travels. It costs the capacity of reading the whole table, as many calls as a scan takes, and the count of a table that changes meanwhile is only an estimate. Limit(int) sets the size of each page and not the count.

public ValueTask<int> CountAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancels the read.

Returns

ValueTask<int>

The number of items that pass the filter. Await it once.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered, or it has a projection, which is no use for a count and which DynamoDB rejects.

ObjectDisposedException

The client is disposed.

ValidationException

DynamoDB rejects the request; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

OverflowException

There are more items than an int holds.

Filter(ExpressionHandler)

Adds a filter in the grammar of DynamoDB, for what the conditions of handles do not cover: .Filter($"size({Attr("tags")}) < {10}"). An interpolated Attr stands for an attribute name and any other interpolated value for a value. DynamoDB applies it after reading, so capacity is charged for every item scanned, not for those returned, and a page can come back empty. A second filter means both must hold.

public ScanCommand<T> Filter(ExpressionHandler condition)

Parameters

condition ExpressionHandler

The condition.

Returns

ScanCommand<T>

The command, which replaces this one.

Filter<TCondition>(TCondition)

Adds a filter. DynamoDB applies it after reading, so it shortens the result, not the read: capacity is charged for every item scanned, and a page can come back empty. A second filter means both must hold.

public ScanCommand<T> Filter<TCondition>(TCondition condition) where TCondition : ICondition

Parameters

condition TCondition

The condition.

Returns

ScanCommand<T>

The command, which replaces this one.

Type Parameters

TCondition

The type of the condition.

ItemsAsync(CancellationToken)

Reads all items of all pages as they come, following pages that a filter left empty. Nothing is sent until the enumeration starts, and only what the enumeration reaches is read, so breaking out early saves the remaining pages. Capacity is charged for the items scanned, whatever the filter leaves; it is not reported here, only on the spans and the metric, see PagesAsync(CancellationToken).

public IAsyncEnumerable<T> ItemsAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancels the read, as does WithCancellation on the result. It is looked at per page.

Returns

IAsyncEnumerable<T>

The items, which can be enumerated once; a second enumeration throws InvalidOperationException, so build the command again to read again.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.

Limit(int)

Limits the items DynamoDB reads per page, before the filter. It does not limit the result: ItemsAsync(CancellationToken) reads on until the end, in pages of this size. A cap on items is .ItemsAsync(ct).Take(n), which reads until it has them, across empty filtered pages, and stops. A small limit is a way to spread a scan out over time, because each page is one call.

public ScanCommand<T> Limit(int items)

Parameters

items int

The most items per page; per segment, for a segmented scan.

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

items is below 1.

MaxAttempts(int)

Sets the number of attempts of this call including the first, in place of MaxAttempts. A retry after a lost response repeats a write that may have been applied, so 1 makes an ADD count at most once, and a lost response then reports a failure after the write was applied.

public ScanCommand<T> MaxAttempts(int attempts)

Parameters

attempts int

The number of attempts.

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

attempts is below 1.

OnMappingFailure(Action<DynamoDbMappingException>)

Skips an item that cannot be read as T and hands its DynamoDbMappingException to onFailure, where the read would otherwise end with it: .OnMappingFailure(failure => bad.Add(failure.Key)), or .OnMappingFailure(static _ => { }) to skip and count only. Ending the read stays the default, because an item that does not fit is a bug of the model or of a writer, which a read that drops items would hide.

public ScanCommand<T> OnMappingFailure(Action<DynamoDbMappingException> onFailure)

Parameters

onFailure Action<DynamoDbMappingException>

What to do with the failure of a skipped item.

Returns

ScanCommand<T>

The command, which replaces this one.

Remarks

The exception says which item it was and why: EntityType, AttributeName, the message, and Key, the key attributes of the table as they are stored, such as USER#alice, also on an index. The key holds values, so nothing logs it: the callback decides where it goes, and the message holds none. After a Select the response holds no key, so Key is null there.

It runs once per skipped item, when the page that holds it has arrived and before its items are given out, outside the call: it is no part of a retry, and an exception it throws ends the read as it is. The pages of a parallel scan and of a fan-out arrive on several threads at once, so there it can run concurrently. Each item it returns from counts in the metric dynasharp.client.operation.skipped_items.

What a skipped item does to the rest of the read: it is not in Items, and Count still counts it, as DynamoDB does, so Count - Items.Count is what a page skipped. Limit(int) counts it, because DynamoDB read it, so a page can hold fewer items. ItemsAsync(ct).Take(n) counts what it gets, so it reads on past a skipped item, and ItemAsync(CancellationToken) gives the first item that can be read. Next continues after the page, so a skipped item is not read again. CountAsync(CancellationToken) reads no item and counts it.

Only a DynamoDbMappingException is skipped: an item of another type in a read keyed by the discriminator, a value a converter cannot read, a missing required attribute, a stored key that does not fit its template. It covers every way of sending the read that gives items, also those of a Select after it. A second call replaces the first.

Exceptions

ArgumentNullException

onFailure is null.

PageAsync(ContinuationToken, CancellationToken)

Reads one page, for paging that a client drives across requests: build the same scan each time and pass the position of the page before. The page can be empty and still have a next one, so a client that wants items keeps going while HasMore holds.

public ValueTask<Page<T>> PageAsync(ContinuationToken after = default, CancellationToken cancellationToken = default)

Parameters

after ContinuationToken

The position to continue from, or default for the first page. It must come from the same scan, and the same segment. A ContinuationToken? passes as after ?? default.

cancellationToken CancellationToken

Cancels the call.

Returns

ValueTask<Page<T>>

The page. Await it once.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.

ObjectDisposedException

The client is disposed.

FormatException

after does not fit what TokenSigner says: a client that signs gets a plain token that came as text, or one whose signature does not hold for this read; a client that does not sign gets a signed one. The message quotes nothing of the token, and no request goes out.

ValidationException

DynamoDB rejects the request, for example a position of another scan; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbMappingException

An item cannot be read as T.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

PageAsync(CancellationToken)

Reads the first page, as PageAsync(ContinuationToken, CancellationToken) does at the default position, for a caller that has a token and no position yet.

public ValueTask<Page<T>> PageAsync(CancellationToken cancellationToken)

Parameters

cancellationToken CancellationToken

Cancels the call. It has no default, because PageAsync() belongs to the overload with the position.

Returns

ValueTask<Page<T>>

The page. Await it once.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.

ObjectDisposedException

The client is disposed.

ValidationException

DynamoDB rejects the request; the exception shows the expressions.

ResourceNotFoundException

The table does not exist.

DynamoDbMappingException

An item cannot be read as T.

DynamoDbException

DynamoDB returns any other error; a throttling or server error only after the last attempt.

PagesAsync(CancellationToken)

Reads all pages as they come, for code that wants ConsumedCapacity or works page by page. It gives every response, also one that a filter left empty. Otherwise as ItemsAsync(CancellationToken).

public IAsyncEnumerable<Page<T>> PagesAsync(CancellationToken cancellationToken = default)

Parameters

cancellationToken CancellationToken

Cancels the read, as does WithCancellation on the result.

Returns

IAsyncEnumerable<Page<T>>

The pages, which can be enumerated once.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.

Parallel(int)

Reads the whole scan in segments at once and merges them. The result is another command, which has only ItemsAsync(CancellationToken): a merged page has no single key to continue from. It does not change what is read or what capacity it costs, it spends the same capacity sooner, so it can use up the throughput of a table that serves other traffic.

public ParallelScanCommand<T> Parallel(int segments)

Parameters

segments int

The number of segments, which is also the most requests in flight at once, from 1 to 1,000,000.

Returns

ParallelScanCommand<T>

The command of the parallel scan, which replaces this one.

Exceptions

ArgumentOutOfRangeException

segments is outside its range.

InvalidOperationException

The command was already sent, extended or rendered, or Segment(int, int) chose one segment.

Project(params ReadOnlySpan<Attr>)

Reads only some attributes, given as handles: Project(Order.Attrs.Status, Order.Attrs.Total). See Project(params ReadOnlySpan<string>).

public ScanCommand<T> Project(params ReadOnlySpan<Attr> attributes)

Parameters

attributes ReadOnlySpan<Attr>

The attributes. A second call adds to the first.

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

ArgumentException

A path is not valid.

Project(params ReadOnlySpan<string>)

Reads only some attributes. It saves bandwidth, not capacity: DynamoDB charges for whole items. A read of an entity needs its discriminator and its keys, so they join the projection on their own, and it needs whatever else the entity requires, so a projection is mostly for Item. A global secondary index gives only what it projects.

public ScanCommand<T> Project(params ReadOnlySpan<string> paths)

Parameters

paths ReadOnlySpan<string>

The names or document paths. Reserved words need no care, and a second call adds to the first.

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

ArgumentException

A path is not valid.

ReturnCapacity(ReturnConsumedCapacity)

Asks for the consumed capacity of each page, which goes to ConsumedCapacity, the span of the page and the capacity metric. ItemsAsync(CancellationToken) has no result to put it on, so there it reaches telemetry only.

public ScanCommand<T> ReturnCapacity(ReturnConsumedCapacity detail = ReturnConsumedCapacity.Total)

Parameters

detail ReturnConsumedCapacity

How much detail to return.

Returns

ScanCommand<T>

The command, which replaces this one.

Segment(int, int)

Reads one segment of a scan that several workers share. DynamoDB splits the table into of parts, and every worker reads its own, with the same number of segments and the same options. A segment can be empty, and HasMore is per segment.

public ScanCommand<T> Segment(int segment, int of)

Parameters

segment int

The segment, from zero.

of int

The number of segments, from 1 to 1,000,000.

Returns

ScanCommand<T>

The command, which replaces this one.

Exceptions

ArgumentOutOfRangeException

of is outside its range, or segment is not one of the segments.

Select<TValue, TConverter>(Attr<TValue, NullableHandleConverter<TValue, TConverter>>)

Reads one nullable attribute instead of the entity, which gives null for an item without it. See Select<TValue, TConverter>(Attr<TValue, TConverter>).

public ScanSelectCommand<TValue?, NullableConverter<TValue, TConverter>> Select<TValue, TConverter>(Attr<TValue, NullableHandleConverter<TValue, TConverter>> attribute) where TValue : struct where TConverter : IDynamoDbConverter<TValue>

Parameters

attribute Attr<TValue, NullableHandleConverter<TValue, TConverter>>

The attribute, a name or a path through maps.

Returns

ScanSelectCommand<TValue?, NullableConverter<TValue, TConverter>>

The command that is left.

Type Parameters

TValue

The type of the attribute.

TConverter

The converter of the attribute.

Exceptions

ArgumentException

The handle is a default value, its path has a list index, or it is not valid.

InvalidOperationException

The read has a Project, which has nothing to add here.

Select<TValue, TConverter>(Attr<TValue, NullableReferenceHandleConverter<TValue, TConverter>>)

Reads one nullable attribute instead of the entity, which gives null for an item without it. See Select<TValue, TConverter>(Attr<TValue, TConverter>).

public ScanSelectCommand<TValue?, NullableReferenceConverter<TValue, TConverter>> Select<TValue, TConverter>(Attr<TValue, NullableReferenceHandleConverter<TValue, TConverter>> attribute) where TValue : class where TConverter : IDynamoDbConverter<TValue>

Parameters

attribute Attr<TValue, NullableReferenceHandleConverter<TValue, TConverter>>

The attribute, a name or a path through maps.

Returns

ScanSelectCommand<TValue, NullableReferenceConverter<TValue, TConverter>>

The command that is left.

Type Parameters

TValue

The type of the attribute.

TConverter

The converter of the attribute.

Exceptions

ArgumentException

The handle is a default value, its path has a list index, or it is not valid.

InvalidOperationException

The read has a Project, which has nothing to add here.

Select<TValue, TConverter>(Attr<TValue, TConverter>)

Reads one attribute of each item instead of the whole entity: await foreach (var total in orders.Scan().Select(Order.Attrs.Total).ValuesAsync(ct)). It saves bandwidth, not capacity: DynamoDB charges for whole items.

public ScanSelectCommand<TValue, TConverter> Select<TValue, TConverter>(Attr<TValue, TConverter> attribute) where TConverter : IDynamoDbConverter<TValue>

Parameters

attribute Attr<TValue, TConverter>

The attribute, a name or a path through maps. A list element cannot be read: DynamoDB returns a shortened list, in which its position is not known.

Returns

ScanSelectCommand<TValue, TConverter>

The command that is left.

Type Parameters

TValue

The type of the attribute.

TConverter

The converter of the attribute.

Remarks

Over Project. The result is the value and no entity, so a required member cannot be in the way. The attribute is projected and nothing else: the filter on the discriminator keeps items of other types away.

Last clause. After Select only the send is left, as after Return: Filter<TCondition>(TCondition), Limit(int), Consistent(), Segment(int, int), ReturnCapacity(ReturnConsumedCapacity) and MaxAttempts(int) come before this, a second Select and a Project after it do not compile, and a Project before it throws.

Last clause. After Select only the send is left, as after Return: Consistent(), ReturnCapacity(ReturnConsumedCapacity) and MaxAttempts(int) come before this, a second Select and a Project after it do not compile, and a Project before it throws.

A missing attribute follows the type of the handle: null for one of a nullable property, an empty set for a set, because DynamoDB has none, and otherwise a DynamoDbMappingException. Never a default, and that is where this differs from a read of the entity, which leaves a property that is not required at its initial value: an int without its attribute is 0 there and throws here, and so does a [Version] on an item without one, which code of another kind writes, with a message that points at VersionMissing.Allow. In a nested path the last handle decides, so ShipTo.At(Address.Attrs.City) throws for an item without ShipTo when City cannot be null, which reading ShipTo itself would not.

A key input. A [KeyOnly] property has no attribute and no handle. A stored property that a key of the table holds has both, and an item that an update created has only the key: a read of the entity restores the property from it, and this throws, which DYNA209 reports.

Exceptions

ArgumentException

The handle is a default value, its path has a list index, or it is not valid.

InvalidOperationException

The read has a Project, which has nothing to add here.

ToRequestJson()

Renders the body of the first page, as DynamoDB JSON, without sending it. It ends the command, and is meant for tests and debugging. The text contains the values.

public string ToRequestJson()

Returns

string

The request body.

Exceptions

InvalidOperationException

The command was already sent, extended or rendered.