Class EnumeratorTokenStream<TToken>
An ITokenStream<TToken> implementation based on an IEnumerator<T>.
Inheritance
- Object
- EnumeratorTokenStream<TToken>
Implements
- ITokenStream<TToken>
Declaration
public class EnumeratorTokenStream<TToken> : Object, ITokenStream<TToken>Type Parameters
| Name | Description |
|---|---|
TToken |
The type of tokens in the enumerator |
Constructors
EnumeratorTokenStream(IEnumerator<TToken>)
Creates an ITokenStream<TToken> implementation based on an IEnumerator<T>.
Declaration
public EnumeratorTokenStream(IEnumerator<TToken> input)Parameters
| Type | Name | Description |
|---|---|---|
IEnumerator<TToken> |
input |
The IEnumerator<T>. |
Properties
ChunkSizeHint
Returns 16.
Declaration
public int ChunkSizeHint { get; }Property Value
| Type | Description |
|---|---|
16 |
Methods
Read(Span<TToken>)
Read up to buffer.Length tokens into buffer.
Return the actual number of tokens read, which may be fewer than
the size of the buffer if the stream has reached the end.
Declaration
public int Read(Span<TToken> buffer)Parameters
| Type | Name | Description |
|---|---|---|
Span<TToken> |
buffer |
The buffer to read tokens into. |
Returns
| Type | Description |
|---|---|
The actual number of tokens read. |