Pidgin

Class EnumeratorTokenStream<TToken>

An ITokenStream<TToken> implementation based on an IEnumerator<T>.

Inheritance
  • object
  • EnumeratorTokenStream<TToken>
Implements
Inherited Members
Declaration
public class EnumeratorTokenStream<TToken> : ITokenStream<TToken>
Type Parameters
Name Description

TToken

The type of tokens in the enumerator.

Constructors

EnumeratorTokenStream(IEnumerator<TToken>)

Creates an ITokenStream<TToken> implementation based on an IEnumerator<T>.

Declaration
public EnumeratorTokenStream(IEnumerator<TToken> input)
Parameters
Type Name Description

IEnumerator<TToken>

input

The IEnumerator<T>.

Properties

ChunkSizeHint

Returns 16.

Declaration
public int ChunkSizeHint { get; }
Property Value
Type Description

int

Methods

Read(Span<TToken>)

Read up to buffer.Length tokens into buffer. Return the actual number of tokens read, which may be fewer than the size of the buffer if the stream has reached the end.

Declaration
public int Read(Span<TToken> buffer)
Parameters
Type Name Description

Span<TToken>

buffer

The buffer to read tokens into.

Returns
Type Description

int

The actual number of tokens read.

Implements

ITokenStream<TToken>