Pidgin

Class ResumableTokenStream<TToken>

An ITokenStream<TToken> implementation which wraps another ITokenStream<TToken> and adds support for resumable parsing.

Inheritance
  • Object
  • ResumableTokenStream<TToken>
Implements
Declaration
public class ResumableTokenStream<TToken> : Object, ITokenStream<TToken>, IDisposable
Type Parameters
Name Description

TToken

The type of tokens returned by the wrapped ITokenStream<TToken>.

Constructors

ResumableTokenStream(ITokenStream<TToken>, ArrayPool<TToken>)

Creates an ITokenStream<TToken> implementation which adds support for resumable parsing to next.

Declaration
public ResumableTokenStream(ITokenStream<TToken> next, ArrayPool<TToken> pool = null)
Parameters
Type Name Description

ITokenStream<TToken>

next

The ITokenStream<TToken> to wrap.

ArrayPool<TToken>

pool

An ArrayPool<T> to use for the internal buffer. Defaults to Shared.

Methods

Dispose()

Return any buffers to the ArrayPool<T>

Declaration
public void Dispose()

Read(Span<TToken>)

Read up to buffer.Length tokens into buffer. Return the actual number of tokens read, which may be fewer than the size of the buffer if the stream has reached the end.

Declaration
public int Read(Span<TToken> buffer)
Parameters
Type Name Description

Span<TToken>

buffer

The buffer to read tokens into.

Returns
Type Description

Int32

The actual number of tokens read.

Return(ReadOnlySpan<TToken>)

Push some un-consumed tokens back into the stream. Parser<TToken, T>s call this method when they are finished parsing.

Declaration
public void Return(ReadOnlySpan<TToken> leftovers)
Parameters
Type Name Description

ReadOnlySpan<TToken>

leftovers

The leftovers to push back into the stream.

Implements

ITokenStream<TToken>
System.IDisposable