singer_sdk.contrib.filesystem.stream.FileStream

class singer_sdk.contrib.filesystem.stream.FileStream

Abstract base class for file streams.

__init__(tap, name, *, filepaths, filesystem)

Create a new FileStream instance.

Parameters:

• tap (Tap) – The tap for this stream.

• name (str) – The name of the stream.

• filepaths (t.Sequence[str]) – List of file paths to read.

• filesystem (fsspec.AbstractFileSystem) – The filesystem implementation object to use.

Raises:

ConfigValidationError – If no file paths are provided.

Return type:

None

get_records(context)

Read records from the file.

Parameters:

context (Context | None) – Stream partition or context dictionary.

Yields:

Record or tuple of Record and child context.

Raises:

RuntimeError – If context is not provided.

Return type:

t.Iterable[dict | tuple[dict, dict | None]]

abstractmethod get_schema(path)

Return the schema for the file.

Parameters:

path (str)

Return type:

dict[str, Any]

abstractmethod read_file(path)

Return a generator of records from the file.

Parameters:

path (str)

Return type:

t.Iterable[Record]

property partitions: list[dict[str, Any]]

Return the list of partitions for this stream.

property schema: dict[str, Any]

Return the schema for the stream.