common.destination.capabilities

LoaderFileFormatSelector Objects

class LoaderFileFormatSelector(Protocol)

[view_source]

Selects preferred and supported file formats for a given table schema

MergeStrategySelector Objects

class MergeStrategySelector(Protocol)

[view_source]

Selects right set of merge strategies for a given table schema

DataTypeMapper Objects

class DataTypeMapper(ABC)

[view_source]

__init__

def __init__(capabilities: "DestinationCapabilitiesContext") -> None

[view_source]

Maps dlt data types into destination data types

to_destination_type

@abstractmethod
def to_destination_type(column: TColumnSchema,
                        table: PreparedTableSchema) -> str

[view_source]

Gets destination data type for a particular column in prepared table

from_destination_type

@abstractmethod
def from_destination_type(db_type: str, precision: Optional[int],
                          scale: Optional[int]) -> TColumnType

[view_source]

Gets column type from db type

ensure_supported_type

@abstractmethod
def ensure_supported_type(column: TColumnSchema, table: PreparedTableSchema,
                          loader_file_format: TLoaderFileFormat) -> None

[view_source]

Makes sure that dlt type in column in prepared table is supported by the destination for a given file format

UnsupportedTypeMapper Objects

class UnsupportedTypeMapper(DataTypeMapper)

[view_source]

Type Mapper that can't map any type

DestinationCapabilitiesContext Objects

@configspec
class DestinationCapabilitiesContext(ContainerInjectableContext)

[view_source]

Injectable destination capabilities required for many Pipeline stages ie. normalize

loader_file_format_selector

Callable that adapts preferred_loader_file_format and supported_loader_file_formats at runtime.

recommended_file_size

Recommended file size in bytes when writing extract/load files

escape_identifier

Escapes table name, column name and other identifiers

escape_literal

Escapes string literal

casefold_identifier

Casing function applied by destination to represent case insensitive identifiers.

has_case_sensitive_identifiers

Tells if destination supports case sensitive identifiers

supports_clone_table

Destination supports CREATE TABLE ... CLONE ... statements

max_table_nesting

Allows a destination to overwrite max_table_nesting from source

max_parallel_load_jobs

The destination can set the maximum amount of parallel load jobs being executed

loader_parallelism_strategy

The destination can override the parallelism strategy

generates_case_sensitive_identifiers

def generates_case_sensitive_identifiers() -> bool

[view_source]

Tells if capabilities as currently adjusted, will generate case sensitive identifiers

merge_caps_file_formats

def merge_caps_file_formats(
    destination: str, staging: str, dest_caps: DestinationCapabilitiesContext,
    stage_caps: DestinationCapabilitiesContext
) -> Tuple[TLoaderFileFormat, Sequence[TLoaderFileFormat]]

[view_source]

Merges preferred and supported file formats from destination and staging. Returns new preferred file format and all possible formats.