universal_transfer_operator.integrations.base

Module Contents

Classes

TransferIntegrationOptions	TransferIntegrationOptions for transfer integration configuration
TransferIntegration	Basic implementation of a third party transfer.

class universal_transfer_operator.integrations.base.TransferIntegrationOptions

Bases: universal_transfer_operator.utils.TransferParameters

TransferIntegrationOptions for transfer integration configuration

conn_id: str

class universal_transfer_operator.integrations.base.TransferIntegration(transfer_params=attr.field(factory=TransferIntegrationOptions, converter=lambda val: ...))

Bases: abc.ABC

Basic implementation of a third party transfer.

Parameters:

transfer_params (TransferIntegrationOptions) –

abstract property hook: airflow.hooks.base.BaseHook

Return an instance of the database-specific Airflow hook.

Return type:

airflow.hooks.base.BaseHook

abstract transfer_job(source_dataset, destination_dataset)

Loads data from source dataset to the destination using ingestion config

Parameters:

• source_dataset (universal_transfer_operator.datasets.base.Dataset) –

• destination_dataset (universal_transfer_operator.datasets.base.Dataset) –

Return type:

Any