universal_transfer_operator.utils

Module Contents

Classes

TransferParameters

Functions

check_if_connection_exists(conn_id)	Given an Airflow connection ID, identify if it exists.
get_dataset_connection_type(dataset)	Given dataset fetch the connection type based on airflow connection
get_class_name(module_ref[, suffix])	Get class name to be dynamically imported. Class name are expected to be in following formats

class universal_transfer_operator.utils.TransferParameters

if_exists: str = 'replace'

universal_transfer_operator.utils.check_if_connection_exists(conn_id)

Given an Airflow connection ID, identify if it exists. Return True if it does or raise an AirflowNotFoundException exception if it does not.

Parameters:

conn_id (str) – Airflow connection ID

Return bool:

If the connection exists, return True

Return type:

bool

universal_transfer_operator.utils.get_dataset_connection_type(dataset)

Given dataset fetch the connection type based on airflow connection

Parameters:

dataset (Table | File) –

Return type:

str

universal_transfer_operator.utils.get_class_name(module_ref, suffix='Location')

Get class name to be dynamically imported. Class name are expected to be in following formats example - module name: test suffix: Abc

expected class names -

1. TESTAbc

2. TestAbc

Parameters:

• module_ref (Any) – Module from which to get class location type implementation

• suffix (str) – suffix for class name

Return type:

str