Configuration
Configuring the database default schema
The Universal Transfer Operator uses the global default schema configuration for the Table
metadata, unless users
specify a different specific Table
(metadata) schema
.
User can define the default schema in two ways: 1. Globally, for all the databases 2. Individually, for each database
The Universal transfer operator applies the global default schema (tmp_transfers
by default) when the user does not
set up a database-specific configuration. Example:
environment variable :
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__SQL_SCHEMA="tmp_transfers"
or by updating Airflow’s configuration
[universal_transfer_operator]
schema = "tmp_transfers"
We can set up the default schema for each database type, such as Snowflake, BigQuery, or Postgres. The database-specific schema overrides the global default schema if both are defined.
[universal_transfer_operator]
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__POSTGRES_DEFAULT_SCHEMA = "postgres_tmp"
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__BIGQUERY_DEFAULT_SCHEMA = "bigquery_tmp"
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__SNOWFLAKE_DEFAULT_SCHEMA = "snowflake_tmp"
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__REDSHIFT_DEFAULT_SCHEMA = "redshift_tmp"
AIRFLOW__UNIVERSAL_TRANSFER_OPERATOR__MSSQL_DEFAULT_SCHEMA = "mssql_tmp"
or by updating Airflow’s configuration
[universal_transfer_operator]
postgres_default_schema = "postgres_tmp"
bigquery_default_schema = "bigquery_tmp"
snowflake_default_schema = "snowflake_tmp"
redshift_default_schema = "redshift_tmp"
mssql_default_schema = "mssql_tmp"