Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AIRFLOW-3086] Add extras group for google auth to setup.py. #3917

Merged
merged 2 commits into from
Oct 9, 2018
Merged

[AIRFLOW-3086] Add extras group for google auth to setup.py. #3917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c2d6aba
[AIRFLOW-3086] Add extras group for google auth to setup.py.
jmcarp Sep 19, 2018
037088c
Update setup.py extras for authentication.
jmcarp Oct 6, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 69 additions & 65 deletions docs/installation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,71 +41,75 @@ these extra dependencies.

Here's the list of the subpackages and what they enable:

+---------------+----------------------------------------------+-------------------------------------------------+
| subpackage | install command | enables |
+===============+==============================================+=================================================+
| all | ``pip install apache-airflow[all]`` | All Airflow features known to man |
+---------------+----------------------------------------------+-------------------------------------------------+
| all_dbs | ``pip install apache-airflow[all_dbs]`` | All databases integrations |
+---------------+----------------------------------------------+-------------------------------------------------+
| async | ``pip install apache-airflow[async]`` | Async worker classes for Gunicorn |
+---------------+----------------------------------------------+-------------------------------------------------+
| celery | ``pip install apache-airflow[celery]`` | CeleryExecutor |
+---------------+----------------------------------------------+-------------------------------------------------+
| cloudant | ``pip install apache-airflow[cloudant]`` | Cloudant hook |
+---------------+----------------------------------------------+-------------------------------------------------+
| crypto | ``pip install apache-airflow[crypto]`` | Encrypt connection passwords in metadata db |
+---------------+----------------------------------------------+-------------------------------------------------+
| devel | ``pip install apache-airflow[devel]`` | Minimum dev tools requirements |
+---------------+----------------------------------------------+-------------------------------------------------+
| devel_hadoop | ``pip install apache-airflow[devel_hadoop]`` | Airflow + dependencies on the Hadoop stack |
+---------------+----------------------------------------------+-------------------------------------------------+
| druid | ``pip install apache-airflow[druid]`` | Druid related operators & hooks |
+---------------+----------------------------------------------+-------------------------------------------------+
| gcp_api | ``pip install apache-airflow[gcp_api]`` | Google Cloud Platform hooks and operators |
| | | (using ``google-api-python-client``) |
+---------------+----------------------------------------------+-------------------------------------------------+
| hdfs | ``pip install apache-airflow[hdfs]`` | HDFS hooks and operators |
+---------------+----------------------------------------------+-------------------------------------------------+
| hive | ``pip install apache-airflow[hive]`` | All Hive related operators |
+---------------+----------------------------------------------+-------------------------------------------------+
| jdbc | ``pip install apache-airflow[jdbc]`` | JDBC hooks and operators |
+---------------+----------------------------------------------+-------------------------------------------------+
| kerberos | ``pip install apache-airflow[kerberos]`` | Kerberos integration for Kerberized Hadoop |
+---------------+----------------------------------------------+-------------------------------------------------+
| ldap | ``pip install apache-airflow[ldap]`` | LDAP authentication for users |
+---------------+----------------------------------------------+-------------------------------------------------+
| mssql | ``pip install apache-airflow[mssql]`` | Microsoft SQL Server operators and hook, |
| | | support as an Airflow backend |
+---------------+----------------------------------------------+-------------------------------------------------+
| mysql | ``pip install apache-airflow[mysql]`` | MySQL operators and hook, support as an Airflow |
| | | backend. The version of MySQL server has to be |
| | | 5.6.4+. The exact version upper bound depends |
| | | on version of ``mysqlclient`` package. For |
| | | example, ``mysqlclient`` 1.3.12 can only be |
| | | used with MySQL server 5.6.4 through 5.7. |
+---------------+----------------------------------------------+-------------------------------------------------+
| password | ``pip install apache-airflow[password]`` | Password authentication for users |
+---------------+----------------------------------------------+-------------------------------------------------+
| postgres | ``pip install apache-airflow[postgres]`` | PostgreSQL operators and hook, support as an |
| | | Airflow backend |
+---------------+----------------------------------------------+-------------------------------------------------+
| qds | ``pip install apache-airflow[qds]`` | Enable QDS (Qubole Data Service) support |
+---------------+----------------------------------------------+-------------------------------------------------+
| rabbitmq | ``pip install apache-airflow[rabbitmq]`` | RabbitMQ support as a Celery backend |
+---------------+----------------------------------------------+-------------------------------------------------+
| redis | ``pip install apache-airflow[redis]`` | Redis hooks and sensors |
+---------------+----------------------------------------------+-------------------------------------------------+
| s3 | ``pip install apache-airflow[s3]`` | ``S3KeySensor``, ``S3PrefixSensor`` |
+---------------+----------------------------------------------+-------------------------------------------------+
| samba | ``pip install apache-airflow[samba]`` | ``Hive2SambaOperator`` |
+---------------+----------------------------------------------+-------------------------------------------------+
| slack | ``pip install apache-airflow[slack]`` | ``SlackAPIPostOperator`` |
+---------------+----------------------------------------------+-------------------------------------------------+
| ssh | ``pip install apache-airflow[ssh]`` | SSH hooks and Operator |
+---------------+----------------------------------------------+-------------------------------------------------+
| vertica | ``pip install apache-airflow[vertica]`` | Vertica hook support as an Airflow backend |
+---------------+----------------------------------------------+-------------------------------------------------+
+---------------------+---------------------------------------------------+-------------------------------------------------+
| subpackage | install command | enables |
+=====================+===================================================+=================================================+
| all | ``pip install apache-airflow[all]`` | All Airflow features known to man |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| all_dbs | ``pip install apache-airflow[all_dbs]`` | All databases integrations |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| async | ``pip install apache-airflow[async]`` | Async worker classes for Gunicorn |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| celery | ``pip install apache-airflow[celery]`` | CeleryExecutor |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| cloudant | ``pip install apache-airflow[cloudant]`` | Cloudant hook |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| crypto | ``pip install apache-airflow[crypto]`` | Encrypt connection passwords in metadata db |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| devel | ``pip install apache-airflow[devel]`` | Minimum dev tools requirements |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| devel_hadoop | ``pip install apache-airflow[devel_hadoop]`` | Airflow + dependencies on the Hadoop stack |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| druid | ``pip install apache-airflow[druid]`` | Druid related operators & hooks |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| gcp_api | ``pip install apache-airflow[gcp_api]`` | Google Cloud Platform hooks and operators |
| | | (using ``google-api-python-client``) |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| github_enterprise | ``pip install apache-airflow[github_enterprise]`` | Github Enterprise auth backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| google_auth | ``pip install apache-airflow[google_auth]`` | Google auth backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| hdfs | ``pip install apache-airflow[hdfs]`` | HDFS hooks and operators |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| hive | ``pip install apache-airflow[hive]`` | All Hive related operators |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| jdbc | ``pip install apache-airflow[jdbc]`` | JDBC hooks and operators |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| kerberos | ``pip install apache-airflow[kerberos]`` | Kerberos integration for Kerberized Hadoop |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| ldap | ``pip install apache-airflow[ldap]`` | LDAP authentication for users |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| mssql | ``pip install apache-airflow[mssql]`` | Microsoft SQL Server operators and hook, |
| | | support as an Airflow backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| mysql | ``pip install apache-airflow[mysql]`` | MySQL operators and hook, support as an Airflow |
| | | backend. The version of MySQL server has to be |
| | | 5.6.4+. The exact version upper bound depends |
| | | on version of ``mysqlclient`` package. For |
| | | example, ``mysqlclient`` 1.3.12 can only be |
| | | used with MySQL server 5.6.4 through 5.7. |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| password | ``pip install apache-airflow[password]`` | Password authentication for users |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| postgres | ``pip install apache-airflow[postgres]`` | PostgreSQL operators and hook, support as an |
| | | Airflow backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| qds | ``pip install apache-airflow[qds]`` | Enable QDS (Qubole Data Service) support |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| rabbitmq | ``pip install apache-airflow[rabbitmq]`` | RabbitMQ support as a Celery backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| redis | ``pip install apache-airflow[redis]`` | Redis hooks and sensors |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| s3 | ``pip install apache-airflow[s3]`` | ``S3KeySensor``, ``S3PrefixSensor`` |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| samba | ``pip install apache-airflow[samba]`` | ``Hive2SambaOperator`` |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| slack | ``pip install apache-airflow[slack]`` | ``SlackAPIPostOperator`` |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| ssh | ``pip install apache-airflow[ssh]`` | SSH hooks and Operator |
+---------------------+---------------------------------------------------+-------------------------------------------------+
| vertica | ``pip install apache-airflow[vertica]`` | Vertica hook support as an Airflow backend |
+---------------------+---------------------------------------------------+-------------------------------------------------+

Initiating Airflow Database
'''''''''''''''''''''''''''
Expand Down
18 changes: 18 additions & 0 deletions docs/security.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,12 @@ and in your DAG, when initializing the HiveOperator, specify:

run_as_owner=True

To use kerberos authentication, you must install Airflow with the `kerberos` extras group:

.. code-block:: base

pip install airflow[kerberos]

OAuth Authentication
--------------------

Expand Down Expand Up @@ -258,6 +264,12 @@ to only members of those teams.
.. note:: If you do not specify a team whitelist, anyone with a valid account on
your GHE installation will be able to login to Airflow.

To use GHE authentication, you must install Airflow with the `github_enterprise` extras group:

.. code-block:: base

pip install airflow[github_enterprise]

Setting up GHE Authentication
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -300,6 +312,12 @@ login, separated with a comma, to only members of those domains.
oauth_callback_route = /oauth2callback
domain = "example1.com,example2.com"

To use Google authentication, you must install Airflow with the `google_auth` extras group:

.. code-block:: base

pip install airflow[google_auth]

Setting up Google Authentication
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
2 changes: 2 additions & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,7 @@ def write_version(filename=os.path.join(*['airflow',
'pandas-gbq'
]
github_enterprise = ['Flask-OAuthlib>=0.9.1']
google_auth = ['Flask-OAuthlib>=0.9.1']
hdfs = ['snakebite>=2.7.8']
hive = [
'hmsclient>=0.1.0',
Expand Down Expand Up @@ -358,6 +359,7 @@ def do_setup():
'emr': emr,
'gcp_api': gcp_api,
'github_enterprise': github_enterprise,
'google_auth': google_auth,
'hdfs': hdfs,
'hive': hive,
'jdbc': jdbc,
Expand Down