Skip to content

Commit 4c286b6

Browse files
holdenkAlice Berard
authored and
Alice Berard
committed
[AIRFLOW-2192] Allow non-latin1 usernames with MySQL backend by adding a SQL_ENGINE_ENCODING param and default to UTF-8 (apache#4087)
Compromised of: Since we have unicode_literals importred and the engine arguments must be strings in Python2 explicitly make 'utf-8' a string. replace bare exception with conf.AirflowConfigException for missing value. It's just got for strings apparently. Add utf-8 to default_airflow.cfg - question do I still need the try try/except block or can we depend on defaults (I note some have both). Get rid of try/except block and depend on default_airflow.cfg Use __str__ since calling str just gives us back a newstr as well. Test that a panda user can be saved.
1 parent 002a48c commit 4c286b6

File tree

3 files changed

+19
-1
lines changed

3 files changed

+19
-1
lines changed

airflow/config_templates/default_airflow.cfg

+3
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,9 @@ executor = SequentialExecutor
8686
# their website
8787
sql_alchemy_conn = sqlite:///{AIRFLOW_HOME}/airflow.db
8888

89+
# The encoding for the databases
90+
sql_engine_encoding = utf-8
91+
8992
# If SqlAlchemy should pool database connections.
9093
sql_alchemy_pool_enabled = True
9194

airflow/settings.py

+11-1
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,7 @@ def configure_orm(disable_connection_pool=False):
155155
engine_args['poolclass'] = NullPool
156156
log.debug("settings.configure_orm(): Using NullPool")
157157
elif 'sqlite' not in SQL_ALCHEMY_CONN:
158-
# Engine args not supported by sqlite.
158+
# Pool size engine args not supported by sqlite.
159159
# If no config value is defined for the pool size, select a reasonable value.
160160
# 0 means no limit, which could lead to exceeding the Database connection limit.
161161
try:
@@ -177,6 +177,16 @@ def configure_orm(disable_connection_pool=False):
177177
engine_args['pool_size'] = pool_size
178178
engine_args['pool_recycle'] = pool_recycle
179179

180+
try:
181+
# Allow the user to specify an encoding for their DB otherwise default
182+
# to utf-8 so jobs & users with non-latin1 characters can still use
183+
# us.
184+
engine_args['encoding'] = conf.get('core', 'SQL_ENGINE_ENCODING')
185+
except conf.AirflowConfigException:
186+
engine_args['encoding'] = 'utf-8'
187+
# For Python2 we get back a newstr and need a str
188+
engine_args['encoding'] = engine_args['encoding'].__str__()
189+
180190
engine = create_engine(SQL_ALCHEMY_CONN, **engine_args)
181191
reconnect_timeout = conf.getint('core', 'SQL_ALCHEMY_RECONNECT_TIMEOUT')
182192
setup_event_handlers(engine, reconnect_timeout)

tests/core.py

+5
Original file line numberDiff line numberDiff line change
@@ -2110,6 +2110,11 @@ def test_password_user_authenticate(self):
21102110
self.password_user.password = "secure_password"
21112111
self.assertTrue(self.password_user.authenticate("secure_password"))
21122112

2113+
def test_password_unicode_user_authenticate(self):
2114+
self.password_user.username = u"🐼" # This is a panda
2115+
self.password_user.password = "secure_password"
2116+
self.assertTrue(self.password_user.authenticate("secure_password"))
2117+
21132118
def test_password_authenticate_session(self):
21142119
from airflow.contrib.auth.backends.password_auth import PasswordUser
21152120
self.password_user.password = 'test_password'

0 commit comments

Comments
 (0)