[AIRFLOW-2544] Guard against next major release of Celery, Flower

[tedmiston]

Make sure you have checked all steps below.

JIRA

• My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"
  • https://issues.apache.org/jira/browse/AIRFLOW-2544

Description

• Here are some details about my PR, including screenshots of any UI changes:

This PR proactively guards against the next major version of Celery (4.2) from being installed automatically until it's been tested. It is currently in RC but not yet officially released. I've also added an upper bound on Flower 1.0 similarly.

Related to https://issues.apache.org/jira/browse/AIRFLOW-1967, and created from discussion in #2806.

Tests

• My PR adds the following unit tests OR does not need testing for this extremely good reason:

n/a - just a dependency version guard

Commits

• My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

Documentation

• In case of new functionality, my PR adds documentation that describes how to use it.
  • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

n/a

Code Quality

• Passes git diff upstream/master -u -- "*.py" | flake8 --diff

[tedmiston]

@Fokko Here is the PR I proposed yesterday in #2806 (comment) that covers Celery. I also added a similar guard for the 1.0 release of Flower as well. It looks like master was recently changed to celery>=4.1.1 so I have preserved that. Let me know if this looks good to you to merge. Thank you!

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@3ed25a9). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #3439   +/-   ##
=========================================
  Coverage          ?   76.68%           
=========================================
  Files             ?      203           
  Lines             ?    15064           
  Branches          ?        0           
=========================================
  Hits              ?    11552           
  Misses            ?     3512           
  Partials          ?        0

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ed25a9...7d396ad. Read the comment docs.

[tedmiston]

^ Not sure what this Codecov warning is about. This PR doesn't change code, so I think it's not relevant, but please let me know if action is required.

[tedmiston]

Rebased off master to see if it makes a difference.

[tedmiston]

@bolkedebruin I noticed you'd done some work around Celery as well. Do you have any input on this one?

[Fokko]

Thanks @tedmiston. Codecov is sometimes a bit off if you're working on an outdated master. Looking good.

[r39132]

LGTM