Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOLD] Race condition between staging and production deploys #14834

Closed
roryabraham opened this issue Feb 4, 2023 · 3 comments
Closed

[HOLD] Race condition between staging and production deploys #14834

roryabraham opened this issue Feb 4, 2023 · 3 comments
Labels
Improvement Item broken or needs improvement. Internal Requires API changes or must be handled by Expensify staff Monthly KSv2 Not a priority

Comments

@roryabraham
Copy link
Contributor

roryabraham commented Feb 4, 2023

Problem

There is a race condition between staging and production mobile deploys that can cause production mobile deploys to fail.

For some background, when we deploy iOS and Android to staging what we're really doing is:

  • Creating a production build
  • Uploading it to the app store as a beta (TestFlight or Internal App Sharing)

Then when we deploy iOS and Android to production, all we're really doing is taking the staging build, submitting it for review + release.

What can happen is:

  1. We have a deploy blocker affecting only one platform (say, web), and it's our last deploy blocker.
  2. The deployer CP's a fix, and as soon as that fix is verified closes the checklist (before the staging deploy from that CP has completed on all platforms, which often takes about an hour)
  3. The production deploy starts before the staging deploy finishes, so when we try to submit the latest version for review + release, Fastlane barfs 🤮 because the build with that version hasn't been uploaded to the app store yet.

Here's a real-world example: https://expensify.slack.com/archives/C07J32337/p1675468753414159

Solution

Let's make it so that a production mobile deploy will wait for the staging mobile deploy of the same version to finish before it begins.

Upwork Automation - Do Not Edit
  • Upwork Job URL: https://www.upwork.com/jobs/~0110288f4cb7c04c91
  • Upwork Job ID: 1621664918611951616
  • Last Price Increase: 2023-02-04
@roryabraham roryabraham added Monthly KSv2 Improvement Item broken or needs improvement. Internal Requires API changes or must be handled by Expensify staff labels Feb 4, 2023
@melvin-bot melvin-bot bot locked and limited conversation to collaborators Feb 4, 2023
@roryabraham roryabraham changed the title Race condition between staging and production deploys [HOLD] Race condition between staging and production deploys Feb 4, 2023
@Expensify Expensify unlocked this conversation Mar 21, 2023
@melvin-bot melvin-bot bot closed this as completed Apr 18, 2023
@MelvinBot
Copy link

@roryabraham, this Monthly task hasn't been acted upon in 6 weeks; closing.

If you disagree, feel encouraged to reopen it -- but pick your least important issue to close instead.

@melvin-bot
Copy link

melvin-bot bot commented Jul 10, 2023

@roryabraham, this Monthly task hasn't been acted upon in 6 weeks; closing.

If you disagree, feel encouraged to reopen it -- but pick your least important issue to close instead.

@roryabraham roryabraham reopened this Jul 10, 2023
@melvin-bot melvin-bot bot closed this as completed Sep 22, 2023
@melvin-bot
Copy link

melvin-bot bot commented Sep 22, 2023

@roryabraham, this Monthly task hasn't been acted upon in 6 weeks; closing.

If you disagree, feel encouraged to reopen it -- but pick your least important issue to close instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Item broken or needs improvement. Internal Requires API changes or must be handled by Expensify staff Monthly KSv2 Not a priority
Projects
None yet
Development

No branches or pull requests

2 participants