Skip to content

In-memory order updater #5872

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 40 commits into
base: main
Choose a base branch
from

Conversation

benjaminwil
Copy link
Contributor

@benjaminwil benjaminwil commented Oct 11, 2024

Summary

This pull request proposes a replacement to the default Spree::OrderUpdater that has new and improved functionality:

  • Increased performance due to the updater making fewer write calls to the database.
  • A built-in update simulator, so that changes to an order can be previewed before persisting them.

There may be some beneficial side-effects that come out of this new order updater implementation:

  • Significantly faster order factories, and thus a significantly faster test suite for Solidus gems and Solidus applications.

We don't expect this to be the default order updater implementation in the next minor version of Solidus, but we would like to propose it as the default for the next major version of Solidus.

Note: The commits on this pull request have a long list of co-authors, as the Super Good team is approaching this as a collaborative mob programming exercise.

Milestones

For this order updater, we intend to achieve the following during updates:

  1. Don't perform writes to the database.
  2. Preload associated records to eliminate reads required.

We appreciate that there is a lot of complexity to achieving these goals (dealing with active promotions, for example).

Notes

  • We should provide more context about performance gains in this PR description. It would be insightful to include the actual number of writes the current Spree::OrderUpdater makes on a typical recalculate.
  • We should further explain why a production store may want to use an order update simulator so it's clear why this feature is worth having. For now, I'll just say that we have worked with stores who could benefit from this feature. Sometimes admins must make significant changes to completed orders, and it would be valuable for them to preview a set of changes before submitting them and causing updates on many order-associated tables.

Checklist

Check out our PR guidelines for more details.

The following are mandatory for all PRs:

The following are not always needed:

  • 📖 I have updated the README to account for my changes.
  • 📑 I have documented new code with YARD.
  • 🛣️ I have opened a PR to update the guides.
  • ✅ I have added automated tests to cover my changes.
  • 📸 I have attached screenshots to demo visual changes.

@github-actions github-actions bot added the changelog:solidus_core Changes to the solidus_core gem label Oct 11, 2024
@benjaminwil benjaminwil changed the title In memory order updater In-memory order updater Oct 11, 2024
@benjaminwil benjaminwil force-pushed the in-memory-order-updater branch 2 times, most recently from 27f19a8 to 340032c Compare October 11, 2024 22:19
@forkata forkata force-pushed the in-memory-order-updater branch from d22210b to d244646 Compare November 8, 2024 22:34
@jarednorman
Copy link
Member

Just making a note that we are waiting on Alistair to rebase #6026 against this.

@AlistairNorman could you just make sure that gets done at some point before the 20th? That way we can have it for our next session.

@AlistairNorman AlistairNorman force-pushed the in-memory-order-updater branch from a3bb1fc to 06e3a2a Compare December 19, 2024 18:27
@stewart stewart force-pushed the in-memory-order-updater branch from 9fd5c8d to 27e4988 Compare December 20, 2024 21:30
@jarednorman jarednorman force-pushed the in-memory-order-updater branch from 4a0f623 to 90f5420 Compare January 17, 2025 22:03
jarednorman added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 17, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Co-authored-by: Chris Todorov <chris@super.gd>
Co-authored-by: Harmony Bouvier <harmony@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
Co-authored-by: benjamin wil <benjamin@super.gd>
jarednorman added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 17, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Co-authored-by: Chris Todorov <chris@super.gd>
Co-authored-by: Harmony Bouvier <harmony@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
Co-authored-by: benjamin wil <benjamin@super.gd>
adammathys added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 30, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Co-authored-by: Chris Todorov <chris@super.gd>
Co-authored-by: Harmony Bouvier <harmony@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
Co-authored-by: benjamin wil <benjamin@super.gd>
adammathys added a commit to SuperGoodSoft/solidus that referenced this pull request Jan 31, 2025
While working on the in-memory updater in solidusio#5872, we found the need to
change how item totals were being calculated, so that we could mark
adjustments for destruction without actually destroying them, while
still keeping tax adjustments intact. This change is completely
backwards-compatible with the current OrderUpdater, so to reduce the
scope of our PR, we wanted to make this change separately.

Since the OrderUpdater is already very large, this helps reduce its
responsibilities and makes it easier to test this behaviour. We don't
see it as necessary to make this a configurable class, but this change
leaves that option open in the future.

Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Co-authored-by: Chris Todorov <chris@super.gd>
Co-authored-by: Harmony Bouvier <harmony@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
Co-authored-by: benjamin wil <benjamin@super.gd>
@adammathys adammathys force-pushed the in-memory-order-updater branch from 9ecbd13 to 92fc679 Compare January 31, 2025 22:42
@github-actions github-actions bot added changelog:solidus_legacy_promotions Changes to the solidus_legacy_promotions gem changelog:solidus_promotions Changes to the solidus_promotions gem labels Jan 31, 2025
Copy link

codecov bot commented Jan 31, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.76%. Comparing base (ca1ae4d) to head (92fc679).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5872      +/-   ##
==========================================
- Coverage   92.42%   88.76%   -3.67%     
==========================================
  Files         389      831     +442     
  Lines        8005    18102   +10097     
==========================================
+ Hits         7399    16069    +8670     
- Misses        606     2033    +1427     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@AlistairNorman AlistairNorman force-pushed the in-memory-order-updater branch from 92fc679 to e7faef3 Compare February 7, 2025 22:23
@senemsoy senemsoy force-pushed the in-memory-order-updater branch from f3d92a4 to a5ed6c6 Compare February 28, 2025 22:28
@stewart stewart force-pushed the in-memory-order-updater branch 2 times, most recently from d0322b2 to 07add2a Compare March 7, 2025 22:28
@forkata forkata force-pushed the in-memory-order-updater branch 2 times, most recently from d22dbf5 to 770e527 Compare March 14, 2025 21:31
@nvandoorn nvandoorn force-pushed the in-memory-order-updater branch from 770e527 to c8e434e Compare March 28, 2025 20:22
@forkata forkata force-pushed the in-memory-order-updater branch from 7bf4adb to 2d497bd Compare April 3, 2025 18:16
@benjaminwil benjaminwil force-pushed the in-memory-order-updater branch from 67186f3 to defde7e Compare April 10, 2025 17:36
@AlistairNorman AlistairNorman force-pushed the in-memory-order-updater branch from 5423a0e to 62209cf Compare April 17, 2025 19:05
@benjaminwil benjaminwil force-pushed the in-memory-order-updater branch from 2b60e3b to 083fafe Compare May 8, 2025 19:03
AlistairNorman and others added 29 commits May 8, 2025 12:08
These methods don't persist so it's more accurate to say that they
recalculate the total instead of saying that they update it.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Benjamin Willems <benjamin@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Bouvier <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendar@super.gd>
Co-Authored-By: Sofia Besenski <sofia@super.gd>
Co-Authored-By: Chris Todorov <chris@super.gd>
Co-Authored-By: Tom Van Manen <tom@super.gd>
Co-Authored-By: Noah Silvera <noah@super.gd>
We want all the methods that might persist data to be called update_
instead of recalculate to be clear that they hit the database.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Benjamin Willems <benjamin@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Bouvier <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendar@super.gd>
Co-Authored-By: Sofia Besenski <sofia@super.gd>
Co-Authored-By: Chris Todorov <chris@super.gd>
Co-Authored-By: Tom Van Manen <tom@super.gd>
Co-Authored-By: Noah Silvera <noah@super.gd>
This is calling the recalculate method not update_adjustments.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Benjamin Willems <benjamin@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Bouvier <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendar@super.gd>
Co-Authored-By: Sofia Besenski <sofia@super.gd>
Co-Authored-By: Chris Todorov <chris@super.gd>
Co-Authored-By: Tom Van Manen <tom@super.gd>
Co-Authored-By: Noah Silvera <noah@super.gd>
This puts all the update and recalculate methods together.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Benjamin Willems <benjamin@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Bouvier <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendar@super.gd>
Co-Authored-By: Sofia Besenski <sofia@super.gd>
Co-Authored-By: Chris Todorov <chris@super.gd>
Co-Authored-By: Tom Van Manen <tom@super.gd>
Co-Authored-By: Noah Silvera <noah@super.gd>
Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Andrew Stewart <andrew@super.gd>
Co-authored-by: Jared Norman <jared@super.gd>
Co-authored-by: Noah Silvera <noah@super.gd>
Co-authored-by: Benjamin Willems <benjamin@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Includes a small refactor to the internal recalculate method to simplify
the code while maintaining the existing logic around only persisting
when the values have changed.

We'll use this persist flag to eventually only save changes to the DB
when requested. Allowing us to use this adjuster to update the order
in-memory.

Co-authored-by: An Stewart <andrew@super.gd>
Co-authored-by: Harmony Evangelina <harmony@super.gd>
Co-authored-by: Kendra Riga <kendra@super.gd>
Co-authored-by: Nick Van Doorn <nick@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
Co-authored-by: benjamin wil <benjamin@super.gd>
Similar to the previous change. We're now passing the persist flag down
to all promotion order adjusters. This does not implement the logic
within the individual adjuster classes to skip persistance when
required, only ensures the flag is pass down from our in-memory order
updater.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Evangelina <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendra@super.gd>
Co-Authored-By: Senem Soy <senem@super.gd>
Co-Authored-By: benjamin wil <benjamin@super.gd>
Previously this would update the eligible column. We now only assign the
value and then save if persist is true.

Co-Authored-By: Adam Mueller <adam@super.gd>
Co-Authored-By: Andrew Stewart <andrew@super.gd>
Co-Authored-By: Harmony Evangelina <harmony@super.gd>
Co-Authored-By: Jared Norman <jared@super.gd>
Co-Authored-By: Kendra Riga <kendra@super.gd>
Co-Authored-By: benjamin wil <benjamin@super.gd>
Co-Authored-By: Chris Todorov <chris@super.gd>
Co-Authored-By: Nick Van Doorn <nick@super.gd>
Co-Authored-By: Sofia Besenski <sofia@super.gd>
We were missing the whole path of doing order level adjustments, e.g. The CreateDiscountedItem benefit.
How dry run behaves is not consistent, and we think we can make not persisting easier if we first update the dry run functionality.
This makes working with unpersisted line items easier because required
attributes are set immediately. This change is in service of creating an
in memory order updater.
This is in service of the in-memory order updater.

Co-authored-by: Adam <adam@super.gd>
Co-authored-by: Alistair <alistair@super.gd>
Co-authored-by: An <andrew@super.gd>
Co-authored-by: Benjamin <benjamin@super.gd>
Co-authored-by: Chris <chris@super.gd>
Co-authored-by: Harmony <harmony@super.gd>
Co-authored-by: Jared <jared@super.gd>
Co-authored-by: Nick <nick@super.gd>
Co-authored-by: Noah <noah@super.gd>
Co-authored-by: Senem <senem@super.gd>
Co-authored-by: Sofia <sofia@super.gd>
See code comments and spec changes. This is similar to how we handled
this in the legacy promotion system.

Co-authored-by: Adam <adam@super.gd>
Co-authored-by: Alistair <alistair@super.gd>
Co-authored-by: An <andrew@super.gd>
Co-authored-by: Benjamin <benjamin@super.gd>
Co-authored-by: Harmony <harmony@super.gd>
Co-authored-by: Jared <jared@super.gd>
Co-authored-by: Nick <nick@super.gd>
Co-authored-by: Senem <senem@super.gd>
Co-authored-by: Sofia <sofia@super.g
We're seeing a spec failure related to changing setting required line
item attributes from `before_validation` to `after_initialize`. We can
"fix" the spec by setting a nil tax category, but this may not be
appropriate.

We've also added a TODO item regarding this change, as we think it may
be a breaking change requiring further investigation.

Co-authored-by: Noah Silvera <noah@super.gd>
Co-authored-by: Kendra Chateau <kendra@super.gd>
This is a similar change to the last commit. We still need to evaluate
if this is appropriate spec setup.

Co-authored-by: Noah Silvera <noah@super.gd>
Co-authored-by: Kendra Riga <kendra@super.gd>
Integration level test using the InMemoryOrderUpdater to ensure we are
not persisting changes during the promotion recalculations.

Co-authored-by: Chris <chris@super.gd>
Co-authored-by: Benjamin <benjamin@super.gd>
Co-authored-by: An <andrew@super.gd>
Co-authored-by: Jared <jared@super.gd>
We can just use the FactoryBot factories instead.

Co-Authored-By: Benjamin <benjamin@super.gd>
Co-Authored-By: Alistair <alistair@super.gd>
Integration level test using the InMemoryOrderUpdater to ensure we are
not persisting changes during the promotion recalculations.

Co-authored-by: Chris <chris@super.gd>
Co-authored-by: Benjamin <benjamin@super.gd>
Co-authored-by: An <andrew@super.gd>
Co-authored-by: Jared <jared@super.gd>
Ran out of time so added this to fix next time.
Co-authored-by: Jared Norman <jared@super.gd>
Co-authored-by: Kendra Riga <kendra@super.gd>
Co-authored-by: Adam Mueller <adam@super.gd>
Co-authored-by: Alistair Norman <alistair@super.gd>
Co-authored-by: Senem Soy <senem@super.gd>
Co-authored-by: Sofia Besenski <sofia@super.gd>
@benjaminwil benjaminwil force-pushed the in-memory-order-updater branch from 3d19c91 to 210eb32 Compare May 8, 2025 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog:solidus_core Changes to the solidus_core gem changelog:solidus_legacy_promotions Changes to the solidus_legacy_promotions gem changelog:solidus_promotions Changes to the solidus_promotions gem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants