Asset-swapper: Fallback orders + refactors

[dorothy-zbornak]

Description

Generate fallback orders

If the allowFallback option is passed into SwapQuoter, fallback orders can be generated. How this works is we first generate an optimized quote. Then if this quote includes native orders, we will generate a second quote for the same size as the native orders and append it to the original quote. This fallback quote will not reuse any sources in the original quote.

We do, however, allow Kyber conflicts in the fallback quote because, even though in theory this should reduce reverts, in reality it increases reverts because with conflicting sources omitted we very often cannot find an fallback path. Even with conflicts, early simulations (w/ bridgeSliipage = 1%) never seem to fail on these quotes, so I think we good?

Gas estimates

Because eth_estimateGas will always fail to correctly anticipate the gas cost of fallback orders, AS now attempts to provide a gas estimate in the bestCaseQuoteInfo and worstCaseQuoteInfo fields. I'm lazy so they're both the same right now but in the future we can get more clever with the best case.

New(ish) optimizer algorithm

I tweaked the algorithm and cleaned it up a lot. We still try to explore as many fill combinations as we can (we have to because fees make price curves non-convex). The algo should also run a bit faster now thanks to Remco's suggestion of iteratively solving two paths at a time, rather than all at once. This is important because we now essentially run it twice in order to generate the fallback orders.

Massive refactors to market_operation_utils

Yeah, that was turning into a hot mess. All the code in that directory are much nicer and simpler now.

Changes to swap quote options

Removed:

• slippagePercentage: The fallback orders essentially double the possible fill amount already.
• dustFractionThreshold: No longer necessary because the optimizer can handle a larger # of paths.
• noConflicts: We never set this to false so it's now always on.

Added:

• allowFallback: If true (default), allows the generation of fallback orders
• gasSchedule: Estimated gas usage of filling a liquidity source-- similar to feeSchedule but expressed in gas instead of ETH.
• maxFallbackSlippage: The maximum rate % change allowed in the fallback (compared to the optimal quote). If the fallback's slippage exceeds this %, no fallback will be provided.

Renamed:

• fees -> feeSchedule

Ordering of orders in quote

Because of how the new optimizer works, there is no longer a strong guarantee that orders will be sorted by decreasing rate. If you relied on this, sorry boo.

Bugs

• I found a small bug related to how DEX paths were generated, which may have led to slightly worse pricing.
• Because the code is so different, I went ahead and addressed [Asset-swapper] Fix bug with batch market buy #2517 in here as well.

Testing instructions

Types of changes

Checklist:

• Prefix PR title with [WIP] if necessary.
• Add tests to cover changes as needed.
• Update documentation as needed.
• Add new entries to the relevant CHANGELOG.jsons.

[dekz]

Taken a quick peek but wanted to discuss some things to get the clear in my mind.

I feel like there are 2 approaches to the goal of lowering reverts with a backup path.

1. Find the best path using on-chain sources, use 0x to improve price if available. So we have a backup to 0x failing.
2. Find the best path, create another backup path incase the best path fails. This feels a bit difficult to reason about. It could be 100% Uniswap, then 100% 0x as a backup (previously this would be just Uniswap, so little to be gained in this scenario). It could also be 100% Uniswap then 100% Kyber as a backup, which could be a conflict. It can also be the inverse of 100% Kyber with 100% Uniswap backup, also a potential conflict.

I am biased to the 1) approach as that seems simple, but keen to hear your thoughts. In my head 0x should never be the backup as I find it difficult to reason the case where Uniswap would fail and 0x orders would still be fillable.

When a path includes Curve, and a backup path includes another Curve, this is now 2million+ faux gas estimate as a "just in case".

[dorothy-zbornak]

Taken a quick peek but wanted to discuss some things to get the clear in my mind.

I feel like there are 2 approaches to the goal of lowering reverts with a backup path.

1. Find the best path using on-chain sources, use 0x to improve price if available. So we have a backup to 0x failing.
2. Find the best path, create another backup path incase the best path fails. This feels a bit difficult to reason about. It could be 100% Uniswap, then 100% 0x as a backup (previously this would be just Uniswap, so little to be gained in this scenario). It could also be 100% Uniswap then 100% Kyber as a backup, which could be a conflict. It can also be the inverse of 100% Kyber with 100% Uniswap backup, also a potential conflict.

I am biased to the 1) approach as that seems simple, but keen to hear your thoughts. In my head 0x should never be the backup as I find it difficult to reason the case where Uniswap would fail and 0x orders would still be fillable.

When a path includes Curve, and a backup path includes another Curve, this is now 2million+ faux gas estimate as a "just in case".

What do you think of:

1. Find the optimal quote.
2. If the optimal quote contains a 0x order:
   1. Generate a fallback quote with the unused sources (+ factoring in Kyber conflicts).

The separate quotes thing is to ensure the optimizer is solving for as close to the actual fill amount as possible, which gives us better pricing.

[dekz]

Want to spend some more time tomorrow just going through the market operations utils deeper. But this is looking good.

[dekz]

Thoughts on removing the feeSchedule and just adding 150e3 to the gasSchedule where feeSchedule is used?

[dorothy-zbornak]

I could go either way. It's probably not good design for AS to assume the protocol fee, even though it already does in a few places (we should remove these).

[dorothy-zbornak]

It's also not particularly hard for the caller to do the operation you described theirself.

[dekz]

Couple of little things but this looks good. Going to simbot now and think more on what this looks like with gasLimit and if we'd need any adjustment. But worse case we wouldn't be introducing more reverts, just transferring the reported revert as OOG.