Redesign `GYTxMonad` and its instances to be able to submit transactions #322

TotallyNotChase · 2024-07-10T22:19:37Z

Summary

Closes #316

This is a long overdue redesign of GYTxMonad. It adds the ability to submit transactions to the monad and splits it into a more lawful hierarchy. In particular:

GYTxQueryMonad - For common utxo and similar queries

Instance: GYTxQueryMonadIO
GYTxQueryMonad => GYTxSpecialQueryMonad - For uncommon queries such as protocol parameters, era history etc. (this could potentially be removed, see note below)

Instance: GYTxQueryMonadIO
GYTxQueryMonad => GYTxUserQueryMonad - For queries while acting as a user (having access to own wallet for querying purposes not signing purposes)

Instance: GYTxBuilderMonadIO - see below.
(GYTxSpecialQueryMonad, GYTxUserQueryMonad, Default (TxBuilderStrategy)) => GYTxBuilderMonad - For building transactions while acting as a user. This allows different instances to choose their own transaction building method. If TxBuilderStrategy is set to GYCoinSelectionStrategy (default), the implementation defaults to the pre-existing transaction building methods. Therefore, simply anyclass deriving this class will be enough to use the transaction building unchanged.

instance: GYTxBuilderMonadIO
GYTxBuilderMonad => GYTxMonad - For interacting with the blockchain in a way that mutates its state. E.g building and submitting transactions etc. Also is able to sign as a user.

Instance: GYTxMonadIO
(GYTxMonad (TxMonadOf m), GYTxSpecialQueryMonad m) => GYTxGameMonad m - This is an entirely new feature. This allows simulating multiple users interacting in an environment, building and submitting transactions etc. This is extremely useful in testing. Where we model an environment with many users interacting, waiting for events etc.

Each "game' monad has its own "tx" monad. The tx monad code can be lifted into the "game" monad and assigned a user to perform it as.

Instance: GYTxGameMonadIO

I suggest putting this brief note about the monad hierarchy in the docs so people can choose which monad they need, instead of always reaching for the most powerful one

Note the distinction between a TxQuery monad and the Tx monad. The query monad classes are meant to be non-mutating. In other words, if you are given a GYTxUserQueryMonad m => m a for example, and you run it (using a relevant interpreter) - you can be sure that it did not mutate anything on the chain. However, that guarantee does not exist for GYTxMonad.

This sort of a lawful type classes helps in "coloring" functions with types to define their expected behavior. Most code in the GY codebase (where the backend only builds transactions and passes it to frontend) can use just a query monad!

This makes running the monad instances (e.g GYTxMonadIO, renamed from GYTxMonadNode) much simpler. There is no traversable hack anymore. It's a simple monad interpreter. As you'd expect from any other proper monad in the ecosystem. As a result, transaction defining, building, signing, and submitting - all under one monad feels much more ergonomic.

I understand that this is a large PR with breaking changes, but I really think it's for the best. I will annotate most of the significant changes with comments in this PR to make reviewing easier. I will post a summary with each important commit.

Furthermore, almost all commits are self-contained (they are granular and they compile). The only exception is the commit titled "Add 'submitTx' and 'awaitTxConfirmed' under 'GYTxMonad' and refine monad hierarchy". The subsequent commit is required to build the project.

Migration guide

For code that uses submission after building, migration is a bit more work but has a simple pattern. This pattern can be observed in Privnet code, for example.

- txBodyPlace <- ctxRunI ctx (ctxUserF ctx) $ do
+ ctxRun ctx (ctxUserF ctx) $ do
      addr <- scriptAddress giftValidatorV1
-     return $ mconcat
+     txBodyPlace <- buildTxBody $ mconcat
           [ mustHaveOutput $ mkGYTxOut addr (valueSingleton goldAC 10) (datumFromPlutusData ())
           ]
- void $ submitTx ctx (ctxUserF ctx) txBodyPlace
+     submitTxBodyConfirmed_ txBodyPlace [ctxUserF ctx]

Code that previously used traversing functions such as rrunGYTxMonadNodeF in order to build a GYTxSkeleton contained within a traversable (e.g for interpeting GYTxMonadNode (Maybe (GYTxSkeleton v))) will simply use well-known helpers to traverse instead, making things much simpler:

- grabGiftsTx' <- ctxRunF ctx (ctxUser2 ctx) $ grabGifts  @'PlutusV2 giftValidatorV2
- mapM_ (submitTx ctx (ctxUser2 ctx)) grabGiftsTx'
+ ctxRun ctx (ctxUser2 ctx) $ do
+    grabGiftsTx' <- grabGifts @'PlutusV2 giftValidatorV2 >>= traverse buildTxBody
+    mapM_ (`submitTxBodyConfirmed` [ctxUser2 ctx]) grabGiftsTx'

For code that only built transactions but did not submit them, it is advised to replace the GYTxMonadNode usage with GYTxBuilderMonadIO. Instead of returning a skeleton, it should use buildTxBody on the skeleton. So you'd have: GYTxBuilderMonadIO GYTxBody. This can be run with runGYTxBuilderMonadIO to obtain the GYTxBody like before. Alternative, define a function buildAndRunGYTxBuilderMonadIO :: GYTxBuilderMonadIO (GYTxSkeleton v) -> GYTxBody, which is simply a composition of buildTxBody followed by runGYTxBuilderMonadIO.

- txBodyPlace <- runGYTxMonadNode nid providers addrs change collateral $ do
+ txBodyPlace <- buildAndRunGYTxBuilderMonadIO nid providers addrs change collateral $ do
      addr <- scriptAddress giftValidatorV1
      return $ mconcat
           [ mustHaveOutput $ mkGYTxOut addr (valueSingleton goldAC 10) (datumFromPlutusData ())
           ]

where:

buildAndRunGYTxBuilderMonadIO :: GYTxBuilderMonadIO (GYTxSkeleton v) -> GYTxBody
buildAndRunGYTxBuilderMonadIO act = runGYTxBuilderMonadIO $ act >>= buildTxBody

This way, a very simple find and replace action will complete most of the migration.

Extended ideas

There could be a GYTxMonad m => GYTxGameMonad m, which could allow a "game situation" where we have multiple users building and submitting contracts, interacting with one another etc. This would essentially have methods like waitUntilSlot, waitUntilBlock etc. There needs to be a function asUser :: (GYTxMonad m, GYTxGameMonad n) => User -> m a -> n a to facilitate the "game" flow. Added.
Related to someUTxO does not actually update envUsedSomeUTxOs in GYTxMonadNode #318 - GYTxBuilderMonadIO should actually have its envUsedSomeUTxO somehow exposed to GYTxMonadIO so that the submitTx implementation of GYTxMonadIO can reset it. I'd suggest making this field mutable. That way, a stateful flow can be achieved - which is necessary since envUsedSomeUTxO is meant to be stateful. I would discourage the usage of the state monad. Just utilize ReaderT _ IO which we already have.
Some day, this codebase should use an effect system instead. effectful will probably be a good choice. Or eff, if it's no longer experimental.

Type of Change

Please mark the relevant option(s) for your pull request:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Code refactoring (improving code quality without changing its behavior)
Documentation update (adding or updating documentation related to the project)

Checklist

Please ensure that your pull request meets the following criteria:

I have read the Contributing Guide
My code follows the project's coding style and best practices
My code is appropriately commented and includes relevant documentation
I have added tests to cover my changes
All new and existing tests pass
I have updated the documentation, if necessary

Testing

Please describe the tests you have added or modified, and provide any additional context or instructions needed to run the tests.

Test A
Test B

Additional Information

If you have any additional information or context to provide, such as screenshots, relevant issues, or other details, please include them here.

TotallyNotChase · 2024-07-10T22:38:34Z

Summary of important commits:

Common MTL transformer instances (also remove ExceptT instance)

This adds some MTL transformer instances so downstream (user code) can easily compose with our monad in easy ways.

I have removed the ExceptT instasnce. Please read the associated note added in the commit.
Reorder GYInScript type variables

Mostly unrelated to the main purpose of the PR, but greatly helpful for users using TypeApps. This is a breaking change. Helped get rid of a verbose type signature in a privnet test.
Add ownChangeAddress and ownCollateral to GYTxMonad

Simple change that allows building transactions within the monad later on

Note the added law on someUTxO. See: someUTxO does not actually update envUsedSomeUTxOs in GYTxMonadNode #318
Remove BuildTxException, rename to GYBuildTxError

Consistent naming and exceptions mechanism

Since transaction building is part of the monad now, transaction build errors ought to also be part of the monad errors.
Add submitTx and awaitTxConfirmed to GYTxMonad

Main commit amending the monad class declaration and hierarchy. Upto GYTxUserQueryMonad denotes query monad capabilities (non chain mutating).

GYTxSpecialQueryMonad could potentially be removed. See note added in the commit.

Note the laws around submission and how it relates to someUTxO.
Make transaction building a part of GYTxMonad utilities

Main commit amending the monad instances and their usage.

This showcases what changes might be required in user code as well. See changes in Privnet code.
Use pre-provided submission utils rather than privnet defined ones.

Note that the submission functions only sign with the payment key. Previously, they checked whether stake key was required and added it if necessary. This is done manually now.

Upon inspection, it seems like this feature of detecting the necessity for stake key was not necessary since the Stake utils test do that on their own anyway.

Note how the Privnet code utilizes ctxRunQuery, ctxRun, and ctxRunBuilder etc. to limit expectations and mutation.

TotallyNotChase · 2024-07-10T22:40:39Z

I suggest reviewing this PR commit by commit. Most of the changes to the Privnet code can be ignored. It's simply updating usage and its summarized in here. My commit summary above might help in reviewing as well.

Before, merging - I need to know how the parallel and chaining building could be amended. I'm thinking of removing the whole Traversable f thing from buildTxCore entirely and then exposing the parallel and chaining functions. They are currently in GeniusYield.TxBuilder but not exposed.

TotallyNotChase · 2024-07-11T16:57:33Z