feat(sqlsmith): differential testing

[kwannoel]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Closes #10427

• Adds differential testing executable.
• Differential testing works by generating stream query, extracting the inner query of it, and running it as a batch query.
• It is expected that after sorting, the results of most queries should match.
• Add to main-cron and PR workflow, but soft fail it, currently seems like there are some unresolved errors.
• Adds similar as a dependency.

---

Further improvements:

• Perhaps this mode should have more inserts? 🤔
• If there are exceptions to differential testing, we can add them later. For now it seems the tests usually pass.
• Support session variable fuzzing.
• Better error reproducibility. Currently all we have are the logs to work with for reproducing errors. We should adapt existing tool to extract DDL, DML, diffed queries.
• For reproducibility this should definitely run in madsim too.

Checklist

• I have written necessary rustdoc comments
• I have added necessary unit tests and integration tests
• I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
• My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
• All checks passed in ./risedev check (or alias, ./risedev c)
• My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

Documentation

• My PR contains user-facing changes.

Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

• Installation and deployment
• Connector (sources & sinks)
• SQL commands, functions, and operators
• RisingWave cluster configuration changes
• Other (please specify in the release note below)

Release note

[kwannoel]

Example of failure:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: 
Different number of rows for:
BATCH:
SELECT min(TIMESTAMP '2023-06-26 07:52:41') AS col_0 FROM bid AS t_0 FULL JOIN (WITH with_1 AS (SELECT tumble_3.col_1 AS col_0, tumble_3.col_1 AS col_1 FROM hop(m1, m1.col_0, INTERVAL '1', INTERVAL '5') AS hop_2 RIGHT JOIN tumble(m0, m0.col_0, INTERVAL '4') AS tumble_3 ON hop_2.col_0 = tumble_3.col_0 AND true GROUP BY hop_2.col_1, tumble_3.col_1) SELECT sum((BIGINT '944')) AS col_0, true AS col_1, (INT '109') AS col_2, 'vf0muPg1rg' AS col_3 FROM with_1 WHERE ((REAL '2147483647') >= (REAL '347'))) AS sq_4 ON t_0.url = sq_4.col_3 AND true AND t_0.channel = sq_4.col_3 WHERE sq_4.col_1 GROUP BY t_0.url, t_0.date_time, t_0.auction, t_0.bidder

STREAM:
CREATE MATERIALIZED VIEW stream_277 AS SELECT min(TIMESTAMP '2023-06-26 07:52:41') AS col_0 FROM bid AS t_0 FULL JOIN (WITH with_1 AS (SELECT tumble_3.col_1 AS col_0, tumble_3.col_1 AS col_1 FROM hop(m1, m1.col_0, INTERVAL '1', INTERVAL '5') AS hop_2 RIGHT JOIN tumble(m0, m0.col_0, INTERVAL '4') AS tumble_3 ON hop_2.col_0 = tumble_3.col_0 AND true GROUP BY hop_2.col_1, tumble_3.col_1) SELECT sum((BIGINT '944')) AS col_0, true AS col_1, (INT '109') AS col_2, 'vf0muPg1rg' AS col_3 FROM with_1 WHERE ((REAL '2147483647') >= (REAL '347'))) AS sq_4 ON t_0.url = sq_4.col_3 AND true AND t_0.channel = sq_4.col_3 WHERE sq_4.col_1 GROUP BY t_0.url, t_0.date_time, t_0.auction, t_0.bidder

SELECT:
SELECT * FROM stream_277

BATCH_ROW_LEN:
2

STREAM_ROW_LEN:
1

BATCH_ROWS:


STREAM_ROWS:
"2023-06-26 07:52:41"

DIFF:
+"2023-06-26 07:52:41"

Example with more diffs:
https://gist.github.com/kwannoel/2e760b5a31fbe2566cf100a24b7e5f4e

[codecov]

Codecov Report

Merging #10558 (3c39cb7) into main (f990ebf) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 3c39cb7 differs from pull request most recent head a2c48ab. Consider uploading reports for the commit a2c48ab to get more accurate results

@@            Coverage Diff             @@
##             main   #10558      +/-   ##
==========================================
- Coverage   70.21%   70.21%   -0.01%     
==========================================
  Files        1270     1270              
  Lines      217722   217722              
==========================================
- Hits       152880   152875       -5     
- Misses      64842    64847       +5     

Flag	Coverage Δ
rust	70.21% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 4 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[jon-chuang]

LGTM

[stdrc]

LGTM!