[Calcite Engine] Push down project and filter operator into index scan

[qianheng-aws]

Description

Push down project and filter operator into index scan. This PR includes change:

1. OpenSearchIndex change to use its specific scan operator, CalciteOpenSearchIndexScan
2. Simplify to use RelRunners to do execution since the current implementation of CalciteOpenSearchIndexScan does not require a "schema" as context.
3. Add OpenSearchFilterIndexScanRule and OpenSearchProjectIndexScanRule to do filter/project operator push down.
4. Add PredicateAnalyzer to build QueryBuilder for filter condition.

Related Issues

Part Resolves #3331

Check List

• New functionality includes testing.
• New functionality has been documented.
• New functionality has javadoc added.
• New functionality has a user manual doc added.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[penghuo]

Thx!

[dai-chen]

Thanks for the changes! Btw, do we have a high level diagram for Calcite integration? Just want to learn and get more context. As I understand, we're trying to adapt v2 OS domain classes to Calcite connector, right?

[dai-chen]

This assumes the given filter can be fully pushed down?

[qianheng-aws]

If analyze exception occurs in PredicateAnalyzer, it will return false and won't push down this filter.

In v2 engine, will there any other case except aggregation block filter push down?

[qianheng-aws]

Thanks for the changes! Btw, do we have a high level diagram for Calcite integration? Just want to learn and get more context. As I understand, we're trying to adapt v2 OS domain classes to Calcite connector, right?

This RFC should be the root issue and should have the overview: #3229

[LantaoJin]

@qianheng-aws seems many ITs fail with this PR in my local.
For example

• Predicates with OR

String.format(
                "source=%s | where (account_number = 25 or balance > 10000) and gender = 'M' |"
                    + " fields firstname, lastname",
                TEST_INDEX_BANK)

• Multiple Tables and Filters

String.format(
                "source=%s, %s gender = 'F' | stats count() as c",
                TEST_INDEX_BANK, TEST_INDEX_BANK)

could you add at least two tests to verify each PR in future:

1. Add a UT such as CalcitePPLBasicTest extends CalcitePPLAbstractTest
2. Add a IT such as CalcitePPLBasicIT extends CalcitePPLTestCase