Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Unify OpenSearch PPL Data Type and Spark PPL Data Type #3339

Open
penghuo opened this issue Feb 19, 2025 · 3 comments
Open

[FEATURE] Unify OpenSearch PPL Data Type and Spark PPL Data Type #3339

penghuo opened this issue Feb 19, 2025 · 3 comments
Labels
enhancement New feature or request v3.0.0

Comments

@penghuo
Copy link
Collaborator

penghuo commented Feb 19, 2025

Is your feature request related to a problem?
Currently, OpenSearch PPL data type and Spark PPL data type are not aligned. For instance, opensearch-project/opensearch-spark#1057.

What solution would you like?
We proposed a breaking change in OpenSearch PPL Data Type.

OpenSearch PPL Data Type Spark PPL Data Type Unified PPL Data Type
byte tinyint tinyint
short smallint smallint
integer int int
long bigint bigint

It is breaking change, and release in OpenSearch 3.0, change includes

  • Integration Test
  • Documentation

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

@penghuo penghuo added enhancement New feature or request untriaged v3.0.0 labels Feb 19, 2025
@penghuo
Copy link
Collaborator Author

penghuo commented Feb 20, 2025

@penghuo penghuo removed the untriaged label Feb 24, 2025
@LantaoJin
Copy link
Member

@penghuo , do you mean in 3.0.0, the PPL data type won't include byte, short, integer and long. instead, they will upgrade to
byte -> tinyint
short -> smallint
integer -> int
long -> bigint

@penghuo
Copy link
Collaborator Author

penghuo commented Feb 25, 2025

@penghuo , do you mean in 3.0.0, the PPL data type won't include byte, short, integer and long. instead, they will upgrade to
byte -> tinyint
short -> smallint
integer -> int
long -> bigint

  • change includes

  • not touched

    • ExprCoreType will not changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request v3.0.0
Projects
None yet
Development

No branches or pull requests

2 participants