Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add codec parameter to hdfs save command #71

Merged
merged 1 commit into from
Aug 28, 2024

Conversation

51-code
Copy link
Contributor

@51-code 51-code commented Aug 26, 2024

Fixes #70 .

Introduced "codec" parameter to the teragrep exec hdfs save command. It will take a String as the value. Parsing of that value will happen in PTH-10 and the requirements for that is documented to PTH-10 #280. The idea is to give the user the possibility to use a specific codec for the compression of saved files and PTH-10 will check if that codec is valid for the format chosen in the command.

Unit tests include a general test for using all parameters together as well as a basic test for using the codec parameter.

@51-code 51-code added the enhancement New feature or request label Aug 26, 2024
@51-code 51-code requested a review from eemhu August 26, 2024 09:01
@51-code 51-code self-assigned this Aug 26, 2024
Copy link
Contributor

@eemhu eemhu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Had a question about using clearly defined rules for each of the codecs instead of a StringType, but it is a conscious design choice agreed upon with the architect.

@51-code 51-code requested a review from kortemik August 28, 2024 07:36
@kortemik kortemik merged commit db2055c into teragrep:main Aug 28, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add an optional compress parameter to HDFS save
3 participants