[#6920] docs(docs): Add doc to Model update #6921

Abyss-lord · 2025-04-14T10:27:24Z

What changes were proposed in this pull request?

Add doc to Model update. Except for Update Comment, this needs to wait for PR merging.

Why are the changes needed?

Fix: #6920

Does this PR introduce any user-facing change?

Add docs of Model update .

How was this patch tested?

local test.

Add doc to Model update.

Abyss-lord · 2025-04-14T11:24:39Z

Hi @jerryshao , could you please review this PR when you have time? I’d really appreciate your feedback.

docs/manage-model-metadata-using-gravitino.md

fix docs Co-authored-by: Qiming Teng <tengqm@outlook.com>

docs/manage-model-metadata-using-gravitino.md

Fix docs.

Abyss-lord · 2025-04-14T16:27:30Z

Hi @tengqm @jerryshao , I've completed the code updates and would appreciate your review of the PR when you have a moment. Here's a summary of the commits:

Fix docs.

docs/gravitino-server-config.md

…module (apache#6922) ### What changes were proposed in this pull request? support fileset multiple locations in the core and catalog module ### Why are the changes needed? Fix: apache#6892 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? tests added

…failed with krb cluster for iceberg issue (apache#6873) ### What changes were proposed in this pull request? Fix use catalog and show databases failed with krb cluster for iceberg issue ### Why are the changes needed? Fix: apache#6871 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? local test. steps: 1、curl -X POST -H "Accept: application/vnd.gravitino.v1+json" -H "Content-Type: application/json" -d '{ "name": "iceberg_catalog", "type": "RELATIONAL", "comment": "iceberg catalog", "provider": "lakehouse-iceberg", "properties": { "catalog-backend": "hive", "warehouse": "/pathforhivetest", "uri": "thrift://HMS HOST:9083", "gravitino.bypass.hadoop.security.authentication": "kerberos", "gravitino.bypass.hive.metastore.sasl.enabled": "true", "gravitino.bypass.hive.metastore.kerberos.principal": "hive/HMS HOST@REALM", "authentication.kerberos.keytab-uri": "PATH OF KEYTAB", "authentication.kerberos.principal": "hive/HOST INSATANCE@REALM", "authentication.type": "kerberos", "authentication.impersonation-enable": true } }' http://localhost:8090/api/metalakes/dev_metalake/catalogs 2、 spark-sql -v --conf spark.plugins="org.apache.gravitino.spark.connector.plugin.GravitinoSparkPlugin" --conf spark.sql.gravitino.uri=http://localhost:8090/ --conf spark.sql.gravitino.metalake=dev_metalake --conf spark.sql.gravitino.enableIcebergSupport=true --conf spark.sql.warehouse.dir=xxxx --conf spark.sql.gravitino.authType=kerberos --conf spark.kerberos.principal=user1/[xxxxxx@MY.HADOOP--conf](mailto:xxxxxx@MY.HADOOP--conf) spark.kerberos.keytab=/user1.keytab 3、use iceberg_catalog;show database ![image](https://github.com/user-attachments/assets/92b9a581-0cb2-45d3-b554-681adea9150f)

### What changes were proposed in this pull request? Support lineage sinks, Linkage sinks manager will load all linkage sinks and register them as event listeners. ### Why are the changes needed? Fix: apache#6781 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? 1. add UT 2. setup a local environment to test spark linage

…T API (apache#6924) ### What changes were proposed in this pull request? support fileset multiple locations in REST API ### Why are the changes needed? support fileset multiple locations in REST API Fix: apache#6893 ### Does this PR introduce _any_ user-facing change? yes ### How was this patch tested? tests added

…iple locations (apache#6935) ### What changes were proposed in this pull request? Java/Python clients supports fileset multiple locations ### Why are the changes needed? Fix: apache#6894 ### Does this PR introduce _any_ user-facing change? yes, Java/Python clients support fileset multiple locations ### How was this patch tested? tests added

apache#6937) ### What changes were proposed in this pull request? support specifying the location name in GVFS ### Why are the changes needed? since the fileset supports multiple locations, the GVFS should support to specify the location when preforming FS ops Fix: apache#6895 ### Does this PR introduce _any_ user-facing change? yes ### How was this patch tested? tests added

…pache#6861) Support update comment for model version - [x] PR1: Add ModelVersionChange API interface, Implement the update comment logic in model catalog and JDBC backend logic, update related event. - [ ] PR2: Add REST endpoint to support model version change, add Java client and Python client for model version comment update. Fix: apache#6815 User can update the comment of a model version. local test. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: roryqi <he@datastrato.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jerry Shao <jerryshao@datastrato.com> Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com> Co-authored-by: FANNG <xiaojing@datastrato.com> Co-authored-by: Eric Chang <e850506@gmail.com> Co-authored-by: Mini Yu <yuqi@datastrato.com> Co-authored-by: mchades <liminghuang@datastrato.com>

Fix docs.

tengqm · 2025-04-15T23:23:23Z

/lgtm

Abyss-lord · 2025-04-16T01:16:35Z

/lgtm

Thx, @jerryshao plz help to check this PR when you have time.

jerryshao · 2025-04-16T01:53:10Z

I would suggest to have doc merged in the last moment. Before that, we'd better hurry up to merge other left PRs for model/model version alteration as possible as we can.

Abyss-lord · 2025-04-16T02:05:17Z

I would suggest to have doc merged in the last moment. Before that, we'd better hurry up to merge other left PRs for model/model version alteration as possible as we can.

@jerryshao I will complete the Model Version update as soon as possible, but at the moment, it seems that only the PR for updating the Model Version comment can be merged. As for the PR related to updating the Model comment, I'm not sure if @TEOTEO520 will be able to finish it before the code is freeze. Personally, I would like to complete the entire Model update feature in version 0.9, although it's unlikely that the full Model Version update can be finished in time. WDYT

jerryshao · 2025-04-16T02:12:10Z

You can continue working on the other model version updating things, don't worry about the time.

jerryshao · 2025-04-16T02:13:48Z

And for the doc, we can merge before the release based on the status at that time.

Abyss-lord · 2025-04-16T02:20:39Z

And for the doc, we can merge before the release based on the status at that time.

@jerryshao Thx, i will complete the Model Version update as soon as possible.

…fileset credential (apache#6985) ### What changes were proposed in this pull request? - using the HTTP header to specify the location name when getting crednetial of the fileset - gvfs automatically retrieves the corresponding credentials based on the current location name. ### Why are the changes needed? Fix: apache#6981 ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? local run `FilesetCatalogCredentialIT` and `GravitinoVirtualFileSystemS3CredentialIT`

Add Model version docs.

Add Model version openapi.

Abyss-lord · 2025-04-22T04:04:18Z

@jerryshao @tengqm , I've completed the code updates and would appreciate your review of the PR when you have a moment. Here's a summary of the commits:

Add detailed documentation covering the ModelVersion changes.
Update the OpenAPI specification to include the ModelVersion change endpoints.

jerryshao · 2025-04-22T04:13:49Z

OK, I will review it today.

docs/manage-model-metadata-using-gravitino.md

jerryshao · 2025-04-22T06:14:53Z

docs/manage-model-metadata-using-gravitino.md

+by 
+sending a `PUT` request to the `/api/metalakes/{metalake_name}/catalogs/{catalog_name}/schemas/


Can you merge this two lines together, line 635 only has one word.

This is not changed from what I saw.

docs/manage-model-metadata-using-gravitino.md

docs/open-api/models.yaml

Fix docs.

Fix yaml.

Fix docs.

Abyss-lord · 2025-04-22T07:17:23Z

Hi @jerryshao , I've completed the code updates and would appreciate your review of the PR when you have a moment. Here's a summary of the commits:

Resolve all syntax errors.
Standardize table formatting in the Markdown file.
Remove the “Update Model” comment section.

docs/manage-model-metadata-using-gravitino.md

Fix docs.

### What changes were proposed in this pull request? Add doc to Model update. Except for Update Comment, this needs to wait for PR merging. ### Why are the changes needed? Fix: #6920 ### Does this PR introduce _any_ user-facing change? Add docs of Model update . ### How was this patch tested? local test. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Qiming Teng <tengqm@outlook.com> Co-authored-by: mchades <liminghuang@datastrato.com> Co-authored-by: gavin.wang <gracejianbo@163.com> Co-authored-by: FANNG <xiaojing@datastrato.com> Co-authored-by: roryqi <he@datastrato.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jerry Shao <jerryshao@datastrato.com> Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com> Co-authored-by: Eric Chang <e850506@gmail.com> Co-authored-by: Mini Yu <yuqi@datastrato.com> Co-authored-by: Kang <zhoukangcn@gmail.com> Co-authored-by: Justin Mclean <justin@classsoftware.com> Co-authored-by: Danhua Wang <danhua@datastrato.com> Co-authored-by: yunchi <yunchipang@gmail.com> Co-authored-by: RickyMa <rickyma@tencent.com> Co-authored-by: Yuhui <hui@datastrato.com>

### What changes were proposed in this pull request? Add doc to Model update. Except for Update Comment, this needs to wait for PR merging. ### Why are the changes needed? Fix: apache#6920 ### Does this PR introduce _any_ user-facing change? Add docs of Model update . ### How was this patch tested? local test. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Qiming Teng <tengqm@outlook.com> Co-authored-by: mchades <liminghuang@datastrato.com> Co-authored-by: gavin.wang <gracejianbo@163.com> Co-authored-by: FANNG <xiaojing@datastrato.com> Co-authored-by: roryqi <he@datastrato.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jerry Shao <jerryshao@datastrato.com> Co-authored-by: yangyang zhong <35210666+hdygxsj@users.noreply.github.com> Co-authored-by: Eric Chang <e850506@gmail.com> Co-authored-by: Mini Yu <yuqi@datastrato.com> Co-authored-by: Kang <zhoukangcn@gmail.com> Co-authored-by: Justin Mclean <justin@classsoftware.com> Co-authored-by: Danhua Wang <danhua@datastrato.com> Co-authored-by: yunchi <yunchipang@gmail.com> Co-authored-by: RickyMa <rickyma@tencent.com> Co-authored-by: Yuhui <hui@datastrato.com>

[apache#6920] docs(docs): Add doc to Model update

6fb0c69

Add doc to Model update.

Abyss-lord self-assigned this Apr 14, 2025

jerqi added the branch-0.9 Automatically cherry-pick commit to branch-0.9 label Apr 14, 2025

tengqm reviewed Apr 14, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

Update docs/manage-model-metadata-using-gravitino.md

e58e87d

fix docs Co-authored-by: Qiming Teng <tengqm@outlook.com>

Abyss-lord requested a review from tengqm April 14, 2025 12:56

tengqm reviewed Apr 14, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

tengqm reviewed Apr 14, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

[apache#6920] docs(docs): Add doc to Model update

4ab0de9

Fix docs.

Abyss-lord requested review from tengqm and jerryshao April 14, 2025 16:28

tengqm reviewed Apr 15, 2025

View reviewed changes

docs/gravitino-server-config.md Outdated Show resolved Hide resolved

mchades and others added 9 commits April 16, 2025 00:13

[apache#6920] docs(docs): Add doc to Model update

ec2bfea

Fix docs.

Merge branch 'main' into doc-model-update

221463d

mchades and others added 4 commits April 22, 2025 11:18

[apache#6920] docs(docs): Add doc to Model update

b5ff0c1

Add Model version docs.

Merge branch 'main' into doc-model-update

e1ab8e2

[apache#6920] docs(docs): Add doc to Model update

02de04f

Add Model version openapi.

Abyss-lord requested a review from tengqm April 22, 2025 04:02

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/open-api/models.yaml Outdated Show resolved Hide resolved

Abyss-lord added 3 commits April 22, 2025 14:32

[apache#6920] docs(docs): Add doc to Model update

3896c38

Fix docs.

[apache#6920] docs(docs): Add doc to Model update

095dd3f

Fix yaml.

[apache#6920] docs(docs): Add doc to Model update

25708b4

Fix docs.

Abyss-lord requested a review from jerryshao April 22, 2025 07:15

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

jerryshao reviewed Apr 22, 2025

View reviewed changes

docs/manage-model-metadata-using-gravitino.md Outdated Show resolved Hide resolved

Abyss-lord added 4 commits April 22, 2025 17:29

[apache#6920] docs(docs): Add doc to Model update

11c8062

Fix docs.

[apache#6920] docs(docs): Add doc to Model update

f73a9dc

Fix docs.

[apache#6920] docs(docs): Add doc to Model update

fd76563

Fix docs.

[apache#6920] docs(docs): Add doc to Model update

a31d73c

Fix docs.

jerryshao approved these changes Apr 22, 2025

View reviewed changes

jerryshao merged commit 509dab1 into apache:main Apr 22, 2025
27 checks passed

Abyss-lord deleted the doc-model-update branch April 22, 2025 10:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#6920] docs(docs): Add doc to Model update #6921

[#6920] docs(docs): Add doc to Model update #6921

Abyss-lord commented Apr 14, 2025

Abyss-lord commented Apr 14, 2025

Abyss-lord commented Apr 14, 2025

tengqm commented Apr 15, 2025

Abyss-lord commented Apr 16, 2025 •

edited

Loading

jerryshao commented Apr 16, 2025

Abyss-lord commented Apr 16, 2025 •

edited

Loading

jerryshao commented Apr 16, 2025

jerryshao commented Apr 16, 2025

Abyss-lord commented Apr 16, 2025

Abyss-lord commented Apr 22, 2025

jerryshao commented Apr 22, 2025

jerryshao Apr 22, 2025

jerryshao Apr 22, 2025

Abyss-lord commented Apr 22, 2025

		by
		sending a `PUT` request to the `/api/metalakes/{metalake_name}/catalogs/{catalog_name}/schemas/

[#6920] docs(docs): Add doc to Model update #6921

[#6920] docs(docs): Add doc to Model update #6921

Conversation

Abyss-lord commented Apr 14, 2025

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Abyss-lord commented Apr 14, 2025

Abyss-lord commented Apr 14, 2025

tengqm commented Apr 15, 2025

Abyss-lord commented Apr 16, 2025 • edited Loading

jerryshao commented Apr 16, 2025

Abyss-lord commented Apr 16, 2025 • edited Loading

jerryshao commented Apr 16, 2025

jerryshao commented Apr 16, 2025

Abyss-lord commented Apr 16, 2025

Abyss-lord commented Apr 22, 2025

jerryshao commented Apr 22, 2025

jerryshao Apr 22, 2025

Choose a reason for hiding this comment

jerryshao Apr 22, 2025

Choose a reason for hiding this comment

Abyss-lord commented Apr 22, 2025

Abyss-lord commented Apr 16, 2025 •

edited

Loading

Abyss-lord commented Apr 16, 2025 •

edited

Loading