-
Notifications
You must be signed in to change notification settings - Fork 47
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
(PXP-6544) Limited PFB Export from the Files Tab (#729)
* Export file PFB if all selected files on same node * Clean up code * Add portal configuration * Add tooltip if file pfb export button is disabled * Show error message on failed PFB export * Revert disabling refreshManifestEntryCount * Use different button types for File tab PFB export - Reason: easier time configuring specific behavior for specific tabs (e.g. Data and Files tab). For example, if the `export` button type is present it triggers `refreshManifestEntryCount` behavior, which is not needed for the files tab. * Clean up code * Use job status for error messages instead of FETCH_ERROR * Revert mistaken debugging change * Fix bug: disable other pfb buttons when file pfb request in-flight * Make export File PFB tooltip text less confusing. * Revert mistakenly uploaded files * Refactor: consolidate duplicated code in isButtonEnabled() * Refactor: move misconfiguration checks to constructor so they only run once * Fix syntax error in misconfiguration check * Fix inconsistent state update * Fix another inconsistent state update * Link to downloadable PFB files * Add documentation * Refactor
- Loading branch information
Showing
4 changed files
with
320 additions
and
27 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,116 @@ | ||
Limited File PFB export adds a limited ability to export PFBs of data files from the Files tab. The limitation is that users cannot export data files of different data types (e.g. `Aligned Reads`, `Imaging Files`, etc.). (More specifically, users cannot export data files that are on different nodes in the graph.) We're accepting this limitation in order to avoid a significant rewrite of the pelican-export job, which assumes all entities to export are on the same node. | ||
|
||
- Link to Jira ticket: [PXP-6544](https://ctds-planx.atlassian.net/browse/PXP-6544) | ||
- Link to design doc: https://docs.google.com/document/d/12FkAYOpDuSdQScEgYBxXsPUdm8GUuUZgD6xU7EQga5A/edit#heading=h.53vab1pwrz1y | ||
- Link to relevant Pelican PR: https://github.com/uc-cdis/pelican/pull/33 | ||
- Link to manual test plan: https://github.com/uc-cdis/gen3-qa/pull/454 | ||
|
||
### How to deploy | ||
- Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 | ||
- Limited File PFB Export is enabled by setting `fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" }` and by adding buttons of buttonType `export-files` (Export to Terra), `export-files-to-pfb` (Download PFB), or `export-files-to-seven-bridges` (Export to Seven Bridges) to `fileExplorerConfig.buttons`. Example: | ||
``` | ||
"fileExplorerConfig": { | ||
... | ||
"buttons": [ | ||
.... | ||
{ | ||
"enabled": true, | ||
"type": "export-files-to-pfb", | ||
"title": "Export All to PFB", | ||
"rightIcon": "external-link", | ||
"tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." | ||
}, | ||
{ | ||
"enabled": true, | ||
"type": "export-files", | ||
"title": "Export All to Terra", | ||
"rightIcon": "external-link", | ||
"tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." | ||
} | ||
], | ||
"enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, | ||
``` | ||
- Limited File PFB Export requires the `source_node` property to be ETL'd -- `source_node` should be added to `props` in the `file` section of `etlMapping.yaml`. Example: | ||
``` | ||
- name: mpingram_file | ||
doc_type: file | ||
type: collector | ||
root: None | ||
category: data_file | ||
props: | ||
- name: object_id | ||
- name: md5sum | ||
... | ||
- name: source_node | ||
``` | ||
- Limited File PFB Export requires a new `export-files` job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to `"file"`, or the name of the Guppy file index if it isn't "file". *For BioDataCatalyst, the EXTRA_NODES environment variable must be set to `""`*. (This is for backwards compatibility: pelican-export includes `reference_file` by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: | ||
``` | ||
{ | ||
"name": "pelican-export-files", | ||
"action": "export-files", | ||
"container": { | ||
"name": "job-task", | ||
"image": "quay.io/cdis/pelican-export:0.5.0", | ||
"pull_policy": "Always", | ||
"env": [ | ||
{ | ||
"name": "DICTIONARY_URL", | ||
"valueFrom": { | ||
"configMapKeyRef": { | ||
"name": "manifest-global", | ||
"key": "dictionary_url" | ||
} | ||
} | ||
}, | ||
{ | ||
"name": "GEN3_HOSTNAME", | ||
"valueFrom": { | ||
"configMapKeyRef": { | ||
"name": "manifest-global", | ||
"key": "hostname" | ||
} | ||
} | ||
}, | ||
{ | ||
"name": "ROOT_NODE", | ||
"value": "file" | ||
}, | ||
{ | ||
"name": "EXTRA_NODES", | ||
"value": "" | ||
} | ||
], | ||
"volumeMounts": [ | ||
{ | ||
"name": "pelican-creds-volume", | ||
"readOnly": true, | ||
"mountPath": "/pelican-creds.json", | ||
"subPath": "config.json" | ||
}, | ||
{ | ||
"name": "peregrine-creds-volume", | ||
"readOnly": true, | ||
"mountPath": "/peregrine-creds.json", | ||
"subPath": "creds.json" | ||
} | ||
], | ||
"cpu-limit": "1", | ||
"memory-limit": "4Gi" | ||
}, | ||
"volumes": [ | ||
{ | ||
"name": "pelican-creds-volume", | ||
"secret": { | ||
"secretName": "pelicanservice-g3auto" | ||
} | ||
}, | ||
{ | ||
"name": "peregrine-creds-volume", | ||
"secret": { | ||
"secretName": "peregrine-creds" | ||
} | ||
} | ||
], | ||
"restart_policy": "Never" | ||
}, | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.