Skip to content

Commit

Permalink
(PXP-6544) Limited PFB Export from the Files Tab (#729)
Browse files Browse the repository at this point in the history
* Export file PFB if all selected files on same node

* Clean up code

* Add portal configuration

* Add tooltip if file pfb export button is disabled

* Show error message on failed PFB export

* Revert disabling refreshManifestEntryCount

* Use different button types for File tab PFB export

- Reason: easier time configuring specific behavior for specific
tabs (e.g. Data and Files tab). For example, if the `export` button type
is present it triggers `refreshManifestEntryCount` behavior, which is
not needed for the files tab.

* Clean up code

* Use job status for error messages instead of FETCH_ERROR

* Revert mistaken debugging change

* Fix bug: disable other pfb buttons when file pfb request in-flight

* Make export File PFB tooltip text less confusing.

* Revert mistakenly uploaded files

* Refactor: consolidate duplicated code in isButtonEnabled()

* Refactor: move misconfiguration checks to constructor so they only run once

* Fix syntax error in misconfiguration check

* Fix inconsistent state update

* Fix another inconsistent state update

* Link to downloadable PFB files

* Add documentation

* Refactor
  • Loading branch information
em-ingram authored Sep 8, 2020
1 parent c4596f6 commit f341b1b
Show file tree
Hide file tree
Showing 4 changed files with 320 additions and 27 deletions.
116 changes: 116 additions & 0 deletions docs/limited_flie_pfb_export.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
Limited File PFB export adds a limited ability to export PFBs of data files from the Files tab. The limitation is that users cannot export data files of different data types (e.g. `Aligned Reads`, `Imaging Files`, etc.). (More specifically, users cannot export data files that are on different nodes in the graph.) We're accepting this limitation in order to avoid a significant rewrite of the pelican-export job, which assumes all entities to export are on the same node.

- Link to Jira ticket: [PXP-6544](https://ctds-planx.atlassian.net/browse/PXP-6544)
- Link to design doc: https://docs.google.com/document/d/12FkAYOpDuSdQScEgYBxXsPUdm8GUuUZgD6xU7EQga5A/edit#heading=h.53vab1pwrz1y
- Link to relevant Pelican PR: https://github.com/uc-cdis/pelican/pull/33
- Link to manual test plan: https://github.com/uc-cdis/gen3-qa/pull/454

### How to deploy
- Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1
- Limited File PFB Export is enabled by setting `fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" }` and by adding buttons of buttonType `export-files` (Export to Terra), `export-files-to-pfb` (Download PFB), or `export-files-to-seven-bridges` (Export to Seven Bridges) to `fileExplorerConfig.buttons`. Example:
```
"fileExplorerConfig": {
...
"buttons": [
....
{
"enabled": true,
"type": "export-files-to-pfb",
"title": "Export All to PFB",
"rightIcon": "external-link",
"tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export."
},
{
"enabled": true,
"type": "export-files",
"title": "Export All to Terra",
"rightIcon": "external-link",
"tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export."
}
],
"enableLimitedFilePFBExport": {"sourceNodeField": "source_node"},
```
- Limited File PFB Export requires the `source_node` property to be ETL'd -- `source_node` should be added to `props` in the `file` section of `etlMapping.yaml`. Example:
```
- name: mpingram_file
doc_type: file
type: collector
root: None
category: data_file
props:
- name: object_id
- name: md5sum
...
- name: source_node
```
- Limited File PFB Export requires a new `export-files` job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to `"file"`, or the name of the Guppy file index if it isn't "file". *For BioDataCatalyst, the EXTRA_NODES environment variable must be set to `""`*. (This is for backwards compatibility: pelican-export includes `reference_file` by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config:
```
{
"name": "pelican-export-files",
"action": "export-files",
"container": {
"name": "job-task",
"image": "quay.io/cdis/pelican-export:0.5.0",
"pull_policy": "Always",
"env": [
{
"name": "DICTIONARY_URL",
"valueFrom": {
"configMapKeyRef": {
"name": "manifest-global",
"key": "dictionary_url"
}
}
},
{
"name": "GEN3_HOSTNAME",
"valueFrom": {
"configMapKeyRef": {
"name": "manifest-global",
"key": "hostname"
}
}
},
{
"name": "ROOT_NODE",
"value": "file"
},
{
"name": "EXTRA_NODES",
"value": ""
}
],
"volumeMounts": [
{
"name": "pelican-creds-volume",
"readOnly": true,
"mountPath": "/pelican-creds.json",
"subPath": "config.json"
},
{
"name": "peregrine-creds-volume",
"readOnly": true,
"mountPath": "/peregrine-creds.json",
"subPath": "creds.json"
}
],
"cpu-limit": "1",
"memory-limit": "4Gi"
},
"volumes": [
{
"name": "pelican-creds-volume",
"secret": {
"secretName": "pelicanservice-g3auto"
}
},
{
"name": "peregrine-creds-volume",
"secret": {
"secretName": "peregrine-creds"
}
}
],
"restart_policy": "Never"
},
```
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,8 @@
width: unset;
font-size: medium;
}

.explorer-button-group__toaster-dl-link {
color: var(--g3-color__white);
text-decoration: underline;
}
Loading

0 comments on commit f341b1b

Please sign in to comment.