Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable HPSS via Globus #3240

Open
wants to merge 141 commits into
base: develop
Choose a base branch
from

Conversation

DavidHuber-NOAA
Copy link
Contributor

@DavidHuber-NOAA DavidHuber-NOAA commented Jan 17, 2025

Description

This enables the archiving or ROTDIR tarballs to HPSS from systems that are not connected directly to HPSS as long as they can communicate to Niagara via Globus.

Transfers are initialized by the Sven client on Hercules or Orion and then finished on Niagara using the Doorman service. Documentation on these services can be found here: Sven User's Guide. These services were developed by George Fekete.

Resolves #1357

Type of change

  • New feature (adds functionality)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? YES
  • Does this change require an update to any of the following submodules? NO

How has this been tested?

  • Cycle test on Hercules
  • Forecast-only test on Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

github-advanced-security[bot]

This comment was marked as outdated.

@DavidHuber-NOAA
Copy link
Contributor Author

@aerorahul aerorahul requested review from WalterKolczynski-NOAA and removed request for WalterKolczynski-NOAA March 6, 2025 16:22
@DavidHuber-NOAA
Copy link
Contributor Author

I tested a C96C48_hybatmDA case on Hercules successfully with ARCHCOM_TO=globus_hpss 🎉!! This PR is ready to go.

@DavidHuber-NOAA
Copy link
Contributor Author

Issue RDHPCS#2025022854000047 has been resolved as well.

Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DavidHuber-NOAA
Thanks for this work. Changes look good to me.

@DavidHuber-NOAA
Copy link
Contributor Author

Launching CI on Hera.

@DavidHuber-NOAA DavidHuber-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Mar 7, 2025
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Mar 7, 2025
@emcbot
Copy link

emcbot commented Mar 8, 2025

CI Passed on Hera in Build# 1
Built and ran in directory /scratch1/NCEPDEV/global/glopara/CI/3240


Experiment C96mx100_S2S_c4c22549 Completed 1 Cycles: *SUCCESS* at Sat Mar  8 09:29:32 UTC 2025
Experiment C48_ATM_c4c22549 Completed 1 Cycles: *SUCCESS* at Sat Mar  8 09:53:53 UTC 2025
Experiment C48mx500_hybAOWCDA_c4c22549 Completed 2 Cycles: *SUCCESS* at Sat Mar  8 10:42:45 UTC 2025
Experiment C96C48_hybatmDA_c4c22549 Completed 3 Cycles: *SUCCESS* at Sat Mar  8 11:31:49 UTC 2025
Experiment C96_atm3DVar_c4c22549 Completed 3 Cycles: *SUCCESS* at Sat Mar  8 11:43:57 UTC 2025
Experiment C96C48_hybatmaerosnowDA_c4c22549 Completed 3 Cycles: *SUCCESS* at Sat Mar  8 11:43:57 UTC 2025
Experiment C48_S2SWA_gefs_c4c22549 Completed 1 Cycles: *SUCCESS* at Sat Mar  8 11:52:38 UTC 2025
Experiment C48_S2SW_c4c22549 Completed 1 Cycles: *SUCCESS* at Sat Mar  8 12:02:34 UTC 2025
Experiment C96C48_ufs_hybatmDA_c4c22549 Completed 3 Cycles: *SUCCESS* at Sat Mar  8 12:38:40 UTC 2025
Experiment C48mx500_3DVarAOWCDA_c4c22549 Completed 2 Cycles: *SUCCESS* at Sat Mar  8 13:09:43 UTC 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable transfers of archived data to Niagara from non-HPSS-connected systems
6 participants