Skip to content

Commit

Permalink
Parameterize reindex job in GitLab with catalog to reindex (#5336)
Browse files Browse the repository at this point in the history
  • Loading branch information
achave11-ucsc committed Feb 28, 2024
1 parent 87ea4f1 commit beb26c1
Show file tree
Hide file tree
Showing 4 changed files with 28 additions and 2 deletions.
13 changes: 13 additions & 0 deletions OPERATOR.rst
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,19 @@ For an example of how to document failures within a PR `click here`_.

.. _click here: https://github.com/DataBiosphere/azul/pull/3050#issuecomment-840033931

Reindexing a specific catalog in GitLab
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

From the GitLab console, select the reindex job for the pipeline that needs
reindexing of any targeted catalog/s. From there, you should see an option for
definings key, value pair of variables to be used as an environement variables
in this job's run.

Set Key=azul_current_catalog and Value to the desired catalog/s to be index.
For example, if we want to only index ``dcp3``, then set Value=dcp3. Once
you've confirm this is the desired configuration, start the reindex by
clicking on ``Run job``.

Updating the AMI for GitLab instances
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
4 changes: 4 additions & 0 deletions environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,10 @@ def env() -> Mapping[str, Optional[str]]:
#
'AZUL_CATALOGS': None,

# The name of a catalog to perform reindex or other operational tasks on.
#
'azul_current_catalog': None,

# The Account ID number for AWS
#
'AZUL_AWS_ACCOUNT_ID': None,
Expand Down
9 changes: 7 additions & 2 deletions scripts/reindex.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,13 @@
nargs='+',
metavar='NAME',
default=[
c for c in config.catalogs
if c not in config.integration_test_catalogs
catalog.name
for catalog in config.catalogs.values()
if not catalog.is_integration_test_catalog
]
if config.current_catalog is None else
[
config.catalogs[config.current_catalog].name
],
choices=config.catalogs,
help='The names of the catalogs to reindex.')
Expand Down
4 changes: 4 additions & 0 deletions src/azul/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,10 @@ def catalogs(self) -> Mapping[CatalogName, Catalog]:
def default_catalog(self) -> CatalogName:
return first(self.catalogs)

@property
def current_catalog(self) -> Optional[str]:
return self.environ.get('azul_current_catalog')

def it_catalog_for(self, catalog: CatalogName) -> Optional[CatalogName]:
it_catalog = self.catalogs[catalog].it_catalog
assert it_catalog in self.integration_test_catalogs, it_catalog
Expand Down

0 comments on commit beb26c1

Please sign in to comment.