Code

Extract common ingestion tasks#5847

Code

Merged

oliverjwroberts merged 8 commits intomainuktrade/data-hub-api:mainfrom fixup/CLS2-995-extract-common-ingestion-tasksuktrade/data-hub-api:fixup/CLS2-995-extract-common-ingestion-tasksCopy head branch name to clipboardDec 12, 2024

Commits

Commits on Dec 10, 2024

• Create ingest.IngestedObject model

  oliverjwroberts

  committed
  3a9c667

  Copy full SHA for 3a9c667

• Add S3ObjectProcessor class

  oliverjwroberts

  committed
  8c53d0b

  Copy full SHA for 8c53d0b

• Add base identification and ingestion tasks

  oliverjwroberts

  committed
  ef7a800

  Copy full SHA for ef7a800

• Extract functionality into is_job_running, _create_ingested_object_instance, and _log_ingestion_metrics methods

  oliverjwroberts

  committed
  1c317d8

  Copy full SHA for 1c317d8

• Fix test_should_process_record_returns_true_when_error_determining_datetime

  oliverjwroberts

  committed
  de6f0c5

  Copy full SHA for de6f0c5

• Add tests for missing lines

  oliverjwroberts

  committed
  ec1e900

  Copy full SHA for ec1e900

Commits on Dec 12, 2024

• Remove superfluous if statements and str conversions

  oliverjwroberts

  committed
  f8f4617

  Copy full SHA for f8f4617

• Fix list_objects and get_most_recent_object_key methods

  Show description for d0d65e3
  oliverjwroberts

  committed
  d0d65e3

  Copy full SHA for d0d65e3