[Feature]: Developer can wait for the deletion of a space to be complete

[gcapizzi]

Background

As a developer
I want to know when a space has finished deleting
So that I can plan other operations accordingly

Dev Notes

• We may want to add timeout logic to check the deletion timestamp on the object set for deletion.
• Note that other jobs default to a 15 minute timeout, but technically the top level delete jobs do not get marked as failed in CF for VMs.
• We're thinking about adding the failed state here along with a small message about checking for the lingering resources like service binding projections
• https://v3-apidocs.cloudfoundry.org/version/3.139.0/index.html#jobs
• Similar to other jobs we suspect this will not require a new crd to track the job, instead encoding the required information in the url

Acceptance Criteria

GIVEN I have a space
WHEN I delete the space
AND I follow the redirect to the deletion job
THEN I see that the job looks like this:

{
  "guid": "space.delete~<the-space-guid>",
  "operation": "space.delete",
  "state": "PROCESSING",
  "links": {
    "self": {
      "href": "https://api.example.org/v3/jobs/space.delete~<the-space-guid>"
    }
  },
  "errors": [],
  "warnings": []
}

GIVEN I have waited long enough for the space deletion to successfully complete
WHEN I get the deletion job
THEN I see it looks like this:

{
  "guid": "space.delete~<the-space-guid>",
  "operation": "space.delete",
  "state": "COMPLETE",
  "links": {
    "self": {
      "href": "https://api.example.org/v3/jobs/space.delete~<the-space-guid>"
    }
  },
  "errors": [],
  "warnings": []
}

GIVEN I have waited long enough for the org deletion to time out
WHEN I get the deletion job
THEN I see it looks roughly like this:

{
  "guid": "space.delete~<the-space-guid>",
  "operation": "space.delete",
  "state": "FAILED",
  "links": {
    "self": {
      "href": "https://api.example.org/v3/jobs/space.delete~<the-space-guid>"
    }
  },
 "errors": [
    {
      "code": 10008,
      "title": "CF-UnprocessableEntity",
      "detail": "something went wrong, check for lingering resources in cfspace"
    }
  ],
  "warnings": [
    {
      "detail": "warning! warning!"
    }
  ]
}

GIVEN the space doesn't exist
WHEN I get the deletion job
THEN I still see a COMPLETE response like the above

GIVEN I have no permissions to get the space
WHEN I get the deletion job
THEN I still see a COMPLETE response like the above

GIVEN a space that hasn't been deleted
WHEN I get the deletion job
THEN I get a 404 Not Found

[matt-royal]

This story passed the acceptance criteria.

When the space was deleting but hadn't completed, I got a 200 with this body:

{
  "created_at": "",
  "errors": [],
  "guid": "space.delete~cf-space-a0ad8c07-fd70-4d70-9458-d870578de5e5",
  "links": {
    "self": {
      "href": "https://localhost/v3/jobs/space.delete~cf-space-a0ad8c07-fd70-4d70-9458-d870578de5e5"
    }
  },
  "operation": "space.delete",
  "state": "PROCESSING",
  "updated_at": "",
  "warnings": null
}

When the space finished deleting, I got a 200 with this body:

{
  "created_at": "",
  "errors": [],
  "guid": "space.delete~cf-space-a0ad8c07-fd70-4d70-9458-d870578de5e5",
  "links": {
    "self": {
      "href": "https://localhost/v3/jobs/space.delete~cf-space-a0ad8c07-fd70-4d70-9458-d870578de5e5"
    }
  },
  "operation": "space.delete",
  "state": "COMPLETE",
  "updated_at": "",
  "warnings": null
}

When I added a fake finalizer to the CFSpace so it would never delete, I eventually got a 200 with this body:

{
  "created_at": "",
  "errors": [
    {
      "code": 10008,
      "detail": "Space deletion timed out, check the remaining \"cf-space-76917ca6-3dff-4baa-ad16-6c36549683b6\" resource",
      "title": "CF-UnprocessableEntity"
    }
  ],
  "guid": "space.delete~cf-space-76917ca6-3dff-4baa-ad16-6c36549683b6",
  "links": {
    "self": {
      "href": "https://localhost/v3/jobs/space.delete~cf-space-76917ca6-3dff-4baa-ad16-6c36549683b6"
    }
  },
  "operation": "space.delete",
  "state": "FAILED",
  "updated_at": "",
  "warnings": null
}

When I requested the job status for a non-existent space, I got a 200 with this body:

{
  "created_at": "",
  "errors": [],
  "guid": "space.delete~nosuchspace",
  "links": {
    "self": {
      "href": "https://localhost/v3/jobs/space.delete~nosuchspace"
    }
  },
  "operation": "space.delete",
  "state": "COMPLETE",
  "updated_at": "",
  "warnings": null
}

When querying the status of a space that I don't have access to I received a 200 with this body:

{
  "created_at": "",
  "errors": [],
  "guid": "space.delete~cf-space-34f3090f-5742-40c8-bcaa-adcc6a034b83",
  "links": {
    "self": {
      "href": "https://localhost/v3/jobs/space.delete~cf-space-34f3090f-5742-40c8-bcaa-adcc6a034b83"
    }
  },
  "operation": "space.delete",
  "state": "COMPLETE",
  "updated_at": "",
  "warnings": null
}

When the job doesn't exist, I got a 404 with this body:

{
  "errors": [
    {
      "code": 10010,
      "detail": "Job not found. Ensure it exists and you have access to it.",
      "title": "CF-ResourceNotFound"
    }
  ]
}