Skip to content

Commit 8f1b62f

Browse files
authored
Add Client Quickstart (#102)
* Add Config CLI parser and store config file. * (wip, docs) Quickstart Page * Revert "Add Config CLI parser" In favour of another PR implementing config handling. This reverts commit 8935a44. * (fixup) Spellchecking of quickstart guide
1 parent aec0761 commit 8f1b62f

File tree

1 file changed

+70
-0
lines changed

1 file changed

+70
-0
lines changed

client/docs/quickstart.md

+70
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Jobq, workflow submission and scheduling
2+
3+
Jobq is a tool that aims to make it easy for you to create, submit and schedule workflows.
4+
5+
To get teams shipping fast, we keep the definition of the workflows inside a familiar environment: Python.
6+
You wrap all your logic into a function that you then decorate with the `@job` decorator.
7+
In this decorator you will define the environment and hardware requirements.
8+
9+
You then use the `jobq` CLI to submit the workflow, stop it, or retrieve logs.
10+
11+
The `jobq` backed is built on top of Kubernetes and is installed in the cluster, orchestrates the workflow management, and interfaces to the client CLI calls.
12+
13+
Let us walk through an example.
14+
The example expects a set-up cluster and API server.
15+
16+
## Workflow definition in your Python files
17+
18+
To restructure your Python script into a workflow that can be executed by `jobq` you need to reorganize your logic such that everything is called from one top level function.
19+
Then you decorate this function with the `jobq.job` decorator.
20+
The decorator takes two arguments, `image` and `options`.
21+
You use the `image` argument to submit `ImageOptions`, a wrapper specifying the image, that is the environment your code will execute in.
22+
The `ImageOptions` take a `spec`, that is a path to a `.yaml` or `Dockerfile` which outlines the environment creation, as well as the name and tag of the image.
23+
24+
The other argument to the `job` decorator are `options`.
25+
Here you specify the desired resources using the `ResourceOptions` wrapper, can attach labels and specify the `SchedulingOptions`.
26+
27+
A defined job looks similar to this
28+
```python
29+
# quickstart.py
30+
from pathlib import Path
31+
from jobq import ImageOptions, JobOptions, ResourceOptions, SchedulingOptions, job
32+
33+
@job(
34+
options=JobOptions(
35+
labels={"type": "hello-world@quickstart"},
36+
resources=ResourceOptions(memory='1Gi', cpu='1'),
37+
scheduling=SchedulingOptions(
38+
priority_class='background', queue_name="user-queue"
39+
),
40+
image=ImageOptions(
41+
spec=Path('dockerfile/path/relative/to/job'),
42+
name="quickstart/hello-world",
43+
tag="latest"
44+
)
45+
)
46+
)
47+
def quickstart():
48+
print("Hello, World!")
49+
```
50+
51+
## Interface with your workflows using the `jobq` CLI
52+
Now that we have the job defined let us execute it.
53+
You can execute the job via the CLI,`jobq submit quickstart.py`.
54+
This execution is locally, which may be useful for debugging and testing.
55+
However, you probably want to execute the job on the remote cluster.
56+
In this case you can add the `--mode kueue` flag to submit the workflow to the cluster queue (orchestrated by Kubernetes Kueue) like so:
57+
`jobq submit --mode kueue quickstart.py`.
58+
59+
This returns the job id which we can use to fetch or stream the logs using `jobq logs <job id>`.
60+
61+
## Setting up the `jobq` Backend and API server
62+
In this quickstart guide we use Minikube to run a Kubernetes cluster locally.
63+
In case you want to deploy the `jobq` backend on your own cluster, follow the instructions [here](/link/to/clustersetup).
64+
65+
Follow the [ instructions ](https://minikube.sigs.k8s.io/docs/start/) to install Minikube in the Minikube docs. You will also need a container or virtual machine manager like [Docker](https://www.docker.com/get-started/).
66+
Then, open a terminal and start Minikube with `minikube start`.
67+
68+
Next, you need to ensure that the registry add-on is enabled.
69+
Do so by running `minikube addons enable registry`.
70+
The registry is where the cluster saves the job images.

0 commit comments

Comments
 (0)