Lifecycle - Kubernetes mode

The lifecycle of a Kubernetes agent in Docker mode is similar to that of the generic Ponos agent. Only the actions specific to task management differ.

Setup

In addition to the generic Ponos agent setup, the Ponos agent in Docker mode will:

Identification

The ponos agent needs a specific environment variable PONOS_NODE_NAME to identify itself using the executing node’s name passed through downward API.

Loop

During its loop, the ponos agent running on a Kubernetes cluster will:

  • list existing jobs using the batch API,

  • create a new job for each task,

  • check upon the running jobs,

  • report their state onto Arkindex,

  • read job logs and publish them on Arkindex S3 storage,

  • stop jobs

Kubernetes job

Kubernetes Jobs are used to run ponos tasks, using the same Docker images as when the agent runs in docker mode.

Each job created by the ponos agent has these settings:

  • only 1 execution (backoff_limit is set to 1, and no restart is allowed), as Arkindex manages restarts,

  • the same environment variables as for docker mode are provided to the container

  • two labels are set on the job:

    1. ponos-agent with the agent ID to allow multiple agents running on the same cluster,

    2. ponos-task-ir with the poonos task ID to allow identifying the ponos task matching the Kubernetes job,

  • a volume to share data from the ponos agent with the job is always mounted as /data,

  • a volume to expose a cached machine learning model may be mounted as /model when the task requires a model.