MLflow¶
Part of the lakehouse (overview and interaction diagram).
The MLflow template (Data science category) deploys an MLflow tracking server: experiments, runs and model registry in a PostgreSQL database (CloudNativePG) created alongside it, artifacts served by the server on an S3 bucket. Web access is protected by the platform login (the MLflow server has no authentication of its own): as with Apache Hop, allowed users and groups are managed from Access.
| Field | Role |
|---|---|
| MLflow version | Tag of the ds/mlflow image. Empty = the chart's version. |
| Artifacts S3 bucket, Prefix in the bucket (Storage tab, bucket required) | Where artifacts are stored, prefix mlflow by default. Fixed at creation: existing runs point at them. |
| S3 endpoint, S3 region, Access Key ID, Secret Access Key (Storage tab) | Filled in by the bucket picker, editable afterwards — type them in for an external S3. |
| Server processes | Number of server workers, 2 by default. |
| Memory limit, CPU limit (Capacity tab) | 2Gi and 1000m by default. |
| PostgreSQL storage size (Capacity tab) | 5Gi by default. Grow-only: a volume cannot be shrunk. |
From an attached notebook (MLFLOW_TRACKING_URI pre-filled, internal access without login): import mlflow; mlflow.set_tracking_uri(os.environ["MLFLOW_TRACKING_URI"]); mlflow.log_metric("accuracy", 0.99). Artifacts go through the server: notebooks don't need S3 keys.
Deletion¶
Deleting a MLflow instance also removes its OpenID client from the platform.