Apache NiFi¶
Part of the lakehouse (overview and interaction diagram).
The Apache NiFi template (Processing category) deploys NiFi 2 with platform-account sign-in: whoever deploys the instance automatically gets edit rights on the flow (the initial administrator is a technical platform account, invisible in NiFi) and authorizes other accounts from the NiFi UI (Users / Policies menu). The flow and repositories (FlowFiles, content, provenance, state) live on a persistent volume.
| Field | Role |
|---|---|
| NiFi version | Tag of the apache/nifi image. Empty = the chart's version. |
| Number of nodes (Cluster tab) | 1 by default. NiFi always runs as a cluster (leader election and shared state natively on Kubernetes, Leases/ConfigMap, no ZooKeeper), one pod and one volume per node. Can be changed at any time (see below). |
| JVM heap, Memory limit, CPU limit (Capacity tab) | 2g, 3Gi and 2 cores by default; keep ~1 Gi of headroom between the heap and the memory limit. |
| Volume size (repositories and flow) (Capacity tab) | 20Gi by default, per node. Fixed at creation: resizing an existing volume is not done from this interface. |
| Apache Iceberg (Polaris) catalog (Lakehouse tab, optional) | Sets POLARIS_URI, POLARIS_CREDENTIAL and the S3_* variables in the pod, to be referenced yourself in a flow: NiFi has no native Iceberg processor, nothing is wired automatically. |
Changing the number of nodes¶
From the application's configuration window (Number of nodes field, or Replicas):
- Adding nodes: the existing nodes keep running; each new node joins the cluster and receives its flow.
- Removing nodes: the highest-numbered nodes stop, then the Control Panel removes them from the NiFi cluster (otherwise NiFi would refuse any flow change while a node remains "disconnected"). A removed node keeps its volume: data still queued on that node comes back if it is added again.
- Stop / Start: stops all nodes, then restarts them with the same count.
After a node count change or a restart, NiFi may ask you to log in again: each node signs its own sessions, so a session opened on another node is no longer recognized. The flow and data are not affected. Right after a start, the interface may also show an error while the nodes form the cluster (usually under a minute): refresh the page.
NiFi applications created before version 2.8.70
An older single-node application is migrated automatically on its first update (configuration change or restart): it is stopped, its volume is cloned under the name expected by the new mode, then it restarts with its flow and data. The old volume is then deleted.
End-to-end TLS
NiFi 2 only listens on HTTPS: an internal certificate is generated at deployment and verified by the platform gateway. The first start takes 1 to 3 minutes (extension loading).
Deletion¶
Deleting a Apache NiFi instance also removes its OpenID client from the platform.