Add OpenEBS v4.6.1 with images repointed to ghcr.io/wrktalk-tech - Loki and Alloy disabled
This commit is contained in:
@@ -0,0 +1,419 @@
|
||||
# mayastor
|
||||
|
||||
Mayastor Helm chart for Kubernetes
|
||||
|
||||
  
|
||||
|
||||
## Installation Guide
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Make sure the [system requirement pre-requisites](https://openebs.io/docs/quickstart-guide/prerequisites#replicated-pv-mayastor-prerequisites) are met.
|
||||
- Label the storage nodes same as the mayastor.nodeSelector in values.yaml
|
||||
- Create the namespace you want the chart to be installed, or pass the `--create-namespace` flag in the `helm install` command.
|
||||
```sh
|
||||
kubectl create ns <mayastor-namespace>
|
||||
```
|
||||
- Create secret if downloading the container images from a private repo.
|
||||
```sh
|
||||
kubectl create secret docker-registry <same-as-image.pullSecrets[0]> --docker-server="https://index.docker.io/v1/" --docker-username="<user-name>" --docker-password="<password>" --docker-email="<user-email>" -n <mayastor-namespace>
|
||||
```
|
||||
|
||||
### Installing the chart via the git repo
|
||||
|
||||
Clone the mayastor charts repo.
|
||||
Sync the chart dependencies
|
||||
```console
|
||||
$ helm dependency update
|
||||
```
|
||||
Install the mayastor chart using the command.
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace>
|
||||
```
|
||||
|
||||
### Installing the Chart via Helm Registry
|
||||
|
||||
To install the chart with the release name `mymayastor`:
|
||||
|
||||
```console
|
||||
$ helm repo add mayastor https://openebs.github.io/mayastor-extensions/
|
||||
$ helm install mymayastor mayastor/mayastor
|
||||
```
|
||||
|
||||
### Uninstall Helm Chart
|
||||
|
||||
```console
|
||||
$ helm uninstall [RELEASE_NAME]
|
||||
```
|
||||
|
||||
This removes all the Kubernetes components associated with the chart and deletes the release.
|
||||
|
||||
*See [helm uninstall](https://helm.sh/docs/helm/helm_uninstall/) for command documentation.*
|
||||
|
||||
## TLS Configuration
|
||||
|
||||
TLS is configured at two levels:
|
||||
|
||||
- **`security.tls`** — shared CA/issuer infrastructure (engine, key algorithm, existing issuer). Common to all services (REST, gRPC, …).
|
||||
- **`apis.rest.security.tls`** — REST-specific leaf certificate settings (`mutualAuth`, `certManager.duration`, `existingSecret`).
|
||||
Per-client overrides live under `apis.rest.security.tls.clients.<component>` (`existingSecret`, `certManager.secretName`).
|
||||
|
||||
| Setting | Effect |
|
||||
|---------|--------|
|
||||
| `security.tls.enabled: false` | No TLS. All services use plain-text. |
|
||||
| `security.tls.autoGenerated.engine: pod` (default) | Server generates a transient cert at startup (`--auto-tls`). No Secrets created. Cannot be combined with `mutualAuth`. |
|
||||
| `security.tls.autoGenerated.engine: helm` | Chart generates self-signed certificates stored in k8s Secrets. Clients verify the server cert. |
|
||||
| `security.tls.autoGenerated.engine: certManager` | cert-manager provisions and rotates certificates. cert-manager must be installed. |
|
||||
| `+ apis.rest.security.tls.mutualAuth: true` | Full mutual TLS (mTLS). Both sides verify. Separate client certs are provisioned for each component (csi-controller, csi-node, obs-callhome, io-engine, operator-diskpool, plugin). Requires `engine: helm` or `engine: certManager`. |
|
||||
|
||||
The `kubectl` plugin (`kubectl mayastor`) automatically discovers the TLS mode from a pod
|
||||
annotation (`openebs.io/rest-tls`) that the chart adds to the `api-rest` pod.
|
||||
|
||||
### No TLS (plain HTTP)
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.enabled=false
|
||||
```
|
||||
|
||||
### HTTPS with ephemeral pod-generated certificate
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.autoGenerated.engine=pod \
|
||||
--set apis.rest.security.tls.mutualAuth=true
|
||||
```
|
||||
|
||||
The server generates a transient certificate at startup. Clients connect to HTTPS without
|
||||
certificate verification. No Secrets are created.
|
||||
|
||||
### Mutual TLS — self-signed certificates
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.autoGenerated.engine=helm \
|
||||
--set apis.rest.security.tls.mutualAuth=true
|
||||
```
|
||||
|
||||
The chart generates a shared CA, a server certificate, and separate client certificates for each
|
||||
component (csi-controller, csi-node, obs-callhome, io-engine, operator-diskpool, plugin). Each
|
||||
component mounts only its own keypair.
|
||||
|
||||
### Mutual TLS — cert-manager
|
||||
|
||||
Install [cert-manager](https://cert-manager.io/docs/installation/) first, then:
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.autoGenerated.engine=certManager \
|
||||
--set apis.rest.security.tls.mutualAuth=true
|
||||
```
|
||||
|
||||
To use your own Issuer or ClusterIssuer:
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.autoGenerated.engine=certManager \
|
||||
--set security.tls.autoGenerated.certManager.existingIssuer=<your-issuer-name> \
|
||||
--set security.tls.autoGenerated.certManager.existingIssuerKind=ClusterIssuer
|
||||
```
|
||||
|
||||
The chart creates a shared CA `Certificate`, a CA `Issuer`, a server `Certificate`, and separate
|
||||
client `Certificate` resources for each component (csi-controller, csi-node, obs-callhome,
|
||||
io-engine, operator-diskpool, plugin). Pass
|
||||
`security.tls.autoGenerated.certManager.existingIssuer` to use your own CA Issuer.
|
||||
|
||||
### Bring your own certificates
|
||||
|
||||
Set `security.tls.autoGenerated.enabled=false` and supply pre-created Secrets:
|
||||
|
||||
```console
|
||||
$ helm install mayastor . -n <mayastor-namespace> \
|
||||
--set security.tls.enabled=true \
|
||||
--set security.tls.autoGenerated.enabled=false \
|
||||
--set apis.rest.security.tls.existingSecret=<server-tls-secret>
|
||||
```
|
||||
|
||||
With mutual TLS, also supply a secret for each client:
|
||||
|
||||
```console
|
||||
--set apis.rest.security.tls.clients.csiController.existingSecret=<secret> \
|
||||
--set apis.rest.security.tls.clients.csiNode.existingSecret=<secret> \
|
||||
--set apis.rest.security.tls.clients.callhome.existingSecret=<secret> \
|
||||
--set apis.rest.security.tls.clients.metricsExporter.existingSecret=<secret> \
|
||||
--set apis.rest.security.tls.clients.diskpoolOperator.existingSecret=<secret> \
|
||||
--set apis.rest.security.tls.clients.plugin.existingSecret=<secret>
|
||||
```
|
||||
|
||||
Each Secret must contain `tls.crt`, `tls.key`, and `ca.crt` keys.
|
||||
|
||||
## Chart Dependencies
|
||||
|
||||
| Repository | Name | Version |
|
||||
|------------|------|---------|
|
||||
| | crds | 2.12.1 |
|
||||
| https://charts.bitnami.com/bitnami | etcd | 12.0.14 |
|
||||
| https://grafana.github.io/helm-charts | alloy | 1.0.1 |
|
||||
| https://grafana.github.io/helm-charts | loki | 6.29.0 |
|
||||
| https://jaegertracing.github.io/helm-charts | jaeger-operator | 2.50.1 |
|
||||
| https://nats-io.github.io/k8s/helm/charts/ | nats | 0.19.14 |
|
||||
| https://openebs.github.io/dynamic-localpv-provisioner | localpv-provisioner | 4.6.0 |
|
||||
|
||||
## Values
|
||||
|
||||
| Key | Description | Default |
|
||||
|:----|:------------|:--------|
|
||||
| agents.​core.​allowNonPersistentDevlink | Allow using non-persistent kernel devpaths for pool disks. Enabling this will let users to use the kernel devpaths e.g /dev/sda, for diskpools. However, this comes with associated risks if the devpaths get swapped among disks, resulting in total data loss especially if encryption is being used. | `false` |
|
||||
| agents.​core.​capacity.​thin.​poolCommitment | The allowed pool commitment limit when dealing with thin provisioned volumes. Example: If the commitment is 250 and the pool is 10GiB we can overcommit the pool up to 25GiB (create 2 10GiB and 1 5GiB volume) but no further. | `"250%"` |
|
||||
| agents.​core.​capacity.​thin.​snapshotCommitment | When creating snapshots for an existing volume, each replica pool must have at least this much free space percentage of the volume size. Example: if this value is 40, the pool has 40GiB free, then the max volume size allowed to be snapped on the pool is 100GiB. | `"40%"` |
|
||||
| agents.​core.​capacity.​thin.​volumeCommitment | When creating replicas for an existing volume, each replica pool must have at least this much free space percentage of the volume size. Example: if this value is 40, the pool has 40GiB free, then the max volume size allowed to be created on the pool is 100GiB. | `"40%"` |
|
||||
| agents.​core.​capacity.​thin.​volumeCommitmentInitial | Same as the `volumeCommitment` argument, but applicable only when creating replicas for a new volume. | `"40%"` |
|
||||
| agents.​core.​encryptedPoolsSoftScheduling | Prefer encrypted pools for volume replicas. If a volume wasn't provisioned with a encryption storageclass, we try to place the replicas of such volume on best-effort basis onto encrypted pools, if this global is set. This is effective subject to volume spec already modified via plugin to request encryption. | `false` |
|
||||
| agents.​core.​logLevel | Log level for the core service | `"info"` |
|
||||
| agents.​core.​minTimeouts | Enable minimal timeouts | `true` |
|
||||
| agents.​core.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| agents.​core.​poolClusterSize | Default blobstore cluster size for diskpools, in bytes. This value is used as a default value of blobstore cluster size on diskpools. This is set to 4MiB internally by default, if nothing specified here. The value is also configurable via Diskpool CR, which takes precedence over this setting. This is an advanced configuration, please refer documentation to understand the usage and implications of this. | `""` |
|
||||
| agents.​core.​priorityClassName | Set PriorityClass, overrides global. If both local and global are not set, the final deployment manifest has a mayastor custom critical priority class assigned to the pod by default. Refer the `templates/_helpers.tpl` and `templates/mayastor/agents/core/agent-core-deployment.yaml` for more details. | `""` |
|
||||
| agents.​core.​rebuild.​maxConcurrent | The maximum number of system-wide rebuilds permitted at any given time. If set to an empty string, there are no limits. | `""` |
|
||||
| agents.​core.​rebuild.​partial.​enabled | Partial rebuild uses a log of missed IO to rebuild replicas which have become temporarily faulted, hence a bit faster, depending on the log size. | `true` |
|
||||
| agents.​core.​rebuild.​partial.​waitPeriod | If a faulted replica comes back online within this time period then it will be rebuilt using the partial rebuild capability. Otherwise, the replica will be fully rebuilt. A blank value "" means internally derived value will be used. | `""` |
|
||||
| agents.​core.​requestTimeout | Request timeout for core agents Default value is defined in .base.default_req_timeout | `nil` |
|
||||
| agents.​core.​resources.​limits.​cpu | Cpu limits for core agents | `"1000m"` |
|
||||
| agents.​core.​resources.​limits.​memory | Memory limits for core agents | `"128Mi"` |
|
||||
| agents.​core.​resources.​requests.​cpu | Cpu requests for core agents | `"500m"` |
|
||||
| agents.​core.​resources.​requests.​memory | Memory requests for core agents | `"32Mi"` |
|
||||
| agents.​core.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| agents.​core.​volumeHealth | Enable extended volume health information, which helps generate the volume status more accurately. | `true` |
|
||||
| agents.​ha.​cluster.​logLevel | Log level for the ha cluster service | `"info"` |
|
||||
| agents.​ha.​cluster.​resources.​limits.​cpu | Cpu limits for ha cluster agent | `"100m"` |
|
||||
| agents.​ha.​cluster.​resources.​limits.​memory | Memory limits for ha cluster agent | `"64Mi"` |
|
||||
| agents.​ha.​cluster.​resources.​requests.​cpu | Cpu requests for ha cluster agent | `"100m"` |
|
||||
| agents.​ha.​cluster.​resources.​requests.​memory | Memory requests for ha cluster agent | `"16Mi"` |
|
||||
| agents.​ha.​node.​logLevel | Log level for the ha node service | `"info"` |
|
||||
| agents.​ha.​node.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| agents.​ha.​node.​port | Container port for the ha-node service | `50053` |
|
||||
| agents.​ha.​node.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| agents.​ha.​node.​resources.​limits.​cpu | Cpu limits for ha node agent | `"100m"` |
|
||||
| agents.​ha.​node.​resources.​limits.​memory | Memory limits for ha node agent | `"64Mi"` |
|
||||
| agents.​ha.​node.​resources.​requests.​cpu | Cpu requests for ha node agent | `"100m"` |
|
||||
| agents.​ha.​node.​resources.​requests.​memory | Memory requests for ha node agent | `"64Mi"` |
|
||||
| agents.​ha.​node.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| alloy.​logging_config.​labels | Labels to enable scraping on, at-least one of these labels should be present. | <pre>{<br>"openebs.io/logging":true<br>}</pre> |
|
||||
| alloy.​logging_config.​tenant_id | X-Scope-OrgID to pe populated which pushing logs. Make sure the caller also uses the same. | `"openebs"` |
|
||||
| apis.​rest.​healthProbes.​liveness.​enabled | Toggle liveness probe. | `true` |
|
||||
| apis.​rest.​healthProbes.​liveness.​failureThreshold | No. of failures the liveness probe will tolerate. | `3` |
|
||||
| apis.​rest.​healthProbes.​liveness.​initialDelaySeconds | No. of seconds of delay before checking the liveness status. | `1` |
|
||||
| apis.​rest.​healthProbes.​liveness.​periodSeconds | No. of seconds between liveness probe checks. | `30` |
|
||||
| apis.​rest.​healthProbes.​liveness.​timeoutSeconds | No. of seconds of timeout tolerance. | `5` |
|
||||
| apis.​rest.​healthProbes.​readiness.​agentCoreProbeFreq | Frequency for the agent-core liveness probe. | `"20s"` |
|
||||
| apis.​rest.​healthProbes.​readiness.​enabled | Toggle readiness probe. | `true` |
|
||||
| apis.​rest.​healthProbes.​readiness.​failureThreshold | No. of failures the readiness probe will tolerate. | `3` |
|
||||
| apis.​rest.​healthProbes.​readiness.​initialDelaySeconds | No. of seconds of delay before checking the readiness status. | `1` |
|
||||
| apis.​rest.​healthProbes.​readiness.​periodSeconds | No. of seconds between readiness probe checks. | `20` |
|
||||
| apis.​rest.​healthProbes.​readiness.​timeoutSeconds | No. of seconds of timeout tolerance. | `5` |
|
||||
| apis.​rest.​logLevel | Log level for the rest service | `"info"` |
|
||||
| apis.​rest.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| apis.​rest.​priorityClassName | Set PriorityClass, overrides global. If both local and global are not set, the final deployment manifest has a mayastor custom critical priority class assigned to the pod by default. Refer the `templates/_helpers.tpl` and `templates/mayastor/apis/rest/api-rest-deployment.yaml` for more details. | `""` |
|
||||
| apis.​rest.​replicaCount | Number of replicas of rest | `1` |
|
||||
| apis.​rest.​resources.​limits.​cpu | Cpu limits for rest | `"100m"` |
|
||||
| apis.​rest.​resources.​limits.​memory | Memory limits for rest | `"64Mi"` |
|
||||
| apis.​rest.​resources.​requests.​cpu | Cpu requests for rest | `"50m"` |
|
||||
| apis.​rest.​resources.​requests.​memory | Memory requests for rest | `"32Mi"` |
|
||||
| apis.​rest.​security.​tls.​certManager.​secretName | Secret name for the REST API server TLS certificate. Defaults to {release}-api-rest-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients | Per-client TLS overrides. Each client can point at a pre-existing secret (when autoGenerated.enabled is false) or customise the cert-manager secret name. | <pre>{<br>"callhome":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>},<br>"csiController":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>},<br>"csiNode":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>},<br>"diskpoolOperator":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>},<br>"metricsExporter":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>},<br>"plugin":{<br>"certManager":{<br>"secretName":""<br>},<br>"existingSecret":""<br>}<br>}</pre> |
|
||||
| apis.​rest.​security.​tls.​clients.​callhome.​certManager.​secretName | cert-manager secret name for the callhome client cert. Defaults to {release}-api-rest-callhome-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​callhome.​existingSecret | Pre-existing Secret for the callhome client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​csiController.​certManager.​secretName | cert-manager secret name for the CSI controller client cert. Defaults to {release}-api-rest-csi-controller-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​csiController.​existingSecret | Pre-existing Secret for the CSI controller client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​csiNode.​certManager.​secretName | cert-manager secret name for the CSI node client cert. Defaults to {release}-api-rest-csi-node-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​csiNode.​existingSecret | Pre-existing Secret for the CSI node client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​diskpoolOperator.​certManager.​secretName | cert-manager secret name for the diskpool-operator client cert. Defaults to {release}-api-rest-diskpool-operator-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​diskpoolOperator.​existingSecret | Pre-existing Secret for the diskpool-operator client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​metricsExporter.​certManager.​secretName | cert-manager secret name for the metrics-exporter client cert. Defaults to {release}-api-rest-metrics-exporter-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​metricsExporter.​existingSecret | Pre-existing Secret for the metrics-exporter client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​plugin.​certManager.​secretName | cert-manager secret name for the kubectl plugin client cert. Defaults to {release}-api-rest-plugin-crt. | `""` |
|
||||
| apis.​rest.​security.​tls.​clients.​plugin.​existingSecret | Pre-existing Secret for the kubectl plugin client cert. Required when autoGenerated.enabled is false and mutualAuth is true. | `""` |
|
||||
| apis.​rest.​security.​tls.​existingSecret | Pre-existing TLS Secret to use when tls.autoGenerated.enabled is false. Must contain tls.crt, tls.key, and ca.crt. The chart mounts it but does not manage it. | `""` |
|
||||
| apis.​rest.​service.​type | Rest K8s service type | `"ClusterIP"` |
|
||||
| apis.​rest.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| base.​cache_poll_period | Cache timeout for core agent & diskpool deployment | `"30s"` |
|
||||
| base.​default_req_timeout | Request timeout for rest & core agents | `"5s"` |
|
||||
| base.​initContainers.​image.​registry | Image registry for init containers | `""` |
|
||||
| base.​logging.​color | Enable ansi color code for Pod StdOut/StdErr | `true` |
|
||||
| base.​logging.​format | Valid values for format are pretty, json and compact | `"pretty"` |
|
||||
| base.​logging.​silenceLevel | Silence specific module components | `nil` |
|
||||
| base.​metrics.​enabled | Enable the metrics exporter | `true` |
|
||||
| base.​metrics.​port | Container port for the metrics exporter service | `9502` |
|
||||
| crds.​csi.​volumeSnapshots.​enabled | Install Volume Snapshot CRDs | `true` |
|
||||
| crds.​enabled | Disables the installation of all CRDs if set to false | `true` |
|
||||
| csi.​controller.​logLevel | Log level for the csi controller | `"info"` |
|
||||
| csi.​controller.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| csi.​controller.​preventVolumeModeConversion | Prevent modifying the volume mode when creating a PVC from an existing VolumeSnapshot | `true` |
|
||||
| csi.​controller.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| csi.​controller.​resources.​limits.​cpu | Cpu limits for csi controller | `"32m"` |
|
||||
| csi.​controller.​resources.​limits.​memory | Memory limits for csi controller | `"128Mi"` |
|
||||
| csi.​controller.​resources.​requests.​cpu | Cpu requests for csi controller | `"16m"` |
|
||||
| csi.​controller.​resources.​requests.​memory | Memory requests for csi controller | `"64Mi"` |
|
||||
| csi.​controller.​snapshotController.​enabled | Run the csi-snapshot-controller container. Disable this if the cluster already runs one. | `true` |
|
||||
| csi.​controller.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| csi.​image.​attacherTag | csi-attacher image release tag | `"v4.8.1"` |
|
||||
| csi.​image.​provisionerTag | csi-provisioner image release tag | `"v5.2.0"` |
|
||||
| csi.​image.​pullPolicy | imagePullPolicy for all CSI Sidecar images | `"IfNotPresent"` |
|
||||
| csi.​image.​registrarTag | csi-node-driver-registrar image release tag | `"v2.13.0"` |
|
||||
| csi.​image.​registry | Image registry to pull all CSI Sidecar images | `"registry.k8s.io"` |
|
||||
| csi.​image.​repo | Image registry's namespace | `"sig-storage"` |
|
||||
| csi.​image.​resizerTag | csi-resizer image release tag | `"v1.13.2"` |
|
||||
| csi.​image.​snapshotControllerTag | csi-snapshot-controller image release tag | `"v8.2.0"` |
|
||||
| csi.​image.​snapshotterTag | csi-snapshotter image release tag | `"v8.2.0"` |
|
||||
| csi.​node.​kubeletDir | The kubeletDir directory for the csi-node plugin | `"/var/lib/kubelet"` |
|
||||
| csi.​node.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| csi.​node.​nvme.​ctrl_loss_tmo | The ctrl_loss_tmo (controller loss timeout) in seconds | `"1980"` |
|
||||
| csi.​node.​nvme.​tcpFallback | Fallback to nvme-tcp if nvme-rdma is enabled for Mayastor but rdma is not available on a particular csi-node | `true` |
|
||||
| csi.​node.​port | Container port for the csi-node service | `10199` |
|
||||
| csi.​node.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| csi.​node.​resources.​limits.​cpu | Cpu limits for csi node plugin | `"100m"` |
|
||||
| csi.​node.​resources.​limits.​memory | Memory limits for csi node plugin | `"128Mi"` |
|
||||
| csi.​node.​resources.​requests.​cpu | Cpu requests for csi node plugin | `"100m"` |
|
||||
| csi.​node.​resources.​requests.​memory | Memory requests for csi node plugin | `"64Mi"` |
|
||||
| csi.​node.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| csi.​node.​topology.​nodeSelector | Add topology segments to the csi-node and agent-ha-node daemonset node selector | `false` |
|
||||
| etcd.​autoCompactionMode | AutoCompaction Since etcd keeps an exact history of its keyspace, this history should be periodically compacted to avoid performance degradation and eventual storage space exhaustion. Auto compaction mode. Valid values: "periodic", "revision". - 'periodic' for duration based retention, defaulting to hours if no time unit is provided (e.g. 5m). - 'revision' for revision number based retention. | `"revision"` |
|
||||
| etcd.​autoCompactionRetention | Auto compaction retention length. 0 means disable auto compaction. | `"100"` |
|
||||
| etcd.​clusterDomain | Kubernetes Cluster Domain | `"cluster.local"` |
|
||||
| etcd.​enabled | Disable when using an external etcd cluster. | `true` |
|
||||
| etcd.​externalUrl | Url of the external etcd cluster. Note, etcd.enable must be set to false. | `""` |
|
||||
| etcd.​extraEnvVars[0] | Raise alarms when backend size exceeds the given quota. | <pre>{<br>"name":"ETCD_QUOTA_BACKEND_BYTES",<br>"value":"8589934592"<br>}</pre> |
|
||||
| etcd.​localpvScConfig.​basePath | Host path where local etcd data is stored in. | `"/var/local/{{ .Release.Name }}/localpv-hostpath/etcd"` |
|
||||
| etcd.​localpvScConfig.​reclaimPolicy | ReclaimPolicy of etcd's localpv hostpath storage class. | `"Delete"` |
|
||||
| etcd.​localpvScConfig.​volumeBindingMode | VolumeBindingMode of etcd's localpv hostpath storage class. | `"WaitForFirstConsumer"` |
|
||||
| etcd.​metrics.​enabled | Expose etcd metrics. | `true` |
|
||||
| etcd.​metrics.​useSeparateEndpoint | Use a separate endpoint for exposing metrics, override the default port (9090) by setting containerPorts.metrics. | `true` |
|
||||
| etcd.​persistence.​enabled | If true, use a Persistent Volume Claim. If false, use emptyDir. | `true` |
|
||||
| etcd.​persistence.​size | Volume size | `"2Gi"` |
|
||||
| etcd.​persistence.​storageClass | Will define which storageClass to use in etcd's StatefulSets. Options: <p> - `"manual"` - Will provision a hostpath PV on the same node. <br> - `""` (empty) - Will use the default StorageClass on the cluster. </p> | `"mayastor-etcd-localpv"` |
|
||||
| etcd.​persistentVolumeClaimRetentionPolicy.​enabled | PVC's reclaimPolicy | `false` |
|
||||
| etcd.​podAntiAffinityPreset | Pod anti-affinity preset Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity | `"hard"` |
|
||||
| etcd.​removeMemberOnContainerTermination | Use a PreStop hook to remove the etcd members from the etcd cluster on container termination Ignored if lifecycleHooks is set or replicaCount=1 | `false` |
|
||||
| etcd.​replicaCount | Number of replicas of etcd | `3` |
|
||||
| eventing.​aggregator.​dirSizeLimit | Maximum total size of event files including rotated history. Used when loki.enabled is false. | `"100Mi"` |
|
||||
| eventing.​aggregator.​enabled | Enable the eventing-aggregator deployment. Requires eventing.enabled to be true. | `true` |
|
||||
| eventing.​aggregator.​logLevel | Log level for eventing-aggregator | `"info"` |
|
||||
| eventing.​aggregator.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| eventing.​aggregator.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| eventing.​aggregator.​resources.​limits.​cpu | Cpu limits for eventing-aggregator | `"100m"` |
|
||||
| eventing.​aggregator.​resources.​limits.​memory | Memory limits for eventing-aggregator | `"32Mi"` |
|
||||
| eventing.​aggregator.​resources.​requests.​cpu | Cpu requests for eventing-aggregator | `"50m"` |
|
||||
| eventing.​aggregator.​resources.​requests.​memory | Memory requests for eventing-aggregator | `"16Mi"` |
|
||||
| eventing.​aggregator.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| global.​analytics.​enabled | Global overide for call home | `nil` |
|
||||
| global.​imagePullPolicy | Global overide for image pull policy | `""` |
|
||||
| global.​imagePullSecrets | Global override for image pull secrets - secret | `[]` |
|
||||
| global.​imageRegistry | Global override for image registry | `""` |
|
||||
| image.​pullPolicy | ImagePullPolicy for our images | `"Always"` |
|
||||
| image.​pullSecrets | docker-secrets required to pull images if the container registry from image.registry is protected | `[]` |
|
||||
| image.​registry | Image registry to pull our product images | `"docker.io"` |
|
||||
| image.​repo | Image registry's namespace | `"openebs"` |
|
||||
| image.​tag | Release tag for our images | `"release-2.12"` |
|
||||
| io_engine.​coreList | If not empty, overrides the cpuCount and explicitly sets the list of cores. Example: --set='io_engine.coreList={30,31}' | `[]` |
|
||||
| io_engine.​cpuCount | The number of cores that each io-engine instance will bind to. | `"2"` |
|
||||
| io_engine.​envcontext | Pass additional arguments to the Environment Abstraction Layer. Example: --set {product}.envcontext=iova-mode=pa | `""` |
|
||||
| io_engine.​interruptMode | SPDK interrupt mode for io-engine reactors. When enabled, reactors sleep on epoll/timerfd instead of busy-polling, which dramatically reduces idle CPU usage. NVMe I/O queues are still polled, but on a periodic timer rather than continuously. | <pre>{<br>"enabled":false,<br>"nvmeIoQueuePollPeriod":"100us"<br>}</pre> |
|
||||
| io_engine.​interruptMode.​enabled | Enable interrupt mode by setting ENABLE_INTERRUPT_MODE=true on the io-engine container (equivalent to passing the --enable-interrupt-mode CLI flag). | `false` |
|
||||
| io_engine.​interruptMode.​nvmeIoQueuePollPeriod | NVMe I/O queue poll period (SPDK NVME_IOQ_POLL_PERIOD). A value of "0" disables timed polling (busy poll). Typical values: "100us", "1000us". Higher values reduce CPU further at the cost of latency. | `"100us"` |
|
||||
| io_engine.​logLevel | Log level for the io-engine service | `"info"` |
|
||||
| io_engine.​nodeSelector | Node selectors to designate storage nodes for diskpool creation Note that if multi-arch images support 'kubernetes.io/arch: amd64' should be removed. | <pre>{<br>"kubernetes.io/arch":"amd64",<br>"openebs.io/engine":"mayastor"<br>}</pre> |
|
||||
| io_engine.​nvme.​ioTimeout | Timeout for IOs The default here is exaggerated for local disks, but we've observed that in shared virtual environments having a higher timeout value is beneficial. Please adjust this according to your hardware and needs. | `"110s"` |
|
||||
| io_engine.​nvme.​rdma.​bufCacheSize | The number of shared buffers to reserve for each poll group | `nil` |
|
||||
| io_engine.​nvme.​rdma.​dataWrPoolSize | RDMA data WR pool size (RDMA only) | `"4095"` |
|
||||
| io_engine.​nvme.​rdma.​inCapsuleDataSize | The max amount of payload data that can be transferred directly within the NVMe-oF Capsule command itself | `nil` |
|
||||
| io_engine.​nvme.​rdma.​ioUnitSize | I/O unit size (bytes) | `"8192"` |
|
||||
| io_engine.​nvme.​rdma.​maxIoSize | Max I/O size (bytes) | `nil` |
|
||||
| io_engine.​nvme.​rdma.​numSharedBuf | The number of pooled data buffers available to the transport | `nil` |
|
||||
| io_engine.​nvme.​tcp.​bufCacheSize | The number of shared buffers to reserve for each poll group | `"64"` |
|
||||
| io_engine.​nvme.​tcp.​inCapsuleDataSize | The max amount of payload data that can be transferred directly within the NVMe-oF Capsule command itself | `"4096"` |
|
||||
| io_engine.​nvme.​tcp.​ioUnitSize | I/O unit size (bytes) | `"131072"` |
|
||||
| io_engine.​nvme.​tcp.​maxIoSize | Max I/O size (bytes) | `"131072"` |
|
||||
| io_engine.​nvme.​tcp.​maxQpairsPerCtrl | Max number of IO qpairs per controller | `"32"` |
|
||||
| io_engine.​nvme.​tcp.​maxQueueDepth | You may need to increase this for a higher outstanding IOs per volume | `"32"` |
|
||||
| io_engine.​nvme.​tcp.​numSharedBuf | The number of pooled data buffers available to the transport | `"2047"` |
|
||||
| io_engine.​nvme.​transportTos | NVMe Transport Type of Service (ToS) value for RDMA QoS/DSCP marking. When using NVMe-oF over RDMA (RoCEv2), set this to mark target (responder) side RDMA traffic with a DSCP value (e.g. 104 for DSCP 26 / AF31) so that switches and NICs can classify storage traffic into a Priority Flow Control (PFC) enabled queue for lossless transport. A value of 0 (the default) means no marking (best-effort QoS). | `""` |
|
||||
| io_engine.​pool.​diskHandleRescan.​enabled | Periodic rescan of the diskpool backend storage file handles. This is used for hot-remove detection without ongoing I/O, as well as updating the disk size | `true` |
|
||||
| io_engine.​pool.​ioAlerts.​errorThreshold | After this many errors a pool alert is raised as Warning. | `64` |
|
||||
| io_engine.​pool.​ioAlerts.​stallDeadline | If an I/O is stuck longer than this period, then the pool is considered stalled and a Critical alert is raised. The pool disk will also be reset and the stall will be cleared once complete and I/O flows again. default: .Values.io_engine.nvme.ioTimeout * 2 | `nil` |
|
||||
| io_engine.​pool.​ioAlerts.​stallTransitionThreshold | After this many transitions within the stallTransitionWindow, a pool alert is raised as Warning. | `3` |
|
||||
| io_engine.​pool.​ioAlerts.​stallTransitionWindow | Time window during which stall ↔ resume state transitions are tracked for flakiness detection. | `"3h"` |
|
||||
| io_engine.​port | Container port for the io-engine service | `10124` |
|
||||
| io_engine.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| io_engine.​pstorRetries | Number of retries for pstor persistence before the volume target self shutdowns | `300` |
|
||||
| io_engine.​resources.​limits.​cpu | Cpu limits for the io-engine | `""` |
|
||||
| io_engine.​resources.​limits.​hugepages1Gi | Hugepage memory in 1GiB chunks | `nil` |
|
||||
| io_engine.​resources.​limits.​hugepages2Mi | Hugepage memory in 2MiB chunks | `"2Gi"` |
|
||||
| io_engine.​resources.​limits.​memory | Memory limits for the io-engine | `"1Gi"` |
|
||||
| io_engine.​resources.​requests.​cpu | Cpu requests for the io-engine | `""` |
|
||||
| io_engine.​resources.​requests.​hugepages1Gi | Hugepage memory in 1GiB chunks | `nil` |
|
||||
| io_engine.​resources.​requests.​hugepages2Mi | Hugepage memory in 2MiB chunks | `"2Gi"` |
|
||||
| io_engine.​resources.​requests.​memory | Memory requests for the io-engine | `"1Gi"` |
|
||||
| io_engine.​runtimeClassName | Runtime class to use. Defaults to cluster standard | `""` |
|
||||
| io_engine.​target.​nvmf.​iface | NVMF target interface (ip, mac, name or subnet) If RDMA is enabled, please set iface to an RDMA capable netdev name from host network. Example, if an rdma device mlx5_0 is available on a netdev eth0 on RNIC, as can be seen from `rdma link` command output, then this field should be set to eth0. | `""` |
|
||||
| io_engine.​target.​nvmf.​maxNamespaces | Maximum number of NVMe namespaces which a given io-engine node can expose. As of today, there's a 1-1 mapping of namespaces to volume targets. | `4096` |
|
||||
| io_engine.​target.​nvmf.​ptpl | Reservations Persist Through Power Loss State | `true` |
|
||||
| io_engine.​target.​nvmf.​rdma | Enable RDMA Capability of Mayastor nvmf target to take RDMA connections if the cluster nodes have RDMA device(s) configured from RNIC. | <pre>{<br>"enabled":false<br>}</pre> |
|
||||
| io_engine.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| localpv-provisioner.​enabled | Enables the openebs dynamic-localpv-provisioner. If disabled, modify etcd and loki storage class accordingly. | `true` |
|
||||
| localpv-provisioner.​hostpathClass.​enabled | Enable default hostpath localpv StorageClass. | `false` |
|
||||
| localpv-provisioner.​localpv.​priorityClassName | Set the PriorityClass for the LocalPV Hostpath provisioner Deployment. | `"{{ .Release.Name }}-cluster-critical"` |
|
||||
| loki.​localpvScConfig.​loki.​basePath | Host path where local loki data is stored in. | `"/var/local/{{ .Release.Name }}/localpv-hostpath/loki"` |
|
||||
| loki.​localpvScConfig.​loki.​reclaimPolicy | ReclaimPolicy of loki's localpv hostpath storage class. | `"Delete"` |
|
||||
| loki.​localpvScConfig.​loki.​volumeBindingMode | VolumeBindingMode of loki's localpv hostpath storage class. | `"WaitForFirstConsumer"` |
|
||||
| loki.​localpvScConfig.​minio.​basePath | Host path where local minio data is stored in. | `"/var/local/{{ .Release.Name }}/localpv-hostpath/minio"` |
|
||||
| loki.​localpvScConfig.​minio.​reclaimPolicy | ReclaimPolicy of minio's localpv hostpath storage class. | `"Delete"` |
|
||||
| loki.​localpvScConfig.​minio.​volumeBindingMode | VolumeBindingMode of minio's localpv hostpath storage class. | `"WaitForFirstConsumer"` |
|
||||
| nodeSelector | Node labels for pod assignment ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ Note that if multi-arch images support 'kubernetes.io/arch: amd64' should be removed and set 'nodeSelector' to empty '{}' as default value. | <pre>{<br>"kubernetes.io/arch":"amd64"<br>}</pre> |
|
||||
| obs.​callhome.​enabled | Enable callhome | `true` |
|
||||
| obs.​callhome.​logLevel | Log level for callhome | `"info"` |
|
||||
| obs.​callhome.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| obs.​callhome.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| obs.​callhome.​resources.​limits.​cpu | Cpu limits for callhome | `"100m"` |
|
||||
| obs.​callhome.​resources.​limits.​memory | Memory limits for callhome | `"32Mi"` |
|
||||
| obs.​callhome.​resources.​requests.​cpu | Cpu requests for callhome | `"50m"` |
|
||||
| obs.​callhome.​resources.​requests.​memory | Memory requests for callhome | `"16Mi"` |
|
||||
| obs.​callhome.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| obs.​stats.​logLevel | Log level for stats | `"info"` |
|
||||
| obs.​stats.​resources.​limits.​cpu | Cpu limits for stats | `"100m"` |
|
||||
| obs.​stats.​resources.​limits.​memory | Memory limits for stats | `"32Mi"` |
|
||||
| obs.​stats.​resources.​requests.​cpu | Cpu requests for stats | `"50m"` |
|
||||
| obs.​stats.​resources.​requests.​memory | Memory requests for stats | `"16Mi"` |
|
||||
| obs.​stats.​service.​type | Rest K8s service type | `"ClusterIP"` |
|
||||
| operators.​pool.​logLevel | Log level for diskpool operator service | `"info"` |
|
||||
| operators.​pool.​nodeSelector | Set nodeSelector, overrides global | <pre>{<br><br>}</pre> |
|
||||
| operators.​pool.​priorityClassName | Set PriorityClass, overrides global | `""` |
|
||||
| operators.​pool.​resources.​limits.​cpu | Cpu limits for diskpool operator | `"100m"` |
|
||||
| operators.​pool.​resources.​limits.​memory | Memory limits for diskpool operator | `"32Mi"` |
|
||||
| operators.​pool.​resources.​requests.​cpu | Cpu requests for diskpool operator | `"50m"` |
|
||||
| operators.​pool.​resources.​requests.​memory | Memory requests for diskpool operator | `"16Mi"` |
|
||||
| operators.​pool.​tolerations | Set tolerations, overrides global | `[]` |
|
||||
| preUpgradeHook.​enabled | Enable/Disable mayastor pre-upgrade hook | `true` |
|
||||
| preUpgradeHook.​image.​pullPolicy | The imagePullPolicy for the container | `"IfNotPresent"` |
|
||||
| preUpgradeHook.​image.​registry | The container image registry URL for the hook job | `"docker.io"` |
|
||||
| preUpgradeHook.​image.​repo | The container repository for the hook job | `"openebs/kubectl"` |
|
||||
| preUpgradeHook.​image.​tag | The container image tag for the hook job | `"1.25.15"` |
|
||||
| preUpgradeHook.​imagePullSecrets | Optional array of imagePullSecrets containing private registry credentials # Ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ | `[]` |
|
||||
| preUpgradeHook.​rolloutTimeout | Set how long we should wait for the Etcd cluster to finish rolling out before giving up. | `"600s"` |
|
||||
| preUpgradeHook.​tolerations | Node tolerations for server scheduling to nodes with taints # Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ # | `[]` |
|
||||
| priorityClassName | Pod scheduling priority. Setting this value will apply to all components except the external Chart dependencies. If any component has `priorityClassName` set, then this value would be overridden for that component. For external components like etcd, jaeger or loki, PriorityClass can only be set at component level. | `""` |
|
||||
| security.​networkPolicy.​enabled | When enabled, the NetworkPolicy will block all HTTP traffic to the REST API service. | `true` |
|
||||
| security.​tls | TLS configuration shared across all service endpoints. The CA, issuer, engine, and per-certificate defaults are cluster-scoped infrastructure common to all services (REST, gRPC, …). Individual services may override leaf-cert settings (mutualAuth, duration, renewBefore) in their own tls block. | <pre>{<br>"autoGenerated":{<br>"certManager":{<br>"caDuration":"87600h",<br>"duration":"2160h",<br>"existingIssuer":"",<br>"existingIssuerKind":"",<br>"keyAlgorithm":"RSA",<br>"keySize":2048,<br>"renewBefore":"360h"<br>},<br>"enabled":true,<br>"engine":"pod",<br>"helm":{<br>"caCertDuration":3650,<br>"certDuration":365<br>}<br>},<br>"enabled":false,<br>"mutualAuth":false<br>}</pre> |
|
||||
| security.​tls.​autoGenerated.​certManager.​caDuration | Duration of the shared CA certificate. | `"87600h"` |
|
||||
| security.​tls.​autoGenerated.​certManager.​duration | Default validity period for leaf certificates issued by cert-manager. Can be overridden per service (e.g. apis.rest.security.tls.certManager.duration). | `"2160h"` |
|
||||
| security.​tls.​autoGenerated.​certManager.​existingIssuer | Optional reference to an existing cert-manager Issuer or ClusterIssuer. When set, the chart uses this issuer instead of creating a self-signed one. Shared across all services — REST and gRPC will use the same issuer. | `""` |
|
||||
| security.​tls.​autoGenerated.​certManager.​keySize | Key algorithm and size used for all leaf certificates. | `2048` |
|
||||
| security.​tls.​autoGenerated.​certManager.​renewBefore | Default renewal window for leaf certificates. Can be overridden per service (e.g. apis.rest.security.tls.certManager.renewBefore). | `"360h"` |
|
||||
| security.​tls.​autoGenerated.​enabled | Enable automatic certificate generation/management. When false, each service's security.tls.existingSecret must point to a pre-existing TLS Secret. | `true` |
|
||||
| security.​tls.​autoGenerated.​engine | Certificate engine (shared across all services): pod: server generates a transient cert at startup (--auto-tls). No k8s Secrets are created. Cannot be combined with mutualAuth. helm: chart generates self-signed certificates stored in k8s Secrets. cert-manager: cert-manager provisions and rotates certificates. cert-manager must be installed. | `"pod"` |
|
||||
| security.​tls.​autoGenerated.​helm.​caCertDuration | Validity period in days for the helm-generated CA certificate. | `3650` |
|
||||
| security.​tls.​autoGenerated.​helm.​certDuration | Default validity period in days for helm-generated leaf certificates. Can be overridden per service (e.g. apis.rest.security.tls.helm.certDuration). | `365` |
|
||||
| security.​tls.​enabled | Enable TLS for all service endpoints. When false, all services use plain-text. | `false` |
|
||||
| security.​tls.​mutualAuth | Default: enable mutual TLS (clients verify the server and present their own certificate). Can be overridden per service (e.g. apis.rest.security.tls.mutualAuth). Has no effect when engine=pod, which uses server-only transient TLS. | `false` |
|
||||
| storageClass.​allowVolumeExpansion | Enable volume expansion for the default StorageClass. | `true` |
|
||||
| tolerations | Tolerations to be applied to all components except external Chart dependencies. If any component has tolerations set, then it would override this value. For external components like etcd, jaeger and loki, tolerations can only be set at component level. | `[]` |
|
||||
|
||||
Reference in New Issue
Block a user