ishu.g/openebs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a14a56846565f6173a0514f6059469d30e5200ef
openebs/charts/mayastor/README.md
T

50 KiB
Raw Blame History

mayastor

Mayastor Helm chart for Kubernetes

Version: 2.12.1 Type: application AppVersion: 2.12.1

Installation Guide

Prerequisites

  • Make sure the system requirement pre-requisites are met.
  • Label the storage nodes same as the mayastor.nodeSelector in values.yaml
  • Create the namespace you want the chart to be installed, or pass the --create-namespace flag in the helm install command. 
    kubectl create ns <mayastor-namespace>
  • Create secret if downloading the container images from a private repo. 
    kubectl create secret docker-registry <same-as-image.pullSecrets[0]>  --docker-server="https://index.docker.io/v1/" --docker-username="<user-name>" --docker-password="<password>" --docker-email="<user-email>" -n <mayastor-namespace>

Installing the chart via the git repo

Clone the mayastor charts repo. Sync the chart dependencies

$ helm dependency update

Install the mayastor chart using the command.

$ helm install mayastor . -n <mayastor-namespace>

Installing the Chart via Helm Registry

To install the chart with the release name mymayastor:

$ helm repo add mayastor https://openebs.github.io/mayastor-extensions/
$ helm install mymayastor mayastor/mayastor

Uninstall Helm Chart

$ helm uninstall [RELEASE_NAME]

This removes all the Kubernetes components associated with the chart and deletes the release.

See helm uninstall for command documentation.

TLS Configuration

TLS is configured at two levels:

  • security.tls — shared CA/issuer infrastructure (engine, key algorithm, existing issuer). Common to all services (REST, gRPC, …).
  • apis.rest.security.tls — REST-specific leaf certificate settings (mutualAuth, certManager.duration, existingSecret). Per-client overrides live under apis.rest.security.tls.clients.<component> (existingSecret, certManager.secretName).
Setting Effect
security.tls.enabled: false No TLS. All services use plain-text.
security.tls.autoGenerated.engine: pod (default) Server generates a transient cert at startup (--auto-tls). No Secrets created. Cannot be combined with mutualAuth.
security.tls.autoGenerated.engine: helm Chart generates self-signed certificates stored in k8s Secrets. Clients verify the server cert.
security.tls.autoGenerated.engine: certManager cert-manager provisions and rotates certificates. cert-manager must be installed.
+ apis.rest.security.tls.mutualAuth: true Full mutual TLS (mTLS). Both sides verify. Separate client certs are provisioned for each component (csi-controller, csi-node, obs-callhome, io-engine, operator-diskpool, plugin). Requires engine: helm or engine: certManager.

The kubectl plugin (kubectl mayastor) automatically discovers the TLS mode from a pod annotation (openebs.io/rest-tls) that the chart adds to the api-rest pod.

No TLS (plain HTTP)

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.enabled=false

HTTPS with ephemeral pod-generated certificate

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.autoGenerated.engine=pod \
  --set apis.rest.security.tls.mutualAuth=true

The server generates a transient certificate at startup. Clients connect to HTTPS without certificate verification. No Secrets are created.

Mutual TLS — self-signed certificates

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.autoGenerated.engine=helm \
  --set apis.rest.security.tls.mutualAuth=true

The chart generates a shared CA, a server certificate, and separate client certificates for each component (csi-controller, csi-node, obs-callhome, io-engine, operator-diskpool, plugin). Each component mounts only its own keypair.

Mutual TLS — cert-manager

Install cert-manager first, then:

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.autoGenerated.engine=certManager \
  --set apis.rest.security.tls.mutualAuth=true

To use your own Issuer or ClusterIssuer:

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.autoGenerated.engine=certManager \
  --set security.tls.autoGenerated.certManager.existingIssuer=<your-issuer-name> \
  --set security.tls.autoGenerated.certManager.existingIssuerKind=ClusterIssuer

The chart creates a shared CA Certificate, a CA Issuer, a server Certificate, and separate client Certificate resources for each component (csi-controller, csi-node, obs-callhome, io-engine, operator-diskpool, plugin). Pass security.tls.autoGenerated.certManager.existingIssuer to use your own CA Issuer.

Bring your own certificates

Set security.tls.autoGenerated.enabled=false and supply pre-created Secrets:

$ helm install mayastor . -n <mayastor-namespace> \
  --set security.tls.enabled=true \
  --set security.tls.autoGenerated.enabled=false \
  --set apis.rest.security.tls.existingSecret=<server-tls-secret>

With mutual TLS, also supply a secret for each client:

  --set apis.rest.security.tls.clients.csiController.existingSecret=<secret> \
  --set apis.rest.security.tls.clients.csiNode.existingSecret=<secret> \
  --set apis.rest.security.tls.clients.callhome.existingSecret=<secret> \
  --set apis.rest.security.tls.clients.metricsExporter.existingSecret=<secret> \
  --set apis.rest.security.tls.clients.diskpoolOperator.existingSecret=<secret> \
  --set apis.rest.security.tls.clients.plugin.existingSecret=<secret>

Each Secret must contain tls.crt, tls.key, and ca.crt keys.

Chart Dependencies

Repository Name Version
crds 2.12.1
https://charts.bitnami.com/bitnami etcd 12.0.14
https://grafana.github.io/helm-charts alloy 1.0.1
https://grafana.github.io/helm-charts loki 6.29.0
https://jaegertracing.github.io/helm-charts jaeger-operator 2.50.1
https://nats-io.github.io/k8s/helm/charts/ nats 0.19.14
https://openebs.github.io/dynamic-localpv-provisioner localpv-provisioner 4.6.0

Values

Key Description Default
agents.​core.​allowNonPersistentDevlink Allow using non-persistent kernel devpaths for pool disks. Enabling this will let users to use the kernel devpaths e.g /dev/sda, for diskpools. However, this comes with associated risks if the devpaths get swapped among disks, resulting in total data loss especially if encryption is being used. false
agents.​core.​capacity.​thin.​poolCommitment The allowed pool commitment limit when dealing with thin provisioned volumes. Example: If the commitment is 250 and the pool is 10GiB we can overcommit the pool up to 25GiB (create 2 10GiB and 1 5GiB volume) but no further. "250%"
agents.​core.​capacity.​thin.​snapshotCommitment When creating snapshots for an existing volume, each replica pool must have at least this much free space percentage of the volume size. Example: if this value is 40, the pool has 40GiB free, then the max volume size allowed to be snapped on the pool is 100GiB. "40%"
agents.​core.​capacity.​thin.​volumeCommitment When creating replicas for an existing volume, each replica pool must have at least this much free space percentage of the volume size. Example: if this value is 40, the pool has 40GiB free, then the max volume size allowed to be created on the pool is 100GiB. "40%"
agents.​core.​capacity.​thin.​volumeCommitmentInitial Same as the volumeCommitment argument, but applicable only when creating replicas for a new volume. "40%"
agents.​core.​encryptedPoolsSoftScheduling Prefer encrypted pools for volume replicas. If a volume wasn't provisioned with a encryption storageclass, we try to place the replicas of such volume on best-effort basis onto encrypted pools, if this global is set. This is effective subject to volume spec already modified via plugin to request encryption. false
agents.​core.​logLevel Log level for the core service "info"
agents.​core.​minTimeouts Enable minimal timeouts true
agents.​core.​nodeSelector Set nodeSelector, overrides global 
{

}
agents.​core.​poolClusterSize Default blobstore cluster size for diskpools, in bytes. This value is used as a default value of blobstore cluster size on diskpools. This is set to 4MiB internally by default, if nothing specified here. The value is also configurable via Diskpool CR, which takes precedence over this setting. This is an advanced configuration, please refer documentation to understand the usage and implications of this. ""
agents.​core.​priorityClassName Set PriorityClass, overrides global. If both local and global are not set, the final deployment manifest has a mayastor custom critical priority class assigned to the pod by default. Refer the templates/_helpers.tpl and templates/mayastor/agents/core/agent-core-deployment.yaml for more details. ""
agents.​core.​rebuild.​maxConcurrent The maximum number of system-wide rebuilds permitted at any given time. If set to an empty string, there are no limits. ""
agents.​core.​rebuild.​partial.​enabled Partial rebuild uses a log of missed IO to rebuild replicas which have become temporarily faulted, hence a bit faster, depending on the log size. true
agents.​core.​rebuild.​partial.​waitPeriod If a faulted replica comes back online within this time period then it will be rebuilt using the partial rebuild capability. Otherwise, the replica will be fully rebuilt. A blank value "" means internally derived value will be used. ""
agents.​core.​requestTimeout Request timeout for core agents Default value is defined in .base.default_req_timeout nil
agents.​core.​resources.​limits.​cpu Cpu limits for core agents "1000m"
agents.​core.​resources.​limits.​memory Memory limits for core agents "128Mi"
agents.​core.​resources.​requests.​cpu Cpu requests for core agents "500m"
agents.​core.​resources.​requests.​memory Memory requests for core agents "32Mi"
agents.​core.​tolerations Set tolerations, overrides global []
agents.​core.​volumeHealth Enable extended volume health information, which helps generate the volume status more accurately. true
agents.​ha.​cluster.​logLevel Log level for the ha cluster service "info"
agents.​ha.​cluster.​resources.​limits.​cpu Cpu limits for ha cluster agent "100m"
agents.​ha.​cluster.​resources.​limits.​memory Memory limits for ha cluster agent "64Mi"
agents.​ha.​cluster.​resources.​requests.​cpu Cpu requests for ha cluster agent "100m"
agents.​ha.​cluster.​resources.​requests.​memory Memory requests for ha cluster agent "16Mi"
agents.​ha.​node.​logLevel Log level for the ha node service "info"
agents.​ha.​node.​nodeSelector Set nodeSelector, overrides global 
{

}
agents.​ha.​node.​port Container port for the ha-node service 50053
agents.​ha.​node.​priorityClassName Set PriorityClass, overrides global ""
agents.​ha.​node.​resources.​limits.​cpu Cpu limits for ha node agent "100m"
agents.​ha.​node.​resources.​limits.​memory Memory limits for ha node agent "64Mi"
agents.​ha.​node.​resources.​requests.​cpu Cpu requests for ha node agent "100m"
agents.​ha.​node.​resources.​requests.​memory Memory requests for ha node agent "64Mi"
agents.​ha.​node.​tolerations Set tolerations, overrides global []
alloy.​logging_config.​labels Labels to enable scraping on, at-least one of these labels should be present. 
{
"openebs.io/logging":true
}
alloy.​logging_config.​tenant_id X-Scope-OrgID to pe populated which pushing logs. Make sure the caller also uses the same. "openebs"
apis.​rest.​healthProbes.​liveness.​enabled Toggle liveness probe. true
apis.​rest.​healthProbes.​liveness.​failureThreshold No. of failures the liveness probe will tolerate. 3
apis.​rest.​healthProbes.​liveness.​initialDelaySeconds No. of seconds of delay before checking the liveness status. 1
apis.​rest.​healthProbes.​liveness.​periodSeconds No. of seconds between liveness probe checks. 30
apis.​rest.​healthProbes.​liveness.​timeoutSeconds No. of seconds of timeout tolerance. 5
apis.​rest.​healthProbes.​readiness.​agentCoreProbeFreq Frequency for the agent-core liveness probe. "20s"
apis.​rest.​healthProbes.​readiness.​enabled Toggle readiness probe. true
apis.​rest.​healthProbes.​readiness.​failureThreshold No. of failures the readiness probe will tolerate. 3
apis.​rest.​healthProbes.​readiness.​initialDelaySeconds No. of seconds of delay before checking the readiness status. 1
apis.​rest.​healthProbes.​readiness.​periodSeconds No. of seconds between readiness probe checks. 20
apis.​rest.​healthProbes.​readiness.​timeoutSeconds No. of seconds of timeout tolerance. 5
apis.​rest.​logLevel Log level for the rest service "info"
apis.​rest.​nodeSelector Set nodeSelector, overrides global 
{

}
apis.​rest.​priorityClassName Set PriorityClass, overrides global. If both local and global are not set, the final deployment manifest has a mayastor custom critical priority class assigned to the pod by default. Refer the templates/_helpers.tpl and templates/mayastor/apis/rest/api-rest-deployment.yaml for more details. ""
apis.​rest.​replicaCount Number of replicas of rest 1
apis.​rest.​resources.​limits.​cpu Cpu limits for rest "100m"
apis.​rest.​resources.​limits.​memory Memory limits for rest "64Mi"
apis.​rest.​resources.​requests.​cpu Cpu requests for rest "50m"
apis.​rest.​resources.​requests.​memory Memory requests for rest "32Mi"
apis.​rest.​security.​tls.​certManager.​secretName Secret name for the REST API server TLS certificate. Defaults to {release}-api-rest-crt. ""
apis.​rest.​security.​tls.​clients Per-client TLS overrides. Each client can point at a pre-existing secret (when autoGenerated.enabled is false) or customise the cert-manager secret name. 
{
"callhome":{
"certManager":{
"secretName":""
},
"existingSecret":""
},
"csiController":{
"certManager":{
"secretName":""
},
"existingSecret":""
},
"csiNode":{
"certManager":{
"secretName":""
},
"existingSecret":""
},
"diskpoolOperator":{
"certManager":{
"secretName":""
},
"existingSecret":""
},
"metricsExporter":{
"certManager":{
"secretName":""
},
"existingSecret":""
},
"plugin":{
"certManager":{
"secretName":""
},
"existingSecret":""
}
}
apis.​rest.​security.​tls.​clients.​callhome.​certManager.​secretName cert-manager secret name for the callhome client cert. Defaults to {release}-api-rest-callhome-crt. ""
apis.​rest.​security.​tls.​clients.​callhome.​existingSecret Pre-existing Secret for the callhome client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​clients.​csiController.​certManager.​secretName cert-manager secret name for the CSI controller client cert. Defaults to {release}-api-rest-csi-controller-crt. ""
apis.​rest.​security.​tls.​clients.​csiController.​existingSecret Pre-existing Secret for the CSI controller client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​clients.​csiNode.​certManager.​secretName cert-manager secret name for the CSI node client cert. Defaults to {release}-api-rest-csi-node-crt. ""
apis.​rest.​security.​tls.​clients.​csiNode.​existingSecret Pre-existing Secret for the CSI node client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​clients.​diskpoolOperator.​certManager.​secretName cert-manager secret name for the diskpool-operator client cert. Defaults to {release}-api-rest-diskpool-operator-crt. ""
apis.​rest.​security.​tls.​clients.​diskpoolOperator.​existingSecret Pre-existing Secret for the diskpool-operator client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​clients.​metricsExporter.​certManager.​secretName cert-manager secret name for the metrics-exporter client cert. Defaults to {release}-api-rest-metrics-exporter-crt. ""
apis.​rest.​security.​tls.​clients.​metricsExporter.​existingSecret Pre-existing Secret for the metrics-exporter client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​clients.​plugin.​certManager.​secretName cert-manager secret name for the kubectl plugin client cert. Defaults to {release}-api-rest-plugin-crt. ""
apis.​rest.​security.​tls.​clients.​plugin.​existingSecret Pre-existing Secret for the kubectl plugin client cert. Required when autoGenerated.enabled is false and mutualAuth is true. ""
apis.​rest.​security.​tls.​existingSecret Pre-existing TLS Secret to use when tls.autoGenerated.enabled is false. Must contain tls.crt, tls.key, and ca.crt. The chart mounts it but does not manage it. ""
apis.​rest.​service.​type Rest K8s service type "ClusterIP"
apis.​rest.​tolerations Set tolerations, overrides global []
base.​cache_poll_period Cache timeout for core agent & diskpool deployment "30s"
base.​default_req_timeout Request timeout for rest & core agents "5s"
base.​initContainers.​image.​registry Image registry for init containers ""
base.​logging.​color Enable ansi color code for Pod StdOut/StdErr true
base.​logging.​format Valid values for format are pretty, json and compact "pretty"
base.​logging.​silenceLevel Silence specific module components nil
base.​metrics.​enabled Enable the metrics exporter true
base.​metrics.​port Container port for the metrics exporter service 9502
crds.​csi.​volumeSnapshots.​enabled Install Volume Snapshot CRDs true
crds.​enabled Disables the installation of all CRDs if set to false true
csi.​controller.​logLevel Log level for the csi controller "info"
csi.​controller.​nodeSelector Set nodeSelector, overrides global 
{

}
csi.​controller.​preventVolumeModeConversion Prevent modifying the volume mode when creating a PVC from an existing VolumeSnapshot true
csi.​controller.​priorityClassName Set PriorityClass, overrides global ""
csi.​controller.​resources.​limits.​cpu Cpu limits for csi controller "32m"
csi.​controller.​resources.​limits.​memory Memory limits for csi controller "128Mi"
csi.​controller.​resources.​requests.​cpu Cpu requests for csi controller "16m"
csi.​controller.​resources.​requests.​memory Memory requests for csi controller "64Mi"
csi.​controller.​snapshotController.​enabled Run the csi-snapshot-controller container. Disable this if the cluster already runs one. true
csi.​controller.​tolerations Set tolerations, overrides global []
csi.​image.​attacherTag csi-attacher image release tag "v4.8.1"
csi.​image.​provisionerTag csi-provisioner image release tag "v5.2.0"
csi.​image.​pullPolicy imagePullPolicy for all CSI Sidecar images "IfNotPresent"
csi.​image.​registrarTag csi-node-driver-registrar image release tag "v2.13.0"
csi.​image.​registry Image registry to pull all CSI Sidecar images "registry.k8s.io"
csi.​image.​repo Image registry's namespace "sig-storage"
csi.​image.​resizerTag csi-resizer image release tag "v1.13.2"
csi.​image.​snapshotControllerTag csi-snapshot-controller image release tag "v8.2.0"
csi.​image.​snapshotterTag csi-snapshotter image release tag "v8.2.0"
csi.​node.​kubeletDir The kubeletDir directory for the csi-node plugin "/var/lib/kubelet"
csi.​node.​nodeSelector Set nodeSelector, overrides global 
{

}
csi.​node.​nvme.​ctrl_loss_tmo The ctrl_loss_tmo (controller loss timeout) in seconds "1980"
csi.​node.​nvme.​tcpFallback Fallback to nvme-tcp if nvme-rdma is enabled for Mayastor but rdma is not available on a particular csi-node true
csi.​node.​port Container port for the csi-node service 10199
csi.​node.​priorityClassName Set PriorityClass, overrides global ""
csi.​node.​resources.​limits.​cpu Cpu limits for csi node plugin "100m"
csi.​node.​resources.​limits.​memory Memory limits for csi node plugin "128Mi"
csi.​node.​resources.​requests.​cpu Cpu requests for csi node plugin "100m"
csi.​node.​resources.​requests.​memory Memory requests for csi node plugin "64Mi"
csi.​node.​tolerations Set tolerations, overrides global []
csi.​node.​topology.​nodeSelector Add topology segments to the csi-node and agent-ha-node daemonset node selector false
etcd.​autoCompactionMode AutoCompaction Since etcd keeps an exact history of its keyspace, this history should be periodically compacted to avoid performance degradation and eventual storage space exhaustion. Auto compaction mode. Valid values: "periodic", "revision". - 'periodic' for duration based retention, defaulting to hours if no time unit is provided (e.g. 5m). - 'revision' for revision number based retention. "revision"
etcd.​autoCompactionRetention Auto compaction retention length. 0 means disable auto compaction. "100"
etcd.​clusterDomain Kubernetes Cluster Domain "cluster.local"
etcd.​enabled Disable when using an external etcd cluster. true
etcd.​externalUrl Url of the external etcd cluster. Note, etcd.enable must be set to false. ""
etcd.​extraEnvVars[0] Raise alarms when backend size exceeds the given quota. 
{
"name":"ETCD_QUOTA_BACKEND_BYTES",
"value":"8589934592"
}
etcd.​localpvScConfig.​basePath Host path where local etcd data is stored in. "/var/local/{{ .Release.Name }}/localpv-hostpath/etcd"
etcd.​localpvScConfig.​reclaimPolicy ReclaimPolicy of etcd's localpv hostpath storage class. "Delete"
etcd.​localpvScConfig.​volumeBindingMode VolumeBindingMode of etcd's localpv hostpath storage class. "WaitForFirstConsumer"
etcd.​metrics.​enabled Expose etcd metrics. true
etcd.​metrics.​useSeparateEndpoint Use a separate endpoint for exposing metrics, override the default port (9090) by setting containerPorts.metrics. true
etcd.​persistence.​enabled If true, use a Persistent Volume Claim. If false, use emptyDir. true
etcd.​persistence.​size Volume size "2Gi"
etcd.​persistence.​storageClass Will define which storageClass to use in etcd's StatefulSets. Options:

- "manual" - Will provision a hostpath PV on the same node.
- "" (empty) - Will use the default StorageClass on the cluster.

"mayastor-etcd-localpv"
etcd.​persistentVolumeClaimRetentionPolicy.​enabled PVC's reclaimPolicy false
etcd.​podAntiAffinityPreset Pod anti-affinity preset Ref: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity "hard"
etcd.​removeMemberOnContainerTermination Use a PreStop hook to remove the etcd members from the etcd cluster on container termination Ignored if lifecycleHooks is set or replicaCount=1 false
etcd.​replicaCount Number of replicas of etcd 3
eventing.​aggregator.​dirSizeLimit Maximum total size of event files including rotated history. Used when loki.enabled is false. "100Mi"
eventing.​aggregator.​enabled Enable the eventing-aggregator deployment. Requires eventing.enabled to be true. true
eventing.​aggregator.​logLevel Log level for eventing-aggregator "info"
eventing.​aggregator.​nodeSelector Set nodeSelector, overrides global 
{

}
eventing.​aggregator.​priorityClassName Set PriorityClass, overrides global ""
eventing.​aggregator.​resources.​limits.​cpu Cpu limits for eventing-aggregator "100m"
eventing.​aggregator.​resources.​limits.​memory Memory limits for eventing-aggregator "32Mi"
eventing.​aggregator.​resources.​requests.​cpu Cpu requests for eventing-aggregator "50m"
eventing.​aggregator.​resources.​requests.​memory Memory requests for eventing-aggregator "16Mi"
eventing.​aggregator.​tolerations Set tolerations, overrides global []
global.​analytics.​enabled Global overide for call home nil
global.​imagePullPolicy Global overide for image pull policy ""
global.​imagePullSecrets Global override for image pull secrets - secret []
global.​imageRegistry Global override for image registry ""
image.​pullPolicy ImagePullPolicy for our images "Always"
image.​pullSecrets docker-secrets required to pull images if the container registry from image.registry is protected []
image.​registry Image registry to pull our product images "docker.io"
image.​repo Image registry's namespace "openebs"
image.​tag Release tag for our images "release-2.12"
io_engine.​coreList If not empty, overrides the cpuCount and explicitly sets the list of cores. Example: --set='io_engine.coreList={30,31}' []
io_engine.​cpuCount The number of cores that each io-engine instance will bind to. "2"
io_engine.​envcontext Pass additional arguments to the Environment Abstraction Layer. Example: --set {product}.envcontext=iova-mode=pa ""
io_engine.​interruptMode SPDK interrupt mode for io-engine reactors. When enabled, reactors sleep on epoll/timerfd instead of busy-polling, which dramatically reduces idle CPU usage. NVMe I/O queues are still polled, but on a periodic timer rather than continuously. 
{
"enabled":false,
"nvmeIoQueuePollPeriod":"100us"
}
io_engine.​interruptMode.​enabled Enable interrupt mode by setting ENABLE_INTERRUPT_MODE=true on the io-engine container (equivalent to passing the --enable-interrupt-mode CLI flag). false
io_engine.​interruptMode.​nvmeIoQueuePollPeriod NVMe I/O queue poll period (SPDK NVME_IOQ_POLL_PERIOD). A value of "0" disables timed polling (busy poll). Typical values: "100us", "1000us". Higher values reduce CPU further at the cost of latency. "100us"
io_engine.​logLevel Log level for the io-engine service "info"
io_engine.​nodeSelector Node selectors to designate storage nodes for diskpool creation Note that if multi-arch images support 'kubernetes.io/arch: amd64' should be removed. 
{
"kubernetes.io/arch":"amd64",
"openebs.io/engine":"mayastor"
}
io_engine.​nvme.​ioTimeout Timeout for IOs The default here is exaggerated for local disks, but we've observed that in shared virtual environments having a higher timeout value is beneficial. Please adjust this according to your hardware and needs. "110s"
io_engine.​nvme.​rdma.​bufCacheSize The number of shared buffers to reserve for each poll group nil
io_engine.​nvme.​rdma.​dataWrPoolSize RDMA data WR pool size (RDMA only) "4095"
io_engine.​nvme.​rdma.​inCapsuleDataSize The max amount of payload data that can be transferred directly within the NVMe-oF Capsule command itself nil
io_engine.​nvme.​rdma.​ioUnitSize I/O unit size (bytes) "8192"
io_engine.​nvme.​rdma.​maxIoSize Max I/O size (bytes) nil
io_engine.​nvme.​rdma.​numSharedBuf The number of pooled data buffers available to the transport nil
io_engine.​nvme.​tcp.​bufCacheSize The number of shared buffers to reserve for each poll group "64"
io_engine.​nvme.​tcp.​inCapsuleDataSize The max amount of payload data that can be transferred directly within the NVMe-oF Capsule command itself "4096"
io_engine.​nvme.​tcp.​ioUnitSize I/O unit size (bytes) "131072"
io_engine.​nvme.​tcp.​maxIoSize Max I/O size (bytes) "131072"
io_engine.​nvme.​tcp.​maxQpairsPerCtrl Max number of IO qpairs per controller "32"
io_engine.​nvme.​tcp.​maxQueueDepth You may need to increase this for a higher outstanding IOs per volume "32"
io_engine.​nvme.​tcp.​numSharedBuf The number of pooled data buffers available to the transport "2047"
io_engine.​nvme.​transportTos NVMe Transport Type of Service (ToS) value for RDMA QoS/DSCP marking. When using NVMe-oF over RDMA (RoCEv2), set this to mark target (responder) side RDMA traffic with a DSCP value (e.g. 104 for DSCP 26 / AF31) so that switches and NICs can classify storage traffic into a Priority Flow Control (PFC) enabled queue for lossless transport. A value of 0 (the default) means no marking (best-effort QoS). ""
io_engine.​pool.​diskHandleRescan.​enabled Periodic rescan of the diskpool backend storage file handles. This is used for hot-remove detection without ongoing I/O, as well as updating the disk size true
io_engine.​pool.​ioAlerts.​errorThreshold After this many errors a pool alert is raised as Warning. 64
io_engine.​pool.​ioAlerts.​stallDeadline If an I/O is stuck longer than this period, then the pool is considered stalled and a Critical alert is raised. The pool disk will also be reset and the stall will be cleared once complete and I/O flows again. default: .Values.io_engine.nvme.ioTimeout * 2 nil
io_engine.​pool.​ioAlerts.​stallTransitionThreshold After this many transitions within the stallTransitionWindow, a pool alert is raised as Warning. 3
io_engine.​pool.​ioAlerts.​stallTransitionWindow Time window during which stall ↔ resume state transitions are tracked for flakiness detection. "3h"
io_engine.​port Container port for the io-engine service 10124
io_engine.​priorityClassName Set PriorityClass, overrides global ""
io_engine.​pstorRetries Number of retries for pstor persistence before the volume target self shutdowns 300
io_engine.​resources.​limits.​cpu Cpu limits for the io-engine ""
io_engine.​resources.​limits.​hugepages1Gi Hugepage memory in 1GiB chunks nil
io_engine.​resources.​limits.​hugepages2Mi Hugepage memory in 2MiB chunks "2Gi"
io_engine.​resources.​limits.​memory Memory limits for the io-engine "1Gi"
io_engine.​resources.​requests.​cpu Cpu requests for the io-engine ""
io_engine.​resources.​requests.​hugepages1Gi Hugepage memory in 1GiB chunks nil
io_engine.​resources.​requests.​hugepages2Mi Hugepage memory in 2MiB chunks "2Gi"
io_engine.​resources.​requests.​memory Memory requests for the io-engine "1Gi"
io_engine.​runtimeClassName Runtime class to use. Defaults to cluster standard ""
io_engine.​target.​nvmf.​iface NVMF target interface (ip, mac, name or subnet) If RDMA is enabled, please set iface to an RDMA capable netdev name from host network. Example, if an rdma device mlx5_0 is available on a netdev eth0 on RNIC, as can be seen from rdma link command output, then this field should be set to eth0. ""
io_engine.​target.​nvmf.​maxNamespaces Maximum number of NVMe namespaces which a given io-engine node can expose. As of today, there's a 1-1 mapping of namespaces to volume targets. 4096
io_engine.​target.​nvmf.​ptpl Reservations Persist Through Power Loss State true
io_engine.​target.​nvmf.​rdma Enable RDMA Capability of Mayastor nvmf target to take RDMA connections if the cluster nodes have RDMA device(s) configured from RNIC. 
{
"enabled":false
}
io_engine.​tolerations Set tolerations, overrides global []
localpv-provisioner.​enabled Enables the openebs dynamic-localpv-provisioner. If disabled, modify etcd and loki storage class accordingly. true
localpv-provisioner.​hostpathClass.​enabled Enable default hostpath localpv StorageClass. false
localpv-provisioner.​localpv.​priorityClassName Set the PriorityClass for the LocalPV Hostpath provisioner Deployment. "{{ .Release.Name }}-cluster-critical"
loki.​localpvScConfig.​loki.​basePath Host path where local loki data is stored in. "/var/local/{{ .Release.Name }}/localpv-hostpath/loki"
loki.​localpvScConfig.​loki.​reclaimPolicy ReclaimPolicy of loki's localpv hostpath storage class. "Delete"
loki.​localpvScConfig.​loki.​volumeBindingMode VolumeBindingMode of loki's localpv hostpath storage class. "WaitForFirstConsumer"
loki.​localpvScConfig.​minio.​basePath Host path where local minio data is stored in. "/var/local/{{ .Release.Name }}/localpv-hostpath/minio"
loki.​localpvScConfig.​minio.​reclaimPolicy ReclaimPolicy of minio's localpv hostpath storage class. "Delete"
loki.​localpvScConfig.​minio.​volumeBindingMode VolumeBindingMode of minio's localpv hostpath storage class. "WaitForFirstConsumer"
nodeSelector Node labels for pod assignment ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ Note that if multi-arch images support 'kubernetes.io/arch: amd64' should be removed and set 'nodeSelector' to empty '{}' as default value. 
{
"kubernetes.io/arch":"amd64"
}
obs.​callhome.​enabled Enable callhome true
obs.​callhome.​logLevel Log level for callhome "info"
obs.​callhome.​nodeSelector Set nodeSelector, overrides global 
{

}
obs.​callhome.​priorityClassName Set PriorityClass, overrides global ""
obs.​callhome.​resources.​limits.​cpu Cpu limits for callhome "100m"
obs.​callhome.​resources.​limits.​memory Memory limits for callhome "32Mi"
obs.​callhome.​resources.​requests.​cpu Cpu requests for callhome "50m"
obs.​callhome.​resources.​requests.​memory Memory requests for callhome "16Mi"
obs.​callhome.​tolerations Set tolerations, overrides global []
obs.​stats.​logLevel Log level for stats "info"
obs.​stats.​resources.​limits.​cpu Cpu limits for stats "100m"
obs.​stats.​resources.​limits.​memory Memory limits for stats "32Mi"
obs.​stats.​resources.​requests.​cpu Cpu requests for stats "50m"
obs.​stats.​resources.​requests.​memory Memory requests for stats "16Mi"
obs.​stats.​service.​type Rest K8s service type "ClusterIP"
operators.​pool.​logLevel Log level for diskpool operator service "info"
operators.​pool.​nodeSelector Set nodeSelector, overrides global 
{

}
operators.​pool.​priorityClassName Set PriorityClass, overrides global ""
operators.​pool.​resources.​limits.​cpu Cpu limits for diskpool operator "100m"
operators.​pool.​resources.​limits.​memory Memory limits for diskpool operator "32Mi"
operators.​pool.​resources.​requests.​cpu Cpu requests for diskpool operator "50m"
operators.​pool.​resources.​requests.​memory Memory requests for diskpool operator "16Mi"
operators.​pool.​tolerations Set tolerations, overrides global []
preUpgradeHook.​enabled Enable/Disable mayastor pre-upgrade hook true
preUpgradeHook.​image.​pullPolicy The imagePullPolicy for the container "IfNotPresent"
preUpgradeHook.​image.​registry The container image registry URL for the hook job "docker.io"
preUpgradeHook.​image.​repo The container repository for the hook job "openebs/kubectl"
preUpgradeHook.​image.​tag The container image tag for the hook job "1.25.15"
preUpgradeHook.​imagePullSecrets Optional array of imagePullSecrets containing private registry credentials # Ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/ []
preUpgradeHook.​rolloutTimeout Set how long we should wait for the Etcd cluster to finish rolling out before giving up. "600s"
preUpgradeHook.​tolerations Node tolerations for server scheduling to nodes with taints # Ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ # []
priorityClassName Pod scheduling priority. Setting this value will apply to all components except the external Chart dependencies. If any component has priorityClassName set, then this value would be overridden for that component. For external components like etcd, jaeger or loki, PriorityClass can only be set at component level. ""
security.​networkPolicy.​enabled When enabled, the NetworkPolicy will block all HTTP traffic to the REST API service. true
security.​tls TLS configuration shared across all service endpoints. The CA, issuer, engine, and per-certificate defaults are cluster-scoped infrastructure common to all services (REST, gRPC, …). Individual services may override leaf-cert settings (mutualAuth, duration, renewBefore) in their own tls block. 
{
"autoGenerated":{
"certManager":{
"caDuration":"87600h",
"duration":"2160h",
"existingIssuer":"",
"existingIssuerKind":"",
"keyAlgorithm":"RSA",
"keySize":2048,
"renewBefore":"360h"
},
"enabled":true,
"engine":"pod",
"helm":{
"caCertDuration":3650,
"certDuration":365
}
},
"enabled":false,
"mutualAuth":false
}
security.​tls.​autoGenerated.​certManager.​caDuration Duration of the shared CA certificate. "87600h"
security.​tls.​autoGenerated.​certManager.​duration Default validity period for leaf certificates issued by cert-manager. Can be overridden per service (e.g. apis.rest.security.tls.certManager.duration). "2160h"
security.​tls.​autoGenerated.​certManager.​existingIssuer Optional reference to an existing cert-manager Issuer or ClusterIssuer. When set, the chart uses this issuer instead of creating a self-signed one. Shared across all services — REST and gRPC will use the same issuer. ""
security.​tls.​autoGenerated.​certManager.​keySize Key algorithm and size used for all leaf certificates. 2048
security.​tls.​autoGenerated.​certManager.​renewBefore Default renewal window for leaf certificates. Can be overridden per service (e.g. apis.rest.security.tls.certManager.renewBefore). "360h"
security.​tls.​autoGenerated.​enabled Enable automatic certificate generation/management. When false, each service's security.tls.existingSecret must point to a pre-existing TLS Secret. true
security.​tls.​autoGenerated.​engine Certificate engine (shared across all services): pod: server generates a transient cert at startup (--auto-tls). No k8s Secrets are created. Cannot be combined with mutualAuth. helm: chart generates self-signed certificates stored in k8s Secrets. cert-manager: cert-manager provisions and rotates certificates. cert-manager must be installed. "pod"
security.​tls.​autoGenerated.​helm.​caCertDuration Validity period in days for the helm-generated CA certificate. 3650
security.​tls.​autoGenerated.​helm.​certDuration Default validity period in days for helm-generated leaf certificates. Can be overridden per service (e.g. apis.rest.security.tls.helm.certDuration). 365
security.​tls.​enabled Enable TLS for all service endpoints. When false, all services use plain-text. false
security.​tls.​mutualAuth Default: enable mutual TLS (clients verify the server and present their own certificate). Can be overridden per service (e.g. apis.rest.security.tls.mutualAuth). Has no effect when engine=pod, which uses server-only transient TLS. false
storageClass.​allowVolumeExpansion Enable volume expansion for the default StorageClass. true
tolerations Tolerations to be applied to all components except external Chart dependencies. If any component has tolerations set, then it would override this value. For external components like etcd, jaeger and loki, tolerations can only be set at component level. []
Reference in New Issue View Git Blame Copy Permalink