4.10.2 Configuring FEPCluster Remote Logging

This section describes how to enable logging in FEPCluster. FEP cluster provides a feature to forward logs to remote Fluentd(FEPLogging) and FEPLogging instance will forward the same logs to Elasticsearch(Optional) & Prometheus.

The following destinations can be used with remote logging:

Fluentd
Elasticsearch
Azure BLOB
Amazon CloudWatch

For more information about log forwarding with the remote logging feature, refer to "Appendix E Fluent Bit Integration Using Custom Secret".

The remoteLogging section needs to be added under fep to define required parameters for remoteLogging configuration.

Following is a sample template:

spec:
  fep
    …
    remoteLogging:
      enable: true
      fluentdName: new-fep-logging
      tls:
        certificateName: fluentbit-cert
        caName: cacert
    …

Below is the list of all parameters defined in the remoteLogging section, along with their brief description:

Custom Resource spec	Required/Optional	Change Effect	Updating value allowed
remoteLogging.enable	Required	The 'enable' is set to true for enabling Logging feature	No
remoteLogging.fluentdName	Optional	The 'fluentdName' is the name of the FEPLogging CR where logs will be forwarded Specify this when transferring logs using the FEP log feature	Yes
remoteLogging.tls.certificateName	Optional	Secret name which contains MTLS certs of fluentbit	No
remoteLogging.tls.caName	Optional	Cacert of Fluentd for ssl verification	No
remoteLogging.image	Optional	Fluentbit image for remoteLogging	Yes
remoteLogging.pullPolicy	Optional	Fluentbit image pull policy	Yes
remoteLogging.mcSpec.limits.cpu	Optional	CPU allocation limit for fluentbit	Yes
remoteLogging.mcSpec.limits.memory	Optional	Memory allocation limit for fluentbit	Yes
remoteLogging.mcSpec.requests.cpu	Optional	CPU allocation request for fluentbit	Yes
remoteLogging.mcSpec.requests.memory	Optional	Memory allocation request for fluentbit	Yes
remoteLogging.fluentbitParams.memBufLimit	Optional	Defines the Mem_Buf_Limit in Fluentbit. This will affect all sections that use this parameter	Yes
remoteLogging.fluentbitConfigSecretRef	Optional	Specify the name of the Secret that contains fluent-bit.yaml when using the log forwarding feature with remote logging. If fluentbitConfigSecretRef is not defined or is defined but the referenced secret does not exist, the FEP operator creates a default Secret <fep-cluster>-fluent-bit-conf and updates this parameter with <fep-cluster>-fluent-bit-conf. If the referenced secret exists, the named secret is mounted to fep-logging-fluent-bit under /fluent-bit/etc.	Yes
remoteLogging.awsCredentialSecretRef	Optional	The 'awsCredentialSecretRef' is the name of a Secret that contains credentials to the AWS service. The credentials are stored in a configuration file and a credentials file. The configuration file must be named "config" and the credentials file must be named "credentials". If the referenced secret exists, the named secret will be mounted to fep-logging-fluent-bit under /fluent-bit/aws.	Yes

Specify this if you want to forward logs to a Fluentd container created by the FEP logging feature.

The enable flag is used to describe that FEPCluster will enable logging feature if set as true.

If enable flag set as true then fluentdName is the mandatory field. It will describe the FEPLogging CR name to which FEPCluster will forwards the logs.

If the enable flag is set as false, the FEPCluster will not enable logging feature.

Example)

    fep:
      remoteLogging:
        enable: true
        fluentdName: new-fep-logging

If user wants to update existing FEPCluster with logging feature then FEPCluster log_destination configuration must be set as csvlogs. For new cluster it will be already set.

Example)

    fep:
      …
      remoteLogging:
        enable: true
        fluentdName: new-fep-logging
      …

    fepChildCrVal:
      customPgParams:
         …
         log_destination = csvlog
         …

When FEPCluster uses secure connection for remoteLogging, then TLS section is mandatory.

In the TLS section, provide the secret name that contains certificate and private key that is used for ssl verification.

For MTLS connection caName is required to mutually validate certificate.

Example)

    fep:
      remoteLogging:
        enable: true
        fluentdName: new-fep-logging
        tls:
          certificateName: fluentbit-cert-secret
          caName: ca-cert

Note

The Elasticsearch server is configured by user and it is NOT part of FEPLogging deployment by operator.

The image property is used to specify other than default Fluentbit image and it’s pullPolicy.

If not specified it will use default image provided by Operator.

Example)

  spec:  
    fep:
      remoteLogging:
        image: 'quay.io/fujitsu/fujitsu-enterprise-postgres-fluentbit:ubi9-18-1.0'
        pullPolicy: IfNotPresent

Specify this if you want to use the log forwarding function via the remote logging feature. ‘fluentbitConfigSecretRef’ is the name of the secret that contains fluentbit.yaml. Refer to "Appendix E Fluent Bit Integration Using Custom Secret".

If not specified, the default secret created by the operator, <fep-cluster>-fluent-bit-conf, is used.

Example)

  spec:
    fep:
      remoteLogging:
        fluentbitConfigSecretRef: ' custom-secret'

'awsCredentialSecretRef' is the name of the secret that contains the credentials to the AWS service.

Example)

  spec:
    fep:
      remoteLogging:
        awsCredentialSecretRef: 'aws-credential'

4.10.2 Configuring FEPCluster Remote Logging

4.10.2.1 FEP Custom Resources - spec.fep.remoteLogging

4.10.2.1.1 Define remoteLogging enable and fluentdName

4.10.2.1.2 Define remoteLogging tls

4.10.2.1.3 Define remoteLogging image

4.10.2.1.4 Define remoteLogging fluentbitConfigSecretRef

4.10.2.1.5 Define remoteLogging awsCredentialSecretRef