Enable tracing

This guide contains the steps to enable tracing with Grafana Tempo for your PostgreSQL application.

To summarize:

• Deploy the Tempo charm in a COS K8s environment
• Offer interfaces for cross-model integrations
• Consume and integratre cross-model integrations
• View PostgreSQL traces on Grafana

Prerequisites

Enabling tracing with Tempo requires that you:

• Have deployed a Charmed PostgreSQL application
  • See How to scale units
• Have deployed a ‘cos-lite’ bundle from the latest/edge track in a Kubernetes environment
  • See Getting started on MicroK8s

Deploy Tempo

First, switch to the Kubernetes controller where the COS model is deployed:

juju switch <k8s_controller_name>:<cos_model_name>

Then, deploy the dependencies of Tempo following this tutorial. In particular, we would want to:

• Deploy the minio charm
• Deploy the s3 integrator charm
• Add a bucket into minio using a python script
• Configure s3 integrator with the minio credentials

Finally, deploy and integrate with Tempo HA in a monolithic setup.

Offer interfaces

Next, offer interfaces for cross-model integrations from the model where Charmed PostgreSQL is deployed.

To offer the Tempo integration, run

juju offer <tempo_coordinator_k8s_application_name>:tracing

Then, switch to the Charmed PostgreSQL model, find the offers, and integrate (relate) with them:

juju switch <machine_controller_name>:<postgresql_model_name>

juju find-offers <k8s_controller_name>:

Do not miss the “:” in the command above.

Below is a sample output where k8s is the K8s controller name and cos is the model where cos-lite and tempo-k8s are deployed:

Store  URL                            Access  Interfaces
k8s    admin/cos.tempo                admin   tracing:tracing

Next, consume this offer so that it is reachable from the current model:

juju consume k8s:admin/cos.tempo

Consume interfaces

First, deploy Grafana Agent from the latest/edge channel.

juju deploy grafana-agent --channel latest/edge

Then, integrate Grafana Agent with Charmed PostgreSQL:

juju integrate postgresql:cos-agent grafana-agent:cos-agent

Finally, integrate Grafana Agent with the consumed interface from the previous section:

juju integrate grafana-agent:tracing tempo:tracing

Wait until the model settles. The following is an example of the juju status --relations on the Charmed PostgreSQL model:

Model     Controller  Cloud/Region         Version  SLA          Timestamp
database  lxd         localhost/localhost  3.5.4    unsupported  21:43:34Z

SAAS        Status  Store       URL
tempo       active  uk8s        admin/cos.tempo

App            Version  Status   Scale  Charm          Channel      Rev  Exposed  Message
grafana-agent           blocked      1  grafana-agent  latest/edge  286  no       Missing ['grafana-cloud-config']|['grafana-dashboards-provider']|['logging-consumer']|['send-remote-write'] for cos-a...
postgresql     14.13    active       1  postgresql                    0  no       

Unit                Workload  Agent  Machine  Public address  Ports     Message
postgresql/0*       active    idle   0        10.205.193.87   5432/tcp  Primary
  grafana-agent/0*  blocked   idle            10.205.193.87             Missing ['grafana-cloud-config']|['grafana-dashboards-provider']|['logging-consumer']|['send-remote-write'] for cos-a...

Machine  State    Address        Inst id        Base          AZ  Message
0        started  10.205.193.87  juju-1fee5d-0  [email protected]      Running

Integration provider       Requirer                   Interface              Type         Message
grafana-agent:peers        grafana-agent:peers        grafana_agent_replica  peer         
postgresql:cos-agent       grafana-agent:cos-agent    cos_agent              subordinate  
postgresql:database-peers  postgresql:database-peers  postgresql_peers       peer         
postgresql:restart         postgresql:restart         rolling_op             peer         
postgresql:upgrade         postgresql:upgrade         upgrade                peer         
tempo:tracing              grafana-agent:tracing      tracing                regular  

View traces

After this is complete, the Tempo traces will be accessible from Grafana under the Explore section with tempo-k8s as the data source. You will be able to select postgresql as the Service Name under the Search tab to view traces belonging to Charmed PostgreSQL.

Below is a screenshot demonstrating a Charmed PostgreSQL trace:

Feel free to read through the Tempo HA documentation at your leisure to explore its deployment and its integrations.