Compare commits

..

4 Commits

Author SHA1 Message Date
7104414bc1 Fix network issues in Docker 2024-04-16 08:11:01 +02:00
d580b0f1d0 Update console container 2024-04-16 08:10:46 +02:00
0312a1e37c Remove commented code 2024-04-16 08:10:27 +02:00
c98e8da2ec Format README 2024-04-16 08:09:41 +02:00
2 changed files with 17 additions and 12 deletions

View File

@@ -1,30 +1,34 @@
# KafkaexLagExporter # KafkaExLagExporter
This project will collect Kafka consumer lag and provide them via Prometheus. This project will collect Kafka consumer lag and provide them via Prometheus.
## Metrics ## Metrics
[Prometheus](https://prometheus.io/) is a standard way to represent metrics in a modern cross-platform manner. KafkaexLagExporter exposes several metrics as an HTTP endpoint that can be readily scraped by Prometheus. [Prometheus](https://prometheus.io/) is a standard way to represent metrics in a modern cross-platform manner.
KafkaExLagExporter exposes several metrics as an HTTP endpoint that can be readily scraped by Prometheus.
**`kafka_consumergroup_group_topic_sum_lag`** **`kafka_consumergroup_group_topic_sum_lag`**
Labels: `cluster_name, group, topic, consumer_id, member_host` Labels: `cluster_name, group, topic, consumer_id, member_host`
The sum of the difference between the last produced offset and the last consumed offset of all partitions in this topic for this group. The sum of the difference between the last produced offset and the last consumed offset of all partitions in this
topic for this group.
**`kafka_consumergroup_group_lag`** **`kafka_consumergroup_group_lag`**
Labels: `cluster_name, group, partition, topic, member_host, consumer_id` Labels: `cluster_name, group, partition, topic, member_host, consumer_id`
The difference between the last produced offset and the last consumed offset for this partition in this topic partition for this group. The difference between the last produced offset and the last consumed offset for this partition in this topic
partition for this group.
## Start ## Start
```bash ```bash
docker run -ti --net="host" -e KAFKA_BROKERS=localhost:9093,localhost:9094,localhost:9095 lechindianer/kafkaex_lag_exporter:0.2.0 docker run -ti --net="host" -e KAFKA_BROKERS=localhost:9093,localhost:9094,localhost:9095 -p 4000:4000 \
lechindianer/kafkaex_lag_exporter:0.2.0
``` ```
Now you can check the exposed metrics at [localhost:4000](localhost:4000). Now you can check the exposed metrics at [http://localhost:4000](http://localhost:4000).
## Developing ## Developing
@@ -41,7 +45,7 @@ KafkaexLagExporter:
docker-compose up --build docker-compose up --build
``` ```
Kowl is served at [localhost:8080](localhost:8080). Kowl is served at [http://localhost:8080](http://localhost:8080).
### Tests ### Tests
@@ -67,4 +71,5 @@ mix dialyzer
Source is on [Gitlab](https://gitlab.com/lechindianer/kafkaex-lag-exporter). Source is on [Gitlab](https://gitlab.com/lechindianer/kafkaex-lag-exporter).
The initial project [Kafka Lag Exporter](https://github.com/seglo/kafka-lag-exporter) was a huge inspiration for me creating my first real Elixir project. Thank you! The initial project [Kafka Lag Exporter](https://github.com/seglo/kafka-lag-exporter) was a huge inspiration for me
creating my first real Elixir project. Thank you!

View File

@@ -32,7 +32,7 @@ services:
- redpanda_network - redpanda_network
console: console:
image: docker.redpanda.com/redpandadata/console:v2.2.4 image: docker.redpanda.com/redpandadata/console:v2.4.6
entrypoint: /bin/sh entrypoint: /bin/sh
command: -c "echo \"$$CONSOLE_CONFIG_FILE\" > /tmp/config.yml; /app/console" command: -c "echo \"$$CONSOLE_CONFIG_FILE\" > /tmp/config.yml; /app/console"
environment: environment:
@@ -63,7 +63,6 @@ services:
image: quay.io/cloudhut/owl-shop:latest image: quay.io/cloudhut/owl-shop:latest
networks: networks:
- redpanda_network - redpanda_network
#platform: 'linux/amd64'
environment: environment:
- SHOP_KAFKA_BROKERS=redpanda:29092 - SHOP_KAFKA_BROKERS=redpanda:29092
- SHOP_KAFKA_TOPICREPLICATIONFACTOR=1 - SHOP_KAFKA_TOPICREPLICATIONFACTOR=1
@@ -78,7 +77,6 @@ services:
container_name: connect container_name: connect
networks: networks:
- redpanda_network - redpanda_network
#platform: 'linux/amd64'
depends_on: depends_on:
- redpanda - redpanda
ports: ports:
@@ -109,6 +107,8 @@ services:
- '4000:4000' - '4000:4000'
environment: environment:
- KAFKA_BROKERS=redpanda:29092 - KAFKA_BROKERS=redpanda:29092
networks:
- redpanda_network
depends_on: depends_on:
- redpanda - redpanda
restart: "unless-stopped" restart: "unless-stopped"