ELK

ELK Stack

Presentation : ELK is a set of tools used for searching (Elastic Search), analyzing (LogStash) and visualizing (Kibana) data in real time.

For the Transcendence project, ELK stack serves as a centralized logging solution. Key benefits are :

• centralization: no need to individually check each container logs
• incident response: filters such as events and reasons help to quickly analyze the scale, gravity and origin of a problem

---

Installation and configuration

Each tool runs as an idependant service, and as such is parameterized in docker-compose.

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.17.0
    environment:
      - discovery.type=single-node
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ports:
      - "9200:9200"

  logstash:
    image: docker.elastic.co/logstash/logstash:7.17.0
    volumes:
      - ./elk/logstash/pipeline:/usr/share/logstash/pipeline
    ports:
      - "5000:5000/tcp"
      - "5000:5000/udp"
    depends_on:
      - elasticsearch

  kibana:
    image: docker.elastic.co/kibana/kibana:7.17.0
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

Configuration

Logstash

input {
  tcp {
    port => 5000
    codec => json_lines
  }
}

output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
    index => "logs-%{+YYYY.MM.dd}"
  }
  stdout { codec => rubydebug }
}

Use cases

Project specific metrics

• specific dashboard in Kibana with : "Number of Pong Games Played", "Active Users" (logged in last 30 days), ...

Do's & Don'ts

✅ Do	❌ Don't
Structure logs in JSON with project log contexts	Log sensitive data (passwords, tokens, PII)
Configure policies to delete old logs	Expose ports publicly
Filter noise : drop debug logs with Logstash filters	Ignore ELK health status

Resources

Type	Ressource	Notes
📄	Official Docs Elastic	-
🎥	ELK Overview	Not checked
🎥	ELK Tutorial	Not checked
🎥	Node.js ms : Monitoring and logging	Not checked

Legend: 📄 Doc, 📘 Book, 🎥 Video, 💻 GitHub, 📦 Package, 💡 Blog