The poller crashed on every S3 or DB error (1162 restarts during the fsn1 object storage degradation on 2026-09-21). Now: - main loop catches errors per cycle, classifies them (S3 5xx/throttling, connection/timeouts, psycopg.OperationalError = transient; everything else logged with traceback) and retries with exponential backoff (5 s .. 5 min, reset after success); db_init runs until it succeeds once - explicit botocore config for the poller: connect/read timeouts, standard retries (total attempts), s3v4, path addressing; S3(cfg) without a config keeps the previous client for the Gundi path - DB connect timeout + TCP keepalives - one S3 client per cycle shared by all workers - a batch stops after a transient S3/DB error (image stays ERROR as before) - no busy loop when ready sets keep failing - [HEARTBEAT] log line with counters, heartbeat file + Docker healthcheck - tools/entrance_inventory.py: read-only listing of icu/entrance orphans - unit tests with simulated S3/DB failures Per-image transaction, status names/order, keys and paths are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
51 lines
1.8 KiB
YAML
51 lines
1.8 KiB
YAML
services:
|
|
melesicumover:
|
|
build: .
|
|
container_name: melesicumover
|
|
restart: unless-stopped
|
|
# Secrets werden aus der lokalen .env in die Container-Umgebung injiziert.
|
|
# Die .env liegt NICHT im Image und NICHT im Git (siehe .gitignore).
|
|
env_file:
|
|
- .env
|
|
environment:
|
|
CONFIG_YAML: /app/config.yaml
|
|
DEBUG: "0"
|
|
# Lebenszeichen: die Hauptschleife schreibt die Heartbeat-Datei bei jedem Zyklus
|
|
# und jedem Bild. Laenger als 15 min unveraendert = Schleife haengt (Backoff max. 5 min).
|
|
healthcheck:
|
|
test: ["CMD", "python", "-c", "import os, sys, time; sys.exit(0 if time.time() - os.path.getmtime('/tmp/melesicumover.heartbeat') < 900 else 1)"]
|
|
interval: 60s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 120s
|
|
# Kein Bind-Mount des Quellcodes mehr (der Code steckt im Image) und
|
|
# standardmaessig KEIN offener Debug-Port.
|
|
#
|
|
# Debugging nur bei Bedarf, lokal:
|
|
# 1) DEBUG oben auf "1" setzen
|
|
# 2) folgende Zeilen einkommentieren - bindet debugpy NUR an localhost:
|
|
# ports:
|
|
# - "127.0.0.1:5678:5678"
|
|
|
|
# Gundi-v2-Push-Empfaenger. Gleiches Image, eigener Prozess: startet und
|
|
# stirbt unabhaengig vom Poller. Nur an localhost gebunden - TLS und
|
|
# oeffentliche Erreichbarkeit uebernimmt nginx (icu.meles.eu).
|
|
gundi-receiver:
|
|
build: .
|
|
container_name: melesicumover-gundi
|
|
restart: unless-stopped
|
|
command: ["python", "/app/gundi_receiver.py"]
|
|
env_file:
|
|
- .env
|
|
environment:
|
|
CONFIG_YAML: /app/config.yaml
|
|
DEBUG: "0"
|
|
ports:
|
|
- "127.0.0.1:8090:8090"
|
|
healthcheck:
|
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8090/healthz', timeout=3)"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
retries: 3
|
|
start_period: 20s
|