[BUG] Dashboard not displaying 7 days of statistics after restart

[EHRETic]

What happened?

Hi,

I did find something mentioning retention period here #3089 but I don't think it's a duplicate.

I noticed the dashboard resets its statistics after a restart, which "voids" what is advertised as "Last 7 days" :

My server restarted this night because of patches and today, my statistics were "pretty low" 😊

Don't know if this can be categorized as bug or feature but if the dashboard could reload 7 days of stats in the past, that would be great and not misleading.

Thx in advance 😉

How to reproduce?

Restart the server and statistics will reset

Configuration file(s) (yaml or .env)

x-bw-api-env: &bw-api-env
  # We use an anchor to avoid repeating the same settings for both containers
  TZ: ${TZ}
  MULTISITE: ${MULTISITE}
  DB_NAME: ${DB_NAME}
  DB_USER: ${DB_USER}
  DB_PASSWORD: ${DB_PASSWORD}
  DATABASE_URI: ${DATABASE_URI}
  USE_REDIS: ${USE_REDIS}
  REDIS_HOST: ${REDIS_HOST}
  BUNKERWEB_INSTANCES: ${BUNKERWEB_INSTANCES}
  UI_HOST: ${UI_HOST}
  MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
  API_TOKEN: ${API_TOKEN}
  API_WHITELIST_IP: ${API_WHITELIST_IP}
  TOTP_ENCRYPTION_KEYS: ${TOTP_ENCRYPTION_KEYS}
  SERVER_NAME: ${SERVER_NAME}
 
services:
  bunkerweb:
    image: bunkerity/bunkerweb:${BUNKERWEB_VERSION}
    container_name: bunkerweb
    hostname: myhostname

    restart: always

    environment:
      # This will set the API settings for the BunkerWeb container
      <<: *bw-api-env

    depends_on:
      mariadb:
        condition: service_healthy
      redis:
        condition: service_healthy
      scheduler:
        condition: service_started

    ports:
      - "80:8080/tcp"
      - "443:8443/tcp"
      - "443:8443/udp"

    volumes:
      - bw-storage:/data

    networks:
      - bw-universe
      - bw-services

    logging:
      driver: json-file
      options:
        max-size: "20m"
        max-file: "10"

  scheduler:
    image: bunkerity/bunkerweb-scheduler:${BUNKERWEB_VERSION}
    container_name: bunkerweb-scheduler
    hostname: scheduler

    restart: always

    environment:
      # This will set the API settings for the BunkerWeb container
      <<: *bw-api-env

    depends_on:
      mariadb:
        condition: service_healthy
      redis:
        condition: service_healthy

    volumes:
      - bw-storage:/data

    networks:
      - bw-universe

  ui:
    image: bunkerity/bunkerweb-ui:${BUNKERWEB_VERSION}
    container_name: bunkerweb-ui
    hostname: ui

    restart: always

    environment:
      # This will set the API settings for the BunkerWeb container
      <<: *bw-api-env

    depends_on:
      mariadb:
        condition: service_healthy

    volumes:
      - bw-storage:/data

    networks:
      - bw-universe

  mariadb:
    image: mariadb:11.4

    container_name: bunkerweb-db

    restart: always

    command:
      - --max_allowed_packet=64M
      - --innodb-buffer_pool_size=256M

    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: ${DB_NAME}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASSWORD}

    volumes:
      - mariadb-data:/var/lib/mysql

    healthcheck:
      test: ["CMD", "mariadb-admin", "ping", "-h", "localhost"]
      interval: 20s
      timeout: 5s
      retries: 10

    networks:
      - bw-universe

  redis:
    image: redis:8-alpine

    container_name: bunkerweb-redis

    restart: always

    command:
      - redis-server
      - --appendonly
      - "yes"

    volumes:
      - redis-data:/data

    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 20s
      timeout: 5s
      retries: 10

    networks:
      - bw-universe

volumes:
  bw-storage:
  mariadb-data:
  redis-data:

networks:
  bw-universe:
    driver: bridge
    ipam:
      config:
        - subnet: 172.30.250.0/24

  bw-services:
    driver: bridge

Relevant log output

BunkerWeb version

1.6.13

What integration are you using?

Docker

Linux distribution (if applicable)

Almalinux 10

Removed private data

• I have removed all private data from the configuration file and the logs

Code of Conduct

• I agree to follow this project's Code of Conduct

[TheophileDiot]

Hi @EHRETic, it's a bug, and the label is the part that's wrong.

Two unrelated sources sit under one "Last 7 days" banner. The blocked request panels, the countries map, top blocked IPs, unique IPs and the blocking timeline, are built from the requests list in Redis. Those do survive a restart with your compose, you have --appendonly yes and a redis-data volume, and the list carries a 30 day TTL. One caveat that has nothing to do with restarts: that list is capped by count, not by time. METRICS_MAX_BLOCKED_REQUESTS_REDIS defaults to 10k, and on a busy site 10k blocked requests is a lot less than seven days, so the window quietly shrinks.

The status code card is neither windowed nor restart safe. It's a plain counter. We keep it per nginx worker in memory and mirror it to Redis with SET, and nothing reads it back at startup. So after a restart the in-memory counter starts at zero and the first sync overwrites the Redis copy with that zero. Redis persistence can't help you there, the value is being replaced rather than lost. That card has never been a seven day figure, it's "since the last restart", and the banner above it says otherwise.

Two things to fix on our side: restore those counters at boot instead of stomping them, and stop the window banner claiming a period the counter cards don't honour.

One thing that would help. On your screenshot, did the countries map and top blocked IPs empty out as well, or only the status code numbers? If those went too then the reports weren't coming from Redis at all, and that's a different problem worth chasing on its own.

[EHRETic]

One thing that would help. On your screenshot, did the countries map and top blocked IPs empty out as well, or only the status code numbers? If those went too then the reports weren't coming from Redis at all, and that's a different problem worth chasing on its own.

Yes indeed, they seems reset themselves too (also this night after another restart - I know I had 5000+ in the US the very first day) :

I say seems because I can't recall the top IPs, but for sure, all stats seems to have been reset, including the timeline at the bottom:

Concerning REDIS, I'll increase gradually METRICS_MAX_BLOCKED_REQUESTS_REDIS to find the "sweet spot" 😉