Operation and Updates

Starting and stopping the stack, logs, health endpoints, updates, versioning and development stack

Audience: Technical administration

Overview

For day-to-day operation you need two scripts in the project directory: start.sh and stop.sh. This chapter also describes where to find logs and service status, how to update oPBWeb and what to keep in mind on the opsi side. Run all commands in the project directory.

Starting and stopping the stack

start.sh

./start.sh [--build] [--logs] [--dev | --test] [--lang de|en]
OptionEffect
(none)starts the production stack (docker-compose.yml) with the existing images
--buildrebuilds your own images before starting
--logsfollows the logs of all containers after starting (logs -f --tail=50); Ctrl+C only ends the output
--devadds docker-compose.dev.yml (development stack, see below); only in a clone with the sources
--testadds docker-compose.test.yml: two opsi mocks as config servers Test and Production, see Test Environment
--lang de or --lang enlanguage of the messages for this run (otherwise OPBWEB_LANG from .env)
-h, --helpshows the help

Sequence: the script aborts if .env is missing („.env fehlt – bitte ./setup-opbweb.sh ausführen (im Entwickler-Checkout: ./setup-opbweb.sh –dev).“ – .env is missing, please run ./setup-opbweb.sh, in a developer checkout ./setup-opbweb.sh –dev), calls scripts/init-secrets.sh, runs docker compose up -d and shows the status with docker compose ps. Running containers whose configuration has not changed are left untouched. start.sh does not pull new third-party images; for that, see Updating .

stop.sh

./stop.sh

The script has no options. It runs docker compose down with docker-compose.yml and the overlays present (docker-compose.dev.yml, docker-compose.test.yml) and thus stops production, dev and test services alike. Containers and networks are removed; the volumes with database and object storage are kept.

Restarting individual services

docker restart opbweb-worker
docker restart opbweb-frontend   # e.g. after a certificate change

Restarting the worker interrupts running jobs. Check under Jobs beforehand whether installations are currently running.

Containers

ContainerServiceNote
opbweb-frontendnginxweb interface, manual, proxy
opbweb-apiAPIruns the migrations at startup
opbweb-workerworkerjobs, cron, opsi connection
opbweb-dbMariaDB
opbweb-redisRedis
opbweb-garageGarageobject storage
opbweb-garage-initGarage setupruns briefly on every start and exits with exit code 0
opbweb-opsi-mockdev onlysimulated opsiconfd (first opsi system)
opbweb-opsi-mock2dev onlysimulated opsiconfd (second opsi system)
opbweb-frontend-devdev onlyVite dev server
opbweb-manual-devdev onlyHugo preview of the manual
opbweb-launcher-builddev onlybuilds the launcher zips at startup and exits with exit code 0
opbweb-launcher-devdev onlyserves the launcher zips

docker compose ps shows the status (production stack); docker compose ps -a also includes exited containers such as garage-init.

Logs

docker logs -f --tail=100 opbweb-api
docker logs -f --tail=100 opbweb-worker
docker compose logs -f --tail=50          # all services
ContainerWhat you find there
opbweb-apimigrations and uvicorn start, creation of the first admin, HTTP requests (one line per request), problems with LDAP sign-in
opbweb-workerjob execution, opsi check (every 5 minutes by default), errors in connections to opsiconfd
opbweb-frontendchoice of HTTP or HTTPS, requests and nginx errors
opbweb-garage-initsetup of layout, key and bucket

LOG_LEVEL in .env controls the verbosity of api and worker (DEBUG, INFO, WARNING, ERROR; default INFO). A change takes effect with the next ./start.sh.

For LDAP sign-in, the API logs why a sign-in fails: user not found (zero hits for the search filter) and wrong password at level INFO, no group mapped to a role at level WARNING. With LOG_LEVEL=WARNING or ERROR you therefore do not see the first two cases. For example:

docker logs opbweb-api 2>&1 | grep "LDAP login"

python-opsi and the HTTP client httpx only output warnings and errors. Their debug output is enabled only by DEBUG=true; this also starts the API with hot reload and is meant for development only, see Configuration .

The complete log of every job is also available in the user interface under Jobs, see Jobs .

Checking the status

EndpointSign-inResponse
/healthnostatic healthy from nginx; only checks whether nginx responds
/api/healthno{"status":"ok","version":"…"}; the API is reachable through nginx
/api/health/detailsyes (bearer token)checks database, redis, storage, worker, opsi with ok, details and duration
/api/system/databaseyes, adminrows and size per database table
/api/system/storageyes, adminobject storage usage per area, deduplication, unused and missing blobs
/api/system/storage/objectsyes, adminobjects of an area with the package versions using them (basis of the bucket browser)
System status pageyes, all rolesthe same checks as a table, refreshed every 10 seconds

/api/health is suitable for simple monitoring:

curl -fsS https://opbweb.example.local/api/health

The detailed check evaluates as follows (the details follow the interface language):

CheckError if …
Databaseno connection; otherwise the details show the MariaDB version and schema revision
Redisno connection
Object storageno connection or bucket missing (“Bucket ‘…’ is missing”)
Workerno heartbeat (“no sign of life”) or last heartbeat older than 90 seconds
opsi servernot checked yet, no config server configured, or for at least one config server: connection failed, no admin rights, new CA certificate not yet accepted, or expiry warning

Administrators also see the database table sizes, the object storage usage and the bucket browser on the system status page, see System status .

Scheduled tasks

The worker runs these tasks itself; you do not need a cron entry on the host.

TaskScheduleEffect
Heartbeatevery 30 secondssign of life for the system status
opsi checkevery 5 minutes (default)connection, depot sync, CA comparison, expiry warning 60 days in advance
Cleanup of temporary filesdaily at 03:17 (default)deletes uploads/ and downloads/ older than 24 hours

The times of the opsi check and the cleanup are set under Settings → Worker ; they apply without a restart.

The automatic task does not remove unused package files (blobs) or build results. An administrator does that when needed with Clean up … on the System status page: first a dry run, then the confirmed deletion.

Updating

  1. Back up the database and object storage, see Backup and Restore . Database migrations cannot easily be undone.

  2. Check under Jobs that no installations are running.

  3. Get the new version, for example with git pull or with a new stack ZIP unpacked over the installation. The ZIP contains no .env, secrets/ or certs/, they are kept.

  4. Bring .env up to date:

    ./setup-opbweb.sh --updateonly
    

    The script keeps all existing values, adds new variables with their default and creates new secrets; existing secrets stay unchanged. If you want to change values at the same time, run ./setup-opbweb.sh without an option.

  5. Update the images and start the stack:

    • Locally built images:

      ./start.sh --build
      

      --build uses the Docker build cache. Security updates of the base images and the system packages do not necessarily arrive that way. For a security update, build without cache and with fresh base images, then run ./start.sh:

      docker compose build --pull --no-cache
      
    • Images from a registry: Enter the new image versions, either interactively with ./setup-opbweb.sh (questions about the image versions; the default is the previous version) or directly in BACKEND_IMAGE, FRONTEND_IMAGE and GARAGE_INIT_IMAGE in .env. --updateonly keeps the previous versions. With a versions.env next to it (stack ZIP), its versions are the defaults, for --updateonly too; unpacking a new ZIP and running --updateonly thus switches the images to its versions. Then:

      docker compose pull
      ./start.sh
      

      If you use the latest tags, docker compose pull and ./start.sh are enough.

  6. The API runs alembic upgrade head automatically at startup. Check the log afterwards:

    docker logs opbweb-api 2>&1 | head -50
    
  7. In the user interface, check the versions in the status bar (“Frontend …”, “Backend …”) and the System status page.

start.sh does not automatically pull new versions of the third-party images (MariaDB 11.4, Redis 7, Garage v2.2.0) within their tags. Example, run when needed:

docker compose pull db redis garage
./start.sh

To also update the base images of your own images (node, nginx, python), build with docker compose build --pull and then start with ./start.sh. This only applies to locally built images.

Versioning

Backend and frontend have separate version numbers following the scheme MAJOR.MINOR.PATCH (milestone, phase, bug fix). The user interface shows both in the status bar; /api/health returns the backend version.

Versions are set with set-version.sh; the affected files are listed in version.yaml (backend: backend/src/core/version.py and backend/pyproject.toml, frontend: frontend/package.json via npm version).

CallEffect
./set-version.shshows the current versions
./set-version.sh 0.14.0sets all components (with confirmation; --force without)
./set-version.sh --backend 0.13.1sets the backend only
./set-version.sh --frontend 0.13.1sets the frontend only
./set-version.sh --component <name> <version>sets one component from version.yaml
./set-version.sh --config <file>uses a different configuration file

A changed version file on the default branch triggers the build of the matching image in GitLab CI, see Build and Delivery .

opsiconfd: admin-networks

The worker calls opsiconfd with admin rights. If the option admin-networks is set in the opsiconfd configuration, the worker’s source address must be within it; otherwise opsiconfd responds with HTTP 403.

Which address opsiconfd sees depends on where oPBWeb runs (derived from Docker’s network behavior; please verify in the opsiconfd log):

  • oPBWeb on the configserver itself: the worker’s address in the Docker network opbweb_opsi-net. To show the subnet:

    docker network inspect opbweb_opsi-net --format '{{range .IPAM.Config}}{{.Subnet}}{{end}}'
    
  • oPBWeb on a different host: usually the address of the Docker host, because Docker masquerades outgoing traffic.

./stop.sh removes the Docker networks; on the next start, opbweb_opsi-net may get a different subnet. Therefore either enter the entire address range from which Docker allocates its networks, or check the subnet after every restart. The opsi manual describes the syntax of the option; opsiconfd must be restarted after the change.

Development stack

The development stack is meant for further development, not for production. In a fresh developer checkout, first create .env (from .env.example, without questions) and then start with locally built images:

./setup-opbweb.sh --dev
./start.sh --dev --build

It adds hot reload (sources as bind mounts, DEBUG=true), two simulated opsiconfd instances, a preview of the manual and the launcher zips to the production stack. API and worker run with the image opbweb-backend:dev.

WhatAddress
Vite dev server (user interface with hot reload)http://localhost:3100
Manual preview (manual-dev, via the Vite proxy)http://localhost:3100/manual/
Launcher zips (launcher-dev, via the Vite proxy)http://localhost:3100/downloads/
API with OpenAPI interfacehttp://localhost:8100/api/docs
nginx frontendport from FRONTEND_HTTP_PORT or FRONTEND_HTTPS_PORT
MariaDBport 3307
Redisport 6380
Garage S3port 3910
opsiconfd mock (opsi-mock)opsi-config.mock and depot1.mock, only in the Docker network
second opsiconfd mock (opsi-mock2)opsi-config2.mock and depot2.mock, only in the Docker network

Each mock simulates a configserver and a depot, including install, uninstall and WebDAV. opbweb-opsi-mock is the first opsi system: as long as no config server is configured, the API registers it at startup as “opsiconfd-Mock” with the stage Test (URL http://opsi-config.mock:4447, user opsiadm, password mock, without TLS verification). opbweb-opsi-mock2 is a second, independent opsi system for tests with several config servers; it is not registered automatically. To use it in the user interface, add it under Depots with + (URL http://opsi-config2.mock:4447, user opsiadm, password mock, TLS verification off).

The end-to-end tests register both mocks as config servers themselves (as “t-Mock 1” and “t-Mock 2”, if not present yet), remove them again afterwards and work only on their depots. They do not assume that the mock is the only or the default config server, and they also run when a real opsi server is configured in addition. The manual preview does not refresh by itself; reload the page after a change.