Technical Administration

• Architecture
• Configuration
• LDAP Sign-in
• Operation and Updates
• Backup and Restore
• Troubleshooting
• Build and Delivery

Architecture

Audience: Technical administration

Overview

oPBWeb runs as a single Docker Compose project named opbweb. The file docker-compose.yml describes production operation; docker-compose.dev.yml extends it for development. This chapter shows which services exist, how they are connected and where each kind of data is stored.

Services

API and worker use the same image; the worker just starts with a different command (./start-worker.sh). The garage-init image carries the :dev tag in production as well.

All services except garage-init run with restart: unless-stopped and start automatically after a host reboot.

Startup order

1. db, redis and garage start and report readiness through their health checks.
2. garage-init waits for the Garage admin API, creates missing resources and writes the S3 credentials to the volume garage-credentials.
3. api starts once db and redis are healthy and garage-init has exited successfully. It waits up to 60 seconds for the database, runs alembic upgrade head and creates the first administrator account as long as the user table is empty.
4. worker starts after the API, writes a first heartbeat and checks the connection to all config servers.
5. frontend starts after the API and chooses HTTPS or HTTP depending on the certificates.

Networks

Only the two frontend ports (FRONTEND_HTTP_PORT, FRONTEND_HTTPS_PORT) are reachable from outside. Database, Redis and Garage have no published ports in production. The API has no route to the outside; only the worker reaches the opsi config servers and their depots via opsi-net (HTTPS, port 4447). The actual Docker networks carry the project prefix, for example opbweb_opsi-net.

Data flow

 Browser
    │  HTTPS 443 (or HTTP 80)
    ▼
 ┌──────────────────────── frontend (nginx) ────────────────────────┐
 │  /           web interface (Vue SPA)                             │
 │  /manual/    manual in German, /manual/en/ in English            │
 │  /health     static response "healthy"                           │
 │  /api        proxy incl. WebSocket ──────────────┐               │
 └──────────────────────────────────────────────────┼───────────────┘
                                   frontend-net     ▼
                                          ┌──── api :8000 ────┐
                                          │ sign-in, REST,    │
                                          │ job events (WS)   │
                                          └───┬──────┬─────┬──┘
                       backend-net (internal) │      │     │ enqueue job
                    ┌─────────────────────────┘      │     ▼
                    ▼                                ▼   redis ◄──────┐
                 db (MariaDB)            garage (S3, bucket packages) │
                    ▲                                ▲                │
                    └──────────────┬─────────────────┘                │
                                   │                                  │
                              ┌── worker ──┐ ─────────────────────────┘
                              │ arq jobs,  │   fetches jobs, heartbeat
                              │ cron       │
                              └─────┬──────┘
                                    │ opsi-net, HTTPS :4447
                                    ▼
                    opsi configserver and depots (opsiconfd)
                    JSON-RPC and WebDAV /repository, /workbench, /depot

Typical flows:

• Editing in the user interface: browser → nginx → API → database and object storage. The worker is not involved.
• Long-running operations (import, build, install, depot sync, workbench, file manager copies): the API creates a job in the database and enqueues it in Redis. The worker runs it and writes progress and log. The browser follows the job live via WebSocket at /api/jobs/{id}/events.
• File manager on the depots: the API passes single file operations (list, read, save) directly to the worker and waits up to 60 seconds for the answer. If the worker does not respond, “Worker not reachable.” or “The worker does not respond.” appears.
• Uploads: nginx accepts uploads up to 20 GB and passes them to the API unbuffered. The API stores them in the object storage under uploads/; a job processes them.
• Downloads: the API serves built packages from the object storage through short-lived download links (valid for 2 minutes). Files from a depot are first copied to downloads/ by a job.

Data storage

Database

MariaDB (database opbweb, character set utf8mb4) holds all metadata: users, audit log, products and versions with control data, properties, dependencies, the file list of each version with SHA-256 checksum, build results, jobs with logs, snippets, application settings, the config servers with their connection data and stages, and the depots, each of which is assigned to exactly one config server. The password of the technical opsi user is stored there encrypted for each config server (RSA-OAEP): the API encrypts it with secrets/opsi_key_public.pem, only the worker can decrypt it with secrets/opsi_key_private.pem.

Object storage

Garage stores all file contents in the bucket packages. Objects are organized by prefix:

Because of deduplication, deleting packages or versions leaves blobs behind that no package file uses any more. An administrator removes them deliberately with Clean up … on the System status page.

Physically, the data lives in two Docker volumes: garage-meta (metadata in LMDB, keys, bucket) and garage-data (data blocks). The prefixes above are object keys in the bucket, not directories in the volume. The layout capacity of 100 GB set by garage-init is a weighting for Garage, not a quota; what counts is the free space on the host.

Volumes

Docker creates the volumes with the prefix opbweb_, for example opbweb_mariadb-data. Which of them to back up is described in Backup and Restore .

Worker and jobs

The worker processes the jobs from Redis:

• up to 10 jobs at the same time, so that short file manager calls do not wait behind long installations,
• time limit for jobs: 8 hours (JOB_TIMEOUT_SECONDS), 3600 seconds for the other worker functions,
• jobs for the same depot run one after another. A depot lock in Redis ensures this; a job waits up to 6 hours for it and otherwise aborts with “Depot has been busy with another job for too long.”

The worker also runs scheduled tasks:

The time of day follows TZ (default Europe/Berlin). The automatic cleanup does not delete blobs or build results.

Manual

This manual is part of the frontend image. During the build, a dedicated Hugo stage generates the static website, and nginx serves it:

• German at /manual/,
• English at /manual/en/.

The book icon in the top bar opens the manual in the language of the user interface in a new tab. The manual is static and reachable without signing in. Changes to the manual appear only with a newly built frontend image, see Build and Delivery .

Configuration

Audience: Technical administration

Overview

oPBWeb splits its configuration into three levels:

• .env in the project directory: non-secret settings such as ports, LDAP server, time limits. Docker Compose reads the file and inserts the values into docker-compose.yml.
• secrets/: all passwords and keys as individual files. They reach the containers as Docker secrets and are never stored in .env.
• User interface: the connections to the opsi config servers (URL, technical user, password, stage, CA certificate; in the Depots view) and the defaults for new packages. These settings are stored in the database, see Config servers and Settings for new packages .

./setup-opbweb.sh creates .env and the missing secrets, see Installation . Afterwards you can edit .env directly or run the script again; the existing values are then the defaults. Changes to .env take effect with the next ./start.sh; Docker Compose then recreates the affected containers.

Variables in .env

The template .env.example contains all variables of the following tables; commented-out lines show the default. If a variable is missing from .env, its default applies. setup-opbweb.sh writes the variables for database, Redis, object storage and CORS_ORIGINS only if they were already in the previous .env; variables the script does not know are kept at the end of the file.

General and sign-in

opsi and packages

BUILD_COMPRESSION only affects new builds. Packages that are already built keep their compression until they are built again.

Frontend and images

If the three image variables are empty, the locally built images apply, see Image overrides and registry mirror .

Database, Redis, object storage

You normally do not need to set these variables. They exist for special cases, such as external services instead of the bundled containers; .env.example lists them commented out.

Secrets

scripts/init-secrets.sh creates missing secrets in the directory secrets/. setup-opbweb.sh and start.sh call the script, start.sh on every start. It never overwrites existing files. Do not share the contents of secrets in tickets or logs.

The secrets/ directory has permissions 750, the files 444; the script sets them again on every run. Docker passes the secret files into the containers with the host permissions, and the processes there run as other users (MariaDB, Redis, the application as appuser with UID 1000). With 600 they could not read the files on a Linux host. The directory with 750 keeps other users of the host out.

The S3 credentials are not in secrets/. garage-init generates the key in Garage and writes it to the volume garage-credentials.

Changing secrets later

Not every secret can simply be replaced after the first start:

The files are read-only (444). To change one, briefly remove the protection and set it again afterwards, for example:

chmod u+w secrets/ldap_bind_password.txt
# edit the file
chmod 444 secrets/ldap_bind_password.txt

You can also enter the LDAP password again with ./setup-opbweb.sh. The services read the secrets at startup. After a change, restart the stack (./stop.sh, then ./start.sh).

TLS and COOKIE_SECURE

On every start, the frontend container decides based on the files in the directory certs/:

Outside the development stack, setup-opbweb.sh always provides a certificate: supplied (fullchain.pem and key.pem next to the script), existing or self-signed, see Installation . The container log shows the choice: „TLS-Zertifikat gefunden – HTTPS auf Port 443“ (TLS certificate found – HTTPS on port 443) or „Kein TLS-Zertifikat – HTTP auf Port 80“ (no TLS certificate – HTTP on port 80). fullchain.pem contains the server certificate with all intermediate certificates. The container gets the target port of the redirect as HTTPS_PORT (= FRONTEND_HTTPS_PORT).

With HTTPS, nginx sends the header Strict-Transport-Security (one year). Browsers that have opened oPBWeb via HTTPS once will refuse HTTP for this host name afterwards.

After replacing a certificate (put the new files next to the script and run ./setup-opbweb.sh --updateonly, or replace them directly in certs/), restarting the frontend is enough:

docker restart opbweb-frontend

In addition, nginx lets at most 10 sign-in attempts per minute and address through (briefly 10 more). Both limits require oPBWeb to see the real client address; if another proxy sits in front or Docker hides the addresses, all users share one address.

COOKIE_SECURE controls the Secure flag of the refresh cookie opbweb_refresh (valid for 7 days, httpOnly, SameSite=Strict). If the sign-in arrives over HTTPS, the API always sets the flag, even with false. For the setting:

Ports

Both ports are always published, but only one of them responds. All other services are not reachable from outside in production. The additional ports of the development stack are listed under Operation and Updates .

Outbound, the worker needs HTTPS on port 4447 to the opsi configserver and the depots.

Time limits and sizes

Image overrides and registry mirror

Without overrides, ./start.sh --build builds your own images locally and names them opbweb-backend:latest, opbweb-frontend:latest and opbweb-garage-init:1.0.0. With BACKEND_IMAGE, FRONTEND_IMAGE and GARAGE_INIT_IMAGE you use prebuilt images instead, for example from the GitLab registry (see Build and Delivery ). setup-opbweb.sh sets the three variables when you choose r at the images question; the prefix comes from OPBWEB_REGISTRY. Done manually, it looks like this (example, adapt it to your registry):

BACKEND_IMAGE=registry.example.com/gruppe/opsipackagebuilder-web/backend:<version>
FRONTEND_IMAGE=registry.example.com/gruppe/opsipackagebuilder-web/frontend:<version>
GARAGE_INIT_IMAGE=registry.example.com/gruppe/opsipackagebuilder-web/garage-init:latest

Then pull the images and start without --build:

docker login registry.example.com
docker compose pull
./start.sh

LDAP Sign-in

Audience: Technical administration

Overview

In addition to local accounts, oPBWeb can sign users in against an LDAP directory or Active Directory. The role in oPBWeb is then derived from the group memberships in the directory. You configure the connection with LDAP_* variables in .env and the password of the service account in secrets/ldap_bind_password.txt. There are no settings for this in the user interface.

How sign-in works

1. If oPBWeb has a local account with the entered user name, oPBWeb signs in locally only. The directory is not queried, even if the password is wrong.
2. Otherwise, with LDAP_ENABLED=true, oPBWeb connects with the service account (LDAP_BIND_DN), or anonymously if no bind DN is set.
3. oPBWeb searches with LDAP_USER_FILTER below LDAP_USER_SEARCH_BASE (empty: LDAP_BASE_DN). The search must return exactly one entry.
4. oPBWeb binds as the user with the DN it found and the entered password, which verifies the password.
5. oPBWeb determines the role from the memberOf attribute (see below). Without a matching group, there is no access.
6. On the first successful sign-in, oPBWeb creates the account with the source Directory (LDAP). Display name (displayName), email (mail) and role are taken from the directory again on every sign-in.

Local accounts therefore always stay local, including the first administrator account. This way you can still get into oPBWeb if the directory is down.

Variables

The service account password is not in .env but in secrets/ldap_bind_password.txt (created empty by init-secrets.sh). The connection has a time limit of 10 seconds for setup and 15 seconds for responses.

Active Directory example

An example for the domain example.local. Replace all names with your values:

LDAP_ENABLED=true
LDAP_HOST=dc01.example.local
LDAP_PORT=636
LDAP_USE_SSL=true
LDAP_TLS_VERIFY=true
LDAP_CA_CERTS_FILE=/run/certs/ldap/ca.pem
LDAP_BASE_DN=dc=example,dc=local
LDAP_USER_SEARCH_BASE=ou=Benutzer,dc=example,dc=local
LDAP_USER_FILTER=(&(objectClass=user)(sAMAccountName={username}))
LDAP_BIND_DN=cn=svc-opbweb,ou=Dienstkonten,dc=example,dc=local
LDAP_ADMIN_GROUPS=cn=opbweb-admins,ou=Gruppen,dc=example,dc=local
LDAP_RELEASER_GROUPS=cn=opbweb-releaser,ou=Gruppen,dc=example,dc=local
LDAP_PACKAGER_GROUPS=cn=opbweb-packager,ou=Gruppen,dc=example,dc=local;cn=client-team,ou=Gruppen,dc=example,dc=local
LDAP_VIEWER_GROUPS=cn=opbweb-leser,ou=Gruppen,dc=example,dc=local

More filters, adapt them to your directory:

Setup

1. Enter the variables in .env as shown above.

2. Write the service account password to secrets/ldap_bind_password.txt (without a trailing line break is safest; oPBWeb strips leading and trailing whitespace).

3. If you use your own company CA, place the CA certificate in PEM format as ldap-ca.pem next to setup-opbweb.sh before the setup (the setup moves it to certs/ldap/ldap-ca.pem and proposes it as CA file) or directly in certs/ldap/. The directory certs/ldap/ is mounted read-only in the container as /run/certs/ldap/; LDAP_CA_CERTS_FILE specifies the path inside the container. Without a CA file, oPBWeb verifies against the container’s default CAs.

4. Restart the stack so that the API reads the new values and the secret:

   ./stop.sh
   ./start.sh
   

5. Sign in with a directory account. Under Users and roles, the account appears with the source Directory (LDAP) and the assigned role.

Groups and roles

• Each LDAP_*_GROUPS variable contains one or more full group DNs, separated by ;.
• oPBWeb compares the DNs case-insensitively, but otherwise exactly with the values from memberOf.
• The highest role wins: Administrator over Releaser over Packager over Viewer.
• No matching group, no access: sign-in fails with “No permission for oPBWeb (no matching group).” If none of the four variables is set, nobody can sign in via LDAP.
• oPBWeb does not resolve nested groups; only what memberOf returns directly counts. The primary group of an AD account (usually “Domain Users”) is not included in memberOf.

What each role is allowed to do is described in Users and roles .

Behavior of directory accounts

Accounts with the same name: if a local account has the same name as a directory account, the local account always applies.

Troubleshooting

The API writes connection and search errors to its log:

docker logs opbweb-api 2>&1 | grep -i "LDAP search failed"

Typical causes are a host name that does not match the certificate, a missing or wrong CA file (path inside the container, not on the host), an empty secrets/ldap_bind_password.txt, or a domain controller that cannot be reached from the Docker host.

To cross-check the search, use ldapsearch on a machine with the OpenLDAP tools. Example, adapt it to your values:

ldapsearch -H ldaps://dc01.example.local:636 \
  -D "cn=svc-opbweb,ou=Dienstkonten,dc=example,dc=local" -W \
  -b "ou=Benutzer,dc=example,dc=local" \
  "(&(objectClass=user)(sAMAccountName=mmuster))" memberOf displayName mail

You can copy the memberOf values from the output directly into the LDAP_*_GROUPS variables.

Operation and Updates

Audience: Technical administration

Overview

For day-to-day operation you need two scripts in the project directory: start.sh and stop.sh. This chapter also describes where to find logs and service status, how to update oPBWeb and what to keep in mind on the opsi side. Run all commands in the project directory.

Starting and stopping the stack

start.sh

./start.sh [--build] [--logs] [--dev | --test] [--lang de|en]

Sequence: the script aborts if .env is missing („.env fehlt – bitte ./setup-opbweb.sh ausführen (im Entwickler-Checkout: ./setup-opbweb.sh –dev).“ – .env is missing, please run ./setup-opbweb.sh, in a developer checkout ./setup-opbweb.sh –dev), calls scripts/init-secrets.sh, runs docker compose up -d and shows the status with docker compose ps. Running containers whose configuration has not changed are left untouched. start.sh does not pull new third-party images; for that, see Updating .

stop.sh

./stop.sh

The script has no options. It runs docker compose down with docker-compose.yml and the overlays present (docker-compose.dev.yml, docker-compose.test.yml) and thus stops production, dev and test services alike. Containers and networks are removed; the volumes with database and object storage are kept.

Restarting individual services

docker restart opbweb-worker
docker restart opbweb-frontend   # e.g. after a certificate change

Restarting the worker interrupts running jobs. Check under Jobs beforehand whether installations are currently running.

Containers

docker compose ps shows the status (production stack); docker compose ps -a also includes exited containers such as garage-init.

Logs

docker logs -f --tail=100 opbweb-api
docker logs -f --tail=100 opbweb-worker
docker compose logs -f --tail=50          # all services

LOG_LEVEL in .env controls the verbosity of api and worker (DEBUG, INFO, WARNING, ERROR; default INFO). A change takes effect with the next ./start.sh.

For LDAP sign-in, the API logs why a sign-in fails: user not found (zero hits for the search filter) and wrong password at level INFO, no group mapped to a role at level WARNING. With LOG_LEVEL=WARNING or ERROR you therefore do not see the first two cases. For example:

docker logs opbweb-api 2>&1 | grep "LDAP login"

python-opsi and the HTTP client httpx only output warnings and errors. Their debug output is enabled only by DEBUG=true; this also starts the API with hot reload and is meant for development only, see Configuration .

The complete log of every job is also available in the user interface under Jobs, see Jobs .

Checking the status

/api/health is suitable for simple monitoring:

curl -fsS https://opbweb.example.local/api/health

The detailed check evaluates as follows (the details follow the interface language):

Administrators also see the database table sizes, the object storage usage and the bucket browser on the system status page, see System status .

Scheduled tasks

The worker runs these tasks itself; you do not need a cron entry on the host.

The times of the opsi check and the cleanup are set under Settings → Worker ; they apply without a restart.

The automatic task does not remove unused package files (blobs) or build results. An administrator does that when needed with Clean up … on the System status page: first a dry run, then the confirmed deletion.

Updating

1. Back up the database and object storage, see Backup and Restore . Database migrations cannot easily be undone.

2. Check under Jobs that no installations are running.

3. Get the new version, for example with git pull or with a new stack ZIP unpacked over the installation. The ZIP contains no .env, secrets/ or certs/, they are kept.

4. Bring .env up to date:

   ./setup-opbweb.sh --updateonly
   

   The script keeps all existing values, adds new variables with their default and creates new secrets; existing secrets stay unchanged. If you want to change values at the same time, run ./setup-opbweb.sh without an option.

5. Update the images and start the stack:

   • Locally built images:

     ./start.sh --build
     

     --build uses the Docker build cache. Security updates of the base images and the system packages do not necessarily arrive that way. For a security update, build without cache and with fresh base images, then run ./start.sh:

     docker compose build --pull --no-cache
     

   • Images from a registry: Enter the new image versions, either interactively with ./setup-opbweb.sh (questions about the image versions; the default is the previous version) or directly in BACKEND_IMAGE, FRONTEND_IMAGE and GARAGE_INIT_IMAGE in .env. --updateonly keeps the previous versions. With a versions.env next to it (stack ZIP), its versions are the defaults, for --updateonly too; unpacking a new ZIP and running --updateonly thus switches the images to its versions. Then:

     docker compose pull
     ./start.sh
     

     If you use the latest tags, docker compose pull and ./start.sh are enough.

6. The API runs alembic upgrade head automatically at startup. Check the log afterwards:

   docker logs opbweb-api 2>&1 | head -50
   

7. In the user interface, check the versions in the status bar (“Frontend …”, “Backend …”) and the System status page.

start.sh does not automatically pull new versions of the third-party images (MariaDB 11.4, Redis 7, Garage v2.2.0) within their tags. Example, run when needed:

docker compose pull db redis garage
./start.sh

To also update the base images of your own images (node, nginx, python), build with docker compose build --pull and then start with ./start.sh. This only applies to locally built images.

Versioning

Backend and frontend have separate version numbers following the scheme MAJOR.MINOR.PATCH (milestone, phase, bug fix). The user interface shows both in the status bar; /api/health returns the backend version.

Versions are set with set-version.sh; the affected files are listed in version.yaml (backend: backend/src/core/version.py and backend/pyproject.toml, frontend: frontend/package.json via npm version).

A changed version file on the default branch triggers the build of the matching image in GitLab CI, see Build and Delivery .

opsiconfd: admin-networks

The worker calls opsiconfd with admin rights. If the option admin-networks is set in the opsiconfd configuration, the worker’s source address must be within it; otherwise opsiconfd responds with HTTP 403.

Which address opsiconfd sees depends on where oPBWeb runs (derived from Docker’s network behavior; please verify in the opsiconfd log):

• oPBWeb on the configserver itself: the worker’s address in the Docker network opbweb_opsi-net. To show the subnet:

  docker network inspect opbweb_opsi-net --format '{{range .IPAM.Config}}{{.Subnet}}{{end}}'
  

• oPBWeb on a different host: usually the address of the Docker host, because Docker masquerades outgoing traffic.

./stop.sh removes the Docker networks; on the next start, opbweb_opsi-net may get a different subnet. Therefore either enter the entire address range from which Docker allocates its networks, or check the subnet after every restart. The opsi manual describes the syntax of the option; opsiconfd must be restarted after the change.

Development stack

The development stack is meant for further development, not for production. In a fresh developer checkout, first create .env (from .env.example, without questions) and then start with locally built images:

./setup-opbweb.sh --dev
./start.sh --dev --build

It adds hot reload (sources as bind mounts, DEBUG=true), two simulated opsiconfd instances, a preview of the manual and the launcher zips to the production stack. API and worker run with the image opbweb-backend:dev.

Each mock simulates a configserver and a depot, including install, uninstall and WebDAV. opbweb-opsi-mock is the first opsi system: as long as no config server is configured, the API registers it at startup as “opsiconfd-Mock” with the stage Test (URL http://opsi-config.mock:4447, user opsiadm, password mock, without TLS verification). opbweb-opsi-mock2 is a second, independent opsi system for tests with several config servers; it is not registered automatically. To use it in the user interface, add it under Depots with + (URL http://opsi-config2.mock:4447, user opsiadm, password mock, TLS verification off).

The end-to-end tests register both mocks as config servers themselves (as “t-Mock 1” and “t-Mock 2”, if not present yet), remove them again afterwards and work only on their depots. They do not assume that the mock is the only or the default config server, and they also run when a real opsi server is configured in addition. The manual preview does not refresh by itself; reload the page after a change.

Backup and Restore

Audience: Technical administration

Overview

oPBWeb’s data lives in two places that belong together: the database holds all metadata, the object storage holds the file contents. The database refers to objects in the storage via SHA-256 checksums and keys (blobs/…, artifacts/…). A backup is only usable if both parts come from the same point in time and the matching secrets are included.

What to back up

Also note the oPBWeb version (Git commit or the versions from the status bar). A database can only be used with the same or a newer oPBWeb version.

Creating a backup

To make sure database and object storage have the same state, first stop the services that change data, then back up the database, and then back up the object storage with the stack stopped.

1. Check under Jobs that no jobs are running or waiting.

2. Create a protected backup directory:

   mkdir -p -m 700 backup
   

3. Stop frontend, worker and API. Database and object storage keep running:

   docker stop opbweb-frontend opbweb-worker opbweb-api
   

4. Back up the database:

   docker exec opbweb-db sh -c 'mariadb-dump -uroot -p"$(cat /run/secrets/db_root_password)" --single-transaction opbweb' > backup/opbweb.sql
   

   The command reads the root password inside the container from the Docker secret; it does not appear on the host’s command line.

5. Stop the stack:

   ./stop.sh
   

6. Back up the two Garage volumes (example with an Alpine helper container):

   docker run --rm -v opbweb_garage-meta:/v:ro -v "$PWD/backup":/b alpine tar czf /b/garage-meta.tgz -C /v .
   docker run --rm -v opbweb_garage-data:/v:ro -v "$PWD/backup":/b alpine tar czf /b/garage-data.tgz -C /v .
   

7. Back up secrets, configuration and certificates:

   tar czf backup/config.tgz secrets .env certs
   

8. Start the stack again:

   ./start.sh
   

Then transfer the contents of backup/ to your backup target. The downtime depends mainly on the size of the object storage.

If a short interruption of the whole stack is acceptable anyway, you can back up the volume opbweb_mariadb-data with the stack stopped, just like the Garage volumes, instead of creating a dump. In that case, the passwords from secrets/db_password.txt and secrets/db_root_password.txt, which are stored in the volume, must be kept together with the database.

Restoring

The order matters: first secrets and configuration, then the object storage, then the database, and finally the start.

1. Stop the stack.

   ./stop.sh
   

2. Provide the program version. Check out the same oPBWeb version that belongs to the backup (or a newer one). If you use images from a registry, their versions are in the backed-up .env (BACKEND_IMAGE etc.).

3. Restore secrets, .env and certificates. Use the original files from the backup, not newly generated ones:

   tar xzf backup/config.tgz
   chmod 750 secrets && chmod 444 secrets/*
   

   The files need permissions 444 so that the processes in the containers can read them; start.sh sets the permissions via init-secrets.sh on every start anyway.

4. Restore the object storage. Remove the existing Garage volumes including the S3 credentials and restore the backup:

   docker volume rm opbweb_garage-meta opbweb_garage-data opbweb_garage-credentials
   docker volume create opbweb_garage-meta
   docker volume create opbweb_garage-data
   docker run --rm -v opbweb_garage-meta:/v -v "$PWD/backup":/b alpine tar xzf /b/garage-meta.tgz -C /v
   docker run --rm -v opbweb_garage-data:/v -v "$PWD/backup":/b alpine tar xzf /b/garage-data.tgz -C /v
   

   You must remove the volume garage-credentials so that garage-init rewrites the credentials from the restored key. If an old volume remains, the credentials do not match the object storage. On the next start, the garage-init log shows “Credential files missing, writing from existing key…”.

5. Restore the database. Remove the database volume, start only the database and import the dump:

   docker volume rm opbweb_mariadb-data
   docker compose up -d db
   docker compose ps db        # wait until the status is "healthy"
   docker exec -i opbweb-db sh -c 'mariadb -uroot -p"$(cat /run/secrets/db_root_password)" opbweb' < backup/opbweb.sql
   

   On the first start with the empty volume, MariaDB creates the database opbweb and the application user with the passwords from secrets/. If you backed up the volume instead of a dump, restore it like the Garage volumes.

6. Start the stack.

   ./start.sh
   

   The API runs any missing migrations if the program version is newer than the backup. It does not create a first administrator account, because the user table from the backup is populated.

7. Verify.

   • System status: all services OK. As an administrator, check under Object storage that no missing blobs are reported; missing blobs indicate that database and object storage come from different points in time.
   • Depots → config server: Test connection for each config server. If sign-in fails, opsi_key_private.pem does not match the database; enter the password of the technical user again.
   • Open a package and download a file or a built package.

Jobs that were waiting or running at the time of the backup are not resumed by oPBWeb after the restore. Start them again if needed.

Consistency

For regular backups, a fixed order in a script that runs all steps of the section Creating a backup one after another helps.

Troubleshooting

Audience: Technical administration

Overview

This chapter sorts typical problems by symptom. For each symptom you find the likely cause and the fix. At the end there is a table of important error messages. Messages of the server follow the interface language; this chapter quotes them as shown in the English interface. Messages of the shell scripts and container logs are German and are quoted with an English translation.

First places to look

Stack does not start

Sign-in

“Wrong user name or password.” For local accounts, every failed attempt counts. After 5 failed attempts from the same address the account is locked for that address for 15 minutes: “Account locked until HH:MM.” The user can still sign in from other addresses; the account is locked for everyone only after 50 failed attempts. After 20 failed attempts from one address (across all accounts) oPBWeb answers “Too many failed sign-ins from this address.” An administrator removes the lock under Users and roles with Unlock; Set password also unlocks the account. See Users and roles .

Password of the only administrator forgotten. There is no tool for resetting it without signing in. Another administrator (local or via an LDAP group) can set the password. So create a second administrator account early on. secrets/admin_password.txt only applies on the very first start.

“Account is disabled.” The account is deactivated in oPBWeb. An administrator reactivates it under Users and roles.

LDAP messages (“Directory service not reachable.”, “No permission for oPBWeb (no matching group).”): see LDAP Sign-in .

Signed out again after reloading the page or after about 15 minutes. The session is renewed via the cookie opbweb_refresh. If COOKIE_SECURE=true is set while oPBWeb is only reachable via HTTP, the browser does not store the cookie. Set COOKIE_SECURE to match the protocol and restart with ./start.sh.

“Session expired.” The token is invalid or was revoked, for example after signing out, after 7 days without renewal, or after secrets/jwt_secret_key.txt was changed. Sign in again.

Connection to the opsi server

The opsi server row on the system status page shows the status, with one entry per config server; in addition, there is the colored dot in front of each config server in the Depots view. The worker checks all config servers every 5 minutes; after a change you can check immediately under Depots → config server with Test connection. The setup is described in Config servers .

Check reachability from the worker container (example; -k only for this test, because the opsi CA is not known inside the container):

docker exec opbweb-worker curl -sk -o /dev/null -w '%{http_code}\n' https://opsi.example.local:4447/

An HTTP status number shows that name resolution and connection work. An error such as “Could not resolve host” or a timeout points to DNS or the firewall.

Building

Build errors are shown in the log of the Build job under Jobs.

Installing and uninstalling

Worker not reachable

Symptoms: On the system status page, the Worker row reports “no sign of life” or “last sign of life N s ago”; jobs stay in the waiting state; the file manager under Depots reports “Worker not reachable.” or “The worker does not respond.”

Causes and fix:

1. Check whether the container is running: docker compose ps worker.
2. Read the log: docker logs --tail=200 opbweb-worker. Common causes are missing secrets, no connection to Redis, or an error at startup.
3. Restart the worker: docker restart opbweb-worker.
4. “The worker does not respond.” can also mean that the worker is busy. It runs at most 10 jobs at the same time; file manager calls wait at most 60 seconds. Wait for running jobs to finish and try again.

Object storage

Uploads and time limits

The worker deletes uploaded files that were not processed after 24 hours.

HTTPS and certificates

Important error messages

Build and Delivery

Audience: Technical administration

Overview

oPBWeb consists of three images of its own: backend (for api and worker), frontend (nginx with user interface and manual) and garage-init. You either build them locally with ./start.sh --build or let GitLab CI build them and push them to the project registry. The CI also generates the manual as PDF. This chapter describes the images, the pipeline and how to build the manual locally.

Images

All Dockerfiles use the project directory as build context. Among other things, .dockerignore keeps secrets/, certs/, .env, node_modules/ as well as the Hugo output and manual/.doc-state/ out of the context.

Backend

backend/Dockerfile, one image for api and worker:

Locally, this produces opbweb-backend:latest (production) or opbweb-backend:dev (dev stack).

Frontend

frontend/Dockerfile:

The manual is therefore an integral part of the frontend image. A change to the manual appears in the web interface only with a newly built frontend image.

garage-init

garage/Dockerfile.garage-init: alpine:3.21 with bash, jq and curl plus the script garage/init-garage.sh. Locally, the image is called opbweb-garage-init:1.0.0. The version is fixed and only changes when the Dockerfile or the script changes.

Network access during the build

For a build, the host needs access to Docker Hub (base images), GitHub (python-opsi, Hugo modules), PyPI, the npm registry and the Go module proxy proxy.golang.org (Docsy). The launcher needs no third-party Go modules.

GitLab CI pipeline

The pipeline is defined in .gitlab-ci.yml. The jobs run on runners with the tag dind and build with Docker-in-Docker.

Stages and jobs

The pipeline reads the versions from backend/src/core/version.py and frontend/package.json. The images end up in the project registry under <registry>/<project path>/backend, …/frontend and …/garage-init (path in lowercase). The registry address is set as CI_REGISTRY in .gitlab-ci.yml.

Stack ZIP

The job stack-release packs everything a Docker host needs to set up and start with the registry images into oPBWeb-Stack-<pipeline no>.zip, without sources:

docker-compose.dev.yml is left out on purpose: the development stack mounts the sources. Without sources, setup-opbweb.sh offers no local build and start.sh --dev stops with a message. The ZIP is stored in the generic package stack/<pipeline no> of the project registry and linked from the release oPBWeb Stack (tag stack-latest), which every run recreates.

Using the images from the registry

The target host then does not need to build the images. In ./setup-opbweb.sh, choose r at the images question and enter the registry prefix (default: the registry of this CI, a different one via OPBWEB_REGISTRY) and the versions you want. The script writes BACKEND_IMAGE, FRONTEND_IMAGE and GARAGE_INIT_IMAGE to .env. Then pull the images (sign in with docker login first if the registry requires it) with docker compose pull and start with ./start.sh without --build. Details are in Installation and Configuration .

When each job runs

On the default branch of the project (set in GitLab under Settings → Repository, e.g. master or main), the jobs start automatically when their triggers change:

A new backend or frontend image is therefore typically created by setting a new version with set-version.sh, see Operation and Updates . A change to the manual alone produces new PDFs but no new frontend image; for that you need a new frontend version or a manual run of build-frontend.

Starting the pipeline manually

When you start a pipeline manually in GitLab, you choose what should run with the input run_job:

The scans check the image with the current version number from the registry. It must therefore have been built and pushed beforehand.

Container scanning

The scan jobs use the GitLab template Jobs/Container-Scanning.gitlab-ci.yml with the threshold HIGH. The reports are available as job artifacts (gl-container-scanning-report-backend.json, gl-container-scanning-report-frontend.json) and appear in GitLab’s security reports.

Manual PDFs

The job build-manuals runs in the image python:3, installs WeasyPrint, Hugo extended version 0.154.5 (variable HUGO_VERSION), Go, Node.js and the GitLab release-cli, and then calls manual/build-pdf.sh output. It produces one PDF file per part and language:

<Version> is the frontend version. The job uploads the PDFs to the generic package manuals/latest in the project registry; it deletes files with the same name first. It then recreates the release manuals-latest („oPBWeb Handbücher“ – oPBWeb manuals) with one link per PDF („DE: Installation“, „EN: User Guide“ and so on) and the build date, pipeline number and frontend version in the description. In addition, the PDFs are kept as job artifacts for 30 days.

Building the manual locally

The sources are in manual/ (Hugo with the Docsy theme as a Hugo module). German texts are in manual/content/de/, English texts in manual/content/en/, images under manual/static/images/<language>/. You do not need to install anything on the host; all methods run in containers.

Preview in the dev stack

The development stack starts the service manual-dev (hugo server in the image hugomods/hugo:exts-0.154.5). You reach the preview via the Vite proxy:

http://localhost:3100/manual/

Live reload is disabled; reload the page after a change. If there are problems, docker logs opbweb-manual-dev helps. The dev stack is described in Operation and Updates .

Building the website as in the image

The frontend image builds the manual itself. To check it without building an image, you can run the same command in a container (example):

docker run --rm -v "$PWD/manual":/src -w /src hugomods/hugo:exts-0.154.5 \
  sh -c 'npm ci --no-audit --no-fund && hugo --minify --baseURL /manual/'

The result is in manual/public/. The directory is excluded from Git and from the Docker build.

Building the PDFs

manual/build-pdf.sh needs Hugo extended, Go, npm and Python with WeasyPrint. Run it in a container, as the CI does. The script reads the versions from backend/ and frontend/, so mount the whole project directory. Example for an x86-64 host, based on the CI job:

docker run --rm -v "$PWD":/work -w /work python:3 bash -c '
  pip install --quiet weasyprint &&
  apt-get update -qq && apt-get install -y -qq wget git golang nodejs npm > /dev/null &&
  wget -q https://github.com/gohugoio/hugo/releases/download/v0.154.5/hugo_extended_0.154.5_linux-amd64.deb &&
  dpkg -i hugo_extended_0.154.5_linux-amd64.deb > /dev/null &&
  manual/build-pdf.sh output'

The PDFs are then in output/. On an ARM machine, use the matching Hugo package (linux-arm64) or start the container with --platform linux/amd64. The files created in the container are owned by root.

For PDF printing, Hugo also generates Docsy’s print output; manual/scripts/combine_print_html.py combines it into one HTML file with a title page per part and language, which WeasyPrint converts into a PDF.

Hugo and Docsy versions

Docsy is pinned to v0.15.0 in manual/go.mod, matching Hugo 0.154.5, which the CI uses for the PDFs. Update Docsy only together with a matching Hugo version, and check the website and PDFs afterwards.