20 KiB
Production guide
How to deploy, secure, operate and recover the Celtic Life Science field-force system on a
single server. This is the operator's document; README.md is the developer's.
Every command below was run against this codebase's Docker stack unless it is marked untested.
Contents
- What you are deploying
- Requirements
- First deployment
- Creating the real users
- Loading reference data
- Configuration
- Domain, HTTPS and reverse proxies
- Security checklist
- Backups and restore
- Updating and rolling back
- Monitoring and logs
- Runbook
- Go-live checklist
- Known limitations
1. What you are deploying
browser ──HTTPS──▶ caddy ──/api /docs /health──▶ api (FastAPI, uvicorn) ──▶ db (PostgreSQL 16)
│ │
└──everything else──▶ web └─▶ pdfstore volume (approved DCR PDFs)
(static React build)
| Service | Image | Role |
|---|---|---|
caddy |
caddy:2-alpine |
The only thing that publishes ports. TLS (Let's Encrypt), gzip, security headers, routing. |
api |
built from Dockerfile |
FastAPI app and PDF rendering (WeasyPrint). Non-root, read-only filesystem, all capabilities dropped. Runs alembic upgrade head on start. |
web |
built from web/Dockerfile |
The React app, built at image-build time and served as static files. |
db |
postgres:16-alpine |
Internal network only. Data in the pgdata volume. |
Persistent state lives in exactly two volumes: pgdata (the database) and pdfstore
(approved DCR PDFs). caddydata holds certificates. Everything else is rebuilt from the code.
Roles: MR (medical representative), AM (area manager), GM (general manager), ADMIN. The MR files daily call reports and company forms, the AM approves, the GM has read access across territories, the ADMIN maintains people, organisation and master data.
2. Requirements
- A Linux server with Docker Engine and the Compose plugin (
docker compose version). 1 vCPU / 2 GB RAM / 20 GB disk is enough for one company and a few dozen users. git,make,python3(used byscripts/setup.shto generate secrets;opensslis the fallback).- For HTTPS with a real domain: a DNS
Arecord pointing at the server, and ports 80 and 443 free and reachable from the internet. - Outbound internet access at build time (base images,
apt,pip,npm).
3. First deployment
git clone <your-repo-url> pharma-web-celtic && cd pharma-web-celtic # or copy this folder
DOMAIN=app.example.com make deploy # omit DOMAIN if you only have an IP address
make deploy runs scripts/setup.sh, then docker compose up -d --build, then waits for the
API to report healthy. It is safe to re-run. setup.sh:
- creates
.envfrom.env.examplewhen missing (.envis never committed and is not part of this repository); - generates a 64-character
SECRET_KEYwhen missing or a placeholder; - sets
ENVIRONMENT=productionandDEBUG=false; - generates
POSTGRES_PASSWORDfor a new database only — an existing database keeps its password; - picks the ports: 80/443 when free, otherwise plain HTTP on the first free port from 2000 (HTTPS side 2443) and tells you which.
Verify:
make status
curl -s http://localhost:<HTTP_PORT>/health
# {"status":"ok","environment":"production","pdf_engine":"weasyprint"}
pdf_engine must say weasyprint. In production the API refuses to start if WeasyPrint's
system libraries are missing or SECRET_KEY is a placeholder or shorter than 32 characters,
so a bad image fails loudly at boot rather than producing broken PDFs later.
Running this next to another copy on the same machine. Compose derives the project name from the folder name, and volumes are named after it (
<project>_pgdata). Two folders with the same name share volumes and fight over ports. SetCOMPOSE_PROJECT_NAME=<something unique>in the environment (or.env) before the firstmake deployof the second copy.
4. Creating the real users
There is no "create first admin" command. (An Administrator can reset any user's password, email and employee code from Users → Edit; users cannot change their own.)
The only way to obtain a first administrator is the seed script, which creates demo accounts
that all share one well-known password (ChangeMe123!, in scripts/seed.py). Treat that as a
short-lived bootstrap and do the following before the server is reachable by anyone else
(firewall closed, or on a private network):
- Seed once:
docker compose exec api python -m scripts.seed(creates the company row, one headquartersBBSR, four demo usersadm001 / gm001 / am001 / mr001 @acmepharma.in, 15 sample doctors and 5 chemists. Idempotent.) - Sign in as
adm001@acmepharma.in. - Setup → Organisation: create the real headquarters. Setup → People: create the real users, one of them an Administrator, each with a strong password. Passwords cannot be changed later from the app, so choose them as final and hand them over securely.
- Sign out, sign in as the new administrator, open Users and deactivate all four demo
accounts (deactivation also revokes their sessions). Verify
adm001can no longer sign in. - Assign an Area Manager to each headquarters, then assign doctors and chemists to MRs (Setup → Assignments).
If you would rather not keep the demo doctors, chemists and the BBSR headquarters, deactivate
them from Doctors & Chemists / leave them unassigned.
5. Loading reference data
Stock / Products list — the Products page reads the stock_inventory table. It is created
and filled (26 products from stock.csv) by the migration h8d4f0a6c3e5, so it is there
after the first start with nothing to do. Administrators can add, edit and delete products from
the Products page.
Sales Projection and Sales & Stock product lists — the Sales Projection form needs its product list (name and PTS) in the database, and the Sales & Stock Statement lists the same products. Load them once with the non-destructive mode of the import script. The CSVs are deliberately not part of the Docker image and the API container is read-only, so mount the two files into a one-off container:
docker compose run --rm --no-deps -T \
-v "$PWD/sales_projection.csv:/srv/sales_projection.csv:ro" \
-v "$PWD/Sales_and_Stock_Statement.csv:/srv/Sales_and_Stock_Statement.csv:ro" \
api python -m scripts.import_csv_data --only-empty
--only-empty loads a table only if it has no rows and prints skipped, already has N rows
otherwise, so it can be re-run safely. Never run the script without --only-empty on a
database that holds real data — the plain form deletes every form table before loading.
6. Configuration
Only variables listed in docker-compose.yml reach the API container. Set them in .env.
| Variable | Default | Purpose |
|---|---|---|
SECRET_KEY |
(required) | JWT signing key, 64+ random characters. Changing it signs everyone out. |
POSTGRES_PASSWORD |
(required) | Database password. See rotation. |
POSTGRES_USER, POSTGRES_DB |
pharma, pharma |
Database user and name. |
ENVIRONMENT |
production |
production enables the boot-time safety checks. |
DEBUG |
false |
Keep false. |
DOMAIN |
localhost |
Host name Caddy serves; :80 means plain HTTP (behind another proxy / no TLS). |
HTTP_PORT, HTTPS_PORT |
80, 443 |
Host ports Caddy publishes. |
APP_TIMEZONE |
Asia/Kolkata |
Business timezone. "Today" and the visit-time check use it, not the server's UTC clock. (Set under api.environment: in docker-compose.yml to change it.) |
BACKDATING_WINDOW_DAYS |
7 |
How many days back an MR may file a report. |
SUBMISSION_DEADLINE_DAYS |
1 |
A report for day D must be submitted by end of D + this many days. |
WEB_CONCURRENCY |
2 |
uvicorn worker processes. |
RUN_MIGRATIONS |
1 |
Set 0 on the api service to skip migrations at start. |
Other application settings (app/config/settings.py) — the letterhead printed on PDFs
(PDF_COMPANY_NAME, _ADDRESS, _WEBSITE, _EMAIL, _PHONE), ACCESS_TOKEN_EXPIRE_MINUTES
(15), REFRESH_TOKEN_EXPIRE_DAYS (7), LOGIN_MAX_FAILED_ATTEMPTS (10) — are not passed
through by the shipped compose file. To change one, add it under api.environment: in
docker-compose.yml and run docker compose up -d. The defaults already carry Celtic's details.
7. Domain, HTTPS and reverse proxies
| Situation | Result |
|---|---|
Ports 80/443 free and DOMAIN=app.example.com (DNS already pointing here) |
Caddy obtains and renews a Let's Encrypt certificate itself. |
| Ports 80/443 free, no domain | DOMAIN=localhost: HTTPS with a self-signed certificate (browser warning). |
| 80/443 owned by another service | DOMAIN=:80 on HTTP_PORT (e.g. 2000). Put your existing proxy in front. |
Existing nginx terminating TLS in front of Caddy:
server {
server_name app.example.com;
listen 443 ssl; # + your certificate directives
location / {
proxy_pass http://127.0.0.1:2000; # the HTTP_PORT chosen by setup.sh
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
Block the published port from the internet when a proxy on the same host fronts it
(sudo ufw deny 2000/tcp; the proxy still reaches 127.0.0.1:2000).
Once HTTPS works on the real domain, enable HSTS: uncomment the
Strict-Transport-Security line in Caddyfile and docker compose up -d. Do not enable it
earlier — it locks browsers into HTTPS for a year.
8. Security checklist
.envischmod 600, owned by the deploy user, and is not in any repository or backup that leaves the server unencrypted.SECRET_KEYandPOSTGRES_PASSWORDare generated values (setup.shdoes this).setup.shwarns if the database still uses the defaultpharmapassword — rotate it.- The four demo accounts are deactivated (section 4).
- Only ports 80/443 (or your proxy) are open. The database and API are not published; do not use
docker-compose.dev.ymlon a server — it publishes them. - HTTPS on the real domain, then HSTS enabled.
- The host is patched, SSH is key-only, and
dockergroup membership is limited to operators. - Backups run and a restore has been rehearsed (section 9).
- Decide about the public API docs:
/docs,/redocand/openapi.jsonare reachable by anyone through Caddy in the shipped configuration (they list every endpoint, though not data). To hide them, delete the correspondinghandleblocks inCaddyfileanddocker compose up -d. - Someone reads the API logs for repeated
401s (accounts lock after 10 failed sign-ins).
What the containers already enforce: API runs as a non-root user on a read-only filesystem with
all Linux capabilities dropped and no-new-privileges; the database is unreachable from outside
the compose network; Caddy sends X-Content-Type-Options, X-Frame-Options: DENY and a strict
Referrer-Policy, and hides the server banner; JSON logs rotate at 10 MB × 5 files per service.
Audit log hardening (optional, not enabled by default). scripts/harden_audit.sql creates a
pharma_app role that can insert into and read audit_log but never update or delete it. The
shipped compose file connects with a single owner role that also runs migrations, so enabling it
means splitting credentials: run migrations with the owner role (set RUN_MIGRATIONS=0 on api
and run alembic upgrade head in a one-off container) and point the running API's
DATABASE_URL at pharma_app, after changing that role's placeholder password in the script.
This flow is untested in this repository — rehearse it on a staging copy first.
9. Backups and restore
What to back up: the database (pg_dump) and the pdfstore volume. Everything else is
code or regenerable. Take a backup before every update.
Database backup (verified):
set -a; . ./.env; set +a
docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > backup-$(date +%F).dump
PDF store backup:
docker run --rm -v "$(basename "$PWD")_pdfstore:/data:ro" -v "$PWD:/backup" alpine \
tar czf /backup/pdfstore-$(date +%F).tgz -C /data .
(Adjust the volume name if you set COMPOSE_PROJECT_NAME; list volumes with docker volume ls.)
Nightly cron (as the deploy user, crontab -e; copy the dumps off the server too):
15 2 * * * cd /opt/pharma-web-celtic && set -a && . ./.env && set +a && docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > /var/backups/pharma/db-$(date +\%F).dump && find /var/backups/pharma -name 'db-*.dump' -mtime +30 -delete
Restore rehearsal — restores into a scratch database so nothing live is touched (verified):
set -a; . ./.env; set +a
docker compose exec -T db createdb -U "$POSTGRES_USER" restore_check
docker compose exec -T db pg_restore -U "$POSTGRES_USER" -d restore_check --no-owner < backup-YYYY-MM-DD.dump
docker compose exec -T db psql -U "$POSTGRES_USER" -d restore_check \
-c 'select count(*) from "user"' -c 'select count(*) from stock_inventory'
docker compose exec -T db dropdb -U "$POSTGRES_USER" restore_check
Real disaster recovery (empty server, backup in hand): deploy as in section 3, then
docker compose stop api, restore into the empty database
(pg_restore -U … -d "$POSTGRES_DB" --clean --if-exists --no-owner < backup.dump), restore the
PDF archive into the pdfstore volume, then docker compose up -d. Use the same SECRET_KEY
if you want existing sessions to survive; otherwise users simply sign in again.
10. Updating and rolling back
# 1. back up (section 9)
# 2. get the new code
git pull # or replace the folder contents
# 3. rebuild and restart — migrations run automatically on api start
make deploy
make status
The API image is rebuilt, so pip/apt/npm need network access. Compose recreates only what
changed; the database and volumes are untouched. Check docker compose logs api | grep alembic
to see which migrations ran.
Rollback: restore the previous code, and if the new release added migrations, step the schema back first, using the new image (it knows those revisions):
docker compose exec api alembic downgrade -1 # once per migration to undo; check with: alembic current
# then deploy the previous code: git checkout <previous-tag> && make deploy
If in doubt, restore the pre-update database backup instead — it is always the safe path.
11. Monitoring and logs
- Health:
GET /health→{"status":"ok", ..., "pdf_engine":"weasyprint"}. Docker health checks run onapianddb;docker compose psshows(healthy). Point an external uptime monitor athttps://<domain>/health. - Logs:
docker compose logs -f api(alsocaddy,db,web). Rotated automatically. - Disk: watch the Docker data directory (
docker system df); thepgdataandpdfstorevolumes only grow. - Errors: no error tracker is wired in. Add Sentry or similar if you need alerting on 500s.
12. Runbook
| Task | Command |
|---|---|
| Status / URL | make status |
| Restart the API | docker compose restart api |
| Stop everything (data kept) | docker compose down |
| Delete all data | docker compose down -v — never on a live system |
| Open a database shell | docker compose exec db psql -U "$POSTGRES_USER" "$POSTGRES_DB" |
| Unlock a user (10 failed sign-ins) | UPDATE "user" SET failed_login_attempts = 0 WHERE email = '…'; in the shell above |
Rotate SECRET_KEY |
edit .env, docker compose up -d — everyone must sign in again |
| Rotate the database password | see below |
Rotating the database password — the database keeps the password it was created with, so
changing .env alone would lock the API out:
NEW=$(python3 -c "import secrets;print(secrets.token_urlsafe(24))")
docker compose exec db psql -U pharma -d pharma -c "ALTER USER pharma PASSWORD '$NEW';"
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$NEW|" .env
docker compose up -d
Common failures
| Symptom | Cause / fix |
|---|---|
make deploy says set POSTGRES_PASSWORD in .env |
Run make setup (or ./scripts/setup.sh). |
API restarts in a loop, log says SECRET_KEY must be a random value |
.env has a placeholder key; run make setup. |
API log: WeasyPrint system libraries are required |
The image was built without Pango/fonts; rebuild with docker compose build --no-cache api. |
| Caddy cannot bind port 80/443 | Another service owns them; run make setup (it moves Caddy to a free port) and put a proxy in front. |
API cannot connect after editing .env |
The database still has the old password — see rotation above. |
| A page shows "This didn't load — Request failed (500)" | docker compose logs api shows the cause; after an update check the migration ran (alembic current). |
| PDFs look plain-text | The API logged the WeasyPrint fallback warning; the image is missing native libraries. |
13. Go-live checklist
make deployfinishes and/healthreportsweasyprint.- Real headquarters and users created; all demo accounts deactivated (section 4).
- Reference data loaded (section 5): Products page shows 26 products; Sales Projection shows the product list.
- One end-to-end pass with real accounts: MR files a DCR → AM approves → PDF downloads and opens; each company form downloads a PDF that matches the printed original.
- HTTPS works on the final domain; HSTS enabled.
- Firewall: only 80/443 (or the proxy) open.
- First backup taken and a restore rehearsed; cron installed.
- Uptime monitor pointed at
/health; someone owns reading the logs. - Business rules confirmed with the client: the back-dating window and submission deadline
(
BACKDATING_WINDOW_DAYS,SUBMISSION_DEADLINE_DAYS) are shipped assumptions.
14. Known limitations
- Users cannot change their own password. Only an Administrator can, from Users → Edit
(also email and employee code). A reset signs that user out everywhere and clears a lockout,
but an access token already issued stays valid until it expires (15 minutes). If the last
Administrator password is lost, create a new admin with a one-off script (see
scripts/) — there is no self-service recovery and no email flow. - Company-form approval is per form, not per row. Tour plan, standard tour plan and fare chart
each have one approval state, stored on the server (
form_approval): the MR sends a form that has rows, the Area Manager approves it, and the PDF signature shows that name. Any change to the rows withdraws the approval. The signature is decided by the server, never by the browser. - Audit-log immutability is opt-in (section 8) and untested with the shipped compose file.
- No built-in alerting or error tracking; only health checks and logs.
- Single server, single database. There is no high availability; recovery time is "restore the latest backup". That is the deliberate trade-off for this size of deployment.
- The test suite needs a reachable PostgreSQL; some tests under
tests/unitdo too.