365 lines
20 KiB
Markdown
365 lines
20 KiB
Markdown
# Production guide
|
||
|
||
How to deploy, secure, operate and recover the Celtic Life Science field-force system on a
|
||
single server. This is the operator's document; `README.md` is the developer's.
|
||
|
||
Every command below was run against this codebase's Docker stack unless it is marked
|
||
**untested**.
|
||
|
||
## Contents
|
||
|
||
1. [What you are deploying](#1-what-you-are-deploying)
|
||
2. [Requirements](#2-requirements)
|
||
3. [First deployment](#3-first-deployment)
|
||
4. [Creating the real users](#4-creating-the-real-users)
|
||
5. [Loading reference data](#5-loading-reference-data)
|
||
6. [Configuration](#6-configuration)
|
||
7. [Domain, HTTPS and reverse proxies](#7-domain-https-and-reverse-proxies)
|
||
8. [Security checklist](#8-security-checklist)
|
||
9. [Backups and restore](#9-backups-and-restore)
|
||
10. [Updating and rolling back](#10-updating-and-rolling-back)
|
||
11. [Monitoring and logs](#11-monitoring-and-logs)
|
||
12. [Runbook](#12-runbook)
|
||
13. [Go-live checklist](#13-go-live-checklist)
|
||
14. [Known limitations](#14-known-limitations)
|
||
|
||
---
|
||
|
||
## 1. What you are deploying
|
||
|
||
```
|
||
browser ──HTTPS──▶ caddy ──/api /docs /health──▶ api (FastAPI, uvicorn) ──▶ db (PostgreSQL 16)
|
||
│ │
|
||
└──everything else──▶ web └─▶ pdfstore volume (approved DCR PDFs)
|
||
(static React build)
|
||
```
|
||
|
||
| Service | Image | Role |
|
||
|---|---|---|
|
||
| `caddy` | `caddy:2-alpine` | The only thing that publishes ports. TLS (Let's Encrypt), gzip, security headers, routing. |
|
||
| `api` | built from `Dockerfile` | FastAPI app and PDF rendering (WeasyPrint). Non-root, **read-only filesystem**, all capabilities dropped. Runs `alembic upgrade head` on start. |
|
||
| `web` | built from `web/Dockerfile` | The React app, built at image-build time and served as static files. |
|
||
| `db` | `postgres:16-alpine` | Internal network only. Data in the `pgdata` volume. |
|
||
|
||
Persistent state lives in exactly two volumes: **`pgdata`** (the database) and **`pdfstore`**
|
||
(approved DCR PDFs). `caddydata` holds certificates. Everything else is rebuilt from the code.
|
||
|
||
Roles: MR (medical representative), AM (area manager), GM (general manager), ADMIN. The MR
|
||
files daily call reports and company forms, the AM approves, the GM has read access across
|
||
territories, the ADMIN maintains people, organisation and master data.
|
||
|
||
## 2. Requirements
|
||
|
||
- A Linux server with **Docker Engine and the Compose plugin** (`docker compose version`).
|
||
1 vCPU / 2 GB RAM / 20 GB disk is enough for one company and a few dozen users.
|
||
- `git`, `make`, `python3` (used by `scripts/setup.sh` to generate secrets; `openssl` is the fallback).
|
||
- For HTTPS with a real domain: a DNS `A` record pointing at the server, and ports 80 and 443 free
|
||
and reachable from the internet.
|
||
- Outbound internet access at build time (base images, `apt`, `pip`, `npm`).
|
||
|
||
## 3. First deployment
|
||
|
||
```bash
|
||
git clone <your-repo-url> pharma-web-celtic && cd pharma-web-celtic # or copy this folder
|
||
DOMAIN=app.example.com make deploy # omit DOMAIN if you only have an IP address
|
||
```
|
||
|
||
`make deploy` runs `scripts/setup.sh`, then `docker compose up -d --build`, then waits for the
|
||
API to report healthy. It is safe to re-run. `setup.sh`:
|
||
|
||
- creates `.env` from `.env.example` when missing (**`.env` is never committed and is not part of this repository**);
|
||
- generates a 64-character `SECRET_KEY` when missing or a placeholder;
|
||
- sets `ENVIRONMENT=production` and `DEBUG=false`;
|
||
- generates `POSTGRES_PASSWORD` for a **new** database only — an existing database keeps its password;
|
||
- picks the ports: 80/443 when free, otherwise plain HTTP on the first free port from 2000 (HTTPS side 2443) and tells you which.
|
||
|
||
Verify:
|
||
|
||
```bash
|
||
make status
|
||
curl -s http://localhost:<HTTP_PORT>/health
|
||
# {"status":"ok","environment":"production","pdf_engine":"weasyprint"}
|
||
```
|
||
|
||
`pdf_engine` must say `weasyprint`. In production the API refuses to start if WeasyPrint's
|
||
system libraries are missing or `SECRET_KEY` is a placeholder or shorter than 32 characters,
|
||
so a bad image fails loudly at boot rather than producing broken PDFs later.
|
||
|
||
> **Running this next to another copy on the same machine.** Compose derives the project name
|
||
> from the folder name, and volumes are named after it (`<project>_pgdata`). Two folders with
|
||
> the same name share volumes and fight over ports. Set `COMPOSE_PROJECT_NAME=<something unique>`
|
||
> in the environment (or `.env`) before the first `make deploy` of the second copy.
|
||
|
||
## 4. Creating the real users
|
||
|
||
There is no "create first admin" command. (An Administrator can reset any user's password, email and employee code from **Users → Edit**; users cannot change their own.)
|
||
The only way to obtain a first administrator is the seed script, which creates demo accounts
|
||
that all share one well-known password (`ChangeMe123!`, in `scripts/seed.py`). Treat that as a
|
||
short-lived bootstrap and do the following **before the server is reachable by anyone else**
|
||
(firewall closed, or on a private network):
|
||
|
||
1. Seed once: `docker compose exec api python -m scripts.seed`
|
||
(creates the company row, one headquarters `BBSR`, four demo users `adm001 / gm001 / am001 /
|
||
mr001 @acmepharma.in`, 15 sample doctors and 5 chemists. Idempotent.)
|
||
2. Sign in as `adm001@acmepharma.in`.
|
||
3. **Setup → Organisation**: create the real headquarters. **Setup → People**: create the real
|
||
users, one of them an **Administrator**, each with a strong password. Passwords cannot be
|
||
changed later from the app, so choose them as final and hand them over securely.
|
||
4. Sign out, sign in as the **new** administrator, open **Users** and **deactivate all four demo
|
||
accounts** (deactivation also revokes their sessions). Verify `adm001` can no longer sign in.
|
||
5. Assign an Area Manager to each headquarters, then assign doctors and chemists to MRs
|
||
(**Setup → Assignments**).
|
||
|
||
If you would rather not keep the demo doctors, chemists and the `BBSR` headquarters, deactivate
|
||
them from **Doctors & Chemists** / leave them unassigned.
|
||
|
||
## 5. Loading reference data
|
||
|
||
**Stock / Products list** — the Products page reads the `stock_inventory` table. It is created
|
||
**and filled (26 products from `stock.csv`) by the migration** `h8d4f0a6c3e5`, so it is there
|
||
after the first start with nothing to do. Administrators can add, edit and delete products from
|
||
the Products page.
|
||
|
||
**Sales Projection and Sales & Stock product lists** — the Sales Projection form needs its
|
||
product list (name and PTS) in the database, and the Sales & Stock Statement lists the same
|
||
products. Load them once with the **non-destructive** mode of the import script. The CSVs are
|
||
deliberately not part of the Docker image and the API container is read-only, so mount the two
|
||
files into a one-off container:
|
||
|
||
```bash
|
||
docker compose run --rm --no-deps -T \
|
||
-v "$PWD/sales_projection.csv:/srv/sales_projection.csv:ro" \
|
||
-v "$PWD/Sales_and_Stock_Statement.csv:/srv/Sales_and_Stock_Statement.csv:ro" \
|
||
api python -m scripts.import_csv_data --only-empty
|
||
```
|
||
|
||
`--only-empty` loads a table only if it has **no rows** and prints `skipped, already has N rows`
|
||
otherwise, so it can be re-run safely. **Never run the script without `--only-empty` on a
|
||
database that holds real data** — the plain form deletes every form table before loading.
|
||
|
||
## 6. Configuration
|
||
|
||
Only variables listed in `docker-compose.yml` reach the API container. Set them in `.env`.
|
||
|
||
| Variable | Default | Purpose |
|
||
|---|---|---|
|
||
| `SECRET_KEY` | *(required)* | JWT signing key, 64+ random characters. Changing it signs everyone out. |
|
||
| `POSTGRES_PASSWORD` | *(required)* | Database password. See [rotation](#12-runbook). |
|
||
| `POSTGRES_USER`, `POSTGRES_DB` | `pharma`, `pharma` | Database user and name. |
|
||
| `ENVIRONMENT` | `production` | `production` enables the boot-time safety checks. |
|
||
| `DEBUG` | `false` | Keep `false`. |
|
||
| `DOMAIN` | `localhost` | Host name Caddy serves; `:80` means plain HTTP (behind another proxy / no TLS). |
|
||
| `HTTP_PORT`, `HTTPS_PORT` | `80`, `443` | Host ports Caddy publishes. |
|
||
| `APP_TIMEZONE` | `Asia/Kolkata` | Business timezone. "Today" and the visit-time check use it, not the server's UTC clock. (Set under `api.environment:` in `docker-compose.yml` to change it.) |
|
||
| `BACKDATING_WINDOW_DAYS` | `7` | How many days back an MR may file a report. |
|
||
| `SUBMISSION_DEADLINE_DAYS` | `1` | A report for day D must be submitted by end of D + this many days. |
|
||
| `WEB_CONCURRENCY` | `2` | uvicorn worker processes. |
|
||
| `RUN_MIGRATIONS` | `1` | Set `0` on the `api` service to skip migrations at start. |
|
||
|
||
Other application settings (`app/config/settings.py`) — the letterhead printed on PDFs
|
||
(`PDF_COMPANY_NAME`, `_ADDRESS`, `_WEBSITE`, `_EMAIL`, `_PHONE`), `ACCESS_TOKEN_EXPIRE_MINUTES`
|
||
(15), `REFRESH_TOKEN_EXPIRE_DAYS` (7), `LOGIN_MAX_FAILED_ATTEMPTS` (10) — are **not** passed
|
||
through by the shipped compose file. To change one, add it under `api.environment:` in
|
||
`docker-compose.yml` and run `docker compose up -d`. The defaults already carry Celtic's details.
|
||
|
||
## 7. Domain, HTTPS and reverse proxies
|
||
|
||
| Situation | Result |
|
||
|---|---|
|
||
| Ports 80/443 free and `DOMAIN=app.example.com` (DNS already pointing here) | Caddy obtains and renews a Let's Encrypt certificate itself. |
|
||
| Ports 80/443 free, no domain | `DOMAIN=localhost`: HTTPS with a self-signed certificate (browser warning). |
|
||
| 80/443 owned by another service | `DOMAIN=:80` on `HTTP_PORT` (e.g. 2000). Put your existing proxy in front. |
|
||
|
||
Existing nginx terminating TLS in front of Caddy:
|
||
|
||
```nginx
|
||
server {
|
||
server_name app.example.com;
|
||
listen 443 ssl; # + your certificate directives
|
||
location / {
|
||
proxy_pass http://127.0.0.1:2000; # the HTTP_PORT chosen by setup.sh
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Forwarded-Proto $scheme;
|
||
proxy_set_header X-Forwarded-For $remote_addr;
|
||
}
|
||
}
|
||
```
|
||
|
||
Block the published port from the internet when a proxy on the same host fronts it
|
||
(`sudo ufw deny 2000/tcp`; the proxy still reaches `127.0.0.1:2000`).
|
||
|
||
Once HTTPS works on the real domain, enable HSTS: uncomment the
|
||
`Strict-Transport-Security` line in `Caddyfile` and `docker compose up -d`. Do not enable it
|
||
earlier — it locks browsers into HTTPS for a year.
|
||
|
||
## 8. Security checklist
|
||
|
||
- [ ] `.env` is `chmod 600`, owned by the deploy user, and is **not** in any repository or backup that leaves the server unencrypted.
|
||
- [ ] `SECRET_KEY` and `POSTGRES_PASSWORD` are generated values (`setup.sh` does this). `setup.sh` warns if the database still uses the default `pharma` password — rotate it.
|
||
- [ ] The four demo accounts are **deactivated** ([section 4](#4-creating-the-real-users)).
|
||
- [ ] Only ports 80/443 (or your proxy) are open. The database and API are not published; do not use `docker-compose.dev.yml` on a server — it publishes them.
|
||
- [ ] HTTPS on the real domain, then HSTS enabled.
|
||
- [ ] The host is patched, SSH is key-only, and `docker` group membership is limited to operators.
|
||
- [ ] Backups run and a restore has been **rehearsed** ([section 9](#9-backups-and-restore)).
|
||
- [ ] Decide about the public API docs: `/docs`, `/redoc` and `/openapi.json` are **reachable by anyone** through Caddy in the shipped configuration (they list every endpoint, though not data). To hide them, delete the corresponding `handle` blocks in `Caddyfile` and `docker compose up -d`.
|
||
- [ ] Someone reads the API logs for repeated `401`s (accounts lock after 10 failed sign-ins).
|
||
|
||
What the containers already enforce: API runs as a non-root user on a read-only filesystem with
|
||
all Linux capabilities dropped and `no-new-privileges`; the database is unreachable from outside
|
||
the compose network; Caddy sends `X-Content-Type-Options`, `X-Frame-Options: DENY` and a strict
|
||
`Referrer-Policy`, and hides the server banner; JSON logs rotate at 10 MB × 5 files per service.
|
||
|
||
**Audit log hardening (optional, not enabled by default).** `scripts/harden_audit.sql` creates a
|
||
`pharma_app` role that can insert into and read `audit_log` but never update or delete it. The
|
||
shipped compose file connects with a single owner role that also runs migrations, so enabling it
|
||
means splitting credentials: run migrations with the owner role (set `RUN_MIGRATIONS=0` on `api`
|
||
and run `alembic upgrade head` in a one-off container) and point the running API's
|
||
`DATABASE_URL` at `pharma_app`, after changing that role's placeholder password in the script.
|
||
This flow is **untested** in this repository — rehearse it on a staging copy first.
|
||
|
||
## 9. Backups and restore
|
||
|
||
What to back up: the **database** (`pg_dump`) and the **`pdfstore` volume**. Everything else is
|
||
code or regenerable. Take a backup **before every update**.
|
||
|
||
**Database backup** (verified):
|
||
|
||
```bash
|
||
set -a; . ./.env; set +a
|
||
docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > backup-$(date +%F).dump
|
||
```
|
||
|
||
**PDF store backup:**
|
||
|
||
```bash
|
||
docker run --rm -v "$(basename "$PWD")_pdfstore:/data:ro" -v "$PWD:/backup" alpine \
|
||
tar czf /backup/pdfstore-$(date +%F).tgz -C /data .
|
||
```
|
||
|
||
(Adjust the volume name if you set `COMPOSE_PROJECT_NAME`; list volumes with `docker volume ls`.)
|
||
|
||
**Nightly cron** (as the deploy user, `crontab -e`; copy the dumps off the server too):
|
||
|
||
```cron
|
||
15 2 * * * cd /opt/pharma-web-celtic && set -a && . ./.env && set +a && docker compose exec -T db pg_dump -U "$POSTGRES_USER" -Fc "$POSTGRES_DB" > /var/backups/pharma/db-$(date +\%F).dump && find /var/backups/pharma -name 'db-*.dump' -mtime +30 -delete
|
||
```
|
||
|
||
**Restore rehearsal** — restores into a *scratch* database so nothing live is touched (verified):
|
||
|
||
```bash
|
||
set -a; . ./.env; set +a
|
||
docker compose exec -T db createdb -U "$POSTGRES_USER" restore_check
|
||
docker compose exec -T db pg_restore -U "$POSTGRES_USER" -d restore_check --no-owner < backup-YYYY-MM-DD.dump
|
||
docker compose exec -T db psql -U "$POSTGRES_USER" -d restore_check \
|
||
-c 'select count(*) from "user"' -c 'select count(*) from stock_inventory'
|
||
docker compose exec -T db dropdb -U "$POSTGRES_USER" restore_check
|
||
```
|
||
|
||
**Real disaster recovery** (empty server, backup in hand): deploy as in section 3, then
|
||
`docker compose stop api`, restore into the empty database
|
||
(`pg_restore -U … -d "$POSTGRES_DB" --clean --if-exists --no-owner < backup.dump`), restore the
|
||
PDF archive into the `pdfstore` volume, then `docker compose up -d`. Use the same `SECRET_KEY`
|
||
if you want existing sessions to survive; otherwise users simply sign in again.
|
||
|
||
## 10. Updating and rolling back
|
||
|
||
```bash
|
||
# 1. back up (section 9)
|
||
# 2. get the new code
|
||
git pull # or replace the folder contents
|
||
# 3. rebuild and restart — migrations run automatically on api start
|
||
make deploy
|
||
make status
|
||
```
|
||
|
||
The API image is rebuilt, so `pip`/`apt`/`npm` need network access. Compose recreates only what
|
||
changed; the database and volumes are untouched. Check `docker compose logs api | grep alembic`
|
||
to see which migrations ran.
|
||
|
||
**Rollback:** restore the previous code, and if the new release added migrations, step the schema
|
||
back first, using the **new** image (it knows those revisions):
|
||
|
||
```bash
|
||
docker compose exec api alembic downgrade -1 # once per migration to undo; check with: alembic current
|
||
# then deploy the previous code: git checkout <previous-tag> && make deploy
|
||
```
|
||
|
||
If in doubt, restore the pre-update database backup instead — it is always the safe path.
|
||
|
||
## 11. Monitoring and logs
|
||
|
||
- **Health:** `GET /health` → `{"status":"ok", ..., "pdf_engine":"weasyprint"}`. Docker health
|
||
checks run on `api` and `db`; `docker compose ps` shows `(healthy)`. Point an external uptime
|
||
monitor at `https://<domain>/health`.
|
||
- **Logs:** `docker compose logs -f api` (also `caddy`, `db`, `web`). Rotated automatically.
|
||
- **Disk:** watch the Docker data directory (`docker system df`); the `pgdata` and `pdfstore`
|
||
volumes only grow.
|
||
- **Errors:** no error tracker is wired in. Add Sentry or similar if you need alerting on 500s.
|
||
|
||
## 12. Runbook
|
||
|
||
| Task | Command |
|
||
|---|---|
|
||
| Status / URL | `make status` |
|
||
| Restart the API | `docker compose restart api` |
|
||
| Stop everything (data kept) | `docker compose down` |
|
||
| **Delete all data** | `docker compose down -v` — **never** on a live system |
|
||
| Open a database shell | `docker compose exec db psql -U "$POSTGRES_USER" "$POSTGRES_DB"` |
|
||
| Unlock a user (10 failed sign-ins) | `UPDATE "user" SET failed_login_attempts = 0 WHERE email = '…';` in the shell above |
|
||
| Rotate `SECRET_KEY` | edit `.env`, `docker compose up -d` — everyone must sign in again |
|
||
| Rotate the database password | see below |
|
||
|
||
**Rotating the database password** — the database keeps the password it was created with, so
|
||
changing `.env` alone would lock the API out:
|
||
|
||
```bash
|
||
NEW=$(python3 -c "import secrets;print(secrets.token_urlsafe(24))")
|
||
docker compose exec db psql -U pharma -d pharma -c "ALTER USER pharma PASSWORD '$NEW';"
|
||
sed -i "s|^POSTGRES_PASSWORD=.*|POSTGRES_PASSWORD=$NEW|" .env
|
||
docker compose up -d
|
||
```
|
||
|
||
**Common failures**
|
||
|
||
| Symptom | Cause / fix |
|
||
|---|---|
|
||
| `make deploy` says `set POSTGRES_PASSWORD in .env` | Run `make setup` (or `./scripts/setup.sh`). |
|
||
| API restarts in a loop, log says `SECRET_KEY must be a random value` | `.env` has a placeholder key; run `make setup`. |
|
||
| API log: `WeasyPrint system libraries are required` | The image was built without Pango/fonts; rebuild with `docker compose build --no-cache api`. |
|
||
| Caddy cannot bind port 80/443 | Another service owns them; run `make setup` (it moves Caddy to a free port) and put a proxy in front. |
|
||
| API cannot connect after editing `.env` | The database still has the old password — see rotation above. |
|
||
| A page shows *"This didn't load — Request failed (500)"* | `docker compose logs api` shows the cause; after an update check the migration ran (`alembic current`). |
|
||
| PDFs look plain-text | The API logged the WeasyPrint fallback warning; the image is missing native libraries. |
|
||
|
||
## 13. Go-live checklist
|
||
|
||
1. `make deploy` finishes and `/health` reports `weasyprint`.
|
||
2. Real headquarters and users created; **all demo accounts deactivated** (section 4).
|
||
3. Reference data loaded (section 5): Products page shows 26 products; Sales Projection shows the
|
||
product list.
|
||
4. One end-to-end pass with real accounts: MR files a DCR → AM approves → PDF downloads and
|
||
opens; each company form downloads a PDF that matches the printed original.
|
||
5. HTTPS works on the final domain; HSTS enabled.
|
||
6. Firewall: only 80/443 (or the proxy) open.
|
||
7. First backup taken **and a restore rehearsed**; cron installed.
|
||
8. Uptime monitor pointed at `/health`; someone owns reading the logs.
|
||
9. Business rules confirmed with the client: the back-dating window and submission deadline
|
||
(`BACKDATING_WINDOW_DAYS`, `SUBMISSION_DEADLINE_DAYS`) are shipped assumptions.
|
||
|
||
## 14. Known limitations
|
||
|
||
- **Users cannot change their own password.** Only an Administrator can, from **Users → Edit**
|
||
(also email and employee code). A reset signs that user out everywhere and clears a lockout,
|
||
but an access token already issued stays valid until it expires (15 minutes). If the last
|
||
Administrator password is lost, create a new admin with a one-off script (see `scripts/`) —
|
||
there is no self-service recovery and no email flow.
|
||
- **Company-form approval is per form, not per row.** Tour plan, standard tour plan and fare chart
|
||
each have one approval state, stored on the server (`form_approval`): the MR sends a form that has
|
||
rows, the Area Manager approves it, and the PDF signature shows that name. Any change to the rows
|
||
withdraws the approval. The signature is decided by the server, never by the browser.
|
||
- **Audit-log immutability is opt-in** (section 8) and untested with the shipped compose file.
|
||
- **No built-in alerting or error tracking**; only health checks and logs.
|
||
- **Single server, single database.** There is no high availability; recovery time is "restore the
|
||
latest backup". That is the deliberate trade-off for this size of deployment.
|
||
- The test suite needs a reachable PostgreSQL; some tests under `tests/unit` do too.
|