Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
node_modules/
/data/
/test-data/
dist/
!packages/plugin-opencode/dist/
!packages/plugin-opencode/dist/index.js
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ FROM scratch
COPY --from=build /diffx-server /diffx-server
COPY --from=build --chown=10001:10001 /data /data
USER 10001:10001
ENV DIFFX_CONFIG_PATH=/data/config.json
ENV HOME=/data
ENV DIFFX_HOST=0.0.0.0
ENV DIFFX_PORT=7981
ENV DIFFX_STATE=/data/state.db
VOLUME ["/data"]
EXPOSE 7981
STOPSIGNAL SIGTERM
ENTRYPOINT ["/diffx-server"]
CMD ["--listen", "0.0.0.0:7981", "--state", "/data/state.db"]
33 changes: 25 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,15 +146,28 @@ review context. Sessions associate with shared reviews; they do not own or
claim authorship of every worktree's edits. Harness, session ID and optional
session name remain searchable even when identical content reuses a review.

Build and run the Docker server (deploy behind HTTPS):
Build and run the container setup for server development:

```sh
mise run docker:up
mise run docker:down
```

Compose publishes HTTP on `127.0.0.1:7981` and preserves the named SQLite volume
when stopped or recreated. Set `DIFFX_PORT=7982` to use another port. Retrieve
the initial credential with `docker compose cp server:/data/state.db.admin-token ./admin-token`.
See [server development](docs/development.md#server-development) for configuration
and persistence details. Deploy behind HTTPS when exposing the server remotely.

Or build and run the image directly:

```sh
mise run docker:build
docker run --name diffx-server -p 127.0.0.1:7981:7981 \
docker run --name diffx-server --read-only -p 127.0.0.1:7981:7981 \
--mount source=diffx-data,target=/data diffx-server:local
```

The volume contains observations, jobs, credentials and configuration. First
The volume contains observations, jobs and credentials. First
startup writes the admin token to `/data/state.db.admin-token`; copy it with
`docker cp diffx-server:/data/state.db.admin-token ./admin-token`.
The image runs without Git or a repository mount, as an unprivileged user.
Expand All @@ -165,11 +178,11 @@ Build/install the remote server with Go:

```sh
go install github.com/flexdinesh/diffx/cmd/diffx-server@main
diffx-server --listen 0.0.0.0:7981 --state /data/state.db
DIFFX_HOST=127.0.0.1 DIFFX_PORT=7981 DIFFX_STATE=./test-data/state.db diffx-server
```

First startup creates `admin` with a generated token saved privately at
`/data/state.db.admin-token`; startup prints its path. The token persists across
`<database>.admin-token`; startup prints its path. The token persists across
restarts. API/MCP requests use bearer authentication; the browser uses HTTP Basic
with the username and token as password. Deploy behind TLS. Multiple individual
users share one SQLite database; credentials select and isolate each user's data.
Expand All @@ -179,9 +192,12 @@ The database stores credential hashes. To add a user, stop the server, run:
diffx-server user create --name alice --state /data/state.db
```

Save the printed token, then restart the server. `--account` and `DIFFX_TOKEN`
Save the printed token, then restart the server. `--account` / `DIFFX_ACCOUNT` and `DIFFX_TOKEN`
can supply initial bootstrap credentials; existing credentials are never replaced.
Use `--retention-days 14` or config/environment to change server retention.
The server defaults to `127.0.0.1:7981` and `./data/state.db`, independent of
personal CLI settings. Its image defaults to `0.0.0.0:7981` and `/data/state.db`;
env vars override image defaults, and explicit flags override env vars.

Collectors select that destination with config `server`, `DIFFX_SERVER_URL`
or `--server`, and authenticate with config `token`, `DIFFX_TOKEN` or `--token`:
Expand Down Expand Up @@ -256,8 +272,9 @@ WebMCP while the page is open; see [docs/webmcp.md](docs/webmcp.md).

Local data lives in `$XDG_STATE_HOME/diffx/state.db`, or
`~/.local/state/diffx/state.db`. `--state memory` disables persistence.
Schema 9 intentionally resets older versioned databases on first open. Current
state persists across restarts; newer or unrecognized databases are refused.
Current schema 9 state persists across restarts. Older, newer or unrecognized
databases are refused without an automatic reset. To start fresh, stop the
server, back up its state directory, and explicitly choose a new database path.
Legacy commands and pending producer payloads are retired.
Stop older processes before upgrading. Local service lifecycle uses a private
authenticated loopback control listener; its token does not authenticate the
Expand Down
41 changes: 21 additions & 20 deletions cmd/diffx-server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@ import (
"flag"
"fmt"
"io"
"net"
"os"
"os/signal"
"strconv"
"strings"
"syscall"

Expand All @@ -22,11 +20,11 @@ func parseSettings(arguments []string, stderr io.Writer) (remoteserver.Settings,
settings := remoteserver.Settings{Token: os.Getenv("DIFFX_TOKEN")}
flags := flag.NewFlagSet("diffx-server", flag.ContinueOnError)
flags.SetOutput(stderr)
flags.StringVar(&settings.Listen, "listen", "0.0.0.0:7981", "HTTP listen address; terminate TLS at the deployment boundary")
flags.StringVar(&settings.Listen, "listen", "127.0.0.1:7981", "HTTP listen address; terminate TLS at the deployment boundary")
flags.StringVar(&settings.State, "state", "", "SQLite database path")
flags.StringVar(&settings.Account, "account", "admin", "bootstrap account name; persisted credentials are never replaced")
flags.IntVar(&settings.RetentionDays, "retention-days", 7, "snapshot retention in days")
path := flags.String("config-file", "", "machine JSON config path")
path := flags.String("config-file", "", "optional read-only server JSON config path")
showVersion := flags.Bool("version", false, "print version and exit")
if err := flags.Parse(arguments); err != nil {
return settings, false, err
Expand All @@ -37,38 +35,41 @@ func parseSettings(arguments []string, stderr io.Writer) (remoteserver.Settings,
if flags.NArg() != 0 {
return settings, false, fmt.Errorf("diffx-server takes no positional arguments")
}
values, err := config.LoadFile(*path)
values, err := config.LoadRemoteFile(*path)
if err != nil {
return settings, false, err
}
resolved, err := values.Settings()
if err != nil {
return settings, false, err
}
listenSet, stateSet, retentionSet := false, false, false
listenSet, stateSet, accountSet, retentionSet := false, false, false, false
flags.Visit(func(value *flag.Flag) {
listenSet = listenSet || value.Name == "listen"
stateSet = stateSet || value.Name == "state"
accountSet = accountSet || value.Name == "account"
retentionSet = retentionSet || value.Name == "retention-days"
})
_, hostEnv := os.LookupEnv("DIFFX_HOST")
_, portEnv := os.LookupEnv("DIFFX_PORT")
if !listenSet && (*path != "" || os.Getenv("DIFFX_CONFIG_PATH") != "" || hostEnv || portEnv) {
port := resolved.Port
if port < 0 {
port = 7981
if !listenSet {
settings.Listen, err = values.ListenAddress()
if err != nil {
return settings, false, err
}
settings.Listen = net.JoinHostPort(resolved.Host, strconv.Itoa(port))
}
if !stateSet || settings.State == "" {
settings.State = resolved.State
settings.State = values.State
}
if !accountSet {
settings.Account = values.Account
}
if !retentionSet {
settings.RetentionDays = values.RetentionDays
}
if retentionSet && settings.RetentionDays <= 0 {
if settings.RetentionDays <= 0 {
return settings, false, fmt.Errorf("retention days must be positive")
}
if settings.State == "" {
return settings, false, fmt.Errorf("state must be a database path or memory")
}
if strings.TrimSpace(settings.Account) == "" || len(settings.Account) > 256 || strings.ContainsAny(settings.Account, "\r\n:") {
return settings, false, fmt.Errorf("invalid bootstrap account name")
}
if _, err := remoteserver.Retention(settings.RetentionDays); err != nil {
return settings, false, err
}
Expand Down Expand Up @@ -117,7 +118,7 @@ func createUser(arguments []string, stdout, stderr io.Writer) error {
flags.SetOutput(stderr)
name := flags.String("name", "", "new account name")
state := flags.String("state", "", "SQLite database path; server must be stopped")
configPath := flags.String("config-file", "", "machine JSON config path")
configPath := flags.String("config-file", "", "optional read-only server JSON config path")
if err := flags.Parse(arguments[1:]); err != nil {
return err
}
Expand Down
55 changes: 51 additions & 4 deletions cmd/diffx-server/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ func TestRemoteConfigMappingAndPrecedence(t *testing.T) {
t.Setenv("DIFFX_CONFIG_PATH", "")
t.Setenv("DIFFX_RUNTIME_DIR", t.TempDir())
settings, _, err := parseSettings(nil, io.Discard)
if err != nil || settings.Listen != "0.0.0.0:7981" {
if err != nil || settings.Listen != "127.0.0.1:7981" || settings.State != "data/state.db" {
t.Fatalf("remote default changed: %#v %v", settings, err)
}
settings, _, err = parseSettings([]string{"--config-file", path}, io.Discard)
Expand All @@ -36,6 +36,12 @@ func TestRemoteConfigMappingAndPrecedence(t *testing.T) {
if err != nil || settings.Listen != "0.0.0.0:6123" || settings.State != "flags.db" {
t.Fatalf("flags mapping: %#v %v", settings, err)
}
t.Setenv("DIFFX_HOST", "invalid")
t.Setenv("DIFFX_PORT", "65536")
settings, _, err = parseSettings([]string{"--listen", "127.0.0.1:0"}, io.Discard)
if err != nil || settings.Listen != "127.0.0.1:0" {
t.Fatalf("overridden listener was validated: %+v %v", settings, err)
}
}

func TestCreateUserPersistsCredentialAndRejectsDuplicates(t *testing.T) {
Expand Down Expand Up @@ -84,17 +90,58 @@ func TestCreateUserValidation(t *testing.T) {
}
}

func TestRemoteConfigAutomaticPortAndVersion(t *testing.T) {
t.Setenv("DIFFX_CONFIG_PATH", filepath.Join(t.TempDir(), "config.json"))
func TestRemoteConfigDefaultsAndVersion(t *testing.T) {
t.Setenv("DIFFX_CONFIG_PATH", "")
settings, _, err := parseSettings(nil, io.Discard)
if err != nil || settings.Listen != "127.0.0.1:7981" {
t.Fatalf("automatic config port: %#v %v", settings, err)
t.Fatalf("default server port: %#v %v", settings, err)
}
if _, showVersion, err := parseSettings([]string{"--version", "--config-file", "/missing/config.json"}, io.Discard); err != nil || !showVersion {
t.Fatalf("version resolved config: %v", err)
}
}

func TestRemoteAccountConfigPrecedence(t *testing.T) {
path := filepath.Join(t.TempDir(), "server.json")
if err := os.WriteFile(path, []byte(`{"account":"file-admin"}`), 0o400); err != nil {
t.Fatal(err)
}
settings, _, err := parseSettings([]string{"--config-file", path}, io.Discard)
if err != nil || settings.Account != "file-admin" {
t.Fatalf("file account: %+v %v", settings, err)
}
t.Setenv("DIFFX_ACCOUNT", "env-admin")
settings, _, err = parseSettings([]string{"--config-file", path}, io.Discard)
if err != nil || settings.Account != "env-admin" {
t.Fatalf("environment account: %+v %v", settings, err)
}
settings, _, err = parseSettings([]string{"--config-file", path, "--account", "flag-admin"}, io.Discard)
if err != nil || settings.Account != "flag-admin" {
t.Fatalf("flag account: %+v %v", settings, err)
}
}

func TestRemoteRejectsInvalidDeploymentSettings(t *testing.T) {
for _, tc := range []struct{ key, value string }{
{"DIFFX_HOST", "localhost"},
{"DIFFX_PORT", "not-a-port"},
{"DIFFX_PORT", "65536"},
{"DIFFX_STATE", ""},
{"DIFFX_RETENTION_DAYS", "0"},
{"DIFFX_RETENTION_DAYS", "-1"},
{"DIFFX_ACCOUNT", ""},
{"DIFFX_ACCOUNT", "invalid:name"},
} {
t.Run(tc.key+"="+tc.value, func(t *testing.T) {
t.Setenv("DIFFX_CONFIG_PATH", "")
t.Setenv(tc.key, tc.value)
if _, _, err := parseSettings(nil, io.Discard); err == nil {
t.Fatal("accepted invalid deployment settings")
}
})
}
}

func TestRemoteRetentionConfigPrecedence(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.json")
if err := os.WriteFile(path, []byte(`{"retentionDays":14}`), 0o600); err != nil {
Expand Down
20 changes: 20 additions & 0 deletions compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
services:
server:
build: .
image: diffx-server:local
environment:
DIFFX_HOST: 0.0.0.0
DIFFX_PORT: "${DIFFX_PORT:-7981}"
DIFFX_STATE: /data/state.db
DIFFX_ACCOUNT: "${DIFFX_ACCOUNT:-admin}"
DIFFX_RETENTION_DAYS: "${DIFFX_RETENTION_DAYS:-7}"
ports:
- "127.0.0.1:${DIFFX_PORT:-7981}:${DIFFX_PORT:-7981}"
volumes:
- diffx-data:/data
read_only: true
restart: unless-stopped
stop_grace_period: 30s

volumes:
diffx-data:
15 changes: 12 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,8 +128,17 @@ uses `X-Diffx-State` to pin submission and polling to the destination identity;
older protocols are rejected. Release artifacts use the explicit `diffx` project
name, independent of the checkout directory.

Schema 9 intentionally resets older versioned databases in one transaction under
the ownership lock. Current state survives subsequent starts; future or
unrecognized schemas are refused. Upgrade server and collectors together.
Schema 9 preserves current state across starts. Older, future and unrecognized
schemas are refused; opening a database never resets it. Durable server volumes
require an explicit reset decision: stop the owner, back up its state directory,
then select a fresh database path. No automatic schema migration is implemented.
Upgrade server and collectors together.
Versioned producer directories ignore old manual/hook pending payloads.
Stop old binaries before upgrading.

Remote configuration is owned by `config` and composed by `cmd/diffx-server`:
defaults, an explicitly selected read-only server JSON file, environment, then
explicit flags. Server startup does not discover, create or lock personal CLI
config. The binary defaults to loopback and a separate `./data/state.db`; the
container supplies overridable env defaults for its listener and `/data` volume.
One process owns the volume's SQLite database, credentials and durable queue.
68 changes: 65 additions & 3 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ List available tasks with `mise tasks`. Keep personal overrides in
| Build the dependency-free CLI at `dist/diffx` | `mise run build` |
| Build and locally install the CLI | `mise run install` |
| Build agent plugin adapters | `mise run plugins:build` |
| Start the container server with persistent SQLite | `mise run docker:up` |
| Stop the container server, preserving SQLite | `mise run docker:down` |
| Install Playwright Chromium | `pnpm test:browser:install` |
| Run web unit and browser tests | `pnpm test:web` |
| Run Go tests | `mise run test:go` |
Expand Down Expand Up @@ -115,9 +117,10 @@ runtime directories (`DIFFX_RUNTIME_DIR`) and in-memory state for lifecycle test
must not register inputs in the personal service. `DIFFX_EXIT_ON_STDIN_CLOSE`
is a foreground development-process lifecycle hook.

Stop running processes before upgrading. Schema 9 resets older versioned
databases transactionally on first open. Current databases survive restart;
in-memory state does not. Server `retentionDays` defaults to seven; fresh
Stop running processes before upgrading. Current schema 9 databases survive
restart; incompatible schemas are refused without resetting data. To start
fresh, back up the state directory while stopped and explicitly choose a new
database path. In-memory state does not survive restart. Server `retentionDays` defaults to seven; fresh
submissions apply the current setting, while exact retries preserve expiry.

For isolated production-user tests, start `diffx-server` with a temporary
Expand All @@ -126,6 +129,65 @@ startup reports its path. Stop the server before `user create --name NAME --stat
DB`, save its printed token and restart. Users share one database, while credentials
scope REST, MCP and events. Do not provision against the running personal service.

## Server development

Build both binaries with `mise run build`, then run server mode directly:

```sh
DIFFX_HOST=127.0.0.1 DIFFX_PORT=7981 DIFFX_STATE=./test-data/state.db ./dist/diffx-server
```

Or use the checked-in Compose setup:

```sh
mise run docker:up
docker compose logs server
docker compose cp server:/data/state.db.admin-token ./admin-token
mise run docker:down
```

The same server binary handles authentication, durable admission, workers and
queries in both setups. The image uses committed embedded assets; run
`mise run web:stage` before rebuilding it after UI changes.

| Setting | Binary default | Container default | Flag |
| ---------------------- | ------------------------------ | ------------------------------ | ----------------------- |
| `DIFFX_HOST` | `127.0.0.1` | `0.0.0.0` | `--listen HOST:PORT` |
| `DIFFX_PORT` | `7981` | `7981` | `--listen HOST:PORT` |
| `DIFFX_STATE` | `data/state.db` | `/data/state.db` | `--state PATH` |
| `DIFFX_ACCOUNT` | `admin` | `admin` | `--account NAME` |
| `DIFFX_RETENTION_DAYS` | `7` | `7` | `--retention-days DAYS` |
| `DIFFX_TOKEN` | Generated for persistent state | Generated for persistent state | None |

`DIFFX_TOKEN` supplies initial bootstrap credentials only; existing credentials
are never replaced. Omit it to generate a private `<database>.admin-token` file.
`memory` and `:memory:` disable persistence and require an explicit token.

Precedence is defaults → explicit JSON file → env vars → explicit flags. Use
`--config-file FILE` or `DIFFX_CONFIG_PATH` to select a server config. The file
supports `host`, `port`, `state`, `account` and `retentionDays`. It must exist and
can be mounted read-only; startup never creates it or a config lock. Personal
CLI config, `DIFFX_RUNTIME_DIR`, producer URL settings and external web assets
do not configure server mode. Restart to apply changes.

Compose keeps the SQLite database, sidecar files, ownership lock and generated
credential in the `diffx-data` volume. The root filesystem is read-only; the
image prepares `/data` for UID/GID 10001. If using a bind mount instead, make
the directory writable by that identity. Keep database files inside `/data`.
`mise run docker:down` preserves the volume; `docker compose down --volumes`
explicitly deletes it. Stop the server and back up the whole directory before
starting with a fresh database; a volume is not a backup.

Set `DIFFX_PORT=7982 mise run docker:up` for a different published and internal
port. Compose also accepts `DIFFX_ACCOUNT` and `DIFFX_RETENTION_DAYS` overrides.
For other settings, use image env overrides or a Compose override file.
Only one server may own a SQLite database; competing starts fail rather than
share ownership. SIGTERM stops HTTP and workers before releasing storage;
unfinished admitted jobs recover from the durable queue after leases expire.

This setup targets one server with SQLite. PostgreSQL, multiple replicas,
browser login and integrations remain separate future changes.

## Build pipeline

The production build generates API types, builds the web application, stages the
Expand Down
Loading
Loading