Run the front door on VMs

Three virtual machines behind a load balancer, forming one raft cluster, with no Kubernetes anywhere. Nothing in the data plane talks to a Kubernetes API, so this is the same binary and the same config the chart installs. You write the cluster block yourself, and the load balancer in front is now yours to configure.

For the Kubernetes version see Deploy the front door with Helm.

Install

Each release publishes a .deb and an .rpm per architecture, and a tar.gz for anything else. The package installs the binary at /usr/bin/kirto-gate, the systemd unit, the kirto-gate service account and an empty /etc/kirto-gate. It starts nothing, because there is no config yet.

$ sudo dpkg -i kirto-gate_1.0.0_linux_amd64.deb      # or: rpm -i ...

Verify what you downloaded first. One signature covers every artifact, because it is over the checksum file that lists them:

$ cosign verify-blob kirto-gate_1.0.0_checksums.txt \
    --signature kirto-gate_1.0.0_checksums.txt.sig \
    --certificate kirto-gate_1.0.0_checksums.txt.pem \
    --certificate-identity-regexp '^https://github.com/kirto-dev/gate/\.github/workflows/release\.yml@refs/(tags/v.+|heads/main)$' \
    --certificate-oidc-issuer https://token.actions.githubusercontent.com
$ sha256sum -c kirto-gate_1.0.0_checksums.txt --ignore-missing

From the tarball instead, the unit is in it and the rest is manual:

$ tar xzf kirto-gate_1.0.0_linux_amd64.tar.gz
$ sudo install -m 0755 kirto-gate /usr/bin/kirto-gate
$ sudo install -m 0644 systemd/kirto-gate.service /etc/systemd/system/
$ sudo useradd --system --no-create-home --shell /usr/sbin/nologin kirto-gate
$ sudo install -d -m 0750 -o root -g kirto-gate /etc/kirto-gate

kirto-gate version prints the build. That is the first thing to paste into a bug report.

The binary is built without cgo but is not static. The region prefilter the pii detector runs on, and the optional ML detector tier, load shared libraries through dlopen at runtime. Any ordinary Linux userland is fine. A FROM scratch container is not. The image ships on distroless rather than distroless static.

The prefilter is Vectorscan, and the binary expects it on the host. The .deb and .rpm depend on libvectorscan5 and vectorscan respectively. From the tarball, install it yourself (apt install libvectorscan5, dnf install vectorscan). A host that cannot have it says prefilter: pure on each pii detector and gets the built-in pass, which is slower on ordinary text and unbounded on adversarial text. A host that has neither the library nor that line does not start, and the error names the setting.

The config

One file per host, /etc/kirto-gate/kirto-gate.yaml, and it configures that host’s process alone. The deployment document is not in it: once the cluster is up, it is written through the admin API, from any host, and lands on every node. Keep it in your configuration management as a file of its own and write it from there.

The cluster block differs per host. This is the part the chart writes for you from the StatefulSet’s ordinals, and the part where a mistake is quiet.

listen: :8484

admin:
  local_socket: /run/kirto-gate/admin.sock

cluster:
  node_id: kirto-gate-1              # kirto-gate-2, kirto-gate-3 on the others
  data_dir: /var/lib/kirto-gate/raft
  secret: ${KIRTO_GATE_CLUSTER_SECRET}
  raft_addr: 0.0.0.0:7000
  forward_addr: 0.0.0.0:7001
  advertise: kirto-gate-1.internal:7000
  forward_advertise: kirto-gate-1.internal:7001
  peers:
    - { id: kirto-gate-1, raft_addr: kirto-gate-1.internal:7000, forward_addr: kirto-gate-1.internal:7001 }
    - { id: kirto-gate-2, raft_addr: kirto-gate-2.internal:7000, forward_addr: kirto-gate-2.internal:7001 }
    - { id: kirto-gate-3, raft_addr: kirto-gate-3.internal:7000, forward_addr: kirto-gate-3.internal:7001 }

Three rules cover most of what goes wrong:

• peers is the same list on all three hosts, and includes the host itself. It seeds the first configuration and nothing after. Once the cluster exists its membership lives in the raft log, and editing this list later cannot change it.
• peers is exclusive with join and bootstrap. A fixed fleet uses peers. A node entering a cluster that is already running uses join. bootstrap starts a cluster of one. Setting peers alongside either is refused at startup, because the mistake it prevents is expensive to unpick. That mistake is a node that meant to join and instead started a second cluster sharing a secret with the first.
• Bind wide, advertise by name. raft_addr is what this process listens on. advertise is what the other two dial and what goes into the raft log. Advertising 0.0.0.0 records an address nobody can reach.

Keep the number of voters odd. Three tolerates one failure. Four tolerates one failure and needs three votes to do it.

The deployment

The deployment document is a file of its own in your configuration management, deployment.yaml, and it is written to the cluster rather than to a host:

auth:
  virtual_keys: true
providers:
  - name: openai
    base_url: https://api.openai.com
    api_key: ${OPENAI_API_KEY}

Once the cluster is up, write it from any host through the local admin socket, as a source named for whoever maintains it:

$ sudo kirto-gate config apply -socket /run/kirto-gate/admin.sock -source platform deployment.yaml
source platform created at revision 1

It lands on all three nodes. A later edit is the same command, which replaces the source at the revision it read and writes nothing when the stored document is already these bytes, so a configuration-management run can apply it on every pass. No host restarts for it. The data plane answers 503 until this first write.

The command is the source’s manager. An edit somebody made to the same source through the console or the API since the last apply is refused, with the objects they changed named, and -force writes over them. kirto-gate config diff -source platform deployment.yaml shows what an apply would change before it runs, -dry-run runs every check on the cluster and writes nothing, and kirto-gate config rollback -source platform -to REV writes a retained revision back.

Credentials go in an environment file, not the config:

$ sudo install -m 0640 -o root -g kirto-gate /dev/null /etc/kirto-gate/env
$ printf 'OPENAI_API_KEY=sk-...\nKIRTO_GATE_CLUSTER_SECRET=%s\n' \
    "$(openssl rand -hex 32)" | sudo tee /etc/kirto-gate/env >/dev/null

The cluster secret is the same string on all three hosts. It authenticates write forwarding and membership changes between them, and each host derives from it the certificate authority under which the three speak TLS to each other on ports 7000 and 7001. No certificate is issued or distributed for those ports.

The systemd unit

The package installs it at /usr/lib/systemd/system/kirto-gate.service. The tarball ships the same file under systemd/. Copy it to /etc/systemd/system/ before editing, a package upgrade replaces the one it owns.

A package upgrade restarts a running unit onto the new binary and leaves a stopped one stopped. Upgrade a cluster’s nodes one at a time, each after the last has rejoined: all of them at once is a moment without quorum.

Four things about it are worth knowing:

• There is no ExecReload. Gate reads its file at start and nowhere else, so an edit to it is systemctl restart kirto-gate. The file configures the process and holds no deployment; the deployment is written through the admin API once the cluster is up, and a config-management run that edits it is a source write, not a restart. See the file configures the process, and nothing else. A file that does not parse stops the start, so run kirto-gate check on the edit before restarting.
• StateDirectory=kirto-gate is the raft log at /var/lib/kirto-gate. systemd creates it, owns it to the service user and keeps it across restarts and upgrades. A node whose log vanishes rejoins as a stranger.
• TimeoutStopSec=60, because leadership transfers on a clean stop and that is worth waiting for rather than making the cluster elect its way out.
• AmbientCapabilities=CAP_NET_BIND_SERVICE so the gateway can hold 443 without running as root.

MemoryDenyWriteExecute=yes is deliberately left out. The optional native detector tier maps executable pages through dlopen, and the seccomp filter that option installs blocks it. The gateway starts and then fails to load a detector, reported as a missing library rather than as a sandbox refusal. Do not add it.

The unit reads /etc/kirto-gate/env for the credentials the config references, and reads it optionally, so a deployment with none still starts.

$ sudo systemctl enable --now kirto-gate
$ journalctl -u kirto-gate -f

TLS

Three shapes, and the choice decides who owns the certificate.

Gate terminates, ACME with DNS-01. The gateway obtains and renews its own certificate. The raft leader is the only node that issues, storage replicates through the log, so all three serve the same certificate. DNS-01 needs no inbound reachability, so it works on a private VM where HTTP-01 cannot. If your CA is not Let’s Encrypt it probably wants External Account Binding.

Gate terminates, static files. tls.cert_file and tls.key_file, rotated by whatever already rotates certificates on these hosts. On AWS this is also the path for an ACM certificate issued as exportable: you export the chain and key onto the host and re-export when ACM renews, because nothing pushes a renewed ACM certificate to an instance.

The load balancer terminates. An ALB or Application Gateway holding the certificate, Gate serving plain HTTP behind it. This is the only way to use a normal, non-exportable ACM certificate, and it means the gateway is no longer the TLS owner. It receives whatever the load balancer decided to forward.

The load balancer

Publish only the data plane. Ports 7000 and 7001 are the raft transport and write forwarding between the three hosts. Port 8485 is the control-plane API. Port 9464 is /metrics. None of them belong on a public listener, put them in a security group that admits the three hosts, and your Prometheus for 9464, and nothing else.

Port	What	Who reaches it
8484	Data plane, /v1/, /mcp/, /healthz	The load balancer
7000	Raft transport, mutual TLS under the cluster secret	The other two hosts
7001	Write forwarding, membership, mutual TLS under the cluster secret	The other two hosts
8485	Control-plane API, when enabled	Your own network, never the front door
9464	/metrics (metrics_listen)	Your Prometheus, or nobody if you push OTLP instead

Health-check the load balancer against /healthz on 8484. It answers from the data plane alone and never grows a dependency on a config store or the cluster, so it stays a liveness answer rather than a readiness one.

Raise the idle timeout, or streams will be cut

This is the failure that does not exist on Kubernetes, where a Service is layer 4 and nothing between client and gateway holds an opinion about how long a response may take.

A load balancer closes a connection that has been idle too long, and the defaults are shorter than a model takes to think. Time to first token on a long reasoning request routinely passes a minute under load. During that window the gateway has sent response headers and no body, which is what an idle connection looks like. The client sees a truncated stream and the gateway sees a write to a closed connection. Neither says “load balancer”.

In front	The setting	Default	Notes
AWS ALB	idle_timeout.timeout_seconds	60s	Layer 7: counts HTTP data. Raise it, the maximum is 4000s
AWS NLB	TCP idle timeout	350s	Layer 4, and TCP keepalive resets it
GCP external ALB	backend service timeoutSec	30s	Caps the whole response, not just idleness
Azure Application Gateway	backend requestTimeout	short	Per-request; raise it for streaming

Two things follow.

TCP keepalive does not fix the layer-7 case. Gate’s listener already has it. Go enables TCP keepalive on every accepted connection by default, probing after 15 seconds idle, but an ALB terminates TCP. Probes on the load-balancer-to-gateway leg never appear on the client leg, and the ALB’s idle timer counts HTTP data, not bare ACKs. Keepalive is the right tool in front of an NLB, which passes the flow through, and no help in front of an ALB.

The gateway will not paper over it. A proxy can hold a connection open by writing SSE heartbeat comments during the gap, and Gate does not, because those are bytes the provider never sent. See Passthrough. The timeout is the load balancer’s policy. Change it there.

WebSocket sessions, OpenAI Realtime, Gemini Live, have the same timeout applied to the upgraded connection, but need nothing from you. The endpoints ping each other, and the gateway forwards those frames verbatim like every other frame.

Check the cluster formed

Run this command:

$ journalctl -u kirto-gate | grep 'cluster joined'

All three should name the same leader. The same facts are gauges on each host’s /metrics (port 9464), and they are the ones to alert on:

Series	Healthy	Otherwise
kirto_gate_cluster_leader_known	1 on every host	0: this host is partitioned, or the cluster has lost quorum
kirto_gate_cluster_is_leader	sums to 1	0: no leader; 2: a partition, briefly
kirto_gate_cluster_members{suffrage="voter"}	3	fewer: a host is gone and quorum is thinner than it looks
kirto_gate_config_rebuild_failed	0 on every host	1: this host could not build the stored document and serves the previous one; its GET /admin/v1/config says why

The policy file and the issuers are per host, read at start, and a host that missed an edit to either enforces something different from its neighbours. Every node reports a hash of each to the cluster, and the members listing shows them side by side:

$ kirto-gate admin GET /admin/v1/cluster | jq '.members[] | {id, policy_hash, issuers_hash, served, rebuild_error}'

Three equal pairs is a fleet that agrees. kirto-gate check prints the same two hashes for a node file before it is deployed, so a host can be compared before it joins. served is the store sequence each host’s configuration was built from, reported after every rebuild: a kirto-gate config apply that landed at revision N is serving on every host whose served is at least N, and a host below it with a rebuild_error is the one whose environment lacks what the document names.

Autoscaling groups

Put the three hosts in an ASG for replacement, not for scaling. Fix min, max and desired at 3. Quorum wants a stable odd fleet, and a group that scales to four voters has bought nothing, four still tolerates one failure.

A replaced instance comes up with a blank disk and a new address, so it cannot use peers (that seeds a first configuration, and one already exists). Give the replacements join instead, naming the forward addresses of the members that are still up, and advertise set to the new instance’s own name. It is admitted as a nonvoter, catches up, and is promoted. Set dead_after on the fleet if you want the members that were replaced to stop counting toward quorum without an operator saying so.

If a whole group is lost at once and the survivors cannot elect, joining is no longer available to them either, because a join is a write. Recovery runs offline on one survivor’s disk: see Recover a cluster that has lost quorum.