g-lola — a Google-Cloud-compatible mini cloud built the lola way: one Rust binary, real services, your own machine
Find a file
2026-08-30 19:10:04 +02:00
docs fix(run, secretmanager): honour Cloud Run traffic splits; seal secret payloads at rest 2026-08-30 17:01:52 +02:00
scripts storage: bucket names validated on the JSON face, delete-bucket/upload race closed on both faces 2026-08-30 18:54:33 +02:00
src Merge branch 'fix/glola-run-secrets' 2026-08-30 19:10:04 +02:00
.gitignore g-lola: scaffold — Google conventions, projects, Cloud Storage buckets; service map 2026-08-29 20:25:56 +02:00
Cargo.lock fix(run, secretmanager): honour Cloud Run traffic splits; seal secret payloads at rest 2026-08-30 17:01:52 +02:00
Cargo.toml fix(run, secretmanager): honour Cloud Run traffic splits; seal secret payloads at rest 2026-08-30 17:01:52 +02:00
README.md Merge branch 'fix/glola-run-secrets' 2026-08-30 19:10:04 +02:00

g-lola — a Google Cloud you own

g-lola is the Google-Cloud-compatible sibling of lola (AWS). Same idea, same discipline: one Rust binary on your own machine that speaks a public cloud's API well enough that the vendor's own SDKs and IaC work unmodified — so a workload can run here, on Google, or on both, and you decide where.

It is not a proxy over Google, and not a mock. Cloud Storage writes bytes to this disk; Pub/Sub delivers messages this process holds; Cloud Run revisions are containers this box runs. Every answer is something this codebase actually did.

Why a sibling and not a mode of lola

The two clouds differ in shape, not just names, and the shape is what the SDKs test:

AWS (lola) Google (g-lola)
resource names ARNs path-shaped: projects/p/buckets/b
tenancy 12-digit account project id + number
auth SigV4 signatures OAuth2 Bearer tokens, metadata server
errors per-service XML/JSON one envelope {"error":{code,message,status}}
async per-service Operation objects you poll
paging per-service tokens nextPageToken everywhere
IaC CloudFormation / CDK Deployment Manager → Terraform / Config Connector

What can be shared is the engine underneath, and the plan is to lift it from lola rather than rewrite: the tenancy model (lola's Tenant already carries a project_id), the WAL and checkpointing, the blob store, the sandbox that runs functions, the SQL engines, the DNS server, the IAM evaluator, backup/mirror. Those would be ports, not inventions. Today, though, g-lola is a standalone binary — Cargo.toml depends on axum/tokio/serde/hyper/rsa and nothing of lola's — and each service module keeps its own store under data_dir/<service>/. The Google-specific work is the surface.

The compatibility contract (what "speaks Google Cloud" means)

Inherited from the earlier gcp-cloud mission and enforced from the first commit — see src/main.rs:

  1. Path-shaped names; the project is the tenancy unit.
  2. Authorization: Bearer <token> — a static token from GLOLA_TOKENS, or one minted by POST /token (jwt-bearer / refresh_token grants) and the metadata server under /computeMetadata/v1/ (src/iam.rs).
  3. Google's JSON error envelope with RPC status names (NOT_FOUND, ALREADY_EXISTS, UNAUTHENTICATED, …).
  4. Long-running operations return an Operation the caller polls.
  5. nextPageToken paging.
  6. JSON/REST first (verifiable with curl); gRPC second, generated from the same protobuf definitions (JSON/REST only so far — see Status).

The test for every service: Google's own SDK works against it unmodified.

Service priority (what to build, in order)

Ranked by what real workloads use first and what lola already has an engine for:

# Google service lola engine to port why first
1 Cloud Storage (JSON + XML API) S3 blob store every app touches it; the XML API is S3-shaped, the JSON API is what SDKs use
2 Cloud Run + Cloud Functions (2nd gen) Lambda sandbox, services/ECS (containers), function URLs the compute people actually deploy on Google
3 Firestore (Native mode) new: document store (built as its own store, see Status) the default app database on Google; no AWS equivalent to port
4 Pub/Sub SQS + EventBridge the eventing backbone; push subscriptions ≈ EventBridge targets
5 Secret Manager SSM (sealed values) trivial port, needed by every deploy
6 IAM (setIamPolicy/testIamPermissions) iam_policy evaluator bindings model differs from AWS policies; the evaluator core is reusable
7 Cloud SQL (Postgres) RDS/pglite services one API over the same databases
8 Cloud DNS Route 53 / knot hybrid steering across clouds needs it on both sides
9 Cloud Logging + Monitoring logs + cloudwatch (EMF-style structured logs) observability parity
10 Artifact Registry ECR face (ecr.rs) in front of registry.abot.at container images for Cloud Run

Rows 16 are built (row 2 as Cloud Run only); rows 710 are not started.

Deployment Manager is dead; the IaC path is Terraform (the google provider against a custom endpoint) and, for the CDK-shaped experience, cdktf. That is the analogue of lola's CDK compatibility.

Capability matrix (all three faces, per resource type, generated from the code): https://matrix.abot.at/

Status (2026-08-30)

One module per service (src/<service>.rs), shared conventions in main.rs. Six services answer; the "verified with" notes name the official client that was run against the module (REST transport — there is no gRPC).

service state
Cloud Storage JSON API (main.rs) buckets (list/insert/get/delete, ownership checked per project, names validated as on Google), objects (uploadType=media and multipart, custom metadata, alt=media and the /download/storage/v1 route, delete); object listing by prefix only (no delimiter, no paging); no copy/rewrite/compose/patch (copy is XML-only); bucket/object index (storage/_index.json) persisted across restarts
Cloud Storage XML API (storage_xml.rs) S3-shaped PUT/GET/HEAD/DELETE /{bucket}[/{object}], listing with prefix/delimiter/paging (marker and list-type=2), server-side copy, x-goog-hash, x-goog-meta-*, Range, If-*/x-goog-if-* preconditions, x-amz-* accepted as x-goog-*; ?acl/?versioning/?uploads answer 501; shared store with the JSON API (verified with boto3 1.34: create/put/get/head/list with delimiter and paging/copy/delete; custom metadata comes back as x-goog-meta-*, as on Google)
Cloud Run v2 (run.rs) services/revisions/operations; with docker or podman on PATH the revision's container really runs (env, ports, resources, startup probe; host ports from GLOLA_PORT+1000), otherwise the resource exists and its URI answers 503; /run/{project}/{loc}/{svc}/* proxy honouring roles/run.invoker (allUsers = public); :getIamPolicy/:setIamPolicy/:testIamPermissions per service; traffic[] is honoured — LATEST = the latest ready revision, REVISION targets by name (must exist, or be the template.revision of the same request), percent splits served by the proxy as a deterministic weighted round-robin (90/10 → 9 of every 10 requests), tag reachable at /run/{project}/{loc}/{tag}---{svc}/* (trafficStatuses[].uri); with one revision at 100% the service uri is its container's port, with a split it is the proxy; a PATCH makes a revision only when the template changes (updateMask=traffic never does, a failed latest revision is retried by a request that carries the template unless the template fixes a revision name), containers of serving revisions are reconciled at boot; retention: a revision's container runs while it has a share above 0% or a tag, is stopped (Active=RETIRED, host port released) once it has neither, and is started again from its template when traffic comes back (verified with google-cloud-run 0.16.1; the split with docker + traefik/whoami)
Firestore (Native, firestore.rs) one database (default) per project (list/get); documents CRUD with updateMask and preconditions, list/paging, runQuery (field/composite/unary filters, orderBy, select, startAt/endAt, limit/offset), runAggregationQuery, batchGet, commit, batchWrite, transactions (beginTransaction/rollback), subcollections, listCollectionIds; own store (in-memory map + JSON-lines log per project) (verified with google-cloud-firestore)
Pub/Sub (pubsub.rs) topics, subscriptions (pull + push with backoff), publish, pull/ack/modifyAckDeadline, ordering keys, filters, dead-letter topics, retry/expiration policies, seek by time (seek to a snapshot answers 501), IAM policy per topic/subscription (verified with google-cloud-pubsub 2.39)
Secret Manager (secretmanager.rs) secrets (labels, annotations, topics, rotation, expireTime/ttl), versions (addVersion with dataCrc32c, access, enable/disable/destroy, delayed destruction via versionDestroyTtl), versionAliases, latest (newest version, any state), lists newest first, IAM policy per secret; payloads are sealed at rest (ChaCha20-Poly1305, a random nonce per version, the version name as associated data, files mode 0600) under a key from GLOLA_DATA_KEY or data_dir/secretmanager/.key — see Environment; plaintext files from before sealing are sealed in place when the store loads; :access returns the plaintext and its dataCrc32c (verified with google-cloud-secret-manager)
IAM (iam.rs) service accounts + keys (Google JSON key files; only the public half is kept), :generateAccessToken, /token (jwt-bearer → access token; refresh_token with a GLOLA_TOKENS token), metadata-server token endpoint, getIamPolicy/setIamPolicy/testIamPermissions on projects (v1 and the v3 alias) and service accounts, predefined + custom roles, roles:queryGrantableRoles, permissions:queryTestablePermissions; signBlob/signJwt/generateIdToken answer 501 (verified with google-api-python-client iam/cloudresourcemanager v1, gapic resourcemanager_v3 + iam_credentials_v1, google-auth)

Runs in production on metal as https://glola.abot.at behind Caddy (systemd g-lola, tokens in /etc/g-lola/g-lola.env, state in /var/lib/g-lola). The binary itself listens on 127.0.0.1:$GLOLA_PORT only.

Every service scopes by project (bearer → project; a foreign project answers 403 PERMISSION_DENIED, the XML API 403 AccessDenied), answers Google's error envelope, and survives a restart (each module persists under data_dir/<service>/). Both Storage faces resolve buckets through one helper (bucket_for in main.rs): a bucket another project owns is 403 on every JSON and XML route, a ?project= that is not the caller's is 403, and a name another project holds is 409 ALREADY_EXISTS on create, and both faces validate bucket names (Google's rules plus this endpoint's route roots), so a name can never leave data_dir/storage (scripts/storage-tenancy-matrix.sh runs that matrix with two tokens against a live server). One caveat: IAM access tokens, Pub/Sub leases and Cloud Run containers are runtime state — after a restart clients re-authenticate (401 → refresh), unacked messages are redelivered, and containers are started again from run/state.json. Not yet: Cloud Functions (skeleton module, no routes), Cloud DNS, Logging/Monitoring, Artifact Registry (skeleton modules), Cloud SQL, gRPC, resumable uploads, Firestore listen/write streams, Pub/Sub snapshots and schemas, service-account signing (signBlob/signJwt/generateIdToken).

GLOLA_TOKENS="tok-ada:sovereign-1" GLOLA_PORT=8080 cargo run
curl -H 'Authorization: Bearer tok-ada' 'http://127.0.0.1:8080/storage/v1/b?project=sovereign-1'

Environment: GLOLA_PORT (default 8080), GLOLA_DATA_DIR (default ./data), GLOLA_TOKENS (token:project,…; each project named here exists — unset means no auth and everything runs in project local), GLOLA_METADATA_PROJECT (the project the metadata server speaks for; default the alphabetically first configured project, else local), GLOLA_DATA_KEY (the Secret Manager at-rest key, 64 hex chars = 32 bytes; unset, a key is generated once at data_dir/secretmanager/.key, mode 0600, and reused — a malformed value fails closed: payload reads and writes answer 500 rather than fall back to plaintext). RUST_LOG filters the tracing output.

The key, said out loud: a key file next to the data protects against a copied data directory or a backup that omits .key, not against theft of the disk it sits on — for that pass GLOLA_DATA_KEY from a secret store (the systemd env file) or encrypt the disk. Lose the key and every secret payload is unrecoverable; back it up separately from data_dir.

Relationship to gcp-cloud

gcp-cloud (same org) was the first attempt: a Java/Quarkus "cell" grown on a VPS rented from lola, iterated by agents, with the mission text this contract is taken from. g-lola is the engine-level sibling of lola: Rust, one binary, meant to share lola's stores and sandbox rather than starting from a JDK.