|
|
||
|---|---|---|
| docs | ||
| scripts | ||
| src | ||
| .gitignore | ||
| Cargo.lock | ||
| Cargo.toml | ||
| README.md | ||
g-lola — a Google Cloud you own
g-lola is the Google-Cloud-compatible sibling of lola (AWS).
Same idea, same discipline: one Rust binary on your own machine that speaks a
public cloud's API well enough that the vendor's own SDKs and IaC work
unmodified — so a workload can run here, on Google, or on both, and you
decide where.
It is not a proxy over Google, and not a mock. Cloud Storage writes bytes to this disk; Pub/Sub delivers messages this process holds; Cloud Run revisions are containers this box runs. Every answer is something this codebase actually did.
Why a sibling and not a mode of lola
The two clouds differ in shape, not just names, and the shape is what the SDKs test:
| AWS (lola) | Google (g-lola) | |
|---|---|---|
| resource names | ARNs | path-shaped: projects/p/buckets/b |
| tenancy | 12-digit account | project id + number |
| auth | SigV4 signatures | OAuth2 Bearer tokens, metadata server |
| errors | per-service XML/JSON | one envelope {"error":{code,message,status}} |
| async | per-service | Operation objects you poll |
| paging | per-service tokens | nextPageToken everywhere |
| IaC | CloudFormation / CDK | Deployment Manager → Terraform / Config Connector |
What can be shared is the engine underneath, and the plan is to lift it from
lola rather than rewrite: the tenancy model (lola's Tenant already carries a
project_id), the WAL and checkpointing, the blob store, the sandbox that runs
functions, the SQL engines, the DNS server, the IAM evaluator, backup/mirror.
Those would be ports, not inventions. Today, though, g-lola is a standalone
binary — Cargo.toml depends on axum/tokio/serde/hyper/rsa and nothing of
lola's — and each service module keeps its own store under
data_dir/<service>/. The Google-specific work is the surface.
The compatibility contract (what "speaks Google Cloud" means)
Inherited from the earlier gcp-cloud mission and enforced from the first
commit — see src/main.rs:
- Path-shaped names; the project is the tenancy unit.
Authorization: Bearer <token>— a static token fromGLOLA_TOKENS, or one minted byPOST /token(jwt-bearer / refresh_token grants) and the metadata server under/computeMetadata/v1/(src/iam.rs).- Google's JSON error envelope with RPC status names (
NOT_FOUND,ALREADY_EXISTS,UNAUTHENTICATED, …). - Long-running operations return an
Operationthe caller polls. nextPageTokenpaging.- JSON/REST first (verifiable with curl); gRPC second, generated from the same protobuf definitions (JSON/REST only so far — see Status).
The test for every service: Google's own SDK works against it unmodified.
Service priority (what to build, in order)
Ranked by what real workloads use first and what lola already has an engine for:
| # | Google service | lola engine to port | why first |
|---|---|---|---|
| 1 | Cloud Storage (JSON + XML API) | S3 blob store | every app touches it; the XML API is S3-shaped, the JSON API is what SDKs use |
| 2 | Cloud Run + Cloud Functions (2nd gen) | Lambda sandbox, services/ECS (containers), function URLs | the compute people actually deploy on Google |
| 3 | Firestore (Native mode) | new: document store (built as its own store, see Status) | the default app database on Google; no AWS equivalent to port |
| 4 | Pub/Sub | SQS + EventBridge | the eventing backbone; push subscriptions ≈ EventBridge targets |
| 5 | Secret Manager | SSM (sealed values) | trivial port, needed by every deploy |
| 6 | IAM (setIamPolicy/testIamPermissions) |
iam_policy evaluator | bindings model differs from AWS policies; the evaluator core is reusable |
| 7 | Cloud SQL (Postgres) | RDS/pglite services | one API over the same databases |
| 8 | Cloud DNS | Route 53 / knot | hybrid steering across clouds needs it on both sides |
| 9 | Cloud Logging + Monitoring | logs + cloudwatch (EMF-style structured logs) | observability parity |
| 10 | Artifact Registry | ECR face (ecr.rs) in front of registry.abot.at |
container images for Cloud Run |
Rows 1–6 are built (row 2 as Cloud Run only); rows 7–10 are not started.
Deployment Manager is dead; the IaC path is Terraform (the google
provider against a custom endpoint) and, for the CDK-shaped experience,
cdktf. That is the analogue of lola's CDK compatibility.
Capability matrix (all three faces, per resource type, generated from the code): https://matrix.abot.at/
Status (2026-08-30)
One module per service (src/<service>.rs), shared conventions in main.rs.
Six services answer; the "verified with" notes name the official client that
was run against the module (REST transport — there is no gRPC).
| service | state |
|---|---|
Cloud Storage JSON API (main.rs) |
buckets (list/insert/get/delete, ownership checked per project, names validated as on Google), objects (uploadType=media and multipart, custom metadata, alt=media and the /download/storage/v1 route, delete); object listing by prefix only (no delimiter, no paging); no copy/rewrite/compose/patch (copy is XML-only); bucket/object index (storage/_index.json) persisted across restarts |
Cloud Storage XML API (storage_xml.rs) |
S3-shaped PUT/GET/HEAD/DELETE /{bucket}[/{object}], listing with prefix/delimiter/paging (marker and list-type=2), server-side copy, x-goog-hash, x-goog-meta-*, Range, If-*/x-goog-if-* preconditions, x-amz-* accepted as x-goog-*; ?acl/?versioning/?uploads answer 501; shared store with the JSON API (verified with boto3 1.34: create/put/get/head/list with delimiter and paging/copy/delete; custom metadata comes back as x-goog-meta-*, as on Google) |
Cloud Run v2 (run.rs) |
services/revisions/operations; with docker or podman on PATH the revision's container really runs (env, ports, resources, startup probe; host ports from GLOLA_PORT+1000), otherwise the resource exists and its URI answers 503; /run/{project}/{loc}/{svc}/* proxy honouring roles/run.invoker (allUsers = public); :getIamPolicy/:setIamPolicy/:testIamPermissions per service; traffic[] is honoured — LATEST = the latest ready revision, REVISION targets by name (must exist, or be the template.revision of the same request), percent splits served by the proxy as a deterministic weighted round-robin (90/10 → 9 of every 10 requests), tag reachable at /run/{project}/{loc}/{tag}---{svc}/* (trafficStatuses[].uri); with one revision at 100% the service uri is its container's port, with a split it is the proxy; a PATCH makes a revision only when the template changes (updateMask=traffic never does, a failed latest revision is retried by a request that carries the template unless the template fixes a revision name), containers of serving revisions are reconciled at boot; retention: a revision's container runs while it has a share above 0% or a tag, is stopped (Active=RETIRED, host port released) once it has neither, and is started again from its template when traffic comes back (verified with google-cloud-run 0.16.1; the split with docker + traefik/whoami) |
Firestore (Native, firestore.rs) |
one database (default) per project (list/get); documents CRUD with updateMask and preconditions, list/paging, runQuery (field/composite/unary filters, orderBy, select, startAt/endAt, limit/offset), runAggregationQuery, batchGet, commit, batchWrite, transactions (beginTransaction/rollback), subcollections, listCollectionIds; own store (in-memory map + JSON-lines log per project) (verified with google-cloud-firestore) |
Pub/Sub (pubsub.rs) |
topics, subscriptions (pull + push with backoff), publish, pull/ack/modifyAckDeadline, ordering keys, filters, dead-letter topics, retry/expiration policies, seek by time (seek to a snapshot answers 501), IAM policy per topic/subscription (verified with google-cloud-pubsub 2.39) |
Secret Manager (secretmanager.rs) |
secrets (labels, annotations, topics, rotation, expireTime/ttl), versions (addVersion with dataCrc32c, access, enable/disable/destroy, delayed destruction via versionDestroyTtl), versionAliases, latest (newest version, any state), lists newest first, IAM policy per secret; payloads are sealed at rest (ChaCha20-Poly1305, a random nonce per version, the version name as associated data, files mode 0600) under a key from GLOLA_DATA_KEY or data_dir/secretmanager/.key — see Environment; plaintext files from before sealing are sealed in place when the store loads; :access returns the plaintext and its dataCrc32c (verified with google-cloud-secret-manager) |
IAM (iam.rs) |
service accounts + keys (Google JSON key files; only the public half is kept), :generateAccessToken, /token (jwt-bearer → access token; refresh_token with a GLOLA_TOKENS token), metadata-server token endpoint, getIamPolicy/setIamPolicy/testIamPermissions on projects (v1 and the v3 alias) and service accounts, predefined + custom roles, roles:queryGrantableRoles, permissions:queryTestablePermissions; signBlob/signJwt/generateIdToken answer 501 (verified with google-api-python-client iam/cloudresourcemanager v1, gapic resourcemanager_v3 + iam_credentials_v1, google-auth) |
Runs in production on metal as https://glola.abot.at behind Caddy (systemd
g-lola, tokens in /etc/g-lola/g-lola.env, state in /var/lib/g-lola).
The binary itself listens on 127.0.0.1:$GLOLA_PORT only.
Every service scopes by project (bearer → project; a foreign project answers
403 PERMISSION_DENIED, the XML API 403 AccessDenied), answers Google's
error envelope, and survives a restart (each module persists under
data_dir/<service>/). Both Storage faces resolve buckets through one helper
(bucket_for in main.rs): a bucket another project owns is 403 on every
JSON and XML route, a ?project= that is not the caller's is 403, and a
name another project holds is 409 ALREADY_EXISTS on create, and both faces
validate bucket names (Google's rules plus this endpoint's route roots), so a
name can never leave data_dir/storage
(scripts/storage-tenancy-matrix.sh runs that matrix with two tokens against
a live server). One caveat: IAM access tokens, Pub/Sub leases and Cloud Run
containers are runtime state — after a restart clients re-authenticate (401 →
refresh), unacked messages are redelivered, and containers are started again
from run/state.json.
Not yet: Cloud Functions (skeleton module, no routes), Cloud DNS,
Logging/Monitoring, Artifact Registry (skeleton modules), Cloud SQL, gRPC,
resumable uploads, Firestore listen/write streams, Pub/Sub snapshots and
schemas, service-account signing (signBlob/signJwt/generateIdToken).
GLOLA_TOKENS="tok-ada:sovereign-1" GLOLA_PORT=8080 cargo run
curl -H 'Authorization: Bearer tok-ada' 'http://127.0.0.1:8080/storage/v1/b?project=sovereign-1'
Environment: GLOLA_PORT (default 8080), GLOLA_DATA_DIR (default ./data),
GLOLA_TOKENS (token:project,…; each project named here exists — unset means
no auth and everything runs in project local), GLOLA_METADATA_PROJECT (the
project the metadata server speaks for; default the alphabetically first
configured project, else local), GLOLA_DATA_KEY (the Secret Manager at-rest
key, 64 hex chars = 32 bytes; unset, a key is generated once at
data_dir/secretmanager/.key, mode 0600, and reused — a malformed value fails
closed: payload reads and writes answer 500 rather than fall back to
plaintext). RUST_LOG filters the tracing output.
The key, said out loud: a key file next to the data protects against a copied
data directory or a backup that omits .key, not against theft of the disk it
sits on — for that pass GLOLA_DATA_KEY from a secret store (the systemd env
file) or encrypt the disk. Lose the key and every secret payload is
unrecoverable; back it up separately from data_dir.
Relationship to gcp-cloud
gcp-cloud (same org) was the first attempt: a Java/Quarkus "cell" grown on a
VPS rented from lola, iterated by agents, with the mission text this contract
is taken from. g-lola is the engine-level sibling of lola: Rust, one binary,
meant to share lola's stores and sandbox rather than starting from a JDK.