Trace Town reads OpenTelemetry. What matters is that your services ship OTel metrics somewhere with a Prometheus query API — not who stores them.
This is a young document and it will grow as more systems get connected. If yours is not here and you get it working, we would like to know.
Open the town, then Data → Connect a source. Two steps: which provider, then where it is. Everything can be changed afterwards.
Signed in, the credential is encrypted and stored against your account, and our server does the fetching — so it survives a reload and a team can share a town without sharing a key with a person. Some providers, Grafana Cloud among them, only work this way: they send no CORS headers, so a browser cannot call them directly. Azure Monitor goes further and has no browser path at all, because what it wants is a client secret exchanged for a token.
Signed out, the key is held in that tab's memory and nowhere else. You retype it after a reload. That is not a crippled mode — for someone pointing at a Prometheus on their own laptop it is the right one — but it only works where the provider allows cross-origin requests.
Also Mimir, Thanos, Victoria Metrics — anything speaking the Prometheus query API.
Trace Town/api/v1/query hangs off, usually http://your-prometheus:9090
Browsing signed out, Prometheus needs --web.cors.origin
set or it will refuse the browser. Signed in, our server does the
fetching and this does not arise.
Grafana Cloud sends no CORS headers, so it cannot be read from a browser directly.
instanceID:token — with the colon, which is what sends HTTP Basicmetrics:read
scope. Copy it — it is shown once. If you lose it, delete it
under Security → Access Policies and make another; the value
cannot be retrieved.
instanceID:token as the credential. Use the
Show button to check it before connecting.
Self-hosted or cloud. Brings its own logs, so there is nothing else to point at.
8080SigNoz accepts two authentication schemes and which one a route wants varies by version. A 403 is more often the key’s account lacking the admin role than a wrong key.
Reads the spans your services already send. Logs come from the same query language, so there is nothing else to point at.
https://api.newrelic.com, or https://api.eu.newrelic.com for an EU accountNRAK-
It has to be a user key, and this is the thing that catches
everyone. New Relic issues several kinds and only one of them can
ask questions. A licence or ingest key — NRII-, or forty
hex characters — is for getting data in and has no read path
at all, so it comes back 403 rather than “wrong
key”. It is a perfectly valid key that simply cannot be used
here.
NRAK-.
curl -sS -X POST https://api.newrelic.com/graphql \
-H 'Content-Type: application/json' \
-H "API-Key: $KEY" \
-d '{"query":"{ actor { accounts { id name } } }"}'api.eu.newrelic.com and try again. Those two failures
are indistinguishable from the response, so rule out both.
accountId:NRAK-…, joined by a colon. A
key that sees exactly one account needs no prefix — Trace Town
works it out. One that sees several is not guessed for; it will
tell you the ids and ask.
An empty town against an account full of data usually means APM,
not OTel. Trace Town groups services by
service.name, which is what an OpenTelemetry resource
sets; New Relic’s own APM agents historically used
entity.name or appName instead. Check with
SELECT count(*) FROM Span FACET `service.name`, `entity.name`
— if the first column is empty and the second is not, that is the
answer, and we would like to hear from you, because we have chosen
not to guess at it rather than guess wrong.
Logs need nothing extra. They are the same query language against the same key, so the Logs tab works as soon as the connection does.
Application Insights, read out of its Log Analytics workspace with KQL.
Application Insights API keys were retired on 31 March 2026. What replaces them is a Microsoft Entra app registration, which is why this is the longest setup here and why it cannot be done from a signed-out browser: a client secret is exchanged for a token, and that exchange belongs on a server.
*/read on
everything in scope. Trace Town reads four tables, so grant four
tables. Save this as trace-town-reader.json, replacing
the scope with your own workspace:
{
"Name": "Trace Town Reader",
"Description": "Query only the tables Trace Town draws a town from.",
"Actions": [
"Microsoft.OperationalInsights/workspaces/read",
"Microsoft.OperationalInsights/workspaces/query/read",
"Microsoft.OperationalInsights/workspaces/query/AppRequests/read",
"Microsoft.OperationalInsights/workspaces/query/AppDependencies/read",
"Microsoft.OperationalInsights/workspaces/query/AppTraces/read",
"Microsoft.OperationalInsights/workspaces/query/AppExceptions/read"
],
"AssignableScopes": [
"/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.OperationalInsights/workspaces/<workspace>"
]
}az role definition create --role-definition trace-town-reader.jsonsharedKeys — so it cannot send
data or hand out an ingestion key — and no table but those four.
tenant, appId and
password: those are the three values Trace Town asks
for, and the password is shown once.
WORKSPACE=$(az monitor log-analytics workspace show \
--resource-group <rg> --workspace-name <workspace> --query id -o tsv)
az ad sp create-for-rbac --name trace-town-reader \
--role "Trace Town Reader" --scopes "$WORKSPACE"az monitor log-analytics workspace show \
--resource-group <rg> --workspace-name <workspace> --query customerId -o tsvGive a new role a few minutes. A freshly granted assignment took about three minutes to reach the query API in our own testing, and until it does, every query fails with insufficient access — which reads exactly like getting the role wrong. Wait before you change anything.
A table you did not grant returns nothing rather than an error.
We measured this: a table with 3,774 rows in it came back empty, with
a 200, for a principal that had not been granted it. So
if you tighten the role further and the town comes up half empty,
suspect the role before you suspect your data — nothing anywhere will
tell you that is what happened.
The service map needs the AppDependencies table to have
rows in it, which is written by automatic dependency collection. If
it is empty, a service instrumented only for inbound requests is all
you have, and the town comes up as buildings with no roads. Check
before you start: AppDependencies | take 5.
If your queue workers come up fed by nothing, that is a
different gap and worth knowing about. A queue hop is only half in
that table: publishing to a broker writes a dependency, but the
delivery is a consumer span, which Azure records as an inbound
request. Trace Town reads the broker from that request's
Source, which the exporter fills in from the span's
server.address — and plenty of messaging
instrumentation does not set it. Check with
AppRequests | where isnotempty(Source).
A companion, not a source of its own — pair it with a metrics provider that keeps no logs.
Standard OpenTelemetry, and nothing bespoke. Trace Town reads whichever of these it finds:
http.server.request.duration the service itself — rate, errors, latency
http.client.request.duration service to service, and to third parties
db.client.operation.duration service to datastore
messaging.client.sent.messages service to broker
messaging.client.consumed.messages broker to worker
If your collector runs the spanmetrics connector, that is
found automatically and preferred — it covers every service that emits a
span rather than only the ones serving HTTP, which is the difference
between seeing your queue workers and not.
Set service.namespace on your services and the town zones
itself into neighbourhoods with a signpost at the edge of each. It is the
single highest-value attribute you can add.
Trace Town reads telemetry you already have. This is the whole of what it takes, what it keeps, and what it never sees — written against the code rather than as a policy, so if any of it stops being true that is a bug and we would like to hear about it.
Pointed at a Prometheus without an account, your browser queries it directly. There is no server in the path, the credential lives in that tab’s memory until you reload, and nothing about your system is transmitted anywhere. Everything below is about signed-in connections, where our server does the fetching on a schedule.
Your traces are not fetched. Trace Town asks your provider for the summary it has already made out of them — how many requests, how many failed, how slow the slow ones were, and which service called which. Individual traces and the spans inside them stay where they are.
Those are aggregates your provider has already computed. None of it is written to a database and none of it is logged.
It is not quite true to say a snapshot is fetched and dropped, so here is what actually happens. Your provider is polled once per connection, not once per viewer, and everyone watching that town is sent the same snapshot — which is why two people in an incident see the same town rather than two towns fifteen seconds apart, and why a crowd costs your Prometheus the same as one person.
That means the most recent snapshot is held in our server’s memory between polls, so somebody opening the page mid-interval is shown the town immediately instead of waiting. A second, shorter memory underneath it stops viewers on slightly different timers each starting their own query.
All of it is process memory only. It never reaches a disk, it does not survive a restart, it is discarded when the last person stops watching, and it is thrown away whenever the connection is edited. It is shared between viewers of the same connection, and a connection belongs to one team — there is no path by which one team’s data reaches another.
Service names are the only thing from your telemetry that we keep, and only because an arrangement has to remember which building you moved. No metrics, no rates, no history.
A saved credential is write-only. No route returns it, not even masked — the API will tell you whether one is set and nothing more. There is no path from a town back to a key for anybody, including whoever typed it.
When you open a building’s Logs tab we fetch recent lines for that one service and show them to you. The timestamp, the severity and the message go to your browser, are held in memory for the couple of seconds described above, and are written down nowhere.
Log messages are your application’s own output, passed through verbatim — so whatever your code writes to a log is what crosses the wire, including anything it should not have written. Nothing is fetched until somebody opens that tab on that service. If that is a line you would rather not cross, connect a metrics provider without a log store and the tab has nothing to show.
Individual traces, span attributes, request or response bodies, headers, query strings, cookies, and URLs beyond a route name. The snapshot every provider is reduced to has no field for any of them.
Every request our server makes to a URL you gave it goes through one guard: https only, every resolved address must be a public one, redirects are followed by hand so each hop is checked like the first, and your credential is dropped the moment a redirect changes origin — a token for your Grafana is for your Grafana.
A plain http URL is refused when you save it, not on the
first poll. Every target we reach is on the public internet, so http
would mean your credential crossing it unencrypted every fifteen
seconds. Running Trace Town inside your own network is the one
exception, because there it is your own network.
The narrowest credential that works is the right one, and the provider pages above say what that is for each: a read-only user key for New Relic, and for Azure a custom role granting the four tables Trace Town reads and nothing else — no write, and no access to ingestion keys.
Only if you accept the cookie notice; nothing is loaded until you do. What it can send is a fixed list of categories — which provider, roughly how big a town, which of four kinds of failure. It carries no URLs, no hostnames, no service names and no error text, because it has nowhere to put them.
Sign-in is Firebase Authentication (Google). The service runs on Railway behind Cloudflare. Analytics, once accepted, is Google Analytics.
A service's shape is guessed from its name and from how it is used, and you can correct it — select a building and change its kind. Height is request rate on a log scale, so five a second still gets you a skyline.
Every figure on the roads is a call. Blue is a clean one and red is a failed one, and how fast they move is the p99 of whatever they are going to see. What they are carrying says something the roads on their own never could: whether anybody is waiting for an answer.
Two of those go to the same building and mean opposite things, which is why the depot has both a counter and loading bays, and why a factory has a door for people and a shutter for vans. You can watch which one a given call uses.
Every signal has one visual meaning, and nothing else shares it.
Requests, errors and p99 are the town's totals, and they mean what you would expect. Two do not:
Population is an accumulator, not a measurement. It counts the traffic this town has served while healthy, and nothing else. Serve a lot, well, for a long time, and it grows. An incident stops it growing rather than knocking it down — the repair sites carry the bad news instead. It is a session's worth: a reload resets it, and so does moving to a different town, because it belongs to the town that earned it.
Level is the same number on a gentler curve, so it keeps moving once population has grown too large for a change in it to show.
Repairs counts the services currently degraded or under repair, out of the total.
An empty town. Usually the query window: a provider only reports what had traffic, and a quiet service reports nothing at all. Widen the window in the connection settings before assuming anything is broken.
Services that come and go. Same cause. Trace Town remembers a service that has gone quiet rather than demolishing its building, but a window narrower than your traffic will still make the town restless.
Nothing but buildings, no roads. Roads come from client metrics —
http.client, db.client,
messaging.client. Without them we know your services exist
but not what they call.
Missing workers. A queue worker never serves an HTTP request, so
it has no http.server metrics and appears only as the far
end of somebody else's. Running the collector's
spanmetrics connector fixes this properly.
A 401 or 403. Check whether the provider wants the credential in a different header, and whether the key's account has the role it needs.