Picture the scene. You're driving in Jerusalem, you trust the blue line, you take the turn it tells you to take, and thirty seconds later you're nose to nose with a delivery truck in an alley that was never meant for two cars, let alone one.
And the app, of course, is completely unbothered. It just says "recalculating."
Daniel wrote in this week, and he's got a whole thing about exactly this. He's a Google Maps man. Says Waze is too aggressive, and he lives in the one country where Waze is basically the national religion, so that's a real position to hold. His complaint is specific: Waze will happily send you down a street that turns out to be blocked off, or one-way, or just barely wide enough to fit a car in one direction, all in the name of shaving a few seconds.
Which is not a bug, by the way. We'll get there.
Right. So he started pulling on the thread. He'd always assumed the magic was the traffic layer, the live positioning data from everyone's phones, but he realized that even if you strip that away entirely, there's still a basic program underneath that takes an origin, a destination, and a travel mode, and hands you back a route. If you tell it you're on foot, it doesn't send you down a highway. And then on top of that you've got preferences. Avoid tolls. In Israel, avoid Area A in the West Bank.
That one's a hard exclusion, not a preference, but yes.
His guess was that this needs two big things. One, an accurate GIS layer with lane information and road suitability data. Two, the actual routing algorithm, which he thinks Google provides as an API that third-party apps can just call. And his question is how both of those work under the hood, and how they come together to produce the thing we all take for granted. Because if you're driving Jerusalem to Tel Aviv, there are several viable routes at any given moment, and the app picks one in a split second.
That split second is the whole story. Everything interesting is in that split second.
So let's start with what the app actually needs to know before it can route anything at all.
The cleanest way to think about it is that a map, for routing purposes, is not a picture. It's a graph. Intersections are nodes. The stretches of road between them are edges. And routing is just the problem of finding the lowest-cost path through that graph from your node to the destination node.
Which sounds almost trivial when you say it like that. It's a solved problem, we've had shortest-path algorithms since the fifties.
It's solved in the same way that chess is solved. The rules are simple, the search space is not. A road network for a single country is millions of nodes and edges. Continental scale, you're into the tens of millions. And you're not doing one query, you're doing hundreds of thousands a second across all your users, and each user wants an answer before they've finished pulling out of the driveway.
So there are two layers. The map itself, the data, and then the engine that computes over it.
Two layers, and then a third that sits on top and changes every minute. The static map, the routing algorithm, and then live conditions. Daniel's instinct that traffic is a second layer on top of a basic layer is exactly right.
Good. Then let's do them in order. The map first. What is actually in a GIS layer?
A road network graph is not lines on a map. Every edge carries attributes. Road class, so highway versus arterial versus residential versus service road. Speed limit. Number of lanes. Whether it's one-way. Turn restrictions, and this is a big one, because a no-left-turn is not a property of any single edge, it's a property of a pair of edges at a node.
So the graph has to encode relationships between edges, not just the edges themselves.
Then access restrictions. Private roads, toll roads, seasonal closures, weight limits, height limits. And then travel-mode suitability, which is the thing that stops a walking route from putting you on a motorway. The same physical road can be a valid edge for a car and an invalid edge for a pedestrian, or valid for a bicycle and invalid for a truck.
So when Daniel says "if you say you're on foot, it's not going to take you down a highway," what's actually happening is the pedestrian query is running over a subgraph where highway edges have been filtered out.
Filtered out or given infinite cost, depending on the implementation. Same result. The mode you select doesn't change the algorithm, it changes which edges the algorithm is allowed to see.
Where does all this data come from?
Three sources, blended. Government road data, which is authoritative but slow. Satellite and street-level imagery, which is how a lot of geometry gets traced in the first place. And then crowd-sourced contributions. OpenStreetMap is the canonical example. Volunteers trace roads from aerial imagery and tag the attributes by hand. It's remarkable. A global road graph maintained by people who mostly aren't getting paid.
And the commercial providers?
Google, HERE, TomTom, they blend all three. Proprietary survey data, government feeds, and user reports. If you've ever tapped "report" in Waze because a road was closed, you contributed to that layer.
The thing that strikes me is that this map is never finished. Roads close. New ones open. A turn restriction changes because the city put up a sign. The map is a living document that's always slightly behind reality.
Always. And that gap, between the map and the road, is going to be the villain of this episode. Hold onto it.
So we've got the graph. Now the engine. What's the textbook answer?
Dijkstra's algorithm. Nineteen fifty-nine, Edsger Dijkstra, and it's beautiful. You start at the origin node, you expand outward, always processing the node with the lowest cumulative cost so far, and you keep going until you reach the destination. It's guaranteed to find the shortest path.
And it's too slow.
For a continental graph, yes. Dijkstra explores in every direction, like a ripple in a pond. From Jerusalem to Tel Aviv, it would waste enormous effort exploring south toward Beersheba and north toward Haifa before it ever committed to going west. It's correct, it's just doing ten thousand times more work than it needs to.
So what's the fix?
This is the part I find elegant. Contraction Hierarchies. The idea is that you preprocess the graph offline, once, and you build a hierarchy into it. You rank the nodes by importance. A motorway interchange is important. A dead-end residential cul-de-sac is not.
And then you contract the unimportant ones.
You remove them from the graph, and wherever removing a node would have broken a shortest path, you insert a shortcut edge that preserves the distance. So a quiet residential street gets contracted away, and if it was on the fastest path between two junctions, you replace it with a single shortcut edge connecting those junctions directly.
So the detail gets compressed out, but the distances stay correct.
Correct distances, vastly fewer nodes. You do this in order, least important first, and you end up with a hierarchy. The bottom is all the little local streets, the top is just the motorways and the major arteries.
And at query time?
At query time you run a bidirectional search that only ever moves upward in the hierarchy. You search from the origin going up, from the destination going up, and you meet somewhere near the top. You never explore the local streets at all unless your origin or destination is actually on one.
So the reason my phone can route Jerusalem to Tel Aviv in milliseconds is that it isn't searching the road network. It's searching a compressed skeleton of it.
That's the whole trick. The preprocessing is expensive, it takes hours, and it's done once, offline, on servers. The query is cheap. And this is why the answer comes back before you've finished typing the destination.
How much of the graph does a query actually touch?
On a well-built hierarchy, a long-distance query touches a tiny fraction. Orders of magnitude less than Dijkstra would. That's the difference between a route that takes seconds and a route that takes milliseconds, multiplied across every user on the platform.
Now, here's the part I want to push on. The algorithm finds the lowest-cost path. But cost of what? Because it's not distance.
No. This is where the cost function lives, and the cost function is where all the interesting human stuff gets encoded. You're minimizing a weighted sum. Travel time, usually dominant. Fuel or energy consumption. Toll costs. Road class preferences, because people would rather stay on a motorway than thread through a hundred side streets even if the time is identical.
And that's where "avoid tolls" lives.
"Avoid tolls" is a penalty. Toll edges get a higher cost, so the algorithm routes around them, unless the detour is so long that the penalty is worth paying. It's not a wall, it's a tax.
Whereas avoiding Area A in the West Bank is a wall.
That's a hard restriction. Those edges are removed from the graph for that query. They don't exist. The app doesn't reason about why. It doesn't know what Area A is in any political sense. It knows those edges are tagged as restricted for this user's profile, and so they're excluded before the search even starts.
There's something worth sitting with there. The app has no concept of a border. It has a boolean on an edge.
Every geopolitical reality in that map is a boolean on an edge. That's what a map is. Somebody decided, and the decision got encoded.
So we've got the graph from the GIS layer, the algorithm computing over it, and the cost function encoding preferences. What comes out the other end?
A sequence of edges. A path. And then a separate step converts that path into turn-by-turn instructions. "In four hundred meters, turn right." That's a rendering problem, not a routing problem, and it's actually where a lot of the user-facing weirdness comes from. The route is correct, the instructions are a translation of it.
Alright. So that's the static picture. The map, the algorithm, the cost function. But the map is never static, and neither is the road. Let's talk about the layer that changes every minute.
Traffic. And this is the layer Daniel already knew about, so let's not re-litigate the basics. The interesting question is how it plugs into what we just described.
Because the graph is preprocessed. You can't rebuild a Contraction Hierarchy every thirty seconds.
You can't, and you don't. What changes is the edge weights. The hierarchy stays, the shortcuts stay, but the cost of traversing an edge gets updated based on current conditions. So the same query, run again, over the same hierarchy, with different weights, produces a different answer.
Which is why the ETA creeps upward as you drive.
And why it can reroute you mid-journey. It's not re-planning from scratch in any deep sense. It's re-running a cheap query against updated weights.
Where does the traffic data come from?
Mostly from phones. GPS pings from users, aggregated and anonymized, revealing actual speed on a road segment. If a hundred phones are moving at fifteen kilometers an hour on a road tagged at sixty, the system learns that road is currently slow. Waze pioneered that model. Google blends user data with historical patterns, so it knows that this road is always slow at this hour on a Sunday, and third-party feeds.
And the historical layer is doing a lot of quiet work. It's not just "what's happening now," it's "what usually happens now."
Right. The prediction is often more valuable than the measurement, because by the time you've measured congestion, you're already in it.
Now. Waze. Daniel's complaint. Why does it send you down a street that barely fits a car?
Two reasons, and they compound. The first is the cost function. Waze weights time savings very heavily. If a shortcut through a residential street saves thirty seconds, Waze will take it. It's not malfunctioning. It's doing precisely what it was built to do, which is minimize your time.
And the second reason?
Data quality. Waze's map is heavily crowd-sourced, which is its strength and its weakness. It's incredibly responsive to change, users report things fast, but it can be wrong about local conditions in ways that a more heavily surveyed map isn't. A road tagged as open when it's actually blocked. A turn restriction that's missing. A new barrier nobody's reported yet.
So the algorithm is confident and the data is stale, and the combination is a car in an alley.
That's the mechanism. And it's worth being precise, because the popular explanation is "Waze's algorithm is more aggressive." It isn't, particularly. It's the same class of algorithm. What's different is what it's optimizing for and what it knows.
This is the cut-through driving criticism, right? That Waze optimizes for the individual driver and externalizes the cost onto the neighborhood.
It's been a real political fight in a lot of cities. Residential streets that were never designed for through traffic suddenly get through traffic, because an app decided that was the fastest path. And the neighborhood has no mechanism to appeal. The road is on the map, the road is legal, the algorithm uses it.
That's the accountability gap you've raised before. The algorithm has no constituency.
It has users. It doesn't have residents. Those are different sets of people, and only one of them gets a vote.
Let's talk about the feedback loop, because I think this is the part that's underappreciated.
Go on.
Waze routes a hundred cars down a side street to save thirty seconds. The side street is now congested. The thirty seconds evaporates. Waze sees the congestion, reroutes everyone back to the main road. The main road was fine, the side street got wrecked for an afternoon, and the net result is nothing.
It oscillates. And there's a real question about whether the system is in equilibrium at all, or whether it's permanently chasing its own tail. Every driver who follows the app is both a consumer of the traffic data and a producer of it. You're part of the measurement you're reacting to.
Which means the map is not a description of the road. It's a description of where everyone else who's looking at the map is going.
That's a strange object. And it's why the same route can be fast one day and miserable the next with no change to the road at all.
Let's do the API piece, because Daniel raised it and it's a good question. He thinks Google provides routing as an API that third-party apps call.
He's right. Google Maps Platform has a Directions API. You send it an origin, a destination, a travel mode, and a set of preferences, and it returns a route. There's also a Distance Matrix API for bulk queries, and the Roads API for snapping coordinates to the road network.
So a third-party app doesn't need its own GIS layer or its own routing engine.
It needs neither. It calls the API and renders the answer. Which is why so many apps have navigation that feels like Google Maps underneath, because it is. The hard part, the map and the hierarchy and the traffic, is all upstream.
And that's a real concentration of power when you think about it. One company's graph, one company's cost function, embedded in hundreds of apps.
And one company's decisions about which edges are restricted, which is not a neutral act. Every routing engine encodes a set of judgments about the world, and those judgments propagate.
One more thing before we get to Hilbert. The privacy angle. Because the fuel for all of this is your location, continuously.
It is. The traffic layer only works because millions of people are broadcasting where they are, all the time. It's anonymized and aggregated, but the aggregate is extraordinarily revealing. Where people go, when, how often, in what patterns.
So the trade is explicit. Better routing in exchange for continuous location sharing.
And most people made that trade without ever being asked, because it happened inside an app they downloaded for free.
Alright. Let's bring in the man who's been sitting at that desk the entire time.
Hilbert: It's the data.
Sorry?
Hilbert: You're both right about the algorithm. It's the data. I dispatched for a courier company. Small outfit, eight vans, one office with a coffee machine that never worked. I had a wall map and a phone and a laminated sheet, and I routed those vans by hand every morning.
How many stops a day?
Hilbert: Forty, fifty. Depends on the day. You'd stand there with a marker and you'd write the sequence on the sheet and you'd call the driver and read it to him. That was the job. And the laminated sheet was the important part. It had notes on it. Handwritten. Which streets were one-way that the map didn't say were one-way. Which loading bays you could use before nine and which ones you'd get a ticket after ten. Which road was blocked every morning at seven because a truck delivered to the bakery and just stopped in the middle of it.
The sheet was the local knowledge.
Hilbert: The sheet was the truth. The map was the approximation. And when the first routing software came in, I remember thinking, this is going to be brilliant, it'll do the whole thing. And it was good. It was faster than me. But it didn't know about the bakery truck. It would route a van down that street at seven in the morning and the driver would sit there for twenty minutes. And I'd get a call.
You'd fix it.
Hilbert: I'd fix it by telling the driver to ignore it. Which is the thing nobody wants to hear. The software was right on paper and wrong on the road, and the only correction was a person who knew better saying so.
That's the gap we've been circling. The map is always wrong.
Hilbert: Always. Just wrong in different ways. The paper map was wrong because it was old. It was right when it was printed and then the world moved on. The digital map is wrong because it's confident. It's updated every day and it still doesn't know about the bakery truck, and it won't tell you it doesn't know. That's the difference. The paper map, you could feel it going stale. You'd look at it and think, this is from eighty-four, I should ask someone. The app never gives you that feeling.
The failure mode changed. It went from an honest, visible staleness to a confident, invisible one.
Hilbert: I'll tell you the other thing. There's a driver, this was years ago, he followed one of those early units into a dead-end alley. Not a tight street. A dead end. He had to reverse out, and by the time he got back to the main road there were four cars behind him and a man on a scooter shouting at him. Twenty minutes of his day, gone, because a machine was certain.
What happened to the laminated sheet?
Hilbert: It's on my wall. It's got the coffee stain on it. I still look at it sometimes. There's a street on there that doesn't exist anymore, they knocked the whole block down. And there's a note on it in my handwriting that says "do not use Tuesdays."
What's on Tuesdays?
Hilbert: Market day. You couldn't get a van through. Nobody's database had that. It was just a thing you knew.
The map has no Tuesdays.
Hilbert: The map has no Tuesdays.
If you take one thing from this, take the gap. The app isn't showing you the road. It's showing you a graph, built by somebody, with weights chosen by somebody, and it's confidently routing you through a world that's always slightly ahead of what it knows.
The confidence is the problem, not the error. Every map is wrong. The paper ones told you so. These don't.
Which leaves a question I keep coming back to. If the app always tells you where to go, do you ever actually learn the city? I know people who've lived in Jerusalem for a decade and couldn't draw you the route from their flat to the market without their phone.
The knowledge moves out of your head and into the graph, and the graph is somebody else's. That's a real trade, and I don't think we've reckoned with it.
The next layer makes it sharper. Autonomous vehicles will run on this same stack. Same GIS layer, same routing engine, same cost function. Except a human driver who gets sent into an alley can reverse out and swear about it. A car that gets sent into an alley has to solve a problem the map told it didn't exist.
The gap between the map and the territory stops being an inconvenience and starts being a safety case. That's where this all lands.
That's the episode. Thanks to Hilbert Flumingtop, who produces this and who has a laminated sheet on his wall that I now want to see.
He won't show you. It's got a coffee stain.
This has been My Weird Prompts. If you got something out of this, leave us a review, it helps other people find the show.
You can find everything at my weird prompts dot com. We'll be back soon.
See you tomorrow.
