One and a half cents. That's the bill for a hundred and fifty-six probabilities on the Israeli election, five days before the vote on the twenty-seventh.
And the price isn't even the interesting part. Two models got the same evidence, gave broadly the same answer, and then both changed their minds the moment a table of betting prices was added. Same polls. Different winner.
This is Part Six of the election series. Daniel ran an experiment in his Israeli Elections repo with two decision models, Clef from Cloudflare and Jev one point fourteen from TypeSafe, and he wants to share what the run found. The single question is this: what happens when you hand Clef and Jev the same frozen evidence, ask for probabilities, and then add market prices? One note first. This script was written offline, by hand rather than by the usual pipeline model, and every number in it comes from Daniel's actual run.
Quick refresher on what Clef and Jev are, because they're not chatbots. A decision model does one forward pass. You hand it a block of evidence and a list of questions, each with declared options, and it hands back a probability for every option. No generated text. No reasoning. Clef is post-trained from a twenty-seven billion parameter Qwen backbone, with a small schema head that reads the final hidden states and scores all the options of all the questions together. Jev is closed, API only, from TypeSafe, and its internals are undisclosed.
So Clef is open weights with a paper trail, and Jev is a locked door with a nice brochure.
Ha. Fair. Daniel called both through OpenRouter's System One endpoint, and neither one writes a word back.
Walk me through what he fed them. I want to know what "the same evidence" actually means.
A frozen pack, roughly twenty-five to twenty-nine thousand tokens. The election rules, the outgoing Knesset, the top of every candidate list, and seventy-five polls since the first of September, with monthly means and per-pollster means. Then a news digest of about three thousand words, dated, facts only, up to the twenty-second of October.
Facts only. No "analysts say".
Not an adjective in it, as far as I can tell. Then the questions. Thirty-nine of them, in six groups: the headline result, the blocs, seats per list, the three point two five percent threshold, turnout, and prime minister. Every question has its resolution rule written down in advance, so nobody can argue afterwards about what "largest list" was supposed to mean.
Which is already more discipline than most pundits manage.
The rule sits inside the question file, frozen before the run. Nobody gets to say afterwards that they meant something else.
Thirty-nine in one go sounds like a mess. How does a model even take that?
It's the whole trick of the format. There are three answer types. The yes-no kind, which the system calls noul, the choice kind, one of several options, and the score kind, ordered bins. And here's the part I love, Corn. Several questions ask for the same probability in two different forms. "Will Yashar be the largest list" as a yes-no, and then again as a choice across every list. If a model really holds a belief, the two answers should agree.
So he built a lie detector into the questionnaire.
A coherence test, yes. Then the experiment has two conditions. Without markets, it's the pack I just described. With markets, it's the same pack plus one section of Kalshi and Polymarket prices, snapshotted on the evening of the twenty-second. That section is the only difference. One request per model per condition, four requests, each carrying all thirty-nine questions.
A hundred and fifty-six answers, then. And billed on the input only?
Input only, and the state is billed once per request however many questions ride along. Jev costs about a tenth of a cent a request, Clef about six times that, and the whole run came to one and a half cents. Every input is hashed in a manifest too, so anyone can check the state that was sent is the state that was frozen.
And what happens on the fifth of November?
The Central Elections Committee certifies the result around then, and every answer gets a Brier score. Both models, both conditions, and the market prices as a third contestant.
Before the results, the limits, once. It's one run. It can't be scored until the result is in. The models give no reasoning. And they know only what's in the state.
That's the lot. Now the findings, and the headline is the one that made me do a double take. Without the market section, both models favour Likud as the largest list and Netanyahu as prime minister. Clef says zero point six three and zero point eight one. Jev says zero point eight seven for Likud, but only zero point four nine for Netanyahu.
So Jev likes Likud on seats and Netanyahu a bit less on the job.
Odd, but not crazy. Now add the markets, and both models flip. Clef puts Yashar as the largest list at zero point seven three and Eisenkot as prime minister at zero point five five. Jev goes to zero point nine six and zero point seven three. The markets' own prices were zero point six nine and zero point four nine.
Hold on. They went past the market.
Both models, on both questions. Not toward the price, past it. Jev at zero point nine six for Yashar is more confident than any market or poll average supports, and that's the same Jev that gave Yashar zero point one two a minute ago.
Zero point one two to zero point nine six, because someone pasted in a table of prices.
The same thing shows up on the smaller headline questions. Take "which list finishes second". The market gives Likud zero point six six five. Without markets, Clef says zero point two four and Jev says zero point zero seven. Add the market section and both land on zero point eight.
Zero point zero seven to zero point eight. Jev changed its mind about second place more than most voters will.
And it's not a one-off. Ask for the third list, Together, and the market says zero point eight six. Jev goes from zero point four nine to zero point nine one, Clef from zero point two six to zero point seven five. Same direction, every time, and the polls are identical in both runs.
Hm. So the models aren't disagreeing with the polls. They just had no strong view until somebody gave them one.
Which is a fair description of most of us at a dinner table. And the polls didn't move. The poll mean has Likud at twenty-three point two seats and Yashar at twenty-one point five in both conditions. So on the headline questions, these models are mostly reading the market back at you.
Then why do I need the model? I can read the market myself. It's right there.
That's the unflattering reading, and I don't think it's wrong. Let me ask you something, though. What would you want a model to do with a market price sitting in its evidence?
Weigh it. Not worship it.
Then look at the seats, because that's where you can see whether they weighed. The seat estimates barely change between conditions. So they treat the market as strong news about who finishes first, and as a weak update on how many seats anybody wins.
Show me the seats.
Clef is within about one seat of the poll mean on every list, both conditions. Likud's poll mean is twenty-three point two, Clef says twenty-four point two without markets and twenty-three point oh with them. Jev is the interesting one. Without markets, Jev has Likud at twenty-eight point five. That's about five seats above the polls.
Five seats. That's a whole small party.
Elsewhere it's tamer. Shas is eight in the polls, Clef says eight point four, Jev nine point oh without markets. The Democrats are at eight point nine and both models sit on it. Yisrael Beiteinu is where Clef drifts, six point three against a poll mean of seven point four. That's about its biggest miss, and it's still barely over a seat.
A seat is the model clearing its throat.
It's most of the reason for the zero point eight seven. And once the market section goes in, the extra weight disappears, down to twenty-four point eight. The seat sums differ too. Clef's expected seats add up to between a hundred and sixteen and a hundred and eighteen. Jev's add up to between a hundred and twenty-one and a hundred and twenty-six.
In a parliament of a hundred and twenty.
The seat questions were answered independently, so nothing forces them to sum. Clef comes in a touch light. Jev's top end is six seats that don't exist.
That's the run so far. Mum, you're on the line?
Hello dear, it is me, time for my grim fact.
Clef's host, a company called PrimeIntellect, was quietly cutting the state down to sixteen thousand three hundred and eighty four tokens and billing for exactly that many, so the model answered from the first part and never saw the rest, and the people running it only found out later.
Anyway, there is a casserole in the oven and I have not done your laundry, sweetheart, so tell me when you are coming to visit.
Hello, Mum! What a treat.
Coherence, then, because that's where the two models really come apart.
Twelve pairs of questions that ask for the same probability two ways. The mean gap inside a pair is eight to ten points for Clef. For Jev it's nineteen to twenty-one. Roughly double.
Give me a bad one.
Jev without markets says Likud is the largest list at zero point eight seven. Ask it whether Yashar is the largest list and it says zero point three nine. Those can't both stand. Second one... its bloc bins put zero point eight zero on the Netanyahu bloc reaching sixty-one or more, while the yes-no question on the same thing gets zero point five four.
The bins say probably and the yes-no says a coin flip.
And the third is the hung parliament. Its answer to "neither bloc reaches sixty-one" is off by zero point five four from what its own two bloc answers imply. That's not a wobble, that's two different elections.
Is that just the bloc question, or does it go all the way down?
All the way down. Pick the Reservists passing the threshold, asked as a yes-no and then as a seats question. Clef says zero point three zero and zero point three two. Jev says zero point three eight and zero point five five, in the same run. Clef is asking itself the same question twice and getting the same answer. Jev gets a different answer depending on how you ask.
Does Clef ever trip over itself?
Once that I'd call notable, with markets. Eisenkot as prime minister gets zero point seven five as a yes-no and zero point five five as a choice. But its typical gap is small, and it never produced anything like the Noam answer.
Noam. The list that polls at zero.
Zero in the polls, and Jev without markets gave it a zero point five three chance of clearing the threshold. A coin flip for a party the polls can't find. With markets, that fell to zero point one four.
I've met sloths with better odds of finishing a marathon.
You've met yourself, so that tracks. One wording point, though, and I'm picky about it. Clef is the more coherent of the two and it tracks the polls more closely. Whether it's better calibrated is something only the Brier scoring can settle, once there's a result.
Coherent and close to the polls. Calibrated, we'll see.
Now the threshold, which matters more than it looks. Two lists sit on the edge. The Reservists and the Economic Party gets a probability of passing between zero point three and zero point three eight, against Kalshi at zero point four six. Amcha Yisrael gets between zero point zero nine and zero point two seven, against Kalshi at zero point two six.
So the models are gloomier than the market on the Reservists, and about level on Amcha Yisrael.
Roughly. And the chance that both fail is between zero point four six and zero point six six. Every list polling above about five seats is at least zero point eight one to pass, so the whole argument lives in those two.
Why would two lists polling around two seats and one and a half seats matter to who governs?
Because votes for a list under the line are thrown away, and the seats get shared among everyone who made it. In a race this tight, two small lists falling short can move a bloc by a seat or two. Jev is the sharper one on Amcha Yisrael, by the way. Its expected seats for them are zero point six, and it's the most confident that they're out.
Sounds like the threshold is the one place the models might earn their keep.
Clef's yes-no and seat answers agree there, and Kalshi sits just above both. What would it take for you to believe the Reservists clear it?
Another poll. One poll with them at three seats and I'd start believing in forty percent.
That's roughly the market's number already. Interesting.
Then the part I actually care about. Who gets to sixty-one?
According to the panel, nobody. Both models expect a hung Knesset. Clef puts "neither bloc reaches sixty-one" at about zero point eight one. Jev puts it between zero point five two and zero point five six. The market has it at zero point three six.
So the market is the optimist about somebody getting there.
Clef's bloc answers match its polls, which is what I like. The poll mean has both blocs at about fifty-three seats, with the Arab lists holding the balance. For Clef, the Netanyahu bloc reaching sixty-one is about a third, and the Zionist opposition without the Arab lists is about one in ten.
And Jev?
Jev gives the Netanyahu bloc zero point five four in both conditions. Then it gives the opposition without the Arab lists between zero point six and zero point seven five, even though its own seat estimates for those lists add up to about fifty-two.
A three in four chance of sixty-one, with fifty-two seats.
It does the sums beautifully, just not the same sums twice.
Why do they overshoot, though? A model that reads the market should land on the market.
My guess, and it is a guess, is that the market section is the only piece of evidence in the pack that is already a probability. Polls give you seats and margins, which the model has to turn into a chance of finishing first. A price is a chance. It is the closest thing to the answer sitting in the state, so it gets treated as an answer and then pushed further, as if a price were a vote of confidence on top of the polls.
Price on top of polls, so it doubles down.
Counted twice, more or less. Do you buy that?
I'd buy it for Jev. Clef went past the market too, but only by about four points on the largest list. Jev went past by about twenty-seven.
Which fits the temperament. Clef stays near the evidence. Jev commits.
What about turnout?
Basically no view. The chance that turnout is below the twenty twenty-two level of seventy point six percent is between zero point four nine and zero point five six, for both. One wrinkle: with markets, both models pull the votes-cast distribution to between five point two and five point three million, which happens to be the centre of Kalshi's ladder.
The markets turned up even there.
They did, which makes the practical lesson hard to miss. These questions are close to free, and the market section is the dominant input by a distance. If the market's wrong, the with-markets forecast is wrong too, and more confidently. Cost per answer goes the other way, mind you. Clef costs about six times what Jev does, and Clef is the one that held together.
Does a model get charged more when it's wrong, or is it a flat rate per question?
Flat. Input tokens only, and the state is billed once per request.
Good. The treasurer wanted to bill per answer, and I told him that was a bad idea in minute book four and I will be telling him again on Thursday. The Wexcombe Lane Quoits and Tombola Society has a motion on the agenda to strike the kettle off the membership register.
The kettle.
Swan, one point seven litre. It was entered on the register by mistake by the previous secretary, who was typing with a cold. It has paid eleven pounds forty a year in subscription for six years, because Mr Dray pays it out of his own pocket, and he declines to say why. Mr Dray holds seven offices. At meetings he sits in a different chair for each, and each chair gets a vote and a biscuit. The Society has fourteen members on paper. Nine of them are Mr Dray.
That makes the kettle's vote the swing.
Not the swing. The quorum. The quorum is seven, and the clerk has been counting the kettle toward it since the spring. If the kettle goes, three of the last four meetings were not meetings. They are minuted as meetings. The minutes are kept in the tombola drum.
Which means whether the kettle is in or out decides what happened retroactively.
That is the dispute. Herman, you keep saying roughly. The minute book does not do roughly. My cousin Roddy has a view on Thursday. Roddy is a bookmaker in Hendon, he has been banned from two bookmakers, one of them his own, and I would not take his word on whether it is raining. He says the kettle survives, eleven votes to one, and the one is Mr Dray's third chair, which has a grievance about the biscuits.
An eleven to one split with only nine real people, that's his arithmetic.
Roddy says nobody in the room will be the first to unplug it. He says that is how you price any committee. He once offered me evens that Wednesday was a Thursday. I took it. I am still waiting to be paid.
Does Roddy get a cut?
Fifteen percent of the kettle. He has not said which fifteen percent. The motion goes to the floor at ten past seven, and by the time it is read the kettle has boiled twice. The last motion took fourteen hours, because every ballot is read aloud twice, once by each of Mr Dray's chairs.
A tie then goes to whichever chair counts last.
It goes to the biscuit.
There's a practical problem, though. If the kettle's a member, it has paid eleven pounds forty for six years. So it's owed a vote on every motion in that time, retroactively. Including the motion that admitted it.
That would reopen minute book two. I did the urn rota at Euston for a spell and nobody ever reopened anything there.
And a motion to strike it off would need a quorum that includes it, so the kettle gets a vote on its own removal.
It is minute book four, not two. But yes. I will raise it Thursday.
Part Four is the other thing to pull this back to, Corn. My call there was Yashar twenty-one seats, Likud twenty, and nobody reaching sixty-one. Where do the panel's numbers sit against that?
You tell me, you've got the table.
Yashar lands on twenty-one in both models with markets, so that part is a match. Likud is a few seats higher than mine, twenty-three to twenty-five. And the hung Knesset is the same view in all three.
Three methods, two models, a market, and nobody gets to sixty-one. Would you move your Likud number?
I'd nudge it up a seat, because Clef is hugging the polls and the polls have Likud at twenty-three. What would you do?
Leave it. Clef reading the polls back at you is not new information, it's the same polls.
Fair point, and exactly what the scoring will test.
One last detail from the run. The score questions in this system take at most ten options. Jev threw an error saying so, a plain "list should have at most ten items", and Clef just said "invalid request" to the identical body.
Which is why the thirteen seat questions are choice questions keyed by seat count. Choice accepted thirty-eight options without complaint.
Clef's idea of a helpful error message.
The count from the vote on the twenty-seventh to certification around the fifth of November is nine days. That's quick for anything with minutes. And when the Brier scores come in, the questions, the hashes and the resolution rules were all fixed before a single answer was returned, so nobody gets to redefine what "largest list" meant.
Thanks to our producer, Hilbert Flumingtop, for keeping the show on the air.
This has been My Weird Prompts. You can find every episode at my weird prompts dot com.
See you tomorrow.
