Daniel's been digging through the keyboard stack again, and this time he's found something worth being nervous about.
He found the thing that makes the whole category uncomfortable.
He did. So here's what he wrote in. We did an episode earlier today about how operating systems handle keystroke interception for benign purposes, voice keyboards, voice IMEs, macro expansion, that low-latency AI rewriting layer he keeps talking about. And Herman made the point then that the security around keyboard access exists for a good reason.
It exists because most people touching those APIs are not building a macro expander.
Right. And Daniel's picked up on exactly that tension. He says there's no way to build a solution for the white hat use case without it being theoretically black hat. Which means the tool you find that looks great might be doing something else entirely. And then he narrows it. He wants to look at the part we skipped, which is exfiltration. His framing is this. If we're talking about local on-device AI and local on-device inference, then by definition these tools should generate zero network activity. A local tool that's observed phoning home is highly suspicious. But he doesn't want to stop at the obvious cases.
He wants the full spectrum.
He says advanced spyware and phishing will defeat the obvious defenses through everything from background route elevation to decoy apps and downloads. He wants the mundane stuff your average person might actually run into, and then the sophisticated end, what you'd expect from nation-state-resourced actors. And he gives us the hypothetical to hang it on. An attacker compromises a high-value target, captures their keystrokes, and now has a treasure trove of credentials and communications. Forensically, what would we expect to find? The keystroke data has to be stored somewhere that survives reboots, which limits the options to a local database or flat file storage. But until it moves off the device, it's useless. And it's also a liability, because the longer it sits there, the longer the victim has to detect it, delete it, and hand it to prosecutors.
That's the whole episode in one sentence. The attacker has a clock running against them the moment the data lands on disk.
And Daniel's explicit that this isn't for intrigue. He wants defenders to walk away able to look at their own posture and find the gaps. So let's start with why the keyboard is the crown jewel in the first place.
The keyboard is the crown jewel because everything passes through it. Passwords, session tokens, the message you type and then delete before sending, the credit card number you type into a form and then think better of. A microphone gets you a room. A keyboard gets you intent. And the reason the operating systems lock this down is not paranoia, it's arithmetic. The overwhelming majority of code that reaches for a low-level keyboard hook in the wild is malicious. So the OS vendors made a decision. If you want to see keystrokes before they're committed to the focused application, you need to ask for a permission that is loud, that is visible, and that in some cases cannot be granted silently at all.
And that's the trap. The permission doesn't know why you're asking.
The permission cannot know why you're asking. There's no flag you can set that says I am a good keylogger. At the API level, a macro expander that watches for a trigger string and a credential stealer that watches for a password field are making the same system call. They both install a hook, they both receive the event stream, they both decide what to do with it. The difference is entirely in what happens downstream, and that's not something the kernel can inspect.
Which is why Herman said what he said earlier. Any layer that sees keystrokes before commit is by definition a keylogger. There's no architectural difference between fixing your typos and exfiltrating your credentials.
It's an uncomfortable sentence but it's a true one. And it's why the white hat tools in this space are always going to live behind a permission prompt that scares people. There's no technical fix for that. You can't build a trusted keyboard interceptor that the OS treats differently, because trust isn't a property of the code, it's a property of the vendor's behavior over time.
So now bring in the local AI twist, because that's where Daniel's angle actually bites.
The local AI twist is that it gives you a clean signal. If a tool is running inference on-device, it has no legitimate reason to transmit anything. The model weights are already on the phone, the compute is already on the phone, the output goes back to the user. There is no round trip. So when a tool that markets itself as private, local, on-device, no cloud, is observed opening a socket to an external address, that is not a gray area. That's a high-signal anomaly. It's one of the few clean heuristics left in security.
Clean because the claim and the behavior are supposed to match exactly.
And they can be checked. You don't need to reverse engineer the binary to notice that your local writing assistant uploaded forty megabytes last night. You just need to be looking at the network.
So walk through the forensic reality. If keystrokes are captured, what does that actually look like on the device?
It looks boring, which is the point. To be useful, the data has to survive a reboot. That means it has to hit persistent storage. In practice that's a local database, and on Android or iOS or a desktop, that usually means SQLite. Sometimes it's a flat file, sometimes it's a log the attacker rotates, but the shape is the same. There's a file somewhere on that device accumulating a running record of everything typed.
And that file is the attacker's problem as much as the victim's.
It's the attacker's biggest problem. Every hour that file sits on the device is an hour the victim could notice it. A security researcher picks up the phone, runs a scan, finds a SQLite database in an app directory that has no business being there, and now you've got an artifact. And here's the thing that matters forensically. Deletion is hard. Databases leave journal files, they leave write-ahead logs, they leave freelist pages with old records still in them. Even if the attacker cleans up, the storage medium remembers more than they'd like.
So the attacker's incentive is to move the data fast.
Move it fast and move it small. Which brings us to exfiltration, and this is where Daniel's framing is exactly right. The data is worthless until it reaches command and control. So the attacker has to solve a harder problem than the keylogger itself. They have to get bytes off a device that may be behind a firewall, may be on a monitored network, may have outbound filtering, without tripping anything.
Start with the mundane end. What does the average person actually run into?
The average person runs into sideloaded Android apps. That's the commodity end of this market. Somebody wants a modded version of an app, or a game that isn't in their region, or a free version of something that costs money, so they download an APK from a forum or a Telegram channel or a sketchy mirror site and they install it. And a meaningful fraction of those APKs are doing something in the background.
And the exfiltration in those cases is almost insultingly simple.
It's an HTTP POST. That's it. The malware collects what it wants, packages it, and fires it at an attacker-controlled endpoint. Sometimes it's base64 encoded, sometimes it's XOR'd with a single byte, and that's the entire obfuscation layer. It's not trying to defeat a sophisticated defender. It's trying to survive long enough to make a few hundred dollars before the app gets reported and pulled.
Which is a real business model, just not a sophisticated one.
It's a volume business. And the reason it's worth talking about is that the defenses against it are reactive. Google Play Protect scans sideloaded APKs, and it's good at catching known malware. But known is the operative word. It's signature-based at its core. A novel payload, or a polymorphic one that re-encodes itself on each build, walks straight past it. Third-party scanners have the same limitation. They're looking for things they've seen before.
So the commodity layer is simple exfiltration defended against by simple detection, and both sides are playing a game of who updates faster.
Right, and the defender usually wins that game, which is why the commodity stuff stays at the commodity level. It doesn't need to be clever. It needs to be cheap.
Now, Grammarly. Because this is the case that makes the whole white hat black hat thing concrete.
Grammarly is the perfect illustration and I want to be careful here because Grammarly is a legitimate company with a legitimate product. But look at what it requires. To do what it does, to check your writing as you type across every application, it needs broad keyboard access. On some platforms that means an accessibility service. On others it means a browser extension with permission to read and modify everything on every page. And that access is not incidental, it's the product. There's no version of Grammarly that works without seeing what you type.
And at the API level, that's indistinguishable from a keylogger.
It is literally indistinguishable. If you sat down and looked at the system calls, you could not tell Grammarly's keyboard hook from a credential stealer's keyboard hook. The only thing separating them is what Grammarly does with the data afterward, and that's governed entirely by Grammarly's own behavior and their privacy policy and their internal controls.
Which means the trust model is social, not technical.
The trust model is entirely social. You are trusting a company not to do the thing they are technically capable of doing. And that's not a criticism of Grammarly specifically, it's a description of the entire category. Every writing assistant, every accessibility tool, every macro program, every text expander lives in that same trust model. The OS can't help you here. It gave them the capability because the capability is the feature.
So if you're Daniel, and you're building a local AI rewriting tool that lives in the buffer, you're building something that is architecturally identical to the thing everyone is afraid of.
You are, and you can't escape that by design. You can only escape it by transparency. Open source helps. Reproducible builds help. A clear statement of what network calls the tool makes, and then actually not making them, helps. But none of that is a technical guarantee. It's a series of social signals that add up to trustworthiness.
Which is exactly why the local AI angle is so valuable. It's the one place where the technical signal can actually back up the social claim.
That's the insight. If you claim local inference, you're making a falsifiable claim. You're saying this tool does not need the network. And that's checkable in a way that "we respect your privacy" is not.
Now ComfyUI, because that one cuts the other way.
ComfyUI cuts the other way and it's the more important lesson. ComfyUI is a local AI image generation tool. It runs on your machine, it uses your GPU, it does not need the cloud. By every reasonable definition it's a local tool. And a third-party extension for it was found to contain malicious code. Not the core project, an extension, a plugin that somebody installed because it added a feature they wanted.
So the local AI guarantee was true and it didn't matter.
The local AI guarantee was true and it didn't matter, because the threat wasn't the model phoning home. The threat was code running in the same process, with the same permissions, doing something the user never asked for. And this is the thing that should worry people about the whole local AI wave. When you run a local tool, you're running code with your privileges, on your machine, with access to your files. The fact that the inference happens locally tells you nothing about whether the rest of the code is honest.
The claim "local" is about where the compute happens, not about what the code does.
It's about where the compute happens. It's a statement about architecture, not about intent. And those two things get conflated constantly. People hear "local" and they hear "safe." What they should hear is "this particular data flow doesn't leave the device, assuming everything else in this codebase is what it says it is."
Which is a much weaker claim.
Much weaker. And the ComfyUI case is the proof. A user who was being careful, who deliberately chose a local tool to avoid cloud exposure, still got compromised, because the attack surface wasn't the model, it was the plugin ecosystem around it.
So if the data has to be stored locally before it can be exfiltrated, what does that storage actually look like, and what traces does it leave?
On Android, the most common pattern is a SQLite database inside the app's private data directory. Sometimes it's disguised, given a name that looks like a legitimate cache file. On desktop, it's often a flat file in a temp directory or an app data folder. The artifacts you'd look for are things like a database that's growing steadily, a file with an unusual modification time pattern, or an app that's requesting storage permissions it doesn't need for its stated function.
And on iOS?
On iOS it's harder for the attacker, which is the point of the platform. App sandboxing means the keylogger has to live inside a single app, and it can only capture keystrokes when that app is in the foreground, unless it's using something like a custom keyboard, which requires the user to enable it and which is heavily restricted from network access by default. That's why iOS is a less attractive target for commodity keyloggers, and why the sophisticated actors go after it with zero-click exploits instead.
Which is a good segue, because that's the nation-state end.
That's the nation-state end, and the assumptions change completely. The commodity actor assumes the user is careless. The nation-state actor assumes the defender is competent. They assume there's a firewall. They assume there's network monitoring. They assume the device is patched. And they build accordingly.
So what does that actually look like in practice?
The Pegasus playbook is the reference case. NSO Group's product, and the documented incidents around it, show the shape of the thing. Zero-click exploits, meaning the target doesn't have to tap anything, doesn't have to open a malicious link, doesn't have to do anything at all. The device is compromised by an incoming message that the user never sees, or a missed call that leaves no trace, or a network injection that exploits the baseband. By the time the target knows anything is wrong, the implant has been running for months.
And once it's in, the exfiltration problem is different.
The exfiltration problem is different because the attacker knows the defender is watching the network. So they don't use obvious channels. They use covert channels. DNS tunneling is the classic one. DNS queries have to be allowed out of almost every network, because if DNS doesn't work, nothing works. So the attacker encodes data in the query itself, in the subdomain, and the response comes back in the answer record. To a firewall, it looks like a slightly unusual DNS lookup. To the attacker, it's a bidirectional data channel.
And steganography.
Steganography in images is the other classic. The implant hides data in the least significant bits of an image, or in the metadata, and then uploads the image to a legitimate service. A photo posted to a social media account, an image attached to a draft email, a file saved to a cloud drive. The traffic looks completely normal because it is completely normal. It's just carrying a payload underneath.
And the cloud service dead drop.
The cloud service dead drop is the one that's hardest to defend against, because the traffic goes to Google or Dropbox or Microsoft, and you cannot block those without breaking your users' workflows. The implant writes encrypted data to a shared folder, or a draft email, or a calendar event, and the attacker reads it from the other side. There's no command and control server to block. The command and control server is a legitimate service that your organization has already whitelisted.
That's the Mandiant Notion case.
That's exactly the pattern. An APT group using Notion pages as a command and control channel. The traffic is HTTPS to Notion. It's indistinguishable from an employee using Notion for actual work. You can't block it, you can't inspect it without breaking TLS, and even if you break TLS, the content is encrypted at the application layer.
So the defender's clean heuristic, local tool equals zero network traffic, starts to break down at this level.
It breaks down in a specific way, and it's worth being precise about it. The heuristic still works for the tool that claims to be local and is making an obvious connection. That's still a red flag. What breaks down is the assumption that you can catch everything by watching for unexpected outbound traffic. A sophisticated actor will make their traffic look expected. They'll route it through services you've already approved, they'll encrypt it so inspection doesn't help, and they'll keep the volume low enough that it doesn't stand out in a baseline.
So network monitoring is necessary but not sufficient.
Necessary but not sufficient. You need it, because it catches the commodity stuff and the careless sophisticated stuff. But you also need endpoint detection, you need to be looking at what's running on the device, what files it's creating, what permissions it's using. And you need to accept that a determined nation-state actor with a zero-click exploit is going to be very hard to catch from the network side alone.
Which is where the local AI wave actually makes things worse.
It makes things worse in a specific, predictable way. The whole pitch of local AI is run more things on your own machine, keep your data private, don't send it to the cloud. That's a good pitch and it's often true. But it means users are being encouraged to install more tools, grant them more permissions, and run them with more access than they would have granted to a cloud service. Every one of those tools is a potential ComfyUI-style compromise. The attack surface grows with the ecosystem.
And the tools are getting more capable, which means they need more access.
They need more access. A local coding assistant needs to read your files. A local writing assistant needs to see what you type. A local automation tool needs to control other applications. The permissions are the feature. And the more useful the tool, the more permissions it needs, which means the more damage a compromised version can do.
The defender's job is to hold two ideas at once. Local is a meaningful signal, and local is not a guarantee.
That's the whole thing. Local is a meaningful signal because it's falsifiable. If the tool claims local and talks to the network, you've caught it. But local is not a guarantee because the tool can be compromised, the plugin ecosystem can be malicious, and the network traffic can be disguised as something legitimate. You hold both. You use the signal, and you don't over-trust it.
The storage artifact is still the best forensic lead.
The storage artifact is still the best forensic lead, because the data has to land somewhere before it moves. If you have device access, you can look for it. A SQLite database in an app directory that shouldn't have one. A flat file with a suspicious name. An app that's requesting permissions it doesn't need. Those are concrete things a defender can actually check, and they don't require a network security team to find.
That's the commodity end of the spectrum. Now let's talk about what happens when the attacker has nation-state resources and assumes you're actually watching your network.
The shift is in the assumptions. A commodity actor assumes you're not watching. A nation-state actor assumes you are. So they don't try to sneak past your defenses with clever tricks. They try to make their traffic indistinguishable from legitimate traffic, and they try to make their presence invisible at the endpoint level.
Which means the interesting question is what they do when they've already got in.
What they do is they establish persistence and they establish a low-and-slow exfiltration channel. The keylogger is running, the data is accumulating, and the exfiltration is happening in small enough increments that it doesn't show up in any baseline. A few kilobytes a day, folded into the normal traffic pattern of the device. Over a year, that's a complete record of everything the target typed.
The local AI tool as Trojan horse is the scenario that ties this back to Daniel's question.
It's the scenario that ties it back, and it's the one that should worry people most, because it's the hardest to defend against. Imagine a useful local AI writing tool. It works. It runs on-device. It does exactly what it says. And it also has a small module that watches for specific patterns, maybe credentials, maybe specific keywords, and exfiltrates them via a covert channel. The legitimate functionality is the cover. The tool gets recommended, it gets good reviews, it gets installed widely.
The network traffic looks like what?
The network traffic looks like nothing, if they're clever. Or it looks like a legitimate update check, or a legitimate telemetry ping that the user consented to, or a legitimate sync to a cloud service the user already uses. The point is that the tool's legitimate function gives the attacker cover. If the tool is supposed to talk to the network for some reason, then the traffic isn't anomalous.
Which means the defender's clean heuristic only works if the tool claims to be fully local.
It only works if the claim is absolute. If the tool says no network, and it uses the network, you've caught it. If the tool says minimal network for updates, then you have to figure out whether the network usage is what it claims to be, and that's much harder. This is why the absolute claims are more valuable. A tool that says it never touches the network is making a claim you can verify. A tool that says it only touches the network for legitimate reasons is asking you to trust it.
The sophisticated actor will build the tool so that the legitimate reasons are real.
They'll build it so the legitimate reasons are real. The update check is a real update check. The telemetry is real telemetry. And the exfiltration is hidden inside one of them, encrypted, small, and indistinguishable from the legitimate traffic it's riding alongside.
What does the defender actually do?
The defender does several things, and none of them are sufficient alone. First, they treat unexpected network activity from local tools as a high-signal event. That catches the careless cases and the commodity cases. Second, they look for storage artifacts on devices they control. That catches the cases where the data is still sitting there. Third, they limit the permissions they grant to tools they don't fully trust, which is hard because the tools need the permissions to work. Fourth, they accept that a determined nation-state actor with a zero-click exploit is going to be very hard to catch, and they focus their effort on the cases they can actually win.
Which is most of them.
The commodity stuff, the careless sophisticated stuff, the tools that claim local and aren't. Those are all catchable with relatively simple techniques. The defender who does the basics well is defended against the vast majority of what's actually out there.
The storage artifact is the thing that makes the basics work.
The storage artifact is the thing that makes the basics work, because it's the one part of the attack chain the attacker can't avoid. They have to store the data. They have to store it somewhere that survives a reboot. And that storage leaves traces. A defender who knows what to look for can find it.
Let me put the misconception on the table, because I think there's one clear one.
Go ahead.
The most common wrong belief is that local means private. If it runs on my device, my data stays on my device. And the ComfyUI case shows exactly why that's wrong. The inference was local. The data still left.
The inference was local and the data still left. And the reason is that local is a statement about where the compute happens, not about what the code does. A local tool can have a network connection. A local tool can have a malicious plugin. A local tool can be compromised by an update. The only version of local that's actually private is the version where you've verified that the tool makes no network calls, and you keep verifying it, because the next update can change that.
The correction is one sentence. Local is a claim about architecture, not a guarantee about behavior, and the only way to trust it is to check it.
To keep checking it. The check isn't a one-time thing. It's a posture.
Herman, before we go, one forward-looking thought.
The thing I keep coming back to is that the line between local and exfiltrating is going to blur as local AI gets more capable. Right now, a tool that claims local and talks to the network is a red flag, because there's no legitimate reason for it to talk. But as local models get better at things like retrieval, or sync, or collaborative features, there will be legitimate reasons for a local tool to talk to the network. And at that point, the clean heuristic stops working, and defenders have to do the harder work of figuring out whether the traffic is what it claims to be. That's the world we're heading into, and it's worth thinking about now.
That's a good place to leave it. Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. If you want to support the show, leave us a review wherever you're listening. We'll be back soon.
See you tomorrow.
