Daniel sent in a question about a photograph of the moon over Jerusalem. His phone camera butchered it. Grainy, blurry at zoom, and no amount of fiddling with brightness or sharpness sliders fixed it. And he makes a point I'd never quite put that way before. All the information is technically in the image. The geometry is there, the color is there. It just looks terrible because the geometry got smeared. So the job for a model is to clarify without changing anything, including the color. Not generate a better moon. Recover the moon that's already in the pixels.
And he's drawing a line. He doesn't want a general-purpose model with a free-text instruction like, make this sharper. That's unbounded. He wants the category of tools that do one precise, predefined thing, like deblurring or colorizing old black-and-white photography, and do it faithfully. He wants to know what that category is called, and what's in it. Open source, commercial APIs, closed source.
So what do you call a model that clarifies without changing anything?
The academic name is image restoration. Low-level computer vision. The researchers call it IR. The marketing people call it AI image enhancement or photo restoration. Same tools, two vocabularies. And you can hear the difference in what each name promises. Restoration says, we are putting back what was lost. Enhancement says, we are making it better. Those aren't the same promise, and the whole field is stretched across that gap.
And the task list?
Denoising, deblurring, super-resolution, deraining, dehazing, inpainting, JPEG artifact removal, face restoration, colorization. That's the standard taxonomy. Each one is a fixed mapping from a degraded input to a clean output. The model is trained to undo one specific kind of damage. A denoiser removes noise. It doesn't colorize. A colorizer doesn't sharpen. That's the predefined workflow Daniel's describing.
And why it's not image-to-image.
Right. Image-to-image in machine learning usually means diffusion-based generation conditioned on an input. You give it a sketch, it gives you a painting. You give it a photo, you ask for a different style. That's open-ended. The output can be anything plausible. Restoration is the opposite. The output is constrained to be the input, just cleaner. Bounded, not unbounded.
And yet the line isn't clean. That's the part I keep chewing on. Even the faithful models make things up.
They do. And the field admits it, sometimes in the documentation. There's a paper on controllable blind deblurring that says it outright. Where high-frequency information is lost, it is challenging to use generative models to produce details that are both photorealistic and faithful to the input. What they mean is, the model will invent texture. It will put detail in that wasn't in the original. It looks better. It isn't real.
So Daniel's moon photo. The deblurrer doesn't recover the craters. It guesses what craters should look like, based on every moon photo it's ever seen.
And if you're lucky, the guess is right. If you're not, you get a plausible moon that isn't yours.
There's a dial for this, isn't there?
CodeFormer has one. It's a face restoration model, and it exposes a fidelity weight. Zero to one. The documentation says a smaller weight produces a higher-quality result. A larger weight produces a higher-fidelity result. That's the entire tension of the field in one parameter. You can have pretty, or you can have faithful. Pick one.
And the fact that the dial exists is the admission. If restoration were purely faithful, there'd be no dial. You'd just run the model.
The dial exists because the model is generative under the hood. It's not recovering lost pixels. It's synthesizing plausible ones. The dial just controls how much you let it synthesize.
So let's get concrete. What's in the open-source toolbox?
The big one is Real-ESRGAN. From Tencent's ARC lab. Thirty-six and a half thousand GitHub stars. BSD license. It's been around since July of twenty twenty-one. Trained entirely on synthetic data. They take clean images, degrade them artificially, and teach the model to undo the degradation. The nice thing about synthetic training is you always know the ground truth. You know exactly what the clean image looked like, because you made the mess yourself.
And it does more than faces?
General image and video restoration. It has a denoising strength flag so you can balance noise against over-smoothing. If you turn it up too high, everything gets that waxy, plastic look. Turn it down too low, the grain stays.
What's the face one?
GFPGAN. Also Tencent. A hundred and fifteen million runs on Replicate. It runs in about two seconds per image. And then CodeFormer, from NTU's S-Lab. Eighteen thousand GitHub stars, fifty-four million runs on Replicate. It does blind face restoration with a codebook lookup transformer. Blind means it doesn't know in advance what kind of degradation it's dealing with. It just gets a bad face and tries to fix it.
And the codebook?
It maps degraded face patches to a learned dictionary of clean face features. So instead of hallucinating freely, it's constrained to pick from a vocabulary of plausible face parts. That's what makes it more faithful than a pure generative model. The codebook is the leash.
And yet still a dial.
Still a dial. The leash has slack.
What else should someone know about?
SwinIR for denoising and JPEG artifact removal. Restormer for motion deblurring, deraining, defocus deblurring. BasicSR is the training framework underneath a lot of these. And then there's the colorization side, which deserves its own conversation.
Before we get there. Daniel's premise was that these are working under the hood in GIMP and SaaS tools. Is that true?
It's uncomfortably true. CodeFormer's own readme lists more than twenty third-party deployments. fal dot ai, Segmind, Sieve, Anakin dot ai, and on. Plus integrations into AUTOMATIC1111 Stable Diffusion WebUI, ComfyUI, ChaiNNer. Many of them unlicensed. The model gets wrapped in an API, stuck behind a paywall, and the original authors get a mention in a readme if they're lucky.
So the invisible layer is invisible in two senses. Users don't know it's there, and the developers don't get paid.
And it's not just CodeFormer. GIMP has plugins. GIMP-ML brings de-blur, de-haze, de-noise into GIMP through Python. Intel has OpenVINO AI plugins for GIMP three, with super-resolution and semantic segmentation. And GIMP's own engine, GEGL, ships classical non-ML filters. There's a denoise operation based on per-patch DCT thresholding.
So GIMP has both the classical tool and the ML tool sitting next to each other. Same menu, same job.
And that's a nice natural experiment. The classical denoiser is deterministic. It does the same thing every time. The ML denoiser is learned. It makes judgments. Daniel said regular tools can't touch these problems. That's mostly true, but GEGL's denoiser is the reminder that some of these problems have been worked on for decades. The ML tools just got dramatically better.
What's the actual gap? Why can't a classical filter deblur a moon photo?
Deblurring is mathematically ill-posed. A blur is a many-to-one operation. Lots of different sharp images could have produced the same blurry one. Classical methods make assumptions. They assume the blur kernel is a Gaussian, or they try to estimate it and invert. But in real photos, the blur is messy. Hand shake, atmospheric distortion, lens imperfections, sensor noise. All combined. The classical inversion amplifies noise and produces artifacts.
And the ML model learns from millions of blurry-sharp pairs what a plausible sharp image looks like. So it's not solving the inverse problem. It's bypassing it.
It's pattern-matching its way to a solution. Which is why it works so well, and why it hallucinates. The model isn't constrained by what's mathematically recoverable. It's constrained by what's statistically plausible.
Which brings us to colorization. The ethically loaded corner.
DeOldify is the canonical open-source tool. Eighteen thousand stars. It uses a self-attention GAN with what the author calls a NoGAN training approach. And it's the model behind MyHeritage In Color. The most advanced version is exclusively licensed to them. So the open-source model and the commercial product are the same lineage.
And what's the ethical problem?
Colorization imputes information that was never recorded. A black-and-white photo doesn't just lack color. It lacks the information about what color was there. The film recorded luminance. That's it. So when a model colors a black-and-white photo, it isn't restoring anything. It's inventing. The color of someone's coat, the color of their eyes, the color of the sky. All guesses.
And people treat it as more scientific than it is.
There's an Hacker News comment from a few years back that put it well. Automatic colorization is viewed differently, as more scientific, when it's still absolutely beholden to the same biases and misconceptions that come from poor training data. The model learned from photos of white people in Western clothing. Give it a photo of a Nigerian wedding in nineteen fifty and it'll guess wrong, confidently.
And the wrong guess looks authoritative. That's the danger. A black-and-white photo has a kind of honesty to it. It says, I don't know what color this was. The colorized version says, this is what it looked like. Full stop.
And yet the applied value is real. There was a remote sensing paper last year that took historical black-and-white aerial photos, ran them through DeOldify for colorization and Real-ESRGAN for super-resolution, and then used them for rooftop detection. The detection accuracy went up by about forty percent over the original black-and-white images.
So the hallucinated color helped a downstream model find roofs. Not because the color was right, but because it added contrast that the detection model could use.
And that's the knock-on effect. The color doesn't have to be true to be useful. It just has to be informative. Which is a strange thing to say about a restoration tool.
Let's talk commercial.
The big closed-source player is Topaz Labs. Gigapixel AI for upscaling, Photo AI for the all-in-one denoise, sharpen, upscale workflow, Video AI for video. There's a comment from twenty twenty-three saying Gigapixel is no longer in active development, they're consolidating into Photo AI. And another user reported running Photo AI on dozens of images for their dad's memoir, and it significantly improved image quality while upscaling in more than ninety percent of cases.
Ninety percent is a strong number for a tool that's fundamentally guessing.
It's strong, and it's also a selection effect. The user probably didn't run it on images that were already perfect. They ran it on the damaged ones. So the baseline was low. But still, ninety percent of damaged images improved is a real result.
Adobe's in this too.
Photoshop has Super Resolution, built in since twenty twenty-one. One click. It doubles the linear resolution of a raw image using a trained model. No plugin, no API, no separate purchase. It's just a menu item. That's the most mainstream this category gets.
And the API side?
Replicate hosts the big three. GFPGAN at a hundred and fifteen million runs, Real-ESRGAN at ninety-four million, CodeFormer at fifty-four million. All pay-per-run, no idle cost. And there are non-official APIs all over the place. fal dot ai, Segmind, Sieve, Anakin dot ai, OpenXLab. DeepAI has a free DeOldify colorizer endpoint.
If Daniel wants to fix his moon photo, he has options. Upload it to Replicate, hit the Real-ESRGAN endpoint, maybe with the denoise flag tuned. Three seconds later, he gets a cleaner moon. Probably.
Probably. And the probably is the whole thing. He won't know if the craters are real.
What about the all-in-one trend? You mentioned earlier that the task-specific models might be collapsing.
Defusion, from CVPR twenty twenty-five, unifies denoising, dehazing, deraining, deblurring, and low-light enhancement in one model. The paper argues that traditional restoration needs distinct models for each degradation type, and that fails in real-world scenarios with mixed or unknown degradations. So they built one model that handles all of it.
Which is what a phone camera needs. You don't know in advance what's wrong with the photo. It might be noisy and blurry and hazy all at once.
There's DRNet-AiO, which handles denoising at multiple noise levels, dehazing, deraining, deblurring, and low-light enhancement. And NanoSD, which is a hundred and thirty to three hundred and fifteen million parameters, runs in twenty milliseconds on mobile NPUs. Twenty milliseconds. That's faster than the shutter lag on most phones.
The category is heading toward a single restoration backbone that runs on-device, in real time, and handles whatever's wrong without being told.
Which raises the question Daniel was really asking. Does the category survive as a distinct thing? Or does it become a mode of a general model?
The all-in-one trend suggests the latter. If one model does denoising, deblurring, dehazing, and colorization, then the predefined workflow isn't a separate tool. It's a parameter.
Yet there's a counter-argument from practitioners. There was a comment on Hacker News about using a general-purpose model to colorize a photo. The person said it would not render the image in vibrant color as if taken with a modern DSLR. It would colorize the photo, but only with washed-out, tinted colors. The task-specific model did better.
The general model is mediocre at everything, and the task-specific model is excellent at one thing. The all-in-one restoration model is somewhere in between. It's general across degradations, but still bounded to restoration. It won't generate a new image. It'll only clean up the one you gave it.
That's the distinction that matters. Bounded versus unbounded. A general restoration model is still bounded. It can't decide to make the moon into a different moon. It can only make the moon cleaner. The free-text editing model is unbounded. You say, make the moon more dramatic, and it'll add clouds, change the color, maybe put a face on it.
Daniel's moon photo is exactly the kind of thing where you want the bounded model. He doesn't want a better moon. He wants his moon, less blurry.
Which brings us back to the fidelity dial. Even the bounded model will invent detail. The question is how much invention you can tolerate.
I keep thinking about the colorization ethics point. The model doesn't know the color of that coat. It guesses. And the guess is shaped by the training data. If the training data is biased, the guess is biased. And the guess looks authoritative. So you've taken an honest black-and-white photo and turned it into a confident lie.
The lie is useful. The rooftop detection paper proved that. The color doesn't have to be true to be useful. It just has to be informative. Which is a strange thing to say about a restoration tool.
It's the same tension as the moon photo. The deblurrer invents craters. The colorizer invents colors. Both are useful. Neither is entirely true. The difference is that with the moon, the ground truth exists. There really are craters. With the colorized photo, the ground truth is gone. Nobody knows what color that coat was. So the invention is permanent.
That's why the purists object. There was another Hacker News comment, from last year, about AI upscales. The person said, AI upscales are exactly what I don't want. Restorations should take great care to restore the original content instead of hallucinating higher resolutions. That's the purist position. Faithful or nothing.
Which is a legitimate position, but it's also a luxury. If you're a historian with a damaged photograph, a hallucinated upscale might be the only way to see the face clearly enough to identify someone. The alternative is a blur.
The purist keeps the blur. The pragmatist takes the hallucination. And the fidelity dial is where they negotiate.
What's the licensing mess you mentioned?
CodeFormer's readme lists more than twenty third-party deployments. fal dot ai, Segmind, Sieve, Anakin dot ai, OpenXLab. And integrations into AUTOMATIC1111, ComfyUI, ChaiNNer. Many of them unlicensed. The model gets wrapped in an API, stuck behind a paywall, and the original authors get a mention in a readme if they're lucky.
The invisible layer is invisible in two senses. Users don't know it's there, and the developers don't get paid.
It's not just CodeFormer. The whole open-source restoration ecosystem is like this. Real-ESRGAN gets embedded in Upscayl, a GUI tool. GFPGAN gets embedded in photo editors. The models are small enough to run on a laptop, so people just download the weights and wrap them.
Which is the open-source bargain. The code is free, the weights are free, and the value is in the integration. The person who wraps the model in a nice UI captures the value. The person who trained the model gets stars.
Stars and citations. Which, in academia, is the currency. But for Tencent ARC lab, the stars are also marketing. Real-ESRGAN is a calling card.
What would you tell Daniel? If he wants to fix his moon photo tonight.
Upload it to Replicate. Use the Real-ESRGAN endpoint. Set the denoise flag low, because he wants to keep the grain structure of the night sky. Maybe run it through CodeFormer if there's a face in the frame, which there probably isn't. Three seconds later, he gets a cleaner moon. Probably.
The probably is the whole thing.
He won't know if the craters are real. That's the trade.
I think there's a deeper point here about the category itself. Daniel said these aren't generative AI workloads. But they are. They're generative in the technical sense. The models synthesize pixels. The difference is the constraint. The output is bounded to the input. But the mechanism is the same. Diffusion, GANs, transformers. The same engines that generate images from scratch are being used to restore damaged ones.
That's the interesting inversion. The most constrained, faithful, boring task in computer vision is being solved with the most unconstrained, generative, exciting technology. The field took the wildest tool and used it for the most careful job.
Then put a dial on it.
Hilbert: I worked a summer at a photo lab in ninety-four. We did manual color correction. Wedding photographers would bring in underexposed rolls and we'd sit with a magnifying glass and a set of dyes, trying to fix the exposure without making the bride's dress look blue. The rule was, you could dodge and burn, you could spot-tone, but you never altered the negative. The negative was the truth. What you printed was your interpretation of the truth.
That's the fidelity dial, before it was a dial.
Hilbert: The lab had a black book. Clients who wanted their photos improved. The owner would sometimes refuse to print a negative if the photographer had pushed the film too far. He'd say, I can't give you detail that isn't there. And the client would say, but you're the expert. And he'd say, that's why I'm telling you no.
The models don't have that option.
Hilbert: They don't. You ask a model to fix a face that's too far gone, it'll happily hallucinate a whole new face. It won't tell you the negative is too thin. It'll just invent a nose and move on.
The judgment call moved from the technician to the parameter.
Hilbert: The dial is just the brush. I had a brush. I'd decide how much to push. The model has a dial. Same decision, different tool.
The difference is, you knew when to stop.
Hilbert: I knew when the owner would fire me. That's a different thing.
The model has no owner. It has a loss function.
Hilbert: The loss function doesn't care if the bride's dress looks blue. It cares if the output matches the training data.
Which is why the colorization thing bothers me more than the deblurring. The deblurrer is guessing at geometry. The colorizer is guessing at history.
Hilbert: The lab had a colorist who'd do hand-tinting. Old black-and-white portraits. She'd ask the client what color the dress was. If the client didn't know, she'd guess. But she'd tell them it was a guess.
The model doesn't tell you.
Hilbert: The model doesn't know it's guessing.
That's the thing. The model isn't lying. It's just wrong, confidently. It has no concept of the difference between a guess and a fact.
Hilbert: The owner had a sign over the counter. We restore photographs. We don't improve them. I think about that sign whenever I see one of these AI tools.
The sign drew the line. The dial blurs it.
Hilbert: The dial lets the customer decide where the line is. That's the whole difference.
The question is whether the line belongs to the technician or the customer.
Hilbert: The technician used to be the one who said no. Now the customer just turns the dial up.
Gets a better moon, whether or not it's their moon.
Hilbert: I'd rather have the blur.
The one thing I'd want Daniel to take from this. These tools are not neutral. The deblurrer guesses at craters. The colorizer guesses at history. The fidelity dial is the only thing standing between you and a confident lie. And the dial is in your hands.
The category is disappearing into the general model. The bounded workflow is becoming a mode, not a tool. So the question isn't just which tool to use. It's whether you can tell, anymore, when the tool stops restoring and starts inventing.
The sign over the counter said it best. We restore photographs. We don't improve them.
If you've used these tools, or if you've got a moon photo that needs rescuing, let us know. We'd like to hear whether the craters looked right.
Thanks to our producer, Hilbert Flumingtop. This has been My Weird Prompts.
Email us at show at my weird prompts dot com. We'll be back soon.
