TECHNICAL GUIDE
How AI Image Upscaling Works (Real‑ESRGAN Explained)
When you upscale an image with AI, the computer doesn’t just stretch pixels — it reconstructs missing detail from what it learned about real photos. This deep‑dive explains the architecture behind Real‑ESRGAN and diffusion upscalers (the models that power this site), why they produce sharper results than Photoshop, and what that means for your everyday photos.
The Problem: Why “Resize” Blurs Images
Every image is a grid of pixels, and a small image has few of them. “Upscaling” means making that grid bigger — say 2× means going from 1000 ×1000 to 2000 ×2000. The new grid has four times as many cells, so three of every four pixels must be invented.
A normal resize (bilinear or bicubic) invents them by averaging neighbours. Blue sky next to green grass becomes a muddy mix; a sharp edge becomes a soft ramp of grey. The math is fast and deterministic, but it adds no real information — it just smooths. That’s why a 2× resize of a tiny logo looks like a blurry watercolour of the original.
Training: Learning From High‑res ↔ Low‑res Pairs
Here’s the key insight that makes AI upscaling different. Instead of giving a single image to the computer and asking it to guess, you give it millions examples of the same scene at two resolutions:
- a high‑resolution photo (the “ground truth”), and
- that same photo downscaled to low resolution (a synthetic “blurry/small” version).
The neural network’s job is to learn the mapping: “given the low‑res version, predict the high‑res one.” It does this by nudging its internal weights via gradient descent until its predictions match the ground truth on millions of examples. Once trained, it has internalised what real textures look like — the way skin pores, brick mortar, or an eye’s iris are arranged — so when it has to fill in missing pixels it draws on that learned knowledge instead of guessing blindly.
The Architecture: CNN + Adversarial Training (Real‑ESRGAN)
The “ESR” in Real‑ESRGAN stands for Enhanced Super‑Resolution. It’s built from two neural networks working together:
| Network | Role |
|---|---|
| Generator (a CNN) | Performs the upscale: reads the low‑res image and emits a high‑res prediction. Convolutional layers let it reason about local texture (edges, corners, patterns) and stitch them into a coherent whole. |
| Discriminator (a CNN) | Acts as a critic: it’s trained to tell real high‑res photos from the generator’s output. The generator is penalised whenever the discriminator catches a fake. This adversarial tug‑of‑war pushes the generator toward outputs that look photorealistic, not just numerically close. |
The discriminator is what makes the result look convincingly sharp rather than a smoothed‑out blur. Without it, the generator minimises pixel error and tends toward the same muddy averages a resize produces. With it, the generator learns to add the kinds of high‑frequency detail (pores, fabric weave, hair strands) that a discriminator recognises as “real”.
Real‑ESRGAN also uses a trick called Relativistic GANs and a perceptual loss (it compares deep features, not raw pixels) so the output looks right to a human eye even if a handful of pixels differ from the original.
Why It Beats Bilinear/Bicubic Resize
Resize and AI upscale solve different problems:
| Resize (bilinear/bicubic) | AI upscale (Real‑ESRGAN) |
|---|---|
| Computes a weighted average of neighbours | Generates texture from learned statistics |
| Only rearranges existing colour information | Adds plausible new detail |
| Same speed regardless of content | Tailors detail to what it recognises (skin, hair, sky…) |
| Soft edges, halos, blur on enlargement | Sharp edges, reconstructed texture |
| Best for: previews, thumbnails | Best for: photos, art, print, 4K |
The AI model is, in effect, a learned prior over natural images: it has seen what real photographs look like at high resolution and uses that to fill in the gaps instead of the featureless averages a resize produces.
Beyond GANs: Diffusion Upscalers (SUPIR)
GANs are fast and produce good detail, but they have a known weakness: they can generate plausible-but‑wrong details (a tooth where there was a gap, a second eye that’s slightly off). Diffusion upscalers — like SUPIR, the “Diffuser” model on this site — take a different approach:
- Start from the low‑res image plus noise.
- Denoise gradually over many small steps, each step guided by a separate “conditioner” network that re‑introduces the original structure.
- At each step the model can read an optional text prompt (“grainy 1970s photograph”, “sharp anime outline”) to steer what kind of detail is added.
Diffusion upscaling is slower and more expensive than GAN upscaling, but it produces the highest‑fidelity texture and gives you control over the kind of detail reconstructed. That’s why the site reserves it for the premium “6 MPx / 4 MPx” targets and lets you pay to unlock it.
Limitations: What AI Can’t Reconstruct
“Learned detail” is still a guess, and it’s only as good as the evidence in the source:
- Completely smooth areas (a blank sky, a plain wall) have no texture for the model to extrapolate — they stay smooth or get a gentle texture, which is usually fine.
- Tiny, unrecognisable inputs (a 40 ×40 thumbnail) have so little structure that the model is essentially inventing — results look “clean” but not “real”.
- No ground truth = hallucination risk. A GAN can happily add a plausible‑looking eye or window that wasn’t in the original. That’s why the face enhancement pass exists — to ground facial detail specifically.
- Artefacts compound. Upscaling an already‑upscaled image twice can amplify ringing or over‑sharpening. One gentle pass is usually best.
Which Model Does This Site Use?
Putting names to the “General/Plus/Diffuser” labels you see:
| Label on site | Underlying approach | Trained on |
|---|---|---|
| General | Real‑ESRGAN (CNN + relativistic GAN) | Photos (16 K‑capable) |
| Plus | Original Real‑ESRGAN (RRDB net) | Illustrations, anime, art |
| Diffuser (premium) | SUPIR (diffusion) + text prompt | Textures, restoration, text guided |
| Diffuser‑lite | SUPIR, capped output | Same as Diffuser, free quota |
Frequently Asked Questions
Is AI upscaling “fake”? Does it invent detail?
It reconstructs detail from learned texture statistics, so the result is plausible rather than pixel‑for‑pixel identical to the original. It’s not “fake” in the sense of being unusable — it’s sharper than a resize — but it’s also not magic: it can’t invent detail that isn’t hinted at.
Why is it faster on a server than on my PC?
The models are compute‑heavy (matrix multiplies the size of the image). This site runs them on GPU‑backed servers, so you get a GPU without owning one — and download the result.
Can I train my own upscaler?
In theory yes — Real‑ESRGAN is open source. In practice it needs a GPU, a dataset of HR↔LR pairs, and days of tuning. For most use cases, the hosted models here cover the common jobs (photos, art, tiny inputs) out of the box.
Does upscaling twice make it better?
Not usually. One well‑chosen pass is sharper and cheaper; chaining 2× twice doubles the pixels but can introduce over‑sharpening. Upscale once to your target, then crop if needed.
The Short Version
AI upscaling isn’t sorcery — it’s a neural network that learned, from millions of photo pairs, how to paint in the detail that a resize would just smear. Real‑ESRGAN (General/Plus here) does it fast with a generator‑discriminator pair; diffusion models (Diffuser/SUPIR) do it slower but with more control via a text prompt. Both beat a plain resize because they add plausible texture, not just averaged pixels.
No account required · Free daily quota · No watermark · Images auto‑deleted
Want the practical side? Start with the how‑to upscale guide, or read the history of image upscaling to see how we got here. Questions? Discord.