Image Generation Metrics: FID, IS, KID, LPIPS, and CLIPScore

How do you score images with no ground truth? You compare distributions and use learned perceptual models: Inception Score, Fréchet Inception Distance, KID, LPIPS, and CLIPScore for prompt fidelity, on one running cat-generator example, with the equations, the catches, and torchmetrics.

Luis Condados · June 4, 2026 · Updated September 27, 2026 · 7 min read Advanced

metrics

Every metric so far compared a prediction to a known answer. Generative models break that: there’s no single “correct” image for “a cat in a spacesuit.” So we measure differently, comparing distributions of real vs generated images, and leaning on learned perceptual networks. We’ll score one toy cat generator every way that matters, with numbers you can move.

Every earlier chapter had a per-sample target: a label for classification, a box for detection, a mask for segmentation, an identity for tracking. Generation has none. A good metric has to capture three things: fidelity (do images look real?), diversity (is the model just memorizing a few?), and for text-to-image, alignment (does it match the prompt?).

The running example: CatGen, a cat-image generator

CatGen invents pictures of cats. We can’t check each one against a “right answer,” so instead we compare the distribution of CatGen’s images to the distribution of real cats. The real metrics describe each image by the 2048 numbers that come out of the last pooling layer of an ImageNet-trained Inception-v3 network [2] (that is what feature=2048 selects in the code below). To keep the arithmetic visible, imagine that network squashes each image to a single “cattiness” feature: real cats land around $\mu_r = 5$ with spread $\sigma_r = 1$ ; CatGen’s outputs land around $\mu_g$ , $\sigma_g$ . The closer those two bell curves, the better CatGen is.

Inception Score (IS)

IS rewards two things at once: each image is confidently one kind of thing (sharp), and across the batch you see many kinds (diverse). The classifier’s per-image class probabilities are the same softmax output as in the multiclass chapter.

The first widely-used one [1]: the KL divergence between each image’s label distribution $p(y \mid x)$ and the overall marginal $p(y)$ :

\text{IS} = \exp\!\Big( \mathbb{E}_{x}\, D_{\mathrm{KL}}\big(p(y \mid x)\, \|\, p(y)\big) \Big).

Worked example · CatGen makes 4 images, 2 classes

Say the classifier knows just two breeds. CatGen makes 4 images: 2 are confidently tabby ( $p(y\mid x) = [1, 0]$ ), 2 confidently siamese ( $[0, 1]$ ). The marginal is balanced, $p(y) = [0.5, 0.5]$ . Each image’s KL is

D_{\mathrm{KL}} = 1 \cdot \ln\tfrac{1}{0.5} = \ln 2, \qquad \text{IS} = \exp(\ln 2) = 2.0.

What it means: IS = 2.0 hits the max for 2 balanced, confident classes: sharp and diverse. If CatGen instead made 4 tabbies (no diversity), the marginal would be $[1,0]$ , every KL = 0, and IS = 1.0, the floor. Higher is better. Its fatal flaw: it never looks at real cats, so it can’t tell “good cats” from “confident nonsense”, which is why FID replaced it.

Fréchet Inception Distance (FID): the headline

Put the real and generated features side by side as two bell curves and measure the distance between them: same place and same spread give a distance of 0.

FID [2] models each set as a Gaussian and takes the Fréchet distance:

\text{FID} = \lVert \mu_r - \mu_g \rVert_2^2 + \operatorname{Tr}\!\Big( \Sigma_r + \Sigma_g - 2(\Sigma_r \Sigma_g)^{1/2} \Big).

Worked example · CatGen's FID (1-D)

With our single feature it collapses to a clean form, $\text{FID} = (\mu_r - \mu_g)^2 + (\sigma_r - \sigma_g)^2$ . CatGen sits at $\mu_g = 6$ , $\sigma_g = 1.5$ vs real $\mu_r = 5$ , $\sigma_r = 1$ :

\text{FID} = (5 - 6)^2 + (1 - 1.5)^2 = 1 + 0.25 = 1.25.

What it means: lower is better (0 = identical distributions). CatGen’s cats are a touch too “catty” on average ( $\mu$ off by 1) and a bit too varied ( $\sigma$ off by 0.5). Train it until both match and FID → 0. Unlike IS, this is anchored to real cats, which is why it’s the field’s headline number.

Now you try

Drag CatGen’s mean and spread and watch the blue curve slide onto the green real-cat curve as FID falls toward 0. Confirm the two ways to score badly: a shifted mean (wrong kind of cats) and a wrong spread (too samey, or too chaotic) each push FID up on their own.

generated mean μ_g generated spread σ_g

FID = (μ_r − μ_g)² + (σ_r − σ_g)² · lower is better

Kernel Inception Distance (KID)

KID measures the same real-vs-generated gap with an estimator that stays honest on small samples. KID [3] is the squared Maximum Mean Discrepancy between the two sets of Inception features, using the cubic polynomial kernel

k(x, y) = \Big( frac{1}{d}\, x^ op y + 1\Big)^3, \qquad d = 2048,

instead of FID’s Gaussian assumption. A cubic kernel compares skewness as well as mean and variance, and KID has an unbiased estimator. FID’s estimator is biased upward: on small sample sets it reads artificially high, and it keeps falling as you add samples (Bińkowski et al. show it still decreasing at 10,000 CIFAR-10 images) [3]. KID is also lower-is-better, so the FID playground above is the right mental model; it estimates that distance more honestly when you only have a few hundred images.

LPIPS: perceptual distance between two images

LPIPS asks whether two specific images look alike to a human, and answers in deep-feature space rather than in pixels.

When you have a pair to compare (a reconstruction, a paired translation, or two samples for diversity), LPIPS [4] gives a perceptual distance from network features that tracks human similarity far better than pixel MSE or SSIM. Lower = more similar (illustratively, two near-duplicate cat frames might score ~0.05, a cat vs a couch ~0.7). The common flip: report higher mean LPIPS among samples as a diversity signal.

CLIPScore: does the image match the prompt?

Alignment here is the angle between the image’s embedding and the prompt’s embedding: vectors that point the same way score high.

For text-to-image, none of the above checks prompt fidelity. CLIPScore [6] does, via the cosine similarity of CLIP’s image and text embeddings, clipped at zero and rescaled by a constant $w$ :

\text{CLIPScore} = w \cdot \max\big(\cos(E_{\text{image}}, E_{\text{text}}),\, 0\big).

Hessel et al. set $w = 2.5$ : real CLIP cosines between an image and a caption sit roughly between 0 and 0.4, and the factor stretches that range toward $[0, 1]$ [6]. TorchMetrics uses $w = 100$ instead, so its CLIPScore prints values on a 0 to 100 scale [9]. Always say which scale a quoted CLIPScore is on.

Worked example · 'a cat' vs CatGen's image

Say the prompt embeds to $E_{\text{text}} = (1, 0)$ and CatGen’s image embeds to $E_{\text{image}} = (0.8, 0.6)$ (already unit length, since $0.8^2 + 0.6^2 = 1$ ):

\cos = \frac{(1)(0.8) + (0)(0.6)}{1 \times 1} = 0.80 \;\Rightarrow\; \text{CLIPScore} = 2.5 \times 0.80 = 2.0 \text{ (Hessel)}, \;\; 100 \times 0.80 = 80 \text{ (TorchMetrics)}.

What it means: a cosine of 0.80 is a small angle: the image broadly matches “a cat.” The 0.80 is a toy value picked for round arithmetic; real CLIP cosines are much smaller, which is why the rescaling exists. And CLIPScore only judges alignment: a six-legged cat can still score high, and it says nothing about aesthetic quality.

Now you try

Drag the image vector toward the prompt vector and watch the cosine climb to 1; swing it 90° away and the clipped cosine falls to 0. CLIPScore is that clipped cosine times the constant $w$ .

Drag the blue dot.

Beyond these, improved precision & recall for generative models [5] disentangle fidelity (precision: do samples fall on the real manifold?) from coverage (recall: does the model span real diversity?), useful when FID alone is ambiguous.

In code (torchmetrics)

TorchMetrics 1.9.0 implements all three [9]:

# torchmetrics==1.9.0 (FID/KID also need torch-fidelity; CLIPScore needs transformers)
import torch
from torchmetrics.image.fid import FrechetInceptionDistance
from torchmetrics.image.kid import KernelInceptionDistance
from torchmetrics.multimodal.clip_score import CLIPScore

# FID: feed real and generated separately. normalize=True => float images in [0,1].
fid = FrechetInceptionDistance(feature=2048, normalize=True)
real = torch.rand(64, 3, 299, 299)
fake = torch.rand(64, 3, 299, 299)
fid.update(real, real=True)
fid.update(fake, real=False)
print("FID", fid.compute().item())          # lower is better; fix the sample count

kid = KernelInceptionDistance(subset_size=50, normalize=True)
kid.update(real, real=True); kid.update(fake, real=False)
print("KID mean/std", kid.compute())         # better for small sample sets

# CLIPScore: prompt fidelity (images uint8 in [0,255]).
clip = CLIPScore(model_name_or_path="openai/clip-vit-base-patch16")
imgs = (torch.rand(2, 3, 224, 224) * 255).to(torch.uint8)
print("CLIPScore", clip(imgs, ["a cat in a spacesuit", "a red bicycle"]).item())  # 0-100 scale (w = 100)

Caveats & common pitfalls

FID is a biased estimator. It shrinks as you add samples, so a number means something only at a stated sample count: compare runs at the same N (10k–50k). Even at equal N the bias can differ between models, enough to flip which one looks better [3]. On small sets, prefer KID, which is unbiased [3].
Implementation details move the number. FID/KID inherit Inception’s ImageNet bias, and Parmar et al. show that the resizing filter alone (an aliased resize in a common library versus a properly antialiased one) and compression applied to the images both shift FID noticeably [8]. Report the toolkit and settings, and don’t compare across papers with different pipelines.
A metric moved to a problem it wasn’t built for still prints a number. The Marigold V2 quantization post shows a depth metric built for strictly positive depth (AbsRel) reading “14% wrong” on relative depth centred near zero, where RMSE and correlation show the two maps agree. Check that a score’s assumptions hold for your comparison before you trust it.
CLIPScore can be gamed. It rewards embedding alignment, not correctness or aesthetics, and the six-legged cat still scores well [6].
Automated metrics are proxies. For generation, human evaluation still settles ties; distribution metrics measure quality+diversity in aggregate, never a single image [7].

Takeaways

No ground truth → compare distributions (FID, KID, IS) and use learned perceptual models (LPIPS, CLIPScore).
FID is the headline (lower better, captures quality + diversity; CatGen = 1.25), but biased by sample count, so always compare at equal N; KID is the small-sample-safe alternative.
IS is legacy. No real reference (CatGen’s max was just 2.0 = its class count), ImageNet-biased.
LPIPS for perceptual pair distance / diversity; CLIPScore (CatGen’s cosine 0.80, so 2.0 on Hessel’s scale) for text-to-image prompt fidelity.
Automated metrics are proxies. Human evaluation still settles ties.

That closes the series. Back to the start: Classification metrics: the foundations.

References

[1] Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved Techniques for Training GANs. NeurIPS. arXiv:1606.03498. (Inception Score.)

[2] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. NeurIPS. arXiv:1706.08500. (FID.)

[3] Bińkowski, M., Sutherland, D. J., Arbel, M., & Gretton, A. (2018). Demystifying MMD GANs. ICLR. arXiv:1801.01401. (KID.)

[4] Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. CVPR. arXiv:1801.03924. (LPIPS.)

[5] Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., & Aila, T. (2019). Improved Precision and Recall Metric for Assessing Generative Models. NeurIPS. arXiv:1904.06991.

[6] Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., & Choi, Y. (2021). CLIPScore: A Reference-free Evaluation Metric for Image Captioning. EMNLP. arXiv:2104.08718.

[7] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning, ch. 20 (deep generative models). MIT Press. Free online.

[8] Parmar, G., Zhang, R., & Zhu, J.-Y. (2022). On Aliased Resizing and Surprising Subtleties in GAN Evaluation. CVPR. arXiv:2104.11222, DOI:10.1109/CVPR52688.2022.01112. Implementation: clean-fid.

[9] TorchMetrics 1.9.0 documentation (stable, accessed 27 Sep 2026): FrechetInceptionDistance, KernelInceptionDistance, CLIPScore (defined there as $\max(100 \cos(E_I, E_C), 0)$ ).

Tagged

metrics

The running example: CatGen, a cat-image generator

Inception Score (IS)

Fréchet Inception Distance (FID): the headline

Now you try

Kernel Inception Distance (KID)

LPIPS: perceptual distance between two images

CLIPScore: does the image match the prompt?

Now you try

In code (torchmetrics)

Caveats & common pitfalls

Takeaways

Further reading

References