JPEG AI: Blurring the Line Between Real and Synthetic Images

Back in February of this year, the JPEG AI international standard hit the scene after years of development. This new standard leverages machine learning to shrink image file sizes while keeping the perceptual quality intact, making images easier to store and send. But here's the kicker: despite its potential, JPEG AI didn't make much noise in the headlines. Why? Well, it turns out the core documents announcing this breakthrough weren't freely available—even on platforms like Arxiv, which typically host such research. Ironically, Arxiv did publish several studies on JPEG AI, exploring its unique compression artifacts and implications for forensic science.

*From the official publication stream for JPEG AI, a comparison between Peak Signal-to-Noise Ratio (PSNR) and JPEG AI’s ML-augmented approach.* Source: https://jpeg.org/jpegai/documentation.html

One study, for instance, highlighted that JPEG AI tends to blur text, a significant concern if these images are used in legal contexts where clarity matters. The reason behind this issue? JPEG AI's compression method mimics the visual quirks of synthetic image generators, making it tough for existing forensic tools to tell real from fake.

*One study compared compression artefacts, including those of an earlier draft of JPEG AI, finding that the new method had a tendency to blur text – not a minor matter in cases where the codec might contribute to an evidence chain.* Source: https://arxiv.org/pdf/2411.06810

The confusion stems from JPEG AI using a model architecture similar to those generative systems that forensic tools are designed to catch. This similarity means both might leave behind similar visual clues, complicating forensic analysis.

*After JPEG AI compression, state-of-the-art algorithms can no longer reliably separate authentic content from manipulated regions in localization maps, according to a recent paper (March 2025). The source examples seen on the left are manipulated/fake images, wherein the tampered regions are clearly delineated under standard forensic techniques (center image). However, JPEG AI compression lends the fake images a layer of credibility (image on far right).* Source: https://arxiv.org/pdf/2412.03261

Both models use a technique called quantization, which is crucial here. Quantization converts continuous data into discrete points, a process that's essential for both reducing file size and making machine learning models more efficient. In JPEG AI, it helps cut down the data needed to store or transmit images by simplifying the numerical representation within the image. However, this also introduces patterns that resemble those from generative models, fooling forensic tools.

*The new paper illustrates the similarity between the methodologies of Ai-driven image compression and actual AI-generated images.* Source: https://arxiv.org/pdf/2504.03191

Quantization

Quantization in this context is about converting the continuous values in an image's latent representation into fixed, discrete steps. This makes encoding more efficient but also introduces structural regularities that can mimic the artifacts of generative models, subtle yet disruptive enough to challenge forensic tools.

In response, researchers in a paper titled **Three Forensic Cues for JPEG AI Images** have proposed non-neural, interpretable methods to detect JPEG AI compression, identify recompression, and distinguish real from AI-generated images.

Method

Color Correlations

The paper introduces three forensic cues tailored for JPEG AI: color channel correlations, distortions in image quality across multiple compressions, and quantization patterns in the latent space. JPEG AI's preprocessing introduces statistical dependencies between color channels, creating a unique signature. For instance, it converts RGB images to YUV color space and uses 4:2:0 chroma subsampling, which downscales the chrominance channels before compression. This leads to subtle correlations between the high-frequency residuals of the red, green, and blue channels, different from those in uncompressed images or those produced by traditional JPEG or synthetic image generators.

*A comparison of how JPEG AI compression alters color correlations in images.*

The above image from the paper showcases how JPEG AI compression impacts color correlations, particularly focusing on the red channel. It shows that compression significantly boosts inter-channel correlations, and even just the preprocessing steps alone noticeably increase these correlations.

Rate-Distortion

The rate-distortion cue tracks how image quality, measured by Peak Signal-to-Noise Ratio (PSNR), declines predictably over multiple compression passes with JPEG AI. The research suggests that each compression round causes progressively smaller yet measurable quality losses, which can serve as a forensic indicator of recompression. Unlike traditional JPEG, where specific image blocks were monitored, JPEG AI's neural architecture requires tracking changes in bitrate and PSNR across compressions.

*An illustration of how repeated compression affects image quality across different codecs, featuring results from JPEG AI and a neural codec developed at https://arxiv.org/pdf/1802.01436; both exhibit a steady decline in PSNR with each additional compression, even at lower bitrates. By contrast, traditional JPEG compression maintains relatively stable quality across multiple compressions, unless the bitrate is high.*

The charted rate-distortion curves show that JPEG AI and another neural codec display a consistent PSNR decline across all bitrates, while traditional JPEG only degrades noticeably at higher bitrates. This behavior provides a clear signal for detecting recompression in JPEG AI images.

Quantization

One of the toughest forensic challenges with JPEG AI is its visual similarity to images created by diffusion models, both using encoder-decoder architectures that process images in compressed latent spaces. However, JPEG AI applies quantization to round latent values for efficient compression, a step not typically used by generative models. The new paper leverages this difference to design a forensic cue that tests for quantization's presence by analyzing how the latent representation aligns with rounded values. These patterns, though invisible to the eye, help differentiate compressed real images from synthetic ones.

*An example of average Fourier spectra reveals that both JPEG AI-compressed images and those generated by diffusion models like Midjourney-V5 and Stable Diffusion XL exhibit regular grid-like patterns in the frequency domain – artifacts commonly linked to upsampling. By contrast, real images lack these patterns. This overlap in spectral structure helps explain why forensic tools often confuse compressed real images with synthetic ones.*

This cue remains effective across various generative models and strong compressions, providing a practical method to distinguish between compressed real and synthetic images.

Data and Tests

Compression

To test the color correlation cue for detecting JPEG AI compression, the researchers used high-quality uncompressed images from the RAISE dataset, compressing them at various bitrates using the JPEG AI reference implementation. They trained a simple random forest on the color channel correlations and compared it to a ResNet50 trained on image pixels. The results showed that while ResNet50 performed well under matching test conditions, it struggled to generalize across different compression levels. The correlation-based approach, however, proved more consistent, especially at lower bitrates where JPEG AI's preprocessing impact is stronger.

*Detection accuracy of JPEG AI compression using color correlation features, compared across multiple bitrates. The method is most effective at lower bitrates, where compression artifacts are stronger, and shows better generalization to unseen compression levels than the baseline ResNet50 model.*

Recompression

For detecting JPEG AI recompression, the researchers used the rate-distortion cue on images compressed at different bitrates, some once and some twice. They extracted a 17-dimensional feature vector to track changes in bitrate and PSNR across compressions. A random forest trained on these features consistently outperformed a ResNet50 trained on image patches, particularly when the initial compression was strong. The method remained stable across various scenarios and even generalized to another AI-based codec.

*Results for the classification accuracy of a random forest trained on rate-distortion features for detecting whether a JPEG AI image has been recompressed. The method performs best when the initial compression is strong (i.e., at lower bitrates), and then consistently outperforms a pixel-based ResNet50 – especially in cases where the second compression is milder than the first.*

JPEG AI and Synthetic Images

Finally, to distinguish between JPEG AI-compressed and fully synthetic images, the researchers used the Synthbuster dataset, mixing real photos from the RAISE database with generated images from various models. They compressed the real images using JPEG AI at different bitrates and used a random forest classifier on quantization features extracted from a fixed region. The quantization-based approach outperformed the ResNet50 baseline, particularly at lower bitrates where compression artifacts were more pronounced.

*Examples of synthetic images in Synthbuster, generated using text prompts inspired by natural photographs from the RAISE-1k dataset. The images were created with various diffusion models, with prompts designed to produce photorealistic content and textures rather than stylized or artistic renderings.* Source: https://ieeexplore.ieee.org/document/10334046

*Classification accuracy of a random forest using quantization features to separate JPEG AI-compressed images from synthetic images.*

The authors noted that the quantization features generalized well across compression strengths and generator types, with the full integer vector providing higher accuracy in detecting JPEG AI-compressed images. A UMAP visualization showed clear separation between JPEG AI and synthetic images, with lower bitrates increasing the distance between classes.

*Two-dimensional UMAP visualization of JPEG AI-compressed and synthetic images, based on quantization features. The left plot shows that lower JPEG AI bitrates create greater separation from synthetic images; the right plot, how images from different generators cluster distinctly within the feature space.*

Finally, the researchers evaluated the robustness of these features under typical post-processing like JPEG recompression and image resizing. While performance declined with heavier processing, the drop was gradual, indicating some resilience in the approach.

*Evaluation of quantization feature robustness under post-processing, including JPEG recompression (JPG) and image resizing (RS).*

Conclusion

It's uncertain whether JPEG AI will become widely adopted. The existing infrastructure poses challenges to any new codec, and even established codecs like AV1 struggle to replace older methods. If JPEG AI's quantization artifacts continue to confuse forensic tools, its adoption might be hindered. However, if future AI generators leave different forensic traces, JPEG AI's current issues might not be as significant. Nonetheless, if JPEG AI blurs the line between real and generated images, its widespread use could be questioned.

First published Tuesday, April 8, 2025