Tutorial: Error Level Analysis

Error Level Analysis (ELA) permits identifying areas within an image that are at different compression levels. With JPEG images, the entire picture should be at roughly the same level. If a section of the image is at a significantly different error level, then it likely indicates a digital modification.

What To Look For

ELA highlights differences in the JPEG compression rate. Regions with uniform coloring, like a solid blue sky or a white wall, will likely have a lower ELA result (darker color) than high-contrast edges. The things to look for:

Property	Expectations
Edges	Similar edges should have similar brightness in the ELA result. All high-contrast edges should look similar to each other, and all low-contrast edges should look similar. With an original photo, low-contrast edges should be almost as bright as high-contrast edges.
Textures	Similar textures should have similar coloring under ELA. Areas with more surface detail, such as a close-up of a basketball, will likely have a higher ELA result than a smooth surface.
Surfaces	Regardless of the actual color of the surface, all flat surfaces should have about the same coloring under ELA.

Look around the picture and identify the different high-contrast edges, low-contrast edges, surfaces, and textures. Compare those areas with the ELA results. If there are significant differences, then it identifies suspicious areas that may have been digitally altered.

Resaving a JPEG removes high-frequencies and results in less differences between high-contrast edges, textures, and surfaces. A very low quality JPEG will appear very dark.

Scaling a picture smaller can boost high-contrast edges, making them brighter under ELA. Similarly, saving a JPEG with an Adobe product will automatically sharpen high-contrast edges and textures, making them appear much brighter than low-texture surfaces.

Lossy & Lossless

There are many different picture file formats. Some formats are lossy, while others are lossless.

• Lossless: Lossless file formats retain exact pixel color information. If you load a picture, save it, and load it again, every pixel will have the exact same value. Even a conversion between two lossless formats will retain the exact same color values. For example, PNG and BMP are two different lossless formats. If you convert a picture from a BMP to a PNG, it will retain the exact same pixel values, even though the file format changed.
• Lossy: A lossy file format does not guarantee that the colors will stay the same. With JPEG, saving requires specifying a quality level. The quality level adjusts the compression amount (lower quality creates smaller files), but it compresses by removing some color information. With JPEG, saving a picture causes the colors to change a little. The resaved file may visually look the same as the source picture, but the exact pixel values will differ.

With lossy picture formats, the first time an image is saved causes a significant amount of color loss. However, loading the picture and then encoding it again in the same lossy format will result in less additional color degradation. The ELA results highlight the areas in the image that are most prone to color degradation during a resave. Edits typically stand out as being a region with a higher degradation potential compared to the rest of the image.

File Formats

There are a wide range of file formats. Some are lossy, while others are lossless. The following table identifies some of the common image formats:

	Lossy	Lossless	Either
Full Color	JPEG2000, JBIG	PNG, BMP, JPEG-LS, PPM, RAW*	TIFF, PDF, WebP, HEIC, HD Photo
Reduced Color	JPEG	GIF, PGM

^* RAW is not a standalone file format. Rather, different cameras use different file formats for storing lossless ("RAW") camera photos. RAW files are typically either lossless TIFF or JPEG-LS file formats.
In addition to lossy and lossless, there are some file formats that only support reduced colors. Full color file formats support at least 3 bytes per pixel (1 byte for red, 1 byte for green, and 1 byte for blue). This results in over 16 million possible colors per pixel, and each pixel is independent. In contrast, reduced color formats must alter the picture's color range to something less than full color. For example:

• GIF: GIF is a lossless format that only supports a maximum of 256 colors. If the picture has fewer than 257 colors (e.g., a grayscale image or clipart), then GIF is a highly compressed lossless format. However, if there are more than 256 colors, then some will need to be combined. This may result in dot dithering or merging similar colors. Although colors must be reduced to fit GIF's 256-color limitation, all subsequent saves are lossless.
• PGM: The Portable Gray Map file format only supports grayscale images.
• JPEG: All lossy file formats cause colors to shift. However, the JPEG algorithm is only capable of reliably storing about 2.4 million of the 16.7 million possible colors. (Technically, this is due to JPEG's integer conversion from RGB to YCrCb, and not due to the lossy compression.)

Some file formats include options to encode pictures in both lossy and lossless modes. For example:

• TIFF may include a lossless bitmap encoding, reduced FAX-based encoding, or a lossy encoding based on JPEG. Many cameras support RAW mode that saves images in a TIFF (or TIFF-like) file format using lossless compression.
• JPEG-LS is the lossless version of JPEG. Typically, JPEG uses the common lossy method, but there is also a lossless encoding method. The lossless encoding is almost exclusively used by a few cameras that record raw photos with JPEG.
• WebP is an image format from Google. It is commonly used for lossy encoding. By some accounts, it generates files that are 25%-35% smaller than similar quality JPEG files. However, it also includes a lossless format that typically compresses better than PNG.
• HEIC (also called HEIF or HEVC) is an alternative to JPEG that is being promoted by Apple. HEIC has virtually no support outside of iPhone and is not supported by any web browsers. HEIC is commonly used for lossy encoding and claims higher compression rates compared to the same quality from JPEG. However in practice, HEIC's lossly mode drops off high frequencies, resulting in blurry pictures.
• HD Photo (also called JPEG XR or JXR) is an alternative to JPEG that is being promoted by Microsoft. This format has virtually no support outside of Windows desktop systems and is only supported by Microsoft's IE and Edge browsers. When used, HD Photo is commonly used for lossy encoding and claims higher compression rates compared to the same quality from JPEG.

The conversion to a reduced color file format typically results in a significant alteration to the pixel values. For this reason, ELA is not applicable on severely reduced color images like GIF. However, ELA is explicitly designed for use with JPEG.

Applying ELA to Lossy Images

JPEG images use a lossy compression system. Each re-encoding (resave) of the image adds more quality loss to the image. Specifically, the JPEG algorithm operates on an 8x8 pixel grid. Each 8x8 square is compressed independently. If the image is completely unmodified, then all 8x8 squares should have similar error potentials. If the image is unmodified and resaved, then every square should degrade at approximately the same rate.

ELA saves the image at a specified JPEG quality level. This resave introduces a known amount of error across the entire image. The resaved image is then compared against the original image.

If an image is modified, then every 8x8 square that was touched by the modification should be at a higher error potential than the rest of the image. Modified areas will appear with a higher potential error level.


When a file is converted from a lossy file format to a lossless format, resave artifacts are retained. This permits ELA to highlight alterations made to a JPEG image that was converted to a PNG.

Not all lossy file formats are compatible. For example, the lossy WebP, HEIC, and HD Photo (JPEG XR) formats use different compression algorithms than JPEG. Even if a JPEG image has been repeatedly saved, it may still result in first-time-saved artifacts if it is converted to these other file formats. This happens because the WebP, HEIC, and HD Photo artifacts are applied for the first time.

FotoForensics can apply ELA to all supported file formats. However, ELA assumes that non-JPEG files were converted from JPEG images. (This is a reasonable assumption since there are no commercially available cameras that natively capture pictures in PNG, WebP, HD Photo, etc.) As a result, all ELA tests are compared against JPEG lossy compression.

Applying ELA to Lossless Images

Lossless file formats do not alter the colors in a picture. When a lossy picture is converted to a lossless format, all of the lossy artifacts are retained. This permits identifying specific types of alterations, such as:

• File format conversions. Converting from JPEG to PNG will retain the previous JPEG artifacts. Since a native PNG should not contain JPEG artifacts, the conversion is detectable.
• Native lossless. A picture that has never gone through JPEG encoding (e.g., converting a camera RAW picture directly to PNG -- without the use of an Adobe product) will have no JPEG artifacts. ELA should report a consistent image quality, and no 8x8 or 16x16 grid-based blocking, since it represents the areas that will change during the first JPEG encoding.

Evaluating ELA

With ELA, every grid that is not optimized for the quality level will show grid squares that change during a resave. For example, digital cameras do not optimize images for the specified camera quality level (high, medium, low, etc.). Original pictures from digital cameras should have a high degree of change during any resave (high ELA values). Each subsequent resave will lower the error level potential, yielding a darker ELA result. With enough resaves, the grid square will eventually reach its minimum error level, where it will not change anymore.

Image	ELA
An original digital photograph (Source: Hacker Factor) has high ELA values, represented by white colors in the ELA. The sections that are black correspond to the solid white book and the black 8x8 squares in the original image. Solid colors compress very well, so these are already at their minimum error levels.
The original image was resaved one time. To the human eye, there is no visible difference between the original and the resave image. However, ELA shows much more black and more dark colors. If this image were resaved again, it will have even lower (darker) ELA values.
The resaved image was digitally modified: books were copied and a toy dinosaur was added. ELA clearly shows the modified areas as having higher ELA values.

It is important to recognize that high frequency areas, such as edges along objects, will usually have higher ELA values than the rest of the picture. For example, the text on the books stands out because the light/dark contrast creates a high frequency edge. In general, you should compare edges with edges and surfaces with surfaces. If all surfaces except one have similar ELA values, then the outlier should be suspect.

Rainbowing

Rather than saving colors by their red, green, and blue components, JPEG separates colors into luminance and chrominance channels. The luminance is effectively the gray-scale intensitity of the image. The chrominance-red and chrominance-blue components identify the amount of coloring, independent of the full color's intensity.

With ELA and resaved images, there may be a visible separation between the luminance and chrominance channels as a blue/purple/red coloring called rainbowing. Drawing tools such as Photoshop can introduce a distinct rainbowing pattern surfaces that have near-uniform coloring.

Image	ELA
Computer-generated hands. ("NMRIH Hands", Matthew Fagan, 2009). The ELA shows red and blue rainbowing as background stripes. In other pictures, rainbowing may appear as large patches.

In general, Photoshop and other Adobe products generate a large amount of rainbowing. However, rainbowing is not an exclusive artifact to Adobe products. For example, the open source GIMP program generates little rainbowing and some high-quality camera photos may also include rainbowing along uniform-colored surfaces, such as white walls or blue skies. Some drawing tools, such as Microsoft's Paint, do not generate rainbowing.

The presence of rainbowing artifacts indicates a color adjustment, but does not identify intentional modifications.

• Prior to 2020, the strong presence of rainbowing suggested that an Adobe product, like Photoshop or Lightroom, was used to save the image. This is because Adobe automatically applies their own color adjustments to pictures.
• In 2020, Apple introduced iOS 14, which also can generate strong rainbowing artifacts due to the camera's automatic color correction system.
• Beginning in mid-2020, some Android devices incorporated color correction systems similar to Apple.

With some of these devices and applications, there is no option to turn off the automatic color adjustments.

Some older digital cameras can produce rainbowing. However, there is an easy way to distinguish a camera's rainbowing from Photoshop. With digital cameras, the rainbowing is usually not restricted to the JPEG grid. The edges of a camera's rainbowing area will appear to have smooth contours. This happens because the chrominance separation is introduced before the first JPEG encoding. With Photoshop and other graphics applications, rainbowing is stictly limited to the JPEG grid because it is applied during or after the JPEG encoding. If the edges of the rainbowing area appear blocky in 8x8 or 16x16 chunks, then the rainbowing is likely caused by a graphics program such as Photoshop or an in-camera adjustment on newer (2020 or later) iPhone or Android devices.

Improving Results

The results from ELA are directly dependent on the image quality. You may want to know if something was added, but if the picture is a copy of a copy of a copy, then ELA may only permit detecting the resaves. Try to find the best quality version of the picture.

For example, many pictures are hosted at Flickr. Flickr provides small, medium, large, and original images. The small, medium, and large are derivative images (resaves) created by Flickr. The "original" is whatever the user sent to Flickr, so the original will be the best quality. Similarly, pictures on news sites are usually resaved. If they have a tagline like "Source: AP Images", then go to the source and use that picture instead. News sites typically recolor, resize, and crop images before saving them at a very low quality. Go for the original source (or get as close as you can to the original source) to improve the image's quality and the ELA results.

Two easy ways to tell that the image is not an original are to look at the image size and attributions. In general, digital cameras do not generate small pictures. Pictures that are sized for the web are likely resized from other pictures, and even those may not be camera-original. Also, many web sites add their logo or URL to a corner of the picture. That means the base picture was resaved and the last modification was likely the addition of the attribution.

For best results, try to find the source picture. If you don't know where to start, then try TinEye. Many pictures on the web are resaved as they pass from user to user. TinEye doesn't know every picture on the web, but it knows many pictures. If the picture is being passed around, then TinEye can help find the source (or at least a better copy of the image). In general, the biggest image is usually the best quality. (But some sites do scale images larger...)

Advanced Uses

With training and practice, ELA users can also learn to identify image scaling, quality, cropping, and resave transformations. For example, if a non-JPEG image contains visible grid lines (1-pixel wide in 8x8 squares), then it means the picture started as a JPEG and was converted to the non-JPEG (e.g., PNG) format. If some areas of the picture lack grid lines or the grid lines shift, then it denotes a splice or drawn portion in the non-JPEG image.

As another example, PNG files are a lossless file format. If a picture is an original PNG, then ELA should produce very high values for edges and textures. However, if ELA generates weak results (dark or black coloring) along edges and textures, then the PNG was likely created from a JPEG. This is because the conversion process from JPEG to PNG is lossless and will retain JPEG artifacts.

When combined with other algorithms, ELA becomes a very powerful evaluation tool.

Caveats

While ELA is an excellent tool for helping detect modifications, there are a number of caveats:

• A single pixel change, or minor color adjustment, may not generate a noticeable change in the ELA.
• Since JPEG operates on a grid, a change to any part of the grid will likely impact the entire grid square. You may not be able to identify exactly which pixel in the grid was modified.
• JPEG uses the YUV color space. High contrast colors in the same grid, such as black and white, orange and blue, or green and purple (opposite ends of the YUV color space), will usually generate higher ELA values than similar colors in the same grid.
• ELA only identifies what regions have different compression levels. It does not identify sources. If a lower quality image is spliced into a higher quality picture, then the lower quality image may appear as a darker region.
• Scaling, recoloring, or adding noise to an image will modify the entire image, creating a higher error level potential.
• If an image is resaved multiple times, then it may be entirely at a minimum error level, where more resaves do not alter the image. In this case, the ELA will return a black image and no modifications can be identified using this algorithm.
• With Photoshop, the simple act of saving the picture can auto-sharpen textures and edges, creating a higher error level potential. This artifact does not identify intentional modification; it identifies that an Adobe product was used. (Remember: if someone needs to download a picture from their camera or resize a picture for the web, they are just as likely to reach for Photoshop as they are to use any other tool.) Technically, ELA appears as a modification because Adobe automatically performed a modification, but the modification was not necessarily intentional by the user.
• As mentioned above, rainbowing is not exclusive to Photoshop. The open source GIMP drawing program may introduce a little rainbowing and some high quality photos may contain rainbowing.

ELA is only one algorithm. The interpretation of results may be inconclusive. It is important to validate findings with other analysis techniques and algorithms.

Sample Image: Oriental Dancer Smiling

I was looking for a public domain picture that would make a good ELA example when I came across "Oriental Dancer Smiling" by Petr Kratochvil.


According to the web site, this picture was captured with a Canon EOS 50D.

Camera Information
Model	Canon EOS 50D
Manufacturer	Canon
Shutter speed	1/250
Aperture	f 8.0
Sensitivity	ISO 100
Focal length of the lens	46 mm

There is no mention that this picture was touched up. But what exactly was done to it? To answer that question, let's apply Error Level Analysis (ELA).

ELA

ELA shows the amount of difference that occurs during a JPEG resave. More white means more change, and black indicates no change.

A real, camera-original picture, should have a lot of white, almost like noise, over the entire picture. As the picture is repeatedly resaved (not copied, but actually loaded into a program and saved again as a JPEG), high frequencies and fine details are removed. With each resave, more frequencies/details are lost until the picture cannot get any worse (returning a black ELA picture).


In this picture, the background is completely black, but the person is not. That means that the background is at a different quality level compared to the rest of the picture; the background was digitally modified. In this case, it was enhanced to make the white look brighter.

Seeing Red

With ELA, all edges should be at about the same error level. Similar surfaces should all have similar error levels, similar coloring should have similar error levels, similar patterns, etc. But that isn't the case here. For example, her red headdress has a different intensity compared to the back of her dress, and she has one red strap that is much brighter than anything else. Even the red dangles from the hat are much brighter than the hat itself. The hat dangles appear brighter than the armband (photo lower left). Interestingly, the red bead in her hair (middle of her back) has a much lower ELA value.

Based on this, we can tell that the red in the hat was brightened, the dangles and one strap were reworked, and the original red color was probably darker -- like the red bead seen in the middle of her back.

Mouseover to toggle between the image and the ELA.

We can also tell that her eyelashes were touched up. For example, her faint lower eyelashes cast a very strong ELA value -- stronger than the edges of her nose and lips. And the lines of her cheeks have just as much contrast as her lower eyelashes, but they have no noticeable ELA variation. This means that her eyelashes were enhanced, while her cheeks were not. (Well, not significantly.)

For bonus points, look at the bright ELA spots in the middle of her eyes. Is that a brightened specular reflection, or were her eyes originally looking in a different direction? We know that her eyes were touched up because we can observe that the sclera (whites of her eyes) contains no red veins. (All eyes have veins, and at this resolution they should be visible.) The veins were removed when the eyes were enhanced.

There may have been other changes made to this picture, but these are the things that ELA can readily identify.

End of the Rainbow

The final item that we can tell from this picture comes from the faint blue and red patches. For example, her left arm (photo left-bottom corner) has a faint blue patch. There is a red patch on her back near her right armpit (lower-right corner). In fact, her face has many small blue patches and only a few small red patches.

The red/blue patches are rainbowing. They are a tell-tail sign that an Adobe product was used on this picture. This is confirmed from the metadata: this picture was last saved with Adobe Photoshop 'Save-for-Web' (and not 'Save As') and the user selected a quality level of 60%.