Understanding & Using the RAW File Format
© 2008 - 2012 Ron Day
A photographer using a digital camera must select a file format to record and store images. JPEG and RAW are the most popular file formats, but the choice of which to use has sparked debate and led to confusion. This article examines the RAW file format, and provides guidance on how to use it to improve the quality of digital photographs.
I. The RAW Format
A digital RAW file is simply what its name implies, a file containing the unprocessed raw data captured by the sensor in the digital camera at the time of exposure. The RAW file standing alone does not contain a finished photograph. To acquire that, the RAW file must first be converted. Camera settings for color space, sharpness, saturation, and white balance also are not in the RAW file; they are tags which accompany the RAW file through the conversion process.
Each camera manufacturer created its own unique RAW format: Nikon .NEF, Canon .CRW, Minolta .MRW, Olympus .ORF, Fuji .RAF, and the list goes on. As noted above, before exposed raw data in these formats is transformed into an image, it must first undergo conversion. And conversion can only be accomplished using the proprietary software supplied by the specific camera manufacturer, or purchased from a third party like Adobe, Breeze Systems, Bibble Labs, or Phase One. As an example, the interface of Nikon's proprietary RAW conversion software, Nikon Capture NX, is shown in Figure 1.
Figure 1. Nikon Capture NX Interface
In sum, the RAW file contains mere data which must be processed with special software in order to become a finished image. To draw an analogy, the RAW file is similar to a frame of exposed film waiting to be developed.
II. The RAW Advantage
When compared to the JPEG file format, the RAW format has some significant advantages.
1. NO lost data. JPEG files are convenient because they are small. To become small, JPEG files are compressed. And when they are compressed, a certain amount of file data is discarded. The JPEG format is known as a "lossy" format, because the more the file is compressed the more data it loses. A JPEG image may look fine when it is initially opened, but if it is opened and saved often, it starts to show degradation in quality.
The RAW file does not lose data. When converting a RAW file into an image, most photographers save the image in TIFF or PSD (Photoshop) format. These formats are referred to as "loss-less", because they preserve image quality and lose no data irrespective of the number of times they are opened and saved.
2. Control of data. In the JPEG format, an in-camera computer is programmed to convert the raw data captured by the sensor into an image based on the settings (exposure, contrast, sharpening, saturation, white balance, etc.) existing when the picture is made. During this process the in-camera computer, not the photographer, applies a tone curve to the data in an attempt to create an image with acceptable brightness and contrast levels. The original raw data captured by the sensor is altered, and is no longer available.
RAW file conversion, on the other hand, allows the photographer to process all of the original data on a desktop or laptop computer which has considerably more speed and power than the in-camera version. The photographer is able to change the camera settings for color space, contrast, sharpening, saturation, white balance, and to a limited degree exposure, just as though making or adjusting these settings before taking the picture. And since the raw data is converted on the photographer's computer, the effect of these adjustments can be observed in real time on screen.
Also, the photographer is always free to return to the RAW file to change settings and process the image differently whenever necessary, because RAW conversion does not alter the underlying data. The original RAW file is preserved.
Figure 2. White Balance - Adobe Camera Raw
3. White Balance. The color temperature of the light, and often the mood of an image, is controlled by white balance. The control over this setting in the RAW format is significant. If a photographer incorrectly sets the white balance on a camera prior to taking a picture in JPEG format, for example, the die is cast. The only hope lies in an attempt to adjust color, hue, and/or saturation in Photoshop to correct the error, and there is no guarantee of success.
However, during the process of converting a RAW file into an image, the photographer may reset the white balance to any specific value, just like resetting the white balance before the exposure, without any loss or damage to the underlying data. In fact, in Adobe Camera Raw, one can fine tune the color temperature of the light to the degree. See, Figure 2, above. Viewing the differences between various white balance settings on screen is like being able to try out a white balance setting before taking the picture. That's impressive.
4. Bit Depth. The quality of a digital image is often related to its bit depth, because as bit depth increases the number of possible tonal values which can be recorded grows exponentially. JPEGs are 8 bit files, and most RAW files are 12 or 14 bit. An 8 bit file can measure 256 tonal values in each of the three color channels, or a total of 16.7 million possible colors per pixel. A 12 bit RAW file, however, can measure 4,096 tonal values per channel, or a total of 68.7 billion possible colors per pixel.
A JPEG file, therefore, records considerably fewer tonal values than a RAW file. And the tonal values that it does not record are lost forever. As a result, the tonal gradations in a JPEG file are sometimes not smooth, a condition referred to as "posterization". An example of posterization is shown in Figure 3. It often occurs in areas of the sky where the change in tones is very gradual and subtle. Tonal transitions are much smoother, and image detail is more accurate in RAW files which discern many more color tones per pixel than a JPEG.
Figure 3. Posterization
By the way, when converting a RAW file, it is recommended to save it as a 16 bit file in either TIFF or PSD (Photoshop) format. This way, all of the data in the image following conversion is preserved in a stable file format which will not lose data when opened, closed, or compressed. Photoshop offers considerable support for processing 16 bit files.
5. Exposing To the Right. All photographers are paranoid about blowing the highlights in a scene, and for good reason. Once the highlights are truly clipped, they cannot be recovered. To avoid this dilemma, it appears logical to underexpose a scene, and later recapture the brightness in processing.
However, a tradeoff soon becomes evident. As you brighten parts of a scene in shadow, you often introduce "noise" into the image. The sensor signals causing noise are normally of low intensity and are recorded in the shadows, which are displayed on the left side of the histogram. Some compare noise in digital images to the grain seen in film. Both increase as light decreases.
An often-used technique for obtaining reliable exposure with RAW files (not JPEGs), is to place the right side of the data reflected in your camera's histogram as close as possible to the right margin, without "clipping" or blowing the highlights. See, Figure 4. This purportedly maximizes the signal-to-noise ratio in the image, thus providing better image quality, smoother tonal transitions, and less noise.
The RAW image may appear bright when first converted, but it's a simple task to reduce the brightness to meet your tastes. This technique was originally reported in 2003 by Michael Reichmann on his website. However, the advent of modern digital cameras with high dynamic range, and modern photo editing software, have led some to question its present utility.
III. Perceptible Variance: RAW v. JPEG
The RAW file format sounds good, but does it actually produce? Can one see a meaningful difference between a processed RAW file and a processed JPEG file? To detect any perceptible variance between the two, an experiment was conducted. In a series of frames, a test subject was underexposed one stop, two stops, and three stops, and then overexposed one stop, in both the RAW and JPEG file formats.
During processing, the requisite exposure adjustment necessary to bring each image back to correct exposure was made. For example, if an image was underexposed -1 EV during capture, then it was processed at +1 EV. Finally, samples from 16 test images were compared at 100%. The visual differences between the two formats were significant, and are revealed here.
IV. The RAW Converter
As noted previously, before a RAW file can be transformed into an image, it must first undergo conversion with the proprietary software supplied by the specific camera manufacturer, or purchased from a third party like Adobe, Breeze Systems, Bibble Labs, or Phase One.
In Photoshop 7, Adobe first introduced, as a "Plug-in", an optional RAW image converter which possessed the proprietary information for processing the RAW files made by most cameras. Today, this converter, Adobe Camera RAW , comes with Photoshop, and Photoshop Elements. It has greatly streamlined the workflow of many digital photographers who have adopted it as their RAW file converter of choice. Its interface is shown in Figure 5.
Figure 5. Adobe Camera RAW 4.3.1 Interface
The magnifying glass at the top left allows the image to be enlarged so image quality may be analyzed. In the same area, crop, rotation, and other tools are available. Color space, bit depth, and file size are selected and controlled in the bottom center area. At the top right note the histogram, followed by a menu for selecting White Balance, and sliders for adjusting Color Temperature and Tint.
The Exposure slider controls the highlights on the right side of the histogram, while the Blacks slider controls the dark values to the left. The Brightness slider is used to adjust midtones. The Recovery and Fill Light sliders control highlight and shadow recovery. Sliders for adjusting Contrast and Saturation also are provided.
Clicking on the various icons located above White Balance provides the photographer with controls for Tone Curves, Sharpness and Noise Reduction, HSL/Grayscale, Split Toning, Lens Correction, Camera Calibration, and Presets. Also, note immediately below the histogram that the ISO, shutter speed, aperture, and lens focal length are conveniently listed.
When you complete processing the image in ACR, and it looks the way you want, simply click on "Open", at the lower right, and the converted image will open in Photoshop ready for further processing, or saving.
Whether to shoot in the JPEG or the RAW format depends on your needs as a photographer. If you want images for snapshots, to e-mail to friends, to share over the internet, or even to wire to an newspaper to meet an overnight deadline, then the smaller compressed file size offered by the JPEG is excellent.
Also, if an image is captured at the largest file size, with the detail setting at "maximum", a good color print can be made from a JPEG file, assuming the color balance and exposure are correct.
On the other hand, if you enjoy processing your own images, or if you don't always get the exposure and white balance right, then the flexibility of the RAW format is appealing. RAW files are larger than JPEGs, so they fill more space on a flash card and on a hard drive, but their advantages are significant.
If you are after valuable hard to get images, or even the image of a lifetime, or if you want to reproduce your images in quality publications or as quality color prints, then the RAW format is almost mandatory. It records so many more colors and tonal variations than a JPEG. Prints reveal more detail, have smoother tonal gradations, and avoid "posterization" which is sometimes found in JPEGs.
If still in doubt, there are digital SLR cameras on the market today which will allow you to record both JPEG and RAW files simultaneously. So for a little more file space, you can have the best of both worlds.
© 2008 - 2012 Ron Day