Popular Photography: How We Test

In the future, we will migrate to DxO’s 15-patch noise target. This is a transmissive rather than reflective target, illuminated from behind by daylight-balanced fluorescent bulbs within a lightbox. The benefit of the new target is that the patches are glass, and thus have very smooth surfaces. DxO Labs, the maker of the DxO Analyzer software, has determined that in cameras with very-high-megapixel sensors, the texture of the surface of the paper used for a test target may show up as noise in the test. Since pixel counts continue to increase, we will be shifting to this target this year and will make note of it here when we do.

When we process the images for the noise test, we apply the default amount of noise reduction in a given manufacturer’s RAW conversion software. Not only does this allow us to set a standard for the way we process our test images, it also encourages manufacturers to provide a reasonable starting point for end users when processing their images. Since manufacturers should know their cameras best, especially when first introduced, we expect that they will apply the amount of noise reduction that provides the best tradeoff between noise reduction and resolution preservation for a given camera model at a given ISO. Some camera makers apply very little noise reduction, or they don’t vary the amount of reduction based on the ISO, in which case we say so in the test. This way our readers know to pay attention to how much noise reduction is applied to their images, and why a given camera’s noise numbers might look odd compared to what they normally see in for its class of camera.

Resolution
People often associate resolution with the number of pixels found on a camera’s imaging sensor. While it is true that the potential for increased resolution does come with an increase in the pixel count, the resolving power of a camera system relies on more than just the sensor (or film, in the case of film cameras). That’s why we rely on actual images of resolution targets in our resolution test. As with some of our other tests, we shoot the resolution test with the camera brand’s 50mm f/1.4 lens at f/8, and at every ISO the camera offers, in whole-stop increments. If a camera’s sensitivity range starts or stops at a nonstandard ISO, we also test that ISO as well as every standard whole-stop ISO above or below it.

We list resolution only at the camera’s lowest ISO setting in the test results. This is mostly because we simply have limited space in the magazine. Instead of listing resolution at every ISO, we typically mention the point at which resolution falls below a certain threshold or we note how much the camera was able to preserve resolving power as ISO increased. We do this to underscore the variability of resolving power based on processing, and to provide more useful commentary to our readers.

We use Applied Image Inc.’s QA-77 target chart, an update of the ISO-12233 chart we had used prior to this one. The main difference between the two is that the QA-77 chart allows us to measure up to 4000 lines per picture height, while the older chart extended to only 2000 lines.

Autofocus Speed
A unique piece of equipment in the Popular Photography Test Lab is our custom-built AF-speed test apparatus. It determines how long it takes for an SLR to focus and capture an image. We set up the tripod-mounted camera, with 50mm f/1.4 lens attached, facing a liquid-crystal shutter that is opaque in its natural state but becomes clear when electricity is applied. On the other side of the liquid-crystal shutter is a very simple target: a bright white board with its left half covered in black velvet to create a vertical border of high contrast. A switch turns on the electrical current to the liquid crystal shutter while simultaneously triggering a timer embedded in the upper corner of the focusing target.

When the camera has locked focus and captures a picture, the time it took to do so (to the nearest hundreth of a second) is captured on the timer display, visible in the image.

We test the camera’s focusing speed at various light levels to show how it slows as the light level dims. Specifically, we test at EV 12, 10, 8 , 6, 4, 2, 1, 0, –1, and –2. At all those light levels in which a camera can reliably and consistently focus, we calculate the average focusing times to plot them on a graph in the test results. Sometimes a camera will become less consistent in dim light—if it does, we mention it in the text. We perform the AF speed test at least 20 times for each light level in the test, though often many more times than that. Once a camera fails to focus at particular light level, we end the test and tabulate the results.