Test Environment

The VCX score is an objective and reliable indicator for the quality of a camera in a mobile device because it is measured accurately in the same test environment thereby excluding environmental variation as much as possible. For these measurements, all test labs are first synchronized and all test environments are very carefully inspected according to the guidelines.

A VCX test measurement currently consists of three parts: Image Quality, Response Time and Motion Control.

Test condition

Variables are controlled and kept constant throughout the test. Any changes in a variable during an experiment or in between tests would invalidate the correlation of dependent variables to the independent variable, thus skewing the results.

Ideally one image contains all the information that we need for the various image quality aspects, e.g. dynamic range and sharpness.

Factory default settings are selected and the default camera app with automatic exposure control is used. The analysis is performed based on the final JPG image data using the camera internal image processing pipeline.

Image Quality

Metrics:

  • Resolution
  • Texture Loss
  • Sharpness
  • Colors
  • Visual Noise
  • Dynamic Range
  • Shading
  • Distortion

All measurements are obtained under 5 controlled lighting situations.
We obtain the metrics with exactly the same exposure setting.

The image quality evaluation based on the TE42v2 test chart is performed under five different lighting conditions:

Bright - This condition is used as the reference. It is performed with an Illumination level of 1000 lux while the device is mounted on a tripod.
Mid - As we do not control the exposure manually, we reduce the light intensity by 2 EV (Exposure Value) which results in an illumination level of 250 lux. This lighting condition reflects a normal indoor light situation without direct sunlight.
Low – A low light situation is the most challenging situation for a camera. In this case, we reduce the illumination by 4 EV, compared to the reference (bright), which results in an illumination level of 63 lux.
Flash – If a situation is too dark, mobile phones mainly use LEDs to illuminate the scene. As a phone is rarely used in cases where there is absolutely no light, the flash is activated for this measurement while the scene is still illuminated with 63 lux (Low).
Zoom – Zooming onto a smaller object is a common action for users of mobile phone cameras. To evaluate the image quality of a zoomed image, we capture an image of the TE42v2 with 4x zoom at 1000 lux. If the device provides an optical zoom, we use the optical zoom first and add digital zoom, if needed, to achieve a 4x zoom. For devices that only offer digital zoom (the standard case of the majority of today’s mobile phones), we use a 4x digital zoom. Devices that achieve less than 4x zoom will receive 0 points for this measurement condition.

Timing Measurement

Metrics:

  • Shutter release time lag measurement images
  • Shooting time lag measurement images
  • Frame rate measurement images (Burst Mode compression)
  • Start-up time measurement images
  • AF failure rate measurement images

The metrics in the part “Response Time” are measured using the Image Engineering timing kit. The start of the exposure can still be determined, as it is the position of the first LED that is of interest.

 

Motion Control

STEVE is an abbreviation of “STabilization EValuation Equipment”. It is a mechanical apparatus that can shake the device under test in a defined way. That way we can simulate the motion introduced due to human tremor while capturing an image. The device simulates a two-axis motion (pitch and yaw). The camera under test has to reproduce a test target showing slanted edges. The chart is the TE261 of Image Engineering inside the AF-Box (also used for timing measurement) and the evaluation is done as described in ISO 20954.

Images for performance test are taken under two different light conditions: 800 Lux (Bright), 40 Lux (Low)

The analysis is made using software that has been shown to comply with the IQ-Analyzer and the VCX chart files. All the data which is extracted from the measurements will be computed without additional subjective analysis. The proportion of each metric in the score is determined by the VCX-Forum members when generating a VCX version.

 

The process is updated annually with new versions. the professional and detailed information can be found on the white paper. If you have any further questions, please contact the VCX coordinator.