Score System

What is VCX score —

The VCX score is a numeric indicator of the camera value of mobile phones and products and is based on an objective assessment of equipment.

The range is designed in such a way that a value of 100 means that the device yields the best possible result in every metric that is achievable with today’s camera technology. The score is the sum of the image quality (70%) and performance (30%).

Bright light conditions
[ISO 50 - f/1.8 - 1/50s]
80/100
Medium light conditions
[ISO 200 - f/1.8 - 1/50s]
73/100
Low light conditions
[ISO 320 - f/1.8 - 1/25s]
65/100
Flash light
[ISO 160 - f/1.8 - 1/50s]
75/100
Zoom
[ISO 80 - f/2.4 - 1/50s]
39/100
Response & performance
-
82/100

How is it designed?

The VCX Score is designed to keep subjective influences as well as human errors as low as possible. It does not contain any visual assessment or other subjective factors.

The only subjective component is the weighting, that is, the decision of which metric is more important for the overall performance compared to others. This weighting has been accurately determined by a group of experts and it is identical for every device, so the comparison between devices is fixed and not influenced by individual opinions.

The testing is accomplished by an independent test lab, the evaluation is automated and 100% objective. It does not contain any visual assessment or other subjective components. Selection of parameters, LGC and HGC, formulae, and weighting are based on physical facts, surveys, and long-term experiences. The score is based on 155 parameters in order to reflect the user experience.

The entire analysis is only based on the captured images of test charts and scenes under defined lighting conditions and the analysis algorithms are applied to these images. For each of the metrics used, a single score is calculated via bespoke algorithms/formulae developed specifically for VCX Furom by the members, derived from use-case studies. Based on a fixed algorithm, the score is calculated using the numerical results. The total score is a weighted sum of the individual scores.

12/100

Why is it objective?

All devices are tested under the same conditions under multiple illumination levels and use cases. All images are analyzed without any human interaction. The same parameters, LGC and HGC, formulae, and weighting are applied to all devices in exactly the same way. The measured results are converted into the VCX score in exactly the same way. To prevent manufacturers from cheating, the detailed scoring process is limited to members who have to sign the association code with ethical standards.

To be improved

The weighting of the different aspects of the image quality is the result of a case study on how mobile phones are used as well as internal research. It correlates well with the outcome of other independent studies. The transformation of metrics into scores is performed under the definition of a theoretical worst and theoretical best value. The scaling is performed in different ways between the extreme points, depending on the metric itself.

For some metrics, the correlation between “metric” and “influence in image quality” is linear, so the score is a linear function of the metric. This would be in the case of a simple “the higher the better” or “the lower the better” assumption. For others, this assumption is not true. Some metrics require a different approach to the one previously mentioned because it would not reflect the perceived quality. Sharpening is a good example of this behavior. No sharpening is not beneficial for the image quality, as an image would appear flat. At the same time, too much sharpening very quickly results in an artificial and unpleasant. So, there is a “sweet spot” below or above which leads to a reduction in the score.

We regularly check the latest development in the camera industry and update every year the test procedure, the score generation process as well as the weighting of the scoring system with the continuous evolution of technology. e.g. it does not reflect the improvements in camera performance or when new technologies need to be included in the procedure.

Mission

Test procedures need to evolve and keep track of technological development. To make sure that VCX produces meaningful results in the future the standard department meets at least three times a year and discusses and implements new test procedures. On a year basis, VCX tries to update the whole procedure and take it to the next generation.

With its effort, VCX offers an objective consumer-oriented test procedure for mobile phone cameras that are widely accepted and used in the industry and known for its high-quality results.