Soomal Reality Camera Mark Mobile Beta: Protocol and Scores
Author:Soomal
Source:eng.soomal.com

Rate:01.00/1

In the past 5 years, Soomal had taken over 8000 photos by smartphone, and wrote 99 camera reviews. We witnessed the huge progress in smartphone cameras, and we had taken them as real cameras rather than just toys in our daily use. As we accumulated large enough experience in shooting based by daily shooting, we consider, we are able to give you quantitative scores of smartphone photo quality and experience.

Protocol

Soomal will be quite different from the main stream. Most reviews or tests are taken in labs, for example, to take ColorChecker photos, while we will take lots photos outside the lab to judge.

Most photos to be valued will be taken by hand hold in auto mode, a few of them will be in manual mode, and for some certain extreme tests, we probably use tripod, which will be noted. We will cover scenes in daily life, including cityscape, landscape, portraits, macro shoot, and still lives.

Why no lab?

1.There existing lot lab tests for references.
2.Simulating users in daily life. Smartphone manufacturers probably operate their product for lab tests, for example, smartphones could lower sensitivity when detecting they are fixed. Such auto settings are good for traditional lab tests, however, in users daily using, it wont be that good.

Valuing by lab tests and valuing by outdoor shooting is not contradictory, they can be seen as supplementary. We hope Soomal smartphone camera score will help you with a new reference.

Benchmark

Samsung Galaxy S8+

S8+ reaches a new high level in all aspects in photographing, becomes one of the best smartphone in camera overall experience. So, we will set S8+ as the benchmark for 100, if a smartphone scores higher than 100, it can be considered better than S8 in overall performance, otherwise, worse.

Weight

Autofocus [speed, pre-focus, focus logic, auto focus point setting] 20%
Metering [Accuracy, detecting] 10%
Image quality [details, noise reduction, HDR] 30%
Colors [style, AWB, texture] 20%
User Experience [usability, functionality, stability, display screen] 20%

Soomal camera scoring system now will focus on still images and related functions and experiences, and we will temperately not value video recording function. Scoring items including autofocus, metering, image quality, colors, and user experience, weights 20%, 10%, 30%, 20%, and 20%.

Autofocus: Autofocus including autofocus speed, pre-focus, focus logic, and focus point setting based on image identification, such as face recognition. Soomal holds this opinion, autofocus is an overall experience, not just speed, even a smartphone has the highest autofocus speed, its experience could still be improved by advancing pre-focus. Focus logic is also veeeeeery important, for example, during macro shooting focus on the background is a defect in focus logic. Face recognition are adopted by most smartphones, and really enhanced shooting experience. We are expecting AI will help the camera to tell who is the leading role.

Metering: A precise metering result will be the best, over or under exposure will deduct points. Optimizing according to scenes, auto HDR helping would be preferred, lagging in changing would be penalty. Metering on focus point, manual exposure compensation will not add points, but specific design will.

Image quality: image quality includes details, noise reduction, HDR, distortion and other indicators relating to hardware and software. Details mainly refers to resolution, which is close related to hardware and software, however, we will definitely not praise fake details brings by over-sharpening. Noise reduction is not only the low lights, we will also concern the dark area in HDR or in daylights. We would prefer moderate HDR, for we do not think those hardly HDR photos are good, for those almost have the same luminance in the dark and bright area. We will add the image quality change bring by HDR in our scoring system, lots smartphone would add noise, and even ghosting, when using HDR. We will also keep an eye on distortion, but this should not be cared about too much, without significantly distortion that can be noticed by human eyes is qualified.

Colors: Colors are mainly about viewing experience, including color style, auto white balance trend, texture, etc. Color style refers to vivid, bright, serene, and so on, we accept different color styles, and we do not think that accuracy is the only feature in valuing, as long as good-looking we will add points. Auto white balance affects color temperature trend, moderate shift could be accepted, for example, iPhone has been keeping a warmth trend, this could be regard as stable shift, while shifting unstable AWB will lose some points. Texture is a sensitive, but less mentioned indicator, Soomal will value texture for colors. If a camera takes metal texture in a photo which looks like plastic, it will lose points.

User experience: Usability, stabilization will be important parts of user experience. Excellent functions will win some points, but we will not deduct those providing less functions. Display performance will be a part of camera user experience, including visibility in strong sunlight, color management. Some smartphone could even lower screen brightness in strong light, we just roast on it before, but now, we can cut its points. Also, without color management will also lose some points.

A+B Mark

Dual-camera design are popular today, so for the sub-camera, we will active B plan.

Autofocus [speed, pre-focus, focus logic, auto focus point setting] 20%
Metering [Accuracy, detecting] 10%
Image quality [details, noise reduction, HDR] 30%
Colors [style, AWB, texture] 20%
Function [Focal length, Bokeh simulation, others] 20%

B mark is similar to A, just changed the 5th from user experience to function.

Function: This mainly refers to focus length and bokeh simulation. Of course, multi-focus-length will add points. Bokeh simulation will be valued according to photos, including accuracy, usability, functionality, etc. Dual-camera will probably affects AR, MR or other applications related to camera, we would prefer some excellent ones.

Rule and Sample

Each item has a 100 point baseline, a camera's mark will be given after weighted.

Autofocus: 110
Metering: 90
Image Quality:100
Colors: 120
User Experience: 90

After weighted, Smartphone A scores 103.

Autofocus: 100
Metering: 90
Image Quality:100
Colors: 110
User Experience: 90

Autofocus: 70
Metering: 80
Image Quality: 60
Colors: 80
Functionality: 60

If smartphone B adopts dual-camera, then after weighted, it will score 99 + 68.

Mark A and B will help you to compare different camera system. We will offer two ranks, one for main camera, and the other for dual-camera system.

In Progress

This beta will be adopted from today on, you will Soomal Camera Mark System in later reviews, because we had accumulated enough experience, and we should provide more quantitative and obvious result for you. As this is a beta system, it is still insufficient, we will patch it.

Soomal
November 2th, 2017

Please Rate 1 2 3 4 5 6 7 8 9 10
Soomal.com