Samsung Galaxy S23 Ultra Selfie test

Exposure and color are key attributes for technically good images. For exposure, the main attribute evaluated is how bright faces are in various use cases and lighting conditions. Other factors evaluated are contrast and dynamic range, e.g. the ability to make details visible in light and dark areas of the image. Repeatability is also important because it demonstrates the camera’s ability to provide the same rendering when shooting consecutive images.
For color, the image quality attributes analyzed are skin tone rendering, white balance, color shading, and repeatability.

Autofocus tests evaluate the accuracy of focusing on the subject’s face, repeatability of accurate focus, and depth of field. While a shallow depth of field can be nice for a single-subject selfie or close-up shot, it can be problematic in specific conditions like group selfies; both situations are tested. Focus accuracy is also evaluated in all real-life images taken, from 30cm to 150cm, and in low-light or outdoor conditions.

Texture tests analyze the level of detail and texture of subjects in images taken in the lab as well as real-life scenarios. For natural shots, particular attention is paid to the level of detail of facial features, such as the eyes. Objective measurements are performed on map images taken under various lighting conditions from 1 to 1000 lux and different types of dynamic range conditions. The charts used are the proprietary SBMARK (DMC) chart and the Dead Leaves chart.

Noise tests analyze various noise attributes such as intensity, chromaticity, grain, and structure on real-life images as well as graph images captured in the lab. For natural images, special attention is paid to noise on faces, but also to dark areas and high dynamic range conditions. Objective measurements are performed on chart images captured under various conditions from 1 to 1000 lux and different types of dynamic range conditions. The graph used is the SBMARK Dead Leaves graph and standardized measurement as Visual Noise derived from ISO 15739.

Artifact assessment examines lens shading, chromatic aberrations, distortion measurement on the Dot and MTF graph, and ringing measurements on the SFR graph in the lab. Particular attention is paid to ghosting, quantization, halos and hue variations on the face, among others. The more serious and frequent the artifact, the greater the deduction of points from the score. The main artifacts observed and the corresponding loss of points are listed below.

Bokeh is tested in a dedicated mode, usually portrait or aperture mode, and analyzed by visually inspecting all images captured in laboratory and natural conditions. The goal is to reproduce a portrait photograph comparable to one taken with a DSLR and a large aperture. The main image quality attributes paid attention to are depth estimation, artifacts, blur gradient, and bokeh blur reflector shape. Vertical image quality attributes (exposure, color, texture) are also taken into consideration.

SBMARK engineers capture and evaluate more than 2 hours of video in controlled lab environments and low natural light scenes, indoors and out, using the default front camera settings. Evaluation consists of visually inspecting natural video taken under various conditions and performing objective measurements on lab-recorded graph video under various conditions from 1 to 1000+ lux and color temperatures from 2,300K to 6,500K.

Exposure tests evaluate facial brightness and dynamic range, e.g. the ability to make details visible in light and dark areas of the image. The stability and temporal adaptation of the exposure are also analysed. Image quality color analysis examines the rendering of skin tone, white balance, color shading, white balance stability and its adaptation when the light changes.

Texture tests analyze the level of detail and texture of real-life videos as well as graphics videos recorded in the lab. Natural video recordings are evaluated visually, with particular attention to the level of detail of facial features. Objective measurements of card images taken under various conditions from 1 to 1000 lux are performed. The chart used is the Dead Leaves chart.

Noise tests analyze various noise attributes such as intensity, chromaticity, grain, texture, temporal aspects on real-life video recording, as well as graph videos taken in the lab. Natural videos are evaluated visually, with particular attention to noise on faces. Objective measurements are performed on graph videos recorded under various conditions from 1 to 1000 lux. The graph used is the SBMARK visual noise graph.

The stabilization rating tests the device’s ability to stabilize footage using software or hardware technologies such as OIS, EIS, or any other means. The evaluation examines overall residual face and background motion, smoothness, and yellow artifacts, during walking and panning use cases under various lighting conditions. The video below is an excerpt from one of the tested scenes.

Artifacts are evaluated with MTF and ringing measurements on the SFR graph in the lab, as well as frame rate measurements using the Universal Timer LED. Natural videos are visually evaluated by paying close attention to artifacts such as quantization, hue shift, and face rendering artifacts, among others. The more severe and frequent the artifact, the more points will be deducted from the score. The main artifacts and the corresponding point loss are listed below