We are happy to introduce a new smartphone scoring system on PhoneArena, which has gone into effect starting in late September 2023!
This new system is a complete overhaul of the way smartphones are rated and it accounts for more nuance and detail, as we now rate smartphones across 12 different categories with carefully selected criteria to ensure that all devices are treated equally and to minimize personal bias.
So what’s this new PhoneArena Smartphone Review Rating system all about and how does it work?
After careful deliberation, we have come up with 12 fundamental categories that describe our modern smartphone experience:
- Speed and Performance for daily smartphone tasks
- Performance in heavy duty tasks and gaming
- Battery life
- Charging speed
- Wireless charging speed
- Camera – Photo Quality
- Camera – Video Quality
- Display Quality
- Biometrics Speed, Performance and Reliability
- Audio Quality
- Software and Updates
- Design
Each of these categories carries a different weight in the scoring system, and some categories have a larger importance for the score than others. That is where we had to use our judgment and readers’ opinions to find a balance that reflects the way most people use their phones. For example, the battery life of a device is a very important category that carries more weight than the charging speed. Both are important, but we feel most people have set their priorities this way.
At the moment, you can still only see the final, holistic score in our reviews, but soon we will be introducing a new visualization where you will be able to see a breakdown of scores by category.
Is the price of a device accounted for in the score?
No.
We are not leaving it at that and we will also show you more context for each score, so you get an idea of where the device stands among its peers too. And remember, that often a lower score will correspond to the lower-end class of the device, but it might still be outperforming its peers!
Explaining each of the categories
So let us quickly guide you through all the individual categories:
Category 1: Daily Performance
The perceived speed and smoothness for easy, daily smartphone tasks like calls, mails, browsing and social media
For this category, we evaluate how well the phone performs in daily tasks. Questions that we ask when we come up with a score in this category are the following:
- Do apps load fast?
- Is the scrolling smooth?
- How does the gesture navigation work?
- Is there any overheating with simple tasks?
Category 2: Extreme Performance
How well the phone performs for heavy duty tasks and gaming
We have a separate category measuring more intense tasks and this score is based on a weighted mix of benchmark scores.
In the mix we have CPU performance tests and GPU / gaming-related benchmarks.
Category 3: Battery life
How long does it last?
Each phone we review goes through three battery tests. Using specialized equipment and software, we set each phone to a fixed brightness level to ensure an equal playing field and all phones are connected to a Wi-Fi network for these tests.
Test #1: Web Browsing — we run a script that cycles through a set of webpages and scrolls around at pre-set intervals, emulating a real world browsing session. This is our lightest battery test.
Test #2: Video Streaming — we run the same video playlist over a Wi-Fi connection and at the same video quality at the full size of the screen, and measure how long the phones last.
Test #3: 3D Gaming — we run a popular gaming title with the same graphics settings and in the same environment / level on each phone we test. This test stresses the GPU.
Based on those three results, we come up with a final battery score for this category.
Category 4: Charging speed
We measure and we rank
We use the recommended by the manufacturer charger and the cable that comes in the box to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.
Category 5: Wireless charging speed
We test the wireless charging speed too!
We use the recommended by the manufacturer wireless charger to measure how long the phone charges from 1% to 100%. Based on the results, each phone is assigned a score.
Category 6: Camera – Photo Quality
Assigning a score to an image is hard, but we have a new complex system just for that
We have created a new complex system that looks at various aspect of image quality and we come up with a score based on extensive testing. We will soon introduce you to that new camera rating system with all the exciting detail that goes into it, so stay tuned!
Category 7: Camera – Video Quality
How good is the video quality on a 1-10 scale
Just like we have for photo quality, we now have a new rating system for video quality. Expect a lot more detail to come in a separate post where you will learn what exactly goes into that brand new video quality rating!
Category 8: Display Quality
We do a number of lab tests measuring various screen characteristics
We use professional equipment and software to measure the characteristics of a phone display: maximum brightness, minimum brightness, color accuracy, and more.
Category 9: Biometrics
Fingerprints and face ID performance ranked by performance and accuracy
We test the performance of a phone’s fingerprint scanner or face recognition system for speed and accuracy, and assign a score based on the results.
Category 10: Audio Quality
Evaluating a phone’s speaker system
We test the phone’s loudspeakers and assign a score based on how loud it plays, and the quality and richness of the sound. The classification is based on typical performance for smartphone loudspeakers.
Category 11: Software and Updates
The software update commitment matters
We investigate the software update commitment for each phone and look if there are limiting factors to the software experience, like a ban on fundamental apps or other major restrictions on software. Each phone is assigned a score based on that.
Category 12: Design
Build quality, materials, looks
How well is this phone designed? That can be a subjective matter, but we have a set of criteria that help us come up with a rating based on the build quality, the materials used, any special design elements, and more.
Got any questions?
You will soon see a new graph with each review that shows the individual scores for each category and will have a clear understanding of why each smartphone review score is the way it is. We are working to deliver that to you in the coming months.
For the first few months of using this system, you will also see a disclaimer right after the introduction paragraph of a review saying specifically when a smartphone score is based on the new system. Older reviews might still use our previous scoring system, which is why at the moment direct comparisons of many scores are not possible. We will be re-evaluating older devices based on the new system over the coming weeks, however, and will be updating this article when the process is complete.
Also, we will be happy to explain the details, so ask away! And… tell us what you think about this new system! Do you like it? Is there something else you would have included? The comment section is now officially open!
