Too many security products trade on fear, uncertainty, and doubt among customers and the media. At the same time, giving a positive review to a flawed product risks putting people’s privacy and even their safety in danger. This is especially true for virtual private networks, or VPNs. When we test VPNs, we consider their performance and available features with the goal of writing reviews that are factual and useful to our readers.
This is harder than it sounds. If we relied entirely upon objective measurements, it would be trivial for a vendor to game the system by inflating particular stats like server count or number of simultaneous connections. If we relied only on subjective observations, we’d miss the features that make it unique. Combining the two—objective measurement and subjective observations—is messier, but leads to better and more comprehensive analysis.
Our readers may not always agree with our conclusions, but we strive to include enough information in our reviews so that readers can form their own opinions, too. In fact, we encourage them to do so.
A Note About Ethics
In an era of fake news, phony reviews, and mounting concern over pay-for-play content, we believe it is important for readers to understand how our company earns money and how our reviews are written. At the top of every review on PCMag, VPN or otherwise, is the following statement:
PCMag reviews products independently, but we may earn affiliate commissions from buying links on this page.
In practice, this means that PCMag may earn a commission either from the company whose product has been reviewed, or some other entity. It’s a common practice among review sites. All handling of affiliate commissions is entirely separate from our editorial process, and managed by a completely separate staff. By design, reviewers do not have any knowledge of the specific ways in which a particular review is monetized. Nor do reviewers or editors receive a cut of that monetization. Reviewers, full-time or freelance, are paid for their work and do not earn a commission or bonuses for the reviews they produce.
Similarly, we at PCMag must be as transparent as possible about our relationships with vendors. Our parent company, Ziff Davis, is nearly a century old and has expanded to encompass many ventures beyond publishing. In the last decade, the company has acquired several technology companies, including some VPNs. We will always disclose whether a product we review is owned by our parent company. Moreover, these companies do not receive any special treatment or access to our work at PCMag. We only extend one courtesy to Ziff Davis companies: We inform them when a review will be published. Nothing else.
Importantly, companies—even those who have affiliate relationships with PCMag or are owned by our parent company—do not dictate the outcome of reviews. This is assured by both PCMag’s code of ethics and our analysts’ collective bargaining agreement, which is a legally binding document. The members of our editorial staff value their reputations and would not stake them on what amounts to bribery or corporate nepotism.
Why are we focusing so much on our editorial ethics? First of all, we’re proud of it, and it bears repeating. Second, it is especially relevant to the discussion of VPNs. Security software in general attracts readers that are extremely concerned with fairness and transparency. The VPN market in particular seems to be awash with suspicion, some of it the result of paranoia, some created by pay-for-play “review” sites, and some—purportedly—stoked by VPN companies themselves. Rest assured, we hear it when readers worry about the ethics of VPN reporting, and it’s important to us, too.
VPNs Are For Privacy, Everything Else Is Gravy
To evaluate a product, you must first understand what it’s for. For example: A MacBook Pro is an excellent laptop, but probably a terrible waffle iron. When it comes to VPNs, we consider them a privacy tool first, and evaluate them primarily on those grounds.
Privacy tools are distinct from security tools. While the two concepts overlap, a privacy tool shields your devices from efforts to track and identity them, or proactively removes identifiable information. A security tool identifies, removes, and (ideally) prevents the use of malicious software or hardware that would harm you, your machines, or your files.
It’s important to make this distinction because of misleading advertising from VPN companies. A VPN cannot protect you against every threat. While we consider VPNs useful tools, that utility is limited and to say otherwise puts people at risk. We strongly encourage readers to use standalone antivirus, enable multi-factor authentication wherever available, and use a password manager to create unique and complex passwords for each login they have.
Our Testing Criteria
When we evaluate the privacy-protecting abilities of a VPN, we look at:
- The technology it uses,
- The servers it makes available,
- The presence of privacy-enhancing tools (primarily multi-hop connections and VPN access to Tor), and
- The measures a company takes to ensure that the VPN itself does not become a threat to user privacy.
This last point can include everything from third-party audits to using an anonymizing login system like the ones employed by Editors’ Choice winners IVPN and Mullvad VPN.
While most of our readers are likely concerned about a VPN’s ability to stream overseas video content, that’s not our primary concern. We test each VPN’s ability to access Netflix, and sometimes compile that information into comprehensive features for readers interested in using a VPN for streaming. But if the best VPN we’ve ever seen is blocked by Netflix, that’s a minor issue at worst.
Some VPNs now include an anti-malware component, usually claiming to stop malicious files before they reach your computer. In general, we believe that the job of stopping malware is better left to dedicated antivirus software, whether it’s a paid service, a free tool, or software that’s bundled with your OS. We’re also wary of the privacy implications that go along with examining files sent through a VPN. Generally, we do not test the efficacy of integrated anti-malware features in VPNs but will sometimes use the tools from the Anti-Malware Testing Standards Organization to verify that the anti-malware components are functioning as expected.
In short, if a VPN doesn’t do much for privacy but is festooned with extraneous features or is built primarily to connect US residents to free BBC streams, we don’t consider it to be a good VPN.
Pricing and Plans
When we compare prices among VPNs, we always state the monthly pricing and report that price in the reviews. While nearly all VPN services offer a discount when you sign up for a long-term subscription, our goal is to report the base-level price for each service and not the discounts you might get for signing up for a year.
Another reason we report the price of the monthly plan is because we want to encourage readers to start with a short-term subscription for a VPN. Too often, readers have told us they spent $60 or more on a year-long subscription to a VPN only to discover it doesn’t work for them. It’s far better to try out a service for a month or three, and decide later to spring for a long-term, discounted billing plan when you’re certain you want to keep it. Consider the extra money you spend up front to be a down payment against buyers’ remorse.
Additional Features
With more and more VPN services popping up, companies have begun adding more and varied features to their offerings in order to stand out. In PCMag’s review of each service, we strive to report as many features as we can but focus on the ones we believe most protect user privacy and reflect the value of a service. The number of devices the VPN service allows you to connect simultaneously, for example, is a concrete measurement of value and a point we always report.
Each VPN review also notes the most significant add-ons available from a VPN service. These add-ons usually include static IP addresses, additional simultaneous connections, and so on. We generally do not test these add-ons, and instead focus on the core product being sold to consumers.
Server Numbers and Distribution
A key differentiator among VPNs is server distribution. If a VPN company offers no servers where you are or where you wish to spoof your location, it’s not going to be very useful. We refer to the server locations and how widespread they are “geographic diversity.” In general, we give preference to services with a lot of servers in many different parts of the world. It’s particularly important to frequent travelers and users overseas, since a VPN server closer to their computer will likely mean a faster and more reliable connection. For users in the US, more VPN server locations means more opportunities to spoof your location.
We do not test each and every connection to ensure it is functional. This is one of the places where we must assume companies are telling us the truth about their products. However, we do investigate if we find a server is unavailable during testing.
Most VPN companies offer servers in Asia (sometimes excluding China, as explained below), Australia, Canada, the US, and Western Europe. Better services include a few servers in Africa, Eastern Europe, the Middle East, South America, and Southeast Asia. We give preference to services with robust offerings in Africa and South America, two areas generally underserved by VPN companies.
Our VPN reviews include a current count of the number of servers provided by the VPN company. With numerous servers available, the VPN company can assign fewer people to each server. That means a bigger slice of the bandwidth pie for each person assigned to a given server.
This figure, however, is just part of the story. Most VPN companies spin up new servers to meet demand as necessary, causing the precise number of servers to change often. It also doesn’t make sense for a small company with only a few thousand subscribers to have as many servers available as a company with a million subscribers. A company might also seek to inflate their server count by using numerous virtual servers, which we explain in greater detail below. We try to balance these considerations in our reviews.
When talking with VPN companies, we ask about the number of virtual locations and virtual servers. Virtual servers are software-defined, meaning a single hardware machine can play host to many virtual servers. Virtual locations are servers configured to appear somewhere other than where they are physically located.
Neither is inherently bad. Virtual servers allow VPN companies to quickly respond to user demand and keep their networks ticking over nicely. Virtual locations can expand a company’s reach, and sometimes provide VPN protection in regions where it’s not safe to physically house a server. In our reviews, we look to see that virtual locations are clearly marked and that the company has standards and practices regarding the security of its server infrastructure.
VPNs and Censorship
VPNs are especially useful for people living or working in countries whose governments have chosen to restrict information and punish dissent. Given those stakes, we believe it would be unethical to choose a service that would be “best” for circumventing censorship.
Additionally, we do not believe our testing is currently adequate for making this determination. It is our hope that experienced organizations will come forward to identify technology and practices that can be used to safely evade censorship without putting individuals at risk.
We appreciate and elevate companies that contribute to a free and open society. We also note as to whether or not a VPN company offers servers in countries with particularly restrictive internet policies. Our understanding is that connecting to one of these servers from within the country will not circumvent censorship but would provide some modicum of privacy and security to the user—particularly for visitors to the country.
When we write stories about a VPN that would be “best” for a region with repressive policies, we base it on the availability of local servers and the region and any documentation provided by the VPN company itself. We try to make clear in each of these stories that readers seeking to circumvent censorship do so at their own risks, and our evaluations cannot guarantee safety. This is a responsibility we take seriously and try to balance the benefits of directing readers toward trustworthy services against the inherent dangers of resisting a repressive regime.
User Experience
There is a false dichotomy in digital security between a product’s ease of use and the value it provides. We frequently see commenters dismiss a product (generally one they have never used) as worthless because it looks pretty. That ignores a truth about humans: No one is going to buy a difficult or annoying product just because it might protect them from hypothetical threats. A well-designed security product that average consumers can actually use is better than a perfect security tool that is only accessible via the command line.
When we review VPNs, we go through the setup process for each service. We also take time to poke around settings and see how easy it is to perform certain functions. It’s important that readers, like you, have a sense of what using a given product will be like from reading our reviews.
Sometimes, an excellent user experience makes a mediocre product better. Conversely, poor user experience undercuts the value of an otherwise stellar product. In general, we place great emphasis on a product being easy to use and accessible to users with all levels of experience. At the same time, we cannot deny the importance of technical excellence, especially when it’s combined with value.
VPN Protocols
There are several different means for creating a VPN connection, but not all of them are equal.
In our reviews, we give preference to the services that offer OpenVPN. This open-source protocol has been picked over by volunteers, helping to quickly find and fix potential issues. It also has a reputation among professionals for providing better speeds and more reliable connections. IKEv2 is another good choice, as it uses newer and more secure technology than older protocols.
Increasingly, we’re looking to see if companies support the WireGuard protocol. This was an experimental protocol, but one that has seen greater acceptance and adoption in recent years. We expect it will soon become the most popular choice.
The other protocols out there are either older or held in less high regard. With the availability of excellent tools, it’s a mark against a VPN company if they can’t offer them.
Some services, such as ExpressVPN, Hotspot Shield VPN, and VyprVPN, have started to deploy their own VPN protocols. We’ve spoken directly with representatives of these companies about their protocols to learn that they’re built on established, validated tools. We would be very concerned by any company that attempted to create its own encryption systems, as it is a notoriously difficult task.
To properly evaluate proprietary VPN protocols requires resources and expertise well beyond our means. Similarly, we have no way to evaluate how companies have implemented existing, established VPN technology. We must therefore rely on the work of independent security researchers to uncover bad practices. As a rule, we assume that VPN vendors are good actors, operating in good faith, until we have reason to believe otherwise.
VPN Speed Testing
Most of our readers are concerned about the impact on internet speeds. That’s understandable since most VPNs increase your latency and slow your overall internet connection. Why this happens is simply a product of taking your internet traffic and running it through extra steps.
While we do take great pains to do a fair evaluation of VPN speed and performance, we do not consider it to be a core criterion. We reward VPNs with exceptional results and devalue VPNs with extremely bad results.
To find the fastest VPNs, we run the Ookla SpeedTest tool 10 times with the VPN active, and then 10 times when the VPN is inactive. We then take the median of each set of results and compare them to find a percent change. The Ookla test returns results for latency, upload speeds, and download speeds, so those are the metrics we use as well.
(Editors’ Note: Ookla is owned by Ziff Davis, PCMag’s parent company.)
The tests have limitations. Issues with the internet connection we use in testing could affect the results. Background system processes on our test computers could muck up the results. Also, while we strive to use a “clean” testing network, it is subject to changes because of local service conditions and other devices connected to the network. We strive to account for these issues as much as possible.
Most significantly, despite gathering numerous test results, it is still only a single data point and not enough to give a definitive judgment on a service’s overall network performance. Consider that when PCMag does our Fastest Mobile Network survey of wireless providers, we test constantly over the course of several days and across several states. To create a truly accurate picture of VPN performance, we would have to replicate the scope and scale of that testing, which is far more expensive, far more time-consuming, and requires tools that currently do not exist.
Because of these limitations, PCMag presents its speed testing not as the final word in a VPN’s performance but instead as a snapshot. It is meant to say that at this given day and time, this VPN performed this way. We have always cautioned readers that our speed test results are best used for comparison and are likely to differ greatly from what readers experience.
Most recently, we’ve had to change how we test VPNs because of the ongoing COVID-19 pandemic. Previously, we tested all the VPN services back-to-back over the course of several days. However, all PCMag analysts and editors transitioned to remote roles in mid-March 2020, and we have had only limited access to the PCMag Labs testing network. As a result, we now test VPNs in batches and publish the results immediately.
The chart you see below appears in all our VPN reviews and updates automatically with the latest results.
Trust and Privacy
When its product is active, a VPN company has the same level of insight as your ISP has into your online activities. Because of that, it’s important that you trust the VPN company you sign up with, and that you are comfortable with the potential pitfalls that using a VPN might entail.
When we review VPNs, we read the privacy policy of each service. In particular, we look for what information a company gathers about customers and their behavior, how the company protects user information, and how a company responds to requests for information from governments and law enforcement. We also directly ask VPN companies to explain their policies and to disclose what legal jurisdiction they operate under, how the companies make money, and the name of any parent company.
It is, of course, entirely possible for a VPN company to lie to the public in their privacy policy and lie to us during our interviews. The goal with these questions is to put them on record, should contrary information surface in the future.
In our reviews, we make judgments about the steps VPN companies have taken to protect their customers. We know that not everyone will agree with our analysis, and so we present the facts as we have them along with our analysis and the context gathered from nearly 40 VPNs. We will always say what we think, but a reader whose priorities differ from ours will have the information to make their own decision.
All companies will have to respond to legal requests for information somehow. The best will present nothing, or very little, because they have no user-identifiable information. We prefer that companies be transparent about their efforts to protect customers, but also about their interactions with law enforcement. Companies should issue transparency reports that record the requests for information and how the company responded.
We are also wary of xenophobia in the guise of seeking the best and most secure option. Rumormongering is not unheard of in the security industry, nor is using baseless fears over race, class, and other factors. For example: China and Russia have been accused of numerous cyberattacks against the US and are known for fostering oppressive environments domestically. Because of this, some consumers refuse to use security products from these countries, believing that they are inherently compromised. By the same token, the US government is responsible for the largest and perhaps most intrusive intelligence gathering operation in the world (if the information from Edward Snowden is to be believed) and has even intercepted domestically made products in transit to install malicious software. Yet US products are sometimes regarded as more trustworthy.
For the time being, we are hesitant to penalize a product for its country of origin alone. Instead, we present the information we collect, provide context, and encourage readers to make their own choices. We support and hope for continued reporting about the risks to individual privacy posed by law enforcement and government intelligence gathering.
We also must acknowledge that no company is immune to attacks or data breaches. Instead of holding companies to an impossible standard, we are more interested in how each company responds to crises. A poor response will likely result in a reduced score from our analysts. We also must give space for companies to change. If a shifty product is reborn and bad past practices abandoned, we must view that change favorably, if sceptically.
Our testing assumes that the VPN companies we review are good actors, operating in good faith, until we learn otherwise. We rely heavily on the work of security researchers who have unmasked some of the worst behavior among VPN providers, and on the robust security community that is quick to point out the flaws in any product.
Is this an ideal system? No. What the VPN industry needs is like what is already common for antivirus products: robust third-party testing and easily deployed tools to confirm that the product is performing as advertised. We hope to foster the creation and dissemination of such tools and standards in the future.
DNS and IP Leak Tests
In 2019, we added testing to see if VPN services leak IP address or DNS information.
To do both, we use the online DNS Leak Test tool. This tool provides a list of which DNS servers are being used, and our public IP address. By comparing the list with and without a VPN, we can see if DNS or IP address information is leaking out.
VPNs Beyond Windows
Windows remains the dominant desktop operating system, and as such the bulk of our VPN testing and analysis is performed on a Windows machine. Speed testing is carried out entirely on our Windows machine. Still, it’s critical that a VPN perform well across all platforms.
In our testing, therefore, we also install native VPN apps on Android, iOS, and macOS devices. We look to see which features are available, as these are sometimes platform specific. We re-run our DNS leak tests on these platforms to ensure the VPN is working as expected.
Each platform has its own design language, and the VPN apps should speak that language fluently. On mobile devices, you interact through a touch screen instead of a keyboard and mouse/trackpad. iOS and Android look as different as Windows and macOS, and VPN apps should blend in with their surroundings. A VPN app should be visually and functionally similar across all platforms but tailored for each—whether it’s a VPN for Android or iPhone. We see ease of use is a major criterion of an excellent VPN, and this is especially true for mobile VPN apps.
The Evolution of VPN Testing
At PCMag, we strive for reviews that are meaningful, based on testing that is reproducible. We eschew including unnecessary information or meaningless testing data that would confuse, rather than elucidate, the reader. Walking this line is always a tradeoff, and it’s no different for VPNs.
As always, we will adapt and improve our testing as well as update our reviews as products change, but also as the landscape around them changes. Perhaps a new technology will completely upend what makes a VPN worthy. Whatever the case, the VPN reviews you read here on PCMag will always be as accurate and useful as we can make them.
