The question “Is the open web dying?” might evoke Betteridge’s law of headlines, which suggests that any headline ending in a question mark can be answered with “no.” But here, the answer is, unfortunately, closer to “yes.” To understand why the open web is faltering and to assess AI’s role in this decline, let’s first define what we mean by “open web.”
For the purposes of this article, the open web is:
- Publicly accessible without barriers like paywalls or subscription fees
- Unfiltered by governmental censorship
- Free from platform-based algorithmic control, allowing content to be discovered through open browsing and linking
- Interoperable, meaning that it fosters connections between sites without restrictions
The open web was once the backbone of online information, enabling anyone with an internet connection to freely share, link, and access content. But a combination of shifting business models, regulatory changes, and technological advancements — most recently AI — is reshaping the internet into a more closed, restrictive environment.
The squeeze on small content providers and diversity
A primary threat to the open web is the gradual exclusion of small content providers. Unlike large corporations with abundant resources, independent websites and niche creators often struggle to compete with the heavily promoted and easily accessible content hosted on major platforms. These smaller providers contribute significantly to the diversity of online information. When they’re squeezed out by competition or forced to hide behind paywalls to sustain themselves, it reduces the variety of voices and viewpoints available on the web.
Algorithm-driven platforms are partly to blame for this problem. On platforms like TikTok, Instagram, and Facebook, what you see is largely determined by engagement-driven algorithms. These algorithms prioritize content that keeps users hooked, which tends to favor big brands, viral content, and sensationalized news over niche, independent content. Smaller creators who can’t afford to engage in paid promotion or optimize their content for these algorithms struggle to gain visibility, further eroding the diversity that once characterized the open web.
Algorithmic search and the downfall of small organizations
Algorithmic control is not limited to social media platforms; search engines are also shifting how they prioritize and display content.
In a TheVerge.com article (“Google is getting even worse for independent sites”), a series of changes to Google’s search algorithm were shown to heavily impact small websites. In one example provided in the article, a website stated that its search traffic decreased 91% within a few months, devastating its business.
Google’s Pandu Nayak was quoted during a Q&A at the most recent Google Web Creator Summit, when asked about the massive drop in traffic for smaller sites following a search algorithm change, “Our goal is to surface great content for users. I suspect there is a lot of great content you guys are creating that we are not surfacing to our users, but I can’t give you any guarantees, unfortunately” (“Google Algorithm Update Due Soon, But Don’t Expect Lost Ranking Recovery,” userp.io).
The platform dependency problem: The case of Facebook and Zynga
When platforms dominate traffic, they gain power over businesses that depend on them. Facebook’s relationship with game developer Zynga is a prime example. For a period, Zynga relied almost entirely on Facebook to drive users to its games. This dependency made Zynga vulnerable to Facebook’s policy shifts, and following one set of changes to Facebook’s policies, Zynga’s reach and revenue collapsed.
This “platform dependency problem” is increasingly common. Businesses and content providers have become overly reliant on platforms, such as Facebook, Instagram, WeChat, TikTok, and YouTube, as gateways to their audiences. With fewer visitors discovering sites through open browsing, websites find themselves at the mercy of platform algorithms and policies. This trend harks back to the early days of the internet when AOL’s walled-garden approach reigned supreme; users were funneled through a closed environment rather than navigating the open web. It now often means users won’t even leave the platforms to experience the diversity of the open web.
The rise of internet filtering and regional “internets”
Countries around the world are exerting increasing control over their domestic web landscapes. Nations like China and Russia maintain tightly controlled “national internets” that restrict content accessible to their citizens. Countries like Turkey and India have also taken steps toward internet filtering, limiting content to align with governmental priorities or ideological positions.
This segmentation of the web into national silos directly undermines the open web’s mission of universal accessibility. The trend threatens to balkanize the internet, creating isolated digital ecosystems where content is available only to specific audiences, subject to government oversight (“Internet Censorship: A Map of Internet Censorship and Restrictions,” comparitech.com).
Even the United States has proposed a ban to TikTok (which seems to have died) that demonstrates how even in democracies there is a similar sentiment that leads to fragmentation of the internet. When the global web becomes partitioned by national borders, the open web becomes a casualty of political agendas.
AI’s impact on the open web: Breaking the covenant
AI, particularly generative AI, is also playing a role in the decline of the open web. Traditionally, web creators shared content with the implicit understanding that they’d receive traffic and recognition in return. Generative AI is disrupting this relationship by enabling users to access summarized or rephrased information without visiting the original source.
For instance, Google’s AI-powered search features provide answers directly on the search results page, often synthesizing information from multiple sources without directing users to those sites. Similarly, services like OpenAI’s recent search technology license content from large sites and position links to the sites in the search results, but smaller organizations don’t have the power to enforce licensing fees or prominent placement, meaning they are denied both licensing revenue and advertising and traffic.
These approaches are a departure from the web’s initial covenant: Creators provide content, and in return, they receive traffic and visibility. AI effectively decouples content creation from website traffic, depriving creators of the benefits they once enjoyed from direct engagement.
When users can obtain comprehensive answers without clicking through to the source, smaller content providers lose out, as do specialized, niche sites that rely on direct user visits for revenue and visibility. This loss of traffic not only reduces ad revenue but also drives more providers toward subscription models, further closing off access to free, diverse content.
The decline of ad-supported content and the rise of paywalls
I never thought I would say something positive about online advertising, but it does (or did) support websites.
Ad revenue has long been a cornerstone of the open web supporting free access to websites. But as advertising and search shifts toward major platforms like Instagram, TikTok, and YouTube, small websites find it increasingly difficult to survive on ad revenue alone. For example, while large publishers can still attract ad dollars, small and niche sites face declining ad rates and reduced engagement at the same time less traffic is being channeled to their sites.
This financial squeeze is pushing many sites to adopt paywalls, subscriptions, or crowdfunding models like Patreon to sustain themselves. While these models can be successful from a revenue perspective, they significantly reduce the reach of the content. It is taking large swathes of content and moving them off the open web and into walled gardens. This is also exemplified by the massive boom in paid newsletters over the past few years. This results in users facing more barriers to accessing diverse content, eroding the principle of an open, accessible web.
This article has gotten long enough, so we won’t get into the impact on the open web of smartphone apps and app stores, as well as ecosystem lock-in.
What does this mean for organizations?
For organizations, the decline of the open web and the shift toward closed, algorithmically controlled platforms mean that digital strategy must evolve to adapt to new challenges around visibility, engagement, and revenue. As platform and search algorithms prioritize content that maximizes engagement, smaller organizations, in particular, face difficulties in gaining organic reach, which can diminish their visibility and limit audience growth. To stay competitive, organizations need to diversify their engagement tactics — exploring (among many things) content personalization and building first-party data to connect with audiences outside of major platforms. Additionally, AI-driven information retrieval tools that aggregate content without directing traffic to original sources compel organizations to rethink their content strategies, ensuring they still receive value from their contributions even when users may not visit their sites directly.
This also requires organizations to consider alternative monetization methods as ad revenue concentrates in larger organizations and platforms. Subscription models, premium content, and value-added services like exclusive data or tools may become increasingly vital for generating revenue. Regulatory shifts, including government censorship and regional “internets,” mean that organizations operating globally must adopt region-specific strategies to remain accessible and compliant in different jurisdictions. By strengthening resilience through diversified engagement, regional compliance, and innovative revenue streams, organizations may better navigate a web that is moving away from open access toward more walled gardens, algorithmic-controlled interaction, and increased domination by platforms.
Learn more about IDC’s research for technology leaders OR subscribe today to receive industry-leading research directly to your inbox.
International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the technology markets. IDC is a wholly owned subsidiary of International Data Group (IDG Inc.), the world’s leading tech media, data, and marketing services company. Recently voted Analyst Firm of the Year for the third consecutive time, IDC’s Technology Leader Solutions provide you with expert guidance backed by our industry-leading research and advisory services, robust leadership and development programs, and best-in-class benchmarking and sourcing intelligence data from the industry’s most experienced advisors. Contact us today to learn more.
Daniel Saroff is group vice president of consulting and research at IDC, where he is a senior practitioner in the end-user consulting practice. This practice provides support to boards, business leaders, and technology executives in their efforts to architect, benchmark, and optimize their organization’s information technology. IDC’s end-user consulting practice utilizes IDC’s extensive international IT data library, robust research base, and tailored consulting solutions to deliver unique business value through IT acceleration, performance management, cost optimization, and contextualized benchmarking capabilities.
