This is part 10 of a 12-part series called Distilling the Web to Zero. The web is far from done improving - particularly as it pertains to building rich web applications that are, to users, developers, and businesses alike, more desirable than their native-app counterparts. This series is prefaced by defining the biggest challenges on the road towards that goal and the 12 essays that follow explore potential solutions using concrete examples.

The uncanny valley of SPAs

"...we need to reckon with the fact that single page apps have kinda ruined the web."
– Rich Harris, creator of Svelte

No doubt, single-page applications have a bad reputation. The fact that Apple’s and Google’s app stores even exist is a daily reminder of the SPA's shortcomings. There'd be no reason for proprietary walled gardens to exist over the open web if users didn't overwhelmingly prefer native apps to SPAs. This isn't true for all of the open web though. Wikipedia is fantastic. StackOverflow is fantastic. There are many such examples but the pattern of whether a website is loved or loathed strongly correlates to whether it's a multi-page application (MPA) or a single-page application (SPA).

"The best SPA is better than the best MPA. The average SPA is worse than the average MPA."
– Nolan Lawson

If it ever feels like people are being too hard on SPAs or any JavaScript-based framework (like React Native), this is due to a phenomenon called the uncanny valley. This comes from robotics professor Masahiro Mori who noticed that people became increasingly positive and empathetic as robots became more human-like until a critical point at which point their responses became strong revulsion.

We see the same phenomenon in our movies. As CGI has become closer and closer to reality, our subconscious can’t help but reject what it is seeing. Considering this essay’s post date, here’s a Halloween-themed video on the topic.

At this point, it's important to point out one ironic misnomer about "single-page" applications. They're usually composed of more than a single page. You can see the many URL routes when navigating from link to link. The "single-page" moniker was meant to describe the page-refresh strategy, not the routing strategy (this distinction will become important later).

Who cares, you ask? As it turns out, this is the very thing that is holding back webapps from being competitive with native apps. It's why people have learned to distrust and avoid the dynamic web.

This series has already covered the many reasons why JavaScript-based apps will never get better so what hope is there for the open web to compete with native apps? The answer lies, not in MPAs or SPAs, but in ZPAs: zero-page applications.

In no way is this meant to imply that URLs were a bad idea. To give credit where it’s due, chopping up the Internet into individual, discrete documents interlinked by URLs is easily one of humanity’s greatest achievements. It’s beautifully simple and incredibly effective. Without this very basic concept, the Internet would simply not be the Internet. It certainly wouldn’t be called “The Web”. It's perfect. Don't change a thing.

URLs are great for documents, but native apps are simply incompatible with URLs. The golden age of native-like webapps will remain on the bench until developers can move beyond a URL-first mindset.

To explain this well, we must cover native apps and their lengthy, contentious past with not just URLs but also its partner in crime, the back button.

Implicit Intents

In 2007 (a year before the iPhone SDK was released) Google released the Android SDK to developers. With it, came a new spin on URLs called Intents. Intents became the foundation for launching anything on Android whether it was UI, a background service, or even broadcasts.

One of Intent’s biggest innovations were implicit Intents which enabled developers to send users to other apps by declaring a user’s intent in a generic, non-prescriptive manner. The user could then be sent to any app installed that had declared to the OS that it was capable of handling such an intent. If multiple apps qualified, the user could either use the disambiguation prompt or pre-configure a set of default apps.

This empowered the user to construct their own custom flows that interlaced a multitude of screens from a multitude of apps. For example, opening a URL could launch their favorite browser where you might then navigate to a “mailto:” link which could then launch your favorite email app where you could add a recipient which might present friends from your favorite contacts app followed by attaching a photo which could present a chooser-screen from your favorite photos app.

This meant Intents needed to support, not just explicit and implicit locations, but also provide a way to return complex data back to the calling app, whether it be a simple string, a strongly-typed value, a complex object with a custom set of fields, or even a resource stream to the file system coupled with the proper set of privileges.

All this extra flexibility came with a great deal of extra complexity. The humble URL was not enough to support it all. Intents made use of Actions, Categories, Extras, Flags, Data URIs, Request Codes, Result Codes, Intent Senders, Pending Intents, and more. In fact, Intents often included multiple URIs as a part of their definition and result-handling. Lastly, handling Intents incorrectly could lead to your app crashing, for example, incorrectly assuming a browser was always installed or running on devices with corporate policies or family safeguards.

Intents proved to be a great next evolution of the URL. Back then, however, native UI was far more simplistic than it is today. Initially there was a pretty clean 1:1 correlation between Intents and their Activities. But when iOS launched, it showed the world a richer way to navigate between screens using navigation controllers which could not be so cleanly defined by web-based routes since the boundaries between each screen was not so discrete and isolated but rather shared and overlapped. Android eventually followed suit by extending their Activity model to include Fragments thus complicating the clean 1:1 correlation Intents initially had with Activities.

Deep links

URL schemes are the section of a URL before the colon “:” e.g. http: or ftp: Android Intents were smart to embrace custom URL schemes as they became very popular on mobile. Desktop consumers rarely saw URL schemes beyond http(s): but on mobile, tel:, smsto:, mailto:, geo: and more were wonderfully simple ways to launch core utility apps. They were simple to understand and they had the added advantage of being easily embeddable in HTML, email, SMS, or even encoded into a QR code, an area where Intents fell short.

Both Android and iOS supported these and even encouraged the use of non-standard schemes like fb://profile/33138223345 or zoommtg://zoom.us/join?confno=...

Adoption of deep links would have spread even further if not for the desktop where launching native apps was, strangely, far less common. Desktop simply lacked many native utilities like Phone, SMS, Maps, etc. What’s more, other categories like social media, didn’t make the same leap from web to native on desktop like it did on mobile.

As convenient as deep links were, they quickly became problematic. As you can imagine, the lack of standardization or regulation led to an explosion of disorganized and poorly documented URL schemes. This also left the door open for some developers to “borrow” the schemes of their competitors. This lack of ownership and control had the unintended consequence of preventing app developers from ever adopting deep links as their app’s primary internal navigation strategy. Instead, usually only a few key routes got mapped while the rest of the internal navigation was handled natively, without consideration for routing patterns.

Web links

Web links are just deep links that use http and https schemes. An app could declare to the OS which routes it was capable of handling using a mix of wildcard- or regex-based routing. This approach aimed to improve the challenge of launching a native app using a more desktop-compatible approach while simultaneously mitigating the app-not-installed issue since it could always just gracefully fallback to using a browser.

While this was an improvement over deep links, it still didn't address the issue of domain ownership and thus required the use of a “disambiguation screen.” This created a poor user experience reminiscent of Vista’s allow/deny fatigue.

App Indexing

Google needed to solve the problem of how to do search with native apps when navigation isn’t always URL-based, there’s no sitemap.xml, and no way for its crawlers to fetch, parse and index the content.

So Google created Firebase App Indexing. Developers could add this SDK which would make their content available in search results but local to the device only. This was great for apps like note-taking apps.

Unfortunately these local-only results were only displayed when searching from Google’s native search UI but not from google.com. Eventually Google killed it, suggesting that developers use Android’s App Links and Apple’s Universal Links instead, which required app developers to mirror their content on the web so Google could crawl it “properly.”

App Links

App Links improved on Web links by addressing the issue of domain ownership. Configuring Intent Filters with autoVerify instructed the OS, upon installing an app, to fetch a specific “Digital Asset Links JSON file” which helped verify ownership through Google’s Search Console.

This negated the need to present a tiresome disambiguation screen for every link click. The OS could simply launch the proper native app without prompting beforehand and without “trampolining” through a browser first. Users could even customize these policies from their system settings.

Granted, this only worked if the user already had the app installed. Where this approach also fell short was in link wrapping. When marketers ran paid ad campaigns or sent emails these tracking URLs did not share the same domain thus negating all the benefits.

Universal Links

Universal Links is Apple’s version of App Links except it used a different JSON file and relied on Apple’s backend services instead of Google’s Search Console.

Universal Links were an improvement over Web links or Deep links but the lack of control hindered deeper adoption for internal navigation. For example, Apple’s CDN would check for the presence of this file within 24 hours but devices would only check for updates once per week after app installation.

There was also the issue of phantom banners. Since Apple generates more revenue from native apps compared to websites they were incentivized to promote the installation of native apps. For any app with Universal Links set up, Apple would use Safari to randomly inject a banner ad above your website and there was no way to control it, customize it, detect or measure it. Like Apple, app developers were also incentivized to direct users towards their native apps since retention is typically higher but without the ability to predict or control Safari, this often led to a barrage of duplicate popups when browsing on mobile.

Dynamic Links

Since retention is usually much higher in native apps, companies naturally sought after a solution to convert their web users into native users as a part of their approach to URLs. Neither iOS or Android ever built support for the use case of how to handle URLs if the app is NOT installed so the private market stepped in to fill this gap with third-party solutions. The most popular of these was ironically Google’s but it did not come from the Android team but rather their Google Cloud and Firebase teams. Fortunately the cloud service was free and unlimited.

This included a highly complicated flow that differed depending on the user’s OS vendor, OS version, whether you typed the URL or tapped a link from inside or outside of a browser and more. Some flows even made use of an intermediate, interstitial screen that would pause the flow to secretly copy the intended URL to the user’s clipboard so that post-installation and on first-launch the app developer could manually navigate the user to the proper deep link. Other features included support for creating short-URLs as deep links to increase shareability across social networks.

It was difficult to set up, difficult to integrate, sometimes required duplicating everything across a secondary domain name, and Apple eventually broke the flow due to security concerns around clipboard-abuse.

In the end, after becoming the de facto standard, Google killed it.

App Clips & Instant Apps

There were parallel efforts to Dynamic Links that did have support at the OS level – Apple’s App Clips and Google’s Instant Apps. These weren’t exactly about converting web users into native users though. Instead they were lightweight duplicates of your native app with strict size requirements (between 5-15 MB) designed to download and launch quickly from either a web search result, QR code, or NFC tag. Their intended use case was for quick, one-time tasks like ordering food or renting a scooter. The goal was to enable native features with the same casual commitment as visiting a website by obscuring the installation step and then uninstalling once the micro-task was completed.

Unfortunately these didn’t see much adoption amongst developers beyond ordering scooters. Hopefully adoption will grow in the near future as companies find better options for cross-platform development that doesn’t bloat apps to hundreds of megabytes, far beyond the limit for these transient applications.

Why such different outcomes?

Non-linear navigation

Incompatibility

Single-page applications

"...we need to reckon with the fact that single page apps have kinda ruined the web."
– Rich Harris, creator of Svelte

Zero-page applications

The goal of web4

Artifacts

Source code: github.com/xui/xui