Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
In the competitive mobile market, application stability is imperative. Downtime leads to poor user experiences, which stunts growth and revenue. Apps that have poor stability suffer the consequences. In fact, new studies show that just a 1% lower stability ranking can lead to nearly a one-star drop in App Store ratings.
Bugsnag recently released its 2021 Application Stability Index (ASI) report, Characteristics of Leading Mobile Apps. The study took 260 top-performing mobile apps across industries and reviewed each’s stability and end-user satisfaction. The study found that apps with solid stability tend to rank much higher in the App Store and maintain increased user engagement. The report also demonstrated that developers are pivoting to weekly releases to quickly address bugs, progressing from the bi-weekly standard for software updates.
Below, I’ll review the specific details of the ASI and the report and consider how stability varies throughout industries. Hopefully, the results here will help other software teams benchmark their app’s performance and release cadence against top-performers within their sector.
Stability = User Satisfaction
When measuring uptime and availability, five nines, or 99.999% uptime, is the golden target for most teams. This report measured stability as the percentage of apps sessions that were crash-free. The median across market-leading apps was 99.80%.
What’s interesting is that apps with more nines in their stability score achieve markedly higher App Store ratings. Apps that scored less than 99.00% had a median rating of 4.15, whereas apps with 99.99% to 100% measured uptime had a median rating of 4.70. The distribution of ratings was extremely wide for low-performing apps, compared to higher average scores for higher stability.
What do these results say? Users expect uninterrupted sessions. Frequently occurring stability issues can damage the consumer’s opinion of an app, leading to dissatisfaction and harsh reviews. The more stable an app is, the better chance it has to drive consumer success.
Stability by Industry
The study also exposed stability differences between industries. B2B SaaS apps ranked relatively high, at 99.85%. This is likely because B2B software must meet service-level agreements (SLAs). Such services have fewer (yet more high-paying) customers to support, the report speculated. Mobile apps directly tied to the exchange of money, such as e-commerce (99.86%), travel and hospitality (99.90%) and financial and banking (99.85%) also ranked consistently high in stability.
Coming in second were mobile apps in the business-to-consumer (B2C) (99.82%), consumer goods (99.80%) and health and wellness (99.78%) categories. Though these areas require decent availability to remain viable, they may not need such stringent SLAs. In such scenarios, non-stop availability isn’t as directly tied to end revenue.
Lastly, apps with the worst stability were found in the mobile games (99.71%) and media and entertainment (99.65%) categories. Mobile gaming had the worst ratings of all industries; mean stability equated to over a full day of outage per year. Increased downtime in both categories is perhaps due to more problems with frequent media updates. These areas also tend to have a lower average revenue per user (ARPU), which may affect how teams value ongoing support.
Deployment Frequencies
According to the report, highly stable mobile apps tend to release at least once per week, replacing the bi-weekly norm for updates. Nearly all industries are pushing at least four releases per month, aside from consumer goods and travel and hospitality, at 2.5 and 4 respectively. E-commerce takes the cake with a mean of seven releases per month.
Nowadays, progressive delivery is made possible due to increased emphasis on CI/CD. Therefore, rapid deployment frequencies are the norm. Releasing new versions more quickly can help appease users with new features. It’s also helpful to quickly patch bugs and decrease mean time to recovery (MTTR) for major incidents.
Users Love Stable Apps
Throughout the pandemic, digital reliance soared—84% of consumers said applications were a lifeline to normality, found a recent AppDynamics report. In this climate, mobile applications are critical to delivering valuable interactions.
Yet, user expectations have evolved. Now, it’s readily apparent that stability directly impacts consumer confidence. The ASI findings behoove engineers to edge closer to the five-nines goal. The more nines you have, the better off you are.
“The results of our second ASI report highlight the need for better proactive error monitoring and stability management to help engineering teams build healthier apps that deliver superior customer experiences,” said James Smith, SVP of the Bugsnag Product Group at SmartBear.
It’s good to reiterate that this study examined a handful of leading apps from each industry. So, these scores shouldn’t be construed as inclusive of the entire app marketplace, only as benchmarks for the top mobile app performers.
