%2520(1).jpeg)
It’s been three years since Uptech reached its cherished milestone – 99% crash-free sessions on Aspiration project! Was it easy? God, no, it was a real challenge for the team. But thanks to dedicated teamwork and expertise, we managed to handle it.
But what is more important – we have managed to keep it on the same level since then. And this milestone turned into a compelling value for Aspiration:
- 40k monthly active users;
- 70% of Aspiration customers use iOS or Android checking account app;
- Aspiration has a 4.9+ rating on App Store
So in this article, we want to share:
- What is a “crash” in mobile app development?
- What does “99% crash-free'' mean and how is it calculated?
- In what steps Uptech achieved the 99% crash-free result?
- What value did this achievement bring to our client?
Let’s dig right in and see how the miracles in development happen! (spoiler: multi-step testing and teamwork).
What Does a “Crash” Mean?
The crash is an exception that happens in a particular environment in an app. Let's break it down at this point.
Let's see code as a black box with input and output parameters. It could be a particular set of input parameters with which the black box gets into an error state, so it throws an exception because it can't perform its logic. For example, you might know that you can't divide a number by zero. This is what the wrong request looks like for the system – dividing by zero.
When a mistake like this occurs in the app's code, the developer should come to the rescue and bravely give an "exception" to the system.
However, if an exception gets away from the developer's eye, the user will see a system dialogue announcing that the app has crashed.
What Does “99% Crash-Free” Mean and Is It Worth It?
99% sounds very impressive, but how do we get this number? And what number of users does it concern?
Firstly, when we talk about the 99% crash-free metric, we include all the app’s versions that have been launched throughout the whole time.
Secondly, we consider active users who use the app regularly. So, the crash-free number is calculated as follows:
Crash-free Rate = 1-(Crashed Users / Active AppUsers)%
However, there is one thing to keep in mind when using this formula. The thing is, all apps use different features at different times. As for Aspiration, let’s assume that 50 % of users actively use Debit Card features, while other 50 % use Credit ones. So when something crashes in the Credit feature, the crash-free rating can still be higher, as only 50% of users (in this example) faced the crash.
So once your app starts scaling, and more than one feature appears (like Aspiration’s Spend&Save, Aspiration Zero, Investment, Plant Your Change) the percentage of crashes will be lower.
Expert’s Insight: The more users the exception covers, the easier it will be to spot it during testing.
How To Achieve a 99% Crash Free Result: Aspiration Case?
Aspiration is an eco-fintech app targeted at the middle class that allows users to save up while helping the environment. On the one hand, this is a classic mobile banking app, but on the other, it has an essential eco-component – it helps users track their ecological footprint and give back to nature by investing in green projects.
Aspiration was one of many projects that has been with us for many years. And the lion’s share of success, in this case, is credited to the project's complex teamwork and well-built workflow.
Here are some of the approaches that helped us in this way:
Separate Feature Testing by Developers
Well, it all starts with developers.
Only a few people know that developers write the code and are the first point of code testing. The developer processes and tests their code against the peculiarities of the Android or iOS systems. We also run unit and instrumented tests. There is a check-out list to be done during this stage:
- Besides testing the code, the developer also tests individual tickets. In this process, not only the code but also the environment around this code is tested, as certain elements could have been affected during the development.
- And if it turns out that there is something to fix in the environment – the developer immediately creates a task for QA engineers, and reanimates the affected element. Once the QA specialist has done their job, the code is added to the whole code base to be later tested during sprint regression testing.
So the key point is that the developer's responsibility lies at this phase.
Uptech’s Best Practice: Thorough documentation in testing is important. At Aspiration, QAs were running the tests through the whole app every sprint. If they caught a bug, it was documented so that the next QA would pay attention to it and test it again. By this trial-and-error method, we created a checklist of cases that every QA had to run every time they tested the app.
Regression Testing by QA Engineers
The next point on the check list is regressive testing. At this stage, QA engineers are ready to carry out regressive testing.
Regressive testing means the specialists go through the app from A to B, testing all the features, and running tests through all the screens.
Uptech’s Best Practice: On the Aspiration project, we use parallel testing in case of new features or for features that have a higher risk of bugs. Thus, two QA engineers run tests for similar features. Think this is a waste of people’s time? Well, let me reassure you of the opposite.
Parallel testing makes sure no human mistake was made. So if the mistake slips the eye of the first QA engineer, the second one will immediately fill in the blank space.
Automation Testing
Automation is another instrument to polish your product from crashes.
Unlike manual, automation tests are run via commands from the Automation QA Specialist. Such commands make it possible to test through the whole app (the job usually done by two manual QAs) automatically via a command to the system.
Sounds compelling, right?
Yes, but in truth, they can also bring problems.
For example, when the development process is too dynamic and facing regular updates, it is impossible to write automation tests, as they should be rewritten too often.
How We Prevent Aspiration from Crashes After the Release?
Sadly, or fortunately, the testing does not stop once the product is on the market. So here are several ways in which we can prevent apps from crashing after the release.
Release app updates with staged rollouts
On Aspiration, we use the method of portion-based release for Android apps. This approach means that we only launched the app partially at a time. Instead, we did it by portions, as in the Android universe there are a large number of diverse devices and thus potentially more crushes.
For example, only 5% of people could use the new version of Aspiration Android app.
Deploying the app by fractions helped us to gather user feedback and prevent any potential crashes for all app users because those 5% who randomly receive the new version first can face a crash.
Could we have launched the app for 100% of users? Yes, but this would have been a risky step, which is only possible when there is a certainty that there are only minor bugs, which only depend on the Android or iOS systems.
Beta Testing
Another way to keep your app safe from crashing after the release is beta-testing, which Google Play Market offers for developers. During beta testing, the app will be available for download for only a part of the users that intentionally agreed to participate in the test. In general, the process looks like this:
- The team develops the app;
- Preparing the release build;
- Launch the internal testing;
- A group of QAs can test the beta version;
- Releasing the beta version for Beta testers only;
- Releasing the app for 100% of users in portions by 25%.
Using Feature Flag
Feature Flag is a programming technique for the back-end that allows users to switch on and off certain functionality in the app without deploying code.
Let’s imagine you’ve released your app, but suddenly, God knows why the screen crashed terribly for 30% of users. In this case, using Feature Flag, you can switch off the crashed feature for the users so that they will not see it until the new build is launched.
Is It Possible to Make an App 100% Crash Free?
While 99% crash-free is an impressive result, the question remains: is it possible to make it to 100%?
The short answer will be “Yes.. but hardly”.
For example, Android is an open-ended system where each device vendor generates their own conditions for the system. Some can change the colors, and others can change how certain things work. As developers, we can hardly predict these specifics and adapt our app to any Android device.
There are cases when the user can use unlicensed updates or hack the app. Users can install specific firmware to replace encryption components to access sensitive data. In this case crashing is a must-have behavior to not allow the user access to the data.
Also, there can be bugs in the libraries used by developers or the Google environment. They are all created by people, and human mistakes are never out of the question.
The same thing works for iOS. Though a smaller one, the risk of unauthorized root access also exists for iOS system. This is called a Jailbroken Device regime – when the user can make modifications to the program, give commands or simply access sensitive data.
So reaching 100% crash-free should always be the purpose. Yet, you can’t control many factors that influence this result.
Drawing Up
On some self-reflection, I would name the following tips that helped us make Aspiration almost crash-free:
- Portion-based approach to the app release;
- Thorough QA process, covering developers, individual testing requests, automation testing and regular complete app testing every one or two weeks;
- Testing one feature by multiple QAs.
This is what worked for Aspiration. However, we choose an individual approach for every project to provide a maximum crash-free level.
Contact our Sales Managers to discuss what will work best for your project.