Open-Source: Advice on the Importance of Cross-Device Testing for Android Apps

Many developers look at cross-device testing as a boring task, so they put it off until the end. That’s exactly what I did when I started creating Omni Notes, a top-rated open-source app. Keep reading to see what happened, and you just may change your mind…

Developers love writing code; it’s what drives us, pushes us to create. But often that leads us to neglect important duties, like testing. However, it doesn’t matter how fast, how functional, how beautiful your app is if it doesn’t work for a huge portion of your users. I learned this lesson firsthand when developing Omni Notes, a top-rated open-source note taking app. 

The Advantages (and Disadvantages) of Android

I love open-source: the community, the knowledge sharing. It creates a level playing field for developers, and there are few places where the benefits of open-source are more evident than in Android nowadays. With almost 82 percent of the mobile market share, it gives every developer the opportunity to have their work available to the vast majority of the world. But there are also added complications when building open-source Android apps. 

Fragmentation—the numerous versions of operating systems or their variations by manufacturers, the mindboggling number of different devices, the skins of the devices themselves—means when you create a new app, you have to pay very close attention to how your code will look and perform on thousands of different devices and OSs. That’s where testing becomes critical.

The Hard Lesson of the Importance of Testing

Like many developers, I work on my personal projects outside of normal business hours—at night, on the weekends, whenever I can squeeze in an hour or two. Unlike my day job, where work is divided between teams—backend, frontend, UX, UI, or different roles on an agile team—when you’re a sole developer, you’re not just the developer. You’re the developer and the designer and the tester. But often we prioritize our core function over the rest, and we put off testing till the end.

That’s exactly what I did when I released Omni Notes. I focused on the logic and the code, but from a user’s perspective, your app is all interface and if something goes bad with that your great coding loses its greatness! So, when I woke up a day after launch, a friend informed me that my app had been reviewed on XDA, which caused an explosion of new users—more than 1,500 in one day! All these users expected my app to work perfectly on their devices and OSs, but since I had put off testing, I was inundated with messages and comments. All I could say was, “It worked for me. Why not for you?” Not the answer they were looking for.

This was a big problem, because if a user downloads your app, they expect that everything will work like it’s supposed to, no matter what device they happen to have. This is especially important with productivity apps, because if the app crashes and their information is lost, their time is lost. Then, the app is removed, and rightly so.

Choosing a Cross-Device Testing Solution

After that experience, I started to work on both unit/integrations testing and BDD with Robotium, but although this was enough to ensure my code would work on Google's Android framework (Nexus devices at the time), the problem was still there: I knew I needed to test Omni Notes on as many Android devices and OSs as possible. However, with all the available phones and tablets and operating systems on the market, there was no way I’d be able to do this on my own. I couldn’t just call 2,000 of my friends and borrow their phones for a day. So, I started looking for a cross-device testing provider that could help. That’s when I discovered TestObject, now a part of Sauce Labs.  

As a sole developer, price was a major concern. So, I contacted TestObject and told them that I was working on an open-sourced project all on my own and asked if they could help me. I couldn’t believe their response. Being a big proponent of open-source, they valued my work and gave me an amazing deal, an opportunity that without I wouldn’t have been able to make Omni Notes what it is today. 

TestObject gave me the ability to test on real devices, so I could see how my app would function on a really big quantity of possible devices and Android OS variants. This meant that I could ensure every user had a fully functional version of my app. Now, everyone can experience the features I love best about Omni Notes, like the ability to upload any attachment—images, sketches, PDF, source code, anything—which is one of the biggest differentiators of Omni Notes from Google Keep.

Then, when TestObject was acquired by Sauce Labs, I was a little concerned that my cost would increase and some of the features I had grown to love would disappear. To my surprise, and delight, Sauce Labs converted my minutes from TestObject into an all-inclusive subscription to their developer tools. It was incredible!

From that moment on, I was able to improve all of my automatic testing. With an all-inclusive subscription, I no longer had to worry about running too many tests and using up all my minutes. Plus, with the addition of continuous integration tools to improve my build process as well as the ability to upload third-party software dependencies, I could do even more than before.

Sauce Labs and TestObject are truly champions of open-source, and I can say that without a doubt, my app wouldn’t be the success it is today and I wouldn't have learnt so much about testing without their support. I have been able to actively contribute to the open-source community, and now have over 1,700 five-star reviews on the Google Play store and an average rating of 4.4—and even more importantly, I’m helping thousands of people all over the world be more productive through my app every day. 

All thanks to cross-device testing and Sauce Labs. Test, test, test!