BrowserStack is a popular browser testing service that allows users to interactively test websites on thousands of real desktop and mobile devices. By installing virtual machines, OSes and browsers on hundreds of remote machines, BrowerStack solved the biggest challenges to getting started with virtual machines - complicated setup procedures, high cost of buying OS licenses, maintaining device hardware and difficulty managing memory on the users machine. For large companies, maintaining device labs to test across real devices became very expensive, very fast due to the high costs of device procurement, transportation, warehousing and maintenance.

Opportunities

Once a BrowserStack testing session started however, the user experience lacked the speed and simplicity that virtual machines provided. We spoke to users and studied feedback by volume of requests over a 3 month period in order to shortlist the primary problems we needed to solve.

Goals and Research

We wanted to build a browser testing experience that matched that of a local virtual machine, but also took it up a notch. To do this we would have to move away from Flash and Java to Canvas and WebRTC, design a more intuitive interface and support established user workflows from the old interface.

BrowserStack vs. Virtual Machines

BrowserStack needed to be at least as good as a local virtual machine in order to increase user sign ups. We studied similarities and differences between virtual machines and BrowserStack and learned that we could improve:

Rendering speed
As a cloud based, real-time browser testing service, a slow testing experience has always been the biggest challenge to overcome. It would often take 4-6 seconds to get the results from the remote browser for a single page scroll. The delay was a result of many things - network latency, connection speeds, page sizes, browser and OS quirks, Flash as a technology, and the distance between the user and the location of the remote browser. In order to improve usability, we needed to get session loading times down to 2-3 seconds with a page rendering time of at most 1.5 seconds.

Usability and User Experience
We wanted to ease users into the experience of using a browser within a browser, and make them as comfortable using BrowserStack as they were using a locally installed virtual machine or better yet, holding a mobile device in their hands.

Device availability
By offering over 1000 desktop and mobile browsers, BrowserStack attracted most users by their high availability of browsers. However, loading and displaying the remote browsers often took too long. We had to refine the steps involved with loading a remote browser and developed fallbacks for when the remote browser did not open in fullscreen.

Learning from other cloud-based services

Like Dropbox and Google Drive, BrowserStack also took traditional desktop software and moved it to the cloud. We studied the onboarding experiences of Dropbox and Google Drive to learn about how they modelled their interfaces around traditional desktop software and leveraged the cloud to provide additional features on top. Users felt more familiar with the product and hence required little to no onboarding.

Both, Google Drive and Dropbox added features to address problems of traditional software. Google Drive offered richer formatting options, embeddable content, collaboration tools and the ability to share documents and resume work seamlessly. Dropbox made files available to you whenever and wherever you needed them and subsequently improved security, availability and accessibility.

Similarly, BrowserStack extended the capabilities of traditional virtual machines by moving them to cloud based machines, giving them virtually unlimited storage and processing power. As a result, users now had thousands of browsers and devices available in a matter of seconds.

Results

In March 2015, users were given an option to switch to the new design and three months later, in May 2015, the new interface was enabled for all users. By the second week of June 2015, traffic from Chrome and Firefox increased to 63% and 30% respectively, with the remaining browsers making up approximately 7%. Three months after launching, user registrations almost doubled and sales of freelance, team and enterprise plans increased by 80%, 120% and 200% respectively. Some enterprise clients who signed up after launching the redesigned Live product included BBC, The Royal Bank of Scotland (RBS), Microsoft and Twitter.

Learnings

Onboarding is as important as the product.

With the redesign, features had been added and removed and the user journey had fundamentally changed. We needed to tell our users about the improvements and how to access them to get the most out of their testing experience.

However, to launch the product in time, we did not invest enough time in the onboarding. And while all users, new and old, found the system more stable and much simpler to use, they were still not aware of all the features available and how to optimize their browser testing workflow. For example, many users were unaware that they could test upload and download functionality on their website. They did not know where to find the files.

Design relatable experiences.

By modeling the new user experience around that of a virtual machine, we were able to design an experience that users felt familiar with, even though many of them were first time users. Since users could port over their knowledge of using VMs, they found it easier to use BrowserStack without having to change their existing workflows of testing.

Learn about your audience and design targeted experiences.

In order to move from Flash and Java to Canvas and WebRTC, we needed to ensure that all users would be able to use the new technologies. Traffic data told us that close to 85% of our traffic came from Chrome and Firefox, the two main browsers that supported WebRTC. This gave us the confidence to make the switch and redesign a whole new experience from scratch.

For the remaining 15% of users, we simply updated the UI and the workflow of the old UI/UX, leaving the technology stack the same. A number of the new features were unavailable to these users. We encouraged users of Safari, IE and Opera to make the switch to Chrome and Firefox, and after 3 months, BrowserStack's traffic from Chrome and Firefox rose to 93%.

By studying our analytics, we were able to design two separate experiences for users coming to the same page. This helped us adopt newer technologies, build a better product and offer more features to users.

Acknowledgements

A big thanks to Arun Pattnaik for guiding the user experience after the initial user tests and being a strong motivation for the team. Thanks to Aaron and Jekin for putting up with my endless icon design requests. From engineering, a shout out to Ahmed, Rahul, Utsav, and Ankur.