Sagar Patil

browserstack

Redesigning the Live user experience.

challenge Redesign the Live product experience to feel as natural to the user as their own desktop and mobile devices.

role User and product research, defining success metrics, wireframing, prototyping and front end coding.

BrowserStack is a popular browser testing service that allows users to interactively test websites on thousands of real desktop and mobile devices. By installing virtual machines, OSes and browsers on hundreds of remote machines, BrowerStack solved the biggest challenges to getting started with virtual machines - complicated setup procedures, high cost of buying OS licenses, maintaining device hardware and difficulty managing memory on the users machine. For large companies, maintaining device labs to test across real devices became very expensive, very fast due to the high costs of device procurement, transportation, warehousing and maintenance.

Opportunities

Once a BrowserStack testing session started however, the user experience lacked the speed and simplicity that virtual machines provided. We spoke to users and studied feedback by volume of requests over a 3 month period in order to shortlist the primary problems we needed to solve.

opportunity tickets emotion optimize
Remote browsers were sometimes unusable due to slow network connections. 350 speed
Testing local files was tedious to setup. Lack of HTTPS made it insecure. 195 security
Users were hesitant to use Java and Flash, particularly over a non-secure connection. 156 security
Common user workflows such as keyboard shortcuts, copying and pasting did not work. 133 user experience
Difficult to find help when needed. 69 user experience
Responsive testing with mobile emulators sometimes generated incorrect results. 44 user experience
No support for streaming media. 38 technology
Pricing too high for infrequent usage. 15 pricing
Most requested fixes/changes between March 2014 - June 2014

Goals and Research

We wanted to build a browser testing experience that matched that of a local virtual machine, but also took it up a notch. To do this we would have to move away from Flash and Java to Canvas and WebRTC, design a more intuitive interface and support established user workflows from the old interface.

BrowserStack vs. Virtual Machines

BrowserStack needed to be at least as good as a local virtual machine in order to increase user sign ups. We studied similarities and differences between virtual machines and BrowserStack and learned that we could improve:

Rendering speed
As a cloud based, real-time browser testing service, a slow testing experience has always been the biggest challenge to overcome. It would often take 4-6 seconds to get the results from the remote browser for a single page scroll. The delay was a result of many things - network latency, connection speeds, page sizes, browser and OS quirks, Flash as a technology, and the distance between the user and the location of the remote browser. In order to improve usability, we needed to get session loading times down to 2-3 seconds with a page rendering time of at most 1.5 seconds.

Usability and User Experience
We wanted to ease users into the experience of using a browser within a browser, and make them as comfortable using BrowserStack as they were using a locally installed virtual machine or better yet, holding a mobile device in their hands.

Device availability
By offering over 1000 desktop and mobile browsers, BrowserStack attracted most users by their high availability of browsers. However, loading and displaying the remote browsers often took too long. We had to refine the steps involved with loading a remote browser and developed fallbacks for when the remote browser did not open in fullscreen.

Learning from other cloud-based services

Like Dropbox and Google Drive, BrowserStack also took traditional desktop software and moved it to the cloud. We studied the onboarding experiences of Dropbox and Google Drive to learn about how they modelled their interfaces around traditional desktop software and leveraged the cloud to provide additional features on top. Users felt more familiar with the product and hence required little to no onboarding.

Both, Google Drive and Dropbox added features to address problems of traditional software. Google Drive offered richer formatting options, embeddable content, collaboration tools and the ability to share documents and resume work seamlessly. Dropbox made files available to you whenever and wherever you needed them and subsequently improved security, availability and accessibility.

Similarly, BrowserStack extended the capabilities of traditional virtual machines by moving them to cloud based machines, giving them virtually unlimited storage and processing power. As a result, users now had thousands of browsers and devices available in a matter of seconds.

Comparison between regular OS folders and a Dropbox desktop folder
Comparison between default OS folders and Dropbox folders highlighting the extra features that Dropbox offers

Solutions

rendering speed

Load pages faster with WebRTC and Canvas.

Our analytics told us that 52% users came from Chrome and 24% from Firefox. Consequently, these were also the 2 major browsers that were experimenting with WebRTC at the time. Hence, we decided to abandon Flash and Java for users coming from Chrome and Firefox, moving them to the faster and more stable WebRTC. Average session loading times reduced to 3.5 seconds on desktops and 4-5 seconds on mobile devices. There was a huge boost to page rendering times and users now only experienced a minor delay between 0.8ms to a second while scrolling and interacting with the browser. Safari, Internet Explorer and Opera users would continue to use Flash.

Canvas allowed us to stream media from the remote browsers using HTML5 video. This streaming capability meant users could now test for audio/video support and quality, quality at different connection speeds and debug immersive websites that used rich media content and CSS3 animations.

The new Live experience for mobile and desktop
USABILITY AND USER EXPERIENCE

Going full screen by default

In the old interface, users only tested in a portion of the screen. As a result, users had to scroll horizontally and vertically within the remote browser if they were testing a large screen resolution.

Going fullscreen made the product experience feel richer and more immersive; a true browser within a browser experience. We initially considered opening a new session in a new tab or a pop up window, but quickly settled on filling the user's screen with the remote browser.

Comparison of wireframes of old and new interfaces
If the resolution of the remote browser was smaller than the size of the workspace, the browser would be fully visible. If it did not fit however, scrollbars would appear on the right and bottom of the workspace
Old BrowserStack Interface A browser within a browser
Old interface versus new interface
USABILITY AND USER EXPERIENCE

Moving all actions into a single toolbar

All actions and information about the remote browser was moved into the new dragable toolbar. The toolbar changes based on whether the user is testing desktop or mobile devices. We also added keyboard support for regular users.

Actions in the toolbar
The toolbar is used to manage the testing session. The actions explain what each button on the toolbar does
Selecting an action opens a modal window with more settings and features.
Selecting an action opens a modal window with more settings and features
Toolbar being used in the remote browser
USABILITY AND USER EXPERIENCE / RENDERING SPEED

Test local files and servers with no setup

We built custom Chrome and Firefox extensions that created a secure connection between the user's computer and BrowserStack’s remote browsers. We moved the installation of the extension into the onboarding experience, to explain what it did and how it worked. If users opted-out, they could enable the extension at a later time from the toolbar.

Local URLs can be copied and pasted into the remote browser. Users can continue making changes to their local files and test changes directly in the remote browser
USABILITY AND USER EXPERIENCE

Quick Launch

Most active users usually tested on a common set of 10 desktop and mobile devices. Freelance designers and developers would test fewer while enterprises would test more. However, there was often a 50-60% overlap in the browsers all users tested. We also found that many users were unaware which browsers to test on.

I created a screen which would allow users to launch their frequently tested browser-OS combinations, through the web UI or a browser extension. Quick Launch is a set of popular, customizable browser-OS combinations ready to test on with a single click. For teams using BrowserStack, individual Quick Launch setups could be shared between team members.

Evolution of the Quick Launch interface
Evolution of the Quick Launch interface
USABILITY AND USER EXPERIENCE

Tabular list of browsers

Scalability of the list of browsers was a huge challenge in the old interface. The UI was generated through bulky JavaScript and and the UI lacked aesthetic. To make things worse, the product team did not want to make any immediate changes to the listing and wanted to show all versions of all browsers and devices.

The new interface was a single column for each browser. Limiting each browser to the latest 10 versions decluttered the lists making it easier for users to find their browser. Older versions were available on demand.

Evolution of the list of browsers
Evolution of the list of browsers
usability and user experience

Matching wallpapers to their OS

Wallpapers help us identify an OS. After matching the default OS wallpapers with the selected OS in the browser listing window, I noticed that the background OS wallpaper of the browser listing gave a perceived speed improvement in the overall experience. I quickly prototyped the idea and had it validated with a user group. 70% of the users felt a sense of continuity in the testing experience as the matching wallpapers of the browser list and the remote OS merged. They felt the overall experience was faster.

Old BrowserStack Interface
As the user changes the OS selection, the corresponding background wallpaper is changed
USABILITY AND USER EXPERIENCE

Responsive testing out of the box

By default all desktop browsers open in Responsive mode. When the user resizes his browser, the remote browser is also resized and switches to the responsive layout if available. This allows users to test real time responsiveness of their websites. This feature made the Responsive product redundant.

Results

In March 2015, users were given an option to switch to the new design and three months later, in May 2015, the new interface was enabled for all users. By the second week of June 2015, traffic from Chrome and Firefox increased to 63% and 30% respectively, with the remaining browsers making up approximately 7%. Three months after launching, user registrations almost doubled and sales of freelance, team and enterprise plans increased by 80%, 120% and 200% respectively. Some enterprise clients who signed up after launching the redesigned Live product included BBC, The Royal Bank of Scotland (RBS), Microsoft and Twitter.

Testing real mobile devices on a computer

Customer reactions

Learnings

Onboarding is as important as the product.

With the redesign, features had been added and removed and the user journey had fundamentally changed. We needed to tell our users about the improvements and how to access them to get the most out of their testing experience.

However, to launch the product in time, we did not invest enough time in the onboarding. And while all users, new and old, found the system more stable and much simpler to use, they were still not aware of all the features available and how to optimize their browser testing workflow. For example, many users were unaware that they could test upload and download functionality on their website. They did not know where to find the files.

Design relatable experiences.

By modeling the new user experience around that of a virtual machine, we were able to design an experience that users felt familiar with, even though many of them were first time users. Since users could port over their knowledge of using VMs, they found it easier to use BrowserStack without having to change their existing workflows of testing.

Learn about your audience and design targeted experiences.

In order to move from Flash and Java to Canvas and WebRTC, we needed to ensure that all users would be able to use the new technologies. Traffic data told us that close to 85% of our traffic came from Chrome and Firefox, the two main browsers that supported WebRTC. This gave us the confidence to make the switch and redesign a whole new experience from scratch.

For the remaining 15% of users, we simply updated the UI and the workflow of the old UI/UX, leaving the technology stack the same. A number of the new features were unavailable to these users. We encouraged users of Safari, IE and Opera to make the switch to Chrome and Firefox, and after 3 months, BrowserStack's traffic from Chrome and Firefox rose to 93%.

By studying our analytics, we were able to design two separate experiences for users coming to the same page. This helped us adopt newer technologies, build a better product and offer more features to users.

Acknowledgements

A big thanks to Arun Pattnaik for guiding the user experience after the initial user tests and being a strong motivation for the team. Thanks to Aaron and Jekin for putting up with my endless icon design requests. From engineering, a shout out to Ahmed, Rahul, Utsav, and Ankur.

Case studies