Cross-browser testing - your thoughts?

This section is for announcements, information and discussions relating to the help community -- for example news about events and seminars of interest, developments in help technology and so on.

Moderator: Tim Green

Post Reply
Simon Dismore
Posts: 454
Joined: Thu Nov 16, 2006 1:29 pm
Location: London, UK

Cross-browser testing - your thoughts?

Unread post by Simon Dismore »

Given today's variety of browsers and screen resolutions, I'd be interested to hear what people are using for checking how their applications and webhelp are displayed. Currently I try things out on kit we happen to have to hand, which gives some idea of possible issues, but the coverage is necessarily incomplete. It's time-consuming, particularly if I want to capture screenshots consistently, and it feels amateur. As we move into more responsive design, we'll need more automation.

My first question is "does that ring any bells?" - is this something H&M authors are encountering in their companies?

The second is "what are you doing about it?". I've been looking at BrowserStack, SauceLabs and Browserling. Any experience of using those?

Broader comments about the testing regime in your/your clients' businesses would be welcome too, especially for web and mobile applications. Can anyone here share experience of services like Soasta, Neotys, Radview, Web Performance, etc?

Thanks in advance

Simon
User avatar
Tim Green
Site Admin
Posts: 23154
Joined: Mon Jun 24, 2002 9:11 am
Location: Bruehl, Germany
Contact:

Re: Cross-browser testing - your thoughts?

Unread post by Tim Green »

Hi Simon,

I try browser emulation solutions occasionally but always give up on them because they are all more or less inaccurate. It is the same as the iPhone and Android phone emulators in their respective development environments. They are OK for quick orientation, but before release you are going to have to test in the real browser on real hardware. There is no way around that -- it is one of the cases where anything less than 100% accuracy is pretty much useless.

Desktop browsers are now all pretty good. On Windows you really only need to test on IE, Chrome, Firefox and Opera. None of the other browsers are even worth installing. IE 11 is now generally good enough that it can actually be trusted as a genuine browser. Problems caused by an arrogant disregard of standards, which used to be Microsoft's speciality, are now increasingly cropping up in Chrome rather than IE. The Microsoft team are genuinely focusing on standard compliance and they are doing an increasingly good job.

On older versions of IE: You should probably perform cursory tests in IE 9 and IE 10, but even that really isn't necessary any more. There is no longer any valid version of Windows that cannot install IE11, and anyone with an earlier version of IE having problems should simply update. I wouldn't recommend spending any time on earlier versions of IE, because supporting them simply prolongs their necessary demise. They are no longer supported and no longer safe and should not ever be used for anything.

You also need at least one Mac running the latest version of OS X, with all the main browsers, a Linux machine (ditto), both a retina and a non-retina iPad, an older iPhone and a current iPhone, and at least five Android tablets and phones with different sizes and resolutions -- more is better. Android is the current web and mobile developers nightmare, and it's not going to get much better any time soon.

On Android you need to make sure that you have devices with all the current Android versions installed, which means everything from 4.0 to the current 4.4. If you want to support anything prior to 4.0 you will radically constrain what you can do for the sake of a dwindling audience -- I wouldn't recommend it. The older versions back to 4.0 are important because everything prior to 4.4 has an older and very bad version of WebKit as the main HTML rendering engine for all browsers except Chrome (which always has its own embedded copy of the latest WebKit) and Firefox (which has its own rendering engine, Gecko). It was only with 4.4 that the current WebKit used for Chrome was installed as the main rendering engine that can be used on Android by all other browsers as well, which has radically increased overall compatibility with current web standards. The older version has many restrictions and bugs, including key things like bizarrely and unpredictably inaccurate interpretations of media queries, screen resolution, device pixel ratio and other things.

This is then compounded by the quirks of the (often shoddy) browsers built around WebKit on Android. You have to decide for yourself which ones you want to support. The stock browser on Samsung devices is unfortunately one of the worst of the entire bunch, but you have to test on it because Samsungs are so common -- it's like IE on Windows. I generally restrict myself to the Samsung browser, Chrome, Firefox, Dolphin and Opera. Chrome and Firefox are both excellent, even on older Android devices, because they always have up-to-date rendering engines. Firefox will even display modern web standards on ancient Gingerbread Android devices -- you can almost always recommend it to Android users having trouble with current websites, no matter what device they are using. Dolphin and the Samsung browser depend entirely on Android's version of WebKit and add their own problems. Opera is now based on Chrome, so it's generally pretty much as good as Chrome. The Amazon Kindle Fire range of tablets have their own browser, Silk, but I haven't bothered with that yet. You have to decide for yourself whether it's worth your while to test on that as well.
Regards,
Tim (EC Software Documentation & User Support)

Private support:
Please do not email or PM me with private support requests -- post to the forum directly.
Post Reply