6 Tips for Website Pre Launch Testing

You've spent months (if not years) building that gorgeous new website. You are thisclose to launching it in all its glory, and you just. can't. wait.

But you should.

As the recent Affordable Care Act (ACA) website's technical snags have shown, thorough pre-launch testing is critical to the success of your project. But because you have been mired in the minutiae of the site for so long, it is easy to overlook even the most obvious tests.

I recently launched a Drupal-based website, and thought I'd share some of the testing protocol we followed:

Design Testing

Now, for this one I don't mean the design testing that happens earlier in the process, when you are getting feedback on the usability of your design (Paul Boag has loads of great stuff to say about this). I am talking about the review you undertake after you've done all your usability and design testing and made the necessary adjustments based upon the results. Among other things, it is a final review to ensure conformity with your design decisions.

Look at the main areas of your site to ensure they conform to your brand identity. Do your colors match your brand palette? Are accent colors and iconography used consistently across the site? Do all logos used meet guidelines for placement?  Have you secured the necessary usage rights for your imagery?

Content Testing

Your website should have a style guide. (It does, right?) Something that identifies style sheets for main pages. On our site, for example, we have attorney biographies. The content style guide for these pages includes guidelines for name use ("Use full name once, then refer to attorney by first name thereafter"); introductory paragraphs ("Single declaratory statement that calls out legal focus area"); and a taxonomy for practice naming.

On this topic, I strongly recommend rethinking how you write for the web. It is not just a digital version of your written materials, and if you keep treating web as just an extension of print you are going to lose your audience to someone who recognizes and adjusts to this fact. There are loads of great articles out there on writing for the web. Jakob Nielsen & co have a few here.

Content testing is also about:

• Content that shouldn't be there. Are your empty tabs rolling up the way they are supposed to? 
• Formatting: are Latin terms such as cum laude italicized?

You might also divide your content testing into static content vs dynamic content. If you are rolling out a brand new content management system, dynamic content needs an extra-keen eye. Transferring a database of thousands of records will always turn up weird little glitches, so you should set aside time for deeper testing. Run multiple searches on your dynamic content; are the correct records being returned? 

Functionality Testing

The objective of functionality testing is to verify the functionality of interactive elements and
hyperlinks. This area is probably the most time intensive. Every page needs to be reviewed for link testing and functionality testing. Some can be tested once and fixed globally. For example, if your logo doesn't take the user back to the home page, that is a global fix that doesn't need to be tested on every single page. Other testing, such as process testing, will require you test many pages. Process testing includes things like your calls to action. They typically involve a multi-step flow made up of if/then logic.

Other considerations of functionality testing:

• Should your links open up in a new window?
• If you use Contact Us forms, does the form get directed to the right person(s)?
• 404 error pages - what will happen should a user stumble upon a non-existent page?  Here are BusinessInsider's 20 Best 404 Error Pages of all time; why not get creative with yours?

Performance Testing

How quickly your website loads, and how it behaves throughout the visitor's stay on your site, is what performance is all about. Typical performance tests are load tests (how many concurrent users can your website handle?), stress tests, and endurance tests. I'm not going to pretend to be well-versed in what it takes to optimize performance of a website- that's why I hired a fantastic agency to worry about this stuff for me -- and here is an article they wrote about testing CSS performance, and an even geekier write up on load testing is found in this Load Impact Blog post.
 
What I did worry about was general site performance. How long will it take my pages to load? Here's a handy tool I found:

• Pingdom tests the load time of the page, and analyzes it to find bottlenecks (no surprise - imagery took up 58% of our home page load time).

 Browser Testing

I'm going to assume you've been tracking your browser visitors using your Google Webmaster Tools, and you know how users are arriving at your site. I also encourage you to be cognizant of what the environment is that your organization uses. Let's say, according to your analytics,  your site visitors overwhelmingly use Safari and Chrome, but your own organization (the folks who will visit the site on launch day and provide you with your first bits of feedback) all use IE7. Well, then I encourage you to be aware of what the site will look like (and how it will perform) on IE7. It matters.

I found browser testing to be the beast of all the testing. You can never - and I mean never - assume that because something works in Chrome (or Firefox, or IE, etc) it will work in Safari (or Firefox, or IE, etc). The  statement of work that you signed with your web design agency should have defined what browsers the site would be designed to work in. Even so, there may be elements of the design that your site will have issues with depending upon the browser. For example, our site was designed to work in IE, Safari, Chrome, and Firefox. And it does. But the parallax on our home page doesn't work in IE8. Should we have designed the site differently because of this? No; but we needed to be aware of what these issues were before we launched the site.

So we tested in all the aforementioned browsers, and fixed the issues we encountered. Areas to note:

• PDFs may open differently depending on your browser, so be sure and test your PDF content
• Print this page - we encountered similar issues as with PDF downloads; each browser rendered somewhat differently
• Image alignment - there were tweaks needed with imagery on different browsers

Device Testing

What devices will your users be on when they attempt to access your website? You can start by looking at your Google Analytics for mobile % (I'd advise you look at trended numbers, because the % when we began the website redesign were much different than when we finished), but that won't tell you everything. Anyway, we all know we are screaming toward mobile, so what you really need to know is: what will my site look like on an iPhone? iPad? Galaxy? Kindle Fire? etc.

QuirkTools has a nifty little (free) tester called Screenfly. Just enter your url and it will show you
what your site looks like on a desktop (through 24" screen); tablet (Kindle, Galaxy, Nexus, iPad mini); mobile (Razr, Blackberry, iPhone, Optimus, Galaxy); custom screen size (you enter px, e.g. 1024x600); even television (480p, 720p, 1080p).

Your launch day should be a festive occasion. If you've done it right, it will be a day of exhausted smiles, shared congrats, and general revelry. BUT...even if you've tested the heck out of your site, trust me when I say you will be finding little issues for days after your launch. This is to be expected, and is totally manageable. Good luck!

Preparing for a Website Redesign

Guest blogger: Paul Boag, founder of UK web design agency Headscape. 

---

Hello all, Cyndy suggested that it might be useful if I shared a bit about the process we are going through to redesign her firm’s website.

My intention is to share a little about each stage of the process from initial requirement gathering to final build. It is still to be decided exactly how much my agency will be involved. However I will share my thoughts for as long as we are.

I will try and keep my comments as generic as possible so that they can be applied either to a website redesign or indeed any other web project (such as an intranet).

Let’s begin with the first stage we carried out for Cyndy. This was a review of their current online presence.

Stepping back and taking stock

In my experience many organisations rushing to major redesign projects without having a clear idea of where they are going or even what is wrong with their current site. We have found that this inevitably leads to scope creep, internal politics and finger-pointing further down the line. That is why we favour a requirements gathering phase at the beginning of projects.

Broadly speaking this falls into 2 phases: a review of the organisation’s current online presence and a discussion with internal stakeholders to establish aims and objectives.

In this post I would like to focus on the first part: the review of the current online presence.

A typical review falls into 4 stages. These are:

• An expert review .
• A heuristic review.
• Competitor analysis.
• An analytics review.

Although we carried out all 4 in our work with Cyndy, not all are necessary for every project. For example, it is not possible to do a competitor review when working on an intranet.

That said, let’s look at each of these stages in more detail, starting with the expert review.

An expert review

Typically it falls to me and my 16 years of experience working with the web to write expert reviews. They normally consist of spending a couple of days trawling the website until I know it back to front. As I work through the site I identify various issues. Many are obvious such as poor navigation or overly verbose copy. However, others can be much more subtle such as no clear calls to action or inconsistent labelling.

Once I have reviewed the site in detail I translate my findings into a report. This document does not just identify flaws it also suggests possible solutions. The document is designed to be circulated to internal stakeholders and so contains a large degree of education about web design best practice.

An example of an expert review.

The exact content of the expert review will vary. However, typically it include sections on accessibility, usability, design, content, social media etc. It also tends to focus heavily on business objectives, calls for action and how return on investment is going to be measured.

In many ways the expert review is similar to a heuristic review with the exception that it doesn’t just observe, it also makes suggestions.

A heuristic review

A heuristic review uses a standard set of criteria to measure the success or otherwise of a website. As with the expert review these cover areas such as usability, accessibility, design, content and more.

The website is measured against the criteria on a 1 to 3 rating with 1 being poor and 3 being high.

This review provides a more objective analysis of the website than an expert review because the reviewer is using a consistent set of criteria and rating to measure the effectiveness or otherwise of the website. These numerical results also enable us to provide clear visual representations of the strengths and weaknesses of the site. This enables you to see at a glance which areas require additional work.

An example of a heuristic review where the site suffers from an obvious weakness in one area.

Another advantage of heuristic reviews is that because they use a consistent set of criteria it is easy to compare one website with another. This can be useful when comparing your site to the competition. However heuristic reviews are time-consuming and so a competitor analysis may often be more appropriate.

A competitor analysis

Depending on the number of competitors an organisation has, a competitive analysis can manifest itself in a number of ways. When there is only one or 2 major competitors in may be appropriate to do a heuristic style review. However, if there are numerous competitors a stripped down version of an expert review is probably more useful.

In this scenario a web design consultant spends a few days looking at the competitors’ websites and identifying their strengths and weaknesses. Where the competitors does something well we learn and improve upon it. Where mistakes have been made, these can be avoided in our own development project.

In certain situations it can also be beneficial to carry out usability testing on the competitors websites. These sites act as a prototype for your own development project and help identify usability issues that can be avoided on your own website.

It is important to stress however that looking solely to the competition for inspiration is a mistake. If you do not look outside of your sector for examples of best practice you are at best going to be following the competition. To truly innovate you need to look further afield for inspiration.

As can be seen from the Higher Education websites above, if you only look to your own sector for inspiration all of the sites quickly begin to look very similar.

An analytics review

The final part of the review process is an analytics review. This requires website analytics (such as Google analytics) to be installed on the existing site. In most cases organisations already have analytics installed, although they are notoriously bad at monitoring them.

Analytics reviews can give a great insight into your users and what they want to achieve.

Analytics are incredibly important in any web project. Without them it is impossible to judge whether the web project generates a return on investment. Existing analytics are necessary to provide a baseline against which the redesigned site can be compared. However an analysis of the existing analytics can act as more than a baseline, it also provides a real insight into the behaviour of users.

The exact details of the information available will vary depending on how the analytics are set up. However, using techniques such as advanced segmentation it is possible to tell how various users behave. For example you can ascertain whether users who have viewed staff biographies are more likely to contact your organisation than a user who has only looked at a practice page.

This type of information is obviously invaluable in designing any future website. For example, if you know users are more likely to contact you if they have read an attorney’s bio then the website can be designed to funnel users to these pages.

Is it worth it?

You may be wondering whether all of this research is entirely necessary before beginning to even discuss business objectives, let alone build the website. This is a fair question and the honest answer is that it is not always necessary to complete all of these steps. However, at the very least this kind of research will inform a major redesign project. It also has the potential to save a project hundreds of thousands of dollars by revealing that what was originally envisaged is not actually required. Nothing is more dangerous than going down the line of thought which results in a website which does not meet users needs or fulfill the organisation’s objectives.

Hopefully these thoughts have proved useful and will help when approaching your next project. Next time I will outline how to take this research and combined it with stakeholder interviews to create an RFP which you can take to various suppliers.

---

Note from Cyndy: I appreciate Paul's generosity in contributing such a lengthy post to the blog. For more from Paul, you can follow him on Twitter, @boagworld.