Synchronous remote usability test

When is this method appropriate?

The synchronous remote usability test is very good to implement if the real use context is supposed to be caught. Another application field is if your user-group is very specific and hence it is very difficult to recruit them in a single location. The procedure of the synchronous usability test resembles a classical usability test in laboratory. But there is the difference that the test subject und the study leader can communicate via internet or webcam and microphone.

Possible questions

• How do visitors use your website?
• Where is potential for improvements on your website and in what areas are the users optimally supported?
• Are the categories, buttons and links conform to expectations, semantic correctly and comprehensively named?
• How do users understand the different navigation elements? Are these elements intuitively operated?
• Do the users find their desired information fast?
• Are the interaction sequences of central operating processes (for example search, contact support, order, newsletter subscription, booking, providing of contents) ‘logical’?
• What is the user’s first impression of this website? What associations, emotional effects do occur?
• How do users perceive the structure and offer on the cover page?
• Is there any ‘information overload’ so that the user is less motivated to get in touch with the website?
• Are the contents and services conform to the expectations?
• Do the contents & services as well as their preparation invite to ‘stay’ and to ‘explore’? Or does the user immediately leave the offer after achieving his goal?

And many more …

In what project phase applicable?

• Analysis
  The test of an ‘old’ or already existing offer can take place on the background of a complete re-working to gain knowledge for the conception of the new, planned website.
• Development/conception
  With the aid of a synchronous remote usability test it is precociously possible to get information about usability problems as well as requirements and expectations of the target-group(s) during the development of clickable prototypes. In the foreground is testing of navigation, usability and interaction processes.
• Programming/implementation
  The synchronous remote test excellently functions as a final milestone of the project. Therefore we need an integrated ascertainment of positive and negative experiences referred to the interface and interaction design as well as the navigation and information architecture of the website.
• Live activity/usage
  During the usage of your website the synchronous remote usability test can identify problems and information about the users’ behaviour in relation to your website. In addition, it can detect the status quo of a website before the re-launch to take the made mistakes into account in future.

What is the procedure of this method?

The synchronous remote usability test tries to extend the classical laboratory test with help of the internet and to transport it to the test subjects’ home. This is possible due to the implementation of the test with the aid of an online meeting tool.

With such kind of tools it is possible to observe the subject via a webcam or a VoIP-connection to communicate with them and to follow their display activities. The test leader observes and reproduces what the test person is doing.

Like in laboratories the test subjects have to fulfil concrete tasks and are asked to comment and rate them during the process (so-called thinking-aloud protocols).

The comments of the test subjects are minuted and their camera picture (gesture and mimic) as well as the activities on the display are recorded on a video file so that they can be used for later evaluations.

In addition, mouse-movements and click-activities can be monitored to detect performance values (for instance required time/clicks to solve the task) and to record the chosen search and surf paths of the test person (so-called click stream analysis).

Excursion:

To create a superlative realistic test or laboratory situation we adopt the following measures:

• Recruiting of the test subjects and screening
  Our online-access-panel offers the possibility to chose potential test subjects according to proved procedures. Besides sociodemographic features, interests, opinions and the web usage behaviour can serve as selection criteria. First of all the potential test attendees have a screening within a pre-survey. Thereby project specific and personal features are detected. The eResult GmbH puts emphasis on a serious acquisition of the test attendees. Our quality management guarantees that no static users falsify the results.
• Incentivation
  To chose an incentivation concept for the test attendees we rely on the creation of highly situative involvements. In an online-shop this works e. g. by using “own money”. This means that the test attendees do no only receive cash money for their attendance (about 65 to 70 euros) as usual. Instead of this they receive a corresponding grant or voucher for a product that is offered in the online-shop. In this matter it is guaranteed that the test attendees use the website under realistic conditions!
• Warm-up phase
  This takes place at the beginning of the test session and basically serves to build up a pleasant and intimate relation between the test subject and the interviewer. In addition, possible technical problems should be clarified ex-ante to guarantee a trouble-free test procedure.

Strengths of this method

• The users attend from home or work so that an authentical user behaviour can be captured.
• The choice of the test subjects is not limited to a district. It is very easy to show the behaviour of hard contactable target-groups.
• It is possible to formulate very concrete tasks or usage scenes.
• Due to the direct interaction between the test subject and the test leader one can individually meet the test person. Misunderstandings can be immediately clarified and if necessary certain questions can be thematised more intensively.
• This method provides comprehensive qualitative data that show how real users operate on your website.
• The video recordings of tests or of the test subject while using the website are very informative and convincing.

What about the sample size?

For a usability test we advise to deal with 8 to 10 test subjects. This advice also applies to the synchronous remote usability test. If the rating of design and layout or the preparation, quantity and kind of information is very important, at least 20 test subjects should attend.

Note: One should generally consider whether all test subjects should test the same object or if the website is tested with the half of planned number of test subjects during the first test phase. This group identifies and solves the main problems. In the second test phase the already optimised website can be examined by the second group of test subjects (further considerations hereunto look: usability test in laboratory).

How much time takes the implementation?

Depending on the sample size the study can be implemented within 3 to 4 weeks (N=10 to 15) or 4 to 5 weeks (N=20).

What input do you have to provide?

Optimally you come to our premises with concrete questions that we enquire within a test. We would like to work out with you this kind of questions in a workshop.

We need at least graphical worked out screens in a current graphic format. But to test interaction processes we need a functional prototype (click-dummy) that already has enough functions to implement a task-based user test. At best your website is already online or callable via the internet.

On demand we would support you for the creation of a functional click-dummy.

What do you receive from us?

You receive an interpretative chart report with the following contents:

• Management summary
• Detailed Illustration of the results from observation and survey data.
• Interpretation of these data: identification of the reasons and causes for the ascertained data.
• Highlight video scenes (click stream, interviews).
• Concrete guidance to remedy identified usability and design problems; verbal described and visual-functional implemented (for instance in terms of optimised wireframes or clickable prototypes).

On demand we present the results and findings of the test within a moderated workshop.

Do you have any further questions? Please do not hesitate to contact us! We would like to consult you.