back to: methods & procedures

Competition analysis/benchmarking

When is this method appropriate?

On the base of competition analyses or benchmarking offers of important competitors user value, design, identification and joy of use can be compared with regard to usability. On the one hand goal of the competition analysis is to determine the position of the own product in relation to the competition. On the other hand best practice examples can be derivated from the offers and measure the competitors. In addition, the method serves to create competitive advantages by idea generation due to focus on direct competitors. Hereby different qualitative and quantitative methods are separately employed and additionally used in combination.

Possible questions

Where is potential for improvement on your website and where did your competitors solve problems in a better way?
Where are your users optimally supported?
What do other websites offer from the same branch? How relevant are these offers from your target-group’s point of view?
Where does your website stand out from the environment?
Where do the visitors/users see the USP (unique selling proposition) of your website?
What competitors already provide a particular function and how user-friendly is this function implemented?
How is your website rated in relation to the important dimensions like usability, joy of use and visual design compared to your competitors?

And many more…

In what project phase applicable?

Analysis
During the re-development of an offer that has already been established by competitors an expert review or a user-based benchmarking by the help of an asynchronous remote usability tests or a task-based online survey is advisable.
Programming/implementation
During the implementation there is time to go on testing. Hereby last changes can be rated by the users. A direct website comparison is most suitable within a usability test in laboratory (so-called comparative usability tests). This means that not only yours but also your competitors’ website is carefully examined in laboratory.
Live activity/usage
At already existing websites an asynchronous remote usability test or a task-based online survey is the most suitable procedure to detect the status quo compared to the competition.

What is the procedure of this method?

(Description of the different approaches or methods)

Expert review:
Usability experts examine your website as well as the websites of your competitors based on current usability categories (look: expert review).
Usability test in laboratory:
Test subjects solve identical tasks on all websites that have to be tested. The test subjects are observed while they solve the tasks. Problems are reflected within a qualitative post-interview. Finally, a direct comparison or a rating of the offers (look usability test in laboratory)takes place.
Asynchronous remote usability test:
A large number of users uses the website in familiar surroundings. Thereby they describe critical (positive or negative) occasions. Finally, a rating takes place on the base of standardised statements or scales. So a direct comparison of the website is guaranteed (look: asynchronous remote usability test).
Task-based online survey:
Within a standardised online survey a particularly comprehensive, differentiated rating of the application or the website is made (look: task-based online survey) Thereby we draw on the members of an online-access-panel..

Strengths of this method:

Enables a comprehensive market view.
Derivation of best practices for the own website.
Identification of unique features (USP).
Offers a high potential of inspiration.

What about the sample size?

Expert review:
Expert reviews are implemented without test subjects and exclusively base on the rating of three usability experts who have gained long term experience in their branches.
Usability test in laboratory:
Depending on the number of offers which are supposed to be examined the sample size varies. Each offer should be examined by at least 8 to 10 test subjects. Each subject should be presented maximal 2 to 3 offers. As far as more offers are examined a rotated master is advisable whereby the total number of test subjects accordingly increases.
If the rating of design and layout or the preparation, number and kind of information is very important, at least 20 test subjects should attend.
Asynchronous remote usability test:
The sample size should be individually discussed together with your questions. With the asynchronous remote usability test it is possible to illustrate a sample up to 250 test subjects.
Task-based online survey:
To gain reliable results for sub-groups a possibly large sample should be interviewed. To measure significant differences between sub-groups each sub-group should have about 250 test subjects.

How much time takes the implementation?

Expert review:
Depending on the number and complexity of the test objects first results can be provided after 3 to 5 workdays.
Usability test in laboratory:
Depending on sample size the study can be implemented within 3 to 4 weeks (N=10 to 15) or 4 to 5 weeks (N=20).
Asynchronous remote usability test:
In average the implementation of such kind of study takes 3 to 4 weeks (ascertainment and evaluation).
If the test is implemented by drawing on our online-access-panel bonopolis.de, already after 2 weeks data and recommendations are provided.
Task-based online survey:
Due to the recruiting by our own online-access-panel within a few days large samples can be realised or invited. Altogether such kind of projects takes about 3 to 4 weeks.

What input do you have to provide?

Optimally you come to our premises with concrete questions and/or site areas which we can carefully examine and thematise during the evaluation. We would like to work out with you this kind of questions.
Generally is regarded: your product or website should have the same detail level in relation to form, content and functionality like the website of the competitor. Otherwise these have to be modified to avoiding falsifying due to formal differences. It does not matter if there are screens (graphical worked out graphics), wireframes (schematic illustrations of websites) or functional click-dummies.

What do you receive from us?

The preparation of the results takes place in dependence on the chosen or the combined methods. Generally, you receive a comprehensive description of the strengths and weaknesses of your website and the websites of your competitors as well as recommendations how to optimise your website or application. The interpretative result report contains best practice examples as well as an estimation of the client’s position compared to the competition.

On demand we present the results and findings of the test within a moderated workshop. In addition, it is possible to have a look at the to data ascertained data during the current study.

Do you have any further questions? Please do not hesitate to contact us! We would like to consult you.