back to: methods & procedures

Online newsletter test -
integrated, qualitative and quantitative!

When is this method appropriate?

With our standardised and multiple successfully used pre-test procedures it is possible to test and optimise online newsletter integrated already before the planned one is mailed. So we can avoid negative effects and predict click rates. The risk of only observing click rates after mailing decreases (in sense of “trial and error” procedures).

Within our standardised test procedure for online newsletter the typical contact phases are simulated with help of online newsletters: from perception of the subject line to click.

Possible questions

By the use of different ascertainment methods – eye-tracking, survey and click-training – answers to the following, exemplary questions can be provided:

What intensity does the call character of the subject line have? … and the upper third of the online newsletter?
Does the newsletter animate to read?
Are there any reception barriers?
Does the newsletter achieve effective eye-tracking?
Does the newsletter convey basic information?
Does the newsletter set up action impulses?
How are the single design elements rated?
Where is need of optimisation according to the test attendees?
How is your online newsletter rated compared to the online newsletters of your competitors?

And many more…

In what project phase applicable?

Development/conception
Already during the development of a concept for your online newsletter it is advisable to test for instance the call character and the design of the newsletter to gain first important findings. Thereby it puts forward to create different concepts and layouts and to test them comparatively. It is also imaginable to implement a comparing test with the newsletters of competitors. Hereby one can learn from others’ strengths and avoid their weaknesses.
Live activity/usage
Of course it is also possible to test the newsletter after the first mailing e. g. to ascertain the status quo, success control, continual quality management or for benchmarking with competitors. In addition, it is possible to use the results of the users’ click-behaviour for the test concept. With help of our test method the causes and reasons for low click rates are identified.

What is the procedure of this method?

Within our standardised test method for online newsletters the typical contact phase with online newsletter is simulated:

Receipt of the newsletters
Effect of the subject line: calling the newsletter with help of a mail program and reading of the subject line.
Calling the newsletters
Aufforderungscharakter des Newsletters: Kurze Call character of the newsletters. short contact situation with the upper third of the online newsletter to clarify the question if there is interest in more information.
Intensive contact situation (if there is interest!)
Rating of the newsletter design, the offered contents & services as well as the measurement of interesting contents (click-behaviour).

Optimally in an experimental examination design alternative design and content layouts are comparatively tested.

People from your target-groups exclusively attend this test (usually subscribers).

The usage of different ascertainment methods – eye-tracking, survey and click-tracking – provides answers for different questions. The test either proceeds ‘online’ by our panel or in our laboratory. By this a combination of qualitative and quantitative procedures and methods of an integrated rating is possible. Furthermore we can present missing contents and services. Besides recommendations it is also possible to point out optimisation approaches of the contentual and functional newsletter design.

Due to a comprehensive benchmarking data base – an integrated rating for more than 100 newsletters from different branches is possible. Furthermore we are able to compare the ratings of your online newsletter with the values of other newsletters. On the base of this comparison (also possible with rating of online newsletters from other branches) valuable findings can be gained for a newsletter optimisation.

Strengths of this method:

Multi dimensional effect analysis.
Test in real contact situations.
Possible combination of online and laboratory tests to compact findings.
Competitor benchmarking is possible!
Low-cost and fast test implementation.
Findings considering positive and negative ratings already before mailing the first newsletter.

Wie groß sollte die Stichprobe sein?

Idealerweise werden in einem experimentellen Untersuchungsdesign alternative Design- und Content-Entwürfe gegeneinander getestet. Jeder Entwurf sollte von mindestens 100 Personen bewertet werden. Für die Tests im Lab – mit einer Messung des Blickverlaufs – sind 30 Testpersonen ausreichend, auch bei einem Vergleich von Design-/ Contententwürfen.

Wie lange dauert die Durchführung?

Der Pretest kann sowohl "online" als auch im Teststudio (Usability-Lab) durchgeführt werden. Dauer: 7 Arbeitstage für den Online-Test, und 2 Wochen für den Test im Lab. Idealerweise werden Online- und Labtest kombiniert und hintereinander durchgeführt (Online-Test vor dem Labtest).

Welchen Input müssen Sie leisten?

Normalerweise werden in einem experimentellen Untersuchungsdesign alternative Design- und Content-Entwürfe gegeneinander getestet. Diese benötigen wir von Ihnen als HTML-Datei oder in einem gängigen Grafikformat.

Gerne unterstützen wir Sie auch bei der Erstellung erster Entwürfe / Konzepte und erarbeiten mit Ihnen gemeinsam konkrete Fragestellungen für den Test. Dabei greifen wir auf unsere umfangreiche Projekterfahrung zurück.

Was erhalten Sie von uns?

Offered services at a glance:

Conception and creation of a questionnaire, or rather, consulting during the creation of the questionnaire.
Recruiting of the survey attendees.
Pogramming of the online questionnaires.
Implementation of the survey & data analysis.
Online reporting: possibility to have a look at the to date ascertained data during the current study.
Interpretative result report with the following contents:
- Management summary
- Comprehensive illustration of the results due to survey data (tabular/graphical preparation).
- Data interpretation: identification of reasons and causes for the ascertained data.
- Concrete guidance and optimisation recommendations to remedy identified usability and design problems for your online newsletter; verbal described and visual-functional implemented (for instance in terms of optimised wireframes).
- Animations and recommendations for an optimisation of the contentual and functional design of the newsletter