The module – in brief

Early Usability Testing and Prototyping aims to quickly discard futile ideas and validate others. The starting point of this module is to test the quality of the various elements in the business model, which is done via a continuous feedback loop and control of assumptions.

The module presents a step-by-step guide to Hypothesis Cards, Interviews and Think-aloud protocol. These test methods can, together with several others, help to validate, further develop and qualify a new digital business model so that it becomes ripe for implementation. 

Testing the business model

This module describes how to test the prepared business models. All the nine elements of a business model can be tested, but especially the three overall value issues reflected in the Model for digital value creation are important to test:

• Value proposition: Do our concrete solution and our sales message have real value for the customer groups or user groups for which they are designed?
• Value creation and delivery: Can the company handle the task of creating, marketing and delivering the digital service or business model?
• Value capture: Can the company make money on it and thus achieve the desired value? When is it likely that this is achieved?

Testing of the business model and the solutions, in the form of digital products and services, will be presented in two modules:

• This module, Early Usability Testing and Prototyping, focuses on the customer side of the business model, including the customer’s experience of the specific solution, of the digital products and services in the solution, of the sales message and the business model – and thus on what the customer is willing to pay.
• The second module on testing, Internal Validation and Testing, is about testing the business model internally within the organization. Thus, this module focuses on the issue of value creation and value delivery, i.e., can your company handle the task of creating the solution and applying the business model?

Therefore, this module on Early usability testing and prototyping will generally consider how to test the hypotheses and scenarios regarding a new business model. This will happen in relation to the customer group’s interests, needs and willingness to enter into the value transaction that the business model represents. For the purpose, this module will present the following tools:

• Hypothesis Cards
• Interviews
• Think-aloud protocol 

We suggest that testing activities takes point of departure in the customers’ experience of the specific solution and its pricing. this is a valuable basis to work from with further small tests of the various sub-elements of the business model. Remember that even if individual elements do not immediately work well in the business model, you will often be able to adapt them via iterations of testing and learning so that you still end up with a viable business model.

Systematic testing

One of the guiding principles of all modules is that it is recommended to work agile and continuously test your assumptions and solutions. This also applies when testing the various elements of the business model. Usually, the business model draft will often contain many assumptions, and it is a good idea to make room for new ideas, even if they are based on assumptions. The most important thing is just that ideas are tested, preferably systematically, so that all results become potential input to a new development process. Thus, both positive and negative test results have value in the form of knowledge.

The logic of this strong focus on testing is that futile solutions should be quickly rejected. At the same time, the knowledge you gather in the process can be a source of great inspiration that can form the basis for new ideas.

The Development Loop for business models, digital products or services is illustrated below.

“Development loop” adopted from Digital Business Models for the Future under license CC BY-SA 4.0

As the Development Loop above illustrates, it is important to test systematically. The system implies that you get answers to your questions and that you can use this knowledge to find new ways and better solutions. This is an important source for learning and continuously adapting your business model and digital products or services.

Hypothesis Cards

When working with the development of a new business model, it is likely that a number of assumptions and hypotheses emerge. This is quite natural and an important aspect of being able to work freely with new ideas because it makes room for more varied inventions. Here, it is important that you formulate hypotheses for all non-documented assumptions so that these can be tested as part of the development of the business model.

Hypothesis Cards are a possible tool for you to use. The good thing about the card is that you can continuously collect assumptions and questions and then work further on them with support from the Hypothesis Card.

An obvious example could be in connection with the dialogue about Designing a New Digital Business Model or Insight into Current Business Model. If you are in doubt about an assumption, or if some of the participants have different views on reality, the obvious thing to do is to make a Hypothesis Card and let the assumption be tested as part of the refinement of the business model. Thus, a Hypothesis Card help to make it visible which questions remain unanswered and to clarify what reality looks like. Please see the Hypothesis Card tool below.

“Hypothesis Card” adopted from Digital Business Models for the Future under license CC BY-SA 4.0

Hypothesis Cards
Print-friendly version of the tool in large format.

Practical information

Set aside 5-20 minutes to complete the Hypothesis Card (one hypothesis per card).

Gather 3-7 people from the company to review which questions and assumptions are still unanswered in the business model. It will be a great advantage to let employees with different professional backgrounds participate in this process, as this will bring more assumptions into play, as well as support the discussions of how the hypotheses can be tested.

You need a stack of printed Hypothesis Cards and a pen for each participant. It is highly recommended to carry out the analysis on paper rather than on a computer, as it is much more flexible and encourages interaction to a greater extent.

Step-by-step guide

1) Start the meeting with allowing a little time for everyone to brainstorm about what unanswered questions and assumptions you see. Everyone should have a stack of hypothesis cards, and they can then write the assumption at the top of the card in the field “hypothesis name”.

2) After this, participants can freely discuss which assumptions should be clarified. If several participants have made cards containing the same assumption, they can be gathered on one card. The date on which you filled in the card should be added to the card.

3) Now, the hypothesis can be formulated. It must be clear, unambiguous and in a way where the core of the original assumption is clear to all.

4) Next, it must be explained what will prove the hypothesis. The same field should also describe how, i.e. a specific test by which this can be confirmed or refuted.

5) Depending on what is to be tested and how, the group should choose the most suitable person(s) to carry out the test. The name(s) are then listed at the bottom of the Hypothesis Card. The hypotheses will then be tested with the chosen test method. This is a prerequisite for being able to fill in the remaining elements of the Hypothesis Card.

6) The result of the test is noted in the last field as a conclusion as to whether the hypothesis was true or false. You should also note what you learned from the test; for example if the test was negative but indications were that something else could be a possible solution. Thus, these learnings become an important part of the further work in the company.

Output

The completed Hypothesis Card now sheds light on the actual test results of the assumption you needed to test. The Hypothesis Card can thus be presented at a meeting or workshop where work is done on the new business model. Here, you can discuss what you learnt, and you can either agree that the relevant element in the business model looks sensible as it is, or that it must be adjusted based on the learnings from the test.

Early validation and pretotyping

Pretotyping is the first and earliest test of an idea. The concept of pretotyping was invented by Alberto Savoia, who is employed by the tech giant Google. Pretotyping sounds a bit like a prototype, and it is no coincidence. The word pretotyping is composed of the words pretend and prototype. We are thus dealing with a “pretend”-prototype.

The difference between a prototype and a pretotype is typically that a prototype is a working version – albeit at an early stage of development. On the other hand, a pretotype is a quick sketch or “pretend”-prototype, which is far from being a working version, for a quick test of an idea to see if it works in the real world.

The basic idea of the pretotyping concept is to be able to eliminate worthless ideas before using too many resources on them. In this case, the statement “Fail often – but fast” is therefore followed. The goal is to test as much as possible and as soon as possible and on that basis to be able to eliminate ideas that will not hold in reality. In other words, when working with digital services and business models, you need to eliminate the ideas that will not work in reality as early as possible before using resources to develop and finish them.

Examples of pretotyping methods are:

• Interviews by, or just simple questions to, customers, colleagues or network.
• Quick sketches or illustrations. For example, a Post-it on the screen of a smartphone to illustrate a new app or a sticker on a machine with the text ‘start’ and another with ‘stop’.
• A ‘dummy’ image on Facebook where visitors can click if a product is of interest in order to identify potential customer interest before using resources to make a prototype.

Using pretotyping does not preclude making a conventional prototype. However, pretotyping should ideally be used before the actual prototype so that you only develop a prototype when pretotyping has shown that there the idea seems viable.

In the next section, we will take a closer look at two tools that can be used in connection with testing by using pretotyping and prototyping. One is the classic Interview, and the other is a tool developed within IT under the name Retrospective Think-aloud protocol. The idea of think-aloud protocol testing is that the test person in a real or simulated situation must say what he or she thinks in order to get a nuanced picture of a user experience from the user’s perspective. Both methods are effective for testing Value Propositions, Value Creation and Delivery as well as Value Capture in relation to the business model’s current and future customers.

Interviews

Interviews, or just a few questions, can be a very effective way to test business ideas. Here, you have to make it clear whom you want to ask, and start with people who are likely to be the target group for the digital service or business model you have in mind. Just testing the idea on three customers, who represent the target group, will give a very good indication of whether you have grasped the right thing.

Examples of key questions:

“If I could offer you ‘XXXX’ so you could get rid of problems with ‘YYYY’, would that interest you?”

• If not: Would others in or around your organisation be interested in this?
• If so: How often do you think you would need this solution?
• If so: What would it be worth to you? And would you, for example, prefer to pay per time you use the product/service or per year?

Questions can be many and can vary depending on which direction you are working towards with your business model. It will always be a good idea to test a few ideas if, for example, you are on a visit to an existing or a potential customer.

Output

The result of Interviews will be the answer to an assumption on a Hypothesis Card. Here, of course, you must be aware that one person’s worldview is not true for the rest of the world. Thus, it will make sense, even in the initial testing and development, always to ask at least three customers. If they have approximately the same opinion, you can continue working with this result in your business model. Another important output from an interview may be that the interviewee highlights other ideas, other potential customers or something else that has value in the customer’s opinion. It is important to listen carefully and use what you have learnt in your development work.

Think-aloud protocol

Think-aloud protocols stems from software development and is frequently used in testing digital services such as web pages. What the test is basically about is that a test person formulates in his/her own words all the thoughts that go through his/her mind while the test is being performed. Hence the name: Think-aloud protocol. The version presented here is the retrospective version, which simply means that the test person first completes the test and then ‘thinks aloud’ while he or she revisits a recording of the test. This ensures that the test person is not disturbed by having to say everything aloud during the test itself, but that as many details as possible are added when the person subsequently has to ‘think aloud’ during the test experience after the active part of the test is completed.

Think-aloud protocols are good for testing everything with user interfaces. This could for example be:

• Physical products with buttons, keyboards or anything that the user must operate.
• Software and web-based digital services.
• Other scenarios in a process flow involving interaction between a human being and a designed product or service.

Please see below the illustration of a simple Think-aloud protocol with a watering can.

“Retrospective think-aloud protocol” adopted from Digital Business Models for the Future under license CC BY-SA 4.0

Practical information

Before a test, you must decide which solution you want to test, as well as with which potential customer or user you will include as test person. In addition, you must decide the time and place for the test. The chosen test site should reflect an environment reminiscent of where the final solution will be used so that the test becomes as realistic as possible. All the things that the test person may need during the test should be prepared, and equipment to record the test must be installed. This can be in the form of video recording of the room where the test takes place or recordings of a screenshot if the solution is digital.

Step-by-step guide

1) The test scenario is described via a number of tasks or questions that the test person (i.e. customer/user) can be asked. These tasks or questions should reflect the intended user situation, broken down into small actions. For example:

1. Pick up the watering can, water the flower, put down the can.
2. Log on to the customer portal with the information on the screen, find your profile information, order a spare part, check if it is in stock.

2) Next, it is time for the actual test where the customer is invited in and asked to behave as in a natural user situation. Here, it is important that you, as the test manager, have your hands free to make notes along the way.

The test manager can now set the test person the first task: “Pick up the watering can”. While the test person performs the action, the test manager notes down if there are any signs of challenges, e.g. if the test person does not seem to be able to spot the watering can, that the watering can is lifted in a different way than intended or if the test person is about to drop the watering can or similar.

Likewise, the subsequent tasks are set until the entire test scenario has been executed

3) Immediately afterwards, the test manager and the test person should watch the recording.

The test person is now asked to ‘think aloud’ and explain (a) what he/she thought in the situation, (b) what was surprising, (c) what was tricky, (d) how things were generally experienced. The test manager takes notes during the entire explanation (the explanation can also be recorded), and via the notes, the test manager can make sure that anything, which seemed to be challenging, or made the test person wonder or excited, is explained in more detail. Hence, all knowledge from the customer test has been documented.

4) You can now use these insights or this knowledge for a new iteration of the watering can setup.

Output

The result of the test can contribute directly to further development of the tested product, the digital service platform or interaction flow. In the example above, the colour of the watering can may need to be changed so that the customer can see it more easily, or perhaps the handle needs to be placed a little differently to create balance, or something completely different.

Expert advice

When you test something on the users, you have to make sure that you do the test from the user’s point of view. Thus, it is also crucial that you trust the customer’s or user’s statements and that you do not end up adding your own interpretation to the statements.

Next step

Testing the elements of a business model can lead the process into a series of loops where you may want to revisit the knowledge found in the other modules: Basic Principles, Pre-Analysis of the Surrounding World, Customer Analysis, Pre-analysis of the Company’s Situation, Service Design and The Development Process. Ultimately, the goal is for you to test your business model to a maturity level where you are ready to move on with Implementation, which is described in a separate module.

The content elements above have been developed through two projects:

‘Digital Business models for the Future’ by Aarhus University, Aarhus School of Marine and Technical Engineering and Danish Technological Institute supported by The Danish Industry Foundation. The material from this project has been adopted in alignment with CC BY-SA 4.0

‘EU-IoT’ by Aarhus University, Martel Innovate, Netcompany-Intrasoft, Fortiss, BluSpecs and funded by the Horizon 2020 Framework Programme under topic ID ICT-56-2020, grant ID 956671.