Test Cards and Learning Cards

definitions
innovation
experimentation
hypotheses

Two Strategyzer card formats that give experiments a fixed shape. A test card is written before an experiment; a learning card is written after it. Together they’re the unit of work in experimentation, and the artefacts the COMP1100 submissions are built from.

Test card

Five lines, filled in before running anything:

We believe that          ______________________
To verify that, we will  ______________________
And measure              ______________________
We are right if          ______________________
Therefore, we will ask   ______________________

The “we are right if” line is what forces falsifiability — you commit to a threshold before you see the data, so you can’t move the goalposts afterwards.

Worked example — users

We believe that our end users are people over 65 who live on their own who would use technology to keep in touch with their family more often because they do not see them much.

To verify that, we will interview 50 people over 65 who live on their own.

And measure how many would like more regular digital interaction with their family.

We are right if 70% of older people say that they would like more regular digital interaction with their family.

Therefore, we will ask “how often would you like to interact with your family members?”

Worked example — paying customer

Same product, different party — note that the user and the customer need separate cards:

We believe that our paying customers are close family members of our end users who would use technology to keep in touch with older family members using a photo-sharing app rather than delegate to a service, because they would like assurance that their family members are doing well in a more personal manner.

To verify that, we will interview 50 people who have family members over 65 that live on their own and have an existing emergency alarm system.

And measure how many would prefer regular (every day) digital interaction with their family member rather than delegating to a service.

We are right if 70% of family members say that they would like more regular digital interaction with their family member rather than delegating to a service.

Therefore, we will ask “how often would you like to interact with your older family member?”

Learning card

Four lines, filled in after:

We believed that         ______________________
We observed              ______________________
From that we learnt that ______________________
Therefore, we will       ______________________

Worked example — hypothesis accepted

We believed that our end users are people over 65 who live on their own who would like to interact digitally with their family more often because they do not see them much.

We observed that 44 out of 50 people we interviewed confirmed that they would like daily digital interaction with their family.

From this we learnt that our end users would consider using a product that supported this.

Therefore, we will accept this hypothesis.

44/50 is 88%, over the 70% threshold the test card committed to — so the decision is already made by the card, not by argument.

Worked example — hypothesis rejected

This is the more instructive one:

We believed that close family members would like to interact digitally with older family members using a photo-sharing app.

We observed that 12 out of 50 people we interviewed confirmed that they would use such an app.

From this we learnt that family members would not use a new product. However, we learnt that some family members would send photos from their smartphone to such an app.

Therefore, we will reject this hypothesis, and test a new hypothesis about how family members would take and send photos for their older family member.

Two things to copy from it. First, the rejection produces a replacement hypothesis, not a dead end. Second, the “however” — the most valuable finding was a side observation that the headline number would have buried. This is what experimentation means by paying attention to outliers.

The templates are in the course’s business model and testing templates.