COMP1100 — Week 5 Notes
Prompt Engineering
Week 5b seminar: the second half of the week 5 session and the last seminar of the semester. Everything from week 6 on is studio and project work. See comp1100 for course logistics.
The AI policy
Under what circumstances can you use generative AI or other related tools to complete your work in COMP1100?
- None. It is prohibited.
- For coding only — no use for writing hypotheses, test cards, etc. c) For anything except generated interview or other data. ✓
- For any task.
The exception is the important part, and large-language-models explains why: a model will fluently generate plausible interview data, and fabricated data defeats the entire purpose of customer-discovery.
Outline
- What is machine learning?
- Large language models — how they work, and where they break
- Prompt engineering: seven tactics
- LLMs as tools for thought
Machine learning and LLMs
See large-language-models — curve fitting as the intuition for machine learning, next-token prediction with the “The woman ran up the ___” probability tables, and the apples example where the model subtracts 300 from both the red and the green count and reports −99 green apples without noticing.
Prompt engineering
See prompt-engineering for all seven tactics with worked prompts:
- Use delimiters
- Ask for structured output
- Ask it to check whether conditions are satisfied
- Specify the steps required
- Iterate
- Be as specific as possible
- Build code incrementally
LLMs as tools for thought
The most transferable idea is to ask the model to interrogate you rather than answer you:
I am doing a project using the Lean Startup methodology. I have a hypothesis about my solution that I want to test. I would like to construct a test card for it. Can you please interact with me to help me construct this? Do not give me answers, just ask a series of questions until I have the test card. Provide feedback on my answers.
The resulting test card contains your team’s hypotheses, not generated ones — which is what keeps it on the right side of the AI policy above.
Homework
Teams: by the submission next Thursday 4:00pm you should have a clear idea of your customer segments and their problems, have those hypotheses formalised as test cards, have tested them, and recorded results as learning cards. You should be starting to formulate what solutions you could provide — some teams may have tested solutions already.
So in week 5 you should be:
- Running final tests for hypotheses related to customer segments and their problems.
- Analysing interview data for common problems and pain points.
- Starting to brainstorm potential solutions leading to a unique value proposition.
- Preparing your repository to submit iteration 1 — see the submission requirements under ‘Assessment’ and in the week 5 studio handout.
Individuals: complete your interviews about your problem hypotheses, record them (with consent), have them transcribed, then commit and push the transcripts to your GitHub repository so the team can use them. Also participate in the data analysis — that’s expected of everyone in the team.
The History of Computing
Week 5a seminar. The second half of the same session is 2025-08-25-prompt-engineering. See comp1100 for course logistics.
Outline
- In what year was the first programmable computer built?
- The eras of computing and AI — the golden age, two winters, the revival
- Another AI winter?
- History and representation
The history
See history-of-computing-and-ai for the full reference: the six defensible answers to when the first programmable computer was built (87 BCE through 1951), the era-by-era timeline, the Turing Test, Dartmouth 1956, both AI winters and what went wrong in each, the revival, and six people the field’s history remembers.
Why this is in an innovation course
This is more than history. The argument is the pattern:
- 1956: “Machines will be capable, within twenty years, of doing any work a man can do.” — Herbert Simon
- 1958: the Perceptron is “the embryo of an electronic computer that will be able to walk, talk, see, write, reproduce itself and be conscious of its existence.” — New York Times
- 1970: “In from three to eight years we will have a machine with the general intelligence of an average human being.” — Marvin Minsky
- 1999: “Artificial intelligence will reach human levels by around 2029.” — Ray Kurzweil
- 2014: “We will have fully self-driving cars on the road by 2017.” — Elon Musk
- 2024: “We are now confident we know how to build AGI as we have traditionally understood it.” — Sam Altman
Twice, that pattern of overpromising ended in a funding collapse. Amara’s Law names the shape of it:
We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
For a course about testing your hypotheses rather than believing them, this is the field’s own cautionary tale — the hype cycle is what happens when nobody makes a falsifiable claim.
“By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.” — Eliezer Yudkowsky
Representation
Six people the field’s history remembers: Ada Lovelace, Alan Turing, Margaret Hamilton (who coined the term software engineering), Grace Hopper, Fei-Fei Li and Timnit Gebru. See history-of-computing-and-ai.
Next
2025-08-25-prompt-engineering — the second half of the same session.
Reference material
Customer Discovery
Customer discovery is the process of turning an initial idea or vision about a market or customers into a set of facts. The process tests whether the business model for a company solves real customer problems, and whether the product features fit those customer problems.
It is the first stage of the customer development process in lean-startup, and the stage COMP1100 spends nearly all its time in — all three assessment submissions are iterations of it.
The four-step process
1. Define hypotheses
↓
2. Test the problem
↓
3. Test the solution
↓
4. Verify, pivot, or refine → (customer validation)
↺
The loop repeats. Each pass turns more of the business-model-canvas from hypothesis into fact.
Step 1 — define our hypotheses
The initial hypotheses are our vision. Write answers to:
- What are our customers’ top problems?
- How does our idea help to solve these problems?
- What would a day in the life of a customer look like before and after our product?
These are guesses or assumptions at best — which is exactly why they get tested rather than built on.
The step also includes sketching a vision. This means a rough sketch, not a polished rendering.
Step 2 — test the problem
Gather data to refute or confirm our hypotheses about the problem. This is where the distinction that runs through the whole course appears:
| Who they are | |
|---|---|
| User | People who use a product |
| Customer | People who pay for or approve a product |
They are frequently not the same people, and a hypothesis needs to say which it’s about.
Step 3 — test the solution
Gather data to refute or confirm hypotheses about our solution. The thing being sought is product/market fit — the match between the customer’s problems and needs on one side, and the product’s features and value proposition on the other.
“Ideas are like babies because everyone thinks theirs is cute, therefore be objective when judging your own ideas.”
— Tina Seelig
Step 4 — verify, pivot, or refine
Refine the product, pivot the strategy, or (rarely) rethink the vision. See lean-startup.
A worked example: the medical alert system
The running example is an emergency alarm for older people living alone, with a speaker, microphone, emergency button, well-being check button, status light and a wireless transmitter.
Initial hypotheses:
- Older people want to be safe.
- Families of older people want to know, on a daily basis, that their older family member is safe.
- Older people want to be able to initiate a call for help and cancel any call for help.
- Older people want to maintain their independence/autonomy.
- Older people want to be cared for.
- Older people do not want to have to press the green button each day to let the care company know that they are alive.
- Older people do not want to wear a visible (and ugly!) emergency alarm around their neck or wrist.
Hypothesis 1 is about both users and customers, while hypothesis 2 is mainly about customers — the family, not the user.
After testing the problem, hypothesis 5 was refuted outright and 7 was partly refuted, producing new hypotheses:
- Older people want to be cared about. (the original said “cared for” — a small wording change carrying a completely different product)
- Some older people wouldn’t mind wearing a wireless transmitter if it was not visible to others.
- Older people wanted to be in touch with their family members, rather than a service provider.
- Older people would prefer a medical alert that allowed them to be mobile — the existing alarms were attached to a phone line and the transmitters had short range.
This is the whole method in miniature: the product that survives contact with real users is not the one that was sketched.
Early adopters
Not all of the market matters at the start. The adoption curve:
| Segment | Share |
|---|---|
| Innovators | 2.5% |
| Early adopters | 13.5% |
| Early majority | 34% |
| Late majority | 34% |
| Laggards | 16% |
The first release targets early adopters, not the majority.
Customer development heuristics
- There are no facts inside your building, so get outside.
- Develop for the few, not the many.
- Early adopters make your product.
- And they know more than you.
- Focus groups are for big companies, not startups.
- Surveys don’t give insight compared to interviews — follow-up questions are difficult.
- The goal for release 1 is the minimum feature set for early adopters.
Heuristic 6 is the reason the course’s assessment requires recorded, transcribed interviews rather than a survey: you cannot ask “why?” to a form.
See value-proposition-canvas for the ground rules on actually conducting those interviews, and test-cards-and-learning-cards for the format the hypotheses and results are written in.
Experimentation
Why experiment?
To gain insights about customers, their problems, and our solutions — before spending the effort to build.
Two types, two reasons
Two types of experiment:
- Generative / exploratory — explore a space.
- Evaluative — find data to support or refute a hypothesis.
Two reasons to experiment:
- Evaluate our understanding of our customers and market fit.
- Evaluate products.
The scientific method
1. State hypothesis → 2. Design experiment → 3. Test with data → 4. Gather insights ↺
The lean startup version of this loop (Steve Blank) expands into seven steps:
- Extract hypotheses
- Prioritise hypotheses
- Design tests
- Prioritise tests
- Run tests
- Capture learnings
- Make progress — and repeat
Both prioritisation steps are easy to skip and shouldn’t be: a team has more hypotheses than it has time, and testing the wrong one first wastes a week.
What makes a good hypothesis?
The full list: testable, reliable (not fake), concise, replicable, specific, ethical. Condensed to four:
| Property | |
|---|---|
| #1 | Measurable/testable ⇒ falsifiable |
| #2 | Specific |
| #3 | Ethical |
| #4 | Simple and clear |
Falsifiability is the load-bearing one. An example of a non-falsifiable hypothesis:
“This solution will help this segment of customers improve their health over the long term.”
Nothing about it can come back false — “improve”, “health” and “long term” are all unmeasurable as stated, so no result could ever refute it. The exercise is to rewrite it until a specific observation could prove it wrong.
Structuring a hypothesis
The test-cards-and-learning-cards format gives hypotheses a fixed shape:
We believe that [target] are [showing behaviour / display interest in] [for reason Y]. To verify that, we will [undertake behaviour] to [identify behaviour]. And measure [particular variable]. We are right if [some measurable statement]. Therefore, we will ask [specific questions].
Types of data collection
Six ways to gather evidence, roughly in increasing order of cost:
| Method | What it is |
|---|---|
| Interviews | Talk with potential customers |
| Observations | Observe people while doing their “jobs” |
| Data collected by others | Find existing data that tests your hypothesis |
| Immersion | Work in your customer’s environment |
| Low fidelity prototyping | Paper prototypes or low-fidelity digital prototypes |
| High-fidelity solutions | Implement new functionality in an existing product |
See prototyping for the last two.
Experiments should
- Be as small and comprehensible as possible.
- Have clear expected outcomes.
- Have a clear definition of success/failure that everyone agrees to — agreed before the results come in, or the team will argue about interpretation afterwards.
- Be as brief as possible when interviewing — preferably under 30 minutes.
More tips
Do: listen; observe people’s behaviour, tone and reactions, not just what they say; ask “why” and “why not”; ask who else you should talk to; ask if you can follow up later.
Don’t: “correct” people (except to clarify a misunderstood question); convince or sell to them.
Learning from experiments
When results come back, interrogate them rather than just recording them:
- If people would not pay for this, what would they pay for?
- If these are not our customers, then who is?
- Did we see any outliers that could open up new ideas?
- Pay attention to why our hypotheses are rejected.
A refuted hypothesis is more informative than a confirmed one — it tells you something you didn’t already believe.
Key takeaways
- Experiments = gain insights about customers and products.
- Hypotheses must be falsifiable.
- Test cards structure our hypotheses; learning cards structure our learnings.
- Refuted hypothesis ⇒ ask why.
The History of Computing and AI
In what year was the first programmable computer built?
The question is a trick. There are six defensible answers:
| Year | Machine |
|---|---|
| 87 BCE | Antikythera machine |
| 1642 | Pascaline |
| 1805 | Jacquard machine |
| 1837 | Analytical Engine |
| 1943 | Colossus |
| 1951 | CSIRAC |
Which one counts depends entirely on what you mean by “programmable” and “computer” — which is the point.
Timeline
| Period | Era | What happened |
|---|---|---|
| 1837 | Charles Babbage invents the Analytical Engine. Ada Lovelace recognises it is a general purpose calculating machine. | |
| 1941–1943 | British mathematicians and engineers build the Bombe and Colossus for cracking codes during World War II. | |
| 1950 | Alan Turing proposes the Turing Test as a goal for artificial intelligence, rather than defining “intelligence”. | |
| 1956 | The Dartmouth Conference is held. The proposal coins the term artificial intelligence. | |
| 1956–1974 | The Golden Era | Foundational search algorithms and artificial neural networks are invented. |
| 1974–1980 | The first AI winter | The hype of the golden era did not eventuate. Funding dried up and excitement dwindled. |
| 1980–1987 | The Knowledge Era | Knowledge-based systems, such as Prolog and expert systems, invented. |
| 1987–1993 | The second AI winter | The hype of the knowledge era did not eventuate. Funding dried up and excitement dwindled. |
| 1994–now | The Revival Era | Intelligent agents, computational power, the Internet, and machine learning. |
The Turing Test (1950)
“The new form of the problem can be described in terms of a game which we call the ‘imitation game’”
— Alan Turing, Computing Machinery and Intelligence, Mind, LIX(236):433–460, 1950
Turing sidesteps defining intelligence and substitutes a test for it. That move — replacing an unanswerable definition with an operational check — is the same instinct as making a hypothesis falsifiable in [experimentation].
Dartmouth (1956)
The founders present: John McCarthy, Marvin Minsky, Claude Shannon, Ray Solomonoff, Alan Newell, Herbert Simon, Arthur Samuel, Oliver Selfridge, Nathaniel Rochester, Trenchard Moore.
“We propose that a 2-month, 10-man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College… The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”
Two months, ten men. The proposal’s confidence is the first data point in the pattern that follows.
Its lasting output was the divide and conquer model of AI, splitting the problem into perception, knowledge representation, learning, reasoning and planning.
The Golden Age (1956–1974)
- Reasoning as search — Shakey the robot and A*, Hart, Nilsson and Raphael (1968); STRIPS (Stanford Research Institute Problem Solver), Fikes and Nilsson (1971), with its preconditions/postconditions action model.
- Perceptrons and neural networks — the single-layer perceptron, Rosenblatt (1958).
The first AI winter (1974–1980)
What went wrong — outcomes failed to live up to the hype:
- Scalability
- Commonsense knowledge
- Perceptron limitations
- Moravec’s paradox
The result: funding dried up, global interest died down, and criticism came from philosophers and cognitive scientists.
The Knowledge Era (1980–1987)
- Knowledge-based systems — Prolog (Colmerauer and Roussel, 1972), and formal ontologies.
- Expert systems — MYCIN, for diagnosis of blood diseases (Shortliffe, Buchanan and Cohen, 1970s).
The second AI winter (1987–1993)
Same headline, different specifics — outcomes again failed to live up to the hype:
- Scalability
- Maintenance
- The qualification problem
- Moravec’s paradox (again)
The result: funding dried up (DARPA declared AI was “not the next wave”), global interest died down, AI companies went bankrupt.
The revival (1994–present)
- Intelligent agents and decision theory — Bayesian networks, Pearl (1988)
- Computational power — Moore’s law
- The Internet and big data
- Machine learning — backpropagation in deep neural networks, Rumelhart, Hinton and Williams (1986)
Another AI winter?
This is the real argument of the history. We have been here before:
“In from three to eight years we will have a machine with the general intelligence of an average human being.” — Marvin Minsky (1970)
“Machines will be capable, within twenty years, of doing any work a man can do.” — Herbert Simon (1956)
“…the embryo of an electronic computer that [the Navy] expects will be able to walk, talk, see, write, reproduce itself and be conscious of its existence.” — New York Times on the Perceptron (1958)
“Artificial intelligence will reach human levels by around 2029.” — Ray Kurzweil (1999)
“We will have fully self-driving cars on the road by 2017” — Elon Musk (2014)
“We are now confident we know how to build AGI as we have traditionally understood it” — Sam Altman (Jan 2024)
Amara’s Law
We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
Which is why the pattern above isn’t simply “the predictions were wrong” — the winters came from the short-run overestimate, and the revival from the long-run underestimate.
What are some of the risks?
“By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it.” — Eliezer Yudkowsky
“People worry that computers will get too smart and take over the world, but the real problem is that they’re too stupid and they’ve already taken over the world.” — Pedro Domingos
“The beauty of AI and what we can do with it is currently being overshadowed by reckless hype, monotheistic techniques, and absurd deification. Yet, it’s not the first time in 65+ years nor it’ll be the last one. We haven’t learnt a thing.” — Dagmar Monett
Representation in computer science
Six people the field’s history remembers:
| Person | For |
|---|---|
| Ada Lovelace | The first “computer programmer”; the first person to consider general purpose computers doing more than numerical calculation |
| Alan Turing | Turing machines, code cracking in World War II, the Turing test, so many other things |
| Margaret Hamilton | Software engineer at the MIT Instrumentation Lab; developed on-board flight software for NASA’s Apollo program; coined the term “software engineering” |
| Grace Hopper | Identified that computer programs can be written in a high-level language and compiled into machine code |
| Fei-Fei Li | Machine learning researcher; founder of AI4ALL |
| Timnit Gebru | Research on bias in machine learning models; founder of Black in AI |
CSSE1001 covers overlapping ground from a different angle — see CSSE1001’s history of computing for the hardware timeline and the machine-code-to-Python language progression.
Large Language Models
Machine learning, briefly
The seminar builds up from curve fitting. Given data on sleep hours against health, machine learning is the business of finding the function that fits:
- \(y = ax + b\) — a straight line
- \(y = ax^2 + b\) — a curve
- \(y = ax + by + c\) — a plane, once there are two inputs (sleep and exercise)
The model is the function; learning is finding its parameters.
How LLMs actually work
An LLM predicts the next token, one at a time, from a probability distribution over the whole vocabulary.
Given the prefix *“The woman ran up the ___“*:
| Candidate | Probability |
|---|---|
| “hill” | 1.2% |
| “stairs” | 1.9% |
| “the” | 1.9% |
| “to” | 1.2% |
| “baking” | 0.005% |
| “package” | 0.0001% |
| “myriad” | 0.000004% |
| “gazpacho” | 0.00001% |
Add one more word to the context — “The woman ran up the” — and the distribution shifts sharply: “hill” jumps to 10.8% and “stairs” to 6.9%, while “the” collapses to 0.00001%.
That’s the whole mechanism. And it explains why *“The woman ran up the ___“* and *“What did the woman run up? ___“* are, to the model, the same kind of problem: continue the text plausibly.
Where it breaks: the apples
The seminar’s worked failure, from ChatGPT:
Q: I have 419 red apples and 201 green apples. I eat 300 of the apples. How many apples do I have left?
A: 620 (initial count) − 300 (eaten) = 320 apples left. ✓
Correct. Then:
Q: How many red apples do I have left?
A: 419 − 300 = 119 red apples left.
Q: How many green apples do I have left?
A: 201 − 300 = −99 green apples. It appears that you’ve eaten more green apples than you initially had…
The model subtracts the whole 300 from each colour independently, producing a negative count — and then narrates the absurdity rather than noticing it. The underlying problem was never specified (which apples were eaten?), and the model doesn’t flag the ambiguity; it generates a plausible continuation for each question in isolation.
The lesson for the course: an LLM produces text that reads like reasoning. Verify anything that has to be true.
Why this matters for COMP1100
The course’s generative AI policy permits AI for anything except generating interview or other data (see comp1100). The apples example is the argument for that rule — a model will happily produce fluent, confident, wrong content, and interview data that was generated rather than gathered defeats the entire point of customer-discovery.
See prompt-engineering for getting useful work out of them anyway.
Prompt Engineering
Seven tactics, aimed at programming tasks. See large-language-models for why they work.
Tactic 1 — use delimiters
Mark where the input starts and stops, and mark the shape of the output:
Here is some code in Python: $$$ … $$$
… format the output as follows: """On [DATE], [NAME] … """
Tactic 2 — ask for structured output
… Put these into an HTML table. The table should have three columns.
… format it as a JSON object: """On [DATE], [NAME] … """
Example:
Write a python program that can generate 5 sample emails, including email header. Each email should contain a body that has a greeting, then the main part, and then a sign-off. The main part should contain an order for a part, of the format:
""" I would like to order [NUMBER] [PARTS]. """
Tactic 3 — ask it to check whether conditions are satisfied
Tell it what to do when the input doesn’t match the expected shape, rather than leaving it to improvise:
Below is an email from someone. Extract the following into a structured format of:
"""On [DATE], [PERSON] ([SENDER EMAIL ADDRESS]) from [COUNTRY] sent an email with a message about [SUMMARY]""". If the email contains a phone number, determine the country number, and insert the country name. Otherwise, write “unknown country” in place of [COUNTRY].
Tactic 4 — specify the steps required
Tactic 5 — iterate
The following is an email from a customer outlining their opinion on a project. Create an email that responds to them, thanks them for their review, and addresses any items in the email.
Tactic 6 — be as specific as possible
A before-and-after:
This:
Create a Python program that downloads emails from the email address “uqtmiller@gmail.com” and gets an image from them.
Versus this:
Create a Python program that: 1. Downloads any email in the account “uqtmiller@gmail.com” 2. Saves any image attachments (JPG or PNG) in the directory called “images” 3. Deletes the email from the email inbox.
A longer example, showing how far the specificity goes — including the error case and the runtime environment:
Create a simple html page that: 1. Use javascript to select a random image from the directory ‘images/’ 2. When the image is selected, display a message that says “a new image has been received” 3. When the user clicks the message, the randomly selected image is displayed. 4. If no image is available, print “There is no image in the directory <DIRECTORY>”, where DIRECTORY is the complete path. 5. A message “An email has been sent to your_family@fake.com” should be displayed. 6. Make sure this runs locally.
Tactic 7 — build code incrementally
Don’t ask for the whole system at once. Ask for the page, then ask for the specific piece:
Prompt 1: Create a simple html page: 1. Use javascript to select a random image from the directory ‘images/’ …
Prompt 2: Create javascript code that selects the file with the most recent timestamp from a list called ‘images’.
LLMs as tools for thought
The most interesting use is not asking for an answer, but asking to be interrogated:
I am doing a project using the Lean Startup methodology. I have a hypothesis about my solution that I want to test. I would like to construct a test card for it.
Can you please interact with me to help me construct this?
Do not give me answers, just ask a series of questions until I have the test card.
Provide feedback on my answers.
This inverts the usual pattern: the model runs the process and you supply the content, so the test card ends up containing your team’s actual hypotheses rather than a plausible-sounding generated one. It stays on the right side of the course’s AI policy for exactly that reason — see comp1100.
Test Cards and Learning Cards
Two Strategyzer card formats that give experiments a fixed shape. A test card is written before an experiment; a learning card is written after it. Together they’re the unit of work in [experimentation], and the artefacts the COMP1100 submissions are built from.
Test card
Five lines, filled in before running anything:
We believe that ______________________
To verify that, we will ______________________
And measure ______________________
We are right if ______________________
Therefore, we will ask ______________________
The “we are right if” line is what forces falsifiability — you commit to a threshold before you see the data, so you can’t move the goalposts afterwards.
Worked example — users
We believe that our end users are people over 65 who live on their own who would use technology to keep in touch with their family more often because they do not see them much.
To verify that, we will interview 50 people over 65 who live on their own.
And measure how many would like more regular digital interaction with their family.
We are right if 70% of older people say that they would like more regular digital interaction with their family.
Therefore, we will ask “how often would you like to interact with your family members?”
Worked example — paying customer
Same product, different party — note that the user and the customer need separate cards:
We believe that our paying customers are close family members of our end users who would use technology to keep in touch with older family members using a photo-sharing app rather than delegate to a service, because they would like assurance that their family members are doing well in a more personal manner.
To verify that, we will interview 50 people who have family members over 65 that live on their own and have an existing emergency alarm system.
And measure how many would prefer regular (every day) digital interaction with their family member rather than delegating to a service.
We are right if 70% of family members say that they would like more regular digital interaction with their family member rather than delegating to a service.
Therefore, we will ask “how often would you like to interact with your older family member?”
Learning card
Four lines, filled in after:
We believed that ______________________
We observed ______________________
From that we learnt that ______________________
Therefore, we will ______________________
Worked example — hypothesis accepted
We believed that our end users are people over 65 who live on their own who would like to interact digitally with their family more often because they do not see them much.
We observed that 44 out of 50 people we interviewed confirmed that they would like daily digital interaction with their family.
From this we learnt that our end users would consider using a product that supported this.
Therefore, we will accept this hypothesis.
44/50 is 88%, over the 70% threshold the test card committed to — so the decision is already made by the card, not by argument.
Worked example — hypothesis rejected
This is the more instructive one:
We believed that close family members would like to interact digitally with older family members using a photo-sharing app.
We observed that 12 out of 50 people we interviewed confirmed that they would use such an app.
From this we learnt that family members would not use a new product. However, we learnt that some family members would send photos from their smartphone to such an app.
Therefore, we will reject this hypothesis, and test a new hypothesis about how family members would take and send photos for their older family member.
Two things to copy from it. First, the rejection produces a replacement hypothesis, not a dead end. Second, the “however” — the most valuable finding was a side observation that the headline number would have buried. This is what [experimentation] means by paying attention to outliers.
The templates are in the course’s business model and testing templates.