Prompt Engineering
Week 5b seminar: the second half of the week 5 session and the last seminar of the semester. Everything from week 6 on is studio and project work. See comp1100 for course logistics.
The AI policy
Under what circumstances can you use generative AI or other related tools to complete your work in COMP1100?
- None. It is prohibited.
- For coding only — no use for writing hypotheses, test cards, etc. c) For anything except generated interview or other data. ✓
- For any task.
The exception is the important part, and large-language-models explains why: a model will fluently generate plausible interview data, and fabricated data defeats the entire purpose of customer-discovery.
Outline
- What is machine learning?
- Large language models — how they work, and where they break
- Prompt engineering: seven tactics
- LLMs as tools for thought
Machine learning and LLMs
See large-language-models — curve fitting as the intuition for machine learning, next-token prediction with the “The woman ran up the ___” probability tables, and the apples example where the model subtracts 300 from both the red and the green count and reports −99 green apples without noticing.
Prompt engineering
See prompt-engineering for all seven tactics with worked prompts:
- Use delimiters
- Ask for structured output
- Ask it to check whether conditions are satisfied
- Specify the steps required
- Iterate
- Be as specific as possible
- Build code incrementally
LLMs as tools for thought
The most transferable idea is to ask the model to interrogate you rather than answer you:
I am doing a project using the Lean Startup methodology. I have a hypothesis about my solution that I want to test. I would like to construct a test card for it. Can you please interact with me to help me construct this? Do not give me answers, just ask a series of questions until I have the test card. Provide feedback on my answers.
The resulting test card contains your team’s hypotheses, not generated ones — which is what keeps it on the right side of the AI policy above.
Homework
Teams: by the submission next Thursday 4:00pm you should have a clear idea of your customer segments and their problems, have those hypotheses formalised as test cards, have tested them, and recorded results as learning cards. You should be starting to formulate what solutions you could provide — some teams may have tested solutions already.
So in week 5 you should be:
- Running final tests for hypotheses related to customer segments and their problems.
- Analysing interview data for common problems and pain points.
- Starting to brainstorm potential solutions leading to a unique value proposition.
- Preparing your repository to submit iteration 1 — see the submission requirements under ‘Assessment’ and in the week 5 studio handout.
Individuals: complete your interviews about your problem hypotheses, record them (with consent), have them transcribed, then commit and push the transcripts to your GitHub repository so the team can use them. Also participate in the data analysis — that’s expected of everyone in the team.