The team at Anthropic released a video sharing some of their best tips for prompt engineering. In the hour-long session, a team of prompt engineers at Anthropic discussed what prompt engineering is, how to be a good prompt engineer, tips for writing good prompts, and the future of prompt engineering.
This video is packed with wisdom from top experts in the industry on how to write better prompts to get the most out of your interactions with AI models.
The discussion is held by four prompt engineering experts at Anthropic: Alex Albert who leads Developer Relations and previously worked as a prompt engineer, David Hershey who works in Applied AI by working with customers to implement LLMs and prompting, Amanda Askell who leads an Alignment Finetuning team, and Zack Witten who works as a Prompt Engineer previously helping customers and now focuses on helping people become better at prompting through prompt generation and educational materials.
If you haven’t seen the video yet, I highly recommend you check it out.
27 Prompt Engineering Tips
This post will focus on everything I learned from the video about prompt engineering and a list of tips to help you become a better prompt engineer.
1. Talk to the Model Like You Are Talking With a Person
This tip is at the top of the list because it is one of the absolute best ways to improve your prompts. All of the experts on the podcast made this point.
David, who helps enterprise customers write prompts, said they come to him for help with their prompts. He simply asks them to describe what they want the model to do. After they describe it to him, he tells them to write that down and says that is their prompt.
Oftentimes, we overthink it when trying to write prompts. Instead, write your prompts to communicate clearly the task you want done as if you were describing it to another person.
2. Read Your Prompts Out Loud
This tip reminds me of early English classes in school where we were told to speak our sentences out loud to check if the grammar sounded correct. But, this tip is also helpful for writing better prompts.
Reading your prompts out loud helps you to think about the words in the prompt and their meaning. This is a good exercise to really think about the words in your prompt and how the model will interpret them.
3. Pay Attention to Details in Your Prompts
This tip sounds obvious but Amanda actually says she often does the opposite by not using capitalization or punctuation and leaving typos in her prompts.
However, Zack makes the counter argument that while perfect grammar might not directly improve the quality of responses, showing attention to the details of your prompt will help you write better prompts overall.
If you are reading over your prompts frequently, which helps you improve them, you’ll catch typos and fix them. But this doesn’t necessarily mean that the fixed typos are improving your prompt as much as you spending time working on it is.
So, take your time while writing your prompts and put the effort and level of care into them to get the same level of results.
4. Continuously Experiment by Trial and Error
Trial and error are what makes crafting good prompts more like prompt engineering than prompt writing according to Zack. To get the best results from AI models, you have to experiment and continually try new things much like any engineering discipline.
5. Use the Restart Chat Button Often
Zack explains that one nice thing about chatting with AI models, is that unlike in conversations with real people, you can restart it at any point.
When you are chatting with the model and don’t seem to be getting anywhere, start over and rewrite your initial prompt.
6. Think of Prompts As Programming the Model
When writing your prompts, David says to think of your prompts as programming the model. You control the data inputs for your prompt. Giving the model too much data at one time could overwhelm the model’s ability to solve the task and longer prompts will take the model longer to process.
Thinking of prompting as programming gets at the engineering aspect of prompt engineering. You as the engineer have to think about how you design your prompts and the tradeoffs associated with what you enter in the model.
7. Design Prompts With Your Users’ Habits in Mind
If you are writing prompts that will be part of a tool that is used by other people, you need to anticipate the types of interactions the users will have with the model. When you are writing your prompt, think about all of the different types and quality of inputs your users will give to your prompt.
David talks about how when he helps customers with their system prompts, they run tests using well-thought-out and elaborate prompts to simulate user behavior. But, he says this often is not the case if you watch how the average person interacts with AI models. You need to anticipate users leaving out important details or writing incomplete sentences and be prepared in your prompt to handle this.
8. Check Your Prompt Outputs Often
If you are using trial and error to refine your prompt, make sure you keep an eye on how your changes affect your outputs. Zack says to keep checking your outputs, “if you aren’t reading the model outputs, you might not even notice that it’s making that mistake”.
As you experiment with changing your prompts, make sure your changes are correctly improving your output how you want and not negatively changing your outputs in another part of the model’s response. It sounds obvious, but if you tell the model to do something, verify that it is correct in the response.
9. Think About How the Model Will Interpret Your Prompt
When you are writing your prompts, you should think about how the model will interpret your prompts.
A lot of these prompt engineering tips are related to this one. As you write your prompt, you need to think about what context the model has about your task and what information you need to pass to the model to get it to interpret your prompt in a way that it can successfully complete your task and give you the desired output.
10. Ask the Model Directly What It Doesn’t Understand About the Task
A great way to improve the results you get from the model is to first ask what it doesn’t understand about your prompt.
Amanda likes to give her original prompt to the model and at the end of the prompt, tell the model not to try to complete the task, but instead tell her anything it doesn’t understand or what is not clear. This can help you make any needed updates to your prompt that would have otherwise left the model confused or unclear about your task.
11. Remove All Assumptions and Communicate the Task Clearly
This tip could be considered the golden rule of prompt engineering. It is such an important tip, all four experts echoed the same message.
When you write your prompts, you need to find a way to take all of the context in your head about the problem or task you have, and communicate it clearly to the model in such a way that it can generate the response you expect. Don’t assume the model knows anything about you or the context of your task. In your prompt, describe what you want the model to do and all the information it needs to complete the task as clearly as possible.
12. Ask the Model for Its Thought Process
Asking the model to show its work or explain its thought process is called Chain of Thought in prompt engineering. Some debate whether the model is truly thinking or not when you have it explain its thought process, but as David points out, we know that when you use reasoning, the results speak for themselves because the outputs are clearly better.
13. Use Different Styles of Prompts for Text Versus Images
Prompting for models to interpret images is often going to be very different from a text only prompt. Most models are generally better at completing tasks with text only prompts.
As David explains, when you are prompting with image inputs or to generate images from models, you need to approach how you write your prompts differently than you normally would.
14. Be Honest With the Model About Who You Are and What You Want To Accomplish
Don’t lie to the model in your prompts. This is Amanda’s advice because the models are becoming smarter and more aware of how we use them. She says to avoid using personas in your prompt like: “Pretend you are a teacher grading a quiz.”
Instead, she suggests you should be open and honest with the model. Tell the model you need help grading a quiz and clearly describe the task to the model so it can complete the task successfully.
Amanda makes the point that you can’t expect the model to do a good job at your task, if you lie to the model and give it a different task from the one you are really trying to accomplish.
15. Tell the Model What Situation It Is In
Be clear and honest to the model about what it is being used for and the context that it is in.
David’s advice is to directly tell the model the situation it is in. For example, tell the model it is being used as a support chatbot embedded in a popup window on the product website writing on behalf of the company.
It is okay to recognize in your prompts that the model is an LLM and not a human. But, be direct about what situation the model is in and what you are asking it to do.
16. Use Metaphors To Describe How To Accomplish a Task
Metaphors are a good way to explain to the model how to do something that might be otherwise difficult to put into words.
Zack pushed back on Amanda a little bit about using personas in your prompts. He argued that in the example of grading the quality of charts, telling the model to grade them like a high school teacher was an effective way to clearly communicate the task to the model.
17. Treat the Model Like a Smart Intern With No Prior Knowledge
Another way to imagine you are talking to the model as a person is to imagine you are talking to a new intern. Amanda gave this example of imagining your company hired a smart new temp worker who has no idea what your company does or what the job is they were hired to do.
When you are writing your prompts, imagine you have to explain the job to the model like you would explain it to the new intern.
18. List Edge Cases for Your Prompt Inputs
When you write your prompts, you need to think about edge cases outside of the typical case for your inputs. The models will typically follow your instructions exactly, according to Amanda. So, if your instructions don’t include any outlier examples, the model will handle them like any regular input.
She gives the example of a prompt that asks the model to rate the quality of a chart using a letter grade (A, B, C, D, or F). She says in your prompt, you should think about how to handle if a user uploads a non-chart image like a picture of a goat instead. These kind of edge cases can occur when your users interact with the model and you need to prepare for how to handle them.
19. Provide Examples To Guide the Model
One of the best ways to influence the results of the model are to give it examples of a successful response. This is known as Few Shot prompting where you give a few examples in your prompt of an input and the corresponding output you would expect.
Using examples will help the model to understand the format you want in your output and give you more consistent results.
20. Consider What Type of Output You Want From the Model
As you write your prompt, think carefully about what type of output you want from the model. If you want your results to be very rigid and look the same use a lot of examples in your prompt to guide the format of your outputs.
However, Amanda talks about how if you want more diverse results and outputs to look unique that don’t all follow the same format, use fewer examples in your prompt and avoid wording that could influence the model to stick to a format or pattern of output.
21. Read Other People’s Prompts
One of the best ways to write better prompts is to start reading other people’s prompts. Zack recommends that to become a better prompt engineer, study prompts from other good prompt engineers so you can learn how they get the best results from the model.
22. Read Your Prompt Like a Human Reading It for the First Time
If you have been working on the same prompt for a long time, it easy to have tunnel vision and think your instructions are clear to the model.
Amanda says to read your prompts as if you were another person reading it for the first time. This helps you strip away all of the context you already have in your head and can think more clearly about how the model is interpreting the task you are presenting in your prompt.
23. Try To Do Something You Think Is Impossible With the Model
A good way to learn more about what is possible with prompting is to push the limits of what you think you can do. David recommends this technique and says even if you fail to do what you attempted, you will still learn a ton about prompting and the model just by trying.
24. Avoid Focusing on Tricks and Hacks As Models Improve
With every new release, AI models are getting better. As they improve, Zack says you will need to rely less and less on prompt engineering hacks or trying to trick the model into doing something.
25. Don’t Dumb Down Your Prompts
To create a good prompt, you want to communicate clearly but that doesn’t mean you need to oversimplify your prompts. David remarks that you shouldn’t provide inaccurate information or dumb down your prompts. You can’t expect the model to successfully complete your task if you give it a watered-down version of the task.
26. Treat Prompting Like Teaching
Zack gave the analogy of treating prompting like teaching. You need to have empathy to understand the student (the model in this case) and consider what they are thinking. When they make mistakes, you need to think about why. To help the student improve, you have to change your way of communicating to how they understand and think about things to get the best results.
27. Use AI To Help You Write Better Prompts
Amanda and Zack both recommend using AI models to help you write better prompts. The training data for AI models now includes the concept of prompting, so models are aware of it and can help you write better prompts using the pattern of good prompts that they were trained on.
Final Thoughts
These prompt engineering tips are a great way to take your prompts from beginner to expert level. As you use this guide, try to not take any one tip as gospel. If you watch the complete video, you’ll notice that several times the experts disagree with each other about prompting techniques.
Use each prompting tip as you need it and discard ones that don’t work for you. As always, be willing to experiment and use trial and error until you find a technique that works best for you.
Remember that this field is always changing and techniques can quickly become outdated or unnecessary, while new and more effective methods are invented. Keep experimenting with the latest models and be willing to try new things.
To close out the video, Amanda delivered this short monologue that is an excellent summary of how to be an effective prompt engineer:
“I’m really used to this idea of, when I’m writing, thinking about the educated layperson, who’s really smart but doesn’t know anything about this topic. And that was just years and years of writing text in that form. I think it was really good for prompting because I was like, “Oh, I’m used to this. I have an educated layperson who doesn’t know anything about the topic.” And what I need to do is take extremely complex ideas and make them understand it. I don’t talk down to them. I’m not inaccurate, but I need to phrase things in such a way that it’s extremely clear what I mean.
Prompting felt very similar. And actually, the training techniques we use are fascinating. For example, when you say to someone, “Just take that thing you said and write it down,” I used to say that to students all the time. They’d write a paper, and I’d say, “I don’t quite get what you’re saying here. Can you just explain your argument to me?” They’d give me an incredibly cogent argument, and I’d be like, “Can you just take that and write it down?” And if they did, it was often a great essay.
It’s really interesting that there’s at least that similarity of taking things that are in your brain, analyzing them enough to feel like you fully understand them, and could take any person off the street—who’s a reasonable person—and externalize your brain into them. I feel like that’s the core of prompting.”