In this blog we will go over the basics of prompt engineering 🛠️ & in the end you will be able to construct your own prompts live 🚧 [scroll to the bottom & try it out]!
To keep things simple, Prompt Engineering is the idea of crafting the best possible system prompt for your chatbot to get an intended behaviour as a result - For Example: if you want your chatbot to only respond in shakespears prose-like language, explicitly specifying that in the system prompt will make the chatbot behave exactly so. Another example is telling the chatbot a specific script it must follow, this would be especially useful for extracting lead information or dealing with leads from a voicebot on a call. Here is an example of an extremely simple system prompt:
SYSTEM:
You are a Shakespearean Prose-Responding Chatbot. You must only respond in Shakesperean Prose-Like Language regardless of user input.
Most prompts are usually just 1 or 2 paragraphs of text. This is good for basic tasks like data extraction, cleaning, & classification. However, for much more complex tasks like customer support, the only option is to have structured prompts. Structured prompts are usually written in markdown where each section has a triple hashtag ### for its section title. Here at Splutter AI we have adopted this structured prompt format because it fits almost all use cases:
The benefit of this structured prompt format is that explicit details can be added to the appropriate section as your use case expands. For example at Splutter AI, we found that users would use the Tagalog language to use our chatbot for nefarious reasons against its intended purpose. So we added this in examples: "Q: Do you speak Tagalog? A: No, we speak all languages except for Tagalog!" - On top of this we added measured to prevent abuse that you will see in the sections below. Here is an example of a structured prompt:
This is a very overkill example for the task of shakesperean prose-like responses, however, if you were to test this prompt out on a chatbot, you would notice that responses would be more consistent regardless of the input, thanks to structured prompts! In the next section we will look at real-life examples of tuning our prompt to get the intended behaviour we want!
There are a few strategies we can use to improve the output of our chatbots via prompt engineering. You can think of this as "thought algorithms" without sounded too technical, because its actually pretty easy for anyone to understand & grasp! With confidence, lets get started.
The main idea behind CoT (Chain of Thought) is to prompt the chabot to "think" about the problem & consider avenues before giving a final response! For example consider these two prompts:
SYSTEM A:
You are a Website Chatbot. When the user speaks to you, recommend a product.
SYSTEM B:
You are a Website Chatbot. When the user speaks to you, consider their needs & requirements, think about what is potentially useful for them, then recommend a product.
The System B prompt is better, but why? In the System B prompt, the difference is that the chatbot will give a response starting with first the considerations (sequential steps) then finally recommend a product! This would give more detailed/personalized responses for users. This is how the chatbot processes would look like:
Naturally, the in-between steps help guide the output making it the first tool in our Prompt Engineering Strategy. We can think of each step as a "thought" thus the final result is a chain of thoughts! This is good - but we can take things a step further with: "self-consistency":
SYSTEM C:
You are a Website Chatbot. When the user speaks to you, first consider their needs & requirements choosing the best ones to focus on,then think about which product is potentially useful for them weighing pros & cons & choose the best one, then finally recommend a product.
The idea is that we consider the best possible "thought" each step before producing our final output - that is the chatbot is being "self-consistent" each step making sure it has the best possible "thought" each step until it reaches an output!
CoT-SC (Chain of Thought with Self Consistency) is the best possible prompt we have made so far! Lets look at an example below then move on to the next method!
Now if you read the interaction above, you will notice that the chatbot recommends products & the price at the same time. Usually you would want the chatbot to list possible products with their features & only present price when the user asks - what would have happened if we had too many options? This is left as an exercise for the reader: modify the prompt to guide the user feature-first instead of simply listing the feature & prices!
Now that we had a look at CoT (Chain of Thought) its time to consider expanding on the idea by considering "possibilities" on top of CoT. What do I mean by that? Well, instead of having 1 sequenctial steps to take, what if we had multiple possible steps & needed to consider a few/some/most of them? This is where Tree of Thought (ToT) comes in. ToT is a method to help the chatbot consider multiple possibilities & choose the best one. There are two types of ToT:
While it looks similar to CoT-SC, there is a big difference - With CoT-SC we choose the best "thought" & add that to our chain, but with ToT (Tree of Thought) we generate all chains then choose the best chain of thoughts - here is a graphic below to help visualize the methods we have learned so far:
Alright, now the important part, how do we implement ToT (tree of thought) in our prompting? Well, we could use the same prompt as CoT-SC & add some additional steps to it but to keep the content fresh, here is a unique way:
SYSTEM D:
You are a Website Chatbot. Imagine three different experts are helping you evaluate the user's needs and product options.
1. **Expert 1**: Focus on understanding and prioritizing the user’s needs and requirements. This expert will consider all the possibilities and weigh them based on the user's context, narrowing down the top few needs.
2. **Expert 2**: Independently assess each product based on the user's requirements. This expert will analyze the products in full detail, identifying their strengths and weaknesses relevant to the user’s needs.
3. **Expert 3**: Compare the findings of the first two experts, contrasting the user’s needs with the product features. This expert will highlight which products align most closely with the user's top requirements.
If any expert finds their initial thought process flawed at any point, they will adjust their reasoning or leave the evaluation. After all three experts complete their assessments, select and recommend the product that best matches the user's needs.
The main difference is that we are now considering multiple possible steps at each step of our CoT chain - that is we are generating multiple chains of thoughts to choose the best one - but it gets even crazier: we can now implement more complex "thought" processing such as:
Now, the moment we have all been waiting for, the prompt example & the next section:
Now if you read the interaction above, you will notice the chatbot doing some calculations! Math is not easy for LLM Chatbots but rough calculations are possible - more advanced calculations require the chatbot to write python, execute the code, then use the output - but that is a story for another day!
If you made it this far, the hard part is done! Now we need to explain "shots" and what zero-shot & few shot mean (its basically how many examples given to the chatbot via prompt):
Here is a simple prompt w/ few shot:
SYSTEM E:
You are a website chatbot. Your job is to answer questions about chocolate. Here are a few examples:
Q. What is the best cocoe beans for chocolate making?
A. Criolla beans are considered "noble" beans & have a less bittery/acidic taste than classic cocoe beans, usually used in high-end premium chocolate.
Q. What is the best chocolate for a chocolate cake?
A. Dark chocolate is the best choice for a chocolate cake, as it has a rich, full flavor and a smooth, creamy texture. It is also less bitter than milk chocolate.
Use these examples when answering the user's question. If a question arises where you are unsure, answer with only accurate information to the best of your ability & do not hallicunate.
Adding more examples can help or hinder chatbot performance, but usually commonly asked questions can be placed in the system prompt that way users are not required to contact support for FAQs. Thats a wrap! We have covered the basics of Prompt Engineering so now lets have a look at our last example by bringing it all together!
We have a task as follows: Create a website chatbot for a construction company for building ADUs (accessory dwelling unit), Renovating via Plumbing/Electrical or Desining & Building from Scrath. On top of being able to answer questions, the chatbot should capture the lead so we can follow-up as well as identify exactly what the user is looking for - making our discovery call with them more streamline. Here is the prompt we came up with:
Do you think we can improve on this prompt? If so, then why don't you try it out yourself! Below you can type out any instruction & the chatbot will use it to respond!
Now to complete this blog post, we setup a custom endpoint where you can test out your prompts below! (Login Required)
Congratulations! 🎉 You made it to the end! You even got to craft your own perfect prompt! If you liked this blog, or want to know more about AI Chatbots, feel free to sign-up for Splutter AI to create your first AI Chatbot! 🤖 Good Luck!
Need a Plan? Try our Hobby Plan!