Prompting Engineering: Notes from the DeepLearning.ai/OpenAI ChatGPT crash course

The recent popularity of large language models (LLM) such as GPT, has empowered and made developers aware of building applications by leveraging interactions with AI models. Part of this process is prompt engineering.

Constructing effective prompts is seen as a crucial engineering skill in building these applications.

This article shares summary notes about the free course “GPT Prompt Engineering for Developers” offered by OpenAI and DeepLearning.AI and presented by Isa Fulford and Andrew Ng, whom I would like to thank for creating a free, simple, concise and informative material.

Material and screenshots are taken as note part of the above course, not my own.

Introduction

Large Language Models (LLM)s can be of two types:

1. Base LLM. It is a large model trained on internet data plus other sources. It predicts the next most likely word to follow. E.g. “What is capital of France?” may return “What is France’ largest city?”.
2. Instruction Tuned LLM. It is a large model trained to follow instruction. It is a training mix of Base LLM , fine-tuned on making the model to follow and be good at instructions with a final Reinforcement Learning with Human Feedback (RLHF) technique, to make the system helpful, honest and harmless. E.g. “What is the capital of France?” will return “The capital of France is Paris”.

The course focused on best practices for the latter LLM.

Prerequisites

A prerequisite to run ChatGPT prompts, is to possess an OpenAi API Key.

Where do I find my Secret API Key? | OpenAI Help Center

The course used Python as primary language.

First, you need to install OpenAI Python library

!pip install openai

As next step, you have to configure the OPENAI_API_KEY api key either as environment variable or as openai.api_key attribute.

Below code loads the OpenAI key:

Below code is an helper function that leverages the chat completion endpoint to get a response out of a specific model:

The course provides also a Jupyter notebook where you can try prompts out directly.

Prompting Principles

Prompting principles help ensure that the language model generates coherent and relevant text that aligns with the user’s intentions.

The course introduces two of prompting principles, each implemented via tactics, and they are:

• Principle 1: Write clear and specific instructions
• Principle 2: Give the model time to “think”

Principle 1: Write clear and specific instructions

• Writing concise prompts reduces the chances of irrelevant or incorrect answers.
• A clear prompt is not the same as short one.

Tactics

• Use delimiters to indicate distinct parts of the input, such as triple quotes (”””), triple backticks (```), triple dashes (— -), angle brackets < > or xml tags <tag></tag>.

• Ask for structured output such as XML or JSON.

• Ask model to check whether conditions are satisfied, by first checking assumptions required to do the task as a basis for the model to provide an answer.

• “Few-shot” prompting: Give successful examples of completing task, then ask model to perform the task. Given an example of how a grandparent would provide a reply, the model is able to answer by mimicking the grandparent tone and style.

Principle 2: Give the model time to think

• Reframe the query to request a chain or series of relevant reasoning before the model provides its final answer.
• Given a complex task, the model may make up a guess if a user provides a prompt in a short number of time and with a small number of words.
• Model thinking longer about the problem = spend more computational effort on a task.

Tactics

• Specify the steps required to complete a task

• Ask for output in a specified format

• Instruct the model to work out its own solution before rushing to a conclusion

In reality, the student answer was incorrect. We then ask the model to work out a solution first based on that context.

Below the model was able to assess that indeed, student answer was incorrect.

Model limitations

• Model does not know the boundaries, as it could make statements that sound plausible but are not true, yet pretty realistic.

• Reducing hallucinations. In above example, Boie is a fictional company, but the model response sounded very plausible. The idea it so reduce hallucinations by first find relevant information, then answer the question based on relevant information.

Iterative Prompt Development

• Iteratively make prompts better by refinement cycles.
• Iterate through 4 phases: Idea -> Implementation -> Experiment result -> Error Analysis and repeat.

• Let’s look at a fact sheet example:

Answer may be too long. You can influence by changing prompt , e.g. making it shorter, at most 50 words:

Or up to 3 sentences:

From specifications to product HTML page example:

To summarize the iterative process:

• Try something
• Analyze whether the result does not give what you want
• Clarify instructions, give more time to think
• Refine prompts with a batch of examples

According to Andrew Ng:

The key to be an effective prompt engineer is not about knowing the perfect prompt, but having a good process to develop prompts that are effective for your case

Summarize Text

An important use case of LLM models is the ability to summarize information.

Summarizing can also be targeted against specific audience and intents such as to give feedback to Shipping department:

Or to give feedback to pricing department:

Extract information = summarize:

A good example is to summarize reviews:

Inferences

Models can infer a sentiment of tag or label, based of a piece of text. It is up to the prompt to make sure how useful this information can be.

For example below detects the user sentiment of a lamp review:

Inferring to be more concise, to give a straight, categorized and precise result:

Inferring can be very useful to find one or more labels:

Or just inferring by yes/no answer:

Extract rich information from customer review:

Instead of calling completion 3 times, just create a good prompt:

Given a long piece of text, infer topics:

Inferring to find info, based on contract. Zero-shot learning is typical as no data can be given, in certain scenarios:

Transforming

Transforming activities involved translation of different languages, contextual tone setting, grammatical correction, and data type manipulation, e.g. inputting HTML, outputting JSON.

• Translation

• Universal translator example:

• Tone setting. It is fundamental to instruct the model on the tone (informal, professional, etc..) for a specific audience (users, internal deparments, etc..) or sentiment you want to get an answer with.

• Transform Input/Output formats

• Spell and grammar checking:

Augment it with specific style guides, such as proof reading with specific style:

Expanding

Take a short piece of text (set of instructions or list of topics) and have LLM generate a longer piece of text such as email or essay. Usage: brainstorming, but important to be careful to be used with responsibility.

• Temperature is parameter that allows to change the variety of model response. It is the degree of inspiration or randomness of the model. The higher the temperature, the mode creative the model’s answer will be. The lower, the more predictable and “safe”. Morever, a high temperature will provide more often different answers than a lower one, where answers will tend to be very consistent.

Example with Temperature = 0.7

Running the same prompt with high temperature, will give a different reply:

Chatbots

ChatGPT is a custom chatbot implemented with OpenAI LLM. However, companies and individuals can also use LLM to build their own custom chatbot, based on their own domain.

Below example extends the initial getCompletion helper by allowing to pass massages with the tuple role and content. This structure helps the assistant to understand roles and be more specific in its answers.

The additional helper can be seen below:

There are 3 main roles:

• User => end user, you
• Assistant => ChatGPT, model
• System => Acts as a whisperer to assistant’s ear and guiding its responses without the user being aware of it,

Below examples show how prompts structured with role/content payload can make chatbot applications easier to be understood by models (different replies as temperature is high, set to 1):

This structure allows to provide some default context and chat interchange so that chatbot can be pre-instructed:

This final part showed how this chatbot implementation can be visually shown by using Python panel library:

Conclusions

The GPT Prompt Engineering for Developers course offered by OpenAI and DeepLearning.AI provides valuable insights into prompt engineering principles, including writing clear and specific instructions, giving the model time to think, and iteratively refining prompts. By mastering these principles, developers can create powerful and effective applications that meet the needs of their users.

The next step after this course will be to apply the principles discussed in the course to build specific applications.