"ChatGPT" from the Command Line

After describing how to access the OpenAI API from Python, I’d like to dig a bit deeper into one particular example application that looks like it could be useful. In this post, I’m going to explain the chat completion endpoint a little bit more, and I’m going to augment the small example program from the linked article with one crucial thing that ChatGPT does extra, which is to allow for a continuing conversation instead of individual questions.

The messages Parameter

For easy reference, this is the example program again:

import openai
from os import environ
 
openai.api_key = environ['OPENAI_API_KEY']
 
messages = [{'role': 'system', 'content': 'You are a helpful assistant.'}]
message = input('message> ')
messages.append(dict(role='user', content=message))
 
response = openai.ChatCompletion.create(
    model='gpt-3.5-turbo',
    messages=messages)
 
print(response.choices[0].message['content'])

The thing that makes a continuing conversation work on ChatGPT, is that, each time you ask a new question, the complete chat up till then is fed into the model again. I don’t know how OpenAI does that in practice for ChatGPT, but it is easy to do through the API.

To see how to do that, let’s look at the messages parameter in the above script. When the call to openai.ChatCompletion.create() is made, the contents of messages could be something like the following.

[{'role': 'system', 'content': 'You are a helpful assistant.'},
 {'role': 'user', 'content': 'What is the smallest prime number?'}]

This sends two pieces of information to the model,

• the system message, which describes how the assistant should behave. In the example I simply use “You are a helpful assistant.”, but it can be much more elaborate;
• the user message, which is your message.

A third type of message that we need to create a conversation is assistant. With that, you can feed back previous responses to build up a conversation. Hence, if you want to continue the conversation with a new question, you need to add both the previous answer (I got “The smallest prime number is 2.”) and that new question to messages:

[{'role': 'system', 'content': 'You are a helpful assistant.'},
 {'role': 'user', 'content': 'What is the smallest prime number?'},
 {'role': 'assistant', 'content': 'The smallest prime number is 2.'},
 {'role': 'user', 'content': 'And what is the largest one?'}]

Note that the question “And what is the largest one?, does not make sense without the context of the previous question. The reply that I got was

There is no largest prime number, as the set of prime numbers is infinite. However, the largest
known prime number as of now is 2^82,589,933 − 1, a number with 24,862,048 digits.

This reply is clearly based on the whole conversation, and not only on the question that was added last.

Python Code

Below is a very basic script that keeps adding your input and the reply from the model to messages.

import openai
from os import environ
 
class Chat:
    def __init__(self, model, temperature):
        self.model = model
        self.temperature = temperature
        self.messages = [
            {'role': 'system', 'content': 'You are a helpful assistant.'}]
 
    def run(self):
        print('Leave an empty message (press return) to end this chat.')
        while True:
            message = input('you> ')
            if message == '':
                return
            self.messages.append(dict(role='user', content=message))
            response = openai.ChatCompletion.create(
                model=self.model,
                messages=self.messages,
                temperature=self.temperature)
            answer = response.choices[0].message['content']
            print('gpt>', answer)
            self.messages.append(dict(role='assistant', content=answer))
 
openai.api_key = environ['OPENAI_API_KEY']
 
chat = Chat('gpt-3.5-turbo', 1)
chat.run()

There are obviously a lot of improvements and additions that could be done, but I’ll leave that for you. Some ideas are the following.

• When exiting a chat, save it to a file.
• Before saving the chat, automatically add one last message like “Create a short title for this chat.”, run it through the model, and use the result in the name of the file.
• Play with the temperature, or with other parameters of openai.ChatCompletion.create().

Also keep in mind the so-called token limit of 4096 for the model gpt-3.5-turbo, or of any other model that you use. For English text, one token is about 4 characters or 0.75 words. ChatGPT automatically drops the first part of the conversation if it becomes too long, but you have to do that yourself if you use the API.

Few-Shot Learning

You can also use a conversation in messages to show the model what you want it to do, instead of explaining it. For example, you can explain that you want a number written out in letters as follows (using the above script).

Leave an empty message (press return) to end this chat.
you> Write the number 5 in letters. 
gpt> The number 5 is written as "five" in letters.

However, you can also use the messages parameter to provide a few examples of what you want, like this:

[{'role': 'system', 'content': 'You are a helpful assistant.'},
 {'role': 'user', 'content': '4'}, {'role': 'assistant', 'content': 'four'},
 {'role': 'user', 'content': '8'}, {'role': 'assistant', 'content': 'eight'},
 {'role': 'user', 'content': '5'}]

This resulted in the answer:

five

This technique is called few-shot learning. This also worked with only a single example (“4”→“four”), by the way (which is then one-shot learning).

Have fun playing with the script!