ChatGPT isn’t having a conversation with you! OpenAI Tutorial

Do you really think you’re having an intellectual conversation with artificial intelligence? You’re not. But the good part is that chatGPT is still way better than those annoying “customer support chatbots” companies wasted money on using. If you want to know what is happening behind the scenes of chatGPT then keep reading.

I’ll answer more questions for you such as “what is the meaning of context size”, “why are my AI responses degrading”, and more importantly, OpenAI. Since I’m technical and like engineering, I became determined to understand the magic behind the LLM boxes. Should you choose to stay with the magic of not knowing the technical aspect of chatGPT and LLMs, then don’t read this article. Otherwise, I will help you utilize chatGPT more effectively.

For a brief introduction to chatGPT and LLMs (large language models), they are massive datasets running as a neural network looking for patterns. The GPT in chatGPT means “generative pre-trained transformer” which is the interface between you, the user, and the AI system. You can interact with an LLM using text or audio but technically, it is just text that is accepted as input because audio becomes transcoded into text, then tokenized and processed by the LLM to provide you intelligent answers.

Should I keep you in suspense or just get to the point about chatGPT not being able to converse with a human? First, you should understand the input-output of these chat models. They accept a list of text messages as inputs and return a text message as output typically in JSON format. Initially, the requests to chatGPT are one-task requests so OpenAI called them “completions” as represented in this code:

messages = [
    {"role": "system", "content": "Describe the role chatGPT should be"},
    {"role": "user", "content": "User request goes here"}
]
response = openai.chat.completions.create(model="gpt-4o-mini", messages=messages)
print(response.choices[0].message.content)

The OpenAI method receives the message and selects the model shown in the parameter. If you sent another another message, chatGPT will treat it as new since it has no inherent memory. You have to program it in as we’ll see in the next step. Now you can see the reason you’re not having a conversation with chatGPT. It is because every request using this chat model is a new message and chatGPT has no recollection of your previous message. How do we solve this? Can we have an ongoing conversation with chatGPT? Yes and no.

As a precaution, the API method openai.chat.completions.create() is for OpenAI and will be different for other AI providers. However, “completions” means AI will attempt to complete your request. If you want to send subsequent messages continuing a topic you have to include your chat history with each request. Additional message increase the message history length therefore increasing the context size being sent to chatGPT.

For the next part, we pass the message history to the model which appears the same as before so we need to put this code block in a loop and accept user input via command line:

while True:
    user_input = input("You: ")
    if user_input.strip() in {"exit", "quit"}:
        break

    messages.append({"role": "user", "content": user_input})

     response = openai.chat.completions.create(
        model=MODEL,
        messages=messages,
        temperature=0.2
     )
     print(messages)
     reply = response.choices[0].message.content
     messages.append({"role": "assistant", "content": reply})

So with each user input, the messages will be appended and re-sent to the LLM. We see this when printing the message variable into the terminal. This means context size (token count) increases but the LLM becomes aware of the previous requests in order to respond with the best answer. Your requests should be separated manually if you’re switching topics or changing tasks. Only keep the conversation going if you want clarification of you previous request or need additional details.

Here’s a brief tutorial if you want to try it yourself running LLM locally on your personal computer. The API used in this example is v1 and may change by the time you’re reading this.

from openai import OpenAI

# Configure OpenAI client to use the local server
API_BASE = "http://127.0.0.1:1234/v1"
API_KEY = "not-needed"  # Placeholder, required even if not used
MODEL = "hermes-3-llama-3.2-3b"

openai = OpenAI(base_url=API_BASE, api_key=API_KEY)

# Initialize chat history with system prompt
messages = [
    {"role": "system", "content": "You are a helpful assistant."}
]

print("Chat with your local LLM. Type 'exit' to quit.")

while True:
    user_input = input("You: ")
    if user_input.strip() in {"exit", "quit"}:
        break

    messages.append({"role": "user", "content": user_input})

    try:
        response = openai.chat.completions.create(
            model=MODEL,
            messages=messages,
            temperature=0.2
        )

        reply = response.choices[0].message.content
        print(reply)
        messages.append({"role": "assistant", "content": reply})
    except Exception as e:
        print(f"Error: {e}"

)