Token

A token is a unit of text that a language model uses when processing and generating language. It can be a word, part of a word, punctuation or another character pattern, depending on the tokenizer and model. It is not the same as a human word, sentence or unit of meaning.

The term matters because it makes the hidden constraint of an AI conversation more visible. Long prompts, long documents and long conversations are handled through token limits, not through human-like reading or memory.

Why it matters

Tokens affect how much information can be included in a request, how much a model response can cost and how quickly a conversation can approach a context limit. They also shape user expectations. A person may feel that a system is reading a whole document in the ordinary human sense, while the model is operating over a technical representation of the input.

Understanding tokens helps readers see why AI work often needs structure: shorter instructions, clear document sections, summaries, checkpoints and verification.

Human-AI angle

Tokens connect a technical layer with a psychological experience. The interface may feel like a conversation, but the system is constrained by units that users rarely see. That gap can lead to misplaced confidence about what the model has taken into account.

For Prompted Psyche, token is a foundational term because it supports clearer thinking about context, cost, model output and the limits of what users call “memory”.

Related concepts

• AI literacy
• Large Language Model (LLM)
• Context window
• Model output
• Human-AI Interaction

Sources and context

• OpenAI. (n.d.). Key concepts. OpenAI API documentation. Retrieved July 2, 2026, from https://developers.openai.com/api/docs/concepts
• OpenAI. (n.d.). What are tokens and how to count them? OpenAI Help Center. Retrieved July 2, 2026, from https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them

Related concepts

Continue through the library

• Context windowThe amount of information a language model can reference in a given interaction, distinct from durable memory or understanding.
• Human-AI InteractionThe relationship between a person and an AI system: how people ask, interpret, trust and integrate model outputs into work and decisions.
• Mental modelA user's simplified understanding of how a system works, shaping expectations, trust and delegation in AI use.

© 2026 Feliks Mamczur / Prompted Psyche. All rights reserved. Short quotations are permitted only with attribution to the author, the title and a link to the source.