Music as Tokens: How AI Models Predict the Next Note
Many modern AI music models work by turning music into a long sequence of small symbols called tokens and then learning to predict the next token, much as a text model predicts the next word. A token might mean "press middle C", "wait a short time", "set the loudness to soft" or "switch to the violin part". After training on many sequences, the model can continue a musical idea one token at a time. MuseNet, a research demo released by OpenAI in 2019, is one well-known public example of this approach.
What a music token looks like
Text is already made of separate symbols, but music is continuous in time. To feed it to a model, researchers usually start from MIDI-style data, which records when each key goes down, when it comes up and how hard it was struck. That data is then written as events.
- Note-on events: which pitch starts sounding.
- Note-off events: which pitch stops.
- Time-shift events: how long to wait before the next event.
- Velocity events: how loud the next notes should be.
- Instrument or style tags: which part or genre the sequence belongs to.
A short piano phrase can easily become hundreds of tokens, because every press, release and pause is spelled out.
How the prediction loop works
The model reads all the tokens so far and gives a probability to every possible next token. The system picks one, adds it to the sequence and repeats. A setting often called "temperature" controls how adventurous the choice is: low values stick to the most likely options, while higher values allow more surprising notes.
Why long pieces are difficult
Each prediction depends on what came before. Models that use a mechanism called attention can look back across long stretches, which helps them repeat a theme later in a piece. Even so, keeping a convincing structure over several minutes remains a known challenge in AI music research.
What tokenised music reveals to learners
Seeing music as tokens is surprisingly practical for anyone who wants to learn piano. It highlights details that beginners often ignore.
- Releases count: a note-off is as much a musical event as a note-on, which is why clean legato and staccato matter.
- Silence has length: time-shift tokens remind us that rests are measured, not empty.
- Loudness is information: velocity changes are part of the content, not decoration.
- Patterns repeat: models learn quickly because music reuses rhythms and intervals, and players can use the same repetition to memorise faster.
How feedback apps use similar data
A piano learning app that listens to your playing also works with events: which key, when and how loud. Instead of predicting the next note, it compares your events with the score. A late note-on shows up as a timing error, a missing note-off may show as an overlapping sound, and a velocity that is too strong appears as an uneven dynamic. The same data format that lets a model generate music lets a tutor explain what happened in your performance.
Limits worth remembering
Prediction models learn from data that people created, so questions about training material, credit and consent are part of the AI music conversation. Generated passages can sound fluent while lacking the intention a human performer brings. For learners, the healthy view is to treat these systems as interesting tools and study partners, while the musical decisions remain yours.
Try thinking in events yourself
Pick a four-bar phrase you already know and describe it as a list of events in plain words: which key goes down, how long it lasts, when it is released and how loud it is. You will probably notice details you usually skip, such as a note held too long or a rest you shortened without realising. Then play the phrase again, paying attention only to releases and silences. Many learners find their playing sounds cleaner straight away. This small habit mirrors how both generative models and practice apps "see" music, and it turns an abstract AI idea into a concrete listening exercise you can repeat with any piece.
FAQ
Q: What is a token in AI music?
A: A small symbol representing one event, such as a note starting, a note stopping, a pause or a loudness change.
Q: What was MuseNet?
A: A 2019 research demo from OpenAI that generated multi-instrument music by predicting the next token in a sequence.
Q: What does temperature mean in music generation?
A: A setting that controls how predictable or surprising the next chosen note is.
Q: Why should a piano student care about tokens?
A: Because they show that releases, rests and loudness are real musical information, just like the notes themselves.
Practise with feedback that listens to every note in TIGA ALPHA
TIGA ALPHA is an AI piano teaching app built by a Thai team, with lessons in Thai, English and Mandarin. Its CEO, Teacher Aum, has 20 years of experience in piano teaching, and that music teaching background shapes every lesson, while piano teaching with AI gives you feedback each time you play piano. Use it as an online piano lessons app for short daily sessions, follow it as a complete AI online piano course, or combine it with a piano course or piano learning course led by a teacher. Whether you want to learn piano from zero, learn piano online after a long break, learn music alongside your children or simply play music for fun, this online piano teaching application keeps your next step clear. New users get a 7-day free Premium trial, then 799 THB/month. Open the piano teaching app below and pick a first lesson that fits your week.
- Download and start: TIGA ALPHA landing page
- In-person lessons in Thailand: TIGA Studio
- About the company: tigaalpha.com
Knowing how machines read music can make you a sharper reader of music yourself.