Tagging
This guide shows you how to retrieve tagging results of an analyzed track, for example its moods, genres, BPM or key, and how to use the most common fields in your application.
What Tagging is
Tagging is the Cyanite product that describes your music. It's made up of
a set of individual model outputs, each describing one aspect of a track,
such as its main genres (MainGenreV2), its moods (MoodSimpleV2), its BPM
(BpmV2) or its key (KeyV2). Your account has access to a specific set of
them.
You don't request an analysis per model output: every track you upload is
analyzed with all the model outputs your account has. You then retrieve the
ones you need from one endpoint, GET /v1/library-tracks/{id}/models.
In the API, the tags of a track are delivered as model outputs, the same form in which AI Music Detection and Segmentation deliver their results. Most model outputs belong to Tagging.
What model outputs are, how they relate to Tagging, which ones exist and how to interpret scores and confidences is explained in Understanding Model Outputs. For the fields of every model output, see Available model outputs.
AI Music Detection is delivered through the
same endpoint, as the AiMusicDetectionV1 model output, but it's a separate
product. You can have it with or without Tagging.
Segmentation is delivered through the same
endpoint too, as the SegmentationV1 model output. It's an add-on to Tagging
that has to be enabled separately.
Prerequisites
- A Cyanite API key, sent in the
x-api-keyheader exactly as in the Quick Start Guide. - The Library Track ID (
libtr_...) of an uploaded and analyzed track. TheanalysisJob/finishedwebhook from the Quick Start Guide tells you when a track is ready. - Access to the model outputs you want to request. If you're unsure which model outputs are part of your Tagging plan, contact Cyanite.
Fetch model outputs for a track
Request the model outputs you need by passing one model query parameter per
model output version:
GET https://rest-api.cyanite.ai/v1/library-tracks/{id}/models?model=MoodSimpleV2&model=BpmV2
API reference: Get Model Outputs
- Typescript
- PHP
- Python
const trackId = "libtr_xxxxxxxxxxxxxxxxxxxxxx";
const params = new URLSearchParams();
params.append("model", "MoodSimpleV2");
params.append("model", "BpmV2");
const response = await fetch(
`https://rest-api.cyanite.ai/v1/library-tracks/${trackId}/models?${params}`,
{
headers: {
"x-api-key": process.env.CYANITE_API_KEY,
},
},
);
const { items } = await response.json();
<?php
use Illuminate\Support\Facades\Http;
$trackId = 'libtr_xxxxxxxxxxxxxxxxxxxxxx';
$response = Http::withHeaders([
'x-api-key' => env('CYANITE_API_KEY'),
])->get("https://rest-api.cyanite.ai/v1/library-tracks/{$trackId}/models?model=MoodSimpleV2&model=BpmV2");
$items = $response->json('items');
import json
import os
import urllib.request
from urllib.parse import urlencode
api_key = os.environ.get("CYANITE_API_KEY")
track_id = "libtr_xxxxxxxxxxxxxxxxxxxxxx"
query = urlencode([("model", "MoodSimpleV2"), ("model", "BpmV2")])
req = urllib.request.Request(
f"https://rest-api.cyanite.ai/v1/library-tracks/{track_id}/models?{query}",
headers={"x-api-key": api_key},
)
with urllib.request.urlopen(req) as response:
items = json.loads(response.read().decode("utf-8"))["items"]
The response contains one item per requested model output (shortened here):
{
"items": [
{
"version": "MoodSimpleV2",
"scores": { "energetic": 0.6145, "dark": 0.4347, "calm": 0.2037 },
"tags": ["energetic", "dark"],
"segments": {
"timestampsSeconds": [5.0, 15.0, 25.0],
"values": { "energetic": [0.5873, 0.635, 0.2871] }
}
},
{
"version": "BpmV2",
"tag": 128,
"confidences": {
"modelCertainty": 0.97,
"predictionStability": 0.94,
"confidence": 0.94
},
"segments": {
"timestampsSeconds": [5.0, 15.0, 25.0],
"values": [128, 128, 127]
}
}
]
}
The order of the items isn't guaranteed. Pick each output by its version, as
shown in the next section.
Pick an output by its version
Every item has a version field that tells you which model output it is and
which fields it has. Look up the output you need by its version instead of by
its position in the list:
- Typescript
- PHP
- Python
const mood = items.find((item) => item.version === "MoodSimpleV2");
const bpm = items.find((item) => item.version === "BpmV2");
$outputs = collect($items)->keyBy('version');
$mood = $outputs->get('MoodSimpleV2');
$bpm = $outputs->get('BpmV2');
outputs = {item["version"]: item for item in items}
mood = outputs.get("MoodSimpleV2")
bpm = outputs.get("BpmV2")
If an output you requested is missing, the track probably hasn't finished analysis yet. See When a model output is missing.
Show the dominant tags
For taxonomy-based outputs such as MoodSimpleV2, MainGenreV2 or
InstrumentsV2, tags holds the labels that describe the track best. Use it
for labels in your UI:
- Typescript
- PHP
- Python
const moodLabels = mood?.tags ?? []; // ["energetic", "dark"]
$moodLabels = $mood['tags'] ?? []; // ['energetic', 'dark']
mood_labels = (mood or {}).get("tags") or [] # ["energetic", "dark"]
To rank all taxonomy values instead, sort the scores map:
- Typescript
- PHP
- Python
const rankedMoods = Object.entries(mood?.scores ?? {}).sort(
([, a], [, b]) => b - a,
); // [["energetic", 0.6145], ["dark", 0.4347], ...]
$rankedMoods = $mood['scores'] ?? [];
arsort($rankedMoods); // ['energetic' => 0.6145, 'dark' => 0.4347, ...]
ranked_moods = sorted(
(mood or {}).get("scores", {}).items(),
key=lambda entry: entry[1],
reverse=True,
) # [("energetic", 0.6145), ("dark", 0.4347), ...]
InstrumentsV2 has no scores map. Its presence map says how much of the
track each instrument plays in (absent, partially, frequently or
throughout):
- Typescript
- PHP
- Python
const instruments = items.find((item) => item.version === "InstrumentsV2");
const instrumentsThroughout = Object.entries(instruments?.presence ?? {})
.filter(([, presence]) => presence === "throughout")
.map(([instrument]) => instrument); // ["drums", "bass"]
$instruments = $outputs->get('InstrumentsV2');
$instrumentsThroughout = array_keys(array_filter(
$instruments['presence'] ?? [],
fn ($presence) => $presence === 'throughout',
)); // ['drums', 'bass']
instruments = outputs.get("InstrumentsV2") or {}
instruments_throughout = [
instrument
for instrument, presence in instruments.get("presence", {}).items()
if presence == "throughout"
] # ["drums", "bass"]
The possible values of each taxonomy are linked from Available model outputs.
Use a musical parameter only when it's confident
Musical parameters such as BpmV2, KeyV2 and TimeSignatureV2 give one
track-level value in tag and a confidences object. A value above 0.9 in
confidences.confidence means a single value describes the whole track well.
Below that, the parameter changes during the track or is ambiguous, so consider
showing the segment values or no value at all:
- Typescript
- PHP
- Python
const bpmLabel =
bpm && bpm.confidences.confidence >= 0.9
? `${bpm.tag} BPM`
: "Variable tempo";
$bpmLabel = $bpm && $bpm['confidences']['confidence'] >= 0.9
? "{$bpm['tag']} BPM"
: 'Variable tempo';
bpm_label = (
f"{bpm['tag']} BPM"
if bpm and bpm["confidences"]["confidence"] >= 0.9
else "Variable tempo"
)
Find where a mood is strongest
segments shows how an output changes over the course of the track. For a
taxonomy-based output, segments.values maps each taxonomy value to one score
per entry in segments.timestampsSeconds. To find where a mood peaks, find the
highest score and look up its timestamp:
- Typescript
- PHP
- Python
const energetic = mood.segments.values.energetic;
const peakIndex = energetic.indexOf(Math.max(...energetic));
const peakStartsAt = mood.segments.timestampsSeconds[peakIndex]; // seconds
$energetic = $mood['segments']['values']['energetic'];
$peakIndex = array_search(max($energetic), $energetic);
$peakStartsAt = $mood['segments']['timestampsSeconds'][$peakIndex]; // seconds
energetic = mood["segments"]["values"]["energetic"]
peak_index = energetic.index(max(energetic))
peak_starts_at = mood["segments"]["timestampsSeconds"][peak_index] # seconds
Start a preview at the most representative part
Request RepresentativeSegmentV2 to get the part of the track that represents
it best, for example as the starting point of a preview player:
{
"version": "RepresentativeSegmentV2",
"startSeconds": 42.0,
"endSeconds": 72.0
}
Handle outputs that can be empty
Some outputs are null when the model has no answer for a track. For example,
SubgenreV2 has null for scores, tags and segments when the track
doesn't fit any subgenre with enough confidence:
- Typescript
- PHP
- Python
const subgenre = items.find((item) => item.version === "SubgenreV2");
const subgenreLabels = subgenre?.tags ?? [];
$subgenre = $outputs->get('SubgenreV2');
$subgenreLabels = $subgenre['tags'] ?? [];
subgenre = outputs.get("SubgenreV2") or {}
subgenre_labels = subgenre.get("tags") or []
Check the schema of each model output in the API Reference, linked from
Available model outputs, for fields that can be null.
Troubleshooting
| Status | Cause | What to do |
|---|---|---|
401 | Invalid API key, or a requested model isn't enabled for your account. | Check x-api-key. The error's detail names the models you can't access. |
404 | The track doesn't exist or doesn't belong to your account. | Check the Library Track ID. |
422 | No model parameter, or an unknown model output version. | Use the version names from the list of model outputs, e.g. BpmV2. |
429 | Rate limit exceeded. | Wait for retry-after seconds, see Rate limits. |
An empty items list usually means the track hasn't finished analysis yet.
Next steps
- Understanding Model Outputs: what model outputs are, how they relate to Tagging, versioning, track-level vs segment-level data, scores, tags and confidence.
- Available model outputs: every public model output, with links to its fields and taxonomy values.
- Filter by metadata: use model outputs to narrow down a search.
- API Reference: the complete request and response schema.