Skip to main content

Tagging

This guide shows you how to retrieve tagging results of an analyzed track, for example its moods, genres, BPM or key, and how to use the most common fields in your application.


What Tagging is​

Tagging is the Cyanite product that describes your music. It's made up of a set of individual model outputs, each describing one aspect of a track, such as its main genres (MainGenreV2), its moods (MoodSimpleV2), its BPM (BpmV2) or its key (KeyV2). Your account has access to a specific set of them.

You don't request an analysis per model output: every track you upload is analyzed with all the model outputs your account has. You then retrieve the ones you need from one endpoint, GET /v1/library-tracks/{id}/models.

Tags are model outputs

In the API, the tags of a track are delivered as model outputs, the same form in which AI Music Detection and Segmentation deliver their results. Most model outputs belong to Tagging.

What model outputs are, how they relate to Tagging, which ones exist and how to interpret scores and confidences is explained in Understanding Model Outputs. For the fields of every model output, see Available model outputs.

note

AI Music Detection is delivered through the same endpoint, as the AiMusicDetectionV1 model output, but it's a separate product. You can have it with or without Tagging.

note

Segmentation is delivered through the same endpoint too, as the SegmentationV1 model output. It's an add-on to Tagging that has to be enabled separately.


Prerequisites​

  • A Cyanite API key, sent in the x-api-key header exactly as in the Quick Start Guide.
  • The Library Track ID (libtr_...) of an uploaded and analyzed track. The analysisJob/finished webhook from the Quick Start Guide tells you when a track is ready.
  • Access to the model outputs you want to request. If you're unsure which model outputs are part of your Tagging plan, contact Cyanite.

Fetch model outputs for a track​

Request the model outputs you need by passing one model query parameter per model output version:

GET https://rest-api.cyanite.ai/v1/library-tracks/{id}/models?model=MoodSimpleV2&model=BpmV2

API reference: Get Model Outputs

const trackId = "libtr_xxxxxxxxxxxxxxxxxxxxxx";

const params = new URLSearchParams();
params.append("model", "MoodSimpleV2");
params.append("model", "BpmV2");

const response = await fetch(
`https://rest-api.cyanite.ai/v1/library-tracks/${trackId}/models?${params}`,
{
headers: {
"x-api-key": process.env.CYANITE_API_KEY,
},
},
);

const { items } = await response.json();

The response contains one item per requested model output (shortened here):

{
"items": [
{
"version": "MoodSimpleV2",
"scores": { "energetic": 0.6145, "dark": 0.4347, "calm": 0.2037 },
"tags": ["energetic", "dark"],
"segments": {
"timestampsSeconds": [5.0, 15.0, 25.0],
"values": { "energetic": [0.5873, 0.635, 0.2871] }
}
},
{
"version": "BpmV2",
"tag": 128,
"confidences": {
"modelCertainty": 0.97,
"predictionStability": 0.94,
"confidence": 0.94
},
"segments": {
"timestampsSeconds": [5.0, 15.0, 25.0],
"values": [128, 128, 127]
}
}
]
}

The order of the items isn't guaranteed. Pick each output by its version, as shown in the next section.


Pick an output by its version​

Every item has a version field that tells you which model output it is and which fields it has. Look up the output you need by its version instead of by its position in the list:

const mood = items.find((item) => item.version === "MoodSimpleV2");
const bpm = items.find((item) => item.version === "BpmV2");

If an output you requested is missing, the track probably hasn't finished analysis yet. See When a model output is missing.


Show the dominant tags​

For taxonomy-based outputs such as MoodSimpleV2, MainGenreV2 or InstrumentsV2, tags holds the labels that describe the track best. Use it for labels in your UI:

const moodLabels = mood?.tags ?? []; // ["energetic", "dark"]

To rank all taxonomy values instead, sort the scores map:

const rankedMoods = Object.entries(mood?.scores ?? {}).sort(
([, a], [, b]) => b - a,
); // [["energetic", 0.6145], ["dark", 0.4347], ...]

InstrumentsV2 has no scores map. Its presence map says how much of the track each instrument plays in (absent, partially, frequently or throughout):

const instruments = items.find((item) => item.version === "InstrumentsV2");
const instrumentsThroughout = Object.entries(instruments?.presence ?? {})
.filter(([, presence]) => presence === "throughout")
.map(([instrument]) => instrument); // ["drums", "bass"]

The possible values of each taxonomy are linked from Available model outputs.


Use a musical parameter only when it's confident​

Musical parameters such as BpmV2, KeyV2 and TimeSignatureV2 give one track-level value in tag and a confidences object. A value above 0.9 in confidences.confidence means a single value describes the whole track well. Below that, the parameter changes during the track or is ambiguous, so consider showing the segment values or no value at all:

const bpmLabel =
bpm && bpm.confidences.confidence >= 0.9
? `${bpm.tag} BPM`
: "Variable tempo";

Find where a mood is strongest​

segments shows how an output changes over the course of the track. For a taxonomy-based output, segments.values maps each taxonomy value to one score per entry in segments.timestampsSeconds. To find where a mood peaks, find the highest score and look up its timestamp:

const energetic = mood.segments.values.energetic;
const peakIndex = energetic.indexOf(Math.max(...energetic));
const peakStartsAt = mood.segments.timestampsSeconds[peakIndex]; // seconds

Start a preview at the most representative part​

Request RepresentativeSegmentV2 to get the part of the track that represents it best, for example as the starting point of a preview player:

{
"version": "RepresentativeSegmentV2",
"startSeconds": 42.0,
"endSeconds": 72.0
}

Handle outputs that can be empty​

Some outputs are null when the model has no answer for a track. For example, SubgenreV2 has null for scores, tags and segments when the track doesn't fit any subgenre with enough confidence:

const subgenre = items.find((item) => item.version === "SubgenreV2");
const subgenreLabels = subgenre?.tags ?? [];

Check the schema of each model output in the API Reference, linked from Available model outputs, for fields that can be null.


Troubleshooting​

StatusCauseWhat to do
401Invalid API key, or a requested model isn't enabled for your account.Check x-api-key. The error's detail names the models you can't access.
404The track doesn't exist or doesn't belong to your account.Check the Library Track ID.
422No model parameter, or an unknown model output version.Use the version names from the list of model outputs, e.g. BpmV2.
429Rate limit exceeded.Wait for retry-after seconds, see Rate limits.

An empty items list usually means the track hasn't finished analysis yet.


Next steps​