Research / Project

Soundscape Annotation

Human listening and machine listening in dialogue. An open source ecosystem for annotating, analysing and teaching the soundscape.

Soundscape Annotation is a research project on the encounter between the first-hand description of the soundscape and its computational analysis. The shared core is a controlled vocabulary of 128 terms across eight referenced taxonomies, from Schaeffer and Smalley to Schafer, Krause, Chion, Truax, Westerkamp and Wishart, the same for the human annotator and for the analysis pipeline.

The working hypothesis is that the divergences between human annotation and automatic analysis are not a flaw to be removed but the most valuable material. The project formalises them into four canonical types (convergences, false positives, false negatives, genuine ambiguities) and uses them in two directions, to train students' critical listening and to iteratively calibrate the computational system.

soundscape machine listening ecoacoustics field recording AFAM pedagogy open source

Artefacts

Three open source artefacts

Analysis toolkit, annotation tool and working paper share the vocabulary, a JSON interchange contract and the research aims. Each component is public, citable and reusable.

Python toolkit / CLI

soundscape-audio-analysis

v0.20.0 · Apache 2.0 + CC BY 4.0

A computational soundscape analysis pipeline. Levels and technical diagnostics (EBU R128 LUFS, true peak, clipping, hum), Schafer band spectrum, ecoacoustic indices (ACI, NDSI, H, BI, ADI / AEI), PANNs semantic classification over 527 AudioSet classes, CLAP auto-tagging with 251 Italian prompts and pluggable vocabularies, segmented critical narrative and PDF reports. Includes a benchmark against 14 academic gold analyses and a compare command measuring human-machine agreement (Cohen's kappa).

Progressive Web App

Soundscape Annotation Atelier

v2.1.1 · Apache 2.0

A web tool for first-hand annotation on a controlled vocabulary. Interactive waveform and spectrogram, time markers, synchronic layers inspired by Aural Sonology, spectromorphological notation with 15 original glyphs, form-building relations. Exports a deterministic JSON consumed by the toolkit, a typographic PDF sheet and a IIIF Presentation 3 manifest with a SKOS vocabulary. No server upload, bilingual IT / EN.

Working paper

Soundscape Annotation (Zenodo)

2026 · Zenodo · CC BY 4.0

A methodology paper formalising the six-stage workflow, the three software artefacts and the six-layer interpretive dossier, with an end-to-end validation case study in Italian higher arts education and the taxonomy of divergences between first-hand description and computational analysis. Published on Zenodo in open access, CC BY 4.0.

Method

An iterative six-stage workflow

The workflow alternates student production, teacher review, automatic analysis and argued critique. Each cycle produces a stratified dossier with six interpretive layers, from the first-hand audio to the final synthesis report.

1
Student

Submission v1

Field recording, annotation in the Atelier on the controlled vocabulary, discursive first-hand sheet, JSON and PDF export.

2
Teacher

Personalised feedback

Detailed remarks on annotation and sheet, with worked examples ready to adapt and a numbered action plan.

3
Student

Submission v2

Dossier revision based on the action plan, re-importing the JSON into the Atelier.

4
Teacher

Computational analysis and three prompts

The toolkit analyses the recording and produces the technical report. A comparative re-reading identifies the divergences and proposes three prompts for the next stage.

5
Student

Self-field-correction

A critical return to the three prompts, arguing where the ear is right and where the machine is.

6
Student

Final critical report

A synthesis of the whole path, from the first listening to the calibration of judgement.

The toolkit's compare command measures the agreement between ear and machine over structural boundaries, source families and annotation coverage, with agreement statistics (Cohen's kappa). The documented divergences feed the calibration of the toolkit, nine atomic interventions in the paper's case study alone.

In numbers

128 terms in the controlled vocabulary
8 referenced taxonomies
527 AudioSet classes in the PANNs classifier
251 Italian CLAP auto-tagging prompts
14 academic gold reference analyses
6 stages in the teaching workflow

In the field

Teaching adoption

Accademia di Belle Arti di Macerata

Multimedia Performance Processes and Techniques and Sound Space Design courses (A.Y. 2025-2026). The workflow was born here, and here the working paper's validation case study is documented, with the iterative calibration of the toolkit fed by student dossiers.

Conservatorio G.B. Martini di Bologna

Ear Training for Sound Technique course, where the soundscape annotation and analysis methodology is applied to training critical listening for sound recording.

Citation

How to cite

The project can be cited through its two main components, the working paper (Zenodo DOI) and the analysis toolkit (GitHub repository). The paper's concept DOI always resolves to the latest version.

@techreport{mariano2026soundscape,
  author       = {Mariano, Francesco},
  title        = {{Soundscape Annotation: workflow iterativo,
                   stratificazione interpretativa e calibrazione di
                   una skill di analisi audio in didattica AFAM}},
  institution  = {Accademia di Belle Arti di Macerata},
  year         = {2026},
  month        = {7},
  type         = {Working Paper},
  number       = {v1.0.2},
  doi          = {10.5281/zenodo.20282495},
  url          = {https://doi.org/10.5281/zenodo.20282495},
  note         = {Pubblicato su Zenodo con licenza CC BY 4.0.
                  Il DOI concept risolve sempre all'ultima versione.}
}

@software{mariano2026soundscapeskill,
  author       = {Mariano, Francesco},
  title        = {{Soundscape Audio Analysis: una skill Claude Code
                   per l'analisi tecnica e compositiva di soundscape
                   e musica elettroacustica}},
  year         = {2026},
  version      = {v0.20.0},
  url          = {https://github.com/francmo/soundscape-audio-analysis},
  note         = {Codice Apache 2.0, vocabolari e tassonomie CC BY 4.0}
}
← All research lines Collaborations and research →