What counts as an evaluated result?
A research object can be technically valid, scientifically limited and useful for a particular purpose at the same time. The thesis makes those judgments separately.
Six questions, rather than a single score
Section 8.1.1 describes a quality vector with six dimensions. It is a framework for making judgments explicit, not a calibrated universal ranking. Each dimension needs its own observations, comparison conditions and uncertainty. A missing assessment means “not investigated”; assigning it zero would confuse missing knowledge with a demonstrated failure.
| Dimension | What to examine | Example of evidence |
|---|---|---|
| Evidence grounding | Can a representation or assertion be traced to appropriate sources? | Identified source fragment, acquisition context, derivation and review decision. |
| Measurement and model validity | Does the measurement or model address the stated property? | Calibration, uncertainty budget, reference data and an independent comparison. |
| Semantic consistency | Do identities, relations and constraints agree? | Schema and SHACL reports, resolved references and consistent contextual scope. |
| Computational reproducibility | Can the stated transformation be repeated or inspected? | Exact input bytes, tool and parameter versions, outputs and execution records. |
| Experience and interaction | Does the interface support the intended task? | A task-specific study of users, interaction, workload or listening judgments. |
| Access and reuse | Can others obtain and use the relevant material? | Identified distributions, formats, dependencies, rights information and working access routes. |
The dimensions need not be independent, and their numbers are not automatically comparable. A weighted total could hide an essential failure: impressive playback cannot compensate for an untraceable historical claim. Define mandatory conditions first, then compare alternatives on the dimensions appropriate to the question. A classroom demonstration and a metrological comparison legitimately require different evidence.
Match each check to the claim it supports
A SHA-256 match establishes the identity of supplied bytes relative to an expected digest. It does not establish that a microphone was calibrated or a builder attribution is correct. A schema check establishes a specified structural contract. It does not show that a modeled room sounds like the measured room. A reproducible program can reproduce a mistaken assumption perfectly. These checks are valuable because their scope is precise.
For an acoustic comparison, specify the reference signal or measurement, alignment, level treatment, bandwidth, sample selection and error statistic. For a timing claim, define the start and end events, device chain, measurement method, trial count and uncertainty. A display refresh rate is not an audio-latency measurement. For a historical claim, inspect source dependence and the instrument state to which each account applies.
Cuntz · Illustrative scenario#
How can a file pass validation but support a limited conclusion?
Conformance means meeting a specified contract. Different checks address parsing, structure, relationships, required capabilities and file identity.
Work through the example 3 STEPS
Take the published observation with value 1281 millimetres. A reader wants to know both whether its record is valid and whether the measurement answers a question about the pipe.
Check the structure
A schema can check that the result has the required form and that a numerical field contains a number. Semantic checks can verify that references point to appropriate records.
Check the evidence links
Resolve the pipe, measurement activity and workbook realization; verify the cited source bytes. These operations establish what the record says and which source it identifies.
Assess fitness for the question
The workbook timestamp still is not the measurement-session date. A valid record without a documented calibration uncertainty cannot support an uncertainty claim simply because its schema passed.
BASIS FOR THIS EXAMPLE
- Cuntz Positiv · VAO 0.5.0-rc.2 ↗
Published manifest: identities, measurement observations, realization digests, profiles and rights. Exact examples checked against this release.
- VAO Standard 0.5.0 ↗
The standard contract is separate from the Cuntz dataset content version 0.5.0-rc.2.
The notebook’s JSON teaching excerpt is deliberately incomplete and is not presented as a complete, independently conforming VAO package.
The useful distinction. Validation and scientific assessment complement one another; each needs an explicitly stated scope.
Studies prepared, but not completed
Section 8.5.2 explicitly identifies five prospective study areas. The dissertation prepares MUSHRA listening comparisons, task-based SUS and NASA-TLX interface studies, external expert panels, apparatus-based action calibration with IMU sensors, and PAMT field validation with uncertainty budgets. These are research plans. They must not be cited as completed listener, usability, expert-panel or field-validation results.
The distinction does not erase the technical checks and corpus or signal analyses that were performed. It makes their contribution interpretable. Development-time listening can identify a troublesome artifact and guide a revision; it does not estimate a population’s perceptual preference. A small diagnostic set can reveal a software failure; it does not establish general accuracy over all instruments or collections.
Keep optimization separate from confirmation
A reconstruction can become self-confirming if its free parameters are repeatedly adjusted until they match the same observations later used to validate it. The thesis calls for separating evidence-based conditions, optimization objectives and independent checks. Document which observations shaped a model and reserve other suitable observations for evaluation where possible. If no independent comparison exists, describe the result as a model-supported hypothesis with an explicit domain of use.
Inspect the reported case-study resultsQuantities, denominators and unresolved evidence.
Check technical conformanceThe separate layers of a VAO validation claim.
Cuntz · Illustrative scenario#
When does a modeled pipe become a historical hypothesis?
A reconstruction combines evidence with assumptions to produce a possible representation. A simulation computes behavior under specified model conditions.
Work through the example 3 STEPS
Suppose a researcher uses the Cuntz GesL observation to help constrain a pipe model. This is an illustrative reconstruction task; the single value is not a complete geometric description.
List what is supplied
The observation identifies one property, one pipe, one result and one source. It does not provide every interior dimension, material parameter or boundary condition needed by an acoustic model.
Label what is assumed
Record the additional dimensions and physical assumptions with their origin. Parameters chosen for convenience should not appear as measured values.
Keep fitting and testing separate
If parameters are adjusted to match a known frequency, that same match is not an independent validation. Compare with suitable independent evidence and retain the model’s domain of use.
BASIS FOR THIS EXAMPLE
- Cuntz Positiv · VAO 0.5.0-rc.2 ↗
Published manifest: identities, measurement observations, realization digests, profiles and rights. Exact examples checked against this release.
- Dominik Ukolov · Musikinstrumente im virtuellen Raum (2026)
Submitted dissertation, 11 September 2026; §§8.2.2.1 and 8.4.1, printed pp. 264–265 and 275–276. The historical description and digital inventory require a resolved mapping; the manuscript is not redistributed here.
No new simulation, measured frequency or complete pipe reconstruction is supplied by this example.
The useful distinction. A plausible reconstruction is a documented hypothesis constrained by evidence, not a recovered certainty.
Sources & further reading
- Dominik Ukolov · Musikinstrumente im virtuellen Raum (2026)
Dissertation submitted to Universität Leipzig, 11 September 2026. §8.1.1, pp. 254–256; §8.5, pp. 280–282. Page numbers refer to the printed manuscript pagination. The manuscript is not distributed by this website.
- VAO Standard 0.5.0
Normative standard, profile index, Dynamic Delivery Profile and conformance specification in the versioned release.
Page editions 2026-09-r6 ↓
Read a fixed snapshot of this chapter, or return to the current notebook.
Edition 2026-09-r6 · SHA-256
fc409ec1d6daPage metadata ↗