What counts as an evaluated result?

Six questions, rather than a single score#

Section 8.1.1 describes a quality vector with six dimensions. It is a framework for making judgments explicit, not a calibrated universal ranking. Each dimension needs its own observations, comparison conditions and uncertainty. A missing assessment means “not investigated”; assigning it zero would confuse missing knowledge with a demonstrated failure.

The dimensions need not be independent, and their numbers are not automatically comparable. A weighted total could hide an essential failure: impressive playback cannot compensate for an untraceable historical claim. Define mandatory conditions first, then compare alternatives on the dimensions appropriate to the question. A classroom demonstration and a metrological comparison legitimately require different evidence.

Match each check to the claim it supports#

A SHA-256 match establishes the identity of supplied bytes relative to an expected digest. It does not establish that a microphone was calibrated or a builder attribution is correct. A schema check establishes a specified structural contract. It does not show that a modeled room sounds like the measured room. A reproducible program can reproduce a mistaken assumption perfectly. These checks are valuable because their scope is precise.

For an acoustic comparison, specify the reference signal or measurement, alignment, level treatment, bandwidth, sample selection and error statistic. For a timing claim, define the start and end events, device chain, measurement method, trial count and uncertainty. A display refresh rate is not an audio-latency measurement. For a historical claim, inspect source dependence and the instrument state to which each account applies.

Studies prepared, but not completed#

Section 8.5.2 explicitly identifies five prospective study areas. The dissertation prepares MUSHRA listening comparisons, task-based SUS and NASA-TLX interface studies, external expert panels, apparatus-based action calibration with IMU sensors, and PAMT field validation with uncertainty budgets. These are research plans. They must not be cited as completed listener, usability, expert-panel or field-validation results.

The distinction does not erase the technical checks and corpus or signal analyses that were performed. It makes their contribution interpretable. Development-time listening can identify a troublesome artifact and guide a revision; it does not estimate a population’s perceptual preference. A small diagnostic set can reveal a software failure; it does not establish general accuracy over all instruments or collections.

Keep optimization separate from confirmation#

A reconstruction can become self-confirming if its free parameters are repeatedly adjusted until they match the same observations later used to validate it. The thesis calls for separating evidence-based conditions, optimization objectives and independent checks. Document which observations shaped a model and reserve other suitable observations for evaluation where possible. If no independent comparison exists, describe the result as a model-supported hypothesis with an explicit domain of use.