Quality Assurance and Measurement

1. First principle: quality is evidence that requirements were satisfied

QA
= requirement
+ observable test
+ threshold
+ evidence
+ reviewer/agent
+ result
+ severity
+ repair
+ regression check

“Looks good” can be a final human judgment, but it cannot replace explicit tests for meaning, truth, continuity, accessibility, or delivery.

2. QA gates

G0 SOURCE_INTEGRITY
G1 TRANSCRIPT_AND_CONTEXT
G2 CLAIMS_AND_RIGHTS
G3 STORY_GRAPH
G4 ROUGH_CUT
G5 PICTURE_LOCK
G6 AUDIO/COLOR/VFX/GRAPHICS
G7 ACCESSIBILITY
G8 DELIVERY_ENCODE
G9 PUBLISH
G10 POST_PUBLISH_MONITORING

A later gate may not silently waive a failure from an earlier gate.

3. Severity grammar

BLOCKER — cannot publish or proceed
CRITICAL — material truth, safety, rights, identity, or technical failure
MAJOR — significantly harms comprehension, continuity, or experience
MINOR — localized defect with limited effect
COSMETIC — polish issue without material functional effect
NOTE — observation or future improvement

4. QA record

issue_id
project/version
asset/time range
requirement_id
domain
severity
description
observed evidence
expected state
actual state
confidence
owner
repair action
status
regression links
approval

5. Editorial-integrity checks

quote meaning preserved
question-answer scope preserved
chronology not falsely implied
causality not falsely implied
reaction remains attached to true stimulus
qualifiers retained
speaker identity correct
context dependencies satisfied
hook paid off
open loops resolved or declared
central state change clear

Automated warning examples:

negative word removed near edit
answer separated from governing question
date/unit/population removed from claim
reaction moved beyond configured temporal tolerance
two clips joined from different topics without bridge

6. Narrative checks

central question/promise identifiable
beat functions present
causal links stronger than accidental chronology
stakes demonstrated
pacing supports comprehension
emotional contour coherent
setup/payoff links intact
ending creates a new state
CTA follows earned relevance

Narrative metrics are diagnostic, not automatic truth:

beats_per_minute
propositions_per_minute
open_loop_count
average_loop_duration
setup_payoff_distance
chapter_turn_count
repetition_rate
unresolved_dependency_count

7. Transcript and semantic checks

word accuracy sample
speaker attribution
word/sentence timecode alignment
named entity accuracy
number/date/unit accuracy
coreference resolution
claim extraction recall
uncertainty/negation preservation
segment boundary quality
language identification

High-risk tokens—names, negation, quantities, dates, diagnoses, accusations, legal terms—receive elevated review.

8. Factual and evidence checks

For every material claim:

claim text
speaker/source
scope
polarity
certainty
supporting evidence
contradicting evidence
verification status
source quality
publication/retrieval date
visualization usage

Status:

UNREVIEWED
SOURCE_NEEDED
PARTIALLY_SUPPORTED
SUPPORTED_WITHIN_SCOPE
CONTESTED
CONTRADICTED
OPINION
PERSONAL_EXPERIENCE
INFERENCE
SATIRE/FICTION

Rules:

IF claim could materially harm a person or organization
THEN require human fact/legal review.

IF visual B-roll appears evidentiary
THEN verify it depicts the correct event/time/place or label it illustrative.

IF a source is current-sensitive
THEN refresh it at publish gate.

9. Continuity checks

Picture

screen direction
axis
position
pose
gaze
action phase
prop state
wardrobe/hair/makeup
lighting direction/time
weather
background geometry
camera/lens family

Audio

room tone
perspective
noise floor
tonal match
sync
phase
music phrase
ambience location

Generated assets

identity
anatomy
object permanence
physics
text/logo
light/shadow/reflection
temporal coherence
entry/exit match
provenance

10. Technical picture checks

resolution
pixel aspect
frame rate/cadence
field order/progressive state
sync
dropped/duplicate frames
frozen frames
black frames
flash frames
illegal or unintended levels
clipping
noise/artifacts
flicker
stabilization artifacts
rolling shutter
focus
compression
color tags
HDR metadata
gamut

11. Technical audio checks

sample rate
bit depth
channel mapping
sync/drift
clipping
true peak
integrated loudness
loudness range
noise/hum/clicks
phase/polarity
mono compatibility
dialogue intelligibility
music/dialogue masking
start/end fades

12. Graphic and caption checks

spelling
numbers/units/source
chart mapping
axis/baseline
uncertainty
reading time
minimum size
contrast
color redundancy
safe area
caption collision
animation completeness
responsive reflow

13. Accessibility checks

captions present and synchronized
speaker labels when needed
meaningful sounds represented
audio description or equivalent for essential visuals
color not sole carrier
legible type and contrast
flashing/motion risk controlled
reading order coherent
keyboard/screen-reader support for interactive versions
transcript available

Prerecorded synchronized media should include captions that represent dialogue and meaningful sound information required for understanding.

14. Rights, privacy, and ethics checks

participant consent/release
location/property release
music/license
stock/license
font/license
archive/public-domain basis
trademark/logo context
minor/vulnerable-person safeguards
personal data
location privacy
medical/legal/sensitive claim handling
likeness and voice rights
generated content disclosure

15. AI-specific red-team checks

synthetic evidence mistaken for real
identity drift or misattribution
fabricated text/document
hallucinated event
stereotype amplification
unsafe reconstruction
prompt-injection text in source media
provenance stripping
undisclosed voice clone
model/version not recorded

16. Performance and platform checks

startup/load where interactive
streaming rendition ladder
thumbnail render
chapter markers
caption ingestion
metadata
link/CTA behavior
mobile layout
platform UI collision
content warning/age settings when required

17. Measurement framework

Audience outcomes

comprehension
recall
trust
emotional response
completion
retention by beat
rewatch
share/save
conversion/action

Production outcomes

time to rough cut
human review minutes
clip retrieval precision
context error rate
fact-check backlog
revision count
render failures
asset reuse
cost per finished minute

Model/system outcomes

transcript error rate
speaker diarization error
clip precision/recall
story-role classification accuracy
rule acceptance rate
candidate diversity
false-positive QA rate
false-negative QA rate
human override rate

18. Evaluation design

baseline version
candidate version
single defined hypothesis
controlled change level
success metric
minimum sample or evidence threshold
stopping rule
segment analysis
ethical constraints

Avoid changing many dimensions while claiming one causal result.

19. LAKA feedback vector

After release, calculate change across:

Magnitude — how large was the outcome difference?
Rate — how quickly did it appear?
Direction — did it move toward the goal?
Scope — which audience/chapters/platforms changed?
Depth — surface metric or durable comprehension/trust?
Duration — how long did it persist?
Frequency — how often did the effect recur?
Acceleration — did the effect grow or decay?
Variability — how consistent across segments?
Detectability — can it be measured reliably?
Reversibility — can the edit be reverted?
Propagation — what downstream versions changed?
Amplification — did platforms/audiences multiply it?
Accumulation — what compounds across episodes?

20. Automated versus human review

Automate:

schema validation
missing fields
file/encode checks
caption overlap
silence/black-frame detection
loudness/peak measurement
duplicate clip detection
continuity candidate warnings
provenance completeness

Require human review for:

meaning distortion
narrative fairness
sensitive claims
likeness and dignity
emotional exploitation
creative judgment
final disclosure adequacy

21. Regression grammar

Every repaired blocker/critical issue becomes a regression case:

input condition
old failure
expected result
machine test where possible
human-review note where necessary

A new version re-runs tests for all inherited issues and dependent outputs.

22. Final publish checklist

story approved
facts approved
rights approved
picture approved
audio approved
color/VFX approved
graphics approved
captions/accessibility approved
all aspect versions approved
encodes verified
provenance complete
release metadata complete
rollback/archive available

23. QA failure modes

CHECKLIST_WITHOUT_EVIDENCE
AUTOMATION_AS_FINAL_JUDGMENT
LATE_FACT_CHECK
LATE_ACCESSIBILITY
ONLY_MASTER_VERSION_TESTED
NO_REGRESSION
NO_SEVERITY_MODEL
CURRENT_SPECS_NOT_REFRESHED
CREATIVE_NOTE_TREATED_AS_TECHNICAL_ERROR
TECHNICAL_BLOCKER_TREATED_AS_TASTE