How Trialdefend Technically Analyzes Evidence

Top-level pipeline

Uploads are classified by file type, staged into a temporary local workspace, extracted with type-specific logic, assembled into a bounded prompt context, converted into a fact pack plus `case_json`, and finally rendered into the attorney report, detailed report, customer report, and evidence ledger with per-document model routing.

1. File classification

At review start, each upload is classified by MIME type and file extension before any extractor runs.

The classifier maps uploads into one of four paths: image, audio, video, or document.

• Image extensions: .png, .jpg, .jpeg, .webp, .gif, .bmp, .tif, .tiff, .heic, .heif
• Audio extensions: .mp3, .wav, .m4a, .aac, .mpga, .mpeg, .webm, .ogg
• Video extensions: .mp4, .mov, .avi, .mkv, .webm, .m4v
• Text-like extensions: .txt, .md, .json, .csv, .tsv, .xml, .html, .log
• Dedicated document extractors are wired for .pdf and .docx.
• Unsupported formats fall back to metadata-only with an explicit note that no extractor is wired for that type.

2. Local staging and temporary workspace

• Each run creates a temporary local workspace under the runtime temp root before extraction.
• Uploaded files are copied from storage into that workspace before extraction.
• The temp root is configurable through `TRIALDEFEND_TMP_DIR`.
• The workspace is removed after the run completes.
• Per-run logs record extraction kind, extracted character counts, and extraction notes per file.

3. Document extraction path

Documents are extracted with format-specific logic rather than a single generic parser.

• Plain text-like files are decoded with fallback encoding attempts: UTF-8, UTF-16, Latin-1, then UTF-8 with replacement.
• DOCX files are opened as ZIP archives, `word/document.xml` is read, paragraph/tab/line-break XML tags are normalized into readable text, and remaining XML tags are stripped.
• PDF files start with `pypdf` page-text extraction.
• Native PDF text is preserved page by page whenever it exists.
• When page-level native text is available, only pages lacking readable text are selected for OCR fallback.
• Fallback rendering uses `scripts/render-pdf-pages.py`, which relies on `pymupdf` / `fitz`.
• Rendered PDF pages become JPEG working images and then go through the normal image-analysis path.
• Default PDF render zoom is `1.5` via `TRIALDEFEND_PDF_OCR_RENDER_ZOOM`.
• Default OCR batch size is 75 pages via `TRIALDEFEND_PDF_OCR_MAX_PAGES`.
• Default large-document notice threshold is 40 candidate pages via `TRIALDEFEND_LARGE_PDF_OCR_NOTICE_PAGES`.
• Long scanned PDFs can continue through multiple OCR batches instead of stopping after the first small pass.
• Current long-document OCR progress events: `large_pdf_ocr_detected`, `pdf_ocr_batch_started`, `pdf_ocr_page_started`, `pdf_ocr_batch_completed`.
• If extraction returns no readable text, the system records that honestly rather than fabricating content.

4. Image analysis path

Images are analyzed as OCR / visible-evidence inputs. The system asks for two structured outputs only: extracted visible text and a short evidence summary.

• Maximum image size for the OCR path: 8 MB.
• HEIC / HEIF files are converted to JPEG first using `skills/criminal-case-intake/scripts/convert_heic.py`.
• Scanned-PDF fallback pages go through this same image-analysis path after rendering.
• Preferred vision path uses `VISION_REVIEW_MODEL`.
• Current configured default vision model: `anthropic/claude-sonnet-4-6`.
• Fallback image path uses OpenAI vision via `gpt-4o-mini`.
• Anthropic image OCR uses structured tool output with fields `text` and `summary`.
• OpenAI image OCR uses JSON-schema output with the same fields.

5. Audio analysis path

Audio-only uploads are transcribed rather than visually summarized.

• When `TRIALDEFEND_AUDIO_TRANSCRIBE_PROVIDER` is set to `aws`, supported audio uploads prefer AWS Transcribe with speaker labels enabled.
• If that path is unavailable for a specific file, Trialdefend falls back to OpenAI audio transcription using `whisper-1`.
• Current direct-transcription size threshold: 24 MB.
• Default AWS region: `us-east-1`.
• Default AWS language code: `en-US`.
• Default AWS max speakers: 6.
• The transcription prompt explicitly tells the engine to preserve names, badge numbers, dates, addresses, times, and uncertain words as heard.
• Current output kind for direct audio: `audio-transcript`.
• AWS diarization returns speaker-labeled transcript lines with timestamps when the media format is supported.

6. Large audio and video-audio chunking

Large audio and video-audio are processed through the chunked-media helper instead of being sent as one oversized upload.

• Helper script: `skills/criminal-case-intake/scripts/transcribe_media.py`.
• ffmpeg normalizes the audio before chunking.
• Output is mono 16 kHz MP3 at 32 kbps.
• Default chunk size is 300 seconds via `TRIALDEFEND_MEDIA_SEGMENT_SECONDS`.
• Optional smoke-test cap is `TRIALDEFEND_MEDIA_SMOKE_MAX_CHUNKS`.
• Each chunk is sent to the OpenAI-compatible `/audio/transcriptions` endpoint.
• The combined transcript file is written incrementally with chunk labels like `[Chunk 1]`.
• Clearly repetitive junk chunks can be excluded conservatively before downstream review when they look like corrupted transcript artifacts rather than real case audio.
• Excluded chunk labels are recorded in run notes so the filtering stays auditable.
• Current output kind for large audio: `audio-transcript-chunked`.
• Current output kind for video audio-only success: `video-audio-transcript`.
• Known limitation: chunk boundaries can reduce cross-segment context.

7. Video visual analysis path

Video visual review is separate from video-audio transcription. The system samples frames across the timeline and asks the vision model to summarize what is visibly happening.

• Helper script: `skills/criminal-case-intake/scripts/extract_video_frames.py`.
• The helper now uses hybrid content-aware sampling: sparse baseline coverage across the whole timeline plus denser sampling inside detected speech windows.
• Default baseline interval: 45 seconds.
• Default speech-focused interval inside detected speech windows: 1.5 seconds.
• Detected speech windows are expanded by ±3 seconds before frame selection.
• Phase 2 silent-video coverage can add targeted scene-change sampling inside quiet regions when `TRIALDEFEND_VIDEO_PHASE2_SCENE_SAMPLING_ENABLED` is enabled.
• Current production Phase 2 settings: scene-change threshold 0.18 and up to 48 silent-region scene-priority frames.
• Maximum sampled frames per video: 180.
• If no usable audio activity windows are available, the helper falls back to baseline-only visual coverage, or baseline plus qualifying silent-region scene changes when Phase 2 is enabled.
• Maximum frames per vision batch: 12.
• Intermediate visual-summary aggregation group size: 20.
• Each batch asks the model for `overview`, `visible_text`, and `notable_details`.
• A second aggregation pass merges batch summaries into a timeline-level visual summary.
• Preferred visual-review model path: Anthropic vision using `anthropic/claude-sonnet-4-6`.
• Fallback path: OpenAI vision using `gpt-4o-mini`.

8. Final video extraction states

• `video-visual-audio-review` = both audio transcription and visual review succeeded.
• `video-visual-review` = visual review succeeded but audio transcription did not.
• `video-audio-transcript` = audio transcription succeeded but visual review did not.
• `metadata-only` = no usable video evidence text was extracted.
• When both channels succeed, the output combines a `Video visual findings` section and a `Video audio transcript` section.

9. Prompt-context assembly

After extraction, Trialdefend builds a bounded review prompt from matter metadata, extraction results, workflow guidance, templates, schema guidance, coverage summaries, and per-file evidence text.

• Skill guidance source: `skills/criminal-case-intake/SKILL.md`.
• Template source: `skills/criminal-case-intake/references/output-templates.md`.
• Schema source: `skills/criminal-case-intake/references/dashboard-case-schema.md`.
• Coverage summary includes counts of transcript-backed media files, video files with visual-review findings, and skipped or metadata-only files.
• Per-file raw text excerpt cap: 16,000 characters.
• Total combined prompt source-text cap: 70,000 characters.
• When the source-text budget is exhausted, remaining files may be represented by metadata only.

10. Fact-pack stage

The final report flow is now fact-pack-first. Instead of making every report generation call reread the full raw evidence prompt, the system first creates a smaller structured downstream package.

• Step 1 output: `case_json` plus `fact_pack_md`.
• The fact pack is instructed to preserve chronology, key evidence, admissions, witness statements, contradictions, defense pressure points, and material missing or uncertain facts.
• Primary fact-pack model: `CLAUDE_FACT_MODEL` → `anthropic/claude-sonnet-4-6`.
• Default fact-pack max tokens: 16,000.
• Retry fact-pack max tokens: 24,000.
• Fact-pack timeout default: 420,000 ms.
• Anthropic fact-pack calls use a structured schema that requires both `case_json` and `fact_pack_md`.
• Current Anthropic structured-output requests also send ephemeral prompt-caching hints so repeated large reruns can reuse provider-side cached context when available.

11. Per-document model routing

Final report generation is split by document rather than generated as one large bundle.

• Attorney report model: `CLAUDE_ATTORNEY_REPORT_MODEL` → `anthropic/claude-opus-4-8`.
• Detailed report model: `CLAUDE_DETAILED_REPORT_MODEL` → `anthropic/claude-opus-4-8`.
• Customer report model: `CLAUDE_CUSTOMER_REPORT_MODEL` → `anthropic/claude-sonnet-4-6`.
• Evidence ledger model: `CLAUDE_EVIDENCE_LEDGER_MODEL` → `anthropic/claude-sonnet-4-6`.
• All Anthropic report calls use structured tool output and request only one target field per call.
• Current section token caps: attorney 20,000; detailed 16,000; customer 12,000; evidence ledger 12,000.

12. Provider transport and runtime details

• General provider timeout default: 180,000 ms.
• Provider fetch retries default to 2.
• Provider fetch retry delay defaults to 1,500 ms.
• Anthropic document and fact-pack calls hit `https://api.anthropic.com/v1/messages`.
• OpenAI paths remain available for image fallback OCR, chunked-media transcription, and fallback review flows still retained in code.
• OpenAI endpoints in use include `/v1/chat/completions` and `/v1/audio/transcriptions`.
• AWS Transcribe is used for supported direct-audio transcription with speaker diarization.

13. Progress visibility during long reviews

• The case page now shows recent run events instead of appearing idle during long OCR work.
• Long scanned-PDF runs surface explicit large-document and OCR-batch progress events.
• Platform-admin views can see more detailed event text, including page-level OCR progress events.
• Client-safe views use simplified wording such as queued, started, reading scanned PDF batch, and finished scanned PDF batch.
• The client-facing running banner switches to a clearer long-document message when a large scanned PDF is being processed.

14. Failure and honesty behavior

• If extraction fails, the file becomes `metadata-only` rather than pretending the file was reviewed successfully.
• If OCR or transcription returns no readable text, the notes record that explicitly.
• If PDF text extraction is weak because the PDF is image-only, that limitation is preserved rather than hidden.
• If media review is missing or partial, downstream outputs are expected to say so honestly.
• If no extractor is wired for a file type, the run records that directly instead of pretending the file was reviewed.