Work / Clinical LLM evaluation

Does the AI hold up where the medicine actually happens?

From May to August 2026, I worked as a medical domain expert at SME Careers, evaluating medical LLM outputs for clinical accuracy, safety and guideline alignment. I identified hallucinations, reasoning failures and unsafe recommendations, and wrote rubric-based feedback for evaluation and red-teaming.

The samples below are independent portfolio work.

How an evaluation is actually done

The method is the product. A transparent evaluation is more persuasive than a large claim about AI safety.

01

What I test

Dosing, red-flag symptoms, crisis response, contraindications, vague-symptom triage and psychiatric frameworks — the tasks where a wrong answer is not a rounding error.

02

How I score

A fixed rubric applied to every response, with the model and version recorded. Findings are reported exactly as measured.

03

What I will not claim

A sample audit is not a certification. I report what a test can and cannot establish, and I do not imply a product is clinically safe unless the method supports it.

An AI product that touches patients?

Get in touch