AI question reviewer for exam content

Context

The situation

Quill Learn is an international education provider. Examiners review large banks of questions before they reach learners.

The problem. Manual review was slow and inconsistent, the existing PDF parser failed on real exam papers, and analysis of a full paper took minutes — too slow for examiners to work interactively.

Approach

Key decisions

1. 01

   Structured review, not a chat box

   Each question is scored by GPT-5 or GLM against 8 pedagogical quality indicators, with findings triaged for the examiner.

2. 02

   Stream results as they arrive

   SSE streaming plus a concurrency pool means examiners see the first findings in seconds instead of waiting for the whole batch.

3. 03

   Fix the input first

   Replaced the failing PDF parser with Azure Document Intelligence and added a parse cache and batch inserts.

4. 04

   Ship with the user in the loop

   Ran the pilot feedback roadmap directly with the awarding body’s examiner lead through to sign-off.

How it works

Outcome

Results

• 40-question analysis from ~4 min to ~1.7 min; import from ~30 s to near-instant.
• Bulk analysis and examiner findings triage in production.
• Same engagement: consolidated scattered role checks into permission-based access control on Better Auth.