USE CASE / Noul + Score

RAG relevance filtering

Judge whether an excerpt supports an answer and score relevance

A representative input (state)

Question and candidate document excerpt

Question type

Noul + Score

Question instruction

Judge whether an excerpt supports an answer and score relevance

When below threshold

Code retains candidates for a text model; citations still need checking

Relevance is not truth

Keep provenance, version, and date for each excerpt. Noul can ask whether evidence is directly relevant and Score can rank against a rubric. Neither proves the source is true or current; verify that the final citation supports the claim.

Three edge cases to test

Outdated refund policy: relevant but unusable
Shared keywords: not necessarily answer evidence
Conflicting excerpts: flag and review provenance

Example request

This is a design draft, not an actual Jev answer or accuracy test.

{
  "state": "Question: What is the refund period? Excerpt: Refunds are accepted within 14 days.",
  "model": "jev-latest",
  "questions": {
    "decision": {
      "type": "noul",
      "instructions": "Does the excerpt contain evidence directly relevant to the question?"
    }
  }
}

Limits and fallback

Relevance is not truth; a score is not fact verification.

What does the support team actually need to decide?

Follow one support ticket from raw text and verified facts through three questions, an illustrative answer, edge cases, and launch evaluation.

Worked example: design a Jev workflow for a missing delivery