Validity matters more than suspicion
Whether a student broke a rule is a narrower question than whether the assessment still supports a sound judgment about their learning. The second question is the one worth designing around.
The Provenance Team
When faculty talk about AI and assessment, the conversation can drift quickly toward suspicion. Who wrote this? Was this generated? Did the student cross a line? What can I prove?
Those are understandable questions, especially when a student submits work that feels strangely polished, strangely generic, or strangely detached from the student’s prior performance. Faculty are responsible for academic standards, and when submitted work no longer gives us the confidence it once did, we should not pretend otherwise.
But suspicion is a poor foundation for assessment design.
Suspicion begins with uncertainty and often stays there. It can trigger a search for evidence, but it does not itself tell us whether the student met the learning outcome. It can lead faculty toward detector reports, writing comparisons, source checks, file metadata, process documentation, and awkward conversations, all of which may sometimes be necessary, but none of which solves the larger instructional problem. Even when suspicion is reasonable, the assessment may still lack the evidence needed to support a clear academic judgment.
Start with validity, not suspicion
A better starting point is validity.
In assessment, validity is about the quality of the inference we make from student work. If a student earns credit for an assignment, what are we justified in concluding? That the student can analyze a case? Write code? Apply a theory? Interpret data? Design a solution? Evaluate evidence? Explain tradeoffs? Use AI responsibly while remaining accountable for the final work?
Those questions matter because grades are not supposed to certify artifact production alone. They certify learning. The artifact is evidence, but the claim faculty make from that evidence is the real issue.
Dawson, Bearman, Dollinger, and Boud (2024) make this point directly: conversations about prohibited assistance often focus too narrowly on whether a rule was broken, while the larger concern is whether the assessment still supports a valid judgment about student learning. A student may violate a rule and produce work that tells us little about their capability. A student may follow the rules and still produce work that gives weak evidence of the intended outcome. A student may use permitted AI tools and, depending on the task, either deepen the work or bypass the learning.
The category of “allowed” or “not allowed” matters, but it cannot carry the whole assessment burden.
Every format is valid for some things and weak for others
Faculty know this from ordinary teaching. A take-home essay can be valid for some outcomes and weak for others. A timed exam can show recall under pressure while missing judgment, revision, source evaluation, and applied reasoning. A group project may reveal collaboration and professional practice, yet obscure individual contribution. A polished final product may demonstrate communication quality while hiding the decisions that produced it.
Generative AI did not create these problems. It made them harder to ignore.
Assessment can’t rest on detection
That is why assessment design in the AI era should not begin with the fantasy of perfect detection. QAA warned in 2023 that AI-generated outputs cannot be reliably detected in ways that provide clear evidence, and it advised institutions to reconsider assessment strategies rather than placing too much weight on after-the-fact identification (Quality Assurance Agency for Higher Education, 2023). The practical implication is uncomfortable but useful: if the validity of an assessment depends on our ability to detect undisclosed AI use after submission, the assessment is already in a fragile position.
Fragile assessments produce fragile decisions.
A faculty member may suspect that a submitted paper does not represent the student’s own understanding, but if the only available evidence is the paper itself, the conversation becomes difficult. The student may deny improper assistance. A detector may produce a probability. A writing comparison may raise questions but not settle them. The instructor may know, professionally and intuitively, that something is wrong, yet still lack the kind of evidence that supports a fair, reviewable decision.
That is bad for faculty, bad for students, and bad for institutions.
It also narrows the educational conversation. Once the focus becomes “Can I prove how this was produced?” the more important question can get pushed aside: “Can the student demonstrate the learning this assignment was designed to assess?”
Ask what the student can demonstrate
That question is more productive because it points toward evidence faculty can actually use.
If the outcome is analytical reasoning, ask the student to explain the claim, identify the evidence, and describe what would weaken the conclusion. If the outcome is programming competence, ask the student to walk through the structure, explain a design choice, trace a bug, and describe a likely failure case. If the outcome is research judgment, ask the student why certain sources were trusted, how conflicting evidence was handled, and what changed between draft and final version. If the outcome is professional communication, ask how the audience shaped the artifact and what tradeoffs were made for clarity, brevity, or persuasion.
These are not tricks. They are ordinary faculty questions, but in the AI era they need to become more systematic, more traceable, and more closely connected to the artifact.
The artifact as the review surface
That is where the artifact becomes the review surface.
The student’s submitted work should remain visible and central, because the goal is not to replace authentic assignments with generic questioning. The goal is to ask students to explain the work they submitted, in relation to the criteria faculty already care about. A good review does not wander away from the artifact; it points to specific sections, claims, functions, examples, sources, or design choices and asks the student to make their understanding visible.
This changes the evidentiary base. Instead of relying only on the finished product, faculty can consider the artifact, the student’s explanation, the question asked, the rubric criterion being assessed, the relevant timestamp, and the final academic judgment. That record is more useful than suspicion because it shows the connection between submitted work and demonstrated understanding.
Provenance Learning is designed for that space.
Provenance supports artifact-based oral assessment built around that question. It exists to strengthen the inference faculty draw from student work, not to adjudicate how the work was produced. Faculty approve the questions and criteria; students talk through their own submission in relation to the outcomes being assessed; and the session leaves behind a timecoded transcript, audio, artifact references, and evidence-linked recommendations faculty review and can override. What that yields is not a ruling about misconduct but a firmer basis for the academic claim a grade is meant to certify: that this student can do this thing.
The product does not need to decide whether misconduct occurred in order to be valuable. In fact, it should not make that decision. Institutions already have processes for academic integrity, grade appeals, accommodations, and faculty review. A responsible assessment system should produce better evidence for human judgment, not replace that judgment with machine confidence.
The same principle applies to AI use. Some assignments may prohibit AI. Some may permit limited assistance. Some may require students to use AI and evaluate its output. In all three cases, the student should be able to explain the artifact and demonstrate the relevant learning. The review surface changes with the assignment, but the evidentiary need remains.
That is the advantage of a validity-first approach. It does not require every faculty member to become an AI investigator. It asks a more familiar question: what evidence would justify the academic claim I am making about this student’s learning?
Faculty deserve tools that support that question rather than tools that intensify uncertainty. Students deserve assessment processes that give them a fair opportunity to explain their work. Institutions deserve records that can be reviewed, audited, and defended without pretending that a detector score can carry the weight of academic judgment.
Suspicion may start the conversation, but it should not design the course.
Validity should.
Key takeaways
- Suspicion is a weak foundation for assessment design; validity — whether a task measures the intended learning — is the stronger one.
- Every format is valid for some outcomes and weak for others; AI changes which formats stay valid, not the principle.
- The productive question is not “was AI used?” but “what can the student demonstrate about this work?”
- Assessments built on evidence rather than detection produce decisions faculty can defend.
References
Dawson, P., Bearman, M., Dollinger, M., & Boud, D. (2024). Validity matters more than cheating. Assessment & Evaluation in Higher Education, 49(7), 1005–1016. https://doi.org/10.1080/02602938.2024.2386662
Quality Assurance Agency for Higher Education. (2023). Reconsidering assessment for the ChatGPT era: QAA advice on developing sustainable assessment strategies. https://www.qaa.ac.uk/docs/qaa/members/reconsidering-assessment-for-the-chat-gpt-era.pdf
Editorial note
This article was drafted with AI assistance and reviewed, revised, and approved by Provenance Learning’s human authors. The claims, sources, examples, and final wording were reviewed before publication. We believe responsible AI use should be transparent, reviewable, and subject to human judgment. See our Editorial Transparency statement.
Provenance is an artifact-based oral assessment platform for higher education. If you’d like to see a session run on one of your own assignments, request a demo.