Are Your Coding Assignments Passing for the Wrong Reasons?

If your coding assignments check only the sample input, a "pass" can hide a broken program: a hard-coded answer, logic that breaks when values repeat, or code that only works with positive numbers. The fix is a small set of hidden test cases, inputs students don't see, that check boundaries, edge cases and the exact rules in your problem statement. Then a pass means the program actually works.
Here's how to spot the problem, which hidden tests every assignment needs, and how to keep them fair.
What does "passing for the wrong reasons" look like?
Take a classic first-year lab question: read n numbers and print the second largest. The sample input is 3 8 1 9 4, and the expected output is 8.
All of these submissions print 8 for that input:
- The hard-coded answer.
printf("8");matches the sample perfectly. It's rare, but it happens, usually close to a deadline. - The duplicate trap. Sort the array and print the second-last element. That works for the sample, but
9 9 4prints 9 when your problem expects 4. - The negative-number trap. Start both "largest" variables at 0. Then an input of
-5 -2 -9gives 0, a number that isn't even in the list. - The tiny-input trap. With only one number, there is no second largest. Many programs read past the array or print whatever is in memory.
- The slow solution. A nested loop that compares every pair is fine for five numbers and far too slow for 100,000.
Every one of these passes the sample. None of them is a working program.
What are hidden test cases in coding assignments?
Hidden test cases are inputs and expected outputs that run against a student's code but aren't shown before submission. Public test cases, like the sample, show the input format and help students get started. Hidden ones check whether the program handles everything the problem statement promised.
You need both. Public cases make the task clear. Hidden cases make a pass mean something.
Which hidden test cases should every coding assignment have?
You don't need dozens. You need one for each behaviour you would deduct marks for. For most first- and second-year programming assignments, these six categories cover it:
| Category | Second-largest example | What it catches |
| Smallest valid input | 4 7 → 4 | Loops that assume more data |
| Too little or invalid input | 7 → the message your problem statement asks for | Reading past the array, missing checks |
| Repeated values | 9 9 4 → 4 | Sort-and-pick shortcuts |
| Negative values | -5 -2 -9 → -5 | Starting values hard-coded to 0 |
| Input order | already sorted, reverse sorted | Logic that depends on order |
| Large input | 100,000 random values | Slow nested loops, overflow |
The repeated-values test only works if your problem statement says that "second largest" means the second largest distinct value. That brings us to the most important rule.
Does your problem statement back up every hidden test?
A hidden test is fair only if the problem statement already covers it. If the statement doesn't say what to print when there's just one number, a test for that case is testing mind-reading, not programming.
Before you add a hidden test, check that the problem statement answers these questions:
- What are the input limits, such as the smallest and largest n and the range of values?
- What should happen with invalid or missing input?
- How should repeated values be treated?
- What is the exact output format: spacing, new lines and capital letters?
Writing hidden tests often exposes gaps in your own problem statement. That's useful: fix the statement, and every student benefits.
How do you keep hidden test cases fair to students?
- Tell students hidden tests exist and what kinds of cases they cover, such as "edge cases and large inputs", without giving away the inputs.
- Don't mark down formatting trivia. A trailing space shouldn't cost marks unless output format is what you're teaching. Ignore extra whitespace when comparing output where you can.
- Give partial credit by category. A program that fails only the large-input test has learned more than one that fails everything.
- Show useful messages after the deadline. "Failed: repeated values" teaches far more than "Test 4 failed".
- Run your reference solution against every test before releasing the assignment. A wrong expected output is the quickest way to lose a class's trust.
Do hidden test cases help with AI-written code?
Hidden tests check behaviour, not authorship, so they won't tell you who wrote the code. What they do is raise the bar from "matches the sample" to "follows your exact rules". A generic answer to the textbook version of a problem may not follow your rules on repeated values, invalid input or output format, and tests built on those rules will catch it.
Pair hidden tests with two habits: a short viva where students explain part of their code, and different variations of the problem for different sections. Our guide to ChatGPT cheating in programming classes covers this kind of assignment design in more detail.
Where does Lab.Computer fit?
Good hidden tests take time to write, and that's the step most often skipped when you're setting labs for several sections. The AI Assignment Generator drafts the problem statement, public and hidden test cases and a rubric from your learning objective. You review them and add the edge case you know your batch always misses. AI Evaluation then runs every submission against those tests and drafts feedback, and you review, edit or override every grade.
For the bigger question of what an assessment should measure, read how coding assessments can measure what students actually know.
Quick takeaways
- Checking only the sample input lets broken programs pass.
- Add hidden tests for the smallest input, invalid input, repeated values, negative values, input order and large input.
- Every hidden test needs a matching rule in the problem statement.
- Tell students hidden tests exist, give partial credit and show clear failure messages after the deadline.
Want hidden tests drafted for your next lab? Book a demo and bring an assignment you've already set.
Frequently asked questions
What are hidden test cases in coding?
Hidden test cases are inputs and expected outputs that run against a student's program but aren't shown before submission. They check edge cases and rules from the problem statement that the sample input doesn't cover.
What is the difference between public and hidden test cases?
Public test cases are visible to students and show the input and output format. Hidden test cases run at grading time to check whether the program handles everything the problem statement asks for.
Why does code pass the sample but fail hidden test cases?
Usually because it doesn't handle an edge case: the smallest or largest input, repeated or negative values, invalid input or a strict output format. Students should re-read the constraints in the problem statement and test those cases themselves.
How many test cases should a programming assignment have?
Write one for each behaviour you would deduct marks for, plus one or two public examples. For a short lab exercise, the six categories in this guide give you about six to eight tests.
Should students be told about hidden test cases?
Yes. Tell them hidden tests exist and what kinds of cases they cover, without revealing the inputs. That keeps the tests fair and encourages students to test their own code.
Can hidden test cases stop students submitting AI-written code?
No test can tell who wrote the code. Hidden tests built on your exact rules raise the bar for every submission, and they work best alongside a short viva and different problem variations for each section.
Related posts
Explore how coding assessments can measure real programming understanding through meaningful test cases, feedback, debugging, and instructor review.
ChatGPT can solve most intro programming assignments in seconds. Here's how instructors are spotting it, what went wrong at Purdue, and how to design work that resists it.
Grading 140 code submissions on a Sunday night? Here's what an autograder actually does, where it falls short, and how to start small without rebuilding your course.


