We use cookies to run the site, and (only with your permission) to understand usage with analytics.Read our Privacy Policy.

Analytics is off until you accept.
Tutorials

Are Your Coding Assignments Passing for the Wrong Reasons?

SANJAY6 minutes
Are Your Coding Assignments Passing for the Wrong Reasons?

If your coding assignments check only the sample input, a "pass" can hide a broken program: a hard-coded answer, logic that breaks when values repeat, or code that only works with positive numbers. The fix is a small set of hidden test cases, inputs students don't see, that check boundaries, edge cases and the exact rules in your problem statement. Then a pass means the program actually works.

Here's how to spot the problem, which hidden tests every assignment needs, and how to keep them fair.

What does "passing for the wrong reasons" look like?

Take a classic first-year lab question: read n numbers and print the second largest. The sample input is 3 8 1 9 4, and the expected output is 8.

All of these submissions print 8 for that input:

Every one of these passes the sample. None of them is a working program.

What are hidden test cases in coding assignments?

Hidden test cases are inputs and expected outputs that run against a student's code but aren't shown before submission. Public test cases, like the sample, show the input format and help students get started. Hidden ones check whether the program handles everything the problem statement promised.

You need both. Public cases make the task clear. Hidden cases make a pass mean something.

Which hidden test cases should every coding assignment have?

You don't need dozens. You need one for each behaviour you would deduct marks for. For most first- and second-year programming assignments, these six categories cover it:

CategorySecond-largest exampleWhat it catches
Smallest valid input4 7 → 4Loops that assume more data
Too little or invalid input7 → the message your problem statement asks forReading past the array, missing checks
Repeated values9 9 4 → 4Sort-and-pick shortcuts
Negative values-5 -2 -9 → -5Starting values hard-coded to 0
Input orderalready sorted, reverse sortedLogic that depends on order
Large input100,000 random valuesSlow nested loops, overflow

The repeated-values test only works if your problem statement says that "second largest" means the second largest distinct value. That brings us to the most important rule.

Does your problem statement back up every hidden test?

A hidden test is fair only if the problem statement already covers it. If the statement doesn't say what to print when there's just one number, a test for that case is testing mind-reading, not programming.

Before you add a hidden test, check that the problem statement answers these questions:

Writing hidden tests often exposes gaps in your own problem statement. That's useful: fix the statement, and every student benefits.

How do you keep hidden test cases fair to students?

Do hidden test cases help with AI-written code?

Hidden tests check behaviour, not authorship, so they won't tell you who wrote the code. What they do is raise the bar from "matches the sample" to "follows your exact rules". A generic answer to the textbook version of a problem may not follow your rules on repeated values, invalid input or output format, and tests built on those rules will catch it.

Pair hidden tests with two habits: a short viva where students explain part of their code, and different variations of the problem for different sections. Our guide to ChatGPT cheating in programming classes covers this kind of assignment design in more detail.

Where does Lab.Computer fit?

Good hidden tests take time to write, and that's the step most often skipped when you're setting labs for several sections. The AI Assignment Generator drafts the problem statement, public and hidden test cases and a rubric from your learning objective. You review them and add the edge case you know your batch always misses. AI Evaluation then runs every submission against those tests and drafts feedback, and you review, edit or override every grade.

For the bigger question of what an assessment should measure, read how coding assessments can measure what students actually know.

Quick takeaways

Want hidden tests drafted for your next lab? Book a demo and bring an assignment you've already set.

Frequently asked questions

What are hidden test cases in coding?

Hidden test cases are inputs and expected outputs that run against a student's program but aren't shown before submission. They check edge cases and rules from the problem statement that the sample input doesn't cover.

What is the difference between public and hidden test cases?

Public test cases are visible to students and show the input and output format. Hidden test cases run at grading time to check whether the program handles everything the problem statement asks for.

Why does code pass the sample but fail hidden test cases?

Usually because it doesn't handle an edge case: the smallest or largest input, repeated or negative values, invalid input or a strict output format. Students should re-read the constraints in the problem statement and test those cases themselves.

How many test cases should a programming assignment have?

Write one for each behaviour you would deduct marks for, plus one or two public examples. For a short lab exercise, the six categories in this guide give you about six to eight tests.

Should students be told about hidden test cases?

Yes. Tell them hidden tests exist and what kinds of cases they cover, without revealing the inputs. That keeps the tests fair and encourages students to test their own code.

Can hidden test cases stop students submitting AI-written code?

No test can tell who wrote the code. Hidden tests built on your exact rules raise the bar for every submission, and they work best alongside a short viva and different problem variations for each section.

Coding AssignmentsHidden Test CasesProgramming AssignmentsAutogradingCS Education