Drowning in Code Submissions? How an Autograder Gives CS Instructors Their Weekends Back

It's Sunday night again

You told yourself you'd finish grading by Friday. Now it's Sunday night, and there are still 140 submissions left for one programming assignment. Each one needs to be downloaded, compiled, run against a few inputs, and given a comment more useful than "check your loop."

If that sounds familiar, you're not alone. Sooner or later, most CS departments end up trying an autograder. So let's look at what one does for you, where it helps, where it doesn't, and how to start without turning your course upside down.

Grading code by hand costs more than your weekend

Grading programming work is slow in a way essay grading isn't. You can't skim code. You have to run it, and when it breaks, you have to work out whether the student misunderstood the problem or just missed a bracket.

When Codio surveyed computer science instructors in higher education, 72% said manual grading time was their biggest grading problem. Almost two thirds had no teaching assistant support at all.

Class sizes keep growing too. At UC Berkeley, the first two intro CS courses serve nearly 2,900 students a term. Most of us don't teach at that scale, but even a class of 120 with a weekly lab adds up to more than 1,400 submissions in a semester.

And the cost isn't only your time. Students wait. By the time feedback on assignment three comes back, they've already handed in assignment four with the same mistakes.

What an autograder actually does

An autograder is a program that runs each student's code against tests you define and reports what passed and what didn't. That's really all it is.

You write your test cases once: give this function these inputs, and expect these outputs. Every submission runs through the same tests in the same environment, so every student is graded the same way. Students get results in minutes instead of weeks, and they can often fix their code and try again before the deadline.

Here's a real example. Patrick Totzke at the University of Liverpool moved his intro object-oriented programming module, COMP122, to automated checks using check50, the tool Harvard built for CS50. Grading used to take about three weeks. With the new workflow, it took just over a week. His TAs still looked at every submission and could overrule the automatic result whenever it got something wrong.

That last bit is important. The best setups still have people in them. The tool just takes the boring, repetitive checking off their plates.

Even Gradescope, one of the best known grading tools today, started with two Berkeley PhD students bonding over how painful grading was. Most grading tools exist because somebody got tired of their own Sunday nights.

What an autograder can't do for you

Tests can tell you what the code does. They can't tell you how the student was thinking.

One student can pass every test with code that's a tangle of copied and pasted blocks. Another can fail a test because of a single boundary mistake while clearly understanding the idea. If your autograder only gives full marks or zero, the second student loses out, and the first one gets rewarded.

A few things still need you:

• Judging code quality and readability
• Giving partial credit when the logic is nearly there
• Noticing work that looks too polished for that student, especially now that ChatGPT can write a passing solution in seconds

So treat the autograder as your first pass, not your final word.

How to start without rebuilding your course

You don't need to automate everything this semester. Pick one assignment, ideally one you've taught before so you already know the common mistakes. Write five to ten tests that cover the normal case, a couple of edge cases, and the errors you see every year. Let students see their test results before the deadline. Then spend the time you save on the comments that really teach.

This is the workflow we built Lab.Computer around. You describe the topic, difficulty, and learning outcomes, and the Assignment Generation Assistant drafts the problem, test cases, and rubric. Students write code right in the browser, so there's nothing to install and no lab to set up. The Evaluation Assistant checks correctness, code quality, edge cases, and efficiency, and drafts feedback you can review and change before anything goes back to students. If the assessment needs to be secure, Proctortrack is built in.

You're still grading just as carefully. You're just not spending your weekends on the parts a machine can check, so you have energy left for the parts only you can do.

Quick takeaways

• Manual grading is the biggest time drain for most CS instructors.
• An autograder runs every submission against the same tests and returns results in minutes.
• Keep a human in the loop for code quality, partial credit, and integrity.

Frequently asked questions

What is an autograder in programming?

It's a tool that runs student code against test cases the instructor writes and reports which tests pass. Most also give students quick feedback, so they can fix mistakes before the deadline.

Can an autograder replace TAs?

No. It handles the repetitive checks, so TAs can spend their time on feedback, partial credit, and helping students who are stuck.

Which languages can an autograder handle?

Most support common teaching languages like Python, Java, and C. Lab.Computer supports more than 40 languages, right in the browser.

Want to see how it handles one of your own assignments? Book a demo and bring your toughest lab.