I built a system for distributing and collecting homework assignments, and the assignments themselves were distributed with automated tests. Students would have instant feedback on their homework with clear tests and point assignments. Great idea, right? A high level required course of 40+ students could be scheduled, collected, and graded all in under an hour.
Not to mention the nightmare of early GitHub for Education used in the first semester (all forks mutually visible, what were they thinking!?), a genuinely shocking percentage of the class tried to cheat their way around it, thinking that I wasn't looking. Cheating felt like it had significantly increased the moment they thought we weren't looking. I expected some cheating, sure, but it really felt like it went higher and stayed higher than before.
The course has moved to others (who I have nothing but respect for), but I hear it is "notorious" for cheating. I was so proud of this system, and the distribution/collection systems are still used by a department, but the experience has really left me with a bad taste in my mouth wrt anything related to automated grading.
I had almost this exact same experience when teaching an algorithms course. I had to stop giving automated feedback when I had a student turn my own code back into me and then lie to my face when confronted about it. It only took 3 weeks. The worst part was how bad I felt for taking something useful away from the students who weren't abusing it.
Not to mention the nightmare of early GitHub for Education used in the first semester (all forks mutually visible, what were they thinking!?), a genuinely shocking percentage of the class tried to cheat their way around it, thinking that I wasn't looking. Cheating felt like it had significantly increased the moment they thought we weren't looking. I expected some cheating, sure, but it really felt like it went higher and stayed higher than before.
The course has moved to others (who I have nothing but respect for), but I hear it is "notorious" for cheating. I was so proud of this system, and the distribution/collection systems are still used by a department, but the experience has really left me with a bad taste in my mouth wrt anything related to automated grading.