You don't succeed in remote teaching and learning by trying to make it as close as possible to the in-person setup. You have to treat it as an entirely different problem.

Consider the synchronous exam. It is a perfect method of grading in-person. It is hard to cheat and all people take the same test in the same place. It is as fair as it can get.

In an online setting however, people can face all sort of troubles in a few hour window in their home. Your internet might stop working, neighbor might be making too much noise etc. Everyone takes the exam in a different setting and it is as unequal as it can get. It is also practically impossible to prevent cheating.

Personal experience : I have declined several MD CS holders this year alone for positions because their demonstrated abilities were not up to par(and these were not particularly difficult questions).

In the end it comes down to trust. We can't trust someone who is willing to cheat to get ahead even if they are a super coder.

- Suramya

I agree at a high level, but practically speaking it's not realistic to expect colleges to completely reinvent their entire teaching systems for a temporary, 1-2 year remote learning period.

Now that we've been dealing with COVID for almost an entire year, it's easy to forget that at one point we thought this would all be over in a matter of weeks or months. The situation was also evolving in real time. Colleges were looking for the most efficient stop-gap solutions, not for ways to completely overhaul their learning experience.

> It is also practically impossible to prevent cheating.

It's a mistake to assume that because we can't eliminate all cheating, we shouldn't bother reducing any cheating.

The advantage of synchronous test taking is that everyone is exposed to the problem set at the same time. The obvious cheat with asynchronous test taking would be for one person to volunteer to take the test early and then send the questions to their peers, all of whom take the test at the last possible minute.

This happens whenever in-person classes offer two time slots for taking a test. The later time slot is always far more packed than the first and comes back with significantly higher scores. Adding the internet and screenshots/camera phones to the equation amplifies this because students can share the exact test, not just what they recall from memory.

> In an online setting however, people can face all sort of troubles in a few hour window in their home. Your internet might stop working, neighbor might be making too much noise etc.

Educators aren't oblivious to this fact. Working with students who have interruptions is just part of the job. Having someone lose internet isn't much different from having someone get a flat tire on the way to the test. It happens, we deal with it, and it's fine.

I also think you're not giving students enough credit. They're not dumb. If noise is a problem, they're going to use headphones. If internet is flakey, they're going to find a better location to take a test.

It's such a weird double standard to see HN champion work from home as unequivocally superior to working in an office, yet whenever the topic of learning from home comes up we get a laundry list of what-about possibilities that might make the experience worse.

I don't really understand what you mean by this though. I graduated college in 2007, and all throughout my time from 2003-2007 I had half my courses online, including the quizzes and tests.

Online courses and asynchronous testing isn't something new to Covid, colleges have been doing it for well over a decade now.

Yes you had people try and cheat their way through asynchronous testing by having friends take them earlier, but why is that more of a big deal now than it was previously?

They aren't, but many are much less privileged than you are making them out to be. My mom teaches students who join her class from their car parked as close as possible to get a weak Wi-Fi signal from their house because they have no quiet places at home.

This seems like a false equivalence; most employers don't use surveillance software to ensure their remote employees keep their microphone and webcam on, continue looking at the screen at all times, etc. One of the benefits of working from home is that your privacy is _increased_ compared to working an office; if everyone needed to allow their boss or HR or whoever to demand microphone and webcam access to me as they worked (and no, this is NOT comparable to Zoom meetings), then of course they wouldn't be praising work as much.

I absolutely agree, but I have to say, I also think it's somewhat bizarre that anyone ever thought that. I know it seemed weird to me at the time.

Where exactly did everyone think the pandemic would go after a few months of lockdown?

We thought we'd combine lockdown with tighter quarantines for visitors and excellent test & trace systems with quarantines for people with covid and that we would, like several countries, get covid under control.

This should have been an option all along, with out the need for a pandemic to force their hand.

>>The advantage of synchronous test taking is that everyone is exposed to the problem set at the same time. The obvious cheat with asynchronous test taking would be for one person to volunteer to take the test early and then send the questions to their peers,

This problem has been largely solved for a long time, because as you noted it is generally impossible to give a test to EVERYONE at the same time.

Thus properly written tests will draw a random selection of questions from a larger pool, the ratio between Pool:Questions the better the security. (i.e a 25 question exam using a 50 question pool is not as secure as a 25 question exam using a 200 question pool)

This method is also used for standardized tests given at the same time, as it cuts out the problem of shoulder surfing or other in-person cheating methods.

Many colleges were already fully capable of distance learning in multiple forms, whether through correspondence courses (what WGU often pitches, complete the project or test and bypass the class, though some of their certificate partners abuse test takers with Respondus or similar) or online learning with systems like Canvas.

Decent colleges offer a mix of these, I can attest to the quality of these programs at Seattle Colleges (specifically North Seattle College & Seattle Central). There is little value in building a panopticon of surveillance in higher education, especially when these divert resources that would otherwise enable students to better master the subject.

It worked out for him but easily could have been a disaster. We live at the edge of the school boundary and it was a localized outage due to a car hitting a pole, so he was the only student in his class that was affected. Good luck convincing the teacher if you don't have alternate access to the test.

Its stupid easy to cheat in this remote world: write your cheat sheet out on a piece of paper and put it just out of sight of the webcam. No technology can beat that.

Anyways, I'd prefer an approach that led to less invasive face tracking - even if it suffered from a lower detection rate.

Their "powerful AI engine" is almost certainly just humans. It might have a few off-the-shelf components like face detection but most of what they claim to do is just so easy to outsource that almost all companies do it. If there is any delay between the system observing a suspect behaviour and the student being told to correct it then they are definitely using humans.

An institution using a service like this is a huge red flag. You should take it as an indicator of a low quality administration if not a low quality institution.

As an engineering problem this task is hard. Ryan Calo (Prof of Law, UW) once presented a fascinating bit of research on trying to automate something as simple as fining someone for speeding. Given perfect information how do you build the system? If someone exceeds the speed limit for 1 second, do you fine them? If everyone around the person is exceeding the speed limit do you use the same rules? If someone oscillates between just above and just below the speed limit, how many times do you fine them? If someone exceeds the speed limit and stays there does this result in fewer fines? How do you square the code written with the law as written? The problems are so extensive it may be that application of rules like this require human level judgement. Proctoring an exam may turn out to be an AI-complete problem.

I wonder how parents feel that an anonymous foreign, most probably male, can watch their daughter in her room working on her exams and she can't opt-out else she'll be treated like a criminal.

Wonder if at some point we'll find a zip file shared among employees with screenshots of students from one of these outsourced proctoring vendors.

Why does foreign matter?

No or weak legal recourse in the event of wrongdoing being discovered.

Plus the outsourced entity can just go bankrupt and you'll never hear about it again.

Sick, twisted? That's the complaint exactly.

Unlikely? Too many things like this happen to dismiss it.

Something else?

A week later he got a half dozen letters in the mail all at once:

* warning do not speed * warning (+fine) * final warning (+fine) * ezpass revocation (+fine) * etc.

I think he ended up arguing that he didn't get the first warning before the others, and they agreed to waive the fines and roll back to the first warning. I.E. human judgement applied after the fines "fixed" the issue.

In any case: toll booths are a workplace, there's people there, go slow.

I'm not sure if any scale would make sense, though. The true thing to be fined for should be "driving unsafely", of course, whether that means driving too fast, braking suddenly, swerving, changing lanes without warning, etc. The thing is, the unsafety of all those things depends in very large part on the cars around you and sometimes on the road. (Driving at 100mph on a straight, flat freeway with no cars nearby is safer than weaving between a bunch of 65-70mph cars to maintain a speed of 80mph.)

But it would be really hard to come up with an objective algorithm to calculate how unsafe a rule violation is, and even harder to implement it without a vast array of cameras. So ... they implement an enforceable set of rules even if it's not a good one. (On a related note, it bothers me that driving with more than some arbitrary blood alcohol level is illegal, but driving after e.g. staying awake all night is not, even though the latter is worse[1]. Likely this is partly because checking BAC can be done fairly directly with cheap equipment, while checking how recently someone slept is... I dunno, there might be ways to mostly do that with good equipment, but I assume it's currently impractical.)

[1] "Being awake for at least 24 hours is equal to having a blood alcohol content of 0.10%. This is higher than the legal limit (0.08% BAC) in all states." https://www.cdc.gov/sleep/about_sleep/drowsy_driving.html

The ticket is merely an incentive to not do it in the future.

This optimizes for both expediency and safety, but, sadly, not for revenue, or for readily-available probable cause to pull over essentially anyone the police want to pull over.

The problem is that such a limit would not be a safe one, you would basically be replacing the limit with a sign that posts the rate that people like to drive at.

Turns out that people do drive at a reasonably safe speed, generally.

Source: https://jalopnik.com/utah-raises-some-speed-limits-to-80-mph...

According to who? The minority who wants the limit set elsewhere. Things like "what speed is safe and reasonable" are matters of social consensus. The "experts" can pontificate all they want and the vocal minority can gripe all they want but the median or average person and society at large is always going to be right on matters of social consensus. It's a tautology.

The main safety increase I can see is with respect to commercial trucking. Not all vehicles have the same acceleration and braking ability. But the fact that people just ignore the speed limit is a hard problem.

So, yes, as far as is practical, that's the idea. We just post the speed people are driving at.

Reasonable drivers conform their driving habits to the environment.

Unreasonable drivers should not be allowed to drive.

> [Wiki] Traffic calming can include the following engineering measures ... Narrowing traffic lanes ... Converting one-way streets into two-way streets forces opposing traffic into close proximity, which requires more careful driving

What is happening here? Did the engineers forget that they were supposed to optimize for safety, and instead optimized for getting people to slow down? Does the net effect of this improve safety, or worsen it?

If everyone drives as fast as they can while meeting some perceived safety level, then does that mean these efforts are unlikely to affect safety and merely to slow everyone down? Is the best set of measures the maximally deceptive set, which looks as dangerous as possible while being as safe as possible?

My understanding is that speed limits have been largely unchanged for the past 50+ years. Car safety, on the other hand, has increased by a great deal.

It seems likely to me that speed limits are conservative when it comes to safety.

I don't know exactly how it goes. If it's chosen (or heavily influenced) by politicians, I suspect politicians can win some votes by saying they'll improve safety by lowering speed limits (particularly around schools or other places where children might be); while they'd be less likely to win as many votes by saying they'll raise the limits (exposing them to the risk of their opponents calling them reckless/irresponsible). If it's chosen by non-elected officials, that's less of an issue, but something like it may still be there; or it may be that raising the limit and then there being a fatal accident will damage your career, while lowering the limit and irritating everyone will not damage your career.

I've driven on many roads in countries that don't have any speed limits. The overwhelming majority are driving a speed that makes sense given the road, driving conditions, etc.

I think a lot of people have problems with strictly enforced rules in finance. Especially when banks re-order daily transactions in order to maximize overdraft fees.

They have problems with it because the rules cost them money, not because it's hard to understand or rigidly enforced. If the rules just resulted in a "transaction declined" they wouldn't really care either way. For instance, I don't think anyone thinks that it's unreasonable for your debit card to get declined if you don't have enough money.

In the case referenced, there were 3 transactions on one day, 2 of $10 & 1 of $20. User has $20 in account. Do you take the true chronological order of transactions (10/10/20) & charge 1 fee, or do you go highest to lowest (20/10/10) & charge 2 fees?

A bank executive would likely argue why it's 'fair' to charge 2 fees because a business day is the relevant period for a bank (closing at end of day, etc.) & that method of accounting is mentioned on page 85 of the checking account TOS that a user signed.

Many consumers would argue that the bank's relevant period is meaningless, especially in the age of computers. They would also argue that it is unfair to lay out complex rules like this in an opaque way because of the asymmetrical information advantage that a bank has. They wouldn't complain that the rule was enforced per se, they would complain that the rule (which is plainly anti-consumer) exists.

Same is true of this cheating stuff - I think in general, students want a fair platform for grading. They just want that platform to be actually fair.

Doesn't that justify my point? If we go back to the speeding example, if the rule was that you can't go over the speed limit (within the capabilities of the measuring device), then everyone would drive a little more slowly. The only reason people 5-10 miles above the speed limit is that 5-10 miles over is generally accepted to be "fine". If anything strict enforcement of speed limits reduce the amount of room for abuse by law enforcement (eg. pulling over someone for going 1 mph over).

Also, at the risk of victim blaming, maybe it isn't such a good idea to have your deposits/withdraws lined up on the same day? Deposits can get delayed/withheld, and withdraws can be moved up unexpectedly. Leaving zero days between a deposit and a withdraw is just asking for trouble. I agree that reordering the transactions from largest to smallest is probably greed motivated, but at the same time expecting it to behave differently is optimistic at best and foolish at worst. It's the equivalent of relying on undefined behavior in programming (eg. assuming that reading one byte after the end of an array wouldn't cause a fault).

I'm not sure you understood the example that GP gave; there was no mention of withdrawals. The idea was that you have $20 in your account, you buy something for $10, then later buy something else for $10, and then later buy something for $20, and the bank reverses the order of applying the transaction and says that you made two purchases after your account was empty, so you get charged two overdraft fees.

The software is a privacy disaster and any computer that has had any of this spyware installed should be considered compromised. I kept a separate hard drive and would swap it in to take tests.

To fix the problem, grading measures need to be changed to accommodate the new world of online classes rather than trying to shoehorn old test proctoring into a remote space. This software only stops bad cheaters anyway.

I'm already imagining the fights I'm going to have with my daughter's schools in the future when they ask us to install this malware.

Unfortunately I think most of the world sees them as one thing :/.

honestly, I think the question was more of "what should the educators do?" not "what should the student do?

Speak for yourself; at my school, even the class that used C# worked with mono.

Because most linux users use their distribution's packages and it is normally easier than hunting for windows binaries.

Need an assembler? `sudo apt-get install nasm`, etc. I am a math/cs undergrad and have not used a windows machine in over 6 years. (In my case it would be editing either editing my home.nix or creating a shell.nix for a course rather than apt/dnf/etc)

If something applies to your low quality university it does not mean that it applies everywhere.

Hell, just reading this article, I thought of various ways by having someone off screen listening to you read aloud and a projector mounted after the test starts, on the ceiling by another person who’s off screen.

You could have a kvm switch to a whole different computer, an AI overlaying data on the screen (reading from the video output, and injecting into the video output), or any number of ridiculously complicated setups.

It probably won’t catch any serious cheaters.

They were already a problem in universities and this is their final and worst form.

University systems have the major problem that a significant portion of students are only there for a degree, and their participation is playing the game in order to get that piece of paper, and the GPA number rating them.

The core of this problem is the question “will this be on the exam?”

Testing of course can be an important part of learning, but making that the metric by which you decide to hand out degrees substantially damages the value of testing as a teaching tool, and damages the value of a university as a place for research and learning.

Another way needs to be found to sort students into degree worthiness.

Raising humans for the first quarter of their lives in a dystopian police state is not what anybody should strive for. We need to figure out how to measure people less.

Being young and tuition-free it was easy to grind through the material just worrying about the next test in the next class. Now, here I am as a tricenarian trying to relax and understand hundreds of pages of anatomy material while everyone else seems to just binge seems so asinine. After 75 pages of reading, exams continue to be a black box. Here I am paying for instruction and the professors can't be bothered to tell me what's important to know for the career let alone the test.

I got one teacher to chat off the record and he basically said to "get through it." What a disservice to students. Why are there still weeding-out classes? How many doctors, surgeons, programmers, engineers would there be in the world if it wasn't for these classes? The Montessori education system and the like seem so superior. College should be pass/fail.

That said, I would like to answer this question:

> Why are there still weeding-out classes?

In majors that are oversubscribed (e.g., pre-med, CS, etc.), these classes weed out the people who like the idea of the major/career more than the actual work involved. Good weeder courses (and I think that these exist) allow later classes in the major path to be more focused on the capable and motivated learners rather than baby-sitting the less-capable and less-motivated.

There are two reasons why this is desirable:

1. There is no way that a department could get through the same amount of material if they had to accommodate the less motivated and less capable in all of the courses.

2. Your department will get a reputation in the field for having students who are capable of X, Y, and Z.

Typically, if someone can't make it in the weeder classes at their school and they still want to enter that career path, then maybe it's better to go to a less competitive school. This sounds elitist, and maybe it is, but that's the workaround in our current system.

This is the high-school-ification of university, and it's part of the problem. This is not mandatory learning / babysitting. Just stop accommodating them. Every class just moves at the pace it needs to move at, and if they can't keep up, so be it.

I will add that the follow up classes can also move more aggressively since the weaker students have been culled by the required weeder course.

My SO once TA'd for a pre-med class. Holy hell, the students were something else as compared to other disciplines. They literally would not leave my SO's office until they had argued up their grades on every assignment. My SO had to get security to escort them out, I am not joking. If they forgot, say, to multiply by 2 and then got the answer wrong, they still argued for full credit despite the very clear error.

There was no shame whatsoever. We figured the other Profs and TAs just didn't want to deal with them, so the pre-meds learned to just wait out the graders to get their grades.

To be clear, this was maybe ~15% of the class, but still a lot of students. The Prof was no help with this either and just advised to pass them along. What a mess.

I understand that med school admissions is about as cut-throat as it gets, but my lord! It's these pressure cookers that lead to MD burnout and mis-selection. After all those shameless hours in undergrad, then med school hell, you finally get to practice on people just looking for their preferred opioid.

Sorry to hear about the mess you are in, but you are right, it's systemic.

One apparently needs close to a 4.0 GPA, plus many extracurricular volunteer hours to qualify for medical school, plus glowing letters of recommendation. I also overheard a number of stories about pre-med students intentionally sabotaging other students by misleading them to think that private group tutoring sessions (to study for the MCAT) had been canceled.

That experience helped form my beliefs about the driver high cost of medical care in the US.

Specifically, that there is no shortage of people who are willing and qualified to be medical doctors, but that doctors and medical schools collude to artificially limit the supply of credentialed medical doctors as a means to increase doctor salaries.

As you alluded to, I think it also creates an adverse selection problem, where the people who become doctors are mostly not those with excellent diagnosis skills or bedside manor or ethics, but those who are the most cutthroat and desperate to get into a high-paying career.

These aren't rumors.

Pre-meds hate other pre-meds and are conditioned to play a zero sum game against one another.

We then expect them to completely flip their attitude and become teams players. Collaborate with other medical professionals (former pre-meds), nurses and social workers.

No wonder healthcare is so dysfunctional.

Honestly, that's typical of most Biology lower division courses. The theory goes 'if you cannot pass this, you have no chance of the MCAT.' which may be true but some of us had no desire to be a physician. Someone replied to such an absurd structure explained to me that's 'how the system works.' Many of us that finally passed would not have anything directly to do with medicine and would AT BEST be considered ancillary service status to Medicine, and we were not amused. I personally will only take Industry-centric certifications, and I will never consider going back for anything degree related.

Google really is making the smartest move out of all the FAANG corps in that regard, by placing grater value on their own rather than degrees.

With that said, that's really what its there to do, isolate those who can commit to high loads of dated and rather antiqued (my near College credit HS Biology graded us on Kingdoms as Domains were already well established in the University lectures I attended) rote learning the best and 'thining the heard' for the upper-divisions which were super-capped when I was in University as all fields in Biology were impacted.

I at least thought now with Online learning being the default for so many it's sad to see the tech still hasn't solved this problem of mainly distribution/class seats.

I think it's one of the 2 elite schools in the UK (Cambridge or Oxford) that have an actual in-person skill assessment for your final rather than a test and most of your grade depends on that. My Inorganic Chemistry professor said his final exams were always the most preferable, albeit impractical, way to go about it. As a regular at his office hour lectures, I wish I had the pleasure of seeing what that may have been like.

> My SO once TA'd for a pre-med class. Holy hell, the students were something else as compared to other disciplines.

Been there, seen that as a Biology major just trying to get my foot in the door to get a graded paper during office hours. It's really sad now that I think about it as an adult that left the Industry long ago, you have no idea the weight on those kids shoulders to make it pass the openly touted 'filter' process, be it self-imposed or from their family. Its a sad reality how few end up using it all in these 2 horrible job markets for young inexperienced graduates.

My heart goes out to anyone who has to graduate in this Market, 2008 was bad but I think this may end up being worse as automation wasn't as big of threat as it was in 2008 as it is now in 2020 with COVID.

PS: Also, its well known amongst Biology majors that adderall abuse doesn't just start in Medschool, its a well established habit by undergrad possibly even the HS level for some.

Sorry could you explain that? Do Google now run courses and place value on having completed them in interviews?

Yes. They are slowly offering those options along with other tech companies.

I think Shopify has the best model: https://devdegree.ca/

Give it a shot, you can survey most courses on Coursera (where Google is offering said certification) for free and pay if you want to stay and earn a certificate. Udemy is another one I used when I wanted to hone my python skills when I was going from my developer role at IBM.

But if you had adequately prepared, the books and notes could really help you craft an excellent written response.

In this scenario, it took the focus off memorizing things and instead focused on understanding the content and being able to communicate something worth reading.

You had to do them, you were completely able to just copy the solution, and when you were working on something you got immediate feedback as to whether you were doing it well. If you wanted to learn you could, if you didn’t you wouldn’t. The lack of a week or two of lag between solving a problem and figuring out if you had done it correctly really made a difference.

We had Homework, Quizzes, and Tests, 33.33 each. Homework was graded like you said. Quizzes and Tests were open notes but on a relatively tight timescale. If people missed a homework, they could turn it in late for some linearly scaled penalty. If they missed a quiz or test question, they could come in and demonstrate/defend a solution off the cuff during office hours for 70% credit or something similarly high.

Students loved it. They no longer had to stress about getting an A, and they could instead focus on understanding the material, since that was the easiest way to (eventually) get the points.

I will say, though, this strategy was very time consuming for a large class. It also requires the TAs to have mastery of the material, which (shocker) is unfortunately not all that common.

Statics is very much a class where the same basic concepts are applied tons of times in different ways, and you kind of need to see and do a lot of problems to get good. My professor had us do online homework with unlimited attempts, but we had to submit our hand written work as well. Even though that class was entirely online, I felt very confident applying what I learned on projects.

The immediate feedback was 10x better than having the week or two of lag, and 9x better than having the answer on the back of the book because I never ran out of problems to try.

In defense of the students, they're just following the incentive gradient in front of them.

For all the romantic notions about how college should be optimized for exploration and "real learning", only a small percentage of people actually want that. And that's fine! Not everyone needs to be that sort of person; you should be able to have a reasonably comfortable career even if you aren't.

But the world is set up right now such that, without a university degree, you fall through the cracks and it's game over. In lots of high-earning jobs, a degree is either de jure required (law, medicine), or de facto required as most major employers won't look at you without one (finance, most tech stuff). So universities become degree mills because that's what the market demands, and students are only there so they can get a job afterward.

To fix the universities, you have to fix the incentives.

It’s very hard to focus on real learning when your financial freedom is at stake. People can not be faulted to feel that every minute spent studying the liberal arts curriculum is a minute that might be better invested in studying something that slays the elephant.

It’s a life or death feeling when the money involved is more than your parents have earned cumulatively.

Slaying the elephant becomes the end goal, everything else is secondary. A high GPA is a spear, and ethical integrity is a cheap way of obtaining it. This doesn’t explain why everyone cheats, but I think it attracts the otherwise “good” people, who are essential for normalizing the practice.

IMO, we need to do away with the elephant, and the rest will follow. Schools need not be so expensive. I hope some startup school will do the job. Otherwise, companies might be the first to do it.

It’s probably cheaper for Google to operate a University than it is to operate their recruiting pipeline :)

Check out CIRR.org which reports audited numbers from bootcamps. Good luck finding a university that reports comprehensive, audited, financial and career outcomes for their students. The numbers don't really exist, because it would be, guessing here, VERY embarassing for the universities.

Basically there’s no incentive for their studies to be scientific, and I don’t think they are. Their most-touted metric was “ROI”, which is super vague and highly dependent on in-field job placement, which was measured in a highly suspect way at my school.

> To fix the universities, you have to fix the incentives

Who better than the universities to drive change? That is sort of their fundamental purpose is it not?

If you need to lead a university around with a carrot, it really has failed to be much of a university.

[0] https://www.nationalgeographic.com/science/2020/11/the-colle...

We have accreditation boards to determine what is required for a common base.

To me, a focus on 'credit-based' system is about acknowledging that the boundaries of those domains are growing and changing. Gaining knowledge and expertise in one area can inform your work in another. The idea would be to broaden abilities, not focus on being 'exclusively prepared'.

> the biggest value of a university degree is teaching someone how to think and learn in combination with some domain knowledge.

I would add also: 'how to work', 'how to learn' and 'who to know'. By 'certs' I didn't just mean tech specializations. Courses can be about broader topics. A course on politics can be absolutely beneficial to an engineer. A more modular approach could also encourage lifelong learning rather than the "checkbox" that a degree satisfies.

>We have accreditation boards to determine what is required for a common base.

That's the problem. Those boards/requirements vary based on Universities and available instructors and resources. Yet, everyone comes out with the same degree.

School boards here, and I'm sure many other places, decided to nix grades from March onwards after the pandemic hit. The announcement wasn't made until closer to the end of the term. I remember some parents being furious claiming that their kids "wasted their time" on remote school if a grade wasn't being assigned.

I've had this discussion many times, and as a university drop out, my opinion has always been similar to yours. One thing I've learned is that debating the merits of the higher education system is right up there with religion and politics in terms of topics to avoid at a dinner party.

People take great pride in their educations (read: pieces of paper). They can take great offense to the institutions and systems that produced those pieces of paper being challenged. These values have been passed on and hardened through generations. This is the first thing that needs to change before there's meaningful reform, and it's going to take time.

Fortunately, he navigated the mess that was AP testing last spring and his test scores prove that he knew the material but as a parent who watched my kid work so hard, even through the remote learning portion, it was quite stressful for both of us.

Now we wait to hear from the initial wave of applications. The system, as it stands, sucks but it is the system we have. Perhaps the great experiment starting with the class of 2021 may drive changes to make it better.

> "... people who have some sort of facial disfigurement have special challenges; they might get flagged because their face has an unexpected geometry.”

So this company is extrapolating "patterns ... associated with cheating" from facial geometry. This is just phrenology laundered through their "powerful artificial intelligence engine" black box. Predicting behavior from the shape of someone's skull is still bullshit pseudoscience, even if the calipers[1] are replaced with a bunch of linear algebra.

[1] http://antiquescientifica.com/phrenology_calipers_George_Com...

It's probably worse. I'd bet that in addition to humans with non-'normal' heads, it would also flag the blind, deaf, sufferers of Tourette syndrome, certain muslim women, bearded men, humans of african descent, humans of asian descent, etc.

> even if the calipers[1] are replaced with a bunch of linear algebra.

I love this line. If your head does not fit the space of eigen-heads, throw an error.

My dad had a pretty severe lazy eye, and I have no doubt eye tracking software would think that he was looking down at his desk constantly.

I stare off into the distance (usually sideways) when I'm thinking.

Those sorts of things are common and barely conscious.

So, what's left for the AI? Trying to detect if people seem nervous? Well, I've got friends who have nervous twitches under normal conditions.

What is normal behavior varies wildly person-to-person. Trying to figure out "is someone glancing somewhere before answering" vs "does someone nod to themselves when they think they got an answer in their head" doesn't sound like something AI can do.

Figuring out "is this person cheating" from a recording of their face sounds pretty close to phrenology. Determining personality from skull-size, determining cheating from facial-motion... they both sound bogus, and like someone's trying to create a correlation that cannot account for the variety of human behaviors and shapes.

This assumes that the AI only picks up on patterns between eye movement and cheating, and not correlations between unrelated dimensions of data in the dataset that the model was trained on.

Famously, such correlations created AI systems that resulted in disparate impact on legally protected classes in the US.

Also, these systems are usually ill-equipped to handle anomalies in an accurate capacity.

People that want to cheat or game the system will find ways, it isn't hard to do. In undergrad we setup a copy of the code checking software that our department used so that we could share code without it getting flagged as copies. I'm sure there are ways to game these systems too if you are motivated enough.

One of the issues is also just academia being so hyper-focused on thinking everyone is there to become an academic which is not the case for the vast majority of people. So these exams and the guidelines for student evaluation is grounded in that expectation instead of the reality.

In her first review after a few weeks on the job, her manager says that she takes too long to do her work; they just need her to take what the client says verbatim, fact check it, and then slap it in a document. She was treating her work like an academic assignment and putting in the effort to craft something unique, when they really just need a fact checking typist.

8 years in higher ed, published study on cancer drugs, thousands of mice died, millions of dollars spent on the lab... all so she can transcribe some text and then validate it against the studies.

Academia is the worst job training program ever.

Academia was never meant to be a jobs training program. The fact that many students treat it as such is not really the fault of the institutions.

No, this isn't true at all. People don't complete PhD's because they looked at their options and thought that one was the best. They do it because other people told them they should, and they just never thought about it.

Pet peeve of mine. Academia is not a training program for jobs. The role of academia is not to produce business-perfect-candidates.

If business wants trained workers, they should train them. Cutting costs by not training them, then blaming universities for not producing trained workers is disingenuous at best.

But no, because the way students (and parents) think about it is "go to college so you can get a good job". And many, many employers require a degree or they won't look at the candidate. In the real world, academia is functioning as a job training program.

Or at least as a gateway to the good jobs. But if it's going to be a gateway, but not do any training... that's pretty inefficient.

As someone who taught a college, that's a flawed statement. Let me propose this to you. You teach Calculus 1000 and there is an end of term exam. Your normal end of term exam is one where everyone sits in the same room, proctors are looking for cheats, things checked, etc. Instead this year you tell your students that this year it will be a take home exam with the following rules: 1) they have 2 hours to do it during the team home peroid; 2) no cheating by the honor system.

Do you think the rate of cheating will be the same? I mean by your logic, it should be.

Another professor devised an exam that used your student ID as a variable of the first question, and subsequent questions used the previous answers as inputs. Impossible to cheat.

We're discussing this in relation in relation to COVID, so no large gathering. This means no open book in person exams, I'm specifically talking about take home exams.

> Another professor devised an exam that used your student ID as a variable of the first question, and subsequent questions used the previous answers as inputs. Impossible to cheat.

Really easy to do so in a take home exam.

I think the larger point is that it is fairly easy in any subject to design a test that is very hard to cheat on. It is much harder to find the resources in modern education to grade that test since each submission is likely unique.

Tests that are easy to grade (like multiple choice), tend to be tests that are easy to cheat....

It's true that you usually can't cheat your way through such an exam provided you actually write the answer yourself, but in a take-home situation you can always ask someone else to solve it for you.

For things such as proofs, it might give you a problem for which the theorem is needed, then ask you to solve the problem, indicate which theorem you used, and then prove the theorem.

Of course if you have 50 students per instructor this is not going to work...

Computer science: Solve a complex problem in code. Include a git history. Be ready to defend your program design if I get suspicious.

Econ: Long answer question: Pick 5 concepts that we learned about that you think are most important. Explain them as you would to a ten year old.

You can always just pay someone to do it for you.

I mean, we're obviously upset about false positives, and we should be. But I'm presuming that some people are caught who were cheating, and without the software they would have cheated and not been caught.

We can suggest that with an honor code in place, maybe some of those cheating students would have not cheated, because... I mean, if they were willing to cheat with software in place, I'm not sure why would have been deterred by an honor code.

I think in about 98-99% of cases, people who claim an honor code prevents cheating are deluding themselves. Yeah, if you don't have any way of catching cheaters, then you can pretend you have a 0% cheating rate. But it's pretend.

P.S. I'm not speaking for my employer at all.

Where would they get reliable training data?

There's a pile of money to be spent on this stuff, and virtually zero accountability. What's that a recipe for?

The flipside is that when you do catch a cheating case, you completely throw the book at them. It's legitimately easier to cheat under an honor system, if that's what you're wanting to do... so my assumption is that if we catch you at it, it's likely part of a pattern, and if we catch you multiple times the pattern is irreformable. It is not uncommon at honor-code institutions to expel students on the second offence (sometimes even on the first).

I do think that cheating is less prevalent at my institution (and my previous institution) than it is in the larger university population.

We may hate the software on ethical grounds, or because it degrades the exam-experience on many levels, or because its use can be considered abusive, but obviously it has an effect in the intended direction.

I'm not sure I follow why this is bad. I think academic rigor and being a decent scholar, as well as being able to parse and produce research are good things (and in my mind, those are the corner stones of being an academic). Did misunderstand you?

FWIW, in germany (and likely other european countries) we have a two tier system for higher education, consisting of universities and "applied universities", with the latter focusing on applied skills and the former focusing on research, which in think is sensible.

The problem I see is that what you talk about as academic rigor isn't what is taught and evaluated in many of the programs and classes that I've seen or been a part of. A lot of these exams and assessments don't particularly evaluate you on your ability to research and understand knowledge. If I know for example that my physics professor uses a bank of questions then it is much more incentivized for me to memorize that bank of questions vs. understanding the content and working the problems myself. Whereas in say a Discrete Math or Algorithm based class, the final exam/grade is based on a proof you have to write yourself, that encourages (or rather at times forces) you to learn and research like you said.

I also think the issue, and this may just be me looking at from my own experience, a lot of people don't want to be scholars, as you put it. They went to a University based on the unfortunate expectation for some jobs that say you need that diploma as your entry ticket.

While it isn't codified, we effectively have this in the US as well.

Most of the "brand name" universities, plus the flagship state universities, conduct research and grant various doctoral degrees.

Then we have the middle-tier colleges (state and private) that grant masters (often only professional degrees like nursing, MBAs, etc).

And thousands of Baccalaureate-only and 2-year community colleges.

Also, in the US, "university" generally indicates a post-graduate degree granting institution. And "college" usually refers to a 2-year and Baccalaureate-only school. But, also not codified and there are notable exceptions (ex: The College of William & Mary is a top-notch full university who's name pre-dates the convention).

For example, there may be a "College of Engineering" and a "College of Arts and Science" that are part of one university.

It's possible to have a stand-alone college that isn't in a university. A good example is Berklee College of Music, which is narrowly focused on music.

https://www.usnews.com/education/best-colleges/articles/2018...

But, you are correct that subject schools within a larger uni are often called College of Such and Such.

Great example of why our educational system isn't all that good.

I took a year of Japanese in college. A big part of the grade was memorizing and performing these dialogs. If you didn't remember the dialog word for word, you'd get points off. I wasted a lot of time trying to memorize those stupid things when I could have been acquiring new vocabulary words or studying Kanji. I took a trip to Japan a couple of years go, and did some brushing up for a couple of months before I left. I learned more in those two months than I did in my entire time at school.

Polygraphs are bullshit and basically select for submissive behavior, which I suppose is what these institutions are looking to reward, but it is conspicuous that nobody called these surveillance schemes what they truly are: degrading.

How about writing open-book tests in ways that are impossible to cheat on if you don't understand the material? You've had almost a year to adapt.

Why outsource student PII to a developing country sweatshop? This all seems absurd to me, it's been much easier to just write a one-liner inline script to hook blur, focus, visibilitychange, and onkeydown and log the userid when the event happens.

If you're curious, this is exactly what Canvas does to detect foul play (although they don't advertise it for that, it just goes into the log)[0][1]. Schools don't think it's good enough, so they spend thousands on these more invasive solutions.

[0]: https://community.canvaslms.com/t5/Instructor-Guide/How-do-I... [1]: https://github.com/instructure/canvas-lms/blob/master/public...

So in a remote test situation, explain how this is possible without use of these software. I can think of many scenarios where a second person can be in the room and doing the exam.

On a more constructive note, it seems pretty simple to require some portion of responses to reference the lectures such that only someone who attended the class in this semester would be able to correctly answer, for example "use the method we discussed in the first half of lecture 3, use the third of the four numbers I told you to write down that day as variable y, if you weren't in class that day instead do X" If a test comes back and someone claims to not remember the material from any classes but still got everything right, that warrants scrutiny. If someone currently enrolled in the class is helping a person cheat, you can use standard anti-cheating techniques like comparing answers.

Then you are measuring class attendance rather than subject matter mastery. If someone has already mastered the year’s material by the third class, why penalize them for skipping the rest of the lectures?

Nothing except these tests were not take home prior to COVID.

>> On a more constructive note, it seems pretty simple to require some portion of responses to reference the lectures such that only someone who attended the class in this semester would be able to correctly answer, for example "use the method we discussed in the first half of lecture 3, use the third of the four numbers I told you to write down that day as variable y, if you weren't in class that day instead do X" If a test comes back and someone claims to not remember the material from any classes but still got everything right, that warrants scrutiny. If someone currently enrolled in the class is helping a person cheat, you can use standard anti-cheating techniques like comparing answers.

So you are expecting the student to remember 100% of what they heard in the online classes? Or more likely the student would write down these details and handle it to the cheater to use during the take home exam. Or the cheater would "do X". In most cases the one doing the helping is not a current student, so it doesn't help.

I've passed college classes not attending any classes. Doing assignments, exams, and mid-terms. So would I be penalized for not attending classes?

Take home tests were definitely a thing before covid.

> So you are expecting the student to remember 100% of what they heard in the online classes? Or more likely the student would write down these details and handle it to the cheater to use during the take home exam. Or the cheater would "do X". In most cases the one doing the helping is not a current student, so it doesn't help.

I expect students to take notes for their open book exams. If you're competent enough to identify all the material needed for the cheater to do an open book test, congratulations you have the knowledge to pass the test. If you just record every piece of information possible, congratulations you have spent way more time and effort than it would have taken to just learn the material.

If you don't attend any classes, do X. You're not being penalized for doing that, it just warrants scrutiny. If this is an entry level course that an intelligent person could teach themselves the material, then it probably doesn't matter if you cheated or not, you'll be found out in higher level courses. If this is a high level course, your department would probably have a good idea of how capable you are from past performance. If the guy struggling to get a C in introductory physics gets a perfect score on his quantum final without going to class once, that is super suspicious.

Yes, and they work pretty well for some subjects. But not for others.

> ... that is super suspicious.

But suspicion is not enough to take disciplinary measures.

In many cases, there really is no solution that is privacy-preserving, anti-cheating, covid-safe and affordable.

Are you claiming that having open-book tests makes it so that everyone can get an MSc? That’s what it sounds like you’re saying, so I assume that I just don’t know what you’re arguing.

Every system is going to catch some percentage of cheaters and wrongly punish some percentage of innocent people. The pandemic has put us in a bad position where we can’t use some of the more effective systems (in-class tests) for assessing knowledge / preventing cheating, so we are forced to come up with some kind of compromise, and in many subjects, open-book, take-home tests work very well (although they require more work from the professors).

But it's the style of justification: because you can't catch them all, just ignore the problem. That's not ok. Education is supposed to teach you something else than cheating.

As a result, most tests consists of multiple-choice questions and standard-format open questions. It is unclear to what extent they test students' understanding, and students do become quite proficient test takers, but at least theses tests can be executed given the current constraints of available staff, acceptable rigor, and student expectations.

Unfortunately, you cannot move these type of test to an online setting and expect students not to cheat at all. It is too easy to talk to classmates, look at the study materials, or even on the Internet for some hints to answers or even the actual answers. The only option many institutions saw was to move to a draconian proctoring solution because they just lack the means to roll-out anything else given the constraints they have to work with.

Ideally, they would re-evaluate their choices regarding quality versus quantity, but because they need the large number of students to stay afloat, nothing meaningful will happen.

How many students do you think Plato, Aristotle, Jesus etc. had? Teach the brightest and the ones with genuine interest in the subject (I don't use the word "passion" because after adoption by corporations it no longer seems to mean anything). The rest will be fine with a high school/secondary school education. Most people retain only a fraction of that knowledge anyway.

Maybe it's time to stop treating (higher) education like an assembly line?

That sounds as if you're steering close to a whiteboarding interview which a lot of people will claim isn't great.

Take home tests aren't perfect but I had plenty of them in school. And even when I didn't I don't think I had just about any engineering class that didn't assign problem sets that counted for a decent portion of your grade. If you can do take-home problem sets, you can do a take-home exam.

I was thinking more about a larger project or paper with quite a lot of freedom for the students that would take the teacher hours to get familiar with and give constructive formative feedback on. As part of that, an oral section to discuss the project and paper with the student would be great as well. In assessment, I would like the teacher and student to be partners rather than opponents.

If a student understands the material, together they should be able to have a conversation where the student can reflect on her understanding and learning to the extent where the teacher can determine that the student has passed the course. Of course, if there is an intense interaction between the student and teacher throughout the course, the whole idea of a final assessment becomes pretty meaningless.

I know, my ideas are a bit far-fetched and there are a lot of practicalities that are difficult to work out. But one can dream!

One major difference is that unlike a whiteboarding interview there are presumably clear expectations as to what one is expected to have learned in the class and therefore what topics the discussion will cover, and at what depth.

Another issue with whiteboarding interviews, specific to computer-related subjects, is being asked to write code on the whiteboard, which is a severely unnatural act in all sorts of ways compared to the way one normally writes code. But an in-person assessment in a proofs-based math class, for example, would not have that problem. Similar for a CS (as opposed to software engineering) class.

And in a software engineering class, the concept of "test" is pretty odd, for the same reasons that whiteboarding interviews are; I'd expect longer-term projects to be closer to the right evaluation tool.

A merge/code review tool could be used to annotate individual problems. Have you seen this done?

We haven't added autograding yet as it is specific, but nbgrader[1][2] comes to mind.

- [0]: https://iko.ai/docs/notebook/#collaboration

- [1]: https://nbgrader.readthedocs.io

- [2]: https://nbgrader.readthedocs.io/en/stable/command_line_tools...

Even so, based on what I've heard from my immediate superior, I'm flagging somewhere between 10-20% of all of the plagiarism and cheating cases in my faculty.

It's not hard to detect (generally good writing mixed with bad on essays / the repetition of similar phrases by different students on exams), but it does take a bit of time to confirm and you need to write a some policy-consistent boilerplate to make it simple to take action.

Some here have suggested that we need to write the equivalent of "open book" exams for the online environment. I'm not sure that would be beneficial. It's still important to exercise your mental faculties. People who can recall information and reason for themselves are still more valuable than those who can cut and paste. Or to put it another way, you need to understand the math in order to wield a calculator well.

It might supply some of the pieces required to solve problems, but the ability to remember all of those pieces by rote vs. looking them up is really orthogonal to the ability to apply them to solving a novel problem. Indeed the ability to find the requisite information to solve a problem in a timely manner is an important skill in itself and more accurately represents the real world problem solving one is preparing for.

This might be my prejudice speaking (I studied math) but isn't this basically because grading is that much more subjective? So it would cut both ways, harder to cheat but also less objective / impartial assuming no cheating.

I was on the other side as an academic assistant for a few days (no role in testing) and I can tell you that oral math exams about algebra can be as hard as you want them to be, especially if you have problems with tests and get nervous. You have to take drugs against that.

The prof I had was a bit of a dick but he was really good at measuring how well you understood the concepts and if you ware able to apply them to problems.

Most of my engineering higher level classes were open everything not electronic. If you don't know how to tackle the problem, you're not going to finish on time.

Similarly, in CS classes, I always enjoyed assignments where I got to go write a program to do something. Again, it could be cheated, but so could lots of other things.

The fact that some students cheat shouldn't ruin classes for everyone else. Since cheaters do lower the value of a degree, maybe universities should start taking legal action against them instead. Provide a huge disincentive without messing everything up for the majority of students.

WRT legal consequences for cheating: I mean, you can get kicked out of university. That’s a loss of thousands of dollars invested. And as a TA, if I knew I could be called in to testify in court every time I caught a student cheating... I’d never call it out. I don’t have time for that! And that still wouldn’t change that we’d have to be on the lookout for cheating.

This software isn't ideal but there's no good solutions here short of major rethink of how these classes, and possible all of the college, is structured.

So one solution is to switch to open book take home exams as much as possible. The other frankly is to have an honor code and, at some point, recognize that some people will cheat but they're mostly hurting themselves.

Students who cheat can secure a higher GPA with less effort. That effort can be redirected towards looking for internships and improving grad school applications.

So I fail to see how cheaters are only “hurting themselves”. Morality aside, if you pursue a cheating strategy your pay-off will be much higher when we consider the scarcity of resources available to students.

This is definitely not the reason why most people cheat.

> Morality aside, if you pursue a cheating strategy your pay-off will be much higher

As with any kind of lie, you will have to cheat more and more to compensate for previous cheating, and eventually your little web of lies will come back and bite you in the ass.

> As with any kind of lie, you will have to cheat more and more to compensate for previous cheating, and eventually your little web of lies will come back and bite you in the ass.

Completely disagree. People cheat to climb the ladder of life without having to put in the work or having the resources required to do bypass that ladder. However, once you climb the ladder high enough, it is much harder to fall back down.

Sure you're not going to be a good doctor if you cheat your way through med school but if you cheat your way through undergrad and get into a better program with better resources, more driven peers and better professors, you are better off regardless.

> if you cheat your way through undergrad and get into a better program with better resources, more driven peers and better professors, you are better off regardless.

Until you end up doing a surgery. You can’t cheat the real world. Your lack of knowledge will show eventually, and then it’ll limit your opportunities.

This assumes that what you learn in university is the useful part, not the grade or the allocation of time that could be used for hackathons, projects, clubs, or internships.

- oral exams via video call

- written exams where students are distributed over a larger area (e.g. the university rents a warehouse for the examination time) so that the COVID-19 spreading risk is nevertheless kept very small.

The desks were so far apart that the current "social distancing" standards would be met. I remember that many courses had multiple variations of the same exam given to students to further reduce cheating.

I'm taking an online masters right now and have taken a few (pre-pandemic) proctored exams. The nearby university offers to proctor any exam for $10. They have a bunch of rooms with desks, and it's no big deal at all.

This proctoring technology thing has gone too far and too fast. It was a knee-jerk reaction, and with a lot of complaints I bet that a lot of it gets dropped.

If so, who cares if cheating numbers / grades have been inflating?

In my professional experience, GPA has little correlation with how well someone can do a job. GPA going up or down by 0.3 points really doesn't mean that much to me.

That way the viewer / reader will be able to compare your results to your class.

Obviously if 80% of your class got A, that A is not going to look as impressive.

On the other hand, if you're the only student that got A, while the class distribution is monotonically increasing towards Failure, that's going to be a good thing for you.

And then there were the take home, open book, open notes, you have three days to finish this exams...

If you can't bother to make a proper exam, don't bother. Rather make the classes pass/fail on project work, or grade the classes on projects / home exams. Trying to bring a 100% replica of the physical exam space into the digital space is just a recipe for disaster.

What do you think it would take to bring educators up to this new standard that is very different?

In the real world of 2020+, almost everyone has a portable Hitchhiker's Guide to the Galaxy in their pocket. Let people use calculators. Heck, let people use Wikipedia! If they copy an incorrect fact from Wikipedia, well, that's their problem and penalize them for that.

Education needs to be considered in the context of the world we live in. If you want to test students, then develop ways of TESTING students - move to live oral exams. Written long-form essays. Develop education techniques that USE the tools we're blessed to have around us instead of fearing them.

I don't have all the answers as I haven't thought about this deeply enough. But I've got to imagine there are ways of testing learning & knowledge that aren't based on fact memorization & regurgitation or performing calculations in your head that can easily be done on a calculator.

The hardest test I ever took was the AI final at CMU. Open book. Books didn't help :(

Yes. The application of principles, concepts and information is not terribly difficult to test in either a timed or take home setting.

They do have the advantage of degrading fairly gracefully from in-person examination to remote examination.

They're also tremendously expensive compared to a test given en masse.

[1] Oral exams in a school setting would feel much fairer to people than programming interviews do, mainly because you know what to expect on an exam in a class you just took.

It would be very, very hard to bluff your way through something like this with Charisma alone. You have to be damn sure you know your topic in and out. For once off exams before graduation I think it is acceptable but I'd pity students who had to go through a grilling like this every couple of months.

Agreed, but I don't think oral exams are the only type of effective live testing. Nor are live/timed exercises the only way to demonstrate knowledge.

Case studies, short/long form writing and creative exercises are all viable options. The primary concern seems regarding scaling grading and review, which with a little effort and ingenuity is solvable.

Caltech had tremendous success with a culture of honesty and take-home exams.

As for tests, there is an easier solution to all this. Write your test as if it was a take home. Open notes. You can't stop people from communicating but often these types of exams/assignments it becomes clear who is doing it. In my undergrad all my upper division classes' exams were take home because "I can't test you on anything worthwhile in 2 hours." Honestly, most of us enjoyed these more. We often did the exams in the same room (we had the back of a building that was dedicated as a lounge to the physics students) and no one really cheated. The closest was "hey, I'm stuck on this problem and I know you are finished. Can I just use you as a rubber ducky?" (more like just explaining the problem to the person and the other person saying "uh huh" and no more) It also made me feel like an adult because our professors trusted us. As someone that frequently does poorly in a testing environment I was also surprised that I was able to get much higher scores on these tests despite the added complexity. The simple fact that I could "take a break and come back" was all the piece of mind I needed (or grabbing a beer when I felt frustrated). This also better reflected, in my opinion, what solving difficult problems were like in the real world. I could grab my book, go to the page that I know is helpful, sit and think, take of my shoes, pace, whatever. I was treated like an adult and it felt good.

The software itself shouldn't have any control over the student's grades. A person should have to review the flags and actually find some wrongdoing. Not just push 'yes' and walk away.

Let’s say you’re only using the software to flag suspicious behavior, and bringing in humans to make the final decision. What happens when (inevitably) the software disproportionally flags people with dark skin because it is not trained to recognize dark-skinned faces? Or when the software disproportionally flags poor people, or people with families?

It means that those groups of people will be targeted by the (human) bureaucracy and tasked with defending themselves, when they’ve done nothing wrong. Humans will inevitably trust the algorithm, they will use the algorithm’s outputs to justify their own biases, and even investigations come with a cost.

There’s this meme going around that the “algorithm isn’t biased, it’s the data”, but that argument doesn’t really hold water—machine learning systems, by default, learn to recognize correlations, and correlations in the real world collected with real sensors contain biases. ML, by its nature, picks up and encodes those biases, and you must make an effort to remove them—you can’t just throw an ML algorithm at a pile of data.

People are getting screwed because they are at the long end of an unbroken chain of crap. Crappy organizations buy crappy software and crappy professors take the results seriously. The fact that there exists a crappy tool that flags all the black people as cheaters (or whatever, point is that the false positives are unacceptable common and unacceptably distributed).

Blaming the gun (the software in this case) is tacitly condoning the unbroken chain of half a dozen people/entities that are failing to do the job they are being paid to do. The software developers shouldn't be building crap software. The companies shouldn't be selling crap software. The universities shouldn't be buying crap software. The professors shouldn't be using the results of crap software. To look at that situation and say "yeah the problem here is that this crap software exists" is beyond naive. The problem is that nobody is being accountable for the bad outcome. I'm not asking for a whipping boy or a scapegoat here, the problem is that when nobody can be held fully responsible it seems like nobody even gets held partly responsible.

There’s really no room for absolutism, where blame is assigned to one source rather than distributed among many contributing factors. Imagine how dangerous air travel would still be if, after an accident, investigators looked for only one cause to blame, and tacitly condoned anything else they came across in their investigations.

Re-read my comment. My point is that fault is distributed sufficiently that accountability seems to evaporate. This is a systemic or organizational problem. Anti-cheating software is just the form this specific instance has taken.

>Your argument assumes that “blaming the gun” is condoning the users