The algorithm:
isSafe = random();
if isSafe >.5 println (“everything is fine\n”);
per the article, it’s rather better than that.
I admit to having grossly oversimplified things. Sorry.
I thought that would have been obvious to everyone already.
Can we stop having AI do… anything?
Algorithms aren’t AI. They’re standardization measures in cases like this. Hell you don’t even need computers for many of them. We use tons in healthcare to classify risk, decide on treatment options, and even decide on how much medication to give.
Pedantic Mathematician here.
If it failed, then it was a heuristic, rather than an algorithm.
Clearly, that’s the most important thing about this post.
You’re welcome.
Why not both? A bad algorithm based on bad heuristics? There are many many algorithms that fail at what they’re supposed to do.
As a non-condescending “mathematician”, I’m happy to help.
Pretty much anything trying to predict human behavior is a heuristic; people using them as if they’ve got some kind of certainty is a problem.
Yes, exactly.
Minority Don’t Report
Cracked me up thanks!
Minority Report: the beta test
Our pigs don’t look as good as [generic Hollywood actor]
Why do we live in a dystopian hellscape
People are complicated creatures that can’t be easily fit into niche categories despite our brains need to do so.
Culture.
Capitalism.
Advocates: take survivors of abuse seriously.
Society: Let’s have computers tell us what to do!I mean I guess the risk of repeated murder-suicide is pretty low…
About 20 new cases of gender violence arrive every day, each requiring investigation. Providing police protection for every victim would be impossible given staff sizes and budgets.
I think machine-learning is not the key part, the quote above is. All these 20 people a day come to the police for protection, a very small minority of them might be just paranoid, but I’m sure that most of them had some bad shit done to them by their partner already and (in an ideal world) would all deserve some protection. The algorithm’s “success” in defined in the article as reducing probability of repeat attacks, especially the ones eventually leading to death.
The police are trying to focus on the ones who are deemed to be the most at risk. A well-trained algorithm can help reduce the risk vs the judgement of the possibly overworked or inexperienced human handling the complaint? I’ll take that. But people are going to die anyway. Just, hopefully, a bit less of them and I don’t think it’s fair to say that it’s the machine’s fault when they do.
Sounds like a triage situation. That really sucks for the women affected.
Is it really too much to want enough resources to respond appropriately to all cases?
Having worked in making software for almost 3 decades, including in Finance both before and after the 2008 Crash, this blind reliance on algorithms for law enforcement and victim protection scares the hell out of me.
An algorithm is just an encoding of whatever the people who made it think will happen: it’s like using those actual people directly, only worse because by need an algorithm has a fixed set of input parameters and can’t just ask more questions when something “smells fishy” as a person would.
Also making judgements by “entering something in a form” has a tendency to close people’s thinking - instead of pondering on it and using their intuition to, for example, notice from the way people are talking that they’re understating the gravity of the situation, people filling form tend to mindlessly do it like a box-ticking exercise - and that’s not even going into the whole “As long as I just fill the form my ass is covered” effect when the responsability is delegated to the algorithm that leads people to play it safe and not dispute the results even when their instincts say otherwise.
For anybody who has experience with modelling, using computer algorithms within human processes and with how users actually treat such things (the “computer says” effect) this shit really is scary at many levels.
More data for the algorithm then.
Computers are only at fault when its convenient to blame them.
I have no issues with using ML to predict outcomes. It’s going to be wrong sometimes, so will humans. The system just needs review and input from humans understanding the model.
In the late 1970s (I was a kid) the computer is always right was a common sarcastic parody of all the people who actually believed it.
We’d discoverin the 1980s it was possible to have missing data, insufficient data or erroneous data.
It’s a sentiment at least as old as the first things that we now call computers.
On two occasions I have been asked, “Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?” … I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.
—Charles Babbage
As if humans can magically make correct decisions with incorrect information lmao. So true.
The way to use these kinds of systems is to have the judge came to an independent decision, then, after that’s keyed in, the AI spits out theirs and whichever predicts more danger is then acted on.
Relatedly, the way you have an AI select people and companies to get spot-checked by tax investigators is not to show investigators the AI scores, but mix in AI suspicions among a stream of randomly selected people.
Relatedly, the way you have AI involved in medical diagnoses is not to tell the human doctor results, but suggest additional tests to be made. The “have you ruled out lupus” approach.
And from what I’ve heard the medical profession actually got that right from the very beginning. They know what priming and bias is. Law enforcement? I fear we’ll have to ELI5 them the basics for the next five hundred years.
But that doesn’t save money and the only reason the capitalists want AI is saving money
I don’t think there’s any AI involved. The article mentions nothing of the sort, it’s at least 8 years old (according to the article) and the input is 35 yes/no questions, so it’s probably just some points assigned for the answers and maybe some simple arithmetic.
Sounds like an expert system then (just judging by the age) which was AI before the whole machine learning craze, in any case you need to take the same kind of care when integrating them into whatever real-world structures there are.
Medicine used them with quite some success problem being they take a long time to develop because humans need to input expert knowledge, and then they get outdated quite quickly.
Back to the system though: 35 questions is not enough for these kinds of questions. And that’s not an issue of number of questions, but things like body language and tone of voice not being included.
so it’s probably just some points assigned for the answers and maybe some simple arithmetic.
Why yes, that’s all that machine learning is, a bunch of statistics :)
so it’s probably just some points assigned for the answers and maybe some simple arithmetic.Why yes, that’s all that machine learning is, a bunch of statistics :)
I know, but that’s not what I meant. I mean literally something as simple and mundane as assigning points per answer and evaluating the final score:
// Pseudo code risk = 0 if (Q1 == true) { risk += 20 } if (Q2 == true) { risk += 10 } // etc... // Maybe throw in a bit of if (Q28 == true) { if (Q22 == true and Q23 == true) { risk *= 1.5 } else { risk += 10 } } // And finally, evaluate the risk: if (risk < 10) { return "negligible" } else if (risk >= 10 and risk < 40) { return "low risk" } // etc... You get the picture.And yes, I know I can just write
if (Q1) {, but I wanted to make it a bit more accessible for non-programmers.The article gives absolutely no reason for us to assume it’s anything more than that, and I apparently missed the part of the article that mentioned that the system had been in use since 2007. I know we had machine learning too back then, but looking at the project description here: https://eucpn.org/sites/default/files/document/files/Buena practica VIOGEN_0.pdf it looks more like they looked at a bunch of cases (2159) and came up with the 35 questions and a scoring system not unlike what I just described above.
Edit: I managed to find this, which has apparently been taken down since (but thanks to archive.org it’s still available): https://web.archive.org/web/20240227072357/https://eticasfoundation.org/gender/the-external-audit-of-the-viogen-system/
VioGén’s algorithm uses classical statistical models to perform a risk evaluation based on the weighted sum of all the responses according to pre-set weights for each variable. It is designed as a recommendation system but, even though the police officers are able to increase the automatically assigned risk score, they maintain it in 95% of the cases.
… which incidentally matches what the article says (that police maintain the VioGen risk score in 95% of the cases).
It looks like the minority report future didn’t consider poor coding or AI “hallucinations”
