I mean this with the biggest offence possible: AI judges make no sense, atleast with the current way of doing AI (LLMs). It’s been known for years that they amplify any bias in their training data. You are black? Higher chance of going to prison and longer serving time. Getting divorced and are male? Your ass is NOT getting custody. Hell, even without that the LLM might just hallucinate some crime not in the data for a case and give you a life time prison sentence. And if you somehow manage to avoid all that, what’s stopping somebody from just shadowprompting it and getting the judgement they want? It would also be an easy target for corruption, the goverment wants their poltical rivals gone? Tweak the model so it’s just that bit harsher, or just a bit more alligned with some other interpretatnion of the law.
Who would even choose the training data? The judges? Why would they, it would be better for them to sabotage and keep their jobs. Some goverment agency then? Don’t want to do that, or you’re gonna find out separation of power has a reason.
I mean this with the biggest offence possible: AI judges make no sense, atleast with the current way of doing AI (LLMs). It’s been known for years that they amplify any bias in their training data. You are black? Higher chance of going to prison and longer serving time. Getting divorced and are male? Your ass is NOT getting custody. Hell, even without that the LLM might just hallucinate some crime not in the data for a case and give you a life time prison sentence. And if you somehow manage to avoid all that, what’s stopping somebody from just shadowprompting it and getting the judgement they want? It would also be an easy target for corruption, the goverment wants their poltical rivals gone? Tweak the model so it’s just that bit harsher, or just a bit more alligned with some other interpretatnion of the law.
Who would even choose the training data? The judges? Why would they, it would be better for them to sabotage and keep their jobs. Some goverment agency then? Don’t want to do that, or you’re gonna find out separation of power has a reason.
Bad idea.