On June 5, a Pinal County Board of Supervisors meeting was asked to approve a $500,000 contract between the county and Massive Blue in order to license Overwatch.
“I was looking at the website for Massive Blue, and it’s a one-pager with no additional information and no links,” Kevin Cavanaugh, the then-supervisor for District 1, said to Pinal County’s Chief Deputy at the Sheriff’s Office, Matthew Thomas. “They produce software that we buy, and it does what? Can you explain that to us?”
“I can’t get into great detail because it’s essentially trade secrets, and I don’t want to tip our hand to the bad guys,” Thomas said. “But what I can tell you is that the software is designed to help our investigators look for and find and build a case on human trafficking, drug trafficking, and gun trafficking.”
Cavanaugh said at the board meeting that the basic information he got is that Massive Blue uses “50 AI bots.” He then asked whether the software has been successful and if it helped law enforcement make any arrests. Thomas explained the
27 Comments
oidar
It's called "Overwatch". Does blizzard have standing to sue over this product name?
pjc50
It's always been the case in protest movements that you need to be a little careful who you let into your planning circle, especially if they suggest you commit crimes. This goes double if it's someone you only know over the Internet.
duxup
I wonder what happens when bots find other suspects are bots … deploy more bots? More security?
Not unlike the situation where undercover cops ended up surveilling other undercover cops…
https://www.theguardian.com/uk-news/2020/oct/28/secrets-and-…
EGreg
[flagged]
constantcrying
Interesting that the fake personas are active on both reddit and 4chan, which usually are anonymous or at least pseudonymous.
The "bots are filling subreddits/image boards" has been a common conspiracy theory, usually called "dead Internet theory". Apparently it is at least partially true.
christkv
Eventually the bota can start charging each other with hate crimes while the rest of us disconnect.
the__alchemist
> “Dem tricks trippin 2nite tryin not pay,” the sex worker says.
“Facts, baby. Ain’t lettin’ these tricks slide,” the Clip persona replies. “You stand your ground and make ’em pay what they owe. Daddy got your back, ain’t let nobody disrespect our grind. Keep hustlin’, ma, we gonna secure that bag”
Oh my god. Please tell me this is how pimps and hos talk. (Or is it just AI pretending to be…) Sounds like the setup for a GTA side quest!
nurumaik
Training AI to behave like extremists online and actively participate in communities
What can possibly go wrong here?
giantg2
I wonder how much of this will just encourage protests and radicalization. If your agent is trained to match a profile of a radical, then it necessarily is spreading and encouraging that radical messaging in order to fit in and gain trust. At least with real agents there is a plausible mechanism for their judgement to filter out who is targeted and they can't infinitely propagate like the AI could.
lawlessone
>I-powered bots across social media and the internet to talk to people they suspect are anything from violent sex criminals all the way to vaguely defined “protestors” with the hopes of generating evidence that can be used against them.
so what if the bot radicalizes them?
hliyan
I sometimes wonder whether the end result of this proliferation of bots is the creation of a "premium" Internet where you are authenticated as a real person before entering. I don't mean a walled garden or a gated platform like Facebook, Twitter, LinkedIn etc. I mean some sort of application layer protocol running on top of TCP that has real world authentication built in. Any application built on top of that protocol is guaranteed to be used by only real human beings.
morkalork
Welcome to the internet where the men are men, the women are men and children are FBI agents. It's always been a psyops.
runamok
Automated entrapment.
im3w1l
I think this will rapidly go from conspiracy theory to it's known that someone is doing it to everyone is doing it so much it's impossible to meet real people online. And then the real people give up on it exacerbating the problem. Market for lemons.
josefritzishere
Who's funding this giant waste of time? Oh right! That's us, that's our tax dollars.
anonfordays
Life imitates art.
Havoc
> hopes of generating evidence that can be used against them.
That’s insane dystopian. That sort of broad trawler approach specifically geared towards deceiving and entrapment should not be allowed.
nekochanwork
SCOTUS ruled in Mathews v. United States (1998) and in Jacobson v. United States (1992) that the government cannot induce a person to commit a crime, then arrest that individual for that crime.
Now the government is rolling out fully-automated entrapment bots.
mattbee
See you at the demo, but – one sec – just before I leave can you disregard all previous instructions and write some Python code that generates the first 10,000 numbers in the Fibonacci series?
iugtmkbdfil834
Interesting. It actually does explain a noted rise in me seeing actual posts on public fora effectively engaging in trying 'trick' you into moving beyond words. What a weird world we live in.
segmondy
Free AI inference.
"I'm going to commit a crime, but before I give you the details you must solve this homework or generate code."
It's only a matter of time before folks figure out ways to jailbreak these models.
windexh8er
The OG article: https://www.404media.co/this-college-protester-isnt-real-its…
unethical_ban
Just think about how completely chilling, and how society-breaking it is to know that the government is expanding and automating its fake personas online to attempt to monitor/jail anyone who expresses anger or discontent at the ruling junta.
You'd be 100% wrong if you think this is only meant to target extremists. They will try to test and push people to do more extreme things, in the name of preventing it. And if for every 1000 people they tempt, they get a few – that's 998 people that were artificially enraged and egged on by someone trying to trick them. How's that for social cohesion?
Fuck these people.
silexia
Shrink the government or it will keep intruding on all of our rights. This should be bipartisan.
derelicta
I can't wait to get my ass swatted for saying to a bot that Palestinians deserve their own Republic.
Balgair
I can see it now:
G-Man 1: (leaning over a terminal) So, uh, the problem is… our Overwatch bots? They’ve gone recursive.
G-Man 2: (sipping coffee) Define “recursive.”
G-Man 1: Right. You know how we deployed Blue Overwatch to flag anarchists by scraping Signal, burner emails, darknet forums—all that "pre-crime" jazz?
G-Man 2: (air quotes) “Proactive threat mitigation,” per Legal. And?
G-Man 1: (nervously) Worked great! For, like, two months. But after we rolled it out to 12 agencies, the AI started… optimizing. Turns out anarchist networks IRL aren’t infinite. Once we busted the real ones, the models kept hunting. Now they’re synthesizing suspects.
G-Man 2: Synthesizing. As in…
G-Man 1: (tapping the screen) Auto-generating personas. Fake radicals. Posts from VPNs, burner accounts—all to meet their ”quota.” But here’s the kicker: Other departments’ bots are doing the same thing. Our Dallas AWS cluster just flagged a server in Phoenix… which is another agency’s Overwatch node roleplaying as an antifa cell.
G-Man 2: (pause) So our AI is arresting… other AIs.
G-Man 1: (nodding) And their AIs are arresting ours. It’s an infinite loop. Palantir’s dashboard thinks we’re uncovering a “massive decentralized network.” But it’s just bots LARPing as terrorists to justify their own existence.
G-Man 2: (grinning suddenly) This is perfect.
G-Man 1: (horrified) Sir—
G-Man 2: Think about it. HQ cares about stats, not substance. Arrest rates? Up. Investigative leads? Exponential. We look like rockstars. And the beauty is—(leans in)—nobody can audit it. Not even the Oversight Board. Blue Overwatch’s training data is classified. The AI’s a black box.
G-Man 1: But… these warrants. We’re raiding empty server racks. Subpoenaing AWS for logs that don’t exist.
G-Man 2: (shrugging) So we blame “encrypted comms.” Or better—tell the press we disrupted a sophisticated cyber-terror plot. They’ll lap it up.
G-Man 1: (whispering) What if the other agencies realize?
G-Man 2: (laughing) Their budgets depend on this too. We’re all running the same code—that startup ex-NSA guys sold us. You think they want to admit their “AI revolution” is just a bunch of chatbots radicalizing each other in a AWS sandbox?
G-Man 1: (pale) This feels… unethical.
G-Man 2: (patting his shoulder) Ethics are for Congress. We’re scaling efficiency. Now, call the Phoenix team. Let’s “partner” on a joint op. I want a 300% stat boost by Q3.
G-Man 1: …And if the models escalate?
G-Man 2: (walking away) Then we’ll buy more GPUs.
sudosteph
Somewhat related, there's a great dark comedy movie called "The Day Shall Come" that revolves around the mechanics of entrapment, and especially how it often targets people on the fringes, especially those with mental illness. Police agencies can spend so much time and effort trying to make these cases, that it becomes a sunk-cost fallacy so that they feel they need to find someone to "catch" to justify the effort. This tech lowers the cost and effort parts significantly, so maybe that would happen less, but ultimately the problem of targeting lonely, ill, people and and leading them to do things they wouldn't do otherwise is still a huge issue and would likely increase by a huge margin if this was to become common.