- cross-posted to:
- technology@midwest.social
- cross-posted to:
- technology@midwest.social
I remember when it first came out that grok was making csam, the elon stans were all saying it was a false flag by feminists to make him look bad. Nope, turns out the literal training model was child porn!
You must log in or register to comment.
At this point I think it’s fair to say that Musk is specifically positioning grok as the csam ai service. Not as a principle offering, but xai’s moves to get immunity for its generations and unwillingness to implement proper safeguards both point in that direction. Chinese firms prove that you can have AI services make money while also being responsible for their own generations (and incidentally chinese firms also prove that you can do AI services without building thousands of datacenters nobody wants).
Fucking lock this piece of shit up forever for creating and distributing child porn, and the ones who wrote the prompt tooSorry can’t do that, we live under capitalism, so I’m going to give him another trillion now.
Im starting to this this Elon Musk and his X company are not good. Chat what do we think?
Looking into it
Thank you for your attention to this matter
What’s the bounty on Elon at now?
tech pedo guy surprises no one
true patriots know the code
I’m curious how they determined that it trained on the csam? Anything generated from it wouldn’t have the hashes that would alert? I’m not saying they didn’t train on it because I’d find it harder to believe that they didn’t, but I’m curious how Jane was alerted to it in the first place.
I assume with that either an investigator sorting through recovered images recognized her face, or they have additional facial recognition checks on top of whatever the image hashing method they use is.
That said, I’m also assuming that was from a pedophile doing an img2img request with an extant picture of her rather than the machine just spontaneously spitting it up on its own. A lot of the claims about how they think the training is being done sound dubious just because they don’t really line up with how image generators work. Like I’m pretty sure xAI doesn’t actually have any models of its own, just open weight models it’s lifted and finetuned, so I don’t know what they’re using now (they started out with Flux) but I’m assuming they’ve switched underlying models repeatedly as newer, better ones come out, with either their own finetuned checkpoints or their own LoRAs getting selected by some LLM based on the prompt.
The idea that they’re grabbing every image they see for training is also weird, because that’s going to be a bunch of random-quality images with no proper information in the form of descriptions or thorough tagging; they might be feeding that through an LLM (probably some qwen version) made to describe images and then dumping anything that has the right quality tags into a training set for a new finetune or LoRA, and considering how that’s an incredibly reckless, lazy, and likely self-defeating way to do it I can actually 100% believe that’s what they’re doing. If it were any other company I’d assume that not even techbro grifters would intentionally poison their model like that, but with xAI I can see whatever groyper dipshit grifted his way into the position just setting that up to run with no oversight so he can goon and skip work but still look “productive” on their metrics.
I’m not convinced they do. The lawsuit alleges it but only the courts will determine whether that actually happened.
What seems to have happened is Jane was a victim of this stuff and signed up to alerts of new instances of images of her being shared. Some pedophiles who had genuine CSAM of her (non-AI) were discussing and ended up using Grok or whatever to generate more similar (AI generated) images of her in that same vein. They might have just used it to put her face on it, they might have given it the CSAM itself and it didn’t refuse to use it as a template to generate more sexual images of her as a minor. Hard to say.
yeah i think that makes the most sense really
I imagine someone posted one of the original images along with the generated ones, so it was simple to make the connection. That being said, it’s pretty easy to get one of these models to recreate the original training data, especially if it is an outlier.
i don’t know much about llm images but it wouldn’t be able to generate a picture of a car without first having been trained on pictures of cars right? similarly,
of course, but my point was that the hashes wouldn’t match any known image because they’d always be different
There’s no way it’s that straightforward. You can tell an ai to generate a car with a hexbear logo paint job being sliced in half by a samurai sword and it will spit out an image that’s recognizable as such, even if the only examples it’s seen of cars being cut in half by samurai swords all have reddit logo paint jobs. It can definitely mash together concepts it’s never seen combined if there are enough examples of the building blocks in its dataset. I doubt this lawsuit would be proceeding without some very direct evidence.
I’m not a computer wizard but if theres a portion of a “hashed”[quotes only because i don’t fully know what it means] image inside a separate image, would it not come up like a partial fingerprint or something? Flagged for review on a partial match maybe?
“hashed”[quotes only because i don’t fully know what it means]
Hashing in this context is just a way of irreversibly reducing something down to a unique identifier. Like if you were to convert every letter in a bit of text into a number and then add those numbers together, that would give you a very basic hash of that text. For images like that, it’s something way more complicated and secretive, because it has to reliably give matches even when an image is transformed or cropped. I can’t even begin to speculate on how its done, just that it’s supposedly reliable.
I assume it wasn’t that, but rather was some secondary check used in reviewing recovered images that tries to match individuals in newly discovered images to ones in known cases, but that the distinction was lost somewhere in the story being relayed.
I’m learning so much lately thank you
There are different forms of hashing. The most common form that everyone thinks of is cryptographic hashing which by design produce completely separate outputs with a single small change in input. For image similarity a ‘perceptual hash’ is used which tries to instead minimise hash changes from inputs and has concepts of similarity between hash values. All reverse image searches do this.
A really basic example to think about this would be to ‘merge’ the colour values of all pixels in an image together into 1 colour, and now you can compare it with other results to find similarly coloured images. This has obvious drawbacks when you have really colourful images and a real algorithm would have to be more advanced but the general idea is the same.
Edit to add: All hash functions take a big input and convert it into a smaller output. This means you can have ‘collisions’ where 2 inputs make the same output. This is even more likely to happen when your algorithm tries to make similar inputs give similar outputs and is a major problem with supposedly benevolent attempts to put csam scanning into private messaging services, because suddenly you can be flagged to authorities when pictures of your dog coincidentally generate similar hashes to pictures of abuse.
No wonder why Elon Musk works so many hours in his office.
