Altman said on a podcast in August that OpenAI has tried to resist the temptation to introduce products that could “juice growth or revenue” but be “very misaligned” with its long-term mission. Asked for a specific example, he gave one: “Well, we haven’t put a sexbot avatar in ChatGPT yet.”
Idaho-based startup Civitai, a platform for AI-generated art, learned the hard way that making money off mature AI won’t be an easy path.
“When we launched the site, it was an intentional choice to allow mature content,” said Justin Maier, the company’s co-founder and CEO, in an interview last year.
Story continues below this ad
Backed by the prominent venture capital firm Andreessen Horowitz, which has also invested in OpenAI, the Idaho startup was one of several that tried to capitalize on the sudden popularity of tools like Stable Diffusion and Midjourney that enabled people to type a description and conjure up almost any kind of image. Part of Stable Diffusion’s initial popularity was the ease with which it could generate a new kind of synthetic and highly customized pornography.
“What we had seen was that there was a lot of interest in mature content,” Maier said. Training these AI systems, known as models, on “mature themes actually made it so that these models were more capable of human anatomy and resulted in actually better models,” he said.
“We didn’t want to prevent the kind of growth that actually increased everything for the entire community, whether you were interested in mature content or Pixar,” Maier said. “So we allowed it early on and have always kind of had this battle of making it so that we can keep things filtered and safe, if that’s not what you’re interested in. We wanted to ultimately give the control to the user to decide what they would see on the site and what their experience would be.”
That also invited abuse. Civitai last year implemented new measures to detect and remove sexual images depicting children, but it remained a hub for AI-generated pornography, including fake images of celebrities. Confronting increasing pressure, including from credit card processors and a new law against nonconsensual images signed by President Donald Trump, Civitai earlier this year blocked users from creating deepfake images of real people. Engagement dropped.
Story continues below this ad
Another company that hasn’t shied away from mature content is Baltimore-based Nomi.ai, though its founder and CEO Alex Cardinell said its companion chatbots are “strictly” for users over 18 and were never marketed to kids. They are also not designed for sex, though Cardinell said in an interview earlier this year that people who build platonic relationships with their chatbot might find it veering into a romantic one.
“It’s kind of very user-dependent for where they’re kind of missing the human gap in their life. And I think that’s different for everyone,” he said.
He declined to guess how many Nomi users are having erotic conversations with the chatbot, comparing it to real-life partners who might do “mature content things” for some part of their lives but “all sorts of other stuff together as well.”
“We’re not monitoring user conversations like that,” Cardinell said.
Story continues below this ad
Altman’s announcement that erotica for adults could arrive on ChatGPT in December came a day after California Gov. Gavin Newsom vetoed legislation that would have banned companies from making AI chatbots available to anyone under 18 years old if it was “foreseeable” that they would engage in “erotic or sexually explicit interactions” with kids or encourage them to harm themselves. The tech industry lobbied heavily against the bill, which Newsom said was too broad, but OpenAI, Meta and others introduced new age restrictions and parental controls for AI-teen interactions.