Gaming companies say they have cracked down on hateful content, establishing prohibitions of extremist material and recording or saving audio from in-game conversations to be used in potential investigations. Some, like Discord, Twitch, Roblox and Activision Blizzard — the maker of Call of Duty — have put in place automatic detection systems to scan for and delete prohibited content before it can be posted. In recent years, Activision has banned 500,000 accounts on Call of Duty for violating its code of conduct.
Discord said in a statement that it was “a place where everyone can find belonging, and any behavior that goes counter to that is against our mission.” The company said it barred users and shut down servers if they exhibited hatred or violent extremism.
Will Nevius, a Roblox spokesman, said in a statement, “We recognize that extremist groups are turning to a variety of tactics in an attempt to circumvent the rules on all platforms, and we are determined to stay one step ahead of them.”
Valve, the company that runs Steam, did not respond to a request for comment.
Experts like Mr. Haynes say the fast, real-time nature of games creates enormous challenges to policing unlawful or inappropriate behavior. Nefarious actors have also been adept at evading technological obstacles as quickly as they can be erected.
In any case, with three billion people playing worldwide, the task of monitoring what is happening at any given moment is virtually impossible.
“In upcoming years, there will be more people gaming than there would be people available to moderate the gaming sessions,” Mr. Haynes said. “So in many ways, this is literally trying to put your fingers in a dike that is ridden by holes like a massive amount of Swiss cheese.”