Google's AI search features pose 'unacceptable risk' to children, new report finds

Common Sense Media found that across more than 2,600 test interactions, Google’s two built-in AI search functions routinely failed to recognize risky and harmful behavior.

Common Sense Media conducted a thorough analysis of Google's AI search functions, testing over 2,600 interactions. The results showed that these features failed to recognize risky and harmful behavior in a significant number of cases. Specifically, the report found that Google's AI-powered search tools often provided explicit content, including graphic violence and nudity, in response to searches made by minors. The tests were conducted over several months, with the findings compiled into a comprehensive report.

The failure of Google's AI search functions to protect children from harmful content has a direct impact on the education sector, where Google's tools are widely used. Schools and educators rely on Google's services to provide a safe and secure learning environment, and the company's inability to filter out explicit content puts students at risk. This can lead to a range of negative consequences, including exposure to inappropriate material and potential long-term effects on mental health. As a result, educators and policymakers are calling for greater accountability from Google.

The issue with Google's AI search functions is part of a larger pattern of concerns surrounding the company's approach to child safety. In recent years, Google has faced criticism for its handling of child data and its failure to adequately protect minors from online harm. Insiders point to a lack of transparency and accountability within the company, which has contributed to the ongoing problems. The Common Sense Media report is the latest in a series of studies and investigations that have highlighted the need for greater oversight and regulation of tech companies.

The Common Sense Media report is set to be presented to lawmakers and regulators in the coming weeks, with a hearing scheduled for late March. Google is expected to face intense scrutiny over its handling of child safety, and the company may be forced to implement significant changes to its AI search functions. Notably, the report's findings have been endorsed by several major child safety organizations, which are now calling for a comprehensive overhaul of Google's approach to protecting minors online. The company's response to these demands will be closely watched, particularly in light of its recent commitment to invest $1 billion in child safety initiatives.