David Parry
This is a very serious incident, not because patient data was accessed, which appears not to have been, but because the agent was attacking “non-consenting” sites. The boundary between searching for information across the web, which is what “crawlers” like Google have been doing for 20+ years, and actively using vulnerabilities to access non-public information is very clear to humans but not apparently to these agents.
The difference between peering into a shop window, ringing the doorbell and trying a door to see if the shop is open and crawling through an open window is obvious to us but these agents have not yet learned this.
Although I’m sure OpenAI will apologise and limit some of this, these models are out there and criminals, the curious and foreign states will not be so kind. It's quite possible that actually teaching these agents “manners” – appropriate behaviour - is actually a more difficult task than we anticipate. It is a wake-up call for organisations to improve cybersecurity testing; the government should support this and also reduce the confidential information they hold and do not need.