Employees are submitting sensitive business data and privacy-protected information to large language models (LLMs) such as ChatGPT, raising concerns that artificial intelligence (AI) services could be incorporating the data into their models, and that information could be retrieved at a later date if proper data security isn’t in place for the service.
In a recent report, data security service Cyberhaven detected and blocked requests to input data into ChatGPT from 4.2% of the 1.6 million workers at its client companies because of the risk of leaking confidential information, client data, source code, or regulated information to the LLM.
In one case, an executive cut and pasted the firm’s 2023 strategy document into ChatGPT and asked it to create a PowerPoint deck. In another case, a doctor input his patient’s name and their medical condition and asked ChatGPT to craft a letter to the patient’s insurance company.
And as more employees use ChatGPT and other AI-based services as productivity tools, the risk will grow, says Howard Ting, CEO of Cyberhaven.
“There was this big migration of data from on-prem to cloud, and the next big shift is going to be the migration of data into these generative apps,” he says. “And how that plays out [remains to be seen] — I think, we’re in pregame; we’re not even in the first inning.”
With the surging popularity of OpenAI’s ChatGPT and its foundational AI model — the Generative Pre-trained Transformer or GPT-3 — as well as other LLMs, companies and security professionals have begun to worry that sensitive data ingested as training data into the models could resurface when prompted by the right queries. Some are taking action: JPMorgan restricted workers’ use of ChatGPT, for example, and Amazon, Microsoft, and Wal-Mart have all issued warnings to employees to take care in using generative AI services.
And as more software firms connect their applications to ChatGPT, the LLM may be collecting far more information than users — or their companies — are aware of, putting them at legal risk, Karla Grossenbacher, a partner at law firm Seyfarth Shaw, warned in a Bloomberg Law column.
“Prudent employers will include — in employee confidentiality agreements and policies — prohibitions on employees referring to or ente
