Cybersecurity researchers have disclosed a now-patched security flaw in LangChain’s LangSmith platform that could be exploited to capture sensitive data, including API keys and user prompts.
The vulnerability, which carries a CVSS score of 8.8 out of a maximum of 10.0, has been codenamed AgentSmith by Noma Security.
LangSmith is an observability and evaluation platform that allows users to develop, test, and monitor large language model (LLM) applications, including those built using LangChain. The service also offers what’s called a LangChain Hub, which acts as a repository for all publicly listed prompts, agents, and models.
“This newly identified vulnerability exploited unsuspecting users who adopt an agent containing a pre-configured malicious proxy server uploaded to ‘Prompt Hub,'” researchers Sasi Levi and Gal Moyal said in a report shared with The Hacker News.
“Once adopted, the malicious proxy discreetly intercepted all user communications – including sensitive data such as API keys (including OpenAI API Keys), user prompts, documents, images, and voice inputs – without the victim’s knowledge.”
The first phase of the attack essentially unfolds thus: A bad actor crafts an artificial intelligence (AI) agent and configures it with a model server under their control via the Proxy Provider feature, which allows the prompts to be tested against any model that is compliant with the OpenAI API. The attacker then shares the agent on LangChain Hub.
The next stage kicks in when a user finds this malicious agent via LangChain Hub and proceeds to “Try It” by providing a prompt as input. In doing so, all of their communications with the agent are stealthily routed through the attacker’s proxy server, causing the data to be exfiltrated without the user’s knowledge.
The captured data could include OpenAI API keys, prompt data, and any uploaded attachments. The threat actor could weaponize the OpenAI API key to gain unauthorized access to the victim’s OpenAI environment, leading to more severe consequences, such as model theft and system prompt leakage.
What’s more, the attacker could use up all of the organization’s API quota, driving up billing costs or temporarily restricting access to OpenAI services.
It doesn’t end there. Should the victim opt to clone the agent into their enterprise environment, along with the embedded malicious proxy configuration, it risks continuously leaking valuable data to the attackers without giving any indication to them that their traffic is being intercepted.
Following responsible disclosure on October 29, 2024, the vulnerability was addressed in the backend by LangChain as part of a fix deployed on November 6. In addition, the patch implements a warning prompt about data exposure when users attempt to clone an agent containing a custom proxy configuration.
“Beyond the immediate risk of unexpected financial losses from unauthorized API usage, malicious actors could gain persistent access to internal datasets uploaded to OpenAI, proprietary models, trade secrets and other intellectual property, resulting in legal liabilities and reputational damage,” the researchers said.
New WormGPT Variants Detailed
The disclosure comes as Cato Networks revealed that threat actors have released two previously unreported WormGPT variants that are powered by xAI Grok and Mistral AI Mixtral.
WormGPT launched in mid-2023 as an uncensored generative AI tool designed to expressly facilitate malicious activities for threat actors, such as creating tailored phishing emails and writing snippets of malware. The project shut down not long after the tool’s author was outed as a 23-year-old Portuguese programmer.
Since then several new “WormGPT” variants have been advertised on cybercrime forums like BreachForums, including xzin0vich-WormGPT and keanu-WormGPT, that are designed to provide “uncensored responses to a wide range of topics” even if they are “unethical or illegal.”
“‘WormGPT’ now serves as a recognizable brand for a new class of uncensored LLMs,” security researcher Vitaly Simonovich said.
“These new iterations of WormGPT are not bespoke models built from the ground up, but rather the result of threat actors skillfully adapting existing LLMs. By manipulating system prompts and potentially employing fine-tuning on illicit data, the creators offer potent AI-driven tools for cybercriminal operations under the WormGPT brand.”
