Arich Lawson | Good pictures
On Thursday, OpenAI declared A plugin system for its ChatGPT AI assistant. These plugins give ChatGPT the ability to interact with the wider world through the Internet, including booking flights, ordering groceries, surfing the web, and more. Plugins are bits of code that tell ChatGPT how to use an external resource on the Internet.
Basically, if a developer wants to give ChatGPT the ability to access any network service (for example: “finding current stock prices”) or any task controlled by a network service (for example: “ordering a pizza over the Internet”), it’s now possible, as long as it doesn’t violate OpenAI’s rules.
Usually, very large language models (LLM) such as ChatGPT are confined within a bubble, so to speak, and can only interact with the world through text conversations with a user. As OpenAI writes in its introduction Blog Of the ChatGPT plugins, “Out of the box the only thing language models can do is output text.”
Bing Chat has allowed this paradigm to search for more recent information on the web, but so far ChatGPT has been isolated from the wider world. When locked in this way, ChatGPT can only retrieve data from its training set (2021 and earlier) and information provided by the user during the conversation. Also, ChatGPT is prone to make factual errors and mistakes (what AI researchers call “hallucinations”).
To overcome these limitations, OpenAI has developed Bubble Popping ChatGPT plugin interface (OpenAI calls ChatGPT’s “eyes and ears”) This allows developers to create new components that “plug” into ChatGPT and allow the AI model to interact with other services on the web. These services can Do the calculations and specify factual information to reduce hallucinations, and if developers create a plug-in for that task, they can interact with any other software service on the Internet.
What kind of plugins are we talking about?
For ChatGPT, OpenAI will allow users to select from a list of plugins before starting a ChatGPT session. They look almost identical to apps in the App Store, with each plugin having its own icon and description.
OpenAI says the first round of plugins were developed by the following companies:
- Expedia (for travel planning)
- Financial Note (For real-time market data)
- Instacart (for grocery ordering)
- Kayak (search for flights and rental cars)
- Clarina (for price-comparison shopping)
- Milo (AI-Powered Parental Assistant)
- OpenTable (for restaurant recommendations and reservations)
- Shopify (to shop on that site)
- Slack (for communication)
- talk (for AI-powered language training)
- Wolfram (for calculations and real-time data)
- Zapier (an automation platform)
In particular, the Zapier plugin appears to be very powerful because it provides ChatGPT access to an existing software automation system, Or as Zapier You can ask ChatGPT to perform over 50,000 actions (including searching, updating, and writing) with any of Zapier’s 5,000+ supported apps. It’s contacts in CRM that turn chat into actions, then update them directly or add rows to a spreadsheet, then send them as a Slack message. The possibilities are endless.”
OpenAI also provides three plugins, a web browser (that can retrieve information from the web in a Bing Chat-like manner), a code interpreter for running Python programs (in a sandbox), and a retrieval tool that allows access. Individual or institutional” sources of information provided elsewhere (basically, extracting information from documents).
Demo video of ChatGPT plugins from OpenAI.
While OpenAI calls the plugin selection process a “plugin store,” the company has not announced plans to sell individual plugins. But by using the “store” label, that decision may be at some point.
Already, developers with early access Quickly prototype plugins to ChatGPT. Compared to other approaches in plugin development, the way ChatGPT plugins work is remarkable. “Instead of an arcane process of usingGlue code” To interface an API with ChatGPT, a developer basically “tells” ChatGPT how to use their service using natural language, and ChatGPT does it.
For those who don’t know: you write an OpenAPI manifest for your API, use human language descriptions for everything, and that’s it. Lets you figure out how to validate the model, serialize calls, process data in between, format the view, etc. There is absolutely zero glue code.
— Mitchell Hashimoto (@mitchellh) March 23, 2023
Beyond that, developers use ChatGPT GPT-4 ChatGPT lets you write plugin manifests (a manifest is “a machine-readable description of a plugin’s capabilities and how to implement them,” according to OpenAI), further simplifying the plugin development process.
This kind of self-assembly feels like growth potential unspecified territory For some programmers. In one case, a Twitter user named Rohit Worry out loud,”friends existential crisis. Finished OpenAI software? What to do besides clean and sweep?“
Sam Altman, CEO of OpenAI replied“No.”
Is it safe?
Constant diffusion
While OpenAI has previously tested its AI models (such as GPT-4) to see if they have the agency to adapt, improve, and propagate themselves in the world’s computing systems, it’s no surprise that OpenAI spends nearly half of its ChatGPT plugins. Blog Talks about security and vulnerabilities. “Plug-ins can have a wide range of social impacts,” the company casually notes in one section Possible impacts on jobs.
Beyond jobs, a persistent fear among some AI researchers involves giving advanced AI models access to other systems, where it could cause harm. An AI system is not necessarily “sentient” or “sentient”, it is only driven to complete a specific task it deems necessary. In this case with plugins, OpenAI seems to be doing exactly that.
OpenAI: Important for security is that AI-generated code does not have direct real-world consequences. So we disabled internet access in the REPL provided by ChatGPT-4
And OpenAI: We’ve partnered with Zapier to power ChatGPT-4 to enable over 50,000 actions across 5,000 apps. pic.twitter.com/IAhOTGDOlx– David 🎇 (@David) March 24, 2023
OpenAI appears to be aware of the risks, often mentioning them GPT-4 system card This describes the type of worst-case scenario testing we described in the previous article. Beyond hypothetical doomsday scenarios, AI-powered harm may come in the form of accelerated versions of current online dangers, such as automated phishing rings and disinformation campaigns. Astroturfingor personal attacks.
“Plugins risk increasing security challenges by performing malicious or unintended actions, increasing the abilities of bad actors to deceive, mislead, or abuse others,” OpenAI writes. “By increasing the range of possible applications, plug-ins can increase the risk of negative consequences due to incorrect or incorrect actions taken by the model in new domains. From day one, these factors have guided the development of our plug-in platform, and we have implemented many safeguards.”
One of these safeguards appears to be phasing out access to plugins. Also, the ChatGPT plugin application is wrapped in the blanket of OpenAI Usage Policy Prohibits its use to create false information and prohibited forms of content, and specifies Rules for plugins, such as a ban on automating conversations with real people. Also, plugins that use content generated by ChatGPT (such as emails) must disclaim that the content was generated by AI.
Individual OpenAI plugins have their own security disclaimers, including the ability to opt out of the ChatGPT crawler robots.txt file and the Python code interpreter running in a “firewall” sandbox. But do those restrictions apply to plugins for other services that can run code? These are the questions that OpenAI and developers will need to work on together in the days, weeks and months to come.
At this time, ChatGPT plugins are only available to select developers and authorized users on an alpha basis. Waiting list. “While we will initially prioritize a small number of developers and ChatGPT Plus users, we plan to provide greater access over time.”