OpenAI CEO Sam Altman.Sebastian Gollnow/picture alliance via Getty ImagesOpenAI is rolling out its Computer Use tools into ChatGPT, part of a push to automate more work.The tools can perform actions in a computer's apps and work online inside a web browser.OpenAI staff spoke about the technology's capabilities and what still needs to be figured out.OpenAI staffers say they're close to realizing one of the AI lab's longest-standing ambitions.Since the company's 2015 founding, leaders have hoped to one day build AI agents that can use a computer — and, crucially, a web browser — with the same dexterity as a human. A huge challenge has been collecting and creating the data to train these agents. OpenAI now has enough faith in the tech to start rolling out Computer Use tools to customers.This year, the company launched a Chrome extension that lets ChatGPT take over the browser, created cloud and in-app browsers for ChatGPT to interact with public websites, and added Computer Use to its Codex coding tool.The same underlying tech now also lets users have ChatGPT complete tasks on other apps. OpenAI's president, Greg Brockman, wrote on X that an April update made his company's tech "no longer just for coders, but for anyone who does computer work."Employees who work on the tools for OpenAI told Business Insider that while there's room for improvement, the tools are at an inflection point. Rival Anthropic is also racing to improve its version of Computer Use and was the first to market in 2024. As of May, at least 600,000 organizations had tried Anthropic's Claude Cowork feature, which uses the tool.Now, OpenAI says its tech's capabilities have caught up, and it's filling its blockbuster ChatGPT with ways to automate computer tasks, betting that users will turn over more and more work to it."Once ChatGPT can use computers and software faster than you or I can, it's going to change the way that you, by default, want to interact with your computer," Ari Weinstein, a manager on OpenAI's Computer Use team, told Business Insider.How OpenAI trained and built Computer UseEach phase of training for OpenAI's newest models included data or processes that improved them for Computer Use. Zhou Yu, an AI researcher at Columbia University who worked on a popular benchmark test for computer-use agents, told Business Insider that this type of extra training builds on a standard AI model's capabilities and teaches it how to handle more tasks.At the earliest phase, where OpenAI builds a model's base with a vast amount of data, the company likely included frame-by-frame screenshots, Yu said, to "front-load" helpful information for Computer Use.Then, in a later step, developers can provide the models with data showing the ideal inputs that produce a given output — in this case, the computer functions that lead to that result, such as filling out a tax form or 3D-modeling an object.This data can come from human trainers: OpenAI, when it announced a 2025 version of Computer Use, said it had used datasets where people demonstrated how to complete tasks. (OpenAI declined to provide specifics about its latest training data.) Yu said this type of data is expensive since it's hard to obtain and clean up for use.OpenAI also likely improves the models' computer-use skills by ordering them to try to complete virtual tasks and rewarding successful behavior — known as reinforcement learning, where the model "can learn to do more of these actions that have gotten rewarded," Yu said.OpenAI has upgraded how its models interact with a computer and process data while in use, Weinstein said. Previously, ChatGPT would take a screenshot of a user's computer, analyze pixels, inject a command, and repeat. Now, when the tool has a website open, it can swiftly read the page's guts — its memory structure, accessibility information, and link connections.The tool still takes screenshots; users are asked to let ChatGPT record their screen as they set up Computer Use, so it can rapidly analyze what's on-screen. Weinstein said the mix of tools helps provide OpenAI's Codex with a way to test the code it's built, find bugs, and make improvements in a "full loop."That's the use case for Cristian Medina Ruiz, a hobbyist coder in the Czech Republic. He showed Business Insider his attempt to use Codex to rebuild the 2013 city-building game "SimCity" from its original source code. The Computer Use tool opened a window on his computer to verify that the game correctly simulated car movements and building images.Ruiz said Computer Use got him wholeheartedly into ChatGPT, and he now leaves his code running and iterating when he's away from his computer."It does these things even when I'm away," he said.OpenAI wants you to use ChatGPT to interact with the open internetWeinstein and James Sun, who works on browser capabilities at OpenAI, told Business Insider that part of the technology's promise is connecting ChatGPT's agents to disparate parts of the internet and its "long t
Don't miss out on breaking stories and in-depth articles.