rewrite this title NVIDIA Wants to Put AI Agents to Work in the Physical World – UC Today

NVIDIA has released a sweeping collection of open source agent tools and skills designed to automate the development of physical AI systems – the kind that power robots, autonomous vehicles, factory inspection lines, and hospital automation.

The release positions NVIDIA’s existing stack of hardware and simulation platforms – including Omniverse, Isaac, Cosmos, Metropolis, Alpamayo, and Jetson – as directly callable by AI coding agents.

The most practical effect is that workflows which previously required significant manual engineering effort can now be orchestrated and executed autonomously by AI agents, end to end.

“AI agents are revolutionising software development, and that shift is now coming to physical AI, extending into the systems that will transform transportation, manufacturing, healthcare and robotics,” said Jensen Huang, founder and CEO of NVIDIA.

“When agents can directly use NVIDIA libraries, models and frameworks, physical AI development will move faster, enabling developers to build the robots, autonomous vehicles and industrial systems of the future at an incredible pace.”

From Code Generation to Physical Orchestration

Until recently, AI agents have largely operated in the realm of software – writing code, summarising documents, answering queries.

NVIDIA’s announcement signals an industry push toward agents that can manage far more complex, multi-step technical processes in the physical world.

The new skills – packages of optimised, repeatable instructions – tell agents which tools to call, what outputs to produce, and how to validate results across the full physical AI development pipeline.

That includes generating synthetic training data, running simulations, fine-tuning models, automating labelling, and managing deployment to edge hardware.

For developers building robots or autonomous systems, this means the gap between a working prototype and a production-ready, continuously improving system gets considerably narrower.

Rather than hand-configuring each stage of the pipeline, teams can direct agents to handle orchestration while engineers focus on higher-level design and validation decisions.

Omniverse at the Centre of the Workspace AI Story

Omniverse – NVIDIA’s platform for building and simulating industrial digital twins – sits at the heart of several key use cases announced alongside the toolkit.

Industrial software companies including Cadence, Dassault Systèmes, Siemens, and Synopsys are using Omniverse libraries and agent skills for engineering data inspection, simulation, and interactive digital twins.

PTC and others are using it alongside OpenUSD-based workflows to convert CAD data into simulation-ready environments.

The implication is that the digital twin – long discussed as a future-state concept in enterprise tech circles – is becoming an active, agent-driven workspace in its own right.

Physical spaces like semiconductor fabs, hospital wards, and manufacturing floors are being modelled, simulated, and optimised by AI before any real-world changes are made. SK hynix, for instance, is building semiconductor fab digital twins using Omniverse as part of its Autonomous Fab 2030 roadmap.

This is immersive and spatial computing technology being applied not to consumer entertainment, but to some of the most demanding operational environments in industry.

Real-World Results Are Already Emerging

Nvidia reported a series of performance figures from early adopters that give a sense of how the tooling is being used in practice.

According to the company, Pegatron – an electronics manufacturer – reported a 67 percent reduction in model training and deployment time by using NVIDIA’s Defect Image Generation skill to produce synthetic training data for visual inspection systems.

Delta Electronics used the same skill to improve defect detection rates by 17 percent on metal busbar soldering lines.

Foxconn also reportedly recorded a three percent improvement in first pass yield on manufacturing lines. Inventec said it reduced defect data collection effort for laptop chassis manufacturing by 30 percent.

In autonomous vehicles, Li Auto, Afari, and DeepRoute.ai are using NVIDIA Omniverse NuRec models to generate more than 300,000 renders and simulations per day, accelerating training and evaluation of their AV systems.

What This Means for IT and Technology Leaders

For IT and technology leaders in enterprise environments, NVIDIA’s announcement is worth tracking closely.

The shift here is from AI as a productivity layer sitting on top of existing workflows, to AI as an active orchestrator of technical infrastructure.

When agents can autonomously manage simulation pipelines, fine-tune models, and deploy to edge hardware, the governance, security, and integration questions that IT teams are already navigating with software-side AI tools become considerably more complex.

NVIDIA has included security and governance tooling in the release – the NemoClaw blueprint and OpenShell runtime offer policy-based security and privacy controls for local or cloud deployments – but enterprise adoption at scale will require IT teams to think carefully about how autonomous physical AI workflows fit within existing operational frameworks.

The toolkit is now available through GitHub and skills.sh, with cloud integrations from Microsoft, CoreWeave, and Nebius. Preconfigured environments for synthetic data generation are available to trial on NVIDIA Brev.