How roboticists are thinking about generative AI

Generative AI: From his time at Bot & Dolly to his current venture, Formant, Jeff Linnell underscores the pivotal role of software in robotics—a lesson reaffirmed by Google’s recent endeavors.

The topic of generative AI has become a frequent subject of discussion in my newsletter, Actuator. I must confess that a few months ago, I had some reservations about dedicating more time to this topic. Having reported on technology for an extended period, I’ve witnessed numerous hype cycles and experienced the resulting disappointments. Covering the tech world demands a balanced approach, involving a blend of healthy skepticism and genuine enthusiasm for the possibilities it offers.

This time around, it appeared that generative AI was patiently waiting in the wings, poised for its moment in the spotlight as the crypto market experienced a significant downturn. As cryptocurrencies faced a decline in interest, projects like ChatGPT and DALL-E were ready to take center stage, becoming the subjects of fervent reporting, hope, criticism, and the various stages of excitement and disillusionment that often accompany tech hype bubbles.

Regular followers of my work are aware that I was never particularly optimistic about cryptocurrencies. However, the landscape is different when it comes to generative AI. One key distinction is the widespread consensus that artificial intelligence and machine learning will play increasingly central roles in our lives in the future.

A prime example can be found in the world of smartphones. I frequently cover computational photography, an area that has witnessed remarkable advancements in recent years. Many manufacturers have managed to strike a commendable balance between hardware and software, resulting in both enhanced final products and lowered barriers to entry for users. Take Google, for instance, which has achieved some truly impressive feats with editing features like Best Take and Magic Eraser.

Certainly, these impressive tricks in computational photography are not just for show; they offer genuine utility rather than mere feature bloat. Looking ahead, the real challenge lies in seamlessly integrating them into the user experience. In an ideal future workflow, most users won’t need to understand what’s happening behind the scenes. They’ll simply appreciate that it works flawlessly, a strategy that closely resembles the classic Apple approach.

Generative AI distinguishes itself by providing a “wow” factor right from the start, setting it apart from previous hype cycles. When even the least tech-savvy family member can sit at a computer, type a few words into a chat box, and witness the creation of paintings and short stories, minimal conceptualization is required. This is a significant reason why these technologies have gained rapid acceptance. Most often, when everyday people are pitched cutting-edge technologies, it requires them to imagine how it might look or function in five or ten years.

With technologies like ChatGPT and DALL-E, you can experience their capabilities firsthand right now. However, this ease of access also makes it challenging to manage expectations. Without a deep understanding of AI, it’s easy to attribute intentionality to these systems. But that’s the current state of affairs. We often lead with attention-grabbing headlines, hoping that people will stay engaged long enough to delve into the intricacies behind the technology.

Spoiler alert: In most cases, they won’t, and suddenly we find ourselves spending months and years trying to bring expectations back to reality.

One of the perks of my job is the opportunity to discuss these topics with individuals much more knowledgeable than myself. They graciously take the time to explain complex concepts, and I strive to translate their insights for readers, with varying degrees of success.

As it became evident that generative AI would have a significant role in the future of robotics, I’ve been proactive in inserting questions about it into my conversations. Most experts in the field concur with the statement in the previous sentence, and it’s fascinating to see the wide-ranging impact they anticipate generative AI will have.

Also Read | With Character.AI, people and AI can chat together in groups

In a recent conversation with Marc Raibert and Gill Pratt, Gill explained the role that generative AI plays in their approach to robot learning:

“We have figured out how to use modern generative AI techniques that allow human demonstration of both position and force to essentially teach a robot from just a handful of examples. The code remains unchanged. This is based on something called diffusion policy, a collaborative effort with Columbia and MIT. We’ve already taught 60 different skills.”

Last week, when I asked Nvidia’s VP and GM of Embedded and Edge Computing, Deepu Talla, why the company believes generative AI is more than a passing trend, he stated:

“I think the results speak for themselves. You can already see the productivity improvement. It can compose an email for me. It’s not perfect, but I don’t have to start from scratch. It gets me 70% there. There are clear indications of significant improvements compared to the past. While it’s not perfect at summarizing, I wouldn’t hesitate to let it read and summarize for me. You can already see signs of increased productivity.”

Meanwhile, in my recent conversation with Daniela Rus, the head of MIT CSAIL, she explained how researchers are using generative AI to design robots:

“It turns out that generative AI can be quite powerful for solving motion planning problems. It provides faster and more fluid, human-like solutions for control compared to model predictive solutions. I think that’s powerful because the robots of the future will be less robotic and more fluid and human-like in their movements. We’ve also used generative AI for design, which is powerful and interesting. It’s not just about pattern generation for robots; the designs must make sense in the context of physics and the physical world. So, we connect them to a physics-based simulation engine to ensure the designs meet their required constraints.”

This week, a team at Northwestern University unveiled its own research into AI-generated robot design. They showcased how they designed a “successfully walking robot in mere seconds.” While it might not look like much now, it’s evident that with further research, this approach could be used to create more complex robotic systems.

“We discovered a very fast AI-driven design algorithm that bypasses the traffic jams of evolution, without relying on the biases of human designers,” said research lead Sam Kriegman. “We told the AI we wanted a robot that could walk on land, and with the press of a button, it generated a blueprint for a robot that looks nothing like any animal that has ever walked the earth. I call this process ‘instant evolution.'”

The AI program autonomously decided to equip the robot with legs. “It’s interesting because we didn’t tell the AI that a robot should have legs,” Kriegman added. “It rediscovered that legs are an efficient way to move on land. Legged locomotion is, in fact, the most efficient terrestrial movement.”

Jeff Linnell, founder and CEO of Formant, expressed his perspective on the impact of generative AI and robotics:

“From my perspective, generative AI and physical automation/robotics are going to revolutionize our understanding of life on Earth. We all recognize that AI is a game-changer and will affect every job, company, and student. I believe it’s symbiotic with robotics. You won’t have to program a robot; you’ll communicate with the robot in English, request an action, and it will figure it out. It’s just a matter of time for that to happen.”

Also Read | ChatGPT Creator OpenAI Says AI Tools Can Be Effective In Content Moderation

Before his involvement with Formant, Linnell was the founder and CEO of Bot & Dolly, a San Francisco-based company renowned for its contributions to the film “Gravity.” Google acquired Bot & Dolly in 2013, with the intention of propelling the robotics industry forward (although, as the saying goes, even the best-laid plans can go awry). Linnell emphasizes that his most significant lesson from that experience is the paramount importance of software in the field of robotics. This sentiment is further underscored by Google’s subsequent developments, such as Intrinsic and the integration of Everyday Robots into DeepMind, which seem to align with this perspective.