As OpenAI’s current VP of Product, Peter runs the company’s product and commercialization efforts. Before that, he played a crucial role in researching and developing one of OpenAI’s most well-known products: GPT-3 API.
But despite being a founding member of OpenAI’s Robotics Research team. Peter actually had reservations regarding robotics. He felt its standard procedures were too slow or too clunky to efficiently meet real world demands.
What changed his mind?
His Academic Endeavors
For the record, Peter was always interested in machine learning and artificial intelligence.
His curiosity was piqued in high school, when he read a book about AI. That initial spark encouraged him to continue exploring similar disciplines, eventually leading to an undergraduate degree in physics. He then pivoted to neuroscience in the California Institute of Technology, believing this was a practical approach to pursuing his passion.
The argument was sound–but it just wasn’t meant to be.
Micro-Implants for Mice
Peter’s most vivid experience in neuroscience involved him sitting for hours at a time in a basement, building micro-implants meant to be inserted in rat’s brains.
While this scene certainly wouldn’t be amiss in a geeky sci-fi television series, Peter grew sick of it in less than a year. The process, he said, was lonely. The micro-implants took three months to build. Then came the actual surgery needed to insert the implant in the rat.
And if, at any point during the project, an error occurred, it was back to square one.
Eventually, Peter realized that neuroscience wasn’t the career for him. He felt that if he continued down this path, it would take him forever to get to grad school. Plus, he wanted to focus more on robotics. So he shifted to get a PhD in Computation and Neural Systems.
A Similar Problem
Unfortunately, robotics had a similar problem. It took way too long to produce useful and usable results. The process of designing the robot, building the robot, and then eventually programming the robot to make sure it worked as intended was, as Peter puts it, probably three quarters of his PhD. That wasn’t including the experiments they would have to run at the end, too.
So he took a more specific, specialized approach this time. He decided to pick just one aspect of the process and see where that would take him.
This allowed him to focus on computer vision–a process that would jump-start his work (and subsequent breakthroughs) with image organization and OCR, or Optical Character Recognition.
Animals & Anchovi Labs
Peter’s work with computer vision and OCR engines inspired him to create his own startup, Anchovi Labs. Their main product was an app that tracked images of animals using computer vision.
It was, for its time, an innovative concept. But it also wasn’t feasible. Costs were too high, demand was too low, and there just wasn’t enough market interest to recoup resources.
But he wasn’t deterred. Peter and his team shifted their focus to creating an app that used computer vision (still!) to independently organize images. This venture caught the attention of–and was later acquired by–Dropbox; one of the world’s largest file hosting and cloud storage service providers.
As its creator, Peter followed suit.
Dealing with the Dark Matter of Dropbox
When Peter first joined Dropbox in 2012, one of the biggest challenges he faced was dealing with the sheer volume of images stored in the server. There were so many photos (billions, he recalls) taking up so much space.
And they were useful to absolutely no one.
They were, according to Peter, basically like dark matter. And he was determined to do something about them. So he started simple; indexing the photos so that users could filter them by general data like date or location.
Once the files were organized, he then focused on helping users extract information from them.
This feature was most useful for business documents. Rather than scan the documents in question, most Dropbox users instead took pictures of them–to preserve them, to have their own copy, to have a digital backup, etc. But since pictures aren’t editable text files, organizing them and retrieving data from them was difficult.
So Peter and his team created a program that allowed users to retrieve only images of text documents (personal photos, family photos, sketches, and the like would not be brought up). Then, this same program would extract the data from the picture using OCR.
But instead of relying on existing OCR engines, they decided to build their own from scratch using deep learning algorithms. They created benchmarks based on the best OCR systems at that time, like Google and ABBYY.
“In three months, we had beaten all the public dataset benchmarks,” Peter says in an interview with Weights & Biases. “That was just mind-blowing to me. That’s the stuff that would have taken so much longer [to build] before.”
On Robots & OpenAI
In 2014, Peter founded Dropbox’s Machine Learning Team. They worked with other departments in the company to “identify, develop, and ship machine learning solutions” so they could improve and/or optimize existing products.
He left Dropbox two years later to become a founding member of the Robotics Research Effort at OpenAI.
Peter’s interest in robotics never really faded. He’d simply set it aside in favor of systems and processes that didn’t take quite as long and weren’t quite as clunky. His success in computer vision validated this decision as well.
But when the team at OpenAI started entertaining the potential of AI and AGI (artificial general intelligence), Peter’s interest was rekindled. He saw this as an opportunity to focus on problem-solving rather than publishing. People were getting results with deep learning and deep reinforcement learning, so he turned his attention there. And he realized soon enough that this was a practical and promising answer to his robotics problem.
Peter’s Projects & Current Career
As the Research Lead at OpenAI, Peter had the opportunity to work on a lot of robotics projects. Some of the more notable ones include:
- Training a robot hand to solve a Rubik’s Cube
- Robot Imitation Learning (where their robot eventually managed to beat the Dota 2 World Champion)
- OpenAI Remote Rendering Backend
- Learning Dexterity
After that, he became the Product, Engineering & Research Lead for the early development stages of OpenAI GPT-3 API. He was hands-on the whole time, leading his team through the grueling process of creating something that had literally never been done before.
Thankfully, Peter has fantastic leadership and critical thinking skills.
He recognized that OpenAI had ambitious goals. But he also believed that “big things” could be achieved with enough team effort. So rather than temper those goals with reality, he instead rose to the challenge.
And we should be thankful that he did. Otherwise, who knows what would have happened to ChatGPT.
Peter’s story is far from over but it already serves as encouragement and inspiration for students who share his passion. His journey is proof that you don’t need to get it right the first (or second, or even third!) time. With enough patience and perseverance, you’ll end up on the path you were meant to take all along.
