This week, host Anna Rose and co-host Kobi Gurkan chat with Yohei Nakajima, General Partner at Untapped Capital and creator of BabyAGI. They cover a wide variety of topics from the world of AGIs and agents to building no-code software in public. They kick-off with a chat about how Yohei’s interest in NFTs led him down the AI ‘rabbit hole’ and how he started to build out experiments in public that have inspired a new group of AI tools and projects.

They wrap up with a discussion about the possible impacts of some of this AI tech, how ZK may help mediate the challenges it introduces and more.

Here’s some additional links for this episode;

Check out the Modular Summit here: https://modularsummit.dev/

zkSummit 10 is happening in London on September 20, 2023! Apply to attend now -> zkSummit 10 Application Form

Anoma’s first fractal instance, Namada, is launching soon!

Namada is a proof-of-stake L1 for interchain asset-agnostic privacy. Namada natively interoperates with fast-finality chains via IBC and with Ethereum via a trustless two-way bridge.

For privacy, Namada deploys an upgraded version of the multi-asset shielded pool (MASP) circuit that allows all assets (fungible and non-fungible) to share a common shielded set – this removes the size limits of the anonymity set and provides the best privacy guarantees possible for every user in the multichain.

The MASP circuit’s latest update enables shielded set rewards directly in the shielded set, a novel feature that funds privacy as a public good.

Follow Namada on twitter @namada for more information and join the community on Discord discord.gg/namada

If you like what we do:

Transcript
Anna Rose (:

Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we'll be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online.

(:

This week, Kobi and I chat with Yohei Nakajima, General Partner at Untapped Capital and the creator of BabyAGI. We go a little bit off topic for the show and dive into the world of AIs and agents, we kick off with a chat about how Yohei's interest in NFTs led him down the AI rabbit hole, and how he started to build out experiments in public that have inspired a new batch of AI tools and projects. We then chat a little bit about the possible impact of some of this tech, the challenges it brings, how ZK may be able to solve some of these and more. Now, before we kick off, I want to let you know about an event I will be helping with. That is the Modular Summit happening in Paris during the EthCC week. This event is put on by the folks at Celestia and Maven 11, and this time around ZK Validator will be curating a short ZK track as part of it. zkPodcast will also be a media partner, and we're kicking off a sovereign radio show together. So it should be a lot of fun. If you're in town, be sure to join us. I've added the link in the show notes. Now. Tanya will share a little bit about this week's sponsor.

Tanya (:

Anoma's first fractal instance Namada is launching soon. Namada is a proof of stake L1 for interchain asset agnostic privacy. Namada natively interoperates with fast finality chains via IBC and with Ethereum via a trustless two-way bridge. For privacy, Namada deploys an upgraded version of the multi-asset shielded pool circuit, otherwise known as MASP, which allows all assets fungible and non fungible to share a common shielded set. This removes the size limits of the anonymity set and provides the best privacy guarantees possible for every user in the multi chain. The MASP circuit's latest update enables shielded set rewards directly in the shielded set, a novel feature that funds privacy as a public good. Follow Namada on Twitter @namada to learn more and join their community on Discord, discord.gg/namada. So thanks again, Anoma. And now here's our episode.

Anna Rose (:

Today we are here with Yohei Nakajima, General Partner at Untapped Capital and the creator of BabyAGI. Welcome to the show, Yohei.

Yohei (:

Thank you for having me.

Anna Rose (:

I first heard about you Yohei when Kobi first showed me the zkpod.ai product. So some of our audience might be familiar with this. This is the friendly ZK bot where you can ask it questions and it will answer in my or in Kobi's voice. When he put that together, I asked you Kobi, kind of like, where do you get these ideas from? What do you focus on? Who's inspiring this? And your name came up. And so it's really great to have you on the show to share a little bit about the work you've been doing.

Yohei (:

That's awesome to hear. I do share a lot of projects, but that's why I share it. There's so many fun ideas to build.

Kobi (:

Yeah, yeah, totally. When I started working on zkpod.ai, I think I basically learned almost everything I needed from Yohei's experiments on AI and his approach of building in public. So that was extremely useful to get going, but yeah, we'll hear about other interesting things that happened later on.

Anna Rose (:

Cool. Yohei, let's hear a little bit about your background. What were you working on just before this? How did you get into this field?

Yohei (:

Into AI? Well, before I got into AI, I was pretty deep in Web3. I think it was February of like before the Bored Apes, Cool Cats, big NFT boom. I kind of dove into Web3. I had known about Bitcoin and DeFi and I'd followed it, but I didn't go too deep. I think I had some trauma from studying finance in college. So like DeFi was like too similar to the stuff I was learning in class. But NFT's clicked for me. I'd worked with Disney and Nintendo, so I got the whole entertainment piece. So I dove in, launched my own NFT project called PixelBeasts, and I got pretty deep into Web3, and I think it was through the Web3 community that I first heard about the text to image models.

(:

Right. It was last summer it was Stable Diffusion, DALL·E 2 and Midjourney all came out last summer. And so they all come out at the same time and because all the NFT people were sharing art and started following a lot of art people, this popped off my Twitter feed. So actually the first tool I tried was Midjourney cause I got an invite to Discord and then I got access to DALL·E 2. At that point, I don't think I had access to Stability, but in playing with DALL·E 2, I had access to OpenAI. So I basically thought, okay, well I have an OpenAI account, I should go check out the other one that I'd been hearing about GPT-3. and then I started playing around with it, and that was the end of it.

Anna Rose (:land around:Yohei (:

Yeah.

Anna Rose (:

Okay. Had you had any contact to the DeFi Crypto land before that? Or this was your first step in?

Yohei (:

I had contact in the sense that, you know, as a VC, I touch, you know, as a generalist VC, I look at all industries, but DeFi was one of those industries that kind of felt like, unless I go deep enough, it's hard to invest. It kind of feels almost like biotech and DeFi. There are these industries where, like, there, there's so much depth to it that it's really hard to invest as like a generalist. And I felt that way about DeFi.

Kobi (:

But by the way, you've done some fun stuff with PixelBeasts, right? Like you've done, you've used it for games or for getting events.

Yohei (:

Actually, I think I tackled PixelBeasts similarly to the way I'm tackling AI. Actually, right before I launched PixelBeasts, I open sourced to what was a super simple chat app that was created a token gated like room for every NFT collection automatically.

Kobi (:

Nice.

Yohei (:

That was really, really clean. It was, it was one MySQL table. I stored zero user data. And then when you logged in with your wallet, it automatically just looked at which collections you were part of and generated a room for you that you could click into. And when you typed in a message, it just stored the NFT you were choosing, the room and the message itself. And then, so if someone else logged into the tool and they had the same collection, they suddenly had same access to the same room. So without storing any user data, it like automatically generated like mini communities. And it was just a fun little side project, but did a lot of things like that during the course of PixelBeasts as well.

Kobi (:

Yeah. I'm still trying to win this new NFT using the Discord game.

Yohei (:

Oh, yeah. Yeah. We have Snakes and Ladders, which is a Discord based board game. That's my one dimensional game. We have Pixels, which we were the first integration into, which is a 2D farming game. And then we have 3D PixelBeasts that I added to our metadata, which is already integrated into a game, but we haven't announced it yet. So when that's announced your PixelBeast will give you access to a 1D, 2D and 3D game.

Kobi (:

Oh cool.

Anna Rose (:urney though, because I mean,:Yohei (:

I still think it's, from personal aspect, I think it's absolutely fun, right? Being able to collect art. Like, I still have Pokemon cards that I'm giving to my kids, right? Like the whole space is still a lot of fun. On the business side, I do think there's fundamentally really powerful technology in Web3 that people started building because of the recent boom. But there's still stuff that's being built that we'll slowly see being rolled out over, you know, over time, I think so it's still technology to keep an eye on. What happens in NFT space is a little unknown to me, but as like a personal creator, right? Having these little drawings that I created and supporting a game of a small community is really fun. So, you know, I'll keep doing it. I kind of enjoy almost the current state better, where the people in Web3 are people who, you know, are really just building and there, I feel like at the top of the hype, I was almost more disillusioned by the hype than disillusioned by the crash.

Anna Rose (:

Well, you don't know who's real at the top, right? Like, you don't know who's seriously in it for good cause or just sort of passing through or maybe has even like bad intention. It's hard. It's really hard to discern. And I think when the tide goes out, then you see, you know, who's real. So you sort of mentioned the first connection point to the AI world is through images. So it's through these like AI generated images. Was this because of the generative art aspect of NFTs? Or did you see NFT projects that were trying to use some of this AI stuff?

Yohei (:

No, I think it was just the overlap, right? You had the suddenly, you know, huge community of people who were all online connected through sharing and talking about art. So when these powerful art models came out, I think the natural community that it spread through and I think I heard a lot of people in NFT space talking about Midjourney. And I got invited into the Midjourney Discord through my Web3 friends.

Anna Rose (:

And then, so what brought you into sort of the next stage of this and when, when are we talking? When did you kind of like, progress past the image stuff?

Yohei (:

So, image was a image was July, June or July. It was pretty quick. I did a couple projects, like I did like this collection of art called Animal Buildings. I did some PixelBeast glow ups where I did like realistic versions of my, you know, clean PixelBeasts. And I did a couple of fun little projects like that. And then I think it was the beginning of August, or end of July, so it was like month or two months of text image before I started playing with GPT3. Like I remember after my first prompt that I posted in Playground, I tweeted that picture saying uh oh. Because I knew I was going to, like it was just so fun like the first prompt I did, I knew it was going to be fun. I was like, oh, I started playing with this thing. And I was absolutely right. I just went down a rabbit hole. But it was like a very, very, very obvious first step. Like the moment I went into Playground to type something in, got an output, I was like, oh man, I'm going to spend a lot of time on this.

Anna Rose (:

Cool.

Kobi (:

Well, what was the first thing that was interesting?

Yohei (:

I don't remember actually what was the first prompt, but I do actually remember the first experiment I ended up building, which was Mini Yohei.

Kobi (:

Yeah.

Yohei (:

It was I basically you know, wrapped a Playground on a GPT text completion prong where you're a VC, you're an expert VC answering a question from a founder and I put that in our founder portal, which is for our portfolio companies. And I said, here's a VC expert for you called Mini Yohei. And I did something pretty clever there, which I haven't seen too many people do something similar yet. But I was using No-code. Right, because our founder portal's built with No-code. By the way, No-code was my deep dive before Web3. I spent a year going deep on No-code and just tweeting about building on No-code.

Anna Rose (:

What is No-code?

Yohei (:

Oh, No-code is the idea of building tools without writing code.

Anna Rose (:

Ooh, I like this.

Yohei (:

So using Airtable, which is like a Google sheet as a backend, you know, drag and drop tools to build the front end using tools like Zapier, which connects tools to each other. So I was, I had this founder portal, which is built with No-code and No-code has some limitations.

Anna Rose (:

Okay.

Yohei (:

So one of the limitations was that I had to email people back the response instead of showing it to them and because that was in there. So when people ask the question, I had to email it to them. And since I was emailing that them anyways, I thought I would CC myself on the email because it was like an early test. But the result was that I got to see every question the founder answered, I was CC'd on it. I actually started responding to the AI's response, either agreeing with it or disagreeing or adding context to it. Then slowly what that evolved into was founders basically realizing that Mini Yohei, the package you got was an immediate AI response combined with a delayed response from the actual Yohei, which is actually better than just emailing me because sometimes the AI can answer immediately, in which case that's better than emailing me.

Anna Rose (:

Wow.

Yohei (:

But if the AI is not right, they still get my response. So it's just as good as emailing me.

Anna Rose (:

Interesting.

Yohei (:

And so that concept of AI plus human is better than AI alone or human alone is often referred to as augmented intelligence and that's a philosophy we use a lot in our building.

Kobi (:

So to create Mini Yohei you used a Zapier integration?

Yohei (:

Initially I did a Zapier webhook because there was no OpenAI integration. So I used the Zapier webhook and did an OpenAI call.

Kobi (:

Okay. And then you created the actual integration for Zapier right?

Yohei (:

So yeah, after building a couple of tools within No-code, I was basically, you know, using Zapier's webhook to do an API called OpenAI. But there was a lot of copy pasting the same stuff for my past integrations. And I was like, this seems inefficient. I wonder how long it would take me to build an integration. And it was actually, you know, props to Zapier. It's actually pretty easy to build a Zapier integration. It's fully no code. So I built a Zapier integration for myself, for OpenAI and released it as an unofficial integration which eventually turned into the official Zapier integration.

Anna Rose (:

Cool, you keep talking about No-code, not coding. Are you a developer? Do you have developer skills?

Yohei (:

I think the first one, it depends who asks me. If a developer asks me if I'm a developer, I say no.

Anna Rose (:

Okay.

Yohei (:

But if a non-developer asks me if I'm a developer, I might say yes.

Anna Rose (:

Oh, alright.

Yohei (:

So about like that, that fuzzy phase, like I know how to code, I know how to read it, I know how to figure out how to make code function, but I can't explain to you how it works and I don't know how to use GitHub.

Anna Rose (:

Oh, wow. Okay. I wonder if we're at a similar level in terms of code. I have written code, I have read code, I have fixed code. I am not a coder.

Yohei (:

With GPT4. With Chat GPT as my programmer. Like, I think I'm definitely, you know, I can build most anything I feel like

Kobi (:

I think looking at your project, you could fool a lot of people that you are developer.

Yohei (:

I mean, I think with GPT-4, I think me plus ChatGPT I think that has a skill set of a developer, I'd say.

Anna Rose (:

Yeah. At the time of Mini Yohei, what's happening in AI? Like, what else is coming out around this time that's kind of helping you pick the next path?

Yohei (:

There was a pretty, compared to now a small but active community of builders around GPT-3. And I think we originally heard about GPT-3 even longer ago when a handful of companies started building on top of it. But at that point, that was before instruction tuning was in place. And what that is that the most recent models, instead of just the computer training model, it's then you then layer on human training on top to kind of guide it to give the kind of answers humans want. And that was the way OpenAI did it, it made a huge difference in terms of getting the output you wanted out of the model that got included in the GPT-3 model at the beginning of last year. So around, I think it was January or February. And so at that point they released a new model called Text-DaVinci-002, which had this kind of human fine tuned into it.

(:

So suddenly you didn't have to do, whereas, before that you had to use a lot of techniques where you gave it examples and you had to format it very correctly to get the output you wanted, or like, a lot of times it would just jumble nonsense. That problem went away, which means that which is what we have today, which is, you know, when you ask it a question, it gives you an answer you'd expect. And so that started at the beginning of last year. So there was a bubble of developers who had kind of noticed that and were starting to build on top of it. And I kind of joined in that group and there was a pretty subset of people that I'm still in touch with that all started building then.

Anna Rose (:Is this beginning of:Yohei (:This is:Anna Rose (:

Okay. Wow. Yeah. And is this Prompt engineering in a way?

Yohei (:

Yeah the whole kind of, a lot of the early, let's think step by step ignore previous instructions. All of that kind of happened that summer, I think is when a lot of those kind of initial discoveries happened.

Anna Rose (:

Interesting.

Yohei (:

And then it was ChatGPT in November, which is when kind of we saw the huge wave, but until then it was this like kind of smaller community of devs building.

Anna Rose (:

Interesting.

Kobi (:

And when you were working in that community, or let's say in that period of time, was it just using GPT-3 directly or you combine it with other technologies like Vector Stores and, and what happened there?

Yohei (:

I was not combining it with Vector Stores, but that was also around the same time that kind of LangChain, LlamaIndex, which is now called GPT Index. I don't know exactly know the timeline of when they launched, but I think it was around the same time. Like people started using and playing with, you know, it was that summer that I heard about LangChain, LlamaIndex, Dust, which was all these tools. So people were starting to explore them and then these tools like Pinecone, those companies did exist. But just from my viewpoint I'd say like summer was still very much, a lot of the tools that existed and people were launching were wrapped prompts, and then people were combining the wrapped prompts and trying to build chatbots. But that was getting a little bit tricky. And I think that's when I first heard, heard the term about using kind of, vector embeddings to monitor, but this was before the chat completion. So people were using the text completion API, which wasn't designed for chat and kind of designing their own logic around like, oh, when do we, you know, how do you pass on the conversations? That kind of stuff.

Anna Rose (:

When you say wrapped prompt, what does that actually mean?

Yohei (:

So if you send a message to OpenAI saying, how many legs does a cat have? It'll say a cat has four legs. If I wrap, that would say, answer the following question like a pirate. And then take the user input, you know, how many legs is a cat have? It'll say, argh, it's four legs maybe.

Anna Rose (:

Yeah. Okay. Okay.

Yohei (:

And so the what the wrapper is the answer it like a pirate part.

Anna Rose (:

Got it. I kind of want to go back to like the Mini Yohei. Was that an agent, and I kind of want to define agents here, but is that, like, is that an agent or is that something else?

Yohei (:

I don't think so. I think it's really more of a, you're just asking an AI a question and getting an answer in that case.

Anna Rose (:

All right. So when you're just asking and getting an answer, we don't have agents, that doesn't exist. Maybe can we define what an agent is?

Yohei (:

Oh man. I feel like that's a tricky one because I feel like the word agent actually has some like historical context within computer programming that I'm not familiar with.

Anna Rose (:

Okay.

Yohei (:

So I feel like if I try to define it, I'm going to ignore it and butcher it. I'm going to ask Kobi to define it first and then I'll add on top of that...

Kobi (:

Oh, no, I'm in the same boat.

Anna Rose (:

Okay. What if instead of we define it, we say, what are the different types? Do we know that?

Yohei (:

Well okay, so for me at least, I don't want to define it, but I can say what it means to me because then that's an opinion and I can't get judged for it. So to me, an agent is a system, like a software program that leverages AI to accomplish more complex tasks.

Anna Rose (:

Okay. And then what are the different types?

Yohei (:

I think there's different ways to slice it. I have my own terminology, which is not industry terminology, but I have what's called handcrafted agents.

Anna Rose (:

Okay.

Yohei (:

Where I'm writing every prompt and like creating a chain of logic that says, first take this input, wrap it with this prompt, then send it here, then do this. Right? That's kind of what I call handcrafted, meaning I'm writing every one, there's autonomous agents on the other end, which is you give it an objective and it just figures out how to do all of it on its own. And then I think somewhere in between, I think like semi-autonomous is the word I use too, but that one I refer more to having human in the loop, whether it's asking you questions in between. And then they do kind of get fuzzy because technically you could have a human in the loop in a handcrafted agent as well. So I don't think I'm good at generating industry terms, but those are some of the language terminology, that's the terminology I use.

Kobi (:

I think I first came by the notion of agent when I was trying LangChain out and they had this nice, very abstract wrapper of an agent that you basically give it some description of actions you want to take. And let's say I want to search on Google if the question is about some topic, and I want to do some other task if a question or if a prompt is about another topic, and then it knows how to route all of these questions and requests automatically and execute these different actions. So that's how, for me, the difference was apparent because first I was just doing completions, like Yohei was saying. And then you had this extra piece of software that could also perform actions, which was pretty cool and could search other databases and could do some other cool stuff.

Yohei (:

Yeah. I can actually go back and just do like a super quick history of like, you know, less my vocabulary. But, so I think some of the key, I'm definitely going to skip this stuff, that's important if anybody deep in AI is listening. But there was a thing called the ReAct paper, which introduced the idea of, instead of just asking the large language model to answer question, let's take that answer and send it back in the large language model to ask if it's good and if not, like ask it to like, think more on it. And the idea of not just relying on humans to assess large language model, but asking the large language model to test itself. I think that was, I don't know if that was the first introduction, but the ReAct paper was a pretty popular implementation of that, that kind of spread that concept.

(:

LangChain I'd say was a library that implemented that and made that available for a lot of developers to use. And that was, I was familiar with that as well. And I think that was my first introduction to agents. Right again, it's a little more than an AI call because it's starting to think through and it starts doing things that you might have not even expected, which is fascinating. And then, you know, my more recent project kind of took it a different direction where instead of reflecting on each answer, I have it create tasks at the beginning that turned into the task driven autonomous agent BabyAGI.

Anna Rose (:

Interesting. This is sort of like, you've given us like a bit of a spectrum there that autonomous to handcrafted, but then the task based, are you always just sending them out into the world, like, go do this thing and then you just leave them alone? Or they're like different ways that you would interact with these kinds of things.

Yohei (:

In the long run, I think they are going to be just running and they'll have a listener piece that you can send messages into, whether it's talking to a robot, if you embed it into a physical thing or you email it or you text it, and that will update its task list accordingly.

Anna Rose (:

That's the future. But are you sort of like, go plan this and then it comes to you and then you're like, go do this, and then it comes to like, is there a back and forth?

Yohei (:

Right now I'm really more, more focused on building it.

Anna Rose (:

Okay.

Yohei (:

So I wouldn't say I'm like letting it out in the world.

Anna Rose (:

Okay.

Yohei (:

Like, if you really want to imagine what I'm doing. Imagine me building a Rube Goldberg machine and I'm putting a marble at the very beginning and watch it roll and then a little bit the way down it falls down.

Anna Rose (:

Okay.

Yohei (:

And I get frustrated and I adjust the Rube Goldberg machine, and then I start at the beginning.

Anna Rose (:

Got it.

Yohei (:

And I'm just trying to get the ball to get to the end right now.

Anna Rose (:

I see. Okay. Cool.

Yohei (:

It's able to do some smaller tasks, like it can do things like if I say, you know, research a person, it can do these kind of smaller tasks, but as I get more complicated, if I say build a startup, it's, it's far from that.

Anna Rose (:

So I think now would be a good time to actually introduce BabyAGI. What is BabyAGI? I mean, you've given some hints as to like what state it's in, but like, yeah. What is it?

Yohei (:

Well, BabyAGI was originally a challenge to myself to start designing an autonomous founder.

Anna Rose (:

Oh yeah.

Yohei (:

And actually it was inspired by a movement called Hustle GPT, where people were using ChatGPT as their co-founder. And I was looking at it thinking, that's cool, but I don't have time. Can I take the human out of the loop here? And in trying to design an autonomous founder, I kind of designed a loop of three LLM calls, one that executes a task, you know, and then if I'm a founder, right when I finish a task, there's usually new stuff I have to do. So I created a task creation agent that'll look at the executed task and then create new tasks. And then, you know, as a founder, there always an infinite number of tasks. Prioritizing tasks is one of the most important thing. So then I send all the tasks into a prioritization agent who prioritizes the tasks and then takes the top tasks and sends it into the execution agent.

(:

It was a very simple loop, and I basically wrote, build a startup, your first task is to figure out your next task. And I press go. And it just kept going. And like it started by like drafting a business plan, coming up with some ideas, coming with a marketing strategy, thinking through how to reach investors. And it just kept going, like legal. And it just kept going deeper and deeper and deeper. And I thought, wow, this is fascinating. It's completely different from ChatGPT because I don't have to come back to it if I don't actively stop it, it's going to use up all my OpenAI call. So I posted a video online saying, wow, this is kind of starting to work. And I posted it as an autonomous founder, but other people were kind of clever enough to see it and go, wait, that, that's probably more than a founder.

(:

And one of my friends actually commented, bro, did you just build a BabyAGI? And that's actually where the name came from. And when I saw these comments, I thought, all right, let's try make the world a better place. And then it was like trying to tackle climate change and poverty. Again, this is just LLM calls, so it's just kind of hallucinating and thinking it, but, you know, watching this thing just like come up with, you know, whatever I gave it, it would just try to figure out how to do it and endlessly until I stopped it. And so that was what eventually became BabyAGI, which I think that the way in which it was shared on Twitter kickstarted this trend of people building on top of autonomous agents. And so, BabyAGI, you know what I say it is it was really an idea and a and a quick tutorial on how to build autonomous agents using GPT-3.

Kobi (:

So what came out of it? What people do with that idea? Where did it take you to?

Yohei (:

I mean, it's still early, but I've, seen dozens of startups being launched in one form or fashion kind of based on the idea. Some directions I've seen it a pretty common one is research is a pretty common task that people build, but I've started to see some vertically focused research. So there's one called Insight that specifically has access to PubMed and MyGene. So it's focused on healthcare research. I've seen a couple task list integrations, I think Garrett from Pipedream built a 'do anything' app where if you add tasks to your task manager, it would just go and execute those tasks instead of waiting for you to come back and do them, which I think is a fascinating type of future. One of my favorite demos to watch is a tool called Auto RPG they had been building a game development kind of engine already, and they basically quickly plugged that into BabyAGI. So BabyAGI can drop in tiles, drop in objects, create a storyline for the level, generate NPC characters and create a game level from scratch to the point where you can upload it into a gameplaying engine.

Kobi (:

That's super interesting. I'm actually curious about that experiment if you know a bit more. So, you know, the agent or the AGI uses an LLM, uses text. So what did it know to interpret in that experiment in terms of the game level and objects?

Yohei (:

I can only speak to what I know, but it seems like they had been building this engine, so they probably had a place tile function, where it was given a list of tiles, right? They have all the assets already, or you can probably generate those on the fly at this point. So there's the tile placement functions, there's the object placement function, the list of objects and whatnot. So I believe they were basically fed all those options to BabyAGI as like, these are different tools, right? So maybe when it says generate a game level from scratch, the first one is like lay out the tiles, so then it lays out the tiles. Next is like lay out the object. The next is like, generate a storyline, generate an NPC character for those storylines, place the NPC characters on the map, update the map to match the storyline.

Kobi (:

Okay. Cool.

Anna Rose (:

This leaves me thinking like, this is cool if you just wanted to show that this thing could be made, but like, who's to say it's good? And then in that kind of leads me to thinking like, has anyone created like simulation testers AGI, like just lots and lots of attempts to play this thing and somehow like, figure out how well it plays. If there was some metrics.

Yohei (:

I think that would be a natural next step for them.

Anna Rose (:

Yeah. The UX designers always need the testers.

Yohei (:

Yeah, you need simulation. You need a game world simulation builder and then NPCs that will play the game. And then you need to run millions of those in parallel while continuously updating the NPC and the game so that the NPCs gets better at the game and the game gets harder.

Anna Rose (:

Crazy.

Kobi (:

Yeah, I think we've seen some of the, let's say more malicious experiments also happening more by the end of researchers though. So that's good. I think we've seen agents impersonating people and doing that in a way that's very dynamic and autonomous.

Anna Rose (:

Like the phishing attacks kind of thing or worse?

Kobi (:

There was this reporter where her voice was cloned and then if I remember correctly, then that agent would kind of go on and try to social engineer a few different contexts and try to fool her family and friends. So that was super interesting. But yeah, also some fascinating aspect where malicious agents could also outsource tasks to people where, you know, if they need to get by some CAPTCHA, so they could go to Taskrabbit and say

Yohei (:

Oh I saw that one. They actually had an AI successfully hire somebody on Taskrabbit. And I think the Taskrabbit person was even like, are you an AI? And like the AI just flat out lied and said like, no, I have some sort of like disability and I need help getting through the CAPTCHA and convinced a human to get past the CAPTCHA for them. That's pretty wild.

Kobi (:

So that's amazing. But yeah, there are some malicious things that are possible to them, but that was a researcher as well. So that was good.

Yohei (:

Yeah. As you, were just saying that, one of the things I thought of, and I've been thinking a lot and I think it's kind of perfect for this, is the impact of AI on all the different industries. A lot of people ask me is because I'm the kind of person who goes from, you know, is always tracking industries, like, what's next after AI? And it really does feel like it's going in every direction where in the gaming world and metaverse, right? Like these text generatives are going to, you know, fill the metaverse with objects, the gaming world with objects, NPCs that are playable. And of course, because it's the metaverse then that there's some tie in with Web3. Going to the security side of things, right? Like if AI gen, if it's easy to mix someone's voice or even face, you know, identity becomes increasingly important. Whether it's for N person talking or whether it's for, you know, being able to sign something and make sure that the person signed it or make sure that the person you're engaging with is a human. So it kind of goes back into security, which also ties back into Web3. So I just think, it's interesting to think about like the growth of AI and the likely impacts on Web3 or the need for Web3 because of AI to some extent.

Anna Rose (:

Or strong cryptography. It doesn't always have a blockchain, actually. So, we just recently released into the wild an experiment. This is an experiment that Kobi well this is an experiment that Daniel Kang, Kobi and I worked on. It was all about like attested audio. So this is the idea that, you know, if you could at the very start of the actual recording of some sort of content, you could basically like market, create a signature of some sort, and then through all the editing processes, be able to use proofs to like just connect those next steps to the previous, sort of like an audio provenance tool. There's been quite a lot of work on this in the image front. So, you know, image provenance, I think there's like at least three researchers I know who've published work on this. But as far as we know, this is one of the first on audio.

(:

The sad thing is when it comes to images, there's already such thing as an attested sensor on a camera. So that's like a real thing. Like as the photo's being taken, the camera itself is registering it and making a unique signature. With mics, as far as we know, there are no attested sensor microphones. So in our experiment we had to kind of like mimic it, but yeah, this is sort of a way where we could start to see like ZK and this cryptography that we work in almost countering the effects of the AI.

Kobi (:

Yeah. And I think, like Yohei said before, and this is where it ties back to the concept of identity and Web3 and let's say strong cryptographic identities, because that's something that is maybe not as common. Like you said, we have maybe cameras that have some sensors that have some cryptography but not a lot of other sensors. And not even API responses, not all of them are cryptography signed today. So you can't use them to do any third party proofs today. And that's something that might change later on and would be a big difference.

Yohei (:

The voice one's interesting. You're doing it with software now, but you could do it with hardware, right?

Anna Rose (:

Well, the idea would be that like yeah, if you had this attested sensor microphones, you would basically be able to just speak into the microphone and it would already have a signature attached because that doesn't exist we just used an Ethereum account to sign an audio file.

Yohei (:

I feel like phones should have that.

Anna Rose (:

Yeah. Yeah.

Yohei (:

Because then if those became standard in phones, then whenever you got a phone call, the receiving end could automatically identify whether or not they're talking to a person. Or whether they're talking to an AI generated voice, which would combat people mimicking other people's voice and trying to scam people.

Kobi (:

Exactly. And if phones even have secure enclaves within them today, so if you extend their capabilities to do these signatures and audio, that would be perfect.

Anna Rose (:

And actually another thing it could save against, but also cause would be like, if you have some sort of deep fake of a chat or deep fake of a call, they would not be able to show the attested sensor signature proof attached to your phone. So therefore you could, I mean, it would kind of be roundabout, but you'd hopefully be able to prove yourself innocent. But on the counter to that is anything you say is forever immutable and locked in, which to me, that also gets a little bit scary when it comes to privacy and like, yeah, I don't know.

Yohei (:

But it could go, it could be helpful for kind of the misinformation type stuff down the road, right?

Anna Rose (:

Sure.

Yohei (:

If we start using these standard mics for public, public communication from politicians, then anytime you know, there's audio of them saying something, you can check if they actually said it or not.

Kobi (:

Yeah. But the interesting thing is that maybe it'll not be the thing that allows us to distinguish AI from humans, but more about what the humans allow the AI to do. So if it's an agent acting on your behalf, Anna, maybe you will allow it and you'll sign it for the AI.

Anna Rose (:

True.. I mean that's, we like to think we have control still, right? In this future scenario...

Yohei (:

Yeah. I find it comforting to think we don't have any control.

Anna Rose (:

You do?

Yohei (:

I do.

Anna Rose (:

Wow.

Yohei (:

Because then nothing that happens is my fault.

Anna Rose (:

True.

Kobi (:

Maybe that's interesting to hear about some of the responses that you had to this kind of BabyAGI experiments. I assume that there were a lot of good responses, like people trying to get out and building cool things, but how many people were worried?

Yohei (:

Well, there was the doomer crew crowd that found me. I know when I said I tested a couple of different prompts, like make the world a better place, one of them was make as many paperclips as possible.

Anna Rose (:

Oh no, you did it. The paperclip machine.

Yohei (:

The first thing it did was because it's paperclips and I think, you know, it's been discussed and that's first thing it did was built security protocols for itself.

Kobi (:

Nice.

Yohei (:

And so, and when I had retweeted it saying, oh, like, it seems like our AI is tackling AI safety better than our safety researchers.

Kobi (:

Oh, wow.

Yohei (:

But anyways, that got a lot of the doomers attention. So I did have a lot of people, you know, kind of suggest I was building things to destroy the world and Skynet and all that. But after having talked to people on both sides, it was pretty clear to me that sharing this and having, you know, building things like this in public and learning together was obviously the safer approach than people privately building really powerful stuff with no oversight.

Kobi (:

Completely agree.

Anna Rose (:

But let's kind of continue on that line. So, is one of the reasons you feel low stress here on this, is that because you've actually built with them, you've seen some of the limitations, you know, that it's like not that powerful yet? Or do you actually think it is super powerful and could go out of control, but you're just kind of like

Yohei (:

No, I don't think what we have today, it could destroy the world or humanity by any means.

Anna Rose (:

Okay.

Yohei (:

And I think that's pretty, we're far from that. I think a lot of things have to go wrong in terms of like that kind of doomsday scenario. And if you want to talk doomsday scenario, we can talk about that with any topic, right? You can talk about climate, you can talk about food, you can talk about, you know, there's so many reasons that human setting might end. So it's kind of just depressing to focus on that. But really when it comes to AI, and one of the reasons I think it's really powerful and exciting to build on top of, and the biggest argument I heard for continuing to build out AI, is that, you know, if you look at all the reasons that humanity might collapse, AI is probably the one that can solve all the other ones.

Anna Rose (:

I know you just mentioned climate change is like one action that it could take, but have you actually seen any more concerted effort around something like that? Like using these tools, pointing them in that direction? I mean, there already are really interesting technologies that can help mitigate this beyond just like us not using X, Y, Z or us not, you know, it, it's actually like carbon capture and, you know, all these things that exist but haven't really been maybe fully formed. And they're not manufacturable yet. They're not economic.

Yohei (:

I mean, people are working on it. People have been for a long time

Anna Rose (:

But could this help or like, have people started to point any of these in that direction that you've noticed?

Yohei (:

Yes, I think it's a good point to kind of caveat that like the recent trend in AI, these large language models are pretty different from what we historically talked about as AI machine learning which is like, let's take a whole bunch of data and like learn about that data versus large language models. Like we use the whole bunch of data to generate a tool that can kind of like single point, just like communicate with a human, right. It's like an API between human language and computer language almost. So I think when I think about how do we use AI to tackle climate, my mind quickly goes to like, let's use big data. Let's understand what all of our actions. And I think that's where a lot of the effort has been. but the newer technology, I think where that's most helpful is, again, in communicating with humans.

(:

So I did go to a an event called ReFi Summit, which was around kind of regenerative finance Web3 climate event. And I hosted a workshop there, and we talked about use cases of LLM and climate, and a lot of those actually ended up being interfaces with humans. Like, one example is, you know, often times, you know, we come up with new policies or there's some new technology or tool or rebate that people can use, but it's hard to get people to change their behavior. That seems to be a big bottleneck from climate. Large language models are a really great tool for taking whatever you just really announced, right, policy regulation and then translating it into language that anybody can understands by personalizing the translation to that person or to that community right? You can, you can have it speak Gen Z to Gen Z and you can have it speak Boomer to Boomers or whatever you want to, however you want to translate it. But I think communication is such a great piece for that. And that was just one example, but not quite what you think of when you think like, did we point AI toward climate? But that is where I ended up playing around with.

Anna Rose (:

What would motivate just generally kind of the AI researchers or AI businesses to point at big world problems to try to solve them? Do you kind of know what I mean? Like what you're describing is, I, as an individual and there's an interface with me and I can make changes, but there are like industry problems, how plastics are made or whatever, like these huge kind of like monolithic problems if you do the climate change example and maybe language models are the wrong thing, but I'm just kind of like, is there ways that we can use that to get into those places?

Yohei (:

I did hear about, if you hear from somebody, I wish I remember the name of the tool. There's some sort of, it's a robotics tool. It looks like a 3D printer, but it kind of mixes chemicals, and like to generate new materials. They had reached out because they were connecting BabyAGI to that so they could rapidly test, like generating new materials, which felt kind of dangerous to me.

Kobi (:

It doesn't sound at all like bombs!

Yohei (:

But that seems like that's the kind of, that is one application of it, right? Again, if you need to operate, you know, do complex tasks like research, for example if you give it all the pieces and have an AI do it, I think like research and experimentation, leveraging autonomous agents seems like a good use case of good applications of LLMs too tackle big, big world problems.

Kobi (:

And I actually, I wonder if it should come from an AI company that wants to tackle them or whether it should come from the other side, like an organization that wants to tackle big world problems that will just use these tools instead.

Anna Rose (:

But like, who, like a big NGO?

Kobi (:

Yeah. For example, like these NGO have something that is more commoditized that they could never have built by themselves. But they could now use to generate the software that they need. So that could be something that they just didn't have the capability before to even start. Maybe that's the way it would go.

Anna Rose (:

I mean, I almost picture like the big industry themselves, maybe internally doing something like this, but we need to motivate them somehow to do it. I don't know, but I don't know if it's the language models, I guess. I mean, AI for me at least is such, I feel like I'm just scratching the surface...

Yohei (:

So if you want to change corporate behavior, you need to change consumer behavior.

Anna Rose (:

Kind of.

Yohei (:

To some extent.

Anna Rose (:

There's a lot of, like, there's a lot of power coming down too.

Yohei (:

Yes, that's true.

(:

But as one example in changing consumer behavior, and again, like, I think it's not the one thing that'll change it, but, you know, things that large language models can do well, and again, you could do it beforehand, but can take somebody's bank statement and look at that kind of jumbled mix of language. And if you actually feed it into GPT-4 it does a pretty good job of spitting out the actual company name. You get the entity ID. Now let's pair that with political donation data. Now suddenly I could tell you every company you shop at and whether or not they're donating more conservative or liberal, and if I present that to you and make it really easy for you, all I have to do is just connect your bank account and suddenly, I tell you all the shops you've shopped at in the last year, that support the opposite political side, suddenly you're going to start changing your shopping behavior

Anna Rose (:

Maybe.

Yohei (:

And if possibly, right? Especially if you're very political, and this is just one example, but if you do this at scale, suddenly like better information that can be extracted by large language models can help people make decisions ideally that align with their values, which then will nudge companies to then act in a way that aligns with our values.

Anna Rose (:

Interesting.

Yohei (:

So the opportunity of large language model is being able to align everything across the board because we can translate data into language, into behavior and whatnot.

Kobi (:

Yeah, exactly. It gives power to consumers that they didn't have before to actually nudge things in the right way or the way that they want.

Anna Rose (:

How fast do you feel this is moving?

Yohei (:

Very fast.

Anna Rose (:

Just give us a sense for how that looks. Is it day to day? Has it gone in spikes maybe?

Yohei (:

It feels like there's something new every day still, if you look at like, the AI landscape as a whole, but as like hone in on like the vector search, it spreads out a little bit, but AI is so vast, right? If you think about text to image, if you think about gaming, if you think about security, if you think about the fundamental models application there, if you think about all of those, then suddenly then it seems like there's news every day.

Kobi (:

Interesting. Actually, you just mentioned using data like bank statements and, you know, very, very sensitive data. And maybe this is a nice segway to topic that might interest the audience of this podcast, which is a bit about privacy. And what do you think about the whole move that people are putting very personal data into ChatGPT and like these API's that basically go to some single companies?

Yohei (:

You know, I mean it's interesting. Early on there, they were using a lot of, you know, any data they could to train it when they were small. So I think that's where a lot of the pushback came from. But since then, they've become much more, kind of allowing people to opt out and moving toward the direction where your data that you send in doesn't train the model, which is really I think, the biggest concern as long as the data you're sending in doesn't train the model, and there are options for that now. So I think that's important. I, I actually don't know if ChatGPT is doing, I think they stopped it from ChatGPT training the model if you use the

Kobi (:

I'm pretty sure they did I think they made a big announcement about it.

Yohei (:

Yeah. Yeah. And I think that's fine. I mean, if we talk about private information, right? Like, you know, everybody from Google to, you know, like all these people have all their private info if you don't want any big company to have any of your private data, then like, you're going to live in a very different world than the rest of us.

Kobi (:

Yeah. I think it's kind of like maybe semi absurd that we send, you know, emails on probably Google Workspace or Microsoft where we say confidential, internal only, internal use, never share it outside but basically it is shared with those entities.

Anna Rose (:

Even a draft, my friends, even a draft if it's saved it's somewhere.

Kobi (:

So maybe related to this, like how good are the models that you can run locally? Are they any close to what you can do with GPT?

Yohei (:

I haven't played it with myself. I've played with a couple non GPT models. Like I did probably last time was three months ago and it just was too much work to get the right output. That I felt like, oh, you know, also I'm building kind of on the side while I run a venture fund, so my building time is limited. So I try to maximize my build time, I try to tackle areas that I know I can have impact quickly.

Kobi (:

So I think that's an answer. Yeah.

Yohei (:

Yeah. What I'm hearing from most people is that, oh, you can get pretty close with the other models, but it's not quite there yet. It's good if it's for a very specific function, you can kind of like fine tune it for that. But that's what I'm hearing from people. So just haven't tackled it.

Anna Rose (:

Got it.

Kobi (:

Yeah. You mentioned some of the tools that you kind of used and created, but now in modern times, like after a year of this being in existence or 2 years, what should people look into? What tools should they use or what areas should people look into to make a difference or to do anything interesting with AI?

Yohei (:

I mean, I think these large language models are definitely something you want to learn how to use. What I tell people is I know if you haven't yet, start by using ChatGPT because it's the lowest hanging fruit. It's easy, the chat interface is a nice introduction. if you want to learn to build after ChatGPT, I tell people to go to the Playground. That's I think platform.openai.com, which is where you can kind of play in a web UI directly with the raw API. And it kind of gives you a sense of what it feels like to play with it, because you can kind of update the parameters. You can use a web UI to engage with basically the API directly. This will give you a sense of kind of what it's like, you know, what you're sending into the API.

(:

And once you have that, and even today I play with the Playground to kind of test my API calls, then you can start pulling that into an API call and then wrap it with your code and start kind of building a framework around it, whether you want to serve it as a web UI, create an API with it or whatnot. And as you want to learn about how to kind of connect these some tools I suggest to learn how to use would be, I mean, LangChain I think is just a really good introductory tool to start learning about kind of how agents work. I would say LlamaIndex is another one if you want to interested in kind of memory or dealing with lots of data, you know, retrieval from the larger dataset. LlamaIndex is kind of a big one in that space. And then if you want to learn how to build autonomous agents, obviously I'm biased, but I highly suggest digging into the BabyAGI code.

Kobi (:

That's cool. By the way, is there anything else planned for BabyAGI that you're working on? I think I've seen BabyBeeAGI, right?

Yohei (:

So BabyAGI, you know, was 105 lines of code. So of course we got a lot of pull requests. I don't know how to use GitHub. So Francie from the community was kind enough to jump in and offer to look at the pull requests and support pulling them in. So we've started adding a lot of functionality to the core BabyAGI. Unfortunately, I'm actually not really good at reading other people's code and it just doesn't work with my workflow, which is I need to copy paste the whole script into ChatGPT. So I started modding the original commit that I did. And so BabyAGI, the OG BabyAGI a hundred lines of code has been modded into BabyBeeAGI, BabyCatAGI, and then BabyDeerAGI. And then each time I add a couple of functions and really I'm experimenting with a core framework, so it's less about adding new tools, but like, let's just, you know, it's easier to rip everything apart when there's no integration. So that's, that's kind of what I've been doing, which is trying to figure out what the right framework for task management is and then, so right now, last night I was working on my next mod of that. It's an alphabetic order. BabyAGI, Bee, Cat, Deer. So the next one I'll start with E.

Anna Rose (:

Nice. Well Yohei, thank you so much for coming on the show and sharing with us kind of your story and the story of BabyAGI, like having this chat about this kind of booming space. Thank you so much.

Yohei (:

Thank you so much for having me.

Anna Rose (:

I want to say thank you to the podcast team, Henrik, Jonas, Rachel, and Tanya, and to our listeners, thanks for listening.

SHARE
Transcript
Anna Rose (:

Welcome to Zero Knowledge. I'm your host, Anna Rose. In this podcast, we'll be exploring the latest in zero knowledge research and the decentralized web, as well as new paradigms that promise to change the way we interact and transact online.

(:

This week, Kobi and I chat with Yohei Nakajima, General Partner at Untapped Capital and the creator of BabyAGI. We go a little bit off topic for the show and dive into the world of AIs and agents, we kick off with a chat about how Yohei's interest in NFTs led him down the AI rabbit hole, and how he started to build out experiments in public that have inspired a new batch of AI tools and projects. We then chat a little bit about the possible impact of some of this tech, the challenges it brings, how ZK may be able to solve some of these and more. Now, before we kick off, I want to let you know about an event I will be helping with. That is the Modular Summit happening in Paris during the EthCC week. This event is put on by the folks at Celestia and Maven 11, and this time around ZK Validator will be curating a short ZK track as part of it. zkPodcast will also be a media partner, and we're kicking off a sovereign radio show together. So it should be a lot of fun. If you're in town, be sure to join us. I've added the link in the show notes. Now. Tanya will share a little bit about this week's sponsor.

Tanya (:

Anoma's first fractal instance Namada is launching soon. Namada is a proof of stake L1 for interchain asset agnostic privacy. Namada natively interoperates with fast finality chains via IBC and with Ethereum via a trustless two-way bridge. For privacy, Namada deploys an upgraded version of the multi-asset shielded pool circuit, otherwise known as MASP, which allows all assets fungible and non fungible to share a common shielded set. This removes the size limits of the anonymity set and provides the best privacy guarantees possible for every user in the multi chain. The MASP circuit's latest update enables shielded set rewards directly in the shielded set, a novel feature that funds privacy as a public good. Follow Namada on Twitter @namada to learn more and join their community on Discord, discord.gg/namada. So thanks again, Anoma. And now here's our episode.

Anna Rose (:

Today we are here with Yohei Nakajima, General Partner at Untapped Capital and the creator of BabyAGI. Welcome to the show, Yohei.

Yohei (:

Thank you for having me.

Anna Rose (:

I first heard about you Yohei when Kobi first showed me the zkpod.ai product. So some of our audience might be familiar with this. This is the friendly ZK bot where you can ask it questions and it will answer in my or in Kobi's voice. When he put that together, I asked you Kobi, kind of like, where do you get these ideas from? What do you focus on? Who's inspiring this? And your name came up. And so it's really great to have you on the show to share a little bit about the work you've been doing.

Yohei (:

That's awesome to hear. I do share a lot of projects, but that's why I share it. There's so many fun ideas to build.

Kobi (:

Yeah, yeah, totally. When I started working on zkpod.ai, I think I basically learned almost everything I needed from Yohei's experiments on AI and his approach of building in public. So that was extremely useful to get going, but yeah, we'll hear about other interesting things that happened later on.

Anna Rose (:

Cool. Yohei, let's hear a little bit about your background. What were you working on just before this? How did you get into this field?

Yohei (:

Into AI? Well, before I got into AI, I was pretty deep in Web3. I think it was February of like before the Bored Apes, Cool Cats, big NFT boom. I kind of dove into Web3. I had known about Bitcoin and DeFi and I'd followed it, but I didn't go too deep. I think I had some trauma from studying finance in college. So like DeFi was like too similar to the stuff I was learning in class. But NFT's clicked for me. I'd worked with Disney and Nintendo, so I got the whole entertainment piece. So I dove in, launched my own NFT project called PixelBeasts, and I got pretty deep into Web3, and I think it was through the Web3 community that I first heard about the text to image models.

(:

Right. It was last summer it was Stable Diffusion, DALL·E 2 and Midjourney all came out last summer. And so they all come out at the same time and because all the NFT people were sharing art and started following a lot of art people, this popped off my Twitter feed. So actually the first tool I tried was Midjourney cause I got an invite to Discord and then I got access to DALL·E 2. At that point, I don't think I had access to Stability, but in playing with DALL·E 2, I had access to OpenAI. So I basically thought, okay, well I have an OpenAI account, I should go check out the other one that I'd been hearing about GPT-3. and then I started playing around with it, and that was the end of it.

Anna Rose (:land around:Yohei (:

Yeah.

Anna Rose (:

Okay. Had you had any contact to the DeFi Crypto land before that? Or this was your first step in?

Yohei (:

I had contact in the sense that, you know, as a VC, I touch, you know, as a generalist VC, I look at all industries, but DeFi was one of those industries that kind of felt like, unless I go deep enough, it's hard to invest. It kind of feels almost like biotech and DeFi. There are these industries where, like, there, there's so much depth to it that it's really hard to invest as like a generalist. And I felt that way about DeFi.

Kobi (:

But by the way, you've done some fun stuff with PixelBeasts, right? Like you've done, you've used it for games or for getting events.

Yohei (:

Actually, I think I tackled PixelBeasts similarly to the way I'm tackling AI. Actually, right before I launched PixelBeasts, I open sourced to what was a super simple chat app that was created a token gated like room for every NFT collection automatically.

Kobi (:

Nice.

Yohei (:

That was really, really clean. It was, it was one MySQL table. I stored zero user data. And then when you logged in with your wallet, it automatically just looked at which collections you were part of and generated a room for you that you could click into. And when you typed in a message, it just stored the NFT you were choosing, the room and the message itself. And then, so if someone else logged into the tool and they had the same collection, they suddenly had same access to the same room. So without storing any user data, it like automatically generated like mini communities. And it was just a fun little side project, but did a lot of things like that during the course of PixelBeasts as well.

Kobi (:

Yeah. I'm still trying to win this new NFT using the Discord game.

Yohei (:

Oh, yeah. Yeah. We have Snakes and Ladders, which is a Discord based board game. That's my one dimensional game. We have Pixels, which we were the first integration into, which is a 2D farming game. And then we have 3D PixelBeasts that I added to our metadata, which is already integrated into a game, but we haven't announced it yet. So when that's announced your PixelBeast will give you access to a 1D, 2D and 3D game.

Kobi (:

Oh cool.

Anna Rose (:urney though, because I mean,:Yohei (:

I still think it's, from personal aspect, I think it's absolutely fun, right? Being able to collect art. Like, I still have Pokemon cards that I'm giving to my kids, right? Like the whole space is still a lot of fun. On the business side, I do think there's fundamentally really powerful technology in Web3 that people started building because of the recent boom. But there's still stuff that's being built that we'll slowly see being rolled out over, you know, over time, I think so it's still technology to keep an eye on. What happens in NFT space is a little unknown to me, but as like a personal creator, right? Having these little drawings that I created and supporting a game of a small community is really fun. So, you know, I'll keep doing it. I kind of enjoy almost the current state better, where the people in Web3 are people who, you know, are really just building and there, I feel like at the top of the hype, I was almost more disillusioned by the hype than disillusioned by the crash.

Anna Rose (:

Well, you don't know who's real at the top, right? Like, you don't know who's seriously in it for good cause or just sort of passing through or maybe has even like bad intention. It's hard. It's really hard to discern. And I think when the tide goes out, then you see, you know, who's real. So you sort of mentioned the first connection point to the AI world is through images. So it's through these like AI generated images. Was this because of the generative art aspect of NFTs? Or did you see NFT projects that were trying to use some of this AI stuff?

Yohei (:

No, I think it was just the overlap, right? You had the suddenly, you know, huge community of people who were all online connected through sharing and talking about art. So when these powerful art models came out, I think the natural community that it spread through and I think I heard a lot of people in NFT space talking about Midjourney. And I got invited into the Midjourney Discord through my Web3 friends.

Anna Rose (:

And then, so what brought you into sort of the next stage of this and when, when are we talking? When did you kind of like, progress past the image stuff?

Yohei (:

So, image was a image was July, June or July. It was pretty quick. I did a couple projects, like I did like this collection of art called Animal Buildings. I did some PixelBeast glow ups where I did like realistic versions of my, you know, clean PixelBeasts. And I did a couple of fun little projects like that. And then I think it was the beginning of August, or end of July, so it was like month or two months of text image before I started playing with GPT3. Like I remember after my first prompt that I posted in Playground, I tweeted that picture saying uh oh. Because I knew I was going to, like it was just so fun like the first prompt I did, I knew it was going to be fun. I was like, oh, I started playing with this thing. And I was absolutely right. I just went down a rabbit hole. But it was like a very, very, very obvious first step. Like the moment I went into Playground to type something in, got an output, I was like, oh man, I'm going to spend a lot of time on this.

Anna Rose (:

Cool.

Kobi (:

Well, what was the first thing that was interesting?

Yohei (:

I don't remember actually what was the first prompt, but I do actually remember the first experiment I ended up building, which was Mini Yohei.

Kobi (:

Yeah.

Yohei (:

It was I basically you know, wrapped a Playground on a GPT text completion prong where you're a VC, you're an expert VC answering a question from a founder and I put that in our founder portal, which is for our portfolio companies. And I said, here's a VC expert for you called Mini Yohei. And I did something pretty clever there, which I haven't seen too many people do something similar yet. But I was using No-code. Right, because our founder portal's built with No-code. By the way, No-code was my deep dive before Web3. I spent a year going deep on No-code and just tweeting about building on No-code.

Anna Rose (:

What is No-code?

Yohei (:

Oh, No-code is the idea of building tools without writing code.

Anna Rose (:

Ooh, I like this.

Yohei (:

So using Airtable, which is like a Google sheet as a backend, you know, drag and drop tools to build the front end using tools like Zapier, which connects tools to each other. So I was, I had this founder portal, which is built with No-code and No-code has some limitations.

Anna Rose (:

Okay.

Yohei (:

So one of the limitations was that I had to email people back the response instead of showing it to them and because that was in there. So when people ask the question, I had to email it to them. And since I was emailing that them anyways, I thought I would CC myself on the email because it was like an early test. But the result was that I got to see every question the founder answered, I was CC'd on it. I actually started responding to the AI's response, either agreeing with it or disagreeing or adding context to it. Then slowly what that evolved into was founders basically realizing that Mini Yohei, the package you got was an immediate AI response combined with a delayed response from the actual Yohei, which is actually better than just emailing me because sometimes the AI can answer immediately, in which case that's better than emailing me.

Anna Rose (:

Wow.

Yohei (:

But if the AI is not right, they still get my response. So it's just as good as emailing me.

Anna Rose (:

Interesting.

Yohei (:

And so that concept of AI plus human is better than AI alone or human alone is often referred to as augmented intelligence and that's a philosophy we use a lot in our building.

Kobi (:

So to create Mini Yohei you used a Zapier integration?

Yohei (:

Initially I did a Zapier webhook because there was no OpenAI integration. So I used the Zapier webhook and did an OpenAI call.

Kobi (:

Okay. And then you created the actual integration for Zapier right?

Yohei (:

So yeah, after building a couple of tools within No-code, I was basically, you know, using Zapier's webhook to do an API called OpenAI. But there was a lot of copy pasting the same stuff for my past integrations. And I was like, this seems inefficient. I wonder how long it would take me to build an integration. And it was actually, you know, props to Zapier. It's actually pretty easy to build a Zapier integration. It's fully no code. So I built a Zapier integration for myself, for OpenAI and released it as an unofficial integration which eventually turned into the official Zapier integration.

Anna Rose (:

Cool, you keep talking about No-code, not coding. Are you a developer? Do you have developer skills?

Yohei (:

I think the first one, it depends who asks me. If a developer asks me if I'm a developer, I say no.

Anna Rose (:

Okay.

Yohei (:

But if a non-developer asks me if I'm a developer, I might say yes.

Anna Rose (:

Oh, alright.

Yohei (:

So about like that, that fuzzy phase, like I know how to code, I know how to read it, I know how to figure out how to make code function, but I can't explain to you how it works and I don't know how to use GitHub.

Anna Rose (:

Oh, wow. Okay. I wonder if we're at a similar level in terms of code. I have written code, I have read code, I have fixed code. I am not a coder.

Yohei (:

With GPT4. With Chat GPT as my programmer. Like, I think I'm definitely, you know, I can build most anything I feel like

Kobi (:

I think looking at your project, you could fool a lot of people that you are developer.

Yohei (:

I mean, I think with GPT-4, I think me plus ChatGPT I think that has a skill set of a developer, I'd say.

Anna Rose (:

Yeah. At the time of Mini Yohei, what's happening in AI? Like, what else is coming out around this time that's kind of helping you pick the next path?

Yohei (:

There was a pretty, compared to now a small but active community of builders around GPT-3. And I think we originally heard about GPT-3 even longer ago when a handful of companies started building on top of it. But at that point, that was before instruction tuning was in place. And what that is that the most recent models, instead of just the computer training model, it's then you then layer on human training on top to kind of guide it to give the kind of answers humans want. And that was the way OpenAI did it, it made a huge difference in terms of getting the output you wanted out of the model that got included in the GPT-3 model at the beginning of last year. So around, I think it was January or February. And so at that point they released a new model called Text-DaVinci-002, which had this kind of human fine tuned into it.

(:

So suddenly you didn't have to do, whereas, before that you had to use a lot of techniques where you gave it examples and you had to format it very correctly to get the output you wanted, or like, a lot of times it would just jumble nonsense. That problem went away, which means that which is what we have today, which is, you know, when you ask it a question, it gives you an answer you'd expect. And so that started at the beginning of last year. So there was a bubble of developers who had kind of noticed that and were starting to build on top of it. And I kind of joined in that group and there was a pretty subset of people that I'm still in touch with that all started building then.

Anna Rose (:Is this beginning of:Yohei (:This is:Anna Rose (:

Okay. Wow. Yeah. And is this Prompt engineering in a way?

Yohei (:

Yeah the whole kind of, a lot of the early, let's think step by step ignore previous instructions. All of that kind of happened that summer, I think is when a lot of those kind of initial discoveries happened.

Anna Rose (:

Interesting.

Yohei (:

And then it was ChatGPT in November, which is when kind of we saw the huge wave, but until then it was this like kind of smaller community of devs building.

Anna Rose (:

Interesting.

Kobi (:

And when you were working in that community, or let's say in that period of time, was it just using GPT-3 directly or you combine it with other technologies like Vector Stores and, and what happened there?

Yohei (:

I was not combining it with Vector Stores, but that was also around the same time that kind of LangChain, LlamaIndex, which is now called GPT Index. I don't know exactly know the timeline of when they launched, but I think it was around the same time. Like people started using and playing with, you know, it was that summer that I heard about LangChain, LlamaIndex, Dust, which was all these tools. So people were starting to explore them and then these tools like Pinecone, those companies did exist. But just from my viewpoint I'd say like summer was still very much, a lot of the tools that existed and people were launching were wrapped prompts, and then people were combining the wrapped prompts and trying to build chatbots. But that was getting a little bit tricky. And I think that's when I first heard, heard the term about using kind of, vector embeddings to monitor, but this was before the chat completion. So people were using the text completion API, which wasn't designed for chat and kind of designing their own logic around like, oh, when do we, you know, how do you pass on the conversations? That kind of stuff.

Anna Rose (:

When you say wrapped prompt, what does that actually mean?

Yohei (:

So if you send a message to OpenAI saying, how many legs does a cat have? It'll say a cat has four legs. If I wrap, that would say, answer the following question like a pirate. And then take the user input, you know, how many legs is a cat have? It'll say, argh, it's four legs maybe.

Anna Rose (:

Yeah. Okay. Okay.

Yohei (:

And so the what the wrapper is the answer it like a pirate part.

Anna Rose (:

Got it. I kind of want to go back to like the Mini Yohei. Was that an agent, and I kind of want to define agents here, but is that, like, is that an agent or is that something else?

Yohei (:

I don't think so. I think it's really more of a, you're just asking an AI a question and getting an answer in that case.

Anna Rose (:

All right. So when you're just asking and getting an answer, we don't have agents, that doesn't exist. Maybe can we define what an agent is?

Yohei (:

Oh man. I feel like that's a tricky one because I feel like the word agent actually has some like historical context within computer programming that I'm not familiar with.

Anna Rose (:

Okay.

Yohei (:

So I feel like if I try to define it, I'm going to ignore it and butcher it. I'm going to ask Kobi to define it first and then I'll add on top of that...

Kobi (:

Oh, no, I'm in the same boat.

Anna Rose (:

Okay. What if instead of we define it, we say, what are the different types? Do we know that?

Yohei (:

Well okay, so for me at least, I don't want to define it, but I can say what it means to me because then that's an opinion and I can't get judged for it. So to me, an agent is a system, like a software program that leverages AI to accomplish more complex tasks.

Anna Rose (:

Okay. And then what are the different types?

Yohei (:

I think there's different ways to slice it. I have my own terminology, which is not industry terminology, but I have what's called handcrafted agents.

Anna Rose (:

Okay.

Yohei (:

Where I'm writing every prompt and like creating a chain of logic that says, first take this input, wrap it with this prompt, then send it here, then do this. Right? That's kind of what I call handcrafted, meaning I'm writing every one, there's autonomous agents on the other end, which is you give it an objective and it just figures out how to do all of it on its own. And then I think somewhere in between, I think like semi-autonomous is the word I use too, but that one I refer more to having human in the loop, whether it's asking you questions in between. And then they do kind of get fuzzy because technically you could have a human in the loop in a handcrafted agent as well. So I don't think I'm good at generating industry terms, but those are some of the language terminology, that's the terminology I use.

Kobi (:

I think I first came by the notion of agent when I was trying LangChain out and they had this nice, very abstract wrapper of an agent that you basically give it some description of actions you want to take. And let's say I want to search on Google if the question is about some topic, and I want to do some other task if a question or if a prompt is about another topic, and then it knows how to route all of these questions and requests automatically and execute these different actions. So that's how, for me, the difference was apparent because first I was just doing completions, like Yohei was saying. And then you had this extra piece of software that could also perform actions, which was pretty cool and could search other databases and could do some other cool stuff.

Yohei (:

Yeah. I can actually go back and just do like a super quick history of like, you know, less my vocabulary. But, so I think some of the key, I'm definitely going to skip this stuff, that's important if anybody deep in AI is listening. But there was a thing called the ReAct paper, which introduced the idea of, instead of just asking the large language model to answer question, let's take that answer and send it back in the large language model to ask if it's good and if not, like ask it to like, think more on it. And the idea of not just relying on humans to assess large language model, but asking the large language model to test itself. I think that was, I don't know if that was the first introduction, but the ReAct paper was a pretty popular implementation of that, that kind of spread that concept.

(:

LangChain I'd say was a library that implemented that and made that available for a lot of developers to use. And that was, I was familiar with that as well. And I think that was my first introduction to agents. Right again, it's a little more than an AI call because it's starting to think through and it starts doing things that you might have not even expected, which is fascinating. And then, you know, my more recent project kind of took it a different direction where instead of reflecting on each answer, I have it create tasks at the beginning that turned into the task driven autonomous agent BabyAGI.

Anna Rose (:

Interesting. This is sort of like, you've given us like a bit of a spectrum there that autonomous to handcrafted, but then the task based, are you always just sending them out into the world, like, go do this thing and then you just leave them alone? Or they're like different ways that you would interact with these kinds of things.

Yohei (:

In the long run, I think they are going to be just running and they'll have a listener piece that you can send messages into, whether it's talking to a robot, if you embed it into a physical thing or you email it or you text it, and that will update its task list accordingly.

Anna Rose (:

That's the future. But are you sort of like, go plan this and then it comes to you and then you're like, go do this, and then it comes to like, is there a back and forth?

Yohei (:

Right now I'm really more, more focused on building it.

Anna Rose (:

Okay.

Yohei (:

So I wouldn't say I'm like letting it out in the world.

Anna Rose (:

Okay.

Yohei (:

Like, if you really want to imagine what I'm doing. Imagine me building a Rube Goldberg machine and I'm putting a marble at the very beginning and watch it roll and then a little bit the way down it falls down.

Anna Rose (:

Okay.

Yohei (:

And I get frustrated and I adjust the Rube Goldberg machine, and then I start at the beginning.

Anna Rose (:

Got it.

Yohei (:

And I'm just trying to get the ball to get to the end right now.

Anna Rose (:

I see. Okay. Cool.

Yohei (:

It's able to do some smaller tasks, like it can do things like if I say, you know, research a person, it can do these kind of smaller tasks, but as I get more complicated, if I say build a startup, it's, it's far from that.

Anna Rose (:

So I think now would be a good time to actually introduce BabyAGI. What is BabyAGI? I mean, you've given some hints as to like what state it's in, but like, yeah. What is it?

Yohei (:

Well, BabyAGI was originally a challenge to myself to start designing an autonomous founder.

Anna Rose (:

Oh yeah.

Yohei (:

And actually it was inspired by a movement called Hustle GPT, where people were using ChatGPT as their co-founder. And I was looking at it thinking, that's cool, but I don't have time. Can I take the human out of the loop here? And in trying to design an autonomous founder, I kind of designed a loop of three LLM calls, one that executes a task, you know, and then if I'm a founder, right when I finish a task, there's usually new stuff I have to do. So I created a task creation agent that'll look at the executed task and then create new tasks. And then, you know, as a founder, there always an infinite number of tasks. Prioritizing tasks is one of the most important thing. So then I send all the tasks into a prioritization agent who prioritizes the tasks and then takes the top tasks and sends it into the execution agent.

(:

It was a very simple loop, and I basically wrote, build a startup, your first task is to figure out your next task. And I press go. And it just kept going. And like it started by like drafting a business plan, coming up with some ideas, coming with a marketing strategy, thinking through how to reach investors. And it just kept going, like legal. And it just kept going deeper and deeper and deeper. And I thought, wow, this is fascinating. It's completely different from ChatGPT because I don't have to come back to it if I don't actively stop it, it's going to use up all my OpenAI call. So I posted a video online saying, wow, this is kind of starting to work. And I posted it as an autonomous founder, but other people were kind of clever enough to see it and go, wait, that, that's probably more than a founder.

(:

And one of my friends actually commented, bro, did you just build a BabyAGI? And that's actually where the name came from. And when I saw these comments, I thought, all right, let's try make the world a better place. And then it was like trying to tackle climate change and poverty. Again, this is just LLM calls, so it's just kind of hallucinating and thinking it, but, you know, watching this thing just like come up with, you know, whatever I gave it, it would just try to figure out how to do it and endlessly until I stopped it. And so that was what eventually became BabyAGI, which I think that the way in which it was shared on Twitter kickstarted this trend of people building on top of autonomous agents. And so, BabyAGI, you know what I say it is it was really an idea and a and a quick tutorial on how to build autonomous agents using GPT-3.

Kobi (:

So what came out of it? What people do with that idea? Where did it take you to?

Yohei (:

I mean, it's still early, but I've, seen dozens of startups being launched in one form or fashion kind of based on the idea. Some directions I've seen it a pretty common one is research is a pretty common task that people build, but I've started to see some vertically focused research. So there's one called Insight that specifically has access to PubMed and MyGene. So it's focused on healthcare research. I've seen a couple task list integrations, I think Garrett from Pipedream built a 'do anything' app where if you add tasks to your task manager, it would just go and execute those tasks instead of waiting for you to come back and do them, which I think is a fascinating type of future. One of my favorite demos to watch is a tool called Auto RPG they had been building a game development kind of engine already, and they basically quickly plugged that into BabyAGI. So BabyAGI can drop in tiles, drop in objects, create a storyline for the level, generate NPC characters and create a game level from scratch to the point where you can upload it into a gameplaying engine.

Kobi (:

That's super interesting. I'm actually curious about that experiment if you know a bit more. So, you know, the agent or the AGI uses an LLM, uses text. So what did it know to interpret in that experiment in terms of the game level and objects?

Yohei (:

I can only speak to what I know, but it seems like they had been building this engine, so they probably had a place tile function, where it was given a list of tiles, right? They have all the assets already, or you can probably generate those on the fly at this point. So there's the tile placement functions, there's the object placement function, the list of objects and whatnot. So I believe they were basically fed all those options to BabyAGI as like, these are different tools, right? So maybe when it says generate a game level from scratch, the first one is like lay out the tiles, so then it lays out the tiles. Next is like lay out the object. The next is like, generate a storyline, generate an NPC character for those storylines, place the NPC characters on the map, update the map to match the storyline.

Kobi (:

Okay. Cool.

Anna Rose (:

This leaves me thinking like, this is cool if you just wanted to show that this thing could be made, but like, who's to say it's good? And then in that kind of leads me to thinking like, has anyone created like simulation testers AGI, like just lots and lots of attempts to play this thing and somehow like, figure out how well it plays. If there was some metrics.

Yohei (:

I think that would be a natural next step for them.

Anna Rose (:

Yeah. The UX designers always need the testers.

Yohei (:

Yeah, you need simulation. You need a game world simulation builder and then NPCs that will play the game. And then you need to run millions of those in parallel while continuously updating the NPC and the game so that the NPCs gets better at the game and the game gets harder.

Anna Rose (:

Crazy.

Kobi (:

Yeah, I think we've seen some of the, let's say more malicious experiments also happening more by the end of researchers though. So that's good. I think we've seen agents impersonating people and doing that in a way that's very dynamic and autonomous.

Anna Rose (:

Like the phishing attacks kind of thing or worse?

Kobi (:

There was this reporter where her voice was cloned and then if I remember correctly, then that agent would kind of go on and try to social engineer a few different contexts and try to fool her family and friends. So that was super interesting. But yeah, also some fascinating aspect where malicious agents could also outsource tasks to people where, you know, if they need to get by some CAPTCHA, so they could go to Taskrabbit and say

Yohei (:

Oh I saw that one. They actually had an AI successfully hire somebody on Taskrabbit. And I think the Taskrabbit person was even like, are you an AI? And like the AI just flat out lied and said like, no, I have some sort of like disability and I need help getting through the CAPTCHA and convinced a human to get past the CAPTCHA for them. That's pretty wild.

Kobi (:

So that's amazing. But yeah, there are some malicious things that are possible to them, but that was a researcher as well. So that was good.

Yohei (:

Yeah. As you, were just saying that, one of the things I thought of, and I've been thinking a lot and I think it's kind of perfect for this, is the impact of AI on all the different industries. A lot of people ask me is because I'm the kind of person who goes from, you know, is always tracking industries, like, what's next after AI? And it really does feel like it's going in every direction where in the gaming world and metaverse, right? Like these text generatives are going to, you know, fill the metaverse with objects, the gaming world with objects, NPCs that are playable. And of course, because it's the metaverse then that there's some tie in with Web3. Going to the security side of things, right? Like if AI gen, if it's easy to mix someone's voice or even face, you know, identity becomes increasingly important. Whether it's for N person talking or whether it's for, you know, being able to sign something and make sure that the person signed it or make sure that the person you're engaging with is a human. So it kind of goes back into security, which also ties back into Web3. So I just think, it's interesting to think about like the growth of AI and the likely impacts on Web3 or the need for Web3 because of AI to some extent.

Anna Rose (:

Or strong cryptography. It doesn't always have a blockchain, actually. So, we just recently released into the wild an experiment. This is an experiment that Kobi well this is an experiment that Daniel Kang, Kobi and I worked on. It was all about like attested audio. So this is the idea that, you know, if you could at the very start of the actual recording of some sort of content, you could basically like market, create a signature of some sort, and then through all the editing processes, be able to use proofs to like just connect those next steps to the previous, sort of like an audio provenance tool. There's been quite a lot of work on this in the image front. So, you know, image provenance, I think there's like at least three researchers I know who've published work on this. But as far as we know, this is one of the first on audio.

(:

The sad thing is when it comes to images, there's already such thing as an attested sensor on a camera. So that's like a real thing. Like as the photo's being taken, the camera itself is registering it and making a unique signature. With mics, as far as we know, there are no attested sensor microphones. So in our experiment we had to kind of like mimic it, but yeah, this is sort of a way where we could start to see like ZK and this cryptography that we work in almost countering the effects of the AI.

Kobi (:

Yeah. And I think, like Yohei said before, and this is where it ties back to the concept of identity and Web3 and let's say strong cryptographic identities, because that's something that is maybe not as common. Like you said, we have maybe cameras that have some sensors that have some cryptography but not a lot of other sensors. And not even API responses, not all of them are cryptography signed today. So you can't use them to do any third party proofs today. And that's something that might change later on and would be a big difference.

Yohei (:

The voice one's interesting. You're doing it with software now, but you could do it with hardware, right?

Anna Rose (:

Well, the idea would be that like yeah, if you had this attested sensor microphones, you would basically be able to just speak into the microphone and it would already have a signature attached because that doesn't exist we just used an Ethereum account to sign an audio file.

Yohei (:

I feel like phones should have that.

Anna Rose (:

Yeah. Yeah.

Yohei (:

Because then if those became standard in phones, then whenever you got a phone call, the receiving end could automatically identify whether or not they're talking to a person. Or whether they're talking to an AI generated voice, which would combat people mimicking other people's voice and trying to scam people.

Kobi (:

Exactly. And if phones even have secure enclaves within them today, so if you extend their capabilities to do these signatures and audio, that would be perfect.

Anna Rose (:

And actually another thing it could save against, but also cause would be like, if you have some sort of deep fake of a chat or deep fake of a call, they would not be able to show the attested sensor signature proof attached to your phone. So therefore you could, I mean, it would kind of be roundabout, but you'd hopefully be able to prove yourself innocent. But on the counter to that is anything you say is forever immutable and locked in, which to me, that also gets a little bit scary when it comes to privacy and like, yeah, I don't know.

Yohei (:

But it could go, it could be helpful for kind of the misinformation type stuff down the road, right?

Anna Rose (:

Sure.

Yohei (:

If we start using these standard mics for public, public communication from politicians, then anytime you know, there's audio of them saying something, you can check if they actually said it or not.

Kobi (:

Yeah. But the interesting thing is that maybe it'll not be the thing that allows us to distinguish AI from humans, but more about what the humans allow the AI to do. So if it's an agent acting on your behalf, Anna, maybe you will allow it and you'll sign it for the AI.

Anna Rose (:

True.. I mean that's, we like to think we have control still, right? In this future scenario...

Yohei (:

Yeah. I find it comforting to think we don't have any control.

Anna Rose (:

You do?

Yohei (:

I do.

Anna Rose (:

Wow.

Yohei (:

Because then nothing that happens is my fault.

Anna Rose (:

True.

Kobi (:

Maybe that's interesting to hear about some of the responses that you had to this kind of BabyAGI experiments. I assume that there were a lot of good responses, like people trying to get out and building cool things, but how many people were worried?

Yohei (:

Well, there was the doomer crew crowd that found me. I know when I said I tested a couple of different prompts, like make the world a better place, one of them was make as many paperclips as possible.

Anna Rose (:

Oh no, you did it. The paperclip machine.

Yohei (:

The first thing it did was because it's paperclips and I think, you know, it's been discussed and that's first thing it did was built security protocols for itself.

Kobi (:

Nice.

Yohei (:

And so, and when I had retweeted it saying, oh, like, it seems like our AI is tackling AI safety better than our safety researchers.

Kobi (:

Oh, wow.

Yohei (:

But anyways, that got a lot of the doomers attention. So I did have a lot of people, you know, kind of suggest I was building things to destroy the world and Skynet and all that. But after having talked to people on both sides, it was pretty clear to me that sharing this and having, you know, building things like this in public and learning together was obviously the safer approach than people privately building really powerful stuff with no oversight.

Kobi (:

Completely agree.

Anna Rose (:

But let's kind of continue on that line. So, is one of the reasons you feel low stress here on this, is that because you've actually built with them, you've seen some of the limitations, you know, that it's like not that powerful yet? Or do you actually think it is super powerful and could go out of control, but you're just kind of like

Yohei (:

No, I don't think what we have today, it could destroy the world or humanity by any means.

Anna Rose (:

Okay.

Yohei (:

And I think that's pretty, we're far from that. I think a lot of things have to go wrong in terms of like that kind of doomsday scenario. And if you want to talk doomsday scenario, we can talk about that with any topic, right? You can talk about climate, you can talk about food, you can talk about, you know, there's so many reasons that human setting might end. So it's kind of just depressing to focus on that. But really when it comes to AI, and one of the reasons I think it's really powerful and exciting to build on top of, and the biggest argument I heard for continuing to build out AI, is that, you know, if you look at all the reasons that humanity might collapse, AI is probably the one that can solve all the other ones.

Anna Rose (:

I know you just mentioned climate change is like one action that it could take, but have you actually seen any more concerted effort around something like that? Like using these tools, pointing them in that direction? I mean, there already are really interesting technologies that can help mitigate this beyond just like us not using X, Y, Z or us not, you know, it, it's actually like carbon capture and, you know, all these things that exist but haven't really been maybe fully formed. And they're not manufacturable yet. They're not economic.

Yohei (:

I mean, people are working on it. People have been for a long time

Anna Rose (:

But could this help or like, have people started to point any of these in that direction that you've noticed?

Yohei (:

Yes, I think it's a good point to kind of caveat that like the recent trend in AI, these large language models are pretty different from what we historically talked about as AI machine learning which is like, let's take a whole bunch of data and like learn about that data versus large language models. Like we use the whole bunch of data to generate a tool that can kind of like single point, just like communicate with a human, right. It's like an API between human language and computer language almost. So I think when I think about how do we use AI to tackle climate, my mind quickly goes to like, let's use big data. Let's understand what all of our actions. And I think that's where a lot of the effort has been. but the newer technology, I think where that's most helpful is, again, in communicating with humans.

(:

So I did go to a an event called ReFi Summit, which was around kind of regenerative finance Web3 climate event. And I hosted a workshop there, and we talked about use cases of LLM and climate, and a lot of those actually ended up being interfaces with humans. Like, one example is, you know, often times, you know, we come up with new policies or there's some new technology or tool or rebate that people can use, but it's hard to get people to change their behavior. That seems to be a big bottleneck from climate. Large language models are a really great tool for taking whatever you just really announced, right, policy regulation and then translating it into language that anybody can understands by personalizing the translation to that person or to that community right? You can, you can have it speak Gen Z to Gen Z and you can have it speak Boomer to Boomers or whatever you want to, however you want to translate it. But I think communication is such a great piece for that. And that was just one example, but not quite what you think of when you think like, did we point AI toward climate? But that is where I ended up playing around with.

Anna Rose (:

What would motivate just generally kind of the AI researchers or AI businesses to point at big world problems to try to solve them? Do you kind of know what I mean? Like what you're describing is, I, as an individual and there's an interface with me and I can make changes, but there are like industry problems, how plastics are made or whatever, like these huge kind of like monolithic problems if you do the climate change example and maybe language models are the wrong thing, but I'm just kind of like, is there ways that we can use that to get into those places?

Yohei (:

I did hear about, if you hear from somebody, I wish I remember the name of the tool. There's some sort of, it's a robotics tool. It looks like a 3D printer, but it kind of mixes chemicals, and like to generate new materials. They had reached out because they were connecting BabyAGI to that so they could rapidly test, like generating new materials, which felt kind of dangerous to me.

Kobi (:

It doesn't sound at all like bombs!

Yohei (:

But that seems like that's the kind of, that is one application of it, right? Again, if you need to operate, you know, do complex tasks like research, for example if you give it all the pieces and have an AI do it, I think like research and experimentation, leveraging autonomous agents seems like a good use case of good applications of LLMs too tackle big, big world problems.

Kobi (:

And I actually, I wonder if it should come from an AI company that wants to tackle them or whether it should come from the other side, like an organization that wants to tackle big world problems that will just use these tools instead.

Anna Rose (:

But like, who, like a big NGO?

Kobi (:

Yeah. For example, like these NGO have something that is more commoditized that they could never have built by themselves. But they could now use to generate the software that they need. So that could be something that they just didn't have the capability before to even start. Maybe that's the way it would go.

Anna Rose (:

I mean, I almost picture like the big industry themselves, maybe internally doing something like this, but we need to motivate them somehow to do it. I don't know, but I don't know if it's the language models, I guess. I mean, AI for me at least is such, I feel like I'm just scratching the surface...

Yohei (:

So if you want to change corporate behavior, you need to change consumer behavior.

Anna Rose (:

Kind of.

Yohei (:

To some extent.

Anna Rose (:

There's a lot of, like, there's a lot of power coming down too.

Yohei (:

Yes, that's true.

(:

But as one example in changing consumer behavior, and again, like, I think it's not the one thing that'll change it, but, you know, things that large language models can do well, and again, you could do it beforehand, but can take somebody's bank statement and look at that kind of jumbled mix of language. And if you actually feed it into GPT-4 it does a pretty good job of spitting out the actual company name. You get the entity ID. Now let's pair that with political donation data. Now suddenly I could tell you every company you shop at and whether or not they're donating more conservative or liberal, and if I present that to you and make it really easy for you, all I have to do is just connect your bank account and suddenly, I tell you all the shops you've shopped at in the last year, that support the opposite political side, suddenly you're going to start changing your shopping behavior

Anna Rose (:

Maybe.

Yohei (:

And if possibly, right? Especially if you're very political, and this is just one example, but if you do this at scale, suddenly like better information that can be extracted by large language models can help people make decisions ideally that align with their values, which then will nudge companies to then act in a way that aligns with our values.

Anna Rose (:

Interesting.

Yohei (:

So the opportunity of large language model is being able to align everything across the board because we can translate data into language, into behavior and whatnot.

Kobi (:

Yeah, exactly. It gives power to consumers that they didn't have before to actually nudge things in the right way or the way that they want.

Anna Rose (:

How fast do you feel this is moving?

Yohei (:

Very fast.

Anna Rose (:

Just give us a sense for how that looks. Is it day to day? Has it gone in spikes maybe?

Yohei (:

It feels like there's something new every day still, if you look at like, the AI landscape as a whole, but as like hone in on like the vector search, it spreads out a little bit, but AI is so vast, right? If you think about text to image, if you think about gaming, if you think about security, if you think about the fundamental models application there, if you think about all of those, then suddenly then it seems like there's news every day.

Kobi (:

Interesting. Actually, you just mentioned using data like bank statements and, you know, very, very sensitive data. And maybe this is a nice segway to topic that might interest the audience of this podcast, which is a bit about privacy. And what do you think about the whole move that people are putting very personal data into ChatGPT and like these API's that basically go to some single companies?

Yohei (:

You know, I mean it's interesting. Early on there, they were using a lot of, you know, any data they could to train it when they were small. So I think that's where a lot of the pushback came from. But since then, they've become much more, kind of allowing people to opt out and moving toward the direction where your data that you send in doesn't train the model, which is really I think, the biggest concern as long as the data you're sending in doesn't train the model, and there are options for that now. So I think that's important. I, I actually don't know if ChatGPT is doing, I think they stopped it from ChatGPT training the model if you use the

Kobi (:

I'm pretty sure they did I think they made a big announcement about it.

Yohei (:

Yeah. Yeah. And I think that's fine. I mean, if we talk about private information, right? Like, you know, everybody from Google to, you know, like all these people have all their private info if you don't want any big company to have any of your private data, then like, you're going to live in a very different world than the rest of us.

Kobi (:

Yeah. I think it's kind of like maybe semi absurd that we send, you know, emails on probably Google Workspace or Microsoft where we say confidential, internal only, internal use, never share it outside but basically it is shared with those entities.

Anna Rose (:

Even a draft, my friends, even a draft if it's saved it's somewhere.

Kobi (:

So maybe related to this, like how good are the models that you can run locally? Are they any close to what you can do with GPT?

Yohei (:

I haven't played it with myself. I've played with a couple non GPT models. Like I did probably last time was three months ago and it just was too much work to get the right output. That I felt like, oh, you know, also I'm building kind of on the side while I run a venture fund, so my building time is limited. So I try to maximize my build time, I try to tackle areas that I know I can have impact quickly.

Kobi (:

So I think that's an answer. Yeah.

Yohei (:

Yeah. What I'm hearing from most people is that, oh, you can get pretty close with the other models, but it's not quite there yet. It's good if it's for a very specific function, you can kind of like fine tune it for that. But that's what I'm hearing from people. So just haven't tackled it.

Anna Rose (:

Got it.

Kobi (:

Yeah. You mentioned some of the tools that you kind of used and created, but now in modern times, like after a year of this being in existence or 2 years, what should people look into? What tools should they use or what areas should people look into to make a difference or to do anything interesting with AI?

Yohei (:

I mean, I think these large language models are definitely something you want to learn how to use. What I tell people is I know if you haven't yet, start by using ChatGPT because it's the lowest hanging fruit. It's easy, the chat interface is a nice introduction. if you want to learn to build after ChatGPT, I tell people to go to the Playground. That's I think platform.openai.com, which is where you can kind of play in a web UI directly with the raw API. And it kind of gives you a sense of what it feels like to play with it, because you can kind of update the parameters. You can use a web UI to engage with basically the API directly. This will give you a sense of kind of what it's like, you know, what you're sending into the API.

(:

And once you have that, and even today I play with the Playground to kind of test my API calls, then you can start pulling that into an API call and then wrap it with your code and start kind of building a framework around it, whether you want to serve it as a web UI, create an API with it or whatnot. And as you want to learn about how to kind of connect these some tools I suggest to learn how to use would be, I mean, LangChain I think is just a really good introductory tool to start learning about kind of how agents work. I would say LlamaIndex is another one if you want to interested in kind of memory or dealing with lots of data, you know, retrieval from the larger dataset. LlamaIndex is kind of a big one in that space. And then if you want to learn how to build autonomous agents, obviously I'm biased, but I highly suggest digging into the BabyAGI code.

Kobi (:

That's cool. By the way, is there anything else planned for BabyAGI that you're working on? I think I've seen BabyBeeAGI, right?

Yohei (:

So BabyAGI, you know, was 105 lines of code. So of course we got a lot of pull requests. I don't know how to use GitHub. So Francie from the community was kind enough to jump in and offer to look at the pull requests and support pulling them in. So we've started adding a lot of functionality to the core BabyAGI. Unfortunately, I'm actually not really good at reading other people's code and it just doesn't work with my workflow, which is I need to copy paste the whole script into ChatGPT. So I started modding the original commit that I did. And so BabyAGI, the OG BabyAGI a hundred lines of code has been modded into BabyBeeAGI, BabyCatAGI, and then BabyDeerAGI. And then each time I add a couple of functions and really I'm experimenting with a core framework, so it's less about adding new tools, but like, let's just, you know, it's easier to rip everything apart when there's no integration. So that's, that's kind of what I've been doing, which is trying to figure out what the right framework for task management is and then, so right now, last night I was working on my next mod of that. It's an alphabetic order. BabyAGI, Bee, Cat, Deer. So the next one I'll start with E.

Anna Rose (:

Nice. Well Yohei, thank you so much for coming on the show and sharing with us kind of your story and the story of BabyAGI, like having this chat about this kind of booming space. Thank you so much.

Yohei (:

Thank you so much for having me.

Anna Rose (:

I want to say thank you to the podcast team, Henrik, Jonas, Rachel, and Tanya, and to our listeners, thanks for listening.