AI für Produktmanager

00:00:00: Speaker: Transform together – das Podcast-Meetup von und mit Johannes Schartau und Holisticon. Welcome to episode eight of Transform Together, the holistic podcast meetup. I'm Johannes Schartau and today I'm talking to Valerio Zanini. Valerio is a product management, AI and agile expert. He's the CEO of the product innovation firm 5G Vision and author of several books, among them the fantastic AI for product Managers and also deliver great products that customers love that I have right here. I have previously recommended these books because they're awesome, and if all of that wasn't enough, he's also a Certified Scrum Trainer, Certified product Innovation Trainer, and a SAFe program consultant. Valerio believes there can be no true agility without product thinking and I couldn't agree more. Valerio, thank you for joining us and welcome to the podcast. Hi, everybody. Thank you. Thank you for having me. So let's talk about your recent book, AI for Product Managers. I have lots of questions, so I would like to focus this podcast today on this topic. Let's start. What are the possibilities of using AI as a product manager. So I know a lot of PMS who say I already ask ChatGPT questions sometimes, but according to you, there's a lot more that can be done. Is that correct? Very true. I think we are scratching the surface and there's so much more. Um, just if you look at just what happened in the last year, the models got so much better. So many new tools came out. Right. So we are still kind of exploring what is possible. I think there are three main areas. The main benefits that AI delivers I call them accelerate, expand, simplify, accelerate is when AI tools allow us to save time, do things faster. So yes, we can go in and ask some questions. We can get a quick answer. Got that? Um, but one example of getting things faster, for example, is getting help to generate a PRD product requirement document. Many product managers are doing that now. Uh, or for example, using notion to summarize notes and documents and kind of things. Or even here in zoom we have an AI companion. And that companion can summarize a meeting. So we are doing a meeting with stakeholders research meeting discovery. So the AI can synthesize that. And that is all an accelerate kind of benefit saving time. But I think the real value comes from the other two areas the expand and simplify. Because there is so much that can be done there. Expand is the the idea that the AI tools allow to to do things that before were not possible. For example, every product manager knows that we need to focus on the problem and we need to talk to customers, right? To do this discovery before we make decisions on product. And for many, doing discovery means preparing an interview guide, scheduling some interview sessions, and then doing the interviews with the customers. This is often a linear process following the steps I just mentioned. And sometimes we get into interviews and we feel like, oh, something is off, something is missing. I didn't ask the right questions. I couldn't go deeper on this. Whatever, all these kind of things. AI expand by giving the possibility to the product manager to actually practicing those interviews with synthetic users. And you cannot talk to AI. So it's actually can be a conversation practices those questions in the interview guide, exploring whether they you get the answers you were expecting from them or something is missing. Maybe there is another question. Follow up question that could come up. So all of these exploring the edge cases, the boundaries, preparing for those interviews is something that before was not possible. I mean, yes, you could practice with, you know, your colleague but was could be could be done, could not be done at scale. Right. So that's one example. Yeah. And I really like when I was reading your book, you kind of mentioned these things. Um, so accelerate expand and simplify. And you also because you just mapped this to the product manager, but you're also saying if you develop any kind of AI product and it doesn't relate to one of these three things, then you're probably just playing around. So you this really helps to nail down your your value proposition I would say. Is that correct? Yes. Because as a manager, we always need to think about what are the outcomes that we want to deliver, the benefits that we want to deliver. And ultimately, AI tools Deliver, accelerate, expand and simplify benefits. Maybe all three. But if they deliver none of this. Well, yes, what is your point, right? What are we doing? Right. Is something missing here? Right. So, for example, one of the things I share in the book but also I, I make available as download on my website is a the three dimensions of the AI products is kind of a chart that people can actually fill in to see how much of accelerate, expand and simplify is their product delivering. And that is a kind of a kind of a mirror, right? Looking into the mirror and saying, are we delivering enough benefits here? Is this idea good enough that we should invest in it? Very cool. Because when I talk to people about using AI as a product manager, it's kind of on the surface level of I'm interacting with ChatGPT, Gemini, whatever it is, But you also really go deeply into this distinction between models and APIs and applications. Could you just unravel this a bit for us so that we know what this is and why it might be important? Yeah. And when I when I think about AI. So let me take a step back before just to split a couple of concepts here. When I think about AI manager, there are really two different areas. One is how can product managers use AI tools and so on to accelerate, simplify or expand their job. Right. The other area is product managers. They want to build products that have some feature or some AI functionality in it, whether it's an ID or an AI first product. Right. So that conversation about model, API and application is really a strategic conversation about a which level do we want to position ourselves if we are building an AI product? Are we building it at the model level? And that is the most complex and expensive and technology heavy, time consuming, uh, play you can do. In fact, there are only a few companies that are doing that right. And we know who they are OpenAI, Google, meta, a few of these, right. Because building a model is such a big, time consuming, energy costly endeavor. So that's one strategic play. The second level is the APIs. The APIs are the interfaces that allow us to interact with the model. And they have a strategic value. Strategic advantage because you are the kind of you control the access to the model. And if you can build the API level, you can create custom filter, custom access to the model, custom kind of Handle customizing a model. For example, you can take an open, open source model like Mistral, for example, and then build a set of APIs on top of it that can, can can be specific to a specific use case, maybe not as complex as building a model, but still a kind of a complex endeavor, right? The third level, that's the top level. That's the application layer. And that's probably where most product people really work at, right? And the idea is let's build a tool, let's build an application, let's build a feature that customers can use that leverages AI functionalities. And we are not building the AI functionality per se. We are leveraging existing models, existing APIs. Now we are just connecting to them. And that's for example, the work we can do with OpenAI using the OpenAI engine that feeds ChatGPT. But we can use that to build our own tools now. So we have a bit more insight in that most of our clients still struggle with adopting a real product life cycle, or managing the organization in a way that we can develop along a product life cycle. Now you're saying that AI is coming in and it's already disrupting this product life cycle. Could you let us know a bit more about how this is happening and what this means for organizations trying to develop products in the future? Oh, yeah. There is actually, you know, if we start digging out that topic about this so much, we can talk about what we can do, like maybe three podcasts on that. Um, so high level, the product development life cycle, I call them the five dimensions. You go through discovery, define, design, drive and finally deliver. Now every team is different. Every team puts a different set of like effort or time into each one of these dimensions. Ideally, they're never linear anyway, but the point here is that AI is actually shifting a lot of what is happening here. For example, now as a product manager, I can actually prepare an interview guide. I mentioned that earlier. I can even practice an interview guide. I can create, synthesizing, synthesize users. I can interview the synthesized users and then synthesize the interviews, generate insights, and then ask AI to do ideation for me so I can go from like an idea of a problem to actually possible solutions within a day. And I can do that all on my own. So that's a huge acceleration of the expansion of the capabilities of a product manager that is disrupting the product life cycle, because now I can get from a problem to a possibility, or even I can use Lovable, Replit or other tools to build a prototype of that. And it could be actually a working prototype I can put in production to test with some users. MVP testing hypotheses again in a day. So huge acceleration of that product lifecycle. I'm not advocating for doing the product management work all in one day, all of your cubicles and then putting in production and and finished product just because you thought it was a great idea, right? So AI can give you this acceleration, but you still need to kind of validate the idea. You still need to test it with real people, get real feedback. Right. This kind of thing. So there is still the work of the product manager is still important, still valuable. Right. We need to do all of that. It's just there is an acceleration happening. One thing I'm really thinking about is we've been talking for the last, uh, well, several years, I guess the product operating model, the triad product manager, designer, researcher and then the engineer. Right. Working together, creating that strong collaboration to ideate, to to research a business problem and the solution, then build and validate that. Right? So that's the idea. And a lot of companies are are trying are being trying have adopted the operating model or some way on the spectrum or getting there right now with all these changes that AI brings, these new capabilities, how is this changing? How is the operating model being impacted? Because if your product manager can do a lot of these things on their own, do I need to work with the researcher or a designer when I can ask lovable to create the design for me, right? Or do I need to work with an engineer? Or I can ask also lovable to build a website for me, right? So there is a lot of capabilities now can be integrated within the product manager role. But again there is also the risk that now we take for granted whatever AI is creating for us and we give too much value to to that result without validation. So there is a problem of trust at some point. How much can we trust? What AI is telling us? There is probably is probably over overvalued trust. Right now the people are putting into the AI results and we still we still need to consider traditional management approaches, right? Talking with customers, validating ideas and so on. So what does it mean for the operating model? I don't have a good answer, but I'm really question how is that going to change. Because ultimately it's going to affect that relationship. And what I think it should happen is all three roles should be empowered to use AI. So it's not the product manager alone, but also the designer research and then the engineering role on the on the triad all collaborating together. Let's use the same tools. Let's do discovery together. Let's build a prototype together. Yes, we can do that faster. And we can do that together so we can bring the different perspectives and avoid being solo, being siloed right in one in one idea. I think that's that's a better approach. So expand the capabilities, but do that for everybody on the team rather than just one one role. So what you're not saying is that I can finally stop talking to other human beings, but there's still interaction that needs to happen. That needs to happen. Is that correct? I don't think as humans we can replace humans, right? I don't think I don't think so. There is a huge boost simplification benefit in expanding our research, looking at the edge cases. Um, I said earlier, practicing interviews, uh, uh, discovery may be unclear or unknown hypotheses or assumptions. Sometimes we are making some assumptions. Some some hypotheses. We may be explicit about them, but we do not see others. And so there is value in that. There is value in doing a pre-mortem exercise with using AI, so that it may help us understand where we may fail if we move forward with that. So there is a lot of benefits here. But again, AI doesn't replace humans. Why? Well, there are many reasons. Maybe it's useful to talk about why. Right? Um, probably many reasons. One of the, one of the main reasons is that the models are trained and fed with existing data. And everybody knows that you cannot ask what is the weather today to judge? Because. Right, as you know, but if you ask what the weather was six months ago, they probably know. It probably knows. So it's all looking in the past. That's all the trained model. So the reasoning, the probability building, all these kind of things. Right. All the parameters are built using data in the past. And we we expect all of that to work for predictions of the future, because that's what we're doing, right? When we build a new product, we are trying to build something for the future, right? For for new user, new use cases. So there is a kind of a gap here. We're using models trained on older data to build and make decisions, something that is for the future. There is there is a risk here. I think the next frontier for AI models is actually predictive analytics. We we are not there yet in terms of predictive analytics predicting the future. But that's what is is the false expectation today that just because AI can make some smart decisions, which are all probabilistic in nature, they are not creative in any way. We falsely expect that the future can be described using past data. That's the risk. That is a huge risk there. Do you have any more examples of certain things that AI tools are just not great at yet, that I might just fall into the trap as a PM, that I feel like I can rely on this or this is something I do. I heard you talking about the fact. For example, I probably don't want to push AI generated code into production immediately. Maybe that's not a good idea. Do you have more examples of that? So several examples. One thing that comes to mind, just connected to what we were talking about a second ago, is really product innovation. The idea of product innovation is that you're building something new. You are solving a problem with the new solution. And because of that, it's hard to make that happen when you're using old data, right? So there was a actually a post written by Melissa Perry, which I, you know, I respect a lot. And she was talking about exactly this problem. And she gave an example of Tinder, the Tinder app. When you swipe left or right? Right. And that idea, that swipe gesture that was an innovation Tinder introduced at the time. Nobody had done that before. It was completely something completely new. And yet this simple gesture revolutionized the way we interact. Now, you know, dating and these kind of things that the point was, I could not come up with that innovation because there was nothing in the data that, you know, that led to that. So if you're seeking real innovation that we need human thinking. You mentioned putting code in production. There is a risk. And there's a lot of conversation happening now about increasing technical debt when we build coding and these kind of things. Right. And we don't have really the the eyes into the engine of the car. So there is a risk of building technical debt. The leader may become very big and difficult to address. Uh, I mentioned predictive analytics as another area that is still under development. But then there are all the all the edge cases where human intervention really is needed. So, for example, for life threatening or security situations, delegating decision to AI is is just risky. So we need to put a human in the middle. So we need to understand what kind of situation we are working on. What is the context. As product manager, I mean what is the context we are working on to decide where are the risks and how do we minimize those risks? Yeah. So an example is AI is great at taking an x ray and analyzing that x ray or MRI. But then because it has seen millions of other x rays and can spot a probabilistic pattern, but then going from there to actually making a decision on whether you need surgery or you need radiation therapy or whatever, or you don't need anything because you're fine. Well, that. Can we trust AI to do that, right? So that's where the human in the middle is important because there is a life threatening situation there potentially. Do you have an example of a mistake that you made yourself with AI early on that maybe taught you something important? Yeah it is. My house would, uh, crash? Let's hear it. That sounds sounds dramatic. I shared this story actually, in the book. Uh, in my book on AI is, um. Last year, I was doing some work at my house. I wanted to take down a a load bearing wall, and I wanted to put a structure, a metal structure to support it. And, um, although I am an engineer by education, I'm not a mechanical or structural engineer. So I asked around, of course, for some help. Structural engineers, I thought, let me ask ChatGPT as well. So I put all my dimensions and you know the details about the job into ChatGPT and ChatGPT came back with something like, oh, you're fine. You, you know, whatever you are thinking is way too big. You don't need something so strong. Okay. I mean, that's good news, but can I see your formulas? Just. I want to double check. What? What are you saying? That. Give me. Show me your formulas. So just show me the formulas. And in particular, one of them was wrong. Big time. Wrong, because it went from square inches to square feet. And this is a one hundred and forty four multiplier between the two. So a kind of under under dimension the design by one hundred forty four times. So not not a good place to be if you are a structural engineering work. Yikes. Yeah. So this goes back to my to my point. The point about trusting, right? Yes, yes. It's a great tool. We need to remember. This is a probabilistic engine works on probabilities and it cannot be one hundred percent accurate. There's always a level of error. And we need to understand that whenever we get the results interpret the results, right? So it's more of an enhancement, but we're still in the driver's seat. It's not like this is taking over anytime yet. We the all the responsibility is still with us. Yeah. Yeah. I like to think, like, um, I take a walk outside, you know, in my street, and I'm. I meet my neighbor, and that guy's very smart. Knows a lot of things. Right? And I ask him a question, and, of course, he's gonna answer the question based on his own knowledge, which is great. I also can be limited, right? Super smart. But for example, if I don't give him enough context about what I'm trying to do, right, he's going to base his answer based on whatever his assumptions are about that. And now we have a gap right understanding. And so that goes to context engineering being being sure that we provide enough context and be clear about what is exactly we want and expect the AI tool to do. So they can give us the answer that we get. Minimizing the error which is always there anyway, cannot go to zero, but at least we try to minimize that. All right, I'm a PM. I think this sounds fascinating. I want to get started using AI. Do you have any tips of some of the first things I might be able to do? Maybe. I've been listening to this podcast. Where do I look? What is something that I can try that is maybe has a low barrier of entry or a high return for relatively little effort? Any tips for me to get started? Yeah, so there are many ways to get started. Is about breaking the barrier a little bit of resistance, about maybe fear, about using AI tools? Um, probably every tool that you're using today already has some AI capabilities. Even a JIRA, for example, has some abilities or notion. Miro. Right. So play with those because nothing can break, right? Just play with those. But one really one thing that I really work for me, at least in kind of breaking my resistance with the AI, was working with a Vibe Coding tool. I started with it. But you know, there is lovable. There are many more, like basecamp and so on. Now, even Claude can generate code inside itself and try to experiment with the creating an interface, an app, a a website, even if it's not specific to the line of work you do. You use those tools every day anyway. So try to experiment with creating one of those, and it doesn't take that long time. It can be just a couple hours in a day, but by doing that, we kind of break the barrier of, oh, I feel so hesitant because I don't know how to use AI, I don't know where to start. It's kind of it's all about starting from one place. There will be my suggestion. I start from that place and and see what happens. Thank you very much. So we'll link to your work in the description of the episode. I just want to really recommend your books again. So we have, uh, deliver great products that customers love. What I really liked about this is it's at this perfect intersection between agile and product thinking. And most books or approaches are kind of in one of the two camps. And what I think you've done so well is really bridge this and make sure that we get the best of both worlds. And then also your new book, AI for Product Managers. Um, before we started recording, I was talking about how you sent me an advanced copy and asked for some feedback, and I really wanted to give you a really good feedback. And then I was reading it and I was just thinking, this is so good, I'm not sure what to add. So this is kind of my endorsement. Everybody please read it. Check it out. At Holisticon, we put into practice what we discussed in Transform Together. We provide companies with holistic support throughout the digital transformation process, from strategy and technology to organizational change. We make our customers more resilient, establish a future proof agile culture, create new business models, inspire people with better services and products if you are facing similar challenges or wondering how you can implement the ideas from this session in your organization, please feel free to contact us. You can find all the information you need at holistic, and don't miss our next Transform Together session on January fourteen at four o'clock pm CDT. So we'll take a short Christmas break, but in January we'll be back. I will be talking to Sylvia Taylor about change management, adaptability, intelligence, and the new game that she created around continuously finding your purpose in an ever changing world. See you then.