Christiaan Verwijs & Daniel Russo

 TRANSCRIPT

Speaker: Miljan Bajic  00:43

All right. So maybe Christiaan, let’s start with you. Who’s Christiaan Verwijs?

Speaker: Christiaan Verwijs  00:50

That’s a very existential question. Well, let’s just say, I’ve been Scrum Master for 13 to 15 years, quite a long time. I started Deliberators together with [unsure 01:04] and familiar face in the community. And I think Barry and I are both very passionate about working with Scrum teams. It’s not even about Scrum, it’s just about what they make possible. We both have very good backgrounds in Scrum teams, we’ve seen how well it works and also what it can create for teams. And I think we start, we noticed that it was very different often in real life for other teams. And I think that’s why we started deliberators just to help more Scrum teams actually get something out of it and make it work for them and enjoy doing it. So partially Scrum Master, but I’m also an organizational psychologist. So that’s sort of the academic background, social more relevant, I think, for the paper that we wrote. And there, the focus is basically leadership, motivation, theme processes. I’m very interested in that and I’m also a professional Scrum trainer at Scrum.org so that’s sort of the three most important things about me. And also important, I have a cat and my wife [unsure 02:03], she’s sitting next door, she’s a photographer, and she’s really cool person. So that’s also important to emphasize, I think.

Speaker: Miljan Bajic  02:11

Nice. Yeah, I was gonna ask you about something, you know, besides, you know, some people might not know about so. I know, I checked out your website and you do talk about it in your website about your cat and your I believe your wife. So what will be something that people might not know about Christiaan?

Speaker: Christiaan Verwijs  02:29

Well, I want to say I’m a huge gaming nerd but if you follow me on Twitter, you probably know so that’s probably not something…. Well, I think most people, what they don’t know is I’m really into exercising, I really like to keep my body fit, especially with Corona and sitting at home, I’ve been doing that a lot. And that’s something I always do at home so people don’t see that. I don’t go out running, I have a treadmill upstairs and a rowing machine so I really enjoy doing that. And I watch a lot of movies during that time. So that’s one of the other upsides of that.

Speaker: Miljan Bajic  03:01

Nice. I do have the equipment but I don’t seem to use it as much as you.

Speaker: Christiaan Verwijs  03:07

Familiar problem I think with that kind of equipment yeah.

Speaker: Miljan Bajic  03:11

Well, thank you. Thank you, Christiaan. Danielle Russo, who’s Daniel Russo?

Speaker: Daniel Russo  03:17

I’m a professor of engineering at Aalborg University in Copenhagen. I am doing research mostly about Agile software development. And right now, I’m looking much more deeper into the effects of the pandemic, of the lockdown to software developers, software engineers and to try to find out how to keep productivity and well-being high.

Speaker: Miljan Bajic  03:50

Nice. What about the fun side of Daniel Russo? 

Speaker: Daniel Russo  03:55

The fun sides of myself. I’m an extremely serious professor right so I don’t have any fun sides. 

Speaker: Miljan Bajic  04:06

I’m sure that’s not true. 

Speaker: Daniel Russo  04:12

You have a lot of confidence in me I see.

Speaker: Miljan Bajic  04:13

I do. You wouldn’t be hanging out with Christiaan if you didn’t.

Speaker: Christiaan Verwijs  04:20

We have so much fun right?

Speaker: Daniel Russo  04:24

Yeah, yes. I love sailing basically right. So I think this is my real passion so I am blessed to live in Denmark where you know we have a lot of water so I, so I loving doing regardless and all this kind of stuff and you know the general day, see environment and atmosphere. Yeah, I very much enjoy it. 

Speaker: Miljan Bajic  04:55

Nice. I’ve never visited that way but I am planning to. Usually, I was born in Sarajevo so like, most of the time, when I go, we just fly to either Serbia or Montenegro or Croatia. And I’m going to spend two months from late August to late October in Europe. But again, it’s like we just try to fly as quickly as we can over there, little take the time to travel to Europe but we’re gonna try to take some time and do that. You guys spent seven years investigating, researching, and I don’t know if you guys did it together or what happened, I want to hear from you. You’ve interviewed or you’ve looked at, you get feedback from around close to 3000 people over 1000 themes. Could you maybe give us a background on what triggered this research and what was the background behind it before we dive into some of the details?

Speaker: Christiaan Verwijs  05:53

Sure, you want me to take this one Daniel and then I’ll pass over to you?

Speaker: Daniel Russo  05:56

Absolutely go ahead.

Speaker: Christiaan Verwijs  05:59

So it started, actually, it started maybe at the beginning of my work with Scrum teams, because I was always, I’ve academic training so I always look for ways to connect my work to scientific insights but I kind of quickly discovered that there’s not a lot of that in our community. And basically, what I’ve done is over a couple of years, I followed the scrum teams that I work with and I observed how they worked. And we use that information as input for the case studies that we share in the paper. So it was a systematic approach to see how do Scrum teams work? What are the challenges they run into? What factors influence their effectiveness? And from that we developed a model that we’ll talk a bit more about that in the coming minutes. But it started quite a long time ago, actually. And the fun thing is that it was sort of a hobby project, because I really like to, I don’t know, create some more reliable knowledge. And I basically started this as a hobby project where I started gathering data also with the strong team survey, and it’s available online. And then Daniel reached out to me to ask if it was possible to either analyze the data or work together on it. And I think that’s where our collaboration also started, which was awesome for the primary reason that it was a great way to connect the practice of working with Scrum with scientific analysis and the rigor that comes with that. And that’s how I met Daniel. And I think, Daniel, that that’s like the starting point, right?

Speaker: Daniel Russo  07:31

I guess so. Yes. Yes. So I also started to work about Agile software development from the early start of my PhD. And so this has always been very interesting research topic for me.

Speaker: Miljan Bajic  07:54

How’d you find Christiaan? Like, how’d you find about what he was doing? Beacause I’m assuming you reached out to him.

Speaker: Daniel Russo  08:02

Yeah, yeah. So I did because basically, I started my, let’s say, research journey into a dry with very, let’s say, standard approach, doing case studies, field studies, and so on, so forth. And basically, during the late years of my of my PhD, you know, I started to become a little bit more quantitative in my research approaches. So I look in much more detailing to structural equation modeling. And where basically also where analysis of his paper is based on. And I was looking for saturated data, which you know, could actually fit potentially SEM, Structual equation modeling. And I went basically through internet browsing, I saw his scrum survey and I thought going for it was actually, it was actually a very nice instrumental and tool And then, you know, I just reached out to Chris and said, Why don’t we do something together?

Speaker: Christiaan Verwijs  09:12

Yeah. And what the cool thing is about that because you just mentioned that briefly Daniel, like, structural equation modeling, which is, it’s this really advanced statistical technique that we’re probably not going to go into in detail. It’s really cool thing. I’ve always been very fascinated about what’s possible with those kinds of techniques. So I was already doing that sort of in my own time, I was analyzing the data with structural equation modeling. But the thing is, there was no one in my vicinity who knew anything about it. So it was also very hard to get help. And Daniel is in his field also one of the few that’s using this technique. So that was pretty cool that we sort of it was possible for us, I would say statistics nerds to connect on that level too. And that was great for the paper because structural equation modeling really is very powerful for this purpose.

Speaker: Miljan Bajic  10:01

Yeah, and I mean like, one of the things that you mentioned that resonated with me is like Scrum is very popular, agile is very popular, it’s been around, but like, there’s not really a lot of good research out there. There are case studies, if you go to save website, if you go to, there is a case study one offs, probably one person, marketing person writing it, there is no real data over time or so it’s surprising the way but it’s also good, you know, in a sense of the platform or maybe example that you’re setting for others because I think we need research like this to give us a little bit more insights and into effectiveness, not just of Scrum teams. But you know, a lot of other things that are surrounded, you know, organizational design, like, you know what. So maybe let’s dive into the some of the research. What was the most surprising thing to you guys as you were collecting this data, analyzing this data? What was the most kind of surprising aspect of this process?

Speaker: Christiaan Verwijs  11:19

Daniel, feel free to take this one.

Speaker: Daniel Russo  11:21

No, no, please go ahead. 

Speaker: Miljan Bajic  11:22

Or maybe there were no surprises. Maybe it was…

Speaker: Daniel Russo  11:25

No, but you know, I think that it was interesting because Chris and myself, we have actually two somehow similar but also very different mindsets right? So I basically, rarely get surprised at somehow, right because I mean, especially if you work with quantitative data, right? So you have an outcome and basically, it’s your job to make a you know, sense of a specific outcome. Sure, I mean, not all our research hypothesis has been supported but it is absolutely normal in any research. So I mean, from a very, let’s say, academic perspective, not much, but from a practitioner perspective, there definitely were a few surprises and therefore I like to ask the question to [inaudible 12:21].

Speaker: Christiaan Verwijs  12:21

Well, maybe just one thing that surprised me at the very start and that’s part of the collaboration with Daniel also is that I discovered for myself that there actually is a substantial amount of scientific research into agile Scrum and so on. But that research is somehow not finding its way to us as practitioners, which it’s just a waste, because there’s such useful research being done. But the other way around is also not happening. So practitioners are not really reaching out to academics to work together. So that was sort of surprising on a meta level on the outside of the analyses. About the paper itself, the study we did what I think was the most surprising was that the model that we had in mind, based on the case studies that we’ve done, actually fitted the data really well. So it means that it makes sense, right, based on the data, it makes sense. But there were also some effects that we couldn’t find in the data. As Daniel said, there are always hypotheses that you are unable to find, for example, that I was personally really expecting that very autonomous teams with higher level of autonomy would be able to respond more quickly to release more frequently. But we did not find that effect in the data. That doesn’t mean it’s not the case. It’s just something that we did not find. And I’m still, like, we’re still trying to figure out what does that mean. And that’s also the nice thing about research, it opens avenues into further research and further investigation into what may be going on there.

Speaker: Miljan Bajic  13:51

Yeah, that is very interesting. And I think that’s something that stood out to me too in a sense, organizational structure has a huge impact on teams and how teams operate. So you could have very decentralized structure. And if you just look at the scrum team in that context, things might be different. If you have a very hierarchical structure, and you’re only zooming into a theme, then that’s going to happen then now even you know, to look into the technical side of things, if you’re looking at the scrum team in a government organization, or large company might be different how they still use cobalt or you know, that whatever the technology or systems they use. So that is I think, something that, from my perspective is worth maybe researching more and understanding the dynamic between those things.

Speaker: Christiaan Verwijs  14:47

Absolutely. Yeah, it’s maybe also good to mention that when we analyze the data, we actually we knew from the teams from what kind of organizations they were from, like very big, very small, what sector so on. And we actually the model that we present in the paper generalize across different sizes of organizations. So the effects we found are not different between small and large organizations. And that was in a way surprising to me, because I was thinking that in large organizations, the dynamics are going to be so different that the model will be very different too, but that actually seems to be not the case. There are small differences of course, but they’re not significant.

Speaker: Miljan Bajic  15:27

That’s interesting. And it was also interesting for me to, I kind of pondered for a couple of days. 

Speaker: Christiaan Verwijs  15:36

You dropped out, your audio dropped out I think.

Speaker: Miljan Bajic  15:38

Let me see. Can you hear me now>

Speaker: Christiaan Verwijs  15:41

Yes, I can hear you. 

Speaker: Miljan Bajic  15:43

Yeah. So I’m not sure, he was saying my internet is unstable somewhere. Maybe that’s why, maybe the hardware. So I was saying, like, I was thinking about the key takeaway and conclusion from your research for a couple of days. So maybe could you describe what you found out? And then we’ll dive into each of the areas and maybe depict but at the end of the day, you know, what is the theory of you know, scrum team effectiveness? What did you find out?

Speaker: Christiaan Verwijs  16:21

Daniel, you want to take that one? 

Speaker: Daniel Russo  16:25

Ah, again please start and I will may be…

Speaker: Christiaan Verwijs  16:29

Sure. So the summary I think of our study is that we’ve identified five core factors that together determine how or at least determine to a very substantial degree how effective Scrum teams are. And those are responsiveness of Scrum teams. So how quickly can they release the concern that teams have for stakeholders, their level of autonomy, the climate of continuous improvement, whether or not the operating one and the support they receive for management. Those five factors together, predict a substantial amount of effectiveness of Scrum teams. And effectiveness, we define that as stakeholder satisfaction, and high team morale. So effective teams have satisfied stakeholders and high morale and from those five factors we described, we can actually predict that to quite a substantial degree, which is really cool.

Speaker: Miljan Bajic  17:22

It is and like, it’s also like common sense, right? You would think that, hey, you know, if I want something and you building it for me, if we continuously collaborate, and I understand what you want, then you know we’ll probably get on the same page. And in real life, you know, it’s very hard for teams to talk to the stakeholders. I had one situation where I was asking a senior leader and describing this, like, teams should be talking to the, developer should be talking to your stakeholders to customers, whoever the stakeholders are, right? And they were like you crazy, Miljan, you want my weird developers to talk to the stakeholders? You know, and like it was a trust issue. It was like, you know… And then you talk to developers, and they’re like, Hell, no, I don’t want to talk to the stakeholders. It’s like, leave me alone, I want to do my work.

Speaker: Christiaan Verwijs  18:20

And you know what’s funny about that? Because that’s something that’s super recognizable from all the teams that I’ve also worked with. But with one team that I’ve worked with the longest, we actually had that as well, like developers were not interested in talking to customers. But at some point, we were building a new product for our customer, a large enterprise customer and I just said, you know what, we’re going to work for a couple of sprints at the customer site, which was a big office, and our company was a smaller one so we were not used to big enterprise offices at all. We just went there and we worked on that location and it was the most fun time we’ve ever had. And if I meet people from that team now, they still remember that, just how cool that was. And it was scary. Of course, it’s scary to talk the actual stakeholders and users and customers but the feedback we got, that was incredible. So absolutely useful.

Speaker: Miljan Bajic  19:12

So let’s maybe dive into the stakeholder concern. You have kind of like four subsets underneath that. And one of them is value focus. And this is another area where it’s weird in a sense. I asked this in every situation. I asked developers, do you know what value is? How value is described or defined by your product owner? And most teams don’t know. I do this in classes too, and you’re trainer, ask them, how do you define value? One or two hands will go up. So when you look at the value of focus, it’s one of the key things but what are your thoughts and if nobody knows what that value is, how can you focus on value?

Speaker: Christiaan Verwijs  20:02

What do people in your classes say when you ask that right? How do they respond if you ask what value is? What kind of words do they use?

Speaker: Miljan Bajic  20:10

Usually like you know, return on investment or you know, customer satisfaction. And I say all of that is true, but usually, it’s multiple factors you know. I’ve worked with a company San Francisco and they’re building essentially, artificial screening of they want to put boards on like Uber and share rides. So it profiles you as you walk down the street to say, hey, here’s Christian, or here’s Daniel, show him this type of ad. So essentially, Google ads and it’s illegal, you can do that. But they’re trying to figure out if they can do it then you have Google ad on Uber. And for them, they get $7 million in funding. So for them, it’s not about customer satisfaction in the sense, maybe it’s a stakeholder, but it’s really about learning and trying to figure out if they can make this work and make it legal. So I tell them, that’s different than when I work with like a publicly traded company, insurance company or bank, where they have different definitions of value or what value is. So I think what’s usually missing is the discussion around what do we define as about? So from your perspective, this value of focus, what did you like, when you look at the data and maybe some of the things around value focus, what are your thoughts?

Speaker: Christiaan Verwijs  21:43

Well, in this case, what’s always tricky with when you sample 1200 Scrum teams is that they work in different environments, right? So value is going to be different. But what we did ask teams is there a strategy behind the work that you’re doing that is focused on creating something that’s valuable? Do you actually talk in your team about what value is? So basically, it’s more on a meta level, like what you said like, right. So do you actually have conversations around it? And one of the things that’s really important that also came out of the case studies is that the more effective Scrum teams work closely with stakeholders. So that’s why stakeholder collaboration is also part of stakeholder concerns, obviously. But if you talk with stakeholders, it will be easier to understand what the value actually is in the work that you’re doing. And product owners can facilitate that but they don’t have to be the only person in the team doing that. And maybe you can even have teams without product owners that also are very good at this. So I think that that’s what we tried to measure in the survey but measurements are always really hard.

Speaker: Daniel Russo  22:50

And if I may also add something on this to try maybe to demystify a little bit of the paper and things from such as basically any measurement instrument available on this planet, which is supposed to be generalizable, it has its limitation. Right? And there is absolutely nothing we can do about that. And as Christiaan said, correctly, it’s much more about the meta level. It’s as I also like to say it’s not a receipt book, right, where if you’re going to do A, B, C, and D, then you know you will great value is defined in these three different categories. And the three different categories are very same for all company around the globe, right? I mean, this is absolutely, totally unrealistic. So what we think actually is of much greater value is to see if actually people, if teams are actually reasoning around certain topic in order to tailor their specific strategy, right? Because I mean, at the end of the day all agile processes are basically a fettish right? So basically, there is no one fits all solution but it it’s also highly dependent on the specific organization. Yes, yes, yes, yes, yes. 

Speaker: Miljan Bajic  24:31

Yeah and I just yeah, I think going back to what Christiaan said about you know, that Meta, it was just the discussion. I think you know, it is like for the teams to start talking about value, whatever you define it, but it’s important then I can, I guess it resonates with me, like with teams that I’m coaching or training or mentoring, I am always trying to help them understand the importance of that discussion around what value is and our focus. So that stood out to me and also like under that same area sprint review quality. You know, in your paper, you talk about like, what’s the quality of sprint review, which goes back to stakeholders, which goes back to the feedback. And I think that’s another area that resonated with me because I said, yeah, absolutely. You know, this sprint review is a good indicator, how much collaboration and how much we’re talking to the stakeholders. From your perspective, and maybe Christiaan from you for as a practitioner, what are your thoughts? I mean, you’ve seen all kinds of, you know, but sprint review is a good indicator of what’s going on, right?

Speaker: Christiaan Verwijs  25:51

Yeah. Well, what we, basically the study that we did just one step back. And what we did in the study is we use 13 case studies to see what patterns can we find in all those Scrum teams that were part of that. So basically, it were 13 organizations with one or more Scrum teams. And what we did is we identified a whole lot of variables. So sprint review quality, stakeholder collaboration, value focus, we already covered some of those and we used statistical techniques and existing literature to identify a structure like a higher level structure. And those are the five core factors that we talked about. So right now we’re talking about stakeholder concern and below that are value focus, spring goals, stakeholder collaboration, and sprint review quality. And then we actually tested this whole model with data from another source. But I think that that would be really cool to talk about next. But basically, for sprint review quality, my own experience is that it absolutely is one of the first things I look for with teams and also one of the first things we tried to change. And I remember there was a sprint review once with a customer that actually was a really cool company. But there was a sprint review, it took five minutes, everyone, a whole lot of people came in, like from all over the place people came in, there was a developer who turned on a projector, they showed an API and a couple of responses that were returned from the API, that was the demo, everyone applauded and they were gone. That was it. Barry and I were both there, we were like, what just happened? No feedback, no questions.

Speaker: Miljan Bajic  27:30

But everybody was happy that they weren’t held hostage for much long.

Speaker: Christiaan Verwijs  27:36

Yes. So they seemed happy, because you were applauding, right. But to me, a good sprint review is about the conversation again, about talking about what actually did we do this sprint? What does it mean for the work that’s coming up? What are the problems we were in into? What else changed in the meantime that we need to account for? And hey, stakeholders, what do you think? Is this actually what you’re looking for or not? But that’s not as often happening as it should. And we can actually say with our research that this, we already know this as practitioners, but it’s really important, it makes teams more effective. But maybe, the analysis would be interesting to talk about more.

Speaker: Miljan Bajic  28:14

Yeah, let’s maybe talk about that. So tell us a little bit more about the analysis and what you guys did there.

Speaker: Christiaan Verwijs  28:20

Yeah. And Daniel can do this properly way better than me. So because…

Speaker: Daniel Russo  28:26

Right, so I think let’s start from the basic at least to provide an understanding of the whole research process, right. So there are more or less and it’s clearly an extremely simplistic view but you know, let’s save it for our two main research philosophical approach, which is basically a very constructive inductive one, where basically we are observing the real world as it is, and we are after from that basically, inferring our theories, right. So what are basically theories to understand which are the relevant phenomena that are happening and how both phenomena relate to each other.

Speaker: Miljan Bajic  29:13

Alright, so that would be just to clarify, that would be like sitting with the teams observing.

Speaker: Daniel Russo  29:19

Absolutely. Yeah, yeah, doing it in an interactive way typically, over a long period of time, until you reach with a call for consideration. So basically, meaning that no additional information adds value actually to remodel your use to so far, right. And this is actually a very, very effective, although very long and expensive way to, you know, to understand the world as it is. Great and, you know, and actually, so far, most of academic papers about the dry are more or less this way. We wanted to go, let’s say one step further and actually see and validate whatever. Our you know, theoretical observation, actually, were also fitted into empirical truth, right? And what does this means? This means that we have to operationalize all of those different phenomena construct we observed. And after verbalization, I mean, I’m telling this in a very short and again, simplistic way, for each of the steps. I mean, I guess if each of these steps should do be in a podcast per se. Anyway, after having operationalize the constructs data were gathered from Scrum teams, right. And most significantly, you know, it’s not 110, 50 Scrum teams, but it’s almost 1200 which is an amount which has never been used in computer science. Alright? So I mean, I think this has been excellently remarkable and I think we should all thank, especially Christiaan that was, you know, the first man in line in collecting it actually.

Speaker: Christiaan Verwijs  31:36

Just started it sort of as a hobby. So…

Speaker: Miljan Bajic  31:38

But this is just a comment on this, I think, as you’re describing this Daniel and like what Christina alluded to, like, we’re clueless as practitioners, most of us are clueless about this stuff. And I think we can work more better, like what you guys did is a great example where we can as practitioners, we can educate ourselves a little bit more about these methods, so we can help and do more stuff like this.

Speaker: Daniel Russo  32:04

Right. Absolutely. Absolutely. I think that generally speaking, the academic community is also very open to that. So one very typical discussion Christiaan and myself are having is, you know, why you know, are both two different communities working on very same topic, just so distant? Right? I mean, it makes absolutely no sense, you know, because, for example, we and with we, I mean we in academics, right, I mean, we really put a lot of time, work, effort, resources, you know, into finding whether base to improve Scrum and to agile in general and right. And, you know, and on the other hand, you know, so I am learning from Christiaan, you know, that basically, someone invents, and I want to [inaudible 32:59] but you know, someone comes up with hierarchy framework, basically, on the top of heirachies mind right without any kind of empirical validation. But as an academic, this looks quite weird to me to be honest. Right? But anyway, the reason is that, you know, this two community, I, you know, I don’t point fingers, I don’t want to blame anyone, you know, it just happens, it’s just this way. So, I would say it my call, both, you know, to my colleagues, and both to the practitioner is let’s try, you know, to build bridges. I think that we both have a lot to learn from each other. I you know, Christiaan and I would not have met but I think that I’ve learned much more from him than he actually has learnt from me so you know that’s just how that it is. 

Speaker: Christiaan Verwijs  34:07

You’re way too kind. 

Speaker: Daniel Russo  34:11

And I think, you know, which is a pity, and I think that we should reason up and we should all… 

Speaker: Christiaan Verwijs  34:18

Well, maybe to add to that, because that’s something that Daniel and have been talking about, like for the years that we’ve been working together. But Miljan, you actually mentioned it before that the model, if you look at it in the paper, it actually feels like common sense, right. As a practitioner, it makes sense if you look at it. And I think that this is a good example. In this case, it makes sense and it actually fits the data, which is great, right? So we can actually empirically verify, okay, this is the case, but I think in our community, a lot of people and that includes myself at times I will absolutely admit to that, say things that sound true but are not actually true when you observe what’s happening. So for example, safe right? I know nothing about safe. I’ll be completely honest about this. But I do know that in the trainer community and the coaching community, people are like safe and it’s stupid, doesn’t help. If you look at the scientific evidence, it’s far more nuanced than that. There are actually papers, they’re like, okay, we see some benefits of it to choose from these kinds of organizations, maybe not so in these kinds of organizations. I think that sort of nuance is missing. So it seems common sense, safe doesn’t work but it’s actually not true. And I think that that’s the pursuit that if we do that together, as academics and professionals, our customers, our clients, the people we do this work for actually benefit because we can help them more, more honestly, and more reliably. And I think that’s our ethical responsibility.

Speaker: Miljan Bajic  35:44

Exactly. And that’s, it’s really, like we’re not aware of our own biases. Right? And it’s the same, it’s easy to, you know, have an opinion without any, you know, data besides, you know, this is what I think over, you know, period. And I think that’s the case with safe, I think that’s the case with Comba and that’s the case even from a different perspective with Scrum, you know. And I just think like you said, the stakeholders and everybody benefits from having better understanding of what’s actually going on. One other one that I would love to have somebody study is you know, how we define success of these agile transformations and what it is. This is something that people debate all the time, you know, the rates are very high, they go, you know, to different research that’s not necessarily relevant to the context of Agile transformations or whatever transformations in general. So, again, I think we’re echoing or saying the same thing, which is, how can we work together? How can we collaborate and break the silos between the two communities to help everybody?

Speaker: Christiaan Verwijs  37:00

Yeah, and Daniel actually has done really interesting research into agile transformations and what contributes to successful transformations or not. So there’s also research about that, which Daniel should absolutely talk about, right? But there is also research about that and it’s really cool.

Speaker: Daniel Russo  37:19

Absolutely. And then we are also very happy to share it at the link of his podcast if someone is interested also can clearly have a look at it. And also Christiaan is helping me to write a non-technical medium article in order to, you know, make it easily available for everyone. Anyone, if I may maybe conclude the research process?

Speaker: Christiaan Verwijs  37:52

We got distracted by a rabbit hole.

Speaker: Daniel Russo  37:54

Yeah. That’s great. That’s great. And yeah, right. So, basically, we collected this huge amount of data and then through you know, the statistical techniques that we said before, so, basically structural equation modeling, we were able to see how the data actually fitted to the model and, you know, good news we actually fit them out so apparently, you know, what we have observed in reality actually is also empirically validated by data on one hand, but also what we can do with actually regression modeling is not to have a much more nuanced understanding about individual relation about how strong each relation is are there any significant difference among for example groups and like so, for example, Christiaan mentioned about big small granger and so on, so forth I mean, we can have really a great understanding of the general phenomena, we are actually looking to and this kind of research approach is called mixed middle approach. So, you know, you have two middle so one constructivism one positivist right. So, basically where you’re inferring your phenomena and afterwards you are validating it and you know, and using this this mix approach really allows you to have actually quite a deep on one hand and once on another hand understanding of in this specific case, quantum effects.

Speaker: Miljan Bajic  39:41

So, that would be for instance, taking the like what I wrote here is, at the end what you wrote like you can have, let me see here, you can’t have one without the other. So when we talk about, do you guys know what I’m talking about here?

Speaker: Christiaan Verwijs  40:00

Yeah. So it’s the responsiveness and stakeholder concern, you mean both right. So we actually, in our model, and this is also something we found in the cases. But in the model, what we found is that it’s great if teams are responsive, so they can release frequently to stakeholders, but it doesn’t really add anything if they don’t, if the team is not also very focused on stakeholder needs, because I think in practice that looks like a team that’s delivering a lot but it has questionable value or is not what stakeholders are asking for. The other way around is also true. So if teams are very focused on stakeholder concern, but are never releasing anything, or very infrequently, which is kind of waterfall, if you think about it, then the effectiveness is also much lower. So stakeholder satisfaction, team morale are very low in those cases. And I think that’s a very interesting finding in our study that we actually were able to see that in the data very clearly.

Speaker: Miljan Bajic  40:56

And that’s why I was bringing that up. But to come back to Daniel’s point about this is where you could do more research, is that where you would say you could take phenomena like that maybe and then do that type of research or am I missing the point? I’m missing the point.

Speaker: Daniel Russo  41:16

No, no, no, no, no, no, no. I mean, I think the beautiful thing of research is, you know, there is never a wrong or right answer. Right. So again, it’s not about that. I think that everything can be investigated and for every question, you have an answer. Now, the best way, you know, to address on specific research question is, you know, a specific research design. Right? So I mean, I talked about this mix middle design, which again is clearly the most effortful and expensive one but you know, it’s also the most reliable one. But you know, what, I mean, but can be an actually there are a lot of research question that may be much more narrow, right, because I mean clearly, this paper has quite broad and quite huge implications for you know, where we retire with a theory for effective Scrum teams because, you know, it’s actually very broad and you cannot really investigate a very broad research question with very narrow techniques. And so, again, so to answer your question, it depends.

Speaker: Miljan Bajic  42:32

Yeah, that’s a very consultant like. 

Speaker: Daniel Russo  42:38

Good, maybe one day we should be going to consultancy one day. 

Speaker: Miljan Bajic  42:44

What about maybe just one more question for Daniel before we move on. Like, how did your community respond? Did you share? I’m assuming you shared this, how did they respond to this paper?

Speaker: Daniel Russo  42:57

To this specific paper? Well, almost in neither way because the paper per se is under review, which means that and this happens with every academic paper, right? Basically, you write your work, then you submit it to a peer review conference and then you know our colleagues of you are basically reviewing making suggestions, questions and so on and so forth. And only after we say yes, okay, it’s great. We publish it, when it’s published, when it’s there, and then basically you’re going to present it and so on so forth. But I’m actually very, very positive about the effect on this specific research also in the academic community. 

Speaker: Miljan Bajic  43:52

What is the process like? How long does it take for them to review? 

Speaker: Daniel Russo  43:56

One year more or less.

Speaker: Christiaan Verwijs  44:00

So we discovered it’s not very agile because it takes a long time, but that’s why we pre-published the paper because we felt it was very valuable already to share. And even if the if the peer review process yields a lot of things we need to change, which is a theoretical possibility somebody could be then that’s also transparent, right? Because so this process is also about seeing what feedback you get and I think it’s good be transparent about that too.

Speaker: Miljan Bajic  44:30

Yeah. And I mean, just like I think bringing more attention to this so we can do like we talked earlier more stuff like this, more collaborations like this, I think is gonna help everybody long. When you look at the other categories like team autonomy, continuous improvement, responsiveness, management support, team morale, stickler like, which ones do you think you know, which ones should we talk a little bit more about? Which ones kind of, well, maybe they’re all important But if we had to prioritize here and create a backlog?

Speaker: Christiaan Verwijs  45:09

Well, we already talked about stakeholder concern and about responsiveness so those are very important. Those are basically the core variables in the model, at least for me if I think about it, but we also have three really important hygiene factors that we identify in the model. So the first one is team autonomy. The second one is continuous improvement. And the third one is management support. And what we are basically say is, these three things need to be in place or they have to be done in place at the same time that teams are working on the stakeholder concern and responsiveness in order for them to be effective. And I think particularly interesting, they’re all interesting, but management support is not at all surprising because I think as practitioners, we know really well how important that is, as academics, we know how important that is. But what we actually found in the data is that management support is particularly important for Team autonomy. So that’s where they seem to be able to have the biggest influence at least in the data that we collected, which I think also makes sense, because team autonomy has to do with constraints with boundaries… 

Speaker: Miljan Bajic  46:15

Decision making rights. 

Speaker: Christiaan Verwijs  46:18

Rights, mandates that teams have to make decisions. And I think that management can really, really be a useful influencer there. So that that’s maybe one to emphasize, the management support. And I think Daniel, you wrote about that as well, of course in other your work right?

Speaker: Daniel Russo  46:33

Yeah, yeah. Yeah. I mean, so for example, we addressed like this model, which, you know, basically, is kind of a similar model, like Chris and myself wrote about successful agile transformations where absolutely mean top management support is a very, very important factor. And, you know, and maybe if I can spend more time with, with top management support, I’m not meaning you know, one specific management style, but it’s much more again related and fitted to the organization, first of all, right? So, if you have actually hierarchy, a very hierarchical organization like for example, the Italian Army, which you know, is the main case study of Agile success model, I mean, in that case, the management support basically mean, you know, I ensure that, you know, the whole structure actually follows the supports. Yeah, I mean, whatever the scrum teams are doing and generally speaking, I mean, Scrum process is put in place. In a more hierarchy or organization, it may be a much more kind of supportive style right? So in saying, okay, so, you well, first of all, it’s perfectly fine to fail, right? So and failure has not to be to blame. It’s about how can we support you software team to, you know, provide the best possible environment to you know, maximize your well-being and productivity.

Speaker: Miljan Bajic  48:35

That will be very interesting. What you just made me think of maybe this is also for Christiaan is  relationship between cognitive capacity of management and emotional capacity because those are really hard to measure. But there are a lot of studies and like, if we go back to maybe Lalou’s work, Fredrick Lalous on re-mentoring organizations or anything that, it depends what type of management and leadership you have, how big their ego is, how much you know their worldviews whatever you want to call it, but that has a huge impact on how they lead and how they support. So you could have hierarchy, but if the leaders are more focused on helping others from a perspective of their needs and wants, then they can work a little bit easier than that. Where if you have very egocentric person that loves command and control in the same environment, you’ll get different results just because of the mindset of the manager, right. So if you have a lot of managers which we have in organizations and is changing, that are very power hungry, let’s just say or ego driven and that’s something, I agree, like this management support is something that I see, so it’s definitely been something that I’ve observed over the years but it’s also something that you’re bringing up as a… what else from the management support, maybe they’re in real life or because I think this is in many different reasons, including Lalou, who spent 10 years or so and I don’t know the quality of his… 

Speaker: Christiaan Verwijs  50:32

I don’t know either. 

Speaker: Miljan Bajic  50:34

But it’s interesting that that keeps coming up. Leadership seems to be the ceiling, you know, the level, and I don’t know who was talking yesterday or maybe it was Yoganna Paulo, I spoke to him this week too and I think he was alluding to, like, you know, that the ceiling is the senior leadership in the organization.

Speaker: Christiaan Verwijs  50:59

Oh, Yoganna is a good person to talk to about this too, of course with management 3.0. I think in general, I can’t really speak to personality styles or ego how that influences teams, I can only speculate based on my own experiences there. But what we asked teams in the survey is, to what degree do you feel supported by management? To what degree do you feel that they understand why you work the way you work? And we can actually see that that has a big influence on their effectiveness. So I think it’s not even necessarily about management style or personality, it’s about whether or not that’s the kind of behavior that you’re exposing as a manager or someone in a management position, where you’re asking questions to the teams to ask what they need from you rather than telling them what to do. And there are so much evidence in other research that that’s important for highly autonomous teams doing complex work that it’s surprising in a way that sort of still not happening in many organizations. But that’s what I can say, based on the analysis. Sorry, Daniel, go ahead.

Speaker: Daniel Russo  52:09

No, no, sorry. And you know, and I think that also, what’s also important to mention is that a lot of companies are claiming to do to, to be agile and to do Scrum and you know, effectively, if you are looking more deeply into various processes, I mean, they are very little agile but you know, it’s very fancy to say so and you know, and if you’re not, you’re out of clamps so you are, you know, pretending to be something you’re not. And but I think that, you know, probably also this pandemic will change very fastly a lot of dynamics. So we see that T. Mattoni for example is extremely important for effective Scrum teams. And so, for example, I’m a big fan of self-determination theory, which is the combination of need for autonomy, relatedness and competence. And I’m actually also doing specific research on that specific aspect. And we see that it’s very, very strongly and positively related to job satisfaction, well-being and productivity. 

Speaker: Christiaan Verwijs  53:38

Absolutely. Yeah. And stress is much lower in those teams. Burnout, so mental health is also much better in teams that have high degrees of autonomy. But not too much. That sort of like there’s always an optimum, right. So it can also create stress if you have too much autonomy, but you don’t feel you have the skills to actually use that autonomy, correctly. We didn’t research that question but that’s also always good to mention that autonomy requires boundaries but they just need to be much broader than they are.

Speaker: Daniel Russo  54:09

[cross-talking 54:09] And you know, and I also think that the way that we will kind of auto negotiate our degree of autonomy and you know, and I mean, in Scrum teams, but I think that this also relates to possibly every knowledge worker possibly, right and also will define the way we will come out from this pandemic, and you know possibly also try to be a little bit happier and a little bit well off that is from our mental health perspective, yeah. So I’m very, I’m very optimistic. And

Speaker: Miljan Bajic  54:45

I am too. I think it is like the COVID and these crisis have forced us to rethink, have forced out of our comfort zone so it’ll be interesting. It’ll be interesting the impact. It’s Crazy, but it’s been an hour. What are some of the last things may be a message or to the audience that are listening? Anything you would like to say in conclusion?

Speaker: Christiaan Verwijs  55:14

Should I take the first one Daniel and you do the second one or the other way around? What would you prefer?

Speaker: Daniel Russo  55:21

Go ahead, please, yes.

Speaker: Christiaan Verwijs  55:23

So I can imagine if you’re listening to this podcast, you may be interested to learn more about the paper now that the paper itself is quite technical because it’s written for an academic audience, it has to be technical because other people have to be able to replicate our work. But we are working on a non-technical version of this, and it will probably be released next week on Monday. Also, for the other paper that Daniel mentioned, that will be in one of the coming weeks. So that’s one thing. And the second thing and that’s the final one for me is if teams are interested to see how they’re actually doing with Scrum, the survey that we created, the measurement instrument that we created, it’s actually available for free, you can try it at scrum teamsurvey.org and you can just use the whole tool for free, if you want to add a couple more teams to track them over time their subscription model because we somewhere need to make some money to generate, to fund all this work and to fund the product itself. But there is a way for teams to actually diagnose how they are doing and that’s also the core message of our paper; diagnose teams on these five factors and then support them in the areas where they’re lacking and make that a conversation between the teams and the rest of the organization. That would be my strong recommendation.

Speaker: Miljan Bajic  56:37

Great. And by the way, I’ll include all the links in the description below soon.

Speaker: Daniel Russo  56:45

Yes, so from my side, I am strongly recommending practitioners to look for some evidence, whatever they are using adopting any kind of approach because as we have seen a lot of things might be of common sense and in several cases, common sense, is our friend but you know, it might also be a false friend. And yeah, I mean, I think that we could save a lot of time and resources if we would have a much more critical reflection of whatever you’re doing. And my probably second recommendation and just about Christiaan said, so I also strongly encourage you to use the quantum survey because first of all, it’s an outstanding way to diagnose the way you’re doing your process. So it actually also helps you think very fairly concrete way also suggesting some very concrete recommendation. And also it helps, you know, science to advance. And also, because all the data, by the way, are totally anonymized and then really released in an open access form so that also our academics and other practitioners, if they want clearly can also use this data for their own research. And please let me stress, in a totally optimized way so absolutely no privacy issue and privacy concerns. So yeah.

Speaker: Miljan Bajic  58:46

That’s great. And thank you for doing that. I think that’s again, going back to the collaboration by willing to share this and being open to that, it invites collaboration. So thank you guys. I really appreciate what you’re doing. I think it’s great for… you know, it’s great for everybody I think. So thank you. And it was a pleasure speaking with you guys. I hope you like this unstructured just isn’t we’re sitting at the bar or something just chit-chatting and…

Speaker: Daniel Russo  59:22

Well, I’m having my coffee anyway. So yeah, so we are actually in a kind of remote bar. 

Speaker: Miljan Bajic  59:28

Exactly. Although it’s still early for me to have any alcoholic beverage otherwise….

Speaker: Christiaan Verwijs  59:36

Well, next time. Well, it was a pleasure for us to be here too and to have a place to talk about this work, and to share it with the community. And we definitely have a lot more questions that Daniel and I are pursuing. So hopefully there will be more publications and we’re working on that.

Speaker: Daniel Russo  59:53

And if you have any question, please feel free also to reach out and I’m saying it also very clearly to everyone who is listening. And also from my side thank you so much for having us. It was pretty fun also from my side yeah.