Oliver Filler, Program Quality Manager at Plan International, talks about his experiences in project management, rolling out program-wide improvements and how PMD Pro has helped provide a foundation for project management learning in the developing world.
Listen to Social Ex Episode 1, Oliver Filler of Plan International
Read full edited transcript of SocialEx Episode 2: Oliver Filler
George Miller: Hello and welcome to this podcast from Humentum. My name is George Miller and I recently had the opportunity to talk to Oliver Filler, the project management lead at Plan International, about his experience of rolling out PMD Pro training. Why did they choose PMD Pro? How did they prepare to introduce it and, critically, how have people found it and what impacts has it had? But before we got onto these big questions I began by asking Oliver about his own background in anthropology. What had got him interested in that?
Oliver Filler: I was just fascinated by people in society and why the world is the way it is, and looking at social constructs and behaviors, and why there is poverty: why are some individuals completely marginalized from society when others seem to have all the luck? I spent a lot of time in India during my university degree and my masters looking at the caste system and looking at the discrimination of Dalits, who are the untouchables within the caste system. That was around the time of the 2004 tsunami, so I thought I’d look into how that has an impact on aid and the distribution of aid. And so I looked at discrimination against Dalits as part of the tsunami response and made a lot of contacts that way with NGOs. It embedded my passion for development and humanitarian work and led into my first job with Save the Children.
George Miller: Was there an alternative course that you could have taken in which you became an academic? It sounds like you were thinking about the big structural issues as well as what was happening on the ground. Is that the path not taken?
Oliver Filler: Yeah, originally I was going to read English literature so I took a massive deviation from the path. People always ask me, do you have these plans in life and set up these goals? My opinion is that you miss opportunities that way and life :is too short to plan it out for 60 years only to realize it wasn’t what you wanted to do. So I have always veered towards the things that interest me, veered towards the things that I care passionately about and think are either great injustices or think that I can contribute something with my skill-set, so I try and maintain a flexibility, which is actually quite a useful skill-set for project management as well.
George Miller:So you began your career in this sector with Save the Children. What were you doing back then?
Oliver Filler:I was in Kashmir looking after education projects and some elements of monitoring and evaluation as well. It was post-conflict, fragile-state work. It pushed me down the path of humanitarian aid, which is most of my background. So fast-changing contexts, insecurity, but meeting huge needs. You’re dealing with very vulnerable individuals, so it was hard work but something I cared very much about. The entry into my career really was through Save the Children. You can deal with the insecurity, you can deal with the hard work, you can deal with the changing contexts and the frustrations because you’re dealing with very vulnerable individuals and you’re trying to support them as best you can. So as long as you’re meeting needs, it’s maybe not easy to forget about it, but you can accept some of the risks that you’re dealing with.
George Miller:Looking at it from the outside, I guess a lot of the time you must be so much in the moment when you’re in the field that it’s difficult to take a step back and think: is this the best way to do it? Are there other ways to do it? To keep an eye on the big picture because you have immediate human needs that you’re meeting.
Oliver Filler: Absolutely, it’s the boiled frog syndrome where you don’t quite realize what’s going on around you because you’re in the middle of it. It’s a real risk with the work that we do in the humanitarian sector that you don’t have time to step back and look at the bigger picture. That is where project management comes in, because the role of a project manager is to provide that oversight and that space. While your team are running around trying to do the best they can in the middle of it, you need someone to be able to step back and look at the bigger picture and organize things. So that was where I first saw the need for that and probably a humanitarian context is the starkest reality to show that.
George Miller: We’re talking about a decade ago, a little bit more than that? What was the status of project management back then in the humanitarian sector?
Oliver Filler: It wasn’t just the humanitarian sector. I’d say the [whole] sector has professionalized a lot. I was coming in really at the cusp of change within the sector, so increasingly it was being professionalized; we were looking at project and programme management; we were looking at monitoring and evaluation and accountability, accountability to beneficiaries, making sure that communities are able to give their opinions and feedback and are at the heart of the projects that are designed. It was a fascinating point in the sector’s history because there was massive change and with massive change you have huge challenges and obstacles. Ultimately we were looking at behaviour change, you know, these new ideas coming in, contradicting the way that the sector had been working previously.
George Miller: And what was pushing that change?
Oliver Filler: I think it was just the flow of the sector. What generally happens, especially in the humanitarian context, is that a huge crisis will happen, the sector will respond and then we’ll learn. We’re lucky sometimes if we just pick up elements of the learning and embed it in the next response. But I think we just hit the cusp of this massive change point where enough had been learned in order to realize that things had to dramatically change. That was reflected in the sector, in the agencies that work within the sector. But I think it was also mirrored by funders, and where funders change their mind and funders start to prioritize things, the sector tends to follow as well. So I think things came together at the right time. It’s not to say that we are perfect where we are now, that we’ve learned everything and we’re doing everything marvelously. That’s certainly not the case, but I think it was a point where we had started to really push for doing things professionally, openly and transparently and that has now helped us improve the sector generally.
George Miller: So you came along on the cusp of change.
Oliver Filler: I think so. I came in at a time where I was able to benefit from the changes that were being made. I’ve been lucky enough to learn as I go and be introduced to new training and new ways of working. But I was also at the heart for Save the Children of pushing a new monitoring and evaluation approach in responses certainly.
George Miller: Can you tell me a bit more about that in practical terms, that new approach and your role in it?
Oliver Filler: At Save the Children we realized that emergencies were being responded to in a knee-jerk way and there was the rationale – which is easy to understand, how we got into this mindset – which is that we must be doing it right because we’re doing it. This is the same with project management: ‘I know how to manage a project because I’m a project manager’. It’s a trap that we always fall into and there was a realization, not just for Save the Children but for the wider sector, that actually we needed to professionalize; we needed to look at our processes; we needed to push for openness and transparency. One of the things that Save the Children pushed earlier than most international organizations, I would say, was monitoring, evaluation, accountability and learning, especially in humanitarian responses. Making sure we’re able to track what we’re doing, tracking our outputs, tracking our outcomes. What are we actually delivering, what quality are we delivering for people? We may be distributing things, but are they the things that people want? Save the Children really drove that early on in the sector. Likewise, running things like real-time evaluations: a few months into a humanitarian response, having an opportunity to step back and ask, is this the right response? In the middle of a manic humanitarian response, are we actually doing the right thing or have we fallen into the trap of just doing the same thing over and over? So real-time evaluations were really pushed by Save the Children early on and that sparked a big change in the sector. Then embedding monitoring and evaluation within that response and making sure that it’s there at the beginning rather than brought in as an afterthought. So instead of ‘we’ve been responding for a couple of months, should we bring in M&E?’, M&E goes in as part of the first phase response.
George Miller: I guess that any cultural shift meets with resistance or inertia or skepticism. Was that something that you remember encountering back then?
Oliver Filler: Yes, the difficulty is that you’re trying to embed a new way of working in a system that doesn’t know how it’s supposed to embed it, and how it’s supposed to work. So you end up with people line-managing M&E specialists who don’t actually know how M&E should work, and that’s just one example. There is a danger that you dilute the impact of it because you’re not utilizing it in the right way (you can see the parallels with project management here as well). You’re looking at a critical mass of people: do we have enough people speaking the same language, with the same understanding, the same competencies to push it over the cusp of the blockage of the old way of doing things? That takes time because it’s behavior change. And in an organization that can be very difficult, especially when you know you’ve been doing the same thing for so many years and change is tough and it requires hard work. So yes, it did take time. The critical thing is to have senior level buy-in married with an understanding of the people who are actually implementing projects as to why it is important. But it’s still challenging, it’s still very tough.
George Miller: Like the initiative Oliver has just described, PMD Pro also dates back about a decade. I ask Oliver to fill in some background for me.
Oliver Filler: PMD Pro was the brainchild, as I understand it, of a number of individuals from Oxfam and others, who identified a need for a project management standard. The corporate world has had things like PRINCE2 and PMI for decades. Despite the fact that in the sector our main business is implementing projects, we didn’t really have anything that we could call on that could support us in that. PRINCE2 / PMI is not particularly relevant for a lot of the work that we do. They’re dealing with very different things. We’re dealing with individuals, with behaviour change at community level, we’re dealing with very complex contexts, so we needed a framework that could be flexible, a framework that could be foundational but then could be built on and expanded and advanced. And so those individuals created the PMD Pro manual and a separate organization called PM for NGOs, which hold the rights and the materials of PDM Pro and now the programme guide and other elements as well.
George Miller: Can you tell me where you were when you first encountered PMD Pro and what your initial experience of it was like?
Oliver Filler: We were at a really interesting point in Plan International. We had stripped back our programming approach – we’d realized that it had become fairly cumbersome, that we had a number of reporting requirements on our country offices and that the organization really had grown kind of organically. We’re a sponsorship organization at our core and yet our grants portfolio had been expanding dramatically to the point where it was 50:50, and that had happened organically – it hadn’t been well controlled. We hadn’t looked at things like procedures for managing projects, we hadn’t looked at the competencies that we would expect project managers to have. This was about three years ago. We stripped everything back and really decided to focus on the foundation. We asked, where do we need everyone to be in terms of the level of project management and programme management? What’s important to focus on and what is actually peripheral and more extractive? So we developed a new project cycle, we developed a new policy which was called the programme quality policy, which has since been superseded, but the programme quality policy outlined what we think quality means, especially within our programming and projects. Out of that came our procedures, three cycles: the country strategy cycle, the annual cycle (a slice of the country strategy each year) and the project cycle, which is how we manage our projects to achieve our strategy. We realized that it’s no good saying ‘these are the procedures’, ‘these are the minimum standards that we expect people to achieve when you’re managing projects’ when we’re not actually giving them the skills and the support to achieve those standards. So we looked around for an option that would help us deal with that issue and PMD Pro jumped out immediately.
George Miller: When planning their pilot, Plan talked to other organizations about their experience with PMD Pro. The message came back that it was important not to link it too quickly to compliance or to wrap it up with other global initiatives. So instead, Plan’s emphasis would be on providing options.
Oliver Filler: We went to our regional offices and pitched it to them and said, ‘you tell us, do you want to do a training of trainers? Do you want to just do level-one PMD Pro training to a range of staff members and see how you go? Do you want to do a level-one training, give it a few months to see how you go, and then do a TOT?’ We talked them through the options and we developed specific rollout plans for them. The first phase of the rollout was geared towards getting trainers into regions, getting enough staff involved and creating a buzz around PMD Pro, and we were perfectly willing to accept that after a year if people didn’t see the value in it, if people didn’t really want to take it up and roll it out, then we would accept that it was the wrong thing to do, but at least we tried. So the first phase was viewed as a pilot and we developed a monitoring evaluation plan around it. We really wanted to learn. We really wanted to make sure that as we went, we identified learning, embedded it within the next training and that we could document it and prove that it was working. So we developed a monitoring evaluation framework. We have followed a number of case studies through the pilot; we’ve tracked things like surveys, so feedback around the training; we’ve been tracking things like the project management KPI, which is a rough indicator of the quality of projects and we’ve been doing semi-structured interviews, FGDs, with participants. So we really try to learn as we go and to embed that learning.
George Miller: A case study: Plan’s first experience delivering PMD Pro training was in East Africa.
Oliver Filler: RESA, our East Africa office, were our guinea pigs and went first as part of the pilot. We discussed all of the options that were available to them around online courses, face-to-face courses, TOT (training of trainers) vs. just a level-one training, and worked with them to come up with a solution that would work for them. The solution they came up with was actually to focus on the level-one training and deliver a five-day face-to-face training in PMD Pro rather than focus on a TOT. The rationale behind it being that they wanted to see how they went with the level one first and then have the training of trainers as an option that they could look at further down the line. So we worked with them to select participants, we came up with a list of criteria for participants and again it was optional. So [we said] we can guide you around who would be appropriate and who wouldn’t. One of the key things that came out from our conversations with the other agencies which have rolled out PMD Pro was that it is best when it is multi-functional. So if we just have a room full of project managers, that’s great for the project managers, but projects are multi-functional by their very nature. We have to work with a number of teams and a number of specialisms, so everyone has some form of responsibility within a project, so it’s important to have those functions in the training as well. So we selected a number of individuals from the East Africa region, across all of the country offices, and delivered the training. We went, in my opinion, a bit too heavy on the monitoring side of things. We were clear that we needed to find an approach that works for Plan, based on the feedback from other agencies, which had all done slightly different things. So we wanted to make sure we had certain decision gates built in as we rolled out PMD Pro that we could reflect on and adapt and change as we went. We were clear that we wanted to track individuals through the course. We were clear that we really wanted to see what the impact was. So not just, ‘have people done the training? ‘Yes, great.’ ‘Have they passed the exam?’ ‘Yes, great. Job done.’ The critical thing for us was the application, so ‘Are they actually applying their knowledge? Are they applying what they’ve learned in their projects?’ And then subsequently ‘Are we seeing an improvement in project quality?’ So we selected case studies, individuals that we could follow through the training. We interviewed them before the training, immediately after the training and then again six months after and asked different questions around what the expectations were and what they liked about the training afterwards and then what their plans to embed things were. Generally, the feedback was very good. It was a fairly new approach to Plan. It’s a really engaging training, really interesting training. And so a lot of the feedback from participants was, ‘Yeah, but I’ve been managing projects for 20 years, so this will be a great way to refresh my skills, but ultimately I’m good.’ And then they would come out of the training and say, ‘I had no idea I had to do risk management.’ So that was really pleasing to hear, that it was having a real impact on individuals and they were really buying into it.
George Miller: I asked Oliver if he could try to encapsulate the shift in mindset that a successful training in PMD Pro can bring about.
Oliver Filler: The consistent feedback about PMD Pro is that it marries everything together around project management. A lot of people will have had a training on log frames. A lot of people will have had a training on how to phase a budget or how to do a risk matrix. But there are very few trainings out there that actually pull everything together from a project management perspective and say, this is how they work together. These are the tools that you use in the disciplines and the principles that you use as a project manager in order to manage projects. The overwhelming feedback has been it has been so useful to step back and look at the bigger picture around how you manage a project. We’ve had multi-functional trainings, so we’ve got project managers in with grants managers, finance managers, M&E managers or coordinators etc. And the really useful thing with that is that often – let’s take M&E as an example – they know where they sit in the day-to-day, but they don’t necessarily know where they should sit throughout the project cycle. Likewise for logistics and procurement. Often they’ll be absolutely hammered during a project because people expect them to deliver immediately. But PMD Pro preaches that actually at the beginning of the project, as you’re designing the project, the functions should be involved in that because they’re expected to deliver the project. So it’s a message that resonates with those functions as well and it really gives an evidence base as well to push back and say, ‘we need to be involved at proposal development stage, we need to be involved as management oversight because we have critical pieces to play in project management’.
George Miller: Oliver was pleasantly surprised that buy-in to PMD Pro didn’t just come from those who’d been on the course. It extended well beyond.
Oliver Filler: We held a learning event a few months ago to look back at the full roll-out of the project. We brought in people from country offices and regional offices who had been central to that roll-out and tried to look at what went well, what didn’t go well etc. As part of that, we interviewed our directors of sub-regions and held focus group discussions with country directors. Now they’re very senior within the organization and would be fairly far removed from the actual implementation of PMD Pro, but they were able to talk about it in a fair amount of detail and not because I had spoken to them before about it. It was coming from their country offices, from their staff, and they had been told how much value they had seen in the roll-out of PMD Pro. So that was hoped for and somewhat expected, but the speed at which we got that buy-in took me by surprise.
George Miller: To get a sense of the impact and effectiveness of PMD Pro, Oliver and his colleagues have been looking at three key performance indicators.
Oliver Filler: One is looking at budget vs. actual: what did we say we were going to spend and what have we actually spent on the project? Second, the achievement of outcomes: are we on track to achieve the outcomes that we stated in our proposal? And third, project risk: what is the risk rating of the project? We ask country offices every quarter to rate every single project either red, amber or green based on those three proxies. We’ve seen an improvement in the KPIs. I was fairly careful around the communication of the expectation of improvement in KPIs, because there is a real risk that the KPI could have gone down because what we’re doing is equipping people with the skills to identify issues in their projects and deal with them. So initially they’re able to identify issues that they weren’t previously able to identify before the PMD Pro training. So there was a risk that KPIs could go down. Actually what’s happened is the KPIs have gone up, so the quality of our projects has gone up. But more importantly to me is a little section at the end of the reporting tool which asks for an explanation as to why they’ve rated the project as they have, specifically ‘Why have you rated it amber or green?’ And an explanation as to what they’re planning to do about it. The improvement in the articulation of what the issue is and what they’re planning to do has been amazing. It’s been really good, and that is the impact of PMD Pro, the ability of individuals and project managers and project teams to say, ‘I know what the problem is. We’ve done this and we should have done that. Therefore this is what we’re going to do about it.’ So that’s been the real impact.
George Miller: So what is its status today would you say within the industry?
Oliver Filler: Increasingly it’s becoming an industry standard. A number of very large INGOs are utilizing PMD Pro and can prove that it is having an impact on the quality of their projects, so increasingly rather than it being seen as a kind of unique selling point of agencies, it’s actually expected. Of course, we should be investing in training our programme managers in a certifiable standard rather than saying, ‘hey, look, we’re better than everyone else because we have PMD Pro-trained staff.’ That’s no longer the case. Donors would expect us to have PMD Pro or equivalent certified project manager.
George Miller: Plan International currently has around 60 PMD Pro trainers internationally and over 1000 staff members have been through the programme. The organization’s going for critical mass. The initial phase of training was centrally funded, but there’s since been a switch to local funding to ensure long-term sustainability. The buy-in from national organizations has been strong in response to grassroots enthusiasm from country offices, and as Plan International continues to evolve, Oliver told me, PMD Pro will play a key role.
Oliver Filler: We’ve embarked on a really ambitious strategy as Plan International: we’re really changing our focus from a pure child rights organization to a focus on equality for girls. And so we’re trying to work out how we can best apply PMD Pro through the changes that we’re making as an organization through that new global strategy. And that’s really exciting. We are increasingly getting demands for follow-up trainings and further detailed project management trainings, which could be quite interesting. We’re also looking at the moment around the application element: PMD Pro is very good at delivering content and knowledge to individuals. One thing that we would like to strengthen is participants’ ability to then take what they’ve learned and explore the application of it. So, making sure that they can go back to their offices and apply what they’ve learned as seamlessly and easily as they can. So we’re looking at a pilot of a project management game, which is ultimately providing a safe sandbox environment for people to practise applying some of the tools before they get back to their country office. It’s a project management simulation developed around a humanitarian setting, so it should be a fairly familiar context. It’s a pilot at the moment but we’re constantly trying to find ways to improve the training. One thing that we’ve just started piloting is PMD Pro Flex, which is an online, gamified version of PMD Pro. The initial roll-out of PMD Pro was exclusively face to face, which was necessary. We needed to embed the training, we needed to create trainers. But now we’re moving towards a global rollout of PMD Pro, the better value for money is through online training, which means that people don’t have to drop everything for five days. And if you consider flight times etc., the impact on a country office can be huge. We’re trying to find ways to minimize the impact that a five-day training could initially have on a country office. Flex is being piloted as we speak. We’re halfway through our first ever Flex, which has been adapted to Plan’s context. The feedback has been very good. We’re able to train a lot more people for the same amount of money as we would a face-to-face training. We’ve got about 70 people on each cohort and we can run four cohorts for the same price as one face-to-face training, which would cover about 30 people. So it offers much better value for money but it also means that people have the flexibility to do their day jobs and then do the PMD Pro training as and when they have free time, which is critical for project managers who are on the front line. It’s very difficult to find one hour let alone five days to focus on training, so we’re really excited by PMD Pro Flex. We think it will add value. We’re not saying we will only do PMD Pro Flex. Again, we’re trying to push the message of options, working with country offices to find the best option for them.
George Miller: I was talking to Oliver Filler of Plan International. My thanks to him and to you for listening. Until next time, goodbye.