196 – Behind the Scenes of 10 Years of Industry Research with Pam Kappelides & Allison Russell – Part 1

January 8, 2026

Episode #196: Behind the Scenes of 10 Years of Industry Research with Pam Kappelides & Allison Russell – Part 1 

To mark 10 years of the Volunteer Management Progress Report (VMPR), Tobi Johnson hands the mic to nonprofit scholar Allison Russell for a behind-the-scenes conversation about what it really takes to produce a decade-long, global research study, without grant funding, institutional backing, or a full research team. 

Joined by longtime research partner Pam Kappelides, this episode explores why volunteer engagement research matters more than ever, how practice-informed research gets done, and what practitioners can learn (and apply) from the VMPR process. 

Industry Research – Episode Highlights

• [00:55] – History of the Volunteer Management Progress Report 
• [13:54] – Importance of Research in Volunteer Engagement 
• [17:25] – Challenges and Innovations in Volunteerism 
• [31:59] – The Origin Story of the Survey 
• [32:43] – The Generosity of the Volunteer Community 
• [33:27] – Planning and Executing the Survey 
• [34:47] – Setting Goals and Philosophical Approaches 
• [36:23] – Steps in Survey Design and Analysis 
• [37:10] – Challenges and Practical Advice 
• [46:02] – Qualitative Analysis and Coding 
• [50:16] – Impact and Reflections 

Helpful Links

• Volunteer Management Progress Report  
• VolunteerPro Impact Lab  
• Engage Journal  
• Volunteer Nation Episode #101: Tap the Power of Psychological Contracts with Pam Kappelides  
• Volunteer Nation Episode #058: Innovative Models in Volunteerism Research with Lucas Meijs 
• Pam Kappelides and Tobi Johnson, A Heavy Load: Challenges and Current Practices for Volunteer Managers in the USA, Australia, and Canada, Journal of Nonprofit & Public Sector Marketing, 32(1)  
• Find Allison on LinkedIn  
• Allison’s UT Dallas Profile 
• Pam’s LaTrobe Profile  
• Find Pam on LinkedIn  

Contact Us

Have questions or suggestions for the show? Email us at wecare@volpro.net.

Rate, Review, & Follow Us on Apple Podcasts

If you love the content Tobi shares on the Volunteer Nation podcast, consider rating and reviewing the show! This helps us reach more people – and help more good causes just like yours – successfully engage enthusiastic, dedicated volunteers with less stress and more joy.

Click here, scroll to the bottom, tap to rate with five stars. Then, select “Write a Review” and let us know what you loved most about this episode!

Also, if you haven’t done so already, follow the podcast so you don’t miss a thing. Follow now!

Subscribe to ProNews: Our Weekly Resource Roundup

If you’d like to stay up to date on all new podcasts, blogs, freebies, and deals posted on our Tobi Johnson & Associates and VolunteerPro websites, subscribe to our weekly ProNews newsletter.

Every Wednesday, we’ll send you a digest of our freshest content, plus a bonus! Once you confirm your subscription, you’ll get our [Free eBook] The New Volunteer Manager: The First 90 Days.

Episode #196 Transcript: Behind the Scenes of 10 Years of Industry Research with Pam Kappelides & Allison Russell – Part 1 

Tobi: Hey, welcome to the Volunteer Nation Podcast, bringing you practical tips and big ideas on how to build, grow, and scale volunteer talent. I’m your host, Tobi Johnson, and if you rely on volunteers to fuel your charity cause membership or movement, I made this podcast just for you. Welcome everybody to another episode of the Volunteer Nation Podcast.

I’m your host, Tobi Johnson, and today we’re going to do something completely different. I’m gonna hand over my interview microphone to Alison Russell, who is the assistant professor of Public and Nonprofit Management at the University of Texas at Dallas. She is also the editor of the Research to Practice section of the Engaged Journal of which this interview will be shared there.

So Alison reached out and I had been been speaking to Rob Jackson in the UK about the volunteer management progress report and the fact that we were going to. Take a pause this year. This last year was our 10th year. We had been creating this fantastic industry research for a decade, which is probably, as far as I know, the longest longitudinal study in volunteer management that I know of.

Maybe there’s something out there I don’t know about, but we’ve been doing it for a really long time and it was time to take a, a little bit of a pause. And Rob said, well, what? Why not talk about the whole history of it and how you put it together and what your insights? And Allison and I got together and she was gonna write an article for the Engaged Journal, and I said, well, let’s do a podcast too.

And, and if we’re gonna do a podcast, then we need to invite Pam Capaldi, my research partner in crime, who’s helped out with several of the additions of the volunteer management progress report. So we invited Pam along. So we are actually like Global Pam’s up early in Melbourne, Australia. We’re up late in the.

West coast and east coast of the US So we’re all around the world and I’m just, I couldn’t be more pleased to have you both here to talk about research and talk about why it’s so important and maybe to inspire some of our listeners to conduct their own research. So welcome to the podcast. Thank you.

Thank you so much for

Allison: having me. Yeah.

Tobi: Before we get going though, I wanna introduce Allison and Pam in case you don’t know them. And if you’re interested, I’ll also link to a previous interview I had with Pam on psychological contract theory. She wrote the meta study on psychological contracts, and if you wanna kind of get the overview on how it, how it all goes together, listen to that episode, but also read hers.

Paper on, on the topic because it really gives you the lowdown on the work that’s been done to date. But Allison Russell, as I said, is assistant professor of public and nonprofit management at the University of Texas at Dallas. I live in Knoxville, Tennessee, and my husband teaches at ut. So that’s the other ut exactly a little.

So Allison’s research and teaching focus on the nonprofit and voluntary sector, including volunteerism and volunteer management, nonprofit ethics, cross sector partnerships and organizational responses to crisis and change. Allison has published papers in a number of academic journals, including nonprofit management and leadership, nonprofit and Voluntary Sector Quarterly, which is a big one, and Volunt task.

These are big journals. Guys in our sector, in case you don’t know and co-authored the book, ethics for Social Impact. Which I am a fan of ethics in our field, as you all know. If you’ve listened to me for any amount of time in a support of fostering stronger connections between academics and practitioners, or as we like to call academics and translating research findings into practice, she currently serves as the editor of the Research to Practice section of the Engaged Journal.

International practitioner focused journal for leaders of volunteer engagement and we will also be posting a link to the engaged journal if you’re interested in, in subscribing. It is very reasonably priced. It is worth every penny. You get so much insight from that journal. I, I’m a subscriber, have been for a few years.

So it’s great. It’s a great resource for us. She also serves as a visiting research at the Gradle Institute of Charity at New College Oxford. Allison received her PhD in social welfare from the University of Pennsylvania, MPA from University of North Carolina, Wilmington and BA in Latin American and Latino studies from Fordham University.

So welcome, Allison.

Allison: Thank you so much, Tobi. I’m so excited to be here. It’s. Is really an honor. I’ve been familiar with your work for a long time, and so it’s really awesome to be here with you.

Tobi: Oh, thank you. Thank you. And Pam, who I’ve known. Geez, Pam. How long have I known you? I knew you when your daughters were teens and tweens.

Oh, we’re in a twenties now, so I think went two years to do a lot. Yeah. Yeah. And stayed at your place with your lovely husband and your two daughters when they were teens and tweens back in the day when I did my tour of Australia. So awesome. Absolutely. So awesome.

Pam: Well, we’re in some of our, so you call me a little bit jealous.

It’s nice and warm here.

Tobi: Well, you know what I am going to, when this airs, I will probably be in New Zealand. I’m, I’m doing a workshop in Wellington. Nice. Yeah, so I think I’ll be enjoying that summer in the southern. Down under in the Southern Hemisphere. But let me introduce Pam. She’s the senior lecturer and student placement coordinator at Latrobe University.

She, her research is focused on volunteer management, sport participation, and inclusion of minority groups such as females, CALD, and disability. And the impact of sport participation on development in sport, health and recreation. She specializes in qualitative research, methodology and evaluations. That’s a big, that’s.

Lots of folks are numbers people. Pam is a, is a person who loves to do that interviewing, that qualitative research, and she’s really good at it. In addition, she’s been a lead researcher on large complex projects. She’s been a practitioner in the sport, health and recreation industry prior to an academic career.

So she’s a person that transitioned from the practical to academic, or I would, I would say practical. Academic can be practical and has a wealth of knowledge in the sector. She has conducted quality research in the area of sport, health and recreation, and has worked with organizations such as the Department of Premier and Cabin in Victoria Vic Health, YMCA, special Olympics Master Sport, and.

On and on. Also, cricket. She’s big into sports. She has recently developed a website on inclusion in the sport, health and recreation sector. With support from the Victorian Government’s Office of Sport and Recreation. I know disability and access is very close to your heart. Pam and she’s affiliated with the Center for Sport and Social Impact and Living with Disability Research Center at Latrobe.

Welcome, Pam. Thanks. Sorry, I think in your as well. So let’s get started. And I wanted to read both of yours full biographies because I are, are full bios because I really wanted our audience to understand that you both come from sort of. Academic and prac practitioner spaces. And you kind of evolved into the academic space that you have ground on the ground experience.

There’s not a lot of, it’s usually people are one or the other. There’s a few other per academics that I can think of off the top of my head, but it’s really an interesting thing to think about, especially when we’re designing research to, to make sure that it is practical for. The folks that actually need to use it in the field, and that can improve our impact as an organization.

So it’s really exciting. And I also think it kind of inspires people who think in the back of their mind, ah, maybe I should get my PhD or maybe I should get my master’s degree. Right? And I would say, yes, you should. If you wanna do the work. It’s hard work. I, I, I never did it. I have a master’s degree and that’s where I stopped.

That doesn’t mean we can’t do research if we want to. That’s really how this, I kind of explained how this podcast episode. Kind of concept came about. I’ve introduced our guests, but I wanna ask a few questions before I hand the mic over to Allison. And the first one is just tell us, I like to talk to talk about our guests, talk with them about their birth story when it comes to volunteerism and community engagement.

Pam, let’s start with you. How did you get involved in volunteerism in the first place?

Pam: Oh, wow. As a volunteer. I was always, yeah, as a volunteerer, we’re a family that has always volunteered. Volunteering is very big in Australia, particularly in the sport and recreation sector. You wouldn’t function, our sports sector wouldn’t function without volunteers, but sorry that my parents were migrated from Cyprus to Australia in the last fifties.

So. They came to a country that they helped out, they built the country, they volunteered for their community, they volunteered for of their challenge. So it was sort of a, just the thing that we did volunteering was just part of my DNAI. So it started off that way. My earliest memory, the volunteering was always like a 12 or 13 sort of a memory.

At school. School where we had the kindergarten kids, say kids who were pre primary school, 3-year-old and 4-year-old, we would go into the kindergarten and take them for walks or them, and that was after school. They just asked for volunteers to go and help out, and it was mainly easily arrived. So for.

Them so they can learn the English language because we were already in primary school, so that was probably my first volunteering when they still volunteer on boards, volunteering, thought paths college. Do all that sort of stuff and, and looking forward to our time. Do a lot more volunteering, sort of going full circle.

Yeah,

Tobi: absolutely. I, I feel you sister. I’m there too.

Allison: Allison, how did you get into volunteering way back when? I think it’s kind of the universal story of people who study volunteering and who work as volunteer engagement leaders is that there was that initial volunteer. Volunteering you did as a volunteer, right, right.

So just like Pam said, you just kind of grow up in an environment where service is part of what you do, whether it’s through school or church. It’s just the idea of giving back to your community and helping other people. It’s just something that I kind of grew up with hearing from my parents. And my mom was very involved as a, a volunteer in our church in different ways.

And so I think that was kind of modeled for me from a young age, but I honestly never really thought about. Volunteering beyond just that active of service until I was doing my Master’s in public administration. When I thought that I was going to become a, a planner or work in local government. And then what happened was I ended up having Jeff Ney as my mentor at UNC Wilmington.

Wow. So he basically sat across his desk from me in the first week of being in this master’s program, and he kind of looked me in the eye and was like, have you ever thought about getting a PhD and going the research route with this? And I had honestly not considered it, but. That was sort of the moment that planted the seed in my head and from working with him and then just working with countless other amazing scholars and also PR, academics, I kind of got into more of the, the research side of volunteerism and volunteer management and what that looks like, and so I kind of was very blessed and fortunate to.

Stand on the shoulders of giants, if you will, really early on in the field. And, and that sort of opened my eyes to all of these other things that people were doing to kind of, like you said earlier, Tobi, like link the practice to the research and be able to kind of make these studies and these projects that would actually help people who are trying to do this work in, in the real world.

Tobi: Yeah. Yeah. And for those of you who don’t know, the late Jeff Ney, he passed away I think two years ago, maybe longer. Time, time. Time is like, I can’t figure out time lately. Yeah, time does fly. Yeah. He was a leader in a nova, which is an association of. Practitioners in our space, and that’s where Pam, Pam and I met at the A Nova, an A Nova conference.

I think it was in Indianapolis, was the first one we met at Pam. We knew of, and I had met, met him also at the national summit several years ago. And Lucas Mays, we’ve had on the, the pod, so I should link to our interview with Lucas Mays, who worked very closely with Jeff on a lot of stuff. Anyway, so the, the circles are small, right?

They, they’re so interconnected. Let’s talk about. Before I hand the mic over to you, Allison, let’s talk about. Why research around volunteer engagement is so important right now, in particular?

Allison: I think there’s a few reasons. From my perspective, I think it’s always been important. Obviously we all think it’s important, but I think right now it’s especially important because we’re in this period of.

Of real flux and change, particularly in the, in the us in the nonprofit sector, there’s been a lot of dramatic shifts around policy and government funding. And before that there was COVID obviously, and the, the post COVID and figuring out. What does volunteerism even look like in this new kind of post COVID moment?

COVID obviously kind of gave us this surge in virtual volunteering and online volunteering in, in ways that we hadn’t seen before, and that’s been really interesting and really exciting in a lot of ways. But I think we’re at this moment where so many things are changing and people who are working in nonprofits and even in public organizations.

Engaging volunteers have to really be thinking about what does that look like in light of all these changes? And particularly if funding is being cut or constrained, you know, we may need to lean on volunteers in new and different ways. And certainly we wanna make sure that we’re galvanizing and keeping these, these our current volunteers engaged.

Right. And sticking with us even through tough times. And I think that is one of the reasons, is this just moment of change that we’re in. And I think another reason for me personally and why I see research to practice at EN at the Engage Journal and and these types of initiatives is so important. Your work with the Volunteer management Progress report, these spaces that link research to practice right, is because also in the United States, we are kind of going through this moment where people are asking the question about what is the value of academic research and what is the value of the university?

And for me, I think like one of the ways I try to answer that question and communicate that. Um, is by talking about the ways in which I try to design studies that are looking towards what are these practical implications and how can these findings be used for organizations to make our communities better, to make these organizations more efficient, more engaged, more responsive to what the community really needs.

And so for me, part of this value proposition of what is. Why is academia and academic research important? Rests with this need to be creating research that is responding to real world challenges. And so for me, this is a big reason why research in this space is really important right now.

Tobi: Hmm. I can’t, I could not agree more.

This thinking around innovation and, and evolution. We have a context that’s that’s really evolving. We need innovation to, to rise up and meet the evolution in society, evolutions in society. This idea of being effective stewards of our, the resources we’re given. I also think there’s an area of.

Maintaining the public and funder trust in nonprofits, especially here in the US because there’s, there’s a fair amount of narrative around whether or not nonprofits are valuable, whether or not they’re trustworthy, whether or not they add. They are a value add to a community. And so I think sometimes, not only being more effective with our work, but being able to have a rationale behind our work and have it be evidence-based, I think is really essential Right now.

You really can’t get away with not having it. Pam, what do you think about research in this moment? Why is it so immor important in our sector?

Pam: I think what Allison was talking about, if up here in Australia and, and probably. Following up with, what else say policy change is really impacting the sector and it’s in the sector where they’re using it and working with volunteers that come to academics to say, we need evidence.

We don’t have the evidence of why volunteers are important. What is the value of volunteers, why we need to expand. Use our volunteers in different sectors. Well, they can’t go and, and I guess only to government that they need support and they need policy and support to make sure that volunteers are supporting and engaged.

Um, as I said earlier, Australia function without its volunteers. We have country fire authorities. Just the other day we were all getting about getting Bush fire safety.

That’s our impactfully. So, you know, they, they impact on our lives as well as volunteers. And what we’re finding is those sector don’t have research to back up while so important to get to get.

So I think the research is really important together to allow our volunteering to, but also to allow volunteering to actually get some credit. I think there hasn’t been much credit for those volunteers who enjoy walking this work and government coming back on on support. Changing policy about putting so much work for volunteers to actually volunteer, but that at the end of the day, if that volunteer isn’t there, we won’t have service in our hospitals, our fire brigade, our schools, our.

So many areas that volunteer to, in that without their support, we wouldn’t have a society. And I think also the research also helps to, to find out why people are not volunteering. We know people are volunteering in different ways. We know that there’s a real impact on changing the that we got, but there are iGen as well.

So the work sector really helps with that, sort of getting the information to find out and think about what, what do we do in the future? How do we get those younger people volunteering now to stay on board? Our society has said previously, but there is a shift in lifestyle challenges and people are time poor who know that.

But you know, if you want the health service one, you need to. If you want the fire to put out a fire at the house, we need to make sure that they’re supported. Someone put their heads up to volunteer. So all those sort of things I think is what research will help with.

Tobi: I think to help contextualize this a little bit, for those of you who are listening who haven’t been to Australia and don’t know.

Much about the volunteering sector there. If you think about here in the us, if you think about the National Football League, the NFL, think about all of the folks who take tickets, all the folks that run concessions, all the folks that help people to their seats. All the folks that make a sport run in Australia, they are all volunteers.

So it would be like a volunteer run, NFL. Or if you think about you live in, I’m in, in Western Washington state right now, eastern Washington. Let’s say you live on, in a rural area and you have a farm. The people who are gonna come, if there is a fire, a wildfire, it’s going to be volunteers who come to help put it out on your property.

So the, the level and infrastructure of volunteerism in Australia is, is awesome. It’s so amazing, but if it’s not there, the country will majorly suffer. And so that research is really important to make sure we’re still, you’re still attracting, or all of us are still attracting volunteers and addressing their changing motivations.

I, I totally agree with that. I think this I idea of connection to resource making the case, making the business case for resources for, I know people are still trying to crack the nut of economic value beyond just wage replacement, which can, it is helpful for some audiences, I think, but the true economic value I think is much more than that, than just wage wage replacement.

I also think that. Pam, you mentioned this I idea of research to policy. Right, and, and really being able to advocate for a particular policy because it’s evidence-based. I think the areas that we would be wonderful to have research around, I would love to have more field research. Like, and it, it is very expensive to go out and like, observe and also conduct quote unquote experiments.

Like if we approach it this way, of course it’s, and it’s very difficult with human, human beings to say, well, this caused this, right? But there’s a lot of correlative analysis that can be done about, we think these are the drivers or the levers of this result. So let’s test it out some more. Right. The other thing I think that is so wonderful about research, and I think it’s a real value add to research, is we often think research is supposed to come up with all the answers, but I think it is equally valuable in posing more questions.

And every year when we do the volunteer management progress report, more questions get posed like, oh, well what about this? I’m like, oh, that’s a good question. It also drives the human sort of psyche and spirit of

Allison: exploration. That’s why you know you’re doing it right, Tobi. When you end up with more questions than answers you, we are onto something

Tobi: and I think this human spirit of exploration.

You know, it needs to be stoked a bit right now. Can I just say that? I will just say that it needs to be stoked. We need to have some more critical thinking. We need to ask questions. We need to not, uh, accept all the answers or all the, we’ve always done it this way, both in our field and outside of it, but I’m just on my soapbox on that phone in our sector for a little bit.

I think. That’s the other value of research is this continuous, it’s, it there, there’s a, the, the human need or the human drive to understand it’s very important that it doesn’t burn out. Especially now when there’s a lot of drivers that are, that keeping people from exploring in the deepest way. And so I think that is a human sort of human.

The human species kind of met. A reason for research to exist is it’s part of who we are and we won’t survive as a species unless we keep that fire going for understanding the world as it changes. Totally agree. Could not agree more. Yeah. Yeah. Hey, are you looking to upgrade and modernize your volunteer program?

Or maybe you’re building one from scratch and you’re just not sure where to start. If so, we’ve got the perfect resource for you. The Volunteer Pro Impact Lab, having built several direct service programs from the ground up, I know that it doesn’t happen by accident. It’s a clear process that takes careful attention with a focus on impact.

In the end, you need a system in place that’s clear. Standardized efficient, and that gets results. In addition, and maybe this is the most important, you need a volunteer program design that directly contributes to your organization’s most critical goals. That’s where the Volunteer Pro Impact Lab comes in.

When it comes to effective volunteer engagement, our bespoke volunteer strategy success path model, which is the heart of our resources and strategic advising, will help you transform your volunteer strategy from fundamental to a fully mature what’s working now approach and all in less time with our online assessment, you’ll quickly gain clarity.

On precisely where to focus your efforts and we’ll provide recommendations for the exact steps needed for sustainable growth regardless of how large or small your organization is, or what your cause impact area or focus is. Our program development and implementation support model will help you build a strong foundation so volunteerism can thrive at your good cause.

If you are interested in learning more, go to ball pro.net/join and we’ll share how to get started and what’s involved. Again, that’s ball pro.net/join.

Well, you know, now, now that we’ve gotten deep into it and I can see already, we’re half an hour into it. I, this is probably gonna be a two-parter, so let’s just, let’s just put that out there right now. ’cause I don’t wanna, I don’t know when everybody’s stop. Time is, but I have a feeling we’re gonna split this into two a part one and a part two.

Interview, but let, let me hand. We’re gonna metaphorically switch seats. We’re gonna metaphorically hand the mic over to Allison and Allison’s got some questions to ask Pam and myself. So let’s do that. So Allison, I’m excited to

Allison: be on the other side of the bike. I know, I feel like so honored that I, this is my first time on the Volunteer Nation and that now I’m going to ask Tobi questions on her own podcast, but it’s.

It’s honestly, it’s just really exciting, really fun. So obviously we’re here to talk about the volunteer management Progress report. You know that you’ve just completed the 10th iteration of that 10 years for the VMPR, and you have announced that, as you mentioned, that you’re gonna be. Putting a pause on that for now.

And so we really wanna talk about the story behind VMPR and some of the, the things you’ve learned along the way and, and just be able to talk about what this process really looks like of doing research in our field and, and give the listeners, I guess, some kind of food for thought around that and just the story of it.

So I just wanna start out really by asking you all, what is the origin story of VNPR and how did you first get the idea to start this project up?

Tobi: Yeah, great. I’ll answer that one because Pam wasn’t here at the very beginning, so I’ll answer this one. But I had started my consulting practice in 2009. I had left a large, I had been the director of a large statewide volunteer program with lots of staff and funding and volunteers and all, and I hung up my shingle and started doing consulting.

In 2015, I decided to open up our volunteer pro community. And so I spent a year developing it and we opened it up. And at the time it was really crazy ’cause people were like, what? Because back then there was no such thing as a paid learning community that just wasn’t part of the world at that point.

But I was a little bit ahead of the scenes, but when I started to think about establishing our presence, and so there’s a couple things. One was. How can we establish our presence? So for those of you who are budding consultants, this was important. I hired a coach, ki Miller at Nonprofit Marketing Guide as a coach to help me launch my community because it, at the beginning, I knew nothing about marketing, digital marketing, et cetera, et cetera.

So I needed help. So I hired smart people. Hire guides. Need guides, right? Smart people hire. People who know their stuff to help them get there faster. And I, I didn’t have time to mess around, so I worked with Kibby. Kibby had a, the nonprofit marketing guide early on and still does a communications. Report and they do research it with marketing and communications folks in the nonprofit sector.

So her business model’s very similar to mine. She does, she works with Marketing Marcos folks and nonprofits. I work with volunteer managers and nonprofits, and I thought this information would be really helpful for the field to start to understand volunteer managers a little bit better, and also would help me understand what kind of content and what kind of learning and what kind of challenges my students and members.

Are struggling with so that I can provide the most direct. To get there, to take action and get traction, as I like to say in the community. How can I best understand this audience and what they really need, not what I think they need based on my past experience with volunteers, but what do they need now?

And that’s really the impetus of that. And the first person I approached about the idea was CCVA, Katie Campbell, who used to be the executive director at the. Council for certification and volunteer Administration. And I pitched the idea and I said, look, this won’t happen unless I work with partners.

Would you support it? And she said, well, yes, of course. Uh, but I think you’re crazy to wanna do this every year. And I said, well, we’ll see. I don’t know. I, I like the idea of a longitudinal study. I like the idea of having this go out every year and being part of the cadence of our, my business. So she got on board and then I started asking other people, which is how you do grassroots organizing.

And I had been a grassroots organizer as one of my jobs, so I knew about building partnerships. So I built a coalition around it to help support. So alive, got on board, Rob Jackson, got on board, lots of folks around different consultants and different folks. Got on board really early, different software companies, got on board really early and so we had a, a group of folks who would help distribute the survey link every year.

And some of those folks helped every single year for 10 years. And I just wanna call those folks out. I just wanna say thank you. You know who you are. I just really appreciate all the support over the years. But yeah, that, that’s how it started. And you know, I had also some friends in my community who were also like, are you crazy?

You don’t wanna do this every year? I’m, it’s a lot of work. I’m like, yes it is. And I’m gonna do it every year. I’m like, now I could, I could write a, i, I could write a dissertation on this data set. There you go. Anyway, that was the, that’s, that’s a due in my retirement. But that’s the, that’s the origin story.

That it was basically. Emulating someone I really respected in the field, in the nonprofit sector who was serving the sector well and who still is Kibby Lou Miller over at Nonprofit Marketing Guide. I’ve had her on the pod here. And then also just to better understand what the heck’s going on out there for folks.

Allison: Yeah, that’s amazing. Thanks for sharing that. And one of the things I wanna just say too is that the field of people working in volunteer engagement, leadership around the world is one of the most. Well open and generous groups of people I’ve ever encountered, and I think it’s one of the reasons why when I started doing research with Jeff and I kind of got into that volunteering research space, and because there was just so many people that were so enthusiastic and so willing to.

To help me and to mentor me. And so it’s no surprise that when you wanted to get this going, that you had such a warm response and supportive response from people. It’s just, it just echoes what I already know about this community and it’s just amazing to hear. But. I think one of the things that I really wanna get outta this conversation too, is for you to be able to share about what this actually looks like to carry out, right?

Because I think so many people, not just not on the academic side, like not on the people working in research, but people who are working in organizations, in nonprofits, in public agencies. Oversee volunteers. They don’t think of themselves as researchers. They don’t necessarily think that they could do this.

Right? And so, mm-hmm. I just want you to talk to us about. What was really the process that went into planning and carrying out this survey? What were some of the steps that you actually had to do to collect the data and decide what you were going to collect and really launch this survey? So walk us through some of the practical, kind of behind the scenes steps, because I wanna try to set this stage and paint this picture for people about what this looks like because my long goal is.

To help see people see that this is something that maybe everybody could do a little bit of.

Tobi: Yeah. Yeah. I, I don’t know if I would encourage everybody to start with a, these are like. 35 question survey. The, the report’s like 45 pages long, maybe like a haired down version. Yeah. We’re looking for nuggets

Allison: to get us

Tobi: started, although, hey, I, I really encourage people to wanna take on the, um, so couple things in my mind just, and then I, I, this would be great for Pam and I to answer together, ’cause we’ve worked on several surveys and she knows the process as well as I, but a couple things just philosophically to get started with, number one.

I wanted, I always set a goal to reach more than a thousand people to get a huge statistically relevant sample size. Because I had seen, and even in academia, uh, there are surveys and there are research studies that have tiny, teeny, tiny sample sizes. And I have to say, Hmm, that’s not very. Yeah, it, it doesn’t add to credibility.

So I was always like, look, let’s get over a thousand. Let’s get 1500 folks. And I always wanted it to be global ’cause I did wanna make. Comparisons and see if there’s differences around the world. And also philosophically, I want it to be very rigorous, although I will say that we never did, we did straight descriptive statistic.

We didn’t do like correlation regression, any of that. I do, some would do some cross tabbing, but the data. Never had the deep dive because I didn’t have the deep pockets to pay for somebody to do all that deep dive statistical analysis, nor the time. And I, I took a statistics class once to try to learn how to do it and I was like, oh, hell no.

Excuse my French. But I was, I was gonna work for me. So part of it, to make it rigorous, what I would work with. Partner with folks who really knew, and I took, I studied survey, I’ve partnered with folks, asked practitioners and academics to review the question, set the survey items, and I also did training myself, did some pre intensive training on survey design and analysis.

So again. I walk my talk when I, I, I believe in training. I believe in professional development, and I believe if you don’t know something, you go out and you figure out how to do it. So I had been well trained up on how to develop surveys. It’s not like I just all of a sudden started writing questions. I designed a research project, but Pam.

Do you remember all the steps we used? It’s been a couple years since you’ve been working on with me on this, but do you remember the step, some of the steps we went through?

Pam: Yeah, yeah. I mean, I think I’m gonna answer Allison’s question about if I was a practitioner and I wanted to find out something in the sector, because I think Allison, that what he asked, he sort of asked, yes, we did this really big research, which could be scary for people.

Yeah. The word research. Sometimes scary work if you not comfortable with the STAs or the, all the words or the analysis. So me, so you and I did a lot of work and a survey together. I wanna inforce. It doesn’t have to be hard and it doesn’t have to be so complicated that people can’t actually find out these.

And I often say this to my post grade students, the, they always come to me with this great idea about everything they wanna do. And I say, let’s think of a funnel. Here are all your ideas. At the end of the day, you got time for this small little chunk, the, the bottom of the funnel. So go away and do lots of brainstorming, lots of, and I pick the safe folks out there in the volunteer sector.

Sit down with your staff or with your volunteers or whoever it might be, your C and think about what do you wanna answer. Do we have a brainstorm? What do you wanna find out? And you’re gonna come up with lots of things, but then think about what the one we really wanna answer, what is the one thing that’s really bugging us?

And it might only teacher and might be how we get different volunteers and might be we wanna find out why we perhaps males or females, and we attract differences. I think break it down. I mean, totally. Our survey did everything, but I think for others they can do little chunks of it to find out what they want.

Go and ask people, just like we did in our survey, that’s not something they all small. Go and ask experts. Do what we call academia desktop analysis. Go and have a look at lots of different how done. Have a look at our survey, see how we’ve done it. But then, okay, what sections of that survey? My blue pilling.

We wanna find out more and maybe replicate that. Because research for us as academics is about when we publish something and Alison would know, says, well, what we’re asked to do, you general is an adult needs to be able to be replicated by someone else. So you can go and have a look at journals and say, how have they done the survey?

How have they done? Try replicate that in your environment. So that gives you a little bit of, so I think even though we’ve done lots of things with the survey, I think for others that that’s a big, think about a smaller set of what new things do and also what do you want?

Tobi: I think too, be careful about intellectual property though, because I’ve had folks copy the VNPR survey questionnaire, redo it, and that was a, a copyright violation.

I had to send a cease and desist order. So I, my perspective on this is. If you are a it, it depends on who you are. If you are a practitioner who wants to know within your organization why something is working or not working, then really focus on what the pain point is. If you are a consultant or someone who wants to get into consulting.

Then figure out what other people aren’t doing because that’s the other thing I did. I was like, okay, what? What are people not doing that I can add value to in the sector? And in our case, there’s plenty. When I looked, when I researched and looked at academic research, there was plenty of research on volunteer motivation.

There’s like hundreds and hundreds of studies on volunteer motivation. No studies on the very few. Jeff Rodney was one who did, did actual research on volunteer management practices. Very few people were doing that at the time. And so I said, well, and besides, those are the folks I wanna, I wanna help. But there are a lot of aspects and we, so every.

Time we do every year we would do a, a survey. I chose certain core questions that I knew I was gonna ask every year for as long as I did the survey. So like, what’s your big number one biggest challenge? Tell us in as many words as you can. And Pam would do the qualitative analysis of that. But there’s also, every year we would do a theme and it would be a theme that was, that we would hear from around that something people are struggling with, like the software edition.

The first few were just sort of general because we didn’t, I wasn’t sure how long we were gonna do this, and at some point it was like, oh, we’re in our fourth year. I guess we better start themeing it up. So we started figuring out what would be the, a little bit of the focus of the. The inquiry for that year.

When Pam and I used to work on several of the additions Pam worked on with me, we would get together and start just spitballing ideas. What do you think we wanna study this year? What, what? What are people working on? What are, what are you interested in learning more about? Because it was also, Pam is an academician who’s busy in her life.

I mean, nobody makes money or gets paid to do this. We don’t charge for it. We don’t get paid for it. We don’t get sponsorship. We don’t get grants. No money exchanges hands for this research at all. So it’s something that we volunteer to do. It had to be a value add for Pam as well ’cause I couldn’t ask her to put on all this time and energy.

So once we spitballed, then we would start to develop the questions and draft up what we call survey items. ’cause some of them are questions statements.

Pam: Which is what I was talking about, that brainstorming. Yes. So it’s very similar. It’s just finding out what one’s out there, what do we wanna know? Yeah.

At that brainstorming complain. Yeah.

Tobi: And then people think that a survey can answer. Some pretty intense questions and we will get, and I would ask also, when I present the results, I go, who? What do you wanna learn next year during my webinars? And I would take notes, anything anybody wants to, any suggestions, I would take notes and I refer to them the coming year in the design process.

But there’s a lot of things that people think a survey can solve or answer, and it can’t. Like, what’s the best way to go about blah blah, blah? And I’m like, okay, well that, what’s the best re recruitment method? I go, okay, A survey on its own will not answer that question. We can find out what methods people are using.

We can ask people what capacity level they’re at with their volunteers, and we can correlate and cross tab those and make some type of correlative analysis that maybe when you use this method or these methods, you might get this result. But we can’t prove that. Right. And some of the questions we would throw out because it’d be like, there’s no way we cannot, this is, this is so far from being able to put a pin in something to be able to like be accurate about it.

We’d also hone it down to the shortest we could get it and then we would send it to a few people to test once we got it uploaded. I, I started with SurveyMonkey, ended up using Zoho surveys ’cause it was less expensive and still does cross tabbing. Which you’re comparing the answers of two questions to get a better, a deeper understanding, and then when the data would come in again, we would.

Distribute through distribution partners who got no money. We didn’t pay them, they didn’t pay us. They just did outta the goodness of their heart. We create all the collateral, the all the cut, cut and paste, copy, all the images, all that, that the survey link, and then we would start promoting it. Usually open for about a month.

Because it took a lot. Sometimes it had to go through a month of somebody’s newsletter cycle to get, make it into their newsletter, so we had to give it a lot of run. Usually with volunteer surveys, I never recommend people wait, go for an retirement. That’s insane within an organization, but within the network globally, you need that much time.

And then when the data would come back, then Pam and I, we’d d download the data and then we’d have a call and and say, what do you see in here? Here’s what I’m seeing. What are you seeing? What questions do you have? Where should we do cross tabbing? Where should we dig deeper? And then Pam would do the awesome.

This last year I did the qualitative analysis, Pam, so I know the detail. I mean, qualitative analysis means that you read every single answer. And you code it and we coded in a main code and a subcode. And so Pam or I would review 1400 responses and code them. And we had a, we sort of created code categories early on.

We would add to them a little bit if they changed. So like in during COVID, those codes, none of the codes, like people are, are not, what’s your biggest challenge was the big one. The big question we coded and folks would answer like COVID and then we’re like, okay, we don’t have a code for that. ’cause COVID didn’t exist last year.

Right. Pam, anything you wanna add on the qualitative side because you’re so good at that?

Pam: I, I think the qualitative side finds make it really rich. But it takes a long time to analyze anyone who’s thinking about doing some qualit, some data, and it would get a lot richer result because it’s a conversation.

You’re having a conversation with someone about their experience in volunteering. So often you’ll get, if you’re gonna do interviews, you’ll get an hour transcript That gives do lots of rich stuff for us, we didn’t limit, I think it was to hundred words because we were finding people in the first few surveys of information that was almost impossible.

So.

Give us your top five, your top issues. Yeah. Was

Tobi: your top? Your top? Sorry. Well, no, I think it was your number one Challenge. Challenge, but then, yeah, and then we would keep the box. Don’t make the box like the whole page. Yeah. But I would say like last year I said. Write as much as you want. We wanna understand, and people did.

Some people write two, two answers. Volunteer recruitment. That’s their answer. I’m like, okay, that doesn’t help me understand at all. Is it that you have too many volunteers? Not enough volunteers? Are the methods that you’re using working? Are they not working? Do you have enough money? Are you’re at Facebook ads?

What is it that you’re doing that isn’t, is to struggle? I really like it when people give me a paragraph, but it takes a long time to. To go through and then we would, we would make the quantitative or the qualitative open-ended comments. Yeah. That we only had two. And we wouldn’t ask a lot. We’d only ask one, what’s your number one biggest challenge?

Mm-hmm. And anything else you want us to know at the end, we’d ask that. And not everybody fills that out and they don’t need to, but. The rest of the questions were quantitative, but we would also make the qualitative quantitative by coding and counting the codes so we could see trends. And I have trend data.

I have charts that show the entire 10 years. And the number one category of challenge is recruitment except for one year in the middle of the pandemic, which it was supervision ’cause people were supervising remote volunteers. Biggest challenge supervising remotely. So that’s kind of how we did it. I kind of competed with myself like, how many, how many responses can we get this year?

And I love to see responses come from places so far away that I can’t imagine they, how they ever heard about the survey, like India or the Middle East. I’m like, Lebanon, how did they hear about the survey in Lebanon? It’s amazing. So. That’s kind how it works. It’s hard work. And then the putting together the report, Pam would analyze all the qualitative, and we do this during the holiday season, so it’s like

Pam: watch for silence during the more summer break.

Your winter break. Yeah.

Tobi: Yeah. And I’m like, this year I’m, I’m editing my, my book. So I, instead of doing the the research report, I’m, I’m doing my book, so I have other stuff to do. But we did it during the holiday and I would, I would do the number crunching and then put the, the graphics of the report together.

Hmm. Yeah, we, it was all DIY y’all. All DIY. That was a lot of work. A lot of work. A lot of work. But worth it. I mean, I feel like it has been so helpful. I hear people, I’ll get emails from people from time to time and they’ll say, you know what, I got a, ’cause sometimes we ask about salary, we don’t ask every year.

’cause it doesn’t go up that much. I’m sorry gang. It doesn’t. So we were like, alright, you are gonna take a break on salary. But I would hear from people, Hey, I got a raise because of your report. Or I got an FTE because of your report, or I was able to make the case for this particular approach because of your report.

And that when you hear that from people, it really motivates you to keep going, I think.

Allison: Yeah. That’s so amazing. Thanks. Thanks for sharing that. I mean, I think it is important to highlight just how much work goes into every phase of it, and I think that’s, this is really useful for people to think that they see this shine.

Finished report, but they don’t see all these behind the scenes, so it’s great to kind of hear more about that. I know Pam has to get going in a couple minutes, so I was just wondering whether, we’ll see if, Pam, if there’s anything that you wanna add or leave us with in terms of your kind of reflections and or your feelings about the work you did with VMPR and anything else you wanna add or leave us with?

Pam: Yeah, I, I, I think the whole survey was amazing. I mean, it, it’s. Allow organizations to really do impactful outcomes and not, you know, like to said, they can go bit more and show some evidence they could. It goes back to what I said, it’s almost in circle that we empty paramount because we we’re volunteering managers, we’re on the ground doing the work.

Having and myself do this survey and being able to access it, I think is really important. It’s important. For so many reasons, but, but I, I wanna discourage people from thinking about going ahead and doing something. And I know Tony talked about copyright of things, but ing the source is important or going and asking permission, but joining alone.

Survey, finding out from your own volunteers, miss.

To your organization because what our survey did was globally was it? But if you want more microlevel stuff in your organization getting together with, say for instance your working in the healthcare area may get together with some other folks managers because has similar things. Good. We survey. To those particular volunteers because they differences compared to if you do a survey for sport volunteers, very different terms of what you might find from being as well.

I could take very quickly what. And I the two different sectors in Australia, they had similarities are very small, but they’re different to the type thing. So you wanna find out from your sector. But I think that’s our survey and look at that macro. I think the gritty stuff still makes that happen at the grassroots, both in that community.

Community areas. Absolutely. And if anyone has any, they’re welcome to reach out and I can help an offer some advice. But I think thank, will I accept macro stuff.

Tobi: Yeah, absolutely. And I’ll put your contact information in your linked to your LinkedIn and your bio in in the show notes. And. We’re at the top of the hour and I know Pam, you have to go.

You know what, we’re gonna wrap up this one and then Allison, if you wanna stick around, we can record part two. Pam, thank you so much for joining me. It’s frigging me. It’s a ra. It’s a walk down remembering lane with you and we gotta catch up, girl. We gotta catch up. Absolutely

Pam: difference when you’re working with other people in our parts of the world.

Put that into perspective out survey. Often Tony and I would be up at all hours of the day and opposite each time to Yeah, try to work

Tobi: together. It was always, it was always my, it was always nighttime for me, and it was always early morning for you. Well, Pam, thanks so much and take care and be in touch and let me know when we can get together and just chat and catch up.

Absolutely. Well, okay everybody, that’s a wrap for this episode of The Volunteer Nation Podcast. So we’re gonna split this into two parts. We didn’t originally think about doing it that way, but as I see the time, we’re gonna take a break and we’re gonna come back next week and go into part two. Now, just behind the scenes.

Allison and I are gonna wait for a couple seconds. We’re gonna start on this second round of questions, but when you come back, we’re gonna talk a little bit more about more advice on how to do research as a consultant, as a practitioner, skills you might need, challenges, all that good stuff. We’ll come back.

Get into that as well. And gang, if you found this episode helpful, I hope you’ll share it with a friend. And of course, we always love five star reviews. It helps us get found on all the podcast platforms. Join us next week for part two, behind the scenes of 10 years of industry research with Pam Capaldi’s and Allison Russell, although part two will be just with Allison and me.

See you then and we’ll be right back.