grantee survey - Arbor Brothers

37 downloads 189 Views 801KB Size Report
Robin Hood Foundation. 3. Social Venture Partners. 0. Strategic Grant Partners. 0. Tipping Point. 0. Because so few AB g
Measuring Arbor Brothers Impact Grantee Evaluation Responses: 2015-6 Grant Cycle Disclaimer: This survey was administered by GlassFrog, an evaluation-focused consultancy, to gather as much constructive feedback as possible from our grantees. Even in an anonymized, third-party administered format, we are cognizant of the dynamics and incentives at play between funders and those they support. While we believe this inquiry is an important element in our selfevaluation, we would emphasize that over time any impact we have will be best measured by our grantees’ outcomes, not what they say about us. More information on how we attempt to triangulate the impact of our work is available here. Arbor Brothers Management Interpretation of Results Positive Takeaways: 1. The dosage of consulting support we provide still seems just right (neither too much nor too little). Grantees also universally endorse the current mix of consulting and $ support. 2. Excepting one outlier, grantees say the combined grant (consulting support plus money) makes AB their most valuable partner. 3. Grantees are uniformly comfortable sharing anything and everything with AB. We are their most trusted advisors. 4. AB’s strategic input is, on average, as or more valuable than input from the grantees’ board chair or most engaged funder. Things We Can Improve: 1. Grantees noted that though this wasn’t a barrier to effective partnership, a more diverse team (by race, gender or even parental status) might offer substantial upside. We agree. 2. Some grantees noted that more time invested in staff training—especially around the financial model—would’ve been helpful. We are moving further into this area this year. 3. A number of grantees, perhaps looking into the not-too-distant future, expressed an interest in ongoing support as alumni. We hope the next phase of our growth will provide for this.

www.arborbrothers.org

Anonymous Survey Results from Nine Grantee Respondents 1. Experiences in and between meetings:

www.arborbrothers.org

2. Comparing AB to other partnerships:

www.arborbrothers.org

3. Feedback on services: When Arbor Brothers' visions and goals for our organization differ from We improve our thought our own, Arbor Brothers processes as a result of respects our viewpoint our conversations with and ultimate ownership. Arbor Brothers. Always Very often Always Very often Always Very often Always Always Always Always Always Very often Very often Always Very often Very often Always Sometimes Always Very often

Arbor Brothers gives conflicting or confusing advice.* Never Rarely Rarely Sometimes Never Rarely Rarely Sometimes Never Rarely

Arbor Brothers does its own homework in between meetings with us. Always Always Always Always Always Always Always Always Always Always

Arbor Brothers' homework contributes positively to achieving the goals of the project. Always Always Very often Very often Always Sometimes Very often Very often Always Always/Very often

We ask Arbor Brothers for advice beyond the scope of the current project. Very often Very often Always Always Always Very often Very often Very often Rarely Very often

Grantee 1 2 3 4 5 6 7 8 9 Modal response

Arbor Brothers pushes us We feel comfortable We can easily negotiate a out of our comfort zone The ultimate value of the sharing confidential compromise with Arbor in a way that improves homework we are information with Arbor Brothers when we the quality of our assigned justifies its Brothers, knowing it will disagree with them about management and difficulty and be handled sensitively our work. leadership. unpleasantness. and appropriately. Always Always Always Always Very often Sometimes Very often Always Always Sometimes Very often Always Very often Very often Very often Always Always Always Always Always Always Very often Sometimes Always Very often Always Very often Always Very often Very often Very often Always Always Rarely Always Always Always Always/Very often Very often Always

Arbor Brothers' advice leads us in productive directions. Very often Always Always Very often Always Very often Very often Very often Always Very often

Arbor Brothers provides clear instructions for doing our homework. Always Very often Always Very often Always Very often Very often Always Always Always

We are comfortable discussing projectrelevant subjects with Arbor Brothers. Always Very often Always Always Always Always Always Always Always Always

Grantee 1 2 3 4 5 6 7 8 9 Modal response

Arbor Brothers helps us with issues not directly related to our core project. Very often Very often Always Always Always Very often Always Always Rarely Always

Grantee 1 2 3 4 5 6 7 8 9 Modal response

Grantee 1 2 3 4 5 6 7 8 9 Modal response

Arbor Brothers' visions and goals for our organization align with our own. Very often Very often Very often Very often Always Very often Always Sometimes Always Very often

We come away from meetings with Arbor Brothers feeling energized about our work. Sometimes Very often Always Very often Always Very often Always Very often Very often Very often

We look forward to upcoming consulting meetings with Arbor Brothers. Very often Very often Always Always Always Very often Very often Always Always Always

We worry about We spend considerable Even when we are as wellupcoming consulting effort to make sure we are prepared as we can be, We improve our practices meetings with Arbor as prepared as possible Arbor Brothers is as a result of our Brothers because we for meetings with Arbor unsatisfied with our conversations with Arbor anticipate disagreements Brothers. work.* Brothers. about project directions.* Sometimes Rarely Very often Rarely Very often Never Very often Never Sometimes Never Very often Never Very often Never Very often Never Always Sometimes Always Never Sometimes Rarely Sometimes Never Very often Rarely Always Rarely Sometimes Sometimes Very often Rarely Sometimes Never Very often Never Sometimes Never Very often Never

Our inherent differences with the members of Arbor Our reputation among Arbor Brothers has helped Arbor Brothers has aided The tools we built with Brothers (e.g. race, gender, nonprofit colleagues has us to understand what it Arbor Brothers has helped us in developing highThe tools that Arbor Arbor Brothers have religion) sometimes cause improved due to our means to have an us to develop our quality tools to support an Brothers has helped us helped improve the way a disconnect in association with Arbor outcomes-focused culture. outcomes-focused culture. outcomes-focused culture. develop are user-friendly. our organization operates. communication.* Brothers. Agree Strongly agree Agree Strongly agree Agree Agree Strongly disagree Strongly agree Neither agree nor disagree Agree

Agree Strongly agree Strongly agree Neither agree nor disagree Agree Agree Disagree Strongly agree Agree Agree

Strongly agree Strongly agree Strongly agree Agree Agree Strongly agree Strongly agree Agree Agree Strongly agree

We come away from meetings with Arbor Brothers with a clear idea of how to move forward, and confidence that we are able to do so. Always Very often Very often Very often Always Sometimes Very often Very often Always Very often

Agree Strongly agree Agree Agree Agree Neither agree nor disagree Strongly agree Agree Strongly agree Agree

Agree Strongly agree Agree Neither agree nor disagree Strongly agree Agree Strongly agree Agree Strongly agree Agree

Disagree Strongly disagree Disagree Strongly disagree Neither agree nor disagree Strongly disagree Strongly disagree Disagree Neither agree nor disagree Strongly disagree

The benefits we receive from working with Arbor Brothers justify the resources (e.g. time, personnel) we devote to Arbor Brothers projects.

The lack of diversity on the Arbor Brothers team has meaningfully diminished the effectiveness of our partnership.*

Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree Strongly agree

Disagree Strongly disagree Disagree Strongly disagree Neither agree nor disagree Disagree Strongly disagree Disagree Neither agree nor disagree Disagree

Strongly agree Strongly agree Agree Agree Agree Neither agree nor disagree Disagree Agree Neither agree nor disagree Agree

4. Balance of financial and consulting support:

Arbor Brothers provides both grant money and consulting. If you could alter the balance of grant money vs. consulting, you would prefer (choose one):

Exactly what we got.

Less consulting and more money overall.

Less money and more consulting overall.

Total

8

1

0

9

5. Usage of “Theory of Change” document (three grantees): # reporting they used TOC for this purpose (total # = 3) Writing grant proposals Fundraising Budgeting Starting new initiatives Beginning, continuing, or ending professional relationships with other organizations or partners Performance management Other: Communications materials Hiring Downsizing Approving expenses

2 2 1 1 1 1 1 0 0 0

www.arborbrothers.org

6. Usage of other tools co-created with AB (nine grantees): # reporting they used tools for this purpose (total # = 9) Setting budgets Making hiring/firing decisions Board development Program evaluation Fundraising Starting new initiatives Performance management Embarking upon/ending professional relationships with other organizations Ending initiatives

7 5 5 5 5 4 4 2 0

7. Usage of systems/processes co-created with AB (nine grantees): # reporting they used systems/processes/data for this purpose (total # = 9) Setting budgets Fundraising Making hiring/firing decisions Starting new initiatives Program evaluation Board development Embarking upon/ending professional relationships with other organizations Performance management Ending initiatives (All grantees reported using systems/processes/data for at least one of the defined purposes.)

6 6 5 5 5 4 3 3 0

8. Markers of outcomes-focused culture:

Grantee 1 2 3 4 5 6 6 7 8 Modal response

I have short-term (1 year or I have short-term targets I have medium-term (2-5 fewer) targets for for how many participants years) targets for outcomes. I want to serve. outcomes. Very true Very true Very true Neither true nor false Somewhat true Somewhat true Neither true nor false Somewhat true Somewhat false Somewhat true Somewhat true Somewhat false Very true Very true Very true Very true Very true Very true Very true Somewhat true Somewhat true Very true Very true Very true Very true Very true Very true Very true

Very true

Very true

I know exactly how I will I am regularly measuring measure the extent to the extent to which I am I periodically realign my My staff can describe our which I hit my outcomes on track to hit my spending to better hit my medium-term outcomes targets. outcomes targets. outcomes targets. targets. Somewhat true Neither true nor false Neither true nor false Very true Somewhat true Neither true nor false Neither true nor false Neither true nor false Somewhat false Somewhat false Somewhat false Somewhat true Somewhat true Somewhat false Somewhat true Neither true nor false Somewhat true Very true Very true Very true Somewhat true Somewhat true Somewhat true Somewhat true Somewhat true Somewhat true Somewhat true Somewhat true Very true Very true Very true Somewhat true Very true Somewhat true Neither true nor false Very true Neither true nor Somewhat true Somewhat true Somewhat true false/Somewhat true

My board members can describe our mediumterm outcomes targets. Neither true nor false Neither true nor false Neither true nor false Somewhat false Neither true nor false Somewhat true Somewhat true Somewhat true Very true Neither true nor false

9. Over the past year, has your organization passed up an opportunity for revenue growth because it did not align with your organization's long-term goals? # (total # = 9) No: we were presented with an opportunity for growth and we decided to pursue it even though it did not directly align with our long-term plans. Yes: we were presented with at least one opportunity for growth and we decided not to pursue it because it did not align with our long-term goals. No: we were not presented with any opportunities like this.

0 7 2

10. Imagine engaging a stakeholder in an hour-long conversation about a large strategic issue of organizational importance. Please rate the relative value of speaking with each stakeholder below compared to speaking with Arbor Brothers.

www.arborbrothers.org

“Arbor Brothers’ input would be _______ valuable than that of this person (or these people)." Grantee

Board chair

Average board member

Most engaged board member

Average funder

Most engaged funder

Front-line staff

Management team

Most trusted staff member

1 2 3 4 5 6 7 8 9 Modal response

About as About as About as About as About as NA NA Somewhat less Somewhat more About as

About as About as Much more Much more Much more Much more Much more Somewhat more Somewhat more Much more

About as About as About as About as About as About as Much more Somewhat more Somewhat more About as

About as Much more Much more Much more Much more Much more Much more Somewhat more Somewhat more Much more

About as About as About as About as About as About as Much more Much more Somewhat more About as

About as About as About as About as About as About as Somewhat more Somewhat more Somewhat more About as

About as About as About as Much more NA Somewhat less Somewhat less Somewhat less Somewhat less Somewhat less

About as About as Somewhat less Somewhat less Somewhat less Somewhat less Somewhat less Somewhat more Somewhat more Somewhat less

11. For each of the following partners, please consider the benefits you receive from this foundation, relative to the time required to work with the foundation (i.e. time spent applying to, working with, and reporting to the foundation). When considering the benefits, please include financial benefits, prestige/halo effect, technical expertise, referrals, and any other benefits that you may receive. On a scale ranging from 1 to 7, how does Arbor Brothers compare to this foundation when considering the ratio of costs to benefits? Because so few AB grantees received funding from these organizations, we cannot report results by foundation. Instead, we show the number of grantees receiving funding from each foundation below. The average score (on a scale of 1-7, with 1=AB is much less favorable and 7=AB is much more favorable) across all foundations was 4.9.

Acumen Fund Ashoka Draper Richards Kaplan Foundation Echoing Green Edna McConnell Clark Foundation GreenLight New Profit NewSchools Venture Fund Robin Hood Foundation Social Venture Partners Strategic Grant Partners Tipping Point

REDACTED [responses not for publication]

# grantees funded by each foundation 0 1 1 2 0 1 1 0 3 0 0 0

12. AB’s role in funder introduction, pitch preparation, and hiring: # reporting yes (total # = 9) Has Arbor Brothers introduced you to a funder from whom you received funding?

6

Has Arbor Brothers helped you to prepare a pitch to a funder that led to funding?

6

Has Arbor Brothers helped locate new potential hires for your staff?

5

13. What, if anything, surprised you about working with Arbor Brothers? Before you started working with Arbor Brothers, what were your expectations of your work with them? Where did these expectations come from? 

"I've worked with other consultants. We'll get a grant and then work on a project and those relationships have been largely frustrating. I thought this would be the same, kind

www.arborbrothers.org

of going through projects for the sake of going through them. But they tailored the work to our needs and what we would find useful. I feel like we have a thought partner, a team that understands what nonprofits go through. I felt like they were on our team, doing what they could do to make things better. 

I hadn't heard much from other grantees, but other funders had said [Sammy and Scott] were really smart. But people say that about a lot of people and it's sometimes not true. "



We really value our work with the Arbor Brothers. We were a little surprised that our second year wasn't as productive as we would have liked it to be. It didn't feel as productive this year. I don't know if that's because we chose the wrong project. I think we got bogged down in details that maybe weren't central to the project and maybe we didn't see eye to eye on the direction of the project.



"Arbor Brothers is very different from other funders. A lot of funders claim to be value added funders, but Arbor Brothers really is. They strike a good balance of understanding the organization, helping support decision making, and also providing really valuable consulting services.



I think a lot of times when people say they want to do value-added funding, I don't take them seriously. But with them, I do take it seriously. It's made a big impact on this organization. "



"Two things surprised me. First, their accessibility. Even when you get funded by a funder that tends to be involved and offers some advisory services, you don't expect to have nearly the kind of access that we have to Scott, Sammy, and Nick, both in terms of sheer quantity of time and also the rate at which they turn around things we request. It's almost like a concierge service in the sense that they make themselves so available and when they commit to helping they do it quickly. You wonder how they support as many organizations as they do. We've been pleasantly surprised by that. They are super dependable and indispensable advisors for us. I have more access to them than I do even to some of our board members who theoretically have made a larger commitment to the organization.



Second, how much they care about you as an individual. Usually, relationships with funders are pretty performative, transactional, you have to always treat it as a professional dynamic between a funder and a grantee. With Scott, Sammy, and Nick, they're informal; it feels like you're interacting with peers and friends, which makes it a warmer and more authentic dynamic than we've experienced before with a funder."



I was surprised by how serious they are about following through on the part of engaged philanthropy. We knew it came with consulting hours, but were mostly just excited about the money. They have a lot of integrity around following through on the work and the hours they commit to. I was surprised by how much we came to rely on it.



I now can't remember what my expectations were. I can tell you what makes them unique --no one else does it-- is providing such concrete assistance, at least to start ups. Maybe

www.arborbrothers.org

later stage orgs get that, but not startups. I think that's unique about them. I don't think I'd talked to anyone who had been their grantees. I was very pleased...I hadn't thought about it enough to necessarily be surprised, but really pleased that they provide such concrete assistance. 

I would say how committed they are to our organization and our success, as well as the amount of time and energy they put into supporting our organization and their willingness to help in other areas outside of the specific project they were working on. I knew a little bit about them through mutual acquaintances, so there was a basic knowledge but definitely not the level or extent of the services they provided or who they are as people.



Sometimes when you work with consultants who have a business background, there is a concern that they're not going to respect your background, skills, and knowledge. I was a little worried about that when I filled out the Arbor Brothers rubric. The language they used and the things they tracked seemed to reflect their values and background. But I have to say that we've been pleasantly surprised that that hasn't been an issue at all. (Grantee later acknowledged that when s/he looked back at the rubric at a later date, s/he had a better understanding of the language used in the rubric, so everything is starting to make sense to them. It just seemed off-putting initially because they weren't familiar with the language or the metrics.)



The level of partnership surprised me. Oftentimes, you think of a granter as people who give you money, but don't invest the time and effort both physically and emotionally that goes into real support. So that really surprised me. It's a real relationship, which I value. I knew the package would be some money and some consulting, but I didn't realize how impactful both would be. The money is always a good thing, but the level of support that they gave us has been nothing short of angelic.

14. Could you tell us about any funding wins or program awards that your organization earned this year that you believe to be “very meaningful?” OneGoal

Big grant from Booth Ferris Foundation, Carnegie Corporation (new this year) Maverick Foundation (new this year) Support from Robin Hood, Heckscher (not new this year) Arbor Brothers put in a good word for us with Booth Ferris. They introduced us to Neuberger Berman. We're going to get a board member and potentially a donationREDACTED from them. The fact that they[responses know everyone andpublication] can say positive things is definitely a not for value add. Everytime we meet, 9 out of 10 times they know someone and 6 out of ten times they know someone well enough to send a note or drop a name.

www.arborbrothers.org

ScriptEd

Kilman Foundation -- 100K 100K from Accenture Salesforce -- 100K this fall (new) Citibank gave us a grant 150K LinkedIn Some of the collateral that Arbor Brothers provided us helped with fundraising. The financial model and TOC model. Funders thought they were impressive and showed we were serious about the work we're doing.

C4Q

We are working on a pay for success model, raising money from social impact investors. We haven't gotten the funding yet; closing is at the end of the month. Arbor Brothers was very instrumental in this, helping us think through how to design it, introducing us to investors. Every step of the way, Arbor Brothers was involved. We kicked off a hiring program with LinkedIn and Pinterest. Others big awards were renewals and we're still waiting to hear on a few more. We were recognized at Robin Hood Heroes Breakfast. Invited to White House toREDACTED speak about their work at South by South Lawn.

[responses not for publication] Springboard We were not in fundraising mode this year. We're raising money in multi-year rounds, so we're not having to ask people for money in the same way each year. Highlights: We expanded to Washington, D.C. and were able to get a vendor agreement with a school district to run K-2 summer programming for kids across the full district. New Heights

Our biggest win was that we hosted a ten-year anniversary gala and raised over a million dollars.

CARA

In the past year, we have expanded our work in a direction of working really closely with the NYC DOE and CUNY on all of our programming. We have 4 different programs and we're expanding all of them either through the DOE or through CUNY. That's definitely a huge win and it's expanded our budget and our reach. It also positions us to do more policy work, because when you're working with groups that are educating a million school children, you get to a scale where you can affect policy. That is really our biggest win this year. Arbor Brothers has been a really helpful voice. As we went through that process, they helped make sure we took the work on in ways that made sense and kept us on mission, so we didn't end up following the money trail. We've called them and said we have 3 new offers and they set up a meeting right away even though it was over the summer. I think they've been really helpful in

www.arborbrothers.org

keeping our heads on straight and making us feel like we had someone to talk to as we went through that.

NAI

$150K grant from AFT Joel Klein - $50K I don't know if I had wins in terms of funding b/c of Arbor Brothers. I don't think that was the case. However, what they prepared us to do -- the first thing was we had no idea how to do a budget. It sounds basic, but oftentimes people like me don't get into the weeds of actually doing things like budgets and REDACTED making sure spreadsheets and reports and decks are ready to go. These kinds of not for things have been [responses a very powerful andpublication] formative for us.

Power of Two

We didn't win any awards or recognition or anything like that. We are in such start up mode; we haven't been looking for that. I haven't applied for anything. We have gotten funding -- I don't know that it's meaningful. We're happy, but not necessarily meaningful. None were particularly huge.

Girl Trek

100K from Ascena Retail Group (parent company of Ann Taylor, etc.). They offered some serious technical assistance that's been amazing for us and beyond what we expected (involving targeted marketing). We're launching something called the "north star group." We're looking for women to give at the hundred thousand dollar level and we've already received some donations. Scott and Sammy helped us with this by coaching us around how to ask for money.

15. What is your definition of an outcomes-focused culture? 

An outcomes focused culture is one that continually executes a cycle of setting goals, making a plan, measuring progress, and having that feed back into the next iteration of goal setting. I've seen it happen where organizations do various pieces of that, but don't necessarily have the metrics feed back into the planning. That's the difference between having outcomes and being outcomes oriented.



A culture in which data and metrics are important and are tracked, and program outcomes and program design are based upon those findings.

www.arborbrothers.org



I guess that it would be that the end goal is kept in mind in the work and that there's a continual circling back to ask whether or not as an organization we're accomplsihing our goals and whether we have evidence of that.



One where eveyrone from front line staff to the board are focusing on outcomes. And that there's a culture that drives toward those outcomes and supports that. The culture supports the focus; it doesn't distract from outcomes.



There are multiple components. It's a clarity of vision such that internal and external partners understand what you're working toward. What you are working towards is an understanding of what the ultimate outcome is and what success will look like if/when you deliver on that promise. That ultimate outcome is the north star of the organization.



We aspire to this: results focused, quantitative and objectively based; accountabilitydriven; understanding and being intentional about what you're trying to achieve, every step of the way, and measuring that, providing analysis, and taking action upon analysis.



I think it is specific benchmarks that you either meet or don't meet in terms of what you value as an organization to accomplish. What is it that you want to do?



A culture in which the entirety of the staff and stakeholders understand what the big, long-term goal is, that we all work toward that goal, and that we use data and metrics to check that we're progressing toward that goal.



Everyone in the organization and on staff is doing intentional work because they understand how our work contributes to the whole, because we've coached them on this.

16. What is one specific way in which Arbor Brothers has helped your organization develop an outcomes-focused culture? Grantee 1

Grantee 2

Arbor Brothers has done a lot of work in analyzing our data, helping us think about comparison points, long term predicted outcomes, how they report out on those, how they celebrate or don't celebrate the numbers, how they take action to drive toward improving those outcomes. There's been a lot of data analysis to help them see the big picture. We have an in-house data team that Scott, Sammy, and Nick have been working with. Arbor Brothers took our internal data and analyzed it to help us see what it really looks like. Arbor Brothers has helped push us to define outcomes we're working toward, how we are reporting them, and how are we working to attain them. Arbor Brothers' value is with helping us with the question of "how are we going to reach these goals." Arbor Brothers helped us start to think about cost per outcome, which we hadn't previously thought about. We are starting to calculate it and use historical data to figure out how much certain types of services cost.

www.arborbrothers.org

Grantee 3

Grantee 4

Grantee 5

First year -creating a theory of change for the main program. Arbor Brothers has given us resources that act as frameworks for thinking through thorny problems and/or benchmarking. This has been very useful. Modeling and providing frameworks for thinking through questions -- e.g., they do a survey of their grantees that includes basic fundraising outcomes. We still return to that to try and understand benchmarks for fundraising practices. This isn't something we've actively worked on with them, but Arbor Brothers helped provide a framework for me to think about the issue. I also like the organizational rubric for sustainability and outcomes-focused culture. My organization uses it and finds it "hugely impactful." Being outcomes oriented and data-driven is one of our three core values, so we didn't need any convincing on why that's important. However, where Arbor Brothers has been invaluable is in building our capacity to do that better. A piece of that has been helping us to develop a robust financial model that helps us connect our outcomes to our financial planning in a way that we haven't been able to do before. They're also helping us as advisors in structuring a four-year longitudinal study that we've been meaning to take on for a year now but haven't had the bandwidth to prioritize. So having support from Arbor Brothers helps in terms of accountability to make sure it doesn't fall off the priority list. They also add capacity-- before we have a full-time team focused just on data analysis, having them as data advisors to expedite the process is helpful. We'd built a pretty robust data system that does most of the analysis already, but they're helping to structure the study and help us think through the larger plan so we can know which data to capture and what are the insights we're trying to gather. They're also helping us manipulate the raw data to make it more actionable than we've been able to do on our end. They helped us create systems to track and evaluate our progress. They helped us create a rubric around our organizational goals for the current fiscal year; we're using it and finding it useful. I think helping facilitate conversation and thought processes around outcomes and putting it at the forefront of our planning and conversations as we move forward has been useful.

Grantee 6

They were incredibly helpful in taking publicly available data about school matriculation outcomes and sifting through our program outcomes and helping us pin down whether and to what degree the schools we've worked with intensively actually have seen increases in matriculation into college. (That's our outcome goal.) They were able to think really creatively about using publicly available data and use Excel to help us start thinking about measuring those outcomes. We pushed back and said there were lots of reasons kids don't get into college or go to college, and they pushed really hard and said, "Okay, but schools that have your program should be doing better than comparable schools." And they really

www.arborbrothers.org

talked us through in a kind and generous way about how we can talk about and measure our outcomes. This has helped us move forward. They were very clear about us needing to be outcomes focused. Grantee 7

I run a network of schools. We value social emotional learning. Most schools focus on math and reading and writing. I believe in both. We spent a lot of time with Arbor Brothers developing a social emotional learning rubric for our network. These are powerful, powerful tools that don't immediately translate into funding, but help flesh out the practical substance of one's ideas.

Grantee 8

We launched with a really strong focus -- our whole reason for being was something that was really driven by outcomes. We had a very specific target audience, very specific outcomes, and in fact we are just scaling one very outcomes-focused program, trying to change 3 parenting behaviors. And if we succeed at that, it's success. I feel like we have an easier time than other organizations being outcomes focused because it's who we are. Our management team is very outcomes focused. Getting the whole team to have that mindset has been more challenging, because our coaches have worked in agencies that weren't very outcomes focused, so we have to help them change their mindset. We were doing manual reports to management about performance and we hadn't given them to coaches yet. We were waiting until we got a data system before we gave the reports to coaches. Arbor Brothers has been very helpful in responding to what we wanted to do, giving advice, helping us draft reports when ours weren't as good. On the concrete side, they've been very helpful.

Grantee 9

In general, some of their comments have been helpful in getting me to think in an outcomes-focused way. Lots of ways. This is the most recent: When the marketing company XXX put us in contact with contacted us, they said they wanted to do a proof of concept before they sent a mailing out to a million people. And they wanted to base this off geography. We were able to pull up a list that Scott and Sammy had forced us to develop where we'd identified U.S. cities that had large shares of our target population. The marketing company asked us about it and we were able to pull up this list. We also used this list to define the areas we would target as an organization. From the very beginning, they pushed back on "what does success look like for a participant in XXX?" We told them we're trying to address the health crisis. They said they understood that, but how do you define and measure it? What is it that makes success? We had to do research around habit formation, behavior change, the industry standard around creating those things, CDC's definitions, and what we want to hold ourselves accountable to. We used all this research to set our benchmark and we use that information to set our goals. We're now holding ourselves responsible for habit foundation -- it's a super clear thing for us to measure, rally behind, and explain. Scott and Sammy spent a year helping us develop that.

www.arborbrothers.org

17. What is one specific area in which your organization still has room to improve, w/r/t an outcomes focused culture? 

"Everywhere and everything. First thing that comes to mind: still don't have performance management services, not every team has key performance indicators, no real-time analytics.



We know where we want to go, but we're lacking systems and capacity. We talk about what we want to do, but it's slower than we'd like."



Our board could be improved in a way that would support the outcome of sustainability. We did a lot of work in terms of developing metrics. I feel good about that. It's just the board is still not where they want it to be, but we're way further along than we were before Arbor Brothers was involved with us.



Tons of areas. Most notably, we haven't figured out how to align programming with our long-term goal in a way that would drive outcomes better.



"We need more structure around our long-term program, more temperature checks on how graduates of our program are doing and what programs are needed to keep them on track.



We are still in the process of centralizing all our data, which is a pain.



We also don't have measurable goals for our operations staff. "



Training and supporting our staff to all collectively know how to collect and track data and think and talk in an outcomes-focused way. I just think our senior team understands and buys into it, but our line staff is not necessarily experienced or trained in data or data collection.



I think the systems for tracking outcomes. We've built out some of the technology ourselves. But the systems we have aren't capable of tracking outcomes as we scale. We are trying to grow and we need to be thoughtful around what types of tech and surveys we roll out.



We need someone on our staff who can ask the right questions and understand our programming from the inside. Someone who can, internally, play the role that Arbor Brothers has been playing for us. Our work with Arbor Brothers has made us realize we need this.



We've done well to optimize each of our offerings/programs using a data driven cycle. The ongoing challenge is figuring out how the pieces come together in order to achieve our long-term mission. We don't have enough visibility into that; our biggest area for growth is being able to embed multiple program offerings into a recipe for fulfilling our mission over time.

18. Arbor Brothers always wants to improve the services it offers grantees by improving their approach, the types of services they offer, and the quality of their work. Could you

www.arborbrothers.org

provide two pieces of constructive feedback that would help Arbor Brothers make improvements in one or more of these areas? Arbor Brothers' leadership: (1) The way they ask organizations to re-apply for second and third year is the organization proposes a project and Arbor Brothers gives feedback. In year 2 or 3, I think it could be better to have Arbor Brothers propose a project based on what they've seen and their knowledge of the organization. They could say, "This is what we've seen and this is our knowledge of the organization, and this is where we want to focus. Do you agree?" We want to make sure we are getting the most bang for our buck and I feel we'd be better off if Arbor Brothers had more direct input and leadership in selecting the project. They may see things that the grantee doesn't see. (2) In the application, we wrote different things we want to do. For us, after the first project, it would have been great to have a check-in about our goals. We're changing all the time. I don't even remember what I wrote or whether I still need it. Maybe they could also look at what I wrote and make suggestions over what could be helpful. I'd love to hear about their ideas, like "When we look at your organization, it strikes us that this might be very helpful." Or, "Now that we're into the second year, we think these things could be helpful." Or, I want them to come to us at different points in time and say, "We want you to be more outcomes-focused. This is what an outcomes-focused organization looks like. Do you agree with this? Are there differences in how you see this?" They've never told me what they think an outcomes-focused organization looks like. It almost feels like they'll help with certain things, but they won't ever tell me what they think a model organization is. (Also from 2) It would be useful if they started the engagement with "this is the way we've helped other orgs. Let's talk about whether this makes sense for your organization." (Also from 2) They are big on the Socratic method, but I'm really looking for their insights. I really need a thought partner and they're great thought partners. Last year, _______________ made an off-the-cuff comment that pushed back on one of my assumptions about my team and, the more I thought about that comment later on, the more I realized it was helpful to have that different perspective. We don't often get to have conversations like this because the meetings are so structured (which I appreciate). And then when we do have that reflection time built into meetings, it feels a little like I'm being quizzed about progress and I get in a mode where I don't feel like I have a thought partner. I go more into performance mode. I don't have a solution here, but I do know that it would be more helpful to have more organic conversations, like peers. I'm a very strong-minded person. I'm not going to feel bullied. I don't need them to be so hesitant. I want to hear which tools can be helpful in which ways. They've worked with a lot of organizations, so I'd love their thoughts. I know it's important to be sensitive about telling me how to run my business, but I would still love their ideas and feedback. (3) In some ways, it might be helfpul for them to say "here's our menu of services." These are our strengths, but we could do other things. That would be better than just letting the grantee drive what they need. Because Arbor Brothers are the experts. They see so many organizations. (4) We need Arbor Brothers to be proactive around suggesting potential funders and helping us figure out how to pursue them. This is an area where we haven't been as productive.

www.arborbrothers.org

New/improved services: (1) I really like the financial model, but most staff don't know how to use it or understand it. I don't have time to train staff on how to use it. We would have liked for Arbor Brothers to provide more full staff trainings. (2) I want more benchmarking data on things like compensation, growth, liquidity, cash flow, key performance indicators etc. I want to know not only what is the standard, but what is the standard for organizations of our size and stage. I think that Arbor Brothers is positioned to collect data like this and would find it very useful. It is especially pertinent for organizations in the Arbor Brothers second-stage. I appreciate the rubric (where organizations can self-rate on scales of 1-5), but would prefer more quantitative data, customized for organizations of different size and stage. (Also from 2) I think that Arbor Brothers could publish more about what they're doing and could help the sector overall (including publishing their toolkits). This would improve their brand, but also be a tremendous service for the field. Improving tools Arbor Brothers built a great financial model that is good for many things. It's a great internal tool, but it's not adaptable for fundraising purposes. The fundraising team can't extract the information they need from it, so they typically have to start from scratch. I'd like for it to be a more flexible tool. Staff diversity: (1) They need more diversity. (2) They should diversify their team so they can be supportive of a wider range of challenges and so they can push the field forward on diversity in the way they do other areas of the field. (3) The fact that none of them have children means they don't always understand the needs of people who do have children. This isn't their fault, but it's something that came up. Alumni programming: (1) They should think about an alumni program that would help sustain the types of services Arbor Brothers is offering now. Alums could get additional funding and coaching -- not 200 hours of coaching, but potentially 50 hours of consulting work. It could be given to all organizations, or organizations that need more help, or organizations that have done really well and would benefit from continued good work. (2) It would be great to have an alumni program.

www.arborbrothers.org

Length of meetings: (1) Sometimes the meetings are a little long. We meet for two hours at a time and, even though the meetings are productive, I sometimes start to zone out. I would benefit from shorter, more focused meetings. (2) This is very minor. I think they throw everything into the agenda (everything we're thinking of and they're thinking of) and we usually only get halfway through the agenda. I think it would be helpful if they either had longer meetings or had a way of structuring the agendas so they weren't so long. We would really like longer meetings (2.5 hours). We're always rushing.

Client management: (1) It could be useful to know how they divide and conquer so the grantee knows how to engage them in the right ways. We go to __________ and _________ joins us for major stuff. Nick is tagged in randomly. They deliver strong work, but it's unclear what the different roles are. It became clear that __________ was our go-to person, but I didn't feel like that was made clear from the beginning. If they have some optimal arrangement, we could help to support that if we know what it is. Like, should we always go through the point person? I think we have a good intuitive sense for it now, but for a new grantee having that made explicit would be helpful. (2) I think there needs to be more clarity around how best to interact with each member of the team so that we're honoring and supporting whatever division of labor they've determined already. I don't really know how they're structured. I thought ___________ was the lead and Nick was the associate. It would be helpful to know how they organize that. I'm not sure who to ask for what. (3) In the first year, I didn't know that they didn't work with grantees in the summer. That would have been helpful to know ahead of time. (4) I have a practical piece of feedback around frequency of the meetings. They're so good with agendas; every meeting has an agenda. It's really helpful, but I think they could be a little more agile or flexible around being "real time" responsive. If things come up, for example, it would be helpful to be able to set up meetings in real time based on emergencies. They did this for us, but we always felt guilty about it. Maybe they should communicate to grantees that they're also available for real-time problem solving. Or it could be communicated better. Having that be an understood agreement up front could be really helpful. (5) It's a little hard that there's this big pause in work as they go through their next round of grants over the summer and into the fall. And then they shut down again in mid-December. We're feeling that squeeze a bit. I know why they do it, but that's one thing that we've been talking about recently. (6) It's not super clear where Nick fits in relative to Scott and Sammy. He's pretty quiet; often the observer. He is obviously super capable, but he's usually supporting Scott and Sammy, playing backup rather than affirming a third and different type of skillset where we'd approach

www.arborbrothers.org

Nick for a question that we wouldn't bring to Scott and Sammy. Maybe that is his role, but it would be useful for that to be spelled out a little more. Whole-group functions/gatherings I don't know if I'm right on this -- one thing we've wondered -- is there not more opportunity for grantees to come together to have shared learning together? We never do this together except at the bbq. On one hand, why force this if it's not what they do well? On the other hand, if Arbor Brothers is your only engaged funder, it's a missing component of the experience. They could curate this in simple ways; an annual one-day retreat where we share best practices around what we've been taught. I want to know about other practices. When I do hear about other grantees' Arbor Brothers projects, I think, "I didn't know we could do that!"

www.arborbrothers.org