Cost-Effectiveness

The findings of the evaluation of the Community Futures Program in Western Canada on the issue of cost-effectiveness are presented in this section of the report.  The evaluation questions which were considered in addressing the issue of cost-effectiveness were as follows:

• To what extent is the CF Program cost-effective?
• Are there other more cost-effective/efficient approaches or alternatives to be considered that would achieve CF Program objectives?

Findings

The overall findings of the cost-effectiveness section are: the Program is still cost effective when the level of activity and return on investment are compared to costs; program costs have been on the rise and level of activity has been declining; the CF delivery model is considered sound and effective by numerous outside parties; those involved with the Program offered suggestions on ways to further improve it. These findings are further elaborated in the following sections.

Cost-Effectiveness

The CF Program is still cost-effective when the level of activity and return on investment (ROI) are compared to costs. However, operating costs have been on the rise over the last few years and the level of activity has been declining, due in part to economic context of the western provinces.

Historically the Program has shown a significant ROI.  A 2002 Impact Study (Ference Weicker) found that the CF program was generating a significant Return on Investment. ROI was calculated by comparing impacts of the loan clients to the value of loans they received. The following is a list of some of the most significant findings:

• In 2001-02, an average loan-only client received $51,215 in loans from Community Futures.
• Loans are generating about $14.30 in accumulated revenues per dollar of loans provided to loan clients.
• Loans are generating one person year of employment for every $2,987 in loans received by loan clients.
• Each dollar contributed by CFs (excluding loan capital) has leveraged $84.00 in public and private sector project funding.
• Each dollar contributed or loaned to CED projects by the CFs has leveraged an additional $15.72 in public and private sector project funding.

The client survey conducted as part of this evaluation has new business clients receiving on average $63,605 in loans from Community Futures and existing businesses receiving on average $89,952. Clients were asked to provide their gross revenue within a range (e.g. $100,000 to $499,000; $500,000 to $999,999 etc.). Clients were also asked the likelihood (scale of 1 to 5 where 1 = very unlikely and 5 = very likely) that they would have been able to start-up or maintain their business without the CF loan. A conservative ROI was calculated by dividing the gross revenues (using the lowest number in the range) by CF loan amount (for only those loan clients who responded that it was very unlikely that they would have been able to start up or expand their business without the CF loan). A somewhat conservative calculation was also done using the higher number in the range of gross revenues for those clients who reported that it was very or somewhat unlikely that they would have been successful without the CF loan.  The conservative estimate results in nearly $6 ($6.40 for a new business and $5.48 for an existing business) in gross revenues being generated for every CF loan dollar. The more liberal estimate results in nearly $11 ($11.14 for a new business and $10.56 for an existing business) in gross revenues being generated for every CF loan dollar. It is difficult to know the exact figure because it is impossible to determine whether or not the business would have been able to start-up or continue in the absence of the CF program and the survey did not ask respondents to clarify what percentage of their gross revenues they would attribute to the loan they received from the CF program.

Table 22: Return on Investment data from Client Survey

	Conservative (lowest in gross revenues range and very unlikely)	Somewhat Conservative (highest in range and very/somewhat unlikely
Gross Revenues	$89,260,000	$291,299,684
CF Loan value	$14,992,932	$26,684,753
ROI	$5.98	$10.84

top of page

As part of the Client survey, CF loan clients were also asked to report the number of jobs that were created or maintained as a result of CF funding. When value of reported CF loans to businesses was compared to total jobs created, it took on average $10,223 of CF loans to create a job for a new business and $12,106 to create or maintain a job for an existing business.

Case studies further corroborate these results. Administrative data, used as part of the case studies, indicate that on average over the 1999/00 to 2007/08 period, the amount of CF loan dollar to create one job was $8,004 in MB, $10,247 in SK, $11,720 in BC, and $14,030 in AB. During the same period, this ratio tended to increase in all four provinces, with marked increase in AB, which is probably a reflection of both the labour shortage and the increase in average loan size in this province.

In the client survey, loan clients were asked to report the total dollar value of additional funding they were able to obtain as a result of the CF loan. 30% of new business loan clients and 20% of existing businesses reported being able to obtain additional funding.  New businesses leveraged an additional $13M, representing 60% of the total value of loans issued to new businesses and existing businesses leveraged an additional $7M, representing 40% of the total value of loans issued to existing businesses (see Table 23 below).

Table 23: Additional Funding Obtained by CF Loan Clients

	Total Value of CF Loans	Total Value of Additional Funding	Percent Leveraged
New Business	$20,953,936	$13,065,614	62%
Existing Business	$17,999,791	$7,123,794	40%

Administrative data, used as part of the case studies, indicate that on average over the 1999/00 to 2007/08 period, CFs' loans recipients clients leveraged $1.34 (including owner's equity) for every loan dollar issued by the CF program.

In addition the CFs benefit from a volunteer board of directors and the volunteer hours of many other partners in the community. The time and dedication of these community members helps to reduce the overall cost of the program. The direct link that these volunteers bring into the community helps to ensure that the CFs are focusing on issues that are relevant to the community.

The evidence demonstrates that historically and recently CF program returns surpass CF program costs. However, a trend analysis of administrative data over the last five years demonstrates that overall, the administration costs of the program are rising and the number of loans being issued and the number of jobs being created out of loans were declining. Hence, the investment (loan dollars) to create a job has risen overall with marked increases in MB, SK and AB. The rising operating costs and decrease of the number of loans issued are significantly associated with the strong economy in the western provinces in the last 6 fiscal years, as explained in sections 3.1.2 and 3.1.6.

The overall costs of the CF program have risen from 2002/03 ($24.0M) to 2007/08 ($29.4M) by 21% with an average annual increase of 4.3%. The CFs received what was considered a much needed injection of operating cash in 2004/05 after years of no increases and even decreases (e.g. 2003/04). The CFs were guaranteed a subsequent 2% increase per year. On average, from 2005/06 onward the CFs have received a 3.7% increase. The current funding level for a CF is on average $292,000. These operating dollars typically support an office that has 3 to 4 personnel.

CF associations received a significant increase in funds (i.e. a 44% increase) in FY 2006/07 and otherwise have remained stable. WD administrative costs rose in 2004/05 from $1.83M to $1.84M and have remained stable since then. WD only tracks the overall administrative costs, not the break down by region. This is problematic because it makes it difficult to confirm the reported disparity of allocated resources in WD's regional offices (each of the four provinces).

top of page

Figure 29: Operating Costs in (000's)

 

Fiscal Year	CFs	Associations	WD Admin	Total
2002-2003	$21,303	$840	$1,827	$23,970
2003-2004	$20,435	$840	$1,827	$23,102
2004-2005	$23,390	$840	$1,827	$26,057
2005-2006	$24,711	$850	$1,840	$27,401
2006-2007	$25,202	$1,227	$1,840	$28,269
2007-2008	$26,313	$1,244	$1,840	$29,397

CFs are active in three areas: issuing loans, offering business advice and participating in community economic development.  WD provides CFs with a lump sum of money to conduct all three types of activities and WD does not keep track of the dollars that are spent in each of the above activity areas. However, evidence from the case studies confirmed that the amount of time spent on each of the three services varies considerably from one CF to another; and that on average CFs' staff spend one-third of their time on each of the three services. The administrative data that is considered most reliable is the number of loans issued. For yearly cost comparison purposes, the total administrative dollars (funds provided to CFs, CF associations and WD) were divided by three (loans represent one-third of CF activities) and compared to the number of loans issued.

The average administrative cost per loan funded has increased by 53% from 2002/03 ($5K) to 2007/08 ($7.5K) with an average annual increase of 9%. When the funds disbursed to CFs by province are compared to the number of loans by province, Manitoba and Saskatchewan demonstrate the greatest increase in cost per loan from 2002/03 to 2007/08 while BC demonstrates the smallest fluctuation in cost per loan (see Figure 30 below).

Figure 30: Cost to Administer a Loan

 

A possible explanation for the decrease in loan activities could be an increase in other activities. However, there is no evidence that there has been a marked increase in the other two activities areas (business services and CED) during this time period. 

Another explanation could be that the number of loans has been decreasing but the total dollar value has been increasing. The administrative data does corroborate this. The total dollar value of loans has increased overall from 2002/03 to 2007/08 by 18.6%. All provinces have shown an increase (BC 35%, MB 31%, AB 14%) with the exception of SK that has had a 7% decrease in the value of their loans (see Figure 31 below).

Figure 31: Total Dollar Value of Loans Issued

top of page

The average administrative cost (total operating dollars for WD, Associations and CFs divided by 3) to administer a dollar of loans has risen a total of 7.3% from 2002/03 to 2007/08 with an average increase of 1.2% per year. The average CF cost to administer a loan has risen 7.8% from 2002/03 to 2007/08 with an average increase of 1.3% pear year. The largest increase can be seen in SK followed by MB and AB. BC experienced an overall decrease over this time period. In Manitoba, the 2007/08 cost to administer a loan is equivalent to the 2003/04 cost but MB experienced relatively large increases in 2004/05 and 2005/06 as compared to other years and other provinces (see Figure 32 below).

Figure 32: Cost to Administer a Dollar of Loans

	AB	BC	MB	SK	Average CF	Average Admin
2002-2003	$0.14	$0.11	$0.27	$0.13	$0.14	$0.15
2003-2004	$0.17	$0.12	$0.22	$0.16	$0.16	$0.18
2004-2005	$0.13	$0.13	$0.37	$0.11	$0.14	$0.16
2005-2006	$0.13	$0.12	$0.37	$0.13	$0.14	$0.16
2006-2007	$0.15	$0.12	$0.25	$0.13	$0.14	$0.16
2007-2008	$0.16	$0.10	$0.27	$0.24	$0.15	$0.16
period average	$0.15	$0.12	$0.29	$0.15	$0.14	$0.16
Average Annual % Change	4.2%	-1.7%	4.9%	20.0%	1.6%	1.5%

When CF operating costs are compared to the number of jobs created, the average cost per reported job created out of loans has risen from 2002/03 ($1,315) to 2007/08 ($2,008) by 39% with an average annual increase of 7.8%. The greatest increases are being seen in SK followed by MB and AB and BC (see Figure 33 below).

The cumulative effect of a decrease in loans and increase in costs results in seemingly dramatic changes in the cost per loan. However, on average there has been only a slight decrease in the average number of loans being issued per CF from 18 in 2002/03 to 14 in 2007/08 representing a decrease of less than one loan per year.  BC consistently has the highest average number of loans per year and MB the lowest (see Figure 34 below).

Average Number of Loans	AB	BC	MB	SK	CF Average
2002-2003	15.8	22.2	13.4	17.2	18.0
2003-2004	14.1	18.9	10.7	14.7	15.4
2004-2005	15.0	18.7	10.4	19.4	16.2
2005-2006	15.5	19.1	8.1	18.2	15.9
2006-2007	13.3	18.0	9.6	15.5	14.7
2007-2008	13.7	18.2	8.9	11.8	14.3
Period Average	14.6	19.2	10.2	16.1	15.8
Average Annual % Change	-2.5%	-3.7%	-6.9%	-5.4%	-4.3%

top of page

There is great disparity from CF to CF in terms of the number of loans that each issues from year to year. Table 24 (below) provides some examples that demonstrate this variation.

Table 24 Sample of the Number of Loans Issued by CFs in each Province

	2002/2003	2003/2004	2004/2005	2005/2006	2006/2007	2007/2008
Alberta
CF1	42	31	0	6	2	5
CF2	5	10	31	12	9	4
CF3	0	4	22	5	1	7
BC
CF1	35	18	12	22	13	3
CF2	0	1	4	11	8	9
CF3	0	2	12	16	12	13
Saskatchewan
CF1	4	1	0	3	2	1
CF2	15	11	6	12	3	6
CF3	9	7	14	9	3	1
Manitoba
CF1	0	0	0	3	3	0
CF2	39	10	16	8	10	3
CF3	8	17	1	9	6	10

Several CFs did not issue any loans in some fiscal years; some issued very few loans on average (less than 2 per year); while other issued a significant number of loans. This is in part a reflection of the different challenges and opportunities, amongst CFs, within the CF program as explained in section 3.1.6. The potential reasons behind the potential decline of number of loans issued have been discussed fully in section 3.1.2, highlighting the potential association with the economic conditions in the west, the overall business lending conditions, as well as the fact that some CFs have adopted some conservative lending practices to conserve their "finite" pool of funds.

There is also a chance that the demand for loans offered by CFs is decreasing. Evidence from the client survey does not seem to support a reduction in demands. The clients (both loan and non- loan clients) expressed a high need for all services (including loans) offered by the CFs.

The increase in dollar value of loans could be an indication that CFs are supporting larger projects. This would be in line with the change in direction at WD. The large loans could also be resulting in greater revenues but there is no evidence to support this. As previously noted, follow-up data on businesses is not collected (e.g. survival rate, revenues).

Some measures are being put in place that may help to reverse this downward trend. WD has put in place minimum performance standards for CFs to ensure the CFs are active to a certain level in all three program delivery areas (loans, business services and CED). These measures combined with a discussion around acceptable levels of risk (see recommendation 3) should help reverse this trend.

It is important to note that this section was mostly focused on loan services only due in part to the availability and reliability of the loan data as opposed to data on business services and CED activities. One of the great strengths of the CF program, which enhances its cost effectiveness, is the extensive use of volunteers. Over the period 1999/00 to 2007/08, each CF was able to generate an annual average of 268,983 volunteer hours.

Furthermore, it is also important to mention that while this section focuses on the jobs created as a result of loans, the reported administrative data revealed business services have led to the creation of a further 27,000 jobs over the period 2002/03 to 2007/08. Thus, this further enhances the overall program effectiveness.

RECOMMENDATION 7:
WD should continue to monitor the trends in level of activity relative to cost to ensure that the measures put in place are having the desired affect and reversing these trends.

Alternatives

The current service delivery model is viewed as a success; there is no major appetite to change models.  Several studies and reviews have strongly supported the CF Program delivery model.

In general, interview respondents supported the current delivery model and did not see a need for change. However, some interview respondents did offer suggestions to improve the efficiency/ effectiveness of the program. The suggestions are outlined below:

• Review the number of CFs and/or the geographical divisions of the program;
• Potential for the creation of satellite offices that would be supported by a central office that houses experts;
• Long term funding;
• Improve reporting processes;
• Ensure that there is more consistency in the way services are delivered by the various CFs;
• The CF Program should be more user-friendly; which means way less bureaucratic (less forms to fill out/create on-line loan applications);
• CF Program should work more independently from WD;
• Increase the level of cooperation between Federal and Provincial Governments (i.e. on economic development, etc.); and
• Consider amalgamating the services of the CFs to a partnering organization (i.e. REDAs) to reduce overhead costs.

Based on the results of focus groups and interviews conducted, as part of the case studies, there is a general agreement that CFs need more operational funding. More immediate and long-term funding would enable CFs to: recruit and retain staff, be more visible, do more marketing of their services, and reach more areas/potential clients and communities. More specifically, remote areas of BC and AB mentioned the need for enhanced information technology capability (video conferencing) to expand and increase access to CF services to more people and more areas.

It was suggested by 6 out of 10 managers interviewed as part of the case studies that WD should consider incentives for good performers and allocation of funds based on performance. Business analysts expressed a need for more resources to be allocated to: increasing the visibility of the CFs and follow up (aftercare). There was also a suggestion for CFs' boundary review to reduce the number of CFs and potentially gain some economy of scale and maximize performance of the CF program as a whole.

Previous evaluations and reviews have strongly supported the delivery model, calling it an outstanding/unequivocal success, innovative, and one of the few signs of the federal government in rural Canada. Following are specific excerpts from some of the reports.

A study conducted by the University of Guelph in 2005 stated that:

"the Canadian Community Futures Program is an outstanding success.  It reflects the conscious crafting of a centrally supported program for endogenous development.  It effectively joins state- to- community with a strong yet flexible governance model and is a good example of place-based policy"¹⁸ .

A 2008 Senate report entitled Beyond Freefall: Halting Rural Poverty¹⁹ found that:

"The challenges of accessing credit in rural Canada have long been recognized at the federal level. Arguably the most successful program to help address this concern has been the Community Futures program, a federally funded but community-based and community-led initiative..."

"The program has been singled out by the OECD (Organisation for Economic Co-operation and
Development) as one of the most innovative and successful rural-oriented policies anywhere in the world"

"The committee believes it is important for the federal government to reaffirm its long-term commitment to the Community Futures program, one of the few unequivocal success stories in federal rural policy and...one of the few visible signs of the federal government in rural Canada. The Community Futures program success is due in no small part to the fact that it is locally run and suited to local conditions...For these reasons, the committee also believes that the federal government should consider expanding the size of loans available under the program as well as the total financing envelope for the program".

top of page

Lessons Learned

The case studies explored noteworthy practices and lessons learned, as well as areas for improvements that are detailed in Appendix I. Following are some of key lessons learned that were identified. For a complete list please see Appendix I.

From a CF perspective, there were two main lessons learned that emerged during discussion with CF managers and business analysts:

• Know the limitations of the CF - CF staff found that while it is good to stay flexible to work on a variety of projects, develop partnerships and stay connected to the community, it is also as important to know the limitation of the CF (time, money, human resources). Therefore, the CF should not over commit and should be able to say "no" when necessary.  Otherwise, the result will more likely be under deliver and lose credibility in the community. It was mentioned in many occasions that building the trust with the community is not easy and takes time but losing that trust might be easy if the CF does not deliver. The CF should understand that it is a catalyst in the community, and it does not exist to do everything for everyone.
• Clarify roles and responsibilities - before engaging in any partnership, the CF should do its due diligence work about the potential partner(s) and document and clarify role and responsibilities.

WD's regional officers interviewed as part of the case studies also noted some lessons learned. Following are some practices that WD staff have found useful:

• Relationship building with CFs - WD's relationships with CFs are best when WD employees take time to know the CF manager and the board chair and spend more time in building trust for long-term sustainable and beneficial relationships.
• Keeping communication channels open - WD keeps the communication channels open and attends the CF's functions as frequently as possible.
• Clear roles and responsibilities - WD clarifies roles and responsibilities and does not send mixed messages.
• Respecting independence of CFs - WD sometimes seems to forget - at WD's own peril - that CFs are independent offices that answer to a local Board of Directors for good reason.  They are effective eyes and ears for WD, but not branch offices.  

For a more comprehensive list and discussion of noteworthy practices, lessons learned, and areas for improvement, please refer to Appendix I.

---