Western Economic Diversification Canada
Symbol of the Government of Canada

Common menu bar links

Challenges and Limitations

Obtaining the population lists of clients from Capital Providers had been a major challenge.  We contacted a sample of 14 Capital Providers that disbursed loans and investments from 21 funds during the time frame covered under this evaluation.  The majority of Capital Providers had privacy agreements in place with their respective clients, preventing them from sharing client contact and financing information with a third party.  In order to comply with the privacy covenants, some Capital Providers had to contact their clients individually to obtain client consent prior to releasing contact and financing information to us.  Some other Capital Providers sent mail-outs to give their clients the option to opt out of our survey by a certain date prior to releasing contact and financing information to us.  As a result, the process of obtaining client population lists was prolonged.  In addition, it had been difficult to locate the appropriate Capital Provider staff for the funds that had expired. In many cases, the staff involved with the expired funds were no longer working at the Capital Provider organizations, making it challenging to locate the documents related to the expired funds.  Furthermore, for pooled funds like the Community Enterprise Investment Fund and the Growth Start Fund, it was necessary to contact each participating Community Futures office individually to obtain client contact and financing information.  While obtaining the population lists of clients took longer than expected, the delay was necessary to ensure a large enough client sample size in order to extrapolate the survey findings.

The full economic impact of the program (in terms of revenues, employment, wages and exports) cannot be assessed from the data the program routinely collects.  Although the client survey data provided some relevant sample information, the lack of comprehensive revenue and business survival information precluded any accurate estimation of economic impact.   Until appropriate data is collected at baseline and at the end of the period for which the client receives assistance, we cannot measure success related to the intermediate outcomes outlined in the program’s Performance Measurement Framework; the intermediate outcomes commit to increased revenue, employment and exports among small/medium sized businesses funded under the Loan and Investment Program.  Furthermore, the LI Program is one of many factors influencing a company’s success and this evaluation investigates the contribution of the LI Program to the achievement of outcomes. 

Another limitation or challenge was the length of time since the loans and other assistance were provided to some clients.  The elapsed time made it difficult for some clients to recall certain data such as the total amount that was invested in their business at the time the loan was received or other details such as their revenues prior to receiving the loan.  However, for most issues, clients were able to recall the importance and impacts of the loans and other assistance provided. 
      

Structure of the Report

This report is divided into three chapters.

  • Chapter I provides an overview of the evaluation in terms of its background, purpose, and methodology including challenges and limitations.
  • Chapter II presents the major findings of our evaluation by the core five evaluation issues related to continued need for the Program, alignment with government priorities, alignment with federal roles and responsibilities, achievement of expected outcomes, and demonstration of efficiency and economy.
  • Chapter III outlines the recommendations arising from our review.