Western Economic Diversification Canada
www.wd-deo.gc.ca
Common menu bar links
Design and Delivery
Appropriate Design and Delivery
Does the program design and delivery remain appropriate?
Key informants and survey recipients were often aware of the WDP prior to their being funded on the current project. They learned about the WDP through a variety of methods, including from departmental related sources, other government organizations, and various associations. Overall, the program design and delivery does remain appropriate. However, some modifications to the current design and delivery were suggested.
Survey recipients and key informants involved in projects reported useful consulting and/or counselling and advice from departmental staff, and this information was primarily used during the initial stages of a project and on project planning. Information on the WDP and the approval process was easily accessible and program officers were available to answer questions and provide helpful information about the monitoring and payment process. Funding recipients agreed that reporting time and effort was reasonable.
Suggestions for improvements primarily dealt with monitoring and measuring success, and included a focus on clarity and streamlining of the approval and reporting process and creating a better awareness of the program.
Awareness/Methods of Awareness of WDP Funding
The WDP and the department in general, are well known in the West for funding projects/ programs designed to enhance and increase western economic development and diversification. According to recipient survey respondents, 64.7% were aware of the program prior to working with the WDP.
| Method | Percentage |
|---|---|
| National or Provincial Association(s) | 28.6% |
| WD/WDP staff | 26.5% |
| WD/WDP web site | 15.5% |
| Regional office | 15.3% |
| Conference/Presentation/Workshop/Forum/Seminar/Kiosk | 13.3% |
| Community Organization | 12.8% |
| Collaboration, word-of-mouth | 11.0% |
| Some form of media (internet, news, mail) | 4.2% |
| Previous association, member of a WD network | 4.2% |
| Other methods less than 3% | 3.9% |
| Don’t Know | 8.5% |
Relationships with Departmental Staff
Key informants spoke unanimously about the usefulness of engaging in consultations and/or planning activities with departmental management or staff. Staff members were described as knowledgeable, encouraging, easy talking to, helpful, skilled, professional, interested, supportive, transparent, open, and innovative.
In terms of discussing the reporting requirements, 79.5% of the recipients mentioned that departmental officers were available to answer their questions about the monitoring and payment process. 75.5% of the recipients agreed that information on the funding approval process and the WDP was easily accessible.
Satisfaction with Process
Are program recipients satisfied with the approval, monitoring and payment process?
Survey recipients were satisfied with all stages of the funding approval process, as well as the monitoring and payment process, and felt that the management structure allowed for efficient approval and timely payment for projects. Some dissatisfaction did occur due to the difficulty with the reporting process, the amount of time it takes to receive approval, and because of the complex nature of the financial issues. Satisfaction with the process occurred as a result of key relationships and effective communication with departmental staff. Non-funded informants from the case studies reported that staff was helpful, but that they had not received sufficient information as to why they were not eligible for funding.
Key informants had some suggestions for improvement including more clarity to and streamlining of the approval and reporting process, longer funding commitments to reduce uncertainty, and increasing information around funding criteria.
Funding Recipient Satisfaction with Approval, Monitoring, and Payment Process
Overall, 76.6% of the survey recipients agreed that they were satisfied with the project screening, approval and monitoring and payment processes currently in use by the WDP. 23.3% reported satisfaction with the level of communication they had with staff (Table 6-2). Dissatisfaction largely pertained to the complicated nature of the reporting process (4.2%) and the length of the approval process (4.1%).
| Category | Percentage |
|---|---|
| Satisfaction | |
| Good communication/information, helpful staff | 23.3% |
| No issues/complaints, satisfied | 21.3% |
| Needs were met, good project, approved project, successful process | 15.4% |
| Program/process is good, timely, efficient, clear | 15.3% |
| Other reasons for satisfaction less than 3% | 1.3% |
| Dissatisfaction | |
| Reporting - too difficult, expensive, time-consuming, too much reporting/paperwork, repetitious | 4.2% |
| Approval process takes too long | 4.1% |
| Other reasons for dissatisfaction less than 3% | 14.6% |
In terms of the monitoring and payment process, 78.5% of the survey recipients agreed that the process was timely, 78.9% felt that payment was timely, and 82.7% felt that reporting requirements were adequately described. Overall, 84.1% of respondents were satisfied with the monitoring and payment process.
Non-Recipient Satisfaction with Screening and Approval Process
Non-funded respondents from the case studies reported that working with staff was helpful; however, communication was not always forthcoming after a proposal had been submitted. Furthermore, all respondents stated that they were unsure why they had not received funding, and had been left to speculate as to whether priorities had been changed, or if some other issue had prevented approval.
Modifications to Design and Delivery
Key informants indicated that the current structure is flexible and that changes have already been implemented that have improved service standards and applications. Some key informants expressed the view that paperwork was burdensome, time-consuming, expensive, and/or difficult to complete (especially for small businesses or not-for-profit organizations). According to funding recipients, auditing rules also created practical difficulties that did not promote optimal outcomes for their projects. Other issues mentioned by key informants include wait times for approval being problematic and the need to improve transparency, regularity, and funding process, in addition to creating more awareness of and publicity for the program. Modifications mentioned by key informants for design and delivery include:
- Streamlining the internal processes, such as assessment, decision-making, and monitoring, as outlined by the Blue Ribbon Panel (see Appendix E for details).
- Improving the clarity of programming and leveraging activities and allowing for cash advances;
- Modifying auditing rules also created practical difficulties that did not promote optimal outcomes for their projects;
- Longer project timelines as implementation of some projects take longer than expected or projects may not be completed as quickly as anticipated;
- A less complex application process by streamlining applications and reporting requirements especially for small organizations;
- Introduction of online applications;
- Longer funding commitments to reduce uncertainty around annual funding and improve effectiveness of long-term planning; and
- Increased information sharing around funding eligibility criteria.
Departmental Management Structure
Has the department created an effective management structure for the various sub-components and sub-agreements that are supported by the WDP?
The department has created an effective management structure for the various sub-components and sub-agreements, but stakeholders see room for improvement. The WDP has a risk management structure that works to ensure accountability and achievement of goals and outcomes, while at the same time supporting the work of partners and networks.
WDP Risk Management Practices
Projects have a risk management system in place.20 The system examines criteria such as materiality, profile, visibility, or reputation; project complexity; threats to and impacts of a project not delivering on results, in order to assess the level of risk (low, medium, or high).
Project officers try to reduce identified risks to acceptable levels during the due diligence stage. Project officers will stipulate special conditions in the contribution agreement that will mitigate those projects with higher risks. These stipulations may occur in the initial stages of a project or be on-going.
If risk factors cannot be reduced or mitigated through contribution agreement stipulations, the project may not proceed. However, if a project is high risk, it still may be approved depending on the types of significant benefits that may be generated and the mitigating factors that the project officers can put into place.
System of Reporting on Outcomes
The project file review showed that DDR's and PAR's are completed for all projects. These DDR's and PAR's contain project-based outcomes measurement systems, which are built into contracts. In addition, the files contain progress and final reports from projects. Management and staff reported that reporting is part of a contractual agreement and that each project goes through a development and assessment process, and a tracking system allows for descriptions of outcomes, baseline indicators, incremental results, and expected outcomes. Both management and staff mentioned the usefulness of the Project Assessment Tool and Project Gateway in following a project throughout its life cycle.
Gaps in Information Provided or Collected
Key informants expressed the view that that the biggest single gap in reporting mechanisms is long-term follow-up and looking at long-term impacts of the department's investments under the WDP. Information pertaining to in-kind contributions was not always identified from the files. This type of information could be a significant form of leveraging for the evaluation purposes. This observation is supported by the file review, which indicated minimal follow-up of projects after departmental funding ends. A number of progress reports and final reports from the file review did not routinely address the indicators outlined in DDR's.
On improvements to the reporting system, key informants provided the following suggestions:
- Building capacity, especially in smaller organizations to enhance reporting requirements.
- Providing clarity on reporting requirements especially on outcomes, indicators, timelines, milestones and financial models.
20 Audit and Evaluation Plan 2006-2009: Risk Assessment System. Retrieved March 13, 2008.
