Findings and Recommendations

This section provides the audit observations and recommendations as well as a conclusion with respect to the audit objectives previously presented. The findings are presented with respect to the three components of the Transfer Payment Life Cycle, which are: Program Design, Program Operations, and Review and Evaluation.

Program Design

As per Sections 3.5 and 3.6 of the Policy on Transfer Payments, “government is committed to ensuring that transfer payments are managed in a manner that respects sound stewardship and the highest level of integrity, transparency, and accountability. Moreover, the government is resolved to ensuring that transfer payment programs are designed, delivered and managed in a manner that is fair, accessible and effective for all involved”. Based on these criteria, the audit assessed the adequacy of the organizational structure, the delivery framework, the funding allocation process, program Terms and Conditions, and program dissemination.

In addition, and in light of the upcoming renewal of WDP Terms and Conditions, the audit also focused on program design. The renewal process is considered as important as the design of a transfer payment program. Management must consider if the program still meets the need for which it was originally established, and if it is still in line with the evolving strategic objectives of the organization.

Organizational Structure

The department uses a decentralized delivery approach for the WDP. Each of the four program-delivery regions has an Assistant Deputy Minister (ADM) and a Director General of Operations responsible for the delivery of regional WDP projects. Each region has evolved independently from each other, setting differences in their structure to meet local delivery issues and requirements.

The department operates two primary organizational models: one model in Saskatchewan and one in the remaining three regions. The Saskatchewan model consists of having Project Assessment Officers cover project management from start to end. In the other three regions, Project Officers manage the initiation of the project and then Monitoring and Payments Officers manage the file once the project has been approved for funding. In addition, there is no peer review system in Saskatchewan to ensure quality of work done by program officers, as used in the other regions. The risk associated with the weaker segregation of duties in Saskatchewan, is compensated, however, by the fact that Project Officers have no approval authority.

The Saskatchewan model was adopted as a pilot project four years ago. It likely is time for the Executive Committee to assess the pilot project and either endorse this model or move to a standard delivery structure across all regions.

Recommendation # 1: The Executive Committee should review its two delivery models and determine whether or not to standardize regional delivery structures across the department.

Financial Reporting For WDP Funding

Appropriation vote information for Grants and Contributions is provided in the DPR at the departmental level. Information on spending is provided by strategic outcomes in the department’s DPR and by program in a table to the DPR.

Our interviews reveal however, that the department showed lapsing of funds at the departmental level in all years from 2003-04 to 2007-08. Lapses were minimal for the core funding with the exception of 2005-06 where we note an over-expenditure of $1.6M as per the 2005-06 DPR (this over-spending was just at the program level, as the total Grants and Contributions vote showed an overall surplus).

Headquarters Finance prepares a continuity schedule that tracks funding changes by region throughout the year. Spending is managed in a manner that maximizes spending for all programs included in the vote approved by Treasury Board. Finance provides financial information to the Executive Committee on a regular basis at a consolidated departmental level, but no details are provided down to the program level.

Reporting is done by the Program Activity Architecture for Western Economic Diversification Canada and the WDP spending is aligned with departmental strategic outcomes in accordance with the Treasury Board Policy on Transfer Payments.

Program Dissemination Strategy

We noted that program dissemination is mainly done through the departmental website, business centers, network partners and other small partners, and other levels of government. These partnerships also allow the department to cost-share initiatives that respond to regional needs and opportunities. For example, Western Economic Partnership Agreements (WEPAs) are multi-year funding commitments to strengthen economic activity and improve quality of life in western communities. WEPAs are cost-shared equally with each of the four western provinces, with a total of $200 million allocated to initiatives identified as federal and provincial priorities.

The departmental website provides very good information on the Western Diversification Program. For example, it provides answers to the following questions: What is the Western Diversification Program? What are the objectives of the WDP? Who is eligible for assistance? What Activities are supported by the WDP? What expenses are eligible? How to apply?

Given the focused and targeted nature of the WDP, the department appears to have well established outreach with its potentially eligible recipients. Under its current design, this program dissemination strategy appears appropriate.

Program Renewal Process

The process for renewing the program was informal, but did bring together the key departmental stakeholders. Corporate Programs, as renewal champion, made a presentation to Executive Committee in June 2008 on the WDP renewal. At that meeting, the Executive Committee decided to proceed with a status quo renewal of the WDP authority. That was the only option offered to the Executive Committee in the presentation.

The department was able to rely on the October 2008 program evaluation of the WDP to assess the continued relevancy and impact of the program. The evaluation concluded that the WDP was well aligned with the Government of Canada and departmental priorities. The evaluation indicated that the impacts from the WDP contributed to all three of the department’s strategic outcomes in some fashion. In addition, the evaluation recommended that the department maintain the current flexibility of the WDP.

top of page

Program Operations

The basis of a transfer payment’s program operation is the approved Terms and Conditions. It is the Terms and Conditions that set the eligibility criteria, the basis for approving applications, performance measurement indicators, the reporting mechanisms, and program dissemination/transparency and fairness. Adequate delivery of a major contribution program requires that resources have adequate competencies and expertise (training), that adequate tools are provided in order to efficiently manage projects, and that effective and efficient delivery processes (approval, decision, segregation of duties) and procedures (site visits, committees, follow-up reports) are in place.

In addition, the Policy on Transfer Payments sets the requirements as to how to manage and ensure stewardship over public funds.

Through our interviews and file reviews, we noted that the contribution agreements properly reflect the approved Terms and Conditions as well as Treasury Board Policy requirements.

We also noted that the project selection and approval process is well structured and documented. The process includes regional G&C committee meetings throughout the approval cycle, with the exception of Manitoba which has adopted a consultative process. Adequate procedures are in place and training ensures adequate application controls over the review and approval process.

Project Start Dates Prior to Agreement Signature Date

As per section 8.2.5 of the Treasury Board Secretariat Guide on Grants and Contributions (which was in place for most of the audit period – up to October 1, 2008), all parties should sign a written agreement before the stated start date, and, more importantly, before eligible expenses are incurred. We noted, however, that a large percentage of project start dates preceded agreement signature dates. Our file review revealed that 77 of the 94 (82%) contribution agreements reviewed had start dates prior to signature dates, and that project expenditures started on average between 68 days to two years prior to the signature of the agreement.

The main cause for start dates preceding the signature of agreements, as noted by the program directors, is the lengthy approval process. They noted that the lengthy project approval process resulted in the department issuing a Project Start Date Policy (March 31, 2006) that requires an explanation for an early start date. This policy, however, has not reduced the frequency of having start dates before signature of agreements. In fact, 22 of the 25 files reviewed (88%) after the introduction of the departmental Policy had start dates prior to signature dates. The 88% frequency is higher than the 82% frequency for the whole audit period. This March 2006 policy talks about exceptional circumstances for using early start dates and puts emphasis on the fact that as much as possible start dates should be after the signing of the Contribution Agreement.

Our file review revealed that all exceptions are well documented; however, with an average of 82% of the projects started before the contribution agreements are signed, these can no longer be considered exceptions. Although this practice does not contravene Treasury Board Policy, it is inconsistent with the principle of the 2006 departmental policy that envisioned these cases as exceptions.

Recommendation #2: The Department should assess the risk associated with the current practice of having a high frequency of start dates occurring before signature of the agreements, and take action to reduce such risks by addressing the cause of the issue.

File Documentation and Rejection of Applications

While files were, in general, well maintained, we found many cases of duplication of documents on the project files. Although it is important to have all information on file, excess documentation could lead to inefficient information retrieval and inefficient storage. The audit team noted that a File Contents Checklist was developed by headquarters, however, there is no policy or procedures that determine the quantity and quality of documentation required on files and there is no monitoring of files documentation by supervisors. This leads to varied and inconsistent documentation practices.

We also noted a lack of proper documentation and audit trail supporting the reason for the rejection of applications. As stated in WD’s Grants and Contributions Internal Control Framework, all applicants whose proposals were rejected are to be notified in writing. Decision letters were found in a minority of cases, and the level of documentation found in files for rejected applications was inconsistent. In addition, there is no reporting done on rejected applications, such as rejection rates, reasons for rejection, comparison with other regions and prior years. The lack of proper documentation leads to informal reporting and increases the risk that management does not have accurate and complete information.

Furthermore, Project Gateway is ineffective for managing documents, as signed documents are not scanned in the system leading to retrieval of hard copy to validate agreements and amendments.

Recommendation # 3: The Department should develop minimum documentation and communication standards for rejected applications.

Compliance

The Delegation of Financial Signing Authorities (Summary of Delegated Financial Signing Authorities matrix) has been approved for all departmental programs funded from regional budgets outlining levels of authority at each level of the organization. However, for the last year of operations, the Minister signs-off on all Project Approval Records (PAR) before a project can be authorized. Therefore, in actuality, all project applications now require approval at the Minister level.

We noted that Payable at year-end (PAYE) set-up for contribution payments is not uniform across all regions. In several cases, adequate documentation was not available to support that PAYE was set-up in accordance with departmental policy. PAYE balances still existed as of March 1 2009, indicating that aging exceeded 11months. In one region, adjustments of $586K were made to PAYE balances in 2008-09. If PAYE balances are over-stated at year-end and require subsequent adjustment, then that money is lost to the department and could possibly have been reallocated prior to year-end to fund other eligible projects.

The department had over $24M in repayable contributions and since the department has provided a limited number of repayable contributions since 1995, many of the receivables are at least 14 years old. These receivables are tracked and repayment agreements are monitored regionally. The status of some of these receivables is not clear and needs to be determined. We noted that some projects have had no action for over seven years. Repayable contributions will be the subject of a separate internal audit on accounts receivable later in 2009-10.

A sample of 97 WDP project files revealed an error/omission rate of approximately 9% (9 of the 97 project files reviewed) for a total of 11 errors. This rate excludes the start date issue (see section 3.2.1) which is present in the majority of the files. From the sample, we noted:

• Payment of invoice prior to start date in two separate files (non-compliance);
• Advance made in one case when should have been a reimbursement or set up as a PAYE (over 90% of funding);
• Lack of supporting documentation in three separate files;
• Payment without adequate supporting documentation on file;
• Three cases of non-compliance to claim review process; and
• One case of a due diligence report without approval, although the project approval record had all the required signatures.

The types of errors could be reduced by enhanced financial training for project officers and managers, combined with adequate monitoring by Corporate Finance to ensure that financial policies and procedures are being followed. Employees have been provided with training documents for Project Gateway and project management, but very little training information was offered on financial management.

Recommendation # 4: Corporate Finance needs to verify that PAYE balances are being set up in accordance the Financial Administration Act and departmental procedures.

Recommendation # 5 Corporate Finance should consider enhancing its financial training for project officers and managers and its active monitoring of project files for key compliance attributes.

top of page

The Recipient Audit Framework

The contribution/recipient audit framework is documented in the WDP Risk-Based Audit Framework (RBAF) dated June 15, 2003. It clearly describes regional audit responsibilities, risk criteria, scope and audit standards. In March 2004, the department updated its recipient audit policy.

The regions conduct a risk assessment for all projects. Those projects that are ranked high are automatically subject to a recipient claims audit. Those projects that are ranked medium or low are then pooled together and headquarters selects an additional sample for recipient claims audit. Headquarters does make an effort to ensure the sample does have some reasonable regional distribution.

The present sampling process does not take into consideration the recommendations from the Independent Blue Ribbon Panel with regards to the requirement for departments to consider a single recipient audit approach. Some recipients, as noted in the project files reviewed, get funding from more than one source (Federal, Provincial, Municipal government etc.). There is no evidence that consideration has been given to working with other governments or departments to meet the requirement of the single recipient audit approach. Management should review its current recipient audit policy in light of the new Treasury Board Policy on Transfer Payments and the Independent Blue Ribbon Panel, both which have been introduced since the last update to the departmental policy in 2004.

The samples for audit are sent to regions for organizing the work with Audit Services Canada (ASC) as contributions audits are all contracted to ASC through a Memorandum of Understanding (MOU). Audits are not done in a timely manner. For example, in British Columbia, there is presently a backlog of more than two years. Furthermore, the department has never compared the cost of doing audits through ASC against a competitive bid process.

Based on claim audit reports reviewed, auditors provide a written opinion on whether claims are in compliance with the terms of the contribution agreements; however, the opinion is provided on a claim by claim basis rather than by recipient or the complete project. As a compensating control, Monitoring and Payment Officers conduct detailed reviews of claims and reconcile them in total for a complete project. As part of the review of the recipient audit framework, management should assess whether recipient claims audits and the reviews done by Monitoring and Payments Officers are complementary or duplicative controls.

An audit log is kept at Headquarters; however, it is not clear who, at headquarters, is responsible for tracking the results or for following up with the regions on the issues identified in the recipient claims audits.

No evidence was found that the management process requires that audit findings are summarized and reviewed at the corporate level, and that lesson learned and resulting changes are communicated in the regions and taken in consideration for training.

Recommendation # 6: The Department should review its 2004 Recipient Audit Policy in view of the new Policy on Transfer Payments and the Independent Blue Ribbon Panel report.

Recommendation # 7: The Department should reassess the memorandum of understanding with Audit Services Canada to ensure fair value for money is achieved and timely service is rendered.

Performance Reporting and Monitoring for WDP

The WDP is the main program through which the department makes strategic investments in initiatives that enhance and strengthen the economy of Western Canada. Increasingly, the WDP is used to collaborate with others and is designed to respond to economic priorities.

Eligibility criteria, as per the present Terms and Conditions of the program are very wide. This has benefits for management since they can address many different needs of Western Canada through the WDP. In terms of performance reporting, we found that program objectives are stated in terms of resource expenditure, activities undertaken, number of projects approved (outputs), rather than in terms of outcomes (short term, midterm and long term). Currently, senior management only gets regular reports on performance against departmental strategic outcomes, not against the objectives and results of the WDP. This is not surprising given the fact that WDP contributes to all three of the department’s strategic outcomes and represents approximately 70% of the department’s core grants and contributions budget.

However, management should collect quantitative and qualitative information to monitor how the program is doing, rather than just how the department is doing. Non financial reporting (on output and outcomes) is presently done by Program Activity Architecture at all levels, but there is no reporting (except on program spending) against the program objectives of the WDP.

As a result, management may not have the adequate program-specific information to support decision making (i.e. improvements, renewal of the program etc.). Presently management can only rely on the evaluations done every five years to assess the relevance, cost-effectiveness and success of the program.

Recommendation # 8: The Executive Director Policy, Planning and Performance Measurement needs to assess whether the WDP performance monitoring framework is adequate to report on results against expected program objectives.

Resources to Effectively and Efficiently Deliver WDP

For the most part, we noted that project officers in all regions have proper resources, competencies, and training to appropriately deliver most projects. However, for large and more complex projects, there is a need for broader expertise and a more complex project management process. When required, WDP assessment officers bring in outside experts to provide advice on large or complex projects.

The main tool for managing and ensuring due diligence for WDP projects at all levels is an in-house system called Project Gateway. This system has evolved to an information management system for the full life cycle of projects. Project Gateway provides very good document templates, but has yet to become a fully integrated project management tool. The inability of Project Gateway to provide for overall project management has led regions to build their own Master Project Inventory Control using an Excel spreadsheet.

In addition, Project Gateway does not contain the most recent/signed documents, thereby, providing incomplete documentation. The maintenance of “black book” systems by officers and regions increases the risk of duplication of information, errors in different data bases, and does not provide readily available information for decision making.

Recommendation # 9: The Department needs to review all of its users’ business needs to see if they are currently being met by Project Gateway. Specifically, the Department should take the necessary steps to increase functionality so that project officers do not need to create and rely upon supplementary systems (e.g., spreadsheets).

top of page

Review and Evaluation

The Deputy Minister must conduct audits or reviews to assess whether requirements of the Policy on Transfer Payments and its supporting directives have been met; that corrective actions are taken; and to properly report on the results achieved.

Internal Audit and Program Evaluation

The June 15, 2003, Risk-Based Audit Framework (RBAF) developed for WDP describes the distinction between ongoing program management and monitoring, and the role of the Audit and Evaluation Branch responsible for the internal audit function. The ongoing monitoring and project audits of the transfers covered by this RBAF is undertaken by the management of WDP while the conduct of periodic audits related to the internal controls of contribution systems and processes is the role of internal audit. In exceptional cases, or when requested, A&E may become involved in project audits of the books, records, operations of recipients on a sample basis, but program management has the primary responsibility for the audit of recipients in keeping with the Policy on Transfer Payments. More specifically, the RBAF outlines the Internal Audit Department’s responsibilities as follows “With respect to audits of the WDP in particular, A&E, on behalf of the Deputy Minister, manages and coordinates the program audit process. A&E is responsible for the overall audit of the program in terms of management policy; practices and controls of all transfer programs and help to ensure compliance by the Department with the Treasury Board Policy on Transfer Payments. A&E is not responsible for the monitoring or audit of individual recipients’ projects or claims”.

This is the first comprehensive audit of the Western Diversification Program. Since 2003, we have seen many audits of the department-wide transfer payments but no specific audit of the WDP. Other audits were performed on the Grants and Contributions Management Control Framework for WD, such as:

• In July 2006, Internal Audit issued a report on the Audit of Financial Management of G&C;
• In July 2006, Internal Audit also performed an audit of Transfer Payments;
• In July 2007, an audit of the administration of G&C was performed; and
• In September 2008, the Audit, Evaluation, and Disclosure Branch performed an Audit of Financial Management Controls.

Each of these audits contained a representative sample of WDP projects or covered processes that applied to WDP projects.

We also noted that none of the audit reports cover the topic of performance measurement for the WDP itself. As for the program evaluation, it appeared to be very comprehensive and provide adequate program coverage, although actions to modify the program are mostly targeted to renewal, and not to program delivery improvement. The WDP was most recently evaluated in October 2008.

top of page

Conclusion

Other than the opportunities for improvement identified in the recommendations in this audit report, the Western Diversification Program has adequate overall delivery, operations and review processes that are working effectively. Sufficient audit evidence was gathered in accordance with the Government of Canada internal audit standards in order to come to this conclusion.