Observations and Recommendations

Management Controls

Audit expectations were:

• That management had an appropriate framework in place to set program direction, precedents, procedures, program objectives and priorities;
• That a contribution agreement setting out the terms and conditions of the project existed;
• That a sound project assessment review framework with decision sign-off stages was in place and well understood by project officers;
• That eligibility of projects, graduates and documentation was communicated and appropriately applied throughout the project cycle
• That authorities and limitations were structured, clarified and communicated;
• That management compared results achieved against expectations on a periodic basis; and
• That program evaluation activities were used to identify program strengths, weaknesses and impacts as well as alternative ways of program delivery.

Management Framework

The FJST program was reasonably managed. There were appropriate controls for screening and approving projects. While procedural memos, precedents and instructions existed in various forms in the regions and headquarters between 1997 and 2002, key uniform strategic directives were initiated March 2002 onwards by the Deputy Minister and the program-operations directorate to guide the renewal and administration of the FJST program until its elimination in 2005. Implementation, administration, eligibility, cut-off dates, promotion and procedural rules were streamlined and clarified in ensuing memos. Such documentation provided the authoritative design and reference sources for staff to effectively and consistently deliver the program.

Projects examined were appropriately approved within the time limitations and within the dollar limits set. Projects and graduates were found to be eligible and met the program criteria. Corporate registry searches were done where warranted. Clarifications were sought where needed. In some cases, the facts were noted on file, while other times there was only a diary that client was spoken to without further details.

A formative evaluation of FJST was done in 2004. The evaluation concluded that the program was relevant, making impacts and contributing to success for small and medium enterprises. There were differing views on whether the program should be delivered by WD or by a third party.

Documentation for more than twenty audit program points that were tested was generally strong. However, 12 of the 64 sample project files had at least one issue associated with the examination against the various audit program points reviewed for each file. Of particular note, at the initial stage of hiring it was sometimes hard to verify the under-employment of the graduate and there was no evidence of file to confirm this fact. In speaking with program officers, many felt that if the employee was willing to accept this job at that salary, then they must be under-employed. Management considered this program, overall, to be a low risk program. As such, WD did not require any further evidence on file to verify the under-employment status of graduates.

Table 1 summarizes the main points from the audit verification results of the sample files examined:

Description	Total	Percentage
Number of active projects in audit period	402
Number of sample projects examined	64
Percentage of sample size to population files		16%
Project Files examined by audit	64	100%
Project Assessment Review Risk Ranking of sample
Low	56	88%
Medium	6	9%
High	0	0%
No assessment found on file - outside audit period	2	3%
Audit Exception Summary:
Files with exceptions to the file review criteria established	5	8%
Files with an unusual matter without clear documentation to support resolution of the matter on file	12	19%

The challenge was that over 88% of the projects examined were rated as low risk. It was apparent from files reviewed that the risk assessment completed was weighted heavily on the fact that the maximum contribution of $20,000 was deemed as being of low significance compared to the higher contribution amounts regularly funded under other WD programs. None of the projects were assessed as high risk, and only 9% were rated as medium risk.

For independent audit analysis purposes, the majority of the clientele served by the program were small and medium enterprises, which often have limited resources to apply to the administration of government programs. That factor in itself becomes an inherent risk of high concern for any direct government assistance program. There have been examples in the past where other government departments have not adequately mitigated these risks.

Internal Audit considered that the 88% rate of low risk ranking across the population was likely far too low considering all the available risk indicators. The low dollar value may have weighed too heavily in the risk assessment.

Overall, the opportunities for improvement were assessed as low risk given consideration to significance, the program's elimination in 2005, and potential impact given that WD currently does not deliver any similar programs. Continuous improvement efforts have already taken place based on recommendations from recent audits around the overall management and administration of grants and contribution programs. Therefore, the recommendations listed will only apply to future programs of similar design and delivery.

Recommendation #1:
If and when WD establishes similar programs in the future, project specific risk should be reassessed periodically. Risk is not static and needs to be revisited throughout the life cycle of the project.

Monitoring and Results

Audit expectations were:

• That external and internal environments were monitored to obtain information that may signal a need to re-evaluate risks and WD's objectives;
• That appropriate and timely progress reports and claim documentation was communicated;
• That financial and non-financial reporting was reviewed and approved;
• That the rigour and evidence of monitoring was commensurate with risk level;
• That processes were in place to monitor financial and operational performance on an ongoing basis; and
• That adequate and appropriate documentation supported claims.

There were appropriate controls for disbursing payments to recipients. Pay stubs, T4s and payroll registers were used to validate salaries as appropriate. Progress reports were requested from recipients and were received. Prior to payments progress reports were requested and were received. Payments made were within the limits indicated in the project approval records. Where funds were not entirely spent, they were de-committed. Companies met their obligations under the WD terms and conditions.

There were notes on file about consultations with supervisors and reasons for decisions made. In a few instances though, rationale for disposal of unusual situations could have been better documented and these conditions are discussed in more details below.

In more than 35% of the samples tested, claims were not filed quarterly, but rather at the end of the project where the entire qualifying amount was then being claimed. A number of the graduate employees had already left the company before the claims were submitted. This would have hindered any opportunity for WD to contact the graduates to confirm that program objectives were still being met.

There was no evidence of site visits by WD officers to validate progress reports. There was little documentation on file to confirm that alternate procedures, such as site visits or interviews, had been done to minimize the risk of misrepresentations in the progress reports. There was heavy reliance on the companies receiving the money to accurately state the duties of the person being hired. There was little verification done by WD to confirm that the employer's claims represented what the employee was actually doing and was still eligible for funding.

For those projects ranked as medium risk in the sample selected, there was no documentation on the file to suggest that their claims processes adopted were any different from those applied to the low risk files. The same standardized claims review method applied whether the recipient had been rated low or medium risk. Despite the fact that some applicants were first-timers and others were repeat applicants, there was nothing on the files to indicate any more or less rigour of monitoring.

Quality and completeness of the documentation on file could have been improved. 19% of sample files tested were missing one thing or another. These included:

• Three instances where limitation on project amounts had changed but this was not realised until the last claims;
• Nine instances where situations of under-employed and its relation to starting salary or salary rate at last employment seemed questionably subjective without insufficient documentation on file;
• Four instances files were not updated with completed amendments and schedules but just preceding emails correspondence;
• Four instances where graduates moved from one company to another separate company and thus effectively they were under FJST for over 12 months; and
• Four instances where graduates were hired before degrees were actually obtained.

WD could have tightened up its due diligence and monitoring processes to ensure that unusual circumstances encountered during the life of the project were fully documented to provide an adequate audit trail.

Internal Audit found that there was no requirement for final project reports to analyze and demonstrate accomplishments to ensure linkages to program objectives and results.

From the files reviewed, it appears that delegation of the project assessment and monitoring to less experienced staff was the norm. However, there was no evidence on file that senior staff had reviewed and agreed with the assessment and the level of monitoring action taken even though claims and project assessment review forms were signed by the appropriate level of authority.

Recommendation # 2:
WD should tailor its due diligence and monitoring processes to be commensurate with the level of risk assessed.

Recommendation # 3:
WD should ensure that files contain adequate documentation when unusual circumstances are encountered. At a minimum, there should be documentation on file to both support and explain any key decisions rendered on the project by an appropriate level of management.

Conclusion

In summary, the controls established for the FJST program by management were reasonably effective in design and operated satisfactorily. Based on internal audit standards and criteria selected, sufficient audit work has been completed and analyzed to support the conclusion that the management control framework for which WD management is responsible was appropriate for the size and risks of the FJST program.

In accordance with the Government of Canada internal auditing standards, a reasonable assurance is provided that relevant and reliable management controls existed for the delivery of the FJST program from 2002 through to 2005 when the program ceased.

In addition, opportunities for improvement have been identified in this report where those controls can be strengthened for future programs of similar design and delivery.