Audit of Information Management and Technology Governance

Executive Summary

The Government of Canada (GoC) established the Department of Western Economic Diversification Canada (WD) in 1987 as the federal regional economic department for western Canada. WD’s corporate headquarters is located in Edmonton, with some corporate functions located in Ottawa. Regional offices are located in Vancouver, Edmonton, Saskatoon and Winnipeg. WD has approximately 400 employees.

Information Management and Technology (IMT) is a key enabler for WD, as all staff deal with information and/or technology on a daily basis. IMT is largely managed by the IMT Directorate at WD headquarters as the Directorate is responsible for both technical infrastructure and information management (IM), and has a total complement of approximately 24 staff led by the Director, IMT and Chief Information Officer (CIO). In addition to IMT Directorate staff, there are 14 regional IMT staff that report to regional management (the regional Manager of Finance) and have a functional reporting relationship to the CIO (although no direct reporting relationship).

IMT governance processes at WD are intended to provide the structure that links IMT processes, IMT resources, and IMT information to WD’s objectives and strategies. IMT governance processes are engaged in by management of the IMT Directorate through day-to-day job performance and participation on relevant committees, and by senior management of the department through membership on relevant IMT governance committees.

There are two governance committees that share the main responsibilities over IMT oversight within the department:

• IMT Council: a cross-representation of departmental business areas and regions with vested interest in the major initiatives and client relationships undertaken by the IMT directorate. The IMT Council is responsible for assessing business cases for major initiatives, setting priorities and making recommendations to Executive Committee for unallocated funding requirements in partnership with the respective Business Project Sponsor.¹
   
• Executive Committee: the departmental standing committee responsible for: providing overall strategic and management oversight to all departmental activities; ensuring that the long-term strategic outcomes of the department and regions are achieved; and, setting priorities for the department to achieve these outcomes.²

The objective of the audit is to determine whether WD has in place an effective IMT governance framework and practices that meet GoC and WD policy requirements.

Key Findings

Findings should be viewed through the context of current IMT -related developments, both within WD and the broader federal government environment that may have a significant impact on WD’s IMT Governance and the audit’s findings and recommendations.  Specifically, this includes:

• The current fiscal environment, including the Federal Government’s Strategic and Operating Review, means that the availability of WD IMT resources may be constrained.
   
• The Government of Canada (GoC), through the new Shared Services Canada entity is intending to streamline and consolidate federal government IT infrastructure and operations, particularly email, data centres and networks, and which may limit WD’s ability to directly control IMT resources.

Strengths

Throughout the audit fieldwork, the audit team observed examples of how controls are properly designed and being applied effectively, including those listed below:

• WD has recently improved processes over IMT governance significantly, including; revising the IMT Council’s Terms of Reference to provide clarification of the IMT Council’s mandate, and instituting quarterly reporting from the IMT Council to the Executive Committee.
   
• Adequate controls exist over IMT budgeting. The IMT Budget is prepared annually by the CIO / Director of IMT and approved by the Executive Director Finance and Corporate Management. On a monthly basis, the CIO prepares a detailed variance analysis of forecasted budget versus spent / committed budget.
   
• Staff and management of the department have expressed that they are satisfied with the level of IMT service desk support provided by IMT staff.

Observations

The audit team also identified areas where management practices and processes can be improved. The following are observations made by the audit team that highlight areas of improvement that should be addressed by WD.

1. There are no formal processes to ensure that IMT planning is fully integrated with the department’s planning processes, and IMT planning is currently performed with limited input from stakeholders across WD (i.e. regional stakeholders, senior management, Planning and Reporting Committee). IMT capacity and capabilities have not been formally assessed to determine if they are appropriate to achieve the IMT priorities of the department. Furthermore, IMT risk management is performed informally at WD, as an IMT risk assessment framework is not in place, and, for example, a risk assessment is not formally performed as part of annual IMT planning.
    
2. Formal performance measures for IMT have not been defined. Some informal monitoring of IMT performance with regards to IMT projects and ongoing operations has been performed within the IMT Directorate; however, there has been limited monitoring / reporting on IMT performance to governance committees.
    
3. Processes over prioritizing IMT projects are maturing as the IMT Project Priority Setting Framework has recently been established; however, evidence of justification of IMT project rankings is not retained, and cost analysis performed as part of prioritization processes does not consistently include all costs associated with IMT projects.
    
4. While WD has a limited number of large projects, oversight processes over IMT projects are not currently formalized at WD. Some oversight of IMT projects has been performed by IMT Council and the Executive Committee; however clarity could be improved over the Committees’ oversight expectations for large, medium or small projects. In addition, change management processes have not been formally defined to provide clear guidance on ensuring significant changes go through appropriate project oversight processes.

Recommendations

1. A formal IMT planning process should be established that allows for integration with departmental planning. Furthermore, planning processes should engage all relevant stakeholders (including regional IMT contacts, senior departmental management, and the Planning and Reporting Committee), and should be periodically monitored by management.
    
2. An IMT risk management framework should be developed that allows WD to identify strategic and operational risks related to IMT , develop mitigation strategies/controls for identified risks, and regularly report on the status of mitigation measures. Furthermore, IMT management should be formally engaged in the departmental planning process to help ensure that IMT risks are considered as part of departmental planning, and that the IMT Directorate is aware of risks at the departmental level that are relevant to IMT activities.
    
3. Formal performance measures should be established for all key IMT activities, including: operational service delivery, key IMT projects, and priorities identified within the IMT annual plan. IMT performance reporting processes should be established that define IMT performance reporting to IMT management and relevant committees at scheduled periodic intervals.
    
4. Some level of justification of IMT project rankings should be provided to the Executive Committee to facilitate the effective oversight and approval of IMT resource allocation decisions.
    
5. Projects costs that are used to perform cost-benefit analysis and produce IMT project rankings should consistently include all relevant costs, including: full time employee labour (at headquarters and regional locations), and post development maintenance costs.
    
6. A formal IMT project oversight framework should be established that helps ensure that adequate oversight is performed consistently for IMT projects. The framework should provide guidance on ongoing monitoring of the performance of IMT projects with regards to cost, quality and schedule and be supported by a documented change management process. Furthermore the framework should clarify the reporting requirements to each governance committee depending on the importance of a project (i.e. large, medium, low value/risk).

Statement of Assurance

In my professional judgment as Chief Audit and Evaluation Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the conclusion provided and contained in this report on the effectiveness of WD’s IMT governance framework and practices.

The assurance is based on a comparison of the conditions, as they existed at the time, against pre-established audit criteria that were agreed upon with management. The assurance is applicable to the policies and processes examined. The evidence was gathered in compliance with Treasury Board policy, directives, and standards on internal audit for the Government of Canada. The evidence has been gathered to be sufficient to provide senior management with the proof of the conclusions derived from the audit.

Conclusion

In conclusion, the IMT governance framework and practices to manage IMT in accordance with relevant acts TBS and Departmental policies, procedures and practices, has control weaknesses with moderate risk exposures that require management attention, related to strategic planning; risk management; performance measurement; project priority setting processes; and, project oversight processes.

Donald MacDonald

Chief Audit Executive
Department of Western Economic Diversification

Audit Team Members

John Hagan
With the assistance of external resources

____________________

¹ WD IMT Council Terms of Reference
² WD Executive Committee Terms of Reference

Introduction and Context

Background

The Government of Canada (GoC) established the Department of Western Economic Diversification Canada (WD) in 1987 as the federal regional economic department for western Canada. WD’s mandate is to promote the development and diversification of the economy of western Canada (Manitoba, Saskatchewan, Alberta and British Columbia) and to advance the interests of western Canada in national economic policy, program and project development and implementation.  WD’s corporate headquarters is located in Edmonton. Regional offices are located in Vancouver, Edmonton, Saskatoon and Winnipeg. WD has approximately 400 employees.

The Information Management and Technology (IMT) Directorate is responsible for both technical infrastructure and information management (IM).  Roles are responsibilities within the Directorate are divided within the following two groups:

• IM team - responsible for corporate applications and databases along with the corporate Help Desk; and
   
• Technology Services team - responsible for managing the IT infrastructure of the Department, including: servers, network, and video conferencing equipment.

The IMT Directorate has a total complement of approximately 24 staff (indeterminate and term positions), including the Director, IMT / Chief Information Officer (CIO), the CIO’s assistant, 16 IM staff, and six Technology Services staff.

In addition to IMT Directorate staff, there are 14 regional IMT staff that report to regional management (the regional Manager of Finance) and have a functional reporting relationship to the CIO (although no direct reporting relationship).  WD is planning on spending $3.72M on IMT (1.9% of WD’s total budget) for 2011-12: $2.73M at headquarters, and $996K within the regional locations (1.4% and 0.5% of WD’s total budget, respectively).

At WD, Information Management Technology (IMT) governance provides the structure that links IMT processes, IMT resources, and IMT information to WD’s objectives and strategies.  Furthermore IMT is a key enabler for WD, as all staff deal with information and/or technology on a daily basis.  The area of IMT governance has not previously been the subject of an internal audit.

There are two governance committees that share the main responsibilities over IMT oversight within the department:

• IMT Council: a cross-representation of departmental business areas and regions with vested interest in the major initiatives and client relationships undertaken by the IMT directorate. The IMT Council is responsible for assessing business cases for major initiatives, setting priorities and making recommendations to Executive Committee for unallocated funding requirements in partnership with the respective Business Project Sponsor. ³
   
• Executive Committee: the departmental standing committee responsible for: providing overall strategic and management oversight to all departmental activities; ensuring that the long-term strategic outcomes of the department and regions are achieved; and, setting priorities for the department to achieve these outcomes.⁴

There are also separate committees in place that provide oversight over specific IMT projects. (i.e. Project Gateway Priority Steering Committee and Information Management Priority Steering Committee).

____________________

³ WD IMT Council Terms of Reference
⁴ WD Executive Committee Terms of Reference

Objective

The objective of the audit is to determine whether WD has in place an effective IMT governance framework and practices that meet GoC and WD policy requirements.

Scope

The scope of the audit included the IMT governance activities related to:

• IMT strategic planning;
• IMT organization;
• IMT project oversight;
• IMT risk management;
• IMT policies;
• IMT investments and budgets; and,
• IMT performance standards.

The audit did not include a review of IMT activities related to IMT architecture management, IMT asset management, IMT application and database change management, IMT operations and support, and IMT security, disaster recovery and privacy.

Approach and Methodology

The approach and methodology used for the audit were consistent with the Internal Audit standards as outlined by the Institute of Internal Auditors (IIA), and were aligned with the Internal Audit Policy for the Government of Canada (GoC).

A risk-based audit program was completed to obtain sufficient and appropriate audit evidence. Specific audit procedures performed included:

• Review of policies and procedures related to IMT governance;
• Review of strategic plans, committee terms of reference and meeting minutes;
• Review of a sample of IMT -related projects (as it related to project oversight); and,
• Interviews with targeted individuals within WD management and the IMT Directorate.

WD strives to maintain a control framework for IMT governance that is reflective of Federal Government policies and industry leading practices. Consequently, relevant sections from the following documents were used in developing the audit criteria:

• Control Objectives for Information and related Technology(COBIT 4.1) framework established by the Information Systems Audit and Control Association (ISACA); and,
   
• Management Accountability Framework (MAF) that sets out the Treasury Board's expectations of senior public service managers for good public service management. Relevant MAF Areas of Management include: Effectiveness of IM, and Effectiveness of IT.

WD is also subject to Treasury Board Secretariat (TBS) Policy requirements related to the Management of IT and IM, and these requirements were leveraged as they relate to IMT governance.

The application of the audit procedures was intended to allow the formulation of a conclusion as to whether the audit criteria established for this audit were being met.  Evidence was gathered in compliance with Treasury Board policy, directives, and standards on internal audit, and the procedures used meet the professional standards of the Institute of Internal Auditors (IIA). Standards for evidence were followed to ensure that information is sufficient, reliable, relevant, and useful to draw conclusions and meet the objective of the audit.

Observations and Recommendations

Based on interviews, analysis, and evidence gathered through the audit, each audit criterion was assessed by the audit team and a conclusion for each audit criterion was determined. Where a significant difference between the audit criterion and the observed practice(s) was found, the risk of the gap was evaluated and used to develop a conclusion for each audit criterion and to document recommendations for future improvement initiatives. Details of these observations and recommendations are provided below.

IMT Strategic Planning and Risk Management

Expectations

We expected to find formal IMT strategic planning processes in place to ensure that IMT planning is integrated with the department’s overall planning processes, includes formal input from stakeholders across the organization, and considers the current IMT capacities and capabilities of the department. Furthermore, we expected to find an IMT risk management framework in place to manage IMT strategic, operational and projects risks, and that is aligned with the department’s overall risk management processes.

Observations

Annual IMT planning is performed by the CIO / Director of IMT , communicated within the IMT section of the Finance and Corporate Management (FCM) Business Plan, and approved by the Executive Director – Finance and Corporate Management. IMT planning is currently performed with limited input from relevant stakeholders: regional IMT contacts, senior departmental management, or the Planning and Reporting Committee.

While the Corporate Management Business Plan is prepared by the Planning and Reporting Committee and approved by the Executive Committee, the department as a whole does not have an integrated planning schedule as functional area business plans are completed at different times during the year. The Planning and Reporting Committee has had limited exposure to IMT priorities / planning in the past; the CIO is not a member of the committee, and IMT has not frequently been discussed.

The audit team performed some analysis of the IMT organizational structure (the audit was not intended to provide a full organizational review), and did not observe any clear gaps in this area; however, WD has not performed a formal assessment of IMT human resource capacity and capabilities to determine if appropriate staffing levels, competencies and organizational structure exist to help achieve IMT objectives and priorities.

Corporate risk management is performed by the Planning and Reporting Committee who leads the development of the Corporate Risk Profile. It was noted that the IMT Directorate does not have direct representation on the Planning and Reporting Committee, and there is limited consideration of IMT risks that affect the department within the Corporate Risk Profile. Furthermore, specific IMT risks / risk mitigation strategies within the Corporate Business Plan or within the IMT portion of the FCM Business Plan have not been identified. Risk management processes are performed informally by IMT management for large IMT projects; however, a formal IMT risk management framework has not been established at WD.

WD and IMT management have identified integrated strategic planning and IMT risk management as areas that can be improved and have taken initial steps to improve processes:

• The department is in the early stages of developing a department-wide integrated planning schedule for the preparation departmental plans, including regional and functional business plans.
• The CIO has prepared an IMT Business Plan for 2011 – 2014 that includes sections on Corporate Alignment and Risk Management; however, this plan has not yet been finalized or approved.

Impact

IMT planning processes should be integrated with departmental planning processes and consider input from all relevant stakeholders. Absence of formal IMT planning processes that are fully integrated with departmental planning increases the risk that IMT planning and resource allocation will not be aligned with the broader priorities of the department.

Without a formal assessment of IMT human resource capacity and capabilities, gaps may not be well understood and addressed in a timely manner to achieve the IMT priorities of the department. It is also expected that the Federal Government’s Strategic and Operating Review, and the Shared Services Canada initiative will have an impact on IMT capacity and capabilities, which further supports the need to perform a formal assessment as the impact of these initiatives becomes clearer.

Without having an IMT risk management framework and formalized IMT risk management processes that are aligned to risk management processes of the department, there is an increased risk that IMT risks will not be appropriately identified and managed to help the Department achieve its objectives.

Recommendations

1. A formal IMT planning process should be established that allows for integration with departmental planning. Furthermore, planning processes should engage all relevant stakeholders (including regional IMT contacts, senior departmental management, and the Planning and Reporting Committee), and should be periodically monitored by management.

2. An IMT risk management framework should be developed that allows for WD to identify strategic and operational risks related to IMT , develop mitigation strategies/controls for identified risks, and regularly report on the status of mitigation measures. Furthermore, IMT management should be formally engaged in the departmental planning process to help ensure that IMT risks are considered as part of departmental planning, and that the IMT Directorate is aware of risks at the departmental level that are relevant to IMT activities.

IMT Performance Measurement

Expectations

We expected to find appropriate IMT performance measures that are clearly defined, and a formal IMT performance monitoring processes established to allow for adequate evaluation of IMT service delivery and IMT ’s contribution to the department by management and relevant governance committees.

Observations

Some IMT performance monitoring is performed by IMT management, IMT Council, and the Executive Committee; however, a comprehensive IMT performance measurement process has not been established.

IMT operational statistics (i.e. service calls received / completed) and IMT project status updates are prepared and provided to the Executive Committee as part of the IMT quarterly update. Beyond these operational statistics, however, performance measures have not been defined at a more strategic level. For example, the IMT portion of the FCM Business Plan does not include formal performance indicators to measure the achievement of IMT priorities within the Plan. Furthermore, there is limited evidence of IMT management or relevant committees (IMT Council, Executive Committee) performing ongoing monitoring of the achievement of priorities within the Plan; a performance measurement framework has not been defined to clarify what should be measured and how performance should be monitored by management and governance committees.

IMT management has identified IMT performance measurement as an area that can be improved and have taken initial steps to improve processes; the CIO has prepared an IMT Business Plan for 2011 – 2014 that includes a section that defines formal IMT performance indicators; however, this plan has not yet been finalized or approved.

Impact

Lack of comprehensive IMT performance measures limits the ability of IMT management and governance committees to effectively monitor IMT initiatives and take corrective action as necessary. As reporting of IMT performance has not been defined, there is an increased risk that IMT management and relevant committees will not have appropriate information to exercise oversight and make informed decisions.

Recommendation

3. Formal performance measures should be established for all key IMT activities, including: operational service delivery, key IMT projects, and priorities identified within IMT annual plans. IMT performance reporting processes should be established that define IMT performance reporting to IMT management and relevant committees at scheduled periodic intervals.

IMT Project Priority Setting Processes

Expectations

We expected to find a process for prioritizing IMT project and ensuring they are aligned with the objectives of the department; we also expected to find that prioritization processes would include analysis of costs, benefits, dependencies and alternatives.

Observations

A new IMT Project Priority Setting Framework has recently been utilized at WD, and is a significant improvement for prioritizing IMT projects to align to IMT strategies and business requirements of the department; however, the use of the Framework is still maturing and could be improved to optimize the effectiveness of the process.

The Framework provides detailed guidance over the roles and responsibilities of IMT management and relevant committees with regards to prioritizing IMT projects. The Framework was used by the IMT Council to prioritize and rank 29 IMT projects, and was approved by the Executive Committee in August 2011; however, justification for IMT project rankings was not compiled by IMT Council and provided to the Executive Committee.

IMT projects are ranked using the Framework according to 5 factors: business value, delivery value, economic value, business risk, and technical risk. In considering value of IMT projects, only incremental costs of IMT projects were considered, and full time employee labour (at headquarter and regional locations) to develop / implement IMT projects were not considered. It was also noted that ongoing maintenance costs (post development / implementation) were not consistently allocated to project costs.

Impact

While WD has a limited number of large projects, not retaining justification for IMT project rankings limits the ability of the Executive Committee to perform effective oversight before approving IMT project rankings, and increases the risk that ineffective IMT resource allocation decisions may be made.

As not all relevant costs are allocated to IMT project cost (i.e. full time employee labour at headquarters and regions, and ongoing maintenance costs), effective cost-benefit analysis is limited, which increases the risk that IMT projects may not be appropriately prioritized.

Recommendations

4. Some level of justification of IMT project rankings should be provided to the Executive Committee to facilitate the effective oversight and approval of IMT resource allocation decisions.

5. Projects costs that are used to perform cost-benefit analysis and produce IMT project rankings should consistently include all relevant costs, including: full time employee labour (at headquarters and regional locations), and post development maintenance costs.

IMT Project Oversight Processes

Expectations

We expected to find an IMT project oversight framework that defines monitoring processes for IMT management and relevant committees to ensure that appropriate oversight is performed consistently for IMT projects. It was also expected that change management processes would be established that ensure significant project changes also follow project oversight processes.

Observations

WD has recently improved IMT project oversight processes with the establishment of the IMT Project Priority Setting Framework and implementing quarterly IMT reporting to the Executive Committee (which includes providing IMT project updates); however, ongoing oversight processes and change management processes can still be better defined to ensure effective oversight is consistently performed, and changes are appropriately managed and approved for IMT projects.

The recently implemented IMT Project Priority Setting Framework includes a description of each stakeholder’s (management and relevant committees) responsibilities in exercising the Framework, including high level responsibilities over monitoring projects; however, the Framework does not include guidance on how these responsibilities should be exercised by relevant committees, or how projects should be measured in terms of quality, cost, and meeting project timelines.

It was noted that WD does use standardized forms to record the approval of IMT changes; however, formal change management processes do not exist at WD.

The first IMT quarterly update was provided to the Executive Committee in August 2011 and included some evidence of IMT project monitoring, including high-level status of key projects, and a snapshot of the IMT project portfolio; there is an opportunity to further improve project oversight reporting to make it more efficient. For example, it is difficult to gather if projects that are not on track are high priority projects, which would better allow the Executive Committee to determine if re-allocation of resources (potentially from low priority projects) is required. When presenting information, the input/decision that is being sought from the governance committee should be made clear.

Ultimately, project oversight processes have not been formalized to detail, based on the importance (e.g. value, risk) of the project, the frequency and performance measures to present to Executive Committee, the IMT Council and project-specific governance committees.

Impact

Without having formally established IMT project oversight processes, there is an increased risk that effective monitoring of IMT projects will not be consistently performed by relevant oversight committees. This also increases the risk that corrective actions may not be taken in a timely manner.

Informal change management processes could result in, amongst other things, changes being implemented that have not received approval from an appropriate level of management.

As specific oversight processes for IMT projects have not been formalized based on the importance of IMT projects (i.e. value, risk), there is also a risk that management and relevant committees may spend too much time monitoring low value / low risk projects, and not enough time monitoring high value / high risk projects..

Recommendation

6. A formal IMT project oversight framework should be established that helps ensure that adequate oversight is performed consistently for IMT projects. The framework should provide guidance on ongoing monitoring of the performance of IMT projects with regards to cost quality and schedule and be supported by a documented change management process. Furthermore the framework should clarify the reporting requirements to each governance committee depending on the importance of a project (i.e. large, medium, low value/risk).