ARCHIVED - Performance Metrics and Benchmarking PSEA (2003)

WarningThis page has been archived.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.



March 31, 2008

Final Report Produced by Deloitte

Table of Contents

Important Notice: The Public Service Commission of Canada respects the Official Languages Act and the relevant Treasury Board policies, and is committed to ensuring all information and services on this site is available in both English and French. However, visitors should be aware that some information from external sources that are not subject to the Official Languages Act is only provided as a convenience and is available only in the language in which it was provided.

1. Executive Summary

The Public Service Commission (PSC) Effectiveness Measurement Framework was developed to measure the progress of the implementation of changes and aspects of the Public Service Employment Act (PSEA) which came into effect on December 31, 2005 for which the PSC is responsible. Deloitte was engaged to conduct an assessment of PSC’s Effectiveness Measurement Framework and to identify the potential for indicators to benchmark performance with other jurisdictions.

The approach to conduct an assessment of the PSC Framework included primary research on performance measurement practices across nine national and provincial/state jurisdictions. The goal of the multi-jurisdictional study was to understand the current state of performance measurement practices within the specific context of the governing legislation, strategic objectives, the staffing business model of the jurisdiction being examined, and to identify potential measures for benchmarking.

An analytical framework was developed and used as a conceptual model to set the foundation for both collection and analysis of the data from the jurisdictional survey. This framework was used to guide the data collection efforts with jurisdictions to understand the current context, their staffing business model and processes, and the focus of their performance measurement efforts. A list of performance indicators used in each jurisdiction was also reviewed to identify those measures which were contextually relevant for PSC and the PSEA staffing objectives, as well as those that were common across two or more jurisdictions, representing a potential for future benchmarking activities.

The inventory of performance measurement practices and indicators from the multi-jurisdictional study was used to assess the extent to which any indicators used by other jurisdictions that could be considered for inclusion in PSC’s emerging framework, as well as to identify any potential refinements to current PSC indicators. In addition to examining the specific measures within the PSC Framework, the assessment also examined the design of the framework in the context of best practice approaches to performance measurement. Considerations in this regard included the integration of PSEA staffing objectives and results into the framework design, the balanced use of leading and lagging indicators, and the availability of information to meet the needs of a range of stakeholders both internal and external to PSC.

Key themes that span several of the jurisdictions in the study were identified. Many jurisdictions reported efforts to transform their approach to recruitment, including continued focus on streamlining the recruitment and selection process and greater focus on a "one employer" brand. Several jurisdictions reported that they were shifting their focus from solely measuring the "process of staffing" and placing greater emphasis on client- and candidate-centric perspectives.

With respect to governance, there is a shift away from "rules-based" policy to more outcome-based directives to departments and agencies which have delegated authority for staffing decisions. Independent appeal bodies to examine recruitment and staffing decisions are commonly observed, particularly at the national level.

Many jurisdictions have enshrined fundamental staffing values such as merit and transparency in legislation that governs the recruitment and staffing process. Recent legislative changes in other jurisdictions are intended to bring about new governance and service delivery models to support modernization of recruitment and staffing activities.

In many jurisdictions, more than one organization has a role in recruitment and staffing. Typical roles observed include policy direction, operational service delivery and independent oversight and appeal. There is a focus on whole-of-government initiatives, such as "one-employer branding" to guide and support service delivery. Most jurisdictions have set specific standards against which departments/agencies are measured, either through self-assessments/reporting or through audit or centralized agency evaluation.

There is significant focus on measuring the effectiveness of the staffing model, including consideration of perspectives from a variety of stakeholders such as hiring managers, employees and candidates. From a reporting perspective, performance indicators are often publicly reported through annual reports. Some indicators are used for internal management purposes to track the achievement of strategic priorities and plans. In general, jurisdictions with greater technology capability demonstrate stronger performance measurement capability.

The assessment of the PSC Framework focused on identifying opportunities to strengthen the framework based on practices observed in other jurisdictions or based on Deloitte’s expertise in performance measurement best practices. In addition, the assessment resulted in the identification of potential indicators that could be benchmarked with other jurisdictions to support the PSC’s effectiveness measurement efforts.

Overall, the measurement framework has several strengths including the specification of PSEA staffing objectives and results to be achieved, and the use of a results-based approach. The linkage to the annual Staffing Management Accountability Framework (SMAF) is also viewed as a strength in terms of alignment with departmental activities and as a source for data to inform performance monitoring. There is generally a balance of leading and lagging indicators across the framework to provide insight into both drivers of performance and achievement of desired results. The identified indicators provide both and oversight and operational perspective, which is appropriate given the purpose of the effectiveness framework and PSC’s mandate. Finally, the indicators are balanced with respect to allowing for reporting to a variety of stakeholders including PSC, Parliamentarians and Departments.

The assessment identified some opportunities for refinement including the removal of indicators that are either considered not relevant to the PSEA staffing objectives or a stronger or related indicator already exists in the framework. In other cases, there are opportunities to refine current indicators to incorporate a wider range of stakeholder perspectives in measuring satisfaction with the recruitment process. Finally, the multi-jurisdictional research identified a few indicators that could be included in the PSC Framework, particularly in support of the PSEA Staffing Objective "HR recruitment needs addressed."

The multi-jurisdictional research identified an opportunity to benchmark a small number of measures which will provide information to challenge or validate the current level of effectiveness of the recruitment and staffing process in the Canadian context. The following indicators emerged as the best candidates for benchmarking: time to hire; quality of hire; employee, candidate and manager perception of adherence to staffing values (merit, fairness and transparency); and employee, candidate and manager satisfaction with the hiring process.

Key recommendations resulting from the assessment of the PSC Framework were made in the following areas:

  • Specific indicators that have potential to be benchmarked with other jurisdictions, as well as approaches to begin discussions with several jurisdictions;
  • The need to continue to engage in broader discussion with Canada Public Service Agency (CPSA) and Privy Council Office (PCO) with respect to how PSC and its measurement practices can effectively contribute to the broader talent management agenda;
  • Specific refinements to the PSC framework, including:
    • Removal or repositioning of indicators, and refinements to some indicators to incorporate a wider range of stakeholder perspectives in measuring satisfaction with the recruitment process.
    • Revision of the presentation of the PSC Framework to provide better context for certain indicators.
  • The need to identify select retrospective indicators to be monitored on an annual basis to track the achievement of outcomes/results.

2. Introduction

2.1 Background

The following report provides the background information, methodology, approach, findings, implications and recommendations related to a study conducted by Deloitte for the Public Service Commission (PSC) of performance measurement practices across various provincial/state and international jurisdictions. The study was conducted in order to provide the PSC with information related to the types of performance measurement practices used by relevant jurisdictions related to staffing and recruitment, and to compare these practices with those included in the PSC's Effectiveness Measurement Framework. The PSC Effectiveness Measurement Framework was developed to measure the progress of the implementation of changes and aspects of the Public Service Employment Act (PSEA) which came into effect on December 31st, 2005 for which the PSC is responsible.

The PSEA provides the legal framework for appointment to and within the Canadian federal public service, describing roles and responsibilities for the PSC and deputy heads of the federal public service. The Act is prefaced by a preamble that contains strategic goals for the legislation in relation to the independent accountability role of the Public Service Commission, delegation of staffing authority to as low a level as possible, staffing criteria for public service excellence, and presents the overall legal framework for the staffing process.

The Public Service Employment Act was one of the key Acts amended as part of the broader Public Service Modernization Act. These new and amended Acts declared the federal government’s priorities of renewing and updating the human resources management system. In essence, the changes were meant to provide a foundation for culture change related to staffing and recruitment in the Public Service which includes a shift from rules-based to values-based staffing and increased flexibility for managers in the staffing and recruitment process. The Act is governed by guiding principles which outline core and guiding values for staffing in departments. These values are reflective of the strategic goals contained in the preamble to the PSEA and are included as objectives to be measured by the PSC Effectiveness Measurement Framework.

The Public Service Employment Act includes a legislative requirement to review the Act five years after it comes into force. As a result, the PSC has developed the PSC Effectiveness Measurement Framework to measure progress in implementing those aspects of the Public Service Employment Act for which the PSC is responsible. The Framework consists of three elements: PSEA based expectations of Deputy Heads; leading indicators; and retrospective/lagging indicators. The purpose of the PSC Effectiveness Measurement Framework is to establish meaningful ongoing performance measures of progress in implementing the PSEA, contribute to continuous improvement of PSC oversight activities based upon key success factors, and provide Parliament with support that the legislation met the required outcomes. As a result, the PSC Effectiveness Measurement Framework (referred to in this document as the PSC Framework) is a key tool which will enable the PSC to report success factors back to Parliament. A copy of the Lead Indicators and Five-Year Evaluation Framework of the PSC Framework is included in Appendix A.

The following outlines the "core values" and "guiding values" for departmental staffing. (Source: Overview of the Public Service Employment Act, Public Service Commission of Canada Publication)

2.1.1 Core Values

Merit

  • Every person appointed must meet the essential qualifications (including official language proficiency) established by the deputy head for the work to be done.
  • The hiring manager may take into consideration any current or future asset qualifications, operational requirements, and organizational needs also identified by the deputy head.

Non-partisanship

  • Appointments and promotions to the public service are made free from political influence.
  • Employees have the right to engage in political activities, while maintaining the principle of political impartiality in the public service.
  • The political activity of employees must not impair, or be perceived as impairing, the employees’ ability to perform their duties in a politically impartial manner.
  • Political activity means any activity in support of, within or in opposition to a political party, any activity in support of or in opposition to a candidate, or seeking to be a candidate in an election.

2.1.2 Guiding Values

Access

  • Potential candidates from across the country have a reasonable opportunity to apply, and to be considered, for public service employment.

Fairness

  • Decisions are made objectively and free from personal favouritism; policies and practices reflect the just treatment of persons.

Representativeness

  • Access contributes to a workforce that embodies linguistic duality and is representative of Canada’s diversity, including the designated employment equity groups.

Transparency

  • Strategies, decisions, policies and practices are communicated in an open and timely manner.

As an independent agency responsible for safe guarding the values of the public service, the PSC has an oversight role with respect to staffing. In executing this oversight role, PSC needs to ensure that operational staffing processes comply with applicable legislation and staffing policy directives. The PSC is responsible for protecting merit, non-partisanship, representativeness and the use of both official languages. Under the PSEA, the PSC has the authority to make appointments to and within the public service and has delegated many of its appointment authorities to Deputy Heads as permitted by the Act. The recent changes to the legislation allow for greater delegation and for Deputy Heads to have more flexibility in the staffing process while adhering to the PSC staffing values such as merit.

Accordingly, Deloitte was retained to provide an assessment of the PSC Framework. This assessment included conducting a study of performance measurement practices across participating provincial/state and international jurisdictions for the following purposes:

  • To validate that the PSC framework encompasses indicators and measures that are meaningful and relevant to the business of staffing in the federal government;
  • To identify other performance measurement practices, including relevant indicators and measures that can be used but are currently not included in the PSC’s emerging framework;
  • To confirm the scope of benchmarking content and practices that can be used across other federal jurisdictions in the future; and
  • In the context of accountability and good governance, and continuous improvement, to ensure that the right performance metrics drive meaningful targets and realized performance to support strategic direction.

2.2 Report Outline

This report is divided into five main sections. This section provides an introduction to the PSC, the legislation governing the staffing process, and other background information that serves as context for this report. Section 3 provides a summary of findings from the multi-jurisdictional interviews and related document review. Key themes based on the multi-jurisdictional research and resulting implications for PSC are identified. Section 4 provides an assessment of the potential for benchmarking selected indicators/measures with other jurisdictions. An assessment of the PSC Framework is provided in Section 5 based on three perspectives: framework design, framework measures, and measures within each specific PSC strategic objective. In conclusion, Section 6 provides recommendations to strengthen the PSC Framework.

2.3 Methodology and Approach

The approach to conduct an assessment of the PSC Framework included primary research on performance measurement practices across international and provincial/state jurisdictions. The goal of the jurisdictional study was to understand the current state of performance measurement practices within the specific context of the governing legislation, strategic objectives, and staffing business model of the jurisdiction being examined. The intent was to develop a summary of performance measurement practices used in other jurisdictions, including an inventory of indicators used to assess the effectiveness of the staffing process.

Interviews were held with senior representatives of each jurisdiction to understand their approach to staffing and related performance measurement efforts. An interview guide was sent to jurisdictions in advance of the telephone interview, along with a synopsis of publicly available information for that jurisdiction related to applicable legislation, governing bodies and basic information on the staffing process.

A total of eleven jurisdictions were invited to participate in the study, with the objective of achieving a 70% participation rate. Nine of the eleven jurisdictions agreed to participate, as summarized in the table below. A summary profile for each jurisdiction was created and validated with the interviewee following data collection.

Category of Organization Jurisdiction
Federal Governments United States
Australia
United Kingdom
Ireland
Provincial Governments Quebec
British Columbia
Alberta
Ontario
State Government California

An analytical framework was developed and used as a conceptual model to set the foundation for both collection and analysis of the data from the jurisdictional survey. This model is illustrated in Figure 1.1. below. The framework depicts the key components of a business model, including overall governance/strategic direction of the business; core business processes which convert inputs into outputs; and enablers of the business. The business operates within the context of broader demands or needs, and delivers outcomes that align with the strategic direction of the business. Performance measurement efforts are focused on monitoring key aspects of the business model (inputs, resource consumption, process, outcomes) as well as the achievement of outcomes. Performance measurement practices will be influenced by the strategic context and the specific approach or business model used to deliver staffing services.

Figure 1.1: Description in paragraphs before and after the image

Figure 1.1: Analytical Framework for Data Collection and Analysis

In the context of this project, the business model being examined is the staffing business model. In Canada, core staffing processes exist primarily at the departmental level to convert inputs (e.g. candidates, staffing specialists, hiring requirements) into outputs (e.g. hires that meet the requirements set out in business plans, high quality hires, etc.). Currently, the staffing business model in Canada is under transformation as a result of public service renewal and amendments to the PSEA, as illustrated by the notation of "implementation" as the broader context in which the staffing business model is operating. Outcomes for the staffing model in Canada include recruitment of a highly qualified, diverse, talented and representative workforce that is non-partisan. The final outcome is "an impartial, representative public service with integrity that strives for excellence in achieving result for the Government of Canada and Canadians".

The analytical framework was used to guide the data collection efforts with jurisdictions to understand the current context, their staffing business model and processes, and the focus of their performance measurement efforts. A list of performance indicators used in each jurisdiction was compiled and analyzed to identify on which component of the business model (input, process, output, outcome) the indicators were focused. The indicators were also reviewed to identify those which were contextually relevant for PSC and the PSEA staffing objectives, as well as those that were common across two or more jurisdictions, representing a potential for future benchmarking activities.

The inventory of performance measurement practices and indicators from the multi-jurisdictional study was used to assess whether the current PSC Framework considers all aspects of the performance of the staffing process. The assessment considered whether there were any indicators used by other jurisdictions that could be considered for inclusion in PSC’s emerging framework, as well as identify any potential refinements to current indicators. In addition to examining the specific measures within the PSC Framework, the assessment also examined the design of the framework in the context of best practice approaches to performance measurement. Considerations in this regard included the integration of PSEA staffing objectives and results into the framework design, the balanced use of leading and lagging indicators, and the availability of information to meet the needs of a range of stakeholders both internal and external to PSC. Recommendations are provided in this report to address any opportunities for improvement noted during the assessment process.

3. Summary of Findings from Jurisdictions

This section of the report provides a summary of the findings from a review and analysis of performance measurement practices across nine jurisdictions at the federal and provincial/state levels. The majority of the findings were derived specifically from the interviews conducted with the jurisdictions. The review covers the following jurisdictions—the United States, United Kingdom, Ireland, Australia, British Columbia, Alberta, Ontario, Quebec and California.

The findings are presented by the components of the analytical framework presented in the methodology section, which was used to collect and analyze the jurisdictional data. Key themes from these findings are presented in three broad categories—Strategic Context, Business Model, and Performance Measurement Practices.

While the focus of this study was on understanding recruitment and staffing performance measurement practices across jurisdictions, it is important to understand the background for these practices including the applicable transformation efforts, governance and legislation. The analytical framework for this study also included understanding the jurisdiction’s business model or approach to delivering staffing and recruitment services, as context for the specific measurement practices used for efficiency and effectiveness assessments. Within this overall context, this report provides a summary of performance measurement practices observed across jurisdictions including approaches to monitor and assess achievement of strategic objectives and the efficiency and effectiveness of the staffing business model, as well as supporting tools or enablers of the performance measurement processes. Specific examples from jurisdictions are provided throughout this section of the report to illustrate the key themes identified.

3.1 Strategic Context

The impetus for this multi-jurisdictional review of performance measurement practices for staffing and recruitment is public service renewal within the Canadian federal government context. The Public Service Employment Act provides a foundation for a culture of change in hiring and promotion, providing managers with authority and accountability to make staffing and recruitment decisions that address program or business needs.

Public service renewal, particularly modernization and transformation of recruitment and staffing was a common theme across all jurisdictions surveyed as part of this study. A discussion of the key themes in these renewal and transformation efforts follows.

3.1.1 Transformation Efforts

Move to a "one employer" brand and use of broader human capital approach

Jurisdictions such as Alberta 1, Ontario 2, British Columbia 3, Australia 4 and the United States 5 have adopted "one employer" branding and talent management strategies with an increased focus on workforce planning and succession management and the linkages to recruitment strategy, while continuing to recognize the multitude of diverse "business lines". Given the need to address critical skills shortages in the marketplace, these jurisdictions have established proactive recruitment strategies to ensure a sufficient supply of talent in the future.

B.C. is taking a "one employer" brand and "one employer" approach through centralized advertising, changes in hiring practices, and implementation of both a new employment website and a new marketing framework. Also, Ontario has taken an "enterprise-wide" approach to recruitment with priorities established around developing and implementing HR strategies and policies that make the Ontario Public Service the "place to work".

The United States is moving towards more of an integrated human capital approach as opposed to a sole focus on staffing and recruitment. Similarly, Alberta has adopted an overall talent management approach with an emphasis on attracting, developing and engaging talent. This integrated human capital strategic focus is influencing the measurement framework and the indicators that are measured. While this practice is more mature in jurisdictions such as the United States and Australia, it is a relatively new practice in the cited Canadian Provinces.

Shift from process focus to client-and candidate-centric perspectives

Across many jurisdictions, there appears to be a shift away from a focus on the staffing process towards the outcomes or results achieved. For example, there is a greater importance placed on the "experience" of the applicant/candidate during the selection process. Jurisdictions such as the Province of Ontario, Ireland 6 and the United States 7 all monitor and collect measures related to applicant satisfaction with the recruitment process.

There is also a shift in focus from selecting the "most qualified candidate". For example, in the Province of Alberta, the concept of hiring a "suitable" candidate has replaced the concept of hiring the "most qualified" candidate. Similarly, Australia has recognized the need to shift the focus from ensuring that the process has adhered to the merit principles to ensuring that the process is meritorious and results in a good and a timely selection decision.

Continued efforts to streamline recruitment and selection process

There are continued efforts to improve the efficiency and effectiveness of the hiring process and to optimize the expenditure of manager’s effort in this process. For example, the majority of jurisdictions are focused on standardizing and streamlining tools for the recruitment and selection process, allowing for better data capture and more effective monitoring of the process. Ireland and California have introduced online candidate assessment, and Ontario has developed software in-house to manage and monitor the recruitment process. These changes are often being driven by a need to increase efficiencies and modernize recruitment and selection methods and to improve on the jurisdiction’s ability to report on the performance of the hiring process.

3.1.2 Governance Approaches

A review of the jurisdictions shows that there are generally three broad roles in relation to recruitment - policy direction, service delivery and independent appeal. Several themes were noted in the review of the specific approaches used in jurisdictions to fulfill these roles.

Decentralization delegated authority across all jurisdictions

A review of the jurisdictions reveals that staffing responsibility is decentralized in all jurisdictions. In the jurisdictions surveyed, authority to hire is delegated to agencies and departments. Staffing responsibility is often delegated to the lowest level practical (i.e. hiring manager). This decentralized approach for authority for staffing decisions has generally been in place in all jurisdictions for many years. In the United States, there is an expectation that managers will be taking an increasingly more active role in all human resource activities including recruitment and selection. In Ireland, public service bodies are required to obtain recruitment licenses to conduct their own recruitment activities, or they can choose to use the services of the Public Appointment Services, a licensed central recruitment agency responsible for appointments to the Irish government.

Shift away from "rules-based" policy direction to more outcome-based directives

All jurisdictions have legislation that governs employment in the public service with components specifically related to staffing. In addition, some jurisdictions have directives specific to recruitment. For example, the U.K. has issued a mandatory Recruitment Code that sets out the procedures that must be followed in the appointment of people to civil service roles. The primary principle enshrined within the code is "fair and open competition" 8. Similarly, both the Provinces of Alberta and Ontario 9 have created directives specific to staffing which are intended to guide managers in staffing and recruitment-related decisions and processes. The United States Office of Personnel Management has released a Human Capital Assessment and Accountability Framework which provides agencies with standards and performance measures aligned to the Merit System Principles outlined in the respective legislation 10. The Commission for Public Service Appointments (CPSA) in Ireland sets standards, which it publishes as "Codes of Practice", for recruitment and selection 11. The CPSA is also responsible for safeguarding these standards through regular monitoring and audits.

Independent appeal bodies to examine recruitment and staffing decisions commonly observed, particularly at the federal level

While a range of approaches are used to provide recourse mechanisms for external candidates or employees who feel they have been unfairly treated in a staffing or recruitment process, many jurisdictions have models where an independent body reviews cases that cannot be resolved at the departmental or agency level. For example, Australian public servants who were not satisfied that a promotion decision was based on merit can request the Merit Protection Commissioner to review the decision. The decision must first be reviewed at the agency or department level before proceeding to the Merit Protection Commissioner. Similar bodies in other jurisdictions include the Office of the Civil Service Commissioners in the UK 12, the Office of the Merit Commissioner in BC 13 and the Merit Systems Protection Board in the US 14.

3.1.3 Legislation

Values enshrined in legislation

Strategic objectives related to staffing and recruitment found to be common across jurisdictions include: fairness, integrity, diversity and a modern recruitment and selection process.

Although "merit" is not explicitly stated as a strategic objective for the majority of the jurisdictions, the notion of "merit" is implicit in the governing legislation and continues to lay the foundation for recruitment and selection policies and directives.

Recent legislative changes to bring about new governance and service delivery models

Many of the recent legislative changes in other jurisdictions provide for the creation of new governance and service delivery models. For example, British Columbia made changes to their legislation to create the BC Public Service Agency which provides leadership and services related to people management. There was also a change in hiring and recruitment practices that previously focused on selection and now focuses on recruitment and sourcing. In Ireland, recent changes to the legislation resulted in the dissolution of existing bodies related to staffing and the creation of two new bodies; the Public Appointments Service (PAS) 15 and the Commission for Public Service Appointments (CPSA) 16. PAS is a central provider of recruitment services, and CPSA is responsible for oversight, policy direction and hearing appeals.

3.1.4 Business Model

Variety of staffing service delivery models with enterprise-wide initiatives

There is a mix between centralized and decentralized staffing service delivery models (i.e. shared services, department-led). For example, Ontario has recently moved towards a more streamlined and centralized approach and is in the process of implementing an enterprise-wide model with Regional Recruitment Centers to provide recruitment and staffing support to Ministries 17. Similarly, in Ireland the Public Appointments Service provides centralized recruitment and consultancy services to agencies, local authorities, health boards, and other public bodies. Agencies in Ireland also have the option to conduct their own hiring with a license from the Commission for Public Service Appointments.

It is noted the majority of jurisdictions have decentralized staffing service delivery models, i.e. departments or agencies have their own recruiters. These departments and agencies are responsible for establishing hiring practices that are consistent with the principles set out in the policy direction and directives from central agencies. Agencies and departments also establish their own processes, practices and tools for recruitment and staffing. Some jurisdictions are establishing an enhanced central support for the provision of recruitment services to agencies, e.g. Ontario, Quebec and Ireland. Others, such as California, have centralized assessment processes used to identify pools of qualified candidates.

Focus on whole-of-government initiatives to guide and support service delivery.

There are many instances of whole-of-government initiatives related to recruitment, which range from the use of a single website as a ‘portal’ for employees (and external candidates) seeking employment in the public service (UK, Australia, British Columbia), to whole-of-government employee branding efforts (Alberta, Ontario, British Columbia) to shared services for the provision of recruitment services for "hard-to-find" skills.

Multiple bodies have a role in recruitment and staffing

In all jurisdictions, there is typically more than one organization with a role in the staffing and recruitment process. Roles consistently observed across the majority of jurisdictions include the presence or combination of an oversight/central policy direction body, service delivery, or independent appeal body. An oversight/central policy direction body, typically from a central agency, establishes the underlying principles to be followed by departments and agencies in recruitment and staffing. An example is the Civil Service Commissioners in the UK which establishes a Recruitment Code to be followed by government-wide 18. The central agencies have a role to monitor the adherence to these principles, using a variety of mechanisms including requirements for reports from departments, audits and evaluations. Tools used in this regard by the UK government include sign-off of a Certificate of Compliance with Civil Service Commissioners’ Recruitment Code and the completion of a Recruitment Code Compliance Self-Assessment (Questionnaire). Similarly, Australia has legislative requirements that require departments/agencies to provide a wide range of information to the Public Service Commissioner on recruitment practices, which are reported annually to the public in the State of the Service Annual Report 19. British Columbia, Alberta and Ontario also have whole-of-government or enterprise-wide staffing initiatives such as employer branding and marketing.

Unique processes for senior level hires, often with significant central oversight and involvement

Many jurisdictions have a unique selection process for senior/ executive-level hires that involves a central agency. For example, one of roles of the Civil Service Commissioners in the UK is to serve as the Chair for the selection process for external hires at the Director General and Permanent Secretary levels in all departments/agencies. In Australia, the Public Service Commissioner must approve hires to the Senior Executive Service.

3.1.5 Performance Measurement Practices

Focus on measurement of the effectiveness of the staffing model

There is a trend towards focusing on measures related to the effectiveness of the staffing model and staffing process as opposed to measures related to the objectives or values that underpin the legislation. Consequently, measures tend to reflect the strategic business or human resource priorities of the jurisdictions. The Provinces of Alberta, B.C. and Ontario, in particular, build performance measurement practices around strategic objectives such as attraction, retention and engagement of talent.

There is a shift from solely processed-based measures to more outcome measures with a focus on the effectiveness of talent management programs and processes as a whole. The focus is on monitoring measures beyond staffing and recruitment and to include measures related to talent development and performance. This focus has been adopted by numerous jurisdictions (e.g. Alberta, Ontario, B.C., United States).

Wide range of mechanisms used to monitor the achievement of strategic objectives, including self-assessment

There are a variety of mechanisms used to monitor the achievement of strategic objectives. These mechanisms include self-identification, surveys, on-line questionnaires, service level agreements, self-assessments, self-accountability programs and third-party audits. For example, the United Kingdom, Australia and Quebec use both audit mechanisms and surveys to monitor the achievement of strategic objectives. Results are reported in such documents as the Civil Service Commissioners Annual Report (U.K.) 20 and the State of the Services Report (Australia) 21. In Quebec, periodic studies or audits are conducted to examine perceptions related to underlying values of the staffing process, for example, merit and accessibility. These reports are published on the Commission de la fonction publique website 22.

Several jurisdictions such as Australia, United Kingdom, United States and British Columbia measure employee and/or applicant perceptions related to merit and fairness in the staffing and recruitment process.

The U.K. uses a departmental self assessment process which requires departments to assess the strengths and weaknesses of the staffing process against specific standards (e.g. whether appointments are made on a fair basis and if exceptions are being managed appropriately) 23. These self-assessments are reviewed by a third party service provider. If the self-assessment points to higher risk (i.e. the department is found non-compliant to the Civil Service Recruitment Code with respect to whether appointments are made on merit on the basis of fair and open competition and that any exceptions are being managed appropriately), an on-site audit will be carried out to ensure that policies are followed. A detailed report of risk and series of actions to be taken is created and monitored by the Civil Service Commissioners. If unlawful appointments are identified in the audit, departments must take action, i.e. end the appointment.

The United States has also implemented a self-accountability program whereby departments and agencies are required to submit an annual Human Capital Management Report which reflects compliance to established standards.

Emphasis on fewer rather than more performance measures

An emphasis on fewer measures rather than more was observed in many jurisdictions. For example, the United States requires departments/agencies to report on eleven required performance indicators. These indicators focus on human capital outcomes and are captured from three different perspectives; organization, employee, and merit system compliance. The Province of Alberta monitors and reports on staffing metrics that relate to three specific priority areas for 2007-2010; attraction, development and employee engagement 24.

Perspectives from a variety of stakeholders form the basis for assessment of the recruitment process

It is noted that multiple stakeholder perspectives are taken into account in assessing the recruitment system (i.e. general public, hiring managers, employees, candidates). For example, the United States monitors applicant satisfaction with the hiring process through an on-line questionnaire posted on the federal government's job website. Ireland also collects measures around client (department) and candidate perspectives through on-line questionnaires. In addition, Ireland collects feedback on satisfaction with the recruitment process from candidates in test centres. Australia collects measures to understand employee, applicant and managerial perspectives 25.

Reporting mechanisms for performance assessment vary

A variety of approaches are used to report measures, including leadership-level scorecards, internal reports, and Annual Reports available to the general public. How measures are reported and frequency of reporting vary across jurisdictions with no common practice emerging.

Jurisdictions with greater technology capability demonstrate stronger performance measurement capability

Jurisdictions with the widest range of performance measurement practices have well-developed enablers. Information technology is viewed as a key enabler to support performance measurement (e.g. recruitment software). Some jurisdictions, such as Ontario, have a single-instance human resources information system, providing a single source of information which facilitates the ability to create a government-wide picture of the performance of the staffing process. There is a trend towards greater enablement of the recruitment process through the use of technology to support activities such as applicant tracking and candidate pools, which also generate data for performance measurement. On-line tools to survey candidates and clients are also prevalent.

Growing interest in measures of "quality to hire"

Several jurisdictions such as the Ontario, British Columbia and the United States indicated a growing interest in measuring "quality of hire". The Province of Ontario, in particular, discussed the need to link the staffing process and outcomes to performance management systems. A measurement approach mentioned by two jurisdictions was to ask the hiring manager to comment on their satisfaction with the performance of the new hire.

3.2 Implications for the PSC Framework

The following chart summarizes the key findings and delineates possible implications for PSC to consider as it moves forward to refine and implement the PSC Framework. These implications are also explored in Section 5 of the report.

Key Findings Implication for the PSC Framework
Majority of jurisdictions focusing on "one employer" branding and talent management strategies as a strategic priority Consider how the PSC Framework fits within the broader talent management agenda in the Canadian context
Existing efforts to measure performance in other jurisdictions resulted in some common areas of measurement
  • Potential for benchmarking exists (see section 4)
  • Further exploration is required to understand the true potential for benchmarking, given that underlying definitions for measures are not necessarily consistent
  • Need to understand whether these common measures will be useful as benchmarks to challenge existing performance levels for the staffing process and its key outcomes
Trend towards focusing on measures related to the effectiveness of the staffing model which are reflective of the strategic priorities of the jurisdictions (e.g. more effective talent management)
  • Potential to adopt measures used by other jurisdictions (see Section 5.5.5)
  • Consider shifting focus of measures from objectives of legislation to more business model focused objectives/outcomes
Emphasis on fewer measures than more
  • Potential to use PSC Framework indicators as the source for reports to a number of key stakeholders, focusing on those which are of most relevance to them and their role in the staffing process
  • Potential to develop and communicate a strategy for implementing the current framework in a practical way
Multiple perspectives considered in the assessment of the recruitment process-candidates, employees, managers, agencies
  • Potential to include measures which incorporate other stakeholder perspectives, particularly those of candidates
  • Consider adoption of measures used in other jurisdictions (or similar ones) related to candidate perspectives
Jurisdictions with greater technology capability demonstrate stronger performance measurement capability
  • Need to assess which measures within the framework can currently be produced with existing enablers and tools (employee survey, survey of appointments, SMAF, etc.)
  • Consider the feasibility and cost of implementing the current effectiveness measurement framework
  • Consider adoption of similar mechanisms used in other jurisdictions such as online surveys to assess satisfaction of candidates with the recruitment process

4. Assessment of Benchmarking Potential

This section of the report provides an assessment of the potential for benchmarking selected indicators/measures with other jurisdictions. The assessment includes the identification of common measures and further analysis is conducted to assess which of those common measures/indicators might serve as useful benchmarks within the Canadian context. The section concludes with the identification of implementation considerations for future benchmarking efforts.

4.1.1 Inventory of Common Measures and Potential for Benchmarking

An inventory of measures/indicators from other jurisdictions identified twenty measures that were common across at least two jurisdictions. An initial review of these common measures suggests that all are contextually relevant for PSC, i.e. they relate to one of the twelve PSEA staffing objectives.

Given the required effort and complexity of benchmarking efforts, there is a need to be focused on a relatively small number of measures and to be selective in identifying those that will provide the best information to improve the effectiveness of the recruitment and staffing process in the Canadian context. Benchmarks are typically used to provide insight into level of results that can be achieved through the use of innovative policies, processes and systems or to validate when a reasonable threshold of performance has been achieved. Benchmarks can also provide information to challenge current practices and to support efforts to set stretch targets for improved performance.

Accordingly, the following criteria were used to determine which of the twenty measures/ indicators are most suitable for PSC to benchmark:

  • Alignment with the PSEA staffing objectives;
  • Prevalence of use in other jurisdictions and degree of similarity in measurement approach;
  • Relates to and reinforces a focal point for change; and,
  • Of interest to stakeholders.

A comparison of the initial twenty measures against these criteria above results in the following indicators emerging as the best candidates for benchmarking:

  • Time to hire;
  • Quality of hire;
  • Employee, candidate and manager perception of adherence to staffing values (merit; fairness, and transparency); and
  • Employee, candidate and manager satisfaction with the hiring process.

4.1.2 Implementation Considerations

A review of the findings related to the indicators identified as potential benchmarking candidates reveals two significant implementation considerations for PSC. First, there are differences between jurisdictions with respect to the descriptions used for these indicators. Of notable difference are the descriptions used for "time to hire".  This may pose limitations on how the data from benchmarking efforts can be used, although it does not necessarily exclude the indicator from consideration in benchmarking exercises. Further discussion on the commonality of specific definitions for indicators is noted as the implementation considerations for each indicator is discussed below.

Secondly, relative to the current PSC framework, there is a much greater emphasis on measuring the "applicant" perspective in other jurisdictions, notably Australia and Ireland. There are opportunities to strengthen the PSC framework by complementing existing indicators that provide manager, employee and successful candidate perspectives with additional indicators that incorporate the applicant perspective. The new Survey of Staffing will provide additional candidate perspectives (both successful and unsuccessful candidates) than were available through the previous Survey of Appointments, which focused only on appointees. The new Survey invitation will be sent only to current public servants (as flagged within the pay system/file). Information on unsuccessful candidates from outside the PS will not be included in the survey. The PSC may wish to consider efforts to expand its reach to attempt to capture the views of this external stakeholder group.

The following discussion summarizes the key implementation considerations for each of the indicators noted as having benchmarking potential.

Time to Hire

With respect to "time to hire", there are several implementation considerations for benchmarking this indicator with other jurisdictions. First, there are a range of definitions for "time to hire" which vary across jurisdictions. As an example, Australia measures "time to hire" from the point of application submission to advisement on success. In the U.S., "time to hire" is measured from the time that an announcement closes to the date that the offer is extended. Finally, in B.C., the process being measured begins when the ad is posted until the start date of the new hire. Given that PSC intends to undertake an in-depth analysis of "time to hire" (including time analysis studies of each stage in the staffing process using multiple definitions), it will likely be feasible to achieve comparability with some other jurisdictions.

Given the intended broad and detailed scope of study for "time to hire", another implementation consideration for benchmarking is to consider the specific measure that can be used to monitor "time to hire" on an ongoing basis following the PSEA evaluation. While a broad and detailed study may be appropriate in support of legislative review, it will likely be much more feasible to select a single "time to staff" indicator which can be used on an annual basis to support PSC’s oversight role.

Finally, Australia and the United States demonstrate relatively mature performance measurement practices with respect to "time to hire" demonstrated in the use of specific targets and public reporting on achievement of results. There is benefit in beginning benchmarking discussions with these two jurisdictions from the perspective of lessons learned with respect to leading staffing transformations and monitoring progress.

Quality of Hire

Based on information collected with comparative jurisdictions, there is notable interest in "quality of hire" as an indicator. The majority of jurisdictions are at the very early stages of developing specific measurement approaches for this indicator. Given that these jurisdictions are generally at the "early stage", an opportunity exists for PSC to work collaboratively with a collection of jurisdictions to develop a common definition and measurement approach. The three jurisdictions that demonstrated some progress or interest in this indicator include the U.S., Ontario and British Columbia. From a Canadian perspective, efforts are already underway to collaborate in the area of HR performance measurement and there may be an opportunity to leverage these efforts to work with benchmarking partners within Canada.

Employee, applicant and manager perspective on staffing values

As a result of there being commonality across jurisdictions related to the values of the staffing process, there is a group of indicators which are intended to assess employee, applicants and manager perspective on staffing values. These perspectives are typically measured through the use of surveys. A key implementation consideration in relation to this group of indicators concerns the specific definitions that are used by other jurisdictions. As expected there is some variation across jurisdictions. The best opportunity for benchmarking on staffing values appears to be with Australia which uses similar statements in their employee surveys, particularly in relation to fairness and merit (the extent to which merit criteria are the basis for appointment). The United States also has a focus on merit which appears to be limited to understanding employee perception of whether merit is applied for promotions. Key implementation considerations include the level of application of values being sought.

Employee, applicant and manager satisfaction with the hiring process

The final potential area for benchmarking relates to employee, applicant and manager satisfaction with the hiring process. The U.S., Ireland and Australia actively invest in efforts to understand the employee/applicant perspective, typically administered through an online survey. The most common dimension of candidate/applicant satisfaction measured relates to the timeliness of various stages within the recruitment process or the length of the recruitment process overall. Australia, however, does include some questions which go beyond topics of timeliness to assess whether applicants had opportunities to seek feedback and overall impressions with the (hiring) agency. The focus for managers is on identifying opportunities to improve the recruitment process and whether results are achieved (i.e. attract and retain employees).

5. Assessment of the PSC Framework

5.1 Overview of Assessment Approach

The PSC Framework (Appendix A) is assessed from three perspectives—the framework design, the framework measures, and the measures within each specific PSEA staffing objective. The criteria used for each of these perspectives is outlined in the table below.

Perspective Assessment Criteria
Framework Design
  • Is the focus for the measurement framework related to the strategic plan or objectives?
  • Is the framework designed to include both drivers (determinants) of performance and outcomes and results?
  • Does the framework provide for frequency of measurement which is consistent with availability of data and need for decision-making?
  • Does the framework allow for measures related to both the overall effectiveness of recruitment and staffing and measures related to the implementation of PSEA?
Framework Measures
  • Does the framework contain a balance between leading and lagging indicators?
  • Does the framework contain a balance between oversight and operational perspectives?
  • Does the framework have a range of measures that will allow for reporting to various stakeholders, e.g. internally at PSC, Departments, Parliamentarians?
  • Can the data for the measures be obtained in a cost-effective manner?
Measures within each Specific Objective
  • Is the performance measure measurable, i.e. defined to a level that others can understand how the indicator will be measured?
  • Does the performance measure support a PSEA staffing objective or result? Are all elements of the PSEA staffing objective and results considered?
  • Is the measure designed to be used on an ongoing basis to assess performance?
  • Were any other performance indicators identified through the multi-jurisdictional study relevant for consideration?

5.2 Context for the Assessment of the PSC Framework

The stated purpose and scope of the PSC Framework should inform the design of the Framework and the selection of measures. Accordingly, the evaluation of the appropriateness of the framework and relevance of identified measures considers the following stated purpose and scope. (Source: PSC Effectiveness Measurement Framework - Results-driven Accountability)

Purpose

The PSC Effectiveness Measurement Framework consists of a core set of activities and integrated HR metrics that comprise the oversight strategy of the Public Service Commission.

The Framework is a change management tool. It measures progress in implementing those aspects of the Public Service Employment Act (PSEA) for which PSC is responsible.

Scope

The PSC Effectiveness Measurement Framework:

  • establishes meaningful ongoing performance measures of progress in implementing the PSEA;
  • Gathers critical benchmarking data (international and domestic);
  • Contributes to continuous improvement of PSC oversight activities based upon key success factors; and
  • Informs the five-year evaluation of the PSEA.

5.3 Assessment of the Design of the PSC Framework

Appendix A contains a summary of the Lead Indicators and the PSEA Five-Year Evaluation Framework. The following table provides a summary assessment of the design of the framework.

Assessment Criteria Assessment
Is the focus for the measurement framework related to the strategic plan or objectives?
  • The framework is aligned with the PSEA staffing objectives and specifies the results to be achieved.
  • The logic between the PSEA staffing objectives and PSEA results stated in the framework is not always clearly evident. For example, PSEA staffing objectives related to enabling infrastructure are connected to three different results and these connections may not be immediately clear to all readers. As such, the PSC Framework is not a stand-alone document and needs some additional contextual explanation including explicit statements of related key change areas (e.g. stronger emphasis on results: flexibility/efficiency) and a depiction of the linkages between the PSEA staffing objectives and PSEA results contained within the framework.
Is the framework designed to include both drivers of performance and outcomes and results?
  • The framework has a results-based approach with key outcomes defined in a logic model.
  • The framework includes both leading indicators based on critical success factors in achieving the staffing objectives and assessment of the PSEA outcomes through use of retrospective indicators. The leading indicators tend to focus on the drivers or levers of performance and the retrospective indicators focus mostly on outcomes and results. In some cases, the retrospective indicators appropriately focus on an in-depth analysis of the drivers of performance to better understand the causal factors of achievement of objectives.
Does the framework provide for frequency of measurement which is consistent with availability of data and need for decision-making?
  • The framework provides for measurement of the leading indicators with some information for this measurement sourced from the annual Staffing Management Accountability Framework.
  • The source of data and timing of measurement for some leading indicators is unclear. For example, the retrospective indicators are identified for the five year evaluation. Some of the indicators have more frequent intervals of measurement, such as the perception of public service employees about competency of people hired (bi-annual). There is a need to clarify the intended frequency of measurement for both leading and retrospective indicators.
  • With respect to retrospective indicators used to demonstrate the achievement of results, the framework sets out a five-year evaluation of the PSEA legislation. Consideration should
Does the framework allow for measures related to both the overall effectiveness of recruitment and staffing and measures related to the implementation of PSEA?
  • The framework provides for tracking on the progress with critical success factors and short-term outcomes of the implementation of PSEA.
  • The framework includes a retrospective view of the achievement of PSEA staffing objectives as part of a five-year evaluation of the PSEA legislation. Monitoring of the overall effectiveness of the recruitment and staffing process could be achieved through monitoring and tracking of some of the retrospective indicators on a more regular basis.

Overall, the measurement framework has several strengths, including the specification of PSEA staffing objectives and results to be achieved, and the use of a results-based approach. The linkage to the annual Staffing Management Accountability Framework (SMAF) is also viewed as a strength in terms of alignment with departmental activities and as a source for data to inform performance monitoring.

Opportunities to enhance the framework include:

  • Ensuring clarity of the logic between the PSEA staffing objectives and PSEA results stated in the framework, as noted above; and
  • Considering whether some of the retrospective indicators could be monitored on an annual basis to track the achievement of outcomes/results. Examples include source of hires, PSST complaints, number of delegations restricted, quality of hire and time to staff. Availability of data is a key decision criteria in determining whether indicators can be monitored on a more frequent basis.

5.4 Assessment of the Framework Measures (Overall)

The following table provides an overall assessment of the framework measures. A more detailed assessment considering each measure is provided in the section that follows.

Assessment Criteria Assessment
Does the framework contain a balance between leading and lagging indicators? Overall, there is generally a balance of leading and lagging indicators across the framework.
Some of the staffing objectives related to enabling infrastructure appear to have a greater emphasis on retrospective indicators.
Does the framework contain a balance between oversight and operational perspectives? Overall, there is generally a balance of oversight and operational perspectives, as is appropriate given the purpose of the effectiveness framework and PSC's need to understand both perspectives. A more detailed analysis examining to what extent the measures provide both an oversight and operational perspectives is provided below.
Does the framework have a range of measures that will allow for reporting to various stakeholders, e.g. internally at PSC, Departments, Parliamentarians? There is a good balance of measures which would allow for reporting to a variety of stakeholders including PSC, Parliamentarians and Departments.
Measures which capture the perspective of common interest to all stakeholders include the following:
  • Time to hire
  • Quality of hire
  • Public Service employee satisfaction with the duration of the process
  • Appointees agreement with SoA Q32b (Internal appointments (actings, terms, promotions) are made fairly.)
  • Perceptions of Merit—Perception of public service employees about competency of people hired
  • Satisfaction of candidates and managers with process by region
  • Perception of public service employees re: transparency of appointment process
Can the data for the measures be obtained in a cost-effective manner? Many of the leading indicators defined in this framework can be sourced through annual reporting from departments through the Staffing Management Accountability Framework. PSC has been identified as the source for other leading indicators and the approach to collecting the data is not defined in the framework.
The retrospective indicators may require the administration of new data collection methods such as surveys and/or reliance on departments for additional information. Some studies, evaluations and audit will also be used to support this framework for effectiveness measurement.

5.4.1 Assessment of Balance Between Oversight and Operational Perspectives

PSC is responsible for providing oversight of the integrity of the staffing process. In this role, PSC is responsible for developing policies and guidelines to be used by public service managers, holding Deputy Ministers and their managers accountable for their staffing decisions, and confirming the effectiveness of the staffing system through audits and investigations.

Given PSC’s role, the PSC Framework needs to provide both oversight and operational perspectives. Oversight measures provide PSC with the information required to carry out their responsibilities with respect to governance of the staffing process. Oversight measures allow PSC to understand legislative compliance, as well as whether the process is producing desired strategic outcomes. Operational measures provide information on the inputs, processes and outputs of staffing activities at departmental levels, allowing PSC to understand the causal factors or ‘levers’ that impact the outcomes and results of the staffing process.

An analysis of the PSC Framework shows that the majority of PSC indicators (65%) are oversight focused. These oversight indicators are predominantly focused on outcomes related to the PSEA staffing objectives and results. Approximately one-third of the indicators are focused on operational aspects of the staffing process. The operational measures provide insight into all of the major components of the staffing process, looking at inputs, resource consumption, process, outputs and outcomes.

Figure 1 - PSC Indicators

Figure 1 - PSC Indicators

Figure 1 - long description

Figure 2 - Operational Measures

Figure 2 - Operational Measures

Figure 2 - long description

Figure 3 - Oversight Measures

Figure 3 - Oversight Measures

Figure 3 - long description

Overall, this balance appears to be appropriate given PSC’s primary role is oversight of the staffing regime.

Consideration to increasing the proportion of input and resource consumption measures in the oversight category may be appropriate for the five-year evaluation to better understand the implementation of the legislative changes.

5.5 Assessment of Measures within Specific Objectives

A detailed assessment of all indicators within the PSC Framework is included in Appendix B of this document. This section of the report provides a summary of the opportunities for change that can be made to the Framework, including the identification of indicators that could be removed or revised. Analysis of the current indicators also identified some indicators that would be better repositioned within the Framework, i.e. moved to be aligned with a different PSEA Staffing Objective as a better indicator of the desired result. Additional observations from the assessment process are documented for future consideration by PSC. Finally, the section concludes with the identification of indicators from other jurisdictions that are contextually relevant for the PSC and could be included in the Framework.

5.5.1 Indicators to be Removed from the Framework

Several indicators can be removed from the framework, either because they are not relevant to the PSEA staffing objective or a stronger (related) indicator already exists in support of another PSEA staffing objective. A brief rationale is noted in the table below and a more detailed rationale for the removal of these indicators is included in Appendix B.

The following indicators can be removed from the Framework:

PSEA Staffing Objective and Result Indicators to be Removed and Rationale
PSEA Staffing Objective: HR recruitment needs addressed

PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
Number of acting appointments
(Indicator related to acting appointments already included under "Fairness")
Percentage casuals/terms going to indeterminate
(Indicator related to acting appointments already included under "Fairness")
Extent to which Value-for Money is achieved through better planning:
  • Percentage of staffing resources spent
(Relevance to this staffing objective is not evident)
PSEA Staffing Objective: Values-based staffing regime - Accessible

PSEA Result: Employees and the public are confident that hiring to and within the public service is accessible
Managers response to SoA Q27 ("Through what route did the appointee enter into consideration?")

(Other indicators already provide information on the use of advertised and non-advertised processes)
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: PSC provides departments with effective coordination and learning.
Perceptions of HRC members on the relevance and timelines of policy information

(Other indicator in this section already covers this concept)

Turnover rate of HR FTEs
(Other indicator in this section already covers this concept)

5.5.2 Indicators to be Revised

The assessment of the PSC Framework identified several opportunities to make refinements to the current indicators. The table below proposes revisions to fifteen indicators from various components of the Framework. The rationale for the revision is noted in Appendix B which contains detailed assessments of each indicator from the Framework.

PSEA Staffing Objective and Result Original Indicator Proposed Revisions
PSEA Staffing Objective: HR recruitment needs addressed

PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
Trends of spending on HR services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/HR transactions) Trends of spending on staffing services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/staffing transactions)
PSEA Staffing Objective: Values-based staffing regime - Accessible

PSEA Result: Employees and the public are confident that hiring to and within the public service is accessible
Restrictiveness of internal staffing appointments (Use of non-advertised processes) Ratio of internal non-advertised processes to total internal processes
PSEA Staffing Objective: Values-based staffing regime - Representative

PSEA Result: Employees and the public are confident that hiring to and within the public service is representative
Staffing strategies regarding EEs Proportion of departments that have staffing strategies regarding EEs
PSEA Staffing Objective: Values-based staffing regime - Fair

PSEA Result: Employees and the public are confident that hiring to and within the public service is fair
Appointees agreement with SoA Q32b ("Internal appointments (actings, terms, promotions) are made fairly") Candidate perception of the fairness of the internal appointment process (actings, terms, promotions)
PSEA Staffing Objective: Values-based staffing regime - Transparent

PSEA Result: Employees and the public are confident that hiring to and within the public service is transparent
SoA appointee Q32a ("Internal appointments are carried out in an open and transparent way") Candidate perception of the openness and transparency of the internal appointment process
PSEA Staffing Objective: Values-based staffing regime - Merit-based appointments

PSEA Result: Timely, merit-based staffing that allows the public service to meet the current and future needs of Canadians.
Appointees agreement with the SoA Q32c & Q32d ("The statement of qualifications or merit criteria reflects the requirements of the position to be filled" and "The posted qualifications and criteria for positions are bias-free and barrier-free") Candidate perception of whether merit criteria reflect requirements of the position to be filled

Candidate perception of whether position qualifications and criteria are bias-free and barrier-free
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: PSC provides departments with effective coordination and learning.
Reduced turnover in HR experts

Views of HR community on quality of advice, and service orientation of HR staffing services at the PSC, and within the department
Reduce turnover in staffing experts

Views of HR community on quality of advice, and service orientation of HR staffing services
Number of HR staff per staffing actions Number of staffing actions per staffing specialist
Number of HR staff per FTEs Number of staffing specialists per FTEs
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: Accountabilities are respected. Problems are corrected. Opportunities for improvement are actively pursued.
Percentage of external audit recommendations implemented (target: 100%) Percentage of external audit recommendations implemented within the specified period (target: 100%)
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: Managers have flexibility in how they use their delegated authority, supported by departmental HR plans linked to overall departmental business planning.
PS employee satisfaction with duration of appointment process

Count & percentage of priority referrals appointed

Count and percentage of qualified candidates in collective processes appointed and not appointed
Manager satisfaction with the duration of the appointment process; Candidate satisfaction with the duration of the appointment process

Percentage of appointments sourced through priority referrals

Percentage of appointments sourced from collective processes

In addition to the revisions above, there are several other opportunities noted to refine some of the indicators to clarify the intent, as it is not clear from the limited description provided in the current Framework. Based on the current description, these indicators appear to be relevant to the PSEA staffing objective, however, some revision or further elaboration may be required to ensure that all users of this framework can understand the intent and purpose of the indicator. An example is "The proportion of appointments to the job that contributed towards meeting planned diversity objectives not currently or previously met for the job or organization".  Based on the analysis conducted, there are approximately seven indicators that PSC will need to revise or elaborate on, based on the intended use of the indicator. Refer to Appendix B for specific details.

5.5.3 Changes in Assignment of Indicators to PSEA Staffing Objectives and Results

Analysis of the current indicators also identified some that would be better repositioned within the Framework, i.e. moved to be aligned with a different PSEA Staffing Objective. The rationale for this repositioning is that the indicator is more closely linked and better demonstrates the desired result for another PSEA Staffing Objective than the one to which it was originally associated.

The table below summarizes the changes that can be made to ensure that indicators are aligned with the most appropriate PSEA Staffing Objective and Result.

Indicator Original Alignment (PSEA Staffing Objective and Result) New Alignment (PSEA Staffing Objective and Result)
Trends of spending on staffing services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/staffing transactions) PSEA Staffing Objective: HR recruitment needs addressed

PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: Managers have flexibility in how they use their delegated authority, supported by departmental HR plans linked to overall departmental business planning.

Stronger Emphasis on Results: Flexibility/Efficiency
Extent of awareness, understanding, and acceptance by stakeholders with respect to accountabilities and acceptance of the new staffing regime (e.g. culture change) PSEA Staffing Objective: Values-based staffing regime - Transparent

PSEA Result: Employees and the public are confident that hiring to and within the public service is transparent
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: The expectations and accountabilities of departments to the PSC are clear, and the PSC is responsible for appointments, oversight, and administration of political activities.
Number of staffing actions per staffing specialist

Number of staffing specialists per FTEs
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: PSC provides departments with effective coordination and learning.
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service

PSEA Result: Managers have flexibility in how they use their delegated authority, supported by departmental HR plans linked to overall departmental business planning.

Stronger Emphasis on Results: Flexibility/Efficiency

5.5.4 Additional Observations from the Assessment Process

Several other observations emerged from the detailed assessment of the indicators for each PSEA staffing objective.

Completion of activities as indicators of performance - A number of indicators identified in the framework relate to whether particular activities or actions have been completed by departments in support of the implementation of PSEA. Examples include linkage between business plans and HR plans and the existence of HR service standards. Many of these activities are important to assess progress towards implementation and to potentially identify departments that require assistance in the implementation of PSEA. These indicators can be removed from the evaluation framework when departments have demonstrated achievement of the fundamental steps, (e.g. sub-delegation instrument development and operational) towards implementing PSEA. The Departmental Staffing Accountability Report provides an ongoing mechanism to monitor expected activities within the implementation period.

Indicator related to range and use of best practices – It is noted that the Framework currently has this indicator in relation to adequacy of the process under the "Fair" PSEA Staffing Objective. The range and use of best practices is an indicator that can be applicable across many dimensions of the Framework, e.g. "Representativeness" and "HR recruitment needs are met". PSC should determine to what extent it wishes to extend the current indicator "range and use of best practices in the staffing process" to support other PSEA Staffing Objectives.

Indicators related to cost to staff (cost-effectiveness) – There are currently several indicators in the Framework related to cost-effectiveness. These measures are aligned with the "Enabling Infrastructure" PSEA Staffing Objective and the PSEA Result which focuses on flexibility in the use of delegated authority. While it is expected that cost-effectiveness indicators would be measured, the relevance of cost-effectiveness indicators is not evident in the current structure and presentation of the PSC Framework. There is a reference to "efficient, effective and economical use of resources" under the PSC Expectations framework, however, it is not clear how this is linked to the PSC Framework. There is a need to refine the PSC Framework to ensure that all readers will understand how the selection of these cost-effectiveness indicators is in support of the PSEA Staffing Objectives and Results.

5.5.5 Additions to the Framework Based on Multi-Jurisdictional Research

The multi-jurisdictional research identified a number of indicators that could potentially be considered for inclusion in the PSC Framework. Each of these indicators was reviewed and an assessment was made whether the indicator could be included in the PSC Framework. The results of this analysis are summarized in the table below.

PSEA Staffing Objective and Result New Indicator Based on Multi-Jurisdictional Research
PSEA Staffing Objective: HR recruitment needs addressed

PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
Conversion rate (Percentage of offers that are accepted by the first choice candidate)

Managerial capacity (Amount of time a manager spends on staffing in relation to other managerial responsibilities)

% of managers who agree that Government of Canada can attract required talent
PSEA Staffing Objective: Values-based staffing regime - Representative

PSEA Result: Employees and the public are confident that hiring to and within the public service is representative
Employee perception of the commitment to recruit and employ members of EE groups
PSEA Staffing Objective: Values-based staffing regime - Merit-based appointments

PSEA Result: Timely, merit-based staffing that allows the public service to meet the current and future needs of Canadians.
Number of complaints investigated by PSC that were founded
PSEA Staffing Objective: Enabling infrastructure - informed, flexible and accountable decision making in a productive public service

Accountabilities are respected. Problems are corrected. Opportunities for improvement are actively pursued.
Proportion of corrective actions identified through investigations that are implemented

6. Recommendations

The following section summarizes recommended actions for PSC to consider related to the PSC Framework.

6.1 Benchmarking

  • The following indicators emerge as having potential for benchmarking:
    • Time to hire;
    • Quality of hire;
    • Employee, applicant and manager perception of staffing values (merit; fairness, and transparency); and
    • Employee, applicant and manager satisfaction with the hiring process.
  • Engage other jurisdictions in a discussion on the potential for a common set of staffing performance measures; taking into account the implementation considerations identified in the analysis

6.2 Refine the PSC Framework and Indicators

  • The PSC should continue to engage in a broader discussion with CPSA and PCO with respect to how the PSC and its measurement practices can effectively contribute to the broader talent management agenda.
  • Identify select retrospective indicators to be monitored on an annual basis to track the achievement of outcomes/results. Examples include quality of hire and time to staff.
  • Remove redundant and less relevant indicators as noted in section 5.5.1.
  • Incorporate refinements to indicators documented in Section 5.5.2 to strengthen the Framework’s ability to assess progress achieved through the introduction of the PSEA. The most significant theme identified by the assessment is to incorporate a wider range of stakeholder perspectives in measuring satisfaction with the recruitment process. Through the use of web-based surveys, expand the current focus of measurement related to satisfaction with the recruitment process from appointees to both successful and unsuccessful candidates on dimensions of process duration and respect for values (fairness, transparency and merit).
  • Reposition indicators noted in section 5.5.3 to provide better alignment with the PSEA Staffing Objectives.
  • Remove activity-type indicators from the evaluation framework when departments have demonstrated achievement of the fundamental steps towards implementing PSEA, (e.g. sub-delegation instrument development and operational).
  • Consider to what extent the range and use of noteworthy practices in the staffing process by departments will be examined. The current framework suggests that this will be done in the context of fairness of staffing process, however, there is an opportunity to examine noteworthy practices in the context of other staffing values (e.g. transparency) and to identify innovative practices in addressing organizational recruitment needs.
  • Refine the framework to improve the relevance of the cost-effectiveness indicators.
  • Consider developing the capability to generate stakeholder specific "snapshots" of relevant indicators within the PSC Framework.

Appendix A

Lead Indicators and Five-Year Evaluation Framework

PSEA Staffing Objective: HR recruitment needs addressed
PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
Leading Indicators Evaluation analysis of in-depth indicators
Stronger emphasis on HR Planning/ Integration with business planning

  • FTE utilization v. budgeted trends
  • Extent to which risk areas (eg: EE, language, skills, and succession planning) inform recruitment targets
  • Number of acting appointments
  • Extent to which HR recruitment targets are identified and met
  • Percentage casuals/terms going to indeterminate
  • Extent to which Value-for Money is achieved through better planning:
    • Percentage of staffing resources spent
  • Key linkage - business plans/HR plans (staffing)/strategies
    • Trend analysis of replacement time (effectiveness in addressing departures from the public service over time)
 
    • Trends of spending on HR services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/HR transactions
PSEA Staffing Objective: Values-based staffing regime - Accessible
PSEA Result: Employees and the public are confident that hiring to and within the public service is accessible
Leading Indicators Evaluation analysis of in-depth indicators
Ratio of non advertised external processes to total Staffing trends
  • Types of appointment process used (source of hires) internal versus external recruitment
Ratio of External processes advertised nationally to total
  • Effectiveness of National Area of Selection
    • Percentage of jobs open to public having a national area of selection (19% in 2005, target is 35% after 2006)
Restrictiveness of internal staffing appointments (Use of non-advertised processes)
    • Sources of geographical representation of candidates versus appointees
Managers response to SoA Q27 ("Through what route did the appointee enter into consideration?")
  • Satisfaction of candidates and managers with process by region
PSEA Staffing Objective: Values-based staffing regime - Representative
PSEA Result: Employees and the public are confident that hiring to and within the public service is representative
Leading Indicators Evaluation analysis of in-depth indicators
Number and percentage of internal and external staffing notices with employment equity (EE) provisions to enhance access of EE group members to public service positions Impacts on EE plans
  • Trend and number of employment equity groups for which representation in the public service is below workforce availability (target zero, one group in 2005-06)
Staffing strategies regarding EEs Impact on Public Service representativeness of Canada's diversity
  • The proportion of appointments to the job that contributed towards meeting planned diversity objectives not currently or previously met for the job or organization
 
  • EE drop-off rates (difference between % of applications and % of appointments for EE group members)
PSEA Staffing Objective: Values-based staffing regime - Fair
PSEA Result: Employees and the public are confident that hiring to and within the public service is fair
Leading Indicators Evaluation analysis of in-depth indicators
Number and percent acting appointments that become indeterminate in the same position Perceptions of Fairness
  • Perceived fairness of the new staffing system by stakeholders (e.g. appropriateness and satisfaction with staffing mechanisms and assessment processes and tools)
  • Perception of public service employees about fairness of appointment processes (target 85%, 64% in 2005)
Number and percent of casuals becoming indeterminate through non advertised processes Adequacy of Process
  • Staffing process and related decisions are documented, justified and communicated
  • Understanding of recruitment/assessment activities, and by managers
  • Range and use of best practices by managers
Appointees agreement with SoA Q32b ("Internal appointments (actings, terms, promotions) are made fairly")  
PSEA Staffing Objective: Values-based staffing regime - Transparent
PSEA Result: Employees and the public are confident that hiring to and within the public service is transparent
Leading Indicators Evaluation analysis of in-depth indicators
HR Plan/staffing strategies on departmental website: Communicated to employees/managers Extent of awareness, understanding, and acceptance by stakeholders with respect to accountabilities and acceptance of the new staffing regime (e.g. culture change)
Organization demonstrates consultation/communication of staffing-related information to union Perception of public service employees re: transparency of appointment process (target 85%)
Length of time job openings posted Branding - perception of Canadians and identified recruitment target groups of the transparency/accessibility of public service staffing
SoA appointee Q32a ("Internal appointments are carried out in an open and transparent way")  
PSEA Staffing Objective: Values-based staffing regime - Official Languages
PSEA Result: The public service is representative of Canada's diversity and linguistic duality.
Leading Indicators Evaluation analysis of in-depth indicators
Percent of non imperative appointments where person does not meet the official language profile within prescribed time period Impact on OL plans
  • Proportion of bilingual/unilingual positions; bilingual non-imperative/bilingual imperative positions
  • Compliance with TBS Official languages policy at time of appointment
Percent of non-compliant positions with respect to official language requirements (target: 0; 242 in 2005)
PSEA Staffing Objective: Values-based staffing regime - Merit-based appointments
PSEA Result: Timely, merit-based staffing that allows the public service to meet the current and future needs of Canadians.
Leading Indicators Evaluation analysis of in-depth indicators
  • Managers satisfaction with the quality of hires SoA:Q37 ("To what extent are you generally satisfied with the quality of hire/s for the reference position/s")
Perceptions of Merit
  • Perception of public service employees about competency of people hired (target 85% in 2007-08 vs 76% in 2005)
  • PSST Complaints (abuse of authority - Merit)
Quality of Hires/Productivity
  • Manager satisfaction with hire
  • Results of performance reviews of new appointees; job retention data
  • Appointees agreement with the SoA Q32c & Q32d ("The statement of qualifications or merit criteria reflects the requirements of the position to be filled" and "The posted qualifications and criteria for positions are bias-free and barrier-free")
Assessment of Merit
  • Well-founded statements of merit criteria that are assessed and are the basis for appointment
  Review of PSST decisions
  • Number of complaints, number mediated, nature of complaints
PSEA Staffing Objective: Values-based staffing regime - Non-partisan public service
PSEA Result: Political impartiality of the public service is ensured while employee's right to engage in political activities is respected
Leading Indicators Evaluation analysis of in-depth indicators
  • Number of complaints related to political influence in staffing
  • Managers response to SoA Q42A ("To what extent did you perceive each of the following in your specific context? A. External pressure to select a particular candidate.")
  • Perceptions of Parliamentarians and Canadians on assurance of the preservation of a non-partisan public service, free from political influence
 
  • Employees aware of their rights and responsibilities with respect to political activities
 
  • Monitoring exists of the state of balance between impartiality, and public servants exercising their right to be politically active
 
  • Investigations allow correction of any incidents of improper political activity of public servants
 
  • Study findings of the effectiveness of policies, procedures and decisions related to the political activities of public service employees
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: The expectations and accountabilities of departments to the PSC are clear, and the PSC is responsible for appointments, oversight, and administration of political activities.
Leading Indicators Evaluation analysis of in-depth indicators
Delegation of Staffing to Deputy Heads
  • Depts/Agencies: sub-delegation instrument development and operational
Roles and Responsibilities
  • Perceptions on the clarity of roles and responsibilities in staffing
  • Perceptions on the utility and flexibility/ appropriateness of delegation models
  • Training/maintenance of competencies for sub-delegated managers
Understanding/Knowledge
  • Evidence of training developed and provided to managers with sub-delegated authorities; % managers having received the training
  • Temporary suspensions of delegated authority to Depts and Agencies by PSC
  • Satisfaction with relevance and effectiveness of training (perceptions of HR units and managers)
  Objectives achievement
  • Number of separate agencies fully meeting compatibility requirements. (baseline from 2005/2006, with analysis of any trends since that year); target 100%
 
  • (Number of departments fully meeting the staffing and political impartiality requirements; target: 100%
  • Number of delegations restricted or removed by the PSC (target zero, five in 2005-06)
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: PSC provides departments with effective coordination and learning.
Leading Indicators Evaluation analysis of in-depth indicators
Improved Organizational HR Service Support Systems
  • Existing number of staffing experts vs Dept/Agency requirements
  • Departments and agencies receive relevant and accurate policy information and advice in a timely manner
  • Reduced turnover in HR experts
  • Increasingly well informed community of sub-delegated managers and human resource specialists:
    • Perceptions of managers on the sufficiency of their knowledge/competence
    • Satisfaction of managers of HR support
    • Perception of HR specialists on manager competency and understanding
  • Percent of staffing experts receiving continuous training
  • Views of HR community on quality of advice, and service orientation of HR staffing services at the PSC, and within the department
  • Timelines/quality of staffing information systems
  • Existence of HR service standards and results of data
 
  • Perceptions of HRC members on the relevance and timelines of policy information
  Capacity of HR units
  • Number of HR staff per staffing actions
  • Number of HR staff per FTEs
  • Turnover rate of HR FTEs
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: PSC provides departments with effective coordination and learning.
Leading Indicators Evaluation analysis of in-depth indicators
Increased Departmental Accountability for Results
  • Variance between planned and actual: assessed/explained/reported
  • Evidence of active monitoring of HR function within the organization (example: key risk areas identified, ongoing reporting of key risks, evidence of follow-up/action plans, quality assurance of information)
  • In-house assessment of any problems (above) plus corrective action)
  • Number of small, medium and large PS organizations with delegated staffing authorities whose staffing performance did or did not fully meet all the essential staffing performance indicators identified in the SMAF, (example: # of acting/casuals/terms/ trends of HR expenditures
 
  • Percentage of external audit recommendations implemented (target: 100%)
 
  • Percentage of coverage of internal oversight activities (monitoring, audits, studies and evaluations) in each risk category
PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: Managers have flexibility in how they use their delegated authority, supported by departmental HR plans linked to overall departmental business planning.
Leading Indicators Evaluation analysis of in-depth indicators
Stronger Emphasis on Results: Flexibility/Efficiency
  • Manager satisfaction with flexibility in making appointments
Perceptions on flexibility/barriers to staffing
  • Views of HR specialists and hiring managers on time to staff and related issues such as barriers to staffing
  • Vacancy time for positions staffed
Value-for Money - Time to staff (Efficiency)
  • Time analysis studies of each stage in staffing process for different selection processes (individual versus collective, external versus internal, by selected classification types) to identify barriers to staffing using multiple definitions for time-to staff:
  • PS employee satisfaction with duration of appointment process
Value-for-Money - Cost to Staff (Cost-effectiveness)
  • Traditional cost per hire metrics
    • Ratio of direct and indirect staffing costs in dollars to the number of persons hired
 
  • Count & percentage of priority referrals appointed
 
  • Count and percentage of qualified candidates in collective processes appointed and not appointed

Appendix B

Detailed Assessment of PSC Framework Indicators

This appendix examines each of the twelve PSEA Staffing Objectives and the indicators used to assess each one to determine whether there are opportunities to refine, remove or augment some of the existing indicators within the framework.

1. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - HR Recruitment Needs Addressed

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to "HR recruitment needs addressed" is provided in the table below.

PSEA Staffing Objective: HR recruitment needs addressed
PSEA Result: Managers have flexibility in how they use their delegated authorities, supported by departmental HR plans linked to overall departmental business planning
Leading Indicators Opportunity for Improvement
Stronger emphasis on HR Planning/ Integration with business planning
  • FTE utilization v. budgeted trends
None
  • Number of acting appointments
The relevance of the indicators to the PSEA staffing objective or PSEA result is not directly evident.

An indicator related to acting appointments appears to be more appropriate in support of an assessment of the fairness of the new staffing regime. Such an indicator is already noted in that element of framework (section 4 of this Appendix).

Recommend that this indicator be removed from the framework.
  • Percentage casuals/terms going to indeterminate
The relevance of the indicators to the PSEA staffing objective or PSEA result is not directly evident. Managers may choose to exercise flexibility by hiring casuals or terms into indeterminate positions through advertised processes.

An indicator related to casuals/terms going to indeterminate appears to be more appropriate in support of an assessment of the fairness of the new staffing regime. Such an indicator is already noted in that element of framework (section 4 of this Appendix).

Recommend that this indicator be removed from the framework.
  • Key linkage - business plans/HR plans (staffing)/strategies
This activity is important to monitor to assess progress towards implementation, although it is not a measure per se.
Evaluation analysis of in-depth indicators Opportunity for Improvement
  • Extent to which risk areas (eg: EE, language, skills, and succession planning) inform recruitment targets
The intent of the measure as stated is not clear.
  • Extent to which HR recruitment targets are identified and met
The intent of the measure as stated is not clear.
  • Extent to which Value-for Money is achieved through better planning:
    • Percentage of staffing resources spent
The relevance of this measure to the PSEA staffing objective or PSEA result is not directly evident.

Recommend that this indicator be removed from the framework.
    • Trend analysis of replacement time (effectiveness in addressing departures from the public service over time)
None
    • Trends of spending on HR services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/HR transactions
The relevance of this indicator to this particular staffing objective or result is not clear. Trends in spending may be more appropriately aligned with the PSEA staffing objective "Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service" where the focus is on stronger emphasis on result: flexibility and efficiency (section 12 of this Appendix).

Recommend that this indicator be revised with a focus on staffing and realigned with the above-noted PSEA staffing objective:
  • Trends of spending on staffing services and planning (benchmarked over time, against like-type organizations, and per-capita FTEs/staffing transactions)

2. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Accessible

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to an "accessible" staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Accessible
PSEA Result: Employees and the public are confident that hiring to and within the public service is accessible
Leading Indicators Opportunity for Improvement
Ratio of non advertised external processes to total None
Ratio of External processes advertised nationally to total None
Restrictiveness of internal staffing appointments (Use of non-advertised processes) Opportunity to consider changing the indicator to focus on the desired behaviour.

Recommend the indicator be revised as follows:

Ratio of internal non-advertised processes to total internal processes
Managers response to SoA Q27 ("Through what route did the appointee enter into consideration?") Given that other leading indicators already provide information on the use of advertised and non-advertised processes, recommend that this indicator be removed from the framework.
Leading Indicators Opportunity for Improvement
Staffing trends
  • Types of appointment process used (source of hires) internal versus external recruitment
None
Geographical representation
  • Effectiveness of National Area of Selection
    • Percentage of jobs open to public having a national area of selection (19% in 2005, target is 35% after 2006)
None
    • Sources of geographical representation of candidates versus appointees
None
  • Satisfaction of candidates and managers with process by region
None

3. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Representative

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to a "representative" staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Representative
PSEA Result: Employees and the public are confident that hiring to and within the public service is representative
Leading Indicators Opportunity for Improvement
Number and percentage of internal and external staffing notices with employment equity (EE) provisions to enhance access of EE group members to public service positions None
Staffing strategies regarding EEs The intent of the measure as stated is not clear.

If concerned with incidence of staffing strategies, recommend the indicator be revised as follows:

Proportion of departments that have staffing strategies regarding EEs
Leading Indicators Opportunity for Improvement
Impacts on EE plans
  • Trend and number of employment equity groups for which representation in the public service is below workforce availability (target zero, one group in 2005-06)
None
Impact on Public Service representativeness of Canada's diversity
  • The proportion of appointments to the job that contributed towards meeting planned diversity objectives not currently or previously met for the job or organization
The intent of the measure as stated is not clear.
  • EE drop-off rates (difference between % of applications and % of appointments for EE group members)
None

4. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Fair

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to a fair staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Fair
PSEA Result: Employees and the public are confident that hiring to and within the public service is fair
Leading Indicators Opportunity for Improvement
Number and percent acting appointments that become indeterminate in the same position None
Number and percent of casuals becoming indeterminate through non advertised processes None
Appointees agreement with SoA Q32b ("Internal appointments (actings, terms, promotions) are made fairly") Opportunity to expand this metric to consider perceptions of all candidates, not just appointees.

Recommend that the indicator be refined, as follows:

Candidate perception of the fairness of the internal appointment process
Leading Indicators Opportunity for Improvement
Perceptions of Fairness
  • Perceived fairness of the new staffing system by stakeholders (e.g. appropriateness and satisfaction with staffing mechanisms and assessment processes and tools)
  • Perception of public service employees about fairness of appointment processes (target 85%, 64% in 2005)
None
Adequacy of Process
  • Staffing process and related decisions are documented, justified and communicated
  • Understanding of recruitment/assessment activities, and by managers
  • Range and use of best practices by managers
The range and use of best practices by managers could also be applicable across many PSEA staffing objectives, for example, the PSEA staffing objective related to enabling infrastructure (flexibility/efficiency).

5. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Transparent

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to a "transparent" staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Transparent
PSEA Result: Employees and the public are confident that hiring to and within the public service is transparent
Leading Indicators Opportunity for Improvement
HR Plan/staffing strategies on departmental website: Communicated to employees/managers This activity is important to monitor to assess progress towards implementation, although it is not a measure per se
Organization demonstrates consultation/communication of staffing-related information to union This activity is important to monitor to assess progress towards implementation, although it is not a measure per se.
Length of time job openings posted None
SoA appointee Q32a ("Internal appointments are carried out in an open and transparent way") Opportunity to refine this metric to consider perceptions of all candidates, not just appointees.

Recommend that the indicator be refined, as follows:

Candidate perception of the openness and transparency of the internal appointment process
Evaluation analysis of in-depth indicators Opportunity for Improvement
Extent of awareness, understanding, and acceptance by stakeholders with respect to accountabilities and acceptance of the new staffing regime (e.g. culture change) The relevance of this indicator to this specific PSEA staffing objective or PSEA result is not directly evident. This indicator may be more aligned with the PSEA objective related to Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service (section 9 of this Appendix).
Perception of public service employees re: transparency of appointment process (target 85%) None
Branding - perception of Canadians and identified recruitment target groups of the transparency/accessibility of public service staffing The reference to "branding" in this context is not clear.

6. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Official Languages

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to "official languages" is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Official Languages
PSEA Result: The public service is representative of Canada's diversity and linguistic duality.
Leading Indicators Opportunity for Improvement
Percent of non imperative appointments where person does not meet the official language profile within prescribed time period None
Evaluation analysis of in-depth indicators Opportunity for Improvement
Impact on OL plans
  • Proportion of bilingual/unilingual positions; bilingual non-imperative/bilingual imperative positions
  • Compliance with TBS Official languages policy at time of appointment
  • Percent of non-compliant positions with respect to official language requirements (target: 0; 242 in 2005)
None

7. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Merit-based Appointments

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to "merit-based appointments" is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Merit-based appointments
PSEA Result: Timely, merit-based staffing that allows the public service to meet the current and future needs of Canadians.
Leading Indicators Opportunity for Improvement
Managers satisfaction with the quality of hires SoA:Q37 ("To what extent are you generally satisfied with the quality of hire/s for the reference position/s") None
PSST Complaints (abuse of authority - Merit) None
  • Appointees agreement with the SoA Q32c & Q32d ("The statement of qualifications or merit criteria reflects the requirements of the position to be filled" and "The posted qualifications and criteria for positions are bias-free and barrier-free")
There is an opportunity to refine this indicator to consider perceptions of all candidates, not just appointees

Recommend that the indicator be refined, as follows:

Candidate perception of whether merit criteria reflect requirements of the position to be filled

Candidate perception of whether position qualifications and criteria are bias-free and barrier-free
Evaluation analysis of in-depth indicators Opportunity for Improvement
Perceptions of Merit
  • Perception of public service employees about competency of people hired (target 85% 2007-08 vs 76% in 2005)
None
Quality of Hires/Productivity
  • Manager satisfaction with hire
  • Results of performance reviews of new appointees; job retention data
There is an opportunity to refine this indicator to specify the time when manager satisfaction or new hire performance is assessed, e.g. performance after one year?
Assessment of Merit
  • Well-founded statements of merit criteria that are assessed and are the basis for appointment
There is an opportunity to provide greater definition of the performance measure and the planned approach to measurement in relation to this indicator
Review of PSST decisions
  • Number of complaints, number mediated, nature of complaints
None

8. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Non-partisan public service

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to "non-partisan public service" is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Non-partisan public service
PSEA Result: Political impartiality of the public service is ensured while employee's right to engage in political activities is respected
Leading Indicators Opportunity for Improvement
  • Number of complaints related to political influence in staffing
  • Managers response to SoA Q42A ("To what extent did you perceive each of the following in your specific context? A. External pressure to select a particular candidate.")
None
Evaluation analysis of in-depth indicators Opportunity for Improvement
  • Perception of public service employees about competency of people hired (target 85% 2007-08 vs 76% in 2005)
None
  • Employees aware of their rights and responsibilities with respect to political activities
None
  • Monitoring exists of the state of balance between impartiality, and public servants exercising their right to be politically active
There is an opportunity to refine this indicator, i.e. how is the right balance between these two activities defined
  • Investigations allow correction of any incidents of improper political activity of public servants
Number of investigations related to improper political activity of public servants which result in corrective action
  • Study findings of the effectiveness of policies, procedures and decisions related to the political activities of public service employees
None

9. Assessment of Indicators Related to PSEA Staffing Objective: Values-based Staffing Regime - Enabling Infrastructure

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to enabling infrastructure service staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: The expectations and accountabilities of departments to the PSC are clear, and the PSC is responsible for appointments, oversight, and administration of political activities.
Leading Indicators Opportunity for Improvement
Delegation of Staffing to Deputy Heads
  • Depts/Agencies :sub-delegation instrument development and operational
This activity is important to monitor to assess progress towards implementation, although it is not a measure per se
  • Training/maintenance of competencies for sub-delegated managers
This activity is important to monitor to assess progress towards implementation, although it is not a measure per se
  • Temporary suspensions of delegated authority to Depts and Agencies by PSC
None
Evaluation analysis of in-depth indicators Opportunity for Improvement
Roles and Responsibilities
  • Perceptions on the clarity of roles and responsibilities in staffing
  • Perceptions on the utility and flexibility/ appropriateness of delegation models
None
Understanding/Knowledge
  • Evidence of training developed and provided to managers with sub-delegated authorities; % managers having received the training
This measure appears to be a leading indicator, rather than a lagging indicator.
  • Satisfaction with relevance and effectiveness of training (perceptions of HR units and managers)
None
Objectives achievement
  • Number of separate agencies fully meeting compatibility requirements. (baseline from 2005/2006, with analysis of any trends since that year); target 100%
None
  • Number of departments fully meeting the staffing and political impartiality requirements; target: 100%
  • Number of delegations restricted or removed by the PSC (target zero, five in 2005-06)
None

10. Staffing Regime - Enabling Infrastructure - Informed, flexible, and accountable decision making resulting in a productive public service

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to enabling infrastructure-Informed, flexible, and accountable decision making resulting in a productive public service staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: PSC provides departments with effective coordination and learning.
Leading Indicators Opportunity for Improvement
Improved Organizational HR Service Support Systems
  • Existing number of staffing experts vs Dept/Agency requirements
None
  • Reduced turnover in HR experts
Recommend that the indicator be refined to focus on staffing:

Reduce turnover in staffing experts
  • Percent of staffing experts receiving continuous training
None
  • Timelines/quality of staffing information systems
The intent of the indicator as stated is not clear.
Evaluation analysis of in-depth indicators Opportunity for Improvement
  • Departments and agencies receive relevant and accurate policy information and advice in a timely manner
None
  • Increasingly well informed community of sub-delegated managers and human resource specialists:
    • Perceptions of managers on the sufficiency of their knowledge/competence
    • Satisfaction of managers of HR support
    • Perception of HR specialists on manager competency and understanding
None
  • Views of HR community on quality of advice, and service orientation of HR staffing services at the PSC, and within the department
The relevance of the views of the HR community on quality of advice "within the department" to the PSEA staffing objective or PSEA result is not directly evident.

Recommend that the indicator be refined, as follows:

Views of HR community on quality of advice, and service orientation of HR staffing services at the PSC
  • Existence of HR service standards and results of data
This activity is important to monitor to assess progress towards implementation, although it is not a measure per se.
  • Perceptions of HRC members on the relevance and timelines of policy information
This indicator appears to be the same as above-Departments and agencies receive relevant and accurate policy information and advice in a timely manner.

Recommend that this indicator be removed from the framework.
Capacity of HR units
  • Number of HR staff per staffing actions
  • Number of HR staff per FTEs
  • Turnover rate of HR FTEs
Two of these indicators - Number of HR staff per staffing actions and Number of HR staff per FTEs relate more to efficiency which may be more appropriately aligned with the PSEA staffing objective "Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service" where the focus is on stronger emphasis on result: flexibility and efficiency (section 12 of this Appendix)

Recommend that this indicator be revised with a focus on staffing and realigned with the above-noted PSEA staffing objective:
  • Number of staffing actions per staffing specialist
  • Number of staffing specialists per FTEs
Recommend that the following indicator be removed from the framework.

The indicator - Turnover rate of HR FTEs - is already covered as a leading indicator above.

11. Staffing Regime - Enabling Infrastructure - Informed, flexible, and accountable decision making resulting in a productive public service

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to enabling infrastructure - Informed, flexible, and accountable decision making resulting in a productive public service staffing regime is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: Accountabilities are respected. Problems are corrected. Opportunities for improvement are actively pursued.
Leading Indicators Opportunity for Improvement
Increased Departmental Accountability for Results
  • Variance between planned and actual: assessed/explained/reported
The intent of the indicator as stated is not clear.
  • In-house assessment of any problems (above) plus corrective action
This activity is important to monitor to assess progress towards implementation, although it is not a measure per se.
Evaluation analysis of in-depth indicators Opportunity for Improvement
  • Evidence of active monitoring of HR function within the organization (example: key risk areas identified, ongoing reporting of key risks, evidence of follow-up/action plans, quality assurance of information)
None
  • Number of small, medium and large PS organizations with delegated staffing authorities whose staffing performance did or did not fully meet all the essential staffing performance indicators identified in the SMAF, (example: # of acting/casuals/terms/trends of HR expenditures
This indicator appears to be an index of many other indicators used throughout the measurement framework. As such, it appears to be an overall analysis, rather than a single indicator per se.
  • Percentage of external audit recommendations implemented (target: 100%)
There is an opportunity to refine this measure by assessing whether the external audit recommendations are implemented within the specified period.

Recommend that the indicator be revised as follows:

Percentage of external audit recommendations implemented within the specified period (target: 100%)
  • Percentage of coverage of internal oversight activities (monitoring, audits, studies and evaluations) in each risk category
The intent of the indicator as stated is not clear.

12. Staffing Regime - Enabling Infrastructure - Informed, flexible, and accountable decision making resulting in a productive public service

An assessment of the leading and retrospective indicators designed for the evaluation of the PSEA staffing objective related to "enabling infrastructure-Informed, flexible, and accountable decision making resulting in a productive public service" is provided in the table below.

PSEA Staffing Objective: Values-based staffing regime - Enabling infrastructure - informed, flexible, and accountable decision making resulting in a productive public service
PSEA Result: Managers have flexibility in how they use their delegated authority, supported by departmental HR plans linked to overall departmental business planning.
Leading Indicators Opportunity for Improvement
Stronger Emphasis on Results: Flexibility/Efficiency
  • Manager satisfaction with flexibility in making appointments
None
  • Vacancy time for positions staffed
None
  • PS employee satisfaction with duration of appointment process
The indicator can be refined to assess satisfaction with the duration of the appointment process from the perspective of key stakeholders, including managers and candidates.

Recommend that the indicator be revised as follows:

Manager satisfaction with the duration of the appointment process Candidate satisfaction with the duration of the appointment process
Evaluation analysis of in-depth indicators Opportunity for Improvement
Perceptions on flexibility/barriers to staffing
  • Views of HR specialists and hiring managers on time to staff and related issues such as barriers to staffing
None
Value-for Money - Time to staff (Efficiency)
  • Time analysis studies of each stage in staffing process for different selection processes (individual versus collective, external versus internal, by selected classification types) to identify barriers to staffing using multiple definitions for time-to- staff.
There is an opportunity to use this measure as a leading indicator.
Value-for-Money - Cost to Staff (Cost-effectiveness)
  • Traditional cost per hire metrics
    • Ratio of direct and indirect staffing costs in dollars to the number of persons hired
The relevance of these indicators to the PSEA staffing objective or PSEA result is not directly evident, i.e. neither the objective nor the result appear to have components related to cost-effectiveness. That being said, cost-effectiveness is implied with supporting material which is given as context for the PSC Framework.
  • Count & percentage of priority referrals appointed
Recommend that the indicator be revised as follows:

Percentage of appointments sourced through priority referrals
  • Count and percentage of qualified candidates in collective processes appointed and not appointed
Recommend that the indicator be revised as follows:

Percentage of appointments sourced from collective processes The revised indicator is a measure of the effectiveness of the staffing process in using established (ready) pools of talent.

1 - 2007 - 2010 Alberta Public Service Workforce Plan. Alberta Public Service Web site. http://www.chr.alberta.ca/Practitioners/Doclist849.cfm. Retrieved on December 3, 2008. (return

2 - HR Transformation in the Ontario Public Service, Presentation to Irish Delegation; October 5, 2007 (return)

3 - Being the Best. Province of British Columbia Corporate Human Resource Plan 2007/08 - 2009/10 (return)

4 - Better, Faster, Australian Public Service Commission, 2007 (return)

5 - HCAAF Practitioners Guide. Office of Personnel Management Web site. https://www.opm.gov/hcaaf_resource_center/. Retrieved on January 29, 2008. (return)

6 - Public Appointments Service: Online Candidate Survey; November 2007. (return)

7 - HCAAF Practitioners Guide. Office of Personnel Management Web site. https://www.opm.gov/hcaaf_resource_center/. Retrieved on January 29, 2008. (return)

8 - http://www.civilservicecommissioners.org/Reference_Documents/
Recruitment_Code_Reference/ (return)

9 - Public Service Commission Directives. Staffing Operating Policy; August 2007 (return)

10 - HCAAF Practitioners Guide. Office of Personnel Management Web site. https://www.opm.gov/hcaaf_resource_center/. Retrieved on January 29, 2008. (return)

11 - http://www.cpsa-online.ie/Default.aspx?SID=7&SSID=21 (return)

12 - http://www.civilservicecommissioners.org/ (return)

13 - http://www.meritcomm.bc.ca/ (return)

14 - http://www.mspb.gov/sites/mspb/default.aspx/ (return)

15 - http://www.publicjobs.ie/en/about/psa.asp (return)

16 - http://www.cpsa-online.ie/Default.aspx?SID=4&SSID=0 (return)

17 - HR Transformation in the Ontario Public Service. Presentation to the Irish Delegation, October 5, 2007 (return)

18 - http://www.civilservicecommissioners.org/Reference_Documents/
Recruitment_Code_Reference/ (return)

19 - http://www.apsc.gov.au/stateoftheservice/index.html (return)

20 - http://www.civilservicecommissioners.org/reports/ (return)

21 - http://www.apsc.gov.au/stateoftheservice/index.html (return)

22 - http://www.cfp.gouv.qc.ca/content/view/71/163/ (return)

23 - 2006 - 2007 Self Assessment Return (return)

24 - 2007 - 2010 Alberta Public Service Workforce Plan. Alberta Public Service Web site. http://www.chr.alberta.ca/Practitioners/Doclist849.cfm. Retrieved on December 3, 2008. (return)

25 - State of the Service Report, 2006 - 2007. Australia Public Service Commission Web site. http://www.apsc.gov.au/stateoftheservice/0607/parttwostaffing.htm. Retrieved on December 7, 2007. (return)