37 Step 4: Know How to Test/Check Your Performance Measures

The ITSM Starter Kit: 2011 Edition 37 Step 4: Know How to Test/Check Your Performance Measures

Horizontal Integration of Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Ensuring Organizational Commitment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Section III: Choosing a Performance Measurement Framework Balancing Measures: One Concept, Different Frameworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 The Balanced Scorecard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 # # # # Why a Balanced Approach? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 NPR’s Balanced Measures Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 DOE’s Balanced Scorecard Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Mark Graham Brown’s Balanced Scorecard Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 DOE/EM’s Critical Few Set of Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 The Balanced Scorecard as a Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 University of California Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Houghton College Dashboard Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Leadership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Strategic Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Customer and Market Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Information and Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Human Resource Focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Process Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Business Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 The ?Critical Few” Set of Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 # Performance Dashboards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 # # # Malcolm Baldrige National Quality Award Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 # # # # # # # Section IV: Developing Performance Measures—Getting Organized Step 1: Establish the Performance Measurement Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 # Guidelines for Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Types of Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Classifications of Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Step 2: Gain an Understanding of the Jargon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 # # Step 3: Consider ?The Considerations” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Step 4: Know How to Test/Check Your Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 # # The Smart Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 The Quality Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 The Performance-Based Management Handbook Table of Contents – 2 170 Volume 2 # # Establishing an Integrated Performance Measurement System The Three Criteria Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 The Treasury Department Criteria Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Step 5: Take a Look at How Other Organizations Measure Performance . . . . . . . . . . . . . . . . . . . . . . 40 Section V: Developing Performance Measures—Sample Approaches The DOE Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 # The DOE Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 The Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 The Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Part 1: Establish Program Level Objectives That Contribute to Strategic Objectives . . . . . . . . . . 46 Part 2: Establish Performance Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 The University of California Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 # # # The Auditor General of Canada Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 # # #

Read more about 37 Step 4: Know How to Test/Check Your Performance Measures :

How To Toolkits and Certification eLearning

Other Sources:

  • Shambhala Sun – How to do Mindfulness Meditation
  • How to do smarter file searches in Windows 8 | PCWorld
  • Price, L.: How to Do Things with Books in Victorian Britain.
  • How To Toolkits and Certification eLearning, Learn more about The ITSM Starter Kit: 2011 Edition HERE:

    How To Toolkits and Certification eLearning and  37 Step 4: Know How to Test/Check Your Performance Measures

    How To Toolkits and Certification eLearning - 37 Step 4: Know How to Test/Check Your Performance Measures

    How To Toolkits and Certification eLearning and  37 Step 4: Know How to Test/Check Your Performance Measures

    How To Toolkits and Certification eLearning - 37 Step 4: Know How to Test/Check Your Performance Measures

    However, it describes what to do, not how to do it

    ITILFoundation Complete Certification Kit However, it describes what to do, not how to do it

    ITIM offers organizations a road map for improving their IT investment management processes in a systematic and organized manner.

    These process improvements are intended to • improve the likelihood that investments will be completed on time, within budget, and with the expected functionality, • promote better understanding and management of related risks, • ensure that investments are selected based on their merits by a wellinformed decision-making body, • implement ideas and innovations to improve process management, and • increase the business value and mission performance of investments.

    ITIM can be implemented as a tool for organizational improvement in a variety of ways.

    For example, an organization can create a separate improvement program, employ external assistance and support, or use the framework as a managerial support tool.

    Regardless of the implementation technique, the following important factors should be considered when using ITIM as an organizational improvement tool: • Many organizations will have a variety of selection, control, and evaluation processes in place.

    ITIM can help these organizations understand the relationships among these processes and determine the key opportunities for immediate improvements. Page 24 GAO-04-394G GAO IT Investment Management Framework 38 Section 4: Uses of ITIM • The framework uses a structured approach that identifies the key practices for creating and maintaining successful investment management processes.

    However, it describes what to do, not how to do it.

    Thus, specific implementation methods can and will vary by organization, based on specific attributes of the organization, such as size, complexity, and culture. • The developmental nature of a maturity model means that process maturation is cumulative.

    Lower-stage processes provide the foundation for upper-stage processes.

    As additional critical processes are introduced into the organization and implemented, the organization attains greater process capabilities and maturity.

    As the organization incorporates additional processes at each successive stage of maturity, it must maintain the lower-stage critical processes that it has previously implemented. • The framework depends on good project management to form the foundation of good performance measurement and the project-level control processes that underlie mature investment control processes. • Where one exists, the use of an EA is a critical frame of reference for making investment decisions, and only investments that move the organization toward its target architecture—as defined by its sequencing plan—should be approved unless a waiver is provided and/or a decision is made to modify the EA. • Critical processes initially may be implemented and practiced within individual bureaus or divisions before they are implemented and are mature across the organization. • Business process improvement initiatives are usually not themselves considered to be IT investments; they are considered to be parallel efforts that may or may not be linked to investments.

    Thus, ITIM assessments do not evaluate individual initiatives.

    However, if such initiatives include IT investments, then the investments should be subject to the organization’s investment management process. • Change management should be a cornerstone of process improvement, because culture affects the nature of investment decisions.

    Investment decisions are about change, and change affects an organization’s culture.

    For example, a decision can be creative or cautious, strategic or tactical.

    Culture emanates from the values of the organization. Page 25 GAO-04-394G GAO IT Investment Management Framework 39 Section 4: Uses of ITIM Tool for Assessing the Maturity of an Organization Just as ITIM can be used as a tool for organizational improvement, it can also be used as a standard against which to judge the maturity of an organization’s IT investment management process.

    For example, ITIM can be used to support assessments to help ensure compliance with industry standards or acceptable practices, independent reviews of organizational maturity by oversight bodies, or other external IT process reviews.

    Regardless of the specific use, however, the following important factors should be considered when using ITIM as an organizational assessment tool: • An assessment using the framework can be conducted for an entire organization (eg, an executive branch department) or for one of its lower-level divisions (eg, a branch, bureau, or agency).

    However, the unit or scope of analysis (eg, branch, bureau, agency, or department) must be defined before an ITIM assessment is conducted.

    Additionally, the assessed maturity stage for a lower-level division is not necessarily indicative of the maturity stage of a higher-level division or of the organization as a whole. • The use and interpretation of ITIM by organizations may vary with their size, culture, and organizational structure—as well as other factors.

    The overriding objective of the framework is to enable senior managers to systemically maximize the benefits of IT investments through the use of a structured investment process.

    In achieving this objective, different organizations may choose different specific implementations of the ITIM, which may be influenced by the factors mentioned above.

    For example, although ITIM addresses the organizational need to align and coordinate multiple investment boards, an organization with only one IT investment board would not need to perform the key practices associated with board alignment.

    Also, small organizations—or those with highly centralized IT management—may not require as extensive written guidance as large organizations, because their investment management processes are executed by a small, cohesive cadre of managers.

    Ultimately, each organization must use its best judgment in determining how to implement ITIM within its own context. • An organization may be concurrently implementing key practices that are associated with several maturity stages.

    In fact, key practices associated with higher stage critical processes are frequently initiated while the organization as a whole is at a lower stage of maturity.

    However, organizational maturity is determined by assessing at what Page 26 — — Page 67 GAO-04-394G GAO IT Investment Management Framework 81 Section 5: Critical Processes for the ITIM Stages •Stage 3: Developing a Complete Investment Portfolio ••Defining the Portfolio Criteria Activities Activity 1: The enterprisewide investment board approves the core IT portfolio selection criteria, including cost, benefit, schedule and risk (CBSR) criteria, based on the organization’s mission, goals, strategies, and priorities.

    The selection criteria should be linked directly to the organization’s broader mission, goals, strategies, and priorities.

    This ensures that the selected IT investments will support these larger organizational tenets and purposes.

    It is important that the criteria also take into account the organization’s IT architecture in orders to (1) avoid unwarranted overlap across investments, (2) ensure maximum systems interoperability, and (3) increase the assurance that investments align with strategy as captured in the EA.

    An organization often chooses to establish multiple portfolios to facilitate the investment process.

    This grouping of investments with similar characteristics can enable the organization to clarify the value of certain types of investments—such as infrastructure or e-government systems—by developing criteria that focus on the contribution each type of investment makes to the organization.

    Also, the organization can determine beforehand how to distribute funding across the portfolios.

    Ultimately, the investment board should assess each investment as part of the single enterprise portfolio—that is, the aggregation of all of the smaller portfolios.

    The selection criteria used for assessing and ranking individual investments and proposals should generally include the four essential investment elements: cost, benefit, schedule, and risk.

    The assessment may also include other criteria, which serves to enhance the evaluation of each investment’s strategic alignment and synergy with other projects.organizations typically focus on these four areas and develop multiple measures under each broad element. • Cost may include life cycle costs broken apart into initial costs, ongoing development costs, and indirect costs. • Benefit may include tangible benefits and intangible benefits estimated using a variety of techniques (eg, cost/benefit analyses using net present value, return on investment calculations). • Schedule may include the life cycle schedule and the schedule of benefits. Page 68 GAO-04-394G GAO IT Investment Management Framework 82 Section 5: Critical Processes for the ITIM Stages •Stage 3: Developing a Complete Investment Portfolio ••Defining the Portfolio Criteria • Risk may include investment, organizational, funding, and technical risks.

    The organization must determine how these criteria are to be used to select IT investments for the portfolio.

    Costs and benefits are both affected by risks.

    A risk-adjusted return on investment could combine all of these categories.

    The selection criteria also may include a description of an investment’s or proposal’s minimum or maximum acceptable CBSR thresholds (eg, a minimum acceptable return on investment hurdle rate or a maximum acceptable schedule length).

    An organization could use a weighting schema when creating the selection criteria.

    The organization would then derive weights for each of the broad categories, as well as any subelements related to each category.

    This would help the organization prioritize those subelements that it considers the most significant (eg, an organization that has limited experience developing systems may give technical risk a greater weight than projected cost).

    Alternatively, other risk analysis methods might incorporate the same “weighting” effect.

    The mixture of weights among the ranking criteria will vary from organization to organization.

    The weighting schema used should take into account the organization’s unique mission, capabilities, and limitations.

    The organization may also create different weighting schemas for different kinds of investments (eg, operational, infrastructure, applications development investments, R&D).

    These weights may need to be refined over time as the organization gains more operational experience using the weighting schema.

    Additionally, as a starting point, the organization may want to borrow selection criteria used by other comparable organizations.

    Ultimately, the criteria should reflect the priorities of the organization.

    Often, the most senior investment decision makers are involved in the development of these criteria.

    Activity 2: Project management personnel and other stakeholders are aware of the portfolio selection criteria.

    The criteria should be distributed to each IT investment board and all of the IT project managers, organizational planners, and any other interested parties.

    The selection criteria should be clearly addressed in funding submissions for IT projects. Page 69 GAO-04-394G GAO IT Investment Management Framework — — Internet x Daily average number of visits to Ecology’s web site Statistics on the usage of Ecology’s web site are available on the agency’s Intranet.

    From 1995 through 2005, usage as measured by the number of visits per weekday doubled every two years.

    However, over the past few years, usage has leveled off and year-over-year there has been very little change in traffic excluding bots .

    X Customer satisfaction Early in the history of Ecology’s Web site, the agency used a number of indirect measures of customer satisfaction with the agency’s web-based information and services.

    Beginning in 2003-2005, an attempt was made to measure customer satisfaction more directly.

    Initially, this effort focused on comments received via email, but no meaningful data was compiled due to a number of limitations with this approach.

    In late CY 2006 and early CY 2007, a survey form was deployed broadly on Ecology’s Web site and approximately 1,000 responses were received.

    These survey responses and other available data were used to design a benchmark usability study conducted in late CY 2007.

    This study yielded an overall average System Usability Score of 61 (0-100 scale).

    In May 2009 another on-line survey was conducted and 238 responses were received; the reported success rate in finding information improved from 57.5% in the prior survey to 80.4%. Voice/Telecommunications x Voice service costs (dial tone, SCAN, cellular, pagers) — $ per FTE per year (See Budget – IT expenditures by major functional area.) Return to Section 1 Table of Contents 1.F.

    Challenges and Opportunities Given the State of IT Currently, What Challenges Face Ecology? Staff and clients of business and government alike demand improved access to needed information and services.

    IT lifecycles are relatively short.

    This environment has existed for a number of years and continues to present many challenges for Ecology.

    X x x x How to fund the implementation of new technologies given the three-year budget planning horizon and the faster pace of technological development.

    How to fund initial IT investments that have a payback period of more than two years (ie a biennium).

    How to help Ecology’s users and IT staff adapt to a change from one set of tools and/or processes to another.

    How to determine the best time for the agency to move to a new technology. 209 x x How to get IT involved up-front in efforts by agency business areas to address business problems.

    How to participate in enterprise initiatives and achieve their longer-term benefits while meeting the shorter-term needs of the smaller business units (programs within an agency or agencies within State government? How to measure the business value of IT? x In the past few years, the agency has been experiencing growing budget challenges and there has been an increased demand for IT services particularly in the area of application development.

    This has made priority-setting more challenging.

    Most recently, the State IT Transformation effort is presenting a number of new challenges: x x x Preparing staff for the change that will come with the IT Transformation.

    Maintaining quality service levels during IT Transformation implementation.

    The need for up-front investments to implement some of the IT transformation concepts. What Does Ecology Need to Succeed? Several factors are and will continue to be key to the agency’s successful use of IT: x x x x x x x x x Obtaining and maintaining executive management’s support and participation in IT priority setting.

    IT participation in the identification and evaluation of business solutions.

    Maintaining a stable, well trained, competent IT staff with appropriate depth in critical functions.

    Providing appropriate and timely IT training to IT staff and all users.

    Encouraging and stimulating the investigation of emerging technologies that hold promise for improving Ecology’s abilities to perform its business functions.

    Viewing and managing all Ecology data as enterprise data.

    Enhancing the agency’s IT infrastructure including desktop resources, server storage, network bandwidth, etc.organizing processes to roll out applications effectively and efficiently on an agencywide basis.

    Determining appropriate times to upgrade and/or move to new technologies balancing issues of training with increased availability of capabilities. Are There Opportunities for Data or Resource Sharing?

    Read more about However, it describes what to do, not how to do it:

    How To Toolkits and Certification eLearning

    Other Sources:

  • Howcast | The best how-to videos on the web
  • How to spot a murderer’s brain | Science | The Observer
  • r/HowTo – Reddit
  • How To Toolkits and Certification eLearning, Learn more about ITILFoundation Complete Certification Kit HERE:

    How To Toolkits and Certification eLearning and  However, it describes what to do, not how to do it

    How To Toolkits and Certification eLearning - However, it describes what to do, not how to do it

    How To Toolkits and Certification eLearning and  However, it describes what to do, not how to do it

    How To Toolkits and Certification eLearning - However, it describes what to do, not how to do it

    “How to Take the Reins At Top Speed”

    June 12, 2013

    “How to Take the Reins At Top Speed” Ultimately, practitioners should seek to design an onboarding strategy that takes individual newcomer characteristics into consideration and encourages proactive behaviors, such as information seeking, that help facilitate the development of role clarity, self-efficacy, social acceptance, and knowledge of organizational culture. Research has consistently shown that doing so […]

    Read the full article →

    • “How to Terminate Cat6a Shielded Keystone Jacks” (http://discountlowvoltage

    June 12, 2013

    • “How to Terminate Cat6a Shielded Keystone Jacks” (http://discountlowvoltage Category 6 and 6a cable must be properly installed and terminated to meet specifications. Incorrect installation practices include kinking or bending the cable too tightly. The cable bend radius should be no less than 4 times the outer diameter of the cable. Incorrect termination practices include […]

    Read the full article →

    “How to Add Google Analytics to Your Facebook Fan Page” (http:/ / web

    June 12, 2013

    “How to Add Google Analytics to Your Facebook Fan Page” (http:/ / web 14 Lite In August 2009, Facebook announced the rollout of a lite version of the site, optimized for users on slower or intermittent Internet connections. Facebook Lite offered fewer services, excluded most third-party applications and required less bandwidth.[62] A beta version of […]

    Read the full article →

    How To U : 123 Research development In 2009 the two conceptions collaborative e….

    June 12, 2013

    How To U : 123 Research development In 2009 the two conceptions collaborative e…. Collaborative e-democracy is based on following core principles: • Self-government, the ideal of direct democracy, and “quod omnes tangit ab omnibus approbetur”, the ancient maxim from the Roman law that says “that which affects all people must be approved by all […]

    Read the full article →

    [67] “How to update Apple TV software” (http:/ / support

    June 12, 2013

    [67] “How to update Apple TV software” (http:/ / support References [1] http:/ / www.apple.com/ appletv [2] Apple TV Now Shipping (http:/ / www.apple.com/ pr/ library/ 2007/ 03/ 21appletv. Html). Apple press release. 2007-03-21. . Retrieved 2007-04-03. [3] http:/ / news. Yahoo.com/ s/ ap/ 20100901/ ap_on_hi_te/ us_tec_apple [4] http:/ / www.ifixit.com/ Teardown/ Apple-TV-2nd-Generation-Teardown/ 3625/ 2 […]

    Read the full article →

    [52] The Coming Technological Singularity: How to Survive in the Post-Human Era (http:/ / www-rohan

    June 12, 2013

    [52] The Coming Technological Singularity: How to Survive in the Post-Human Era (http:/ / www-rohan References [1] A Ping-Pong-Playing Terminator (http:/ / www.popsci.com/ technhttp:/ / en. Wikipedia.org/ wiki/ Special:Contributions/ Train2104ology/ article/ 2010-02/ ping-pong-playing-terminator). Popular Science. . [2] Best robot 2009 (http:/ / www.gadgetrivia.com/ 8164-best_robot_international_robot_exhibition). Www.gadgetrivia.com. . [3] Zunt, Dominik. Who did actually invent the word […]

    Read the full article →

    “A Study of How to Decrease the Costs of Collecting, Processing and Transporting Slash”

    June 12, 2013

    “A Study of How to Decrease the Costs of Collecting, Processing and Transporting Slash” 1.4 Transportation Transportation of chips or hog fuel from the forest location to an energy facility can be accomplished by truck with the following configurations: x x x Tractor-Truck & Open Top Chip-Van Tractor-Truck & Enclosed Chip-Van Roll-On/Off Transport Containers Tractor-Truck […]

    Read the full article →

    How To U : 8 Appetizer software 35 External links Appetizer home page 9….

    June 12, 2013

    How To U : 8 Appetizer software 35 External links Appetizer home page 9…. Appetizer is an open source dock for Windows 2000, XP, Vista and 7, which allows organizing the user’s shortcuts and applications. It can be customized via skins and plugins and is available in 25 languages. Various other features are available including […]

    Read the full article →