Practice Analysis

Before items can be written or exams created, a guiding connection is needed between the real world in which the credential will be used and test content. That critical link is the practice analysis study, also known as a competency profile, job analysis or role delineation.

ACT® ProExam, an ACT® affiliated company, has deep expertise in practice analysis studies, applying recognized procedures that have been developed and refined over seven decades across a range of professions, industries and institutions. All with the objective of building a trusted and legally defensible credentialing program that is tied to real-world requirements.

Description

We start by analyzing your professional, occupational or educational space in detail. ACT® ProExam works with content experts who are recognized authorities in the field, carefully chosen to represent the diversity of practice and key stakeholder perspectives. Together, we build a detailed content- or process-based description of practice that includes:

  • Major performance domains;
  • Key competencies, tasks and activities within the performance domains;
  • Knowledge and skills underlying demonstration of competencies and performance of tasks and activities.

We gather information using a wide range of proven techniques, such as interviews with thought leaders, daily logs, critical incidents interviews with supervisors or recruiters, independent reviews and focus groups. We scrutinize the common threads in, and distinctive aspects of, job descriptions, curricular materials and other relevant literature within the profession or industry.

Validation

This description of practice is only the first step.  The initial description is validated, following ACT® ProExam’s established – and at the same time, uniquely tailored – process.

  1. We work closely with you to identify a representative sample of current practitioners – locally, nationally or internationally – to take part in a web-based survey;
  2. Survey participants review domain, competency-task-activity and knowledge-skill statements, rating them with validation scales that are designed specifically for your program, and that address frequency and criticality of task performance along with cognitive complexity and acquisition of knowledge and skills;
  3. We analyze the survey ratings to refine the detailed description of practice and validate profiles of your profession, highlighting similarities and differences related to professional or demographic variables such as years of experience, geographic location, industry sector and work setting.

ACT® ProExam prepares reports that document the results of the practice analysis study, providing a robust description of practice. And, at your request, we can present those results at a national conference or prepare articles for a professional journal.

Results

Practice, competency or role are now accurately described. ACT® ProExam uses the ratings from the validation survey to establish weightings for the specific components of the profile of practice. Working with subject-matter experts, we review and augment the study results. This creates the foundation for developing test specifications that determine the content of your program’s examination. The examination, along with decisions that are made on experience and education requirements, leads to a valid, legally defensible credential that reflects actual responsibilities and tasks.

In addition to test specifications, a practice analysis study can be used to achieve other organizational and credentialing insights:

  • Establishing continuing competency or re-credentialing criteria;
  • Creating professional development materials;
  • Identifying emerging areas of practice and opportunities for specialty, advanced or other new credentials;
  • Evaluating the value and meaning of your credential to stakeholders.

Based on industry standards, ACT® ProExam generally recommends that a practice analysis be conducted every four to seven years. We’ll work with you to determine an appropriate time frame based on the rate of change within your profession or industry.

“ACT® ProExam is good at helping people understand what they need to do, and then guiding them through complex tasks without intimidating them.”

Edwin L. Weinstein, PhD, C.Psych.
The Brondesbury Group
Toronto, Canada