Job Analysis Development
Best Practices in Item Writing
Developing Items to Measure Each Competency
Developing task-based items to measure each competency or factor on the assessment tool requires
effort and sound professional judgment. In developing items, extract important or paramount duty
statements from the source documents as indicators or items to measure each competency. Each
competency is defined in terms of the types of tasks an individual might perform in the type of job
being filled or by tasks that more generally represent and define the competency. In other words, the
tasks used to define a competency are not necessarily among those that are performed on the job.
Rather, they are among those that require possession of the competency in order to be performed
successfully, and would therefore serve as effective indicators of that competency.
Note that the competencies or rating factors will not appear on the questionnaire. Generally, only the
task statements will be visible to the applicant but the factor headings will appear in the
documentation.
Ideally when developing questionnaire items, you will be able to look at the task frequency and
importance ratings by grade level from the job analysis report, selecting those tasks that are most
critical (i.e., rated as most important and performed most frequently). Consider also the
consequences if the task is not performed correctly. In addition, remember that the task statements
may cover more than one grade level. Therefore, you must use judgment when selecting item/task
statements that enhance the meaning of the major job functions across grade levels.
Best Practices in Item Writing
Crafting quality items for automated assessment tools requires time and effort. The goal is to define
the competency or rating factor in terms of observable behaviors.
Following the formula and writing tips below will help to ensure quality items.
Follow the Formula. The formula for writing an effective task statement is as follows:
Perform What? + To Whom or What? + To Produce What? OR Why? How?
(Action Verb) (Direct Object) (Qualifying Statement)
For example:
Sort incoming mail into functional groups for distribution
Distribute policy report to all employees by email
Proofread and edit letters or other documents to address format or grammatical, spelling,
or typographical errors.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 1
Rev. 5/1/2020
The task statement should begin with an action verb in the present tense that describes a specific
behavior (e.g., writes, categorizes, meets, corrects, delegates, applies, etc.) The action verb
addresses what the worker does. You want the action to be as specific and observable as possible so
that the statement is interpreted in the same way by everyone. Two verbs should only be included in
the same statement if they are closely and directly related.
A direct object follows the action verb to identify what or who the behavior is performed on.
Examples include client, contract, coworker, telephone, assignment, spreadsheet, and training.
Qualifying statements are added to describe when, where, why, and how the behavior is performed
(Brannick & Levine, 2002; Whetzel & Wheaton, 1997). They specify the purpose of the task, any
critical materials that are needed to perform the task, under what conditions the task is performed,
etc.
It is understood that the worker will be performing the actions, so it is unnecessary to include
"worker" or "incumbent" in the task statement. The focus is on the actions that are being performed,
not on the person performing them.
Below are additional tips for writing quality items.
Write Tasks in Behavioral, Observable, and Verifiable Terms. The wording of tasks is critical for
helping to ensure accurate self-rating. Write task items in terms of observable aspects of work
behavior. Pick a strong action verb to begin your task statement. Use words like 'writes' instead of
'develops.' 'Makes presentations' is a statement often used, but what does it really mean? Think about
the crux of what is important. Is one 'giving written material to another,' 'orally briefing someone in
a group setting' or 'giving speeches to large and varied audiences?'
As another example, 'prepares' is often used in task statements. What is the intent of this word? Does
it mean assemble? Write? Compiles information from colleagues? When writing task statements, be
a nit picker. Specificity is critical in the development of task statements.
In addition, tasks should be written so that the behaviors being described are observable or verifiable
(a colleague or supervisor could verify that the applicant performed this behavior). Specific,
unambiguous, and verifiable tasks are less susceptible to exaggeration and inflation and will help to
increase the reliability and validity of the assessment tool.
Write Clearly and Succinctly
Clear and succinct tasks also are critical for helping to ensure accurate self-rating. Clear and succinct
tasks take more time and effort to write, but they are easier for the applicant to respond to accurately
and reliably. In writing clear and succinct statements, focus on the key or unique aspects of the task
you are describing. Select only words essential to the meaning of the statement while following the
formula outlined above. Below is an example of a succinct item written from an original task
statement for the position of Personnel Supervisor.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 2
Example:
Original task statement Checks time reports for persons under own supervision for
errors in addition and hours claimed, uses daily work
summary sheets as a resource if necessary, signs time reports,
and/or payroll sheets, routes to payroll department before pay
p
eriod deadline, in order to authorize payment for time
worked.
Clear and succinct item
for assessment tool
Audits and signs time reports to authorize payment for time
worked.
Develop Items that Meaningfully Distinguish Among Candidates. An important part of developing
items is determining what critical behaviors differentiate those who are exceptional or highly
successful in the position from more average performers. Use the distinguishing value ratings from
the job analysis to help you identify competencies that are good differentiators. If they are not
available, make every effort to collect them from your subject matter experts. From there, develop or
select items for those competencies with distinguishing value. For the other competencies, look at
the job analysis ratings to determine which related tasks were rated as most critical (very important
and performed frequently). Think critically about what behaviors relevant to that factor would
distinguish a highly successful employee from an average performer- what are the critical behaviors
that someone would need to be able to display on the job that would differentiate them?
Keep Your Competencies Unidimensional. In taking a competency-based approach to developing
automated assessment tools, it is important to focus on what you are trying to measure under any
given competency or factor. Competencies should be unidimensional in nature. In other words, you
want to measure only one construct or underlying human trait, characteristic or competency.
Otherwise, the ratings can average themselves out and you have not really measured anything well.
For example, most jobs require some combination of technical/specialized and general training and
experience. However, it is generally appropriate to keep technical or specialized content separate
from items designed to measure general competencies. For example, for a Transportation Specialist,
GS-2101, the defining tasks for the knowledge of transportation industry operations may be fairly
specific.
1. Evaluate day-to-day operations to ensure efficient and economical use of carrier
services.
2. Analyze transit time data to determine carrier effectiveness and plan shipments.
3. Advise on procedures and operations for receipt of incoming material, temporary
storage, and shipment of items through the country and overseas.
4. Review contracts to verify accuracy of transportation-related items.
However, the defining tasks for written communication may be much broader and may not
necessarily include transportation issues. The tasks need only be indicative of written
communication.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 3
1. Compose complex correspondence or other written work (for example, position
papers, briefings, contracts) to educate or inform others.
2. Review reports, documents, records, data, or other materials to verify
completeness, correctness, consistency, compliance, or authenticity.
3. Proofread and edit letters or other documents to address format or grammatical,
spelling, or typographical errors.
As another example, if measuring Oral Communication, the statement 'Provide training to new
employees on payroll operations' may be too specific. You may do better to measure 'Knowledge of
payroll operations' under a separate factor. An item such as 'Explain specialized or other complex
information orally to inform or educate others' may be a more appropriate item to measure under a
general Oral Communication factor.
Similarly, as previously mentioned, you may want to keep items related to Written Communication
and Oral Communication separate, unless the method of communication does not matter (i.e., the
employee could use one or the other forms of communication).
Note, if as you are constructing your assessment tool you develop/identify a task that is absolutely
critical but cannot be assigned to an existing competency, you may need to develop another
competency. Again, keeping your competencies unidimensional or 'pure' is likely to improve the
measurement properties and thus the validity of your assessment tool.
Quality Over Quantity One question often asked is 'How many task statements should be written?’
There is no exact number. You will want to write as many as you need to adequately describe each
competency as it is utilized on the job. Generally speaking, you will want to have 5 to 10 items for
each competency. There is some evidence to suggest that 'applicant fatigue' is a factor when there are
too many items to which to respond. All else being equal, fewer well written items will be better
than more poorly written items. Avoid developing 'duplicate' or 'redundant' items representing
multiple levels of a given competency. (The default rating scales are designed to differentiate
applicants in terms of their training and experience with regard to any given task). When the
competency has been adequately described, then the optimum number of questions or task
statements has been reached.
Potential Pitfalls
Common pitfalls in item writing include using: unnecessary words, double-barreled items; overly
specialized items; evaluative, vague or ambiguous terminology; and abbreviations. Examples of
these common pitfalls and ways to address them are highlighted below.
Unnecessary words add to the length of a task statement but do not add to the meaning. They can
make a task seem more complicated than it actually is. You want your items to be concise so that
they are easily understood.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 4
Example:
Relay information, both in oral and written form, to individuals at all levels and on
Wordy: multi and varied acquisition topics.
Relay oral and written information on varied acquisition topics to individuals at all
Concise:
levels.
A double-barreled item is a task statement that includes multiple tasks or components. Including
more than one task in a statement forces applicants to choose which part of the statement they will
respond to, giving you incomplete or ambiguous information as to what experience the applicant has.
Example:
Double-Barreled:
Resolve customer problems independently and deal with difficult
customers with courtesy and tact.
Two Statements:
Resolve customer problems independently.
Deal with difficult customers with courtesy and tact.
Combined: Resolve customer problems independently with courtesy and tact.
Overly specialized items or writing items based on job-specific task statements in the position
description when not warranted may unnecessarily omit qualified applicants. Make sure that you are
honing in on the critical aspects of the competencies needed for the position at the time of hire.
Although there may be certain specialized or technical expertise that is required, when assessing
general competencies such as oral communication or written communication, for example,
experience in other job settings may be equally as valuable. Writing items often requires a delicate
balance. You do not want to exclude an applicant by focusing on specialized aspects of the position
when it is not warranted. At the same time, as work statements become more general, interpretation
by applicants would be expected to become more subjective. When writing statements for general
competencies, you need to write in specific, behavioral, and verifiable terms.
Example:
Overly Specific:
Add, subtract, multiply, divide, and compute percentages to
calculate pay, time, and benefits. (grade 4, Math Computations)
Appropriate
Level of
Generality:
Uses addition, subtraction, multiplication, and division, and
computes percentages to perform basic calculations.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 5
Vague, Ambiguous, or Evaluative Terminology
Use behavioral terms that are clear and specific. Without a description of specific behaviors (i.e.,
that the applicant can visualize), applicants are likely to interpret vague or ambiguous statements
differently and generally in their favor if there is doubt about how they should be interpreted.
Do not use adjectives like 'excellent', 'effective', or 'adequate' in item writing. Statements that require
the applicant to be more evaluative rather than descriptive in assessing their training and experience
are likely to affect their variance in frankness (honesty) resulting in inflation. Statements that are
perceived as less threatening to the applicant are more likely to be rated more accurately.
Example:
Performs quality control procedures on time and attendance reports in an
Vague:
effective manner.
Verifies the accuracy of data or information in a report or database
Behavioral:
following standard procedures.
Abbreviations save space and time, but they may not be understood by or hold the same meaning for
everyone. Encountering an unfamiliar abbreviation that is not spelled out or an abbreviation that can
stand for more than one thing can be frustrating to applicants and can lead to differences in the way
that the task statement is interpreted.
OPM – Supplemental Handbook for Developing Automated Assessment Tools Using USA Staffing
Prepared for Use by the National Business Center - August 2006
Page 6
