Test Development Contract Lead - 17443BR

Test Development Contract Lead

Achievement & Assessment Inst
University of Kansas - Lawrence
Employee Class: 
U-Unclassified Professional Staff

Position Overview

The Achievement and Assessment Institute (AAI), which oversees the development and implementation of the Kansas Assessment Program (KAP) and the Kansas English Language Proficiency Assessment (KELPA), is seeking a Test Development Contract Lead to manage assessment development and delivery of that assessment in the content areas of English language arts (ELA), English language proficiency, mathematics, and science. The Test Development Contract Lead will provide guidance to staff on the content teams with regard to assessment development best practices, Evidence-Centered Design (ECD) principles and other measures designed to bolster validity on both the item and test levels. This role will ensure that development processes are implemented and documented consistently to meet professional standards, relevant quality assurance criteria are identified and implemented, and that delivery deadlines are met.

The Test Development Contract Lead will oversee an annual scope of work that includes multiple overlapping phases of a 30-month end-to-end development cycle that governs the assessment development process. The process will include creating and updating test specifications, construct definition development, item writing, overseeing of internal content development and validity-building steps, external reviews of items, stimuli and data. Additionally, it will include construction of forms, development of educator resources, and validity studies related to assessment content. The Test Development Contract Lead will also partner with the Implementation team to work with the state client on a variety of collaborative endeavors, including test design, annual scope of work specifications, setting of qualifications and selecting of external review panelists, and providing informed consultation on assessment validity and best practices.

A successful candidate will have deep understanding of best practices in assessment design, development and validity evidence drawn from test content, well-grounded in industry standards and Evidence-Centered Design literature and will show strong evidence of being able to successfully lead people and overlapping projects in a deadline-driven environment. The successful candidate will also have a comprehensive knowledge of next generation curricular standards in at least one, and preferably multiple, content areas.

This is a full-time (40 hours/week), unclassified position at the Lawrence campus. We will consider remote work for well-qualified candidates.

Job Description

50% - Lead test development efforts for multiple assessment programs with the primary focus on the Kansas Assessment Program and Kansas English Language Proficiency Assessment. Examples of job activities include:
  • leading content teams in developing blueprints, tests, item specifications and construct definitions grounded in ECD principles
  • developing plans for item pool development based on test blueprints
  • contributing to the professional growth of content team members, particularly with regard to understanding validity evidence drawn from test content, basic psychometric ideas and industry-standard cyclical development processes
  • leading content teams in developing assignments for item writers to ensure the construction of high quality, aligned test items
  • ensuring that internal content reviews result in polished, accurate items that tightly align to close reading of curricular standards
  • identifying and implementing process improvements
  • implementing budgets and administrative procedures for test development projects
  • working collaboratively with leadership, content area managers, project coordinators and other contract leads to ensure staffing resources are distributed across programs so as to ensure work progresses toward key milestones and deliverables.
30% - Coordinate internal resources to support assessment projects. Examples of work performed may include:
  • coordinating the establishment of a work plan with multiple teams
  • coordinating the production and review of technical reports
  • collaborate with the program director, implementation team and psychometric team on identifying deliverables for the Kansas Assessment Program and Kansas English Language Proficiency Assessment
  • schedule and lead cross-team meetings for special projects
  • contributing intellectual leadership to other center-wide initiatives and projects

10% - Communicate with external audiences including researchers, state education agency staff, and professional organizations.
  • preparing and delivering presentations for technical and non-technical audiences
  • providing consultation and collaboration on all aspects of test design, scope, development and validity evidence drawn from test content to state education agency staff
  • responding to client requests and coordinating communication with the program director

10% - Develop proposals for external contract and research opportunities. Examples of work performed may include:
  • leading the development of responses to RFPs
  • collaborating with center staff on proposals for externally-funded research projects

Required Qualifications

  1. Master’s degree in a related field
  2. Five years of large-scale test development experience, at least two years of which is as the person with primary responsibility for a subject and/or multiple grade levels within a testing program
  3. Experience developing and implementing work plans for assessment development that include blueprints, test and content specifications, construct definitions, and item development and review
  4. Experience leading projects focused on next generation curricular standards (such as Common Core State Standards or the Next Generation Science Standards) in at least one content area, and operational familiarity with curricular standards for multiple content areas
  5. Experience developing technical reports for assessment programs and developing text for validity study reports, cognitive lab reports or similar
  6. Excellent written and oral communication skills

Preferred Qualifications

  1. Experience managing complex relationships with stage education agency representatives and peer colleagues within an organization
  2. Experience applying a variety of theoretical approaches to validity and current research within a large-scale assessment context
  3. Experience managing item and test development across multiple content areas
  4. Experience with assessment software development
  5. Record of published research or dissemination on large-scale assessment

Additional Candidate Instructions

A complete application includes online application, resume or vita, cover letter highlighting qualifications and including specific examples of experience, a professional writing sample and contact information for three professional references. Only complete applications will be considered.

Application review begins July 20, 2020 and continues until the position is filled. For first consideration, please apply before the application review date.

Contact Information to Applicants

Advertised Salary Range

Minimum $70,000, dependent on qualifications

Application Review Begins


Anticipated Start Date


Search Jobs
Posting Information
Posting ID:
Achievement & Assessment Inst
University of Kansas - Lawrence
Employee Class:
U-Unclassified Professional Staff
Application Review Begins: