Test Development Contract Lead - 22163BR

Test Development Contract Lead

Achievement & Assessment Inst
University of Kansas Lawrence Campus
Work Location Assignment: 
Employee Class: 
U-Unclassified Professional Staff

Position Overview

Accessible Teaching, Learning and Assessment Systems (ATLAS) at the University of Kansas promotes learning and improved outcomes for all students. We create technology-based learning and assessment systems that are accessible and academically rigorous. Our center serves all students, including students with disabilities and struggling learners, as well as their educators.
We specialize in map-based learning systems that are informed by research on skill development, assessment design, and teacher professional development. Our grant-funded research includes writing instruction, science professional development, support for teacher trainers, and science assessment design. We also foster partnerships with diverse organizations that share our focus and commitment to quality learning experiences for all students.
Our center develops and delivers state assessments. The Dynamic Learning Maps® (DLM®) Alternate Assessment System assesses students with the most significant cognitive disabilities in English language arts, mathematics, and science. We also offer custom social studies alternate assessments. Our broader state assessments include the Kansas Assessment Program, the Career Pathways Assessment System, and the Kansas English Language Proficiency Assessment.

ATLAS is seeking a Test Development Contract Lead to manage assessment development. The Test Development Contract Lead will provide guidance to staff on the content teams about assessment development best practices, Evidence-Centered Design (ECD) principles and other measures designed to bolster validity on both the item and test levels. This role will ensure that development processes are implemented and documented consistently to meet professional standards that relevant quality assurance criteria are identified and implemented, and that delivery deadlines are met. The role will coordinate with the Test Development Contract Lead for the Kansas Assessment Program (KAP) and the Kansas English Language Proficiency Assessment (KELPA) to maximize consistency of practice and process across two of the Test Development team’s central workstreams, and with the program director to develop and deliver externally facing communications and presentations about the Test Development Team’s activities.

The Test Development Contract Lead will work with a project team, which will include project managers, content specialists, the program director and relevant members of other teams, to improve and streamline existing processes in order to establish a recurrent, efficient test development cycle. The Lead will then oversee an annual scope of work that includes multiple overlapping phases of a 30-month end-to-end development cycle. This work will include creating and updating test specifications, construct definition development, item writing, overseeing of internal content development and validity-building steps, and reviews of test content and data with external educator panels. Additionally, it will include collaboration with the Test Production team and the Research and Psychometrics team to ensure appropriate depth and test domain coverage within a computer-adaptive testlet pool. The Lead will coordinate with content area teams to develop educator resources and validity studies related to assessment content. The Test Development Contract Lead will also partner with the Implementation team to work with DLM partner states on a variety of collaborative endeavors, including test design, annual scope of work specifications, setting of qualifications and selecting of external review panelists, and providing informed consultation on assessment validity and best practices.

Job Description

A successful candidate will have deep understanding of best practices in assessment design, development, and validity evidence drawn from test content; will be well-grounded in industry standards and Evidence-Centered Design literature; and will show strong evidence of being able to successfully lead people and overlapping projects in a deadline-driven environment. The successful candidate will also have specific, relevant expertise with students who take alternate assessments students with the most significant cognitive disabilities. The candidate should have a working knowledge of next generation curricular standards in at least one, and preferably multiple, content areas.

This is a full-time (40 hours/week), unclassified position at the Lawrence campus. We will consider remote work for well-qualified candidates.

50% - Lead test development efforts for multiple assessment programs with the primary focus on the Dynamic Learning Maps (DLM). Examples of job activities include:
  • leading and coordinating content teams’ contributions to the development of test designs, blueprints, item specifications and construct definitions grounded in ECD principles
  • evaluating existing processes and identifying efficiency gains
  • developing plans for testlet pool development based on test blueprints
  • contributing to the professional growth of content team members, particularly with regard to understanding students with significant cognitive disabilities who take alternate assessments, validity evidence drawn from test content, basic psychometric ideas and industry-standard cyclical development processes
  • leading content teams in developing appropriate, content-focused orientation materials and assignments for item writers to ensure the construction of high quality, aligned test items
  • ensuring that internal content reviews result in polished, accurate items that tightly align to close reading of curricular standards
  • implementing budgets and administrative procedures for test development projects
  • working collaboratively with leadership, content area managers, project coordinators and other contract leads to ensure staffing resources are distributed across programs.
25% - Coordinate internal resources to support assessment projects. Examples of work performed may include:
  • coordinating the establishment of a work plan and repeatable efficient test development cycle with multiple teams
  • coordinating the production and review of technical reports
  • collaborate with the program director, implementation team, test production team and psychometric team on identifying deliverables for the DLM assessment program
  • schedule and lead cross-team meetings for special projects
  • contributing intellectual leadership to other center-wide initiatives and projects
15% - Communicate with external audiences including researchers, state education agency staff, and professional organizations.
  • preparing and delivering presentations for technical and non-technical audiences, including partner states, the DLM Technical Advisory Committee and educator panels
  • providing consultation and collaboration on all aspects of test design, scope, development and validity evidence drawn from test content to state education agency staff
  • responding to partner state requests and coordinating communication with the program director
5% - Develop proposals for external contract and research opportunities. Examples of work performed may include:
  • leading the development of responses to RFPs
  • collaborating with center staff on proposals for externally-funded research projects
5% - Other duties as assigned

Position Requirements

  • This position allows for a telework or hybrid work location for well-qualified candidates. A telework position may be required to travel to Lawrence up to four times per year.
  • This position will require attendance at regularly scheduled meetings, onsite or through telecommunication platforms.

Required Qualifications

  • Master’s degree in education or related field
  • At least three years of large-scale test development experience, at least two years of which is as the person with primary responsibility for a subject and/or multiple grade levels within a testing program
  • At least three years working with students with significant cognitive disabilities, either directly in classrooms, developing educational products and/or assessments for these students, or both.
  • At least two years of experience leading projects focused on next generation curricular standards (such as Common Core State Standards or the Next Generation Science Standards) in at least one content area, and operational familiarity with curricular standards for multiple content areas
  • At least one year of experience developing and implementing work plans for assessment development that include blueprints, test and content specifications, construct definitions, and item development and review
  • At least one year of experience developing technical reports for assessment programs and developing text for validity study reports, cognitive lab reports or similar
  • Excellent written and oral communication skills

Preferred Qualifications

  • At least one year of specific experience collaborating with providers or developing standardized assessments for students with significant cognitive disabilities.
  • At least two years of experience applying a variety of theoretical approaches to validity and current research within a large-scale assessment context
  • At least one year of experience managing complex relationships with state education agency representatives and peer colleagues within an organization
  • At least one year of experience managing item and test development across multiple content areas
  • At least one year of experience with assessment software development
  • Record of published research or dissemination on large-scale assessment

Additional Candidate Instructions

A complete application includes:
  • Online application
  • Résumé
  • Detailed letter matching your professional experiences to the required and preferred qualifications
  • Names and contact information for three professional references.

Only complete applications will be considered.

Application review begins on 05/23/2022. Priority will be given to applications received by this date

Contact Information to Applicants

Cynthia Perez ATLAS-Admin@ku.edu

Advertised Salary Range

Minimum $80,000, commensurate on qualifications

Work Schedule

M-F; 8-5

Application Review Begins


Anticipated Start Date


Search Jobs
Posting Information
Posting ID:
Achievement & Assessment Inst
Primary Campus:
University of Kansas Lawrence Campus
Work Location Assignment:
Employee Class:
U-Unclassified Professional Staff
Application Review Begins: