AP Placement Validity Study
This customized study will help you evaluate the success of your institution’s Advanced Placement Program (AP) Exam placement policies. It gives you information on how well your current policies for granting AP credit for introductory courses are working for each higher-level course you choose to study.
To assess whether a particular policy is working as intended, the study will compare the performance of students who entered the higher-level course through AP credit to the performance of students who entered the course by completing the introductory AP-equivalent course or courses at your institution.
Designing Your Study
In this section, get an overview of how to design your study. For detailed step-by-step instructions, download the ACES AP Placement Validity Study Guide in the Resources of this page.
Choosing a Criterion and Predictors
In an ACES study, a criterion means a measure of student success that can be used to validate your admission or placement policies. The standard criterion for an ACES placement validity study is an end-of-course grade in a specific college course. For an AP placement validity study, you should use the course into which you place your students rather than the course out of which you will place students.
A predictor means a data point that you’re using to make your decisions and whose effectiveness you want to analyze. The standard predictors for an ACES AP placement validity study are AP Exam scores or final course grade in the course for which your institution offers AP credit.
Submitting Your Student Data
Which Students to Include
To be included in the sample, students must have a valid course grade on the criterion. Your results will help you make more accurate predictions if you keep these rules in mind:
- The sample of students used for the study should be similar to the group of students whose success you want to predict. Characteristics of the population such as gender, ethnicity, and age should be represented in the sample. We recommend including at least one entire cohort of students and two academic years of course records.
- The data from the predictor must have been collected before the criterion. That is to say, AP Exam scores or prerequisite courses must have been completed before the student took the criterion course.
For help in analyzing the validity of a predictor for a sequence of courses, contact the ACES staff.
Which Data to Include
You’ll need to supply identifying information on the students in your sample as well as data on your institution's AP credit and placement policy and the criterion (e.g., their final grade in the relevant course at your institution). Identifying information will be used to include College Board records on AP Exam scores.
Submitting Your Data File
You’ll be prompted to submit a data file when you design a new study. You can upload a new data file or reuse one from a previous study.
Your data file format must be Microsoft Excel, Comma Separated Value (CSV), Tab Delimited (.txt), or SAS Transport (XPORT).
For detailed information on preparing your data file, see the ACES Data Preparation Guidelines in the Resources section. You can also download an Excel template with the correct data layout for the AP placement validity study.
Getting Your Report
When your study is complete—within 20 business days of your completed request—we’ll notify you that your report is available on the ACES website and you can sign in to access it. You’ll get two deliverables:
- A complete, printable report in PDF format that shows the strength of your chosen predictors of success—alone and in combination—with charts, tables, and detailed explanations.
- A report in HTML format featuring interactive graphs. You can click to display or hide data, compare data, zoom in and out, take a snapshot, sort table columns, and more.
FAQ
Do I need to submit a new data file for a placement study based on students analyzed in our admission study?
Provided that the previously submitted admission validity study file incorporates all data elements required for a placement validity study, you can request a new placement validity study and indicate that you plan to use an “existing data set.”
Can we get placement information for our math and English courses at the same time?
Yes. ACES provides for analysis of up to 10 courses for any placement study request.
If my institution primarily gives Pass/Fail grades, can we conduct a placement validity study?
In order to effectively evaluate how well a predictor is working, ACES requires a range of at least four points on a scale for the criterion. Students with Pass/Fail grades cannot be analyzed in the study.
Can we include students with Fail, Incomplete, or Withdraw grades?
Generally, we have found that at most institutions failing grades, such as F or E, are calculated into the students' grade point average and are equivalent to zero grade/quality points; and Incomplete or Withdraw grades, such as I or W, are not incorporated into the GPA calculation. In these cases, students with failing grades should be included in your placement validity study, while students with Incomplete or Withdraw grades can be included in your file but will not be included in the placement validity analyses. When you are completing your grade mapping table you will note that grades such as W or I have “No Value” and will therefore not be analyzed in the study, while grades such as F or E are equivalent to a grade of 0.
These are only given as general guidelines. If you have a large number of students who failed the course you wish to study for any reason other than doing poorly on the tests and/or class work (such as, dropping the class or leaving the school without officially withdrawing), or if your Incomplete or Withdraw grades are calculated into the GPA, please contact the ACES staff to discuss your specific situation.
How should we collect data for a placement validity study?
The ideal way to obtain data for a placement validity study would be to assign students randomly to the alternative learning situations (different courses, sections, levels, etc.) so that students with a full range of academic ability are in each learning situation. However, since this is rarely feasible, a study using the chosen test, without any randomization, can still provide useful information.
For SAT and ACCUPLACER placement validity studies, information from the tests or measures that are used as predictive variables is collected prior to the student entering the course of interest. The time between the testing of the student and the entrance into the course should not be lengthy. There should also not be any instructional intervention between the predictive test or measure and the start of the course.
For example, suppose you’re looking at students who took the SAT Math Test and then took the introductory math course at your institution. In this case, we don’t recommend including students who not only did those two things but also took your remedial math course before taking the introductory (credit-bearing) math course. Including these students will not give you an accurate analysis of the validity of SAT Math Test scores to predict success in your introductory math course.
What’s the minimum number of students needed for a placement validity study?
An AP placement validity study requires a minimum of 15 students whose AP scores and course-taking records indicate that they entered the higher-level course with AP credit. It also requires a minimum of 15 students without AP credit whose course-taking records indicate that they entered the higher-level course by completing the required introductory AP-equivalent course(s) at your institution.
A CLEP placement study requires a minimum of 15 students whose CLEP scores and course-taking records indicate that they entered the higher-level course with CLEP credit, and a minimum of 15 students without CLEP credit whose course-taking records indicate that they entered the higher-level course by completing the required introductory CLEP-equivalent course(s) at your institution.
For SAT and ACCUPLACER placement studies, course data for at least 30 students are required for a course placement analysis involving a single predictor measure. If one to three predictors are used, a minimum of 30 students is required; for four predictors, a minimum of 40 students; and for five predictors, a minimum of 50 students.
How many semesters of data are required for a study?
A minimum of two semesters or terms of course records are required for an AP placement validity study. AP credit-holding students may enter the sequent or subsequent course immediately upon matriculation, but non-AP students need time to take the AP-equivalent course(s) and the sequent or subsequent course. If your policy offers credit for more than one course, you may need more than two semesters of data.
Can I include more than one cohort?
Yes. Some institutions may need to include more than one cohort to reach minimum sample requirements. Note: Your institution’s AP credit/placement policy must be the same for all included cohorts.
What if there is no specified sequent course?
If there is no prescribed sequent course for students to take, you can identify up to three courses that have the AP-equivalent course(s) as a prerequisite and are taken after completion of the equivalent course(s). Note: At least 15 students in each pathway group are required for a course to be included in the study.
What if we only give placement for AP Exam scores and not course credit?
Course credit is not required for inclusion in the placement validity study, but you must specify courses that are equivalent to AP in order to identify a non-AP comparison group. One recommendation is to use the course or courses that serve as prerequisites for the course into which AP credit-holders are placed.
What if we only give course credit for AP Exam scores and not placement into a more advanced course?
In order to complete a study, you must be able to identify courses students can place into with AP Exam score credit, as well as courses that are equivalent to AP courses. If your institution awards non-specific credit or elective credit, a traditional placement validity study cannot be conducted.
What if our policy awards credit for more than one course?
You can specify up to three courses for which AP credit can be awarded.
What if students can earn course credit through other pathways, such as dual enrollment?
AP placement validity studies can include one additional pathway for comparison with AP credit-holding students and native course takers. You will need to identify those students in a separate variable. Students in the third pathway group should all be the same credit type in order to make accurate conclusions based on comparison results.
Can I conduct subgroup analyses or include control variables in the study?
Because AP policies are not conditional, based on a student’s race, gender, locality, etc., the comparisons do not include these variables. However, some subgroup descriptive information is provided in your report: AP Exam score means and standard deviations are provided for sex and race/ethnicity subgroups with at least 15 students. Currently, the system does not include custom subgroups.
Can we include other commercial exams as a predictor in my ACES study?
No. ACES cannot be used to validate non-College Board commercial products.
How do I analyze and articulate my institution’s AP credit and placement policies within ACES?
The AP Placement Validity Study Detailed Documentation below provides useful information on how to specify different placement policies for an AP Placement Validity Study, the information needed to conduct a study, and the logistics around getting started.
Which College Board testing programs are supported by ACES?
ACES supports the SAT, Advanced Placement Program (AP), ACCUPLACER, and the College-Level Examination Program (CLEP).
Are grades required for an AP Placement Validity Study?
Yes. These placement validity studies rely on course performance in sequent or subsequent courses to compare AP credit-holding students to native course takers. However, grades do not need to follow a traditional scale. You will map course grades in the study design process.