Admission Validity Study
This customized study will help you evaluate your use of SAT scores, high school GPA, and other measures in admission decisions. It will show you how well these criteria work alone and in combination so you can better predict how applicants will perform at your institution if admitted.
You’ll learn:
- Which measures are the most useful predictors of success at your institution.
- How you might narrow the number of factors you consider without loss of predictive ability.
- How to construct the optimal equations for predicting the success of future students.
- Which specific students in the data set you supply are at risk for not returning.
Designing Your Study
In this section, get an overview of how to design your study. For detailed step-by-step instructions, download the ACES SAT Admission Validity Study Guide in the Resources section of this page.
Choosing a Cohort and Criterion
You’ll select a cohort to examine—for example, last year’s admitted class—and a criterion. In ACES studies, a criterion means a measure of student success that can be used to validate your admission or placement policies.
The standard choice of criterion for an ACES admission validity study is first-year GPA. You can also select first-semester GPA or cumulative GPAs over other time periods.
Choosing Predictors
In ACES studies, a predictor means a data point that you’re using to make your decisions and whose effectiveness you want to analyze. All ACES admission validity studies use SAT score(s) and high school GPA as predictors. You can add up to five additional predictors, including custom ones.
Choosing Additional Subgroups (Optional)
All ACES studies break down your results on the basis of gender and ethnicity whenever your sample includes 50 or more students for at least two levels of a subgroup (e.g., 50+ males and 50+ females) with the relevant study variables in the file. For an SAT admission validity study, you may request up to three additional subgroups, using ACES-supplied data, your own data, or a combination of the two.
Submitting Your Student Data
Which Students to Include
For a standard admission validity study, you should include all first-time, first-year students (domestic and international) that entered your institution last fall.
Which Data to Include
You’ll need to supply identifying information on the students in your sample as well as college performance data (e.g., their first-year GPA). If you are supplying any additional predictors or subgroups for your study, these will need to be included in your file.
Submitting Your Data File
You’ll be prompted to submit a data file when you design a new study. You can upload a new data file or reuse one from a previous study if it meets the new study requirements.
Your data file format must be Microsoft Excel, Comma Separated Value (CSV), Tab Delimited (.txt), or SAS Transport (XPORT).
For detailed information on preparing your data file, see the ACES Data Preparation Guidelines in the Resources section. You can also download an Excel template with the correct data layout for the SAT admission validity study.
College Board Matched Data
After we receive your data file, we’ll use the personally identifiable information contained in it to find matches to those student records in our College Board ACES database. Then we’ll combine our data and your data into one file.
When you receive your ACES admission validity report, you’ll also get what we call a “matched data file”—the complete set of combined data for all the students in your study.
The matched data file is invaluable for future research. It also comes with indicators that show which students are at risk for not returning, so you can target retention efforts.
Getting Your Report
When your study is finished—within 20 business days of your completed request—we’ll notify you that your report is available on the ACES website and you can sign in to access it. You’ll get three deliverables:
- A complete, printable report in PDF format that shows the strength of your chosen predictors of success—alone and in combination—with charts, tables, and detailed explanations.
- A report in HTML format featuring interactive graphs. You can click to display or hide data, compare data, zoom in and out, take a snapshot, sort table columns, and more. You may want to insert these graphs into campus reports or presentations you are preparing for various constituents.
- A matched data file that combines your data with College Board data and includes at-risk indicators for specific students.
FAQ
What’s the minimum number of students needed for an admission validity study?
Admission validity studies require data from a minimum of 50 students. See The cohort we want to study has fewer than 50 students; can we do an admission validity study?
The cohort we want to study has fewer than 50 students; can we do an admission validity study?
Yes; you will be able to combine data for several recent student cohorts. Note that the oldest class loaded in the system is the entering class of 2017. If you need to conduct a study with multiple cohorts combined, contact us for assistance.
How should we choose predictors for an admission validity study?
You can create your own custom predictors and supply data for them, or select predictors from the ACES system that ACES has data for, collected from the SAT Questionnaire (that SAT takers opt in to) or AP Exam data. These include average AP Exam score (calculated prior to senior year of high school), number of different AP Exams taken, number of honors or AP courses taken during high school, and number of activities during high school (limit of 10).
The primary purpose of an admission validity study is to evaluate measures used in admissions decisions—the predictors—to determine how well they work alone and in combination to predict student success. A good predictor, however, has several other important qualities. It should be widely available, reliable, and fair to all students. Predictors should show enough variation in scores to differentiate student ability, without large clumps of students at the top or the bottom scores.
When selecting predictors to include in your study, consider any possible contributors to your admissions decisions. ACES can sort through them for redundant or weak contributors and also provide prediction equations for future students.
The ideal prediction equation has multiple predictors that measure relatively different characteristics and consequently aren’t highly correlated. In such a situation, the correlations between the individual predictors and the criterion are more or less "additive." A less-than-ideal situation occurs when the individual predictors measure similar constructs and consequently are highly correlated (e.g., high school rank and high school GPA). The worst case is when two predictors are perfectly correlated.
Can we include part-time students in our sample for an admission validity study?
Part-time students can be included in your ACES study. However, part-time students can be very different from full-time students (e.g., they may be, on average, much older than traditional full-time students right out of high school). Therefore, you might want to include a school-supplied subgrouping variable stating the status of the student (e.g., 1= full-time, 2= part-time) so that you can compare them.
How should we handle student course grades of Fail, Incomplete, or Withdraw?
Generally, we have found that at most institutions failing grades, such as F or E, are calculated into the students' grade point average and are equivalent to zero grade/quality points; and Incomplete or Withdraw grades, such as I or W, are not incorporated into the GPA calculation. If this is true for your school, then you should include students who have received any (or all) of these grades.
Note that prior to conducting your first ACES study that includes course grades in the analysis, you will be asked to supply your institution’s comprehensive grading scale in the ACES system. This will involve mapping each letter grade that you assign to student performance to their corresponding numeric value or in the case of a grade like W for Withdraw, you would indicate “No Value.” The system will smoothly guide you through this process but it’s important that you are comprehensive and not forget to include grades that may have unusual characters like asterisks (e.g., A*, B+H for special honors course grades, or TA for an A in a transfer course).
Can we include students who don't complete their first year in our admission validity study?
If the student completed at least one semester, you should definitely include their data; if not, it is probably still a good idea. Although these students' records will not be used in the validity analyses if no criterion is present, they will be included on the electronic files you get back from ACES. This might help you to do internal retention analyses for students who drop out.
Can we include students who are missing SAT scores in our admission validity study?
Note that the starting point for constructing your sample for an ACES admission validity study should not be your SAT takers, it should be your entering class or the particular student sample (e.g., students admitted to your engineering school) you are focusing on for your study. Sometimes students have SAT scores on record at the College Board that the institution doesn't have, and the SAT scores in the ACES study are taken from the College Board database. Note that it's not possible to use concorded SAT scores in an ACES admission validity study, only official SAT scores.
Can we include international students in our admission validity study?
Yes. If international students are missing any key identification variables for matching, you may still include their records and we will attempt to match these records to the ACES database. Provided that the international student has the criterion measure (e.g., FYGPA), a high school measure of achievement, and SAT scores after the matching process has taken place, the student will be included in your study.
We don’t capture HSGPAs in our student information system; is this a problem?
All ACES admission validity studies require that you examine a HSGPA value along with SAT scores. Many institutions choose to include students’ HSGPAs in the file they submit to ACES. Sometimes, institutions do not have students’ HSGPAs on file, and this is not a problem. Instead, you can choose to use the “ACES-supplied” HSGPA taken from the student’s SAT Questionnaire data. You will indicate this choice when you begin the ACES study design process online.
If you opt to upload HSGPA, the HSGPA values in your file should not be from very different grading scales (e.g., a mix of 4-point and 100-point grade scales). If your institution rescales HSGPA for admission, then it would be best to submit the rescaled HSGPA. Otherwise, such a mixed scale predictor can adversely impact model performance and results.
How can I include special subgroup analyses in my admission validity study?
Subgroups are categorical variables that can be included in your data file/study to enhance the study results by providing subgroup analyses (or further breakdown of results). All ACES studies break down your results on the basis of gender and race/ethnicity whenever your sample includes 50 or more students for at least two levels of a subgroup (e.g., 50+ males and 50+ females). You may also specify up to three additional subgroups, again, using either ACES-supplied data (see below for these options), your own data, or a combination (adding up to three). Examples of possibly useful subgroup analyses include colleges within the institution (e.g., Arts and Sciences, Agriculture, Architecture, etc.) or First Generation College Student (yes/no), resident versus commuter student, Pell Grant Eligible (yes/no), etc. When submitting your data, you will be prompted to provide the label/description of the code values presented in your data (e.g., 1=Yes, 2=No).
Options for additional subgroups from ACES-supplied data are:
- First language (English only, English and Another Language, Another Language)
- Best language (English only, English and Another Language, Another Language)
Why would we want to request additional subgroups in an admission validity study?
There are a number of reasons for considering subgroup results. For example, the criterion might be different for each subgroup, such as when the college first-year GPAs for engineering majors differ from the first-year GPAs of education majors.
It is also appropriate to consider subgroup results when it is suspected that one or more of the predictors might function differently for each subgroup. This could be the case with high school GPA, which may give a different result for older applicants, who have been out of high school for a few years, than for younger applicants recently graduating from high school.
Although you will probably not use separate prediction equations for admission purposes, information provided about subgroup performance may be of interest to you in monitoring how special groups of students, perhaps ethnic or gender groups, perform after they are admitted into your institution. If the particular subgroup's actual performance is considerably above or below its predicted performance, you may want to pursue possible explanations for this outcome.
How many predictors can I include in my admission validity study?
Predictors are variables that can be included in your data file/study to provide additional information to explain/predict the outcome. Unlike subgroups, predictors should not be categorical, but continuous or dichotomous (e.g., coded 0 or 1 for absence or presence of a characteristic), and can take any numeric value. You will have the option to label and describe these variables. A study can have up to five additional predictors. An example of a predictor provided by an institution may be an academic rigor index or number of years of Math coursework.
Which College Board testing programs are supported by ACES?
ACES supports the SAT, Advanced Placement Program (AP), ACCUPLACER, and the College-Level Examination Program (CLEP).