Retention Study
This customized study will inform your understanding of student retention at your institution and assist you in predicting student retention outcomes. It will help you to understand whether some students are at greater risk for not returning, and show you how to calculate and monitor this risk among applicants and enrolled students.
You’ll learn:
- Which measures are the most useful predictors of retention at your institution.
- How you might narrow the number of factors you consider without loss of predictive ability.
- How certain predictors might overlap or co-occur for nonreturning students and can be used to consider student risk for not returning.
- How to construct the optimal equations for predicting the retention outcomes of future students.
- Which specific returning students in the data set you supply are at risk for not returning in the future.
Designing Your Study
In this section, get an overview of how to design your study. For detailed step-by-step instructions, download the ACES Retention Study Guide in the Resources section of this page.
Choosing a Cohort and Retention Outcome
You’ll select a cohort to examine—for example, last year’s admitted class—and a retention outcome. In ACES retention studies, the retention outcome is whether or not the student returned to your institution in a particular academic year/term.
Choosing Predictors
In ACES retention studies, a predictor refers to a data point that you’re using to make your decisions and whose effectiveness you want to analyze. All ACES retention studies analyze SAT score(s) and give you the option to also study high school GPA and college GPA as predictors. You can add up to five additional predictors.
Choosing Additional Subgroups (Optional)
For ACES retention studies, you may request up to three additional subgroups (e.g., college within institution, in-state versus out-of-state) to further analyze your results.
Submitting Your Student Data
Which Students to Include
For a standard retention study, you should include all first-time, first-year students (domestic and international) that entered your institution in the starting academic term of interest. For example, if you're interested in studying second-year retention rates for the fall 2017 entering cohort, you would include all students who began at your institution in fall 2017 and their retention outcome (0, 1) from fall 2018.
Which Data to Include
You'll need to supply identifying information on the students in your sample, as well as retention data (e.g., 0, 1). If you're supplying any additional predictors or subgroups for your study, these will need to be included in your file.
Submitting Your Data File
You'll be prompted to submit a data file when you design a new study.
Your data file must be in one of the following formats in order to successfully upload: Microsoft Excel; Comma Separated Value (CSV); Tab Delimited (.txt); or SAS Transport (XPORT).
For detailed information on preparing your data file, see the ACES Data Preparation Guidelines in the Resources section. You can also download an Excel template with the correct data layout for the retention validity study.
College Board Matched Data
After we receive your data file, we will use the personally identifiable information contained in it to find matches to those student records in our College Board ACES database. Then we will combine our data and your data into one file.
When you receive your ACES retention report, you'll also get what we call a “matched data file”—the complete set of combined data for all the students in your study.
The matched data file is invaluable for future research. It also comes with indicators that show which students are at risk for not returning in the future, so you can target retention efforts.
Getting Your Report
When your study is finished—within 20 business days of your completed request (including a clean data file)—we’ll notify you that your report is available on the ACES website and you can sign in to access it. You will get three deliverables:
- A complete, printable report in PDF format that shows the strength of your chosen predictors of retention with charts, tables, and detailed explanations.
- A report in HTML format featuring interactive graphs. You can click to display or hide data, compare data, zoom in and out, take a snapshot, sort table columns, and more. You may want to insert these graphs into campus reports or presentations that you are preparing for various constituents.
- A matched data file that combines your data with College Board data and includes retention probabilities for the students in your file.
FAQ
What’s the minimum number of students needed for a retention study?
You'll need at least 200 students in your study sample to conduct a retention study. In order to analyze retention results by subgroup, you'll need at least 100 students in at least one level of the subgroup.
How should we choose predictors for a retention study?
In addition to selecting SAT scores, you can create your own custom predictors and supply data for them.
The primary purpose of a retention study is to evaluate the measures used in predicting and understanding retention and to learn which students may be at greater risk for not returning in the future. A good predictor, however, has several other important qualities. It should be widely available, reliable, and fair to all students. Predictors should show sufficient variation in scores to differentiate student ability, without large clumps of students at the top or the bottom scores. Retention studies also support two-category (0,1) predictors that can represent student characteristics.
When selecting predictors for your study, you should consider including any possible contributors to understanding retention at your institution. ACES can help to sort through them for redundant or weak contributors, as well as to provide prediction equations for future students.
Can we include students who are missing SAT scores in our retention study?
Yes. You should include all students in the cohort that you're choosing to study. You should not provide students’ SAT scores in your file as they'll be matched to your student data from the College Board database. You may want to include ACT scores in your file when available. The system can then use students’ concorded SAT scores (based on the ACT scores) in analyses along with actual SAT scores for the rest of the cohort who have SAT scores on record.
If you are examining SAT total scores in your ACES retention study, then include students’ ACT composite scores. If you’re studying SAT Evidence-Based Reading and Writing (ERW) scores and Math scores, then include students’ ACT Reading, English, and Math scores in your file.
If a student has both SAT and ACT scores on record, which score(s) will be used in the analysis?
The system will use the student’s SAT scores when both SAT scores and ACT scores (in the data file to be concorded to SAT scores) are available. Note: ACT scores should not be submitted as an "Additional Predictor" in the retention study as they can adversely impact the model performance and results.
We don’t capture HSGPAs in our student information system. Is this a problem?
If you want to study HSGPA in an ACES Retention Study, you'll need to supply the HSGPA in your file. Note: HSGPA is an optional variable in retention studies.
How can I include special subgroup analyses in my retention study?
Subgroups are categorical variables that can be included in your data file/study to enhance the results by providing subgroup analyses (or further breakdown of results). You may specify up to three additional subgroup variables using your own data. An example of a possibly useful subgroup for analysis is “College within the institution” (e.g., Arts and Sciences, Agriculture, Architecture, etc.). Note: In order for subgroup analyses to run, you'll need at least 100 students in at least one level of the subgroup (e.g., 100 or more students in Arts and Sciences).
How many predictors can I include in my retention study?
Predictors are variables that can be included in your data file/study to provide additional information to explain/predict the outcome. They can either be continuous numeric predictors or predictors with two distinct categories. SAT scores are always included as predictors in ACES retention studies, and you can also choose to include HSGPA and college GPA as predictors in your retention study (both supplied by your institution).
A retention study can have up to five additional predictors above SAT scores and the optional HSGPA and college GPA. An example of a predictor that an institution may choose to additionally provide is student credit load.
What is a retention descriptor in an ACES Retention Study? And how is that incorporated in the study?
Retention descriptors are indicators you optionally create when designing your retention study. They define characteristics potentially associated with nonreturning students and are constructed by specifying a cutoff value for any of the SAT score, HSGPA, or college GPA predictors.
Students at or below the cutoff point on the predictor are identified and several tables and graphs in the study present how many nonreturning students exhibit these characteristics, singly and in combination, providing additional insight into nonreturning students. For example, if HSGPA is a predictor, then at your institution, students with a HSGPA at or below 3.0 might be considered more at risk and a retention descriptor could be defined on this basis.
Which score(s) will be used in the analysis?
The system will use the student’s SAT scores when both SAT scores and ACT scores (in the data file to be concorded to SAT scores) are available. Note: ACT scores should not be submitted as an "Additional Predictor" in the completion study as they can adversely impact the model performance and results.