Completion Study
This customized study will inform your understanding of student completion at your institution and assist you in predicting student completion outcomes. It will help you to understand whether some students are at greater risk for not completing and show you how to calculate and monitor this risk among applicants and enrolled students.
You will learn:
- Which measures are the most useful predictors of completion at your institution.
- How you might narrow the number of factors you consider without loss of predictive ability.
- How certain predictors might overlap or co-occur for non-completing students and can be used to consider student risk for not completing.
- How to construct the optimal equations for predicting the completion outcomes of future students.
Designing Your Study
In this section, get an overview of how to design your study. For detailed step-by-step instructions, download the ACES Completion Study Guide in the Resources section of this page.
Choosing a Cohort and Completion Outcome
You will select a cohort to examine—an earlier admitted class that is expected to have completion information—and a completion outcome. In ACES completion studies, the completion outcome is whether or not the student completed at your institution by a particular academic year/term.
Choosing Predictors
In ACES completion studies, a predictor refers to a data point that you are using to make your decisions and whose effectiveness you want to analyze. All ACES completion studies analyze SAT score(s) and give you the option to also study high school GPA and college GPA as predictors. You can add up to five additional predictors.
Choosing Additional Subgroups (Optional)
For ACES completion studies, you may request up to three additional subgroups (e.g., college within institution, in-state versus out-of-state) to further analyze your results.
Submitting Your Student Data
Which Students to Include
For a standard completion study, you should include all first-time, first-year students (domestic and international) that entered your institution in the starting academic term of interest. For example, if you are interested in studying four-year completion rates for the fall 2017 entering cohort, you would include all students who began at your institution in fall 2017 and their completion outcome (0, 1) from fall 2021. You would also include their completion date. Please note that the earliest cohort you can analyze in the ACES system is the fall 2017 entering class.
Which Data to Include
You'll need to supply identifying information on the students in your sample, as well as completion data (e.g., 0, 1) and the completion date (e.g. 05/15/2021). If you're supplying any additional predictors or subgroups for your study, these will need to be included in your file. If you expect that not all of your students will have matched SAT score information, we suggest including ACT scores for all students.
ACT scores are only used by the ACES system as concorded SAT scores when students do not have official SAT scores on record at the College Board. The ACT scores you provide are concorded to the corresponding SAT scores using the 2018 concordance tables within the ACES system itself.
Submitting Your Data File
You'll be prompted to submit a data file when you design a new study.
Your data file must be in one of the following formats in order to successfully upload: Microsoft Excel; Comma Separated Value (CSV); Tab Delimited (.txt); or SAS Transport (XPORT).
For detailed information on preparing your data file, see the ACES Data Preparation Guidelines in the Resources section. You can also download an Excel template with the correct data layout for the completion study.
Getting Your Report
When your study is finished—within 20 business days of your completed request (which must include a clean data file)—we will notify you that your report is available on the ACES website and you can sign in to access it. You will get two deliverables:
- A complete, printable report in PDF format that shows the strength of your chosen predictors of completion with charts, tables, and detailed explanations.
- A report in HTML format featuring interactive graphs. You can click to display or hide data, compare data, zoom in and out, take a snapshot, sort table columns, and more. You may want to insert these graphs into campus reports or presentations that you are preparing for various constituents.
FAQ
What is the minimum number of students needed for a completion study?
You will need at least 200 students in your study sample to conduct a completion study. In order to analyze completion results by subgroup, you will need at least 100 students in at least one level of the subgroup.
How should we choose predictors for a completion study?
In addition to selecting SAT scores, you can create your own custom predictors and supply data for them.
The primary purpose of a completion study is to evaluate the measures used in predicting and understanding completion and to learn which students may be at greater risk for not completing in the future. A good predictor, however, has several other important qualities. It should be widely available, reliable, and fair to all students. Predictors should show sufficient variation in scores to differentiate student ability, without large clumps of students at the top or the bottom scores. Completion studies also support two-category (0,1) predictors that can represent student characteristics.
When selecting predictors for your study, you should consider including any possible contributors to understanding completion at your institution. ACES can help to sort through them for redundant or weak contributors, as well as to provide prediction equations for future students.
Can we include students who are missing SAT scores in our completion study?
Yes. You should include all students in the cohort that you are choosing to study. You should not provide students’ SAT scores in your file as they will be matched to your student data from the College Board database. You may want to include ACT scores in your file when available. The system can then use students’ concorded SAT scores (based on the ACT scores) in analyses along with actual SAT scores for the rest of the cohort who have SAT scores on record.
If you are examining SAT total scores in your ACES completion study, then include students’ ACT composite scores. If you’re studying SAT Evidence-Based Reading and Writing (ERW) scores and Math scores, then include students’ ACT Reading, English, and Math scores in your file.
Which score(s) will be used in the analysis?
The system will use the student’s SAT scores when both SAT scores and ACT scores (in the data file to be concorded to SAT scores) are available. Note: ACT scores should not be submitted as an "Additional Predictor" in the completion study as they can adversely impact the model performance and results.
We do not capture HSGPAs in our student information system. Is this a problem?
If you want to study HSGPA in an ACES Completion Study, you will need to supply the HSGPA in your file. Note: HSGPA is an optional variable in completion studies.
We have different HSGPA scales in our student information system. Is this a problem?
If you opt to include HSGPA as a predictor in an ACES Completion Study, the HSGPA values in your file should not be from very different grading scales (e.g., a mix of 4-point and 100-point grade scales). If your institution rescales HSGPA for admission, then it would be best to submit the rescaled HSGPA. If not and you are unable to align very different HSGPA scales to a common scale, then it would be best to not include HSGPA as a predictor, since such a mixed scale predictor can adversely impact model performance and results. Note: HSGPA is an optional variable in completion studies.
How can I include special subgroup analyses in my completion study?
Subgroups are categorical variables that can be included in your data file/study to enhance the results by providing subgroup analyses (or further breakdown of results). You may specify up to three additional subgroup variables using your own data. An example of a possibly useful subgroup for analysis is “College within the institution” (e.g., Arts and Sciences, Agriculture, Architecture, etc.).
Note: In order for subgroup analyses to run, you'll need at least 100 students in at least one level of the subgroup (e.g., 100 or more students in Arts and Sciences).
How many predictors can I include in my completion study?
Predictors are variables that can be included in your data file/study to provide additional information to explain/predict the outcome. They can either be continuous numeric predictors or predictors with two distinct categories. SAT scores are always included as predictors in ACES completion studies, and you can also choose to include HSGPA and college GPA as predictors in your completion study (both supplied by your institution).
A completion study can have up to five additional predictors above SAT scores and the optional HSGPA and college GPA. An example of a predictor that an institution may choose to additionally provide is student credit load.
What is a completion descriptor in an ACES Completion Study, and how is that incorporated in the study?
Completion descriptors are indicators that you optionally create when designing your completion study. They define characteristics potentially associated with noncompleting students and are constructed by specifying a cutoff value for any of the SAT score, HSGPA, or college GPA predictors.
Students at or below the cut-point on the predictor are identified and several tables and graphs in the study present how many noncompleting students exhibit these characteristics, singly and in combination, providing additional insight into noncompleting students. For example, if HSGPA is a predictor, then at your institution, students with a HSGPA at or below 3.0 might be considered more at risk and a completion descriptor could be defined on this basis.
When can I conduct an ACES Completion Study?
The 2017 cohort is the earliest cohort that can be analyzed in the ACES system. You can begin studying four-year completion in the summer of 2021, five-year completion in the summer of 2022, and six-year completion in the summer of 2023. In keeping with this example, you can conduct an ACES four-year completion study for the 2018 entering cohort in 2022.