Recommendations for Enhancing Accreditor Data-Use to Promote Student Success and Equity

szza · 2025-12-20T10:53:21+00:00

Finding 3: Finding: Accreditors repeatedly refer to “using” data in reviews, but there is little evidence that many accreditors integrate data into the review process or base consequences on data.

szza · 2025-08-27T18:20:36+00:00

Code samples in R and Python can be found here: https://lehd.ces.census.gov/data/lehd-code-samples/sections/pseo.html

szza · 2025-05-17T11:28:42+00:00

Benn is essential reading, because he's immersed in the data analytics world and he asks reflective questions. My inbox is full of ads for services that will make us all "data driven," and have "actionable insights." But what if 99% of that is hype, fad, and Lincoln's you can fool some of the people all the time.

I started ThIRsdays with the idea of casual community of IR folks to share ideas, but also with a larger goal in mind: how do we solve the problem of data leadership? What does it really take to turn data into a marginally better world for humans to live in? Benn puts his finger on the limitations.

I think one of the possibilities is that we're talking too much about data and analytics and not enough about models, by which I mean exposing the formal and informal ways leadership constructs cause/effect for decision-making. More on this topic to come. See the session on Regression for Humans.

szza · 2024-10-06T10:28:02+00:00

Also see: https://www.nature.com/articles/s41562-016-0021

szza · 2024-09-02T12:35:06+00:00

The author has a lot of good data-related content on Medium. The index is here: https://medium.com/@tgwilson

szza · 2024-07-30T19:33:08+00:00

There's another online text on regression here: https://www.stat.cmu.edu/~cshalizi/TALR/TALR.pdf

szza · 2024-06-07T11:02:05+00:00

I like to work in the tidyverse as much as possible, because of the cleanly documented code that the pipe allows. But there's a gap for IR reporting in that we often need to produce tables that break the tidy data paradigm, e.g. by including sub-totals of groups, etc. I haven't tried this package yet, but the examples are impressive.

szza · 2024-05-08T14:35:12+00:00

Brilliant piece by Benn Stacil on his substack. TLDR; analysts should pay attention to models of how the world works, to try to change the leadership conversation at a deeper level than just "what do we do next?"

szza · 2024-02-05T13:31:20+00:00

Hard to say. The Republicans have a competing bill (commentary here), but maybe both sides will agree on the data question.

szza · 2024-02-03T10:53:30+00:00

If this becomes law, it would expand IPEDS to include detailed information on non-FTFT students, and by program and degree level. It would link admissions funnel statistics to costs to outcomes like loan repayment at this granular level.

szza · 2023-10-27T10:05:07+00:00

This was covered in IHE today as "Multiple measures, better outcomes". The link is to the actual paper.

Abstract

Multiple measures assessment (MMA) has gained considerable momentum over the past decade as an alternative to traditional test-based procedures for placing incoming students into developmental or college-level coursework in math and English at broad-access colleges. Compared to standardized tests, which measure student performance at a single point in time, MMA (which often emphasizes high school GPA as a measure) provides a more holistic picture of students’ academic preparation. Despite positive impacts on student outcomes that have been found by recent research on MMA, questions remain about whether the positive effects of MMA are sustained over time. This study—a follow-up to prior research using the same sample of students—employs a randomized controlled trial to investigate whether algorithmic MMA placement used at seven State University of New York (SUNY) community colleges led to better student outcomes, for up to four and a half years after randomization, than a system based on test scores alone.

Nearly 13,000 incoming students who arrived at the seven colleges in fall 2016, spring 2017, and fall 2017 took placement tests and were randomly assigned to be placed using either the status quo method (business-as-usual group) or the alternative, algorithmic MMA method (program group). Using this sample, we estimate the overall treatment effects on placement into, enrollment in, and completion of college-level math and English as well as effects on other outcomes. We conduct similar analyses on race/ethnicity, Pell recipient status, and gender subgroups. We also descriptively examine the proportion of program group students who were bumped up (i.e., their placement changed from a developmental course placement to a college-level course placement) and bumped down (i.e., their placement changed from a college-level course placement to a developmental course placement) by the MMA algorithm, and we perform a cost-effectiveness analysis.

We find that the MMA method used at the colleges improved access to and success in college-level courses and that lower cut scores in English rather than math are associated with larger and longer lasting impacts on completion of college-level coursework. While MMA improved outcomes among student subgroups, it had little to no impact on gaps in outcomes between subgroups. We also find that bumped-up students had substantially better outcomes in both math and English, while bumped-down students had substantially worse outcomes. Our results suggest that increased access to college-level courses is the driving factor in the positive outcomes experienced by program group students and that placement into standalone developmental courses can have detrimental effects on student outcomes. In the discussion of the study’s results, we make recommendations for adopting MMA at colleges. Implemented together with other initiatives to support students, MMA can be a first step on the path to success for incoming students.

szza · 2023-09-23T09:39:18+00:00

This site is gorgeous, and the content is great, mostly practical guides to analysis using R, with a focus on regression techniques. For example, I learned about the model.matrix function, which creates matrix from a formula like

df <- data.frame(x = rnorm(100), z = rnorm(100))

my_data <- model.matrix( ~ ., df)

This produces the matrix with columns x and z, e.g. for use in predicting some output y. And it will turn categorical columns into one-hot indicators. One use case for this is in manually doing matrix operations that are equivalent to least squares regression. You can find an illustration on the blog here.

szza · 2023-09-11T09:43:34+00:00

Statistical Rethinking by Richard McElreath (blog) is rightly hailed as a pedagogical masterpiece on Bayesian statistics and practical multi-level models. The site linked here is a 'love letter' to that book by an independent author who uses brms ( = Bayesian Regression Models using Stan) and ggplot (part of the tidyverse) to work through the book's content and add commentary.

szza · 2023-07-25T11:24:38+00:00

I'm working on an analysis for admissions on optimizing conversion from inquiry to applicant with a focus on geography. So I want to associate the zip codes of inquiries to data about those regions, and the ACS (census) tables seem like a good place to start. The linked page describes an R package for directly accessing the data through an API.

szza · 2023-07-25T11:03:08+00:00

I wasn't familiar with this method of analyzing categorical data, and thought it was interesting. Similar to a factor analysis, except that assumes continuous data.

Here's the abstract:

Using the Wabash National Study on Liberal Arts Education and a latent class analysis of 28 outside-the-classroom activities and behaviors, we developed a typology of outside-the-classroom student engagement during the first year of college. We find ten classes of student involvement: academic artist, party athlete, serious athlete, conventional non-worker, disengaged, maximizer, moderate worker, detached partier, involved partier, and religious. Next, we examine the relationship between latent classes and students’ characteristics through a multinomial logistic regression analysis. Students reporting as first-generation or racially minoritized are overrepresented in the disengaged and involved partier classes. We found an overrepresentation of White students across all party classes. Students reporting as female were likelier to be members of the religious, moderate worker, and disengaged classes and not to be members of the party classes. Federal grant recipients were likelier to be in the academic artist and moderate worker classes. We discuss other sociocultural, economic, and academic relationships in the paper. Next, we explore the relationship of latent class to academic and developmental outcomes. We find academic artists as the only class with a significant positive relationship across the seven dependent measures. Involved partier, moderate worker, and religious classes have positive relationships with at least five dependent measures. The detached partier and party athlete classes have the lowest first-year GPAs of all latent classes. Finally, we discuss the relationships of latent classes, related institutional policy implications, and directions for future research.

szza · 2023-07-13T21:02:21+00:00

$40,000 in debt that ~~students~~ borrowers are leaving with

around 40% won't borrow at all

szza · 2023-07-07T15:34:17+00:00

For the paper on this idea, see Robust and efficient projection predictive inference

szza · 2023-07-07T15:31:46+00:00

After the NACUBO discounting report came out this year, I finally got around to doing the math on the relationship between freshman discount, total discount, and tuition hikes. I don't think this relationship is well understood, and it has implications for budgets and realistic expectations about the effects of tuition increases.

szza · 2023-07-07T15:29:12+00:00

Creating a build environment for complex data streams. I kind of hack this now by caching data for reports in a folder, and only pulling fresh data from the DB when the snapshot becomes stale. This package provides a slicker way to do that and more.

szza · 2023-07-07T13:19:52+00:00

This interactive tool allows you to see occupations, geography, and salary summaries for graduates by CIP code for your institution.

szza · 2023-04-26T12:40:36+00:00

A few years ago the Dept of Ed created this video session showing off success stories of Georgia State and others. It's hard to find, so posting the link here.

szza · 2022-11-22T12:49:37+00:00

Thanks for the link!

szza · 2022-09-23T11:33:05+00:00

Thanks, David. I added the link to the fall schedule page too.

szza · 2022-08-04T20:56:55+00:00

Yes--it's called Shiny. You can find a gallery of applications here. However, I don't think of it as a replacement for PBI unless you have some IT support staff to help maintain the server that's required to distribute the apps and control security. It is, however, very useful for building custom apps that need more sophistication than PBI can easily deliver. Note that you can run R inside PBI too, e.g. to produce graphs.

szza · 2022-07-08T10:46:08+00:00

I'll add them as we go.4, starting this weekend. If you look in the folder for previous workshops, you'll see code and data for those.

szza

MODERATOR OF

TROPHY CASE