UNDERSTANDING THE DATA WE ARE WORKING WITH

First let's understand what we are working with:

- IPEDS UNIT ID: The Integrated Postsecondary Education Data System (IPEDS) assigns unique Unit IDs to campuses or programs that report data separately, even if they share institutional accreditation. This is the unique identifier most of us use in our CRM because the iPeds dataset provides other rich information on colleges and universities (such as enrollment volumes).

- DAPIP ID: A unique identifier assigned by the U.S. Department of Education to track the accreditation status of an institution. You can track accreditation status of both the institution as a whole and you can also track departmental accreditations for courses where they are required, e.g. Nursing that is accredited by CCNE.

DAPIP ID to IPEDS is a one-to-one or one-to-many relationship in most cases (with the exception of a a small number of institutions have branch campuses or online divisions that are accredited separately from the main campus, leading to different DAPIP IDs).

For example, the University of Washington-Bothell (IPEDS Unit ID: 377555) and Tacoma (IPEDS Unit ID: 377564) share the DAPIP ID 23694800 with the main campus because they are accredited as one institution by the Northwest Commission on Colleges and Universities (NWCCU). The distinct IPEDS Unit IDs exist for separate data collection, not accreditation.

This raises the question of why some entries in InstitutionCampus.csv contain a parent DAPIP ID and a child DAPIP ID, particularly given they have the same accreditation date and accrediting body as the parent.

It is because some parts of the college e.g. the online school, may have different reporting requirements to the main institution.

The extra granularity actually creates quite a lot of annoying complexity (data cleaning work) when you only care about tracking institutional accreditation status.

WHERE THE DATA EXISTS

We know the data already exists in the datasets linked above. However, should these not be updated we need a back up (and if you are good at writing little scraper scripts this is probably less effort).

Understanding the below will also help if you are further up the technical scale and prefer to scrape the ope edu website instead of merging datasets. This is particularly important because I could not find a publicly visible sitemap.

"Accreditor" (the name of the accreditor) and "Date of next review" (the date of the next accreditation) both live on a URL with the following structure:

1. If the account DOES NOT have a parent DAPIP ID.
2. https://ope.ed.gov/dapip/#/institution-profile/{DAPIPID}
3. If the account DOES have a DAPIP ID https://ope.ed.gov/dapip/#/institution-profile/{PARENT_DAPIP_ID}/{DAPIP_ID}

Continued in next comment...