Magazine article The Washington Monthly

A Note on Methodology: 4-Year Colleges and Universities

Magazine article The Washington Monthly

A Note on Methodology: 4-Year Colleges and Universities

Article excerpt

There are two primary goals to our methodology. First, we considered no single category to be more important than any other. Second, the final rankings needed to reflect excellence across the full breadth of our measures, rather than reward an exceptionally high focus on, say, research. Thus, all three main categories were weighted equally when calculating the final score. In order to ensure that each measurement contributed equally to a school's score within any given category, we standardized each data set so that each had a mean of zero and a standard deviation of one. The data was also adjusted to account for statistical outliers. No school's performance in any single area was allowed to exceed five standard deviations from the mean of the data set. Thanks to rounding, some schools have the same overall score. We have ranked them according to their pre-rounding results.

To establish the set of colleges included in the rankings, we started with the 1,727 colleges in the fifty states that are listed in the U.S. Department of Education's Integrated Postsecondary Education Data System as having a Carnegie basic classification of research, master's, baccalaureate, and baccalaureate/associate's colleges, are not exclusively graduate schools, and participate in federal financial aid programs. We then excluded 134 baccalaureate and baccalaureate/associate's-level colleges that reported that at least half of the undergraduate degrees awarded in 2012 were below the bachelor's degree level, as well as eleven colleges with fewer than 100 undergraduate students in fall 2012. Next, we decided to exclude the five federal military academies (Air Force, Army, Coast Guard, Merchant Marine, and Navy) because their unique missions make them difficult to evaluate using our methodology. Our rankings are based in part on the percentage of students receiving Pell Grants and the percentage of students enrolled in the Reserve Officers' Training Corps (ROTC), whereas the service academies provide all students with free tuition (and thus no Pell Grants) and commission graduates as officers in the armed services (and thus not the ROTC program). Our final set of exclusions was to not rank colleges that had not reported data on the three main measures used in the social mobility section (percent Pell, graduation rate, and net price) at least once in the past three years. This resulted in a final sample of 1,540 colleges and includes public, private nonprofit, and for-profit colleges.

The primary change in this year's rankings is the use of the three most recent years of data (each equally weighted) instead of the most recent year of data, as we had done in the past. This helps reduce wild swings in rankings, particularly at smaller colleges where a few more students graduating or defaulting on their student loans would have substantial implications for their rankings. Using the average of multiple years would hurt the ranking position of colleges that have exhibited rapid improvements in their outcomes, but the truth is that few colleges can move the dial this quickly. This will reduce size of the year-to-year changes in a college's rankings going forward, which may sell fewer magazines but paints a more accurate picture of performance.

Each of our three categories (service, research, and social mobility) includes several components. We have determined the community service score by measuring each school's performance in five different areas: the size of each school's Air Force, Army, and Navy ROTC programs, relative to the size of the school; the number of alumni currently serving in the Peace Corps, relative to the size of the school; the percentage of federal work-study grant money spent on community service projects; a combined score based on the number of students participating in community service and total service hours performed, both relative to school size; and a combined score based on the number of fulltime staff supporting community service, relative to the total number of staff, the number of academic courses that incorporate service, relative to school size, and whether the institution provides scholarships for community service. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.