| RED HERRINGS |
| | |
Q: Of course alumni of private colleges are going to give their experiences high marks! Not many would be likely to say, “I really screwed up by going there." |
A: Survey respondents are simply told they will be asked questions about their college experience, not that we will be making comparisons. We wouldn’t assume that alumni of public institutions would be any less defensive of their choices. All responding alumni—public and private—have an equal opportunity to look back with rose colored glasses, with an ax to grind, or in any other way they choose. |
| |
Q: It could be that students who aspire for graduate degrees or are interested in issues of faith are more likely to attend private colleges, explaining those findings. |
A: True. And one way to interpret these results would be to hypothesize that the fit between institutional mission and student intent matter. |
| |
| Q: There was no control group, therefore the study is invalid. |
A: We have never cast the comparative alumni research as objective proof that private colleges generally deliver greater educational effectiveness. The comparative alumni research measures perceptions and reported experience. Analysis of the findings is wholly consistent with earlier controlled studies by higher-education scholars and with National Survey of Student Engagement data assessing the role of engagement in student learning. Whether the school is big or small, coeducational or single sex, engagement and interaction produce greater effectiveness and benefits for students. We don’t know of anyone who argues with this. Our research simply documents how frequently such engagement and interaction occurred for and benefited the graduates we surveyed, according to them. Analysis of these reported responses shows that this happens far more frequently at private institutions. |
| |
| GOOD QUESTION, HERE ARE THE DETAILS |
| | |
Q: Aren’t students who attend private colleges likely to come from wealthier families with college-educated parents who sent them to the best high schools and pay for test prep courses? How do you know those factors aren’t what account for the differences between them and public university alums? (A variation is, “Aren’t private colleges more selective, and doesn’t that explain it all?”) |
A: We haven’t been able to obtain uniform, objective entering-student data such as test scores, grade point average, class rank, and other indicators for graduates across the range of graduation years we survey (going back to 1970). An alternative we considered was to ask respondents about a selection of these variables. We dropped this idea as a result of testing. Too many alumni could not remember things like their SAT scores, presumably because they devote their memories to information with some actual utility. So no, we don’t have reliable data about alumni entering academic, economic, or aspirational profiles. Instead, we used selectivity of each college as a proxy for these entering factors, reasoning that the reported experiences and benefits and outcomes would vary with the ability of the college or university to select its class and to not admit those students unable to handle the work or not suited for the particular institutional mission. In the Women’s Colleges Coalition survey, the coed private colleges and public institutions as a group were more selective than the women’s schools. This suggests rather strongly that the survey findings are not attributable to an academically elite incoming student profile. And while we do not have entering student family income, we do have objective proof from other research that public flagship universities in many states enroll a somewhat higher family-income cohort than do private colleges in those states. |
| | |
| Q: Don’t college selectivity ratings change over time? |
| A: We double-check the selectivity ratings of each college in the consortia when we do new interviews to make sure they haven't changed, and none have in the last ten years. There are probably some—especially public universities—that have become more selective since 1970, but no one college or university makes up more than a very small proportion of any group. (As in investing, there's safety in diversification!) |
| |
| Q: Don’t people’s answers change over time? |
A: That’s a question we asked ourselves. Might the same people answer the same question differently in 1998 than in 2003 (perceptual drift)? We conducted a test with two panels of alumni drawn at random from two national marketing list sources. We made sure they matched on year of graduation and selectivity of college. Out of the 57 questions we asked, respondents’ replies differed significantly on 19—10 of which were about religion and faith while the other 9 were scattered among topics. It makes sense that a person’s views on religion and faith would change over time, while their college experience would remain more fixed. |
| | |
Q: Who are these guys Astin, Pascarella, and Terenzini and why do you keep referring to them? |
A. Hardwick-Day’s comparative alumni research design and survey instrument were devised to test, based upon the reported experiences and perceptions of graduates, the findings of their earlier scholarly research, which found that certain types of engagement and interaction produce greater effectiveness and benefits for students. (Astin, A.W., What Matters in College? Four Critical Years Revisited. San Francisco: Jossey-Bass, 1993 and Pascarella, E.T., and Terenzini, P.T., How College Affects Students. San Francisco: Jossey-Bass, 1991.) |
| |
Q: If you’re calling people off an alumni list provided by a college, they’re probably already biased toward the place because they’ve given it their phone number. |
A: One would think so, so we tested this. We compared two samples—one from a national marketing data base and one from a client list. Both were composed of alums of private institutions. Of the 57 questions we asked, only 8 responses differed significantly between samples. Again, most of the difference had to do with respondents from the client list citing higher involvement in and placing more importance on religion and faith. This only makes sense, since the majority of the client colleges have faith-based missions. |
| | |
Q: Since this study was sponsored by XYZ Private College Coalition, maybe you just didn’t show the negative outcomes? |
A: Hardwick-Day provides the results of every variable examined to our consortia clients. That can be up to 375 variables. The margins of error have been between 4.9 and 6.1 at a 95% confidence level. It is up to consortia to interpret and disseminate the data. Clients present a variety of variables, some where they perform better and some where they are not different than the comparison groups (no "significant" differences). Each chooses what makes the most accurate description of the college, while keeping the story succinct. The Women's Colleges Consortia made everything public. In the study, alumnae of women’s colleges were much less likely to participate in intramural athletics (25%) than alums of other private colleges (39%). That could be good or bad depending on your point of view. Also in the women’s study, alums of women’s colleges were much more likely to believe that “It is extremely important to be well-off financially” (33%) than either flagship alums (22%) or graduates of other private institutions (17%). If this is a negative outcome, the Women’s College Coalition didn’t shy away from presenting it. And, all clients who contribute to the data base by commissioning this research agree that their data will become part of the larger data base and will therefore be available for future studies, as well as scholarly analysis by qualified scholars whose research interest indicates a public purpose. |
| |
| THAT WOULD BE A GREAT QUESTION TO EXPLORE FURTHER |
| | |
Q: Why didn’t they compare current income or some other quantifiable metric? |
A: That's a great question we would love to explore. Naturally, there would be more to it – we’d have to define “income” to exclude spouse’s income – or would we? Many people meet there spouses in college. Does income vary by where your spouse went to college? |
| |
Q: What about gender? Do men have the same experiences as women at private vs. public institutions? |
A: It would also be interesting to know more about first-generation students, low-income students, students of color, and any number of other characteristics. |
| |
Q: Why are there differences in the skills (thinking, writing, leading) that alumni of private colleges report compared with those who attended public institutions? |
A: This question was posed by Susan Lennon, head of the Women’s College Coalition. Obviously there are endless research avenues to follow, and as with everything in higher education, what gets priority boils down to leadership, personnel, time, and funding. |
