• Executive Summary
  

• Introduction
  
  • Background
    

  • Sample
    

  • Quality assurance
    

  • Rating scales used
    

  • Interpretation of rating scales
    

• Survey findings
  

• Goals of Undergraduate Education
  

• General Education Preparation
  

• Summary and Conclusions
  
  

Return to OIRP Survey Page
Return to OIRP Home Page

Executive Summary

Background

The NC State Alumni Survey of baccalaureate alumni who graduated from December 1990 through August 1993 sought among other things to identify the extent to which bachelor's alumni valued and were satisfied that they had met goals established for undergraduate education at NC State. This report, the fourth in a series of reports on data gleaned from the survey, focuses on how all responding alumni (n = 3,179) rated items concerned with the importance of various goals for undergraduate education, the extent to which alumni were satisfied that their undergraduate education at NC State had met those goals, and the differences between those two ratings. The differences between satisfaction and importance ratings are called gap scores; they are used to measure the extent of any disjuncture between perceptions of alumni preparation and importance of goals for undergraduate education. Mathematically, this may be expressed by the formula:

Other items on the NC State Alumni Survey sought to assess the extent to which agreement was present among alumni concerning their preparation in terms of general education requirements. These items were developed to specifically address aspects of the assessment plan for general education requirements developed by the Council on Undergraduate Education.

Goals of Undergraduate Education

Among the goals established for undergraduate education at NC State, preparing for a career was rated by alumni as the single most important facet, followed by viewing learning as a lifelong process. Being involved in public and community affairs was rated as one of the least important facets, a situation that speaks to a lack of understanding of the role of land-grant universities in community service among our alumni. In terms of alumni satisfaction, the highest ratings among these goals were given to acquiring a broad general education, the ability to critically analyze events, information and ideas, and viewing learning as a lifelong process. This finding speaks very positively to the efficacy of undergraduate programs at NC State in the three vital areas of general education preparation, critical thinking skills, and lifelong learning.

The item rated as most important, preparing for a career, also showed the largest gap score for any item in this group. The gap between importance and satisfaction ratings on this item was greatest for the School of Design, the College of Management, and the College of Humanities and Social Sciences.

General Education Preparation

Eight items on the NC State Alumni Survey were specifically developed to measure progress towards achievement of general education requirements as outlined through the work of the Council on Undergraduate Education at NC State. On the positive side, the literacy and current events measure (reading newspapers and magazines) received a high level of support. Similarly, the item concerned with whether or not NC State graduates had been prepared to be competitive with graduates of other institutions received strong support. This finding may be interpreted as another bottom-line indicator of the efficacy of undergraduate preparation at NC State. On the other hand, respondents were less inclined to agree that foreign language or social science courses had been particularly helpful to them so far. Results for the other literacy item, the frequency of leisure reading, were mixed. While 62.2 percent of respondents reported reading for leisure at least once a week, 12.4 percent indicated that they read for leisure seldom or not at all.

Summary and Conclusions

Findings from this study are mixed, and raise several implications for undergraduate education at NC State. On the positive side, responding alumni appear to be essentially satisfied with the broad general education they acquire at NC State, and gave high ratings to their competitive preparation, ability to access information, creative thinking processes, current events literacy, their view of learning as a lifelong process, and their ability to critically analyze events, information, and ideas. On these measures, it is clear that responding alumni regard their preparation at NC State as successful.

On several other measures, alumni ratings raise more questions than provide answers. Especially noteworthy among these are measures of public and community affairs involvement, preparing for a career, understanding one's abilities and interests, and frequency of leisure reading. Alumni also offered substantially lower levels of agreement regarding the usefulness of foreign language and social science courses. The findings speak to a desire for better career preparation and more knowledge of self, yet betray a certain narrowness of focus among responding alumni, as if preparation for both local and global citizenship were not a vital function of the university.

Return to Table of Contents
Return to OIRP Survey Page

Introduction

Background

The NC State Alumni Survey of baccalaureate alumni who graduated from December 1990 through August 1993 sought among other things to identify the extent to which bachelor's alumni were satisfied that they had received quality training in professional skills and abilities at NC State. This report, the fourth in a series of reports on data gleaned from the survey, focuses on how alumni responded to questions concerned with the goals of undergraduate education and general education preparation.

Sample

Data obtained from this survey represent the largest and most comprehensive effort in the history of NC State to gain feedback from alumni. The survey sample included 7,491 alumni. Three full mailings and a telephone follow-up resulted in a 51.2% response rate, yielding 3,179 usable questionnaires.

Quality assurance

Data quality assurance steps taken yielded the following findings:

• Precision: The overall margin of error for items included this report was extremely narrow at 1.9 percent (at a 95 percent confidence level).
• Reliability: Individual item reliabilities were high (Cronbach's Alpha = 0.96).
• A telephone follow-up survey demonstrated that late (telephone) respondents did not think less of their undergraduate preparation than did earlier (mail) respondents; indeed, telephone respondents were significantly more positive (p = .05 or better) in their assessment of the education gained at NC State than were earlier respondents on several items.
• Significant differences were found between the population and obtained sample in proportion of gender, race, and college, but no differences were found by year of graduation. Though these differences were significant (p = .001), they were not of sufficient magnitude (i.e., 5% discrepancy or greater) to suggest that the obtained sample was appreciably different from the population of bachelor's graduates from 1990-1993.
• This report is therefore based on the conclusion that results obtained are both accurate and broadly representative of the population of NC State bachelor's graduates for those years. Further methodological information on this survey is available from University Planning and Analysis.

Rating scales used

All items concerning the goals of undergraduate education (items 14-30 on the alumni survey instrument) were rated by respondents on five-point scales. For items assessing the importance of various areas, the scales ranged from 5 = very important to 1 = not important. For items assessing the level of satisfaction with each area, the scales ranged from 5 = very satisfied to 1 = very dissatisfied. Responses in the "not applicable" category were omitted in calculation of the mean score for both importance and satisfaction items. Seven items concerning general education preparation (items 31-37 on the alumni survey instrument) were rated on five-point scales where 5 = agree and 1 = disagree. Responses in the "no opinion" category were omitted in calculation of the mean score for these items.

Interpretation of rating scales

In this report, three distinct pieces of information are presented on each survey item concerned with the goals of undergraduate education. First, mean ratings of the relative importance to alumni of each item to alumni are presented. Next, mean ratings of alumni satisfaction that their undergraduate education met each goal are shown. For items disaggregated to the college/school level, tests of significance were performed on these means; significant departures from the university-level means are noted. Finally, the means of observed differences between levels of importance and levels of satisfaction are presented. These means of observed differences are labeled "gap scores" and are included to provide an indicator of the size and direction of areas for improvement (indicated by a negative gap score) or areas in which level of satisfaction exceeds perceived importance of the goal (indicated by a positive score). T-tests were conducted to determine whether or not these gap scores were significantly different from zero; significant gap scores are noted in the tables. When viewing the results of this survey, it is important to remember that all three elements should be considered - importance, satisfaction, and gap scores - before arriving at a judgment as to the correct interpretation of the findings.

As a guide to interpreting the relevance of the gap score, in any instances where the gap score is larger than -1.0, the item most probably merits further investigation. In other cases, it may be useful to examine the standard deviations for both importance and satisfaction. In situations where the gap score is negative and larger than the standard deviation of either the importance or satisfaction rating, further investigation may be warranted. In situations where the gap score for a particular item is negative and larger than the standard deviation for both importance and satisfaction ratings, a real difference between importance and satisfaction is implied and further investigation is indeed warranted. Note: due to rounding and to the differing number of respondents to items on importance and satisfaction, mean gap scores may not exactly total the difference between the mean importance and mean satisfaction ratings.

Survey findings

Goals of Undergraduate Education

Eight items in this section of the survey were rated as important by responding alumni. Not surprisingly, preparing for a career was rated as the single most important facet, followed by viewing learning as a lifelong process. Nine other items were rated as moderately important. The three items rated least important in this section were being involved in public and community affairs, understanding the present as it relates to historical events/processes, and advancing my appreciation of the arts, music, and literature. In terms of satisfaction, the three highest ratings were given to acquiring a broad general education, ability to critically analyze events, information, and ideas, and viewing learning as a lifelong process. These findings are displayed in table 1.

To permit a visual comparison of these findings, ratings of importance and satisfaction are displayed in chart 1.

An examination of chart 1 provides several useful insights. To begin with, the goal of acquiring a broad general education was not only rated as important, but also received the highest rating of any item in this group in terms of alumni satisfaction. This is a very positive finding, and indicates that, as far as responding alumni are concerned, the general education they received at NC State has essentially met their needs. To permit a more detailed examination of the response to this item, results are displayed by school/college in table 1a below.

Results displayed in table 1a demonstrate that respondents across the university (except for College of Engineering) considered acquiring a broad general education to be an important goal of undergraduate education. Furthermore, respondents in the College of Agriculture and Life Sciences and the College of Humanities and Social Sciences expressed a significantly higher degree of satisfaction with their preparation in this area than did respondents university-wide. On the other hand, respondents from the College of Engineering expressed a significantly lower degree of satisfaction and respondents from the School of Design expressed the lowest degree of satisfaction with general education preparation across the university.

Second, being involved in public and community affairs was not only perceived as of lesser importance than most other goals, but it was also rated the lowest of any item in this group in terms of alumni satisfaction. This finding reveals a lack of understanding among responding alumni of the importance of community service as well as a lack of preparation in this area at NC State - clearly an undesirable situation for a major land-grant university.

Third, the item rated as most important, preparing for a career, also showed the largest gap score for any item in this group (-.99). This may be taken as an indication that responding alumni perceived a disjuncture between the importance of career preparation as a goal for undergraduate education and how satisfied they were with the career preparation received at NC State. Given the size of the gap score, a separate analysis of this item was done by school/college and appears below in table 1b.

Results displayed in table 1b indicate that while responding alumni rated the importance of preparing for a career consistently across the university, respondents from the School of Design and the College of Humanities and Social Sciences evinced significantly lower satisfaction with their career preparation at NC State than did respondents from other colleges. Moreover, the gap scores on this item for the School of Design, the College of Management, and the College of Humanities and Social Sciences are substantial and indicative of a real difference between how important this goal was to responding alumni and their satisfaction with the preparation they perceived.

A less sizable but still troubling gap score (-.77) was observed on understanding my own abilities and interests. Again, this may be taken as an indication that responding alumni perceived a difference between the importance of understanding one's abilities and interests as a goal for undergraduate education and how satisfied they were that they had met this goal at NC State. Given the size of the gap score, a separate analysis of this item was done by school/college and appears below in table 1c.

As can be seen in table 1c, respondents across the university indicated that understanding their abilities and interests was an important goal to them for undergraduate education at NC State. In terms of preparation, respondents from the School of Design and the College of Humanities and Social Sciences expressed significantly higher satisfaction than university respondents overall, while respondents from the College of Engineering noted the lowest level of satisfaction on this item across the university.

General Education Preparation

Eight items on the NC State Alumni Survey were specifically developed to measure progress towards achievement of general education requirements as outlined through the work of the Council on Undergraduate Education at NC State. The scale used for these items was a Likert-type scale, where 5 = agree and 1 = disagree. The mean ratings for these items are given below in table 2.

To permit a visual comparison of these findings, ratings of importance and satisfaction are displayed in chart 2.

As can be seen from chart 2, respondents expressed the highest levels of agreement on four items, and were somewhat less inclined to express support for three other items. On the positive side, the literacy and current events measure (reading newspapers and magazines) received a high level of support, as did the item on whether or not NC State graduates had been prepared to be competitive with graduates of other institutions. On the other hand, respondents were less inclined to agree that foreign language or social science courses had been particularly helpful to them so far.

Responses to these items on general education preparation were disaggregated to the school/college level to ascertain any differences in response. On the three items concerned with (1) knowing how to access and use information, (2) foreign language preparation and (3) frequency of reading newspapers/magazines, little variation was evident across schools or colleges. On four items, responses varied significantly by school or college. Responses to these items by school or college are displayed below in tables 2a - 2d.

As shown in table 2a, alumni survey respondents from the College of Textiles showed significantly higher agreement than respondents university-wide on this item. However, respondents from the College of Education and Psychology reported significantly lower agreement than did respondents across the university.

Results displayed in table 2b clearly demonstrate that respondents from colleges connected with the social sciences were significantly more inclined to show agreement with this item than were respondents from other schools or colleges. While not surprising, this finding demonstrates that the goal of providing every student with an understanding and appreciation of the role of social sciences may still be some distance ahead.

Responses displayed in table 2c show that respondent agreement with this item tended to be significantly higher in the School of Design and in the College of Humanities and Social Sciences, whereas respondents from the College of Management showed a significantly lower level of agreement.

The results shown in table 2d are another important bottom-line indicator of undergraduate preparation at NC State. This item deals with alumni perceptions of how well their skills and abilities match up against the skills and abilities displayed by graduates of other institutions. Overall, the level of agreement expressed on this item is quite high. Clearly, responding alumni from the College of Engineering and the College of Textiles were in agreement that they had been competitively prepared. Respondents from the College of Humanities and Social Sciences, the College of Management, and the School of Design were less positive about their competitive preparation.

One final item on general education preparation was included in the NC State Alumni Survey. This item, a literacy measure, was also designed to measure progress towards the satisfaction of general education requirements as put forward by the Council on Undergraduate Education. Results appear below in table 3.

To permit a visual examination of the results for this item, chart 3 appears below.

Chart 3 indicates mixed results for this measure of general education. While 62.2 percent of responding alumni reported reading for leisure at least once a week, and 79.5 percent of responding alumni estimate that they read for leisure at least once a month, 12.4 percent of respondents indicated that they read for leisure seldom or not at all.

Summary and Conclusions

Findings from this study are mixed, and raise several implications for undergraduate education at NC State. On the positive side, responding alumni appear to be essentially satisfied with the broad general education they acquire at NC State, and gave high ratings to their competitive preparation, ability to access information, creative thinking processes, current events literacy, their view of learning as a lifelong process, and their ability to critically analyze events, information, and ideas. On these measures, it is clear that responding alumni regard their preparation at NC State as successful.

On several other measures, alumni ratings raise more questions than provide answers. Especially noteworthy among these are measures of public and community affairs involvement, preparing for a career, understanding one's abilities and interests, and frequency of leisure reading. Alumni also offered substantially lower levels of agreement regarding the usefulness of foreign language and social science courses. The findings speak to a desire for better career preparation and more knowledge of self, yet betray a certain narrowness of focus among responding alumni, as if preparation for both local and global citizenship were not a vital function of the university.

Return to Table of Contents
Return to OIRP Survey Page
Return to OIRP Home Page