• Executive summary
  

• Introduction
  
  • Background
    

  • Sample
    

  • Quality assurance
    

  • Rating scales used
    

  • Interpretation of rating scales
    

• Survey findings
  
  • Communication skills
    

  • Technical skills
    

  • Computer skills
    

  • Workplace skills
    

  • Professional traits, skills, and attitudes
    
  
  
  

Background

The NC State Alumni Survey of baccalaureate alumni who graduated from December 1990 through August 1993 sought among other things to identify the extent to which bachelor's alumni were satisfied that they had received quality training in professional skills and abilities at NC State. This report, the third in a series of reports on data gleaned from the survey, focuses on how all responding alumni (n = 3,179) rated items concerned with the importance of various skills and abilities, the extent to which alumni believed that their undergraduate education at NC State had provided them with those skills and abilities, and the differences between those two ratings. The differences between preparation and importance ratings are called gap scores; they are used to measure the extent of any disjuncture between perceptions of alumni preparation and workplace requirements. Mathematically, this may be expressed by the formula:

Communication Skills

Preparation in all communication areas was judged to be between average and good by respondents, a positive finding. Fairly wide negative gap scores are evident on communication skills overall and on listening skills. This finding may be interpreted as an indication that alumni judge the level of importance of overall communication skills and listening skills in the workplace they experience to be greater than the level of preparation they received at NC State in these areas.

Technical Skills

Judging from the minimal gap scores, the importance of technical skills and the degree to which responding alumni perceived they had been prepared at NC State were quite similar. This can be interpreted to mean that, overall, NC State graduates perceived they are adequately prepared in technical skills relative to the importance of these skills in the workplace. When disaggregated to the college level, substantial variation by college became apparent on overall technical knowledge. The gap score on this item for the School of Design was negative and fairly large.

Computer Skills

Overall, respondents indicated they had received average preparation in computer skills at NC State. When disaggregated by college, substantial variation is evident. Concerning overall knowledge of computer applications and basic computer skills, the gap scores were negative and fairly large for three colleges (Management, Agriculture and Life Sciences, and Humanities and Social Sciences. For the School of Design, the gap score was negative and quite large on both items. These findings indicate an uneven level of preparation among NC State alumni by college. The sizable gap scores found on these two items for some schools or colleges on overall knowledge of computer applications and basic computer skills reinforces the notion that student access to and understanding of computers and computing technology will continue to require substantial attention if NC State is to retain its competitive edge in attracting, retaining, and graduating highly qualified students.

Workplace Skills

All items in this area were rated between important and very important by responding alumni; preparation was judged as average to good. For leadership and management skills, a fairly large negative gap score was evident. This may be taken as an indication that alumni judge the importance of leadership and management skills in the workplace to be substantially greater than the level of preparation they received at NC State.

Professional Traits, Skills, and Attitudes

Items in this area were rated as important by responding alumni, although ability to work with persons from diverse cultural and ethnic backgrounds was rated the lowest in this group. Preparation in this area was rated between average and good. Judging from the magnitude and negative direction of the gap scores, it is evident that alumni perceive the level of importance of three areas in particular to be greater than the level of preparation they received at NC State: professionalism, confidence in your ability to perform well and ability to grow on the job. This last item, in particular, is cause for concern since it is a component skill of lifelong learning.

Return to Table of Contents
Return to OIRP Survey Page

Introduction

Background

The NC State Alumni Survey of baccalaureate alumni who graduated from December 1990 through August 1993 sought among other things to identify the extent to which bachelor's alumni were satisfied that they had received quality training in professional skills and abilities at NC State. This report, the third in a series of reports on data gleaned from the survey, focuses on how alumni responded to questions concerned with the importance of various skills and abilities, the extent to which alumni believed that their undergraduate education at NC State had provided them with those skills and abilities, and the differences between those two ratings.

Sample

Data obtained from this survey represent the largest and most comprehensive effort in the history of NC State to gain feedback from alumni. The survey sample included 7,491 alumni. Three full mailings and a telephone follow-up resulted in a 51.2% response rate, yielding 3,179 usable questionnaires.

Quality assurance

Data quality assurance steps taken yielded the following findings:

• Precision: The overall margin of error for items included this report was extremely narrow at 1.9 percent (at a 95 percent confidence level).
• Reliability: Individual item reliabilities were high (Cronbach's Alpha = 0.96).
• A telephone follow-up survey demonstrated that non-respondents did not think less of their undergraduate preparation than did earlier respondents; indeed, telephone respondents were significantly more positive (p = .05 or better) in their assessment of the education gained at NC State than were earlier respondents on nine items regarding quality of preparation and not significantly different from earlier respondents on other items dealing with level of preparation.
• Significant differences were found between the population and obtained sample in proportion of gender, race, and college, but no differences were found by year of graduation. Though these differences were significant (p = .001), they were not of sufficient magnitude (i.e., 5% discrepancy or greater) to suggest that the obtained sample was appreciably different from the population of bachelor's graduates from 1990-1993.
• This report is therefore based on the conclusion that results obtained are both accurate and broadly representative of the population of NC State bachelor's graduates for those years. Further methodological information on this survey is available from University Planning and Analysis.

Rating scales used

All items were rated by respondents on five-point scales. For items assessing the importance of various areas, the scales ranged from 5 = very important to 1 = not important. For items assessing the level of preparation in each area, the scales ranged from 5 = excellent preparation to 1 = poor preparation. Responses in the "not applicable" category were omitted in calculation of the mean score for both importance and preparation items. In order to facilitate interpretation of the results of this survey, similar items are grouped together in the discussion below.

Interpretation of rating scales

In this report, three distinct pieces of information are presented on each survey item concerned with professional preparation. First, mean ratings of the relative importance of each area to graduates' jobs are presented. Next, mean ratings of the quality of preparation received in each area are shown. For items disaggregated to the college/school level, tests of significance were performed on these means; significant departures from the university-level means are noted. Finally, the means of observed differences between levels of importance and levels of preparation are presented. These means of observed differences are labeled "gap scores" and are included to provide an indicator of the size and direction of areas for improvement (indicated by a negative gap score) or areas in which level of preparation exceeds perceived workplace requirements (indicated by a positive score). T-tests were conducted to determine whether or not these gap scores were significantly different from zero; significant gap scores are noted in the tables. When viewing the results of this survey, it is important to remember that all three elements should be considered - importance, preparation, and gap scores - before arriving at a judgment as to the correct interpretation of the findings.

As a guide to interpreting the relevance of the gap score, in any instances where the gap score is larger than -1.0, the item most probably merits further investigation. In other cases, it may be useful to examine the standard deviations for both importance and preparation. In situations where the gap score is negative and larger than the standard deviation of either the importance or preparation rating, further investigation may be warranted. In situations where the gap score for a particular item is negative and larger than the standard deviation for both importance and preparation ratings, a real difference between importance and preparation is implied and further investigation is indeed warranted. Note: due to rounding and to the differing number of respondents to items on importance and preparation, mean gap scores may not exactly total the difference between the mean importance and mean preparation ratings.

Survey findings

Communication skills

Communication skills were judged by survey respondents to be of high importance to their current positions. The one exception was foreign language skills, which was judged to be of much lower importance. Among specific dimensions of the communication process, listening skills received a notably high importance rating. Preparation in all communication areas was judged to be between average and good by respondents. Preparation in reading skills received the highest rating (3.65), while preparation in foreign language skills (2.98) was rated lower relative to other communication dimensions. These results are given in table 1.

To permit a visual comparison of these findings, ratings of importance and preparation are displayed in chart 1.

The large positive gap score for foreign language skills is a result of low ratings on importance and much higher ratings on preparation. Fairly wide negative gaps are evident on communication skills overall and on listening skills. This finding may be interpreted as an indication that alumni judge the level of importance of overall communication skills and listening skills in the workplace they experience to be greater than the level of preparation they received at NC State in these areas. Listening skills may be thought of as the ability to receive and understand information transmitted orally, whether from clients, co-workers, or supervisors.

Technical Skills

To a land-grant research university, the importance of undergraduates acquiring a thorough grounding in technical skills cannot be overemphasized. Three items designed to capture alumni estimates of importance and preparation in these skills were included on the survey instrument: overall technical knowledge, ability to apply scientific principles, and ability to apply mathematical skills. Of these, overall technical knowledge received the highest rating in terms of importance (4.04) and ability to apply math skills received the highest rating in terms of preparation (3.83). These results are displayed in table 2.

To permit a visual comparison of these findings, ratings of importance and preparation in technical skills are displayed in chart 2.

Judging from the minimal gap scores, the importance of technical skills and the degree to which responding alumni perceived they had been prepared at NC State were quite similar. However, the large standard deviations on these items should also be taken into account when interpreting the ratings. Not only is there a wide variance in how important survey respondents perceived technical skills to be, but there is a similarly wide variance in the perception among respondents as to how well they were prepared at NC State. To facilitate an understanding of these ratings, a further analysis by college was performed on all three items; this is set forth in tables 2a - 2c.

Results displayed in table 2a support the notion that substantial variation exists by college with regard to importance and preparation in overall technical knowledge. Negative gap scores are largest on this question for the School of Design and smallest for the Colleges of Textiles and Engineering. A large negative gap score on this item indicates that responding alumni reported a discrepancy between the overall technical preparation they received at NC State and the importance of technical knowledge overall in the workplace they experience.

Table 2b clearly shows that respondents' judgments on the importance and preparation in ability to apply scientific principles varies somewhat by college. On this item, most colleges display positive gap scores; no significant negative gap scores were found. This indicates that respondents felt rather better prepared in their ability to apply scientific principles than was demanded of them in the workplace.

Alumni survey results displayed in table 2c confirm that little disparity exists by college on respondents' ratings of the importance and preparation in ability to apply mathematics skills. Gap scores were either positive or not significantly different from zero on this item by college. Again, this indicates that respondents felt somewhat better prepared in their ability to apply mathematics skills than was demanded of them in the workplace.

Computer Skills

Three items on the alumni survey were concerned with computer skills: overall knowledge of computer applications, basic computer skills (word processing, etc.), and technical computer skills (programming, etc.). Basic computer skills were rated the most important of these three items, and respondents judged their preparation about equal in overall knowledge of computer applications and in basic computer skills. These findings are given in table 3.

To permit a visual comparison of these findings, ratings of importance and preparation in computer skills are displayed in chart 3.

To facilitate an understanding of these ratings, a further analysis by college was performed on all three of these items; this is set forth in tables 3a - 3c.

Results displayed in table 3a clearly demonstrate a differential level of preparation by college. For three colleges, Management, Agriculture and Life Sciences, and Humanities and Social Sciences, the gap scores exceed -1.00 and demonstrate an area of concern. For the School of Design, the gap score of -2.20 is especially noteworthy and is cause for in-depth investigation.

Table 3b clearly shows that, as in table 3a, substantial variation exists by college with regard to basic computer skills (such as word processing, spreadsheets, etc.). Gap scores exceed -1.00 for four schools and colleges: Design, Management, Agriculture and Life Sciences, and Humanities and Social Sciences. Again, this finding is cause for concern about the level of preparedness of NC State graduates for dealing with technology in the workplace.

Table 3c demonstrates little variation among colleges with regard to importance and preparation in technical computer skills, and little discrepancy for most colleges and schools between importance and preparation ratings. One important exception is the School of Design, where a substantial gap score exists for this item. Given the highly technical graphics and design packages in current workplace use within Design disciplines, this is a special cause for concern.

A recent authoritative national poll of students conducted by the Art and Science Group, Inc. (1997) found that "a majority of the [new student] market rates the quality of a college's computer facilities as very important in college choice - equal in importance to a college's science and research laboratories" (p. 10), and that "access to e-mail, the Internet and the software required for academic work in college, among other computer resources, is very important to a high percentage of students" (p.15). The sizable gap scores found for some schools or colleges on overall computer skills and basic computer skills reinforces the notion that student access to and understanding of computers and computing technology will continue to require substantial attention if NC State is to retain its competitive edge in attracting, retaining, and graduating highly qualified students.

Workplace skills

Exploring those skills and abilities that connect the academy to the world of work was a major theme of the NC State Alumni Survey, and a number of survey items were designed with that in mind. Among these dimensions were the ability to work in teams, leadership and management skills, and various ways in which alumni might be expected to apply their learned skills in project planning and problem solving activities. All items in this area were rated between important and very important by survey respondents; items concerned with the importance of problem solving abilities were rated especially high. Preparation in these areas was judged overall between average and good. Preparation in areas concerned with solving problems achieved the highest ratings, whereas preparation in leadership and management skills was rated the lowest of the skill sets included in these items. These results are presented in Table 4.

To permit a visual comparison of these findings, ratings of importance and preparation in workplace skills are displayed in chart 4.

Most items in this group display gap scores that, while significant, are smaller than the standard deviations of importance and preparation ratings and as such are not particular areas for concern. However, a large negative gap score is evident for leadership and management skills. This may be interpreted as an indication that alumni judge the importance of leadership and management skills in the workplace to be substantially greater than the level of preparation they received at NC State in this area, and argues for more specific exposure to leadership and management concepts and practices in undergraduate curricula.

Professional traits, skills and attitudes

Continuing the theme of exploring commonalities between NC State and the world of work, another broad group of sixteen items was designed to elicit information on professional traits, work attitudes and skills, and professional development. All importance items were rated between important and very important by respondents, although ability to work with persons from diverse ethnic and cultural backgrounds received the lowest rating (4.02) among this group. Items concerned with the quality of preparation at NC State were rated between average and good with the exception of the ability to work independently, which was rated as good (4.06).

To permit a visual comparison of these findings, ratings of importance and preparation in professional skills, traits and attitudes are displayed in chart 5.

Judging from the magnitude of the gap score, it is evident that alumni perceive the level of importance of three areas in particular to be greater than the level of preparation they received at NC State: professionalism (gap score = -1.00), ability to grow on the job (gap score = -0.96), and confidence in your ability to perform well. Professionalism may be thought of as the ability to meet or exceed professional standards of conduct and practice in the workplace, and confidence in your ability to perform well serves as a measure of the assurance an alum perceives relative to the quality of their overall preparation. Ability to grow on the job is one of the component abilities of being a lifelong learner, and as such is an important outcome indicator for NC State.


Return to Table of Contents
Return to OIRP Survey Page
Return to OIRP Home Page