THE IMPACT OF SURVEY RESEARCH-BASED ASSESSMENT AT NORTH CAROLINA STATE UNIVERSITY

Table of Contents

• Abstract
• Executive Summary
• Project Background, Purpose, and Methodology
• Findings
• Summary and Conclusions
• Recommendations
• References

Institutional Strategy and Analysis, June 1998
J. Joseph Hoey, Assistant Director, Evaluation and Policy Studies
Denise Gardner, Coordinator of Survey Research

Abstract

In January, 1998, Institutional Strategy and Analysis surveyed administrators (N = 80) at NC State to ascertain the impact of providing student, alumni, and employer survey information. Forty-four responses were gained, a 55% response rate. Questions were concerned with the extent to which units had used survey information to assess their performance, to improve their environmental scanning efforts, and to take specific actions to realign, change, or improve their performance. Results revealed extensive impact on academic and administrative units. The greatest impact is apparent at the department level, where the use of survey information for planning and the extent of curricular change and innovation are particularly noteworthy. In administrative units, survey data is being used to focus on improving customer service, either the way it is measured or to improve service levels and customer satisfaction. Recommendations include the continuation of all surveys using the current large-sample methodology to enable trend comparisons for all subunits. To assure reliable and valid information at the department level, student response to surveys should be established as a requirement.

Executive Summary

Project Purpose and Background

To ascertain the impact on colleges, schools, departments and units of providing student, alumni and employer survey information, an electronic survey of administrators was undertaken in January 1998 by Institutional Strategy and Analysis. The instrument was sent out to a population of 80 associate deans, directors, and department heads at NC State. Forty-four usable responses were gained (7 associate deans, 27 department heads, and 8 administrative unit directors), for a response rate of 55%.

The instrument used to measure the impact of assessment consisted of questions following a nationally-established set of guidelines for the improvement of organizational effectiveness: the Malcolm Baldridge National Quality Award 1997 Criteria for Performance Excellence (as cited in North Carolina Quality Leadership Foundation, 1998). Respondents were asked to rate each component of NC State's survey research program (Entering Student, Graduating Senior, Alumni/Employer) on each area. Questions were primarily concerned with:

• the degree to which survey information had enabled colleges/schools, departments or units to assess their performance;
• what actions colleges/schools, departments, and units had taken to realign, change, or improve their performance on the basis of survey information; and
• the extent to which survey information had enabled colleges/schools, departments and units to use survey information for environmental scanning to position themselves with regard to external expectations.

Conclusions

• Survey research-based assessment has had substantial impact on NC State.
• In colleges/schools and academic departments, much of this impact focuses on curricula -- including specific changes to curricula in response to survey results, as well as an improved ability to plan for curricular needs and changes.
• In administrative units, survey data is being used to focus on improving customer service, either the way it is measured or to improve service levels and customer satisfaction.
• Over 50 percent of respondents use data from one or more UPA surveys to support strategic planning, determine key program features and their importance to stakeholders, open up departmental/unit discussions on program or service performance, and improve performance measurement capabilities.
• Over 55 percent of respondents use data from one or more UPA surveys to set direction and higher performance targets for their department or unit; 45 percent use survey data as a basis for following up with constituents on programs, activities and services.
• Over 50 percent of respondents use data from one or more UPA surveys to update, keep current, or improve their ability to listen and learn from stakeholders and markets as well as to take their needs and expectations into account.
• The deepest levels of assessment impact are apparent at the department or unit level. Department heads and unit directors expressed strong preference for data that can be applied to decision making at their level. Respondents directed UPA to continue providing data disaggregated to the department/unit level.
• As a corollary, departments and units want more customized instruments and items that would allow them to measure specific outcomes. Respondents directed UPA to continue to work with departments and units on unit-specific inserts to be included with the university-level survey instruments.
• Reliable long-term trend data from surveys and benchmarking information from comparable student surveys at other institutions emerged as strong decision support needs of respondents.
• Respondents requested simplifying reports and putting statistical data and tables out on the web as two ways of improving access to survey research information.

UPA Response So Far

• For the 1997 entering student and graduating senior surveys, we changed the reporting process and disseminated printed copies of shorter summary reports, with more detailed reports (along with tables of responses) available on the web.
• Three-year trend analyses of the entering student surveys and graduating senior survey will be available in July 1998.
• Planning for the next cycle of alumni/employer surveys is well underway, and academic personnel have been informed on four occasions so far that their input is needed to ensure that departmental questions on alumni and graduating senior surveys meet their needs for assessment information.

Recommendations

• The survey program should be continued. Specifically, graduating senior surveys should be conducted at least annually if not each semester. Alumni surveys and employer surveys should be continued in order for comparative data across time (trend analysis) to be made available at the department and unit level. Entering student surveys should continue to be administered at orientation.
• UPA should continue to involve associate deans, department heads, undergraduate coordinators, and unit directors in revisions to university-level survey instruments and departmental/unit inserts that accompany those university-level instruments.
• Require student response to entering student, sophomore, and graduating senior surveys as a condition of matriculating, continuing or graduating at NC State. This will enable the institution to meet GA-mandated response rates to their survey questions, and will assure that information gained is reflective of the views of all students in a particular program and can be used with confidence for decision making at the department/unit level.
• Continue using large sample sizes on alumni surveys in order to have the ability to disaggregate findings to the department or unit level.
• Continue to provide more timely data, disseminate summary printed reports only, and place more detailed information on the web.
• Work with peer institutions to compare alumni and graduating senior survey information, and make this information available (where possible) on a disaggregated basis to colleges/schools, departments and units at NC State.

Project Background, Purpose, and Methodology

Since 1993, a survey research-based program of student assessment has been underway in Institutional Strategy and Analysis at NC State University. Beginning with the revision of the entering first-year student survey in 1993, the program has been expanded to encompass a graduating senior survey (since 1995), an entering transfer student survey (since 1995), alumni and employer surveys (first administered in 1995-96), and a sophomore survey (since 1997). The graduating senior survey and sophomore survey were put in place as a direct result of a mandate from UNC-General Administration (GA); the first-year student survey also contains several items mandated by GA every other year. All surveys feature items specifically designed for NC State as well as any GA-mandated items.

Results of these surveys have been disseminated to the campus community in a number of ways. A series of reports on university-wide results have been sent to administrators, deans, associate deans, department heads, and unit heads. These reports have also been published on the Survey Research web page in UPA. College and departmental analyses, including analyses of any unit insert questionnaires, have been sent to the appropriate academic personnel on a recurring basis. Presentations of findings have been made to the Administrative Council, the Council on Undergraduate Education, groups of administrative unit heads reporting to vice chancellors for student services and university advancement, and other selected groups.

The bottom line in assessment research is certainly whether it is has a measurable and positive effect upon educational and administrative processes. To ascertain the impact on colleges, departments and units of providing student, alumni and employer survey information, an electronic survey of administrators was undertaken in January 1998. The instrument was sent out to a population of 80 associate deans, directors, and department heads at NC State. Forty-four usable responses were gained (7 associate deans, 27 department heads, and 8 administrative unit directors), for a response rate of 55%. While this response rate precludes generalization of findings to the population of interest, the data nevertheless offer substantial insights into the impact of a survey-based assessment program at NC State, and demonstrate the wide extent to which the survey research program is having a positive effect on the university's ability to become a more self-regarding institution (Ewell, 1984).

The instrument used to measure the impact of assessment consisted of questions following a nationally-established set of guidelines for the improvement of organizational effectiveness: the Malcolm Baldridge National Quality Award 1997 Criteria for Performance Excellence (as cited in North Carolina Quality Leadership Foundation, 1998). Respondents were asked to rate each component of NC State's survey research program (Entering Student, Graduating Senior, Alumni/Employer) on each area. Questions were primarily concerned with:

• the degree to which survey information had enabled colleges/schools, departments or units to assess their performance;
• what actions colleges/schools, departments, and units had taken to realign, change, or improve their performance on the basis of survey information; and
• the extent to which survey information had enabled colleges/schools, departments and units to use survey information for environmental scanning to position themselves with regard to external expectations.

Several open-ended items were also included to allow respondents to describe more fully how the information provided had affected their units (including providing examples of specific actions in which survey information was used), which reports were most useful, how the reporting process could be improved, and any other overall comments respondents cared to make concerning survey information.

Findings

Usefulness of Surveys for Planning and Assessing Unit Performance

In a series of seven items, survey respondents were asked to rate the extent to which survey information had enabled them to effectively assess the performance of their department or unit. Results are detailed below in Table 1 in terms of what percentage of respondents indicated that assessment surveys had been moderately or very useful to their college, department, or unit for each purpose.
Table 1. Usefulness of Surveys for Planning and Assessment.

	Percent reporting surveys were moderately or very useful for this purpose
Survey Item	Entering Student	Graduating Senior	Alumni/ Employer
Develop or revise strategic plans, including such factors as your target markets and stakeholders; the learning environment; opportunities and threats; the capabilities of your department/unit; and supplier and/or partner capabilities	44.4%	57.9%	48.6%
Integrate the information you gained from survey findings and reports into measurements that can be used to track and improve the performance of your department/unit	36.4%	55.3%	35.5%
Led to more open discussions and sharing of information between members of your department/unit	31.4%	53.8%	39.4%
Assess your current performance relative to your past performance	29.4%	47.3%	25.8%
Enabled you to determine key program, activity and service features and their relative importance/value to employers, alumni, students, the general public, and other stakeholders	27.7%	55.0%	48.6%
Track your department/unit performance relative to your strategic plan	27.3%	50.0%	32.3%
Compare your department/unit's performance with that of similar department/units at this or other institutions	23.5%	44.4%	29.1%

These findings are illustrated in Charts 1a through 1c.

• For entering student surveys, revising strategic plans and integrating information gained into performance measurements were the most frequently cited uses.

• Graduating senior surveys were rated high across the board for usefulness in planning and assessing unit performance. This was especially noteworthy concerning the revision of strategic plans, integrating information gained into performance measurements, determining key program and service features, and opening up department/unit discussions on performance.

• Determining key program, activity and service features and developing or revising strategic plans were the two areas in which alumni and employer survey results were found to be most useful, followed by opening up department/unit discussions on performance.

Open-ended comments

One respondent noted that "My general response is that all the surveys are giving us empirical foundations on which to develop our plans and goals." More specifically, several respondents noted the need for survey trend and comparison data to assist their assessment efforts:

• "Trend information would greatly help, also peer comparisons."
• "Need information from peer institutions to make comparisons."
• "A longer series of surveys is required to confidently track and change."
• "Data from one or two years of surveys has somewhat limited impact in small departments with relatively low numbers of graduates. This information will be much more valuable as we are able to develop and assess trends."
• "[The] alumni survey will be more useful when it has been repeated several times and when departments are able to customize the instrument."

Other respondents proffered insights concerning the level at which the greatest impact of survey data had been or would be felt:

• "Surveys provide useful data in preparing reports for our [departmental] accrediting agency, the Council on Social Work Education."
• "Departmental reports are most useful."
• "Department had full discussion of alumni results."
• "In order for the reports to directly affect our behavior and plans it has to be taken down to the departmental level…I also indicated that the reports have not been very helpful in tracking year to year changes. That is simply because we have not seen enough yearly reports yet to make those comparisons."

Taking Action to Change, Realign and Improve Department/Unit Performance

The next series of items requested information on whether or not survey reports had been used as a basis for changing, realigning or improving departmental or unit performance. Results are shown in Table 2 in terms of what percentage of colleges/schools, departments or units had taken specific actions on the basis of survey results.

Table 2. Actions Taken on the Basis of Survey Results.

	Percent reporting action was taken in this area on the basis of survey results
Survey Item	Entering Student	Graduating Senior	Alumni/ Employer
Update and improve strategic and action plans for the next planning cycle	36%	55%	33%
Set direction for your department/unit and seek future opportunities; set higher targets for your department/unit's performance	33%	57%	24%
Improve your performance measurement capabilities in the future	26%	33%	21%
Reorganize the work of your department/unit to enable you to meet current and changing stakeholder and operational requirements	24%	36%	24%
Follow-up with students, alumni and employers on programs, activities and services as a result of what you learned from the surveys	21%	45%	31%

These levels of impact are graphically illustrated in Charts 2a through 2c.

• For entering student surveys, the most frequent action taken was to update and improve strategic and action plans on the basis of findings. One third of respondents reported using entering student survey data to set department direction and higher performance targets.

• Graduating senior surveys were much more frequently used as a basis for action across the board than other components of the survey program, notably so for setting department direction and performance targets, updating and improving strategic and action plans, and following up with constituents on the basis of findings. It is worth noting that graduating senior surveys have been administered at NC State since 1995, and meaningful trend analysis is possible at the department level.

• For alumni and employer surveys, the most frequently reported actions taken on the basis of survey findings were to update strategic and action plans for the next cycle, as well as to follow up with stakeholders on programs, activities and services.

Open-ended comments

In an open-ended follow-up to these items, respondents were asked to describe examples of specific activities, projects, or other actions in which they had used survey information to realign, change, or improve their department or unit. The numerous responses received fell broadly into four thematic categories: (1) curriculum revision, (2) student advising, and (3) improving planning processes, including planning for reaccreditation, and (4) customer service improvement.

Curriculum revision: numerous comments alluded to various curricular initiatives and changes that had been put into place as a result of survey information gained. A selection of these comments included:

• "A capstone course was developed during the first year of senior and alumni surveys - findings from those surveys have helped us make changes in the capstone course in response to identified needs."
• "Enhanced writing/communication components of course work, recognized need to increase business and economics skill levels among undergraduates."
• "We are also reconsidering the undergraduate and graduate curricula in light of what employers indicate they want of graduates."
• "Incorporation of more problem solving into courses. Addition of courses in professionalism and ethics. Addition of course exercises that require team work."
• "More emphasis on evaluating writing in our undergraduate curricula…We have become aware/concerned with the need to expose our students to the world of fine arts."
• "Development of new concentrations identified by students as important areas for development of competencies in technical agriculture. Faculty appointments have been adjusted to provide more opportunities for extension faculty to teach and interact with students in the extension option."
• "The graduating senior survey has been used successfully with an exit interview conducted by the department chair to improve teaching/learning, communication skills, and computer skills. We have an excellent association with industry and they have also provided us with 'feedback' on areas of improvement for our graduates."

Advising: Several comments detailed how departments had made revisions to their advising processes as a result of alumni and employer feedback gained. Comments included:

• "An associate department head for academic affairs has been appointed to oversee and coordinate all academic and advising matters at the undergraduate and graduate levels."
• "Our emphasis on advising minority students has been strengthened."
• "Graduating senior surveys and alumni surveys have been useful in identifying problems related to advising, communication, and curriculum."
• "Appointed new undergraduate coordinator and agreed on goal of improving advising and building a stronger feeling of ownership among faculty for the undergraduate program."
• "In the graduating senior report we noticed that we were not doing quite as good a job at advising as we thought. That has led us to consider implementing a direct evaluation of advising similar to teaching. We are currently piloting such a program."

Planning: A number of comments were received that related to how departmental or unit planning processes had been positively affected by the information supplied in alumni and employer survey reports. Some of these included:

• "Entering student surveys have helped us plan for recruitment of new undergraduates (niche selection)."
• "The library presented the results of the graduating seniors survey to its staff and to the University Library Committee as one component of data­gathering in the planning process. They helped to confirm the Libraries' expansion to a 24­hour facility and the introduction of the TRIPSaver service."
• "Departmental strategic plan included improving student support services in light of alumni survey findings."
• "Employer surveys have contributed to our planning for, e.g. implementation, of the writing and speaking general education requirements."
• "Preparing faculty for the realities of ABET 2000 [engineering accreditation criteria], which is outcomes based."
• "In the process of working…to align information collected in these surveys with information needed to meet ABET Criteria 2000."
• "Our strategic planning process will now center on how to focus the Alumni Association on those things that alums have said they want (such as continued career planning assistance and professional development opportunities). We also funded a .5 FTE position to work directly with alumni in providing those resources they said they want. For example, we offered a pilot regional career fair for young alumni in Charlotte where we pulled together 167 alums and 23 corporations. We had great feedback, so we will do one in the Triad next year."

Customer service improvement: Several comments received related to how service and process improvements had been undertaken as a result of survey data, including:

• "The entering student survey supplies vital demographic and income data which is used to ensure the correct product mix for all individuals attending N C State is supplied. The graduating survey has prompted the bookstore to begin a series of internal, random surveys of the students in critical areas such as our buyback location which furthers our understanding on staff responsiveness, and products and services as viewed by our customer. Additionally, a new delivery mechanism, point­of­sale registers, has been purchased and there has been a significant increase in "throughput" of our customer base. In the long run this should help the customers perception, in one way, of staff responsiveness."
• "Extended Hours of UCO [University Cashiers Office], Implemented Feedback system."
• "We developed in-house 'rate our service' cards to collect continuous feedback from customers; updated written materials to get better information to students to clarify the aid/applications award process; designed and implemented a survey of all aid applicants to gauge satisfaction with services (asking the specific questions the senior/sophomore surveys do not); and created a Financial Aid Advisory Group composed of students who have volunteered to participate in assisting the office to better our services."
• "Survey information has been instrumental in staff meeting discussions to keep staff focused on key issues of importance regarding integrating academics and residential living. Some staff have used the information in training new residence hall staff members. Others see it as a means of supporting programs and activities planned by students and staff."
• "More extensive collaboration with Career Counseling - the career counselor for CHASS attends all meetings with coordinators of advising."
• "To promote Ag Foundation financial support for 1) Spend-A-Day recruitment program, 2) faculty teaching workshops, 3) HS science teacher workshops, 4) Jefferson Scholars program support, 5) better alumni relations with CALS."

Environmental Scanning

The next section of the survey contained four items concerned with the extent to which survey information increased the ability of colleges/schools, departments and units to use survey information for environmental scanning to position themselves with regard to external expectations. Responses are given in Table 3 in terms of what percentage of respondents indicated that assessment surveys had been moderately or very useful to their college, department, or unit for each purpose.

Table 3. Usefulness of Survey Information for Environmental Scanning.

	Percent reporting surveys were moderately or very useful for this purpose
Survey Item	Entering Student	Graduating Senior	Alumni/ Employer
Update, keep current, or improve your ability to listen to and learn from your stakeholders and markets	35.3	55.2	41.7%
Helped you take into account the needs and expectations of key stakeholders as you provide leadership for your department/unit	30.3%	51.3%	45.7%
Enabled you to anticipate public concerns	23.5%	29.0%	29.4%
Address current and potential impacts on our community, state, and society of your programs, activities, services, facilities and operations	18.8%	27.0%	32.3%

These various levels of impact of survey information are graphically illustrated in Charts 3a through 3c.

• The most frequently cited usage in this area for entering student survey data was in keeping up with market trends, followed by using survey data to take account of stakeholder expectations.

• Concerning graduating senior surveys, the most notable finding among these items is the extent to which departments and units found survey information useful for obtaining direct feedback from their customers and stakeholders.

• Similar to graduating senior surveys, with regard to alumni and employer surveys the most notable finding among these items is the extent to which departments and units found survey information useful for obtaining direct feedback from their customers and stakeholders.

Open-ended comments

Few follow-up comments were provided on this section, although one respondent did note that college-level questions on an insert to the alumni survey provided by far the most important information and were used by the college in a comprehensive review. Another respondent noted that their department was developing a core course focused on leadership and group dynamics as a result of employers' expressed needs for skills in leading public groups in decision making.

Which Survey Reports Have Been Most Useful?

Several respondents noted that all the surveys were of some utility to them in improving the operations of their unit. Representative comments included:

• "I wish I could answer this. To me there is one survey with different parts. If I had to, I would place the senior survey above the others­ It lets my 'product' talk to me."
• "All have been useful. They have covered the extensive range of activities here at NC State and provide an invaluable statistical overview."
• "I find the entering student and graduating senior data the most helpful, although the alumni/employer survey is interesting as well. The information from the entering and senior surveys allow us to focus on these target populations in the [residence] halls. Granted, there are many more first year students than seniors, so efforts are most likely to be in this area. However, what we learn from the senior and alumni/employer surveys is helpful when thinking of programming over the course of a student's life in the residence halls."

Numerous respondents voiced the opinion that the graduating senior surveys had been the most useful. Reasons given varied, but the common theme underlying many of the comments was that timely corrections to programs and services could be made by using data from students who are just graduating. Examples include:

• "Graduating Senior Surveys, including the department questions, because it provides feedback from individuals who have most recently participated in the programs of the Department."
• "The graduating senior report has so far been most useful because it most specifically indicates where problems might lie."
• "Senior survey. For curriculum renewal and ABET accreditation (VERY IMPORTANT)."
• "Departmental surveys of graduating seniors. This information has provided the department and committees involved with undergraduate and graduate education a valuable tool to monitor the opinions of graduates."
• "Entering and exiting student surveys. Provides us with some feedback for improving our programs."
• "Seniors survey, because it contains a few questions relating to the Libraries."
• "Graduating Senior/Sophomore surveys ­­ at least compared to prior year's results we are able to gauge whether any improvements have been noted by our customers."

Alumni and employer reports were also cited as highly useful by a number of respondents. The chief reason for this had to do with the believable, experienced and in-depth perspective that alumni and their direct supervisors are able to offer. Representative comments in this vein included:

• "Alumni/employer: gives additional feedback from the individuals/groups in the best position to judge the effectiveness/quality of our programs."
• "Senior and alumni surveys tell us about success/problems from a population that probably is honest in reporting and that has perceived there to be little to lose by being honest."
• "Alumni: they are our best evaluators after they've had time to reflect on the value of their education to their lives."
• "The alumni report which showed low correlation between employment success and GPA has enabled us (Career Planning and Placement Center) to make presentations to corporations, so that they may modify their hiring criteria in this tight market. With this analysis, we are providing a level of service to some high quality corporate customers of ours that they cannot get anywhere else."
• "In regional hiring workshops for employers, we used the findings on young alumni and employer opinions about skill sets, employee training and development programs that are needed. This gave us a way of generating outreach to employers. A whole section of our workbook uses the findings to discuss how to hire, evaluate their skills, and retain graduates by providing appropriate professional development."

Many comments received spoke to the usefulness of disaggregating survey information to the departmental level. Typical comments included:

• "College/Dept. level reports. Focused reports done for alumni/employer survey."
• "The ones that give departmental info and compare to both college and university."
• "Departmental surveys…This information has provided the department and committees involved with undergraduate and graduate education a valuable tool to monitor the opinions of graduates."
• "More surveying done at the program, not the university, level would be more useful."

The importance of having trend information from multiple administrations of the surveys over time was underscored by an associate dean who commented that "Although the alumni survey has been least useful, it can become the most useful if repeated for more recent alumni. We are interested in post-1995 graduates."

How Could Survey Information be Provided in a More Useful Fashion?

A final open-ended question asked respondents to provide feedback on how survey information could be made more useful for their department or unit. Responses fell into several broad areas:

• Six respondents directed UPA to continue to provide disaggregated data at the department or unit level, and to keep on including respondents' written comments. One respondent summed this up: "The departmental reports and summary of comments relevant to [the]department are most useful."
• Five respondents indicated that more customization of survey items to department- or unit-specific questions would provide the greatest benefit. For example, one respondent advised UPA to "tailor the questions to the needs and concerns in the department and permit the department to ask more questions and direct questions about specific topics of interest in the department." Three other respondents affirmed the need for UPA to continue working closely with departments and units and listening to their data needs. For example, one associate dean advised: "Continue interacting with us to ensure questions meet college and departmental needs."
• Three respondents requested comparative data from other departments at comparable institutions. As one associate dean phrased this, "Obtain similar info from our peer institutions for comparisons. What are the benchmarks?"
• Another theme that emerged concerned reduction of information. Respondents advised UPA to simplify the presentation of information, to continue to provide executive summaries, but to put disaggregated data in tables on the web. As one respondent put this, "I like getting the executive summaries. Putting the rest of the material on the web would work best for me. The full reports are long."

Summary and Conclusions

From project planning to ultimate effect upon the institution, discernible impact is evident as a result of survey research-based assessment at NC State. Colleges, schools and academic departments are sharing survey information and discussing the results among themselves. They are incorporating feedback from students, alumni and employers into revisions of strategic and action plans for their units. They are also using the data provided to understand the needs expressed by their customers and stakeholders and subsequently altering or creating new courses, curriculum emphases, and services that address these needs. Administrative units are using survey data to gauge student opinion and to evaluate the effect of customer service innovations.

While all surveys included in the NC State program of survey research have been used by campus constituents for a variety of purposes, graduating senior and alumni/employer surveys have been cited by departments as having unbiased points of view, and therefore some of the most valuable information for program improvement. Entering student surveys are being used by colleges, schools, and departments for enrollment planning and by administrative units for program or service planning. To maximize the usefulness of these data, departments and units requested that the information continue to be provided in a format where they can compare their results to other departments or units, to their college, to the university as a whole, and to other similar institutions as applicable. Comparison of results across time was also a prevalent request, one which entails collecting survey data on a longitudinal basis for trend analysis.

UPA has already begun to respond to these findings, as follows:

• For the 1997 entering student and graduating senior surveys, we changed the reporting process and disseminated printed copies of shorter summary reports, with more detailed reports (along with tables of responses) available on the web.
• Three-year trend analyses of the entering student surveys and graduating senior survey will be available in August 1998.
• Planning for the next cycle of alumni/employer surveys is well underway, and academic personnel have been informed on four occasions so far that their input is needed to ensure that departmental questions on alumni and graduating senior surveys meet their needs for assessment information.

Recommendations

• The survey program should be continued. Specifically, graduating senior surveys should be conducted at least annually if not each semester. Alumni surveys and employer surveys should be continued in order for comparative data across time (trend analysis) to be made available at the department and unit level. Entering student surveys should continue to be administered at orientation.
• UPA should continue to involve associate deans, department heads, undergraduate coordinators, and unit directors in revisions to university-level survey instruments and departmental/unit inserts that accompany those university-level instruments.
• Require student response to entering student, sophomore, and graduating senior surveys as a condition of matriculating, continuing or graduating at NC State. This will enable the institution to meet GA-mandated response rates to their survey questions, and will assure that information gained is reflective of the views of all students in a particular program and can be used with confidence for decision making at the department/unit level.
• Continue using large sample sizes on alumni surveys in order to have the ability to disaggregate findings to the department or unit level.
• Continue to provide more timely data, disseminate summary printed reports only, and place more detailed information on the web.
• Work with peer institutions to compare alumni and graduating senior survey information, and make this information available (where possible) on a disaggregated basis to colleges/schools, departments and units at NC State.

References

Ewell, P.T. (1984). The self-regarding institution: information for excellence. Boulder, CO: National Center for Higher Education Management Systems.

North Carolina Quality Leadership Foundation. (1997). Education performance excellence criteria. North Carolina Quality Leadership Foundation: Raleigh, NC.

Return to top