North Carolina State University
2008 Staff Well-Being Survey
Introduction, Research Methods, and Response Rates
Introduction
In spring 2008 NC State's Office of Institutional Planning and Research office (UPA) administered the Staff Well-Being Survey (SWBS). The primary goal of the survey was to collect information from the staff to better enable the University to work towards meeting its Strategic Plan's Investment Priority "to help attract, develop, and retain staff of the highest quality." The SWBS, sponsored by Provost Larry Nielsen and Vice Chancellor for Finance and Business Charles Leffler, was modeled on a similar survey conducted of NC State faculty in fall 2006. An advisory committee with representatives from various units across campus was responsible for identifying key areas of staff concern to investigate, administering the survey, conducting the data analyses, and (in on-going efforts with input from the entire campus community) interpreting the results, preparing various reports, and making them publicly available.
This document provides a brief overview of the history of the project and the development of the questionnaire, describes the survey population, details how the survey was administered, and gives response rates broken down by various sub-groups of the population. Links to reports providing more detailed information are provided. A copy of the questionnaire with overall results to the survey, an executive summary on the overall results, and tables of results broken down by various sub-groups of the survey population (e.g., division/college, employment profile characteristics, and demographic characteristics) are available online at https://isa.ncsu.edu/srvy/empl/staff.
In the coming months UPA will work with the campus community to identify specific areas of interest from the survey to explore more fully. Reports and/or presentations on these areas of interest (e.g., the performance review process, diversity concerns, campus leadership) will be shared as available.
History and Development1
Although various staff members at NC State might have been asked to participate in surveys over the years to collect information on specific issues of concern (e.g., campus safety), the University had apparently never before conducted a broad-based, campus-wide survey of its staff. UPA's Associate Director for Survey Research, Dr. Nancy Whelchel, organized an advisory committee to identify key areas of interest and develop the questionnaire. In addition to Office of Institutional Planning and Research staff, the advisory committee consisted of top-level representatives from the Staff Senate, Office for Equal Opportunity and Equity, Office for Diversity and Inclusion, Committee on the Status of Women, Human Resources, and the University Director for Assessment, and staff members representing various constituents across campus (e.g., Building Maintenance and Operations, NC State Libraries, CALS Cooperative Extension Services).2 The major goal of the survey project was to collect information that would be useful to the university community in identifying what NC State is doing well and what challenges the university faces in promoting staff well-being. Ideally, the resulting data would better enable the university to work towards making improvements in areas of concern identified by the survey.
The committee built on the questionnaire originally developed and used for the Faculty Well-Being survey project. In addition, the committee explored staff surveys administered at other universities as well as in the private sector. When possible, the NC State questionnaire included questions similar or identical to these surveys in order to allow for comparisons with other universities. Ultimately the questionnaire included 12 topical sections related to staff well-being, and a final section collecting demographic information not available through personnel records. Topical sections were:
- Overall Satisfaction
- Working Relationships
- Communication
- Leadership
- Vision and Direction
- Diversity and Multiculturalism
- Work Activities
- Support and Professional Development
- Performance Evaluation
- Pay and Compensation
- Campus Infrastructure and Physical Environment
- Campus Activities
Altogether there were about 300 forced-choice questions on the survey, most of which employed a 4-point Likert response option scale (e.g., "very satisfied," "satisfied," "dissatisfied," "very dissatisfied"). Response options did not generally include a middle or neutral response option (e.g., "neither satisfied nor dissatisfied") but did include a "don't know" type option when it was judged likely that some respondents would not have sufficient experience on which to base an opinion. The questionnaire also included a limited number of open-ended questions.
During the summer and fall of 2007 and early 2008, Dr. Whelchel gave presentations on the project to the Staff Senate, NC State University Executive Officers, Vice Provosts, and Human Resources' Personnel Connections, and sought their input on the topics covered in the questionnaire. In addition, members of the SWBS advisory committee were encouraged to talk with their constituents about the project and to provide feedback on the questionnaire, in particular to make suggestions for topics and/or specific items to include (or exclude). The questionnaire was also pre-tested with a group of staff an Occupational Activity Code of "professional," a group of "technical" and "clerical" staff, and two groups of "skilled crafts" and "service/maintenance" staff (for a total of four separate pre-tests). Revisions were made to the questionnaire based on feedback from the presentations and the pre-tests. Dr. Whelchel also met with the with the university's Regulatory Compliance Administration to confirm that the project was following appropriate regulations regarding research with human subjects.3
Survey Population
The survey population was designed to be as inclusive as possible. In general, all staff with Staff Senate representation were eligible to participate in the survey. Specifically, the population included all non-instructional EPA and SPA permanent and time-limited staff with an FTE of 0.5 or higher who had been employed at NC State for at least 6 months at the time of the start of the survey (i.e., non-probationary). Librarians and Cooperative Extension staff were included in the population. All NC State employees on the '3D' list (essentially those in administrative positions), post-docs, and students were excluded from the population. No sampling was done - - all staff in the population as defined above were invited to participate in the survey. The final survey population size was 5,841. (More details on the demographic make-up of the survey population are included in discussions of response rates below.)
Survey Administration
The SWBS was available both online and on paper, with the paper version being available in both English and Spanish. All staff were eligible to use up to one hour during their normal working hours to complete the survey.
Online survey: UPA staff did all programming to create the online survey and hosted it on their secure server. Only members of the survey population, who were required to log on to the survey using their university Unity ID and password, could access the survey. A randomly generated unique ID number, and not the Unity ID, was retained on each individual survey record. For the duration of the data collection process UPA maintained a datafile linking the randomly generated IDs to contact information for each member of the survey population in order to facilitate follow-up reminders and the data cleaning process. As per item 47790 in the North Carolina Program Records Retention and Disposition Schedule, this latter datafile was destroyed after the data cleaning process, thereby assuring the confidentiality of all respondents.
Because the questionnaire was long (with about 300 questions in total) there were concerns with both respondent burden and computer "time out" problems. In order to minimize these potential problems and enhance participation rates, each of the 12 sections on the questionnaire were created as separate "pages." At the end of each "page" (section) the respondent was required to submit that series of questions, and elect whether to continue on with the next section (for which the specific topic was given) or exit the survey and return to complete it at a later time. The number of questions remaining on the survey was indicated by letting respondents know when they had completed 25%, 50%, and 75% of the survey. When a respondent who had exited the survey before completing the final section returned to the survey at a later time, he/she was automatically directed to the next section he/she needed to complete. Ultimately, this method proved quite successful, with 85.7 percent of those who started the survey completing all 12 substantive sections, and 91.4 percent completed at least half of the survey (see Table 1). Thus, we have partial data for about 14 percent of the respondents included in the final results.
Table 1: Number and Percentage of Respondents Completing Each Section of the Online Survey| Section | N | Percent |
|---|---|---|
| A: Overall Satisfaction | 2,828 | 100.0% |
| B: Working Relationships | 2,783 | 98.4% |
| C: Communication (through 25% of questions) | 2,743 | 97.0% |
| D. Leadership | 2,689 | 95.1% |
| E: Vision and Direction | 2,632 | 93.1% |
| F: Diversity and Multiculturalism | 2,604 | 92.1% |
| G: Work Activities (through 50% of questions) | 2,586 | 91.4% |
| H: Support and Professional Development | 2,529 | 89.4% |
| I: Performance and Evaluation | 2,500 | 88.4% |
| J: Pay and Compensation | 2,470 | 87.3% |
| K: Campus Infrastructure/Physical Environment (through 75% of questions) | 2,458 | 86.9% |
| L: Campus Activities | 2,423 | 85.7% |
Almost 20% of those participating in the survey took advantage of the option to take the survey in more than one sitting. Among those completing all sections of the online survey in one sitting, the median time for completion was 38 minutes, and average time 41 minutes.
Paper survey: In order to better enable participation in the survey, particularly to those with limited access to the Web, the survey methodology included a paper questionnaire option. The printed scannable form contained identical question wording, and essentially mirrored the formatting of the online survey. All staff classified as skilled crafts, service, or maintenance (N=941) were mailed packets with a paper copy of the survey, a cover letter, and a pre-addressed postage-paid return envelope. All staff in this group with a race/ethnicity of Hispanic (as indicated in personnel data) were sent both an English and Spanish version of the cover letter and survey in their packet (N=107). Paper surveys were also available on request for those in other occupational classifications. Those receiving paper survey packets were also informed that they could complete the online survey if they chose to do so.
Survey Promotion, Announcements, and Invitations4
A great deal of effort was put in to promoting the survey to the NC State staff, and encouraging them to participate in it. In addition to the presentations noted above, articles about the survey were published in the Fall 2007 and Spring 2008 issues of Human Resources' Wolfbeat newsletter, and in the March 12, 2008 issue of the Bulletin. UPA distributed flyers to be posted in work locations across campus, and hosted an informational booth about the SWBS at the Employee Appreciation Benefits and Wellness Fair in early March 2008 (which included a drawing for a 30 minute massage, donated by Campus Recreation). Various administrators on campus also sent targeted emails to their constituents informing them about the survey. Further, a number of different incentives were used to help encourage staff to participate in the survey. Specifically, these consisted of: a drawings for two tickets to an NC State football game and for two tickets to an NC State men's basketball game (donated by the Athletics' office); four $50 gift cards for the University Bookstore (donated by the University Finance and Business office); ten vouchers worth two tickets each for an ARTS NC STATE performance (donated by the Office of Student Affairs); and a $50 gift certificate to the NC State Alumni Online Store (donated by the Alumni Association).
The research design called for members of the survey population to receive a pre-notification letter, either paper survey packet or an email announcing when the online survey went live, and at least two follow-up reminders for non- and partial-respondents. About 3 weeks before the survey went live, Provost Nielsen and VC Leffler sent all staff in the survey population a hardcopy pre-notification letter about the upcoming survey (including a Spanish version for Hispanic staff). Pre-notification letters were sent via campus mail to staff members' on-campus address, or through US mail to those with an off-campus work address. The letter informed staff about the upcoming survey, and explained the survey's importance in making progress on the University's Strategic Plan's Investment Priority "to help attract, develop, and retain a staff of the highest quality." The letter explicitly stated that participation in the survey was voluntary and that all responses would be kept confidential. The letter also included information about the drawing for incentives.
On March 27, 2008 all members of the survey population with a valid email address were sent an email from Provost Nielsen and Vice Chancellor Leffler announcing that the Staff Well-Being Survey was now available online. (This and all subsequent email announcements about the survey also informed faculty that a paper copy of the survey was available on request.) All those classified as skilled crafts, service, or maintenance were sent a paper copy of the survey to their work address, either through campus of US Mail.
In order to keep the names of those who had and had not responded to the survey confidential, all follow-up reminders to non-respondents and partial-respondents, and all queries about the survey were handled directly by Dr. Whelchel. On April 2, Dr. Whelchel sent targeted follow-up email reminders to non-respondents and to those who had submitted some, but not all sections of the online survey. Non-respondents classified as skilled crafts, service, or maintenance were sent a postcard reminder to their work address through either campus or US mail. On April 16, skilled crafts, service, and maintenance staff who had not yet submitted the survey either online or on paper were mailed another complete survey packet, and all others were mailed the postcard reminder. A final email reminder was sent to all non- and partial-respondents on April 28. The survey closed on May 1, 2008.
Response Rate
The Staff Well-Being Survey was in the field (either online or on paper) for a total of 45 days. As of that time, 3,243 of the 5,841 faculty in the survey population had submitted either some or all of the survey, either online (N=2828) or on paper (N=377 English version and N=38 Spanish version), for a 55.5 percent response rate, and a margin of error of plus or minus 0.8 percentage points.5
Table 2 presents a breakdown of response rates by various subgroups of the survey population, specifically: division/college, EPA/SPA classification, occupation, on- or off-campus work location, number of years employed at NC State, gender, race/ethnicity, and age. For the most part, response rates were remarkably consistent across these various groups - with the exception of gender (with women being more likely than men to have participated in the survey) there are literally no statistically significant differences in response rates between any of the aforementioned subgroups. Other than slightly over-representing females, the final survey sample closely mirrors the actual survey population in terms of its distribution across all groups listed above. For example, 31.0 percent of the survey population is in the College of Agricultural and Life Sciences, as are 30.4 percent of survey respondents. Off-campus staff (e.g., cooperative extension) make up 14.3% of the survey population and 15.0% of the staff. 12.4 percent of the staff in the population have worked at NC State for twenty or more years, as do 12.0% of respondents. African Americans make up 18.5% of the survey population and 16.5% of respondents. As a results, the overall survey results are generalizable to the survey population as a whole, and results of sub-groups analyses generalizable to the respective groups.
Given their relatively low number in both the survey population and among respondents, results are not presented separately for Native Americans. However, responses from Native Americans are included in the overall results, as well as in their respective group for other categories (e.g., women, SPA). In addition, in order to assure confidentiality, results for groups with less than 5 respondents are not presented.
Table 2: Survey Response Rate and Margin of Error (overall and by subgroups)| Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
|||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| Overall | 5,841 | 100.0 | 3,243 | 100.0 | 55.5 | ±0.8 |
| Division | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| Academic Affairs | 449 | 7.7 | 282 | 8.7 | 62.8 | ±2.2 |
| Chancellor's | 198 | 3.4 | 66 | 2.0 | 33.3 | ±8.0 |
| Extension, Engagement, & Economic Development | 149 | 2.6 | 83 | 2.6 | 55.7 | ±4.8 |
| Finance & Business | 1183 | 20.3 | 721 | 22.2 | 60.9 | ±1.4 |
| Office of Information Technology | 250 | 4.3 | 159 | 4.9 | 63.6 | ±2.8 |
| Research and Graduate Studies | 101 | 1.7 | 63 | 1.9 | 62.4 | ±4.6 |
| Student Affairs | 436 | 7.5 | 204 | 6.3 | 46.8 | ±3.7 |
| University Advancement | 73 | 1.3 | 44 | 1.4 | 60.3 | ±5.9 |
| College of Agriculture & Life Sciences | 1812 | 31.0 | 986 | 30.4 | 54.4 | ±1.4 |
| College of Design | 27 | 0.5 | 18 | 0.6 | 66.7 | ±7.7 |
| College Of Education | 57 | 1.0 | 29 | 0.9 | 50.9 | ±8.9 |
| College of Engineering | 342 | 5.9 | 183 | 5.6 | 53.5 | ±3.4 |
| College Of Humanities & Social Sciences | 83 | 1.4 | 48 | 1.5 | 57.8 | ±6.0 |
| College of Management | 37 | 0.6 | 19 | 0.6 | 51.4 | ±10.9 |
| College of Natural Resources | 99 | 1.7 | 64 | 2.0 | 64.6 | ±4.3 |
| College of Physical & Mathematical Sciences | 115 | 2.0 | 62 | 1.9 | 53.9 | ±5.7 |
| College of Textiles | 66 | 1.1 | 24 | 0.7 | 36.4 | ±12.7 |
| College Of Veterinary Medicine | 363 | 6.2 | 187 | 5.8 | 51.5 | ±3.5 |
| Classification | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| EPA | 1,759 | 30.1 | 919 | 28.3 | 52.2 | ±1.5 |
| SPA | 4,082 | 69.9 | 2,324 | 71.7 | 56.9 | ±0.9 |
| Occupation | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| Administrative | 257 | 4.4 | 162 | 5.0 | 63.0 | ±2.8 |
| Professional | 2,140 | 36.6 | 1,156 | 35.6 | 54.0 | ±1.3 |
| Technical | 1,462 | 25.0 | 888 | 27.4 | 60.7 | ±1.3 |
| Clerical | 1,041 | 17.8 | 527 | 16.3 | 50.6 | ±2.1 |
| Skilled Crafts | 356 | 6.1 | 203 | 6.3 | 57.0 | ±3.0 |
| Service/Maintenance | 585 | 10.0 | 307 | 9.5 | 52.5 | ±2.7 |
| Campus Location | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| On campus | 5,004 | 85.7 | 2,755 | 85 | 55.1 | ±0.8 |
| Off campus | 837 | 14.3 | 488 | 15 | 58.3 | ±1.8 |
| Years at NC State | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| 1 year or less | 950 | 16.3 | 537 | 16.6 | 56.5 | ±1.8 |
| More than 1 year, but less than 5 years | 1,258 | 21.5 | 688 | 21.2 | 54.7 | ±1.7 |
| 5-9 years | 1,711 | 29.3 | 969 | 29.9 | 56.6 | ±1.4 |
| 10-19 years | 1,199 | 20.5 | 661 | 20.4 | 55.1 | ±1.7 |
| 20 years or more | 723 | 12.4 | 388 | 12.0 | 53.7 | ±2.3 |
| Gender | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| Male | 2,568 | 44.0 | 1,239 | 38.2 | 48.2 | ±1.4 |
| Female | 3,273 | 56.0 | 2,004 | 61.8 | 61.2 | ±0.8 |
| Race/Ethnicity | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| White | 4,345 | 74.5 | 2,486 | 76.8 | 57.2 | ±0.8 |
| African American | 1,069 | 18.3 | 531 | 16.4 | 49.7 | ±2.1 |
| Asian | 181 | 3.1 | 89 | 2.8 | 49.2 | ±5.3 |
| Hispanic | 199 | 3.4 | 108 | 3.3 | 54.3 | ±4.3 |
| Native American | 23 | 0.4 | 16 | 0.5 | 69.6 | ±7.5 |
| Not specified | 12 | 0.2 | 6 | 0.2 | 50.0 | ±20.0 |
| Age | Survey Population |
Survey Respondents |
Response Rate |
Margin of Error |
||
|---|---|---|---|---|---|---|
| N | % | N | % | % | ||
| Less than 35 years of age | 1,445 | 24.7 | 758 | 23.4 | 52.5 | ±1.7 |
| 35-44 years of age | 1,453 | 24.9 | 798 | 24.6 | 54.9 | ±1.6 |
| 45-54 years of age | 1,802 | 30.9 | 1,042 | 32.1 | 57.8 | ±1.3 |
| 55 or more years of age | 1,141 | 19.5 | 645 | 19.9 | 56.5 | ±1.7 |
General Feedback on the Survey Process and Instrument
Although relatively few people provided feedback on the survey process or instrument during the data collection phase, anecdotal reactions to the survey itself (via email, phone calls, etc.) were general positive. Several staff called or emailed to question why they were not in the survey population, and adjustments were made as necessary. A few staff wrote or called to note that the survey took longer than anticipated to complete, and a few commented on the lack of a 'neutral' response option for most questions. A few also contacted Dr. Whelchel to discuss the confidentiality of the survey. Many staff, however, expressed their appreciation for being given the opportunity to share their opinions in such a survey.
For more information
For more information about the survey project or results, to provide feedback, or to suggest topics for further exploration, please contact Dr. Nancy Whelchel at 515-4184 or ncsu_surveys@ncsu.edu.
1. See Appendix A for timeline of important activities related to the project. (back)
2. See Appendix B for a complete list of committee members. (back)
3. Because of the nature of the survey - internal assessment - - the project did not fall under the purview of the IRB. However, IRB procedures were followed through all stages of the project. (back)
4. See Appendix C for an example of all correspondence with staff about the survey. (back)
5. That is, if 78.0 percent of the respondents answered a question saying they are "satisfied" working at NC State, we can be 95 percent sure that the true figure would be between 78.9 percent (78.0 + 0.9) and 77.1 percent (78.0 - 0.9) if all faculty had responded to the survey. The margin of error increases as the sample size decreases, so statements for various subgroups, such as the separate figures reported for full professors and assistant professors, are less precise than statements based on the total sample (see Table 2 for the margin of error for sub-groups). (back)
Posted: August, 2008
To download an MS Word version of this document, click here.
Return to 2008 Staff Well-Being Survey Table of Contents Page