Chapter III

A comprehensive assessment plan provides an opportunity for students, faculty, and alumni to express their perceptions of a course or program. Accrediting bodies frequently ask departments to provide input from students and alumni. Surveys are an excellent way of collecting such information. A survey can be locally constructed or purchased. It can be designed for specific groups such as students in a particular course, or alumni of the program. Surveys can be one page or several pages. They can be given in class or mailed. A variety of information can be obtained from a survey, as illustrated in the checklist below. You can choose which types of information will provide the most complete picture of the group you plan to survey.

As part of its assessment plan, the department may want to target a survey to a specific audience such as entering students, current students, exiting seniors, alumni, faculty, or employers. Each group can provide specific types of information as suggested below.

There are several types of data that can be obtained on a survey instrument. The department may be interested in finding information about students' perceptions of the department, levels of involvement, perceived competence at various skills, students' satisfaction, employment or education status, and faculty opinions about the department. The following are examples of questions that may be included on a survey used for the purpose of assessing the departmental major.

The main purpose of using surveys for assessment of the General Studies curriculum is to determine if students and faculty feel that the goals and objectives of the course or program are being met. A survey administered for the General Studies evaluation has questions that are about the course rather than the instructor.

Questions concerning the students' background, study techniques, academic performance, content evaluation, and faculty opinions and perceptions can provide information about a General Studies course.

The following are examples of questions that can be included on a survey used for the General Studies evaluation:

Types of Questions

Types of Questions	Use	Advantages	Disadvantages	Examples
Open Ended Questions	Stimulates free thought from the respondents. Elicits information that respondents can recall without difficulty when there are a very large number of possible answers and listing all of them as response choices makes answering the question difficult.	These questions stimulate free thought, solicit suggestions, probe people’s memories, and clarify positions.	These questions require people to find the terms with which to express themselves. Answers may be incomplete, uninterpretable, or irrelevant. Information may be difficult to analyze.	1. What should be done in order to improve the department? 2. To what professional organizations do you belong?
Closed-Ended Questions With Ordered Answer Choices	Determines intensity of feeling, degree of involvement, and frequency of participation. Ordered choices provide specific limits to responses.	These questions are less demanding to answer. Answers may be combined to form a multiple-item scale.	The responses may not be exhaustive.	1. How many hours a week do you study? a. 0-3 hours b. 4-7 hours c. 8-11 hours d. 12-15 hours e. 16 or more hours
Closed-Ended Questions With Unordered Answer Choices	Provides independent choices representing different concepts.	Questions of this type are often used to establish priorities among issues and decide among alternative policies.	Preferred options of all respondents may not be stated. Respondents must balance several ideas at once, especially if asked to rank 10-20 items.	1. Rank in order of importance the following reasons for attending this university. –Reputation of the university –Reputation of the department –Close to home –Friends attend –The size of the university
Partially Close-Ended Questions	Provides for responses which might be overlooked by researchers.	These questions allow respondents to give answers when the available choices do fit them.	A sufficient number of additional responses to warrant analyses may not be obtained.	1. What are your plans for next year? a. Continue prior job b. Start a new job c. Continue graduate study d. Other_____

Common Response Categories

Types of Questions	Description	Advantages	Disadvantages	Examples
Likert Scale	This scale is used with attitude and opinion questions. Respondents are asked to indicate the degree to which they agree or disagree with statements. Statements are usually worded fairly strongly, and can be worded both positively and negatively.	Questions are easily understood and quantified. Undecided responses can be accommodated. Allows for depth of response. Provides a meaningful way to group a series of items. Overall scores can be computed.	Method is less direct than using some other answer categories that more closely match the questions.	1. General Studies classes are very important. __Strongly Agree __Agree __Undecided __Disagree __Strongly Disagree
Semantic Differential Scale	This scale is best used to describe a series of attitudes toward a complex concept. The question presents the topic or issue, and the semantic differential scale asks the respondent to choose a number between two opposite adjectives.	Generally strong at finding particularly favorable or objectionable aspects of multi-faceted issues and concepts. Provides an overall scale score (average) for the concept.	Limited applicability	1. Do you feel that computer instruction is: Efficient 3 2 1 0 1 2 3 Inefficient Useful 3 2 1 0 1 2 3 Useless Boring 3 2 1 0 1 2 3 Interesting
Importance	These categories can be used when you need to find the importance of goals, objectives, or activities.	These categories simplify analysis, survey construction, and data entry. Works well to force respondents to give meaningful replies.	Not much variance is allowed, and statistical uses are limited. Does not provide much discrimination between items.	1. How important is…? __Very Important __Somewhat Important __Not Important
Participation	This scale can be used for questions about the frequency of activities, when respondents aren’t likely to know the actual number of times they participated.	Easy to complete, and works well to describe several activities which would have a wide range of expected frequencies.	Not very precise. Use numeric ranges such as 1-5, 6-10, etc. for additional precision, if the items have similar expected ranges.	1. How often did you attend…? __Very Often __Often __Sometimes __Rarely __Never

A) The answer to this question depends on who your audiences are and what they need to know. Do you want to know how students learned about your program and what they expect from it? In this case, a survey of students just entering the major is appropriate. Are you interested in exploring students' perceptions of the major program and how well it met their expectations? Surveying graduating seniors would give you this information. A General Studies evaluation may be concerned with only those students entering or exiting the particular course or courses being evaluated. Accrediting agencies sometimes require information about department alumni.

A) One suggestion would be to ask faculty members to submit survey items – what would they like to learn from the students? Some departments designate a committee or even one faculty member to design a survey. A good place to start is by looking at surveys that have been done in departments similar to yours. Commercially prepared surveys are available from places such as the Educational Testing Service (ETS) or the American College Testing Program (ACT).

A) One of the most crucial factors affecting response rate is the length of the survey. A survey should only take about 15 minutes or so to complete. Appearance is, of course, important and the survey should be easy to fill out, with clear and concise instructions. A cover letter that explains the purpose of the survey and assures the confidentiality of the respondents' answers can make a significant difference in whether a survey is completed and returned.

A) Whether or not you ask students to put their names on surveys depends in part on whether the surveys are to be mailed or given in class. Asking students in class to put their names on surveys could make students uncomfortable and reluctant to respond candidly. A mail survey is less personal so it is easier to ask for identification – either a name or Social Security number. There are several advantages to identifying respondents. First, identifying the respondents allows you to do a follow-up mailing and thus increase your response rate. Secondly, identified surveys allow you to match the survey data to other information which helps you describe the academic and demographic characteristics of the respondents. When the respondent is identified, be sure to stress that survey responses will be kept confidential.

A) Follow-up mailings can be helpful in boosting the return rate – often by as much as ten percent or more. However, they do require that the respondents be identified either through a label on the survey or by asking the respondent to supply their name. A follow-up mailing will also increase the cost of doing the survey.

A) Copying and mailing are the biggest expenses for a mail-out survey. A department may choose to do their own copying or have it done elsewhere. The cost of copying is affected by several factors such as the number of surveys, and the style and length of the survey. Mailing costs vary depending on whether you choose to send the surveys bulk rate or first class, whether you include a postage-paid return envelope, as well as the size of envelope. Departments sometimes choose to reduce the cost of surveying students by administering surveys in class or making them available in accessible locations such as the department office.

A) In deciding whether to survey all your students or a sample, you need to consider factors such as cost, available support staff, and time constraints, as well as your assessment needs. Your choice will depend in part on the size of the student population involved. A survey of students in a course that has several sections would probably require only a sample of students. A survey of the departments' majors might be more effective if all the majors are invited to participate.

A) Using the traditional survey, in which students put their answers directly on the survey rather than on a scan sheet, has the advantage of allowing for open-ended responses. Students generally prefer this style of survey since it seems more personal, less like an exam. Scan sheets require a particular type of pencil and need to be filled out very carefully by the respondents. On the other hand, scan sheets allow for quicker data analysis, and do not require support staff to handle the data entry. Using scan sheets is not recommended for mailed-out surveys since they generally cannot be folded.

Converse, Jean M. and Stanley Presser. Survey Questions: Handcrafting the Standardized Questionnaire. Newbury Park: SAGE Publications. 1986.

Dillman, Donald. Mail and Telephone Surveys: The Total Design Method. New York: Wiley-Interscience Publication, 1978.