"Focus groups provide an excellent opportunity to listen to the voices of students, explore issues in depth, and obtain insights that might not occur without the discussion they provide." (Palomba & Banta, 1999, p.196-7)

Focus groups are "fundamentally a way of listening to people and learning from them." (Morgan, 1998, p. 9) They can be used:

• to examine attitudes or opinions
• to explore why opinions are held
• to identify the strengths and weaknesses of programs
• to interpret results from other assessment projects
• to provide information for designing surveys

Focus groups should not be used:

"A critical aspect of preparing to conduct a series of focus groups is selecting a precise set of questions." (Palomba & Banta, 1999, p.202)

Questions can also be in the form of group exercises. Group members can be asked to write a word or phrase on index cards and then share their responses. This ensures that all members share opinions. Groups can be asked to generate lists or brainstorm ideas. Individuals can be asked to rate aspects of a program and then explain what led them to give that particular rating. Sentence completion exercises can also be used (e.g., What I liked best about this program was.). The openness of the focus group format allows for a multitude of activities that can guide or generate discussion.

"The quality of the moderator is the most important element that determines the ultimate usefulness of the output of focus group research…" (Greenbaum, 1988, p. ix)

The moderator begins the focus group with a short introduction, which should include:

• a brief welcome
• an overview of the topic
• some guidelines or ground rules
• an opening question.

The welcome should include a thank you for participating and an introduction of the moderator(s). The overview is often short and simple, explaining what the topic is and why the participants were chosen. Guidelines are used to lay groundwork such as speaking one at a time or using first names only. They also function as a time to explain recording devices, confidentiality, and the role of the moderator. The opening question should serve as an ice-breaker and should give each participant a chance to talk. Many books on conducting focus groups have sample introductions (see References).

An important part of moderating a focus group is listening. Some suggestions for improving listening skills include: make a conscious effort to actively listen, focus on listening rather than talking, make eye contact with each participant as they speak, and be "intimately familiar" with the questions and objectives of the project. (Greenbaum, 1999, pg 131-132) Another feature of moderating is time management — the moderator must weigh the length of discussion needed for each question with the time limits of the focus group. Finally, the moderator must probe for further information. He/she must pick up on nonverbal clues and draw out information. He/she must follow up with unclear statements.

Facility problems usually involve distractions or recording equipment. A little planning can often prevent both of these problems. Examining a room ahead of time can alert a moderator to potential distraction problems. Precautions such as closing doors or windows may be all that is needed to eliminate noise and other distractions. Sometimes, a new room will need to be chosen — which is easier to do in advance than at the time of the focus group. A moderator should also have a back up plan for equipment failure, whether it is a back up tape recorder or a skilled note taker.

A dominant group member can influence the tone of the group or inhibit comments from other participants. A dominant group member may take the form of an expert or just someone who likes to talk. Greenbaum suggests using the dominant person "as a straw man against which the moderator can play the reactions of the rest of the group." (1988, p.65) Asking other group members whether they agree or disagree with the dominant member and why they feel that way allows others to talk and encourages a variety of opinions.

Another common difficulty is the quiet or shy group member. The moderator may need to encourage this person with eye contact, call on the person by name, and use follow up questions to generate a useful response from the person.

Transcript Analysis

-using data from the student database to examine the course-taking or grade patterns of students

Use transcript analysis when:

• you would like a "snapshot" of a group of students at a particular point in time.
• you need to know what classes students took and in what order.
• you are interested in patterns in student grades.

Transcript analysis allows the department to obtain a more complete picture of their students. Are majors who follow a particular course-taking path through their college years more likely to succeed? Do the department's General Studies courses attract students to the major program? What path through the General Studies program seems to be most effective for the department's majors?

Suggestions for Using Transcript Analysis:

1. A department may be interested in determining if students change majors as a result of taking General Studies courses offered in the department. The university student database (see Chapter VI) would allow the department to relate a student's choice of major to the initial course they took in that department.
2. A department may want to explore the entering qualifications of students enrolled in a required qualifying course for their majors. The database could be searched to select the students enrolled in the course and report information on entering qualifications (i.e., grades in General Studies English or Math courses).
3. Investigating the General Studies course-taking pattern of their majors is often useful to departments. The student database may be used to produce a report which tracks department majors, listing the General Studies courses taken by each with term and grade awarded.

Unobtrusive Measures

-observation or keeping records of students' use of facilities and services. This form of assessment provides data that can be correlated with test scores and/or course grades.

Use unobtrusive measures when:

• you are interested in how students study.
• you are concerned about the effectiveness of study sessions or other supplements to classroom activities.
• you are focusing on the relationship between out-of-class behavior and in-class performance.

While test scores and survey results can indicate how much students are learning and how they feel about that process, observing the behavior of students can tell you quite a bit about how they learn. What do students do outside the classroom? Which out-of-class behaviors and activities seem to lead to better in-class performance?

Suggestions for Using Unobtrusive Measures:

1. An indicator of student participation in a course can be how frequently students are using services such as study sessions, department computer labs, or reserved supplemental readings in the library. Even tracking class attendance can be useful.
2. A department can determine how frequently students attend museums, concerts, plays, recitals, lectures, etc.
3. It may be helpful to a department to know how frequently students visit the department advisor, chair, or other faculty members. Are successful students more likely to seek advice or guidance? Of interest, too, may be how often students in the department make use of facilities such as the Learning Center.

Review of Current Curriculum Materials

-systematic review of course syllabi, textbooks, exams, and other materials.

Use curriculum materials when:

• you want to clarify learning objectives.
• you want to explore differences and similarities between sections of a course.
• you want to assess the effectiveness of instructional materials.

Basic to any instruction are the materials used in the classroom – textbooks, hand-outs, exams, and so forth. A review of these provides invaluable information to enhance any assessment effort.

Suggestions for Using a Review of Curriculum Materials

1. Faculty members can review the current course syllabi for the department's General Studies course to determine if the course is meeting the goals and objectives that were established for it.
2. A review of the course exams for the different sections might reveal enough standardization of exam items to allow those exams to be used to assess the course. For example, five different instructors who are each designing an exam for their section of a course could be asking similar questions on common subjects. Those common items could be used as an assessment tool much the same as if a standardized exam had been administered to all five sections.

Anecdotal Records, Logs, and Journals

-maintaining records of classroom activities, students' responses, or faculty impressions

Use anecdotal records when:

• you are concerned with how students and faculty react to particular instructional methods.
• you are piloting new textbooks or other materials.
• you are interested in students' perceptions of certain aspects of the course.

While assessment usually relies on systematic collection of information, often the more informal observations prove useful. What is going on in the classroom? What works and what does not?

Suggestions for Using Anecdotal Records, Logs, and Journals

1. Faculty members can maintain journals on their experiences in the classroom. If, for example, a new textbook is introduced, a journal can be kept to record the students' responses to the book, as well as the instructor's reaction.
2. Students can be asked to evaluate visiting speakers or instructors.
3. Faculty members can ask students to write reaction papers in response to classroom activities, videotapes, field trips, and so forth. Did students find the activities helpful? How could the activities be improved?

External Examiners

-using an individual or group of individuals from outside the department to provide an objective assessment of a course or program.

Use examiners when:

• you need an objective observer of your program.
• you want an assessment expert to provide insight and advice.

Asking someone from outside the department or university to observe and respond to particular aspects of a program or course can yield valuable information that could not otherwise be obtained.

Suggestions for Using External Examiners

1. A department may invite a member of the English faculty in as a consultant to provide insight into the department's use of writing in the major. The consultant can answer questions such as: Does the program provide ample opportunities for students to write? How can the department increase those opportunities? How could the department effectively assess their students' writing ability?
2. An external evaluator from another university can be invited to moderate a focus group or to assist a planning session on designing a departmental assessment plan.

For Further Reading

Albrecht, T. L., Johnson, G. M., & Walther, J. B. (1993). Understanding communication processes in focus groups. In D. L. Morgan (Ed.), Successful Focus Groups: Advancing the State of the Art (pp. 51-64). Newbury Park, CA: Sage Publications.

Brown, J. B. (1999). The use of focus groups in clinical research. In B. F. Crabtree & W. L. Miller (Eds.), Doing Qualitative Research (2^nd ed., pp. 109-124). Thousand Oaks, CA: Sage Publications.

Crabtree, B. F., & Miller, W. L. (Eds.). (1999). Doing Qualitative Research (2^nd ed.). Thousand Oaks, CA: Sage Publications.

Fowler, Floyd J., Jr. and Thomas W. Mangione. Standardized Survey Interviewing. Newbury Park: SAGE Publications, 1990

Greenbaum, T. L. (1988). The Practical Handbook and Guide to Focus Group Research. Lexington, MA: D.C. Heath and Company.

Greenbaum, T. L. (2000). Moderating Focus Groups: A Practical Guide for Group Facilitation. Thousand Oaks, CA: Sage Publications.

Krueger, R. A. (1993). Quality control in focus group research. In D. L. Morgan (Ed.), Successful Focus Groups: Advancing the State of the Art (pp. 65-85). Newbury Park, CA: Sage Publications.

Krueger, R. A. (1994). Focus Groups: A Practical Guide for Applied Research (2^nd ed.). Thousand Oaks, CA: Sage Publications.

Krueger, R. A. (1998a). Developing Questions for Focus Groups: Focus Group Kit 3. Thousand Oaks, CA: Sage Publications.

Krueger, R. A. (1998b). Moderating Focus Groups: Focus Groups Kit 4. Thousands Oaks, CA: Sage Publications.

Morgan, D. L. (Ed.). (1993). Successful Focus Groups: Advancing the State of the Art. Newbury Park, CA: Sage Publications.

Morgan, D. L. (1998). The Focus Group Guidebook: Focus Group Kit 1. Thousand Oaks, CA: Sage Publications.

Morgan, David L. Focus Groups as Qualitative Research. Newbury Park: SAGE Publications, 1988.

Palomba, C. A., & Banta, T.W. (1999). Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education. San Francisco: Jossey-Bass.

Sapp, Mary M. and M. Lewis Temares. "Learning How to Improve the Quality of Services Using Focus Groups". Paper presented at the 32d Annual AIR Forum, 1992.

Stewart, D. W., & Shamdasani, P. N. (1990). Focus Groups: Theory and Practice (Applied Social Research Methods Series Vol. 20). Newbury Park, CA: Sage Publications.