Department of Political Science
Knowledge and skill goals for this undergraduate degree program are recorded in the most recent CU-Boulder catalog.
In some summaries of assessment activity, goals are referred to by number (e.g., K-2 is knowledge goal 2).
Assessment Activities in 2002-03
(Undergraduate Curriculum Committee: Prof. Mapel-Chair, Prof. Mewes, Prof. Bair, Andrew Thangasamy-graduate student)
This report summarizes the Departmentís ongoing process of evaluating undergraduate degree program performance. After taking a close look last year at faculty teaching portfolios, the Department returned this year to its regular practice of focusing primarily on student portfolio assessment.
The Process of Portfolio Assessment
In 2003, the UCC performed two tasks: reviewing a sample of portfolios of graduating seniors and collecting portfolio materials for continuing majors, both by gathering tests and essays from a new cohort of freshman majors begun last year.
These portfolios contain a history of student performance as indicated by final exams and papers finished between Fall of 1999 and Fall of 2003. Identifying information was deleted from this material, leaving course number, assignment, and semester in which the work was done. The professorís name and grade assigned, if originally indicated, were also removed. Three class tests and papers constituted a minimum portfolio; most portfolios had four or five assignments. Seven portfolios were evaluated, totaling 23 exams and essays altogether. This was a randomly drawn sample, with GPAs varying from 2.25-3.89.
Each studentís written work was evaluated on four characteristics plus an estimated GPA. Those characteristics were:
Students were rated on achievement during their fourth year of examinations and papers as well as on improvement over the entire period. Achievement was rated on a scale running from "Excellent" to "Good", "Fair", "Poor", and "Not Applicable" when there was insufficient evidence, e.g. when an assignment failed to require a particular intellectual task such as making comparative judgments. "Improvement" reflected a global impression of a studentís performance across all four characteristics. Raters were asked simply to indicate whether they perceived any improvement in performance between studentsí freshman/sophomore and junior/senior level course work by answering "yes", "some", and "none." The complete set of scores for the set of portfolios is shown in the table below.
As one can see from the attachment, there are two cases where the estimated GPAs of students were considerably higher than their actual GPAs. We have no ready explanation for these two discrepant scores, although it may simply be that students have a tendency to perform somewhat better on assignments for classes in their own major than they perform in classes generally. Otherwise, the faculty (and graduate student) scoring of students was fairly consistent, and after discussion of the results, the committee unanimously agreed on the following conclusions. The committee concurred that, with one exception, all of the students showed clear improvement. The committee also agreed that this improvement was particularly evident in the writing skills of the students. The students who showed less improvement clearly had better skills at the beginning, a result that we have consistently found over several years of doing portfolio assessments. In contrast, students who were below par at the beginning of their careers showed the most dramatic improvement, once again in their writing skills in particular. It is hard to say, however, whether this means that the Department does especially well at teaching students who have poor skills at the outset of their education, or only that any improvement in the work of such students is bound to show up with particular clarity. Whatever the explanation, the overall results of the assessment are encouraging.
Another thing that the department seems to do fairly well is in striking a balance between creative and critical thinking. In addition to the usual exams and essay questions, for example, there were assignments that asked students to demonstrate their understanding of a particular set of issues by writing short stories or plays. When assignments were more conventional in character, professors did well in forcing students to engage in critical thinking by making assignments particularly explicit in this regard. Almost all of the assignments asked students "to make a counter-argument and refute it" or "consider alternative explanations." There were a few assignments, however, that did not provide sufficiently explicit directions to engage in critical thinking. This failure may have made a couple of students score lower than otherwise, through no fault of their own. This is certainly one place where we can continue to improve as a department, by being more consistent across courses in forcing students to engage in analysis that requires the consideration of alternative explanations or interpretations.
Assessment Activities in 2001-02
The political science department has assessed success and failure in undergraduate teaching in a number of ways in the past. We have looked at the historical development of our honors program, for example, and have also critically examined the results of several different kinds of surveys. Our primary method of assessment for many years, however, has been to review the portfolios of a random sample of graduating seniors. These portfolios contain a history of student performance as indicated by final examination and written papers collected from students as freshmen and again when these students are seniors. Portfolios are stripped of information that would identify a specific student, and student performance is then evaluated by members of the Undergraduate Curriculum Committee in terms of an estimated GPA plus progress made in four areas: basic knowledge of different areas of political science, ability to make comparative judgments and to think critically, ability to draw reasoned conclusions from evidence, and writing ability. The department continues to collect these data from freshmen, and plans to do a portfolio evaluation in 2002-03.
This year, however, we have elected to use a different approach to evaluating and trying to improve our undergraduate teaching. We have adopted this approach because we believe that it is also crucial to begin assessing the teaching process itself as a way of determining what works and what doesn't in educating our undergraduates. For this reason, the Undergraduate Curriculum Committee collected and critically evaluated the teaching portfolios of sixteen members of the department. These portfolios contain a very extensive set of materials, including syllabi, exams, paper assignments, web site pages, reading and studying guides, bibliographies and other kinds of teaching aids. In addition, the committee met with instructors teaching the large lecture courses to discuss how they are being taught and how they might be improved in order to do a better job helping students learn critical thinking skills.
General Findings and Recommendations
The committee was generally encouraged by the obviously great effort that has gone into faculty teaching materials, as well as by their wide variety. Virtually without exception, the political science faculty devote a great deal of thought and energy to course design. Given the very wide variety of materials contained in the teaching portfolios, it was also clear to the committee that it would be inappropriate to suggest a single model for constructing courses. Nevertheless, the committee does believe that faculty members can learn a great deal from one another about teaching, and that the more discussion and cooperation we can generate, the more students will benefit. We begin by noting a few of the specific innovations that our faculty have introduced in their courses. We suggest that it might be worthwhile for other members to experiment with similar projects (the committee can put faculty members together who wish to collaborate). Our report then moves to some more general recommendations.
Much of the specific experimentation we noted in the teaching portfolios involved the use of different media and technology in and outside of the classroom. One faculty member who teaches Comparative Politics, for example, has a quite ambitious, free-standing "film festival" of relevant foreign movies running side by side with her more traditional lectures, exams and paper assignments. A second faculty member has set up distance-learning, allowing students in his course to collaborate electronically with students at another university where a parallel course is being taught. Many other faculty members have built sophisticated web sites which give students much greater ability to interact and to make use of a wide range of resources on the net. In cases where course web sites are long and complex, at least some of these sites have internal links to make them more student-friendly. The general lesson that the committee draws from such experiments is that our faculty need to encourage each other to make greater use of technology in their teaching. One of the best ways in which this can be done is through the summer Faculty Teaching Excellence Program (FTEP) workshop on integrating technology in the classroom. The committee notes that several of our colleagues have now taken advantage of this course. The FTEP workshop discusses the pedagogy of using technology to its full advantage, teaches faculty how to employ various tools in the classroom, such as Powerpoint, and helps faculty build and evaluate their own course web sites. The FTEP workshop appears to be the single best resource available on campus for learning make use of new technology in the classroom. The committee strongly recommends that more political science faculty make use of it.
Turning to other areas, our first concern is with the large, introductory lecture courses. As part of our effort to make these courses more effective, the department passed a motion to include in these classes an assignment that requires TAs to work individually with students on critical thinking. One example might be a requirement that each student works with his or her TA on several drafts of a critical thinking paper. Another example might be a set of problems that must be tackled one-on-one with a TA. Thus far, writing assignments of the sort just described have been assigned in the large introductory lecture courses in American politics, international relations, comparative politics, and political theory.
The committee met with faculty teaching these kinds of courses to find out how these assignments have worked and to see what might be done to improve them. We were particularly interested in having faculty explain what was unique in their approach to these classes and how they used TAs. Are TAs encouraged to develop a separate "story-line" in their sections, for example, or are they expected to stick closely to the material in lecture? This discussion was particularly valuable in establishing the need to continue experimenting with the use of TAs and in establishing norms about working closely with TAs on how these requirements are implemented. One of the committee's recommendations is that we make it a regular practice for faculty teaching these large courses to meet and discuss how these courses should be taught. While it is obvious that each faculty member will use TAs somewhat differently, faculty should be expected to meet extensively with TAs every week to discuss both the course material and the way that material is being taught in sections.
On the basis of our examination of the teaching portfolios, the committee has two other major recommendations to offer. First, we suggest that the department develop a general web-page or set of pages that can be linked to all of the courses taught in the department. In particular, we believe it would be useful for students to be able to link to a departmental web-page on how to read critically, another page on critical thinking, and yet a third on stages in writing a research paper. A good example of how the reading page might be done can be found on one faculty web site that uses a special acronym to help students remember different aspects of critical reading. A web page on critical thinking might also be devised drawing on several different ideas contained in these teaching portfolios. For example, one faculty member first teaches a range of informal fallacies and mistakes in reasoning, and then has his students write a paper deliberately committing as many of these errors as possible. We would also like to note that where an assignment is both detailed and difficult, some syllabi include words of encouragement. We suggest that faculty think of their syllabi as something that can be humorous, encouraging, perhaps inspiring and otherwise communicative, rather than as just a list of readings and requirements.
The committee's second general recommendation, or set of recommendations, concerns the design of syllabi and course material. Again, we caution that one size does not fit all. Nevertheless, the committee had a chance to look at over thirty syllabi in political science, and we believe that faculty members might improve their teaching by including some of the following features in their syllabi (assuming they don't already):
Use of assessment in instructional planning/design
We have been asked how our student assessment process has influenced our curricular development and other educational programs related to unit instruction. In particular, we have been asked: 1) whether we have spent more time discussing the teaching of critical thinking in all classes; 2) what has been done more specifically in the large lecture courses, including information on the effects of our assessment process.
With respect to the first question, there is on-going discussion of the problem of teaching critical thinking, both in the Undergraduate Curriculum Committee and in the department as a whole. In the summer of 2002, for example, the committee assessed the teaching portfolios of sixteen members of the department and drafted a report with several recommendations about how to improve course materials to enhance critical thinking. At the departmental level, we have also been engaged in discussions of teaching critical thinking, in particular during our program review process (PRP) last year, when we made use of the results of outcomes assessment in our report on the undergraduate program. PRP included several recommendations about improving critical thinking in all classes, including a systematic assessment of the teaching process itself (which we have started, as noted above), supplementing FCQs with evaluation of more extensive teaching materials, etc. In addition, the department made more specific recommendations about teaching the large lecture classes. All of the large, lecture courses below include a mid-term exam (sometimes two), a final exam, and various recitation assignments. Here this report focuses only on the extra assignment that was included in order to strengthen the critical thinking component. Here are some examples of what was done:
The Undergraduate Curriculum Committee met in mid-September with all of the faculty responsible for these courses in order to discuss how better to teach the critical thinking component of the respective courses. Topics included: getting everyone to use their TAs in the required way; uniform expectations about supervising TAs; whether the critical thinking assignments focused primarily on factual questions or on questions dealing with critical thinking; whether paper topics should be open-ended or tailored; increasing the number of grade points. The Department is committed to pursuing this matter and plans to have a regular meetings in the future to monitor and discuss these classes.
A Note on Graduate-Level Assessment
At the graduate level, we emphasize the training of the next generation of political scientists in the knowledge and best practices of the discipline. Together with our students, we seek to explore and expand what we can know about politics and government and to reason about the classic questions of
L:\ir\Outcomes\0A0102\polisci.doc and OA0203\polisci.doc
Last revision 05/05/03
PBA Home | Strategic Planning |   Institutional Research & Analysis |  Budget & Finances |
Questions? Comments? | Legal & Trademarks | Privacy
15 UCB, University of Colorado Boulder, Boulder, CO 80309-0015, (303)492-8631
© Regents of the University of Colorado