Instructor Use

Jalloh, C., Collins, B., Lafleur, D., Reimer, J., & Morrow, A. (2019). Mapping session learning objectives to exam questions: How to do it and how to apply the results. Medical Teacher, 42, 66-72. These authors mapped 536 multiple-choice questions in an exam bank to 358 instructional LOs in an undergraduate population health course. Instructional LOs were revised to create more multiple-choice items that could measure each LO and required higher-level thinking. This exercise resulted in better alignment of assessment and teaching with the LOs. Learning objectives were revised to engage students in higher-level thinking. Validity of exam score meaning was improved in several ways. For example, new questions that assessed higher-order thinking were created. All questions were explicitly mapped to ensure the assessment of all LOs, improving content coverage. The reliability of exam scores (as measured by KR-20, an estimate of internal consistency reliability equivalent to coefficient alpha) increased post-mapping. Student survey data indicated a 47% improvement in student perception of the clarity of the LOs and a 23% decrease in negative comments about exams. The authors offer instructors a step-by-step guide to mapping LOs and assessment items to ensure that LOs 1) reflect program goals/professional practice, 2) improve score reliability, and improve the validity of interpretation and use of scores.

Chasteen, S.V., Pepper, R.E., Caballero, M.D., Pollock, S.J., & Perkins, K.K. (2012a). Colorado Upper-Division Electrostatics diagnostic: A conceptual assessment for the junior level. Physical Review Special Topics-physics Education Research, 8, 020108-1-020108-15. As part of an upper-division electrostatics course transformation effort, researchers engaged faculty to create course-level LOs. The researchers then developed a new assessment explicitly designed to test student mastery of the new course-level LOs because those objectives were not aligned with traditional exam questions. A follow-up paper showed that student performance on this assessment in the reformed course was higher than in conventional sections, even though there was no change in scores on traditional exams (Chasteen et al., 2012b). Instructors will find a comprehensive description of the development of an open-ended exam, with reliability and validity evidence, designed to measure higher-level thinking and identify student learning difficulties.

Chasteen, S.V., Pollock, S.J., Pepper, R.E., & Perkins, K.K. (2012b). Thinking like a physicist: A multi-semester case study of junior-level electricity and magnetism. American Journal of Physics, 80, 923-930. This article is a follow-up paper to the results of redesigning an upper-division physics course transformation using LOs to design instruction (Chasteen et al., 2011). A concept-based assessment, with supporting validity evidence, was developed to measure learning gains of the course-level LOs (Chasteen et al., 2012a). The learning gains demonstrated by students in the redesigned courses were the equivalent of two letter grades higher than students in the standard lecture courses. These results were observed for students at all performance levels and at all participating institutions. Instructors will find additional analysis of outcomes reported for the redesign of an upper-division physics course transformation using LOs to design instruction.

Minbiole, J. (2016). Improving course coherence & assessment rigor: “Understanding by Design” in a nonmajors biology course. The American Biology Teacher, 78, 463-470. The author used the Understanding by Design framework to align assessment with explicit LOs that articulated the goals of a non-majors biology course. LOs were shared with students to guide their preparations for quizzes and exams. Assessment shifted from a topic-based approach to assessing deeper understanding via more short answer questions requiring justification. Exams were designed to gather student progress towards the LOs by assessing knowledge and skills. Exam questions shifted from 70% multiple-choice/30% short answer before use of the planner to 70% short-answer and essay questions focused on the LOs after the redesign. Prioritizing course content decreased lecture time and allowed for increased class discussions, yielding opportunities for formative assessment that informed future instruction. Despite increased rigor on exams, final exam grades improved after LOs were articulated, with significantly more students earning A’s or B’s. Based on these outcomes, instructors should be encouraged to develop LOs to guide assessment that emphasizes measuring student skills and requiring students to justify their thinking.

Ricke, A. (2019). Mapping assessment in anthropology: Using team‐based qualitative methodology to create learning objectives and evaluate outcomes. The Annals of Anthropological Practice, 43, 53-71. The author reports on developing and using an assessment framework for introductory anthropology courses focused on course-level LOs aligned with the university- and state-level LOs. The assessment framework is based on research methodology used in the discipline—specifically, team-based qualitative data analysis–and emphasizes collaboration to protect instructors’ agency and creativity. The overall goal was to create an assessment framework that addressed faculty resistance to administrative requirements for assessment and could provide data on performance in the same course across multiple instructors and sections. Instructor use indicated that the model was not burdensome; dissatisfaction was found when collaborative efforts were low. The author provides instructors with an example of a framework for curriculum mapping used to gain instructor buy-in by protecting instructor creativity in the course and assignment design. Instructors will find a protocol described whereby assessment is aligned with LOs, providing consistency across multiple class sections.

Keshavarz, M. (2011). Measuring course learning outcomes. Journal of Learning Design, 4, 1-9. The author reviews key ideas in assessing student mastery of course-level LOs and suggests a systematic methodology for doing so: creating a linkage matrix that associates each LO with student performance on each of multiple assessments. This approach provided insight into the consistency of the various assessment tools used to measure student learning and an overview of student performance on each course-level LO. Instructors will find a linkage matrix described that can be applied to any course and can track student progress to meeting course or instructional LOs and provide insight on the consistency of the assessment tools being used to measure each LO.

Rodriguez, M.C., & Albano, A.D. (2017). The college instructor’s guide to writing test items: measuring student learning. Routledge. The authors give practical guidelines to improve instructors’ assessment practice. Topics discussed include the foundational role of LOs for planning assessment, creating a test blueprint, item-writing guidelines for writing multiple-choice items, and items from various fields to critique using the item-writing guidelines provided. Instructors will find this resource helpful in constructing assessments that have validity evidence to support their interpretation and use, that align with instructional LOs, and that are representative of classroom instruction.

HHMI BioInteractive estimates that the assessment database website will be live in the Summer of 2022. Creating a placeholder for this here.

ASK BIO network regional workshops. (2021). ASK-BIO Assessment Skills in Biology. The Advancing Assessment Skills in Biology (ASK-BIO) Network is an NSF-supported initiative that supports Biology instructors in improving question-writing and assessment development, emphasizing linking assessment items to instructional LOs. Free regional workshops are offered throughout the US.

McTighe, J., & Wiggins, G. (2012, March). Understanding by design framework. Association for Supervision and Curriculum Development. The authors outline a framework for course design that begins with considering course LOs. Using this framework, instructors consider the knowledge and skills students should have acquired upon successfully completing the course. Assessments are then designed to evaluate mastery of the stated knowledge and skills. Finally, classroom instruction is planned to allow students the opportunity to learn and practice those learning in advance of higher-stake assessments. Instructors will find this a valuable resource for aligning their course assessments and activities with their LOs.

Minbiole, J. (2016). Improving course coherence & assessment rigor: “Understanding by Design” in a nonmajors biology course. The American Biology Teacher, 78, 463-470. The author used the Understanding by Design framework to create explicit LOs that articulated the goals of a non-majors biology course and formed the basis for assessment and instruction. Key understandings and core ideas were identified, assessments were redesigned based on these understandings and core ideas, and course elements were prioritized for instruction. Elements identified as desirable but not essential were removed from the lecture, reducing the time spent lecturing by half. Classroom activities were designed to support student achievement of the LOs and included activities, video clips, and student discussions. LOs were shared with students to make course priorities transparent. Final exam grades increased after LOs were articulated, with significantly more students earning A’s or B’s. Instructors are provided with a 3-stage planning template that uses LOs to guide the redesign and reevaluation of a course based on the Understanding by Design framework.

Reynolds, H.L., & Kearns, K.D. (2017). A planning tool for incorporating backward design, active learning, and authentic assessment in the college classroom. College Teaching, 65(1), 17-27. The authors introduce a backward design-inspired lesson planner that documents LOs and their alignment with assessment and course activities. This planning matrix is adaptable for individual lessons or major units. The researchers claim that using the tool increased planning efficiency, instructor creativity, and opportunities for feedback on student learning. Using the planner in a non-majors biology course resulted in a shift from 90% lecture and 10% active learning to 45% lecture and 55% active learning. Instructors should note that the success of this 3-step lesson-planning matrix is based on establishing measurable LOs that guide the backward design process.

Smith, G.A., Stark, A., & Sanchez, J. (2019). What does course design mean to college science and mathematics teachers? The Journal of College Science Teaching, 48, 81-91. The authors analyze how college math and science teachers view course design. They interviewed nine instructors who were completing a 1-yr professional development program on STEM course redesign. This effort included revision and creation of new LOs, new formative and summative assessments, and the addition of new active learning activities. Peer interaction between teams facilitated feedback. Each of the instructors interviewed had taught their redesigned course. By coding semi-structured interviews, the researchers identified five levels in conceptualizing course design. Each represented a distinct level in instructors’ overall awareness and understanding of what it means to design a course, beginning with instructors’ awareness of what to teach and how to teach it, and progressing to the incorporation of reflective practice, the inclusion of departmental collaboration, and efforts to incur collective change across a discipline. This research can be used to inform and enhance course design. Instructors should use this analysis to support reflection on conceptualizing their course design, prompting consideration of content, teaching methods, continuous improvement efforts, departmental collaboration, and collective change across institutions.

  Fata-Hartley, C. (2011). Resisting rote: The importance of active learning for all course learning objectives. The Journal of College Science Teaching, 40, 36-39. The author reports observations on the impact of student engagement through active learning activities to teach stated LOs, versus relying on a lecture-based approach with the expectation of rote memorization. A table is provided to illustrate a strategy for aligning assessments and classroom activities with the LOs. Active vs. passive instructional methods employed to teach the LOs were compared. Students performed significantly better on higher-level thinking items that assessed content taught using active engagement compared to lower-level items that assessed content presented by lecture requiring rote-memorization. Instructors will find a table that includes example LOs, examples of aligned assessment items, and examples of aligned classroom activities that vary in their degree of student engagement.

Leone, E.A., Salisbury, S.L., Nolen, Z.L., Idema, J.L., Parsley, K.M., Stefanik, K.L., & Daniel, K.L. (2019). Identifying the breakdowns in how students and faculty interpret course objectives. Bioscene: The Journal of College Biology Teaching, 45, 16-23. The authors present a survey-based study of 424 students and two participating instructors from an introductory biology course designed for non-science majors. They found that instructors’ intended objectives- those that instructors use to plan instruction- are not always connected or aligned to the course-level LOs explicitly stated in the syllabus. Further, students more accurately recognize intended objectives based on classroom activity vs. the explicit course objectives listed in the syllabus. The study concludes that classroom activities communicate LOs more effectively than the syllabi. Instructors should note that these outcomes support and underscore that classroom instruction communicates instructor priorities over that which is listed in their syllabi and that effective use of LOs requires ongoing and frequent reinforcement as a part of classroom instruction.

Bowen, R.S. (2017). Understanding by design. Vanderbilt University Center for Teaching. The author provides an overview of backward design, elaborating on its benefits, explaining the three phases, and providing a template for implementation.

Milkova, S. (n.d.). Strategies for effective lesson planning. Center for Research on Learning and Teaching (CRLT). The author describes strategies for planning classroom instruction that is aligned with LOs. Specific steps are outlined for preparing classroom instruction, and recommendations are given for use and reflection.

Simon, B., & Taylor, J. (2008). What is the value of course-specific learning goals? The Journal of College Science Teaching, 39, 52-57. The authors discuss the value of using LOs in two upper-level computing literacy courses and one upper-level microbiology course from the student and instructor viewpoint. Student feedback (n=553 student comments) on the use of LOs was 85% positive, with the dominant positive value identified by students was that LOs helped them in “knowing what I need to know.” Students identified LOs as valuable in organizing the information based on what the instructor identified as important. Instructor comments support LOs as an effective communication method with students. Negative feedback from students included complaints of “unclear wording.” Survey results supported the authors’ hypothesis that students have difficulty identifying what is essential in their courses and that LOs help them organize their efforts more effectively. Instructor feedback indicated that LOs enhanced communication with both students and other instructors. Instructors should note that the outcomes reported in this study underscore that using instructional LOs to guide instruction positively affects both students and instructors.

Bowman, D.R., & Stephan, E. (2012). Using GEARSET to promote student awareness of learning objectives. Computers in Education Journal, (3)2, 27-33. The authors report on the implementation of a web-based module in large enrollment, freshman engineering courses. The module was designed to increase student understanding of each learning activity connected to the instructional, unit, and course LOs. The tool allows students to check off tasks completed and LOs they feel they have mastered and gives instructors real-time information about student progress. The authors report increased student awareness of the connections between course activities and LOs at the instructional, unit, and course levels. Analytics indicate that increased engagement with the tool can be used as an early warning system to alert instructors to at-risk students and is associated with higher grades. The outcomes reported here encourage instructors to make explicit connections between learning activities and instructional LOs. Instructors should encourage students to use LOs to self-evaluate their understanding.

  Babik, D., & Lending, D. (2020). Teaching tip: Visualizing IS course objectives and marketable skills. Journal of Information Systems Education, 31, 260-268. The authors created visual presentations of course-level LOs in two information systems courses to get students to engage with and internalize the LOs. The objectives were presented at the beginning of the course and again mid-course. Student feedback indicated that (a) the visualization helped them understand what was expected, even before course content was learned, (b) presenting LOs in any form supported high-confidence in terms of learning gains, and (c) visual vs. text-based LOs supported higher confidence in using skills gained from the class to perform specific tasks. Instructors should consider increasing students’ interest in and engagement with LOs by visually presenting them in a structured, practical, simple, and appealing form.

Reynolds, H.L., & Kearns, K.D. (2017). A planning tool for incorporating backward design, active learning, and authentic assessment in the college classroom. College Teaching, 65(1), 17-27. The authors report on outcomes from using a backward design-inspired lesson planner that facilitates LOs and aligns assessment and course activities in a non-majors biology course. After using LOs to establish priorities for class time and planning assessment, activities were designed and implemented to promote student-centered learning. This resulted in a shift from 90% lecture and 10% active learning to 45% lecture and 55% active learning. Student feedback was generally favorable and indicated that students appreciated having class time to interact with and discuss course material. Instructors can use this tool to prioritize and plan activities that facilitate active learning opportunities–allowing students to practice and demonstrate what they have learned.

Minbiole, J. (2016). Improving course coherence & assessment rigor: “Understanding by Design” in a nonmajors biology course. The American Biology Teacher, 78, 463-470. The author used the Understanding by Design framework to create explicit LOs that articulated the goals of nonmajors biology. Core ideas were identified and prioritized for instruction, and the amount of time spent lecturing was reduced by half. Classroom activities were designed to support student achievement of the LOs and build on their understanding of these core ideas and included activities, video clips, and discussions. Student discussions yielded opportunities for formative assessment that informed future instruction. Final exam grades increased after LOs were articulated, with significantly more students earning A’s or B’s. Instructors can use these outcomes to support efforts that shift class time from content delivery to creating opportunities for students to practice skills and content priorities established by instructional LOs.

Fata-Hartley, C. (2011). Resisting rote: The importance of active learning for all course learning objectives. The Journal of College Science Teaching, 40, 36-39. The author reports observations on the impact of student engagement through active learning activities to teach stated LOs, versus relying on a lecture-based approach with the expectation of rote memorization. A table is provided to illustrate a strategy for aligning assessments and classroom activities with the LOs. Active vs. passive instructional methods employed to teach the LOs were compared. Students performed significantly better on higher-level thinking items that assessed content taught using active engagement compared to lower-level items that assessed content presented by lecture requiring rote-memorization. Instructors should note that these results suggest better exam outcomes result from active learning activities that engage students with explicit LOs and encourage them to construct their understanding.

Kiser, S., Kayes, L. J., Baumgartner, E., Kruchten, A., & Stavrianeas, S. (2022). Statewide curricular alignment & learning outcomes for introductory biology: Using Vision & Change as a vehicle for collaboration. The American Biology Teacher, 84(3), 130-136. The authors present course-level LOs (CLOs) designed to (a) provide a framework for a one-year majors biology curriculum that increases the implementation of Vision and Change in Undergraduate Biology Education (V&C), (b) promote vertical articulation from introductory biology to upper-level courses, and (c) support articulation agreements between institutions. These CLOs were developed during a five-year collaboration between introductory and upper-division biology teaching faculty from two-year community colleges, regional comprehensives, private liberal arts colleges, and large R1 institutions across Oregon and Washington. The network participants engaged in a three-phase approach. In phase one, the focus was on building a network of instructors and defining the learning goals for introductory biology. During this phase, 53 participants engaged in a workshop with two main goals: 1. Defining the broad learning goals for one year of majors biology and 2. Develop aligned content modules emphasizing backward design, content depth rather than breadth, and competencies. In year five, ~70 participants met to focus on the importance of vertical articulation to transfer student success and collaboratively developed a set of CLOs. The CLOs were shared, edited, reviewed, and consented to by ~85 faculty members from 34 institutions. This project was associated with an increase in participants’ familiarity with V&C over time, and 92% of faculty agreed that developing the CLOs as a group was a valuable effort. This article provides biology instructors 33 CLOs that address 21 content-specific LOs and 12 competencies. Further, the collaborative approach to establishing a regional network to reach consensus on these CLOs is described.

Ezell, J.D., Lending, D., Dillon, T.W., May, J., Hurney, C.A., & Fulcher, K.H. (2019). Developing measurable cross-departmental learning objectives for requirements elicitation in an information systems curriculum. Journal of Information Systems Education, 30, 27-41. The authors summarize a departmental effort to create and align classroom instructional activities with the LOs for a key disciplinary skill within individual courses and across a modern information systems curriculum. The effort began by articulating the expectations for a graduate in the field, which then guided the development of an assessment rubric. This rubric was used to determine students’ baseline skills and measure the success of a curriculum redesign. The redesign began by developing LOs that reflected the thinking skills required for each of the six levels of Bloom’s taxonomy. Care was taken to produce well-written, clear LOs that would guide instructors in developing classroom activities and provide students clear guidance in assessing their learning progress. Learning objectives were then mapped to each existing core course in the curriculum to identify gaps. Faculty members took the lead in revising or creating instructional activities to help students master the revised and new LOs. The authors emphasize the need for a shared vision and identify the effort to establish LOs that reflect the field-expected criteria as a key to their success. The assessment rubric revealed a significant increase in student skill levels after the first year of implementation. Student response to the changes made was positive, with students reporting increased confidence in their abilities and skills. Instructors will find a detailed description of a faculty-led, team-focused effort that developed new, interconnected LOs across the curriculum, summarizing the process and lessons learned from the process.

Pepper, R.E., Chasteen, S.V., Pollock, S.J., & Perkins, K.K. (2012). Facilitating faculty conversations: Development of consensus learning goals. AIP Conference Proceedings, 1413, 291. The authors describe a department-wide effort to develop course-level and instructional LOs for four upper-division physics courses. They recommend: (a) scheduling a series of sessions and ending each on time, (b) engaging a person with disciplinary expertise as a facilitator, (c) recruiting faculty via endorsements from authority figures and personal contact, (d) allowing faculty to air grievances about courses or discuss appropriate topic coverage before moving on to developing LOs, and (e) sending out post-meeting summaries as well as a post-effort report indicating how faculty input was used. With this framework, faculty discussion moved beyond developing LOs to include curriculum alignment, student attitudes and conceptual difficulties, and pedagogical innovations. Instructors will find advice and experience on leading diverse faculty discussions to develop consensus LOs and course assessments. The authors outline: difficulties encountered, benefits of discussion, and other key elements for success to help instructors as they engage with colleagues to develop their own consensus LOs.

Richlin, L., & Cox, M.D. (2004). Developing scholarly teaching and the scholarship of teaching and learning through faculty learning communities. New Directions for Teaching and Learning, 97, 127-135. Faculty Learning Communities (FLCs) can support instructors’ development in the scholarship of teaching and learning (SoTL). A survey of 132 institutions with FLCs found that course redesign, course design, and teaching projects were the most commonly reported scholarly teaching activities. The ability to articulate LOs and assess student mastery of them is presented as integral to several stages in the development of a scholarly teacher. The FLC approach is analyzed and reported as a practical community support framework for advancing instructors’ teaching expertise. The authors provide instructors with evidence of the success of FLCs and how engaging with colleagues through FLCs can be used to support the development and use of LOs in SoTL projects and assessment of student learning.

Smith, G.A., Stark, A., & Sanchez, J. (2019). What does course design mean to college science and mathematics teachers? The Journal of College Science Teaching, 48, 81-91. The authors analyze how college math and science teachers view course design by interviewing nine instructors who were completing a 1-yr professional development program on STEM course redesign. This effort included revision and creation of new LOs, new formative and summative assessments, and the addition of new active learning activities. Peer interaction between teams facilitated feedback. Each of the instructors interviewed had taught their redesigned course. By coding semi-structured interviews, the researchers identified five levels in conceptualizing course design. Each represented a distinct level in instructors’ overall awareness and understanding of what it means to design a course, beginning with instructors’ awareness of what to teach and how to teach it, and progressing to the incorporation of reflective practice, the inclusion of departmental collaboration, and efforts to incur collective change across a discipline. This research can be used to inform and enhance course design. This analysis provides instructors with outcomes of a one-year professional development program where faculty worked in faculty learning communities (FLCs) to give and receive peer input in their course redesign efforts.

ASK BIO network regional workshops. (2021). ASK-BIO Assessment Skills in Biology. The Advancing Assessment Skills in Biology (ASK-BIO) Network is an NSF-supported initiative that supports Biology instructors in improving question writing and assessment development. Free regional workshops are offered throughout the US.