RSVP 

The “ARC” of Assessment Design to Foster Student Participation

Michael D. Smith, PhD 
John H. Finley, Jr. Professor of Engineering and Applied Sciences; Senior Fellow of the Society of Fellows, ex officio; Harvard University Distinguished Service Professor
LInc Faculty Fellow: 2020-2021

H. Deniz Marti, PhD
LInc Pedagogical Fellow
Harvard John A. Paulson School of Engineering and Applied Sciences

Overview: 

In the spring of 2020, COVID-19 and the sudden movement to online instruction forced many educators to consider whether the pedagogical methods and assessments that had worked in their in-person classrooms would work as effectively online. For us, our primary concern was to find ways to encourage and measure student participation. Historically, we had simply told the students we expected them to participate and then relied heavily on our own observations of the students’ engagement in class and with the course materials. As we moved online, this approach fell apart. A student could be deeply engaged and yet have their camera turned off. Another student may never raise their virtual hand, but would use the chat facility to make insightful comments on the material. It quickly became clear that our traditional norms around student participation were not valid in this online paradigm.

While our concerns about assessment began with participation, we quickly realized that what we were facing was a more general challenge: How do we motivate students to engage deeply with the material and lean into the types of classroom activities that will produce meaningful and long-lasting intellectual growth? This led us to investigate the principles upon which real, deep learning is based and create a theoretically-motivated framework for assessment design, which we call ARC. It stands for Autonomy, Reflection, and Community, and we used it to create an assessment approach that fostered student participation. With ARC, we created an environment where students are given the opportunity to take ownership of their learning, reflect on their participation, and feel motivated to collaborate, rather than compete, in a community of learners. In this talk, we will describe the ARC framework and our experience applying it to participation in a master’s level, discussion-based data science class in Spring 2021. We will also demonstrate how the ARC framework might be used in a different type of course. We will leave time toward the end of the session for the audience to explore how they might apply ARC to design their own assessments and guide their students toward intellectual growth.

Biographies: 

Michael D. Smith is the John H. Finley, Jr. Professor of Engineering and Applied Sciences and a Distinguished Service Professor at Harvard University. He spent 11 years as the Edgerley Family Dean of the Faculty of Arts and Sciences, leading Harvard’s oldest and largest school. His leadership was characterized by a deep commitment to undergraduate teaching, to faculty development, and to research, both fundamental and applied. He is delighted to be back in the classroom. He is currently developing new in-person and online classes, and writing a book on teaching.

Mike sits on the Board of Trustees of The Peddie School in Hightstown, New Jersey, and he is an advisor to 1984 Ventures, GSV Ventures, and Xfund. He was actively involved in Harvard and MIT’s launch of edX, and he served on its board from 2012-2018. He enjoys exploring the interplay of technology with other fields, from the life sciences to business to education.

Earlier in his career, Mike spent time in industry building a range of computing hardware for Honeywell Information Systems, and in 2001, he co-founded the data security company Liquid Machines, which was acquired in 2010 by Check Point Software Technologies. While at Harvard, he received a prestigious National Science Foundation Young Investigator Award, the Alpha Iota Prize for Excellence in Teaching, and the W.E.B. Du Bois Medal.

Mike received his Ph.D. from Stanford University in 1993. He has a Masters from Worcester Polytechnic Institute and a Bachelors from Princeton University. He is a member of the ACM and Senior Member of the IEEE.

Deniz Marti is a Postdoctoral Fellow with Harvard SEAS Learning Incubator (LInc). She works to improve learning environments in STEM education for both faculty and students alike. She focuses on evidence-based research to explore expertise development, online pedagogy, and active learning techniques, specific to STEM education.

Deniz completed her undergraduate degree in Industrial Engineering at Bogazici University, graduating with Dean’s Honor, and completed her Ph.D. degree in Systems Engineering at George Washington University. During her doctoral studies, she explored the intersection between engineering and cognitive psychology by taking an interdisciplinary approach that incorporated social science research into engineering solutions. She specialized in specific research tools such as experimental design, survey development, and expert elicitation thereby using these tools to examine underlying mechanisms of experts’ cognitive decision-making processes in STEM. In particular, she studied the association between the cognition of expert judgments and the development of expertise. Her doctoral work also disentangled two seemingly similar yet distinct cognitive phenomena, information transfer and expertise transfer, in various decision-making contexts. The implications of her research include improving pedagogical tools that enhance students’ decision making, creating learning processes that facilitate expertise development, and designing curricula that experts (educators) can use to effectively transfer knowledge to novices (students) in STEM.

Deniz is deeply interested in pedagogy, specifically in STEM learning environments. By drawing upon her extensive research experience in cognitive processes in engineering systems, she aims to understand more deeply how STEM students impart knowledge and skills most effectively, use them in solving complex problems, and eventually develop expertise. Deniz’s goal is to work on creating STEM learning and teaching platforms that are adaptable to an ever-changing world so that new generations can rapidly and efficiently address the needs of technology-driven venues. Deniz's future work trajectory focuses on using and developing novel research-based pedagogical methods that assess, evaluate, and advance STEM education.