Tag: Formative Feedback

Applying Formative Assessment in Professional Development

In these past few weeks, I have been exploring professional development (PD) models that optimize adult learning. The primary focus of these posts has been on the characteristics of adult learning and various professional development formats that honor these characteristics.  While understanding these models is important so that participants gain the most out of their professional development, in this post I’d like to focus on applying these concepts to incorporate content, exploring educational technology best practices described in the ISTE coaching standard 4b:  “Design, develop, and implement technology rich professional learning programs that model principles of adult learning and promote digital age best practices in teaching, learning, and assessment,” (ISTE, 2017).

In investigating digital age best practices, formative assessment appeared as a reoccurring theme. Formative assessment as part of a feedback loops empowers learners to engage in the trial and error of learning safely and with minimal risk. Applying formative assessment to professional development could offer similar results. In applying this idea to the ISTE standard, I began wondering what digital tools could be implemented to teach teachers about the importance of formative feedback?

What is “Formative Assessment” and why is it a best practice?

Feedback loops are often used as a teaching best practice in aiding students build 21st century skills. As described in other posts in this blog, of the four different types of assessment, traditional, or summative, assessment measures learning after an assignment has been turned in.  Summative evaluation assumes that a student has “learned” after an intervention (such as teaching) and the educator evaluates the extent of that learning, (Vlad-Ortiz, 2018). While summative assessment is useful for formal evaluation, it may not be timely nor help students improve if only offered as one-time feedback, (Vlad-Ortiz, 2018). Where summative assessment is formal and final, formative assessment is more casual and on-going as the evaluation occurs during the learning, (Vlad-Ortiz, 2018). Formative assessment therefore provides a checkpoint for student understanding, (Office of Educational Technology, n.d.)

I explore the benefits of feedback loops for students in this post, I’d like to expand the investigation to including formative feedback as a tool in adult learning.  The Office of Educational Technology found that formative feedback when coupled with technology tools may be more complete than traditional assessment and may “reduce time, resources, and disruption” to conduct the assessment, (Office of Educational Technology, n.d.) These benefits help educators as formative assessment may provide an avenue for capturing teaching qualities that open opportunities for “self-reflection, peer reflection, feedback, and supervisor evaluation,” (Office of Educational Technology, n.d.). Extending these concepts further, formative assessment can be used in professional development as a means to inform instructional practice where participants track their own learning, (Office of Educational Technology, n.d.). This means that meaningful evaluation can occur more rapidly and frequently, offer more insight, and help guide professional development needs.

Tech tools that can be used for formative assessment.

There are several educational technology tools that can be used for formative assessment. Common Sense Education created a list of the top 27 tools for formative assessment available here.  These formative feedback tools include the following features: student progress tracking, interactive and collaborative activities, student-paced learning, and instant feedback to both students and teacher. Formative feedback is given by utilizing interactive slideshow presentations, video responses, multi-multimedia platforms, content-mapping, quizzes (including clickers and polling), and backchannel chats. In creating the list, Common Sense Education agrees with the Office of Educational Technology stating that the best formative assessment tools help students (and participants in this case) self-reflect and assess so that they understand their current level of learning and self-identify areas of improvement, (Common Sense Education, n.d.).

Integrating formative assessment into professional development.

Incorporating formative assessment in adult learning must assume that participants are learners who are joining the professional development for a variety of different motives that are relevant to their work situations. Though are quite a few professional development resources available on the internet on formative feedback tools, I’d like to use this professional development video I found through YouTube entitled, “10Tips for Formative Assessment with Technology: Meaningful, Sustainable, & Scalable” as an example. In the video Dr. Monica Burns walks participants through her tips by highlighting main features and how to use some formative feedback tools.  A summary of her tips is provided in figure 1.1 below.

infographic on tips for incoporating technology tools with formative assessment.
Figure 1.1. Tips for Formative Assessment with Technology

Though the video is purely informational as Dr. Burns lectures for about 30 minutes on her ten tips, this could be a useful resource for participants that are highly motivated. The professional development model used assumes that the participants already have an awareness of formative assessment and simply need guidance or ideas on how to implement this in their teaching practice.

According to the ISTE standard, best practices for the effective PD includes modeling, (ISTE, 2017). While the workshop above may model ways to use each tool through verbal and visual description, it fails to include participant buy-in and interaction. Formative feedback could have been included into the professional development itself, allowing participants an opportunity to experience instant feedback through the lens of a learner. For example, demonstrating how to gauge comprehension to better understand the audience’s needs could have been accomplished by using a backchannel chat or using the polling/quizzes apps described in the video.  This tangible and experiential approach could help increase self-efficacy of technology tools for mixed audiences where the presenter modifies their role to facilitation at certain periods of the professional development.  When presenters start thinking about their participants as learners, professional development becomes stronger, more impactful which can yield better improvements in teaching and learning.

References

Common Sense Education, (n.d.) Top tech tools for formative assessment. Available from: https://www.commonsense.org/education/top-picks/top-tech-tools-for-formative-assessment

Office of Educational Technology, (n.d.) Section 4: Measuring for Learning. Available from: https://tech.ed.gov/netp/assessment/

Vlad-Ortiz, C. (2018). Incorporating feedback loops to develop an empowered student [blog]. Available from: http://digitaleducationblog-cvo.org/incorporating-feedback-loops-to-develop-an-empowered-student/

Vlad-Ortiz, C. (2018). Instructional coaching: Using rubrics to quantify qualitative data for improved teaching outcomes.
Available from: http://digitaleducationblog-cvo.org/instructional-coaching-using-rubrics-to-quantify-qualitative-data-for-improved-teaching-outcomes/

Lessons from the Six Facets of Understanding and Backward Design Process

For the past ten weeks, my cohort and I have been exploring techniques to get more out of the classes we teach.  I have been personally exploring teaching methods that truly achieve student understanding. Interestingly, authors of the book, Understanding by Design, argue that our interpretation of the word “understanding” is narrow and doesn’t encompass the word’s full translation.  In my field of higher education, academic application of “understanding” typically means the “ability to explain”. Students who can explain demonstrate their understanding through academic performance such as achieving high test scores or through products such as essays, where they explain how things work, what they imply, and how the concepts are connected, (Wiggins & McTighe, 2005).  While this skill is important, we shouldn’t rely solely on explanation to demonstrate whether or not students are understanding, as we could potentially deemphasize the other meanings that hold an equal value, (Wiggins & McTighe, 2005). In fact, there are six facets of understanding which are highlighted in figure 1.1 below.

Infographic of Understanding by Design's six facets of understanding.
Figure 1.1 The Six Facets of Understanding from Understanding by Design.

One of the best practices for accomplishing student understanding (in one or multiple facets) is to lesson plan using the “backward design” approach. In this approach, educators are encouraged to look at their objectives, identify what they want students to learn and accomplish, then design a lesson plan that achieves those goals.  This lesson planning begins by first reviewing and refining objectives and/or learning outcomes. By establishing the lesson plan objectives early on, it ensures that the ultimate mission of the class is clearly defined. In other words, the objectives help set the destination of the lesson.  This step is followed by developing how these objectives/outcomes will be evaluated, setting the road map  for the learning journey.  Lastly, the actual plan with the learning activities is designed ensuring that the objectives are appropriately met, this will where the journey begins.  Figure 1.2 explores the backward design process from Understanding by Design more in-depth.

Figure describing the backward design process.
Figure 1.2 Understanding by Design’s Backward Design Process.

Implementing Backward Design

In our case, it wasn’t enough to understand what backward design is through explanation alone, our cohort was challenged to interpret and apply this design method.  We were given the option of designing a new lesson that we would use in the future, or choose an existing lesson to improve. I chose to focus on a unit from a project-based class I teach, whose main focus is mastering scientific writing while also developing research skills.  The ultimate assessment item of this unit is a final draft of the “Introduction” and “Methodology” sections of the research paper. This assessment focuses on appropriately and expertly incoportating components necessary to set the purpose and procedure of the research project.

Lesson Background. Before reaching this assessment, there are several steps that the students must accomplish.  By the time they turn in the final intro and methods draft, the students have already picked their research food (the topic of the research project and paper), created their hypothesis(es), designed their experiment, and are conducting several experiments a week. In order to successfully craft their experiment, they should have prepared a good annotated bibliography, which is the basis for the introductory section of the paper.  

In this introductory section, students develop a mini literature review exploring the properties and potential outcomes of their foods. Students understand that they are showcasing the work and results of other researchers, what literature is missing, and how their experiment contributes to the body of literature. The final paragraph introduces their experiment along with their hypothesis(es).

The methodology section of the paper is a brief, yet descriptive, mention of the procedure for producing the research food, its variations (typically students choose 2 variations), and other relevant how-to details of their experiment. The idea behind these few paragraphs is that anyone should be able to pick up their paper and clearly understand how to reproduce their experiment.

The Challenge. Historically, students struggle with the concept of a “final” draft, submitting for formal evaluation something that resembles a paper closer to a first rough draft. Students are then disappointed by their low assessment scores.

From the professor’s perspective, this assignment is frustrating to grade and disappointing to see the low quality effort from students. Despite the fact that students take an entire class dedicated to research writing prior to this class, it is evident that they have not mastered it.  In particular, they struggle with the content of these two sections. The two most common comments made in their writing is that some sections have far too much “fluff” or unnecessary explanation while other sections are too vague or lack clarity. They have a hard time writing concisely but descriptively.

From the student’s perspective (based on course evaluations and face-to-face feedback) the assignment is hard, they need more instruction on the writing process, and they have a misunderstanding of what the term “final draft” means. Students always comment that the writing portion is the most frustrating component of the course.

Students are not motivated to practice writing skills on their own though they are encouraged to write several drafts prior to the final draft due date. To help understand what content should be included, students  examine examples of scientific writing by identifying the necessary components of the intro and methods sections. Students become very good at identifying these pieces yet still struggle to apply them to their own work. This is likely because most students wait to write their first rough draft the night before the final draft is due, are not familiar with the proper draft writing process, or underestimate the difficulty of scientific writing and do not seek outside assistance. 

Revising the lesson. In an effort to resolve frustration from both the professor’s and student perspectives, my mission is to find simple, actionable solutions to address the issues present above. I would like to see students moving away from frustration to feeling challenged and having the intrinsic motivation to practice becoming great scientific writers.  One possible solution is making this draft process more collaborative. Since students become very good at identifying necessary components in the works of others, by providing more peer and instructor formative feedback, any clarity issues and missing content would be identified earlier. Students would also be encouraged to review their own work more frequently using the RISE model, addressing the issue of last-minute drafts.

By incorporating more collaboration, this provides an opportunity to focus on building digital citizenship.  In particular, I wish to address the ISTE student standard of digital citizenship that “develops safe, legal, and ethical behavior” when using technology by allowing students to write their drafts using a Google Doc collaboration, (ISTE, 2017).  Another way to implement this standard is through the curation process leading to the annotated bibliography using the web app, Diigo.  A second aspect of the digital citizenship standard I wish to address is “responsibly using and sharing intellectual property”, (ISTE, 2017).  Students will encounter this at various aspects of the class as they will rely heavily off of the works of others.

By working backwards to design a solution, I realized that all of the challenges faced by students in writing the final draft was actually pretty easy to overcome once I had all of the right tools and techniques.  My solution did involve significant re-arranging of existing helpful class topics, removal of unhelpful topics, and implementation of topics that previous students had identified as missing. Figure 1.3 summarizes the unit lesson planning with the new topics highlighted in bolded, yellow font.

Chart depicting a summary of the intro and methods unit learning and teaching activities.
Figure 1.3. Summary of the Intro and Methods Unit Learning and Teaching Activities.

As depicted by Figure 1.3 above, the concept of digital citizenship is introduced through an online literature curation process in which the students collect, organize, and annotate relevant research articles.   This new assignment is a spin-off of an existing assessment, annotated bibliography, that allows students not only to cultivate new skills, but provide a helpful tool to better capture information from the articles they read. Students are still required to submit an annotated bibliography but the artifact has been changed to include self-reflection.

The biggest change in this unit is the introduction of the three-step formative feedback process using the RISE model where students undergo peer, self, and instructor feedback.  Through this new process, it will help students write multiple drafts prior to the submission of the final draft. Sharing their work and thoughts are made simpler through the use of Google Docs.  This new collaboration effort allows students to work together and share their expertise to gain a better understanding of the draft writing process.

Final Thoughts on the Backward Design Process.

Wiggins and McTighe admit that is it difficult to follow this design process step by step without fighting the desire to skip to the next step or write one area with another in mind, (Wiggins & McTighe, 2005).  This was the case for me. The objectives and the evaluation criteria were clear as they were based off of accredited standards and those featured elements of scientific writing. The challenge existed in the preparation steps necessary to help students achieve those objectives. However, the most illuminating moment was the emphasis on the evaluation process.  By taking a closer look at my unit planning and through considerable reflection, I had realized that there were missing components that were not setting up my students to achieve the desired outcomes. It was like I had the the destination in mind, I knew the road I needed to take but I forgot which vehicle was going to get me there most efficiently.  Though I did fight the urge to jump straight into lesson planning, the backward design process helped remind me of what was important for this unit and better equipped me to  address the existing problems that I was previously unsure how to solve.

What I’ve also learned to appreciate is that as an educator, you are never quite done with this process.  One benefit that I had as I was revising my unit planning was the previous feedback I received from my students.  If they hadn’t voiced their frustrations in a constructive way, I wouldn’t have been able to address these issues so specifically. I didn’t need to reinvent the wheel, but rather just fix the small area that was not working. Thanks to their feedback, my design process was streamlined and poignant. As I gear up to implement these changes in the upcoming quarters, I look forward to the improved successes of my students while also being cognisant of the fact that I will, at some point, need to revisit the backward design process and make small yet significant changes again.

References

International Society for Technology in Education, (2017).  The ISTE standards for students. Retrieved from: https://www.iste.org/standards/for-students.

Wiggins, G., & McTighe, Jay. (2005). Understanding by design (Expanded 2nd ed., Gale virtual reference library). Alexandria, VA: Association for Supervision and Curriculum Development.

Incorporating Feedback Loops to Develop An Empowered Student

Being a successful professor means preparing students to be successful. Delivering knowledge-centered classes on a particular topic is no longer the primary task of professors. Gone are the days of the large lecture halls, professor front and center, exhibiting knowledge for students to somehow absorb.  Scholars are now calling for students and professors to engage in a new learning paradigm that provokes the development of specific skills for the 21st century.  This paradigm includes teaching five major career skills that are highly sought after by employers today.  Mastering these five essential skills means that students: 1) thrive on change by being receptive to feedback, 2) are able to get things done independently and without direction, 3) are open-minded, understand their own biases, and appreciate differences in others, 4) know how to prioritize tasks, and are good at influencing behavior of others, 5) facilitate activities and relationships within an organization, (Kivunja, 2014).  This is not an easy feat as skills need time and practice to be cultivated. The first ISTE standard for students calls for the empowered learner as a mechanism to help build 21st century skills.  The empowered learner is one that, “…leverages technology to take an active role in choosing, achieving, and demonstrating competence in their learning goals,” (ISTE, 2017). An empowered student is one that is at the forefront of their learning by thinking beyond the lecture and is autonomous because they have intrinsic motivation, (Stefanou et. al., 2004).  

Figure 1.1 Empowered Student Flowchart

So if students need to develop self-determination and become autonomous in order to thrive in the current workforce, are we, as educators, doing our part in preparing them to do so?  This question can only be answered positively if we adopt a student-centered approach to teaching.  The authors of the book, Understanding by Design, challenge educators to consider the backward design approach. In this design approach, the educator starts their plan with the desired results, determines which indicators are appropriate for measuring the outcomes of their results, then plans the experiences and/or instruction required to achieve these outcomes, (Wiggins & McTighe, 2005).  When students are informed of the desired results and are allowed to take part in the creation process, that’s when self-determination and autonomy develops, (Stefanou et. al, 2004).

It is also important to remember that students are still developing these skills so simply stating the purpose or goal of an assignment and leaving them to their own devices will not help them develop autonomy.  Coupled with the student-centered approach, formative feedback must be included to help guide and remind students of the big-picture results.  Formative assessment when conducted as a feedback loop helps to “enhance performance and achievement,” (Wiggins, 2012).  Essentially, this means that students are given consistent, on-going, and immediate feedback as a way to encourage continual practice of skills.  Formative feedback is not evaluated formally (i.e. no grades are assigned to the feedback) and does not offer extensive evaluation, advice, nor it is purely praise.  Instead, formative feedback offers the student a “gauge of their efforts to reach a goal”, (Wiggins, 2012).  In order to provide good feedback, the assessor must first observe, then comment or ask questions on those observations, (Wiggins, 2012). Figure 1.2 summarizes Wiggin’s strategy on formative feedback.

Figure 1.2

Putting the Theory Into Practice: The Investigation.

In our digital education leadership program, we were asked to create a question(s) related to the classes we teach and investigate a resource(s) that would aid in addressing the first ISTE standard for students.  I teach a nutrition research class whose main purpose is to develop not only students’ research skills but also build autonomy as researchers. Students must  investigate a food-related issue, then design and implement an experiment, later report their findings through a final research paper. This class explores the research process including hypothesis creation, experiment -building and -testing, and scientific writing.  The current challenge is to allow enough freedom for autonomy to develop while providing  direction to ensure correct research protocol is established.  

I began my brainstorming process for a student-centered approach to the issue by first identifying the important design outcomes. I started with a goal: Allow students to take their research project into their own hands while working toward a common goal and using the research protocol. Though students will be developing autonomy and need to be self-driven, they will also need appropriate feedback in order to gauge their work at critical points in the quarter. With this goal in mind, two main questions developed: 1) What feedback timeline would be most effective to design a researcher-centered approach to teaching nutrition research classes? and 2) What computer driven-tools would effectively provide timely and ongoing feedback?  The findings of my investigation and potential resources are explored below.

Question 1: What feedback timeline would be most effective to design a researcher-centered approach to teaching nutrition research classes? Upon further investigation, this question can’t be answered directly. Each assessment will vary in scope and length, therefore a prescribed timeline is not feasible. However, according to education leaders Hicks and Wiggins, they both agree that formative feedback is the best approach using the student-centered or researcher-centered approach.  As a reminder, formative feedback is not formally assessed but rather allows the student/researcher an opportunity to take a step back to evaluate and reflect upon their own work in relation to their research goals. The timing of feedback should be immediate, ongoing, and consistent,(Hicks 2014, Wiggins 2012).  Feedback should follow a specific format which does not make judgements nor evaluates the work.  Hicks references the RISE model (see figure 1.3) to format formative feedback in a meaningful way, which is why I’ve chosen the model as the resource of choice for this question.

Figure 1.3

The RISE model can be used for self-assessment, peer-review, or evaluator review in formative feedback.  The process begins by assessing the degree to which the current work meets the goals/objectives of the assignment.  The subsequent steps allow for specific, tangible, and actionable suggestions to the author for improvements on their current version and future version of the work. The benefit of using this model is that as the feedback advances towards higher steps, it also involves higher level of thinking. RISE allows the user to get at the heart of student-centered learning by allowing students to evaluate and create works. I have not used this model in action but my predictions for any drawbacks may involve peer-feedback where students skip a level or provide judgements without fully understanding the model itself.  These concerns could be combated with scaffolding and more detailed instruction on the feedback process.

Question 2: What computer-driven tools would effectively provide timely and ongoing feedback? For an assessment item such as a research paper, using a collaboration tool such as G suite or the Google Doc Collaboration feature in CANVAS is ideal.  Google Docs are available to anyone that holds a gmail sign-in, along with several other features of the G suite including: to-do lists, calendar, google hangout, and gchat, just to name a few.  The Google Doc collaboration feature in CANVAS allows students to access a google doc on one google drive (usually belonging to the instructor).  The owner of the google drive would then have access to all of the collaboration pages for the class. The use of these collaboration tools is appealing because the docs are easily accessible by students, the professor, or individual providing the feedback.  Formative feedback is simple to provide using the “comment” feature. Google Docs also track changes throughout the life of the document and provides comment notifications in gmail. Using Google Docs would also help address issues related to equality of work among team members (i.e. members doing their fair share of the collaboration). To further my justification of this technology, it would help me improve my current assignment by achieving M and R from the SAMR model.  Google Doc collaboration also scores roughly a 14 on the Triple E rubric (according to my assessment of intended use).

The only downside related to the collaboration tool feature in CANVAS. The feature is not intuitive and somewhat difficult for students to access. It is also not well integrated with Google Docs, for example, simply placing students into groups on CANVAS and assigning these groups to a Google Doc collaboration does not automatically give students access to their group’s Google Doc in the drive.  The instructor has to manually give permission to each student. The collaboration feature also does not link instantly to the gradebook or back to CANVAS where other course materials/resources would be kept.

The Next Steps.

The RISE model and Google Doc tool were well received by my colleagues when evaluating them as resources that resolve my two questions on formative feedback. Not surprisingly, others also shared similar concerns with using Google Doc as a collaboration feature in CANVAS. Since Google Docs can be used independently of CANVAS, this is not a big issue particularly since formative feedback is not associated with a formal grade therefore an association with CANVAS materials or gradebook is not necessary.

Interestingly, most of their feedback on these two resources related to implementation, namely what assessment tools would/could be used to implement the RISE model and would/could Google Apps for Education help facilitate this assessment function? My initial reaction on creating an assessment tool to implement the RISE model was to create “guiding questions” students would answer as part of their feedback comments.  By answering the questions fully, the students would effectively go through the entire model without skipping steps. I have yet to investigate other Google Apps for Education for feedback features.  Though I do not have complete answers to these great questions, I do have the beginning of of my next investigation: Feedback Implementation.

References

Hicks, T. (2014, October 14). Make it count: Providing feedback as formative assessment. Edutopia. Retrieved from: https://www.edutopia.org/blog/providing-feedback-as-formative-assessment-troy-hicks

International Society for Technology in Education, (2017).  The ISTE standards for students. Retrieved from: https://www.iste.org/standards/for-students.

(Kivunja, C. (2014). Teaching students to learn and to work well with 21st century skills: Unpacking the career and life skills domain of the new learning paradigm. International Journal of Higher Education, 4(1), p1. Retrieved from http://files.eric.ed.gov/fulltext/EJ1060566.pdf

Stefanou, Candice R., Perencevich, Kathleen C., DiCintio, Matthew, & Turner, Julianne C. (2004). Supporting Autonomy in the Classroom: Ways Teachers Encourage Student Decision Making and Ownership. Educational Psychologist, 39(2), 97-110.

Wiggins, G., & McTighe, Jay. (2005). Understanding by design (Expanded 2nd ed., Gale virtual reference library). Alexandria, VA: Association for Supervision and Curriculum Development.

Wiggins, G. (2012, September). 7 keys to effective feedback. Education Leadership. 70 (1).

Wray, E. (2018). RISE Model. Retrieved from: http://www.emilywray.com/rise-model.

css.php