BLOG.ANNEDAVIES.COM
Issue 86: Research on Assessment OF Learning
22B
SUBJECT: Research on assessment OF learning
If you are trying to give students as much time as possible to learn, then you are likely engaged in discussions regarding how to best assess and evaluate the quality of student work in relation to the required standards or learning outcomes. If so, then you might be interested in examining a report published by the Assessment Reform Group that is focused on assessment OF learning.
It is based on four working papers that analyze relevant research. In summary, they clearly say that when teachers evaluate, they need to be able to make a professional judgment regarding the quality of student work. In order to do so in such a way that reliability and validity issues are taken care of, they need to engage in processes which involve working with colleagues and co-constructing shared criteria around quality expectations. Then they need to practice assessing student work, checking with colleagues to see if there is agreement. This is a moderation process that leads to teachers making “informed reliable and valid professional judgments” – and in fact, teacher professional judgments in this context have better reliability and validity than external tests. Here is a link to the report: http://www.assessment-reform-group.org/ASF booklet English.pdf
When one takes this perspective, then students who continue to improve their work are observed to be learning and adding to the evidence of learning that can be considered. Subsequently, when it comes to making a professional judgment, teachers will examine the entire collection of evidence. The work handed in by a student that was submitted *multiple times as part of a feedback cycle* will be viewed differently than the same assignment *without feedback.* Professional judgment frees educators from being slaves to scores, marks, grades, and numbers; instead, educators attend to quality expectations in relation to course standards or outcomes.
____________________________________________________
“Assessment and testing have a strong effect on the lives and careers of young people… When the results of tests and examinations are used to pass judgments on teachers and schools, they also affect the ways in which pupils are taught.”
~ ARG report on Assessment OF Learning
SUBJECT: Research on assessment OF learning
If you are trying to give students as much time as possible to learn, then you are likely engaged in discussions regarding how to best assess and evaluate the quality of student work in relation to the required standards or learning outcomes. If so, then you might be interested in examining a report published by the Assessment Reform Group that is focused on assessment OF learning.
It is based on four working papers that analyze relevant research. In summary, they clearly say that when teachers evaluate, they need to be able to make a professional judgment regarding the quality of student work. In order to do so in such a way that reliability and validity issues are taken care of, they need to engage in processes which involve working with colleagues and co-constructing shared criteria around quality expectations. Then they need to practice assessing student work, checking with colleagues to see if there is agreement. This is a moderation process that leads to teachers making “informed reliable and valid professional judgments” – and in fact, teacher professional judgments in this context have better reliability and validity than external tests. Here is a link to the report: http://www.assessment-reform-group.org/ASF booklet English.pdf
When one takes this perspective, then students who continue to improve their work are observed to be learning and adding to the evidence of learning that can be considered. Subsequently, when it comes to making a professional judgment, teachers will examine the entire collection of evidence. The work handed in by a student that was submitted *multiple times as part of a feedback cycle* will be viewed differently than the same assignment *without feedback.* Professional judgment frees educators from being slaves to scores, marks, grades, and numbers; instead, educators attend to quality expectations in relation to course standards or outcomes.
____________________________________________________
“Assessment and testing have a strong effect on the lives and careers of young people… When the results of tests and examinations are used to pass judgments on teachers and schools, they also affect the ways in which pupils are taught.”
~ ARG report on Assessment OF Learning
Issue 85: One Chance to Learn?
Starting with this issue, we will be emailing you our newsletters every second Tuesday, rather than weekly. We welcome feedback from our readers on whether you prefer this less frequent schedule.
In some discussions about formative and summative assessment, the controversy arises about giving students second chances to improve their assigned work. I think part of the challenge stems from the traditional practice of ‘assign and assess’ where teachers used to address the topic under study by giving a lecture, assigning some practice work and then evaluating students’ learning through a performance task or a test. In this world view, some people believe that it isn’t fair to let some students know what they’ve done wrong and then have a chance to try again to improve their marks, when others have achieved a good grade the first time.
Today, educators recognize that some students need more time than others to learn. Teaching involves assessing what students know, teaching to the needs that are evident through the assessment data, and then, after students have had time to learn, evaluating the evidence of learning. In order for all students to show what they know, it is often important to be flexible in terms of ‘what counts’ for evidence or proof of learning.
If you and your colleagues are still engaged in conversations about how to work with the numbers, marks, grades and scores related to assessing and evaluating student work, find Ken O’Connor’s work. His book, **How to Grade for Learning K–12, Third Edition** is a wonderful resource for conversations at this level.
More on this next week…
____________________________________________________
“Grades must be directly related to the learning goals for each grading period.”
~ Ken O’Connor
Check out “A Conversation about 21st Century Learning” by Dr. David Peat and Barry Allen, on our STORIES page:
http://www.annedavies.com/assessment_for_learning_stories.html
In some discussions about formative and summative assessment, the controversy arises about giving students second chances to improve their assigned work. I think part of the challenge stems from the traditional practice of ‘assign and assess’ where teachers used to address the topic under study by giving a lecture, assigning some practice work and then evaluating students’ learning through a performance task or a test. In this world view, some people believe that it isn’t fair to let some students know what they’ve done wrong and then have a chance to try again to improve their marks, when others have achieved a good grade the first time.
Today, educators recognize that some students need more time than others to learn. Teaching involves assessing what students know, teaching to the needs that are evident through the assessment data, and then, after students have had time to learn, evaluating the evidence of learning. In order for all students to show what they know, it is often important to be flexible in terms of ‘what counts’ for evidence or proof of learning.
If you and your colleagues are still engaged in conversations about how to work with the numbers, marks, grades and scores related to assessing and evaluating student work, find Ken O’Connor’s work. His book, **How to Grade for Learning K–12, Third Edition** is a wonderful resource for conversations at this level.
More on this next week…
____________________________________________________
“Grades must be directly related to the learning goals for each grading period.”
~ Ken O’Connor
Check out “A Conversation about 21st Century Learning” by Dr. David Peat and Barry Allen, on our STORIES page:
http://www.annedavies.com/assessment_for_learning_stories.html
Issue 84: We Evaluate What We Value
Systems define success in relation to their purpose and goals. If the measure of success is limited to only quantitative data, the destination may also become limited. What does your system value? What do you evaluate? Are the indicators of success appropriate and do they include multiple sources? How can you use assessment FOR learning to provide evidence of working towards or achieving your mission?
Remember, we **evaluate what we value.** If we collect simple, comparative numerical data, we communicate that we value that which can be measured simply and in numerical terms. We end up doing what Paul LeMahieu (1996) warns us against – we end up ACCOUNTING for learning rather than BEING ACCOUNTABLE for it.
Sometimes the push from leaders is to collect ‘objective’ data, yet all data is subjective – it just looks objective when it is in numerical form. When it comes to classroom assessment, leaders can support teachers to work towards better reliability and validity of their evaluations by encouraging them to collect evidence of learning from multiple sources over time. One way to do this is through *triangulation* – collecting evidence through making observations, having written, oral, or recorded conversations and collecting the products created as students learn. Then, when teachers come to evaluate and report, they can be confident in their judgment.
Sometimes educators find it hard to imagine using qualitative data as part of a summative grade. In this Assessment OF Learning video clip I give examples of how some teachers calculate percentage grades using both quantitative and qualitative data. You can view the video online at http://72.47.193.84/canada/wordpress/calculate-percentage-grading. I encourage you to think about how you might be able to include both qualitative data as well as quantitative data in your summative grades so as to report on all learning outcomes or standards, using more of the evidence of learning.
____________________________________________________
“Your true value depends entirely on what you are compared with.”
~ Bob Wells
TRANSFORMING BARRIERS TO ASSESSMENT FOR LEARNING is “jammed with examples” of how barriers are being transformed daily by leaders at all levels of education. Check it out at: http://connect2learning.com/store/publications/transforming-barriers-to-assessment-i-for-i-learning-lessons-learned-from-leaders
Remember, we **evaluate what we value.** If we collect simple, comparative numerical data, we communicate that we value that which can be measured simply and in numerical terms. We end up doing what Paul LeMahieu (1996) warns us against – we end up ACCOUNTING for learning rather than BEING ACCOUNTABLE for it.
Sometimes the push from leaders is to collect ‘objective’ data, yet all data is subjective – it just looks objective when it is in numerical form. When it comes to classroom assessment, leaders can support teachers to work towards better reliability and validity of their evaluations by encouraging them to collect evidence of learning from multiple sources over time. One way to do this is through *triangulation* – collecting evidence through making observations, having written, oral, or recorded conversations and collecting the products created as students learn. Then, when teachers come to evaluate and report, they can be confident in their judgment.
Sometimes educators find it hard to imagine using qualitative data as part of a summative grade. In this Assessment OF Learning video clip I give examples of how some teachers calculate percentage grades using both quantitative and qualitative data. You can view the video online at http://72.47.193.84/canada/wordpress/calculate-percentage-grading. I encourage you to think about how you might be able to include both qualitative data as well as quantitative data in your summative grades so as to report on all learning outcomes or standards, using more of the evidence of learning.
____________________________________________________
“Your true value depends entirely on what you are compared with.”
~ Bob Wells
TRANSFORMING BARRIERS TO ASSESSMENT FOR LEARNING is “jammed with examples” of how barriers are being transformed daily by leaders at all levels of education. Check it out at: http://connect2learning.com/store/publications/transforming-barriers-to-assessment-i-for-i-learning-lessons-learned-from-leaders
Response to Rubrics
Thank you to the recent blogs on rubrics. Initially, they can be difficult and many times the teacher will find it an exercise in word-smithing. Together with colleagues and students, the task of developing and refining rubrics does become easier. The dialogue that goes on will really help to clarify expectations and make them "friendly" to students.
Developmental continuums are wonderful tools and there has been great work done around them. One of the differences that I see is that at times, the descriptors in continuums may be what students exhibit at that stage. For example, when students are learning to write, "inventive spelling" may be part of the development continuum or "letters are strung together with no spacing". These provide descriptions of where that student is in relation to their development in writing. In rubrics, the language is more of quality. For example, a descriptor of quality on the trait of organization may read "The organization enhances and showcases the central idea or
storyline. The order, structure or presentation of information is
compelling and moves the reader through the text. " (Six Trait Rubric)
Other perspectives? Would love to hear other ideas.
Issue 83: Co-constructing Criteria
Co-constructing criteria with students is one of the most powerful facets of assessment FOR learning. By engaging them in the process of defining the learning goal and what success looks like, teachers are often amazed at how much MORE learning happens. Students begin to speak the language of assessment and take ownership of their own learning journey.
Educators of students with Special Needs are often experts at this process. They are used to working as a team with other professionals to identify the challenges of individual students and are very clear about each one’s specific learning targets. Their systems of keeping track of evidence of learning for these students are typically based on more qualitative data (using multiple sources over time) rather than relying on numerical data.
Many teachers model the language of assessment for their students with Special Needs by involving them in co-creating criteria to show what the learning will look like for each of them. Being able to communicate about what they have learned helps students become more confident and capable of learning on their own.
A group of educators in Ohio have seen such outstanding results from co-constructing criteria with their students (elementary, middle and high school) that they offered to share their stories with us. Visit the STORIES section on our website at http://www.annedavies.com/assessment_for_learning_stories.html and check out “Vignettes of Co-Constructing Criteria.”
____________________________________________________
“A guiding question for me is “What qualities and skills do I, as an educator, want our students to have?... Mostly, I want them to have a belief in themselves that they are capable of great things.”
~ Philip Divinsky
Phil Divinsky teaches high school students with Special Needs. His chapter, *Creating the Classroom Culture* describes his strategies for developing students’ communication skills so they get to know their peers and work together, sharing specific feedback for learning. Read more in CLASSROOM ASSESSMENT: WHAT’S WORKING IN HIGH SCHOOLS? at:
http://connect2learning.com/store/publications/classroom-assessment-what-s-working-in-higschools
Educators of students with Special Needs are often experts at this process. They are used to working as a team with other professionals to identify the challenges of individual students and are very clear about each one’s specific learning targets. Their systems of keeping track of evidence of learning for these students are typically based on more qualitative data (using multiple sources over time) rather than relying on numerical data.
Many teachers model the language of assessment for their students with Special Needs by involving them in co-creating criteria to show what the learning will look like for each of them. Being able to communicate about what they have learned helps students become more confident and capable of learning on their own.
A group of educators in Ohio have seen such outstanding results from co-constructing criteria with their students (elementary, middle and high school) that they offered to share their stories with us. Visit the STORIES section on our website at http://www.annedavies.com/assessment_for_learning_stories.html and check out “Vignettes of Co-Constructing Criteria.”
____________________________________________________
“A guiding question for me is “What qualities and skills do I, as an educator, want our students to have?... Mostly, I want them to have a belief in themselves that they are capable of great things.”
~ Philip Divinsky
Phil Divinsky teaches high school students with Special Needs. His chapter, *Creating the Classroom Culture* describes his strategies for developing students’ communication skills so they get to know their peers and work together, sharing specific feedback for learning. Read more in CLASSROOM ASSESSMENT: WHAT’S WORKING IN HIGH SCHOOLS? at:
http://connect2learning.com/store/publications/classroom-assessment-what-s-working-in-higschools
Issue 82: Rubrics
Most professional educators are sophisticated when using rubrics for evaluation, but in the ‘Assessment FOR Learning’ context, things can sometimes go wrong. In this little video clip from the Facilitator’s Guide, I talk about some of the ways to think about using rubrics in your classroom assessment practice: FG Rubrics video
After you take a look at the clip, you may consider setting up some rubrics for yourself. Here are some questions to guide rubric development from *The Facilitator’s Guide to Classroom Assessment K–12* at http://www.annedavies.com/PDF/21B_rubric_guide.pdf. I hope you find them helpful.
____________________________________________________
“If you don't ask the right questions, you don't get the right answers. A question asked in the right way often points to its own answer.”
~ Edward Hodnett
For more info about THE FACILITATOR’S GUIDE TO CLASSROOM ASSESSMENT K–12, go to: http://connect2learning.com/store/the-facilitator-s-guide-to-classroom-assessment-k-12
After you take a look at the clip, you may consider setting up some rubrics for yourself. Here are some questions to guide rubric development from *The Facilitator’s Guide to Classroom Assessment K–12* at http://www.annedavies.com/PDF/21B_rubric_guide.pdf. I hope you find them helpful.
____________________________________________________
“If you don't ask the right questions, you don't get the right answers. A question asked in the right way often points to its own answer.”
~ Edward Hodnett
For more info about THE FACILITATOR’S GUIDE TO CLASSROOM ASSESSMENT K–12, go to: http://connect2learning.com/store/the-facilitator-s-guide-to-classroom-assessment-k-12
Issue 81: Assessment & Technology
Over the past 100 years, our schools have changed dramatically to meetthe needs of the workplace. Historically, students were trained tofollow directions and tested with standardized methods that reflectedthat skill. Today’s students need to be flexible thinkers to take theirplace in the technologically advanced world. How has learningassessment evolved to keep pace?
In 2002, Maine took theunprecedented step of providing laptop computers to all the state’s 7thgrade students. In following years, they also provided computers to 8thgrade students and during the last few years have begun to implementthe program in high schools. “Assessment & Technology” is a videoprogram created in 2004 to explain the changes in classroom assessmentthat began to take place.
How can teachers assess studentlearning in a one-to-one computing environment? Does the use ofindividual computers have an effect on student assessment? Find outresponses to these questions and more when you view this video: http://72.47.193.84/canada/wordpress/assessment-technology
As you watch, consider the changes that have occurred since 2004 in technology-rich classrooms.
____________________________________________________
“Onlyif we expand and reformulate our view of what counts as human intellectwill we be able to devise more appropriate ways of assessing it andmore effective ways of educating it.”
~ Howard Gardner
ProfessionalLearning can happen online in your own home for far less cost and freeof the hassles of air travel! For information on Assessment FORLearning web-conferences and web-institutes, go to: http://annedavies.com/catalog2009/index.html
In 2002, Maine took theunprecedented step of providing laptop computers to all the state’s 7thgrade students. In following years, they also provided computers to 8thgrade students and during the last few years have begun to implementthe program in high schools. “Assessment & Technology” is a videoprogram created in 2004 to explain the changes in classroom assessmentthat began to take place.
How can teachers assess studentlearning in a one-to-one computing environment? Does the use ofindividual computers have an effect on student assessment? Find outresponses to these questions and more when you view this video: http://72.47.193.84/canada/wordpress/assessment-technology
As you watch, consider the changes that have occurred since 2004 in technology-rich classrooms.
____________________________________________________
“Onlyif we expand and reformulate our view of what counts as human intellectwill we be able to devise more appropriate ways of assessing it andmore effective ways of educating it.”
~ Howard Gardner
ProfessionalLearning can happen online in your own home for far less cost and freeof the hassles of air travel! For information on Assessment FORLearning web-conferences and web-institutes, go to: http://annedavies.com/catalog2009/index.html
Issue 80: What educators really need to know...
At the Symposium in New Zealand last year, a subgroup of 12 knowledgeable and informed assessment experts from around the world created a summary of what teachers and leaders need to know about Assessment FOR Learning. An excerpt from this Dunedin report appears below; the entire summary is available on our website at www.annedavies.com/pdf/20D_Dunedin_CPD_Team.pdf
The Essential Content for Teachers – Key Lessons to Learn:
** Understanding learning to learn – how to be a critical friend
** Understanding the inquiry process
** How to be clear about and differentiate assessment purposes
** Mastery of clear learning targets
** How to gather dependable evidence of learning
** Ways to analyze and understand what the evidence implies about needed learning
** Negotiating with the learner about how to close the gap
** How to use the assessment (inquiry) process and its results to support learning, including when and how to involve students
** How to communicate effectively in the assessment FOR learning process, including when and how to involve students
____________________________________________________
“On a group of theories one can found a school; but on a group of values one can found a culture, a civilization, a new way of living together among men.”
~ Ignazio Silone
There is a new option to consider for Professional Learning in July 2010:
The 5-day Institute on Assessment FOR Learning for teachers and leaders is an event that Connections Group and Anne Davies would like to bring to your School District. For details, go to:
http://connect2learning.com/store/events/leading-the-way-to-quality-assessment-practices
For more information, please call Lisa Robinson at 1-877-603-9888 or email lisa@connect2learning.com
The Essential Content for Teachers – Key Lessons to Learn:
** Understanding learning to learn – how to be a critical friend
** Understanding the inquiry process
** How to be clear about and differentiate assessment purposes
** Mastery of clear learning targets
** How to gather dependable evidence of learning
** Ways to analyze and understand what the evidence implies about needed learning
** Negotiating with the learner about how to close the gap
** How to use the assessment (inquiry) process and its results to support learning, including when and how to involve students
** How to communicate effectively in the assessment FOR learning process, including when and how to involve students
____________________________________________________
“On a group of theories one can found a school; but on a group of values one can found a culture, a civilization, a new way of living together among men.”
~ Ignazio Silone
There is a new option to consider for Professional Learning in July 2010:
The 5-day Institute on Assessment FOR Learning for teachers and leaders is an event that Connections Group and Anne Davies would like to bring to your School District. For details, go to:
http://connect2learning.com/store/events/leading-the-way-to-quality-assessment-practices
For more information, please call Lisa Robinson at 1-877-603-9888 or email lisa@connect2learning.com
Issue 79: Goals for the New Year
Welcome to 2010! Whew, where did that year go? Thank goodness we have markers, like the New Year, to prompt us to step back and take a break from this frantic pace of life and think about where we are in our goals and where we want to go over the next year. It’s interesting to reflect back on the plans we made last year at this time and give ourselves feedback on whether we met the criteria that we set for marking our steps to success. Some of us may need to adjust our goals to fit better with our present situation, but we can all celebrate our achievements, whether large or small.
I believe it is possible to create our life to be the way we want it to be. We just need the courage and commitment to begin planning for success, both in our own lives and in our teaching practice. Goethe said, “Whatever you can do or dream, begin it. Boldness has genius, power, and magic in it. Begin it now.”
As a first step in helping your students plan for a new outcome (or for helping us to achieve our own goals), we suggest you try an idea called ‘Goal Envelopes’ from CONFERENCING AND REPORTING (pages 17-20). An outline is available for the first part of the exercise and a template; samples and adaptations are included in the book. They can be viewed at www.annedavies.com/pdf/20C_GoalEnvelopes.pdf and www.annedavies.com/pdf/20C_p58_Conferencing&Reporting.pdf
____________________________________________________
“The future belongs to those who believe in the beauty of their dreams.”
~ Eleanor Roosevelt
CONFERENCING AND REPORTING is part of the three-book series called ‘Knowing What Counts’ which is available at:
http://connect2learning.com/store/publications/knowing-what-counts-series-3-books
I believe it is possible to create our life to be the way we want it to be. We just need the courage and commitment to begin planning for success, both in our own lives and in our teaching practice. Goethe said, “Whatever you can do or dream, begin it. Boldness has genius, power, and magic in it. Begin it now.”
As a first step in helping your students plan for a new outcome (or for helping us to achieve our own goals), we suggest you try an idea called ‘Goal Envelopes’ from CONFERENCING AND REPORTING (pages 17-20). An outline is available for the first part of the exercise and a template; samples and adaptations are included in the book. They can be viewed at www.annedavies.com/pdf/20C_GoalEnvelopes.pdf and www.annedavies.com/pdf/20C_p58_Conferencing&Reporting.pdf
____________________________________________________
“The future belongs to those who believe in the beauty of their dreams.”
~ Eleanor Roosevelt
CONFERENCING AND REPORTING is part of the three-book series called ‘Knowing What Counts’ which is available at:
http://connect2learning.com/store/publications/knowing-what-counts-series-3-books
Issue 78: The Gift of Time
Many years ago, I started looking for a way to simplify my life. Taking some clues from the writing of Stephen Covey and what I knew from my work with strategic planning, I began to review my goals and establish a picture of my life for myself.
I reviewed all the things I did that gave me pleasure and contributed to my quality of life and the lives of others: my family, my community, and my profession. I considered what my talents and gifts were and how they best found expression. I looked at those parts of my life that gave me little or no pleasure. I decided I wanted time to do the things that were of value to me. I made some decisions…
Now I try to use my newly-created time to do the things I value. I have more time to write – an activity I value. I have more time to talk with other educators about issues of concern and challenge to me – an activity I value. I have more time to be with my family – an integral part of the gift of life and a great value to me. My life is not perfect, but it does make more sense to me. When I stepped back and reflected on my life, these were some of the choices I made for me. They have given me the gift of time. What about you? How can you give yourself the give of time?
(Read more on this in my article called Gift of Time: Stepping Back and Making Sense at www.annedavies.com/images/PDFs/gift_of_time.pdf
____________________________________________________
“Regret for the things we did can be tempered by time; it is regret for the things we did not do that is inconsolable.”
~ Sidney J. Harris
“What we desperately need in business is reflection. Not more planning. Not more strategizing. More reflection.”
~ John Dalla Costa
Anne Davies has begun to share some of her tranquil moments through her paintings and writing. Find out more at:
http://connect2learning.com/store/publications/art-reflection
I reviewed all the things I did that gave me pleasure and contributed to my quality of life and the lives of others: my family, my community, and my profession. I considered what my talents and gifts were and how they best found expression. I looked at those parts of my life that gave me little or no pleasure. I decided I wanted time to do the things that were of value to me. I made some decisions…
Now I try to use my newly-created time to do the things I value. I have more time to write – an activity I value. I have more time to talk with other educators about issues of concern and challenge to me – an activity I value. I have more time to be with my family – an integral part of the gift of life and a great value to me. My life is not perfect, but it does make more sense to me. When I stepped back and reflected on my life, these were some of the choices I made for me. They have given me the gift of time. What about you? How can you give yourself the give of time?
(Read more on this in my article called Gift of Time: Stepping Back and Making Sense at www.annedavies.com/images/PDFs/gift_of_time.pdf
____________________________________________________
“Regret for the things we did can be tempered by time; it is regret for the things we did not do that is inconsolable.”
~ Sidney J. Harris
“What we desperately need in business is reflection. Not more planning. Not more strategizing. More reflection.”
~ John Dalla Costa
Anne Davies has begun to share some of her tranquil moments through her paintings and writing. Find out more at:
http://connect2learning.com/store/publications/art-reflection
