"Who am I assessing?": The problem of authenticity in remote instruction

In all its forms, and on any given day of regular teaching, assessment is complex. I’m on family leave so didn’t teach through the current interruptions to regular schooling, but I suspect that the shift to remote instruction only magnified many complexities of assessment for teachers.

Over the coming months, schools will no doubt find useful ways to “capture the upsides” from the period of remote teaching, but we should also pause to interrogate the areas of practice that may have suffered; there is a lot to be learnt from the downsides, too.

In partnership with the Victorian Department of Education, I recently facilitated a number of Bastow Institute webinars, which I titled “Assessment Considerations for Remote Teaching”. I’m very grateful to the 800+ teachers who took part, eagerly exchanging problems and practices. In each session, I asked participants to answer one opening question: “What is your single greatest assessment challenge in the context of remote teaching?” As you’d expect, responses were varied, but in each of the twelve webinars, a set of connected concerns dominated the chat feed:

“I don’t know whose work I’m assessing.”

“Some of the work submitted is clearly not the students’ own.”

“I don’t recognise some students in their work.”

“The improvement in some students’ writing is nothing short of a miracle!”

“I think some parents want the work submitted to be perfect, but I just want to know what the kids can do.”

These comments are of course not representative of all teachers’ experiences, and many other prominent problems emerged (not least the issue of non-submission). Nevertheless, I want to explore the issue of authenticity; I wonder what we can learn from these observations, and what they might mean for future online instruction.

Authenticity: What is it, and when does it matter?

Assessment information is deemed “authentic” if we can verify that the person demonstrating proficiency is the person being assessed; put simply, it’s about establishing that a student’s work (or performance) is his or her own. While this can impact any assessment, we are generally most aware of it in high stakes or standardised assessments (e.g. VCE examinations, admissions testing, NAPLAN, etc.). While it’s clearly critical in those cases, the vast majority of a teacher’s assessment is low-stakes and used for formative purposes. So, what about authenticity in everyday assessment? To what extent does it matter then?

Authenticity in everyday assessment

Teachers continually examine student responses (in real time, or from lesson to lesson) to draw conclusions about what students know and can do. In turn, they make important decisions about what to do next – a process often called “formative assessment”. Whether instruction is face-to-face or online, teachers need to engage in this kind of iterative assessment to make well-founded instructional decisions. A compromise in the authenticity of assessment information can in turn impact practice.

Let’s take the example of Miss Jones, who – after a period of delivering explicit instruction online – sets a number of fractions problems for students to work through at their own pace, from home.

In Scenario A, student James completes the problems. When Miss Jones reviews his answers, she notices that he holds a misconception in the way he has added several fractions. With this information, Miss Jones has choices; she can address his misconception by doing more worked examples, or she can explicitly address this in a note to James, or she can direct him to a helpful resource, and so on.

In Scenario B, James gets his brother to complete the problems. The answers submitted are all correct, and so Miss Jones is denied critical information about what James can do. With misrepresentative evidence, she may never find out that James carries detrimental misunderstandings about how fractions work; in turn, Miss Jones might make a poorly-founded decision about what to do next.

Teachers are well aware that authenticity can be an issue. It’s probably fairly common in the context of homework – but when all work becomes homework, you can quickly see how managing authenticity in everyday teaching becomes a problem. Over prolonged periods, inauthentic information can distort perceptions of students' capabilities, and hinder decision-making.

Managing factors which impact authenticity

Teachers don’t normally contend with authenticity as a major issue in day-to-day assessment, I suspect in part because they have close physical proximity to students; as any teacher will tell you, it’s darn helpful to be able to see and hear the kids in front of you! Teachers normally talk to students while they complete set work. Teachers can scan the room, making important observations about students' work habits, ability to follow routines, and attitudes to various aspects of learning. What is James doing? He looks confused. Is Jessica copying from Elissa’s work? Why is Daniel asking Belinda for help? Fraser has opened his laptop – is he using translation software to complete the comprehension questions?

Being able to monitor the learning environment in this way probably helps teachers short-circuit many potential issues of authenticity in real time – so, over the last few months, with students working on tasks in different places, and at different times, with variable technological/human supports, and away from their teachers, it’s easy to imagine the many ways in which authenticity issues have loomed large.

What can we learn?

No one had time to thoughtfully plan every aspect of the shift to online instruction; it was a move made in the context of a global emergency. Far from being critical of anything that teachers have done, I want to prompt conversation about why authenticity may have been compromised, and how we might adjust future online practices in light of these experiences.

1. Reviewing task difficulty

Task difficulty alone does not determine whether a student will do his or her own work, but if a task is much too hard for independent completion, it follows that students will need help. If we’re lucky, they may receive this in the form of a very useful parent. But they may also get it in the frantic Googling of an answer, or in the form of friends sending photos of their work with the generous caption: “Here, copy mine”. In any case, I suspect that tasks delivered remotely carry a much higher degree of difficulty just by virtue of the fact that students will complete them without the regular supports of the classroom. Teachers always factor in task difficulty, but I think we can be doubly cautious when setting tasks for remote completion.

2. Flagging the intended mode of completion

It’s not always obvious to parents (or carers, or students) whether a task is designed for supported or independent completion. For example, a parent might help his child with an online standardised reading assessment, giving prompts and clues which inadvertently interfere with reliability of the assessment. Being clear (perhaps overly clear) about the intended administration of assessment is helpful for everyone.

3. Communicating ways in which parents (or others) can assist

Irrespective of the formality of the task, it’s helpful to determine (and then communicate) how assistance can best be given.

Let’s take the example of a parent who assists her child with narrative writing by writing a story herself, before asking her child to copy it out. A teacher may well feel frustrated by this ("The student has now completed a handwriting task!"), but simultaneously that parent may feel that they’ve been very helpful. The point is that parents (and others) do not instinctively know how to be useful without compromising the rigour of assessment, so a few dot points explaining how people can (and perhaps shouldn’t) assist could go a long way.

Below I note a few ways in which parents (and others) can assist in ways that don’t usually interfere with accurately determining competence:

- assisting with planning (how to plan responses, chunk tasks, what to work on first, etc)

- helping students to interpret assessment tasks (what is the task really asking of you?)

- encouraging students to seek help through available channels (if really stuck)

- supporting students to manage the learning environment (e.g. removing distractions, setting timers, organising materials, etc.)

- helping students with aspects of the task that are not central to the capabilities being assessed (e.g. spelling – unless of course it is an assessment of spelling)

4. Establishing the level of support received

For the purposes of determining how much help a student has received, it may be worth developing some mechanism by which parents (or students themselves) can give feedback. This may take the form of a quick rating scale, or a code. Did a parent have to help the child with every single part of the task? Was the task completed independently, without any support? Or was it somewhere in between? As a classroom music teacher, I’ve never needed to do this myself, but the point is that if teachers are left wondering, “How much help was given?”, there may be ways to get a sense of this.

5. Developing a positive culture around assessment

When I first started teaching, I didn’t have a good language for talking about everyday assessment with parents and students; I wasn’t good at establishing purpose. I think I got better at this as the years went on, and I suspect it helped me to elicit more authentic information from students.

For example, when I first started, I rarely gave tests and when I did, I’d say, “I want to see your best”. Over time, self-testing just became part of what we did, and my framing changed:

“This is another chance to test your memory, which we know is great for learning.”

“If you don’t know an answer, it’s great information for me.”

“This isn’t a test of you as a person – it’s a way to find out what has stuck.”

“You’ll get lots of chances to review what we cover today – this is just a snapshot.”

This framing might be unhelpful in some contexts– but, for me, it helped to establish a culture of safety around formative assessment in the primary school music classroom.

6. Capturing your own hunches

Currently practicing teachers will have a much more contextually accurate view of these issues on the ground. If authenticity has been identified as problematic in your setting, I hope this post invites thinking/discussion about what might have been happening, and what may be adjusted in any future remote teaching.

Wrapping up

The move to remote emergency teaching threw up a number of dilemmas of practice. While maintaining authenticity may have proved an emerging problem in some circles, I’m sure on the flipside that the tireless efforts of countless parents helped deliver authentic assessment information in the most trying of circumstances. Through the crisis, I’m in awe of what parents and carers did with (and for) their children, and I’m equally stunned by what teachers achieved on such short notice.

Should teachers and leaders choose to “hold on” to any aspects of remote instruction, it’s going to be important to carefully interrogate the elements of task design and delivery that may have seriously compromised accuracy of data. My hunch is that it’s been a bit of a minefield, and many teachers probably feel they’re “starting from scratch” to get an accurate picture of proficiency for some students.

While it’s not entirely controllable, and isn’t all that important in isolated cases, a diminishing of authenticity in everyday assessment matters over time. The more accurate information teachers have about their students (from day to day), the better they can teach. To this end, authenticity of assessment is well worth trying to preserve.