This is one in a series of weekly blog posts in which I show how a journal article or book has inspired me to improve my teaching practice. Each post ends with a practical resource or activity. Today I show how books by Christodoulou (2017) and Wineburg (2001) inspired me to try and improve my students’ understanding of source analysis.

When I was moving house two months ago, it was my pleasure to come across several of my husband’s medical exam preparation books. At the bottom of one pile (to be moved from one loft to another), I found an entire book based on multiple-choice questions. My husband, an anaesthetist and intensive care doctor, has been answering multiple-choice questions for nearly all of his many (many) years of medical training. 

British educational researcher Christodoulou has recently outlined the many opportunities presented by the multiple-choice question, including:

• The concept that a multiple-choice question can test a fine gradation of understanding by offering several related options, that require the student to think deeply about an issue. This does mean that the question and available options have to be very carefully constructed. This is related to the second opportunity:
•  The options can, if carefully designed, home in on important and frequent misconceptions. These can serve to make the question more challenging and provides an excellent opportunity for teachers to highlight and dissect those misconceptions with their students. 

I have started to use multiple-choice questions in the rather unexpected area of source analysis, with positive results. My case for using multiple-choice questions in supporting pupils to handle sources rests on how well it can help a teacher to tackle several issues. My case is shown through the lens of the demands made upon my post-16 students, undertaking an AQA-examined A-Level on Religious Conflict and the Church in England, c.1529-1570.

My intention with the multiple-choice statements is to use them as a discussion tool.  At no point have I ever taken in the resource sheets to grade them. As Professor Wineburg has pointed out:

‘a blackened squiggle can mean anything. Right answer for wrong reasons, wrong answer for right reasons, or sheer guessing’. 

However, we as a class have spent up to 30 minutes discussing a single sheet of questions. To give them out and grade them afterwards without discussing the statements is to miss the point of using them entirely. 

One: Source analysis can be so complex that pupils often do not know where to begin when confronted by a new source

First, there are issues in understanding the vocabulary used in the source. Second, the need to move continually between source provenance, source content and contextual knowledge can overload the working memory. American researcher Wineburg has argued that analytical writing about evidence is  difficult because it involves making numerous decisions, and that pupils should be taught how to make those decisions. He has also argued that analysing sources is an ‘unnatural act’. The reconstruction of an intellectual context in order to analyse a source demands ‘skill’ of the highest order.

Multiple-choice questions can break down more complex tasks or, to use the words of Christodoulou, the ‘end goal’, into smaller and more manageable chunks. The helpfulness of ‘chunking’ has been emphasised recently in education (although the concept is not new, e.g. Miller, 1956) because it helps to avoid overloading pupils’ working memories.

My multiple-choice questions based on a source (see the resource below)  asks pupils three questions to interpret source content (Q1-3), then asks them to connect the source content to their own knowledge (Q4), and finally asks them to consider how the provenance of the source has potentially affected the source content and language (Q5-8).

In the question below, which asks post-16 students to analyse a source outlining several injunctions made by Bishop Hooper in 1552 (the source is included in this blog post at the bottom as well as in the resource that you can download), students are asked to consider just one aspect of source provenance: the date. 

The provenance of the source – in that it was written in 1552 – is useful because…

1. This was the year in which the Six Articles were repealed, so the source is useful to demonstrate a reaction to the state of limbo when Catholic doctrine had been removed
2. This was the year in which the 42 Articles were ratified by Parliament, so the source is useful to demonstrate a reaction to the final doctrine decided upon
3. This was the year in which the Second Book of Common Prayer was written, so the source is useful to demonstrate the continued need to confirm and enforce the liturgy outlined in that Book
4. This was the year when Edward VI died, so the source is useful as a final overview of the religious changes brought about by the reign of Edward VI

 I have carefully chosen four real events that took place during the time period in question in order to make this source question challenging but accessible. I judge that it is accessible because pupils are honing in on one aspect of source provenance – the date. Something else that makes this question effective for students’ learning is that it implies that students must have an excellent grasp of chronology in order to judge what the author may have been responding to. This implication brings me onto modelling.

Two: Pupils often need help to communicate their ideas about source content and provenance

Modelling structure and language can help students make progress in handling sources. Of course, too much scaffolding can lead to empty phrases and thoughtless banalities, but a little can go a long way. In my MEd thesis (which has since been published in Masterclass in History Education, a book edited by Counsell, Chapman and Burn), I outlined how other teachers have successfully modelled the type of language needed to present ideas about sourcework:

LeCocq (2000) suggested that pupils needed language to present their conclusions, through helpful words and phrases that cement higher-level thinking. Wiltshire (2000) then taught her pupils to communicate the results of their evaluation of sources through the use of ‘language of uncertainty’. Smith suggested a year later (2001) that the use of adverbs such as ‘probably’ and ‘perhaps’ helped pupils to realise the relative strength of the claim they were making. Wiltshire introduced the idea of teaching pupils to use ‘tells’, ‘suggests’ and ‘implies’ to support their thinking about establishing and stating degrees of certainty/uncertainty.

My use of multiple choice questions is modelling in action. In the blue example above, I have modelled how I might comment upon the temporal context of the source. In the green example below I model how I have selected the word ‘implies’ when making an inference about source content, because of my judgement about the strength of the claim that I can make. I have also modelled how to use quotes from the source to support that judgement. In the brown example, I model how I might refer to the author’s standpoint and interpret it as something of value.

Q3. Another way in which the source is valuable to an historian is that it implies that there have been:

1. Changes to church liturgy, including the removal of requiem mass, as in ‘purgatory…is contrary…to the honour of Christ’
2. Physical changes to the church, such as iconoclasm, as in ‘the veneration…of…images, is contrary to and harmful to the honour of Christ’
3. Further doctrinal changes to the church, such as a move towards the idea of Royal Supremacy, as in ‘the worshipping of saints’
4. Very little changes made to the Church, for the laity are still asked to ‘receive the confirmation and augmentation of the merits and deservings of Christ spiritually’

_______

Q6. It is useful for an historian to canvass the views of Bishop John Hooper because… 

1. He exaggerates the negativity of Edward VI’s religious changes, which tell us about the views of regional leaders towards the Protestant Reformation
2. He overemphasises the extent of religious changes, including the move to transubstantiation
3. His position as regional leader of a church demonstrates that at local level there was a need for Edward’s doctrinal and liturgical changes to be enforced, implying that there may have been issues with some members of the clergy accepting those changes
4. He is a particularly radical Protestant reformer, demonstrating that the country was moving in a Calvinist direction

Crucially, even in the ‘wrong’ answers I am still modelling a ‘thread’ of how my thoughts about source content and provenance might be communicated clearly and cogently. Purposeful and constructive inclusion of ‘wrong’ answers is yet another strength of using multiple-choice questions with sources. The ease with which a pupil can misconstrue source content leads me onto:

Three. Pupils often need to ‘slow down’ their reading of sources

Wineburg has demonstrated in his research that sources need to be read ‘deeply’ in order to be understood, commenting on one particular historian who read a source eight times before coming to a conclusion about its meaning. In my experience, students have a tendency to read a source rather quickly (almost as if the process is very painful) and jump to conclusions about content based on a single phrase that they have taken out of context – possibly because it is the only one that they fully understood at first reading.

Source analysis is hard because a student must be able and willing to rethink a judgement they made at first glance in the light of the second reading. Careful formation of a multiple-choice question can help pupils to identify (and, hopefully, remedy) this habit of reading fast because I include phrases that are easy to read on their own, but actually mean something very different when read in the context of the whole. Take, for example, the example in purple below (where I have also modelled how I might refer to the tone and language in a source):

Q5. The tone of the source is useful to an historian because:

1. It is formal and cogent, as in ‘that the clergy diligently teach’ and ‘the doctrine of purgatory, pardons and prayers…is…harmful’ – this shows that church leaders such as Hooper felt it necessary to reinforce religious change with their dioceses and ensure conformity with it
2. It is angry and inflammatory, as in ‘Good works…are of no value or estimation’ – this shows that church leaders such as Hooper believed passionately in their religious convictions
3. It is positive and admiring, as in ‘the merits and deservings of Christ’ – this shows that church leaders such as Hooper felt it necessary to remind their parishioners to always serve God and refrain from sinning
4. It is negative and mournful, as in ‘prayers for them that are departed from this world’ – this shows that church leaders such as Hooper were sorry for those that have been killed on the road to Protestantism (such as Thomas Cromwell)

Reading those quotes alone, it is easy to see that the ‘deservings of Christ’ could be positive; reference to the ‘departed’ could be seen as mournful; reference to good works as having ‘no value or estimation’ could be seen as inflammatory. Yet reading the source carefully, in context, should lead you to select Option 1 as a ‘best fit’ statement. 

I use the word ‘best fit’ deliberately, because the questions have been crafted, after all, on my personal interpretation of the source. It is perfectly possible to come to a reasonable conclusion that contradicts my interpretation, perhaps finding limitation when I see value. Which brings me onto…

Four: To make progress in evidential thinking, pupils are required to do what educational researcher Shemilt has called a ‘reverse somersault’

This involves moving from the viewpoint that source bias is limiting and distorting, to the idea that source bias is revealing and useful.  Evidence can, after all, be used to support propositions not intended by the author, as ‘unwitting’ testimony. This ‘somersault’ is, in my experience, an extremely difficult feat for some students to make. Using multiple-choice statements can model the process of interpreting bias in both a positive and negative light. Compare, for example, Question 5 above (in purple) with Question 8 below (in pink):

Q8. On the other hand, the source provenance may limit its value somewhat in that:

1. The source was written only half-way through Edward’s reign, so we cannot see the religious changes that took place during the second half
2. The author was not an eyewitness to the changes made personally by Edward VI – he lived in Gloucester and not in London
3. Bishop John Hooper was unusual in his anti-transubstantiation stance, so his views are atypical for the time
4. The injunctions were issued at a local level only, and so cannot give us a picture of religious change in other districts or at a national level

Not only does this example demonstrate that multiple-choice questions can model both usefulness and limitation, this question demonstrates how a teacher could spend just as much time on analysing why Option 2 is not a good answer, helping to overcome the issue of ‘stock evaluation’ (which is rife in my school), as they spend on highlighting why Option 4 is a better (although perhaps not the very best) fit. Again, ‘best fit’ is important. Perhaps the student has highlighted an even more important limitation?

Five. Pupils often struggle with question relevance 

In the context of the AQA exam, Year 12 are asked to consider the value of a source in relation to a certain topic. Again, the inclusion of ‘wrong’ answers can help pupils to understand the difference between a general point that they can infer from the source, and a point that is relevant to the examination question. For example, this examiner wants to know whether the source is useful to find out about changes to liturgy and doctrine. In Question 3 above (in green), Option 1 and Option 2 are both valid points – yet only Option 1 is relevant to the examination question.

Conclusion

As always, this is no ‘holy grail’. There are two very important limitations that I need to contend with before I write any more multiple-choice questions.

The first limitation is that it interrupts the more organic, hermeneutic movement between part and whole – the constant movement between content and provenance, wider context and personal context, that historians would naturally go through in order to interpret a source. If used sparingly and as a scaffold to ‘get going’, then I may be forgiven, but this is still an important caveat.

A second limitation, which I have already noted, is that the meaning to be made from the sources is pre-generated by me. There are a myriad of ways of interpreting source content and provenance and, much like a writing frame, I may have ‘closed off’ other interpretations. Yet, if multiple-choice questions are interleaved with the more traditional ‘open’ questions, then hopefully they will serve more as scaffolds and less as prisons. 

Multiple-choice has come out of the closet. Is it a magic bullet? No, it is not. Can it serve as just one of several ways in which to analyse sources? Perhaps it can. 

Download my resource: Source Multiple Choice Hooper

Main references:

Christodoulou, C. ‘Closed questions and higher order thinking’. From the blog: A Wing to Heaven: https://thewingtoheaven.wordpress.com/2013/10/06/closed-questions-and-higher-order-thinking/

Christodoulou, D. (2017). Making Good Progress? The future of Assessment for Learning. Oxford: OUP.

LeCocq, H. (2000). Beyond bias: making source evaluation meaningful to year 7. Teaching History, 99, Curriculum Planning Edition.

Shemilt, D. (1987). Adolescent ideas about evidence and methodology in history. In C. Portal (Ed.). The History Curriculum for Teachers (pp. 29–61). London: Heinemann.

Smith, P. (2001). Why Gerry now likes evidential work. Teaching History, 102. Inspiration and Motivation Edition.

Wiltshire, T. (2000). Telling and suggesting in the Conwy Valley. Teaching History, 100, Thinking and Feeling Edition.

Wineburg, S. (2001). Historical Thinking and Other Unnatural Acts. Philadelphia: Temple University Press.

Worth, P. (2016) ‘Evidential Thinking: Language as Liberator and Gaoler’ in Counsell, C., Burn, K. and Chapman, A. (Ed.s) Masterclass in History Education: Transforming Teaching and Learning (London: Bloomsbury).

 

Source A: Injunctions issued by Bishop Hooper in 1552 to the Diocese of Gloucester

7. Item, that the clergy diligently teach and preach that man is justified only by the faith of Jesus Christ and not by the merit of any man’s good works; although good works necessarily follow justification. Good works without faith are of no value or estimation before God.
8. Item, that the doctrine of purgatory, pardons and prayers for them that are departed from this world, the veneration, invocation and the worshipping of saints and images, is contrary to and harmful to the honour of Christ our only Mediator and Redeemer
9. Item, that in the sacrament of the Body and Blood of the Lord there is no transubstantiation of the bread and wine into the Body and Blood of Christ, or any manner of corporal and local presence of Christ in, under, or with the bread and wine. We receive the confirmation and augmentation of the merits and deservings of Christ spiritually, by faith