Rajiv Jhangiani, Ph.D.

Open Education, SoTL, Psychology

Category: Pedagogy

Why have students answer questions when they can write them?

Questions by Alan Levine (CC-BY 2.0). Retrieved from https://flic.kr/p/mrQ1x1

Questions by Alan Levine (CC-BY 2.0). Retrieved from https://flic.kr/p/mrQ1x1

I recently trialled a new assignment in my Social Psychology class: During each of the 10 weeks when there was no scheduled exam I asked my students to write multiple-choice questions. That’s right, they wrote questions instead of merely answering them.

From a pedagogical perspective, I really wanted my students to achieve a deeper level of understanding (e.g., the level it takes in order to craft three plausible distractors). However, this assignment also served a pragmatic purpose in that the open textbook that I use for this course (and that I helped revise) does not yet have a readymade question bank.  By asking my students to craft and peer-review multiple-choice questions based on the concepts covered that week (and scaffolding this process over the semester), I considered I had a budding open pedagogy project on my hands.

Here’s how it went:

  1. The students were asked to write 4 questions each week, 2 factual (e.g., a definition or evidence-based prediction) and 2 applied (e.g., scenario-type).
  2. For the first two weeks they wrote just one plausible distractor (I provided the question stem, the correct answer, and 2 plausible distractors). They also peer reviewed questions written by 3 of their (randomly assigned) peers. This entire procedure was double blind and performed using Google forms for the submission and Google sheets for the peer review.
  3. For the next two weeks they wrote two plausible distractors (the rest of the procedure was the same).
  4. For the next two weeks they wrote all 3 plausible distractors (the rest of the procedure was the same).
  5. For the remainder of the semester they wrote the stem, the correct answer, and all the distractors.

I adapted existing guidelines about how to write effective multiple-choice distractors and how to provide constructive peer feedback and produced these two brief guides:

Guidelines for writing effective distractors for multiple-choice questions

Guidelines for providing constructive peer feedback

The result? My small class of 35 students wrote 1400 questions in the span of 10 weeks! And although I wouldn’t consider this a polished question bank ready for use by other instructors, I still consider this assignment to have been a success because the questions steadily improved over the semester (the experience of serving as peer reviewers was especially useful to the students when constructing their own questions). The students were also buoyed and motivated by my practice of including a few of their best questions on each of the three course exams. Looking forward, I plan to have my next cohort of Social Psychology students revise and add to this bank. I figure that it will take only a couple of semesters for us to provide the commons with a high-quality question bank, something that will enable even more instructors to adopt this open textbook.

If you have attempted something similar or would even like to collaborate with me on this assignment, please write a comment below or otherwise get in touch. Your feedback is very welcome.

The opposite of open is broken

The opposite of open is not closed; the opposite of open is broken. The more I think about it, the more this cogent observation, made by John Wilbanks, resonates with me. Continue reading

Idealism or pragmatism? A false dichotomy in four tweets

Continue reading

Are open textbooks the end game?

“I don’t want to be part of a movement that is focused on replacing static, over-priced textbooks with static, free textbooks.” I hope Robin DeRosa’s thoughtful post about open textbooks provokes some reflection on the tone and goals of the open textbook movement and its advocates. It begs the question of whether we are waging the wrong war here, at least in part. Continue reading

Pilot testing open pedagogy

This summer, as has become usual practice for me, I adopted open textbooks for my Introductory Psychology and Social Psychology sections (produced by NOBA and the BC Open Textbook Project, respectively); however, my desire to enjoy a semester entirely free from traditional textbooks was challenged by the absence of a high quality open textbook for Cognitive Psychology. Continue reading

OER for Psychology

Later this week Farhad Dastur and I will present at a symposium at the Annual Conference on the Teaching of Psychology in Atlanta, organized by the Society for the Teaching of Psychology. Our symposium is titled “Opening up psychology: Adopting open textbooks, open pedagogy, and an open philosophy in the classroom.” The following is a list of some Psychology-specific OER for those who attend our symposium. It is not an exhaustive list by any means but is a start: [Click here to download the list: Psychology OER]

NOBA Project: http://nobaproject.com/

BC Open Textbook Project: http://open.bccampus.ca/

Resources for Teaching Research and Statistics in Psychology: http://www.teachpsychscience.org/

Go Cognitive: http://www.gocognitive.net

Cognitive Psychology Wiki: http://cognitivepsychology.wikidot.com/

STP Office of Teaching Resources in Psychology: http://teachpsych.org/otrp/resources/index.php

APA Online Psychology Laboratory: http://opl.apa.org/

Personality Pedagogy: http://personalitypedagogy.arcadia.edu/

Animations for Teaching: http://garyfisk.com/anim/index.html

Social Psychology Teaching Resources: http://www.socialpsychology.org/teaching.htm

Timothy Bender’s Memory & Cognition Demonstrations: http://courses.missouristate.edu/timothybender/mem/mydemos.html

Rossman/Chance Applet Collection: http://www.rossmanchance.com/applets/index.html

Consortium for the Advancement of Undergraduate Statistics & Education: https://www.causeweb.org/webinar/activity/

Interpreting Correlations: http://rpsychologist.com/d3/correlation/

The Brain from Top to Bottom: http://thebrain.mcgill.ca/

Research Methods Knowledge Base: http://www.socialresearchmethods.net/kb/

Fostering Sustainable Behavior (textbook): http://www.cbsm.com/pages/guide/preface/

TED Ed: Psychology: http://ed.ted.com/lessons?category=psychology

Psychology Tutorials & Animations: http://psych.hanover.edu/Krantz/tutor.html

 

Other General Repositories & Providers of OER

MIT Open Courseware: http://ocw.mit.edu/courses/brain-and-cognitive-sciences/

Saylor Academy Library: https://saylor.longsight.com/

OpenStax College: http://openstaxcollege.org/books

OpenStax CNX: http://cnx.org/

Merlot: http://www.merlot.org/

OER commons: https://www.oercommons.org/browse/keyword/psychology

The Great Psychology Testbank Sprint

Well, we did it. Seventeen psychology faculty from six post-secondary institutions in British Columbia came together on July 18 & 19 and worked intensively for two days to create a testbank designed to accompany open textbooks for introductory psychology. As I have previously written about, the absence of ancillary materials (a testbank most of all) presents a significant challenge to instructors who wish to adopt an open textbook. Although some observers will turn their noses up at the thought of faculty not writing every single one of the questions on their exams, the reality is that many (overworked) instructors of large, multi-section survey courses do start working from the publisher-supplied testbank, writing, revising, and editing as necessary over time. The need for this crutch becomes even more pronounced for new faculty that do not have years of classroom and assessment experience to draw on, sessional faculty assigned to a course only weeks ahead of the first class meeting, and large sections with limited teaching assistant support. For those of us that advocate for the adoption of open educational resources in the classroom, ignoring or criticizing this reality is self-defeating.

Fast forward to this May when, following a series of productive conversations with Mary Burgess (Director of Open Education at BC Campus) and Peter Lindberg (NOBA Project), I was able to secure the financial support of both organizations to coordinate a “testbank sprint.” Modelled on the efforts of textbook sprints that, for example, bring together a number of faculty members for a few days to write a textbook from scratch, the idea was to invite faculty from a number of psychology departments in BC to come together for two days for a retreat-style sprint.

At the best of times, writing questions is not enjoyable work, so I knew that in order to recruit enough participants and to create a productive environment the sprint would have to be interspersed with opportunities for socializing and rejuvenation. Another goal was to gather sprinters from a variety of institutions (the wider the representation, the more natural the sense of ownership and buy-in). Finally, the challenge of producing a resource like this for Introductory Psychology is the sheer number of topic areas (15) that need coverage. Thus we would have to find a way of collecting data about potential participants’ areas of expertise and mapping these onto the topic areas in order to ensure adequate coverage.

Enter the Organizing Team

Following a couple of discussions with members of my department at Kwantlen Polytechnic University, three wonderful souls in Dianne Crisp, Cory Pedersen, and Levente Orban stepped forward to offer their assistance with organizing the sprint. Together we dealt with the questions of when to hold the sprint (mid-July, in time for Fall adoptions), where to hold it (the Cheam Golf Course in Chilliwack with accommodations at the nearby Coast Chilliwack hotel – central enough for those travelling from Victoria or Kamloops but far enough away to feel like a retreat), how to recruit faculty (the Articulation listserve + direct emails to those with past involvement in open education projects), how to map availability & areas of relative expertise onto the 15 topic areas of Introductory Psychology (enter the technical wizardry of Levente with custom online forms and formulae-laden Excel worksheets), and how to structure the event (see below).

Nineteen faculty signed up and, although two dropped out just 48 hours before the sprint was to begin, seventeen faculty representing the psychology departments at Kwantlen Polytechnic University, Thompson Rivers University, Camosun College, Northern Lights College, Capilano University, and the University of the Fraser Valley met in Chilliwack on July 18. The amazing Clint Lalonde from BC Campus was also on hand to provide logistical and technical support for both days of the sprint.

The Cheam Mountain Golf Centre welcomes us

The Cheam Mountain Golf Centre welcomes us

The logo for our sprint (yes, I got carried away with my role)

The logo for our sprint (yes, I got carried away with my role)

Sprint Structure

The sprint was structured around 3 90-minute working sessions each day for two days, interspersed with breakfast, coffee breaks, and lunch. Following an early breakfast we began on both days with an orientation at 8:30am. The working sessions ran from 9-10:30am, 11am-12:30pm, and 1:30-3pm, with a final progress update & wrap up from 3-3:30pm. Dragon boating and a hosted dinner were scheduled for the afternoon/evening of July 18 while the sprinters were free to play 18 holes of golf on the afternoon of the 19th.

The seventeen faculty were assigned to one of 7 working groups of 2-3 faculty for each session, with each group responsible for writing questions for a specific topic area. The composition of the groups and the topics changed every session such that by the end of the second session on Day 2 of the sprint each faculty member had participated in 5 working groups and each topic area had been visited by two working groups. From an organizational point of view this was a bit like playing with a Rubik’s cube with 17 squares and 15 sides. Only the squares had feelings and the saturation of the colours was not always the same.

The target for each working group during their 90 minute session was to write an average of 10 high-quality multiple-choice questions per group member (~20-30 questions per topic per session and ~50 questions per topic or ~750 questions in total by the end of the sprint). To help with this we circulated the topic assignments and a document outlining some best practices for good multiple-choice questions ahead of the sprint (the sprinters were encouraged to think about or even write questions ahead of time and to bring their own laptops).

The final session on Day 2 was devoted to the vetting and categorization (Applied, Conceptual, or Factual) of questions. I assigned one faculty member to go through and vet all of the questions written for one topic area in which they had intermediate-to-high expertise. Minor problems could be fixed immediately but larger problems had to be flagged. Flagging a question meant that another faculty member with overlapping expertise would have to be consulted to see if the question could be resuscitated or whether it had to be deleted (deletion was always a two-person decision). I should note here that we did not have an easy template to follow for any of this so the structure and rules of the sprint really resulted from my sense of what would work well. Mercifully it all went to plan and when I solicited feedback and suggestions at the end of the first day the group appeared very happy with how things were going.

Battle stations

Battle stations

Vetting questions

Vetting questions

The Finish Line

So how did we do? By the end of the fifth session we had collectively written 870 questions across the 15 topic areas (comfortably surpassing my initial target of 750 questions). Following the vetting process in the final session on Day 2, we finished with 851 questions (296 Applied, 219 Conceptual, & 336 Factual). Very soon this entire testbank will be made available by BC Campus to any recognized psychology faculty, in a format that can imported into any LMS. In other words, mission accomplished, under budget and on schedule.

Perhaps most importantly though, we truly enjoyed ourselves. Both process and outcome, journey and destination, were gold.

Happy Sprinters

Happy Sprinters

Some Observations

  • Although some working groups began writing questions collaboratively, most found that they worked more productively when they divided the learning objectives/sub-topics and worked in parallel. Having said this, some truly enjoyed writing questions collaboratively.
  • The target of 10 questions per faculty member per 90 minute session was found to be reasonable and well-calibrated.
  • Although we had installed testbank creation software on a KPU-issued laptop at each of the seven stations Clint and I decided ahead of the sprint that it would be easier and a time-saver to have everyone enter questions (using the correct format) into a more familiar tool (MS Word) instead. The software permitted the easy import of questions from Word.
  • Our hosts at the golf course remarked several times at how quiet we were while working. This was generally true as the soundtrack of each session was a steady chorus of fingers on keyboards punctuated with laughter (one of the sources of laughter was the amusement we spontaneously derived from embedding one another’s first names into many of the applied questions!).
  • Given the nature of the work we were there to perform I remain amazed by the level of enjoyment we derived. The dominant and universal theme in the feedback I have received since the end of the sprint has been how much fun it was.
  • As noted by someone during our dinner at the end of Day 1, dragon boating on Harrison Lake on the first evening proved to be the ideal metaphor for the sprint. We were in it together, we had to coordinate our efforts well, we had one person (me) providing continual feedback to the group while staying mindful of our pace and progress, we rested in between sessions of hard work, we laughed a lot, truly cherished one another’s company, and experienced a rather unforgettable adventure.

The Heroes of the Sprint

My thanks to my organizing team (Dianne Crisp, Cory Pedersen, & Levente Orban from Kwantlen Polytechnic University), other fellow sprinters (Betty Rideout, Carla Maclean, David Froc, Farhad Dastur, John Marasigan, Kurt Penner, Kyle Matsuba, and Richard LeGrand from Kwantlen Polytechnic University, Jacqueline Kampman from Thompson Rivers University, Randal Tonks from Camosun College, Istvan Geczy from Northern Lights College, Hammond Tarry from Capilano University, Wayne Podrouzek from the University of the Fraser Valley, and  Clint Lalonde from BC Campus), ed-tech whiz Meg Goodine from Kwantlen Polytechnic University, and project sponsors (Mary Burgess from BC Campus and Peter Lindberg from the NOBA Project). Thanks also to the Mavericks Dragon Boat Team for hosting us on the evening of July 18.

See Also

Clint’s blog post from Day 1 of the sprint

Press coverage of our work by the Chilliwack Times

A Faculty Perspective on Open Textbooks

Recently I have found myself at many meetings and events (e.g., the Open Textbook summit) centered on Open Education. Despite my non-representativeness due to self-selection, I am often called upon at these meetings to represent the “faculty perspective.”

As much as I would love to do this, in my experience, there is no single faculty perspective on open education in general and open textbooks in particular. Some, like myself, are early adopters. Others are willing to go along if their concerns are addressed. Still others remain skeptical and resistant. And there are many views in between, many of which contain a mixture of curiousity, interest, and concern. And this is not a bad thing. Academic freedom is sacred. And, at least in my experience, a faculty members’ teaching philosophy is often intimately connected with their openness to openness.

Most faculty I know consider the cost to students when assigning a textbook for their courses. For them, shifting to an open textbook (assuming one is available) provides a clear advantage. Students can download and use a digital copy of the textbook for free or even print a physical copy at a fraction of the cost of a traditional textbook. At a time when a growing number of students are attempting to hold down full-time jobs while pursuing their post-secondary educational ambitions, this is a tangible benefit with a human face. Every semester I notice students in my classes who elect not to purchase the course textbook (despite cautionary notes from me) due to financial constraints. My colleagues report the same. In the battle between groceries and a textbook, the textbook loses every time.

This is especially true given the increase in the price of a traditional textbook over the past decade. We cannot fault our students for questioning the value of their (forced) purchase. As an example, the textbook I previously assigned for Research Methods in Psychology (which just happens to be the most popular textbook for this course in BC) is a softcover book printed in black ink that runs 416 pages long and retails for $114.95 + taxes & shipping. I should say that it is a great book and well written. But, in contrast, the Canadian edition of the open textbook for Research Methods in Psychology that I revised includes colour graphics throughout, runs 378 pages long and costs my students nothing. If they wish to order a print copy of the book it will cost them $13.06 + taxes & shipping. At least on price, there is no contest.

So why are faculty not yet adopting open textbooks more widely?

1. Quite simply, for many disciplines and courses, there is no open textbook available. So other than putting together a set of existing open educational resources, the nontraditional options are limited. I should say here that cost-saving alternatives like e-textbooks put forward by the big publishers are often a terrible option for students because they come with a time-limited license and have no resale value, which means that they often end up costing the students the same (or even more) in the long run, as compared with biting the bullet and buying the assigned traditional textbook.

2. In my experience, reason #2 has to do with concerns about quality (e.g., comprehensiveness, clarity, currency, etc.). Some faculty are instantly skeptical of open textbooks and hold them to a higher standard than traditional textbooks. This is fair, because traditional textbooks typically have several sets of eyes on them through their development and are later sent to many other faculty for their review. Although some open textbook initiatives (such as the BC Campus Open Textbook Project) collect and post comprehensive faculty reviews for the books in their repository, others do not. Where available, open textbooks or chapters written by leading scholars (e.g., the NOBA project) are especially helpful in countering doubts about quality.

3. But let’s imagine that a high quality open textbook is available for a particular course. Sometimes these are entirely text-based – no illustrations, charts, or graphics to aid comprehension. No questions or critical thinking exercises embedded. No online learning management system available that students can rely on for formative feedback. And, crucially for many faculty, no testbank, which means that the instructor is then obligated to write every question for every assessment for their course. Considering the amount of time it takes to write good test questions that are reliably able to distinguish between different levels of understanding, this is a tall order.

4. The choice of textbook is sometimes not an individual one. Especially for large, multi-section introductory courses (sometimes offered in two halves), in order to facilitate student mobility, academic departments often mandate that faculty adopt the same textbook across all sections. This reality often makes switching to an open textbook a less nimble decision.

One of the myths I often try to dispel is that faculty are the enemy and have some great stake in upholding the traditional textbook model. To be clear – assuming they are not the author, faculty do not receive any benefit when they assign a particular publisher’s textbook. Faculty are, however, deeply concerned about student learning. For this reason I believe that if faculty are presented with an open textbook alternative that has been favourably reviewed by other faculty, embeds good pedagogical features, and has an available testbank, it would be more difficult for the majority to continue upholding the status quo.

Beyond merely speaking to the legitimate concerns of faculty, however, I find it more refreshing to speak to the additional advantages that open textbooks bring to faculty:

1. Faculty have the ability to adapt and remix the textbook to suit their needs. They may choose to delete specific chapters or sections or even write and insert sections for their open textbook, making it possible to incorporate recent developments in research and theory much faster than the traditional textbook’s five-year review cycle permits. In other words, an increase in academic freedom!

2. There is some evidence to suggest that when an open textbook is carefully adapted to suit a particular program, student performance and retention is actually enhanced.

3. The ideal textbook does not exist. My colleague Takashi Sato at Kwantlen Polytechnic University recently made this excellent point. There are always tradeoffs that faculty make when adopting a textbook. Often it is a question of whether the content is “good enough,” assuming that several other resources are in place. For the reasons listed above, open textbooks are very often better than “good enough.”

4. As soppy as this sounds, the looks on your students’ faces when you tell them that you have adopted an open textbook. You have them at hello.

5. Although this post is about open textbooks, I would be remiss if I did not point out that the open research movement is a natural and strong ally, particularly when addressing faculty. Open access journals like PLOS ONE have become mainstream as researchers have come to appreciate the need for the fruit of their labour (and public tax coffers) to be shared with the public. In many ways, open textbooks are merely an extension of this same philosophy and permit faculty to live more closely in concert with their values.

Of course there are many remaining issues to work on before open textbooks can go mainstream. The sustainability of who will continue to revise and update the open textbooks is one such question. Government support and resource sharing agreements help a great deal. But ultimately I believe that it is institutional culture that will need to shift. A university’s strategic priorities need to include moving towards open education. From the president’s office down, open education initiatives need to be supported for these to develop and mature. This includes time releases for faculty adapting/adopting open textbooks, institutional recognition of this work, practical and regularly offered professional development workshops, and the consideration of the development of open educational resources in the files of those on the tenure-track.

I recently met a student from the University of Regina who spoke eloquently about why we should avoid pitting different stakeholders (e.g., faculty and students) against one another. I believe she is correct. There is not just one reason to consider adopting open textbooks. The benefits to students are obvious, the benefits to faculty can be highlighted, and the benefits to the institution (e.g., with recruitment) may need to be spelled out. Open textbooks represent a rare win-win-win scenario, the kind we do not see very often in post-secondary education.

To finish, I ask you to engage in a useful thought exercise: Imagine a world in which open textbooks, open research, open pedagogy, and open educational resources are the norm. In this future world, imagine that a representative from a for-profit publishing house approaches a faculty member in order to persuade them to adopt one of their textbooks. What would their pitch look like? And what could they possibly say that would convince faculty to adopt their product?

A Nice Finish

This term I taught another graduate seminar in Applied Social Psychology at UBC. The group included nine students from counselling psychology, business, and language & literacy education. We covered a lot of ground over the semester and I was trying to think of a way to review our journey while providing some closure.

I ended up sticking post-it notes onto the double chalk-board, each listing a week’s topic (arranged chronologically). I then invited the students to use the chalk to write around those notes to list the concepts, theories, or ideas that stood out the most for them. I was thrilled to see the exercise work so well. This collective mind-mapping exercise vividly illustrated our journey (it is often easy to forget how much we accomplish over four months). By the end of it we all marvelled at the space and everyone began taking photographs of our intellectual journey (wish you were here?). The final step involved distilling seven general social psychological principles that ran through all of the topics. These were as follows:

1. The power of the situation

2. The power of the individual (to alter the situation for all)

3. Perceptions are often more important than reality

4. Dual processing (e.g., implicit vs. explicit levels)

5. Attitudes can lead to behaviour or vice versa

6. Culture and self are mutually constitutive

7. Multiple levels of analysis (e.g., individual, group, culture, etc.) and multiple forces impacting behaviour (e.g., biology, evolution, culture, social norms, personality, etc.)

Nice.

Journey mapping exercise

Journey mapping exercise

IMG_4556

Journey mapping exercise

Final class exercise

Journey mapping exercise