Publishing with Students – Collecting and Analyzing SoTL Data
This is the fifth post from a series I wrote in 2012 about writing and publishing with undergraduate students. I’ll be publishing this series every other Thursday over the summer. For more recent research and resources, visit the new International Journal for Students as Partners and a special issue of Teaching and Learning Inquiry that I co-edited on students as co-inquirers.
When analyzing and writing up SoTL data with my students, ethics has been the most important and challenging aspect of the work. Ethics in SoTL work is an important issue anyway – how can we study our students without taking advantage of them or privileging those who agree to take part in the research? The foundations of SoTL say that we are the experts in our own disciplines and classrooms, so we are best positioned to study the effects of teaching practices and the process of learning in our own classrooms. This can seem at odds with our graduate training and our respect for IRB and human subjects regulations. Empirical data collection with human subjects obviously requires careful attention to research ethics, and the empirical nature of SoTL is one of the things that differentiates it from teacher research or scholarly teaching.
When collecting SoTL research with or without your students, here are some simple measures to take to ensure ethical data collection and treatment of subjects. The first two tips are for SoTL projects in general:
Carefully create and submit your IRB application early. Some IRB’s, especially at more traditional research institutions, are still learning to deal with the challenges and requirements of SoTL research, so it might take more than one pass to get the application through. My university is a teaching institution, but we of course went through the same challenges. Here is the statement that our Center for the Advancement of Teaching and Learning posted to help both researchers and IRB members navigate these waters. For us, anything that records students, voice or video, is still a challenge, and those methods will require and extensive justification. But methods that are part of the class structure are much easier to support. Here’s a good page with a collection of sample methods.
Ask a colleague to handle the informed consent forms. SoTL isn’t really about control groups and sterile research practices. If I’m the expert in my classroom, and I want to try something based on my observations that I think will make a positive impact on learning, I’m not going to offer that benefit to some students but not others. To be sure that I don’t think of my students as research subjects during the semester, I ask a colleague to attend my class early in the semester for about 20 minutes. I step out, and my colleague explains the study, takes questions, collects the consent form, and seals the envelope. That envelope then lives in the colleague’s files until after my grades have been submitted for the semester. This ensures that I treat every student the same way throughout the course regardless of their participation in the research study.
Develop data collection instruments with your student co-researchers. In the last post, we discussed the process of creating research questions with the students; creating the instruments is equally important. This is an opportunity to help students determine the best means for collecting data to answer the research questions we developed. This is also the opportunity to discuss the ethics of the data collection and interacting with their peers as research subjects. I frequently try to stick to methods that are somewhat less personalized like a survey; though interviews and focus groups might be equally as appropriate.
Include the student researchers in the process of drafting the IRB application. The application is the first opportunity to really codify the research plan and talk through not only how the data will be collected, but also how the researchers will engage with participants. Student researchers might not understand or even consider the ethical implications of research, so this is a great teaching moment. I often work in two phases with my IRB applications – I will submit one as the sole researcher before a class that meet the attributes of my research agenda, and if something special or interesting outside or tangential to that agenda occurred in the course, I will invite students to collect more data and write with me the next semester.
Discuss research ethics regularly. In my most recent foray into writing with students, I was faced with a perplexing ethical dilemma that caught me by surprise for some reason. I had collected, with IRB support, 10 reflections from each student in the class we were studying. The student co-researchers and I then created and survey that we sent to students in the course the next semester. We also used our own experiences and observations as the third aspect of our triangulation. Once we decided what we were looking for in the study, in this case student motivations to collaborate well with peers and community partners in a service-learning course, I had to code the student reflections. I took on the coding myself because I was uncomfortable sharing those reflections with the students.
As I was coding, I realized that it was going to be pretty easy for the student researchers to figure out which of their peers said what in their reflections anyway. After talking though the ethical issues with a wonderful mentor at CATL, I decided to use the blinded data, muddy though it was, to avoid putting the student researchers in a particularly awkward position. We promised to make it clear in the write-up how the data had been coded and why we had chosen that path. We overdid it in the data by constantly attributing the data analysis to me (“In RPR’s findings in the reflections” – times 50), but we ended up with the occasional reference and a good footnote in the version we submitted for publication.
Working with student co-researchers, like any SoTL work, has its own ethical challenges. Identifying and thinking through the challenges with the students and developing responsible solutions is part of good scholarship. In the next post we will look at some strategies for writing up the data for your target journal – especially the issue of voice.