Student Data and Program Assessment

I had an interesting conversation with a colleague today. We were discussing PSU’s assessment of our General Education program and what the process might be for a pilot program that we are running in the Fall. We’re using a tool called TaskStream (which I think is a product of Watermark). I have not had the chance to use and check out the tool yet.

But my colleague’s understanding of how it would work is that an instructor would identify particular student work to be part of the assessment pilot, create a submission point in Moodle (our Learning Management System), and instead of submitting to Moodle, the link would take the student right to TaskStream for the actual submission. The instructor would then grade the student work in TaskStream using whatever rubric they had uploaded. Since we aren’t requiring the use of standardized rubrics to grade the individual student work, we would then assess the student work using the rubrics for the Gen Ed Program learning outcomes to determine the level of proficiency demonstrated by the work. We call the Gen Ed Program learning outcomes “habits of mind” and we call the rubrics to assess them “benchmarks.”

One question we were discussing is who does this second level of assessment since this is not tied to a grade for the student? Instead, we are gathering this second level of assessment as a way to assess the program, not the student. So does the instructor assess the student work using the benchmarks? Or do we have another, separate group that assesses the student work? For example, if we are assessing the habit of mind called “purposeful communication” in a First Year Seminar, the student might be at the “basecamp,” or lowest level of proficiency, but still receive an A on the assignment. This is because we expect first year students to mostly be at the “basecamp” level of proficiency. As they move through their Gen Ed program, we would expect them to move to the next level of proficiency (“climbing”) and by the end of their capstone Gen Ed experience, we would expect most of them to move to the highest level of proficiency (“summit”). This is our hypothesis. If we find something other than this, our intervention would be at the program level, not at the student level. So if we found that by the capstone, most students still aren’t attaining the summit level of proficiency in one of the habits of mind, we would need to look at our program to see what we might do differently. Alternatively, if we find that most of our first year students are at the summit level of proficiency in one of the habits of mind, we would have to figure out whether our expectations are too low or whether we are assessing inappropriately in some way.

Anyway, during the conversation, I was struggling with the idea that our students would submit their work to Taskstream so that we could use their work as data for our program assessment. What if a student wants to opt out of allowing us to use their work for purposes other than grading the individual assignment? This leads to the question: Who owns student work?

If you’ve been reading my blog, you probably know that my answer is that the student owns their work. And if we want to use it to assess our program, we need to get explicit permission from the student to do so. If, for example, I want to use sample student work in a Promotion and Tenure folder, I get permission from the students before doing so. Why wouldn’t we have to get student permission to use their work in a program assessment task? And if we do need to get their permission, we can’t just automatically have them submit their work to our assessment tool. So we need to figure out a work flow that allows the faculty member to grade the student’s work and then only upload the work from students who have given their permission to do so. Apparently, TaskStream doesn’t make this easy (although I don’t know that for a fact since, as I said, I haven’t used the tool).

One of the reasons I feel kind of passionate about this is because I think we in higher ed have done our students a disservice by not talking to them about these kinds of issues in their digital lives. I think companies like Cambridge Analytica find it very easy to exploit data on social media sites like Facebook because those sites don’t make it easy for users to understand what they’re giving up when they take a quiz or read a news story on the site. I don’t want higher education to exploit our students in similar ways. We need to teach our students what the issues are and that they should think about what they share and with whom. I think most students would willingly share their data if they understood how it was going to be used to improve the education that future students receive. But they should be given the choice.

I’m curious about other people’s thoughts about this. Are there any institutions of higher education that have policies about student data and how it can and cannot be used? Does anyone use an “opt-in” model for use of student data? What are the challenges with federal reporting requirements if we aren’t reporting data about ALL of our students but instead only reporting data about students who give us permission to do so? Are we legally allowed to do something like this?

Image Credit