Assessment Conference Debrief

I went to the 3rd Annual Higher Education Assessment Conference in Henniker last week. It’s kind of an amazing conference because it is so cheap to attend–$50 for the main day of the conference and $20 for each half-day workshop the day before the main conference. So I got two days of assessment professional development (which included continental breakfast, lunch, and snacks) for $90. That’s amazing!

I went to a half-day workshop by Dan Doerr from the University of Connecticut called Connecting Planning and Assessment. The overall question of the workshop was: how do we integrate assessment with strategic planning? The strategic plan should provide a framework of priorities that are dynamic and adaptable as well as action-driven. The strategic plan should explicitly state our mission, vision, and values. As we make decisions about our daily work, we want to be able to understand how those decisions support (or not) the mission, vision, and values. The strategic plan provides the intermediate structures to connect our daily decisions to the mission, vision, and values. In other words, the strategic plan helps us to align our daily work with what we’re trying to achieve in the long run. Doerr then provided us with three frameworks for helping to provide the structures for this alignment and guided us through exercises that would help us to understand the frameworks. The three frameworks he used were logic models, fishbone diagrams, and performance measures. One of the the things that he said about all of the frameworks is that they don’t automatically help an individual understand the alignment. That understanding can only happen with open and frequent dialog about the strategic plan. The frameworks simply provide the structure through which we can have those discussions.

For the afternoon workshop on the first day, I went to another session with Dan Doerr but this time in partnership with his colleague Christine Wilson, also from UConn. The session was called Assessment Methods. They handed out information about a lot of tools and talked about those tools. But the main thing I took away from this session is that all assessment should start with a question or set of questions. What do we authentically want to know? Only then can we understand which tool is appropriate to use. They talked explicitly about many assessment activities actually being data collection activities, rather than assessment. They also talked about using the right sized assessment method. In other words, doing a ton of work to answer small questions is not a wise use of our time. I really liked that they talked about assessment as a disposition toward inquiry and curiosity. What you want to know about your program and how to improve it should be the motivation for assessment activities.

The keynote on the main day of the conference was by Natasha Jankowski. She is from the National Institute for Learning Outcomes Assessment (NILOA). She was inspiring and funny, something I never thought I would say about someone speaking about assessment. She told us that local contexts matter for assessment so you will not find any (good) step by step lessons for how to do it at your institution. She also said that we should trust that our accountability to our outside accrediting agencies will be met if we work to answer our authentic questions about how we are doing. That was refreshing and reassuring. She talked about three schools of thought when it comes to assessment: measurement (concerned about reliability, validity, and objectivity with a drive to narrow the curriculum to things we can measure), compliance (get 100% of faculty to comply and get reports in on time but it isn’t clear where the reports go and no one uses the results to actually change anything), and using evidence of student learning to improve higher education. You can imagine which one she advocated for. She co-authored a book called Using Evidence of Student Learning to Improve Higher Education which I bought immediately. She then talked about two assessment models: institutional or program improvement (look at students at the start and end of their time with us and use what learn about what they know to change the institution or program), learning improvement (again look at students at start and end but check on individual learning all along the way using course-embedded assignments so students know where they are and how they are doing). Again, you can probably guess which one she advocated for. To improve learning then requires us to be explicit with students about what we are trying to get them to learn. The thing that I liked best about her is that she talked about assessment as an exercise in evidentiary reasoning. We are providing evidence that what we are doing matters to student learning. She also talked quite a bit about telling evidence-based stories about the ways in which what we are doing matters to student learning.

I spent the rest of the day at four sessions that provided several insights about the detail of assessment practices. But the insight that I would really like to make sure we’re implementing at PSU came from the last session that I attended. It was presented by Gavin Henning from New England College. The session was about creating a learning organization, an organization that is interested in answering questions about itself and its processes and practices. He said that learning organizations share several characteristics: change is embraced, conflict is accepted and managed, there is a commitment to a cycle of continuous improvement, there is a feedback mechanism integrated into daily work, assessment results are not negatively judged and punished but instead are seen as opportunities for improvement, and failure is encouraged and celebrated because we can learn a lot from it. In order for all of this to be a part of the culture, people need to be held accountable for the process, not the results (unless someone just isn’t doing their job). He also said we should examine our successes in as much detail as we examine our failures. We can learn a lot from both. PSU is not yet a learning organization but I hope that we can become one.

Overall, this was an informative conference that left me with a lot to think about. It was definitely worth $90!