AACR Recruiting Faculty to Pilot Test Questions, Portal

Sign Up Here if you are interested in participating, or would like more information about the pilot program.

The AACR project is recruiting faculty who teach introductory biology or statistics to try our questions with your students and pilot test the automated analysis and reporting web portal we are developing.

What is the purpose of the questions?

Student ideas are not either "right" or "wrong" but they are often a mix of scientific, naive and incorrect ideas.  Student writing reveals student thinking much more accurately than multiple choice questions.  Much like concept inventories, the AACR questions are intended to be used as formative feedback to instructors about your students' thinking so that you can adjust your instruction to address any conceptual problems.  The questions are not intended for high stakes, summative assessment to assign grades to individual students.

We suggest using the questions as homework in your Learning Management System (e.g., Blackboard) and giving your students a small amount of credit for answering the questions.  We then can provide you a report that shows the concepts in your students' writing.

No way, computers can't understand writing

That's right, computers do not understand language the way humans do. However, we use statistical models and machine learning tools, much like those used by Google and Amazon, to predict how experts would score the writing.  We do this by having experts score hundreds of student responses using carefully constructed rubrics and then creating computer models that predict human scoring.  Our models have inter-rater reliability with experts as good as the inter-rater reliability between experts.

How will it work?

  • Click on the Questions button above to locate questions.  You can narrow the listing by discipline and topic.  
  • Click on a particular question to get more information about that question.
  • A drop-down box allows you to narrow the search to questions for which reports are available, or to questions for which we are asking for faculty to collect data that we can use to develop questions.
  • When you locate a question you wish to use, copy the text of the question and download any images that are associated with that question to your computer.
  • Add the text as an online assignment in your Learning Management System (LMS) to which the students may enter a text response. 
  • Download the student responses as a spreadsheet or CSV file.  Check your LMS documentation for to determine how to do this.
  • Login to the AACR web site and return to the original question.
  • You will be able to upload your spreadsheet to our analysis portal and receive a detailed report about your students' responses in a couple of minutes.

Sounds great, what's the catch?

We are continuing to develop  the web portal and additional questions.  What this means is:

  • In exchange for providing reports to you on questions that indicate Report provided, we ask you to collect data on questions marked Data collection ONLY so we can develop reports for those questions.
  • We ask for your feedback about the questions, reports and portal so we can improve them
  • Because this is a pilot, things may not always work as we hope and we may make changes as we try to improve it, so we ask for your patience in this process.

I'm game, sign me up!

If you are interested in participating, or would like more information about the pilot program, please Sign Up Here.

thumbnail of small NSF logo in color without shading

This material is based upon work supported by the National Science Foundation (DUE grants: 1438739, 1323162, 1347740, 0736952 and 1022653). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF.