Abstract. Our objective is to devise a mechanism to characterize and assess upper division and graduate student thinking in hydrology. We accomplish this through development and testing of an assessment tool for a physical hydrology class. The instrument was piloted in two sections of a physical hydrology course. Students were asked to respond to two questions that probed understanding and one question that assessed their ability to apply their knowledge, both prior to and after the course. Student and expert responses to the questions were classified into broad categories to develop a rubric to score responses. Using the rubric, three researchers independently blind-coded the full set of pre- and post-artifacts, resulting in 89% inter-rater agreement on the pre-tests and 83% agreement on the post-tests. The majority of responses made by students at the beginning of the class were characterized as showing only recognition of hydrology concepts from a non-physical perspective; post surveys indicated that the majority had moved to a basic understanding of physical processes, with some students achieving expert understanding. Our study has limitations, including the small number of participants who were all from one institution and the fact that the rubric was still under development. Nevertheless, the high inter-rater agreement from a group of experts indicates that the process we undertook is potentially useful for assessment of learning and understanding physical hydrology.