My posed question: How can the criteria “to grow cognitively, performatively and affectively in nursing education” be objectively measured?
By exploring the College of Nurses
The College’s definition of competence speaks to the achievement of cognitive, performative and affective learning. “COMPETENCE: The quality or ability of a registered nurse to integrate and apply the knowledge, skills, judgments, and personal attributes required to practise safely and ethically in a designated role and setting. Personal attributes include but are not limited to attitudes, values and beliefs.” The definition pertaining to the competencies themselves also refer to “knowledge” (cognition), “abilities and skills” (performance) and, “attitudes and judgments” (affective thinking) required of nurses to perform safely.
Inherent in these definitions is the idea of growth or learning, plus the document clearly states that critical inquiry (cognitive learning) is an expected skill: “CRITICAL INQUIRY: This term expands on the meaning of critical thinking to encompass critical reflection on actions. Critical inquiry means a process of purposive thinking and reflective reasoning where practitioners examine ideas, assumptions, principles, conclusions, beliefs and actions in the context of nursing practice. The critical inquiry process is associated with a spirit of inquiry, discernment, logical reasoning and application of standards.”
In addressing professional presence, the National Competencies document lists the following affective characteristics: “confidence, integrity, optimism, passion, and empathy” and goes on to address the performative skills of “verbal and nonverbal communications and the ability to articulate a positive role and professional image, including the use of name and title”.
Finally, in discussing requisite skills and abilities (RSA), the College lists seven categories of expected RSAs” cognitive, communication, behavioural, interpersonal, physical, sensory perception and environment. The list addresses all three learning domains.
The document proceeds to list all 123 competencies required of nurses, in specific statements are listed in measurable terms. Each competency is comprised of an action verb, such as modifies, recognizes, modifies, applies, accepts, incorporates, uses, facilitates… These actions can be measured by behaviours and outcomes.
When I apply my chosen definitions of objective measurement for my posed question [“Objective measurement is a repeatable, consistent reference standard, not influenced by subjective influences, that is based in facts.”] I see that it might be too rigid, as my peer suggested it could be. Some of the competencies take subjective, albeit professional, learned, expert subjectivity, to be able to defend. For instance, to objectively prove that a nurse “accepts responsibility”, judgement of his/her overt behaviours needs to occur. To use a consistent reference standard “not influenced” subjectively, is not practical when one is attempting to measure affective learning, attitudes and values. Behaviours are always subject to judgement.
With this in mind, I suggest that the key term of objective measurement be modified to read: “Objective measurement is a repeatable, consistent reference standard, minimally influenced by subjective influences, that is based in as much factual data as possible.”
Reference:
CNO. (2008). National Competencies from the Jurisdictional Competency Process: Entry-level Registered Nurses. Toronto , ON College of Nurses Ontario
*My Question of Interest:* How can the criteria “to grow cognitively, performatively and affectively in nursing education” be objectively measured?
ReplyDeleteIn exploring how other professions objectively measure the three domains of learning, I found a textbook on teaching Lawyers in clinical settings. Ron Stucky’s overall focus seemed to be on how poorly US lawyer students are evaluated on their overall performance. He provided a review of a variety of methods on how to better conduct evaluations, but none was particularly definitive, nor specifically addressed each domain, nor used ojective measurements.
Although there was a succinct and promising segment on the importance of objective measurement of cognitive, affective and performative learning in Chapter 5, the subject seems to subsequently be dropped. Multiple methods were proposed, but each suggestion was either objective measurement on the students’ understanding of the aspects of ethics & professionalism (affective learning), or it was subjective on the part of the instructor. No specific competencies were outlined as in the National Competencies for nurses in Canada. In Chapter 7, Stuckey pointed to physician assessment tools and techniques for potential answers, yet no definitive, valid and reliable method of objective measurement of learning was proffered.
Much attention in this book was given to the importance of clinical learning in order to develop knowledgeable, skilful and ethical lawyers. The problem remained at the end of the book that there was no practical or definitive method for objectively measuring growth in the 3 domains of learning.
The “current grading practices of legal educators function less as a means for measuring student learning than as a means for… “weeding out” students who are not developing the requisite knowledge, skills, and values to pass a bar examination. Stuckey seem to be arguing for the benefit of the profession for the need for learning outcomes and marking rubrics: “We should also consider how accurately we can evaluate student performances without first establishing performance benchmarks.” Stuckey concludes: “We do not propose an answer, but evaluating student learning in client representation courses is an issue that legal educators have not yet adequately addressed.” Wow! This seems to set evaluative procedures (within US academic legal education), for prospective, practicing lawyers, in an archaic, ineffectual, dark age.
Alternatively, in the Journal of American Medicine Drs. Epstein and Hundert state that the three most commonly used assessment methods in medical schools are subjective assessments by supervising clinicians, multiple-choice examinations, and standardized patient assessments. Their main complaint was that assessment formats currently used “ may underepmphasize some important domains of professional medical practice, including interpersonal skills, lifelong learning, professionalism, and integration of core knowledge into clinical practice.” They go on to state that there are better assessment systems that are in use, but identify them as “subjective” versus the desired “objective” measures.
In looking at the affective domain, Epstein and Hundert state there is an under-emphasis on measurement of learning in professional competence and physician-patient relationships. In the performative and cognitive domains, the integration of knowledge and skills are lacking.
In reviewing two professional entities in order to compare nursing’s use of objective measures of learning, the two most promising journals were sadly less than helpful. I must expand my search beyond my key terms and will try specific journals to narrow my quest! Any suggestions would be most appreciated! I hope the rest of you are having better success!
References:
Epstein, R. M. & Hundert, E. M., Defining and assessing professional competence, JAMA, Jan. 9, 2002, at 226, 226-27.
Stuckey, R. (2007). Best practices for legal education, Clinical Legal Education Association. Retrieved from http://www.cleaweb.org/best-practices
The College of Nurses of Ontario National Competencies document seems to provide with you a way to measure competence within the three domain. With your question there are two items that stand out for me. First, is the issue of objective measurement, which you discuss in your post and comment. Second, is the issue of growth. I'd encourage you to rethink the use of this word, because growth implies progression and that is not the same as competence.
ReplyDeleteYour comments about lawyers doesn't surprise me, especially having talked to Amanda (from our class last year). Another idea I had was, since the assessment is performed by an assessor and based on his or her's opinion - who assesses the assessors?
Thanks for your post, Jules.
DeleteI have had the same haunting thought that the word "growth" in my posed question might need to be altered to something else, such as "knowledge" or "learning", but "competence" seems to be to limited, as you suggest. Thanks for helping to clarify.
To answer your question re: who assesses the assessors is two fold: The CNO has standards of practice all nurses must meet and there is a yearly process of reflective practice whereby each nurse must look to his or her own learning needs in order to maintain competence and grow; common practices nurses use for evaluative processes are self and peer reviews and supervisor reviews, with each using the CNO standards.
That supervisor's evaluative opinion you mentionned is an 'informed' opinion based on the supervisor's expertise and experience, the CNO standards and the institution's policies and standards. The common phrase used in nursing is evidence informed practice. This is applied to not only practicing nursing with clients, but applies to all aspects of nursing: management, administration, policy development, research, teaching, entrepreneurship, leadership, and even learning. As an extreme example, nursing unions will protect their members with a "prove it" attitude. If a supervisor cannot defend his or her decision with evidence and facts to discipline a nurse, the union will ensure that the member is reinstated.
The summative feedback a supervisor provides in annual reviews is a means for nurses to not only strive to maintain competence but to grow in knowledge, learning, critical thinking and improved/increased practice.
In my search for literature to prove/answer/round out mu posed question has not been as fruitful as hoped.
Having spent hours scanning material and finding the lawyer's book and the physicians' article as hopeful sources, I feel I'm back to ground zero in my search. More to come...
I'm going to write my comment here, but it applies to your follow-up post of 30th September, too!
ReplyDeleteYou've found and referred to a list of competencies used by the CNONC and included a couple of the definitions they propose. Are they defensible definitions? What happens when you evaluate them?
The most pressing issue here is to determine whether and how the information you've found helps you answer your question. It isn't clear from the post, and I'm having trouble seeing the relevance. That doesn't necessarily mean that this information isn't relevant, just that it isn't obviously relevant. So you need to figure that out and make it clear to yourself and your readers.
In doing that, you'll need to define and clarify your terms. For instance, although your question doesn't ask about competence or achievement (rather, growth), the rest of the post seems to assume that competence, achievement and growth are either the synonymous or closely related. Are they? If so, you need to show your readers the connection.
The first definition you proposed for objective measurement was a good start, though it still faced some problems. The revisions you've made after feedback are improvements, definitely. So: excellent! Now I'm wondering whether most of the things you're proposing to measure are actually measurable (for instance, of them could actually have valid units of measurement?). It seems to me that what you're really looking for here are defensible judgments. The information you share in your comment, regarding law and medicine, reinforces that last point, I think. In each case it seems to be people searching for ways to objectively measure what can't be objectively measured.
The major point here is: If what you wish to measure can't be measured, then whatever measurements you end up with will never be valid. A concern!
The Sept 30th post ends with assertions that well-defined competencies can be used to objectively measure nurses' growth, and that the document supports your assertion that it's possible to objectively measure cognitive, performative and affective growth -- but I don't see any connections between the information you've shared and those conclusions. The reasoning needs to be explained, please, so I can see what I'm missing.
Looking forward to the next post!
Hi, Prof Potter,
ReplyDeleteThanks for the rich food for thought!
I assumed credence for the CNO definitions as it is the College that declares nursing competence. I see that my definition of growth needs, as you and Jules have pointed out, to be clarified. The issue of competence as a core part of growth in learning is what I need to clarify.
The relevance of discussing the CNO the competencies comes from my desire to explore and defend how growth from a novice to experienced nurse is measured. The definition of growth requiring a core component of competence is made more obvious by how the CNO and the AACN measure a nurse’s ability to do the art and science of nursing well. When I typed into the search engine “competence, achievement growth”, it took me immediately to a list where the CNO National Competencies was second on the list! I was surprised! First on the list was an article about kindergarten.
I think, Prof Potter that you have nailed what I have been unable to wrap my brain around… that the answer to my question is that I cannot objectively measure the three domains purely objectively. There is always an aspect of subjectivity. One can only hope that nurses will use evidence informed decision making and professional judgment when evaluating another nurse’s competence and growth. I do respectfully disagree with your statement that the three areas can’t be measured and thus will never be measured; it begs a definition for measurement. This is right where my question asks about objectivity (versus subjectivity). Because there are defined competencies than evidence can be gathered to make an informed judgment whereby professionals can measure whether a competency has been met on a scale of high degree or low degree. There are tribunals whereby the evidence is judged as sufficient to rule if competencies were met, at minimum, or if they are substandard and require on-going education or whether the behaviours exceed expectations. If objectivity can be measured by a high bar then it was either cleared, hit by the high jumper with danger of falling or missed and knocked to a lower level. The evidence of consistent and sometimes one-time behaviours is usually the determination used to judge a clear, knocked or missed high bar.
The clarity of hit, miss or cleared is not as objective as nursing would like to have it thus the competencies are measured in behavioural terms – taking subjective affective and cognitive learning and put it in a similar measurement pool as performative learning.
Thanks for your comments this week, Beth under my previous post. Thanks to everyone else's contributions! I was looking at competencies as objective measures but needed to acknowledge that there is a high level of subjectivity in their interpretation. The presence of professional, ethical, knowledgeable nurses to interpret those competencies means that the human element is very near and clear and subjective. You are all right. My question: "How can the criteria “to grow cognitively, performatively and affectively in nursing education” be objectively measured?" cannot be answered, because my evidence shows that there is too much of a subjective element. That begs the need to re-state my question, as we all discussed in class. I really appreciate everyone's contribution and the better sense of clarity regarding my question. Thank you!
ReplyDeleteMaybe it should read "How can scholarly teachers evaluate students' cognitive, performative and affective learning in nursing education?"