You are on page 1of 9

Clinical Nurse SpecialistA Copyright B 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins

Feature Article
Validity and Interrater Reliability of the Moline-Roberts Pharmacologic Sedation Scale
Beverly Moline, MS, RN-BC, ACNS-BC n Melanie Roberts, MS, APRN, CCNS, CCRN n Janet Houser, PhD, RN
Cohen 0 ranged from 76.4% to 97.4%. The Cohen 0 P value for all components at all points in time was statistically significant at P G .001. Conclusion: The Moline-Roberts Pharmacologic Sedation Scale demonstrated content validity and strong reliability. Implications: The sedation scale has clinical value in providing a standardized assessment and quantitative assignment of pharmacologically induced sedation that is reflective of the continuum of sedation. Information obtained regarding the patients sedation should be documented, trended, and incorporated into the decision-making process regarding additional administration of agents that produce or potentiate sedation. Further research is needed in populations not included in this study. KEY WORDS: arousal/drug effect, clinical nursing research, nursing assessment

Purpose: The objective of this study was to determine validity and reliability of the Moline-Roberts Pharmacologic Sedation Scale. Design: A multidisciplinary expert panel was used to establish content validity. Reliability was determined by a prospective, randomized, psychometric evaluation of sedation assignment made by 2 nurse research assistants. Setting: The study was conducted in a 260-bed nonprofit community hospital. Sample: Eighty-six subjects were enrolled in the study. Inclusion criteria were as follows: receiving opioids, benzodiazepines, or anesthetic agents; ability to understand English; and normal or near-normal hearing. Methods: Two bachelor of science in nursingYprepared nurses observed each subject and independently documented sedation levels at 3 or 4 points in time for each patient. Findings: Content validity resulted in 100% agreement that the sedation scale reflected the concept of pharmacologic sedation. Internal reliability as measured by Cronbach " was .983 to .996. For each of the scales components, interrater reliability using
Author Affiliations: Acute Pain Clinical Nurse Specialist (Ms Moline), Poudre Valley Health System, Poudre Valley Hospital, Fort Collins, Colorado; Critical Care Clinical Nurse Specialist (Ms Roberts), Medical Center of the Rockies, Loveland, Colorado; Academic Dean (Dr Houser), RueckertHartman College for Health Professions, Denver, Colorado. This study was supported in part by the Colorado Trust/Five Million Lives Campaign, the Poudre Valley Hospital Foundation, and Medical Center of the Rockies Foundation. The Moline-Roberts Pharmacologic Sedation Scale is a copyright of Poudre Valley Health System. The authors report no conflicts of interest. Correspondence: Melanie Roberts, MS, APRN, CCNS, CCRN, Medical Center of the Rockies, 2500 Rocky Mountain Ave, Loveland, CO 80538 (mgr@pvhs.org). DOI: 10.1097/NUR.0b013e3182503fd6

xcessive sedation has long been considered a clinical end point that reflects imminent respiratory depression.1 Sequential sedation assessment is an important nursing function in the prevention of respiratory depression as sedation precedes opioid-induced respiratory depression.2,3 The value of periodic sedation assessment has been underscored by the American Pain Society, which clearly states: No patient has succumbed to respiratory depression while awake.4 In addition to opioids, sedation is a property of other pharmacologic agents including neuroleptics, anxiolytics, hypnotics, antidepressants, antihistamines, and some muscle relaxants.5Y6 These agents increase the risk for respiratory depression particularly in opioid-naive patients.4 The physiologic mechanisms that induce and influence sedation and respiratory depression are diverse, complex, and modulated by such factors as sleep, pain, genetics, stress response, drug interactions, and agent-specific pharmacokinetics and pharmacodynamics.7Y13
May/June 2012

140

www.cns-journal.com

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

A review of a hospitalized patients pharmacologic profile will often reveal the use of more than 1 sedating agent. In addition to opioids for analgesia, the patient may be receiving a benzodiazepine (eg, lorazepam) to reduce anxiety,6 promethazine to treat protracted nausea,6 and/or a central-acting skeletal muscle relaxant such as cyclobenzaprine hydrochloride for muscle spasms in both the surgical and medical patient.6 Even anesthetic agents and techniques may influence sedation and respiratory depression beyond discharge from the postanesthesia care unit.4,14,15 For these reasons, a sedation scale that is not limited to the influence of 1 drug class increases functionality in the clinical setting. Importance Monitoring for excessive sedation is important not only in pain management but also in the setting of moderate sedation/analgesia. Sedation is a continuum,16 and vigilant monitoring is required to prevent transgressing the physiologic boundaries of moderate sedation/analgesia. Dentist C. R. Bennett17 was the first to describe conscious sedation, which is currently referred to as moderate sedation/ analgesia.16 Three objectives of this technique were identified in 1985 and remain relevant today: (1) maintain adequate sedation with minimal risk, (2) relieve anxiety and produce amnesia, and (3) provide relief from pain and other noxious stimuli.18 When providing moderate sedation/analgesia, the intent is not to achieve a specific level of sedation, but rather to titrate medications to provide the patient adequate sedation and analgesia to meet the demands of the procedure.11 Using the least amount of sedation to accomplish the objectives is a goal emphasized in Kosts19 Manual of Conscious Sedation. At the same time, it is essential that the monitoring nurse recognizes when a patient has exceeded the boundaries of moderate sedation/analgesia. Purpose Because excessive sedation can lead to respiratory depression, the clinical imperative is early recognition of increasing or excessive sedation. This is best achieved through vigilant monitoring, documentation, and trending of sedation as there is considerable variability in individual patient response to medications.1,4,7,8,14,20 A valid and reliable sedation scale that provides a standardized method of assessing and quantifying sedation can facilitate monitoring, trending, and communicating an individuals response to potentially sedating medications across care environments.

sedation scales.21Y38 Table 1 provides a summary of the review. All of the studies, except the alertness sedation scale of Chernik et al,35 used intensive care unit (ICU) populations. This study used a small sample of healthy, young men receiving only midazolam. No studies were identified that examined the use of a pharmacologic sedation scale for nonYgoal-directed sedation in adults outside the ICU setting. The Pasero Opioid-Induced Sedation Scale3,38 is used for sedation assessment outside the ICU but is limited to the assessment of opioid-induced sedation associated with pain management. This deficit in the literature underscored the potential value of the Moline-Roberts Pharmacologic Sedation Scale.

REVIEW OF THE LITERATURE A literature search was conducted to review information on published validity and reliability studies of adult
Clinical Nurse Specialist
A

SEDATION SCALE DEVELOPMENT In 1999, the authors desired to develop a tool for early recognition of increasing or excessive sedation. The intent of the tool is to alert nurses to a change in a patients sedation level so that a further assessment, increased monitoring, or other appropriate interventions may be initiated. The goal of which is to decrease respiratory depression. In collaboration with anesthesia department, a pharmacologic sedation scale was developed. This scale had a numeric ranking from 1 to 7 representing the continuum of sedation from alert to unresponsive. In the original scale, the sedation level was based on the patients appearance, degree of stimulation required to awaken the patient, and ability to interact with the environment once awake. Although not published, reliability and validity were established, and the scale was implemented within the organization. The scale was used across care settings to alert nurses to the potential of excessive sedation and impending respiratory depression. Nursing staff value the usefulness of the scale in clinical decision making. In 2007, the authors initiated a revision of the scale that involved merging 2 levels, modifying descriptors, adding airway and ventilation evaluation, and reformatting the scale. The sedation scale was revised, and a research project to establish content validity and interrater reliability was designed and submitted to the organizations internal review board. The project was approved without the need for patient consent based on the following rationale: (a) the patients sedation level was currently being assessed with the previous version of the sedation scale, and (b) no patient care decisions were being made based on the research sedation assessments. Upon completion of the study, the tool was copyrighted. The Moline-Roberts Pharmacologic Sedation Scale represents the continuum of sedation as described by the American Society of Anesthesiologists (ASA)16 and complies with the Joint Commissions guidelines for moderate sedation/analgesia.20 The entire scale is divided into 7 columns. The first 4 columns are sedation assessment parameters, and the fifth column reflects the sedation level. These 5 columns (Table 2) were the focus of the
www.cns-journal.com

141

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Feature Article
Table 1. Sedation Scales Reviewed
Author/Year Olson et al, Riker et al,
22

Scale 2007 1999 Ramsey Sedation-Agitation Scale (SAS) SAS compared with visual analog scale SAS

Population 237 critical care nurses 45 Intensive care unit (ICU) patients 60 ICU patients 39 cardiac surgery patients 69 ICU patients, random 25 ICU mechanically ventilated patients 31 ICU mechanically ventilated patients 246 consecutive ICU encounters 38 patients ICU patients ICU patients

Results

0 = 0.28
Interrater correlation r2 = 0.83, P G .001 Interrater agreement 0 = 0.92, P G .001 Interrater agreement 0 = 0.93, P G .001, Spearman D = j0.77, P G .001 SAS correlated with visual analog scale (r = 0.91, P G .001), bispectral index, 3.2 (r = 0.60, P G .001) Weighted 0 = 0.82 agreement between doctors and nurses Linear trend between MAAS and visual analog scale (VAS) (P G .001) 0 = 0.83 MAAS more reliable than the VAS MAAS less variation Pearson r = 0.75Y0.92, Luer scale r = 0.37Y0.94 Correlation coefficient 0.81 (MAAS) vs 0.79 Interrater reliability r = 0.922Y0.983, 0 = 0.64Y0.82 Interrater reliability 0.89Y0.90, internal consistency = 0.95

24

Brandl et al,25 2001 Riker et al,26 2001

Ryder-Lewis and Nelson,27 2008 Devlin et al,28 1999 Hogg et al,29 2001

SAS Motor Activity Assessment Scale (MAAS) MAAS

Sessler et al,30 2002 de Lemos et al,31 2000 Olleveant et al,32 1998 Laing,33 1992

Richmond Agitation-Sedation Scale (RASS) Vancouver Interaction and Calmness Scale New Sheffield Sedation Scale A new sedation scale for ICU (no name)

0 = 0.73
Favorable reaction from nursing staff, scale easier to understand, more practical, and straightforward than Ramsey, no validity or reliability Weighted 0 = 0.82, r2 = 42% Correlation with Ramsey rs = 0.74 (P = .01) Correlations with visual analog scale 0.90, P G .001

Binnekade et al,34 2006 Chernik et al,35 1990 Weinert and McFarland,36 2004 De Jonghe et al,37 2003

Sedic Scale

46 consecutive ICU mechanically ventilated patients 18 healthy subjects 93 ICU mechanically ventilated patients 80 medical ICU patients

Observer_s Assessment of Alertness/Sedation Scale Minnesota Sedation Assessment Tool Adaptation to the ICU Environment (ATICE)

0 Between raters for arousal scale, 0.85; motor scale, 0.72


Cronbach " consciousness and tolerance domains, .87 and .67, respectively. Intraclass correlation coefficient (interrater reliability) for the same domains, .92Y.99 Reliability POSS Cronbach " = .903 vs RASS = .770 POSS showed higher percent correct scores and correct actions, P e .05

Nisbet and Mooney-Cotter,38 2009

Pasero Opioid-Induced Sedation Scale (POSS), RASS, and Inova Health System Sedation Scale

96 medical-surgical nurses

research study. The sixth and seventh columns of the original and current scale provide guidance in decision making for pain management and moderate sedation and are specific to the organizational environment of the authors. These 2 columns were not included in the validity and reliability research study and were revised in October 2009 to provide greater clarification and direction for nursing actions, particularly for the novice nurse. The reverse side of the tool provides instructions for use.
142
www.cns-journal.com

Color is incorporated into the scale as an alert to the degree of patient risk: sedation levels 1 and 2 are green and represent low risk; 3 and 4 are yellow, reflecting the caution needed at these levels; the dangers of 5 and 6 are reflected by the color red. Specific descriptors for all levels of the scales assessment parameters (presentation, auditory stimulus, tactile stimulus, and response) are provided to ensure a standardized approach to nursing assessment. To assign a sedation level, the patient is
May/June 2012

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Table 2. Components of Sedation Assessment With the Moline-Roberts Pharmacologic Sedation Scale
Presentation Awake, aware, alert Airway and ventilation adequate Restful, drowsy, dozing, lightly sleeping Airway and ventilation adequate Sleeping Airway and ventilation adequate Sleeping Airway and ventilation adequate Sleeping Airway and ventilation may be impaired Sleeping Airway and ventilation likely impaired Auditory Stimulus None Tactile Stimulus None Response Spontaneous, sustained interaction Sedation 1 None to minimal 2 Anxiolysis

Soft voice or ambient noise

None or light touch, rubbing or tapping

Sustains interaction

Soft to normal voice Normal to loud voice Loud voice

Light touch, rubbing or tapping Light touch, rubbing or tapping Intense to noxious

Limited or brief interaction

3 Moderate sedation 4 Moderate sedation 5 Deep sedation 6 General anesthesia

Follows simple commands

Purposeful response or nonpurposeful movement No response, unarousable

Loud voice

Noxious

Copyright B 2008 Poudre Valley Health Care, Inc. d/b/a/ Poudre Valley Health System. Reprinted with permission from Poudre Valley Health Care, Inc. d/b/a Poudre Valley Health System.

observed (presentation) prior to providing any stimulation. Presentation evaluates the appearance of wakefulness, for example, eyes open, alert, and interactive with the environment, or eyes closed as if sleeping. In addition, airway and ventilation are observed to confirm there is no obvious sign of airway obstruction and that respiratory exchange is satisfactory. Obtaining respiratory rate alone is inadequate. Observation of depth and quality (eg, regularity, chest excursions, and sound of breathing) is also an important evaluation.3 If the patient appears to be sleeping, then stimulation (auditory stimulus with or without tactile stimulus) as described in the tool is provided. Stimulation is initiated at the lowest level (lightest sedation) with progression to more intense stimulation (greater/higher sedation) until a defined response has been elicited, or noxious stimulation fails to produce a response. Failure to start at the lowest level may result in inaccurately placing the patient at a higher sedation rating. Lastly, the extent of the patients ability to interact with the environment (response) is assessed. The fifth column, sedation level, provides a numerical sedation rating ranging from 1 (no sedation) to 6 (a level of sedation that is consistent with a patient in a state of general anesthesia). The sedation rating selected represents the assessment findings from the first 3 columns. All parameters, presentation, stimulus (auditory and tactile), and response, are considered in totality to assign a sedation level but may not always be given equal weight. For the Clinical Nurse Specialist
A

safety of the patient, airway or ventilation impairment or decreased ability to interact with the environment should be given priority in the assignment of a sedation level. This scale is intended to be utilized by nurses as nursing judgment may be required in some patient evaluations. For example, nurses must determine whether a hearingimpaired persons need for greater auditory stimulation is due to level of sedation or sensory impairment.

METHODS Setting The study was carried out in a 260-bed nonprofit hospital. Clinical areas included in the study were those that provided the greatest opportunity to assess pharmacologically induced sedation changes over time. Areas included cardiac procedure unit, center for gastroenterology and special procedures, ICU, radiology, inpatient units, and surgical environments of the operating room including preanesthesia and postanesthesia care units and a 1-day surgery center. Many patients were enrolled in the preanesthesia care environment and followed through the operating room, postanesthesia care unit, 1-day surgery center, or postoperative bedded unit.
Sample Inclusion criteria were receiving opioids, benzodiazepines, or anesthetic agents; the ability to understand English; and normal or near-normal hearing. The exclusion criteria were
www.cns-journal.com

143

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Feature Article
cognitive impairment (prior to medication administration), those who had a hearing impairment with or without a hearing aid, neurologic injury affecting ability to respond, and any patient receiving sedating medications other than those listed in inclusion criteria. A total of 86 patients were enrolled in the study: 71 patients had sedation assessments at 4 points in time; 15 patients had sedation assessments at 3 points in time. The timing of assessments was not stipulated in the research protocol. Table 3 shows the sample demographics. Assessment of Validity Content validity was appraised using the expert panel method. Panel members were selected for their knowledge of pain management, sedation standards, participation in establishing organizational pain and sedation policies/protocols, and/or for their depth of experience in patient sedation. Thirteen sedation experts represented a diversified professional panel, consisting of 6 physicians representing anesthesia, pulmonology, trauma surgery, and gastroenterology; 3 advanced practice nurses representing anesthesia, cardiac surgery, and adult health; 3 pharmacists; and 1 experienced nurse manager from a sedation unit. Sedation experts were sent a letter requesting participation in the content validity study along with a preaddressed and stamped return envelope and 4 additional documents: (1) a conceptual blueprint of sedation, which reflected the first 4 columns (presentation, auditory stimulus, tactile stimulus, and response) of the scale; (2) the revised Moline-Roberts Pharmacologic Sedation Scale including the instructions for use; (3) definitions of minimal sedation, moderate sedation sedation/analgesia, deep sedation/analgesia, and anesthesia as defined by ASA19 and the Joint Commission20; and (4) an expert panel evaluation form. Table 4 portrays questions on this form. Assessment of Reliability Interrater reliability is an important consideration if an instrument may be used by multiple caregivers to assess the same patient. This is often the case with sedated patients, and as such, changes in the patients responses must be Table 3. Study Group Demographics
Descriptor Male Female Aged 18Y60 y Aged 61Y80 y Received anesthesia on day of assessment % Study Group 37 63 59 34 78

Table 4. Expert Panel Responses


Expert Panel Responses Do you believe the blueprint reflects the concept of pharmacologic sedation? Do you believe the blueprint reflects the breadth of pharmacologic sedation in the following columns? a. Presentation b. Auditory stimulus: 1 comment regarding patient with hearing impairment c. Tactile stimulus d. Response Do you believe the sedation scale reflects the blueprint? Note: One responder failed to check yes or no 100 90 100 100 100 % Agreement 100

attributable to physiological changes in the patients condition, not to the way the instrument was applied. Second, internal consistency is an important consideration in scalebased measures. Reassurance that each individual item reflects the overall construct is important for efficiency and validity. Finally, stability over time is a critical consideration when an instrument may be used for trending a patients condition over time. Design Interrater reliability was tested using a prospective, randomized design. Two bachelor of science in nursingY prepared nurses were used in conducting the interrater reliability testing. Education was provided to the nurses on the continuum of sedation, the purpose of the scale, and instructions for use. Upon arrival in a clinical area, the nurses collecting data for the research study made contact with the nurse in charge for assistance in identifying potential subjects using inclusion and exclusion criteria, which were printed at the top of the data collection tool. Subjects for the study were selected randomly using the roll of a die. This procedure was followed each day and in each care environment for a total of 6 days. The nurses would simultaneously assess a sedation level for each patient. If the patient appeared to be sleeping, one nurse would provide the stimulus (auditory and tactile if needed) specified on the Moline-Roberts Pharmacologic Sedation Scale. Without consulting each other, the nurses would document findings on a data collection form. The form reflected the blueprint of sedation with presentation, auditory stimulation, tactile stimulation, and response columns. Six sedation levels were represented in rows under each column and contained descriptors for each sedation level. All subjects were assigned a sequential
May/June 2012

144

www.cns-journal.com

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

number, and no identifying information was included on the data collection form. Assessment of interrater reliability was tested using Cohen 0 and percent agreement, along with determination of item-level diagnostics. Evaluation of test-retest reliability was conducted on measures from multiple periods using Pearson product-moment correlation, and internal consistency was measured using Cronbach " statistic.

Table 6. Interrater Reliability Range and Miscoding Diagnostics


Item-Level Diagnostics Presentation Auditory Tactile Miscoding 2/3 Miscoding 4/5 Miscoding 1/2 Non-specific Miscoding 5/6 Interrater Reliability Range 76.4%Y92.2% 81.0%Y87.8% 77.2%Y88.5% 84.1%Y97.4% 90.6%Y97.0%

RESULTS Content Validity A response rate of 77% represented all professional disciplines. There was 100% agreement that the conceptual blueprint reflected the concept of pharmacologic sedation. Table 4 shows additional responses.
Interrater Reliability Interrater reliability was established by calculating agreement between raters within each period for each component of the scale. Rater-by-rater tables were constructed, and Cohen 0 statistic was used to quantify the level of agreement and the probability that agreement was achieved Table 5. Interrater Reliability Scores
Rater 1 Rater 2 Interrater Reliability Period 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 4 4 4 4 4 Scale Presentation Auditory Tactile Response Sedation level Presentation Auditory Tactile Response Sedation level Presentation Auditory Tactile Response Sedation level Presentation Auditory Tactile Response Sedation level
A

Response Sedation level

Cohen 0 92.2 87.7 86.7 97.4 95.1 81.7 87.8 83.1 89.7 91.8 85.3 85.8 88.5 92.2 97.0 76.4 81.0 77.2 84.1 90.6

P
G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001 G.001

by random error. Table 5 demonstrates the interrater reliability for each period and scale, the 0 statistic, and statistical significance. As time passed, interrater reliability was reduced, with the lowest reliability in the tactile scale in period 4, and the highest in the response scale in period 1. All scores were well within the acceptable range of 77.2 to 97.4 (with a preset standard of 70%). In all 4 periods, sedation level was consistently greater than 90 and ranged from 90.6 to 97. Only 2 period/scale scores were in the 70- to 80-point range; all others exceeded 80%. A full set of item-level diagnostics was calculated for each period. These diagnostics allowed the determination of which items generated the most confusion between raters. The most common miscodings were in sedation level and the least in tactile. Table 6 reflects the interrater reliability range and miscoding diagnostics by scale. Reliability of the Instrument Over Time Stability of the instrument over time was tested using correlation analysis. The instrument was only weakly reliable between time 1 and time 2, with item-specific correlation coefficients ranging from .07 to .174. None of these were statistically significant. Correlation coefficients for stability between periods 2 and 3 and periods 3 and 4 steadily improved. Correlation coefficients from periods 2 and 3 ranged from .319 to .549 and from periods 3 and 4 ranged from .399 to .729. All of these correlations were statistically significant. The weakest stability over time was in the respiratory assessment, and the strongest in the response category, followed by overall sedation. These data indicate that the instrument becomes more stable as time passes, perhaps as raters become familiar with the instrument. Test-retest correlations by scale are shown in Table 7. Internal Reliability Internal reliability was established for the overall instrument at each period to reflect internal consistency. The instrument had strong internal reliability as measured by Cronbach " at all periods, ranging from .983 in period 4 to .996 in period 1. Internal reliability is intended to
www.cns-journal.com

Clinical Nurse Specialist

145

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Feature Article
Table 7. Test-Retest Correlation Coefficients use this instrument to assess sedation level with assurance of its validity and stability. To prepare the scale for use throughout the organization, column 6, action for attentive pain management, and column 7, action for moderate sedation/analgesia, were revised. These 2 columns were part of the original scale and were developed to assist the novice nurse with decision making and promote consistency of care. The content in these 2 columns is specific to the organization and was not part of the research project. These columns provide direction for when to decrease, stop, or reverse opioids/sedating medication; further assess a patients risk; increase monitoring; and activate additional resources such as rapid response team.

by Scale
Scale Respiratory From Period 1 2 3 Presentation 1 2 3 Tactile 1 2 3 Response 1 2 3 Auditory 1 2 3 Sedation level 1 2 3
a

To Period 2 3 4 2 3 4 2 3 4 2 3 4 2 3 4 2 3 4

>
0.07 0.319a 0.399a 0.174 0.556a 0.633a 0.161 0.482a 0.668a 0.152 0.549a 0.729a 0.09 0.508a 0.648a 0.163 0.539a 0.711a

CONCLUSIONS The study demonstrated the internal stability of the scale with reliability over time and across individuals. The Moline-Roberts Pharmacologic Sedation Scale is a valid and reliable tool for quantifying sedation in the populations studied. Routine sedation assessment, documentation, and trending can provide valuable input into the decision-making process related to further administration of agents that produce sedation. IMPLICATIONS The ongoing administration of medications is a dynamic process and requires knowledge of both the desired outcomes and the potential adverse effects. Repeated and vigilant nursing assessments are essential as individuals have unique and unpredictable responses to pharmacologic agents even within the same hospital stay.1,14,20,39 Currently, the literature does not offer a valid and reliable sedation scale that will quantify the impact of multiple sedating medications in nonYintensive care settings. Having 1 sedation scale to assess sedation induced by opioids and other pharmacologic agents offers an advantage to nursing staff and promotes improved patient outcomes by facilitating consistency in sedation assessment and communication as the patient moves across the care continuum. The Moline-Roberts Pharmacologic Sedation Scale mandates the evaluation of airway and ventilation, which are important variables in assessing respiratory status and the risk for respiratory depression. The assessment of ventilation facilitates early identification of hypoventilation, a common adverse effect of opioids and other sedating agents. This pharmacologic sedation scale meets the essential criteria for clinical tools: feasibility and clinical utility.40 It is easy to understand, takes seconds to use, and supports quality outcomes by standardizing assessment, quantifying sedation, and providing a consistent method for monitoring, trending, and communicating sedation. In addition, this information is utilized by the
May/June 2012

P G .001.

reflect how strongly each individual item represents the overall construct being measured by the instrument. Given that all of these scales were intended to represent sedation, one would expect " values near 1.0.

DISCUSSION The Moline-Roberts Pharmacologic Sedation Scale demonstrated content validity and reliability. A panel of content experts had strong agreement that the instrument reflected the predetermined blueprint, which presented the continuum of sedation from alert to unresponsive. The raters using this instrument achieved moderately strong to strong interrater reliability. Cohen 0 statistic, representing percent agreement adjusted for random error, ranged from 76.4% to 97.4%. It is of note that only 2 periods on 2 scales fell below 80% agreement. Seven periods and scales exceeded 90% interrater reliability. From a clinical perspective, the most important measure would be the assignment of a sedation level. It is reassuring that the strongest scale on all measures was sedation, with Cohen 0 from 90.6 to 97.0 with a P G .001. The instrument possesses good content validity, internal consistency, and test-retest and interrater reliability. Nursing staff can
146
www.cns-journal.com

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

nurse in the decision-making process regarding the safe administration of additional pharmacologically sedating agents. Color coding of the scale serves as a visual queue for increasing risk to the patient and the need for increased monitoring and vigilance by the nurse. The components and descriptors of the scale support the ASA and Joint Commissions statements regarding the definitions and continuum of sedation.25,26 To facilitate consistent education and establish baseline competency prior to implementation, all nursing staff (a) were provided 4-page self-learning packet about the revised Moline-Roberts Pharmacologic Sedation Scale and (b) viewed a video that included a review of each sedation level followed by a nurse assessing a patient demonstrating the characteristics of that sedation level. At the end of the video, participants view 7 patient scenarios representing the 6 sedation levels. After each patient scenario, participants assign a sedation level based on their nursing assessment of the patient portrayed in the video. At the time of implementation, 97% of approximately 600 nurses attained a score of 100%.

STUDY LIMITATIONS Submitting the scale to only content validity is a limitation of this study. Criterion-related validity and construct validity were not undertaken in this study. In addition, the populations studied were primarily limited to surgical and procedural patients. Future studies should focus on medical, intensive care, and other patient populations both in our organization and in other acute care settings. The nurse researchers were provided education regarding the revised Moline-Roberts Pharmacologic Sedation Scale used in this study; in addition, they were employees of the organization and had extensive experience with the existing sedation scale. Their familiarity and experience with the current tool may have resulted in greater interrater agreement with the revised tool if compared with nurses with no previous exposure to the sedation scale.
ACKNOWLEDGMENTS The authors thank the nurse research assistants and Jerry Carlson, medical librarian for Poudre Valley Health System. References
1. Dolin SJ, Cashman JN. Tolerability of acute postoperative pain management: nausea, vomiting, sedation, pruritus, and urinary retention. Evidence from published data. Br J Anaesth. 2005; 95(5):584Y591. 2. Wells N, Pasero C, McCaffery M. Improving the quality of care through pain assessment and management. In: Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality; 2008. AHRQ Publication 08-0043. http://www.ahrq.gov/qual/nurseshdbk. Published June 12, 2008. Accessed July 30, 2009. 3. Pasero C. Assessment of sedation during opioid administration for pain management. J Perianesth Nurs. 2009;24(3):186Y190.

4. American Pain Society. Principles of Analgesic Use in the Treatment of Acute Pain and Cancer Pain. 5th ed. Glenview, IL: American Pain Society; 2003;30Y31. 5. Bourin M, Briley M. Sedation, an unpleasant, undesirable and potentially dangerous sideYeffect of many psychotropic drugs [article taken from Wiley Interscience]. Hum Psychopharmacol Clin Exp. 2004;19:135Y139. Published January 27, 2004. doi: 10.1002.hup561. Accessed May 20, 2009. 6. Thomson Reuters. Micromedex Healthcare Series: DRUGDEX Drug Point Web site. http://www.thomsonhc.com/hcs/librarian/ND_T/ HCS/ND_PR/Main/CS/C0D39D/DUPLICATIONSHIELDSYNC/ 91ACC5/ND. Accessed November 3, 2009. 7. Holden JE, Forrest JM. The endogenous opioid system and clinical pain management. AACN Clin Issues. 2005;16(3): 291Y301. 8. Paice JA. Pharmacokinetics, pharmacodynamics and pharmacogenomics of opioids. Pain Manage Nurs. 2007;8(3):S2YS5. 9. Trescot AM, Datta S, Lee M, Hansen H. Opioid pharmacology. J Am Soc Intervent Pain Physicians. 2008;11(2s, opioid special issue):S133YS153. Published April 16, 2008. http://www .painphysicianjournal.com/2008/march/2008;11;S133-S153.pdf. Accessed April 16, 2009. 10. Pattinson KTS. Opioids and the control of respiration. Br J Anaesth. 2008;1000(6):7474Y7458. 11. Gan TJ. Pharmacokinetic and pharmacodynamic characteristics of medications used for moderate sedation. Clin Pharmacokinet. 2006;45(9):855Y869. 12. McCrimmon DR, Alheid GF. On the opiate trail of respiratory depression. Am J Physiol Regul Integr Comp Physiol. 2003: 255:R1274YR1275. 13. Lotsch J, Dudziak R, Freynhagen R, Marschner J, Geisslinger G. Fatal respiratory depression after multiple intravenous morphine injections. Clin Pharmacokinet. 2006;45(11):1051Y1060. 14. Overdyk FJ, Moon R, Oglesby H, et al. Challenges in prevention of postoperative respiratory depression and opioid therapy. Initiatives Safe Patient Care. 2009;1:2Y4. Published January 16, 2009. http://www.initiatives-patientsafety.org/Initiatives1%20.pdf. Accessed October 10, 2009. 15. American Society of Anesthesiologists, Inc. Practice guidelines for the prevention, detection, and management of respiratory depression associated with neuraxial opioid administration. Anesthesiology. 2009;110(2):1Y13. 16. American Society of Anesthesiologists, Inc. Practice guidelines for sedation and analgesia by nonanesthesiologists. Anesthesiology. 2002;96:1004Y1017. 17. Bennett C. Conscious Sedation in Dental Practice. 2nd ed. St Louis, MO: CV Mosby; 1978. 18. Scammon FL, Klein SL, Choi W. Conscious sedation for procedures under local or topical anesthesia. Ann Otol Rhinol Laryngol. 1985;94:21. 19. Kost M. Manual of Conscious Sedation. Philadelphia, PA: WB Saunders Company; 1998:5. 20. The Joint Commission. Hospital Accreditation Standards. Oakbrook Terrace, IL: Joint Commission; 2008. 21. Ramsay MAE, Savege TM, Shimpson BRJ, Goodwin R. Controlled sedation with aphaxalone-alphadolone. Br Med J. 1974;2:656Y659. 22. Olson D, Lynn M, Thoyer SM, Graffagnino C. The limited reliability of the Ramsay scale. Neurocrit Care. 2007;7:227Y231. 23. Riker R, Fraser G, Cox P. Continuous infusion of haloperidol controls agitation in critically ill patients. Crit Care Med. 1994;22(3):433Y440. 24. Riker R, Picard J, Gilles F. Prospective evaluation of the sedation-agitation scale for adult critically ill patients. Crit Care Med. 1999;27(7):1325Y1329. 25. Brandl K, Langley K, Riker R, Dork L, Qualls C, Levy H. Confirming the reliability of the sedation-agitation scale administered by ICU nurses without experience in its use. Pharmacotherapy. 2001;21(4):431Y436.
www.cns-journal.com

Clinical Nurse Specialist

147

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Feature Article
26. Riker R, Fraser G, Simmons L, Wilkins M. Validating the sedationagitation scale with bispectral index and visual analog scale in the adult ICU patients after cardiac surgery. Intensive Care Med. 2001;27:853Y858. 27. Ryder-Lewis M, Nelson K. Reliability of the sedation-agitation scale between nurses and doctors. Intensive Crit Care Nurs. 2008;24:211Y217. 28. Devlin J, Boleski G, Mlynarek M, et al. Motor activity assessment scale: a valid and reliable sedation scale for use with mechanically ventilated patients in an adult surgical intensive care unit. Crit Care Med. 1999;27(7):1271Y1275. 29. Hogg L, Bobek MB, Mion L, et al. Interrater reliability of 2 sedation scales in a medical intensive care unit: a preliminary report. Am J Crit Care. 2001;10:79Y83. 30. Sessler C, Gosnell M, Grap M, et al. The Richmond agitationsedation scale validity and reliability in adult intensive care patients. Am J Respir Crit Care Med. 2002;166:1338Y1344. 31. de Lemos J, Tweeddale M, Chittock D. Measuring quality of sedation in adult mechanically ventilated critically ill patients: the Vancouver interaction and calmness scale. J Clin Epidemiol. 2000;(53, sedation focus group issue):908Y919. 32. Olleveant N, Humphris G, Roe B. A reliability study of the modified New Sheffield sedation scale. Nurs Crit Care. 1998;3(2):83Y88. 33. Laing A. The applicability of a new sedation scale for intensive care. Intensive Crit Care Nurs. 1992;8:149Y152. 34. Binnekade J, Vroom M, de Vos R, de Haan R. The reliability and validity of a new and simple method to measure sedation levels in intensive care patients: a pilot study. Heart Lung. 2006; 35(2):137Y143. 35. Chernik D, Gillings D, Laine H, et al. Validity and reliability of the observers assessment of alertness/sedation scale: study with intravenous midazolam. J Clin Psychopharmacol. 1990; 10:244Y251. 36. Weinert C, McFarland L. The state of intubated ICU patients. Chest. 2004;126:1883Y1890. 37. De Jonghe B, Cook D, Griffith L, et al. Adaption to the Intensive Care Environment (ATICE): development and validation of a new sedation assessment instrument. Crit Care Med. 2003; 31(9):2344Y2354. 38. Nisbet AT, Mooney-Cotter F. Comparison of selected sedation scales for reporting opioid-induced sedation assessment. Pain Manage Nurs. 2009;10(3):154Y164. 39. Olson DM, Graffagnino C, King K, Lynch JR. Toward solving the sedation- assessment conundrum: bispectral index monitoring and sedation interruption. Crit Care Nurs Clin North Am. 2005;17:257Y267. 40. Gelinas C, Loiselle CG, LeMay S, Ranger M, Bouchard E, McCormack D. Theoretical psychometric and pragmatic issues in pain measurement. Pain Manage Nurs . 2008;9(3): 120 Y130.

148

www.cns-journal.com

May/June 2012

Copyright 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

You might also like