The justification of informed consent requires that a patient be provided with the information necessary for deciding treatment and able to use such information based on reasonable thinking. The clinical decision to consider anyone who has mental disorder as incompetence without objective assessment does not only encroach human rights of the persons with mental illness, but seriously prevent them from being recovered. Hence the objective assessment of competency is needed in mental health. Our study aimed to develop the Korean Tool of Competency to Consent to Psychiatric Hospitalization and to analyze the reliability and validity of this tool.
Totally 98 patients with mental illness who were hospitalized in mental hospital, participated in this study. For the subjects a questionnaire composing of 22 questions of understanding, appreciation, reasoning and expression of a choice was used. To investigate validity of this tool, MMSE-K, insight test, estimated IQ, BPRS were conducted. Its reliability and usefulness were examined with Cronbach's alpha, ICC and ROC analysis respectively and criterion related validation performed.
As results, this tool shows that agreement between raters is relatively high and the confirmatory factor analysis for constructive validation shows that the tool is valid. Also, for criterion related validation, estimated IQ, insight and MMSE are significantly correlated to understanding, appreciation and reasoning. However competence to express a choice did not show any significant correlation with criterion variables, nor showed BPRS any significant correlation with sub-competences.
Our study developed the Korean Tool of Competency to Consent to Psychiatric Admission Treatment in the Mentally Ill, verified the reliability and validity of the tool and analyzed the optimum cutoff to distinguish between competence and incompetence in sub-competences. Korean Assessment Tool of Competency to Consent to Psychiatric Hospitalization (KATOC), analyzed the reliability and validity of this tool and presented the cutoff points by subarea. As a result, the reliability and validity of satisfactory levels were verified, the ROC analysis was implemented based on the clinical assessment and the cutoff points were found in understanding, appreciation, expression of a choice and reasoning. Such findings showed that the tool developed by researchers could be very favorably used in Korea.
In the health care, the justification of informed consent requires that a patient be provided with the information necessary for deciding treatment and able to use such information based on reasonable thinking. In other words, it is prerequisite for such justification that sufficient information is offered to a person who can make a decision in the circumstances that enables him/her to make a voluntary choice. However, when treating physical disease, the decision of an adult must be respected; in case of mental disease, he/she is judged to have no ability even without special reason, if he/she refuses to accept the treatment proposed. This means that different standards are applied to evaluate the autonomy of physical and mental diseases by clinician.
In fact, a psychiatric patient has not been historically recognized as a decision-maker of life-related matters for the lack of recognition of their disorder and for the reason of disability of cognitive functions. Accordingly, WHO presented 25 principles for the protection of persons with mental illness, but showed the paradox of permitting a forced treatment for the reason of optimum treatment and protection from physically harming himself/herself or others at the same time.
In the mental health, not recognizing the competency to consent of psychiatric treatment usually damages treatment relationships and a patient's self-esteem and prevents him/her from regaining his/her civil right by carrying out a responsible role. Accordingly, the mere existence of mental disease does not allow to assume that every patient is incapable, and even if incapacity is confirmed in some areas, such fact does not enable to restrict the competences in other areas
Clinically, social study, current and past histories, mental status examination, and memory test have been used to assess the competence to consent to hospitalization without objective tool. However, memory test solely serves to sort out who is absolutely incompetent, but cannot be used to identify the people who can remember any information but cannot understand or reasonably manage them. And mental status examination is a method to assess the adaptation functions of mental disease patients, but it alone does not discriminate competence status well.
Accordingly, to develop useful assessment tool of competency to consent a number of researchers
Such consent competences, including four sub-competences, are assessed with MacArthur Competency Assessment Tool-Treatment (MacCAT-T), Structured Interview for Competency and Incompetency Assessment Testing and Raking Inventory (MacCAT-T) and Competency Interview Schedule. Each tool has merits and flaws, but MacCAT-T is a tool most used. This tool offers a mental disease patient the information on disease, including symptom, diagnosis and process, characteristics, advantages and dangers of proposed treatment and alternative treatment. Thereafter, the area-specific ability is assessed in terms of understanding, appreciation, reasoning and expression.
Our study developed the Korean Tool of Competency to Consent to Psychiatric Hospitalization based on the MacCAT-T that is popularly used in relevant studies. To develop the tool, we composed the items representative of script, understanding, appreciation, expression and reasoning by the help of mental health professionals, revised grading standards and questions through preliminary survey and conducted this test. The objective of this study is to analyze the reliability and validity of this tool and to present the optimal cutoff points of competence and incompetence by sub-competence.
The participants in this study were 98 psychiatric inpatients. The research ethics committee (Institution Review Board: IRB) in each hospital approved the study respectively. After describing the study to the participants, we obtained the informed consent from them. According to the DSM-IV criteria, there were 72 schizophrenia (73.5%), 25 mood disorder (25.5%) and one obsessive-compulsive disorder (1%). Of these 54 (55.1%) were male and 44 (44.8%) were female with a mean age 36.58 (SD=10.67) years old.
This study used mental status examination, insight examination, intelligence scale and clinical evaluations of psychiatrists to verify criteria-related validity of the tool to assess the ability to consent to hospitalization developed by our researchers.
K-MMSE
To assess insight, this study used a scale developed by David.
To assess the estimated intelligence of an individual, the estimated intelligence was calculated using 4 of 11 subscales of the K-WAIS
The brief psychiatric rating scale (BPRS), developed by Overall and Gorham,
The expert panel, including a doctor in charge, measured the hospitalization consent competences of patients: 0=no ability; 1 (a few); 2 (many). They evaluated patients' competency based on the clinical assessment such as interview with patients and their primary care takers, and observation on patients' behaviors.
The tool to assess hospitalization consent competences was developed by developing scripts, drawing up questionnaires, conducting and revising preliminary tests and conducting the main test. Specifically, each stage is as follows:
1) Script development: We drew up a script that contained the general psychiatric symptoms (concentration reduction, sleeplessness, gloom, uncertainty, auditory hallucination etc), treatment method (medication), treatment merits (symptom improvement), problems non-compliance (symptom deterioration, recurrence etc), merits at the time of hospitalization (regular medication, parallelism with other treatments etc). Symptoms were not restricted to a certain illness, and a wide range of treatment methods, merits and problems were presented. They were applicable to any patient with mental illness. This script was composed of 13 sentences of about 17 lines that can be understood by any person who graduated from primary school. It took approximately 1 minute 30 seconds to 2 minutes to read the script.
2) Development of question items: The items were made to question the script content. The items representative of each area were developed by collecting and analyzing the tools used for assessing the hospitalization decision-making ability of patients through literature review and by deciding the areas of the ability to consent to hospitalization through the discussion of experts. The items developed for the first time amounted to a total of 21: 5 items (understanding); 7 items (appreciation); 3 items (expression of a choice); 6 items (reasoning). Thereafter, a preliminary test was conducted, inappropriate items were deleted or revised through the discussion of experts, and one item was added to the expression of a choice. Finally, the tool to assess the ability to consent to hospitalization, developed by researchers, was composed of a total of 22 items: 5 items on comprehensive ability, 7 items on appreciation, 4 items on expression of a choice and 6 on reasoning.
3) Preliminary study: Two raters conducted a preliminary test for 5 hospitalized patients with schizophrenia. The preliminary test used 21 questionnaires on the competence to consent that were primarily developed by researchers. We discuss on the results and revised questionnaires, script and grading criteria. At this time, patients' answers were transcribed in maximum detail by raters. In questionnaires, the items were deleted or revised that patients were hard to answer or could be misunderstood or raters were hard to assess, and the items were complemented that were deemed to be additionally necessary. Grading criteria were divided in more detail by discussing problems based on the opinions of 2 raters who interviewed patients and assessed their answers respectively. Then we adjusted the grading criteria in order to minimize the discrepancy among the raters.
4) Implementation and grading of the main study: The patients admitted in mental hospital had a test conducted by 12 raters that had master degree or higher in mental health care sector. For consistent testing and grading, test developers carried out training for raters, who communicated with the developers at any time, when questions appear during testing and grading. To determine the reliability between raters, the transcripts were allotted to the two other raters who had participated in develop the test and were remarked by them.
A script on the general symptoms (concentration reduction, sleeplessness, uncertainty, abnormal behavior, auditory hallucination etc) and the necessity for medication and hospitalization etc) was read to them. The script was read not mechanically, but as if a story were slowly told, in a colloquial style in order patients to be easy to understand, important words in the script were emphatically read to them.
Thereafter, to assess their competences of understanding, they were asked in 5 items about how well they remember and understand symptoms, proposed treatment, treatment's advantages and dangers, comparison with alternative treatment etc. Each item was rated in 0-2 points, so the maximum total mark amounted to 10 points. To assess appreciation, they were asked in 7 items about whether they thought treatment and hospitalization were necessary, which symptoms they thought would be improved and which problems they thought would occur. Although the patients did not agree with the proposed treatment, he/she presented reasonable reasons, we evaluated he/she has competence of appreciation. Each item was measured in 0-2 points, so the maximum total mark amounted to 14 points. The competences of expression and reasoning were assessed together. First, to assess the competence of expression, it was measured whether they would follow the proposed treatment, wanted to make such decision, wanted to be hospitalized, and thought they were entitled to make such decision to be hospitalized. In this regard, they were given 4 items, asked to give yes/no to each item. If they could give any answer, they were given 1 point; if not, they were given 0 point. To assess the competence of reasoning, they were asked in 6 items about why they gave yes or no to each item. Specifically, they were asked about why they made such decision and which influence they thought such decision would have. Each item was rated in 0-1 point or 0-2 points, so the total marks ranged from 0 to 8.
SPSS 15.0 for windows and AMOS 7.0 program were used to make descriptive statistical analysis, correlation coefficient analysis, Cronbach's alpha analysis, ICC analysis, factor analysis, confirmatory factor analysis, Receiver Operating Characteristic (ROC) analysis etc, which are used to develop the tools to decide the hospitalization consent competences of mental disease patients.
This study checked the ICC and Cronbach's alpha between raters.
AMOS 7.0 program was used to verify model appropriateness, i.e. whether the developed evaluation tool is made in a way theoretically hypothesized. Kline
This study analyzed whether the developed tool was correlated to Korean Version of Mini-Mental State Examination points, insight, estimated intelligence and BPRS.
To verify the efficiency of this tool, this study used the ROC Curve calculated out between the groups judged to be competence and incompetence by the expert panel and the relevant indexes. This curve, used to assess the accuracy of diagnostic classification, offers a graph of true positive rate, sensitivity vs. false positive rate, 1.0-specificity. The Area under Curve (AUC) shows the efficiency of assessment: the broader area is, the more efficient assessment is. Based on the assessment efficiency of the tool developed, the optimum cutoff point was decided to distinguish the people with consent competence and the people without consent competence.
Means, standard error of means (SEM) and standard deviation of KATOC and criterion variables used in this study were calculated. In the Wechsler's intelligence scale, the average intelligence of subjects was 91.31 (SEM=2.14, SD=20.28), showing an intellectual ability of normal range. KATOC full scale's means, SEM, and SD were 27.12, 0.72, 7.10, respectively. KATOC subscale understanding means was 6.65 (SEM=0.30, SD=2.98); appreciation 10.77 (SEM=0.30, SD=2.98); expression of a choice 3.81 (SEM=0.06, SD=0.57); reasoning ability 5.90 (SEM=0.20, SD=2.01). Criterion variables' means were as follows; MMSE 25.67 (SEM=0.31, SD=3.10), Insight 21.19 (SEM=0.58, SD=5.71), BPRS 31.76 (SEM=1.57, SD=15.54).
The hospitalization consent competence scales for mental disease patients developed by researchers was validated through the procedures of construction validation and criterion-related validation. First, in terms of construction validation, it was checked how appropriate it was to compose the scales of hospitalization consent competence proposed by researchers of 4 subscales. As a result, the appropriateness levels of a 4-factor model showed χ2=72.40, GFI=0.90, TLI (NNFI)=0.95, CFI=0.96, RMSEA=0.05 and SRMR=0.08. Such indexes show the four-factor structure model hypothesized in this study according to the model appropriateness standards proposed by Hu and Bentler
All factor loadings were statistically significant. Such results show that construction validity is seen in the hospitalization consent competence test of mental disease patients proposed by researchers.
In the meantime, this study checked the correlation between the marks of the test and the criterion-related variables. As seen in
The efficiency of the developed tool in determining whether a mental disease patient has hospitalization consent competence was checked using the ROC Curve calculated out between the group (n=27) judged to have no consent decision ability and the group (n=25) judged to have consent decision ability and the relevant indexes. This curve, used to assess the accuracy of diagnosis examination, offers a graph of true positive rate, sensitivity vs. false positive rate, 1.0-specificity. The AUC shows the efficiency of tool: the broader area is, the more efficient evaluation is.
First, the mean value of understanding and expression are 6.65 and 3.81 respectively. Accordingly, the cutoff points of understanding and expression are 6.65 and 3.81 respectively. According to the ROC analysis results, accuracy is 0.71, when the sensitivity and specificity of this scale are 0.84 and 0.57 respectively. Accordingly, the appreciation value with an accuracy of 0.71 is 10.5. When adopting this value as a cutoff point, the efficiency of test is optimal. When its value is 4.5, the competence of reasoning is the highest, with sensitivity, specificity and accuracy having 0.88, 0.39 and 0.64 respectively. Accordingly, when adopting 4.5 with an accuracy of 0.64 as a cutoff point of reasoning, the efficiency of test is optimal.
In short, when understanding, expression, appreciation and reasoning meet the conditions of 6.65 points or more, 3.81 points or more, 10.5 points or more and 4.5 or more respectively, its is judged that there is consent competence; if there is even one scale lower than the cutoff points of the four scales, it is judged that there is no consent competence.
The Korean Tool of Competency to Consent to Psychiatric Hospitalization based on MacCAT-T shows that the agreement between raters is relatively high, as seen in the values from 0.831 to 0.958 and that the inner consistency coefficient is reliable, as seen in the values from 0.908 to 0.979. This is relatively high, as compared to the fact the agreements of two raters for MacCAT-T range from 0.33 to 0.71 according to Crains et al.
Such results correspond to those which the consent sub-competences measured with MacCAT-T are related to MMSE
However, this study shows that BPRS is not significantly related to consent sub-competences. Such result is different from the results that BPRS scores are higher in the group of incompetence
In terms of correlation between consent sub-competences, this study shows that the r values between sub-competences except the correlation between expression competences were 0.212-0.657, showing static correlation. In particular, appreciation showed high correlations of 0.505 and 0.657 with understanding and reasoning respectively. However, Vollman et al.
In addition, this study showed that expression had no significant correlation with criterion-related variables. This is because subjects were 3.81 on the mean in expression score, approaching 4, maximum point. Vollman et al.
Vollmann et al.
Implication. Our study developed the Korean Tool of Competency to Consent to Psychiatric Admission Treatment in the Mentally Ill, verified the reliability and validity of the tool and analyzed the optimum cutoff to distinguish between competence and incompetence in sub-competences. Korean Assessment Tool of Competence to Consent to Psychiatric Hospitalization, analyzed the reliability and validity of this tool and presented the cutoff points by subarea. As a result, the reliability and validity of satisfactory levels were verified, the ROC analysis was implemented based on the clinical assessment and the cutoff points were found in understanding, appreciation, expression of a choice and reasoning. Such findings showed that the tool developed by researchers could be very favorably used in Korea where 90% or more of hospitalized patients are coercive admitted to hospital based on the mere clinical judgment without objective assessment tool. In particular, as psychiatric symptoms show no significant correlation with consent sub-competences, it is difficult to effectively distinguish between consent competence and incompetence with clinical judgment alone, so it is more necessary to develop an objective assessment tool.
It is expected that a involuntary hospitalization will significantly decrease if the forced admission is limited to incompetence judged with an objective tool; if involuntary hospitalization is unavoidable, the more ethical practice of clinicians will be more strengthened by demonstrating the incompetence of the patient involved. Also, the findings showed that the consent competence of a patient is not one dimensional, so it is possible to diminish the malpractice of generalizing the incompetence in one area into those in other areas.
However, this study is limited in selecting subjects. First, all of the subjects consented to the participation in the study and had their symptoms controlled in hospitalization, so the study excluded the patients whose psychiatric symptoms were serious or had a very low consent competence from the beginning. Also, 73.55% of subjects had schizophrenia, making it difficult to make a diagnosis-specific comparison. Accordingly, it is necessary to verify the efficiency of the tool that can generalize patients with various diagnoses in the following studies. In addition, this study developed a tool to assess hospitalization consent competence for hospitalized patients, so it is expected that another tool will be developed that can be used for various areas such as treatment consent competence and study participation consent competence and for outpatients. Moreover, as consent competences themselves can be changed and improved by personal psychosocial circumstances,
This study was supported by a research grant from the Korean Government (MOEHRD), Basic Research Promotion Fund (KRF-2008-321-B00138).
The Receiver Operating Characteristic Curve, in which the sensitivity is plotted against 1-specificity for various possible settings of the cutoff score.
Item statistics, the ICC, and Cronbach's Alpha for KATOC subscales
Pearson correlations between admission competency and criterion variables
*correlation is significant at the 0.05 level (2-tailed), †correlation is significant at the 0.01 level (2-tailed)
Area under the curve in ROC analysis
*under the nonparametric assumption, †null hypothesis: true area=0.5
Coordinates of the curve in the KATOC
The test result variable(s): full scale and subscales has at least one tie between the positive actual state group and the negative actual state group. a The smallest cutoff value is the minimum observed test value minus 1, and the largest cutoff value is the maximum observed test value plus 1. All the other cutoff values are the averages of two consecutive ordered observed test values