INTRODUCTION
Background/rationale
The most commonly used definition of pornography is any form of a representation containing an explicit display of sexual organs or behaviors intended to arouse or increase sexual feelings and thoughts [1]. Forms of pornography include videos, photographs, and writings, and, in recent years, the number of internet pornography has increased. Anonymity, easy accessibility to the internet, and reasonable internet service price make the internet a powerful medium for pornography [2]. The diagnostic criteria of the addictive disorder have been modified in DSM-5 (Diagnostic and Statistical Manual of Mental Disorders. 5th ed.) and now include craving criterion in addition to tolerance, withdrawal, repeated attempts to control use, and physical and psychological problems related to use [3]. Kraus stated that craving is one indicator of addiction and a major characteristic of excessive pornography use [4]. Thus, craving is an essential factor when screening for and treating people with pornography use problems. Kraus applied the definition of substance craving to his definition of pornography craving, which states pornography craving is a temporary but strong desire and impulse to use pornography and, similar to other addictive behaviors, include emotional, cognitive, physiological, and behavioral factors [4-6]. Kraus developed a 12-item, self-report Pornography Craving Questionnaire (PCQ) designed to screen for problematic excessive pornography use [4].
South Korea’s sex industry is rapidly expanding, and internet pornography has been widely circulated in the country [7]. A study showed that approximately 85% of the Korean adult male population use pornography [8]. According to a national survey in 2010, approximately 38% of adolescents reported viewing pornography videos, and about 36% responded using pornographic websites [9]. Another study showed that 47% of adolescents aged between 14 to 16 years use pornography [10].
Various negative aspects related to pornography use have been studied in Korea. Problematic pornography use was associated with the experience of role-playing sexual scenes from pornography and a preference for using pornography to achieve sexual excitement over having sex with a partner [8]. There was a negative correlation between first exposure to obscene material and sexual attitudes and a positive correlation between first exposure to obscene material and intercourse experience [11]. The characteristics of exposure to internet pornography harmed sexual attitudes [12]. Problematic pornography users may have an antisocial sexual attitude [13]. Pornography-friendly sexual orientation influences exposure to internet pornography, and exposure to internet pornography affects sexual drive and behavior [14]. Exposure to sexual stimuli is related to anger and aggressive behavior [15]. Adolescents with no experience encountering internet pornography had a more positive attitude towards sexual consciousness, while adolescents with experience encountering pornography had more uninhibited libidos and were more open to romantic relationships [16].
As the number of reported cases of various problems associated with excessive use of pornography increased, pornography’s problematic use of pornography is now being perceived as a personal issue and a social issue. There is an increasing need for psychological treatments for people with problematic pornography use. A good screening measure is needed to provide appropriate treatments to people.
Kraus and Rosenberg developed the self-report pornography craving questionnaire. It is the measurement tool for male undergraduate students in the united states. It intended to measure the craving for pornography after the three-round validity tests, and the original 20 item tool was shortened to a 12 item tool. Individual items were presented in Table 1. Test-retest reliability was 0.82 (p=0.001). The mean inter-item correlation coefficient was 0.46 so that it can be estimated to be a unidimensional tool. Internal reliability presented as Cronbach alpha was 0.91. There were limitations to this tool. First, it is primarily Caucasian, male undergraduates excluding women students. Second, subjects were relatively young people of pornography users. It is expected to be used for predicting the high-risk situation of pornography craving in the clinic. It was translated into Korean so that it is necessary to evaluate the validity, reliability, and goodness-of-fit of this instrument’s Korean translation. It is essential to identify cultural differences and differential factors that arise when translating the original English questionnaire into Korean. These findings help us determine whether the Korean version of the questionnaire applies to Korean culture. This measurement tool was developed for men University students. However, women students can also crave pornography. Therefore, the women students were included in the subjects.
Objectives
This study aimed to determine the applicability of the PCQ, developed by Kraus and Rosenberg in 2014, as a measuring tool in Korea by validating the Korean translation. Specifically, the followings were executed.
Content validity and consequential validity were tested. Construct validity was tested by confirmatory factor analysis. Reliability was tested by exploratory factor analysis. Criterion validity was tested by the correlation analysis with the results of the other two similar tools. A unidimensionality test was done for analyses based on the item response theory so that the item goodness of fit test and the differential item function test by gender were possible. Cut-off value of this tool was suggested.
METHODS
Ethics statement
It was approved by the Institutional Review Board of the Eulji Medical Center, Seoul (IRB No.: EMCIRB 16-26). Informed consent was obtained from the subjects.
Study design
It is the questionnaire survey-based cross-sectional observation study. It was described according to the STROBE Statement (https://www.strobe-statement.org/).
Setting
The survey was done from April 4, 2016, to June 30, 2016, by dispatching the questionnaire sheet to the students at Hallym University and Eulji University, who agreed to respond to the questionnaire. The questionnaire sheets were distributed to the students who attended the liberal art class. Questionnaire sheets were collected one week after the distribution at the same class. The responses were coded by researchers in the spreadsheet.
Participants
The students of seven liberal art classes were the target of the sampling. The number of target students was about 500. Out of them, 275 students from four classes at Hallym University and 93 students from three classes at Eulji University participated in the survey. Out of 368 responses, ten students were removed due to a lack of valid responses. Out of 358 students, only 226 persons who had viewed pornography within the last 6 months were included. Participants were not limited to the medical students but all Department students in the University. The inclusion criterion is the students at Hallym University and Eulji University in Korea. There was no exclusion criterion. There was also no limitation according to age.
Data sources/measurement tools
The Korean translation (Supplementary Table 1 in the online-only Data Supplement) of the self-report Pornography Craving Questionnaire, including 12 items (Table 1), developed by Kraus and Rosenberg, 4 was used. Two Korean psychiatrists participated in the translation process to ensure the original English questionnaire’s psychiatric meanings are not lost in the translation. This translated version was back-translated into English by a native English speaker. The back-translation was compared and checked with the original English questionnaire to minimize the differences in meaning. The items of the questionnaire had a 7-point Likert scale.
Measurement tool
The reliability, validity, and goodness-of-fit of the measuring tool were tested based on the classical test theory and item response theory. The aspects examined were content validity, construct validity, item goodness of fit, item reliability, generalizability validity, consequential validity, and criterion validity. The detailed evaluation process and methods of analysis provided by the questionnaire validity assessment standards are as follows:
First, for the reliability of the Korean version PCQ, the Cronbach-alpha and correlation between each item were analyzed. The Cronbach-alpha indicates the internal consistency of a measuring instrument. Also, the reliability value of the entire questionnaire was measured in order to evaluate the item reliability.
Second, content validity reflects the degree to which a given measuring instrument measures the latent trait that the instrument purports to measure, based on experts’ feedback. The Korean version PCQ was developed in the United States in 2014 and was reviewed and validated by experts. Thus, it is not necessary to further analyze whether the content of this questionnaire is valid. In this study, however, we examined whether there was any difference in meaning in the translated contents than the English content.
Third, construct validity refers to evaluate whether the instrument intend to measure. Items of a measuring instrument need to be evaluated on whether they measure the researcher’s intended latent construct set to check the appropriateness of inferences made based on the measurements. For the construct validity, exploratory factor analysis and confirmatory factor analysis were performed.
Fourth, to determine the item goodness of fit, the Rasch model was used. Out of the item response theory models, the Rasch model, which is the most commonly used model, was used to evaluate item goodness of fit. The item response theory models the relationship between a study subject’s response and the subject’s latent trait. The assumptions entailing the item response theory are both a unidimensionality, which assumes the instrument’s items measure one trait, and local independence, which assumes that a subject’s response to one item does not influence responses to the other items. The measurement tool was tested for unidimensionality.
Fifth, in order to examine generalizability of gender, gender difference was analyzed with the responses to the questionnaire and checked whether this questionnaire can be used for both genders. Also, based on the item response theory, a differential item analysis was performed on 226 subjects (male 150, female 206) as two students did not specify their gender.
Sixth, consequential validity refers to how the results of a questionnaire fit the purpose and how much the scores contribute to the intended measure. Consequential validity reflects whether the given instrument is appropriate to what is intended to be measured and evaluated based on the feedback from experts. The Korean version PCQ was based on the original PCQ developed by Kraus and Rosenberg in 2014. Thus, further analysis of whether this questionnaire is an appropriate instrument for measuring pornography craving is unnecessary.
Seventh, criterion validity refers to the extent to which the given measuring instrument predicts an outcome of an instrument that has been validated. There was no available instrument for measuring the degree of pornography craving. Therefore, two measurement instruments-Korean translatgion of Young’s Cybersexual Addiction Index [17-19] and the Sexual impulsivity scale of Korean [19]-that are used to measure other traits related to pornography craving were used for the correlation analysis.
Cut-off value of the K-PCQ
It was suggested based on the test information function. The measure of a latent variable at the maximum function corresponded to the raw score of the K-PCQ.
Statistical methods
AMOS ver.18 (SPSS Inc., Chicago, IL, USA) was used for the confirmatory factor analysis. SPSS version 20.0 (IBM Corp., Armonk, NY, USA) was used for exploratory factor analysis. DBSTAT version 5.0 (DBSTAT, Chuncheon, Korea) was used for comparison analysis. Poly-DIMTEST (Assessment Systems Corporation, Minneapolis, MN, USA) was used for the unidimensionality test (Poly-DIMEST program is not commercially available in 2020). Winsteps® program version 4.8.0 (https://www.winsteps.com) was used for item goodness-of-fit test, differential item function, and test information function based on the Rasch model.
RESULTS
Participants
Demographic findings of the participants were collected for gender and ages. From 358 subjects, there were 150 men and 206 women. Two students did not respond to gender items. After excluding the students without viewing pornography, responses from 226 students were included for the analysis. The number of men university students was 136 and that of women was 90. The students were divided into three age groups: 56 students at 18 to19 years old, 82 at 20 to 21 years old, and 88 at 22 years old and older. Raw response data from 226 university students are available from Supplementary Table 2 (in the online-only Data Supplement).
Reliability
The correlations between each item were shown in Table 2. The Cronbach’s alpha value was 0.9363; the degrees of freedom were 221, Significant level at alpha=0.05 was t=1.9709.
Construct validity
In order to explore the number of possible factors, the eigenvalue was explored using exploratory factor analysis. The results of exploratory factor analysis showed that constructs with an eigenvalue greater than 0.5 were factor 1 (7.428), factor 2 (0.878), factor 3 (0.786), factor 4 (0.565), and factor 5 (0.527) (Figure 1). The eigenvalue of factor 1 was much higher than those of the other factors.
The model fit of Korean version of the K-PCQ in the confirmatory factor analysis was presented as follows: χ2 value was 241.947; degree of freedom was 54; Tucker-Lewis index (TLI) was 0.865; comparative fit index (CFI) was 0.907; and root mean square error of approximation (RMSEA) was 0.124. The recommended values for TLI and CFI are larger than 0.95, and RMSEA is less than 0.06 [21]. The TLI, CFI and RMSEA of K-PCQ do not meet the recommended values, but they are close to recommended values. Figure 2 shows the confirmatory factor analysis of K-PCQ. The regression coefficients between K-PCQ and each item are significant at level 0.05.
Item goodness of fit
The goodness-of-fit test was performed to determine whether the questionnaire measures what its developer intended to measure. It also decided if subjects responded according to the questionnaire items and that each item was designed appropriately to the subjects’ level. The results were shown in Supplementary Table 3 (in the online-only Data Supplement). INFIT and OUTFIT are fit statistics index that shows the extent to which an item or study subject does not fit the Rasch model. Infit is more sensitive to the responses to items targeted on the subjects. In contrast, the outfit is more sensitive to the responses to items that are far from the subjects’ ability (outlier). Mean square (MSQ) means mean square fit statistic and has the expected value of 1. The outfit mean square value of item 11 was 2.3367. In the person goodness-of-fit analysis, the number of subjects with outfit mean square value of 2.0 or greater was 29, and those with infit mean square value of 0.5 or less was 42 (Supplementary Table 4 in the online-only Data Supplement).
Generalization for gender and age
For the generalization for gender, the difference in item scores according to gender were analyzed and differential item functioning (DIF) analysis for gender was conducted. The average total score of the questionnaire for the male students was 28.51 (SD: 13.6), and that of the female students was 22.50 (SD: 8.4). Wilcoxon Rank Sum test was done to compare the difference by gender because the two groups did not show the normal distribution. There was a significant difference by gender (p<0.0005). The item scores according to the age group were 23.68 (SD=9.71), 27.06 (SD=11.70), and 26.78 (SD=14.32) from the age groups 18 to 19 years old, 20 to 21 years old, and 22 years-old and older, respectively. The difference in item scores among the three age groups was not statistically significant (p=0.33420) according to the Kruskal-Wallis test. This non-parametric analysis was done because the three age groups did not show normal distribution.
DIF analysis of item response theory was performed to examine whether the questionnaire can be used generally regardless of gender. The analysis showed that all items’ differences in DIF values were not greater than 0.5 (Supplementary Table 5 in the online-only Data Supplement, Figure 3).
Criterion validity
The Cybersexual Addiction Index [17-19] and the Sexual impulsivity scale of Korean [20] were used to examine the criterion validity of the Korean version of PCQ. These two measuring tools have been previously validated and measure the characteristics similar to pornography craving. Thus, they were used as external criteria measures to which the scores of the Korean PCQ were compared. The comparison of scores of the Korean version of the PCQ with scores of the CAI and the Sexual impulsivity scale of Korean showed that the correlation coefficients were (r=0.6683, p<0.0001, and (r=0.3702, p<0.0001), respectively.
Unidimensionality test
Data of 12 items of a 7-point Likert scale with 226 examinees were analyzed by Poly-DIMTEST for unidimensionality test. For the analysis, the grouping of items was required. AT item list was randomly consisted of 1st, 2nd, 3rd, and 4th items. Supplement 2 was a response data of the examinees to 12 items 7-point Likert scale. SSCPARM was inputted as Supplementary Table 6 and Supplementary Table 7 (in the online-only Data Supplement) was a result of the Poly-DIMTEST statistic report, SSCMSG file. P-value was 0.266937.
The cut-off value of the tool
The cut-off value of this scale was -0.098 as theta value (latent trait) of the examinees based on the test information curve (Figure 4, Supplementary Table 8 in the online-only Data Supplement). The highest information was 16.0584, and the corresponding theta value (the latent trait of the students) was -0.08). It corresponds total score of 46 (54.8%) out of the maximum 84 (Supplementary Table 4 in the online-only Data Supplement). If this cut-off value is applied, 18 students are in the positive trends of pornography craving.
DISCUSSION
Key results
The construct validity, item goodness of fit, item reliability, generalizability validity, and criterion validity of the Korean version of the PCQ, which was originally developed by Kraus and Rosenberg in 2014, were evaluated. The above results showed that the Korean version of the PCQ could be used with sufficient validity and reliability in clinical settings. Furthermore, this translated measurement tool can be used for women university students, although the original PCQ was for men university students. Cu-off value of -0.098 as a theta value was suggested for the decision of the pornography craving tendency. This value corresponds to the score of 46 out of a maximum of 84.
Interpretation
The difference in item score by gender
The higher item score of men means that men have more pornography craving tendencies than women. It is not possible to compare it with other studies on pornography craving because there was no data with the same measurement tool, including gender difference.
Construct validity evaluation
The eigenvalue of factor 1 from the confirmatory factor analysis was very high, 7.428. Thus, factor 1 is the underlying factor of this questionnaire, and the rest of the factors are single factors that measure pornography craving. It means that the measurement tool may be explained by one factor, and the entire component of the questionnaire contains pornography craving. As all questionnaire items may be explained by one factor, this questionnaire satisfies the unidimensionality assumption of the item response theory. When applying the item response theory, the unidimensionality assumption and local independence assumption are usually made. There are several ways to evaluate whether the unidimensionality assumption is satisfied, and one of the methods is confirmatory factor analysis. Local independence assumption states that the examinee’s response to one item does not influence his or her response to other items in a questionnaire. All items in the Korean version of the PCQ are sufficiently independent, so the local independence assumption is satisfied.
Item and person goodness of fit
In the item goodness of fit test, an outfit means square values greater than 2.0 mean that an item is too underfit from the content that the researcher is trying to measure. In other words, it may be possible that the item may be misfitting as an outlier with a too extreme value. The fit statistic is used as a fitness index in the Rasch model, and, in general, the fit statistic greater than 2.0 is unpredictability. Fit statistic of 2.0 or greater means that the probability of observing the same response is less than 0.5 based on the principle of statistical hypothesis testing. It is necessary to examine whether the item needs to be removed from the questionnaire if the item is unfit with other items as it may distort the entire test results. The Rasch model analysis showed that, of the 12 items in the Korean version PCQ, only item number 11 had an outfit mean square value greater than 2.0. Item 11 states “My heart would beat faster if I were watching porn right now,” and it is necessary to check why this item underfit the Rasch model. It can be speculated that item 11 had inappropriate terms or was ambiguous, which could have confused the subjects. It is also possible that other traits than the trait influenced the responses to this item it was intended to measure and thus, the unidimensionality assumption was not satisfied. The original English text of item 11 was “My heart would beat faster if I were watching porn right now,” where the expression ‘the heart would beat faster’ was used to mean a physiological change that occurs when sexually aroused. In Korean, however, the expression ‘My heart beats faster” is seldom used as an expression to show sexual arousal and is also used when experiencing negative feelings, including anxiety and fear. In the Korean language, the phrase ‘My heart pounds’ is more often used than ‘My heart beats faster’ when one is experiencing a positive feeling, such as excitement and joy. In American culture, the English expression ‘the heart would beat faster’ is often used to express a positive emotional state, including excitement and joy. This difference may explain why item 11 had an outfit mean square value greater than 2.0. There was no items that had an infit mean square value less than 0.5, and thus none of the items were too overfit to the pattern expected by the Rasch model. The results of these two indexes, infit mean square and outfit mean square, show only one item of the 12 items is out of fit, and the items of the Korean version PCQ are, overall, fit. However, it is necessary to check whether item 11, when revised, will have better item goodness of fit.
The subjects’ goodness of fit test showed that of 226 subjects, 29 (12.8%) subjects had outfit mean square value of 2.0 or greater, whereas 42 (18.6%) had infit mean square value of 0.5 or less. The subjects’ goodness of fit test score of 2.0 or greater means the subject is an outlier. This may result from the subject responding inappropriately to the question due to assumption or mistake, being anxious while taking the test, failing to understand certain parts of an item, responding inaccurately, and an error with the test technique. To analyze the cause of the unfit subjects, further analysis of the subject’s response is needed to check how the subject’s response differs from the expected response and identify the possible cause. The subjects with outfit mean square value of 2.0 or greater did not respond appropriately to the items and were unfit for this questionnaire. There is a possibility that these subjects may have responded to the questionnaire half-heartedly or were uncomfortable with the content of pornography craving that this questionnaire was trying to measure.
On the other hand, the subjects with an infit mean square value of 0.5 or less are overfitting to the questionnaire than expected. These subjects may have consistently responded high or low through the test or have traits that fit this type of questionnaire. The subject’s goodness of fit is useful for interpreting the test results rather than generating question items as each subject’s traits need to be examined.
Item reliability
The Cronbach’s alpha of this measurement tool was 0.9363, which showed that the probability of obtaining the same results is very high when the questionnaire is completed multiple times. Cronbach’s alpha value, which is greater than 0.6 is acceptable. It shows that the consistency among questionnaire items was high, indicating that this questionnaire’s item reliability was high. In general, the reliability of a questionnaire increases as the number of items increases. It is because the quality of response decreases as the number of measuring items increases. In this study, the total number of items in the questionnaire was 12. The reliability of the 226 subjects was very high, which shows that the Korean version of the PCQ has high reliability.
Generalization validity
There was no differential item between the genders in the generalizability validity test. In other words, there was no item with a difference of 0.5 or greater between the men and women students. P values were greater than 0.05 in both the Rasch-Welch model and the Mentel model. In K-PCQ, women students were included. Their response pattern of all items was not different from that of men students according to the DIF results on gender. It means that K-PCQ may be a valid tool for measuring pornography craving for both men and women students. It is the merit of the K-PCQ from the original PCQ. The present survey found that women students (43.7%) also viewed pornography, although the rate was lower than that of men (90.7%).
Criterion validity
A previously validated measuring instrument that measures the same construct is needed. As there was no available measuring instrument that measures pornography craving, two instruments, the Korean translation of Cybersexual Addiction Index [17-19] and the Sexual impulsivity scale of Korean [20], that are accepted to be related to pornography use or measure similar construct, were used. The results showed that the Korean version PCQ was correlated to both the Korean translation of the Cybersexual Addiction Index (r=0.6683, p<0.0001) and the Sexual impulsivity scale of Korean (r=0.3702, p<0.0001).
Cut-off value for the clinician’s assessment
Although this measurement tool is not for the summative evaluation, the cut-off value was suggested for clinicians to discriminate the tendency of pornography craving. It should be interpreted cautiously if the theta value is around the cut-off value. Also, it does not mean the specific value is the criteria of the clinical intervention. Clinical intervention is dependent on the client’s recognition as a problem.
Limitation
A structural equating model may be used in addition to the aforementioned analyses. However, the use of that model may not hold much meaning as the eigenvalue of factor 1 from the confirmatory factor analysis was very high. The use of this tool is limited to university students. It is not a randomized study; therefore, there may be some selection bias of participants.
Generalizability
This study’s results may be applied to University students in Korea. Another survey for other groups with the same tool is recommended for generalizability.
Conclusion
Except for item 11, which had an outfit mean square value greater than 2.0 and require revision, the Korean version PCQ valid in all aspects of item reliability, construct validity, item goodness of fit, generalizability validity, and criterion validity and may be used in clinical settings in Korea regardless of gender with specific cut-off value.
