Methods/Statistics
Agreement between Two Methodologies in the Development of a Sexual Violence Risk Index Seema Nayak* Seema Nayak Noelle Horth Eileen Shields
Sexual violence is a significant social and public health problem. The New York State Department of Health developed a county-level sexual violence risk index (SVRI) to identify counties with high proportions of populations at risk for sexual violence. To validate identification of high-risk counties, we developed the index using two method and compared results. Comparing processes and results of the two methods provides insight and validation for future index development efforts.
Twelve individual, relationship and community-level risk factors for sexual violence victimization and perpetration were identified from literature and other online resources. County-level percentages and rates for risk factors were obtained from federal and state resources. In the first SVRI-calculation method, values for each risk factor were standardized by calculating their Z-scores for each county. Z-scores for each county were averaged across factors to create its composite SVRI score. The second SVRI-calculation method used a principal component analysis (PCA) where correlated risk factors were reduced to three components. Composite SVRI scores were calculated by aggregating component scores. In both methods, counties were categorized as low, low-medium, medium-high, and high-risk categories using quartiles of the composite SVRI distribution. County risk categories were compared using percentage agreement and Cohen’s Kappa coefficients to assess agreement between the methods.
With the Z-score methodology, 27% of the 62 counties were identified as high-risk including two in New York City (NYC) and 15 counties in Rest of the State (ROS). Using PCA, about 18% of counties were identified as high-risk including three in NYC and eight counties in the ROS region. Simple and weighted Kappa coefficients of 0.34 and 0.37 showed fair agreement between methods. About 47% of counties showed exact agreement, with highest agreement in the high-risk category. The Z-score method categorized 17 counties as high-risk and included 91% of the 11 counties identified as high-risk by PCA method.
The Z-score methodology was determined as more appropriate as it standardizes values allowing us to make comparisons between counties on the same scale and identify those with highest proportion of risk factors compared to the regional average.