Adapting and Validating Scales: Translation, CFA and Measurement Invariance

A large share of survey research in educational administration rests on instruments that were first written in English and then carried into another language and school system. Scales of principal leadership, organisational trust, teacher burnout and school climate are routinely translated for use in Turkey and elsewhere, and a thesis often devotes a full chapter, or at least a substantial methods section, to showing that the adapted version still measures what the original measured. Scale adaptation is the process of translating an instrument, checking that its content suits the new context, and gathering statistical evidence (usually through confirmatory factor analysis, reliability estimates and, where groups are compared, measurement invariance testing) that scores from the new version can be interpreted in the intended way.

When to use it

Adapt rather than develop when a well-established instrument already captures the construct you have in mind, when you want your findings to speak to an international literature that uses the same instrument, or when you intend to compare countries. Develop a new scale when the construct is defined differently in your setting, when the original items refer to structures that do not exist locally (for instance, school board roles or tenure arrangements with no Turkish equivalent), or when no instrument with acceptable evidence exists. A half-way position, rewording several items and dropping others, is common in theses but should be described honestly: the further you move from the original, the weaker your claim to be using “the same” instrument.

Before any translation begins, check who holds the rights. The International Test Commission’s guidelines open with a pre-condition that permission be obtained from the intellectual property holder (International Test Commission, 2018). Many scales published in journal articles can be used with the developer’s written consent, but others are sold under licence. The Maslach Burnout Inventory, including its Educators Survey form, is distributed commercially by Mind Garden, whose terms of service require a purchased licence to reproduce or administer its instruments and assign the rights in any translation to the publisher, so contact the publisher before translating. Keep the permission correspondence and mention it in the method section.

Key concepts

Concept	What it refers to	Typical evidence in a thesis
Linguistic equivalence	The translated items carry the same meaning as the originals	Forward and back translation, expert committee review, pilot interviews
Content validity	Items cover the construct and suit the target context	Expert ratings, for example Lawshe’s (1975) content validity ratio
Factorial validity	Items group into the factors the theory predicts	CFA of the original structure; EFA only when the structure is uncertain
Reliability	Consistency of scores within a factor	McDonald’s omega, with Cronbach’s alpha for comparison
Measurement invariance	The scale functions the same way across groups	Multiple-group CFA: configural, metric, scalar models

Step by step

Translate with more than one person

Single-translator versions are a frequent weakness. Beaton et al. (2000) describe a sequence that has become a common reference point: at least two independent forward translations into the target language (one translator aware of the construct, one not), a synthesis of the two, back translations by people who have not seen the original, review by an expert committee, and a pretest in which roughly 30 to 40 respondents from the target population are asked how they understood each item. The ITC guidelines (International Test Commission, 2018) also call for translators who understand both the language and the culture, and for documented judgemental review. A narrow back-translation design, in which the source and back-translated texts are compared but the target-language version is never reviewed in its own right, can miss items that are literally accurate but sound unnatural to Turkish teachers. A committee that discusses every item and records its decisions protects against this.

Check content and cultural fit

Ask subject experts (educational administration academics, experienced principals) whether each item is relevant and clear in the new setting. Lawshe’s (1975) method asks panellists to classify each item as essential, useful but not essential, or not necessary, and converts the proportion rating it essential into a content validity ratio. Pair this with cognitive pretesting, which tells you how teachers actually read the items.

Choose between CFA and EFA

When you adapt an instrument with a known factor structure, the natural test is confirmatory factor analysis: you specify the original model and ask whether your data are consistent with it. Exploratory factor analysis suits situations where the structure is genuinely in doubt, for instance when the original study reported unstable factors or the construct may be organised differently in the new culture. If you run both, use separate samples, or split a large sample at random, so that the CFA is not simply confirming a structure derived from the same responses. Çelik and Üstüner (2019), discussed below, followed this approach.

Estimate reliability properly

Cronbach’s alpha remains the most widely reported coefficient, but it rests on assumptions, notably that all items relate equally strongly to the factor, which rarely hold in practice. McNeish (2018) reviews these limits and recommends alternatives with less restrictive assumptions, among them omega total. Omega is computed from the factor loadings, so it fits naturally after a CFA. In R, the compRelSEM() function in the semTools package calculates omega-type composite reliability from a fitted lavaan model; jamovi’s reliability analysis also reports McDonald’s omega alongside alpha. Report reliability for each factor, not only the full scale.

Judge model fit with care

Most theses cite Hu and Bentler (1999), whose simulations suggested values close to .95 for CFI and TLI, close to .06 for RMSEA and close to .08 for SRMR, and recommended reading SRMR together with another index. These values were never meant as universal pass marks. Marsh et al. (2004) warned against treating them as “golden rules”, and McNeish and Wolf (2023) showed that what a given index value means depends on features of the model such as the number of items and factors and the size of the loadings. They proposed dynamic fit index cut-offs, derived by simulation for the model at hand, and provide a web application for computing them. A careful thesis reports several indices, cites the source of any benchmark, and inspects local misfit (large residuals or modification indices) rather than declaring a model “good” on one number.

Test measurement invariance before comparing groups

If you plan to compare male and female teachers, primary and secondary schools, or Turkish and other national samples, you need evidence that the scale works the same way in each group. Otherwise a difference in means may reflect the instrument rather than the people. Invariance is tested through a sequence of increasingly constrained multiple-group CFA models (Putnick & Bornstein, 2016):

• Configural: the same pattern of items and factors holds in each group.
• Metric (weak): factor loadings are equal across groups, which supports comparing relationships such as correlations or regression paths.
• Scalar (strong): item intercepts (or thresholds) are also equal, which is needed before comparing factor means.
• Residual (strict): item residual variances are equal; this level is not required for comparing latent means.

Each model is compared with the previous one. Because the chi-square difference test is sensitive to sample size, researchers also look at changes in fit indices. Cheung and Rensvold (2002) proposed that a drop in CFI greater than .01 signals non-invariance. Chen (2007) suggested, for metric invariance, a CFI change of about .01 accompanied by an RMSEA change of about .015 or an SRMR change of about .030, with a stricter SRMR criterion at the scalar level; her simulations also showed that these changes behave differently with unequal group sizes. If full scalar invariance fails, partial invariance (freeing a small number of intercepts or loadings) may still allow cautious comparisons, but there is no agreed rule on how many parameters may be freed. In lavaan (Rosseel, 2012) these models are fitted with the group argument, equality constraints are added with group.equal (for example, “loadings”, then “intercepts”), and selected parameters can be freed for partial invariance with group.partial.

What to report

• The original instrument, its developers, factor structure and evidence, and written permission or licence.
• The translation procedure: number and background of translators, committee composition, how disagreements were resolved, and pilot or cognitive interview results.
• Content validity evidence, including the expert panel’s size and the statistic used.
• Sample sizes for each analysis, and whether EFA and CFA used different data.
• Estimator (for example, maximum likelihood with robust standard errors, or a categorical estimator such as WLSMV for ordinal Likert items), handling of missing data, and software with version.
• Standardised loadings for every item, factor correlations, and several fit indices with the sources of any cut-offs.
• Omega (and alpha, if your field expects it) for each factor.
• For invariance: fit of each model, the changes between models, the criteria used, and any parameters freed.

Common mistakes

• Running EFA and CFA on the same sample and treating the CFA as independent confirmation.
• Relying on one translator, or on back translation alone, without a committee review of the Turkish text.
• Reporting only alpha for the total score when the scale has several factors.
• Presenting Hu and Bentler’s values as strict thresholds, or listing indices that look good while ignoring those that do not.
• Deleting items or adding correlated residuals until fit improves, without theoretical justification or reporting.
• Comparing group means (by gender, school type or country) with t-tests or ANOVA without first testing scalar invariance.
• Calling an instrument “valid and reliable” in general, rather than presenting evidence for a specific use and population.

Examples from Turkish research

Turkish educational administration journals contain many adaptation studies, and they vary considerably in method, which makes them useful for comparison. Readers can browse further adapted scales on our instruments page, and related constructs under distributed leadership and organisational trust.

• Özdemir (2012), in Kuram ve Uygulamada Eğitim Yönetimi, adapted Hulpia and colleagues’ Distributed Leadership Inventory with 160 teachers in Ankara, using translation and back translation by educational administration and English language experts. Validity evidence came from EFA, with reliability estimated by alpha and split-half coefficients; no CFA was reported.
• Ersözlü and Ulusoy (2016), in the European Journal of Educational Research, adapted Davis’s Distributed Leadership Scale with 386 high school teachers in Tokat, using both EFA and CFA. The Turkish form retained 34 items across seven factors.
• Çelik and Üstüner (2019), in Kuramsal Eğitimbilim Dergisi, adapted Ferris and Kacmar’s Perception of Organizational Politics Scale for schools. After translation, linguistic checks and a 30-teacher pilot, they ran EFA on 316 middle school teachers and first- and second-order CFA on a separate group of 856, reporting CFI, TLI, RMSEA and SRMR alongside alpha.

Key takeaways

• Obtain permission first, and use a documented, multi-person translation process with committee review and pretesting.
• Use CFA to test the original structure; keep EFA for genuine uncertainty and run it on separate data.
• Report omega for each factor and treat fit-index cut-offs as attributed guidelines, not pass marks.
• Establish at least scalar invariance before comparing group means, whether across genders, school types or countries.

References

• Beaton, D. E., Bombardier, C., Guillemin, F., & Ferraz, M. B. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 3186–3191. https://doi.org/10.1097/00007632-200012150-00014
• Çelik, O. T., & Üstüner, M. (2019). Örgütsel politika algısı ölçeğinin okul örgütleri için Türkçe’ye uyarlanması: Geçerlik ve güvenirlik çalışması [Adapting the Perception of Organizational Politics Scale into Turkish for school organisations: A validity and reliability study]. Kuramsal Eğitimbilim Dergisi, 12(1), 77–96. https://doi.org/10.30831/akukeg.425250
• Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. https://doi.org/10.1080/10705510701301834
• Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
• Ersözlü, A., & Ulusoy, T. (2016). Adaptation of distributed leadership scale into Turkish: The validity and reliability study. European Journal of Educational Research, 5(2), 43–52. https://doi.org/10.12973/eu-jer.5.2.43
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
• International Test Commission. (2018). ITC guidelines for translating and adapting tests (second edition). International Journal of Testing, 18(2), 101–134. https://doi.org/10.1080/15305058.2017.1398166
• Lawshe, C. H. (1975). A quantitative approach to content validity. Personnel Psychology, 28(4), 563–575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
• Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling: A Multidisciplinary Journal, 11(3), 320–341. https://doi.org/10.1207/s15328007sem1103_2
• McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
• McNeish, D., & Wolf, M. G. (2023). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods, 28(1), 61–88. https://doi.org/10.1037/met0000425
• Özdemir, M. (2012). Dağıtımcı liderlik envanterinin Türkçe uyarlaması: Geçerlik ve güvenirlik çalışmaları [Turkish adaptation of the Distributed Leadership Inventory: Validity and reliability studies]. Kuram ve Uygulamada Eğitim Yönetimi, 18(4), 575–598.
• Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004
• Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

Last reviewed: October 2026