Development and validation of the prompt engineering competence scale (PECS)

Gibreel O., Arpaci İ.

INFORMATION DEVELOPMENT, 2025 (SSCI, Scopus) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2025
  • Doi Numarası: 10.1177/02666669251336455
  • Dergi Adı: INFORMATION DEVELOPMENT
  • Derginin Tarandığı İndeksler: Social Sciences Citation Index (SSCI), Scopus, Index Islamicus, Information Science and Technology Abstracts, INSPEC, Library, Information Science & Technology Abstracts (LISTA)
  • Bursa Uludağ Üniversitesi Adresli: Hayır

Özet

In a digitized world where artificial intelligence (AI) is rapidly infiltrating every aspect of our lives, it is crucial to utilize generative AI tools effectively. While the demand for AI technologies is increasing rapidly, challenges arise in their practical use. Accordingly, this study aims to develop a novel scale to measure users' level of competence in prompt engineering. The psychometric properties of the Prompt Engineering Competence Scale (PECS) were evaluated using data obtained from 437 users. An exploratory factor analysis was performed to investigate the factor structure of the PECS. The results revealed a one-factor structure with a Cronbach's alpha value of 0.92. Confirmatory factor analysis results indicated that the one-factor model provided a good fit to the data. In the final stage, the item discrimination index was calculated to improve further the reliability and validity of the newly developed scale. The results suggest that the scale items were able to reliably discriminate users. These findings collectively indicate that the PECS is a valid and reliable instrument for measuring users' level of competence in prompt engineering.