ARTIFICIAL INTELLIGENCE (AI) ASSISTANTS’ EVALUATION OF ENVIRONMENTAL, SOCIAL, AND GOVERNANCE (ESG): HOW CONSISTENT AND RELIABLE ARE THE ASSISTANTS?

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper aims to investigate how consistent and thus reliable individual popular generative artificial intelligence (AI) assistants are in evaluating the environmental, social, and governance (ESG) performance of the top companies/stocks among the S&P 500. The three assistants employed in the underlying study were Meta Llama, Google PaLM, and Microsoft Copilot, which were independently requested to award rating scores to the three ESG performance components, namely, (1) Environmental, (2) Social, and (3) Governance, of the top 40 companies/stocks among the S&P 500. For each of the three assistants, the minimum, the maximum, the range, and the standard deviation of the rating scores for each of the three components were calculated across all the 40 companies/stocks. The rating score difference for each of the three components between any pair of the above three assistants was computed for each company/stock. The mean of the absolute value, the minimum, the maximum, the range, and the standard deviation of the differences for each component between each pair of assistants were calculated across all the companies/stocks. A paired sample t-test was then administered to each component for the rating score difference between each assistant pair over all the companies/stocks. Finally, Cronbach’s coefficient alpha of the rating scores was computed for each of the three components between all the three assistants across all the companies/stocks. These computational results were to signify whether the three assistants accorded discrimination in evaluating each component across the companies/stocks, whether each assistant, vis-à-vis each other assistant, erratically or systematically overrate or underrate any component over the companies/stocks, and whether the three assistants were consistent and reliable in evaluating each component across the companies/stocks. Apart from some ancillary results, it was affirmed that the three assistants were marginally consistent and thus reliable, at least in a sense analogous to convergent validity and internal consistency, in evaluating all the three components of the top 40 companies/stocks among the S&P 500.

Original languageEnglish
Title of host publicationProceedings of the International Conferences on Applied Computing and WWW/Internet 2024
EditorsPaula Miranda, Pedro Isaias, Pedro Isaias, Luis Rodrigues
PublisherIADIS Press
Pages285-292
Number of pages8
ISBN (Electronic)9789898704627
Publication statusPublished - 2024
Event21st International Conference on Applied Computing 2024, AC 2024 and 23rd International Conference on WWW/Internet 2024, ICWI 2024 - Zagreb, Croatia
Duration: 26 Oct 202428 Oct 2024

Publication series

NameProceedings of the International Conferences on Applied Computing and WWW/Internet 2024

Conference

Conference21st International Conference on Applied Computing 2024, AC 2024 and 23rd International Conference on WWW/Internet 2024, ICWI 2024
Country/TerritoryCroatia
CityZagreb
Period26/10/2428/10/24

Keywords

  • Environmental
  • ESG
  • Generative Artificial Intelligence (AI)
  • Governance
  • S&P 500
  • Social

Fingerprint

Dive into the research topics of 'ARTIFICIAL INTELLIGENCE (AI) ASSISTANTS’ EVALUATION OF ENVIRONMENTAL, SOCIAL, AND GOVERNANCE (ESG): HOW CONSISTENT AND RELIABLE ARE THE ASSISTANTS?'. Together they form a unique fingerprint.

Cite this