TY - GEN
T1 - Building AI Competency Knowledge Graphs with LLMs
T2 - 14th International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2025
AU - Tang, Zhuoyuan
AU - Wei, Wei
AU - Yang, Yi
AU - Zhang, Shile
AU - Lam, Chi Kin
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025
Y1 - 2025
N2 - With the rapid advancement of artificial intelligence, the demand for AI talent is constantly evolving. Traditional expert-driven competency modeling approaches suffer from slow update cycles, hindering their ability to provide timely educational guidance. This study proposes an AI competency knowledge graph constructed using large language models (LLMs), enabling the transformation of unstructured recruitment texts into structured educational knowledge through an end-to-end automated framework. A total of 1,142 industry job postings were collected, and competency entities were extracted using few-shot prompt engineering. A two-stage strategy combining semantic embedding and LLM-assisted validation was employed for entity alignment and standardization. The method achieved a micro F1 score of 72.5% on a validation set of 120 samples, resulting in a knowledge graph containing 5,793 standardized competency entities. Application cases such as core skill identification and personalized career planning demonstrate the graph's applicability in curriculum design, career guidance, and learning support. This research establishes a data-driven approach for translating dynamic labor market demands into structured educational knowledge, providing a digital foundation for AI education.
AB - With the rapid advancement of artificial intelligence, the demand for AI talent is constantly evolving. Traditional expert-driven competency modeling approaches suffer from slow update cycles, hindering their ability to provide timely educational guidance. This study proposes an AI competency knowledge graph constructed using large language models (LLMs), enabling the transformation of unstructured recruitment texts into structured educational knowledge through an end-to-end automated framework. A total of 1,142 industry job postings were collected, and competency entities were extracted using few-shot prompt engineering. A two-stage strategy combining semantic embedding and LLM-assisted validation was employed for entity alignment and standardization. The method achieved a micro F1 score of 72.5% on a validation set of 120 samples, resulting in a knowledge graph containing 5,793 standardized competency entities. Application cases such as core skill identification and personalized career planning demonstrate the graph's applicability in curriculum design, career guidance, and learning support. This research establishes a data-driven approach for translating dynamic labor market demands into structured educational knowledge, providing a digital foundation for AI education.
KW - Competency
KW - Educational Technology
KW - Entity Extraction
KW - Knowledge Graph
KW - Large Language Model
UR - https://www.scopus.com/pages/publications/105033234887
U2 - 10.1109/TALE66047.2025.11346567
DO - 10.1109/TALE66047.2025.11346567
M3 - Conference contribution
AN - SCOPUS:105033234887
T3 - TALE 2025 - 2025 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, Proceedings
BT - TALE 2025 - 2025 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 4 December 2025 through 7 December 2025
ER -