TY - JOUR
T1 - From Job Postings to Curriculum Feedback
T2 - An LLM-Driven Framework for Constructing Role-Stratified AI Competency Knowledge Graphs
AU - Tang, Zhuoyuan
AU - Wei, Wei
AU - Liang, Kai
AU - Lam, Chi Kin
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2026
Y1 - 2026
N2 - AI-related occupations are changing rapidly, whereas higher education curricula typically follow longer institutional revision cycles. Job postings provide timely but unstructured evidence of employer-stated competency demand. Following a design science research approach, this study develops and evaluates a modular framework that transforms AI job postings into a role-stratified competency knowledge graph for curriculum-oriented analysis. We applied the framework to 4,612 job postings from Shenzhen, China, integrating four-dimensional competency extraction, five-stage entity alignment, occupational-role classification, and graph construction. On a 461-record human-annotated reference set, the full Qwen3-Max extraction protocol achieved a precision of 0.898, recall of 0.786, and Micro-F1 of 0.838, with the highest precision and Micro-F1 among the four extraction configurations. On a common stratified review sample of 1,286 manually reviewed entity pairs, the full alignment pipeline achieved a sample-based pairwise cluster precision of 0.960, recall of 0.590, and F1 of 0.731. A separate relaxed-candidate audit found a conditional missed-merge proportion of 6.5% within the relaxed same-dimension candidate space. In a 200-record predicted-label-stratified audit, the equally weighted role-assignment precision was 94.5% (Wilson 95% CI: 90.4%-96.9%). The final graph contains 30,063 nodes and 157,858 relationships and enables descriptive role-level comparisons of employer-stated competency demand. A case analysis using a public talent-cultivation plan demonstrates how graph-derived role profiles can be compared with an existing curriculum to inform program-level review. The study contributes a reusable framework that connects recruitment-derived competency evidence with broader AI education goals, together with a module-level validation strategy that supports transparent assessment and role-informed curriculum review.
AB - AI-related occupations are changing rapidly, whereas higher education curricula typically follow longer institutional revision cycles. Job postings provide timely but unstructured evidence of employer-stated competency demand. Following a design science research approach, this study develops and evaluates a modular framework that transforms AI job postings into a role-stratified competency knowledge graph for curriculum-oriented analysis. We applied the framework to 4,612 job postings from Shenzhen, China, integrating four-dimensional competency extraction, five-stage entity alignment, occupational-role classification, and graph construction. On a 461-record human-annotated reference set, the full Qwen3-Max extraction protocol achieved a precision of 0.898, recall of 0.786, and Micro-F1 of 0.838, with the highest precision and Micro-F1 among the four extraction configurations. On a common stratified review sample of 1,286 manually reviewed entity pairs, the full alignment pipeline achieved a sample-based pairwise cluster precision of 0.960, recall of 0.590, and F1 of 0.731. A separate relaxed-candidate audit found a conditional missed-merge proportion of 6.5% within the relaxed same-dimension candidate space. In a 200-record predicted-label-stratified audit, the equally weighted role-assignment precision was 94.5% (Wilson 95% CI: 90.4%-96.9%). The final graph contains 30,063 nodes and 157,858 relationships and enables descriptive role-level comparisons of employer-stated competency demand. A case analysis using a public talent-cultivation plan demonstrates how graph-derived role profiles can be compared with an existing curriculum to inform program-level review. The study contributes a reusable framework that connects recruitment-derived competency evidence with broader AI education goals, together with a module-level validation strategy that supports transparent assessment and role-informed curriculum review.
KW - AI competency knowledge graph
KW - curriculum feedback
KW - design science research
KW - entity alignment
KW - large language model
KW - recruitment text mining
UR - https://www.scopus.com/pages/publications/105050490567
U2 - 10.1109/ACCESS.2026.3731650
DO - 10.1109/ACCESS.2026.3731650
M3 - Article
AN - SCOPUS:105050490567
SN - 2169-3536
VL - 14
SP - 140015
EP - 140039
JO - IEEE Access
JF - IEEE Access
ER -