A Hybrid AI Framework for CV Screening with CNN-Based Layout Classification and Open-Source LLMs

Setiawan, Johan (2025) A Hybrid AI Framework for CV Screening with CNN-Based Layout Classification and Open-Source LLMs. In: 8th International Conference on New Media Studies (CONMEDIA), 14-17 Oktober 2025, Malacca, Malaysia.

Text A Hybrid AI Framework for CV Screening with CNN-Based Layout Classification and Open-Source LLMs.pdf Download (2MB)

Abstract

The growing influx of digital Curriculum Vitae (CV) has created significant challenges for recruiters in efficiently identifying qualified candidates. This study proposed a dual-stage hybrid artificial intelligence (AI) framework that integrated Convolutional Neural Network (CNN)-based visual classification with semantic analysis using an open-source Large Language Model (LLM) to streamline the recruitment workflow. In the first stage, the CNN model classifies CV with 98% accuracy and evaluates their compatibility with Applicant Tracking Systems (ATS), achieving 92% classification accuracy. In the second stage, a fine-tuned LLM extracts and evaluates candidate information, generates concise summaries, and identifies missing key competencies. Unlike proprietary API-based solutions, the proposed system leverages open-source models to ensure transparency, enable local deployment, and support ethical handling of personally identifiable information (PII). Experimental results show an F1-score above 0.86 for skill extraction and summary evaluation. The framework outputs structured data in JSON format for seamless integration with recruitment platforms. Despite limitations related to dataset availability and evaluation diversity, the proposed approach demonstrates strong potential for scalable, interpretable, and customizable CV analysis in modern human resource systems. Future work would explore job-matching capabilities, multilingual support, bias detection, and compliance with global data protection regulations. This research underscores the feasibility of integrating vision and language models to develop ethical and intelligent recruitment workflows.

Item Type:	Conference or Workshop Item (Paper)
Creators:	Setiawan, Johan
Contributors:
Keywords:	Applicant Tracking System ,Convolutional Neural Network, CV Screening, Large Language Model, Recruitment Automation
Subjects:	000 Computer Science, Information and General Works > 000 Computer Science, Knowledge and Systems > 004 Computer Science, Data Processing, Hardware > 004.2 Systems Analysis and Design, Information Architecture, Performance Evaluation 000 Computer Science, Information and General Works > 000 Computer Science, Knowledge and Systems > 006 Special Computer Methods > Artificial Intelligence, Machine Learning, Pattern Recognition, Data Mining
Divisions:	Faculty of Engineering & Informatics > Information System
Date Deposited:	03 Feb 2026 10:04
URI:	https://kc.umn.ac.id/id/eprint/44545

Actions (login required)

View Item