ANALISIS SISTEM DETEKSI DINI FRAUD PADA TRANSAKSI PERBANKAN MENGGUNAKAN LONG SHORT-TERM MEMORY (LSTM) DAN TRANSFORMER

Abstract

The development of digital banking services in electronic payment channels has led to a significant increase in transaction volumes, accompanied by higher fraud risk. Fraud patterns are dynamic and temporal, making detection based solely on individual transactions ineffective. This study aims to develop an early fraud detection system using a cluster-aware sequential deep learning approach. Transaction data are processed through data cleansing, behavioral feature extraction, and customer clustering based on transaction characteristics. Long Short-Term Memory (LSTM) is employed to learn temporal transaction patterns, while Transformer is used to capture global context and nominal transaction deviations. Both models are integrated through a dynamic ensemble approach with adaptive thresholds for each cluster. Model evaluation is conducted in a supervised manner using PR-AUC as the primary metric, supported by ROC-AUC, Precision, Recall, and F1-Score. The results demonstrate that the cluster-based ensemble approach improves detection stability, reduces false positives, and adapts effectively to differences in customer behavior. Experimental results show that models trained without oversampling provide more stable precision–recall performance on datasets where fraud manifests as extreme behavioral outliers, while SMOTE is used as a comparative scenario.

 Keywords: Fraud Detection, Deep Learning, LSTM, Transformer, Bank