Comparison of tabular data generation algorithms using gans
Yükleniyor...
Dosyalar
Tarih
2023
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
İzmir Bakırçay Üniversitesi Lisansüstü Eğitim Enstitüsü
Erişim Hakkı
info:eu-repo/semantics/openAccess
Özet
Veri mahremiyeti, eksik veri, tutarsız veri ve veri yetersizliği gibi çeşitli veri kaynaklı sorunlar nedeniyle sentetik verilere olan ihtiyaç her geçen gün artmaktadır. Sentetik veri üretmek için GAN (Generative Adversarial Network), VAE (Variational Autoencoders), kopula vb. birçok yaklaşım geliştirilmiştir. Bu tezde farklı mimarilere ve özelliklere sahip 5 GAN modeli (C-GAN, CT-GAN, DAT-GAN, DPC-GAN, Copula-GAN) 5 iyi bilinen veri seti (adult, credit, census, insurance, cardio-train) üzerinde deneylerle karşılaştırdık. Değerlendirme sırasında sentetik verilerin gerçek verilere ne kadar benzediğini anlamak için çeşitli istatistiksel testler uyguladık ve elde edilen sonuçları paylaştık. Ayrıca, makine öğrenimi modellerini kullanarak gerçek veriler yerine sentetik verilerin nasıl kullanılabileceğini de gösterdik. Karışık veri türleri, uzun kuyruk problemleri, normal olmayan (çarpık) dağılımlar, çok tepeli dağılımlar, seyrek kodlanmış vektörler, yüksek derecede dengesiz kategorik sütunlar gibi bazı açık sorunların deneylerimizde nasıl sonuç verdiğini grafiklerle sunduk. Deneysel sonuçlara bakıldığında kullanılacak GAN modelinin başarısı veri setlerinin özelliklerine ve boyutlarına göre değişmekle birlikte deneydeki en tutarlı ve başarılı sonuçlar Copula-GAN, DPC-GAN ve CT-GAN modellerine ait gözlemlenmiştir.
The need for synthetic data is increasing day by day due to various data problems such as privacy regulations, missing data, inconsistency, and insufficiency. Many approaches, such as GAN (Generative Adversarial Network), VAE (Variational Autoencoders), copula etc. have been developed to generate synthetic data. In this research, we evaluated five GAN-based synthetic tabular data generation models in a comprehensive and versatile way, namely C-GAN, CT-GAN, DAT-GAN, DPC-GAN, and Copula-GAN. We conducted experiments on 5 well-known datasets from different fields and with different characteristics: adult, credit, census, insurance, and cardio-train. During the evaluation, we applied various statistical tests and shared the results to understand how similar the synthetic data were to the real data. We also demonstrated how synthetic data can be used instead of real data using machine learning models. We showed how some open issues, such as mixed data types, long tail problems, non-gaussian (skewed) distributions, multimodal distributions, sparsely hot-coded vectors, and highly imbalanced categorical columns, yielded results in our experiments. Considering the experimental results, although the success of the GAN model to be used varies according to the characteristics and size of the datasets, the most consistent and successful results belong to the Copula-GAN, DPC-GAN, and CT-GAN models.
The need for synthetic data is increasing day by day due to various data problems such as privacy regulations, missing data, inconsistency, and insufficiency. Many approaches, such as GAN (Generative Adversarial Network), VAE (Variational Autoencoders), copula etc. have been developed to generate synthetic data. In this research, we evaluated five GAN-based synthetic tabular data generation models in a comprehensive and versatile way, namely C-GAN, CT-GAN, DAT-GAN, DPC-GAN, and Copula-GAN. We conducted experiments on 5 well-known datasets from different fields and with different characteristics: adult, credit, census, insurance, and cardio-train. During the evaluation, we applied various statistical tests and shared the results to understand how similar the synthetic data were to the real data. We also demonstrated how synthetic data can be used instead of real data using machine learning models. We showed how some open issues, such as mixed data types, long tail problems, non-gaussian (skewed) distributions, multimodal distributions, sparsely hot-coded vectors, and highly imbalanced categorical columns, yielded results in our experiments. Considering the experimental results, although the success of the GAN model to be used varies according to the characteristics and size of the datasets, the most consistent and successful results belong to the Copula-GAN, DPC-GAN, and CT-GAN models.
Açıklama
Anahtar Kelimeler
Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol, Computer Engineering and Computer Science and Control
