SeekDeeper: Minimal Implementations of Popular AI Models

[📖中文ReadMe]

Motivation

Official code repositories often include extensive engineering details, which may increase the learning burden for beginners. This repository aims to provide minimal PyTorch implementations of representative models, thereby facilitating conceptual understanding and experimental reproduction. In addition, many tutorials focus primarily on model architectures while omitting the complete workflow of data loading, preprocessing, training, and evaluation. Therefore, each model implementation is designed to expose relevant training frameworks, configuration utilities, and data-loading pipelines, so that learners can study the full experimental process in a coherent manner.

Models

Model	Paper	Official or Reference Repository
Transformer	Attention Is All You Need	https://github.com/hyunwoongko/transformer
GPT	Improving Language Understanding by Generative Pre-Training	https://github.com/openai/finetune-transformer-lm https://github.com/openai/gpt-2 https://github.com/karpathy/nanoGPT https://github.com/karpathy/minGPT
GPT-2	Language Models are Unsupervised Multitask Learners
ViT	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	Hugging Face ViT implementation
Bert	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	Hugging Face Bert implementation
GAN	Generative Adversarial Networks	https://github.com/goodfeli/adversarial
DCGAN	Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks	https://github.com/Newmu/dcgan_code
WGAN-GP	Improved Training of Wasserstein GANs	https://github.com/igul222/improved_wgan_training

Directory Structure

For each model, the typical directory structure is as follows:

<model name>/
├── checkpoints/
├── modules/
├── datasets/
├── assets/
├── README.md
├── data.py
├── config.py
├── train.ipynb
└── inference.ipynb

• checkpoints/: Contains pre-trained model weights for direct use in inference.ipynb. Sometimes, pre-trained parameters from official repositories are loaded directly.
• modules/: Contains modules necessary for model implementation.
• datasets/: Contains datasets required for training or inference validation, which may sometimes be downloaded to this directory via code.
• assets/: Contains original papers and images used by the model README.
• README.md: Introduces the implemented task and describes the implementation details.
• data.py: Defines Dataset, Dataloader, or data preprocessing.
• config.py: Defines hyperparameters needed for the experiment.
• train.ipynb: Clearly presents the process from data loading, preprocessing, to training and evaluation.
• inference.ipynb: Loads model parameters from the checkpoints/ directory for inference.

License

This project uses the MIT License.