Implemented common machine unlearning (MU) algorithms such as zero-glance and zero-shot unlearning via error-maximizing noise and gated knowledge transfer respectively, selective synaptic dampening (SSD), incompetent teacher unlearning, etc., exploring the nascent field of MU and its applications in the right to be forgotten, debiasing, influence functions and model interpretability, and more.
Built and trained decoder transformer models in parallel from scratch on TinyStories, investigated scaling laws of dataset and model size with validation loss and created story generation demo.
Fine-tuned encoder-decoder model T5 with LoRA for extractive question answering on reading comprehension dataset SQuAD v1.1.
Implemented and compared DCGANs and Creative Adversarial Network (CAN)s to generate paintings, performed hyperparameter tuning and metric evaluations, developed interactive Streamlit demo.
Implemented a Switch Transformer alongside a conventional autoregressive transformer and trained on TinyShakespeare to research effects of mixture-of-experts architecture on validation loss, sample-efficiency and training time.