I'm a computer scientist (and a self-proclaimed tinkerer), specializing in computer vision and machine learning. I’m designing methods and systems for data-driven solutions to real-world complex problems.
I am building generative multimodal models for a new kind of ML hardware accelerator. Earlier, I was Staff ML Scientist @ Nference where I built probabilistic multimodal ML models for biomedical applications based on EKGs, EHRs, CTs, MRIs, Ultrasound imaging, Surgical Planning via Echo-to-3D machine learning models. I earned my Ph.D. in CS at Max Planck Research School for Intelligent Systems, researching uncertainty estimation in ML models like VLMs, Diffusion Models, and GANs. Even earlier, I was an undergrad at CS @ IIT-Bombay. During grad school, I interned at Amazon Science, Microsoft Research, and worked with early-stage startups.
Selected Publications (full list @ GoogleScholar)
Uddeshya Upadhyay, Sairam Bade et.al., Transactions on Machine Learning Research (TMLR) - 2023
ProbVLM: Probabilistic Adapter for Frozen Vision-Language Models
Uddeshya Upadhyay*, Shyamgopal Karthik*, et.al.,
International Conference on Computer Vision (ICCV) - 2023
[code]
Vikrant Rangnekar*, Uddeshya Upadhyay*, et.al.,
The Conference on Uncertainty in Artificial Intelligence (UAI) - 2023
BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks
Uddeshya Upadhyay*, Shyamgopal Karthik*, et.al.,
European Conference on Computer Vision (ECCV) - 2022
[code] [demo]
Uncertainty-Guided Progressive GANs for Medical Image Translation
Uddeshya Upadhyay, Yanbei Chen, et.al.,
International Conference on Medical Image Computing and Computer Aided Diagnosis (MICCAI) - 2021
[code]
Robustness via Uncertainty-aware Cycle Consistency
Uddeshya Upadhyay, Yanbei Chen, Zeynep Akata
Advances in Neural Information Processing Systems (NeurIPS) - 2021
[code]
Selected Projects (More @ Github)
Neurapse is a package in python which implements some of the fundamental blocks of SNN and is written in a manner so that it can easily be extended and customized for simulation purposes. Spiking Neural Networks (SNNs) are called 3rd Gen and they attempt to simulate biological neural networks closely. Broadly the framework consists of Spikes, Neurons, Synapses, Networks.
IBM launched TrueNorth chip in 2014, Intel announced Loihi in 2017 both are attempts to do neuromorphic computations efficiently on chip.
BayesCap. This is the official PyTorch implementation of BayesCap (from ECCV 2022) that allows estimating calibrated uncertainty for pre-trained (frozen) computer vision regression models in fast and efficient manner.
Uncertainty_Guided_Progressive-GAN. This repository provides the code for the MICCAI-2021 paper titled "Uncertainty-guided Progressive GANs for Medical Image Translation". We take inspiration from the progressive learning scheme demonstrated at MedGAN and Progressive GANs, and augment the learning with the estimation of intermediate uncertainty maps (as presented here and here), that are used as attention map to focus the image translation in poorly generated (highly uncertain) regions, progressively improving the images over multiple phases.
Transformer Based Reinforcement Learning for Games.
This repository contains experimental models, written in PyTorch, which incorporate transformers in the Deep Q-Learning tasks, to see if they perform better than the RNN based version (DRQN) or simple DQN.