Machine Learning and Perception Lab, Georgia Tech
abhshkdz at gatech dot edu
News
- [Jul 2018] Presenting a tutorial on Connecting Language and Vision to Actions at ACL 2018.
- [Jun 2018] Organizing the VQA Challenge and Visual Dialog Workshop at CVPR 2018.
- [Mar 2018] Speaking on Embodied Question Answering at NVIDIA GTC. (Video)
- [Dec 2017] Awarded the Adobe Research Fellowship. (Department’s news story)
- [Dec 2017] Awarded the Snap Inc. Research Fellowship. (Department’s news story)
- [Oct 2017] Presenting our paper on Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning as an oral at ICCV 2017. (Video)
- [Jul 2017] Speaking about our work on Visual Dialog at the Visual Question Answering Challenge Workshop, CVPR 2017. (Video)
- [Jul 2017] Presenting our paper on Visual Dialog as a spotlight at CVPR 2017. (Video)
Bio
I am a Computer Science PhD student at Georgia Tech, advised by Prof. Dhruv Batra, and working closely with Prof. Devi Parikh. My research focuses on deep learning and its applications in building agents that can see (computer vision), think (reasoning/interpretability), talk (language modeling) and act (reinforcement learning). My CV is available here.
2011 - 2015
Summer 2015
2015 - 2016
2017 - Present
Summer 2017, Spring 2018
I spent summer 2017 at Facebook AI Research (FAIR, Menlo Park), working with Georgia Gkioxari, Devi Parikh and Dhruv Batra on training embodied agents capable of navigating and answering questions in House3D virtual environments (see embodiedqa.org for more details).
Prior to joining grad school, I worked on neural coding in zebrafish tectum as an intern under Prof. Geoffrey Goodhill and Lilach Avitan at the Goodhill Lab, Queensland Brain Institute.
I graduated from Indian Institute of Technology Roorkee in 2015. During my undergrad years, I’ve been selected twice for Google Summer of Code (2013 and 2014), won several hackathons and security contests (Yahoo! HackU!, Microsoft Code.Fun.Do., Deloitte CCTC 2013 and 2014), and been an active member of SDSLabs.
On the side, I built neural-vqa, an efficient Torch implementation for visual question answering (and its extension neural-vqa-attention), and maintain aideadlin.es (countdowns to a bunch of CV/NLP/ML/AI conference deadlines), and several other side projects (HackFlowy, graf, etc). I also help maintain Erdős, a competitive math learning platform I created during my undergrad. I often tweet, and post pictures from my travels on Instagram and Tumblr.
Blog posts from a previous life.
Publications
Embodied Question Answering
CVPR 2018 (Oral)
Paper
embodiedqa.org
Evaluating Visual Conversational Agents via Cooperative Human-AI Games
Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
ICCV 2017 (Oral)
Paper
Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
ICCV 2017, NIPS 2016 Interpretable ML for Complex Systems Workshop
Paper
Code
Demo
Visual Dialog
PAMI 2018, CVPR 2017 (Spotlight)
Paper
Code
visualdialog.org
AMT chat interface
Demo
Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?
CVIU 2017, EMNLP 2016, ICML 2016 Workshop on Visualization for Deep Learning
Paper
Project+Dataset
neural-vqa-attention
Talks
Side projects
neural-vqa-attention
Torch implementation of an attention-based visual question answering model (Yang et al., CVPR16).
The model looks at an image, reads a question, and comes up with an answer to the question and a heatmap of where it looked in the image to answer it.
Some results here.
aideadlin.es
aideadlin.es is a webpage to keep track of CV/NLP/ML/AI conference deadlines. It's hosted on GitHub, and countdowns are automatically updated via changes to the data file in the repo.
neural-vqa
neural-vqa is an efficient, GPU-based Torch implementation of the visual question answering model from the NIPS 2015 paper 'Exploring Models and Data for Image Question Answering' by Ren et al.
Erdős
Erdős by SDSLabs is a competitive math learning platform, similar in spirit to Project Euler, albeit more feature-packed (support for holding competitions, has a social layer) and prettier.
graf
graf plots pretty git contribution bar graphs in the terminal.
gem install graf to install.
HackFlowy
Clone of WorkFlowy.com, a beautiful, list-based note-taking website that has a 500-item monthly limit on the free tier :-(. This project is an open-source clone of WorkFlowy. "Make lists. Not war." :-)
AirMaps
AirMaps was a fun hackathon project that lets users navigate through Google Earth with gestures and speech commands using a Kinect sensor. It was the winning entry in Microsoft Code.Fun.Do.
HackView
Another fun hackathon-winning project built during Yahoo! HackU! 2012 that involves webRTC-based P2P video chat, and was faster than any other video chat provider (at the time, before Google launched Hangouts).
8tracks-downloader
Ugly-looking, but super-effective bash script for downloading entire playlists from 8tracks. (Still works as of 10/2016).