Computer Science PhD Student
Georgia Tech
abhshkdz at gatech dot edu

Co-founder, Caliper

News

• [Nov 2019] Organizing the Visual Question Answering and Dialog workshop at CVPR 2020.
• [Sep 2019] Organizing the Visually-Grounded Interaction & Language Workshop at NeurIPS.
• [Jun 2019] Presenting Targeted Multi-Agent Communication as an oral at ICML 2019 (Video).
• [Mar 2019] Co-founded Caliper. Caliper helps recruiters evaluate practical AI skills.
• [Feb 2019] My work was featured in this wonderful article by Georgia Tech.
• [Jan 2019] Awarded the Facebook Graduate Fellowship.
• [Jan 2019] Awarded the Microsoft Research PhD Fellowship (declined).
• [Jan 2019] Awarded the NVIDIA Graduate Fellowship (declined).
• [Jan 2019] Organizing the 2nd Visual Dialog Challenge!
• [Oct 2018] Presenting Neural Modular Control for Embodied Question Answering as a spotlight at CoRL 2018 (Video).
• [Sep 2018] Presenting results and analysis of the 1st Visual Dialog Challenge at ECCV 2018.
• [Jul 2018] Presenting a tutorial on Connecting Language and Vision to Actions at ACL 2018.
• [Jun 2018] Organizing the 1st Visual Dialog Challenge!
• [Jun 2018] Presenting Embodied Question Answering as an oral at CVPR 2018 (Video).
• [Jun 2018] Organizing the VQA Challenge and Visual Dialog Workshop at CVPR 2018.
• [Mar 2018] Speaking on Embodied Question Answering at NVIDIA GTC (Video).
• [Dec 2017] Awarded the Adobe Research Fellowship. (Department’s news story)
• [Dec 2017] Awarded the Snap Inc. Research Fellowship. (Department’s news story)
• [Oct 2017] Presenting our paper on Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning as an oral at ICCV 2017 (Video).
• [Jul 2017] Speaking about our work on Visual Dialog at the Visual Question Answering Challenge Workshop, CVPR 2017 (Video).
• [Jul 2017] Presenting our paper on Visual Dialog as a spotlight at CVPR 2017 (Video).

Bio

I am a 4th year Computer Science PhD student at Georgia Tech, advised by Dhruv Batra, and working closely with Devi Parikh. My research focuses on deep learning and its applications in building agents that can see (computer vision), think (reasoning/interpretability), talk (language modeling), and act (reinforcement learning).

My research is supported by fellowships from Facebook, Adobe, and Snap. My CV is available here.

IIT Roorkee
2011 - 2015

Queensland Brain Institute
Summer 2015

Georgia Tech
2017 - Present

Facebook AI Research
S2017, W2018, S2018

DeepMind
W2019

I’ve spent three wonderful semesters as an intern at Facebook AI Research — Summer 2017 and Spring 2018 at Menlo Park, working with Georgia Gkioxari, Devi Parikh and Dhruv Batra on training embodied agents for navigation and question-answering in simulated environments (see embodiedqa.org), and Summer 2018 at Montréal, working with Mike Rabbat and Joelle Pineau on emergent communication protocols in large-scale multi-agent reinforcement learning.

In 2019, I was fortunate to get the opportunity to spend time at DeepMind in London working on grounded language learning with Felix Hill, Laura Rimell, and Stephen Clark, and at Tesla Autopilot in Palo Alto working on differentiable neural architecture search with Andrej Karpathy.

Prior to joining grad school, I worked on neural coding in zebrafish tectum as an intern under Prof. Geoffrey Goodhill and Lilach Avitan at the Goodhill Lab, Queensland Brain Institute.

I graduated from Indian Institute of Technology Roorkee in 2015. During my undergrad years, I’ve been selected twice for Google Summer of Code (2013 and 2014), won several hackathons and security contests (Yahoo! HackU!, Microsoft Code.Fun.Do., Deloitte CCTC 2013 and 2014), and been an active member of SDSLabs.

On the side, I built neural-vqa, an efficient Torch implementation for visual question answering (and its extension neural-vqa-attention), and maintain aideadlin.es (countdowns to a bunch of CV/NLP/ML/AI conference deadlines), and several other side projects (HackFlowy, graf, etc). I also help maintain Erdős, a competitive math learning platform I created during my undergrad. I often tweet, toot, and post pictures from my travels on Instagram and Tumblr.

Blog posts from a previous life.

Publications

Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

Vishvak Murahari, Dhruv Batra, Devi Parikh, Abhishek Das
Paper Code

Improving Generative Visual Dialog by Answering Diverse Questions

Vishvak Murahari, Prithvijit Chattopadhyay, Dhruv Batra, Devi Parikh, Abhishek Das
EMNLP 2019
Paper Code

Unsupervised Discovery of Decision States for Transfer in Reinforcement Learning

Nirbhay Modhe, Prithvijit Chattopadhyay, Mohit Sharma, Abhishek Das, Devi Parikh, Dhruv Batra, Ramakrishna Vedantam
ICLR 2019 Task-Agnostic RL Workshop
Paper

TarMAC: Targeted Multi-Agent Communication

Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Michael Rabbat, Joelle Pineau
ICML 2019
Paper

Embodied Question Answering in Photorealistic Environments with Point Clouds

Erik Wijmans*, Samyak Datta*, Oleksandr Maksymets*, Abhishek Das, Georgia Gkioxari, Stefan Lee, Irfan Essa, Devi Parikh, Dhruv Batra
CVPR 2019 (Oral)
Paper

Audio-Visual Scene-Aware Dialog

Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Stefan Lee, Peter Anderson, Irfan Essa, Devi Parikh, Dhruv Batra, Anoop Cherian, Tim K. Marks, Chiori Hori
CVPR 2019
Paper Code video-dialog.com

End-to-end Audio Visual Scene-Aware Dialog Using Multimodal Attention-based Video Features

Chiori Hori, Huda Alamri, Jue Wang, Gordon Wichern, Takaaki Hori, Anoop Cherian, Tim K. Marks, Vincent Cartillier, Raphael Lopes, Abhishek Das, Irfan Essa, Dhruv Batra, Devi Parikh
ICASSP 2019
Paper video-dialog.com

Neural Modular Control for Embodied Question Answering

Abhishek Das, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra
CoRL 2018 (Spotlight)
Paper embodiedqa.org Presentation video

Embodied Question Answering

Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, Dhruv Batra
CVPR 2018 (Oral)
Paper embodiedqa.org Code Presentation video

"Embodied Question Answering" by Abhishek Das

"... a goal-driven approach to autonomous agents" by Dhruv Batra, Devi Parikh

"... an AI scavenger hunt that could lead to the first useful home robots" by Will Knight

Evaluating Visual Conversational Agents via Cooperative Human-AI Games

Prithvijit Chattopadhyay^*, Deshraj Yadav^*, Viraj Prabhu, Arjun Chandrasekaran, Abhishek Das, Stefan Lee, Dhruv Batra, Devi Parikh
HCOMP 2017
Paper Code

Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning

Abhishek Das^*, Satwik Kottur^*, Stefan Lee, José M.F. Moura, Dhruv Batra
ICCV 2017 (Oral)
Paper Presentation video

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra
IJCV 2019, ICCV 2017, NIPS 2016 Interpretable ML for Complex Systems Workshop
Paper Code Demo

Visual Dialog

Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José M.F. Moura, Devi Parikh, Dhruv Batra
PAMI 2018, CVPR 2017 (Spotlight)
Paper Code visualdialog.org AMT chat interface Demo Presentation video

Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?

Abhishek Das^*, Harsh Agrawal^*, C. Lawrence Zitnick, Devi Parikh, Dhruv Batra
CVIU 2017, EMNLP 2016, ICML 2016 Workshop on Visualization for Deep Learning
Paper Project+Dataset neural-vqa-attention

"Is Artificial Intelligence Permanently Inscrutable?" by Aaron Bornstein

"Deep learning is creating computer systems we don't fully understand" by James Vincent

"Robot eyes and humans fix on different things to decode a scene" by Aviva Rutkin

"Robots and humans see the world differently – but we don't know why" by Duncan Geere

"AI Is Learning to See the World—But Not the Way Humans Do" by Jamie Condliffe

Talks

Side projects