AI organizations divide their work into data engineering, modeling, deployment, business analysis, and AI infrastructure. The necessary skills to carry out these tasks are a combination of technical, behavioral, and decision making skills. Deep learning skills are sometimes required, especially in organizations focusing on computer vision, natural language processing, or speech recognition.

The deep learning case study interview focuses on technical and decision making skills, and you’ll encounter it during an onsite round for a Deep Learning Engineer (DLE), Deep Learning Researcher (DLR), or Software Engineer-Deep Learning (SE-DL) role. You can learn more about these roles in our AI Career Pathways report and about other types of interviews in The Skills Boost.

I   What to expect in the deep learning case study interview

The interviewer is evaluating your approach to a real-world deep learning problem. The interview is usually a technical discussion on an open-ended question. There is no exact solution to the question; it’s your thought process that the interviewer is evaluating. Here’s a list of interview questions you might be asked:

• How would you build a speech recognition system powering a virtual assistant like Amazon Alexa, Google Home, Apple Siri, and Baidu’s DuerOS?
• As a deep learning engineer, you are asked to build an object detector for a zoo. How would you get started?
• How would you build an algorithm that auto-completes your sentence when writing an email?
• In your opinion, what are technical challenges related to the deployment of an autonomous vehicle in a geofenced area?
• You built a computer vision algorithm that can detect pneumonia from chest X-rays. How would you convince a radiologist to use it?
• You are tackling the school dropout problem. How would you build a model that can determine whether a student is at-risk or not, and plan an intervention?

II   Recommended framework

All interviews are different, but the ASPER framework is applicable to a variety of case studies:

1. Ask. Ask questions to uncover details that were kept hidden by the interviewer. Specifically, you want to answer the following questions: “what are the product requirements and evaluation metrics?”, “what data do I have access to?”, ”how much time and computational resources do I have to run experiments?”, ”how will the learning algorithm be used at test time, and does it need to be regularly re-trained?”.
2. Suppose. Make justified assumptions to simplify the problem. Examples of assumptions are: “we are in small data regime”, “the data distribution won’t change over time”, “our model performs better than humans”, “labels are reliable”, etc.
3. Plan. Break down the problem into tasks. A common task sequence in the deep learning case study interview is: (i) data engineering, (ii) modeling, and (iii) deployment.
4. Execute. Announce your plan, and tackle the tasks one by one. In this step, the interviewer might ask you to write code or explain the maths behind your proposed method.
5. Recap. At the end of the interview, summarize your answer and mention the tools and frameworks you would use to perform the work. It is also a good time to express your ideas on how the problem can be extended.

III   Interview tips

Every interview is an opportunity to show your skills and motivation for the role. Thus, it is important to prepare in advance. Here are useful rules of thumb to follow:

Show your motivation.

In deep learning case study interviews, the interviewer will evaluate your excitement for the company’s product. Make sure to show your curiosity, creativity and enthusiasm.

Listen to the hints given by your interviewer.

Example: You’re asked to automatically identify words indicating a location in science fiction books. You decide to use word2vec word embeddings. If your interviewer asks you “how were the word2vec embeddings created?”, she is digging into your understanding of word2vec and might be expecting you to question your choice. Seize this opportunity to display your mastery of the word2vec algorithm, and to ask a clarifying question. In fact, maybe the data distribution in the science fiction books is very different from the data distribution of the text corpora used to train word2vec. Maybe the interviewer is expecting you to say “although it will require significant amounts of data, we could train our own word embeddings on science fiction books.”

Show that you understand the development life cycle of an AI project.

Many candidates are only interested in what model they will use and how to train it. Remember that developing AI projects involves multiple tasks including data engineering, modeling, deployment, business analysis, and AI infrastructure.

Avoid clear-cut statements.

Because case studies are often open-ended and can have multiple valid solutions, avoid making categorical statements such as “the correct approach is …” You might offend the interviewer if the approach they are using is different from what you describe. It’s also better to show your flexibility with and understanding of the pros and cons of different approaches.

Study topics relevant to the company.

Deep learning case studies are often inspired by in-house projects. If the team is working on a domain-specific application, explore the literature.

Example 1: If the team is building an automatic speech recognition (ASR) software, review popular speech papers such as Deep Speech 2 (Amodei et al., 2015), audio datasets like Librispeech (Panayotov et al., 2015), as well as evaluation metrics like word error rate used to evaluate speech models.

Example 2: If the team is working on a face verification product, review the face recognition lessons of the Coursera Deep Learning Specialization (Course 4), as well as the DeepFace (Taigman et al., 2014) and FaceNet (Schroff et al., 2015) papers prior to the onsite.

Example 3: If you’re interviewing with the perception team of a company building autonomous vehicles, you might want to read about topics such as object detection, path planning, safety, or edge deployment.

Articulate your thoughts in a compelling narrative.

Your interviewer will often judge the clarity of your thought process, your scientific rigor, and how comfortable you are using technical vocabulary.

Example 1: When explaining how a convolution layer works, your interviewer will notice if you say “filter” when you actually meant “feature map”.

Example 2: Mispronouncing a widely used technical word or acronym such as NER, MNIST, or CIFAR can affect your credibility. For instance, MNIST is pronounced “ɛm nist” rather than letter by letter.

Example 3: Show your ability to strategize by drawing the AI project development life cycle on the whiteboard.

Don’t mention methods you’re not able to explain.

Example: If you mention batch normalization, you can expect the interviewer to ask: “could you explain batch normalization?”.

Write clearly, draw charts, and introduce a notation if necessary.

The interviewer will judge the clarity of your thought process and your scientific rigor.

Example: Show your ability to strategize by drawing the AI project development life cycle on the whiteboard.

When you are not sure of your answer, be honest and say so.

Interviewers value honesty and penalize bluffing far more than lack of knowledge.

When out of ideas or stuck, think out loud rather than staying silent.

Talking through your thought process will help the interviewer correct you and point you in the right direction.

IV   Resources

You can build AI decision making skills by reading deep learning war stories and exposing yourself to projects. Here’s a list of useful resources to prepare for the deep learning case study interview.

• In deeplearning.ai’s course Structuring your Machine Learning Project, you’ll find insights drawn from Andrew Ng’s experience building and shipping many deep learning products. This course also has two “flight simulators” that let you practice decision-making as a machine learning project leader. It provides “industry experience” that you might otherwise get only after years of ML work experience.

• In Stanford’s Deep Learning class (CS230), Andrew Ng and Kian Katanforoosh present insights on how to lead successful machine learning projects.
  • Deep Learning intuition is an interactive lecture illustrating AI decision making skills with examples from image classification, face recognition, neural style transfer, and trigger-word detection.
  • In Full-cycle deep learning projects and Deep Learning Project Strategy, you’ll learn about the lifecycle of AI projects through concrete examples.
  • In AI+Healthcare Case Studies, Pranav Rajupurkar presents challenges and opportunities for building and deploying AI for medical image interpretation.
• By looking at well-designed deep learning Git repositories, you can get a lot of insights. Ask yourself questions such as “why is the code structured the following way?”, “what evaluation metrics are used?”, etc. Here are some examples:
  • The popular real-time object detector YOLO (Redmon et al., 2015) was originally written in a framework called Darknet. Darkflow (Trieu) translates Darknet to Tensorflow and allows users to leverage transfer learning, retrain or fine-tune their YOLO models, an export model parameters in formats deployable on mobile.
  • OpenPose (Cao et al., 2018) is a real-time multi-person system that can jointly detect human body, hand, facial, and foot keypoints on single images. You can find the authors’ code in the Git repository openpose.
  • Learn about simple and efficient implementations of Named Entity Recognition models coded in Tensorflow in tf_ner (Genthial, 2018).
  • By studying the code of ChatterBot (Cox, 2018), learn how to program a trainable conversational dialog engine in Python.
• Search for deep learning case studies from the companies you’re interviewing with. Here are some examples:
  • Companies use convolutional neural networks (CNNs) for an assortment of purposes. They care about how accurately a CNN completes a task, and in many cases, about its speed. In Faster Neural Networks Straight from JPEG, Uber scientists (Gueguen et al.) describe an approach for making convolutional neural networks smaller, faster, and more accurate all at the same time by hacking libjpeg and leveraging the internal image representations already used by JPEG, the popular image format. Read carefully, and scrutinize the decisions making process throughout the project.
  • Prediction models have to meet many requirements before they can be run in production at scale. In Using Deep Learning at Scale in Twitter’s Timelines, Twitter engineers Koumchatzky and Andryeyev explain how they incorporated deep learning into their modeling stack and how increased both audience and engagement on Twitter.
  • Network quality is difficult to characterize and predict. While the average bandwidth and round trip time supported by a network are well-known indicators of network quality, other characteristics such as stability and predictability make a big difference when it comes to video streaming. Read Using Machine Learning to Improve Streaming Quality at Netflix (Ekanadham, 2018) to learn how machine learning enables a high-quality streaming experience for a global audience.