CSCA 5642: Introduction to Deep Learning

Get a head start on program admission

 Preview this course in the non-credit experience today! 
Start working toward program admission and requirements right away. Work you complete in the non-credit experience will transfer to the for-credit experience when you upgrade and pay tuition. See How It Works for details.

Cross-listed with DTSA 5511

  • Course Type: Breadth
  • Specialization: Machine Learning: Theory & Hands-On Practice with Python
  • Instructor: Dr. Geena Kim, Adjunct Professor of Computer Science
  • Prior knowledge needed: TBD
    • Programming languages: Basic to intermediate experience with Python, Jupyter Notebook
    • Math: Intermediate level experience with Probability and Statistics, Differential Equations
    • Technical requirements: Windows or Mac, Linux, Jupyter Notebook

  View on Coursera

Learning Outcomes

  • Apply different optimization methods while training and explain different behavior.

  • Use cloud tools and deep learning libraries to implement CNN architecture and train for image classification tasks.

  • Apply deep learning package to sequential data, build models, train, and tune.

Course Grading Policy

Assignment

Percentage of Grade

Week 1

 

Week 1 Lesson 1 Quiz

3%

Week 1 Lesson 2 Quiz

3%

Week 1 Programming Assignment: Neural Networks

2%

Week 1 Peer Review: Neural Networks

4%

Week 2

 

Week 2 Quiz

3%

Week 2 Peer Review: Stochastic Gradient Descent

7%

Week 3

 

Week 3 Quiz

3%

Week 3 Peer Review: CNN Cancer Detection Kaggle Mini-Project

15%

Week 4

 

Week 4: NLP Disaster Tweets Kaggle Mini-Project

15%

Week 5

 

Week 5 Peer Review: GANs

15%

Week 6

 

CSCA 5642 Introduction to Deep Learning Final Project

30%

Total

100%

Course Content

Duration: 10 hours

In this week, you will learn the overview and motivation of deep learning. We will also introduce the concepts of artificial neurons, activation functions, and multilayer perceptrons. We are starting off the course with a busy week.  In the first part, after a quick introduction to Deep Learning's exciting applications in self-driving cars, medical imaging, and robotics, we will learn about artificial neurons called perceptrons. The second introduces the backpropagation algorithm, which trains a neural network through the chain rule. We will finish by learning how deep learning libraries like Tensorflow create computation graphs for gradient computation. This material, notably the backpropagation algorithm, is so foundational to Deep Learning that it is essential to take the time necessary to work through and understand it. We are excited for you to build pivotal knowledge for this groundbreaking field! 

Duration: 6 hours

Last week, we built our Deep Learning foundation, learning about perceptrons and the backprop algorithm. This week, we are learning about optimization methods. We will start with Stochastic Gradient Descent (SGD). SGD has several design parameters that we can tweak, including learning rate, momentum, and decay. Then we will turn our attention to advanced gradient descent methods like learning rate scheduling and Nesterov momentum. Besides vanilla gradient descent, other optimization algorithms include AdaGrad, AdaDelta, RMSprop, and Adam. We will cover general tips to reduce overfitting while training neural networks, including regularization methods like dropout and batch normalization. This week, you will build your DL toolkit, gaining experience with the Python library Keras. Assessments for the week include a quiz and a Jupyter lab notebook with an accompanying Peer Review. This assignment is your last Jupyter lab notebook for the course. For the next three weeks, you will build hands-on experience and complete weekly mini-projects that incorporate Kaggle challenges. Then you will submit a final Deep Learning project in Week 6. It is a good idea for the final project to start early and give yourself plenty of time to iterate to achieve better results. Now that you learned about the Keras library this week, it would be a good idea to find a project dataset, start performing EDA and define your problem. 

Duration: 15 hours

This week will teach a type of neural network called convolutional neural networks, suitable for image analysis tasks. We will learn about definitions, design parameters, operations, hyperparameter tuning, and applications. There is no Jupyter lab notebook this week. You will have a brief quiz and participate in a clinically relevant Kaggle challenge mini-project. It is critical to evaluate whether cancer has spread to the sentinel lymph node for staging breast cancer. You will build a CNN model to classify whether digital pathology images show that cancer has spread to the lymph nodes. This project utilizes the PCam dataset, which has an approachable size, with the authors noting that "Models can easily be trained on a single GPU in a couple of hours, and achieve competitive scores." As you prepare for the week, look over the rubric and develop a plan for how you will complete it. It will be necessary for a project like this to work on a timeframe that allows you to run experiments. The expectation is not that you will cram the equivalent of a final project into a single week or that you need to have a top leaderboard score to receive a good grade for this project. Hopefully, you will have time to achieve some exciting results to show off in your portfolio. Remember to carve out some time this week for your final project. We expect that you will be looking at some research papers, GitHub repositories, and Medium articles to understand your topic and plan your approach. Once you've finalized the dataset and EDA, do any data preprocessing to prepare for your deep learning models. 

Duration: 13 hours

This week will teach you another neural network called recurrent neural networks (RNNs) to handle sequential data. So far, we have covered feed-forward neural networks, including Multi-layer Perceptrons and CNNs. However, in biological systems, information can flow backward and forwards. RNNs do a backward pass closer to biological systems. Using RNNs has excellent benefits, especially for text data, since RNN architectures reduce the number of parameters. We will learn about the vanishing and exploding gradient problems that can arise when working with vanilla RNNs and remedies for those problems, including GRU and LSTM cells. This week, we don't have a quiz, but we have a Kaggle challenge mini-project on NLP with Disaster Tweets. The project is a Getting Started competition designed for learners building their machine learning background. The challenge is very doable in a week, but make sure to start early to run experiments and iterate.This week, work on the primary analysis of your final project. Experiment with a single model, optimize the hyperparameters or compare multiple models.  

Duration: 13 hours

This week will focus on neural network models trained via unsupervised Learning. We will cover autoencoders and GAN as examples. We will consider the famous AI researcher Yann LeCun's cake analogy for Reinforcement Learning, Supervised Learning, and Unsupervised Learning. Supervised Deep Learning has had tremendous success, mainly due to the availability of massive datasets like ImageNet. However, it is expensive and challenging to obtain labeled data for areas like biomedical images. There is great motivation to continue developing unsupervised Deep Learning approaches to harness abundant unlabeled data sources. This week is the last week of new course material. There is no quiz or Jupyter notebook lab. Generative adversarial networks (GANs) learn to generate new data with the same statistics as the training set. This week, you will wrap up one final Kaggle mini-project. This time, you will experiment with creating a network to generate images of puppies. Keep running experiments and working on the primary analysis for your final project. Ideally, it would be excellent to finish experimenting and iterate with your models this week so that next week, you can focus on preparing your final project deliverables. 

Duration: 8.5 hours

For the final peer reviewed project, you will identify a Deep Learning problem to perform EDA and model analysis. The project has 140 total points. In the instructions is a summary of the criteria you will use to guide your submission and review others’ submissions.  You will submit three deliverables

Notes

  • Cross-listed Courses: Courses that are offered under two or more programs. Considered equivalent when evaluating progress toward degree requirements. You may not earn credit for more than one version of a cross-listed course.
  • Page Updates: This page is periodically updated. Course information on the Coursera platform supersedes the information on this page. Click the View on Coursera button above for the most up-to-date information.