CS-7643 - Deep Learning
| DL |
Toggle to Select Spcific Semesters
Reviews
My background: Handful of years experience programming in C. My previous classes were: Computer Vision, Intro to OS, Advanced OS, Intro to High Performance Computing, Human-Computer Interaction, Graduate Algorithms, and Machine Learning.
This is the single most practical course I’ve taken in terms of new skills I didn’t have before, that I expect to use at work regularly especially for my side hustles. A close 2nd place is Intro to High Performance Computing.
If you want to learn the mathematical nuts bolts of ML models (implement from scratch with numpy) AND gain a working proficiency with Pytorch, this is the class for you.
It’s a slog with a lot of things to submit, but 70% of the class gets an A historically. The final group project had a median score of 58/60 (points, not percent). They graded 200-400 of them in 3 days and were not picky at all. Just touch on all parts of the rubric in some depth.
My average quiz score was 55%, but I got a 97% overall thanks to doing the extra credit assignment and spending a lot of time on the report/theoretical portions of the assignments.
First of all, my comment is from a beginner’s view, please ignore if you have good amount of background in DL. About my self: This is my 7th course in OMSCS. I have taken ML4T but have no background in DL. I am not from CS or engineering major, but I work with big data and am comfortable with basic algebra or calculus. My final grade in DL is A.
I definitely recommend anyone who works with data to learn DL, but this course is not the best way to start your journey. Although the topics sounds interesting and super useful, course quality is mediocre at best compare to others in OMSCS. Please lower your expectation and prepare for a lot self-learning. Overall, I would like to rate 5/5 for the topics, 4/5 for assignment 1 and 2, 2/5 for assignment 3 and 4, 2/5 for lectures and organization. The first half is challenging and fun, but then things start to fall apart drastically. I need to remind myself several times a week that the class is almost over, in order to not melt down in the last two months.
Please consider the following things before signing up. If you meet 2 or more of the descriptions below, this course will likely to be a rough ride for you.
- You have never learned any DL courses or topics. (General ML courses only have limited help so they don’t count);
- You do not know and do not like to learn linear algebra, multivariable functions, derivatives, etc;
- You have never used pytorch or numpy;
- You have other thing going on in life that constantly requires > 30h/week.
Other reviews already mention most comments I would write. The biggest issue to me is this course tries to cover everything in one semester, but the lectures never spend enough time to explain concepts. DL is such an interesting topic, it is unbelievable how boring these lectures are. I ended up completely rely on Stanford CS231n which is a really great course, highly recommend. Another issue is they put a lot on the to-do list to fill up your time, so you might burnout and lose motivation quickly. I do appreciate there are a lot of office hours. Many thanks to TAs Alex Shum (assignment 1 & 2), Farrukh Rahman (assignment 1 & 2) and Sangeet Dandona (assignment 4). Do remember you need to attend OH to get hints and fulfill hidden requirements of assignments.
This is my 4th OMSCS course but I took the first 3 (including ML) back in 2015. In the month prior to the course starting I took Andrew NG’s deep learning course which i felt was very good preparation for the course. In short, I felt I really learned a lot and my understanding is much deeper than what i could’ve gotten from the Andrew NG course alone. The quizzes involve a fair amount of preparation (taking detailed notes on lecture videos and spending a few hours or so studying the notes on quiz day). I think the questions were very fair on the first 3. The last 2 had a few nitpicky questions and while I didn’t do as well on those, it didn’t really impact my overall grade too negatively. Overall I felt that the quizzes in terms of prep time required were fair for a graduate level course. The projects were really great, but of course must start early and give yourself plenty of time. I didn’t really bother doing any of the reading, and I agree the DL book is only useful if you already know the subject matter very well (so is therefore not useful for learning in a first course). My only real complaint, as stated by other students, is I really disliked the Facebook lectures. They don’t explain things in decent lecture style detail and it feels as though the presenters are just cursorily describing algorithms and techniques. After watching all those, I watched the Andrew NG lectures and read some things on towarddatascience which gave me a an understanding that the lecture videos just did not. Professor Kira’s lectures are pretty good, but I do feel it helps to watch Andrew NG’s lectures to supplement and gain intuition.
This is my 7th class in the program and I took AI and ML right before DL. First half of the class (lectures + A1 & A2) were well organized and benefited my understanding a lot. Second half (starting with Meta lectures) and A3 & A4 were not that well organized. A very good aspect of this class is that the teaching staff have been the most enthusiastic that I have ever witnessed (they offered office hours more than all my other classes combined). Another highlight is that you get to have exposure to the most cutting-edge deep-learning technologies the industry and academia are using. The downside is the lack of coherence of the material that is being taught.
I have mixed feelings on this class. Overall, there’s a ton of content and things to learn. Out of necessity, you will have to ramp up really quickly or it’s easy to fall behind and play catchup the entire time.
I won’t reiterate everything but I agree with most of the sentiment from reviewers particularly those taking the course in Spring 2022: quizzes are unnecessarily hard with frequent curveballs, at times you feel like you’re on an island for assignments which were pretty taxing at times, with some challenges not really benefitting your learning at all.
All that being said, there’s current content in this class so it does a good job trying to keep up, as this field is constantly evolving.
Just prepare to spend a LOT of time on this class to get the most out of it. Coding assignments were my favorite, lectures were not very engaging as others stated, especially the meta guest lectures. There were times where I felt like I’d never get through an assignment but so far (4/4), I have been able to get things finished (right up to the deadline).
Looking at the current pain matrix I’d say an estimate of < 20 hours is pretty generous unless you already have DL experience of some kind.
Don’t take this class unless you feel you must, its not worth it, either learn on your own or take a course elsewhere it will be a better experience.
I want to preface this in that I have 5 years experience as data scientist focusing on deep learning, I have published research on deep learning prior to taking this class. I am on track to get an A.
Quizes:
I feel like these are designed to produce a grade distribution and serve no other function. The TAs may feel like providing examples of what is on the quizzes, but studying will likely be largely pointless for anything beyond reviewing lectures and hoping that you can do the math they expect.
Class experience:
TAs can flat out delete posts that have any potential useful discussion just because they have extremely restrictive information sharing rules. This stifles any useful discussion basically.
Projects:
A1, and A2 were wonderful experience of doing backprop from scratch, I really liked these. I really like the theory questions.
A3 and A4 were exercises in coding to the autograder, I feel like this was a waste of time and money.
The test they provide for the homework’s are brittle and incomplete, and some of the grade scope test are rather pointless be we are graded on them regardless.
There was an optional A5 that was rough around the edges but I really enjoyed the challenge here.
Graded Discussion:
This are a waste of time, you read a paper, write answers to a few questions, and then respond to a few other students. Nothing here drives a genuine discussion, and despite how well you write your responses, the TAs find ways to doc you points without feedback on how to improve.
Final Project:
This works well in theory if you have good team mates, but if you have team mates that are unresponsive and don’t do their share of the work its awful. Really this is why you should not take this class, it makes what could be an enjoyable solo project and makes it a crap experience. Seriously don’t both with this class because of this.
Lectures are very dry and soporific. Quizzes are extremely difficult, testing random facts from lectures and readings. A lot of effort is needed to get high marks on them, and each individual quiz is worth little of the grade. Assignments are decent and are where the core learning happens, but the culture of the class discourages helping others and therefore hinders some of the learning. One TA straight up cancelled students’ posts. The course overall just has poor organization, and the core learnings from the class can be done individually. Expert competency with numpy is assumed.
Pros: Great assignment content, interesting to see the results of the implementations. They are very hard though. Project seems like a good opportunity to explore areas of interest in DL Pytorch is nice to finally learn
Cons: Very dry lecturer. I’m sure the prof is a nice guy but he manages to make the lectures extremely boring on such an interesting subject. Very little passion in delivery, dry, monotonic. Quizzes. Why are these even a thing. It’s an unbelievable stress inducer, they test extremely random factoids from the lectures and you pretty much have to know perfectly word for word what is presented on the slides. They are only worth around 4% each but add up. TA policing. As another review mentioned, the TAs are extremely annoying when it comes to policing the class forum and not only taking down things that could be considered “too revealing of the assignment”, but also threatening to impose penalties! Why in the world do I have incentive to help on ed if I am at risk of getting a penalty just for simply suggesting to use one function over another? It’s absolutely ridiculous.
I really don’t know what happened to this course this semester, but it’s definitely been one to remember. We’re halfway through this semester, but I wanted to go ahead and put a review up to warn others before they sign up for this course.
First of all, this class has the worst group of TAs I’ve seen yet (and yes, that includes ML). They completely ignore questions, office hours are terrible, and the only thing they seem to be interested in is policing the project threads (if anything could even remotely be considered a hint, they remove it). The grading feedback is comical. You’ll get something like “we weren’t expecting this,” so you ask for additional feedback… they respond with “this value wasn’t what we were expecting.” And be careful requesting a regrade, they will lower your grade in a heartbeat.
The class organization is freaking ridiculous. Assignment dates are constantly changing. You never know whether to believe canvas, grade scope, or the syllabus because they all three have different times. Then they randomly start changing due dates while the assignments are on going. They once changed the due date for a quiz to be 12 hours earlier so a lot of folks rushed to take it, then said it was a mistake and the folks that missed it got three additional days to take it. Also had a problem where they didn’t make the calculator in one of the quizzes obvious, so some folks were able to get some math questions thrown out but not for everyone. The assignments page and grades page on canvas are still screwed up halfway through the semester and they don’t seem to really care.
So 20% of the grade comes from five quizzes, and these things are BRUTAL. You can’t just watch the lectures and feel confident enough to take the quizzes, you need to understand the material in every aspect, read the additional materials, and do some independent research, and hope to score at least a 50. Previous semesters only had math on 2 quizzes, but that’s changed. There’s at least 20-30% of the quiz that is heavy math and that’s on all of the quizzes.
The projects are rip-offs from Stanford’s version of this class. You know what that means? Right, the TAs don’t have a clue what’s going on with them. We’ve started project 4 and the libraries they are using don’t even run on newer graphics cards (30 series). Their response? “Just run it on Google colab and see if it works.” Projects 1 and 2 weren’t horrible. Project 3 required you to read 6 papers and attempt to decipher the algorithms (we had to beg for an extra week because they said this project was too easy and took a week away from us). Oh, and make sure you read the class forums, they’ll hide hidden requirements about the projects in there.
Unlike previous semesters, they are nit picking the write ups for the projects (without explanation of course). Make sure you do a PhD thesis for each of the questions. The grading is all over the place. You may be 0 points for your explanation but someone else got full credit with the same explanation. They’ve also increased the difficulty of the reports this semester and added math questions to them.
Grading is SLOW. I mean months slow. They even completely forgot to grade an assignment until someone asked about it, then they pretty much gave everyone full credit if you turned something in (was only worth 0.5%).
The only improvement I have found this semester is they no longer test on the Facebook lectures. Thank goodness because those lectures are a joke.
I’ll complete this review when the semester ends, but if you’re thinking about signing up for this class, you might want to think twice. This is my 8th class and I wish I wouldn’t have taken it. It’s been extremely frustrated dealing with this TA group and has made a hard class unbearable.
My background: As an OMSA student, I am taking Deep Learning as the last one in my program. I do not have a CS background and enrolled in OMSA for a career change to data science. Prior to this course, I took Andrew Ng’s Deep Learning specialization on Coursera to get a high-level understanding of deep learning concepts.
Course itself: Although I am less than halfway through the course, this is already my favorite course in the OMSA program. Although I have attempted to study deep learning through MOOCs and hackathons before, this course gave me a deep dive into deep learning I needed to make all the concepts really stick.
The course is well-designed. TAs are very responsive and their office hours are good for getting unstuck. Their office hours include occasional tutorials dedicated to trickier parts of the assignment. The readings are interesting and easy enough for someone like me (no STEM background prior to OMSA) to follow.
Difficulty: The most difficult parts of the course are the coding portions of the assignments but all assignments of the course have been doable. The theory portions of the assignments are easy. Although doing well on the quiz requires watching the lecture videos/doing the readings more closely, there have been no trick questions on the two quizzes I have taken so far.
Workload: Varies. I have been putting in 20-30 hours/week to finish coding portions of the assignments (there are four assignments total) well ahead of the deadline. On weeks where I am just watching the lectures or finishing up my assignment, the workload is around 5-10 hours.
Rating: I strongly recommend this course. I am glad I set my alarm early in the morning to register for the course back in December.
I agree with the other students that this is the most “contemporary” course in the OMSCS journey. Most other courses only have the relevant research papers dated 5 to 10 years ago, but the amount of research papers on Deep Learning (DL) are really overwhelming in these few years (since the revival of neural network at approx 10 years ago). Meanwhile, this course is being actively updated on each semester, and it will guide you through even the newest/hottest topic like GPT-3.
The course is easier than I thought (though still hard) since the core algorithm is nothing but gradient descent. Of course there are many different networks and optimisers and schedulers, but it seems most of them do not involve complex math equations. As professor Zsolt Kira said (and probably a sad fact), most neural network (NN) models are empirically found, rather than deduced from a math model.
When your DL or NN models do not work well, it is this empirical nature of NN models that makes them hard to tuned or optimised. Often it is very hard (if not impossible) to mathematically prove a NN model is correct; the model still works somehow, but at sub-optimal quality. Life is even harder when DL needs massive computation power before a single empirical test can converge (i.e. a week on MacBook, days on Google Colab, hours on latest GPU), which makes the software development cycle extra long.
Regarding GPU, the course organiser is very kind to invite Google and Amazon to offer few cloud computing credits to the students. However, Google Cloud Platform (GCP) only gives each student “USD50” while any public user can already receive “USD300”… on the other hand, Amazon came in very late when most students have already started the projects, so switching from Google Colab or GCP to Amazon Cloud might not worth the trouble. Also I am not satisfied about Google, since GCP charges me a much higher price (though subscription will only charge after USD300 is used up) with a 8-year old GPU Nvidia K80 offer after complex application, while Colab Pro can already give me a 5-year old P100 at a much cheaper price; though on the other hand Colab terminates my program execution every 24 hours even if I paid for the most expensive Colab Premium (with background execution). Not mentioning that Colab UI has issues (bugs?) that hide the “background execution” option from time to time, and allocates & shows the CPU / GPU memory incorrectly so that my program crashes on a memory-full error without any early warning…
Tests and challenges in the course are homework assignments, quizzes, and a project. The homeworks are fine, which progressively guide students through the DL concepts, and teach you how to use PyTorch (though it is best to run PyTorch in Colab to avoid rare local CPU errors). The project is not very difficult except on project management and choice of project objective. I was very lucky to have good teammates, one of whom was so heroic to consolidate all our writings into a big & elegant research paper. The project instructions mandate each students in the project team to work equal share on all tasks, though I still think it would be better for the project if each of us can assume a different role, e.g. programmer, writer, project manager, etc.
Nevertheless, beware of those “quizzes”! As many students said, quizzes in this course are never quizzes; they are full-blown exams!! And very difficult!! They are closed books and some concepts are very confusing, so I think most students lost most marks on the quizzes, rather than failing an assignments or project.
Regarding the video lectures, I think most videos offered by professor Kira are great, but those offered by Facebook are usually (50% chance) poor… The problem is that some FB speakers speak strange English with difficult accent (which is hard to interpret when without subtitles), when they present diagrams or equations they do not explain or prove at all… Some materials are more like sales presentations than concept overview or worked examples. Sorry Facebook, I really cannot appreciate your video lectures (though I thank you for PyTorch).
Overall, this is really a great course. Enjoy!!
Excellent course as an intro to DL. Expect to spend a lot of time trying to figure out why your code does not work since TAs do not offer much help beyond “there is something wrong with your code.”
Pros:
-
Projects. This is where you truly learn the material. Yes, you will struggle with the code. Yes, the last assignment has incomplete unit tests and you will be re-writing the code for the sections you just finished and passed local tests. Having said that, this struggle is good in a sense that once you finish this class, you will have the confidence to teach yourself pretty much anything since there is no help offered.
-
Sangeet Dandona and Farrukh Rahman are two amazing TAs who actually know and understand the subject they are teaching. They explain the material clearly and, more importantly, they explain the intuition and not regurgitate how to code an equation. The rest of the TAs are not so great.
-
Prof. Kira’s lectures are amazing. I wish he did all lectures and not outsourced to FB/Meta researchers.
-
Graded discussions. At first I did not like it since it feels like you are forced to write a specific amount of words. Reading articles is crucial to keeping up with the developments in the field. At first you read because you have to, later - because you want to. Writing responses to comments of others forces you to think deeper about the subject you just read. Overall, I think this is a good pedagogical tool.
Cons:
-
FB/meta lectures. These are simply pedagogical disasters. Do not expect to learn anything from them. Read comments of other reviewers about this subject, they are pretty accurate.
-
Quizzes. I am not sure what is the point of them outside of being a grade differentiation. The quizzes cover a spectrum of topics and ask fairly detailed questions. People who claim to do great on quizzes are the ones who work in the DL domain as their regular job or have some exposure to this domain. It is not that the quizzes are hard, but given the amount of time you spend on projects, you will have less, if any, time to prepare for these if you have a career and a family.
-
The class is front-loaded. I put on average of 20 hours per week but the distribution is not even. There were weeks I spent up to 60 hours for some projects, then there are weeks you do not do anything at all towards the end of the semester.
-
Group projects. You will have a free-rider problem.
Overview
For context, this was my seventh course, having taken AI4R, RL, AI, ML, BD4H, and ML4T beforehand, in that order.
Deep Learning is a course which I have very mixed feelings on. The material is very interesting and well worth learning, but the course itself is highly variable in quality and feels like it has artificial difficulty.
Deep Learning covers over the course of the semester the following material, All the material is quite fascinating and can feel like drinking from a firehose, which is great!
- Feedforward Networks
- CNN (convolution and pooling)
- Visualization of networks
- RNN, LSTM, & Embeddings
- Transformers, Deep Reinforcement Learning
- Unsupervised/Semi-supervised DL & GANs
Lectures
The lectures are a bit of a mixed bag. They mostly stay pretty high level, and you cover the material quickly over the semester. But… they don’t go very deep and then later on you are expected to know that lower-level detail on the assignments/quizzes. The book is only okay at supplementing at this so you need to search out other material, like other school’s lectures, to really understand the material. And then the Facebook (Meta now…) lectures are basically useless and too high level to be of use.
Assignments
- Assignment 1: Building a NN from scratch, very good assignment to learn the basics.
- Assignment 2: Building a CNN from scratch and then using PyTorch, pretty good overall.
- Assignment 3: Visualizing CNNs, which is kind of fun, however, Gradescope doesn’t have near enough tests to get full coverage and you have to compare your images to images in the PDF, which can seem close to your human eye but apparently aren’t close enough and you’ll get ducked points. In theory this is a great assignment, but they need to give you better tools to self-assess. It’s removed in summers and almost worth taking in the summer just to skip it.
- Assignment 4: Making an RNN, LSTM, and transformer for NLP. I rather enjoyed it, but you really want a GPU to test this out and get your losses low enough.
Graded Discussion
On the one hand you do review fairly interesting papers, on the other hand, the discussions don’t add much and I’m pretty sure they just have an automated word counter for whether or not you finish this section.
Group Project
The group project is great in theory; however I feel you’re not given enough time to do it justice. In BD4H you had over a month to focus on the project, whereas in DL, you have two weeks after the last assignment and quiz to focus just on the project. With assignments and quizzes you’re kept constantly busy. On the plus side, the grading is very generous, perhaps too much so.
Quizzes
The quizzes will make or break your grade. And honestly they feel kind of like bs. There are questions on the quizzes which feel like they come out of nowhere from the lectures and even the papers and unless you have a perfect knowledge of what was in all of those you will miss plenty of points. Earlier on they gave some good examples of computational questions on the quizzes, but later they don’t at all. The TAs/Professor should do a better job given examples of what to expect and I feel they let the class down in this regard.
Ed Discussion
On the plus side, the TAs/Professor are very active on Ed, and quickly answer questions. It is possible to get extra credit on Ed by getting “endorsed” posts, which generally are best done via finding papers about SOTA techniques and/or making detailed notes to help other classmates.
Tips for Success:
- I highly recommend watching the CS231n (https://cs231n.stanford.edu/2017/syllabus.html) and EECS598 (https://web.eecs.umich.edu/~justincj/teaching/eecs498/FA2020/schedule.html) lectures from Stanford and UMichigan to supplement the course lectures (frankly I think they’re better). Both EECS 598 and 7643 were based on CS231n originally, and 598 is taught by a former Stanford CS231n TA.
- Take copious notes of lectures, review the papers mentioned in lecture, post the notes to Ed, and make some quizlets to test yourself. Quiz materials can come from the lectures, the papers, or frankly related sources.
- Brush up on your calculus, especially with regards to partial differential equations and linear algebra. The first six chapters of Goodfellow’s Deep Learning book are good to review and understand intimately.
- If you can, get a good GPU or get used to working in Colab, they make the assignments and project far easier, especially A4.
This is the third course that I take, with AI4R and ML4T as the first two. I completed a M.S in CS already and have good understanding of ML and with some prior knowledge of DL (Completed part of DL Specialization by Andrew Ng.)
Overview
Overall, according to my experience, this is a fairly organized course, which covers a broad range of topics in DL such as Gradient Decent, CNN, Language Model, Semi-supervised learning, Deep RL and the advanced topics such as GAN. The lecture itself is very informative and you will get to know all these up-to-date topic and DL techniques.
Quizzes
Quizzes are demanding and require effort and study time to devote and really understand the concept, and first two quizzes includes more computational questions that requires you the calculate the Gradient Decent and input the number for it for example, the later quizzes are more on the conceptual questions but still entails computational questions. You need to study the detail concept covered in the lecture. For the first two quizzes, they provided TA tutorial and some samples questions and solutions which help you to get prepared a lot. The third and the fourth ones are quite difficult. And the rest three, usually, some students put a lot of relevant study materials and that help. In general, going through lecture slides and understand the concepts are necessary for combating quizzes.
Assignments
The second assignment is the most challenging one as it requires you to implement backprop from scratch but still provided code template for doing that. The first one is easy as long as you understand the gradient decent. The first half of the course is more math heavy with gradient decent. The third assignment is a bit easier and very interesting to visualize the learning algorithm of CNN and implement style learning and transfer learning. The last one (4th) is challenging as well, but worth the time you put in as you will learn a lot about the transformers and machine translation. Overall, the assignments are well designed to cover broad range of topics too.
Final Project
Final project is a group project, and it depends on various factors such as finding good team members and choose the project that is feasible in short period of time. Don’t be too ambitious, but the rubric said they focus on you gain and learn DL techniques but not exactly care about if you succeed or fail to accomplish your project. Thus, it’s okay to show not good experiment results or discuss your failure experiments as part of your project, but make sure to showcases your understanding through quantitative and qualitative analysis of your experiments.
Graded discussion
This is an outstanding feature of this course and you will learn a number of recent research papers and discuss it in summary and comments with your classmates and learn from one another. I find it quite interesting and a lot learning from it.
OA & Tutorial
The Prof. K was very dedicated and organized a lot of OA time with prof himself or tutorials with TA to explain difficult concepts, or overviews for the assignments. I would recommend to go through the overviews of the assignments OA as it will give you directions and save you a lot of time understand it. Other than this, FB Office hours or OA with AWS are accessible as AWS provides some free computing powers (GPU) to your final projects too.
Summary
In summary, I personally had a good learning experience this semester and think I learn a lot from this course, and highly recommend this course if you want to learn DL. Meanwhile, I found the quizzes, assignments are challenging but worth the time you devoted to. I personally learnt a lot from them. The entire semester is busy, some weeks with quiz due only is a bit easier. But after the semester, I feel very accomplished personally. Time spending on each week various, weeks with assignment or final project due demanded more time, but weeks with quiz due are a bit easy but still needs the effort. Hope this help to others who are considering this course.
I loved this course. I learned so much, and so much of it plays right into my work in a computer vision startup where I work in product management in a role that lets me get my hands on our technology. This was my ninth course in OMSCS (btw I have not taken ML, officially a pre-req, but I didn’t feel like I was missing any of the content not having taken ML first).
The coding components of the assignments are auto-graded, which I always prefer to non-autograded coding assignments. You can keep hacking away at at it until it passes, and there are plenty of people on EdStem or Slack to ask questions of when you’re stuck, who have encountered that exact same error message as the one you’re currently stuck on. The report components of the assignments were of the type where, if you answer the questions in the template, you get most if not all if the points. I missed 1 or 2 points out of 30 on some of the reports for submitting OK but not great answers. All in all, getting scores in the high 90%s on the projects isn’t terribly difficult, but it requires doing the programming and then completing the report. That being said, I learned a fair amount about how pytorch works through these assignments, and I hope to keep that facility with pytorch up going forward.
The “quizzes” are definitely the most challenging part of the course grade-wise. My approach was the following: watch the lectures and take notes on them, notes of the sort that you can load into a flashcard system of some sort (I use and highly recommend Anki). Then watch the Stanford CS 231n or 224n lectures corresponding to the gatech lecture in question, and take notes on them as well. I found that the Stanford lectures came from a slightly different angle and that watching both gave me a much fuller understanding of the material. Armed with those flashcards, I would review them in the week prior to the quiz when out on a walk or at other times when I was waiting in line at the store or something. This approach resulted in quiz scores that were all closer to full marks than to the mean score. Oh, for the quizzes that had official study guides to go with them, definitely work through those, you will regret it if you don’t.
Your experience with the final project depends quite a bit on your teammates and the project you choose. We chose an ambitious project, which didn’t really end up rejecting the null hypothesis in the end, but, we put together a complete report that was responsive to the rubric and we got full marks on it. Find teammates that you get along with and respect, and then, do your best to be a good teammate. There were definitely times when I felt like I wasn’t pulling my weight, due to family or work obligations or whatever, which inspired me to try harder at other times to catch up and do what I could to move the project forward. At the end, each teammate fills out a form reviewing the effort put in by each of their teammates. If you are reviewed as “never helping”, my understanding is that you could earn a 0 (for 20% of the grade) for the final project, despite the project getting a perfect score for the other teammates. Don’t be a deadbeat!
The graded discussions were a new-to-me course mechanic, and I thought they were an interesting new way to get people to learn material via research papers. I preferred this approach to the one in AOS in which you have to summarize papers.
I might as well chime in that I agree that the FB lectures weren’t great. Could I have done better if I were in their shoes? I have no idea. I applaud the individuals at FB who stepped up and recorded lectures for our use, but I can’t help but wonder if there might have been a way to better support those lecturers such that the final product were closer to the quality level of the rest of the lectures.
Overall: Great course if you want to learn a lot about how DL works, as well as get experience with using PyTorch to build things.
This was my 6th course in OMSCS after: Comp Photography, AI4R, Software Arch, CV, ML
This was my most difficult course so far, despite the fact that I’ve worked as a Data Scientist and have studied neural networks before. Assignments take a long time. Due to the hidden nature of the autograder, I spent hours and hours trying to debug insignificant and esoterric elements of the code. You’ll get to understand how to code in PyTorch more than you will actually understand some of the course content. This is the tradeoff of enrolling in an online program- the instructors must rely on autograder which means the students end up focusing on matching the results exactly with the expectation as opposed to studying and learning the course concepts.
The quizzes were very difficult. Even with watching all the lectures, taking notes, and doing the readings, I ended up with around average score on the quizzes which sometimes was down around 60%.
Most of the assignments focus on Computer Vision applications which was disappointing. It was interesting but I would prefer a more common application since many ML/DL Engineers don’t work in CV. I enjoyed the sections on NLP but felt the Deep RL section were unnecessary.
That being said, the material is very interesting and presented well. There is A LOT of material to learn, and instead of going deep into a few important concepts, this course prefers to go wide and touch lightly on many subjects within deep learning.
I liked the course. Got what I wanted from the course. I had taken RL and ML4T.
Early in the course, there were office hours on math background required for the course (matrix calculus, backprop etc). Watching these helped with initial quizzes and assignments.
Lectures from Prof. Kira are good and concise. However FB lectures are not organized well and most of them are bad. I feel they should be redone for future semesters. I used other sources to compensate for FB lectures.
Quizzes contribute to 20% of the grade. They are difficult but are good for testing your understanding.
I liked the assignments. They helped reinforce the learnings from lectures. However I felt fourth assignment could use some improvements in instructions.
Graded discussions encourage to read some interesting research papers from past couple of years. They account for about 5% of the grade.
Although we got a good score in the group project, I didn’t have good experience with it. This depends on the team you end up with. To me, team members not attending scheduled meetings and leaving things to the end, was stressful. I would have liked if an additional assignment replaced the group project.
The grade distribution and absence of a curve requires one to do well on all the components.
Very nice course - probably going to be my favorite and the one I come back and refer to many many times in the future. The same deep fashion project I wasn’t even able to properly load in the data, this course helped (or forced lol) me to complete it. I can tell I visibly had some serious improvements in deep learning skills. The assignments are quite challenging, but I learned a lot by doing them. Understanding the assignments directly lead to the success of my final project, which pretty much saved me from getting a B.
The other reviews mentioning the drawbacks of this course, however, are true. The quizzes are hard and stressful, losing marks on them does add up at the end. The facebook lectures are very poor quality, I would say future improvements will hopefully remove them from making lectures, instead they will focus on providing guidance and mentorship on projects.
Overall, this is a great course. Very much worth your money, and some hard work. You should give it a try
This is a challenging class that covers a lot of ground. Like the other ML classes I have taken in the program so far (AI and ML) it is like drinking from a firehose. Deep Learning is one of the frontiers in terms of what ML has been able to achieve in recent years and is responsible for a lot of praise the field gets (e.g large language models like GPT3 and other popular advances in Computer Vision and NPL, which is something you will understand after this course). In this sense, this is an essential class to take if you are in the ML track or are seriously interested in ML.
Little bit about the class, as most others have said below, the first 1/2 of the class is really good. Be wared that it is math heavy but I feel like diving into the math really lets you absorb how deep neural networks are working. The second half gets progressively worse especially with the FB lectures. I think the class tries to cover too many difficult topics too quickly in the end. Hopefully they improve that in the future.
The 7 quizzes are difficult and can be a little annoying, especially given everything else you need to keep up with. I did like the exposure to all the really interesting research papers we had to read for this class and is one of my most valued take-aways from the class. Being able to read the most important DL papers on your own seems like an essential skill to keep up with this quickly growing field.
Hope you enjoy the class if you take it, prepare for a difficult ride and know that it will be worth it in the end. See you on the other side.
Completely no quality control on Facebook lectures. Completely unfair for TA grading on Final projects. Just don’t waste your time and money on this course and all other ML tracks.
I really enjoyed the content of this course, i think that Prof Kira is one of the best professors I’ve had! The assignments were interesting, the quizzes were weirdly challenging (they did force you to really pay attention to the material, but with little to no guidance), and the final project was just okay. I would have much rather had a couple of other homework assignments to cover all the topics.
My only complaint is the very poor quality of lectures by the Facebook team. The biggest problem is that this class is one of the few, if the only one, that actually teaches NLP, and it was poorly covered by a few Facebook researchers who are good at reading off the slides, but not great at actually teaching.
Class starts out strong but continues to get worse in every aspect. The lectures get worse, the assignments get worse and the quizzes are pointless.
The first two assignments are pretty good I felt like I learned something from them. The last two assignments are basically trying to figure out how to exactly implement the awful instructions that are provided and don’t aid a lot in learning.
The quizzes are absolute garbage. At the beginning they aren’t too bad but very focused on doing math exactly in the style taught. Then they just become useless trivia that in no way measures your understanding of the material.
Group projects don’t work in this program and the teaching staff made absolutely no effort to try to make it work. The provided projects are very poorly described and there is no check in or follow ups or anything to ensure everyone in the group is capable of contributing.
Ohh and the graded discussions are just a waste of everyone’s time
Overall there is some good material in the class and then it is ruined by the worst structured class I’ve had in the program. The class structure needs to be done which is pretty pathetic given how new the class is
TLDR: Great course, demanding workload and conceptual difficulty. Roll with the punches (quizzes, FB lectures). You will learn a lot and feel like you have earned it.
Instruction: As mentioned repeatedly below, Prof. Kira’s lectures are great and communicate useful, interesting rules of thumb about deep learning. The FB lectures are not good - too high-level and hand-wavy especially relative to the degree of knowledge that is tested on the quizzes. The TAs (special shout-out to Farrukh, Alex and Sangeet) are excellent, patient, and ran high-quality tutorial sessions.
Assignments: Hard & time-consuming, but worth the effort to get comfortable with architecutres. Best to come into the course feeling confident in Python and data structures. A4, relative to the others, was painful as the test harnesses weren’t as well-constructed as the others and the instructions were less clear. Getting a subscription to Google Colab is basically a must for the portions where you have to tune PyTorch networks to find good hyperparameters. It might feel a bit painful at first, but I highly recommend writing up a hyperparameter tuning script to save down best configurations and running it overnight. I tuned all my models manually and that was more stress than it was worth.
Quizzes: The complaints are mostly valid, lol. They are overfit to the lecture videos, so those that correspond to FB lectures are brutal. To minimize the damage you take from these, be overly-focused on every word said in the lecture videos (said, not just written on the slides).
Group project: It is what you make of it. I had an extremely good group. We attempted a novel architecture for our project which didn’t perform how we had hoped, but I’m glad we gave it a shot. Probably would have been easier/smoother to try something more vanilla.
In summary, I highly recommend this course. Easily the best course I’ve taken so far (prev. CN, DBS, DVA, SDP) and the most worth the workload. I came away with a strong and crisp understanding of a lot of concepts within DL, and a better programmer to boot.
Good overall. Prof Kira’s lectures are good. The FB ones, not so much. So you’ll need to take the initiative to find other resources for that material (e.g., Fei-Fei Li’s lectures). Note that you need to have some calculus and linear algebra background. If you don’t, you’re not going to survive the first couple of assignments. The assignments were fairly hard, but you learn a lot. If you’ve just been through a couple MOOCs on DL and think you understand NN’s and backprop, trust me, you don’t. Once you can get assignment 1 to work, you will. I see some people complaining about group projects, but if you form a group early and find smart, reliable people (there are plenty), it’s not going to be an issue.
Strongly recommend to have an introductory understanding of deep learning. Need to implement NN from scratch. Forced to read papers and think about them.
Update:
I would say it was a bad experience in Fall 2021. The first half of this course went well. However, the last half was totally trash.
Assignments:
Assignments took 55% weights of your final grade. There were four assignments. A1 and A2 were well-organised. As for A3 and A4, their instructions were ambiguous and should be clarified further. TAs graded the assignments leniently. But their feedbacks were always similar to “we did not expect that answer”.
Quizzes:
Most complaints were from quizzes. 7 quizzes took 20% of your score and had more multiple choices than before. As other comments said, some of them expected you understood the concepts that were only covered 2-3 seconds in the FB lectures but contributed 25% of one quiz. You worked hard on recitating lectures but usually achieved trivial or no improvements in the quizzes.
Graded discussions & Final Project:
I have no words about the Graded discussions and Final Project.
I hope I could not take this course if there was a time machine :( . Standford cs231N and Umich EECS 598 are better choices if you want to learn deep learning. Most content (included assignments) of this course was copied from cs231n.
11/22/2021, 3:01:09 PM:
FB lectures are awful.
Assignments are less organized. Instructions for Assignment4 are so ambiguous.
Graded discussion is good. It helps us review some famous papers.
The most annoying things are 7 quizzes!!! If profs and TAs want to examine our learning outcome, please use mid/final exams! These quizzes are time-consuming and useless. They will determine your final grade. It is a discouraging thing.
See “Difficulty” and “Workload”.
I have very mixed feelings about this course. During the first half, I loved it. I feel the course is very well organized and the course materials are fantastic. However, starting from the sequence models, I think this course is absolutely trash, the FB lectures are disasters.
I think the Deep Learning teaching is facing a very serious problem, in the sense that, people want more students to learn deep learning, so they reduce the difficulty and expand the topics to cover as much as they can. However, the quality is reduced dramatically.
-
Mathematically, this course started strong, and ended poorly. It has a good introduction on backpropagation and covers quite a bit about how to derive it.
-
The worst thing I don’t like about this course, or maybe Deep Learning education in general, is that, they ignore the fact that Deep Learning course should serve as a starting point as students go on their deep learning application adventure. It’s good that they focus on a lot of advancement in this field, and deep learning truly is constantly evolving. But by making deep learning course a memorization of all advanced methods is not a good idea, as least for me. Why should I remember each architecture without even knowing what is this? Like, what is ResNet, what is their advantage? This is becoming a literature review class. And for sequence models, it’s so bad. I admire people from Facebook, they are super smart for sure. But consolidating their years of research in a 15 minutes lecture is beyond me. They even had a hard time explaining embeddings…. We have to read extra materials on YouTube or medium.com to learn it. Then what is the point of including this in this course.
-
I saw many discussions about people wanting to have a natural language processing class. And maybe we should.
In summary, I think if we want to know more detailed stuff in Deep Learning, we should not take this class. But if this is the first class of your Deep Learning adventure, maybe it’s OK. I really hope the Deep Learning educators treat this as a “Computer Science Version of Calculus”. Not some research sharing, or research showcase. They are indeed useful, but not as something in this introduction course. Even worse, please do not quiz people with pros and cons of different architectures without letting people understand it.
This is a very hard course. The first half focuses on MLP and CNN. You need to implement back propagation from scratch, then in the last assignment of CNN you’ll have a chance to work with pytorch. The second half course focuses on NLP. It’s a very busy course, you’ll have quizzes, assignments, readings almost every week. There is also a big project to replicate a paper, which is not easy also.
Tips for this course is to start everything early, especially for assignments. Use the piazza as a resource, since often people would share their solutions on it (without codes). Also the office hour is very helpful for quizzes and assignments.
First couple of lectures were really good and after that it looked like rushed and incoherent. This is the one of the few courses with TAs least helpful, may be because I took the course it was first offered in OMSCS
I found the course extremely demanding but rewarding. I had some previous experience in Deep Learning through Udacity, but this course is a completely different ball game! The syllabus is complete: it covers the basics, the math behind deep learning, ConvNets, sequential models/NLP, attention models, and a little of more advanced topics. I loved the assignments and the final group project. (Not so much all the quizzes and graded discussions.) But I can say for sure my level of understanding and proficiency in Deep Learning is in a different level now, thanks to this course. If you are serious about AI/Machine Learning/Deep Learning, this course is a must do!
4th OMSCS class after RL, HDDA, and CN. Taking it over the summer, you miss a few sections on scalable training, deep RL, unsupervised/semi supervised learning, and generative models.
You also miss one of the four projects - you only do project 1, 2, and 4 in the summer session. Project 1 was about building a simple fully connected deep NN from scratch, using no ML libraries. Project 2 dealt with building a CNN from scratch also, and then using PyTorch to build a few CNNs and experiment with. Project 4 dealt with building language models using Pytorch: RNN, LSTM, Seq2Seq, and Transformer Architectures. The projects are worth 55% of your grade.
I feel a lot of the learning comes from the projects, both practical in application, and theoretical from the fact that you do make it from scratch - admittedly, I did feel I didn’t learn as much on project 4, however, since it is “less” from scratch; felt I was more following directions.
There were 5 quizzes for the summer session, closed book, proctored, untimed, 4% each, totalling 20% for quizzes - all multiple choice or “enter a number” (computation). They would cover lecture video content and reading - I did no reading, however. As everyone else has said, the lectures by the professor are impeccable, but the Facebook guest lecturers are pretty trash in comparison… they did do enough for me to get through the quizzes and assignment, however. The professor’s lectures have a good balance of theory and practice in them, while the Facebook lecturers are pretty much like medium posts. The math in this class is very light, and they manage to teach the subject adequately from a CS practitioner’s perspective.
Participation is 5% of your grade, which comes from 3 paper reading discussions - you basically pick a paper, read it, answer some questions posted on Canvas forum, and then have to have two replies to other peer’s posts. Didn’t enjoy this section at all, but I like the effort they made, as paper reading is invaluable. I felt maybe they could have made one of their projects a more paper-intensive task. I actually didn’t read a single paper, just skimmed a few and tried to answer to the best of my ability.
There is also a 20% of your grade group project, and with all group projects, are only as good as your group - I luckily got a very good and enjoyable group, so I had a very good experience with it, especially it being my first group project.
Overall, a must take for anybody who wants to learn deep learning! I do feel like RL is still the superior ML elective - much more math and paper writing in there. However, I’ve been trying to learn deep learning for so long, and this class finally taught me it adequately, so I am very grateful for that. My workload number is not very accurate, by the way.
Before taking this course, 1) I had only used neural networks as black box models to do trivial analysis 2) was not familiar with PyTorch.
This course demanded a lot of hours for completing assignments. The assignments help you learn a lot and are very well set up. They require understanding of OOP in Python.
The exams are pretty hard, but they don’t hurt you as much because they carry less weightage.
There was something due every week and so the pace of this course is relentless. In the end, it was worth it because I came out of it learning a lot in a very short period of time.
Feedback to instructors: a) At the start of the course, a lecture video is needed for calculating the computational graph for a 3-4 layer network. There was no full example shown and although there was an office hour session where this was covered, TA glanced over the crucial steps. b) NLP portion of the course did not have a good build-up of basics. As another review pointed out that there is really no good textbook. A ground up explanation of fundamental NLP would help students who are taking this course to learn from zero as opposed to being already experienced in the topic.
CS7643 Deep Learning - One Overhyped Class!
Disclaimer - I took this class in summer (for the love of OMSCS, don’t do this) and I ended up in a bad project team. So YMMV
Background - Non-CS bachelors; Comfortable with Python; Reasonable exposure to ML basics; Completed AI and CV so far
Grade - A (98% in assignments, 80% in quizzes, 96% in project)
Introduction
I have no idea why this course has so many rave reviews. It was not a terrible class (hence the dislike and not strong dislike), but nowhere near as good as some of the reviews suggested. The lectures are mediocre (don’t even get me started on the Facebook lectures. That has a rant section of its own below), half of the assignments were pointless and the group project was just two weeks of frustration with absolutely no meaningful end result. If you really want to get started with deep learning, I would strongly recommend looking at Dr.Andrew NG’s Deep Learning specialization on Coursera. I ended up doing that in parallel with this course and it did pretty much everything better than this course. The lectures were miles better in quality and the assignments were easier but much more meaningful and it was about $700 cheaper. I apologize if I come across as promoting a different course here, but I was so disappointed that an MOOC can offer so much content that is better in quality than a GaTech course.
Here is a section wise breakdown along with my ratings:
Lectures and Content (5/10)
In the beginning of the course, I was delighted that this course had its own dedicated lectures and slides and not some Udacity cut and stitch job. I enjoyed the first few (say 25%) of the lecture videos. I felt the professor - Dr.Zsolt Kira put in a lot of effort into introducing the fundamentals of deep learning. Basic neural network concepts, optimization, CNNs were all covered very well. The CNN architectures is where things began to fall apart for me. The lectures started to feel more rushed with a lot of diagrams thrown in but no proper context. (Dr. Andrew NG’s specialization does a great job here and teaches you the intent behind each of the architectures with a lot of insights on the implementation). From here, it was downhill. The Faebook lectures start to take over and the whole experience is ruined. Also, the textbook was not great and I ended up returning it in 10 days.
Facebook Lectures (0/10)
One of the reviewers used the term “pedagogical disasters” to describe these lectures. Those words could not have been more accurate. This is easily the worst aspect of the course for me. These guys may be world class software engineers and I respect them for that, but they should stay away from teaching for the rest of their lives. Every single video was beyond terrible and most of them go like this: Self introduction -> throw in some random technical term with absolutely no context -> brag about some cool stuff that Facebook did in this domain -> the end. Please Prof. Kira - get rid of these ASAP. These concepts are valuable and need to be taught well. Even if you provide links to YouTube videos on this content, I would be okay with that.
Assignments (5/10)
Massive disappointment. According to me, the assignments are the most important aspect of any course and can potentially save an otherwise bad course. Not here unfortunately.
- Assignment 1 - Really good. You build your own neural network from scratch, implement loss functions and backprop, visualize the learning, etc. Medium - hard in difficulty and I learnt a lot. Included a detailed write up section as well. In general for write-ups, as long as you explain your tuning process instead of just showing a massive table of hyperparameters, you should be fine.
- Assignment 2 - First part was building CNNs from scratch and was pretty brutal. But it was worth the slog. The second half was CNNs using PyTorch - good introduction to this library but nothing beyond that. The third part was class imbalance and focal loss. In theory this was interesting, but was not executed well in the assignment. The instructions were vague and we had no idea what end result they were expecting.
- Assignment 3 - Removed due to short summer semester. Why they removed a potentially valuable assignment and kept a useless group project is beyond me (same 2 week time frame for both!)
- Assignment 4 - Without doubt, the worst assignment I have ever worked on in any course so far. Had to build encoder-decoder type sequence models and transformers. It was setup in a terrible way without any introduction to what you were actually trying to achieve. 90% of it was wrestling with tensor dimensions and blindly trying to pass the autograder. I got 100/100 but learnt absolutely nothing.
Overall, the assignments (A4 and most of A2) didn’t teach me anything particularly useful or relevant to the real world. My advice is start early and keep learning expectations to a bare minimum
Graded discussions (7/10)
There were three of these - You essentially read cutting edge DL papers, answer some subjective questions and respond to others’ posts on Piazza. The papers were great, the questions were great, but the responding to others part was forced and you just end up making generic comments. On the plus side, they were graded leniently.
Group project (2/10)
Absolutely hated it. I really wish OMSCS gets rid of all group projects. I just don’t see the value of these in a setting where most of the people are working professionals and are in different timezones. We worked on this project for about 10 days. Technically, we had about a month, but in the first two weeks, there were a gazillion other deliverables. I am pretty sure most of the folks spent under two weeks on the project. (The longer semesters should definitely be better in this regard) The project itself was weak - pick some preprocessed dataset, tune some really basic models and write 6 pages on how we changed the world. Absolutely nothing meaningful was achieved. I don’t think it is possible to do a research paper type DL project in 10 days unless all team members are already proficient in this field. To make this worse, I ended up in a bad team - one person who didn’t bother to review the work others had done and suggested last minute changes to everything and another person who hardly showed up to meetings or did anything valuable. I am not even sure if the peer reviews are factored into grading. Overall, a hugely frustrating experience.
Quizzes (10/10)
The lone savior! This is the most challenging part in this course and can make or break your grades. There are five proctored quizzes (longer semesters have 11 I think) that are a mix of conceptual and computational questions. They are hard and you really need to understand the content well to get through them. I really liked the quizzes and felt a sense of accomplishment when I did well on these. Also, they were my main motivation to slog through some of the god awful lecture videos.
Grading (8/10)
Assignment grading was fair. Discussions and projects are graded leniently. Pay attention to the assignment write-ups which carry about 30% weight. You can potentially lose points here. The grading process is extremely slow and this creates unnecessary anxiety. Grades for Assignment 2 and 4, Discussions 2 and 3 were all released only in the last week. Overall, if you do the assignments, discussions, project well and somehow manage 60% in the quizzes (which is not easy), you have a shot at an A.
TA support/Office hours/Slack (8/10)
The professor was attentive and held office hours. Thank you sir! (However, based on past reviews, looks like he was more involved during the longer semesters). TA office hour support was stellar during the first few weeks and was helpful to get through assignment 1 and quiz 1. But then, it dropped off steeply. The later sessions were not very helpful. Piazza was a dumpster fire especially during assignment 4. TAs just couldn’t handle the chaos and there were numerous questions left unresolved. Surprisingly, the slack channel which is usually bustling with activity was relatively quiet. A shoutout to TA Farrukh Rahman who was active and remained extremely polite throughout the semester.
Note on Workload
This is a high workload class (20 hours or more per week). Without exposure to ML and Python, this will be borderline unmanageable. I strongly recommend you take this course in the longer semesters. This is definitely not a summer course. You end up doing 75% of the work in 67% of the time. It is a constant barrage of deliverables week after week. It will probably be similar in spring or fall but you would at least have more time to work on the project.
Overall (5/10)
In short, this course was a disappointment. Unfortunately, it is not a case where there are a few rough edges and will be eventually sorted out. It needs significant rework - assignments need to be made more meaningful, Facebook content needs to be redone, project needs to have a better definition and scope, etc.
Take this course if you want CS7643 - Deep Learning to show up on your transcript or if it helps you meet your specialization requirements. But if you want to actually learn deep learning, look elsewhere. There are tons of good resources online.
TL;DR its a good course and you’ll learn a lot but take the course in the fall or spring. its too much work in the summer
This course was a mess in the summer. They did not scale down the workload properly. In ~67% of the time we needed to do ~75% of the quizzes, discussions and assignments and 100% of the final project.
The assignments are the best part and I learned a lot. They were time consuming but manageable (this is a theme in OMSCS. most assignments are difficult but manageable if started early and a disaster if started late). I learned a lot while doing the assignments.
The final project was fine. I would have preferred a solo project but my team was pretty good and I learned a lot during it.
The quizzes were OK. There were some interesting things we needed to study in preparation for them (computation graphs, parameter/dimensions calculation) but some questions felt like trivia and we had to memorize equations and I don’t see the point of memorizing these kinds of equations in order to do arithmetic at this level.
The discussions were a waste of time. The papers were interesting to read but no real discussion was facilitated. People simply left generic comments probably because we were all swamped with other work.
I would recommend this course in the fall or spring but not summer. That should make the pacing of this course more manageable. It felt like we had two things due every week and coupled with summer vacations it was this constant feeling of dread of due dates.
I’m writing this review from the perspective of someone who took it in the Summer, so it may not be indicative of your experience in other semesters.
Pros:
- Professor shows concern about improving the class.
- There is a level of math understanding required that is higher than in other CS courses in the program
- The assignments are useful to build intuition by coding low-level operations manually. The only one that I didn’t enjoy was Assignment 4 where I finished but felt I didn’t understand transformers completely.
- Deep learning is still cool I guess.
- Quizzes are good, but they rely too much on the lectures/slides and some questions resemble trivia at times.
Cons:
- The Facebook lectures are TERRIBLE and there are many (roughly half) by them. Each one is taught by a different researcher from Facebook, but the common thing is that they all suck at teaching or don’t care about the quality of their videos. They are overly short, have no build up (they often start speaking about concepts at a deep level without properly introducing them to the students), and are mostly inferior compared to even random Medium posts at explaining similar topics. Most of these lectures were clearly based on Stanford’s CS231 and CS224n content, with many of the slides straight up copied from it, and it would be wiser to seek this original content instead as they’re available for free on the internet.
- Summer had the last module, the most advanced and interesting, as optional. While the lectures were still available, there was no quiz to help test your knowledge and to act as incentive to walk through the lectures. Less content overall compared to the full semesters was a let down.
- Professor was mostly absent this term. He held weekly office hours and made sporadic appearances on Piazza, but based on past reviews I believe he was less involved in this semester.
- Personally, I found many questions on Piazza going unanswered for very long compared to other classes. And the students/instructor answers were often insufficient and lacked depth.
- I didn’t find the office hours by the TAs very helpful. They felt similar to the Facebook’s lectures, as in the TA didn’t seem comfortable with explaining the material in a structured, pedagogic way.
- The book could have been explored in more depth. Particularly, I felt the mathematical rigor present in the book wasn’t explored in the classroom activities.
Prof Kira had some good lectures and was active on Piazza.
TA and student quality of responses on Piazza was pretty lacking though. Response time on detailed and thoughtful questions was pretty slow with not always the most insightful reply.
Facebook lectures were lacking in depth. Ex. they talked about beam search, but never mentioned length normalization, then the quiz had a question about length normalization in it. Having already done Andrew Ng’s courses this wasn’t a problem personally, but be warned the lectures are not all you need. I recommend prepping for the course by doing those.
I liked the quizzes, they actually tested if you truly had a grasp on the material. The above about length normalization is an example of why people may not like the quizzes. However, if you stop at lectures you’re hurting yourself. Papers, the book, and some further self study are crucial if you want to do DL for real. You get out what you put in. For example, the book had a great section on the loss function surface for RNN/LSTM type models and mathematically proving why gradient clipping was needed. That next level of understanding will take you a long way at work, since DL is sensitive and missing details will be crushing. They give a very generous study guide on what areas to focus on for the quizzes.
For the project, pay attention to their request of the effort of one assignment per person for the project. DL is fun but takes time to get to the cutting edge stuff, especially if its a newer subject for you. A lot of teams set out to change the world and ended up changing scope. The example projects are designed to push you but focus on quality of experimentation and analysis over how fancy your project is.
For my background I have a couple years of experience working as an MLE.
- Assignment 1: 8 hours, straightforward calc
- Assignment 2: 20 hours, CNN backprop takes a while to think through
- Assignment 4: 25 hours, the random seed initialization to get past the LSTM auto grader was a pain
- Project: 30 hours, group was meh
Grades:
- Quizzes 90%
- Assignments 99%
- Project 97%
Good class, not as incredible as others may say, a good balance of work.
First time this course is being done in the Summer. This is my 6th class in the program and has had easily the fastest ramp up in work. This will be the end of the third week and given that I have a full-time job, I feel like I’m way behind. For all of you taking this in the Summer, I wish you good luck, god knows I will need it.
Have now finished two quizzes and the first assignment. First quiz went well but relied on being comfortable with the calculus. Second quiz was rough. I spent a lot of time going over lectures and I felt like I understood the material well. Ended up feeling like the quiz was just trying to trick people and did not reflect the lectures well on several of the questions.
Assignment 1 took a lot of work. Definitely needs a couple weekends on it at least. The code we were given was good, I am happy with how the assignment was built. However, the guidance within the pdf and the comments in code left a lot to be desired. I would say some of the comments in code were even misleading in a couple spots.
Lastly, the lectures are passable. Definitely near the bottom relative to the other classes I have taken. The audio quality is poor and can be distracting. I don’t think the material covered is divided into all the lectures very well. That is, an individual lecture is probably fine, but I usually was left thinking that it was not connected very well to the rest of the content. And lastly, it drives me crazy that there is no pointer that is used for me to follow where in the slide I should be looking. Often times there is a lot of content in the slide and I have no idea where I should be looking. I had to rewatch lectures just from trying to decipher when I should look at what.
I now have just the group project left to do. I’ve hated this class. The subject matter is super interesting and that’s helped me get through the semester, but I’ve hated this class. If you would ever need to know DL, then take it. But with a full-time job that is about 50hrs/week, this class has added a high level of stress to my life this summer. In particular there are always overlapping deadlines. For example, you might have an assignment (worth 20%) due next Sunday so you’d like to start this (previous) weekend, or you can study for the quiz which is due this Sunday which is only worth 4%. I always ended up studying the whole weekend and that barely got me to the average Quiz grades. And then the next week was an absolute hell trying to work on the assignment. I always felt behind and so it’s been 3 months of stress. The subject matter alone is not ‘Very Hard’ but I am rating it as such because it has been the hardest class I have taken so far after two years in the program, and because consequently it has had the worst impact on my day-to-day life.
This course is a must-take for anyone doing the ML specialization. It gives a broad overview of many of the deep learning techniques currently being used in industry and research. It is overall well-taught and the material is fascinating.
Pros: -Professor Kira made himself very available with weekly office hours and a lot of support during the semester. He was clearly very invested in his students’ learning outcomes. -The combination of programming assignments (building neural network components from scratch in assignments 1-2 and using PyTorch modules for assignments 2-4), quizzes, final project, and research paper discussions helped me understand and retain a lot of the material from a few different perspectives. -While the quizzes covered a lot of material and really demanded a strong understanding, each one was accompanied by a study guide that included most of the major topics to focus on. -Professor Kira is very open to feedback and is working on continuously improving the course.
Cons/Areas for improvement: -There was some uninteresting ambiguity in one of the assignments. Sometimes a specific function had to be used to match the expected output or a neural network architecture was not clearly described. -The lectures co-taught by Facebook employees had inconsistent quality and depth of coverage.
Things to be aware of: -The course hits the ground running and there is a lot of material to wrap one’s head around in the first month.
This course is very similar to CV, especially the assignments and final project, but it is easier and takes less time than CV.
Every assignment is about implementing some DL methods (e.g. backprop, CNN, RNN, Attention) by filling some unimplemented functions. Local test code and an online auto grader are provided. There is also a write-up portion but some brief explanations are enough.
Quizzes are the only difficult part of the course. Average was below 80%. The questions really test your solid understanding of the concepts covered.
Group project may sound challenging at first, but it is graded very leniently. As long as you cover all the points in the provided rubrics, you will get most of the scores. Report seems more important than code and experiments.
Overall an easy-A and light course.
The course content is very good overall. I suppose I should classify it a survey course. If you have previous experience with deep neural nets, as I did from CV, then this adds to that knowledge pretty effectively, but frankly it isn’t a very challenging course. There are very many activities to keep up with (several office hours per week, lectures, readings, graded discussions, assignments, quizzes, final project). I didn’t find anything in particular very difficult, but it is a little overwhelming scheduling-wise. Of all the graded assignments I found the (proctored) quizzes to be the trickiest to do well. Assignments are quite easy compared to other classes (as I remember, they are mostly filling in TODO sections with pytorch code), especially if you have numpy or pytorch experience (since pytorch operations are very similar to numpy), and the assignments are unit tested, so you know at the time of submission what your graded score will be (except possibly by some fluke). Some of the tests within the notebooks they provide were a little lacking in coverage, but it was only a small difficulty. Written analysis portions of the assignments are very easy compared to e.g. ML. The final project is a group project, with all of the potential pitfalls, but for me it was my favorite part because it required the most coding, the most time, and was the biggest challenge of the course. If you are thinking of also taking CV, I can say that I took CV first and did the CNN (individual) final project in that course, so I was well prepared for this class. Taking them in the opposite order could also work well, I’d only be guessing. I can say there is some overlap.
tl;dr I’d give this course 12/10 and you should definitely take it.
I’ll be real, I took a few days to recover before writing this up. Sure, I worked right up to the last hours at the end (earlier part of the course is easier to time manage), and, yeah, I and my teammates had a hard time being available for each other and keeping up with our individual workloads in the course (alongside FT jobs and late pandemic burnout), but this is hands-down one of the very best classes I’ve taken. The workload is constant but well delineated and hours put in will almost always translate to progress. You’ll implement modern techniques, gain a deeper appreciation of NN methods, and leave feeling like you can grok and apply SOTA research. This is a rapidly changing field, but I have little doubt that Prof. Kira will keep this course updated with meaningful developments, if not in directly in lecture than in other aspects of the course. If you have experience with multivariate calculus (even if you’re rusty) and a broad understanding of ML algorithms and challenges, you should take this course. I personally don’t think a GPU is required in this class, depending on your ambition for the final project. I did all of it (even the project) on local compute, and the GPU available on colab would be enough for the assignments (which, again, you can run on CPU just fine, it’s just faster to tune on GPU).
1] Quizzes: If people have a gripe about an aspect of this course, it’s likely the quizzes. I think a lot of that is the mental association we have with the word “quiz”; unlike the low-stakes high-level knowledge that word evokes these are more akin to mini-exams. Because of the rest of the coursework is graded quite fairly (bordering on generously), this 20% is in practice the differentiator of your letter grade. I personally found them fair but challenging; they do require you to have thought through the implications and meaning of topics which sometimes explicitly spelled out for you in lecture and sometimes something you need to stop and think about on your own. I found that, provided I stopped the lectures frequently and really thought about what was just said or explained, I did great on the quizzes (YMMV). There was only one quiz that felt a bit punishing in that it had some “chained” calculation questions (so if you got the first one wrong you’d actually get 4 wrong), but that was an outlier. Early on, TAs ran dedicated OH and made preparation worksheets for the calculation questions.
2] Assignments: The current 4 assignments are building your own NN (including differentiation for gradient descent), building your own CNN then implementing on in PyTorch, visualization of features and style transfer, and building RNN, LSTM, and Transformer solutions for NLP. Code is autograded (unlimited submissions) and most of the points; for the written part just make sure you actually offer an explanation of why you think you got what you got. Instructions are generally really clear (and there are copious office hours if you’re stuck). The only assignment that I think needs a little tweaking is the final one; it felt a bit more high level vs. the more granular nature of the others.
3] Graded Discussion: This might feel like busywork to some, and I won’t argue with that much. I will say that it was a prompt to read and think about papers that either extended the lecture materials or were a glimpse into late-breaking developments. It feels like a good part to keep in the course as a way to explore new findings or techniques, and while the “conversation” can feel a little artificial given the length rubrics for a substantive contribution, it was interesting to see how other students interpreted and responded to the papers.
4] Project: Yes, take this class even though there’s a group project. If you’re the type that wants to get a paper out of a class, you can go for it. If you want to be ambitious and play with some Facebook problems, you can do that in this course. Everyone else, you’re OK, too. People of the first two types should try and find a group by the halfway point of the class and start working, because the rest of the course keeps up the pace and those few weeks at the end aren’t as much time as you may need; there were occasional glimpses of data accessibility issues and group squabbles on Piazza. Everyone else, it’s OK if you just find a niche part of DL that extends the ideas in the class (that’s what we did, and I was grateful our project was so contained and could run on local compute). Remember, the two biggest issues you’re going to have with your project are sourcing your data and compute, so brainstorm with those constraints in mind.
5] Office Hours: This part really shines. There are weekly OH with Prof. Kira, whose passion for the topic and commitment to students is beyond evident, if you want to get his take on recent DL developments. There were also multiple OH with Facebook engineers, which is just a phenomenal opportunity (even if I found them a little wan). And we’re not done; the TAs ran recurring OH 6 days a week, some days even had multiple time slots. During some of these they’ll do breakout sessions and offer you a code review if you’re truly stuck (you can also get code review via private piazza post). Compared to most courses in the program, this course’s OH are an embarrassment of riches. The TA presence is outstanding, and so is Prof. Kira’s. Everyone is genuinely very helpful and positive.
I loved this class. Hard, but doable, which means the instruction and assignments were on point relative to the rest of this program. For reference, I got an A, am almost done with OMSCS, and have extensive ML experience from work.
Notes
- To get an A, finish the assignments and find a good group for the project.
- I averaged 60-70% on quizzes just because they personally took way too much effort to study for and I wanted to use my time elsewhere, so I just watched the lectures once after the first few. My friends who did better read the papers and external materials thoroughly.
- Assignments are the best part of this course aside from a few “I have no idea what’s going on” moments and having to lean hard into slack / Piazza for direction. They really teach you modern deep learning and the PyTorch parts were so much fun
- Use Google Colab. Yes, you’ll need a GPU, but Colab will get you through all assignments + Pro is only $10/mo if you need it for the final project. My group ended up using Pro which has a 3 parallel notebooks and a 24hr consecutive limit to GPUs and got everything needed done by splitting up the training.
Final thoughts: if you’re on the ML track, this class might seem like another hard class after ML, but is completely worth it. If not, but are interested in ML, take this over anything else.
I like the assignments. The concepts are developed from ground up. TAs are very helpful. The link shared on by the colleagues helped to understand challenging concepts from different dimensions.
Deep learning is one of the best and most useful courses I’ve taken via OMSCS.
This class will teach you what deep learning is, how it works, and how to use it in the real world. If you are serious about going into deep learning, this is an excellent way to get your foot in the door. I’m pretty sure I could tackle some compelling Kaggle problems at this point.
Thinks to Know
-
This class requires a good deal of math in the beginning. If you aren’t fresh on calculus and linear algebra, be prepared to spend a lot of time in the first ~5 weeks grinding. You don’t need to know all the math to succeed on the homework/quizzes, but you’ll be severely limiting how much deep learning intuition you’ll get out of the class. I highly recommend spending the time on the math early and often to both make your life easier and improve your learning outcome.
-
This class requires a GPU. I’ll repeat that. THIS CLASS REQUIRES A GPU. While you can technically finish this course without one (I did) and get a good grade, it is way more painful and you won’t learn as much. Instead of spending time doing deep learning you’ll be spending time pushing your code to expensive AWS instances or wrangling with GCP/Google Colab environments. If you’re like me, and you are in this program to learn as much as you can, spend the money. Buy a 20x or 30x series GPU (ideally 2080+). It’s 100% worth it and you can sell it after if you don’t want it. NOTE: DO NOT BUY AMD. The ecosystem for deep learning is based around NVIDIA GPUs. Don’t try to go against the herd, it’s not worth it. If you’re reading this in 2023 it might be different, but if it’s 2021 for you it’s 20x/30x series NVIDIA or bust. Tl;DR: Go buy a desktop on which you’d play Crysis at full settings.
-
The final group project is your time to shine. The last ~5 weeks of the class is a final project. We had no final for this class (not sure if that will change), so it’s really ~5 weeks of uninterrupted group project. Pick a compelling, challenging problem and be prepared to bang your head on it learning how Pytorch works. If you use this project time wisely, you will know how to train a “real world” DL model.
Pros
- As stated above: you will know what deep learning is, why it works, and how to use it.
- Professor is awesome, TAs are great, course pacing is good, assignments are good, quizzes force you to study, and the final project is well-structured.
Cons
-
Facebook content is weak. Compared to the professor-generated lecture content, the Facebook lectures are pedagogical disasters. Hopefully this content gets removed or re-recorded, but you’ll just have to suffer through it for now. Fortunately there isn’t too much of it and nearly all the critical lecture material is produced by the professor (and is therefore awesome).
-
Quizzes are brutal. I put this here but it’s not really a con: you will be forced to thoroughly review the concepts of the course frequently, which is a good thing. However, be prepared for some pain. Study early and take good notes.
-
The textbook is (IMO) quite poor. It’s either too meandering or too difficult for introductory deep learning students. I think there just isn’t a good deep learning textbook yet, which is why the course textbook is so poor. Hopefully this will change soon, but in the meantime don’t feel too bad if you can’t figure out what the textbook is saying. The professor will clarify on Piazza exactly which sub-sections/sub-chapters to read. I recommend waiting for that post and only reading those subsections (but read them thoroughly).
Overall, this is a fantastic course and you should take it if you can.
TL;DR: Take this class, BUY A GPU BEFORE CLASS STARTS, and don’t fall behind.
Even after ML and RL, this course was not easy, but worth it. I liked it overall but found a few things annoying. The annoying parts:
- Projects could have had better descriptions and instructions. It was more like guess what they want to pass unit tests (some of which were flaky). Too much time was spent on guessing and googling
- Lectures by FB were mostly crap. One I couldn’t understand half of what the girl from FB was saying (very strong accent). I have nothing against accents (I have one myself) but you have to think of the wider audience when the speaker is halfway unintelligible.
- Reinforcement Learning lectures by a grad student = WTF. I took RL class and it was still too hard to follow. They squeezed the whole course into 40 min and expected people to just follow it.
- Final project.. Very little guidance, difficult to pick just the right amount of work to split between multiple people. I was VERY lucky with my group (smart, hard-working, and helpful) but this could’ve gone bad as well. I was just lucky
The good parts:
- Lectures by prof were mostly great
- The TA’s held very frequent Office Hours
- One of the most responsive and attentive profs in the program. First time in my 10 classes where the prof held weekly Office Hours as well. And those were good, whenever I could attend.
- You can learn a lot. Now I want to continue with some of the areas independently. It’s still a high-level overview of many areas but they are a recent development in the field, some SOTA.
Overall, this course can be GREAT. Just drop FB lectures and group Final Project.
Edit: Final Project is graded fairly leniently. As long as you put some effort, do background research and explain your experiments you are likely going to do well.
Final course in the program which is also the one I spent most time vs. other courses (had BDH, CV, ML, AI, DVA, ML4T, AI4R, RL, GA). There’s something due almost every weekend e.g. 4 coding assignments, 7 quizzes, 4 paper reading/discussions, and 1 final project which easily takes 3 weeks of dedicated efforts. So be prepared for this especially if you have a demanding job and family/kids.
On the other hand, I have to say that I learnt a lot from this course. I have much deep understanding on how back propagation works and how CNN learn its filters. I think the reason is that you really get the opportunity to get hands on exp to write the actual code that’s the core of TensorFlow or PyTorch (of course it’s toy size coding here…). TBH this is a must have if you are or want to be a ML practitioner, or you’re on ML track. I felt lucky that they open this course right on time so I had the chance to take it.
For sure there’re things to improve, e.g. course videos are too shallow and sometimes can be hard to follow due to lack of detailed description, the FB office hours and content can be more helpful if they include more prepared, detailed discussion of a topic and not just answering questions. Also I wish the final projects provided by FB can have more help available on troubleshooting their existing code/libs (we spent 2 weeks just to get their ‘base’ code run…). But overall a great course if you manage your expectation/workload well.
Definitely one of the better courses in the program. Content is current and for the most part well presented (see below). Assignments are fairly involved, but plenty of time is given for them. If you can prepare for the assignments by watching the lectures and finish early, there will be some weeks where you won’t have that much to do, unless you decide to do all the optional reading, which gives some opportunities to breathe for a bit throughout the semester. Compared to the ML course, I liked this one a lot more, and the workload is slightly less I’d say (because you aren’t spending your entire weekends writing reports). My only complaint would be that some of the Facebook lectures are pretty weak (it depends on the person that prepared them; there are 5-6 different Facebook lecturers). I didn’t get much out of at least a few of them, either because they didn’t go into much depth, or because they assumed too much background knowledge and/or glossed over stuff too fast. Wish the instructor would re-do some of these lectures, or at least supplement them a bit so that they’d be easier to follow. His lectures were much better.
Great class overall. The assignments still need some work - I learned to start slightly later than I would have liked to, just so some other students can iron out the bugs (thanks to all you trailblazers!). They are mostly auto-graded, and the report section is, in my experience, graded fairly leniently.
Lecture quality varies depending on topics. I believe (hope) that they are also improving that aspect as the class matures.
Group project is kind of an annoyance. I did end up with mostly fine group members but all the common issues that usually comes with group projects still apply - nothing really specific about this course.
Quality of quizzes also varies quite a bit - they’re probably still figuring things out as well. Professor will post a thread about the main topics covered by the up-coming quiz, which, again, varies in helpfulness.
All in all, a great course but still in the making. Follow the schedule, turn in your assignments, and you should be fine!
An OMSA Student Perspective
I took this course in Spring 2021, the first semester that it was offered to OMSA students. I am part of the OMSA program and don’t come from a direct computer science background.
Summary
I highly recommend this course to anyone that is part of the computational analytics track! It is very thorough and detailed, but also very difficult. If you actually want to learn and understand the material, it will take a lot of time out of your day. It teaches you the fundamental DL skills, and also the basics of state-of-the-art technologies. It was the most challenging course that I’ve taken in my life, but also one of the coolest and most rewarding. I feel that this course has taught me all the skills that are needed to keep up-to-date in the ever changing field that is Deep Learning.
Overview
This course was divided into 4 parts: Assignments, Quizzes, Participation, and a final project. The lectures were like drinking through a firehose, but I wanted to learn everything and it all seemed important.
Dr. Zsolt is a great prof. His lectures weren’t too long, and everything that was taught was things that I wanted to learn. He was the most active professor that I’ve seen on Piazza in the entire program. The TA team was hit and miss, but he was always there to answer questions if it was needed.
Assignments
There are 4 assignments and are worth 55% of the final grade in total. They are very time-consuming, but you are proud of your work after they are done. They involve everything from manually programming a simple CNN, to using PyTorch for language prediction. They capture the fundamentals and teach you the basics of Computer Vision and Natural Language Processing. I found they captured both theoretical understanding and practical application very well.
The assignments are very programming-heavy. It is important to understand OOP in Python going into the course. I feel that if you come from a computer science background, the assignments would be much less time-consuming, but having to learn programming methods and practices is a skill that I will carry forward throughout my career.
Some parts of some assignments are autograded, but I never had any issues with this. Some reports need to be submitted, but they are as simple as copying a photo or table into a PowerPoint slide template - no LaTeX or unnecessary explaining required.
Many of the projects involve PyTorch. This benefits strongly from having access to an Nvidia GPU. I have an AMD GPU that is not compatible with PyTorch which was a real bummer (there are workarounds if your AMD GPU is Vega, but support for NAVI doesn’t exist yet). The workaround is to use Google Collab, which is a free service that utilizes cloud GPU processing power. It’s a little more clunky than having a local GPU, but it certainly does the job.
There was an assignment that would only pass local tests if you had an Intel CPU. I suspect that AMD CPUs will be supported in future iterations of the course.
Quizzes
There were 7 quizzes, worth 20% of the final grade. I did not like the quizzes at all. They were highly theoretical that tested your comprehension ability rather than your understanding of the material. They were proctored, closed-book, closed internet and you couldn’t use your calculator. There were both theoretical questions and calculations. The quizzes were very difficult and did not seem to fit the theme of the course. They tested both lecture and reading material.
Participation
4 graded discussions counted for 5% of the final grade. These involve reading selected research papers, summarizing them and answering some questions, and then replying to other people’s summaries and answers.
Originally, I did not like these at all, but as I did more of them, I grew to like them. They are an opportunity to learn state-of-the-art technologies, and since Deep Learning is such a dynamic field and a very active domain of research, I think that this is an important part of the class. In this field, it is important to know how to learn about new technologies.
Project
At the time of writing, the project is currently in progress. It seems like any other project in this program - form a group, write a proposal, submit a report. The project guidelines are quite general and it is a fun and exciting way to apply your ideas or explore new ones.
This is a great class, and anyone who is interested in Machine Learning should really consider taking it. It’s very practical and hands-on. It is very math-heavy, especially in the beginning. Hard to imagine a more relevant machine learning topic today.
To prepare…brush up on your matrix calculus skills and check that you have some basic ML skills. Although the requirements say you should have taken ML, I did not. I did take ML4T and AI before though, and this plenty. I didn’t feel left out at really any point.
The lectures by Dr. Kira are excellent, top-notch. The ones by Facebook are…varying in quality. Some are good, but I feel like Dr. Kira could do a much better job. Overall though, definitely some of the best lectures.
The assignments are like any other auto-graded class. You have a series of steps that build on one another, you pass or you fail, iterate, yada-yada. Most of the assignments have good unit tests that give you that warm feeling of getting a good grade. The only thing is there is a written report you have to do which is about ~30% of the assignment grade, but it’s not bad. (Definitely not like those dreaded CP reports)
There are discussions in the course, but I actually enjoyed the papers they centered around. These will not take much of your time.
I do have a few complaints so far. One is that there is some roughness-around-the-edges. I knew that going in but it has actually cost me several days of work only to figure out the instructions of one of the assignments were wrong. In this case, it pays NOT to finish assignments early, or else you will fall prey to this problem. (Ahem…Focal Loss).
I also think more of these assignments should be focused on some of the applications rather than designing the networks from scratch. There is a lot of “create this from scratch and struggle through it” and then “muahaha, see how it only takes two lines of code with PyTorch”. Maybe it would be better to spend that time doing some interesting things instead that we can walk away with.
Also, if they made the assignments a little smaller, they could squeeze another one in. For example, at the end there are modules on RL and Unsupervised Learning. The RL one was so briefly done by the Facebook lecturer, it doesn’t do the subject any justice.
Another is that the quizzes can be worded strangely and in such a way that studying any amount of time will not fix it. I don’t know how to describe it, but it is like some of the questions are designed to trick you. They seem to me like they have been designed by someone who really has a lot of experience with DL and forgets we are students still learning. It’s really frustrating being super confident walking into a quiz and feeling utterly defeated when you walk away. PS: The class average on one of the quizzes was a 59%. Thankfully they are only 20% of your grade.
The final project varies, and you are pretty much required to be in a group. I suggest you look early and in your time zone. My team so far has been great and the content pretty interesting. The Facebook projects are both really complicated and interesting.
Finally, the workload is probably 15-20 hours a week, much like AI sans the crazy exams. Definitely a more front-loaded course. Overall, good feelings for this course and I learned a ton.
I recommend this course. While the assignments were rough around the edges as far as deliverables, in 1-2 semesters they should have it down pat. I learned a lot from the coding assignments (4 assignments of about 2 weeks worth of work each) and the project (1 project about a month of work), so in that sense they should be considered well done. The weekly quizzes keep you honest in keeping up with the lectures. Some of them helped me understand the lecture content, while some were more like knowledge checks. I anticipate I will not retain a lot of the info tested on the quizzes, but not so with the assignments.
The project is a group project, and (no fault of the class) I had a not-so-good group experience, as happens sometimes in OMSCS. Never-the-less it greatly increased my understanding of the content.
The group discussions were ok (4, about a days worth of work each). Reading the paper, writing it up, and commenting on others’ write-ups definitely increased my understanding of deep learning, but I would say it failed as a “discussion” per se. This is mostly due to the format of Canvas, which alerts you when anyone in the class comments on anyone’s response. The only thing to do with the notifications are to ignore them. If we only got notified when our own contribution was commented on, perhaps a better discussion could be facilitated.
I wish there were more assignments on the later topics like deep reinforcement learning, generative models , and unsupervised learning, which are all very complex and very interesting topics which build on the earlier topics of CNNs, encoders-decoders, and other building blocks.
The professor and TAs had great engagement, although that is expected since this is the first semester it was offered.
Overall, I would say the workload was similar to AI. The course is not as polished, nor the assignments quite as open-ended as the first two in AI, so I would consider AI a better course overall. Hopefully within a couple of semesters this course will get there. I wouldn’t recommend waiting for the perfect time, though, as this course is still very good as it stands.
The weakest part of the course is the guest lecturers from Facebooks’ AI researchers, but it wasn’t a huge hindrance, as the lecture slides provided the needed content.
I really enjoyed this class!
There were weekly readings, lectures, and quizzes. I could not figure out how to do well on the quizzes, ended with an 80% average on them. Luckily they were worth only 15% of the final grade, with the lowest one dropped. I found the lectures to be very well done and explained the material well.
I felt a little personally conflicted that the class was co-taught by Facebook, as I don’t think they use deep learning in an ethical manner.
The assignments were great! I am a serial procrastinator so by the time I started on it most of the bugs were worked out. The tests and Piazza conversation made them very straightforward and reinforced the material. I didn’t personally get much out of the project, as the way we divvied up the work I didn’t get my hands as dirty with the DL work.
Overall I loved this class and think it’s a must take in addition to ML and RL for the ML specialization.
I’m not going to go over what was already discussed, but wanted to chime in with a few of my thoughts:
Prerequisites:
- Machine Learning - the course assumes that you have the necessary knowledge regarding concepts taught in the Machine Learning course, although I wouldn’t say it’s a hard requirement. Just note that you may lose some points whether it’d be on the quiz or project because you don’t know something they assume you already know.
- Read the paper “The Matrix Calculus You need for Deep Learning” (https://arxiv.org/abs/1802.01528) and you should be fine with the math in the course:
My Concerns during the class:
- Lectures were released on a weekly rolling basis. I believe that they heard the negative feedback from students and are now releasing all of the lectures at once.
- The lectures provided by Facebook weren’t that informative and only provided a really high overview of the topics. They were a very stark contrast to the lectures provided by Dr. Zsolt.
- Quizzes were either hit or miss: the content that we were tested on was either conceptual or applied and the difficulty ranged from too time consuming or could be completed in under five minutes. For the applied quizzes, we had to do calculations on paper that didn’t really feel appropriate. It would have been better off as practice problems. Although towards the latter part of the semester, Dr. Zsolt provided some insight on what was important to know for the quizzes.
-
However, I wouldn’t worry about the quizzes as they are only worth a total of 15% of your grade. For our term, the lowest score was dropped and we were offered an optional quiz to replace another low score.
- The assignments had ambiguity and the instructions were unclear initially, but the TAs worked hard on fixing and making the assignments more clear quickly.
- The final project was rough for my group, but in the end we were able to pull it off. I would say to contribute in the Slack channel and figure out who you want to work with early. Ideally, you get a good group and have no hiccups, but anticipate for some problems especially during crunch time.
Overall, Deep Learning was my favorite course so far due to the content discussed as well as the teaching staff. It seems like the grading was very lenient for the assignments and with the way the class is weighted, you should be able to get an A (>60% of the class).
EDIT: The grade distribution ended up being: A - 70.0% B - 9.3% C - 4.0% W - 14.0%
Summary
Overall, this was a good course, but I think it has the potential to be great with some modifications. The class is heavily front loaded with more difficult assignments and well prepared / delivered lectures by Prof. Kira. In the second half the lecture quality drops substantially with Facebook engineers delivering most of the lessons. This class is technically a “collaboration” with Facebook, which I interpret as Facebook trading lectures for first dibs at recruiting after the course (Georgia Tech definitely got the short end of this stick). The final project is a group project with no individual option, which I felt was more of a lesson in logistics than it was deep learning. Grading is very generous compared to ML, RL, CV.
Professor
Professor Kira was incredibly engaged. This is my 8th OMSCS class and I think this the most engaged I’ve seen a professor. He clearly has a passion for DL and teaching it. 10/10.
Lectures
As I mentioned above, the early lectures are quite good and well organized. Not all information for completing the assignments can be found in the lectures so further reading or researching is required to complete the assignments (which is fine). However, after around the half way point each lecture is taught by a different Facebook engineer. The lectures range in quality, but the average is quite low and I was disappointed that Georgia Tech decided to partner with Facebook on this class.
Assignments
There are a total of four assignments
Assignment 1 requires implementing the training pipeline (backprop and cost function) for two network architectures. Working through the linear algebra took me some time, but ultimately I thought this was a great project for understanding the math going on under the hood.
Assignment 2 focuses on CNNs. The first part of the assignment requires implementing a CNN training pipeline from scratch (similar to Assignment 1 except there are some nuances in dealing with the pooling and conv layers). The second part introduces pytorch. Overall, a good assignment, although by this point I felt I’d done enough “implementing from scratch” and was excited to start using pytorch.
Assignment 3 deals with visualizing network layers and performing style transfer. Overall a really fun project that helped build intuition around how CNNs work.
Assignment 4 focused on RNNs and encoder / decoder architectures for the application of language translation. I was honestly disappointed by this assignment. The autograders were too simple and didn’t catch bugs early on, which allowed you to get pretty deep into the assignment before you’d find an issue in some block. At the end, it felt mostly like an exercise in debugging tensor dimensions. Even though I received a perfect score on the assignment I felt at the end that I learned little.
Quizzes
There is a proctored quiz each week to check comprehension. The quizzes are overall worth a small part of the grade, but served as good motivation to stay up to date on lectures. I thought this was good.
Graded Discussions
There are four graded discussions where you’re given two papers and asked to choose one to review. Once you’ve submitted your answers you are required to comment on others’ submissions. Overall, I thought this was a nice way to connect to the literature in the field of deep learning. Reading the papers and throughly understanding them could be time consuming, but I felt I learned quite a bit.
Final Project
Students are required to form groups of 2 to 4 and either select a project from a list or propose one. There was no option for individual projects. This was by far my least favorite part of the course. In classes like ML, CV, RL I derived immense value from working through all parts of the problem and coming out the other end feeling satisfaction in the amount I had learned and worked through. Instead, in deep learning we spent way too much time working through logistics of finding common times when we could meet (dealing with jobs, time zones families, etc) and who would do what. I really hope they decide to make the group component of this project optional. I left the group project exhausted and didn’t feel I had learned nearly as much as I should have.
Pros
- Good survey of deep learning topics
- Great professor who is actively engaged in the class
- Gaining familiarity with PyTorch
- Learning and working through math of backprop / training
- Exposure to some literature in the field
Cons
- The group project (did I mention this yet? ;) ).
- Lectures delivered by Facebook engineers / the whole Facebook collaboration. I think Georgia Tech’s decision to partner with Facebook was a mistake. Hopefully they can replace this content in subsequent years.
- Some assignments (namely #4) need some polishing, but I think this will happen over time
Liked the topics covered. Time consuming. TAs not helpful
I was pleased with this class. It was definitely front-loaded with some very difficult assignments doing hand-crafted backpropagation. Good to understand, but not easy to implement as there are a lot of moving pieces, especially for the CNNs. It felt a little like overkill at times to be working on the backpropagation for CNN when there are so many other things we could be doing, like experimenting with more of the deep architectures, but I guess the class is more about understanding the basics of deep learning, not really playing with the really deep models ourselves.
The math for those early quizzes and assignments is quite intense, but by midway through the course there are really no math requirements (as far as work you do, understanding the math behind softmax attention etc. is still important).
There were some hiccups with the assignments where they needed to push out several versions as students ran into issues. I think they got enough feedback that future classes won’t have this kind of trouble. But to this point, I found that the discussions on Slack were invaluable for overcoming errors and learning from others on those harder assignments. The later assignments were (to me) much easier and I stopped going to Slack for advice.
The lectures were pretty good and covered a fair share of breadth and depth. They had plenty of errata that made reading some slides quite confusing, but I think they’ve been alerted of most of those issues. Sometimes the lectures also breeze over the math as if it should be quite clear but to me it wasn’t always that way, so be prepared to do some outside studying if you want to really grasp the underlying math.
Overall though, I really enjoyed it and I learned a lot of things that I think will be very impactful to me and my future career. I’m also much more confident when reading research papers on the topic. And I didn’t realize how much I would like PyTorch. Moral of the story is that you should take the class, but definitely hone in your linear algebra and partial derivatives beforehand so you don’t have the initial desperate drowning experience at the beginning.
A note on effort: I probably spent about 20 hours a week for the first 6 weeks, but after that, I was able to get things done in about <= 10 hours a week. I would not recommend pairing with another class unless you have little to no other commitments during the first several weeks. Depending on your project and team you will need more or less time at the end of the semester. My team was great and our project was not Facebook level hard, so the last bit was not bad at all. A good way to finish the program!
My favourite course alongside RL. I’d say the course is a fair amount of work but a bit too easy. Assignments are front loaded, and the project adds a hefty backload. Feels like there’s a fire hydrant of material to read and study but one can easily do well without it if you so chose. I tried to absorb as much as possible so even it wasnt too difficult it still took a significant amount of time to read and digest. Course staff and Prof are awesome! Interactive and lots of OH.
Took a nose dive reluctantly into the first class of DL. I was skeptical and worried about being the first in the line of battle. But boy I am glad I did. Let me first start by introducing the work load.
11 proctored quizzes spaced every week (there are some off weeks), 15%
4 Coding assignments, 60%
4 Graded discussion, 5%
Final Project (including proposal), 20%
Assignment1: Implement ANN from scratch. Here you implement ANN or MLP. You implement the model and the optimizer.
Assignment2: Build CNN from scratch. You also implement CNN on pytorch and test your model on CIFAR-10 dataset.
Assignment3: CNN model visualization. You will implement Saliency Maps, Gradcam, Fooling Images and Class Visualization.
Assignment4: Implement RNNs and LSTM, seq2seq model and Transformers.
Assignment 1 and 2 were hard. You need to implement ANN and CNN from scratch. You do a forward pass and a back prop. You need to figure out the math for the backward pass by writing the partial derivate on paper and use chain rule as the gradient flows backward. While the code itself is not hard and only few lines per layer, figuring out the math and getting an intuition is hard.
Assignment 3 was all about visualization of CNN. You have to implement algorithms that will visualize parts of an image that the CNN uses (activations) in its decision making.
Assignment 4 was all about RNNs. RNNs are hard and the most difficult of all Neural network algorithms to understand and get an intuition on. You won’t implement RNNs from scratch thankfully but use pytorch to implement LSTMs, seq2seq and Transformer models.
The weekly quizzes forces you to periodically watch the lectures as they are released at least twice. While the quiz itself was not hard, preparing for the quiz was stressful.
There are 4 graded discussion where you have to read a paper they ask. You pick one of the 2 papers and post a short review on it and also answer 2 questions on it. You then post your response on Canvas. You then have to read at least 2 response from other students and comment on their post. This is not a peer review but the GD is designed to exchange thoughts and ideas with other students. While not very hard, it is time consuming. You need to spend at least 4-5 hours on this task.
Finally there is a group project. For this term you were not allowed to work solo and groups were mandatory. You can choose your project or choose from one of the Facebook project. The Facebook projects are all real life and hard. The data itself will run into 100s of GBs and a GPU is a must for most of them. So choose them with caution. Using online cloud resource is not very practical. I would recommend that at least one member of the team has a 1080 ti or better GPU available to run the code locally.
Overall this class gets the job done in explaining practical ML. While I won’t call this class very hard, it is stressful for sure. Prof Kira and his TAs did a fantastic job. The first class had only 150 students and were all OMSCS veterans and very motivated. The drop rate was only 10% for this term. The TAs were on campus students who took this class while on campus. Overall the quality of students in this class was very high due to limited class strength. It felt like an on campus class unlike other classes I have taken in OMSCS. But I don’t expect this experience to continue with increased class size. The first class was super competitive and the class mean was very high. Thankfully the class is not curved and 90 or above constitute an A. It is not difficult to get an A if you do well on the project and the assignments. This class is a must take for ML specialization students.
This is the type of course I was hoping all courses in OMSCS would be like when I enrolled. Great combination of theory and practice. Paper discussions keep you updated on some of the many new developments that are always occurring. The assignments are solid and the open-ended nature of the final project is awesome. Professor Kira is great and is really involved in the class. One major caveat with my review is that there were only 150 seats in this first iteration, so hopefully the quality can be maintained (looks like next term it’s 500 + 50 for OMSA). On the other hand, there were some kinks with the assignments and quizzes to work out this first term which shouldn’t be as prevalent in the future. Since DL is an area so many students are really excited about, there’s a lot of good discussion and helpful students on piazza and in slack. Awesome course and really glad I was able to squeeze it in in the last term!
This class is a must take if you’re interested in ML or doing the ML specialization. The assignments in DL have you implement a bunch of fundamental things from scratch including neural networks (forward and backward pass), convolutional neural networks (forward and backward pass). But it’s not all from scratch; after implementing a lot of fundamentals from scratch, the assignments have you use PyTorch to explore more advanced topics. Implementing things from scratch also help you better understand how all of the components of PyTorch fit together like lego blocks and how you can modify different parts to achieve different goals. One interesting application was in an assignment where we had to create custom loss functions and then use PyTorch’s built in backprop to optimize over images to do style-transfer. I hadn’t been exposed to this side of DL–using backprop to optimize over your input, so it was quite illuminating to me. We also explored how we can optimize over image to visualize what CNNs are learning, and generate adversarial examples that can expose the blind spots of a neural network. In the last assignment, we used PyTorch to implement recurrent neural networks, LSTMs, and transformers to perform neural machine translation. We couldn’t use the high level APIs, but instead had to implement custom nn.Modules that defined the forward passes based on the equations. Luckily, in this assignment we could leverage the back-prop utility baked into PyTorch, so we didn’t need to implement the backward pass.
It’s also worth noting that parts of the class were developed by FB AI researchers. Some of the project prompts were defined by them (Motion prediction, Self-supervised MRI reconstruction, Hateful Memes Classification, Natural Language Translation Quality Estimation, something about Transformers/Adapters). During the last month, we actually had 9 Office Hours scheduled with the FB AI researchers in a wide array of topics (Data Wrangling, Transformers, Neural Machine Translation, Language Models, Scaling Deep Learning from Experiment to Production, PyTorch & Scalable Training, Fairness, Accountability and Transparency (FAT), World2Vec).
But wait, there’s more! There are weekly proctored quizzes. They were pretty annoying, but I think they’re annoying in a good way. Assuming you want to do well on the quiz, they force you to review the lectures perhaps a 2nd time and practice a few calculations on paper to make sure you really did understand what you saw. In many other classes, you’re not really assessed on your understanding of every detail in the lectures, so it’s fine to just breeze through them not really caring to make sure it all sticks. Overall, DL really increases your depth of understanding of various topics in ML. I’ve taken ML, and RL, but DL is the class that puts the ML in the ML specialization.
IMO this course is the only one that is comparable to the graduate-level, on-campus version of course offered in Top-level colleges such as Standford CNN course (cs231n). I felt I’m fortunate to take this course as the last course before graduation.
As a person who watched all the lectures and did assignments of cs231n before, I felt this course is similar to cs231n in terms of high-quality of lectures, the difficulty of assignments, and open-ended projects including FB’s ideas which are not trivial but promising. I was also surprised about active engagements from both Prof.Kira and TAs.
Here are a few tips:
Assignments: you need to clearly understand basic DL algorithms such as forward/backward pass so that you can implement this in the lowest level using numpy. For assignments 3 and 4, you need to implement recent DL models using pytorch. I’d recommend to brush up your numpy skill and watch tutorials about pytorch unless you have experience with DL frameworks.
Quiz: quiz is provided weekly and most of the problems are either conceptual or simple calculations. The only thing to do well in the quiz is understanding lectures well, reviewing the contents thoroughly, and don’t abstract them. Hints from Prof.Kira were also helpful.
Project: you can propose any subject regarding DL or tackle one of FB’s ideas. You might need GPUs or additional cloud credits if you tackle FB projects as those might involve heavy computation. Team project would be recommended but you can do it alone if you have a thorough idea about your topic. Personally, I took one of FB projects as I have quite decent GPUs and those seem promising and potentially publishable ideas. If you want to shoot for publication, I’d recommend tackling FB ideas as most of the projects are well-organized and straight-forward to do though those aren’t trivial.
Overall a great and long overdue DL course. My one complaint is that it took GT so long. Should be a course you take right at the beginning after taking AI so you start your OMSCS journey in ML with a good foundation of what you want to focus on more.
I’m glad we finally got a rigorous academic course on DL that really tries to delve in to the mathematical nuts and bolts of the algorithms even if it ignores most theoretical aspects which imo is fine given the amount of material to cover. I have said this many times before but DL is not something you just learn through cheap MOOCs and this class proves why.
This class is not difficult in the unreasonable brutish kind of way. It is however a time consuming course in the sense that you need to code everything and assignment points are all or nothing pretty much and you never have a week off because there is something due every week.
Overall though it ranked in the middle for me time wise averaging 15 hours/week and I took this with GA which I averaged 20 hrs/week.
4 assignments with only one (A2) I would classify as Hard. A1 took me 15 hrs and is a fairly gentle introduction to neural networks, A2 30 hours was the hardest and involves some beastly level of coding and derivations, A3 the easiest to implement 20 hrs, and A4 took 20 hrs but has a lot of material to cover.
The quizzes are mostly jokes with a few math ones that have no business being quizzes. But even if you fail those it makes little impact on your grade.
The project can be as easy or hard as you want it to be. Group format makes it a pain. I am yet to see the grade but I would advise you to not do the FB projects if you are very new to DL as those are open research topics and absolutely not meant to be trivial
Pros:
1) Prof Kira has some of the best lectures and some of the best engagement I have seen in any class. Some of his lectures on CNNs I truly have not found anywhere which cover the material with the same kind of rigor. Thank you Professor for finishing my experience on a high not. Coming into OMSCS i thought this is what the program would be like. I am glad i got at least this class (+AI4R and AI) where it felt like I was in a class and not some cheap autopilot MOOC.
2) Student quality was some of the best and the small size makes for excellent discussions. Unfortunately this may decline next semester with a bigger cohort that is not biased towards curious and motivated OMSCS veterans.
3) Project is a good way to get exposure to a problem you want to learn about. I spent about 100 hours only because I enjoyed my problem. You definitely don’t have to. 40-50 hours sounds more reasonable
Cons:
1) Facebook lectures and involvement in general is actually very bad. Some of the FB lecturers (SWES and PMS) are probably less knowledgeable about DL than the top 10% of the student body taking the class. Their so called projects are open source data competitions which you don’t need to take this class for. It is disappointing that FB wanted to be involved in this course and yet they won’t provide any of their data or cloud GPU resources for us to work with. They scheduled mostly useless office hours (during work hours too) and their project involvement included maybe 1 or 2 hours talking to someone who didn’t really care much about non PhD students. An unethical place like FB should also not be teaching an AI Ethics module. I found it very tone deaf and marginalizing for GT to let FB speak on this sensitive topic.
2) Weekly quizzes are too many and can be useless T/F type questions. They need to be condensed to 5 or 6 biweekly and the math questions are not suited for a quiz format. Either put them as part of take home problem sets or provide examples before the quiz.
3) Group project format does not work for DL research work environments in real life. DL is not SWE work. It is not PM work. DL requires independent research and people collaborate only when their research interests align and they have something of value to add. Best solution would be to what CV or BD4H does. Give a list of predetermined and vetted 6 projects and you get 3 weeks to implement 1 or 2 research papers from scratch on your own. There are also some teams I think that imploded because there are too many hyper competitive types in this class who want to prove how smart they are to everyone at the expense of actually writing an Introduction to DL level paper. At work people would be fired immediately if there was a mismatch of expectations and skills at most places something you can’t do in a class format.
Overall, I thought this was an excellent course. One of my favorites in the entire program. This was my 8th class. The class provided a great split of foundational knowledge/depth and higher level breadth and exposure to deep learning topics.
There were 4 programming assignments. The first 2 have you implementing neural networks from scratch. These were the most difficult assignments as they require you to have a good handle on linear algebra and the chain rule in calculus. The math wasn’t super hard, but keeping track of everything when you have hundreds of variables across multiple network layers got complicated. I hadn’t done calculus or linear algebra in ~5 years so I was scrambling to re-learn for the first month of class.
The last 2 are higher level assignments where you use PyTorch to implement different network architectures. I had no pytorch experience before this class and thought that the assignments did a good job of teaching you the library. You won’t suffer too much if you haven’t used it before. As others have mentioned, the quizzes were fairly difficult and you can spend a lot of time preparing for them if you want good grades on them. Luckily they weren’t worth too much of your final grade, so it doesn’t hurt you too much if you bomb a few of them.
The assignments all had several included unit tests that the TAs wrote along with an autograder. These helped immensely since it can be hard to tell if your code is doing the math correctly. As others have said, there were some assignments that the TAs were fixing problems with while students were actively working on the project, but that’s to be expected in a new course. The TAs and the professor were always very responsive on Piazza. This is one of the few classes I’ve taken where the professor is actively engaging with the students, even on the Piazza posts that were just discussions rather than post about the lectures/assignments.
The class also included several “graded discussions” where you chose 1 paper out of a few that the professor selected, read the paper, answered some questions about it and provided your opinions, then responded to a few other students posts. These were always very enjoyable. Many times, we were reading papers that had been published within the last year. It was really cool to be able to see what researchers were actively working on and be able to understand what they’re talking about in the papers. In other classes were we read academic papers, they were always super old papers, or the material felt very inaccessible to someone who wasn’t already a PhD in the field. These deep learning papers always felt somewhat accessible, even if I wouldn’t be capable of re-implementing them.
The final project had you in groups of 3/4 students working on a project of your choice. As I write this review, the project is due tonight, so I don’t yet have a grade for it. But the TAs and professor have said the grade is based more on what you learned and could articulate in your final report than how good your results are. As with all group projects, if you are proactive and find your own group members before the project starts, you will probably have a good time. If you get auto-assigned to a group, you will probably have a bad time.
Overall, this is one of the best courses in the program that I would recommend anyone take, even if you’re not doing the ML specialty (I am not doing ML). The professor, TAs, and content are all top notch.
Deep learning CS7643 single-handedly changed how I felt about machine learning at OMSCS.
I have taken numerous other machine learning courses and always felt machine learning CS7641 never did the subject justice. Taken in isolation, 7641 machine learning is at best an okay course and when compared to how other programs and departments teach machine learning, CS7641 really pales in comparison. By contrast, deep learning CS7643 is really one of the best courses whether you evaluate it in isolation or you compare it to similar courses at other universities.
Simply put I think CS7643 is THE course to take in OMSCS whether you are in the ML track or not.
This course isn’t just a “run the model with an ML package” type of course. In the first half you will implement large parts of multi-layer perceptrons and CNNs by hand and including back propagation for these models. This really makes you learn in detail how these models work under the hood and is already far more detailed than how many other courses treat this part of the material.
The second half the course includes older topics LSTMs and sequence to sequence models but also more recent topics like attention and transformers.
There is a proctored quiz roughly weekly but it’s not a big deal if you read over the slides especially the parts the instructors say are important.
Overall I did like this course, and the material covered is very good and in-depth. Without this course and the materials covered, I don’t feel like the Machine Learning specialization would have been complete, by far. The material covered is essential for understanding deep learning techniques. By that criteria, this course is a must-take.
Additional considerations:
- The professor and TAs are engaged and quick to interact via Piazza and even in the Slack channel. Definitely appreciate the engagement and help there.
- The assigned papers were cutting edge, and essential reading. The assigned book is OK, and a bit dry, but it’s fair game for the quizzes.
- The weekly quizzes got old fast, and they are not easy. Some nights you have to forgo working on a given assignment to get through preparing for the quizzes, and that caused some frustration. If you have limited time on nights and weekends, this needs to be managed.
- The assignments went through some churn fixing errors as can be expected for a first-run course, but eventually got sorted. If they go with the final versions of the project descriptions and codebase for future semesters, this shouldn’t be an issue.
- I miss (maybe just nostalgia at this point) the Udacity videos with their built-in quizzes. That format with bite-sized sections in the 5-8 minute mark would be better to break up the material and have details sink in. Sometimes felt like a grind to get though 25 minute videos (especially the Facebook ones) where the audio is not always crisp and there is a lot of information covered quickly.
- Working on the assignments brought back some flashbacks from AI, where you’re searching for some clue to break through an issue and the clue is buried deep in the assignment Piazza post, or even in the Slack (which had a lot of activity, not always pertinent).
- Got a great working knowledge of Pytorch and its peculiarities, something you only get by banging your head against the wall solving tensor size mismatch issues.
Overall, if you are here for ML, you can’t skip this course. Without this knowledge, you’d be shooting in the dark when working with deep neural network architectures.
One of the best course in omscs. Workload is high but the content is amazing and so latest. Learning pytorch and implementing latest research papers were so much fun. This was the only course till now which I never wished to end.
The toughest and most rewarding class in the program. You will have a very thorough understanding of how neural networks are designed from scratch after taking this class through rigorous programming exercises. The workload for the class surpassed ML. Do NOT pair this with another class unless you’re full-time. Huge shout out to the professor and TAs for being extremely active on Piazza and willing to make adjustments in this first semester as was deemed appropriate (shifting deadlines, updating assignments, correcting quiz errors).
Excellent course. Good assignments and lectures. A must take for students interested in ML.
Really a great class. Lots of math at the beginning, need to calculate chain rule by hand for backprop, convolutions by hand ..etc. Was able to touch on most of the major DL topics, GANs, VAEs, CV, NLP. Would 100% recommend.
Would be hard to pair with another course.