I joined Georgia Tech Master of Science in Computer Science at the end of 2017 with the idea to expand my knowledge in an area I am interested: machine learning. At the time I am writing this blog entry, I have completed four semesters with a grade of “A”, and six more remain to complete the program. At the dawn of the Summer 2019 semester, I decided to drop-out the course I had started two weeks earlier named Reinforcement Learning. The major reason is the arrival of my second children. Nonetheless, I wanted to give some insight into the online program for those who might be interested in. Complimentary to my opinion, you can find online the website OMSCentral where students write reviews on each class. I decide to write non-anonymously my genuine unsatisfaction with the program.
Your Past Drive your Present
My journey started smoothly with my first two semesters with database systems concepts and design and network security classes. I was familiar with most of the topics and took these two classes to refresh my knowledge, to get accustomed to school after 10 years of hiatus and because Georgia Tech requires to have in the first year two semesters with a grade of “B” or higher.
Why is your past driving your experience at Georgia Tech? Let’s start with my context. I have been working with databases for more than 15 years and also working with the web for 20 years. While a lot of students were expressing their difficulties with some assignments that needed database or web knowledge to accomplish the primary goal, I was fine. However, in hindsight, it should have rung a bell. To be able to spend under 15 hours per week, you need to understand at least half of the topics or master assignments technologies before debuting the semester. For example, in the network security class, one assignment was using Wireshark, the next one a MySql/PHP database/website and another one a Chrome extension in less than two months. I am not even talking about the nitty gritty details of having to know different languages (Java, Python, etc.), libraries (Pandas, Numpy, Mathplotlib) and environment (virtual machine, Ubuntu). In the database class, there was a considerable amount of the final assignment directed toward building a PHP website which is not needed to evaluate database knowledge. It was not a predicament for me, but you can foresee that it could be a burden to add on top of a lot of new theories. The factor of serendipitous knowledge to have gathered play a significant role in the grade. However, this is not the worse. The worse is that satellite knowledge plays a vital role in your success. Even fast learners will feel strain because while learning the main subject, the numerous homework, assignments, and projects on top of the exams appear as obstacles instead of learning vehicle. The rapid pace impeded a full understanding. The numerous tangential topics, not well weaved together, constitute in mixed knowledge that looks like a bag of feeble crumbs — ready to be blown away on the next gust of subject. Riding on a slight understanding and moving on and forget what you barely explored to try to catch the next train is the modus operandi.
Instructors Do Not Teach
My third class was machine learning for trading. I learned Python on top of an overview of machine learning technics. I overall liked the class but the head instructor was mostly absent with acerbic interactions. A trend that will get worse with machine learning and reinforcement learning class were instructors are answering student in a bully way. There is absolutely no way that the words used by the instructors would be accepted in any workplace I have been working. When students ask questions, you can bet to read a scathing response. An example of a reply: “You know it in your heart” is a classic of one of the instructors instead of answering. The responses disincentivize people who misconceive subjects and would like further explanation. The public humiliation of receiving a stark reply stem, in some classes, the propagation of anonymous question (which is possible with the system) or simply in my case demotivate to ask a question.
The lessons recorded could be condensed by half or with better more thorough examples. Most of the lessons are full of pleasantries which sway the main topic out of attention and also increase anxiety because you want to understand but you keep having an instructor taking the subject lightly. In some class, it’s a recording from an actual class at Georgia Tech. The recording is made from a laptop. You can imagine the sound and video quality.
Finally, the instructors refer regularly to whitepapers. Not as a complementary explanation but as a way to avoid providing a thorough explanation in the lesson. I acknowledge that delivering clear explanations require a considerable amount of preparation but I entail, by joining a top 8 University in the USA, to not be deflected constantly toward whitepapers. Swimming in new concepts and having someone referring that the remaining of the information is in a whitepaper (and insinuating that students already have read them), is a lack of professionalism and it makes student losing track of the lesson. I found not educational to stop the lesson and to jump into these scholar papers, which takes hours to read instead of taking a couple more minutes to explain the essential parts.
Interaction on the Student Forum
This is a good segway into the type of questions. Most student will ask questions in the school’s student forum called “Piazza.” Most questions are on the assignments and not on the lecture or reading. The focus is on where to get points, not to dive into the topics. Because my goal is to learn and not to get points, I asked several questions in all my classes. Most of the time, students were answering me. The instructors and TA (teaching assistant) were absent or focusing on the assignments. Many TA couldn’t respond properly, either. An example, I took the time to screen grab a portion of the lecture asking why the instructor was mentioning an increment of +1 in a formula and after more than four messages the TA was saying that I could use anything. But, when I challenged him about why the lecture was explicitly using +1 or if I can use any constant, the TA was answering me that it would be better to have decay. I still couldn’t get an answer about the +1. I understand that the ratio of TA/students is enormous, and they are overwhelmed to grade assignments, but I was expecting more. I’ll skip some frustration with the delay on getting grade which does not help to give you a pulse if you are working on the expected way. In most courses, you will get your grade for the first assignment once you have submitted your second, hence could not really apply the feedback from the former on the latter.
As briefly mentioned in this section, what astonished me the most is that questions are all related to assignment not the lecture or the books or papers to read. I interpret (and also read in some Slack discussions) that people skipped the lecture because of the reasons discussed in this blog article: people are aiming to get the grade and move on. I am on the side that I want to understand and move on.
Grading and Subjectivity
The grading is also subjective and questionable. During my machine learning semester, I jumped throughout the session between 55% and 96% in my grade giving the same amount of thoroughness. The change was different TA was grading me. Also, even if I was answering every requirement there was always a place to subjectivity about that I should have discussed furthermore on different tangential topics. I noticed similar grading fuzziness in other classes where the report has an unclear requirement. Asking clarification on the assignment does not result in iterating on the assignment requirements but to many scattered posts in the student forum which make it hard to follow. I also found that it penalizes student that was not last minute. Many time, a clarification was changing the meaning of something and I had to re-write. Furthermore, asking for re-grade is heavily skewed to make you fail even more. In two of the four class, they had a blank statement that a request to re-grade would lead to a penalty if they deem that it is not justified.
Time Invested
I finally withdraw my fifth semester because of the arrival of my new child. On the first day of the semester, my son had three days. I naively thought that I would be able to handle one class while being in parental leave. I was wrong.
So far, I spent roughly 14, 14, 18, 23 hours per week on the network security, database systems concepts and design, machine learning for trading and machine learning. However, with this fifth semester, with reinforcement learning, the first two weeks cost me 28 hours each. The difference was that after these 56 hours invested was that I was mesmerized if the assignment was really for the class I had studied or not — a complete disconnect other than few keywords.
The time invested varies depending on how you understand the topic has mentioned earlier but still there is something wrong. It is supposed to be possible to mary work and school while you have to sacrifice a lot. In the two first weeks, I had to do two homework and starting to work on the first of three projects but also to listen to several lectures of the class and the complimentary David Silver lecture. These lectures are highly recommended but because the explanation is more detailed. However, it is from another school. I was projecting a considerable investment in time because that in reinforcement learning you cannot copy the code from other students. You read it properly. The machine learning class is basically copying from previous student assignment and write a report. It is explicitly mentioned to do so in the assignments. Naively started by not doing so, and spent way too much time.
Nonetheless, in reinforcement learning, even with all that time spent studying the theory and linking all possible source of information, the reality of misunderstanding stroke: the homework and projects were so disconnected that I had little idea what to code in Python. The additional stress of not having any base to start or any clues was un-educational. I agree with struggling to learn and working hard but not to ferment in limbo waiting to salvage pieces of information from a flood of questions asked by different and lost students. What was worth the majority of the points with the growing misunderstanding of the topics stirred with the lack of sleep led me to take a break. While the time invested every week varies greatly depending on if you read and watch all the material there is also group homework that can play for a big factor to increase the charge.
Team Assignments
The value of teamwork is often undervalued, however, is inadequate for a remote master program. The outline is that people are spread across different timezone. The variance between each individual in term of skills can be considerable. The motivation of each member between wanting to perform or just getting a fast degree can add additional load on your shoulder. And, the classic burden of teamwork where half of the team is doing everything and the other half just wait is influencing quite a lot the time you will invest every week.
The first two courses I took had team works and I got lucky to have people with more or less 3 hours different timezone and also found people who were not last minute. In one of my team, one guy was almost 12 hours difference influencing my choice of also communicating with him before my daily work. Nonetheless, consolidating family, work and school is already a challenge. Teamwork is adding an additional layer of difficulty. It makes the idea of flexibility out of the equation if you are diligent in your assignments.
Flexibility
One aspect of online school is that you have control of the time you want to study. But, the reality is far from being flexible. All the course I had, so far, was giving so many homework (up to eight), with additionally many projects (up to three), with a mid-term and final exam. And, most works require prior reading; hence, when you schedule all that needs to be done you are entirely on a rigid schedule. The critical path of success is on a thin line where shifting a couple of days is possible but increase the backlog of work later.
Also, most assignments are not in a final state when delivered. I had several graded works being in a draft for several days, and then being amended during the short allowed time to perform the work. The fluctuation of requirement disrupt people who like to start ahead. I am someone who likes to start and finish as soon as possible and got bitten often by assignments that were changing throughout the weeks hence favoring people working late. The result is that you have barely any room of flexibility. Once again, if you add that you need to schedule with many teammates, then you are simply in a corset with a low room to breath.
Quality of Content
Concerning the quality of the content, it varies but mostly poor. You can see it by yourself because most are available on Youtube. The problem is, depending on the instructor, you will get a lot of low production quality classes. Audio is often poor, examples borrowed from books, calculus are skipping step making it hard to follow, many jokes that make the week lecture up to 3 hours (the recording varies between 45 minutes to 1 hour 30 minutes but if you take notes it will go up to 3 hours). My main complains is that in 2019 we have so many good quality content for free on Youtube that Georgia Tech must level up their production quality with the aim to teach instead of just covering a topic. For example, covering a critical algorithm (I already forgot the name) in less than 5 minutes video is a blasphemy. Also, there is a lot of variation on the notation used compared to the whitepapers which might be accurate but very confusing for a neophyte to decipher whitepapers.
The lecture were made not with student trying to learn in mind. For example, when the instructor is writing formula, often, they will in post-production cut the time when the instructor is writing — it fast forward into the end of the writing. It forces the student to pause the video when taking note. Thus, the lecture are most of the time taking way more time because of all the time we need to pause. In real classes, student can write at the same time of the teacher, or take time to digest the message before moving on. Not that time, this responsibility is on the student. Furthermore, in real class, you can raise your hand to ask a question, get unblocked and being able to continue to follow. With recorded lecture you can only continue to watch and ask later in the student forum, or search online. The disconnect does not seem harsh for someone who understand but for someone not getting a specific key point in the theory can cause to not master well the remaining of the lecture. I am autodidact and found myself in few situations not totally sure about what the instructor meant. If the instructor is using personal jargon then you are on yourself to find out what it really mean because Internet will not be helpful. Relevant to the format of the lecture, because they were recorded many years ago without alteration, unclear passage remains unclear. In traditional lecture, the teacher can learn from the questions of the student and clarify next time this one give the course. This natural iterative improvement is not present in Georgia Tech because they do not actively work on the content.
In two classes (network security and machine learning for trading) I had to buy a completely useless book. The former one was a hacking guide that had literally 1 useful page. The latter, a book in which the instructor was the author — 60$ for a thin hundred pages. The concent could fit under 30 pages in letter format. It was a summary of the lecture. In my perspective, it reflects the quality in the sense that there was a lack of diligence on how useful was the recommendation. But, also an absence of a neutral party to asses how self-promoting material that compensates the instructor directly.
In each class, the enormous quantity of different topics can be reduced by half in most of the time. For example, in machine learning for trading, there is an entire class on “options trading,” but it is shallow without any real use in term of machine learning. In this same class, there is a whole week on the 2008 crashes. It is not related to machine learning. Hence, a couple of week of finance could be trim to go deeper into the machine learning part. Similar in the two classes, machine learning, and reinforcement learning. In the machine learning curriculum, we touch reinforcement learning. If you take reinforcement learning, two of the lectures are copy-pasted. Instead, in the machine learning class, a better explanation of non-reinforcement learning should be approached.
Finally, in every class I took, students who completed the course suggested to watch David Silver or Andrew Ng. But why? Because Georgia Tech courses are built in a very bumpy way. The lessons start slow, with (too many) jokes and then get shallow on many topics and then dive very deep in calculus, so much that you have a hard time to grasp concepts because of all the shortcut taken and references to whitepapers to finally get back very into a totally different aspect of the main class. For example, you can have 2 hours on game theories and then 1 hour on Markov.
Fearful Approach
Not every course, but some are driven by fear of expulsion of the program. For example, sharing student notes are forbidden. Even, sharing a few lines of code is forbidden while asking for guidance. I understand to not share a whole project or homework but there is a valuable learning value to have examples. The paucity of example provided by professors/instructors in the content lead to dry land of application. Fears drive students to stop asking a question or helping others. In the end, the only benefits are the TA who are grading the same assignments every semester because they do not have to change the requirements. Once again, students pay the price of not fully spread their wings to understand but are pushed to stay alone in their corner. That being said, side “Slack” discussion that vanish a few hours later are useful and seem to be one way to avoid the circumspect law that reign.
TA/Instructors Video Interactions
I was able to assist to a single online “live” video interaction with TA and instructor. For a program driven for “online” education, it is very awkward. The time for these “live” video-chat are supposed to be a time where you can ask questions and to have a discussion but it is more driven by students asking questions in the forum (Piazza). My experience is that you cannot rely on these because the time is mostly the wrong one for your life. For example, in one class, the time was while I was at my job and the other one in my commute back home. That being said, they are recorded, so you can listen to them later with the disadvantage that you cannot ask questions or clarification. In reality, I watched twice a recorded TA/instructors video because they are a very slow pace. Usually, the modus operandi is that the lead will introduce all the TA then read the Piazza questions and try to explain something. Most of TAs do not have a microphone outside there headphone or laptop hence the quality is low. For me, the worse is that every question launch discussion that digresses the main question. 10 questions could lead to 1 hour, and as mentioned earlier, time is scarce.
Am I alone?
In most of the classes, there is some side-discussion in Slack channel where I was able to catch the pulse of panic of most people not understanding what to do. In this last class, I asked the question to the few people that were already doing homework and projects in advance on how they were able to be so productive. Their answers are that they already work in the domain, hence understand most of the theory and only need to ace the work, or it was that they had another class that was similar and could relate better what was needed. Finally, the answer is that the “grading curve is generous” meaning that I could continue to walk head down and get a low score (as 40%) and be able to get a “B”. I was astonished by this last answer but I realized that it was true: most people can get an “A” or a “B” with a very poor understanding of the topics taught.
Browsing the OMSCentral or Reddit, and you can find several people aligning with the overview I am presenting in this blog article. The problem is that there is no direct was to provide impactful feedback. I would have flag many instructors and TA as inappropriate during the semesters, but there was only a last week feedback form which does not seem to do anything because a lot of complain about some classes lectures are still the same five years later of their inception. Even worse, one of the most inappropriate interactions from Georgia Tech I witnessed lead the professor to become a dean. It is clear that the quality of teaching is not a priority.
Why is Georgia Tech not Improving?
Georgia Tech hopefully knows, but move slowly which is paradoxical for a top University in technology. In machine learning, all the assignments have the mention to “copy” from previous students. A shortcut justified by letting students focus on the analysis but it is a sign that students lack the knowledge to complete the task. Instructors decide to employ different learning shortcut instead of guiding the student from the base to ground up. The issue is that the affinity between the classes is not well defined. A well-tailored program in machine learning should guide the student into several logical and ordered classes that go from the base to something complex. Instead, each class is distinct without any relation (and can be taken in any order which is confusing). Hence, each class is shallow and opt to different technic to compensate like allowing students to copy, or to have a big grading curve, or to repeat content from class to class.
Georgia Tech online system (student portal) to register to class is similar to the website you were visiting in early 2000. While it might not be a big deal, the scarcity of guidance on which class to take cause headache. Students must rely on an external source of information built by other students. Again, Georgia Tech is a top university and should have a system that guides users to thrive instead of delegating. I already mention that Georgia Tech delegates to whitepapers for depth in concept, and to other online videos for the poor quality of their own, they delegate to other books for the limited examples provided and now they delegate how to manage student curriculum to students. I’ll cut short in the tech-infrastructure pitfalls because this could be a post on its own (they also leaked student social security number and password). There the saying that you can judge a restaurant’s kitchen by own clean is the restaurant’s restroom. Unfortunately, you can also do the same for system and infrastructure to evaluate the quality of the teaching.
Solutions exist that Georgia Tech can put in place. A process of having someone to genuinely asses the quality of each course should be in place. Instead, I witness that the institution rather close their eyes and pat themselves on the back. For example, one of the most unsparing instructor in his way to behave with student got promoted to dean recently. He has a history of being a jerk. His contribution might be stellar in the research community does but is not a prerogative to teaching and acting poorly. Georgia Tech has a feedback system only available at the end of each semester. Nothing seems to get taken seriously. Instead, an on-going feedback mechanism should be in place without feering retaliation. It should be hosted by a third-party and acknowledged by the institution.
I understand that building a course involves a considerable amount of time. However, keeping heads down for many years (sometimes more than five years) with courses that barely involves (lecture are not re-recorded for example) foster the inadequacy of the system to evolve and to have the student as the priority.
What Now?
While I may sound negative about the Georgia Tech Master of Science in Computer Science it has something good: it really forces you to not give up and to really dig to understand. However, the school is not there to teach you anything. Georgia Tech throws on the wall as many topics they can. Then, they grade you on how you catch everything before it falls on the floor.
There is a huge opportunity for other online programs to annihilated Georgia Tech here: have a program that the staffs and materials stewards students to understand with a realistic life-balance approach. For example, the course machine learning, machine learning for trading and reinforcement learning have a lot of overlap that could be avoided in favor of a better and longer time on each topic. Also, one additional flaw of the course is the level of example that is often none or a single one taken from a whitepaper or book which is merely a requirement to understand. To alleviate these problems, clearer lecture with clear materials instead of having a list of +20 white papers would help. Creating a set of examples with step by step explanation where newcomer could consult and experts could skip is not a genius idea — it is what you get when you try to understand who consume your service.
Additionally, aiming for a maximum of 10 hours per week for a neophyte and testing that assumption would make the program available for an adult with a family and a daily job. But, quality in education requires iteration of the material. Re-recording, adjusting assignments, acknowledging feedbacks and acting on it, improving exams, etc
I patron a program that is not a walk-in-the-park, but there is a limit about how impractical you can be to make a life of people hard for no beneficial reason. Teaching is the art of making topics hard easier; not to try to make it looks harder than it is. Education is to be able to put yourself into your student shoes and to mold a better future by opening these doors that spark these “Ahhhh” moments. On my side, I believe I can succeed in the reinforcement learning class once my newborn gives me more than four hours of sleep per night. However, in the end, my goal is to “master” machine learning — not to play a game of aiming for the minimum (having my grade curved up) and hanging a degree on my wall. This is where I question Georgia Tech’s motives.
