Lex Fridman Interview - Andrew Ng

"Andrew Ng is one of the most impactful educators, researchers,  innovators, and leaders in artificial intelligence and technology space in general. He co-founded Coursera and Google Brain, launched deeplearning.ai, Landing.ai, and the AI fund, and was the Chief Scientist at Baidu. As a Stanford professor, and with Coursera and deeplearning.ai, he has helped educate and inspire millions of students  including me." ~ from Lex Fridman's website.

Author's Note

There wasn't much technical discussion about artificial intelligence, except for certain sections that talk about self-supervised learning and other less known techniques that might be gaining more traction these days, like Sparse Coding and Slow Feature Analysis. The interview is more of Andrew Ng's philosophy in teaching and his passion for Machine Learning, especially in Reinforcement Learning which is the topic of his Ph.D. thesis.

Many, including me, have been inspired by his CS229 lectures on Machine Learning and took up this data science and artificial intelligence route with focus. Here is the CS229 YouTube playlist. If you have not watched it, I strongly recommend you sit through it. Have fun! :)

Note 1: How Did Andrew Ng Start?

At a very young age, Andrew Ng was given books on Expert Systems and Neural Network to read. And he picks up programming during that time too. During his attachments, he was asked to do a lot of repetitive work like photocopying and shredding so he was thinking given that he knows programming, it will be great to automate such processes. From here we can see that Andrew Ng actually see Artificial Intelligence as a tool for automation. AI is a better and smarter tool to turn more manual processes previously, into automated processes.

Note 2: Programming

Andrew Ng sees programming as a tool for interacting with computers, to get them to do the things humans wanted. With the advent of data science and many more companies are collecting lots of data these days, the demand for data science and analysis will grow. This increasing demand can serve as a motivator to get more people on board into programming.

Note 3: Unsupervised Learning

The next step for Deep Learning research is unsupervised learning. Andrew Ng mentioned that the direction should have been set earlier on it, and believed that unsupervised learning may become the next breakthrough for Artificial Intelligence. Some of the techniques mentioned are Self-supervised Learning, Sparse Coding, and Slow Feature Analysis. They represent different techniques of Representation Learning.

The author vaguely remembers learning about Sparse Coding and went to do a YouTube search on it. It is actually a section in an online Neural Network course by Hugo Larochelle, a prominent researcher in Google Brain. Here is his course on YouTube.

Note 4: How to Study Deep Learning

To get started on Deep Learning, start with deeplearning.ai specialization on Coursera. The prerequisite is a basic programming background and preferably basic knowledge in Calculus and Linear Algebra.

To be in the Deep Learning field, you need to read a lot. Start with creating a habit to learn. Set aside time in your schedule to learn. For Andrew Ng, he always sets aside time on Saturday and Sunday to research and read research papers. Consistency and regularity is the key. To help with the absorption of new knowledge, Andrew Ng recommends that the readers take handwritten notes because it forces the brain to distill to the learning point.

Start working on projects as well, it need not be big but more importantly, a passion project. While working on the project, pick up more learning along the way by reading blog posts and research papers.

If you are looking for career advice by Andrew Ng, check out this YouTube video, mentioned in the interview. :)

Note 5: How to Build a Successful AI Startup

Andrew Ng mentioned that one of the common failures for startups in Silicon Valley is building products that no one wants. There is a strong need to be customer-focused and -oriented, creating features that customers like to have. This is where data science can come in, doing A/B testing, etc.

Andrew Ng pointed out that entrepreneurship is a lonely journey and lots of important decisions need to be made. These decisions may need to be made with or without additional knowledge and information, to avoid pitfalls and manage risk, it is best to walk the journey with a group or community.

Note 6: Real-World AI

Being able to train a good machine learning model that performs well on the test set is definitely the first step but not the only step.

There are issues to consider, such as scalability, process design (how to put the machine learning model into the process and re-designing the other stages of the process around it), and also circumstances may change, as in the model did perform well on the test set but once it moves to business implementation, the data presented might have changed. For instance, perhaps a computer vision model was trained and doing well for the test set, but perhaps there is no change in the lighting intensity of the factory floor which may invalidate the deployed model.

Deploying the model, making it robust in production will require some software engineering work.

If your company is looking to build and deploy AI capabilities, have a look at Andrew Ng's book and if you need a consultant, consider having a chat with me. Here is the book "AI Transformation Playbook". :)

Note 7: Artificial General Intelligence

In this section, Andrew Ng mentioned the trolley problem and alignment problem need to be considered but should not take the attention away from the fact that Artificial Intelligence, as it gets better, has a winner-takes-all dynamic and may concentrate wealth and exacerbate wealth inequality. So the priority right now is how to ensure there is a concentration of power. Another urgent consideration is bias and ethical usage of machine learning models, not creating nefarious usage of AI.

If you will like to discuss the topic on AI, feel free to Tweet me or add me on LinkedIn.  To keep up to date with my learning and sharing, consider subscribing  to my newsletter below. Each subscription is a vote of confidence in my work.