My resolution for 2018 was to get into deep learning. I had stumbled upon a website called fast.ai in October 2017 after reading an article from the New York Times describing the shortage of people capable of training a deep learning model. It sounds a bit clichéd to say it changed my life, but I could hardly have imagined that, one year later, I would be helping prepare the next version of this course from behind the scenes. So in this article, I’ll tell you a little bit about my personal journey into Deep Learning, and I will share some advice which I feel could have been useful to me six months ago.
Who am I and where do I come from?
My background is mostly in Math. I have a Master’s degree from a French University; I started a PhD but stopped after six months because I found it too depressing, and went on to teach undergrads for seven years in Paris. I’m a self-taught coder, my father having had the good idea to put an introduction to ‘Basic’ in my hands when I was 13.
My husband got relocated to New York City, so I moved there three and a half years ago, and became half stay-at-home dad, half textbook writer for a publisher in France. When I first looked at fast.ai, I was curious about what the hype around Artificial Intelligence was all about, and I wanted to know if I could understand what seemed to only be accessible to a few geniuses.
I have to confess that I almost didn’t start the course; the claim it could explain Deep Learning to anyone with just one year of experience of coding and high school math sounded very suspicious to me, and I was wondering if it wasn’t all bogus (spoiler alert: it’s not). I did decide to go with it though; I had finished my latest books and finding seven hours a week to work the course while my little boys napped didn’t seem too much.
Although I started the first version of the MOOC with a clear advantage in math, I struggled a lot with what I call the geeky stuff. I’m a Windows user and I had never launched a terminal before. The setup took me the better part of a week before I was finally able to train my own dogs and cats classifier. It felt like some form of torture every time I had to run some commands in a terminal (that still bit hasn’t changed that much!)
If you’re new to the field and struggling with a part (or all) of it, remember no one had it easy. There’s always something you won’t know and that will be a challenge, but you will overcome it if you persevere. And with time, it will become easier, at least a little bit… I still need help with half my bash commands, and broke the docs and course website twice during the first lesson. Fortunately, everyone was too busy watching Jeremy to notice.
Do you need advanced math to do deep learning?
The short answer is no. The long answer is no, and anyone telling you the opposite just wants to scare you. You might need advanced math in some areas of theoretical research in Deep Learning, but there’s room for everyone at the table.
To be able to train a model in practice, you only need three things: have a sense of what a derivative is, know log and exp for the error functions, and know what a matrix product is. And you can learn all of this in a very short amount of time, with multiple resources online. In the course, Jeremy recommends Khan Academy for learning about derivatives, log, and exp, and 3 Blue 1 Brown for learning about matrix products.
In my opinion, the one mathy (it’s a bit at the intersection of math and code) thing you’ll really need to master (or at least get as comfortable with as you can) is broadcasting.
Make your own boot camp, if you’re serious about it
After I finished the first part of the course, it was clear to me that I wanted to work in this field (hence the good resolution). I contemplated various boot camps that were promising to turn me into a Data Scientist in exchange for a substantial tuition fee. I found enough testimonials online to scare me a bit from it, so fortunately I quickly gave up on that idea.
There are enough free (or cheap) resources online to teach you all you need, so as long as you have the self-discipline, you can make your own boot camp. The best of all being the courses from fast.ai of course (but I’m a bit biased since I work there now ;) ).
I thought I’d never be selected to join the International Fellowship for the second version of the second part of the course so I was a tad unprepared when the acceptance email came in. I booked a coworking space to distance myself from the craziness of a baby and a toddler at home, hired an army of sitters in a rush until we found a nanny, then worked from 9 to 5 each day, plus the evenings, on the course materials. I thought I’d follow other MOOCs, but with all the challenges Jeremy left on the forum, and the vibrant community there, I never got the time to look elsewhere.
Even though the course is intended for people to spend seven hours a week on homework, there is definitely enough to keep you busy for far longer, especially in the second part. If you’re serious about switching carreers to Deep Learning, you should spend the seven weeks working your ass off on this course. And if you can afford it money-wise/family-wise, fly yourself to San Francisco to attend in person and go every day to the study group at USF. If you can’t, find other people in your city following the course (or start your own group). In any case, be active on the forum, not just to ask questions when you have a bug, but also to help other people with their code.
Show what you can do
I’m shy and I hate networking. Those who have met me in person know I can’t chitchat for the love of God. Fortunately, there are plenty of ways you can sell yourself to potential employers behind the safety of your computer. Here are a few things that can help:
- Make your own project to show what you learned. Be sure to completely polish one project before moving to another one. In my case, it was reproducing superconvergence from Leslie Smith’s paper then the Deep painterly harmonization paper.
- Write a blog to explain what you learned. It doesn’t have to be complex new research articles, start with the basics of a model even if you think there are thousands of articles like this already. You’ll learn a lot just by trying to explain things you thought you understood.
- Contribute to a Deep Learning related open source project (like the fastai library).
- Get into Kaggle competitions (still on my to-do list, maybe it’s going to be my 2019 resolution ;) ).
- Get a Twitter account to tell people about all of the above.
I was amazed and surprised to get several job offers before the course even ended. Then, Jeremy mentioned he was going to rebuild the library entirely and I offered to help. One thing led to another one and he managed to get a sponsorship from AWS for me to be a Research Scientist at fast.ai.
Behind the mirror
In my opinion, there are always three stages in learning. First you understand something abstractly, then you can explain it, and eventually you manage to actually do it. This is why it’s so important to see if you can redo by yourself the code you see in the courses.
As far as Deep Learning is concerned, following the course was the first stage for me; writing blog posts, notebooks or answering questions on the forum was the second stage; re-building the library from scratch with Jeremy was the third one.
I have learned even more in those past few months than during the time when I was following the courses. Some of it in areas I had discarded a bit too quickly … and a lot of it by refactoring pieces of codes under Jeremy’s guidance until we got to the outcome you can see today. Building a fully integrated framework means you have to implement everything, so you need to master every part of the process.
All in all, this has been one of the years in which I’ve learned the most in my life. I’ll be forever thankful to Rachel and Jeremy for creating this amazing course, and I’m very proud to add my little stone to it.