# How to change careers and become a data scientist - one quant’s experience
Rachel Thomas
2017-03-01

*This post has been translated into [Chinese
here](https://www.jiqizhixin.com/articles/2016-10-10-3).*

*I sometimes receive emails asking for guidance related to data science,
which I answer here as a data science advice column. If you have a data
science related quandary, email me at
<a href="mailto:rachel@fast.ai" class="uri">mailto:rachel@fast.ai</a>.
Note that questions are edited for clarity and brevity. Other
installments of the data science advice column include:*

- [How to encourage your child’s interest in science and
  tech](../2017/05/07/parent/)
- [How to structure your data science and engineering
  teams](../2016/12/08/org-structure/)
- [Does machine learning as a service (MLaaS) work? Do you need a
  PhD?](../2017/03/17/not-commoditized-no-phd/)
- [Alternatives to a degree to prove
  yourself](../2017/04/06/alternatives/)
- [How should I focus my study time? How can I find a
  specialty?](../2017/03/23/focus-on-coding/)
- [Advice to a student interested in deep
  learning](../2016/12/19/favorite-posts/)

**Q:** This question is a composite of a few emails I’ve received from
people with some limited programming skills, living outside the Bay
Area, and interested in becoming data scientists, such as the following:

Q1. I’m a financial analyst at a major bank. I’m in the process of
pivoting to tech as a software engineer and I’m interested in machine
learning, which lead me to your post on [*The Diversity Crisis in
AI*](https://onthefuture.co/the-diversity-crisis-in-ai-and-fast-ai-diversity-fellowship-886dd1407363#.9racfdqun).
Do I need a masters or PhD to work in AI?

Q2. I am in the 6th year of my PhD in pure mathematics and am going to
graduate soon. I am really interested in data science and I want to know
if I want to get a job in this area, what can I do or what should I
prepare myself so that I can have the skill sets that the companies
need? I’m now reading books and trying to find some side projects that I
can do. Do you have any ideas where I can find these projects that will
interest employers?

Q3. I have a graduate STEM degree and have worked as both a researcher
and a teacher. I am currently in the midst of a career transition and
looking for industry roles that might need both analytical and
instructional skills from their employees. My knowledge is more on the
science side though rather than in software. The Internet can get to be
a pretty overwhelming place to pull information from without the actual
sharing. Do you have recommendations for programming courses and
workshops that are also friendly to a teacher’s budget? And what coding
languages or skills would you say would be most helpful to focus on
developing?

**A:** I think of myself as having a somewhat *non-traditional
background*. At first glance it may seem like I have a classic data
science education: I took 2 years of C++ in high school, minored in
computer science in college (with a math major), did a PhD related to
probability, and worked as a quant. However, my computer science
coursework was mostly theoretical, my math thesis was entirely
theoretical (no computations at all!), and over the years I used less
and less C/C++ and more and more MATLAB (why oh why did I do that to
myself?!? Somehow I found myself even writing web scrapers in MATLAB…)
My college education taught me how to prove if an algorithm was
NP-complete or Turing computable, but nothing about testing, version
control, web apps, or how the internet works. The company where I was a
quant primarily used proprietary software/languages that aren’t used in
the tech industry.

After 2 years working as a quant in energy trading, I realized that my
favorite parts of the job were programming and working with data. I was
frustrated with the bureacracy of working at a large company, and of
dealing with outdated and proprietary software tools. I wanted something
different, and decided to attend the data science conference
[Strata](https://conferences.oreilly.com/strata/strata-ca) in February
2012 to learn more about the world of Bay Area data science. I was
totally, absolutely, blown away. The huge enthusiasm for data, all the
tools I was most excited about (and several more I’d never heard of
before), the stories of people who had quit their previous lives in
academia or established companies to work on their passion at a startup…
it was so different and refreshing to what I was used to. After Strata,
I spent a few extra days in San Francisco, interviewing at startups and
having coffee with some distant acquaintances that I found had moved to
SF - everyone was very helpful and also apparently addicted to [Four
Barrel Coffee](https://www.fourbarrelcoffee.com/) (almost everyone I
spoke with suggested meeting there!)

I was star-struck, but actually switching into tech made me feel totally
out of it… For my first many conversations and interviews with people in
tech, I often felt like they were speaking another language. I grew up
in Texas, spent my 20s in Pennsylvania and North Carolina, and didn’t
know anyone who worked in tech. I’d never taken a statistics class and
just thought of probability as real analysis on a space of measure 1. I
didn’t know anything about how start-ups and tech companies worked. The
first time I interviewed at a start-up, one interviewer boasted about
how the company had briefly achieved profitability before embarking on
rapid expansion/hiring. “You mean this company isn’t profitable!?!?” I
responded in horror (Yes, I actually said that out loud, in a shocked
tone of voice). I now cringe in embarrassment at the memory. In another
interview, I was so confused by the concept of an
[“impression”](https://en.wikipedia.org/wiki/Impression_(online_media))
(when an internet ad is displayed) that it took me a while to even get
to the logic of the question.

I’ve been here five years now, and here’s some things I wish I’d known
when I was starting my career move. I’m aware that I’m white, a US
citizen, had a generous fellowship in grad school and no student debt,
and was single and childless at the time I decided to switch careers,
and someone without these privileges will face a much tougher path.
While my anecdotes should be taken with a grain of salt, I hope that
some of these suggestions turn out to be helpful to you:

## Becoming ready for a move to data science

1.  Most importantly: find ways to work whatever you want to learn into
    your current job. Find a project that involves more coding/data
    analysis and that would be helpful to your employer. Take any boring
    task you do and try to automate it. Even if the process of
    automation makes it take 5x as long (and even if you only do the
    task once!), you are learning by doing this.

2.  Analyze any data you have: from research for an upcoming purchase
    (i.e. deciding which microwave to buy), data from a personal fitness
    tracker, nutrition data from recipes you’re cooking, pre-schools
    you’re looking at for your child. Turn it into a mini-data analysis
    project and write it up in a blog post. E.g. if you are a graduate
    student, you could analyze grade data from the students you are
    teaching

3.  Learn the most important data science software tools: Python’s data
    science stack (pandas/numpy/scipy) is the \#1 most useful technology
    to learn (read [this
    book](http://shop.oreilly.com/product/0636920050896.do)!), followed
    closely by SQL. I would focus on getting very comfortable with
    Python and SQL before learning other languages. Python is widely
    used and flexible. You will be well-positioned if you decide to
    switch to more software development work or to go full-steam into
    machine learning.

4.  Use [Kaggle](https://www.kaggle.com/). Do the tutorials, participate
    in the forums, enter a competition (don’t worry about where you
    place - just focus on doing a little better every day). It’s the
    best way to learn practical machine skills.

5.  Search for data science and tech meetups in your area. With the
    explosion of data science in the last few years, there are now
    meetups in countries all over the world and in a wide variety of
    cities. For instance, Google recently held a [TensorFlow Dev
    Summit](https://events.withgoogle.com/tensorflow-dev-summit/) in
    Mountain View, CA, but there were viewing parties around the world
    that watched the livestream together ([including in Abuja,
    Nigeria](https://twitter.com/jennifazor/status/831922571469869057),
    [Coimbatore,
    India](https://twitter.com/GDGCoimbatore/status/832231191042367488),
    and [Rabat,
    Morocco](https://twitter.com/GDGRabat/status/831913259028643841)).

## Online courses

Online courses are an amazing resource. You can learn from the world’s
best data scientists in the comfort of your own home. Often the
assignments are where most of the learning occurs, so don’t skip them!
Here’s a few of my favorites:

- Andrew Ng’s classic [machine learning Coursera
  class](https://www.coursera.org/learn/machine-learning) (He does a
  great job explaining the algorithms, although don’t spend too much
  time on Octave/MATLAB. It’s probably better to do the assignments in
  Python, [like
  this](http://www.johnwittenauer.net/machine-learning-exercises-in-python-part-1/).)
- The fast.ai [Practical Deep Learning for
  Coders](http://course.fast.ai/) course. Disclosure: I helped create
  this course, and we’ve gotten lots of [positive
  feedback](http://course.fast.ai/testimonials.html) from students. The
  only prerequisite is 1 year of coding experience.
- Udacity’s [data science
  sequence](https://www.udacity.com/courses/data-science).
- Udacity’s [Web Development
  Course](https://www.udacity.com/course/web-development--cs253).
  Technically you don’t need to know this to be a data scientist, but
  [becoming a better software developer makes you a better data
  scientist](https://quomodocumque.wordpress.com/2017/02/26/mathematicians-becoming-data-scientists-should-you-how-to/),
  and it is so useful to know how to build your own web apps.

As one of the questioners highlighted above, the amount of information,
tutorials, and courses available online can be overwhelming. One of the
biggest risks is jumping from thing to thing, without ever completing
one or sticking with a topic long enough to learn it. It’s important to
find a course or project that is “good enough”, and then stick with it.
Something that can be helpful with this is finding or *starting a
meet-up group* to work through an online course together.

Online courses are very useful for the knowledge you gain (and it’s so
important that you do all the assignments, as that is how you learn).
However, I have not seen any benefit to getting the certificates from
them (yet– I know this is a newer area of growth). This is based on my
experience as having interviewed a ton of job applicants when hiring
data scientists, and having interviewed for many positions myself.

## News sources

- Twitter can be a surprisingly helpful way to find interesting articles
  and opportunities. For instance, my collaborator Jeremy Howard has
  provided over 1,000 links to his favorite [machine learning papers and
  blog posts](https://twitter.com/jeremyphoward/likes) (NB: you’ll need
  to be signed in to Twitter to read this link). It will take some time
  to figure out who to follow (and may involve some following and
  unfollowing and searching along the way), although one short cut is to
  look at who wrote the tweets you like in the link above, and follow
  them directly. Look up data scientists at companies that interest you.
  Look up the authors of libraries and tools that you use, or are
  interested in. When you find a tutorial or blog post you like, look up
  the author. And then look up who these people retweet. If you are
  unsure what or how to tweet, I think it can be helpful to think of
  Twitter as a way to (publicly) bookmark links that you like. I try to
  tweet any article or tutorial that I think I may want to reference
  back to in a few months time.
- The [machine learning
  subreddit](https://www.reddit.com/r/MachineLearning/) is a great
  source of recent news. You may find a lot of it inaccessible at first,
  but after a couple of months you’ll start seeing more and more that
  you recognize
- It’s helpful to sign up for newsletters such as [Import AI
  newsletter](http://us13.campaign-archive1.com/home/?u=67bd06787e84d73db24fb0aa5&id=6c9d98ff2c)
  and [WildML news](http://www.wildml.com/newsletter/)

## Moving to the Bay Area

Do whatever you can to move to the Bay Area! I realize that this won’t
be possible for many people (particularly if you have children or for a
variety of visa/legal residency issues). There are so many data science
meet-ups, study groups, conferences, and workshops here. There is also
an amazing community of other bright, ambitious, hungry-to-learn data
scientists. I had trouble even figuring out which were the most useful
things for me to learn from afar. Although I’d started studying machine
learning on my own before moving here, coming to SF rapidly accelerated
my learning.

My first year in San Francisco was a period of intense learning for me:
I attended tons of meetups, completed several online courses,
participated in numerous workshops and conferences, learned a lot by
working at a data-focused start-up, and most importantly met scores of
people who I was able to ask questions of. I completely under-estimated
how amazing it is to be able to interact regularly with the people who
are building the tools and technology that excite me most. I’m
surrounded by people who love learning and are pushing the cutting edge
of what is possible. That TensorFlow Dev Summit I mentioned above that
people watched from around the world? I was lucky enough to be able to
attend it live, and [my favorite part was the people I
met](https://twitter.com/math_rachel/status/832047022215569408).

One good approach to moving here is taking a “not-your-dream-job”;
i.e. try to get to a place where you’re surrounded by people you can
learn from, even if it’s not a role you’d otherwise be interested in. I
decided to switch careers in early 2012, before Insight or other data
science bootcamps existed. I applied to a few of what were my “dream
jobs” at the time and was rejected. In hindsight, I think this was a mix
of my lacking some needed skills, not knowing how to market myself
properly, and doing a fairly brief job search. In March 2012, I accepted
an analyst position at a start-up I was excited about, with the hope and
an informal agreement that I could move into an official data
science/modeling role later. Overall, it was a good choice. It let me
move to San Francisco rapidly, the company I joined was great in a
number of ways (including having a weekly reading group that was working
through [Bishop’s Pattern
Recognition](http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf)
and including me on a field trip to meet [Trevor Hastie and Jerome
Friedman](https://statweb.stanford.edu/~tibs/ElemStatLearn/)), and my
manager was supportive of me doing more engineering-intensive projects
than the role was officially scoped for. One year later, I landed what
on paper was my dream job: a combined data scientist/software engineer
role with a startup that had fascinating datasets.

There are also some good bootcamps in the area, which generally also
provide many opportunities to connect with interesting people and
companies in the data science space. - [Insight Data
Science](http://insightdatascience.com/) is a 7-week, free, intensive
bootcamp for graduates with PhDs in STEM fields. Potential downsides:
Since it is only 7 weeks, part of which is focused on networking and job
search, I believe it’s mostly for people who already have most of the
skills they need. Also, it’s very competitive to get into. - Data
Science boot camps such as
[Galvanize](https://new.galvanize.com/san-francisco/data-science) or
[Metis](https://www.thisismetis.com/data-science-bootcamps). Positives:
These are 12-week immersive experiences, that provide structure and
networking opportunities. Downsides: These are rather expensive. Some
factors to consider: How close is your background to what you need? That
is, if you have little programming experience, it may be necessary to do
something like this, but if you are transitioning from a closely related
field, this may be overkill. Also, how motivated are you with
independent self-learning? If you struggle with it, the accountability
and structure of a bootcamp could be helpful.

There are many factors in deciding whether to do a bootcamp. A big one
is how much structure or external motivation you need. There are a lot
of amazing resources available online. How much discipline do you have?
Note that it’s important to accept what you need to best learn. I find
the motivation of online courses and having assignments really works for
me, and I used to feel embarrassed that this was easier for me than
having a completely independent side project. Now, I’ve accepted this
and try to work with it. Other questions to ask: how much do you need to
learn, and how quickly can you learn on your own? If it’s a lot, a
bootcamp may really speed that up. One area where I think bootcamps can
particularly shine is teaching you how to put a bunch of different
tools/techniques together.

You can also move here without a job. This requires a number of things,
including: ample savings, US legal residency status, and not having
children, so it won’t be an option for many people. However, if you are
able to do it (i.e. a US permanent resident, coming from finance), it
can be a good option. Searching for a job in tech can be a full-time
job, as data science and engineering interviews require a lot of
studying to prepare and many require time-intensive take-home
challenges. In hindsight, I’ve often done rushed job searches when I was
working full-time and job-searching, and that has lead me to a few
sub-optimal decisions. You will certainly find plenty of ways to fill
your time with studying for interviews, coding side projects, and
attending workshops and study groups. Also, two things that surprised me
when I switched to tech are how frequently people switch jobs, and how
normal it is to take time off between jobs for learning new things or
traveling (so there is less reason to worry about gaps in your resume,
as long as you have good answers about what you were learning during
that time).

The huge caveat: I was unaware of how sexist, racist, ageist,
transphobic, and morally bankrupt Bay Area tech is (despite its
grandiose claims to be creating a better future) 5 years ago when I made
my move. A few years later, I became so discouraged that [I considered
leaving the tech industry
altogether](https://medium.com/tech-diversity-files/if-you-think-women-in-tech-is-just-a-pipeline-problem-you-haven-t-been-paying-attention-cb7a2073b996#.c65hq1a3j).
Tales of betrayal, callousness, and cruelty abound: for instance,
someone I’m close to had his family medical emergencies exploited for
profit by his coworkers, and many of my friends and loved ones have had
similarly awful experiences. However, the community of passionate,
fascinating people and access to cutting edge technology keeps me here,
and given the choice, I’d choose to move here all over again. I
currently feel very lucky with fast.ai to be working on the problems
that I find most interesting and believe will have the greatest impact.

## Answers to miscellaneous related questions

*Do I need a masters or PhD to work in AI?* I firmly believe the answer
is NO, and I’m working to make that even more of a reality than it
already is. In fact, AI PhDs are often not that well-positioned to
tackle practical, relevant business problems, because that is not what
their training is for. Academia is focused on advancing the theoretical
bounds of the field, and is motivated by what can be published in top
journals (very different from what will create a viable business!). Read
more about fast.ai’s educational philosophy
[here](http://www.fast.ai/2016/10/08/teaching-philosophy/) and check out
our free, online course [Practical Deep Learning for
Coders](http://course.fast.ai/).

*Should I learn Ruby after learning Python?* There’s no reason for an
aspiring data scientist to learn Ruby. It’s similar enough to Python
that it won’t teach you new concepts (the way learning a functional
language or lower-level language would), and there is not much of a data
science eco-system.

*Where can I find side projects that will interest employers? I think I
can find random data sets online, but I guess employers do want to see
how I handle a real situation?* Don’t feel like your side project needs
to be completely unique, or involve a unique or unusual data set. It’s
fine to use a dataset you get from Kaggle for a side project. It’s fine
if your project isn’t ground-breaking. When creating side projects, blog
posts, or tutorials, think of your audience as the person who is
one-step behind you, as they are the one who you are best positioned to
help. You may be worried that a project or post wouldn’t be interesting
to someone senior in the field, or that someone else may have done
something similar. That’s fine! It’s good just to get your work out
there.
