Hire the Top 3% of Freelance Data Scientists

Data science is an exploding field. According to the Linkedin Workforce Report, demand for data scientists in 2018 was off the charts.

Data science is a multidisciplinary field that uses scientific methods, processes, algorithms, and systems to extract knowledge and insights from structured and unstructured data. (Wikipedia)

There is a huge movement in the industry toward the democratization of artificial intelligence and data science. This means that it’s now easier than ever to pick up relevant skills, and more and more people are trying to break into this field. This is a positive trend, yet at the same time, it is a double-edged sword. It makes hiring data scientists more difficult due to high demand and the constant influx of new people into the field.

This data science hiring guide should help you select the right person for the job.

Is a Data Scientist What You Really Need?

Before we dive into the how, you really need to answer the why. Is a data scientist really what you need? With the democratization of data science came a flood of tools and solutions that can be used as a black box. Tools like Google Vision API and Cloud AutoML help you achieve a lot without having to worry about what’s happening behind the scenes.

Before you start a full-fledged search for a data scientist, know your needs, and search if there is a way to use existing APIs and tools to get the same (and in some cases better) result than an in-house custom solution. The age-old adage “Don’t reinvent the wheel” is very relevant in this case.

Challenges of Hiring Data Scientists

One of the challenges of hiring a data scientist is judging whether the candidate has in-depth knowledge or just relies on blackbox libraries that do all the work for you. These work, but if the candidate has no knowledge of how it works, debugging any problem will be an issue.

The other challenge is knowing what you need the data scientist for, as there are many problems under the umbrella of data science (e.g., big data, Natural Language Processing (NLP), Computer Vision (CV), etc.). The first step before hiring data science talent is to define the scope of the problem you are trying to solve.

Finding Candidates

Obviously, the traditional ways of finding candidates are still there—such as word of mouth or posting on various websites like LinkedIn or Stack Overflow—but one new place to find good experienced data scientists is Kaggle.

Kaggle is an online community for data scientists and ML practitioners. However, this isn’t a one-size-fits-all type of recommendation. There are candidates who are great at Kaggle but might not be right for your organization. The flip side is also true, as there are candidates who are great data scientists but are not on Kaggle.

Selecting the Right Candidate

The first thing you need to do is define the problem a data scientist will solve for you, and then try to find the bucket of data science it falls into. If you have a language task, such as keyword classification or building a text summarizer, you need an NLP specialist. If the task is more of dealing with pictures and photos, you need a computer vision expert. Similarly, if the job involves dealing with labeled data (facial recognition, spam classification, etc.), you need someone with expertise in supervised learning. If the job is more along the lines of stats, such as A/B testing or analyzing results of controlled trials, you need someone who has a strong skillset in stats (ideally, all data science candidates should have statistical knowledge).

This figure offers a good generalization of different branches of data science:

To be fair, this is a generalization; in reality, the lines are fuzzy and there is some overlap, so the skill chart looks like this:

Note: A good data scientist should know about all these skills and branches and have expertise in one or more of them.

Data Scientist Hard Skills and Experience

Any data scientist should have statistical analysis knowledge, data analytics experience, and a real-world understanding of maths behind the algorithms they use. Other than that, software development and coding experience in Python, R, MATLAB, or Octave is a must. Experience with Hadoop and Spark is always important.

If you want to hire an NLP expert, the candidate should have ideally used NLP packages like NLTK, Stanford NLP, etc.

For computer vision, they should have experience working with CNN (Convolutional Neural Networks), OpenCV, etc.

Experience with algorithms like random forest, decision tree, naive Bayes, linear regression, or SVM (Support Vector Machine) is also necessary.

If the job is more on the side of using deep learning and neural networks, they should have experience building neural networks with TensorFlow, PyTorch, Theano, Torch, Sonnet, Keras, and MXNet.

Asking the Right Questions

1) What is the difference between supervised and unsupervised machine learning?

In supervised machine learning, the algorithm is trained on a labeled dataset, with the task being that the algorithm will find labels given new data.

In unsupervised learning, the data is unlabeled.

2) Can you provide some examples of supervised and unsupervised ML algorithms?

Supervised: random forest, decision tree, naive Bayes, linear regression, SVM, KNN (K-nearest neighbors).

Unsupervised: K-mean clustering, autoencoders, DBSCAN, hierarchical clustering.

3) What is the “exploding gradient”?

A gradient is the magnitude and direction of the update that is calculated while training a neural network.

An exploding gradient is a problem that happens when the gradient gets accumulated and the value becomes so large that it causes overflow, and the gradient becomes NaN, causing the training to stop.

4) What is a confusion matrix?

A confusion matrix is a 2x2 table that contains the metrics to judge the performance of a binary classifier.

• True Positives: values that belonged to the positive class, and the classifier labeled correctly.
• False Negatives: values that belonged to the positive class but the classifier labeled them as negative.
• False Positives: values that belonged to the negative class but the classifier labeled them as positive.
• True Negatives: values that belonged to the negative class, and the classifier labeled correctly.

5) What is the ROC curve and how does it work?

The ROC curve is a graph that tells us about the true positive rates (TPR) and false positive rates (FPR) at various thresholds. It is used to tell the tradeoff between TPR (sensitivity) and FPR (1 - specificity).

6) What is a decision tree and how does it work?

A decision tree is a classification/regression algorithm that uses a tree-like structure to learn the structure in data.

Each node in the tree represents a test. For example, one node can be whether the asset’s input is low and depending on the answer (yes or no), the decision tree will output bad risk or good risk.

7) How does the random forest algorithm work?

Random forest algorithm is an ensemble learning technique which creates multiple decision trees and trains them on the given data. The final prediction is the mode of all the classes predicted by the decision trees in case of a classification problem, and the average of all the values in case of a regression problem.

8) What are overfitting and underfitting?

Overfitting is a phenomenon where a machine learning algorithm fits too closely to a training dataset, which causes high accuracy/low loss on the training set, but low accuracy/high loss on the test/validation set.

Underfitting is a phenomenon where a machine learning algorithm is not able to learn the structure of the dataset, and hence this causes low accuracy/high loss.

9) How do you prevent overfitting in a decision tree?

There are two ways to prevent overfitting in a decision tree: pre-pruning and post-pruning.

In pre-pruning, we stop the decision tree before it becomes a full-grown tree.

Here are the typical stopping conditions for a node:

1. Stop if all instances belong to the same class.
2. Stop if all attribute values are the same.

In post-pruning, we let the decision tree grow to full length, then trim the nodes of the tree in a bottom-up fashion.

10) What is a normal distribution?

Data is often distributed with a bias toward the left or right of the mean. When the data is symmetric around the mean, i.e., more frequent near the mean and less so the further away you go from the mean, then the data follows a normal distribution.

11) What problems can arise from a too high learning rate in a deep neural network, and why?

If the learning rate is too high, the loss can jump around the minima. Since the learning rate decides how big the updates to the weights of the network are, a high learning rate can cause large updates, which in turn cause divergent behavior.

12) While working on a dataset, you find that two variables have negligible Pearson correlation value. Does this mean that the two variables are independent of each other? Why/why not?

Correlation is defined as the relationship between two variables. Positive correlation is when a variable increases, prompting the other variable to increase as well. Similarly, negative correlation is when a variable decreases but the other increases.

Pearson correlation is a measure of linear relationship between two variables. Just because two variables have close to zero Pearson correlation doesn’t mean that they are independent; it just means that there is no linear relationship between them, but a relationship of higher order can still exist.

13) There is a deep neural network with 15 hidden layers, all of which have the same activation function, i.e., tanh. While training, you notice that training loss stays the same. What seems to be the problem and what is the simplest change you can do to tackle this problem?

This is a vanishing gradient problem. Neural networks are usually trained with gradient-based training methods and backpropagation. Essentially, we send each weight an update proportional to partial derivative of the error with respect to the current weight. Sometimes, this update becomes so small as an error is back-propagated that weights don’t change, and hence, training loss stays the same.

To fix this problem, we can reduce the number of layers and replace tanh with ReLU activation function.

14) You are training a simple RNN network to predict the new word in a sentence, but while training, the accuracy is not good. After debugging, you find that the RNN is unable to follow the context beyond a couple of words. Which deep learning model do you use to fix this?

A Recurrent Neural Network (RNN) is a deep neural network with such a structure that they are able to use their internal state to remember the previous inputs. Unlike feedforward neural networks, in RNN, the output from the previous step is fed as input to the current step. This forms a loop allowing information to persist.

RNNs suffer from a problem where they are unable to “remember” a lot of data due to their simple structure. Using something like LSTM (Long Short-Term Memory) can help solve this problem.

LSTMs are a special kind of RNNs capable of learning long-term dependencies.

In LSTM, there is an explicit memory state which can be updated by the LSTM cell using carefully controlled gates. This memory state allows the LSTM to retain the memory much longer, and hence, it can follow the context for a much longer sequence.

15) During training on a binary dataset, you find that the accuracy of your algorithm is high but while analyzing the dataset, you find that there is a very high disbalance between the samples of one class. For example, the positive samples make up almost 99% of the dataset. Can you still trust the accuracy metric? If not, what metric should you use?

When dealing with an imbalanced dataset, accuracy is not the best measure of the performance, because if in this case the algorithm marks all data as positive, it will still get 99% of the answers right, and will thus have 99% accuracy. In such cases, metrics like balanced accuracy, f1 scores are used.

• Precision: the value of true positives/(true positives + false positives)
• Recall: the value of true positives/(true positives + false negatives)
• F1 Score: the harmonic mean of precision and recall
• Balanced Accuracy: the average recall of each of the classes in the dataset

16) You are working on a clustering problem and you have a high-dimensional dataset. While training a basic k-means clustering algorithm on the data, you notice that no matter how much you tweak the hyperparameters, the clusters keep changing between runs. Why do you think this is the case? What can you do to overcome this problem?

K-means clustering works on distance-based metrics. While working on a high-dimensional dataset, distance-based metrics is rendered almost useless by the curse of dimensionality since all the data points in the dataset appear to be equidistant due to the large number of dimensions. Hence, in multiple runs, the algorithm is unable to get consistent results. To fix this, you should try applying PCA/t-SNE or some other dimensionality reduction algorithm.

17) While using Stochastic Gradient Descent on an optimization problem with a large number of parameters, you find that the algorithm is oscillating and is unable to achieve the global optima. Why is this? How will you fix it?

One of the biggest cons of Stochastic Gradient Descent (SGD) is that the learning rate is the same for all parameters. So if one of the parameters needs a smaller learning rate and some other parameter needs a larger learning rate, SGD is unable to perform well in such a scenario.

This problem is magnified with the optimization problem that has a lot of parameters, as the probability of such a situation arising is also large. In such cases, you should use algorithms with adaptive learning rate like Adam optimizer.

Don’ts

These are some common issues of hiring data scientists, and you should avoid them as much as possible.

1. Don’t ask “gotcha questions.” Gotcha questions are questions that are very hard to solve unless you know a particular technique. These questions may seem tempting, but avoid them since knowing or not knowing the solution to such a question is not a good indicator of the candidate’s knowledge.
2. Don’t take projects at face value. Projects are great but now, more than ever, it is easier to do machine learning in just a few lines of code. While this is great news, it also means that some candidates might just use prebuilt Python libraries without really understanding the working of the same. You must ask questions and make the candidate explain the work they did.
3. Don’t insist that the candidate should know your exact technology stack. You should probe the candidate for their willingness and ability to learn new techniques and methods, rather than insisting that they know what you have been using. You might find a great candidate who doesn’t know your tech stack but is able to learn it quickly.

Wrapping Up

The growing amount of data in organizations creates an ever-growing demand for data scientists, data analysts, and data engineers.

With the democratization of machine learning, it’s now easier than ever to gain relevant skills in this field. But there are also challenges in finding the right candidate for your team—as if looking for your very own needle in a haystack.

Our selection of tips and questions can aid in hiring the right data scientist for your team; however, it is meant to be an addition to your overall hiring strategy, as only you know who the right candidate for your team is. This guide is designed to help you with the decision-making process, and we hope that the above questions and tips will help you identify the right candidate for your organization and project.