Skip to main content

what data scientist spend the most time doing


Generally we think of data scientists building algorithms,exploring data and doing predictive analysis. That's actually not what they spend most of their time doing however , we can see in the in the graph most of the time Data scientist are involved in data cleaning part , as in real world scenario we are mostly getting the data which is messey, we can feed the data after cleaning , ML model will not work if the data is messey, Data cleaning is very very important so mostly data analyst and data scientists are involved in this task.

60 percent: Cleaning and organising Data

According to a study, which surveyed 16,000 data professionals across the world, the challenge of dirty data is the biggest roadblock for a data scientist. Often data scientists spend a considerable time formatting, cleaning, and sometimes sampling the data, which will consume a majority of their time.Hence, a data scientist, the need for you to ensure that you have access to clean and structured data can save you a lot of time and will help you get done with the work quickly.

19 percent: Collecting data

One of the major challenges that Data Science professionals face is finding the relevant data sets to work with. Many a time organisation’s data lakes are nothing but a dumping ground with relevant and irrelevant data sets. 

9 percent: Modelling/machine learning

Once the first two use cases have been sorted, a data scientist is then left with the task of suggesting machine learning and predictive modelling as per business requirements.

It is said that one of the hardest parts of being a data scientist is not exactly developing a problem, rather it is about defining a given problem and finding means to measure the solution. This is even more pertinent when the clients do not have a clear idea of what they want. So if your models do not deliver the outcomes in correlation with the business requirement, then you are left with the daunting task of explaining discrepancies and understanding what went wrong and where.

“Often, analysts are given vague goals by the business. “Help me improve my bottom line by 15%” or “Identify the biggest problems our customers are facing” are not precise enough problem statements for the analysts. Enough time needs to be spent on understanding the exact business problem and then converting this business problem into an analytics problem that can be solved with data,” Gaurav Vohra co-founder & CEO of Jigsaw Academy notes.

5 percent: Other

Since Data Science is a mix of business use-cases, mathematics, statistics, programming and communication skills, data scientists are not singularly tasked with data handling alone. As another Quora user sums up, a data scientist is also required to perform a number of other tasks which include:

Undirected research and frame open-ended industry questions
Explore and examine data from a variety of angles to determine hidden weaknesses, trends and/or opportunities
Communicate predictions and findings to management and IT departments through effective data visualizations and reports
Recommend cost-effective changes to existing procedures and strategies

4 percent: Refining algorithms

This process might take months before to make the necessary changes and this can be achieved through a number of ways, often leaving the data scientist with perplexing questions choosing the right way to do so.

3 percent: Building training sets

Data Sets are the essential component or the building blocks upon which the data scientist builds his project. At times, the data scientist will have to perform scaling, decomposition, aggregation transformations on the data before they can train their models.
Labels:

Comments

Post a Comment

Popular posts from this blog

Introduction to Datascience

Data Science has become one of the most demanded jobs of the 21st century. What is Data Science? “Data Science is about extraction, preparation, analysis, visualization, and maintenance of information. It is a cross-disciplinary field which uses scientific methods and processes to draw insights from data. ” As a data scientist, you take a complex business problem, compile research from it, creating it into data, then use that data to solve the problem. A Data Scientist, specializing in Data Science, not only analyzes the data but also uses machine learning algorithms to predict future occurrences of an event. Therefore, we can understand Data Science as a field that deals with data processing, analysis, and extraction of insights from the data using various statistical methods and computer algorithms. It is a multidisciplinary field that combines mathematics, statistics, and computer science. Why Data Science? So, after knowing what exactly Data Science is, you must explore ...
Post a Comment
Read more

15 Common questions for Machine Learning...!!

  1. What is logistic regression? Logistic regression is a machine learning algorithm for classification. In this algorithm, the probabilities describing the possible outcomes of a single trial are modelled using a logistic function. 2. What is the syntax for logistic regression? Library: sklearn.linear_model.LogisticRegression Define model: lr = LogisticRegression() Fit model: model = lr.fit(x, y) Predictions: pred = model.predict_proba(test) 3. How do you split the data in train / test? Library: sklearn.model_selection.train_test_split Syntax: X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42) 4. What is decision tree? Given a data of attributes together with its classes, a decision tree produces a sequence of rules that can be used to classify the data. 5. What is the syntax for decision tree classifier? Library: sklearn.tree.DecisionTreeClassifier Define model: dtc = DecisionTreeClassifier() Fit model: model = dtc.fit(x, y) Predictions: p...
Post a Comment
Read more

Data Science Interview Questions -Part 2

1) What are the differences between supervised and unsupervised learning? Supervised Learning Unsupervised Learning Uses known and labeled data as input Supervised learning has a feedback mechanism  Most commonly used supervised learning algorithms are decision trees, logistic regression, and support vector machine Uses unlabeled data as input Unsupervised learning has no feedback mechanism  Most commonly used unsupervised learning algorithms are k-means clustering, hierarchical clustering, and apriori algorithm 2) How is logistic regression done? Logistic regression measures the relationship between the dependent variable (our label of what we want to predict) and one or more independent variables (our features) by estimating probability using its underlying logistic function (sigmoid). The image shown below depicts how logistic regression works: The formula and graph for the sigmoid function is as shown: 3) Explain the steps in making a deci...
Post a Comment
Read more