{"id":633,"date":"2018-03-28T01:55:47","date_gmt":"2018-03-27T22:55:47","guid":{"rendered":"http:\/\/kusuaks7\/?p=238"},"modified":"2025-08-22T12:46:09","modified_gmt":"2025-08-22T12:46:09","slug":"data-preprocessing-for-non-techies-feature-exploration-and-engineering-part-two-checklist-of-most-common-practices","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/data-preprocessing-for-non-techies-feature-exploration-and-engineering-part-two-checklist-of-most-common-practices\/","title":{"rendered":"Data Preprocessing for Non-Techies: Feature Exploration and Engineering,\u00a0Part Two\u200a\u2014\u200aChecklist of Most Common Practices"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tReady to learn Data Science? Browse courses<\/a>\u00a0like\u00a0Data Science Training and Certification<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Now that we have covered the basic terms and definitions for data types and structure on my\u00a0previous\u00a0post<\/a>\u00a0<\/strong>let\u2019s dive into the creative and most time consuming side of data science \u2014 cleaning and feature engineering.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
What are some of the basic strategies that data scientists use to clean their data AND improve the amount of information they get from it?<\/strong><\/blockquote>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

The type of cleaning and engineering strategies used usually depend on the business problem and type of target variable, since this will influence the algorithm and data preparation requirements.<\/p>\n

Therefore, I will provide you a basic\u00a0checklist\u00a0<\/em>that can help any beginner brainstorm what to do with the data at this stage. (including me)<\/strong><\/p>\n

The most important part of data cleaning is the experimentation, and checking how applying one or many of this strategies affects your ability to actually predict or classify in the model.<\/p>\n

Also, although there is some logic in the order, keep in mind that these steps always happen in iteration, and you will always go back and forth between:<\/em><\/p>\n

\u2192 Exploration, Cleaning, Creation, and Selection<\/strong><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Data Exploration<\/strong><\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

A. Variable Identification:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
    \n \t
  1. Context of Target Variable (logical connection)<\/li>\n \t
  2. Data Type per Feature (character, numeric, etc)<\/li>\n \t
  3. Variable Category (Continuous, Categorical, etc.)<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    B. Uni-variate Analysis:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t
    \n\t\t\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
      \n \t
    1. Central Tendency & Spread for Continuous<\/li>\n \t
    2. Distribution(levels) for Categorical<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t\t
      \n\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      C. Bi-variate Analysis:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t\t
      \n\t\t\t
      \n\t\t\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t
        \n \t
      1. Correlation of Continuous Variables<\/li>\n \t
      2. Two-Way Table or Stacked Columns for Categorical<\/li>\n \t
      3. Chi-Square Test for Categorical<\/li>\n \t
      4. Z-Test for Categorical vs Continuous<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t

        Data Cleaning<\/strong><\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t

        A. Remove\u00a0Noise:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t \t
      5. Duplicates<\/li>\n \t
      6. Paragraph Columns<\/li>\n \t
      7. Erroneous Values<\/li>\n \t
      8. Contradictions<\/li>\n \t
      9. Mislabels<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t

        B. Missing\u00a0Values:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\t
          \n \t
        1. Delete<\/li>\n \t
        2. Mean\/Mode\/Median Imputation<\/li>\n \t
        3. Prediction Model<\/li>\n \t
        4. KNN Imputation<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t\t
          \n\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t

          C. Outliers:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t\t
          \n\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t\t\t\t\t
            \n \t
          1. Cut-Off or Delete<\/li>\n \t
          2. Natural Log<\/li>\n \t
          3. Binning<\/li>\n \t
          4. Assign Weights<\/li>\n \t
          5. Mean\/Mode\/Median Imputation<\/li>\n \t
          6. Build Predictive Model<\/li>\n \t
          7. Treat them separately<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t

            D. Variable Transformation:<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t\t\t\t\t
              \n \t
            1. Logarithm<\/li>\n \t
            2. Square \/ Cube root<\/li>\n \t
            3. Binning \/ Discretization<\/li>\n \t
            4. Dummies<\/li>\n \t
            5. Factorization<\/li>\n \t
            6. Other Data Type<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t\t
              \n\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t

              Feature Creation<\/strong><\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t\t
              \n\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t

              A. Indicator Features<\/strong><\/h4>\n<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t\t
              \n\t\t\t
              \n\t\t\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t\t\t\t\t
                \n \t
              1. Threshold (ex. below certain price = poor)<\/li>\n \t
              2. Combination of features (ex. premium house if 2B,2Bth)<\/li>\n \t
              3. Special Events (ex. christmas day or blackfriday)<\/li>\n \t
              4. Event Type (ex. paid vs unpaid based on traffic source)<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                \n\t\t\t\t\t\t
                \n\t\t\t\t\t
                \n\t\t\t
                \n\t\t\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t

                B. Representation Features<\/strong><\/h4>\n<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                \n\t\t\t\t\t\t
                \n\t\t\t\t\t
                \n\t\t\t
                \n\t\t\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\t
                  \n \t
                1. Domain and Time Extractions (ex.purchase_day_of_week)<\/li>\n \t
                2. Numeric to Categorical (ex. years in school to \u201celementary\u201d)<\/li>\n \t
                3. Grouping sparse classes (ex. sold, all other are \u201cother\u201d)<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                  \n\t\t\t\t\t\t
                  \n\t\t\t\t\t
                  \n\t\t\t
                  \n\t\t\t\t\t\t
                  \n\t\t\t\t
                  \n\t\t\t\t\t

                  C. Interaction Features<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                  \n\t\t\t\t\t\t
                  \n\t\t\t\t\t
                  \n\t\t\t
                  \n\t\t\t\t\t\t
                  \n\t\t\t\t
                  \n\t\t\t\t\t\t\t\t\t
                    \n \t
                  1. Sum of Features<\/li>\n \t
                  2. Difference of Features<\/li>\n \t
                  3. Product of Features<\/li>\n \t
                  4. Quotient of Features<\/li>\n \t
                  5. Unique Formula<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                    \n\t\t\t\t\t\t
                    \n\t\t\t\t\t
                    \n\t\t\t
                    \n\t\t\t\t\t\t
                    \n\t\t\t\t
                    \n\t\t\t\t\t

                    D. Conjunctive Features<\/strong><\/h4>\n<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                    \n\t\t\t\t\t\t
                    \n\t\t\t\t\t
                    \n\t\t\t
                    \n\t\t\t\t\t\t
                    \n\t\t\t\t
                    \n\t\t\t\t\t\t\t\t\t
                      \n \t
                    1. Markov Blanket<\/li>\n \t
                    2. Linear Predictor<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                      \n\t\t\t\t\t\t
                      \n\t\t\t\t\t
                      \n\t\t\t
                      \n\t\t\t\t\t\t
                      \n\t\t\t\t
                      \n\t\t\t\t\t

                      E. Disjunctive Features<\/strong><\/h4>\n<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                      \n\t\t\t\t\t\t
                      \n\t\t\t\t\t
                      \n\t\t\t
                      \n\t\t\t\t\t\t
                      \n\t\t\t\t
                      \n\t\t\t\t\t\t\t\t\t
                        \n \t
                      1. Centroid<\/li>\n \t
                      2. PCA<\/li>\n \t
                      3. LDA<\/li>\n \t
                      4. SVD<\/li>\n \t
                      5. PLS<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                        \n\t\t\t\t\t\t
                        \n\t\t\t\t\t
                        \n\t\t\t
                        \n\t\t\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t

                        F. Programming<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                        \n\t\t\t\t\t\t
                        \n\t\t\t\t\t
                        \n\t\t\t
                        \n\t\t\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t\t\t\t\t
                          \n \t
                        1. Logic (FRINGE)<\/li>\n \t
                        2. Genetic<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t\t
                          \n\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t

                          Feature Selection<\/strong><\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t\t
                          \n\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t

                          A. Filter\u00a0Methods<\/strong><\/h4>\n<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t\t
                          \n\t\t\t
                          \n\t\t\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\t
                            \n \t
                          1. Correlation<\/li>\n \t
                          2. Statistical Score<\/li>\n \t
                          3. Ranking (Relief Algorithm)<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                            \n\t\t\t\t\t\t
                            \n\t\t\t\t\t
                            \n\t\t\t
                            \n\t\t\t\t\t\t
                            \n\t\t\t\t
                            \n\t\t\t\t\t

                            B. Wrapper\u00a0Methods<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                            \n\t\t\t\t\t\t
                            \n\t\t\t\t\t
                            \n\t\t\t
                            \n\t\t\t\t\t\t
                            \n\t\t\t\t
                            \n\t\t\t\t\t\t\t\t\t
                              \n \t
                            1. Forward Step Wise<\/li>\n \t
                            2. Backward Step Wise<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                              \n\t\t\t\t\t\t
                              \n\t\t\t\t\t
                              \n\t\t\t
                              \n\t\t\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t

                              B. Embedded\u00a0Methods<\/strong><\/h4><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
                              \n\t\t\t\t\t\t
                              \n\t\t\t\t\t
                              \n\t\t\t
                              \n\t\t\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\t
                                \n \t
                              1. Ridge Regression<\/li>\n \t
                              2. Lasso Regression<\/li>\n \t
                              3. Decision-Trees<\/li>\n \t
                              4. Elastic Net<\/li>\n \t
                              5. XGBoost<\/li>\n \t
                              6. SVM<\/li>\n \t
                              7. LightGBM<\/li>\n<\/ol>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

                                We have covered the basic terms and definitions for data types and structure on my previous post, let’s dive into the creative and most time consuming side of data science — cleaning and feature engineering. What are some of the basic strategies that data scientists use to clean their data AND improve the amount of information they get from it?The type of cleaning and engineering strategies used usually depend on the business problem and type of target variable, since this will influence the algorithm and data preparation requirements.<\/p>\n","protected":false},"author":268,"featured_media":3417,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[94],"ppma_author":[1785],"class_list":["post-633","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-data-science"],"authors":[{"term_id":1785,"user_id":268,"is_guest":0,"slug":"melody-ucros","display_name":"Melody Ucros","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Ucros","first_name":"Melody","job_title":"","description":"Melody Ucros<\/a>, an entrepreneurial Techie with an interest in Sales Strategy and Digital Transformations, is Director of Operations at Fundie Ventures, an impact investment consultancy startup. A full-time student at IE Business School's Masters in Big Data, she is Passionate about helping start-ups, playing with data & exchanging knowledge with impact-makers around the world."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/633","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/268"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=633"}],"version-history":[{"count":7,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/633\/revisions"}],"predecessor-version":[{"id":38069,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/633\/revisions\/38069"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/3417"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=633"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}