{"id":2102,"date":"2019-11-29T01:04:38","date_gmt":"2019-11-29T01:04:38","guid":{"rendered":"http:\/\/kusuaks7\/?p=1707"},"modified":"2024-02-19T15:47:42","modified_gmt":"2024-02-19T15:47:42","slug":"how-to-extend-scikit-learn-and-bring-sanity-to-your-machine-learning-workflow","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/how-to-extend-scikit-learn-and-bring-sanity-to-your-machine-learning-workflow\/","title":{"rendered":"How to Extend Scikit-learn and Bring Sanity to Your Machine Learning Workflow"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tWe usually hear (and say) that machine learning is just a commercial name for Statistics. That might be true, but if we’re building models using computers what machine learning really comprehends is Statistics\u00a0and<\/em>\u00a0Software Engineering<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

\n

To make great products: do machine learning like the great engineer you are, not like the great machine learning expert you aren't - Rules of Machine Learning: Best Practices for ML Engineering [1]<\/em><\/strong><\/h4>\n<\/blockquote><\/h4>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThis combination of Statistics and Software Engineering brings new challenges to the Software Engineering world. Developing applications in the ML domain is fundamentally different from prior software application domains, as Microsoft researchers point out [2].\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tWhen you don’t work in a large company or when you’re just starting out in the field it’s difficult to learn and apply Software Engineering best practices because finding this information is not easy. Fortunately, open-source projects can be a great source of knowledge and can help us address this need of learning from people that have more experience than us. One of my favorite ML libraries (and source of knowledge) is\u00a0Scikit-learn<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nThe project does a great job of providing an easy-to-use interface while also providing solid implementations, being both a great way to start in the field of ML and also a tool used in the industry. Using scikit-learn tools and even reading maintainer’s answers on the issue discussions on Github is a great way to learn from them. Scikit has a lot of contributors from industry and from academia, so as these people make contributions their knowledge gets \u201cembedded\u201d in the library. One rule of thumb of scikit-learn’s project is that user code should not be tied to scikit-learn \u2014 which is a library, and not a framework [3]. This makes it easy to extend scikit functionalities to suit our needs.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tToday we’re going to learn how to do this, building a custom transformer and learning how to use it to build pipelines. By doing so our code becomes easy to maintain and reuse, two aspects of Software Engineering best practices.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

The scikit-learn API<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tIf you’re familiar with scikit-learn you probably know how to use objects such as estimators and transformers, but it’s good to formalize their definitions so we can build on top of them. The basic API consists of three interfaces (and once class can implement multiple interfaces):\n