{"id":1266,"date":"2019-02-15T10:32:02","date_gmt":"2019-02-15T10:32:02","guid":{"rendered":"http:\/\/kusuaks7\/?p=871"},"modified":"2021-05-17T18:14:54","modified_gmt":"2021-05-17T18:14:54","slug":"why-i-think-machine-learning-enhanced-software-systems-are-the-future","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/why-i-think-machine-learning-enhanced-software-systems-are-the-future\/","title":{"rendered":"Why I think machine learning-enhanced software systems are the future."},"content":{"rendered":"

Ready to learn Machine Learning? Browse courses<\/a> like Machine Learning Foundations: Supervised Learning<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n

\"experfy-blog\"<\/p>\n

I have been brewing the idea of using machine learning to improve software systems since 2016. It was pretty vague and broad, without an actionable plan. I just had the intuition \u2014 the software configuration and tuning, especially after the adoption of microservices, was getting too complex.<\/p>\n

The increasing complexity of configuring and tuning systems<\/strong><\/h3>\n

If you have enough experience in the software industry, then it\u2019s very likely that you\u2019ve struggled with either a configuration problem or a tuning problem.<\/p>\n

Configuration and tuning problems are pretty common and can lead to really bad outages. They often occur when:<\/p>\n

    \n
  1. Some parts of the system are poorly or wrongly configured, or<\/li>\n
  2. A configuration that worked before now doesn\u2019t work because the context of the system has changed.<\/li>\n<\/ol>\n

    Think of a number of database replicas and their writing schemes. Or in Postgresql, think of the number of shared buffers, effective cache size, and the min and max wal size.<\/p>\n

    If wrongly configured from the start, it won\u2019t work in the given context, plain and simple. What\u2019s more interesting, though, is if it\u2019s correctly<\/em> configured, it might work at a given time. But as the context changes \u2014 system workload, system resources usage, overall system architecture \u2014 the system will behave poorly. Or, even worse, an outage might happen.<\/p>\n

    This will, inevitably, lead to manually-performed operations and the creation of heuristics. In other words, it will lead to:<\/p>\n

    Oh, we should set X to A, when workload is T, but it should be A+10 when workload is T+100 and we have system resources usage above 80%\u2026 I guess. Or maybe let\u2019s just up a queue in front of this component, queues solve everything, right?<\/em><\/p>\n

    Now multiply this scenario by tens or hundreds of services. Think for a second about the cognitive burden resulting from these configurations.<\/p>\n

    This is not a new concern. In 2003, Ganek and Corbi discussed<\/a> the need for autonomic computing to handle the complexity of managing software systems. They noted that managing complex systems became too costly, labor-intensive, and prone to error due to the pressure engineers felt while maintaining them. This increased the potential of system outages with a concurrent impact on business.<\/p>\n

    Even nowadays, most of the configurations and tuning of the systems are performed manually, often in run-time, which is known to be a very time-consuming and risky practice. Check out these two links (here<\/a> and here<\/a>) to read more about it.<\/p>\n

    \"experfy-blog\"<\/p>\n

    The need for autonomic computing<\/strong><\/h3>\n

    Most decisions to configure and tune the system are made based on the context \u2014 there are many different variables such as workload, number of instances of some services, resources usage, and more. So why not delegate these tasks to something that excels at exactly that? Machine learning sounds like a feasible tool for the job.<\/em><\/p>\n

    After starting my Masters at the University of British Columbia, I kept working on this idea. It seemed interesting although quite weird, and, sometimes, unpractical and impossible to implement.<\/p>\n

    To my surprise, I realized I wasn\u2019t alone. Some very interesting people were working on these ideas \u2014 so it might not be that weird, unpractical, and impossible.<\/p>\n

    Recently, Jeff Dean \u2014 a man that I admire a lot \u2014 gave a talk at NIPS 2017 talking about machine learning for systems<\/a>, where he stated:<\/p>\n

    Learning should be used throughout our computing systems. Traditional low-level systems code (operating systems, compilers, storage systems) does not make extensive use of machine learning today. This should change!<\/em><\/p>\n

    Computer Systems are filled with heuristics: compilers, networking code, operating systems. Heuristics have to work well \u201cin general case\u201d. [They] generally don\u2019t adapt to actual pattern of usage and don\u2019t take into account available context<\/em><\/p>\n

    Learning in the core of all of our computer systems will make them better\/more adaptive.<\/em><\/p>\n

    I was in complete awe when I read this. One of the engineers I admire the most was talking about the very same ideas I\u2019ve been thinking about and working on.<\/p>\n

    This led me to think that it\u2019s not only interesting but natural to think about enhancing software systems with machine learning.<\/strong> Throughout the whole software stack, we have many heuristics that, although they work well, could be improved by machine learning.<\/p>\n

    Is it challenging and potentially risky? Yes, most definitely. Especially given that interpretability, apparently, has become a secondary goal in the machine learning community. How can we interpret and explain the decisions made by neural nets?<\/p>\n

    However, with that said, these obstacles shouldn\u2019t hinder scientific and technological progress. Yes, we should question old paradigms <\/a>and try to improve things.<\/p>\n

    \"experfy-blog\"<\/p>\n

    Towards machine learning-enhanced software systems<\/strong><\/h3>\n

    As Jeff Dean pointed out: we need to find practical<\/strong> ways to make systems data-aware. We need systems that collect metrics and metadata about themselves. To achieve this, we could learn a thing a two from the ideas in systems observability and instrumentation. We have been instrumenting systems for decades, and the data is already there.<\/p>\n

    We also need to find practical<\/strong> and clean<\/strong> ways to integrate<\/strong> machine learning components into software systems, making learning a first-class citizen in the system. This will lead to systems that learn how to improve themselves,<\/strong>beating heuristics and manually-performed operations. Think about this for a second. It does sound cool and<\/em> feasible.<\/p>\n

    I would also add that we need practical<\/strong> and clean<\/strong> ways to propagate the decisions made by the learned models to the rest of the system. This would allow the system to have self-adaptive capabilities. Here, we could learn something from the control theory community.<\/p>\n

    The general idea is fairly simple: make a system learn about its behavior by training a model on its context. Then allow it to change its structures and configurations in order to optimize for a certain scenario. Now implement this idea in such a way that it could be possible to integrate it into many kinds of systems.<\/p>\n

    Summary<\/strong><\/h3>\n

    The most interesting questions I have in mind are:<\/p>\n

      \n
    1. Can self-adaptation by learned models lead to more stable, faster, safer software systems? Can it reduce the need for manually configuring and tuning systems, allowing engineers to focus on more important tasks?<\/li>\n
    2. Can this be easily integrated into software systems, requiring only small changes to the codebase?<\/li>\n
    3. Can this work with low overhead?<\/li>\n<\/ol>\n

      It is worth noting that this would not<\/strong> replace good engineers, but would rather free the engineers\u2019 cognitive abilities to focus on what matters.<\/p>\n","protected":false},"excerpt":{"rendered":"

      The general idea is fairly simple: make a system learn about its behavior by training a model on its context. Then allow it to change its structures and configurations in order to optimize for a certain scenario. Now implement this idea in such a way that it could be possible to integrate it into many kinds of systems.<\/p>\n","protected":false},"author":189,"featured_media":21930,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-post-2.php","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[97],"ppma_author":[2790],"class_list":["post-1266","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-artificial-intelligence"],"authors":[{"term_id":2790,"user_id":189,"is_guest":0,"slug":"rodrigo-araujo","display_name":"Rodrigo Ara\u00fajo","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Ara\u00fajo","first_name":"Rodrigo","job_title":"","description":"Rodrigo Araujo worked as software engineer serving highly scalable ML models and systems. His research is on using machine learning to build self-adaptive systems; systems that can learn how to improve themselves and adapt to different contexts."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1266","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/189"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1266"}],"version-history":[{"count":3,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1266\/revisions"}],"predecessor-version":[{"id":21933,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1266\/revisions\/21933"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/21930"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1266"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}