{"id":1568,"date":"2019-03-12T04:13:10","date_gmt":"2019-03-12T04:13:10","guid":{"rendered":"http:\/\/kusuaks7\/?p=1173"},"modified":"2023-09-08T12:20:34","modified_gmt":"2023-09-08T12:20:34","slug":"scaling-machine-learning-from-0-to-millions-of-users-part-2-training-ec2-emr-ecs-eks-or-sagemaker","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/scaling-machine-learning-from-0-to-millions-of-users-part-2-training-ec2-emr-ecs-eks-or-sagemaker\/","title":{"rendered":"Scaling Machine Learning from 0 to Millions of Users\u200a\u2014\u200aPart 2 Training: EC2, EMR, ECS, EKS or SageMaker?"},"content":{"rendered":"

In Part 1<\/a>, we broke out of the laptop, and decided to deploy our prediction service on a virtual machine. By doing so, we discussed a few simple techniques that helped with initial scalability\u2026 and hopefully with reducing manual ops. Since then, despite a few production hiccups due the lack of high availability, life has been pretty good.<\/p>\n

However, traffic soon starts to increase, data piles up, more models need to be trained, etc. Technical and business stakes are getting higher, and let\u2019s face it, the current architecture will go underwater soon. Time\u2019s up: in this post, we\u2019ll focus on\u00a0scaling training to a large number of machines<\/strong>.<\/p>\n

<\/p>\n

Two hundred\u2026 and fifty\u2026 six\u2026 GPUs!!!\u00a0Ja!<\/span><\/p>\n

This is an opinionated series, remember? It\u2019s also based on what I hear when talking to real-life customers, not ideal ones. Reality is often ugly, and only tech hipsters (and top management) think that it looks exactly like that fancy article your read on <insert_name_here>\u00a0\ud83d\ude09<\/p><\/blockquote>\n

Special thanks to rockin\u2019 Evangelists Abby Fuller, Jerry Hargrove, Adrian Hornsby, Brent Langston and Ian Massingham for their tips and ideas.<\/p><\/blockquote>\n

Scaling up will fix it\u2026\u00a0right?<\/strong><\/h3>\n

Yes and no. Yes, it can be a\u00a0short-term solution<\/strong>\u00a0to use a large server for training and prediction. Amazon EC2 has a ton of\u00a0instance types<\/a>\u00a0to pick from, and all it takes is\u00a0stopping<\/a>\u00a0your instance,\u00a0changing the instance type<\/a>, and\u00a0starting<\/a>\u00a0it again.<\/p>\n

You should only be using APIs from now on. When it comes to scaling tech & ops, manual work in any GUI is the Antichrist.<\/p><\/blockquote>\n

No, because it\u2019s only a\u00a0temporary solution<\/strong>:<\/p>\n