{"id":1046,"date":"2018-12-28T02:46:31","date_gmt":"2018-12-27T23:46:31","guid":{"rendered":"http:\/\/kusuaks7\/?p=651"},"modified":"2021-05-24T11:51:19","modified_gmt":"2021-05-24T11:51:19","slug":"machine-learning-for-trading","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/machine-learning-for-trading\/","title":{"rendered":"Machine Learning for Trading"},"content":{"rendered":"<p><strong><em>Ready to learn Machine Learning? Browse<\/em><\/strong> <strong><em><a href=\"https:\/\/www.experfy.com\/training\/tracks\/machine-learning-training-certification\">Machine Learning Training and Certification courses<\/a> developed by industry thought leaders and Experfy in Harvard Innovation Lab.<\/em><\/strong><\/p>\n<h4>How AI helps traders make better decisions &amp; improve high-frequency trading<\/h4>\n<p style=\"text-align: center;\"><img decoding=\"async\" alt=\"image\" src=\"https:\/\/images.ctfassets.net\/2yr4wv2jga4w\/6pNLfuSH6wsQSYOQWEYyoG\/ea46920a66cfc38a0917f96c063e68ec\/Trading.png\" style=\"width: 700px; height: 390px;\"><\/p>\n<p>&nbsp;<\/p>\n<h2 id=\"summary\">Summary<\/h2>\n<p>Trading is a gruesomely competitive world. And with AI being painted as the new wonderweapon for everything, it\u2019s understandable that there\u2019s a huge amount of interest in discovering how to use AI for trading.<\/p>\n<h3 id=\"disclaimer\">Disclaimer<\/h3>\n<p>AI does play an important role in trading \u2013 but maybe not in the way you\u2019d expect. Unfortunately, AI can\u2019t be used to power a superhuman trading machine that steals human traders\u2019 lunches in every market. Not even Renaissance Technologies has that capability. At least&nbsp;<em>not yet<\/em>.<\/p>\n<p><em>Why not?<\/em><\/p>\n<p>The short answer:&nbsp;<strong>human competition<\/strong>&nbsp;\u2013 more about that&nbsp;<a href=\"https:\/\/www.datarevenue.com\/en\/usecases\/machine-learning-for-trading#ai-today-is-not-smarter-than-humans\" rel=\"noopener\">below<\/a>.<\/p>\n<p>Meanwhile, the battles AI actually wins are much more incremental \u2013 but still significant. AI is more a trader\u2019s side-kick than their replacement.<\/p>\n<p>Below are the top four ways AI applies to trading today:<\/p>\n<h2 id=\"analysing-sentiment\">Analysing Sentiment<\/h2>\n<p>Humans can\u2019t process all the information out there \u2013 but machines can get close. With AI, machines can now do much more \u2013 including analysing and summarising texts.<\/p>\n<p>They can summarise things like sentiment: scrapers collect daily news, tweets, and other social media posts about a particular topic, and then AI algorithms (in particular Natural Language Processing) summarise whether positive or negative views are stronger. They can even categorise texts into topics and build human-readable text summaries automatically.<\/p>\n<p>This is highly valuable information for traders, who have to be up to date on what\u2019s happening as quickly as possible.<\/p>\n<h2 id=\"forecasting-real-world-data\">Forecasting real-world data<\/h2>\n<p>Traders also use AI to improve the reliability of input data forecasts \u2013 elements in the real world that help traders succeed.<\/p>\n<p>Like the prediction of:<\/p>\n<ul>\n<li>Weather patterns in the northeast over the next two weeks;<\/li>\n<li>Solar energy supply in Europe;<\/li>\n<li>The outcome of a political election.<\/li>\n<\/ul>\n<p>These forecasts are based on other algorithms developed by other companies \u2013 but that doesn\u2019t mean these predictions can\u2019t be improved.<\/p>\n<p><strong>One handy trick<\/strong>&nbsp;is to train an algorithm to&nbsp;<strong>combine multiple expert forecasts into another prediction<\/strong>&nbsp;\u2013 which is then more accurate than any of the forecasts it was based on. This is called&nbsp;<a href=\"https:\/\/en.wikipedia.org\/wiki\/Ensemble_learning\" rel=\"noopener\">ensembling<\/a>, and it works pretty well.<\/p>\n<h2 id=\"finding-patterns\">Finding patterns<\/h2>\n<p>Trading is about identifying localised patterns \u2013 which are often limited in time and space \u2013 and then guessing how to exploit them. The process of finding patterns is arduous and time consuming.<\/p>\n<p>But&nbsp;<strong>AI algorithms are basically pattern-finding machines<\/strong>. If an analyst suspects irregularities in a particular dataset, they can save time by using AI to find them.<\/p>\n<p>So AI can find useful patterns, as long as it\u2019s guided by an experienced analyst who knows what to look for. These patterns are then used by traders, who mix them with their experience and intuition, then apply them. Or you can use them to design automated trading machines \u2013&nbsp;see the next section.<\/p>\n<h2 id=\"tuning-high-frequency-trading-machines\">Tuning high-frequency trading machines<\/h2>\n<p>In high-frequency trading \u2013 as the name suggests \u2013 machines execute thousands or millions of trades per day, trying to take advantage of inefficiencies that only exist over very short time spans.<\/p>\n<p>Human beings can\u2019t make these trades \u2013 there are simply too many \u2013 but humans define the rules by which these machines operate.<\/p>\n<p>However, because the market keeps changing, these machines need to be adjusted constantly. That takes a lot of time and effort.<\/p>\n<p>AI can automate these recalibrations \u2013 and do a lot of the repetitive statistical work that analysts would otherwise need to do.<\/p>\n<h2 id=\"warning-signs--things-to-watch-out-for\">Warning signs \u2013 Things to watch out for<\/h2>\n<p>The promise of finding a miracle algorithm that can really print money is so alluring, lots of smart people have bought into it. Here are the most important traps to watch out for:<\/p>\n<h3 id=\"ai-today-is-not-smarter-than-human-beings\">AI today is NOT \u201csmarter\u201d than human beings<\/h3>\n<p>In truth, even today\u2019s most advanced AI algorithms are&nbsp;<strong>very naive compared to human brains<\/strong>.<\/p>\n<p>When an algorithm beats a human at chess or GO, it\u2019s like a car beating a human runner in a quarter-mile race: yes, the machine is faster, but that doesn\u2019t make it superior. It just means that we built a machine that can do a very narrowly defined task \u2013 under certain narrowly defined conditions \u2013 really well.<\/p>\n<p>It might seem like it at first, but trading is not such a narrowly defined task. Why? Because in trading you are competing with&nbsp;<strong>other humans<\/strong>&nbsp;\u2013 who will use all their brainpower trying to outsmart you.<\/p>\n<h3 id=\"published-trading-strategies-often-dont-work-in-real-life\">Published trading strategies often don\u2019t work in real life<\/h3>\n<p>There is a lot of research and lots of blog articles out there promising a profitable AI-based trading algorithm. But these models don\u2019t tend to work in real life, for several reasons.<\/p>\n<p><strong>Faulty setups<\/strong><\/p>\n<p>A surprising number of papers actually make mistakes in how they set up their training and testing framework. They use variables that wouldn\u2019t have been available at the time when the AI needed to make a decision (data leakage) or evaluate the predictions against the current price, not the future price. Surprising mistakes \u2013 but time-series datasets are a complicted thing for human brains to handle.<\/p>\n<p><strong>Selection bias<\/strong><\/p>\n<p>Fund managers are often critiqued for attributing their better returns to superior skill rather than luck. But if a lot of fund managers make a lot of random guesses, then in the end there will be some who made a few good guesses. The losers close down, and we never hear about them \u2013 so it looks like there are lots of fund managers who have the skills to beat the market.<\/p>\n<p>In reality though, the number of fund managers who beat the market is exactly in line with what you would expect based on random guesses.<\/p>\n<p><em>The same is true for research papers<\/em>. If you try a lot of algorithms, you\u2019ll eventually find one that seems to produce certain profits. If you don\u2019t tell everyone how many experiments you had to run to get there, it will seem like you just stumbled on a superior approach.<\/p>\n<p>However, there is absolutely no guarantee that this strategy will work outside the particular data you tested it on.<\/p>\n<p><strong>Transaction fees and slippage<\/strong><\/p>\n<p>Actually, building a trading strategy that outperforms the market is often quite simple \u2013 IF you forget about the real-world costs of doing trades. Transaction fees (the fees you pay for every trade) and slippage (the fact that the price might change between the time you make your order and the trade going through) eat up a lot of profit. And in almost every case, that\u2019s enough to delete the profit you saw in simulation.<\/p>\n<p><strong>Patterns change over time<\/strong><\/p>\n<p>One of the most important concepts in Machine Learning is finding patterns in past data and using them to make correct forecasts of the future.<\/p>\n<p>However, this doesn\u2019t work in trading. Other traders are competing to find the same patterns \u2013 so patterns get found, exploited, and then disappear. That means patterns rarely exist for long, and you have to constantly find new ones.<\/p>\n<p>This requires immense adaptability \u2013 something human beings are currently much better at than machines.<\/p>\n<h3 id=\"algorithms-alone-will-never-give-you-an-advantage\">Algorithms alone will never give you an advantage<\/h3>\n<p>It\u2019s easy to get carried away and focus on the algorithm as the main competitive advantage between one trading strategy and another.<\/p>\n<p>This is essentially what companies like&nbsp;<a href=\"https:\/\/numer.ai\/\" rel=\"noopener\">Numerai<\/a>&nbsp;are proposing:<\/p>\n<ol>\n<li>Combine lots of good models into a super model.<\/li>\n<li>Beat the stock market.<\/li>\n<\/ol>\n<p>But&nbsp;<strong>this does not work<\/strong>. Why? Because&nbsp;<strong>data beats algorithms<\/strong>. The data you&nbsp;<em>give to your algorithm<\/em>&nbsp;has a much larger effect on your model\u2019s performance than how good the algorithm is. The data Numerai gives you is fixed \u2013 you can\u2019t add to it. So the forecasts they make will always be worse than those of traders, who aren\u2019t limited in the data they can use \u2013 traders who have access to an open pool of data, who can try, test, and add new data points to their algorithms continuously.<\/p>\n<h3 id=\"a-bit-of-hope---inefficent-markets-might-still-be-fertile-ground\">A bit of hope &#8211; Inefficent markets might still be fertile ground<\/h3>\n<p>Markets with&nbsp;<strong>few trading participants, high barriers to entry, a limited trading volume<\/strong>, and few players able to use Machine Learning might offer some opportunity for pure-play AI trading success.<\/p>\n<p>In these markets, automated trading \u2013 especially the use of Machine Learning \u2013 is still just beginning, and traders who build automated trading engines could score enough of an edge to produce a good profit.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Trading is a gruesomely competitive world. And with AI being painted as the new wonder weapon for everything, it&rsquo;s understandable that there&rsquo;s a huge amount of interest in discovering how to use AI for trading. AI does play an important role in trading &ndash; but maybe not in the way you&rsquo;d expect. Unfortunately, AI can&rsquo;t be used to power a superhuman trading machine that steals human traders&rsquo; lunches in every market.&nbsp;Then, how AI helps traders make better decisions &amp; improve high-frequency trading?<\/p>\n","protected":false},"author":314,"featured_media":23514,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-post-2.php","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[92],"ppma_author":[2069],"class_list":["post-1046","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-machine-learning"],"authors":[{"term_id":2069,"user_id":314,"is_guest":0,"slug":"markus-schmitt","display_name":"Markus Schmitt","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Schmitt","first_name":"Markus","job_title":"","description":"Markus Schmitt is the founder and head of data science at Data Revenue, a Machine Learning Agency based in Berlin, Germany, where he builds custom end-to-end machine learning systems for Medical, Finance and Marketing clients. Before Data Revenue he developed new ventures for the company builder Team Europe and studied Mathematics &amp; Economics at Warwick."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1046","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/314"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1046"}],"version-history":[{"count":2,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1046\/revisions"}],"predecessor-version":[{"id":22036,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1046\/revisions\/22036"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/23514"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1046"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1046"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1046"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1046"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}