{"id":24845,"date":"2021-06-17T11:19:47","date_gmt":"2021-06-17T11:19:47","guid":{"rendered":"https:\/\/www.experfy.com\/blog\/?p=24845"},"modified":"2023-08-19T11:39:27","modified_gmt":"2023-08-19T11:39:27","slug":"duration-estimation-in-an-a-b-test","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/ai-ml\/duration-estimation-in-an-a-b-test\/","title":{"rendered":"Duration Estimation In An A\/B Test"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"24845\" class=\"elementor elementor-24845\" data-elementor-post-type=\"post\">\n\t\t\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-116d175 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"116d175\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-70c3ffb\" data-id=\"70c3ffb\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-1e8646c elementor-widget elementor-widget-text-editor\" data-id=\"1e8646c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>While running an experiment, waiting for data is often the most challenging period as you are likely to get impatient. All you want during that period is for the A\/B test to end as quickly as possible so you can go in a\u00a0 full-scale execution mode. And, the anxiety adds up when you don\u2019t know how long you need to wait for the test to reach statistical significance.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-13dd2e5 elementor-widget elementor-widget-text-editor\" data-id=\"13dd2e5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The impatience is entirely understandable as you do not want to lose conversions on suboptimal variations. Nothing much can be done about that anxiety as the statistical test will end when it ends. But, if you can have an estimated waiting time for the A\/B test to end, it could undoubtedly appease the anxiety to some extent.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-c3f331e elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"c3f331e\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-bd11e57\" data-id=\"bd11e57\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-ec69571 elementor-widget elementor-widget-heading\" data-id=\"ec69571\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Let me explain how to estimate the duration of an A\/B test:<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-bde18e8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"bde18e8\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-4d86d34\" data-id=\"4d86d34\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-a2889f8 elementor-widget elementor-widget-heading\" data-id=\"a2889f8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Visitor Sample Size Calculator<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-42186fe elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"42186fe\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d352803\" data-id=\"d352803\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-18e1e39 elementor-widget elementor-widget-text-editor\" data-id=\"18e1e39\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>In statistics, you can never say with 100% confidence that an A\/B test will end after X number of days. Instead, you say there is an 80% (or a 95%, whatever you choose) probability of getting a statistically significant result if it exists after X number of days.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6c81f1c elementor-widget elementor-widget-text-editor\" data-id=\"6c81f1c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>There could be the case when there is no difference in the performance of the variations, and no matter how long you wait, you will never get a statistically significant result. Thus, it becomes essential to estimate the number of visitors required to conduct an A\/B test for statistical significance before even running a test.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b12fb78 elementor-widget elementor-widget-text-editor\" data-id=\"b12fb78\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>There are three pieces of information you would need to determine the number of visitors for the A\/B test \u2013\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-da9567b elementor-widget elementor-widget-text-editor\" data-id=\"da9567b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n<li>A<strong> base Conversion Rate(CR)<\/strong> \u2013 value that you are expecting the campaign would get at the least.<\/li>\n<li><strong>Expected Uplift \u2013<\/strong> What percentage difference in CR you want to detect on the base CR (lower the uplift you wish to test, the more visitors it will need)\u00a0<\/li>\n<li><strong>Number of variations<\/strong> to test (the more variations you test, the more traffic you need)<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d1a2fe7 elementor-widget elementor-widget-text-editor\" data-id=\"d1a2fe7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The following image was taken from <em>Statistical Rules of Thumb,<\/em> by Gerald van Belle.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0332e8c elementor-widget elementor-widget-text-editor\" data-id=\"0332e8c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The formula described above is known as <strong>Lehr&#8217;s equation<\/strong>, which is obtained by using frequentist statistics.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7e16b6c elementor-widget elementor-widget-text-editor\" data-id=\"7e16b6c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n<li>Type I Error (\u03b1) is the probability of rejecting the null hypothesis when it is true (if \u03b1=0.05, then it means that out of 100 independent tests where variations are the same, 5 tests will say variations are statistically different)\u00a0<\/li>\n<li>Type II Error (\u03b2) is the probability of not rejecting the null hypothesis when it is false (if \u03b2=0.20, then it means that out of 100 independent tests where variations are different, 20 tests will say variations are the same)<\/li>\n<li>z is the Z-score value obtained from the Z-table. Visit <a href=\"https:\/\/www.investopedia.com\/terms\/z\/z-test.asp\" target=\"_blank\" rel=\"noreferrer noopener\">this<\/a> to know more about the Z-test.<\/li>\n<li>\u03c3 is the standard deviation of the visitor\u2019s Bernoulli distribution. Hence,<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d457592 elementor-widget elementor-widget-text-editor\" data-id=\"d457592\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>By putting the values in Lehr\u2019s equation, you\u2019ll get the number of visitors (n) needed to get statistically significant results between two variations.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1bdcf0e elementor-widget elementor-widget-text-editor\" data-id=\"1bdcf0e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>If there are multiple variations, then multiplying (n) with the number of variations (V) will give the overall number of visitors needed (n*V).\u00a0\u00a0\u00a0\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cbc4321 elementor-widget elementor-widget-text-editor\" data-id=\"cbc4321\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Divide the obtained result by the average number of daily visitors and you\u2019ll get the number of days the A\/B test is likely to take in order to find the best variation. You can use the calculator at <a href=\"https:\/\/vwo.com\/blog\/ab-test-duration-calculator\/\" target=\"_blank\" rel=\"noreferrer noopener\">ab-test-duration-calculator<\/a> built upon the same formula.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-4e0cbeb elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"4e0cbeb\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-316c460\" data-id=\"316c460\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e8bc31c elementor-widget elementor-widget-heading\" data-id=\"e8bc31c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Lehr's Equation\u2019s Mathematical Intuition<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-6c853a9 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6c853a9\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a9d8758\" data-id=\"a9d8758\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-2969159 elementor-widget elementor-widget-text-editor\" data-id=\"2969159\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>There are two key ingredients to sample size calculations: the difference between the two variations\u2019 conversion rates, and the variability in their measurements.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d40b8ed elementor-widget elementor-widget-text-editor\" data-id=\"d40b8ed\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>Each distribution in the above image is a model that represents the differences of conversion rates between two variations where the x-axis is the absolute difference scale of conversion rates (\u0394=y0\u2212y1).\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6792c46 elementor-widget elementor-widget-text-editor\" data-id=\"6792c46\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>One distribution\u2019s center is at 0, and the other\u2019s center is at \u03b4(\u03b4=CR*Uplift). The null hypothesis that there is no difference between the two variations is represented by the distribution on the left (\u0394=0). The alternative hypothesis that there is some difference between the two variations is represented by the right curve (\u0394=\u03b4). Each distribution also has a variance\u00a0 (\u03c3<sup>2<\/sup>), which is usually assumed to be the same for both.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3fb5e84 elementor-widget elementor-widget-text-editor\" data-id=\"3fb5e84\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The relationship between the standard error (SE), the absolute difference of conversion rates of the two variations, and the standard deviation of the distribution allows us to set up calculations for the sample size, n.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-db93c68 elementor-widget elementor-widget-text-editor\" data-id=\"db93c68\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>By multiplying SE with an appropriate z score, we add the confidence level we want in our estimation.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-085b9e0 elementor-widget elementor-widget-text-editor\" data-id=\"085b9e0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>The critical value is where the \u03b1 region of the null curve and the \u03b2 region of the alternative curve meet. This point is:\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f7840c4 elementor-widget elementor-widget-text-editor\" data-id=\"f7840c4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul>\n<li>distance away from the mean of the null curve, and<\/li>\n<li>\u00a0distance away from the mean of the alternate curve.\u00a0<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-290a711 elementor-widget elementor-widget-text-editor\" data-id=\"290a711\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>As the sum of the two distances is \u03b4, just by rearranging the resultant linear equation, you can get the number of visitors needed to obtain a statistical difference between the two variations. The errors in our tests would be <strong>\u03b1<\/strong> and <strong>\u03b2<\/strong>. The lesser the value of <strong>\u03b1<\/strong> and <strong>\u03b2<\/strong>, the more will be the visitor estimate. Thus, the equation we get is:<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f41798c elementor-widget elementor-widget-text-editor\" data-id=\"f41798c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>You can read this <a href=\"http:\/\/www.vanbelle.org\/chapters\/webchapter2.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">sample size chapter<\/a> to get a more in-depth understanding of how this equation is derived.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-ff6c156 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ff6c156\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-53940a8\" data-id=\"53940a8\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e5368c6 elementor-widget elementor-widget-heading\" data-id=\"e5368c6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Conclusion<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"has_eae_slider elementor-section elementor-top-section elementor-element elementor-element-5b687b6 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"5b687b6\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"has_eae_slider elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-2ecb5af\" data-id=\"2ecb5af\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4332307 elementor-widget elementor-widget-text-editor\" data-id=\"4332307\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>By using the method described above, you can estimate the time duration of the test needed to check statistical significance in a frequentist A\/B test. However, if you perform a test using <a href=\"http:\/\/www.experfy.com\/blog\/ai-ml\/frequentist-vs-bayesian-statistics-which-one-is-best\/\" target=\"_blank\" rel=\"noreferrer noopener\">bayesian statistics<\/a>, you can read the <a href=\"https:\/\/engineering.wingify.com\/posts\/maths-behind-bayesian-duration-calculator\/\" target=\"_blank\" rel=\"noreferrer noopener\">maths behind the bayesian duration calculator<\/a> in order to understand its implementation.<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0b7ed64 elementor-widget elementor-widget-text-editor\" data-id=\"0b7ed64\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>It is a common practice to perform sample size calculations before starting an experiment to avoid bias in results. If we include very few subjects in an experiment, the results cannot be generalized to the population as this sample will not represent the target population. On the other hand, if we study more subjects than required, we could waste resources. Adequate sample size calculation thus becomes crucial in any statistical experiment to arrive at scientifically valid results.\u00a0<\/p>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>While running an experiment, waiting for data is often the most challenging period as you are likely to get impatient. All you want during that period is for the A\/B test to end as quickly as possible so you can go in a\u00a0 full-scale execution mode. And, the anxiety adds up when you don\u2019t know<\/p>\n","protected":false},"author":1153,"featured_media":24847,"comment_status":"open","ping_status":"open","sticky":false,"template":"single-post-2.php","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[183],"tags":[94],"ppma_author":[3666],"class_list":["post-24845","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","tag-data-science"],"authors":[{"term_id":3666,"user_id":1153,"is_guest":0,"slug":"anshul-gupta","display_name":"Anshul Gupta","avatar_url":"https:\/\/www.experfy.com\/blog\/wp-content\/uploads\/2021\/05\/anshul_gupta-150x150.jpeg","user_url":"https:\/\/datascienceinsider.wordpress.com","last_name":"Gupta","first_name":"Anshul","job_title":"","description":"Anshul Gupta\u2019s expertise lies in building interpretable machine learning solutions. He works in the Data Science team at VWO where he handles the mathematical function of their Bayesian optimization platform."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/24845","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/1153"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=24845"}],"version-history":[{"count":6,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/24845\/revisions"}],"predecessor-version":[{"id":30643,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/24845\/revisions\/30643"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/24847"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=24845"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=24845"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=24845"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=24845"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}