{"id":1969,"date":"2019-09-24T02:22:43","date_gmt":"2019-09-24T02:22:43","guid":{"rendered":"http:\/\/kusuaks7\/?p=1574"},"modified":"2024-04-01T05:49:55","modified_gmt":"2024-04-01T05:49:55","slug":"big-data-in-the-financial-services-industry-from-data-to-insights","status":"publish","type":"post","link":"https:\/\/www.experfy.com\/blog\/bigdata-cloud\/big-data-in-the-financial-services-industry-from-data-to-insights\/","title":{"rendered":"Big Data in the Financial Services Industry – From data to insights"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

1. Introduction<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tJust as “Cloud”, “IoT” (Internet of Things), “Open Banking” and “Machine Learning”, “Big Data” is\u00a0one of the most written buzzwords in the financial services industry<\/strong>\u00a0today, but just like the other mentioned terms a clear definition is not so easy to provide.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tEspecially as “Big Data” is often used as a synonym for\u00a0customer analytics, real-time analytics or predictive analytics<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe general consensus is that “Big Data” is the collective term used for the contemporary methodologies and technologies used to collect, organize, process and analyse\u00a0large, diverse (structured and unstructured) and complex sets of data<\/strong>, while “customer \/ real-time \/ predictive analytics” mainly refers to specific types of\u00a0analyses done on these data sets to find patterns and create business value<\/strong>. However, since the ultimate business goal of Big Data is not the data itself, but to get business insights into the data, the analytics part of the chain is the most visible and important for a business user, which explains why the terms are often interchanged.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAccording to a study of IBM in 2015, it is estimated that every day we create 2.5 quintillion (1018) bytes of data and that 90% of the data in the world today has been created in the last 2 years.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nThese figures show that the\u00a0scale of Big Data has taken an exponential increase<\/strong>\u00a0in recent years and will continue to rise in the coming years, especially due to the further adoption of mobile technologies and IoT.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

2. Impact on the Financial Services sector<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tAs the financial services sector is probably\u00a0the most data-intensive sector in the global economy<\/strong>, the impact of Big Data on the sector is hard to overestimate.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tBanks have enormous amounts of customer data (i.e. deposits\/withdrawals at ATMs, purchases at point-of-sales, payments done online, customer profile data collected for KYC\u2026 ), but due to their silo, product-oriented organisations, they are\u00a0not very good in utilizing these rich data sets<\/strong>.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThis while the financial services industry has been\u00a0investing heavily for more than a decade in data collection and processing technologies<\/strong>\u00a0(such as data warehouses and Business Intelligence) and is one of the\u00a0forerunners in investments in Big Data technologies<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\nDue to the\u00a0increasing and changing customer expectations<\/strong>\u00a0and the\u00a0increased competition of Fintech players<\/strong>, the financial services sector can simply not permit itself to leave those huge amounts of data unexploited. Instead banks and insurers should leverage the existing (and new) data sets to maximize customer understanding and gain a competitive advantage.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tSeveral players in the market are already using Big Data techniques to\u00a0deliver compelling use cases<\/strong>, but many organisations are still lagging behind.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

3. Drivers<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tThe recent rise of Big Data is driven by several factors, which strengthen each other, resulting in an exponential increase of data and the need to derive value from it:\n
    \n \t
  • Change in customer behaviour and expectations<\/strong>:<\/li>\n \t
  • Customers are\u00a0interacting with their bank or insurer more and more digital<\/em>, meaning that personal interaction is reduced, but at the same time it is possible to collect in an automated way much more data about the customer (e.g. his browsing history, geo-location data on his mobile phone, exact timing of the interactions\u2026 ) than when he visits a branch. This data should be leveraged to compensate the reduced customer engagement, caused by the loss of personal interaction.<\/li>\n \t
  • Customers\u00a0use more and more social media<\/em>: where these media used to be limited to closed private circles of friends, customers now use these media more and more in their day-to-day live, e.g. to interact with companies. This means banks and insurers should interact more through these channels to offer services and to gain insights about their customers.<\/li>\n \t
  • Customers expect\u00a0more and more a high-quality, low-friction, around-the-clock, customer-centric experience across multiple channels<\/em>. In order to deliver such a personalised service, an in-depth holistic knowledge about the customer is required. This can only be achieved by leveraging all available customer data through Big Data techniques.<\/li>\n \t
  • Technological evolutions leading to larger amounts of input data<\/strong>:<\/li>\n \t
  • The\u00a0rise of IoT<\/em>\u00a0(Internet of Things) will further explode the amount of customer data, as it will result in new, continuous (even if customer is not interacting with the bank or insurer) streams of data.<\/li>\n \t
  • New\u00a0advanced authentication techniques<\/em>, such as biometric authentication and continuous authentication (e.g. mouse movements and keyboard rhythm or accelerometer and gyro sensor readings on mobile phone), will also considerably increase the amount of data to be processed in near real-time.<\/li>\n \t
  • The rise of\u00a0Open Architectures<\/em>\u00a0(Open APIs), allows banks and insurers to collect valuable data about their customers from data stored at competitors.<\/li>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
  • Competition of Fintech players using already Big Data techniques for new financial services<\/strong>. E.g. the recent success of the Fintech robo-advisors, offering automated digital investment advise using their gathered customer profile information, shows that Fintechs are already able to convert Big Data into new compelling customer services. Unless banks can deliver quickly similar services, they are likely to lose considerable business to these Fintech companies.<\/li>\n \t
  • Regulatory pressure<\/strong>: the recent tsunami of new regulations (Basel III, FRTB, MiFID II, AML\/KYC, FATCA\u2026 ) force banks to disclose more diverse data and more granular data to central banks and regulators. Furthermore, the fines associated when not complying to these regulations are climbing. This forces banks to collect more and more data in a controlled way, so that the necessary regulatory reporting can be generated automatically, but also that all data is available for ad-hoc inquiries of the regulators.<\/li>\n \t
  • Increased cyber-security<\/strong>: with fraud and financial crimes increasing, banks need to protect their most valuable asset, namely the “trust” that customer give to their bank. This increases the pressure to further secure the interaction channels and the customer data, through different security techniques. One of the most promising is\u00a0risk-based authentication<\/em>, in which a fraud-detection engine calculates a risk profile for each channel request, determining the required level of security (authentication). This fraud-detection engine uses customer analytics to identify irregularities in the user\u2019s behaviour.<\/li>\n \t
  • Pressure to reduce operational costs<\/strong>: due to the increased competition and low interest rates, profit margins in the financial services industry are dropping. Banks and insurers are forced therefore to reduce operational costs, by improving business efficiency. Many of these efficiency gains can be driven by the insights gained from Big Data.<\/li>\n \t
  • Technological evolutions to support the processing of huge amounts of complex and diverse data in real-time<\/strong>: with data sets growing so large and complex, traditional tools are no longer able to process this data at sufficiently low cost and in reasonable time. Luckily a set of new technologies provides an answer to this issue, allowing to process these data sets in near real-time and at lower costs, i.e.<\/li>\n \t
  • Event streaming<\/em>: streaming of large volumes of events in real-time.<\/li>\n \t
  • NoSQL databases<\/em>: databases which allow to store and retrieve data in a much more scalable and flexible way than the traditional relational databases (RDBMS).<\/li>\n \t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t
  • In-memory data stores<\/em>: data structure which resides entirely in RAM and is distributed among multiple servers.<\/li>\n \t
  • Distributed processing<\/em>\u00a0(= distributed computing): using a network of computers to split up a task in smaller tasks, which are executed in parallel over the different computers (often commodity hardware), after which the result is aggregated.<\/li>\n \t
  • Machine-learning<\/em>: give computers the ability to learn without explicitly being programmed.<\/li>\n \t
  • Advanced data visualization<\/em>: tools to assist users in visualizing in a user-friendly way the large amounts of data and the insights derived from it (e.g. through bubble charts, word clouds, geospatial heat maps\u2026 ).<\/li>\n \t
  • Cloud solutions<\/em>: cloud solutions offer a cheap and flexible (i.e. elastic scalability) infrastructure (but also higher-level services) to support these Big Data technologies.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t

    4. Characteristics of Big Data<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
    \n\t\t\t\t
    \n\t\t\t\t\t\t\t\t\tBig data is characterised by the 3 V\u2019s, i.e.\n
      \n \t
    • Volume<\/strong>: a vast quantity of data (i.e. terabytes or petabytes) to be handled. These huge amounts of data make it impossible to be processed by traditional data processing tools within reasonable time delays.<\/li>\n \t
    • Velocity<\/strong>: Big Data technologies should be able to process both batch and real-time data. For real-time data, quick analysis for (near) real-time insight generation can be a necessity for the business.<\/li>\n \t
    • Variety<\/strong>: multiple types of data should be supported, i.e. from highly structured data to unstructured info like text, video, audio, blogs, tweets, Facebook status updates\u2026<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      5. Use Cases<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\tThe financial services industry, being a data-driven industry, allows to define a multitude of use cases, where Big Data and Customer Analytics can bring added value.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\tIn this chapter, a few of these use cases are presented.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t

      5.1. Sales and Marketing<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\tMcKinsey research estimated that sales and marketing consume about 15 percent of the costs of financial service companies, meaning that improving the efficiency of these processes can lead to significant cost savings.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
      \n\t\t\t\t
      \n\t\t\t\t\t\t\t\t\tWhen looking at the customer lifecycle, we identify 3 stages, i.e.\n
        \n \t
      • Acquisition<\/strong>: the art of attracting new customers, by targeting prospects through campaigns.<\/li>\n \t
      • Activation<\/strong>: this stage includes re-engaging old customers or dormant customers to buy new products\/services, but also the art to maximize the first purchase of new customers.<\/li>\n \t
      • Relationship management (Cultivation)<\/strong>: cultivating the customer\u2019s relationship with the company.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t\t\tThese stages can be supported by Big Data, demonstrated by below use cases:\n
          \n \t
        • Improve the efficiency of acquisition through Big Data<\/strong>: mass marketing campaigns are costly and often ineffective, especially nowadays when customers are overloaded by marketing campaigns through different channels. Effective marketing campaigns should therefore target the right set of customers, with the right (personalized) message and through the right channel (direct mailing, email, channel advertising, social media, TV, radio\u2026 ). Big Data can provide an answer to this by:<\/li>\n \t
        • Segmenting prospects based on publicly available information about the prospects and insights gained from information from the existing customer base. This allows to select the right set of prospects for targeting a new product\/service (hyper-targeted marketing<\/em>).<\/li>\n \t
        • Determine the\u00a0best channel (mix) for the marketing campaign<\/em>, based on the gained insights in the selected prospect segment.<\/li>\n \t
        • Personalization of marketing messages<\/em>\u00a0(contact optimization)<\/li>\n \t
        • Monitor what customers say<\/em>, i.e. monitor social media and other information sources (e.g. call centre records) to get direct feedback on marketing campaigns, allowing to adjust ongoing campaigns or take lessons for future campaigns (and product design\/development).<\/li>\n \t
        • Identify influential customers<\/em>, i.e. identify and engage with influential customers (i.e. customers who have high impact on company brands or products) to boost word-of-mouth marketing.<\/li>\n \t
        • Improve the efficiency of activation through Big Data<\/strong>: once a prospect has replied to a campaign, it is important to maximize the first sales opportunity. At the same time, sales to existing customers should also be boosted. Big Data can also support those processes through:<\/li>\n \t
        • Segmentation of customers, based on the available data (e.g. customer profiling, analyzing transaction patterns, past and immediate customer behaviour\u2026 ) to get real-time customer insights. This allows to\u00a0predict the products or services customers are most likely to be interested in (i.e. predictive analysis)<\/strong>\u00a0for their next purchase, thus allowing to determine\u00a0next-best-offers<\/em>\u00a0(next-product-to-buy<\/em>) and what his most likely next action will be. These products can be specifically marketed to the customer and proactive offers can be generated.<\/li>\n \t
        • Bundle products<\/em>\u00a0based on the gained insights to boost cross-selling.<\/li>\n \t
        • Optimize pricing<\/em>, i.e. apply dynamic pricing, based on estimation of how much a customer is willing pay for the product or service.<\/li>\n \t
        • Generate\u00a0cross- and up-selling opportunities<\/strong>\u00a0based on customer insights and current customer behavior. These opportunities can result in notifications, call-backs or in specific pop-ups in the front-end channels, e.g.<\/li>\n \t
        • Customer has received a large inflow of cash: investment cross-selling opportunity<\/li>\n \t
        • Customer has less money on his current account than he requires based on the budget he has created in the Personal Finance Management module of the bank: credit line (overdraft) cross-selling opportunity<\/li>\n \t
        • Customer has been simulating car loans on the internet and steps into a branch: car loan cross-selling opportunity<\/li>\n \t
        • Customer has cancelled a demand in middle of the flow and steps into a branch: cross-selling opportunity for continuing the demand<\/li>\n \t
        • Customer has an expiring term deposit: term deposit reinvestment selling opportunity<\/li>\n \t
        • Customer arrives in foreign country (identified based on geo-location information): opportunity to authorize credit card for the country (if not the case) and to temporarily increase his credit limit (e.g. to pay hotel bill)<\/li>\n \t
        • Customer is performing home renovations (i.e. identified based on transaction information): renovation loan cross-selling opportunity<\/li>\n \t
        • Customer is buying a bond on the stock market: upselling opportunity for similar structured note (if customer insights show that customer would be open to this and has enough knowledge of the product).<\/li>\n \t
        • Customer is requesting consumer loan, but payment history identifies lot of home renovation expenses: upselling opportunity for home renovation loan<\/li>\n \t
        • Customer does not have a home yet and is currently located at a house for sale (based on geo-location information and public information of houses for sale): selling opportunity for mortgage<\/li>\n \t
        • Customer modifies certain customer information (e.g. change of address due to move\/relocation, change of civil status e.g. following a wedding): selling opportunities for loans (e.g. mortgage, car loan\u2026 ) or insurances (home insurance, car insurance\u2026 ).<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t

          5.2. Customer Cultivation<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t\t\t\t\tThis paragraph describes the use cases to optimize the\u00a0management of an existing customer relationship<\/strong>, i.e. the so-called customer cultivation.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t\t\t\t\tBig Data can also support this life-cycle step by:\n
            \n \t
          • Transforming the business to a customer-centric business<\/strong>: empower employees with a Single Customer View (i.e. 360\u00b0 holistic view of the customer), providing a broad, centralized view of the customer information, a full view of the history of inquiries and transactions (regardless of the interaction channel) and insights on the customer\u2019s family, business and bank employee relationships. These insights allow to have more focused and in-depth interactions with the customer.<\/li>\n \t
          • Identifying high-value and most profitable customers<\/strong>: identifying those customers allows to provide a premium service to these relationships, i.e. offer more attractive products and services, provide attractive pricing or get insights on how they behave, how they best should be reached and what motivates them to buy more.<\/li>\n \t
          • Enhancing the loyalty<\/strong>\u00a0of existing customers through:<\/li>\n \t
          • Targeted next-best-offers<\/li>\n \t
          • Loyalty programs, e.g. based on card usage habits<\/li>\n \t
          • Partnerships with retailers to send discount offers to cardholders, who use their card near the retailer\u2019s stores<\/li>\n \t
          • Retention management<\/strong>: detect customers with high risk of leaving (indicators can be e.g. cancellation of automatic payments, customer complaints in call centre calls or on social media\u2026 ) and provide retention offers to these customers.<\/li>\n \t
          • Adaptive channel interactions<\/strong>: based on the data of current and past customer behaviours, it is possible to predict future customer trends and what their most likely next action will be. Front-ends could use this information to show dynamic buttons\/menu items (i.e. dynamic screen adaptation), which put these next actions forward.<\/li>\n \t
          • Improve efficiency of products, services and channel interactions<\/strong>: monitor the customer journey, i.e. interactions of customers, to gain insights to improve existing channels, processes and products. Such improvements will also result in an improvement of customer service.<\/li>\n \t
          • Feedback of customers on social media can also be used for this.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t

            5.3. Risk Management<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t\t\t\t\tBig Data can help banks and insurers to significantly improve risk management, through improved and (more) real-time insights in the customer behavior.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t\t\t\t\tThis paragraph provides some examples per type of risk:\n
              \n \t
            • Cyber (identity) fraud detection and prevention<\/strong>: use Big Data to feed fraud-detection engines, allowing to continuously assess the risk of identity fraud and determine near real-time whether additional security measures (e.g. additional authentication techniques or restriction of access) are required.<\/li>\n \t
            • Liquidity risk management<\/strong>: get better insights on the incoming and outgoing cash flow, to optimize liquidity management. This technique can be useful for both physical money at branches as for the overall liquidity management of the bank\/insurer.<\/li>\n \t
            • Credit risk management<\/strong>: based on customer insights, improve the credit models for private and corporate customers, thus allowing to improve credit scoring. These insights can be derived from transaction history, public information (e.g. annual reports of companies), IoT data (e.g. inventory sensors, home sensors, car sensors\u2026 )\u2026 This data can also be used to better manage the collateral of credits, thus also reducing credit risk for the bank.<\/li>\n \t
            • Cards fraud detection<\/strong>: analyse the card transaction patterns (location, timing, amount, type of merchants\u2026 ) to identify frauduleus transactions, so that they can be blocked.<\/li>\n \t
            • Insurance fraud detection<\/strong>: improve detection and prevention of fraud at opening of a new insurance policy (i.e. policy data not matching with reality) and fraud when introducing claims. Several indicators can be used for this identification, e.g. IoT data (sensor data can give directly information about car accident or home damage), customer performing several simulation attempts with different configurations before demanding an insurance policy, a high number of view request to the insured amount of life insurances or fire\/theft insurances preceding a claim, unpaid premiums\u2026<\/li>\n \t
            • Legal Claim Management<\/strong>: Big Data can also help to better avoid, prepare for and react to legal matters involving large amounts of data, e.g. collect all information related to a legal matter, assessment of cases to determine probability that case will lead to a legal claim, tracking of regulations to avoid fines and sanctions\u2026<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t

              5.4. New Data Driven Products and Services<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t\t\t\t\tBig Data allows also to\u00a0deliver new, innovative products and services to customers<\/strong>, which use the insights derived from the data streams.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
              \n\t\t\t\t
              \n\t\t\t\t\t\t\t\t\tSome examples:\n
                \n \t
              • Home insurance in combination with IoT<\/strong>\u00a0(utilities smart meters, smoke and carbon monoxide detectors, fire suppression systems, advanced alarm systems) allowing to improve protection, dynamically adapt pricing and provide value-added services (e.g. statistics on utilities consumption).<\/li>\n \t
              • Car insurance in combination with IoT<\/strong>\u00a0(black-box in car) allowing to improve protection (e.g. car recovery in case of theft), dynamically adapt pricing (based on driving style) and provide value-added services (e.g. fleet management services to SMEs).<\/li>\n \t
              • Trade Finance contract in combination with IoT<\/strong>\u00a0(supply chain sensors) allowing the automatic execution of the contractual conditions defined in the contract.<\/li>\n \t
              • Personalized Wealth Management Advise<\/strong>: use Big Data to identify customer goals, family situation, risk aversity, financial situation, financial goals\u2026 and propose automatically investment advise, tax advise and financial planning based on these insights.<\/li>\n \t
              • Personal Financial Management<\/strong>: use Big Data to automatically classify financial transactions in categories, which are best suited to the type of customer, propose budget plans in line with customer\u2019s goals and compare budget plans and actuals with “people-like-me”.<\/li>\n \t
              • Algorithmic trading<\/strong>: analyse massive amounts of market data in fractions of a second to identify investment opportunities<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t

                5.5. Internal Management Support<\/h3><\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\tWhere all use cases up till now are all focusing on getting better insights about the customer and consequently better servicing these customers and generating more profits, this paragraph focusses on using Big Data to\u00a0support internal management decisions<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                \n\t\t\t\t
                \n\t\t\t\t\t\t\t\t\tSome examples:\n
                  \n \t
                • Examine customer feedback<\/strong>: collect and analyse customer feedback from different sources (e.g. call centre comments, social media\u2026 ) to identify improvements to products and services. Using these techniques allows much faster reactivity to this feedback than traditional surveys or focus groups, which tend to be slow, costly and inaccurate (due to the limited size of the sample group).<\/li>\n \t
                • Determine branch location\/relocation strategy<\/strong>: use Big Data to understand where customers live, where they shop and how much they spend to determine optimal locations of branches.<\/li>\n \t
                • Big Data can also be used to\u00a0better comply with regulations and improve regulatory reporting<\/strong>. Some examples of regulations which are very data driven are KYC\/AML, FATCA, MiFID2 and Basel III. Getting better insights into the customer will allow banks to reduce the risk of sanctions and fines, due to regulation breaches.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                  \n\t\t\t\t
                  \n\t\t\t\t\t

                  6. Benefits<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                  \n\t\t\t\t
                  \n\t\t\t\t\t\t\t\t\tAs showed above, Big Data can support multiple use cases which will bring\u00a0significant benefits<\/strong>\u00a0for banks or insurers:\n
                    \n \t
                  • Drive growth<\/strong>\u00a0in the business<\/li>\n \t
                  • Drive better risk management<\/strong>\u00a0(better identification, assessment, prevention and mitigation of risks)<\/li>\n \t
                  • Reduce and better control costs<\/strong><\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                    \n\t\t\t\t
                    \n\t\t\t\t\t\t\t\t\tThrough:\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                    \n\t\t\t\t
                    \n\t\t\t\t\t\t\t\t\t
                      \n \t
                    • More\u00a0personalized and targeted marketing<\/em>\u00a0(maximizing lead generation potential)<\/li>\n \t
                    • Improved\u00a0measurement of marketing effectiveness<\/em>\u00a0across all channels<\/li>\n \t
                    • Optimized funnel conversion<\/em><\/li>\n \t
                    • More\u00a0personalized customer servicing<\/em>\u00a0(relevant content per channel, dynamic pricing, next-best-offer\u2026 )<\/li>\n \t
                    • Faster reactivity<\/em>\u00a0(seconds rather than hours) to customer related transaction issues<\/li>\n \t
                    • Holistic and forward-looking view<\/em>\u00a0(predictive analysis) of customers<\/li>\n \t
                    • Increased\u00a0customer loyalty<\/em>\u00a0(better servicing, loyalty programs, knowing which customers are going to churn and when)<\/li>\n \t
                    • Enhanced usability<\/em>\u00a0(by dynamic adaptive front-ends and better monitoring and feedback cycles on usability)<\/li>\n \t
                    • Identify\u00a0hidden connection between seemingly unrelated data<\/em><\/li>\n \t
                    • Identify\u00a0customer trends<\/em>\u00a0and\u00a0changing customer preferences and expectations<\/em>\u00a0faster than competitors<\/li>\n \t
                    • Guiding customers to low cost channels<\/em><\/li>\n \t
                    • Reducing time lost searching information<\/em><\/li>\n \t
                    • Supporting\u00a0optimal management decisions<\/em><\/li>\n \t
                    • Improved\u00a0modelling of credit scoring and fraud detection<\/em><\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                      \n\t\t\t\t
                      \n\t\t\t\t\t

                      7. Challenges<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                      \n\t\t\t\t
                      \n\t\t\t\t\t\t\t\t\tDelivering these Big Data enabled use cases and reaping the benefits from them poses\u00a0significant challenges<\/strong>\u00a0for banks and insurers:\n
                        \n \t
                      • Data sits very often in\u00a0many silos<\/strong>\u00a0(i.e. large, monolithic legacy systems), making it difficult to centralize data. Many financial service companies have already invested in extracting specific data from these silos towards data warehouses (often in batch), but Big Data ideally requires more data and a faster (near real-time) delivery. Therefore, significant data integration efforts are expected.<\/li>\n \t
                      • Current technologies are not able to\u00a0handle the high-volume, high-velocity, high-variety data sets<\/strong>\u00a0and analyse these data sets in a timely manner. Banks and insurers will therefore have to invest in new Big Data technologies to support the new use cases. These investments will require application selections (several Big Data solutions are already available on the market) and a good understanding of the new technologies and the specific complexities associated with distributed systems.<\/li>\n \t
                      • Big Data consists of a\u00a0lot of unstructured content<\/strong>, which makes it difficult to interpret. Furthermore, given that Big Data can identify hidden connections between seemingly unrelated data, it is\u00a0difficult to know in advance which Big Data will bring the biggest business value<\/strong>.<\/li>\n \t
                      • Introducing Big Data requires a\u00a0cultural shift<\/strong>. Instead of considering data as an IT asset, the ownership of data should be moved to the business users, making data a key asset for decision making. This means that a\u00a0closer cooperation between business and IT<\/strong>\u00a0is required and the\u00a0organisational product-silos should be torn down<\/strong>.<\/li>\n \t
                      • These new Big Data technologies and methodologies result in new profiles such as data scientists and Big Data IT specialists. As all companies start investing heavily in these processes, a\u00a0war for talent<\/strong>\u00a0is expected.<\/li>\n \t
                      • Banks and insurers should consider the\u00a0restrictions of regulatory requirements and privacy concerns<\/strong>. For example, the European Global Data Protection Regulation (GDPR<\/strong>) imposes companies to store only personal data when there is a direct use for it. Furthermore, a customer should be in full control of his data, meaning he should be able to request at any moment to see or modify his personal data and the company should be able to explain for all stored personal data why it is stored. This makes Big Data use cases certainly a grey zone for breaching this regulatory law.<\/li>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t\t\t\t\t
                      • Regulations also enforce more and more transparency on the algorithms used to e.g. calculate credit acceptance and pricing, insurance acceptance and pricing or algorithmic trading (cfr.\u00a0MiFID2<\/strong>\u00a0regulation).<\/li>\n \t
                      • Data Quality<\/strong>: already today the principle “garbage in – garbage out” applies, when dealing with data quality. This statement is however exponentially enforced by the sheer volume of data, the real-time nature (meaning that cleansing actions are not possible) of the processing and the fact that Big Data leverages often third-party, publicly available sources (for which bank or insurer cannot manage the reliability).<\/li>\n \t
                      • Security<\/strong>: due to the centralization of customer data out of the well-protected silos, data security risks are amplified. A reliable data security policy (encryption, access restrictions\u2026 ) is therefore also a real challenge.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t

                        8. Data Sources<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t\t\t\t\tBig Data will leverage a\u00a0multitude of internal and external data sources<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                        \n\t\t\t\t
                        \n\t\t\t\t\t\t\t\t\tSome examples are:\n
                          \n \t
                        • Structured data<\/strong>\u00a0present in the company\u2019s databases, like<\/li>\n \t
                        • CRM<\/em>\u00a0information (e.g. KYC\/AML information)<\/li>\n \t
                        • Product silo information<\/em>\u00a0(e.g. credits, accounts, payments, securities, insurances\u2026 )<\/li>\n \t
                        • Security information<\/em>\u00a0(e.g. authentication, authorization and signature data)<\/li>\n \t
                        • Structured data gathered by\u00a0new compelling services<\/em>, like PFM (Personal Financial Management), Digital Investment Advise, IoT\u2026 This consists of information like budget plans, categorization of expenses, saving goals, investment profiling questionnaires, financial plans, IoT sensor data (like information from car telematics, wearables, home sensors, geolocation data\u2026 )\u2026<\/li>\n \t
                        • Structured and unstructured\u00a0data transiting through the organisation, but often not stored<\/strong>, like<\/li>\n \t
                        • Unstructured communication<\/em>\u00a0like emails, chat sessions, voice and video recordings (e.g. between branch or contact center employees and customers). This also includes unstructured summaries of communication, like call logs or comments written by call centre employees.<\/li>\n \t
                        • Customer behavior on websites<\/em>, i.e. browsing patterns.<\/li>\n \t
                        • Meta-data on channel interactions<\/em>, like browser name and version, IP address, customer operating system, cookie data, URL redirects, geo-location data\u2026<\/li>\n \t
                        • Customer surveys<\/em><\/li>\n \t
                        • Information gathered from\u00a0external partners<\/strong>, like<\/li>\n \t
                        • Social media data<\/em>, i.e. feedback, likes, (re-)tweets, shares, opinions, feeling and attitudes about the company brand, topic or keyword\u2026<\/li>\n \t
                        • Blogging<\/em><\/li>\n \t
                        • Data gathered from\u00a0API ecosystems<\/em>, e.g. data collected by partners, which deliver services on top of APIs exposed by the financial services company or data collected from APIs publicly exposed by competitors (e.g. PSD2 account information stored by competing banks)<\/li>\n \t
                        • Financial Market news, analyst reports, securities data<\/em>\u2026<\/li>\n \t
                        • Publicly available information<\/strong>, like<\/li>\n \t
                        • Demographic data<\/em><\/li>\n \t
                        • Financial data<\/em>\u00a0about companies (like annual reports)<\/li>\n \t
                        • Media<\/em>\u00a0feeds (news agencies, newspapers\u2026 )<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t

                          9. Technological Evolution<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tAs indicated in the first chapter of this article, Customer Analytics has been around for more than a decade. Big Data and Real-Time Analytics should therefore not be considered as a revolution, but rather as an evolution of technologies and processes which have been around for several years.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tIn this paragraph, we give a short overview of this\u00a0evolution<\/strong>. Since financial service companies have already deployed several of these “older” technologies, it is essential that the analytical solutions from different generations work effectively together, rather than having to perform a big-bang replacement of the existing data architecture.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tSeveral financial service companies have been\u00a0aggregating data from different channels and silos<\/strong>, by creating\u00a0operational data stores (ODS)<\/strong>\u00a0and\u00a0data warehouses (DWH)<\/strong>, which are based on relational databases.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tAfterwards these companies created\u00a0OLAP (On-Line Analytical Processing) cubes<\/strong>\u00a0(i.e. multi-dimensional data tables) on these data warehouses or used\u00a0Business Intelligence (BI) tools<\/strong>\u00a0like Business Objects to slice and dice the data, to calculate KPIs and to better understand customer behavior. More recently these BI tools have also evolved towards\u00a0more statistical and mathematical analysis<\/strong>, like e.g. SAS.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tWith data volumes increasing, data streams become more and more real-time and complex (structured and unstructured), these\u00a0data warehouses and business intelligence architectures are no longer meeting the business needs of today<\/strong>. Big Data technologies provide an answer to these challenges, by using new distributed technology stacks, like Hadoop-MapReduce and technologies to analyse unstructured data, like text analytics, natural-language processing\u2026\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tThese Big Data technologies require also new ways of storing the data, which gave rise to\u00a0Operational Data Lakes (ODL)<\/strong>. These data lakes store all data (i.e. data in use today, but also data that may be used someday), both structured and unstructured, in their raw unprocessed form (allowing easy adaptation to change). This in contrast to data warehouses, which only store the data required by the business and store this data specifically structured for flexible querying and intuitive viewing and analysis (i.e. data is denormalized with a lot redundancy). Operational Data Lakes form a modern replacement for Operational Data Stores (ODS), supporting both structured as unstructured data and handling much larger volumes of data. Often companies consider the replacement of an ODS with an ODL as a good first step towards a Big Data architecture.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tThe combination of these new technologies, with the current technology stack, is called a\u00a0Big Data Warehouse (BDW)<\/strong>, which is a hybrid data warehouse architecture. The structured data is still using the conventional DWH setup, but for the unstructured data a more modern ODL approach is taken. Big Data technologies can be setup in such a way that they can aggregate and analyse data from both sources.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tFinally, there is also a trend from batch-oriented replication of data on which analytics are executed, towards online real-time streaming of data, on which\u00a0(near) real-time analytics<\/strong>\u00a0are performed.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t

                          10. Big Data Processes<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                          \n\t\t\t\t
                          \n\t\t\t\t\t\t\t\t\tFrom collecting data all the way to getting business added-value results (i.e. discovering useful information, suggest conclusions and support decision-making) is a complex process, requiring several steps to go executed. High level these steps can be split in 2 high level blocks, i.e. Data Management and Data Analytics.\n
                            \n \t
                          • Data Management<\/strong>: this includes all processes required to prepare the data for analysis:<\/li>\n \t
                          • Data collection<\/strong>\u00a0(i.e. the data source layer): gather from different internal and external sources (data scraping) the raw data.<\/li>\n \t
                          • Data storage<\/strong>\u00a0(i.e. the data storage layer): storage and\/or staging (i.e. temporary storage of the data) of the data for further processing.<\/li>\n \t
                          • Data manipulation<\/strong>\u00a0(i.e. the data processing layer): prepare the data to perform analysis more efficiently:<\/li>\n \t
                          • Data filtering<\/em>: filtering out of irrelevant data<\/li>\n \t
                          • Data cleansing<\/em>: improving quality of the dataset by filtering out or correcting incorrect data samples<\/li>\n \t
                          • Data transformation<\/em>: transforming the data to a different format, applying simple business rules to the data\u2026<\/li>\n \t
                          • Data aggregation<\/em>: generate synthetic data at different levels of granularity (i.e. grouping at different levels)<\/li>\n \t
                          • Data join or merge<\/em>: joining \/ merging different data together<\/li>\n \t
                          • Data sorting<\/em>: sorting the data in a pre-defined order (e.g. alphabetically)<\/li>\n \t
                          • Data comparison<\/em>: compare data with previous (old) datasets to identify the differences<\/li>\n \t
                          • Data calculations<\/em>: execute calculations on the data, like averages, sums\u2026<\/li>\n \t
                          • Data Analytics<\/strong>: this includes all processes, where new insights are gained from the data. This includes 2 sub-steps, i.e.<\/li>\n \t
                          • Modelling and analysis<\/strong>\u00a0(i.e. the data analytics layer): there is a huge variety of techniques to model and analyse data and often techniques are combined to get the best result. In the different analytic techniques, different groups can be identified:<\/li>\n \t
                          • Unstructured data to structured data conversion<\/em>: these techniques transform unstructured data to structured data, on which other Big Data techniques can be applied. This includes “Text Analytics”, “Picture Analytics”, Audio Analytics” and “Video Analytics”.<\/li>\n \t
                          • (Descriptive) Data Mining<\/em>: this refers to techniques using algorithms to discover hidden patterns, relationships, dependencies and unusual records or dependencies.<\/li>\n \t
                          • Predictive analytics<\/em>: a variety of techniques to make predictions (determine likelihood of future events, i.e. future trends or likely behavior) from historical and current data patterns. Often based on time-series analysis, this type of analysis is typically used for determining the “next-best-offer” and implementing adaptive user interfaces.<\/li>\n \t
                          • Machine learning<\/em>: this group of techniques consists of applying one of the above techniques but adding the element of automated learning to it. This means the analytic technique will learn itself to provide better insights into the data, i.e. the model compares expected outcome with the real outcome and adapts accordingly to better align for future predictions.<\/li>\n \t
                          • Social network analysis<\/em>: this group of techniques will typically use a combination of above techniques, but due to their widespread usage, it is often considered as a separate group. It allows to represent, analyse and extract patterns and trends from social media data. A typical example is sentiment analysis, which aims to derive conclusions from the subjective information of the customer sentiment.<\/li>\n \t
                          • Data visualization<\/strong>\u00a0(i.e. the data output layer): visualize the data and gained insights with different visualization methods like charts, graphs, decision trees, traffic light indicators, heatmaps\u2026 aggregated in dashboards and reports. It also includes the representation of the analytical model itself and reports needed for model monitoring, benchmarking and back testing.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                            \n\t\t\t\t
                            \n\t\t\t\t\t\t\t\t\tWhen looking more in detail to “Modelling and Analysis”, we can identify various techniques and algorithms to achieve different types of modelling and analyses:\n
                              \n \t
                            • Text Analytics<\/strong>\u00a0(also referred to as text data mining): this refers to several algorithms to derive conclusions from unstructured text. It includes parsing the input text, adding derived linguistic features to it and then deriving patterns from this structured data.<\/li>\n \t
                            • Example usage<\/em>: measure customer opinions, product reviews, feedback<\/li>\n \t
                            • Anomaly or Outlier Detection<\/strong>: search for data items (i.e. outliers) in a dataset that do not match a projected pattern or expected behaviour (i.e. significantly deviate from the general average).<\/li>\n \t
                            • Example usage<\/em>: fraud detection engines<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tAssociation Rule Learning<\/strong>: search for interesting relations (interdependencies) between different variables in large databases.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tExample usage<\/em>: identification of cross-selling opportunities\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tClustering Analysis<\/strong>: discover data sets that are alike to understand the differences as well as the similarities within the data (i.e. finding meaningful patterns within a data set).\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tExample usage<\/em>: targeted marketing, identifying “people-like-me”\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\t\nClassification Analysis<\/strong>: task of applying a known structure (classification) to new data. Often implemented using decision trees and decision rules.\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tExample usage<\/em>: credit risk engine\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tRegression Analysis<\/strong>: tries to define the dependency between variables or even a function which models the data with the least error.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                              \n\t\t\t\t
                              \n\t\t\t\t\t\t\t\t\tExample usage<\/em>: algorithmic trading (i.e. correlations between stock prices) or churn prediction\n
                                \n \t
                              • A\/B testing<\/strong>: technique in which a control group is compared with one or more test groups to determine which changes improve a given objective variable, e.g. marketing response rate.<\/li>\n \t
                              • This technique is often used when deploying new software versions, i.e. user population is split in 2 parts, i.e. 1 group receives the new software version, while the other group remains on the current version. This way the impact of the version change can be measured and analysed.<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                \n\t\t\t\t
                                \n\t\t\t\t\t

                                11. Big Data Technologies<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                \n\t\t\t\t
                                \n\t\t\t\t\t\t\t\t\tAs mentioned in previous paragraphs, Big Data requires several\u00a0new technologies to deal with the volume, velocity and variety of the Big Data sets<\/strong>.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                \n\t\t\t\t
                                \n\t\t\t\t\t\t\t\t\tThese technologies are typically based on the\u00a0Hadoop open source framework\/platform<\/strong>. This technology stack allows to distribute computing problems across a (large) number of (commodity hardware) servers, thus enabling massive parallel processing. This permits very fast and scalable processing and eliminates dependency on expensive hardware (making the process much more economical).\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                \n\t\t\t\t
                                \n\t\t\t\t\t\t\t\t\tThe\u00a0Apache Hadoop-oriented technology stack<\/strong>\u00a0consists of multiple elements:\n
                                  \n \t
                                • HDFS<\/strong>: Hadoop Distributed File System, enables the storing of large files by distributing the data among a pool of data nodes.<\/li>\n \t
                                • MapReduce<\/strong>: engine for the simultaneous processing of data across multiple nodes<\/li>\n \t
                                • YARN<\/strong>: Yet Another Resource Negotiator, i.e. the cluster-coordinating component of the Hadoop stack<\/li>\n \t
                                • Spark<\/strong>: Big Data framework, like Hadoop, which is more modern and significantly faster than Hadoop-MapReduce. Where Hadoop-MapReduce is mainly batch-oriented, Spark provides both batch processing and online stream processing. As Spark does not have a file system, it is typically installed on top of Hadoop, where Spark replaces MapReduce, but makes use of HDFS as file system. Spark can however also be deployed standalone, when paired with a storage layer (e.g. an in-memory data grid like Apache Ignite)<\/li>\n
                                • Pig<\/strong>: high level programming language that simplifies the common tasks of working with Hadoop, i.e. loading data, data transformation and storing the results<\/li>\n \t
                                • Hive<\/strong>: enables Hadoop to operate as a data warehouse, i.e. provides an SQL like interface and relational model<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                  \n\t\t\t\t
                                  \n\t\t\t\t\t\t\t\t\tOther options for Big Data processing (often used on top or in combination with Hadoop) are:\n
                                    \n \t
                                  • Storm<\/strong>: Big Data framework, specifically oriented to stream-only workloads.<\/li>\n \t
                                  • Samza<\/strong>: Big Data framework, like Storm, specifically oriented to stream-only workloads.<\/li>\n \t
                                  • Flink<\/strong>: Big Data framework, like Spark, which allows both batch-oriented and online streaming processing.<\/li>\n \t
                                  • HBase<\/strong>: distributed, versioned, non-relational database modeled after Google\u2019s Bigtable (a NoSQL database running on top of the Hadoop cluster)<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                    \n\t\t\t\t
                                    \n\t\t\t\t\t\t\t\t\tAll those open source applications are\u00a0available for free<\/strong>, but to simplify usage and provide support,\u00a0enterprise versions like Cloudera, MapR and Hortonworks exist<\/strong>. But even then, this Hadoop stack is\u00a0not an out-of-the-box solution<\/strong>\u00a0and requires extensive configuration and programming to perform the analytics desired by the financial service company.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                    \n\t\t\t\t
                                    \n\t\t\t\t\t\t\t\t\tApart from these open source frameworks, which are generic toolkits, allowing to define any analytics for any industry, some companies have also started to create solutions (using the Hadoop technology stack in the back) to implement the specific customer analytic techniques in the financial services industry. E.g. NG Data delivers Lily Enterprise, which is a solution to collect data about the customer (i.e. Customer DNA) from different sources and generates insights from analytics. This solution is built on top of the Hadoop platform.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                    \n\t\t\t\t
                                    \n\t\t\t\t\t\t\t\t\tThe different cloud providers also provide several offers for Big Data analysis:\n
                                      \n \t
                                    • Amazon (AWS): Amazon EMR, Amazon Redshift, Amazon Kinesis\u2026<\/li>\n \t
                                    • Google (GCP): Cloud Pub\/Sub, Cloud Data Transfer, Cloud Dataflow, Cloud Dataproc, BigQuery\u2026<\/li>\n \t
                                    • Microsoft (Azure): HDInsight, APS (Analytics Platform System) \u2026<\/li>\n<\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                      \n\t\t\t\t
                                      \n\t\t\t\t\t

                                      12. Conclusion<\/h2><\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                      \n\t\t\t\t
                                      \n\t\t\t\t\t\t\t\t\tBanks have access to enormous amounts of data about their customers, but due to multiple constraints this data is not yet sufficiently converted into useful insights.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
                                      \n\t\t\t\t
                                      \n\t\t\t\t\t\t\t\t\tWith competition in the financial services sector getting fiercer, banks need to adopt a data-driven approach if they want to stay competitive. As opportunities for incumbent banks and insurers from these insights are almost unlimited, Big Data will be a strong differentiator in the future competitiveness of financial institutions.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"

                                      1. Introduction Just as “Cloud”, “IoT” (Internet of Things), “Open Banking” and “Machine Learning”, “Big Data” is\u00a0one of the most written buzzwords in the financial services industry\u00a0today, but just like the other mentioned terms a clear definition is not so easy to provide.Especially as “Big Data” is often used as a synonym for\u00a0customer analytics, real-time<\/p>\n","protected":false},"author":645,"featured_media":4028,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[187],"tags":[95],"ppma_author":[3386],"class_list":["post-1969","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bigdata-cloud","tag-big-data-amp-technology"],"authors":[{"term_id":3386,"user_id":645,"is_guest":0,"slug":"joris-lochy","display_name":"Joris Lochy","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/?s=96&d=mm&r=g","user_url":"","last_name":"Lochy","first_name":"Joris","job_title":"","description":"Joris Lochy is Product Manager & Business\/Functional Analyst at The Glue, a contemporary, new solution which enables financial institutions to quickly develop innovative financial services."}],"_links":{"self":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1969","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/users\/645"}],"replies":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/comments?post=1969"}],"version-history":[{"count":4,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1969\/revisions"}],"predecessor-version":[{"id":36528,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/posts\/1969\/revisions\/36528"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media\/4028"}],"wp:attachment":[{"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/media?parent=1969"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/categories?post=1969"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/tags?post=1969"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.experfy.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=1969"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}