Big Data, Cloud & DevOps

Personalization remains a tough spot in the rush for better customer experiences. However, there’s little doubt about the benefits it can bring. It’s just a question of how. For many businesses, getting a significant number of existing customers to make another purchase in the next month is little more than a pipe dream. For the many different organizations looking for a slice of the pie, it’s a highly lucrative offer – and the services offering the best personalization will likely be the ones that come out on top.  

Many machine learning practitioners may have come to a crossroads where they have to choose the number of features they want to include in their model. This decision is actually dependent on many factors. Machine Learning practitioners often work with high dimensional vectors to create a model. Here is an illustration of the Curse of Dimensionality, a powerful concept where the volume of the vector space grows exponentially with the dimension, causing our data to be highly sparse and distant from one another.

In computer science, divide and conquer is an algorithm design paradigm based on multi-branched recursion. A divide and conquer algorithm works by recursively breaking down a problem into two or more sub-problems of the same or related type, until these become simple enough to be solved directly. A typical Divide and Conquer algorithm solves a problem using following three steps: Divide: Break the given problem into sub-problems of same type. Conquer: Recursively solve these sub-problems. Combine: Appropriately combine the answers.

As a data scientist, one of your most important skills should be choosing the right modeling techniques and algorithms for your problems. You need a model that will use a deeper semantic understanding of the documents. Deep learning with small data is still in its early stages as a research field but it looks like it’s gaining more popularity especially with pre-trained language models and hope that researchers and practitioners will find more methods that will make deep learning valuable to every dataset.

Many people make the mistake of trying to compensate for their ugly dataset by improving their model. This is the equivalent of buying a super car because your old car doesn’t perform well with cheap gasoline. It makes much more sense to refine the oil instead of upgrading the car. This article explains how you can easily improve your results by enhancing your dataset, with the task of image classification as an example, but these tips can be applied to all sorts of datasets.

A data scientist is not ‘only’ a physician or a mathematician who knows how to implement code in Python. He or she, develops his/her abilities use case by use case, leveraging best practices but often exploring new ways to approach old problems, combining different learning techniques or chaining different classes of algorithms to optimize data to improve prediction quality and performances or to overcame previously unseen obstacles and challenges. And similarly to their alchemist ancestors who, in their stride of transforming rocks into gold, paved the way to modern chemistry, our modern data scientists, in their effort of extracting gold out of data, are laying the foundations for future generations of AI.

Collaborative Data Preparation and Data Analytics will combine business users’ need for agility and data access for analytical purposes with IT’s desire for with governed, secure processes to manage data usage. Enterprise, team-based Data Preparation and Analytics will give companies the agile, intelligent infrastructure for true data-driven decision-making. Firms are already implementing integral parts of this new approach to Data Preparation and Analytics. Enterprises create an intelligent Data Management platform that will ensure positive business results and operational processes.

Goal-setting is arguably the most important step to start any project. While research isn’t clear on the benefits of proper goal-setting, we can deduct the advantages and disadvantages. If we fail to state a clear goal, co-workers cannot collaborate, actions are not aligned and we don’t know if we’ve reached the goal. In short, havoc looms. Accordingly, every Data Science project aims to fulfill a goal. The breadth of goals might vary from researching a new model to creating a prototype for improving an existing system.

In an algorithm design there is no one ‘silver bullet’ that is a cure for all computation problems. Different problems require the use of different kinds of techniques. A good programmer uses all these techniques based on the type of problem. In this blog post, I am going to cover 2 fundamental algorithm design principles: greedy algorithms and dynamic programming. A greedy algorithm always makes the choice that seems to be the best at that moment. The core idea of Dynamic Programming is to avoid repeated work by remembering partial results and this concept finds it application in a lot of real life situations.

Big data and machine learning are part of a single architecture, a powerful duo that together can protect against even the most complex threats. A strong cyber security platform requires an inbuilt data management platform that collects and organizes big data, in combination with machine learning algorithms that analyze this data, respond to threats, and prevent against new attacks. Without big data analytics and machine learning, it would be impossible for security professionals to gather and organize the heaps of security events and to interpret all potential threats. 

A more refined framework is needed to provide a richer common lexicon for thinking and communicating about data in machine learning. A framework along the lines of the one  in this article should lead practitioners, especially newer practitioners, to develop better models faster. With 7 Data Types to reference we should all be able to more quickly evaluate and discuss the encoding options and imputation strategies available. Hope that this article will provide a useful taxonomy of groups that for more actionable steps for data scientists.