R package for media data validation and cleaning engine

Project Description

Every day we are receiving media data featuring several media metrics that has a business logic that needs to be upheld. Some of this business metrics is easy to uphold. Others might be more tricky. At Blackwood Seven we rely on massive quantities of data and the correctness of this data is naturally crucial. 

The following needs to be understood and completely grasped. 

• Online media metrics
• Offline media metrics
• CPM, CPC, CPA
• ROI
• Marginal cost analysis
• Time domain filtering

For programming

• R
• R-Studio
• Python
• R6 Classes

Specifically the R package that needs to be developed should handle the sanity checks of data as well as methods for fixing issues on a best effort. The approach could be to use Deep learning on generated sane and insane data. The package needs to be structured and built using R6 classes. Worst case S3 can be used. Alternatively all of it can be implemented in python with an interface to R. 

Sample sane and insane data are provided as CSV's which is of course not enough to train a network but just to reveal some of the potential issues.

Explanation of the data

The data set here consists of Impressions, Clicks and Net as metrics. As dimensions we have Date, Channel and Supplier. 

• Impressions: The number of times a banner has been shown to a user
• Clicks: The number of Impressions that users clicked on. Thus the following MUST be true always Clicks < Impressions
• Net: The amount of money paid for the banner which is usually reconciled as CPC=Cost per Click or CPM=Cost per thousand impressions. In other words (Impressions > 0) => (Net > 0) And (Clicks > 0) => (Net > 0) while the reverse is not true. Just because you paid does not mean you received any clicks. It is however very unlikely.