Before we take a deep dive on the subject, let’s first get on the same page on the definitions. An organizational unit is considered a Cost Center if, from accounting perspective, only the expenses are considered, and revenues are not. It is considered a Profit Center where all elements of expenses and revenue are considered. For the purpose of this article, Managed Services is classified as a unit (either within or outside of the organization) providing post-production support.

The Return on Investment (ROI) calculations for transactional and operational systems such as ERP, office productivity etc. in general invariably add the post-production support as a cost while calculating the total ROI for the project. This, however, should not be the case for Advanced Analytics projects. In this article, I will detail one scenario of Advanced Analytics – Decay of Performance of Model – and provide a perspective on why post-production support is a revenue stream for organizations, hence making the case that managed service should be considered as profit center.

Figure 1 depicts a typical project work breakdown structure for a scoring model development project.

Any scoring model (except for quantum computing models) will have 4 outcomes. These outcomes are: True Positive (TP) and True Negative (TN) wherein the model correctly predicts the intended experiment results; False Positive (FP) and False Negative (FN) wherein the model incorrectly predicts the intended results. A good scoring model is one which has the high prediction accuracy. Accuracy is defined as ((TP+TN)/(TP+FP+TN+FN)) or simply a ratio of the number of correct outcomes divided by total number of test. Once the target level of accuracy is achieved, through iterative training of the model, the project is classified as a success and model is deployed.

And here lies the central point of this article – Performance decay of the scoring model.

1.   Dataset may not be comprehensive

Datasets are the key to success. Richer the datasets more accurate the predictive outcome will be. At the inception of the project, efforts are made to identify as many datasets as possible to model. Despite the best of efforts, it is not possible to capture all the data streams, and if miraculously it is done, there remains a gap where new data comes to the surface forming a new dataset or modifying the current dataset. For example, 10 years ago for the gender dataset, there were only two options modeled which were Male (M) and Female (F). Advanced modeling considered the blank values as well. A scoring model developed on this basis will need to be “updated” today to incorporate values other than the three listed above.

2.   New Data/Channel set may become available

“Customer 360 degree” with and without “location” data has very different usability. When the models are enhanced with locational data, it opens totally new revenue streams thus enhancing the ROI. Similarly, in a multi-channel shopping world, it would be prudent to enhance the model based on the channel data. For example, who knew that people will be buying houses using Blockchain and Centra Card (Centra card is a pre-pay card wherein transactions are carried out using crypto currencies)

3.   Modeling assumptions change

Every model has some underlying assumptions and it is critical to periodically evaluate if those assumptions hold true. One of the main assumptions is that the future will follow a similar trajectory to that of past. Occasionally (and it is becoming more frequent now) comes along a disruptive technology that totally invalidates the underlying model assumptions. Uber has certainly changed the model for public transportation. People may have forgotten that when Apple launched its cellphone business, it met with much skepticism only to see many leading companies such as Motorola go out of business.

4.   Outliers become the norm

No scoring model can be 100% accurate as there are always some outliers which are often ignored. As the trends shift some of the outliers start to become a norm. A classic example is that Airbnb was an outlier when modeling for tourism industry’s hotel’s capacity and utilization planning during the early stages of Airbnb. Today the capacity offered by Airbnb far exceeds that of the largest hotel chain in the world.

In summary, all models will decay with the passage of time and as a result the ROI will erode over time. By treating the Managed Services as a profit center and integrating it with the overall initiative, organizations will continue to reap the benefits of additional revenue due to updated model with minimal of additional expenses.

We at HEXstream believe that Data Analytics is not a static exercise but rather a process that needs to be monitored and tailored based on the dynamic analysis needs.  At HEXstream our engagement model extends beyond building data exploration and analytics system. Our managed services team seamlessly takes over the newly built system and provides ongoing support and maintenance to make sure our customers get the full return on their investment.