Posts Tagged Analytics

May 12 2016

Big lessons from big data implementation – Part I

Each day 23 billion GB of data are being generated and the speed of generating big data is going double in every 40 month! Apart from their business data, organizations now also have humongous data available from google, Facebook, amazon, etc. They wish they can use all the available data to find useful information for doing their business better.  Let us look into big data deployment of a few organizations and learn from their experience.

Case 1: Rabobank

Rabobank is a Dutch multinational banking and financial services company headquartered in Utrecht, Netherlands. It is a global leader in food and agro financing and sustainability-oriented banking. Rabobank started with developing a big data strategy July 2011. They created a list of 67 possible big data use case. These use cases included:

  • To signal and predict risks, prevent fraudulent actions that the bank is running
  • To identify customer behavior  and to obtain a 360-degrees customer profile;
  • To recognize the most influential customers as well as their network;
  • To be able to analyses mortgages;
  • To identify the channel of choice for each customer.

For each of these categories they roughly calculated the time needed to implement it as well as the value proposition. In the end the Rabobank moved forward with big data application to improve business processes as the possibility for a positive ROI. A dedicated, highly skilled and a multidisciplinary team was created to start with the big data use cases. They were using Hadoop for analyzing big data. They selected social data, open data and trend data were integrated. So there data approach with a deluge of semi and unstructured mess. Hadoop is only part of a big data strategy. The key to success was the multidisciplinary team and that they embraced uncertainties and accepted mistakes to be made help them to overcome situation.

Problems faced during implementation

Rabobank didn’t store raw data, due to the costs and capacity issues. The data quality was not constant and the security issues were very high. Rabobank noticed that it was often unclear who owned the data as well as where all data was stored. Hadoop is different from older database and data warehousing systems, and those differences confused the users.


  1. Specialized knowledge as well as visualizations is very important to drive big data success.
  2. Start with the basics & don’t stop at stage one. Big data implementation is continuous journey to reap data-driven insights.
  3. Not having the right skills for the job can be a big problem.
  4. The dangers of underestimating the complexity of a big data system implementation so focus on data management.


In 2010 the newly elected president of the United States of America government introduced Patient Protection and Affordable Care act. The main purpose of this act was the best of public and private insurance coverage for the population, and thereby controlling and reducing healthcare costs and requires them to interact with the government via a website to do so. The system is in essence a big data implementation problem with data being collected on a potential population in Excess of 300 million people across the entire country. Unfortunately the project not progressed as planned, and has become mired in technological controversy.

Problems faced during implementation

  • This act brought the country to default on its debt
  • Cost of Obama care – $1.6 trillion
  • Estimation of cost 2014-2024 – $ 3.8 trillion

Anticipation to prevent the problem:

  • They can take special knowledge as well as visualizations can prevent the loss.


  1. The dangers of underestimating the complexity of a big data system implementation to focus on data management.
  2. The prior analysis and prediction complexity of data can prevent cause.
  3. Most of the data collected and stored in an agency’s transaction processing systems lacks adequate integrity so make sure that captured data meet integrity standard.
  4. Specialized knowledge as well as visualizations is very important to drive big data success.
  5. Not having the right skills for the job can be a big problem.


We shall analyze a few more cases tomorrow. Keep watching this space.


Views expressed on this article are based solely on publicly available information.  No representation or warranty, express or implied, is made as to the accuracy or completeness of any information contained herein.  Aaum expressly disclaims any and all liability based, in whole or in part, on such information, any errors therein or omissions therefrom.


  1. The process of big data solution adoption By Bas verheji.
  2. Case study on Rabobank by Hewlett Packard enterprise.
  3. Big data for all : Privacy and user control in age of analytics by Omer Tene and Jules Polonetsky
  4. Realizing the promise of big data implementing the big data projects by Kevin c Desouza
  5. Case study on Obama care big data problem on patient protection and affordable care act


Sep 19 2015

Building Data Science in your organization – Is it really important?

Analytics for businessDoes data science bring value to your organization? Hang on, what is this data science really? Why should we really care about? You got to really care because this one might change the way you run your business in the near future whether you like it or not. Just like Software/IT changed the world a few decades before!  IT is omnipresent and those who didn’t care to change the wheels suddenly had to pay steep price to change their course. Today, thankfully the cost of IT adoption is very low since the industry is much matured to provide right solutions at low price. But the initial period was very crucial. People were extremely cautious about implementing, evaluated ROI, questioned why they should invest considering there were new systems, servers, recruitments, etc. Those were the times where clear ROI from IT cannot be calculated. IT was nascent. There were gross blunders like Y2K issues. But the world got changed and IT has touched almost all facets of life.

Whether we like it or not, data science is going to bring another transformation in the day-to-day activities. There is a wide perception that data science is applicable for bigger organizations and not for small and medium businesses. Of course, there are successful early adopters. A few have burnt their fingers with analytics adoption. Money ball showcases how Billy bean uses data science effectively for making a good team to compete in base ball. Of course, there are several criticisms for the same strategy in other games. The success requires careful adoption to the business context.

The analytical adoption is getting more matured. It is time for the companies to realize quick data driven insights, if they want to achieve a competitive edge over other companies. How should one adopt the analytics strategy? Come and attend our analytics event “Analytics for CXO” -a must not miss program for the CXOs who want to adopt/implement analytics for their organization.

  1. Update yourself with bleeding edge analytics developments happening in the industry
  2. Customized Analytics roadmap specific to your organization/department
  3. Consultation with industry experts – subject to prior appointments.
  4. Practical use cases with analytics implementation, benefits and value
  5. Focused group discussions on the benefits , challenges, issues, faced by the organizations
  6. Learn best practices from the industry, academia and peers
  7. Analytics Jumpstart Kit – Small data or big data. This Kit is definitely a must have for your organization.

Registration @ Explara -
Date: Saturday, November 21, 2015, 9:00 AM to 5:00 PM
Venue: IIT Madras Research Park, Chennai

Jun 25 2014

Marketing Mix Modelling from Multiplicative Models

E-Commerce or Digital Marketing has emerged as a separate and rapidly growing business domain with many businesses thriving solely on them. With internet acting as a strong channel to perform business, there also arises a simultaneous need to use this channel effectively to add value to your business.


2014 – The year of Digital Marketing Analytics

According to a popular study on the Forbes magazine, 2014 is the year of Digital Marketing Analytics.

DMA Businesses that use digital marketing analytics for improving customer acquisitions, increasing brand loyalty, increasing ROI from their marketing channels, etc have a better competitive advantage over other businesses who have not yet ventured out into this space. The internet search giant company Google Inc, has also created a massive impact in the field of E commerce analytics through their Google Analytics platform which provides a variety of KPI’s and statistics to help digital businesses track and measure their online sales and marketing. AdWords, E commerce reporting, real time analytics are some of the popular services provided in this platform.


Digital Media Analytics

While these services are good enough to track high level trends and provide basic directional metrics, there is often a need to dwell the past data much deeper with sound understanding and usage of advanced analytical techniques to get better insights. One such area in the field of digital market analytics which relies heavily on the use of econometric models for decision making is Market Mix Modelling. The Gartner IT glossary defines Market Mix Modelling as “analytical solutions that help marketers to understand and simulate the effect of advertising (volume decomposition), and to optimize tactics and the delivery medium”.

In simple terms Market Mix Modelling refers to the estimation of statistical models to measure and analyze how various marketing channels are effectively contributing to your sales. If is often used to find the optimal mix of the market channels and forecast future mix which would help one to maximise the ROI from various marketing channels and sales. In particular it helps the business user address key questions such as:

  1. How do we decompose our sales to key drivers to understand the combination of these channels which would contribute to sales, market share or profits?
  2. How are these key drivers impacting sales over time?
  3. How do we calculate the efficiency or ROI from these market channels?
  4. What happens when we change the budget by channel


Market Mix Modelling – Methodology and Illustration

Market mix modelling relies on three popular model forms to understand the impact of various market mix variables on sales. Users use either of the three based on their requirement from the analysis.


Table 1. Summary of Market Mix Models functional forms


Additive models tells us how much of a change is generated in the response variable in unit terms with one unit increase in the explanatory variable and is only used in scenarios where the impact of each additional unit of the explanatory variable is identical. For example this model is not suitable in determining sales decomposition of seasonal brands such as yogurt or ice creams. However the estimation of additive models are easy and sales decomposition can be directly implemented to understand the variable contributions. Semi logarithmic models on the other hand show how much the absolute level of the response increases/decreases proportionately with the independent variable. An even better logarithmic model gives us the variable elasticities which exactly measures the responsiveness of the independent variable to the dependent variable. This model comes very handy to understand how price and promotion discount elasticity impact sales and hence the most preferred model in industry standards. However modelling and decomposing logarithmic or semi logarithmic models which are in the multiplicative form are indirect and quite challenging. A good understanding of the data and techniques is very essential to model market mix variables in the multiplicative form and to decompose it to arrive at something similar to the below figure.


Figure – Decomposing sales to key drivers

Market Mix Modelling from Multiplicative Models

In the above figure sales of a popular ice cream brand has been decomposed to understand how the key drivers such as its own price, the competitors spend, and its market channel spend from Press and TV impact the sales volume of the product. Sales volume has been decomposed here to its base effect (shown in blue) which usually can be interpreted as the sales derived from the brand value of the product and the incremental effect derived from the sales drivers. Impact of the market mix variable TV can be seen clearly in the months of Jun/July where the TV spend (shown in yellow) is made. A multiplicative model has been implemented here to capture the right effects of the variables and to understand the seasonality of the product sales as the product has a very seasonal demand.


A link is available to experience the true value from Market Mix Modelling here.


Right information at the right time through the right means

Adoption of modelling techniques in business and the insights gained from it are vital to the survival of any organisation. Here in this article I have showcased how the concept of market mix modelling developed in the right fashion could add much value to your business and help you foresee things which you could not have before. Right information at the right time through the right means is really worth your investment of time and money!







Jun 10 2014

Crouching “NoSQL” Cautious “SQL”

“According to analysis by Wikibon’s David Floyer (and highlighted in the Wall Street Journal), the NoSQL database market is expected to grow at a compound annual growth rate of nearly 60% between 2011 and 2017. The SQL slice of the Big Data market, in contrast, will grow at just a 26% CAGR during that same time period.” NoSQL adoption will go big bang but will it impact all the organizations? Will SQL lose this battle? Does everybody need to care about NoSQL? First of all, there is no battle to begin with. NoSQL doesnt mean NO SQL. It simply means “Not only SQL”. Wikipedia defines NoSQL database as

                  “A NoSQL or Not Only SQL database provides a mechanism for storage and retrieval of data that is modeled in means other than the tabular relations used in relational databases. Motivations for this approach include simplicity of design, horizontal scaling and finer control over availability. The data structure (e.g., key-value, graph, or document) differs from the RDBMS, and therefore some operations are faster in NoSQL and some in RDBMS. There are differences though and the particular suitability of a given NoSQL DB depends on the problem to be solved.”


Let us understand SQL and NoSQL a bit more better!



Clearly each one has its own merits and shortcomings. There is no standard solution that fits to all business requirements. Sometimes it is more shocking to see how people react/make decisions on this front. See for yourself one such fight.



Platforms, technologies should be chosen to fit business and not vice-versa.  While there are problems SQL isn’t suitable for, it still got its strengths. Lots of data models are simply best represented as a collection of tables which reference each other. Therefore SQL atleast for now should be good enough for most of the organizations. Right now, it’s fair to say NoSQL is only relevant to a very minor proportion of businesses. But that proportion is very very significant! The changing data arena would sooner see a technical innovation driving business innovation very similar to what we have seen with smart phones?

There wasn’t a need for smart phones but the technical innovation provided business opportunities. Likewise, big data promises a huge opportunity for NoSQL.