In 1996, when a top of the range PC harddrive was about 30MB, the cost of digital storage dropped below the cost of storing data on paper (Morris, 208). The efficiencies of digital storage have continued to increase at about 100 percent per year and digital storage is now hundreds of thousands times more efficient than in 1996. Our ability to store, gather, sort and analyze consumer data has transformed our capacity in market research. But will the majority of businesses be able to gain useful market insight from this revolution? Or will we just continue to ‘drown in the data’? (Carmody, 2015), (Thomson, 2013).
While some companies that are in a position to take advantage of big data in their marketing research are finding useful insight, the majority in 2013 were struggling to make efficient use of the data they already had (Thomson). Two years later there seemed to be little progress (Carmody). There are three possibilities for this lack of progress: 1/ technological barriers; 2/ psychological barriers and 3/ ethical barriers.
Potential areas of technological challenge with big data are the sheer volumes of data involved, the variety of the data and an inability to integrate it. In addition there are difficulties in quality and accuracy of the data over time (Carmody). Critically sorting and analyzing this data in a way that can provide businesses with strategic insights that will improve profitability seems allusive and risky to all but the largest of companies. However, entrepreneurial businesses are arising that specialize in big data analytics, even predictive analytics, boasting among other things that they can predict what product your customer will want to buy tomorrow! It does not seem likely that technological barriers will hold back this opportunity in market research for long.
One potential barrier suggested for slow utilization of big data in marketing strategies is simple change management issues within organizations. Additionally there is recognition that Big Data does not only measure consumer behavior but also employee and managerial performance. A simple human reluctance to be measured may be more pervasive amongst the employees of a company than amongst its customers (Thomson). However, similar to the Total Quality Assurance revolution of the 1970’s and 1980’s, it seems unlikely that a resource that can be utilized to make more sales and provide the customer with just the product they are looking for will not be resisted for long.
In 2015 up to 35% of companies surveyed considered brand damage as a risk related to the use of big data, especially when combined with inaccurate data (Carmody). Similarly 36% cite distrust amongst employees as a risk (Carmody). While not addressing the issue of the ethics of big data directly, the perceived risk to brand and employee trust indicates an uncertainty about the ethical standard. Customers, employees and the companies themselves are unsure what information is appropriate to record, store and analyze. Unsurprisingly 40% of the surveyed companies rate regulatory risk as high.
Unlike the traditional survey principle of the anonymity of those being surveyed, the whole point of big data is that individuals are identified and targeted. As the data collection is driven by companies looking for a competitive edge, companies live in the tension of self regulation.
The power of Big Data combined with predictive analytics may mean that this scene from the 2002 movie minority report is technically possible now:
Perhaps with a scan code to instantly confirm the purchase…
Or, alternatively, predictive analytics may be able to anticipate what product you will buy tomorrow and have a 30 day free trial delivered to your doorstep as you ‘sleep on it’. The technological and psychological barriers to this marketing research reality will be soon overcome, if they haven’t been already. Social, cultural and ethical barriers will take more time perhaps overcome through clear boundaries and the building of trust between buyer and seller.
Carmody, Bill, ‘Still Drowning in Big Data, and Starving for Insights’
[online] 4/8/2015, Available at:
[Accessed 15 Apr. 2016]
Morris R. J. T, Truskowski, B. J, “The evolution of storage systems” in IBM Systems Journal, Vol 42, No 2, 2003.
Northway, James , ‘What Will You Buy Tomorrow?’
[online] 24/3/2016, Available at:
[Accessed 17 Apr. 2016]
Press, Gil ‘A Very Short History Of Big Data
[online] 9/5/2013, Available at:
[Accessed 15 Apr. 2016]
Thomson, Jeff, ‘Why CFOs Are Drowning In Data But Starving For Information’
[online] 30/10/2013, Available at:
[Accessed 15 Apr. 2016]
Submitted By: Simon Koefoed
Student id: 214486705
Wordpress Username: @skoefoed