Ad tech firms have been selling the dream of big data for over a decade, but how close are we really to this supposed promised land? Teavaros Nico Pizzolata examines whether well ever truly break down the silos.

For a decade now, the advent of big data has promised to fulfil the marketers’ dream of targeting the right customers at the right moment with the right offer, leading to a steep rate of conversions, juicy commission bonuses and prolonged time on the golf course. Like any dream, it ended when marketers woke up to the reality: the data wasn’t simply going to surrender its value. The power of data analytics for marketers depended on one key step: getting hold of their marketing data in the first place.

But bringing all data sources, silos and assets together, online and offline, has often proven an intractable problem, even for first-party data. The competitive advantage of big data truly lies in going deeper into analysis, and hunting for insights that go beyond those easiest to harvest. A trove of data still sits in the marketer’s lap, yet, somehow, it remains elusive.

Reconciling incompatible systems
Despite the innumerable ad tech vendors offering tools to collect, process, and integrate data from different sources, preparing it for the right use requires effort and time. The data does not come marketing-ready; this involves manipulating the structure of the information, down to field-level mapping, to fit the desired purpose. Even the same kind of data points – for instance, those collected by a customer services call centre – might be tricky to align if the company has recently upgraded to a CRM software that uses a different taxonomy.

Similarly, different departments might store data in both relational (SQL) and non-relational (NoSQL) databases; sales data from physical stores may not map readily to the same data collected from online purchases. IT departments do not often think with a marketer’s hat and if a company has been around long enough, it has probably accumulated its share of incompatible systems.

Data lakes seemed to prefigure a world without silos, but diverse naming conventions, onboarding problems, obstinate legacy data and security challenges have created a gap between expectations and delivery. Data fabric is the latest incarnation of the vision to create a unified data environment. Companies operating in this space battle with challenges that now go beyond the fragmented enterprise’s data centres. For instance, the use of devices in customers’ hands – such as smartphones or the Internet of Things – introduce new challenges for data unification. So too the multiple locations of data, in cloud computing for instance, and the different standards and interfaces, from REST API to HDFS (Hadoop distributed file system) necessary to access that data.

At Teavaro, we have been working closely with clients to transform their data architecture, to combine information from different online sources and to overcome the restrictions of legacy systems. We often encounter inconsistent, mostly siloed IT systems – most of which do not represent the end-to-end business processes of their owner holistically – and thus we know the problems they introduce.

Traditionally, there were two possible approached for the solution. The first – a complete rebuild of the company’s stack by exchanging legacy systems for a fancy, built-for-purpose system architecture – is usually not feasible due to costs, duration, negative business impact during the update and the tremendous requirement for disciplined interworking between all units within the company.

The second lead to the dead-end companies are currently finding themselves in; selecting new technology to address isolated issues individually by focusing on their specific requirements and priorities is what led to a marketing stack of randomly-integrated partner solutions. It is obvious from the results of the latter that some of the habits of the former need to be adopted. An aligned business strategy between all relevant business units and IT is needed to transform system architecture consistently. Starting from that point, the introduction of new and additional platforms can fulfil both requirements – leveraging the needed business support and transforming the landscape.

Data cultures
When different departments produce and collect data, the demand across the company is to create a sense of shared ownership of the different data assets. According to Edd Wilder-James of Silicon Valley Data Science, “Data isn’t a neutral entity — you must interpret it with knowledge of its history and context. This sense of proprietorship can act against the interests of the organisation as a whole”.

This collective interest is the transformation of individual data points into usable information, but if the data owner does not realise that value then it is lost. Managing data across units, organisational functions and CRM systems might involve bringing together marketers, data scientists, and IT staff, with data from all the brand’s touchpoints. In the deluge of data accumulated by the company (including logistics, billing, manufacturing, etc.), those teams without a marketing focus might not understand which data points are likely to lead to the best business outcomes. In the process of integrating teams, changing the data culture may become a pre-requisite, a catalyst for organisational change tout court.

A crowded space
The tendency to increase the stack of marketing tools has so far been part of the problem, not the solution. The proliferation of martech is well documented by the annual Lumascape graphic. The numbers of logos increase exponentially, by roughly one third every year. Such is the proliferation of vendors and the expansion of sub-sectors, the individual players of the ecosystem have become almost indistinguishable to the naked eye when displayed on the average computer screen or printout. This has come to mean that any big data architecture strategy is not only about integrating data points but systems.

Another consideration pertinent to digital marketing is regulatory law. Any solution aimed at providing an integration of data sources must adhere to the strict guidelines of the GDPR. So why not place the requirements of GDPR at the centre of this solution, and use the standard to tackle the integration of legacy stacks and silos?

Teavaro’s FunnelConnect enables the integration of online and batch processes with the capability of real-time data processing as needed in the online world. Without adding to the martech stack, we design systems to help our clients merge profiles of identified visitors (online) in different channels with legacy customer information (offline) after full identification (e.g. login). This integration generates rich first-party data such as customers profiles of interactions to helps deduce the next best action.
A truly agnostic data solution, it is, by design, able to provide more than simple connection, but a future-proof system capable of providing consent and permissions management in line with the GDPR. Using customer identification within the given GDPR framework, it is not only possible to identify customers and prospects in different online channels but also to extend this to existing legacy stacks. That’s when the dream comes true.

Nico Pizzolato is Senior Lecturer at Middlesex Univeristys Business School, and an investor in Teavaro.