The Pros and Cons of Fusion
(speech delivered by Roland Soong at the Staying Tuned 2002 conference, February 11, 2002, Toronto, Canada)

Data fusion refers to the process whereby two separate respondent-level research databases are statistically matched to form a new respondent-level database containing information from the two original databases.  Historically, data fusion products in media research appeared first in Europe (Germany, United Kingdom, Poland) and Asia (India, Australia, New Zealand).  But now the Americas have made up the many years of lagging behind in a hurry.  As of 2001, commercial data fusion products have appeared or are announced in six countries in the Americas.  In each case, the fusion was between a television people meter panel and a consumer survey of multimedia and product usage.  Here is the table that summarizes those developments, in chronological order:

In this talk, we will discuss the experience and lessons that were gained with these fusion products.  We will review the standard pro-fusion and con-fusion arguments that we have heard from many people in many places for many times.  Finally, we will extrapolate as to what might happen for Canada.

PRO-FUSION ARGUMENTS

Data fusion is essentially a free market development, for it would not be here if there was not any need.  The demand for fusion comes from two principal directions.

The first is the need for target group ratings.  For many advertising campaigns, the target group is a very specific subgroup of the population.  As an example, for an anti-depressant drug, the target group would be people who have experienced mental depression recently.  As another example, for an allergy cure, the target group would be people who suffer from seasonal or chronic allergies.  For most target groups, a consumer study such as the Target Group Index (TGI) provides product and multimedia usage data.  Thus, for example, we may know a lot about the magazine preferences and habits of allergy sufferers.  Although TGI studies also contain some television data for target groups, they are crude compared to the people meter data.

This puts us in a quandary when we need to draw up a television schedule, since there is usually no target group information in the people meter panel.  The most typical procedure is as follows: from the TGI study, we find the demographic group that has the highest incidence of the target group.  For example, we find that women 35-64 was the group with the highest incidence of mental depression, and we draw the media plan using this surrogate demographic group within the people meter panel.

Unfortunately, this tactic results in misclassification errors.  The following example is drawn from the MARS study:

 There are two types of errors --- not all target group persons are in the surrogate demographic group (Type I Error = 5.3% in the above example) and not all the surrogate demographic group is in the target group (Type II error = 23.7% in the above example).  Since the demographic group is an imperfect surrogate for the target group, these misclassification errors will result in inefficiencies in the selected media plan.  For the above example, according to the NTI-MARS fused database, we obtained these results:

• For the surrogate demographic group of women 35-64, we used the best programs (defined as those programs that have the highest incidence of women 35-64 in their audiences) to obtain 50 gross ratings points (at 17% reach).  For this same list of programs, we found that we achieved 37 gross ratings points (at 15% reach) for the target group.  Therefore, this schedule, which was optimized for the surrogate demographic group, delivered less than the nominal GRP value of 50 for the target group.  Generally speaking, we expect that a schedule optimized for the surrogate demographic group would be sub-optimal for the target group.  Historically, only the absence of a single source (or fused) database has shielded us in blissful ignorance.
   
• From this same fused database, suppose we had used the best programs for the target group (defined as those programs that have the highest incidence of the target group in their audiences), and we obtained 50 gross rating points (at 31% reach).  For this same list of programs, we achieved 40 gross rating points (at 22% reach) for the surrogate demographic group.  The implication here is that we were able to achieve our goals for the target group for significantly fewer demographic rating points.

In this example, we showed that we can achieve significant improvements in goal delivery and cost effectiveness.  We have done almost 500,000 such similar analyses for different target groups in the NTI-MARS fused database, and we have found gains of this magnitude (namely, between 10% to 30%) to be quite typical.

The second need that data fusion addresses is the evaluation of mixed media schedules.  For a pure television schedule, the evaluation (in terms of gross rating points, frequency, reach, effective reach, exposure distribution, etc) is done with the people meter data.  For a pure print schedule, the evaluation is done with the print study such as TGI.  In real life, advertising campaigns may contain both television and print vehicles.  The evaluation of a mixed media schedule is either not performed at all or else based upon some modeling assumptions.  The most prevalent modeling assumption is that of random duplication between television and print, which usually overstates reach and understates frequency to an unknown degree.  A fused television-print database permits the direct evaluation of mixed media schedules.

The two principal needs described above have been known to exist for many years.  It is not as if these needs were just newly uncovered in 2001.  But in the past, there was no acceptable solution.  Specifically, the data fusion methodologies usually end in diminishing one or more of the databases, in the form of losses in sample sizes or distortions in the audience 'currencies.'  In 2001, there emerged a sound methodology for data fusion that would preserve both the sample sizes and the audience 'currencies.'  In today's talk, we will not  deal with the details of this methodology.  For those audience members who are interested in the details, we refer to our paper, The Anatomy of Data Fusion, that was delivered at the 2001 Worldwide Readership Research Symposium in Venice. 

CON- FUSION ARGUMENTS

We will now enumerate some of the arguments against data fusion.  These arguments are not just coming from Luddites who gainsay by reflex, so we must give serious thoughts to them. 

(1)  Gut Instinct Arguments ("You're making up data!")

This is true in the sense that the television study did not collect information about product usage and magazine readership, and the multimedia/consumer study did not have minute-by-minute television viewing information.  Instead, a pseudo-"single source" database was constructed by statistically matching respondents under some optimal criteria.

In general principles, data fusion is related to other methodology that appears to be broadly accepted.  The best example is the use of ascription in two-phase studies (such as the Return To Sample product booklets used by BBM).  Another example is the use of PSYTE geodemographic clusters.  In all these cases, the reported data had not been collected directly, but some form of statistical estimation was used.  The rejection of data fusion should be accompanied by an re-evaluation of these other related methodologies.

It is unfortunate that the data fusion cannot be validated directly without a true single-source database.  But then again, if we have a true single-source database, we would not need to conduct data fusion.  As much as one may be intellectually dissatisfied with the validity of data fusion, one would still have to balance this methodologically rigorous but not totally validated technique against the other choices.  That is to say, can one continue to use surrogate demographic groups for television planning without apologies when the misclassification errors are so glaring?  And can one either advocate random duplication as a superior approach to evaluate mixed media schedules, or forego the evaluation of mixed media schedules altogether?  Our point here is that the rejection of data fusion means that one may be embracing alternative positions that are much more difficult to justify.

(2)  Applicability ("This stuff is useless for me!")

It is true that people have been able to get along for many years without data fusion, but that was only due to the lack of availability.  Target group ratings have obvious applicability, since its logic is simple and powerful.  Assuming that one accepts the validity of the data fusion, target group ratings will be a powerful element in the media planners' toolkit.

Consider the financial implication of our statement that we should be able to realize saving of 10% to 30% through target group ratings from the NTI-MARS fused database.  Suppose a pharmaceutical advertiser spends $100 million advertising a brand.  From one direction, we claim that we could achieve the same goals by spending 10% less.  That is a net savings of $10 million.  From another direction, we note that there is a common belief right now that every advertising dollar in the pharmaceutical category generates $6 in sales.  For the same $100 million advertising expenditure, we claim that we could use make it work 10% better, as if we had spend $110 million for which we would get $660 million in sales.  That is a net increase of $60 million.  These numbers are mind-boggling, and nobody can walk away from it.

The evaluation of mixed media schedules is a more complicated issue and may be open to substantive debate.  A common question is, "Should one television impression be equal to one magazine impression?"  Maybe ... maybe not ... or maybe some comparability factors have to be developed.  The exploration of these issues will be a learning experience.  This is not just an issue about the relative merits of data fusion, but it resides at the core of the theory and practice of media allocation.

(3)  Financial Issues ("It costs too much" and "What's in it for me?")

For users, there is the fear of exorbitant costs for data fusion products.   In reality, the bulk of the costs for a fused database was incurred during the collection of the primary data, such as for people meter equipment and interviewer labor.  Data fusion is just post-processing whose costs are shared among all users.  In the Americas, the current data fusion products are licensed quite cheaply to the subscribers of both studies.

Given this type of pricing structure, the media research suppliers are not motivated by direct sale revenues from the data fusion products.  Rather, they are motivated by new sales to those who previously subscribe only to the other study, such as the new sales of TGI studies to television channels.  Furthermore, data fusion is a means of adding value to already well-established studies and meeting market demands.

(4)  Organizational Issues ("I don't want to deal with unhappy clients!")

Data fusion requires consent and cooperation by the two research organizations, who have to protect the interests of their subscribers.  In the past, certain data fusion methodologies have the unfortunate effect of losing sample sizes and/or distorting audience 'currencies' in one or more of the databases.  It would be difficult for research organizations to rationalize the shrinking sample sizes as being good, or accept distortions in audience estimates that favor some subscribers over others.  The current generation of data fusion products manages to preserve sample sizes and audience 'currencies,' so this is no longer a stumbling block.

Apart from the technical issues, the media research organizations will also have to manage anxiety.  There may be the fear by the television and print communities that this is a zero-sum game in which a new tool will bring about a disequilibrium such that someone wins while someone else loses.  These types of anxiety will be allayed by the actual experience, as when television people can highlight their inventories with respect to specific target groups, or when print people can document the unique contribution of print vehicles.

(5)  Methodological Issues ("Does this stuff really work?")

Realistically, data fusion cannot be expected to work perfectly all of the time.   In fact, when implemented incorrectly, it can fail spectacularly.  The bottom line is about the management of expectations, to acknowledge that it is not perfect all of the time and to know how to identify and avoid bad situations.

Every data fusion project has its unique elements, and we cannot hope to provide a one-size-fits-all manual.  So we will give five examples in which care needs to be exercised.

Example 1 (Misalignment of sub-universes):  This is a specific example from the Mexican fusion.  Ordinarily, we would be fuse within major sub-universes (such as the Mexico City TAM with Mexico City TGI, the Monterrey TAM with Monterrey TGI, etc).  Within Mexico City, it was actually necessary to maintain three further sub-universes: Cablevision subscribers, Multivision subscribers and non-pay-tv persons.  Cablevision and Multivision are TAM subscribers to their separate sub-universes in Mexico City.  Failure to recognize this situation would be disastrous.  The more general lesson is that the data fusion must respect usage and marketing needs.

Example 2 (Respondent weighting):  Certain types of TGI studies uses single respondent per household, with design weights used to compensate for the differential probabilities of selection.  Any matching procedure that ignores these design weights may result in systematic biases.  The more general lesson is that the data fusion must respect the sample design.

Example 3 (Ineffective matching variables):  Historically, TAM panels collect only a short list of demographic variables (such as age, sex, geography, head of household characteristics such as age, sex, race, occupation and education, socio-economic characteristics, household composition and cable/satellite television status).  These are the standard demographic variables that are initially available for statistical matching in data fusion.

In the USA, we observed that NTI began to collect information on pets, cars/trucks, computers/internet and movie attendance over time.  We take this as evidence that the standard demographics were inadequate with respect to these applications.  By inference, data fusion, which is based upon the predictive power of standard demographics, will encounter similar problems.  The more general issue is the identification of these situations and the development of the solutions (such as collecting new matching variables into the studies).

Example 4 (Fusion Did Not Meet Expectations):  For example, we expect golf magazine readers to almost surely watch the major golf tournaments.  Data fusion based upon standard demographic variables may give some lift, but not near 100%.   Again, the more general issue is to recognize the limits beyond which we should not push our data, such as microscopic analyses of specific vehicles.

Example 5 (Fusion Results Don't Even Make Sense):  Suppose we are in a situation when we are fusing a people meter panel with a radio diary sample, by statistically matching on standard demographic variables.  Within the fused database, we may see 'too many people' who are watching cable television and listening to radio in their cars for specific quarterhours.  In this case, it is too difficult to require quarterhour-by-quarterhour consistency in a time-based database, but the fused data may be viable on daypart averages.  The general issue is to recognize the limits on the granularity of the data.

We now moved to the current situation in Canada.  The major Canadian research studies are:

This configuration of multiple studies is quite similar to that found elsewhere in the Americas --- Brazil, Mexico, USA, etc.  We will take as a given that the two principal drivers of data fusion are also present in Canada: target group ratings for television planning and mixed media schedule evaluation

These needs are not being fulfilled right now, for much the same reasons that were present elsewhere:

• The people meter panels have only limited demographic/product usage information, out of concern that a more intensive data collection effort will negatively affect cooperation and compliance rates.
• Each study has been optimised for its particular industry, and cannot hope to be expanded to accommodate the measurement of other media.
• Product usage varies in scope across the studies (e.g. number of categories, brands, buyergraphics, etc)
• Product usage varies in content across the studies (e.g. recency, frequency, volumetrics, time periods, etc)

Just like elsewhere, there is little or no more technical or financial barriers for data fusion to occur in Canada.  A (TAM-PMB)-type fusion would be the same as everywhere else, with well-documented and well-understood properties.  Such a fusion could be either a pure tv-print fusion, or a pure tv-product usage fusion, or a joint tv-print-product usage fusion. 

Another type of fusion in Canada would be to consolidate the product usage information, with a single organization (such as PMB) to collect the product data to fuse onto all other studies.  The perceived advantages would be the standardization of the product usage information and cost savings by no longer duplicating the data collection efforts.  Of course, this would be predicated upon a data fusion that is acceptable.   The validation of this type of fusion is made easier by the fact that the studies have collected actual product usage information which can be compared against the fused data.

Perhaps the most important lesson that we can impart from our experience is that the objective conditions do not guarantee that commercial data fusion products will emerge.  After all, it took many years before data fusion appeared in the Americas.  There is always the matter of political will.  A data fusion project involves four parties: the two research suppliers, the data fusion supplier and the user community.  In the four Latin American countries, Kantar Media Research is the owner/partner in all the TAM and TGI studies as well as being the data fusion supplier, thus getting the total commitment from all the organizations.  In USA, Kantar Media Research is the owner of the MARS study and the data fusion supplier, again expediting the process.

In the case of Canada, the prospect of data fusion will depend on the positions of the key players.  Will the media research suppliers be pro-active, resistant or be dragged into it kicking and screaming?  We bear in mind that BBM and PMB are tri-partite, non-profit organizations that are governed by agencies, advertisers and media, and must therefore act according to the wishes of their constituencies.  And will the media (television, radio, publishers) see this as an opportunity to gain or lose?   And how hard will advertisers/agencies press?

(posted by Roland Soong, 2/11/2002)