Byline: MARY MCKENNA-TREPP
WHY IS database modeling important? Which models should we use and when? The three areas of publishing where we tend to apply our efforts in direct database marketing are acquisition and list rental; billing and renewals applications; and advertising applications.
We build 85 models a year in my group, and possibly 80 percent of those models are for acquisition and list rental. At Time Inc., we do a smaller amount of that work for billing/renewal, and a lot of it is exploratory, or analysis related to segmentation. The other application I will touch on, that is relevant at different points along the way, is advertising.
I've been in this field for twenty years. Historically, people were focusing on using intuition or business experience to do their modeling. People with no experience would make selections from programs based on their intuition. Some companies, particularly those who were database-oriented or who had large customer files, would have included modeling in their programs. Over the last 15 years, database marketing and modeling have been extended more broadly. More companies are using them more extensively, and tools that handle things like cluster segmentation are available to a wider audience.
Aside from multi-title marketing, which has very unusual challenges, we use modeling techniques for all our single-title direct mail promotions. This is particularly helpful in a weak economy when mail costs are going up and we need to increase the efficiency of our mail efforts.
CHOOSING YOUR SELECTS
Starting out, most people use basic criteria, all different types of selects that are fundamentally variations of hotline selects. For example recency, things like purchase or lifetime dollars, and the all-important variable for all direct mail is, "Were they direct mail sold?" You can get very far with just those few elements. The other piece might be affinities - a purchase based on an interest in a particular category, such as sports is a good category for a Sports Illustrated offer.
At Time Inc, in our single title direct mail, those types of hotline or other straight selects account for about half of the volume we mail each year. The other half is modeled, because as recency falls off, it's a better tactic to exploit names that have not ordered in the last 30 or 60 days. Modeling is also helpful if the product that you want to cross-sell is not too similar to what they purchased before. For example, cross-selling SI for Kids to someone who bought SI previously. That is not a clear cross-sell; knowing that they purchased SI is somewhat powerful, but not in itself enough to know that they are a strong candidate for SI for Kids.
The simpler step for cross-sell before you get to modeling is to test whether there are some simple pieces of data that can be used to sharpen up that select. Demographics, ie. knowing this is someone who recently purchased SI and has a child in the house, would be important for cross selling SI for Kids.
But as recency falls off, you have applications where it's not just a couple of variables that are very powerful. It's actually a range of variables that adjust the likelihood of response. Modeling is the more efficient way to go, because it will find the subset of the universe within all the SI active subscribers, and who will be the best candidates. It is also helpful because there's more than one way to be a good candidate for SI for Kids - you might have a young child of the right age in the home, or you might be a generous aunt, grandparent, etc.
So with modeling, we help identify some of the other groups of people who would be good candidates for the offer. It helps to extract more volume out of that total pool of SI actives, because multiple variables that flag good prospects are identified. With straight selects, you tend to cherry pick the very strongest names, and there is an opportunity cost for some of the other names that are perhaps strong candidates because of their values or a number of different, less strong variables. When you add a set of weaker, but complimentary variables together in a model, it has a very positive impact on performance.
SCORING A UNIVERSE
We are going to talk about scoring a universe and some examples of the financial issues involved. The most powerful approach is to start with a database that includes both responders and non-responders. The goal is to ID a set of variables that compliment each other and are non-redundant, that explain as much of the variability in response as possible.
You don't want the variables to be redundant with one another, that is called high correlation. If you have variables that are highly correlated, it can make the model very unstable. There are statistical issues that result and the coefficients cannot be estimated accurately. The coefficients are critical since these are the weights that are applied to each variable to determine the impact it has on the final score.
The goal of the model is to come up with an equation to be your forecasting tool to estimate response among people who were not mailed to yet. The purpose of the model is to make the best possible guesses about who is likely to respond if a similar group of people is mailed in the future. We don't know if future prospects will respond or not, but we do know other things that can predict the likelihood of response: Whether they are homeowners, whether there are children in the house, etc.
Let's say that we started out with a list test of a potential universe that didn't meet our financial goals, which might have been mailed as a small test first. What we might find is that in order to meet our financial target, we need to achieve a 2% response rate; but this list test only responded at a 1.5% rate. Then we can say that this list won't work.
There's still opportunity there, we just have to sift through the chaff and find the wheat. The modeling helps you do that. By using that equation you can make a forecast for all the members of that universe. You have a tool to make a prediction about the response rate among everyone who is in that SI active universe. Once you have that prediction for each of the customers, you can sort them into groups and prioritize the most-likely performers vs. the least likely. Because of that, you might find that 20% of the universe is mailable.
Taking that universe and applying the equation to it is called "model scoring." Typically, when a model is scored, you can order a list from an outside vendor or from your internal analysis group, based on their score: "I want people who have a score greater than .1256," is not typically the way to go. To make it easier to manage, lists are usually cut into deciles or percentiles if the universe is very large.
A gains chart is a tool the modelers use to evaluate the effectiveness of a model. What you are looking for is a sharp differentiation from the top of the model to the bottom. For example, in a model that's not so strong, the mailing effort in the top decile might be a 120, and the bottom decile a 75. It's not a sharp model, perhaps because zip code level data was used.
If you had household level data, where the predictors were very effective, then it would be typical to see a more dramatic lift. The index at the top would be at least 200-250, and the bottom would be 50.
The level of ranking will depend on the detail of the data, and also on what you are trying to predict.
USING THE AVAILABLE DATA
I had an interesting challenge to try and build a model to predict how much someone would be willing to spend on a visit to a casino in Atlantic City. The information I had to work with was limited to demographic data and very basic lifestyle data. That was a tough modeling project! There are some personality issues that impact gambling behavior that we don't have readily available in most database files. But luckily, you have magazine data available - prior purchase of magazines, or prior direct mail response data, which is much more effective for the modeling projects that you are facing.
The most important gains chart of all is the gains chart on the validation sample. The validation sample is a set of customers who are set aside, that the analyst uses to demonstrate that the model they built will hold up, that they haven't done something called overfitting. (Overfitting is what occurs when a model works very well on the sample the analysts built the model on, but isn't stable at rollout).
There are a few tools to use to evaluate a model that someone else has built. You want to ask them to show you the gains chart for the analysis sample and for the validation sample. Look at those indices and see if those indices track a very similar pattern.
Continued from page 1.
If the indices on the data analysis set look much better than on the validation, there are probably variables in that model that are not powerful, but are variables which, just by chance, appear to be powerful in the first model and not in the second. A modeler can control for that by looking at the variables one at a time before the model is built, as well as by using some common sense and experience to say, for example, that the color of their third car is probably not a good predictor of whether they will respond to SI for Kids.
Let's speak for a moment about the issue of campaigns. What we find is that because we have such a large database of 70 million households, we are able to build cross-sell models or multiple cross models for each of the single title mailings. If you are thinking of starting an intensive modeling or database marketing program for your title, one thing to think about is how to build the database, how many models might I build in a year, how many might I score?
Make sure that there is a production process that is appropriate your level of activity. If you work in an environment where there aren't that many mailings per year, and you don't anticipate you'll have more than 5 models being scored simultaneously, it might be the case that the analysts who develop the models could be responsible for scoring them as well.
Analysts and production people who score our models are not necessarily cut from the same cloth. If you have heavy volume, the best thing to do is to have separate dedicated production people with a tight focus on efficiency and quality control when executing the models for the campaigns. The campaigns need to go out on time. If you have analysts do it for you, that's fine; make sure that you start far enough in advance, and that they are checking the accuracy of the scoring every step of the way.
ECONOMICS OF MODELING
The way we think about acquisitions modeling is that ultimately, it saves money. It involves an initial investment, above and beyond the investment in the database or the infrastructure. What we do in our group is charge our customers for the models that we build. We charge $9000 for the models that we build, and that covers the majority of the staff costs involved with the model build.
In addition to the model building costs, there's the cost of mailing a sample of people who, by definition, did not meet the acquisitions allowable. And there's a cost associated with mailing a large group of people that don't meet your financial targets. Once the model has been built, the cost of selecting names internally is much lower than selecting outside list names. In addition, you can use the same model for 2 or 3 years. You can offset that investment in the model through savings associated with a large number of lower cost names selected over time.
Another type of model is the prospect model: you don't mail a sample of customers in this case. Instead, you match a file of your best customers to an outside file. Sometimes it's called a look-alike model or a profile model, and what it is designed to do is identify people who look like your good customers. The only potential downside is if you have a bias in the kinds of customers you mail to, then the prospect model will continue to push you in that direction.
The prospect model is not a good tactic if your goal is to identify some new pockets of opportunity, some sub-segment you have not explored in the past. However, it is very quick and less expensive in the initial term. So we recommend a prospect model if you participate in a cooperative database, or if you are having a model built by outside vendors. We use it to recommend good names for our list rental clients and also for internal cross-sell, if there hasn't been a lot of cross-marketing in the past. When Time4 Media joined the Time Inc. database, we were able to use these tactics successfully.
Then there is the response model. First we use test lists to evaluate good potential model universes. It could be that these universes are products of knowing what was successfully marketed in the past on a hotline basis, or perhaps we've done some analysis, like the prospect type of analysis.
I recommend using a response model on a large universe, with 600,000 to 2 million names. Because you are making an investment in a model, you want to be sure this universe isn't going to dry up next week. You want to get two to three years of life out of it. And you want to make sure that the number of quality names you can get out of it is large enough. We find it particularly useful for inactives and expires, because you want to get every possible good name. The less recent names from universes that have been very strong in past hotline selects are a very good place to start for cross-sell modeling. Expire names are ideal for reactivation mailings.
ANALYSIS TOOLS
Next, we need to match the tools to the business problem. One technique you should be aware of is profiling. Profiling consists of doing descriptive analysis of one group vs. another. When someone is doing a profile for you, make sure you are getting information that is statistically significant. For example, this can be used to profile differences between renewers and non-renewers, people who come in through one source vs. another. In an advertising example, you might want to look at who came in from one source vs. another, because one source is yielding the kind of prospects your advertisers are interested in, while another source is not.
Another technique is discriminant analysis, which is similar to regression modeling, except that you have the ability to make predictions about three or more groups. In regression modeling, you are predicting between one or two outcomes. With a discriminant model, you can develop it to say if this person is likely to renew by phone, mail or non-renewal. One word of caution: with discriminant analysis, don't let analysts build on too many different categories. Once you go beyond five or six categories, especially if they aren't sharply defined, the results will get muddy and the predictive power will not be as strong.
Cluster analysis is different from the other techniques because there is no specific outcome to predict. Your goal is to come up with groups of people that are similar to one another, then identify what this similarity is - this is the most challenging part.
Demographics are often used to create clusters or other segmentation because they are very versatile. This is the technique being used when an outside company is creating tools, like Prizm or Personix. My recommendation is to use household level segmentation rather than zip level, where possible, because the precision will be much greater. This can be a start, as an alternative to prospect modeling, and it can be used to then mail a sample of people and build a response model on just that segment.
BILLING AND OFFER SEGMENTATION MODELS
Billing applications are really not about whether you should market to people: they're about what is the best offer. What we want to do is balance the best offer based on the different segments and what the rollout economics are. Be careful: think about what your costs will be including scoring costs to assign people into the different segments; what the loss of efficiency or economy of scale could be if you separate people into too many creative segments, or if you want to keep a different inventory of premiums for one group vs. another.
There are costs associated and you should bear that in mind at the beginning of your planning. Give your analysts some input about what are the reasonable number of categories from the financial perspective. Often it will be two or three.
In one project, we built a model for credit card payment, and identified who was likely to pay by credit card and who was not. It was interesting to see that with the right type of offer, all the customers paid at a higher rate.
In another project, a segmentation project, we broke customers into three different groups. One group was ideal for a non-sweeps offer (our standard offer) - affluent and typically middle aged. They had a far superior net response rate than the older, more sweeps-like segment or the younger active segment, suggesting that a different offer might be appropriate for the other two segments.
The bottom line is that database marketing is actually a diverse area, but fortunately the tools and techniques are also very versatile. We can use response models or regression models for new business. Clusters and other techniques can be useful for getting insight about your customer and/or segmenting customers to assign the optimal offer. In general, investing in your database modeling can be worthwhile and far superior to intuition for even the most experienced marketer.
Continued from page 2.
Mary McKenna Trepp, executive marketing director, Database Marketing Services, has been with Time Inc. since 1997. Her responsibilities include all database marketing analysis activities in Time Consumer Marketing, including modeling, ad hoc analysis of prospects for acquisitions, renewals, targeting for advertising editions, etc. Prior to working at Time, she was manager of analysis for relationship marketing at Advanta, a credit card company.
COPYRIGHT 2003 Copyright by Media Central Inc., A PRIMEDIA Company. All rights reserved.
COPYRIGHT 2003 Gale Group
