[Skip Global Navigation]

SPSS Tech Tips

SPSS Direct Home

Creating and comparing models in Clementine® 11.1

This is a complimentary tip available only to SPSS Direct subscribers.

Question

I have Neural Net, C5, and CHAID models in my stream, and I would like to connect all three to one Evaluation node so that I can see the lift stream for each model on the same graph. I've tried to do this via a SuperNode, but I receive an error message that reads: "Cannot determine where to insert connectors." Is it possible to analyze the lift for all three models and see the results on the same graph?

Answer

If you use Clementine 11.0 or Clementine 11.1 and have a license for the Clementine Classification Module, then you can use the Binary Classifier node to do this.

The Binary Classifier node allows you to create and compare models for binary outcomes (Yes/No, Churn/Don’t, etc.) using Neural Net, Decision Tree (C5.0, CART, QUEST, and CHAID), and Logistic Regression algorithms. To evaluate the lift of the models, follow the steps below.

Begin by attaching the Binary Classifier node to your stream:
1. Go to the model palette and double-click to customize its settings
2. In the Expert tab of the Binary Classifier node, deselect the models that don’t interest you (see Figure 1 below)

Figure 1: Select the models you’d like to use by accessing the Expert tab of the Binary Classifier node.
(Click to enlarge)

3. Execute the stream

From the results window, select the models you would like to see on the evaluation chart by checking the appropriate boxes in the “Generate” column. Next, go to the drop-down menu at the top of the window, and click on Generate -> Evaluation Chart.

Figure 2: Specify which models you’d like to generate evaluation charts for in the Binary Classifier Results window.
(Click to enlarge)

Then select what chart types you would like to create, as well as the plot and line options. In this instance, select “Lift” from the next window and click “OK.”


Figure 3: Choose a chart type for your evaluation chart.
(Click to enlarge)

As shown in Figure 4, your evaluation chart will be a lift chart which depicts the lift for all of the models that you selected, along with any requested plots and lines.

Figure 4: This lift evaluation chart displays the lift for C5, CHAID, and Neural Net models.
(Click to enlarge)

In this example, the C5.0 model appears to provide the best lift. For a good model, lift should start well above 1.0 on the left, remain on a high plateau as you move to the right, and then trail off sharply toward 1.0 on the right side of the chart.

For more information regarding the Binary Classifier node, please refer to the Clementine Node Reference Guide, which is found in the documentation folder within your installation location. For guidelines on the interpretation of the model evaluation results, please reference the Clementine online help topic, “Reading the Results of a Model Evaluation.”