Churn Norris – Retain your customers with ML

Presentación del tema: "Churn Norris – Retain your customers with ML"— Transcripción de la presentación:

1 Churn Norris – Retain your customers with ML
Pau Sempere Churn Norris – Retain your customers with ML

2

3 About me Pau Sempere Working with SQL Server since 2009
Microsoft Machine Learning MCSA SolidQ LinkedIn:

4 Agenda What is Churn? In balance there is virtue KISS
How did we get here?

5 What is Churn?

6 What is Churn? CUSTOMERS STAY ABANDON

7 What is Churn? CUSTOMERS 70% 30% STAY ABANDON

8 What is Churn? CUSTOMERS 95% 5% STAY ABANDON

9 What is Churn?

10 What is Churn?

11 What is Churn? CUSTOMERS WHO? WHY?

12 What is Churn? //debate entre los asistentes, cada uno lo definirá de una manera particular. Si no participan, darles pie, proponer diferentes puntos de vista

13 Key concepts about Churn
CLEAR DEFINITION HISTORICAL DATA CONTEXT PUNTOS CLAVE Esto aplica a todos los problemas de ML del universo, pero especialmente a Churn ya que habla de comportamiento humano

14 DEMO Basic Churn

15 Looking for a better model
Random forest has: N_estimators Max_Depth Max_leaf_nodes … We need to find a better model exploring such combinations

16 DEMO Tuned model

17 In balance there is virtue

18 In balance there is virtue
Churn is an unlikely event

19 In balance there is virtue
OVER SAMPLING UNDER SAMPLING HYBRID METHODS

20 In balance there is virtue

21 In balance there is virtue

22 In balance there is virtue

23 In balance there is virtue

24 In balance there is virtue
Extra techniques (algorithm based) WEIGHT VECTORS BALANCED EXPLORATION

25 DEMO Balancing Churn

26 KISS

27 KISS

28 Tree-based methods Decision Trees Random Forests Boosted Trees (lightgbm, xgboost, rxFastTrees…) DISCARD THE LESS USED FEATURES GAIN COVER FREQUENCY* Todos los métodos basados en árboles se basan en partir el subespacio de datos para encontrar subespacios más “puros”. De ahí nace “Gain” (cada algoritmo lo llama de una manera), representa la reducción de la entropía o la ganancia de información  generación de subespacios más seguros de la clasificación Cover representa el numero relativo de observaciones relacionadas con esa feature  Si tienes 100 observaciones, 4 feats y 3 arboles, y la feature 1 se ha usado para decidir los nodos hoja para 10, 5 y 2 observaciones en los arboles 1, 2 y 3 respectivamente, el cover será = 17. Frequency es la cantidad de veces que la feature se usa. No todos los algoritmos ofrecen esta

29 Recursive Feature Elimination
Different combinations of features are tried iteratively and performance data is gathered Requires an estimator (model)

30 DEMO A simpler life

31 How did we get here?

32 How did we get here? Churn is a business problem Business is key If our model is ONLY precise, is not enough

33 Why do we need to interpret?
To understand the results we are receiving from our models Debug and improve models Avoid bias Prescriptive analytics

34 What can we see? Returned values Measures
Regression Classification Measures Can we rely ONLY in measures?

35 Clever Hans

36 Partial dependency plots

37 DEMO Show me the model!

38 QUESTIONS?

39 THANK YOU!