PROGRAM EVALUATION Quick and Dirty (actual versus expected) Propensity matching Identifying the best programs.

Presentación del tema: "PROGRAM EVALUATION Quick and Dirty (actual versus expected) Propensity matching Identifying the best programs."— Transcripción de la presentación:

1 PROGRAM EVALUATION Quick and Dirty (actual versus expected) Propensity matching Identifying the best programs

2 Quick and Dirty Develop a risk equation (e.g. recidivism) Estimate the risk for each participant (Yhat) Average the risk for the service participants Compare the average risk with actual recidivism

3 Quick and Dirty Benefits Easy Acknowledges risk of participants No control group necessary Downsides Does not identify who service is effective with No bootstrap

4 PROPENSITY MATCHING Develop risk equation Identify variables significantly associated with outcome Identify program participants and non-participants For every participant, find their identical twin Compare the rates for the two groups Boot strap

5 Participant 1 BestClose Pretty close OK Participant 2 Best Next best Participant 3 Best Next best OK Participant 4 Best Next best CloseOK Participant 5 Matched Control Matching Process

6 Matched control 1 Participant 1 Matched control 2 Participant 2 Matched control 3 Participant 3 Rate for participantsRate for control group

7 IDENTIFYING THE BEST PROGRAMS Develop equations for each program (who are they effective with) Disassociate participants from programs Randomly select one participant and place in “best” program Randomly select second participant and place in their best program The first program to fill is your best program (if matching were used to place youth

8 Identifying the “Best” Programs 12 Not 11 1 1111111 2 NOT 2 2 11111112 2 2 211111112 NOT 2 2 22 2 2 2 1 1212 11111 2 NOT 2 2 1 2 2121 222 1 2121 22 1 2 NOT 2 2 2 122 2 11111