PROGRAM EVALUATION Quick and Dirty (actual versus expected) Propensity matching Identifying the best programs
Quick and Dirty Develop a risk equation (e.g. recidivism) Estimate the risk for each participant (Yhat) Average the risk for the service participants Compare the average risk with actual recidivism
Quick and Dirty Benefits Easy Acknowledges risk of participants No control group necessary Downsides Does not identify who service is effective with No bootstrap
PROPENSITY MATCHING Develop risk equation Identify variables significantly associated with outcome Identify program participants and non-participants For every participant, find their identical twin Compare the rates for the two groups Boot strap
Participant 1 BestClose Pretty close OK Participant 2 Best Next best Participant 3 Best Next best OK Participant 4 Best Next best CloseOK Participant 5 Matched Control Matching Process
Matched control 1 Participant 1 Matched control 2 Participant 2 Matched control 3 Participant 3 Rate for participantsRate for control group
IDENTIFYING THE BEST PROGRAMS Develop equations for each program (who are they effective with) Disassociate participants from programs Randomly select one participant and place in “best” program Randomly select second participant and place in their best program The first program to fill is your best program (if matching were used to place youth
Identifying the “Best” Programs 12 Not NOT NOT NOT NOT