New M
N
Measures off Ri
Risk-Adjusted
k Adj t d Rates
R t
of Complications
p
of Elective Surgical
g
Procedures Reveal Substantial
Variation in Hospital Performance
Michael Pine,
Pine M.D.,
M D M.B.A.
MBA
Michael Pine and Associates, Inc.
mpine@consultmpa.com
p @
p
773-643-1700
Overview
 Study
Design
 Direct
Composite Measures of Complications
 Variations
in Risk and Complication Rates
 Surrogate
Measures of Complications
 Comparison
 Summary
of Direct and Surrogate Measures
and Conclusions
© 2010 Michael Pine and Associates, Inc.
Study Design
 Procedures
within 2 Days of Elective Admission
 Each Case Assigned to 1 of 24 Types of Procedures
 Assignment based on Procedure and Diagnosis Codes
3
Years of Hospital Claims Data (1,623,579 Cases)
 324 Hospitals in California and New York State
 Present-on-Admission Coding Passed Quality Screening
 Types
of Risk-Adjusted Outcome Measures
 Composite Based on Hospital-Acquired Complications
 Surrogate Based on Postoperative Length of Stay
 Any Coded Hospital-Acquired Surgical Complication
 Prolonged Risk-Adjusted
Risk Adjusted Postoperative Length of Stay
© 2010 Michael Pine and Associates, Inc.
Types of Elective Surgical Procedures
1.
Brain, Spine, Peripheral Nerve
13. Pancreas
2.
Face, Head, Neck
14. Hernia
3.
Lungs
15. Abdomen, Peritoneum, Spleen
4
4.
Thorax
Thorax, Esophagus
16 Adrenal
16.
Adrenal, Kidney,
Kidney Urinary Tract
5.
Heart, Pericardium
17. Male Reproductive
6
6.
Pacemaker
Pacemaker, Defibrillator
18 Female Reproductive
18.
7.
Coronary Angioplasty
19. Cesarean Section
8.
Blood Vessels
20. Bone, Minor Joint
9.
Lymph Nodes
21. Major Joint Replacement
10. Stomach, Intestines
22. Vertebrae
11. Liver
23. Muscle, Soft Tissue, Skin
12. Gall Bladder, Bile Ducts
24. Amputation
© 2010 Michael Pine and Associates, Inc.
Quality of POA Coding
Pe
ercent of Hospitals
40%
30%
20%
10%
0%
0%
>95%
>90% to 95%
>80% to 90%
>70% to 80%
Score on POA Screens
New York State
© 2010 Michael Pine and Associates, Inc.
California
<=70%
Discrete Versus Composite Measures
 Discrete
Outcome Measures (e.g., sepsis)
 E
Easily
il Related
R l t d to
t Clinical
Cli i l Care
C
 Low Incidence Often Reduces Statistical Power
 Same
S
Case
C
M
May B
Be C
Counted
dA
As M
Multiple
l i l Events
E
 Composite
Measures (e.g., death or high-risk
high risk
inpatient surgical complication)
 More Difficult to Interpret
p
Clinically
y
 Higher Incidence Increases Statistical Power
 Can Be Structured to Avoid Overweighting
g
g Individual
Cases with Multiple Complications
© 2010 Michael Pine and Associates, Inc.
Types of Coded Surgical Complications
2.1%
4.2%
90.2%
9.8%
3.1%
0.3%
No Complication
Dead (†)
C
Complication
li ti
Hi h Ri k Complications
High-Risk
C
li ti
(>4% †)
Moderate-Risk Complications
Low-Risk Complications
p
(<2%
(
†)
© 2010 Michael Pine and Associates, Inc.
Composite Coded Complications
C
Complicat
ion Rate (percent))
12%
10%
8%
6%
4%
2%
0%
Very Severe
Complication
Severe Complication
Death
Moderate‐Risk Complication
© 2010 Michael Pine and Associates, Inc.
Moderately Severe
Complication
High‐Risk Complication
Low‐Risk Complication
Variation in Rates of Very Severe Complications
Death or High-Risk Complications by Type of Elective Surgical Procedure
20%
Perrcent Co
omplications
18%
16%
14%
12%
10%
8%
6%
4%
2%
0%
19 23 18 17 6
7 20 2
9 21 22 1 12 25 14 10 16 8 24 15 4
Proced ral Category
Procedural
Categor
© 2010 Michael Pine and Associates, Inc.
3 11 13 5
C-Statistics for Risk-Adjustment Models
1.00
0 95
0.95
C-Sta
atistic
0.90
0 85
0.85
0.80
0.75
0.70
0.65
0.60
0.55
0.50
Mortality
High-Risk
Moderate-Risk
T
Type
off Complication
C
li ti
© 2010 Michael Pine and Associates, Inc.
Low-Risk
Percentage of Outlier Hospitals
One or More Composite Measures of Coded Surgical Complications
18%
Incrreasing Sensitiivity
Pe
ercent o
of Hospitals
16%
14%
12%
10%
8%
6%
4%
2%
0%
High Outlier
Very Severe
© 2010 Michael Pine and Associates, Inc.
Low Outlier
plus Severe
plus Moderately Severe
Reported Events Versus Surrogates
 Reported
Events (e.g., surgical wound infection)
 Are Clinically Meaningful
 Are Subject to Reporting Biases
 Objective
Surrogates (e.g., prolonged hospitalization)
 Are More Difficult to Interpret
 Are Less Subject to Reporting Bias
 Combined
Measures (e.g., coded complication with
prolonged risk-adjusted length of stay)
 Maintain Clinical Credibility
 Reduce Reporting Bias
© 2010 Michael Pine and Associates, Inc.
Risk-Adjusted Postoperative Lengths of Stay
All Live Discharges After Knee Replacement at a Single Hospital
OBS L
LOS minus
s PRED LO
OS (days)
60
50
40
30
20
10
0
-10
1
31
61
91
121
151
181
211
241
271
301
331
361
391
421
451
481
511
Sequence Identifier
Average
© 2010 Michael Pine and Associates, Inc.
3 Std Dev
Normal LOS
Long LOS w Cpl
Long LOS w/o Cpl
Long Risk-Adjusted Post-op Lengths of Stay
Percent with L
Long Po
ost-op L
LOS
By Type of Reported Complication
60%
50%
40%
30%
20%
10%
0%
High-Risk
Moderate-Risk
Low-Risk
T
Type
off Complication
C
li ti
© 2010 Michael Pine and Associates, Inc.
None
Comparative Hospital Performance
All Coded Complications Versus Only Complications with Long LOS
55%
Percent of Hosp
pitals
50%
45%
Good
A
Agreement
Poor Agreement
40%
35%
30%
25%
20%
Complete
Disagreement
g
15%
10%
5%
0%
Coded Compl =
Long LOS
Coded Compl
Coded Compl
Both Outliers with
Outlier; Long LOS Average; Long LOS 1 High & 1 Low
Average
Outlier
(color = long LOS)
High Outlier
© 2010 Michael Pine and Associates, Inc.
Low Outlier
Average
Comparative Hospital Costs
All Coded Complications Versus Only Complications with Long LOS
Hospital Cost (d
dollars)
$70,000
$60,000
$50,000
$40,000
$30,000
$
,
$20,000
$10,000
$0
Colon Resection
Hip Replacement
No Complication
C d dC
Coded
Complication
li i with
ihL
Long LOS
© 2010 Michael Pine and Associates, Inc.
CABG Surgery
Only Coded Complication
Risk-Adjusted Complication Rates with Long LOS
Magnitude of Differences between Observed and Predicted Rates
14%
Average
Quality
High
g Quality
y
Pe
ercent o
of Hospitals
12%
Poor Quality
y
10%
8%
6%
4%
2%
0%
<=-1.5%
-1% to
>-1.5%
-0.5% to
>-1%
0 to >0.5%
N.S.
(52.5%)
0 to
<0.5%
0.5% to
<1%
Observed minus Predicted (percent)
© 2010 Michael Pine and Associates, Inc.
1% to
<1.5%
>=1.5%
The Bottom Line
Properly
P
l constructed
t
t d composite
it
surrogate outcome measures
can improve the inferential power
off analyses
l
off clinical
li i l quality.
lit