A Mental Health Professional Weighs in on the Ja
Report
The enclosed “Review of the Davis Ja Report” was written
by Dee Roth, MA, a Board Member of the Pacific Institute for Health
Innovation (PIHI). Ms. Roth was Chief of Program Evaluation
and Research for the Ohio Department of Mental Health for 36 years.
She has been the principal investigator on two National
Institute of Mental Health (NIMH) grants, and has extensive mental
health experience at the national level, including with NIMH and
with the Substance Abuse and Mental Health Services Administration
(SAMHSA).
Ms. Roth’s review of the Ja Report reflects PIHI’s
commitment to promote clinical quality management approaches to
ensure that mental health services provided are effective.
Among other insights, Ms. Roth notes:
- The Ja Report’s goal to “determine the level of
community expertise available to promote better discharges and
linkages” does not appear to have been evaluated.
- In well-done, credible evaluations, it is critical to state
clearly what the central question being investigated is. She
notes the Ja Report confuses the central question it put forth,
because the Ja Report didn’t at least measure services provided
in the community (or at LHH) against a set of established quality
standards, and didn’t evaluate the effectiveness of services
provided. The Ja Report didn’t measure the quality
or quantity of behavioral
health care services at LHH or in the community.
- Though the Ja Report claimed it had used a “random sample”
in reviewing LHH patient charts, Roth notes this is incorrect,
because the Study Group was 29.4% black and 44.0% white, but
that the chart review sample was 46% black and 28% white, so
the chart review group cannot have been regarded as either a
random, or a representative, sample of the overall behavioral
health patient group. Instead, she suggests that race could
be a strong interactive variable around discharges. [Although
Roth didn’t discuss this, one has to consider whether Ja’s
overall recommendations may apply to some demographic clusters,
but not to the needs of other demographic clusters.]
- That though the Ja Report focuses extensively on so-called
“grounded theory methodology” (a technique used to
analyze qualitative data) Ja used to interpret information
gleaned from 41 interviewees, there are indications Ja’s
grounded theory methodology wasn’t followed. [Roth
did not comment on how this single problem may negate many of
the Ja Report’s findings.]
Roth sees a more serious problem being that the “Key Questions
for Interviews” on page 54 in Ja’s Report may have
severely biased what was heard from interviewees, and how interviewers
coded and interpreted the grounded theory data. The “Key
Questions” lead-in asserts at least four times that some
question whether LHH staff will tell the truth, honestly, introducing
a severe bias that interviewers may have approached their task
assuming (or being instructed) to doubt with pre-conceptions
the veracity of information provided by LHH staff chosen to be
interviewed. [Note: Many people are deeply concerned by
the Ja Report’s possible implication that LHH staff would
deliberately “lie” (not be “truthful”) to
Ja’s interviewers.]
- That while percentages of LHH residents with “severe
mental health illnesses” Ja presented were technically correct,
evaluators experienced with the policy
use of findings would have calculated the percentages differently,
providing a clearer, more accurate picture. A careful read
of Roth’s review compared to Ja’s Report shows Ja moved
the denominator from 12 to 9, misreporting that five (55.6%)
of those with a severe mental illness did not have
documentation of psychiatric evaluation or service in their charts,
inflating that only 41.7% did not have such documentation.
Ja also claimed (by reducing the denominator) that 44.4% of patients
with a severe mental health diagnosis had received psychiatric
treatment during the study period, when, in fact, only 33.3%
needed such treatment. Roth was too kind by not commenting
on why Ja inflated both indicators by double digits, possibly
to claim an inflated need to hire additional behavioral staff.
- That the system-level group being formed in response to Ja’s
recommendations needs to be made a broadly-representative group,
and that it focus on Laguna Honda’s currently-mandated responsibilities
and mission. The system-level group that was actually appointed
is meeting outside of the public view (its meetings are not open
to the public), and hasn’t released any of its deliberations.
- That Ja Report’s recommendation that “higher salaried
physicians be replaced by registered nursing staff, social workers
and psychologists,” came completely out of the blue. Roth
sees no evidence in the data presented in the Ja Report that
would support such a premature conclusion made even before policy
and operations groups sat down to figure out appropriate treatment
needs of patients. Roth notes
this very specific recommendation “about replacing doctors
seems to echo politics, not data.”
But Roth doesn’t specifically address a key admission
in the Ja Report: Ja acknowledged that the five patients
who had no documentation of psychiatric evaluation were actually
being treated for psychiatric conditions by their primary care
MDs. Doesn’t that portend that cutting LHH’s primary
care MDs might then inadvertently cut psychiatric care to a significant
number of patients, in addition to cutting the medical care that
they need?
If Ja did not stop to consider that cutting MDs might have
an unintended consequence of cutting psychiatric care at the same
time, what kind of nonsense is his recommendation to eliminate
MDs at Laguna Honda Hospital, where 100% of its patients need
medical care? How could he have overlooked such a glaring
inconsistency? (Or did Ja just assume nobody would notice
the unintended consequence?)
Top
_______
Copyright (c) 2009 by Committee to Save LHH. All rights
reserved. This work may not be reposted anywhere on the
Web, or reprinted in any print media, without express written
permission. E-mail the Committee
to Save LHH.