By Ahmed Shaikh

May/June 2023

In 2021, the Khalil Center, Maristan Muslim and other mental health organizations publicized a statement in the Journal of the American Medical Association Psychiatry (JAMA; 2021 Sep; 78(9): 1041–1044.) that Muslims are far more suicidal than non-Muslims or atheists. They then spread this claim throughout much of the media to raise money, especially zakat.

Upon closer inspection, this deeply unethical claim came about due to conflicts of interest and cooking numbers to get a contrived result, followed by outright fraud.

An Opinion Poll

The Institute for Social Policy and Understanding (ISPU) annual “American Muslim Poll” selects a small sample to arrive at generalized conclusions. As with all polls, there is a margin of error.

In 2019, part of the process involved the Institute of Muslim Mental Health (Dr. Hamada Altalib), the Stanford Muslim Mental Health & Islamic Psychology Lab (Dr. Rania Awaad) and the Khalil Center (Dr. Hooman Keshavarzi), who purchased questions on suicide from ISPU to be included in the survey, which focuses on Jews, Muslims and the “general population.”

They found that 7.9% of Muslims, 5.1% of Protestants, 6.1% of Catholics and 3.6% of Jews reported trying to kill themselves at some point. While this is not twice as many Muslims attempting suicide as others, it does look like Muslims in the sample attempted suicide at a higher rate. However, according to ISPU, these numbers are meaningless because the margins of error were quite large. So, comparing the groups with the largest gaps, Jews and Muslims, is actually something not worth discussing. The margin of error for the Muslim survey was ±4.9%, while for Jews it was a whopping ±7.6%.

Indeed, when ISPU first announced the results of their 2019 Muslim poll, they did not mention suicide at all, as doing so would have been pointless.

The authors of the JAMA article used another method to compare religion and suicide attempts. They compared a reference group — Protestants — to Muslims, calculating an “odds ratio.” The authors found that the numbers had a statistically insignificant p-value of 0.1. So, unless they had another way to calculate the rate of suicide among members of different religions, they were out of luck. As it happens, though, they devised a way to adjust the numbers and get headline-grabbing results.

Adjusting for Unknown Demographic Factors

The claim that Muslims were twice as likely as others to attempt suicide was the result of “adjusting for demographic factors” for which the authors had no data. They used “regression analysis” — a technique to adjust for race. This technique found that Muslims are twice as likely to attempt suicide as are non-Muslims.

In a comment and a long-form analysis, Dr. Osman Umarji (statistician and scholar, the Yaqeen Institute) made two important criticisms: (1) Since the authors started with nothing, it was inappropriate for them to perform a “regression analysis” and (2) they were doing regression analysis for non-existent variables. One of the biggest problems here was considering “Arab” as a race — a race assumed to be exclusively Muslim, which is not the case. The ISPU data that the authors worked with contained no non-Muslim Arabs, which caused a “correlation” between the groups “Muslim” and “Arab” (Dr. Umarji’s full reanalysis can be found at tinyurl.com/4n2j6buk).

Regression analysis isn’t designed to work with data that is incomplete in a way that causes a correlation. Statisticians call this effect, which is a little bit like stacking the deck, a “suppressor effect.”

Attack in Public, Agree in Private

Umarji was attacked vigorously online by the study’s authors — Awaad accused him of peddling “disinformation” and Altalib accused him of lying. The authors claimed they had three unnamed statisticians from top institutions look at the numbers, all of whom reportedly deemed the “suppressor effect” found by Umarji “baseless.”

The problem was that before the authors accused Umarji, they had privately written a response to JAMA and conceded that he did have a point about the suppressor effect: “In this sample, being Arab may be a confounder and serve as a suppressor variable. However, as we argue above, it is critical to include race in the model so that readers can see the potential effect. Our intent of publishing this Research Letter is to draw attention to an under-recognized issue and promote further discourse on suicide across communities” (emphasis added).

Elsewhere, the authors claim that a “suppressor” may be a good thing for reasons that appear hypothetical and not grounded in fact. So, they contrived a “fact” that Muslims were twice as likely as others to attempt suicide to “promote further discourse.” One would hope that the envisioned discourse would be based on facts and not fiction.

If you run an organization that advocates for the view that Muslim mental health is in horrible shape and your organization happens to offer a solution — if we send you more money — that may very well color what your research results will look like. A Muslim mental health advocacy group is unlikely to herald a study finding that Muslims attempt suicide at the same rates as everyone else or that they ran the numbers and have no idea about anything either way (leaving aside the question of who would publish such a thing).

The Incredible Shrinking P-Value

One area where the authors did push back both privately and publicly (in a public comment to the JAMA article) was the notion that it was improper for them to do a regression analysis in the first place when they started with nothing.

They claimed that they had in fact started with something — the odds ratios I told you about earlier. The odds ratios had a p-value (for probability) of 0.1 with the reference group, Protestants. Thus, the authors had only statistical noise. However, apparently to bolster their argument while attacking Umarji, they brazenly changed the p-value to 0.05 with no explanation for why an already published number was changed. In other words, they were defending their work with a fake number.

From the JAMA article with the “unadjusted odds ratio.” I point to the critical p-value, which shows no statistical significance.

“Changing the ‘p-value’ here appears to be an attempt to defend doing ‘regression analysis.’ Without the numbers needed to create something that could be published in a medical journal, they ‘tortured’ the data to make it happen. If you torture the data enough, it’ll tell you whatever you want to hear.”

Driven by Conflicts of Interest

Here is the claim (from the JAMA article) that funders had no role:

Dr. Keshavarzi and Dr. Awaad failed to disclose the obvious conflicts of interest (https://jamanetwork.com/journals/jama/fullarticle/2712191). Both belong to organizations that stood to benefit from their claims.

The authors also claimed that the funders — the organizations headed by the authors themselves — of the study had no role in the “design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review or approval of the manuscript; and the decision to submit the manuscript for publication.”

“Funding/Support: The Stanford Muslim Mental Health & Islamic Psychology Lab, the Khalil Center, and the Institute for Muslim Mental Health contributed funding to include mental health-related questions in the Institute for Social Policy and Understanding national survey. Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study: collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Additional Contributions: We would like to thank the support of the Institute of Muslim Mental Health, the Stanford Muslim Mental Health & Islamic Psychology Lab, and the Khalil Center.”

All three funding organizations were headed by article’s authors. In the same article, it’s clear that the authors were involved with all aspects of the study:

“Author Contributions: Dr Awaad had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: Awaad, Keshavarzi, Mogahed, Altalib. Acquisition, analysis, or interpretation of data: Awaad, El-Gabalawy, Jackson-Shaheed, Zia, Altalib. Drafting of the manuscript: Awaad, El-Gabalawy, Jackson-Shaheed, Zia, Keshavarzi. Critical revision of the manuscript for important intellectual content: Awaad, El-Gabalawy, Zia, Mogahed, Altalib. Statistical analysis: El-Gabalawy, Jackson-Shaheed, Zia. Obtained funding: Awaad, Altalib. Administrative, technical, or material support: Awaad, El-Gabalawy, Keshavarzi, Mogahed. Supervision: Awaad, Altalib.”

Virtually nothing about this study looks right. The claim was contrived, and much of the Muslim mental health sector comes out looking bad here.

What about Mental Health?

Awaad and the other JAMA Psychiatry paper authors produced a conclusion about Muslim Americans they could not legitimately support and then lied to support it. There should be no confusion about this.

The authors hyped this in the media and raised money, including zakat. They took full advantage of their obvious conflicts of interest, which they did not bother to disclose, slandered a Muslim scholar when they knew he was telling the truth and privately admitted as much. They then doctored the results of a p-value to make themselves look good. Our community needs to know about this, even if it means that the public will have less confidence in Muslim mental health professionals.

When we discuss the stigma surrounding mental health, it doesn’t mean that Muslims fail to seek mental health because they are ignorant. Rather, it’s because of a long history of mental health professionals using mental health as a cover for nefarious purposes, including episodes in recent history like torturing Muslims (www.nytimes.com, May 1, 2015) and Countering Violent Extremism (https://muslimjusticeleague.org/for-health-care-professionals/). Stigma comes from dishonest efforts like the Muslim suicide study.

Unfortunately, too many Muslim donors, especially zakat donors, as well as imams have been taken in by this sham.

Muslim mental health professionals can be a benefit to society. This is more likely to happen without lying and cheating.

Ahmed Shaikh is an estate planning attorney in Southern California and the co-author of “Estate Planning for the Muslim Client” (ABA Publishing, 2019). This article is adapted from his newsletter on Muslim nonprofits and leadership, which is available at ehsan.substack.com.

Tell us what you thought by joining our Facebook community. You can also send comments and story pitches to [email protected]. Islamic Horizons does not publish unsolicited material.  

The post Opinion: A Questionable Claim appeared first on Islamic Horizons.