The AI revolution can be leveraged to strengthen sex and gender health equity, rather than reinforce bias.
From assisting doctors with diagnoses to suggesting advanced treatments, Artificial Intelligence (AI) is transforming health and medicine.
But AI has predominantly been developed by men, based on data sets that prioritise men’s bodies and health needs. That means many AI models are riddled with gender and sex biases — posing a health risk to women, as well as nonbinary patients.
With these biases in medicine coming under the spotlight in recent years, will AI widen existing healthcare inequities — or can it be harnessed to help bridge the gap?
Biased data
The calibre of AI totally depends on the quality of the large data sets that are fed into the underlying machine learning algorithms within its software programs.
If data either excludes or under-represents relevant sectors of the global population, ill-informed AI can pose serious health risks — from missed diagnoses, to compromising the interpretation of medical imaging, to incorrect intervention recommendations.
Problems start with gender biases underlying the very coding of the AI software language.
Infiltration of masculine stereotypes into AI have emerged — from the apparently unconscious default to the male pronoun “he” when options are ambiguous, to alarming healthcare applications that threaten diagnosis and treatment.
For example, in the field of psychiatry, when men describe trauma symptoms, they are more likely to be diagnosed with post-traumatic stress disorder (PTSD), while women describing the same symptoms are at higher risk of receiving a personality disorder diagnosis.
This kind of gender bias can (and often does) influence a women’s access to health care or her management within the healthcare system — and it appears this bias is replicated in AI models.
A 2020 study from the US found that Natural Language Processing AI models used in psychiatry demonstrate significant gender biases.
The research paper, published in PLos One, warned that AI models that screen for psychopathology or suicide will make mistakes if they are trained predominantly on data written by white men, because language is shaped by gender. Men and women express suicidal distress differently, for example.
Crucially, awareness of these types of issues is gathering and initiatives to avert bias are emerging — often driven by women, such as Bioinfo4women-B4W, a program of the Barcelona Supercomputing Centre.
This example also reminds us that considerations around bias and gendered language in AI must extend beyond the English language, in order to be relevant to AI development around the globe.
Opportunities for inclusive design
But concerns don’t stop at the level of language. What if something so basic as our body build is not considered when AI is being developed?
As the use of AI expands into safety product design, we have an unprecedented opportunity to build better products by crafting in features that adequately cater to our human bodies — female and male.
Average female and male bodies have proportionality differences; we can’t simply scale from one to the other.
This point was driven home during the COVID pandemic, when wearing personal protective equipment (PPE) became mandatory.
Despite around 70 percent of global healthcare workers being women, PPE has been designed around a male body. A Canadian survey identified that ill-fitting PPE was not only responsible for a failure to offer adequate protection, but also that oversized and ill-fitting gear posed a significant accident risk.
More studies are needed on this topic, but researchers have already proposed building AI-designed PPE. Ensuring that sex traits are considered in PPE design could be expected to improve safety.
Moves in the right direction
The accuracy of AI-assisted clinical diagnoses is completely reliant on the robustness of the underpinning data sets. Without actively accounting for sex and gender bias in historical data sets, AI may contribute to missed or mis-diagnoses.
Fortunately, adjusting for such biases appears to lead to better healthcare outcomes for women.
For example, the traditional risk assessment score for heart attacks, the Global Registry of Acute Coronary Events (GRACE), was updated in 2022 to incorporate AI predictive models that account for sex-specific disease characteristics.
This update has revolutionised the performance of this assessment tool. The success stems from separate analysis of male and female data– which guides more female patients to lifesaving early intervention, helping overcome structural biases in patient management.
A practical example of an AI model designed to address and reduce gender bias is SMARThealth Pregnancy GPT. This tool, developed by The George Institute for Global Health, aims to improve access to guideline-based pregnancy advice for women living in rural and remote communities in India.
The concept was to develop a large language model chatbot that would be contextually sensitive and clinically accurate — and avoid entrenching harmful stereotypes.
The George Institute team worked closely with community health workers, clinicians and women living in rural communities, to co-create and refine the tool’s algorithm. Clinicians also scored AI-generated answers on accuracy, appropriateness for community health workers, completeness and risk of bias, which helped improve the chatbot’s responses.
The chatbot showcases AI’s potential in building healthcare worker capacity and enhancing health education in resource-limited settings — while avoiding bias and promoting women’s rights.
Gender-sensitive AI development could similarly improve countless other medical technologies that rely on data diversity and integrity for accuracy: for example, tailoring personalised treatments; predicting treatment responses; performing certain robot-assisted surgeries; monitoring patients remotely; virtual health care; and acceleration of drug discovery.
Initiatives to advance improved sex and gender equity in healthcare have begun to emerge in recent years, too. They include the newly launched Australian Centre for Sex and Gender Equity in Health and Medicine and the UK Medical Science Sex and Gender Equity.
These programs are actively advocating for routine consideration of sex and gender from discovery to translational research, including AI applications, to ensure scientific rigour as a robust foundation for advancing health and medical care.
AI is the future of healthcare, and we can’t afford to replicate the past mistakes of health inequities perpetrated by ignoring sex and gender. It is time to program AI to chart our course toward an ethical destiny.
The information in this article is provided for educational purposes and is not to be taken as medical advice.
This article is a collaboration between Dr Sue Haupt, Prof Bronwyn Graham and Prof Jane Hirst.
- Prof Bronwyn Graham is the Director of the Centre for Sex and Gender Equity in Health and Medicine at the George Institute for Global Health and a Professor in the School of Psychology, UNSW.
- Prof Jane Hirst is the Program Director of Women’s Health at the George Institute for Global Health, School of Public Health, Imperial College London.
Originally published under Creative Commons by 360info™.