Meet iToBoS Series - Ethical AI in iToBoS

Online, 31/05/2024.

On Friday 31st of May 2024, the MPNE hosted the latest instalment of the “Meet iToBoS” online workshop series.

The live webinar, titled “Ethical AI in iToBoS,” was moderated by the MPNE and presented by Zita McCrea, a senior research analyst from Trilateral Research, to members of the MPNE community and iToBoS partners.

The webinar started with an introduction to Trilateral Research and its expertise in Ethical AI, and  continued with an introduction to the concept of Ethical AI and how it should be considered at various stages in a product lifecycle. This includes the innovation and R&D stages (iToBoS), but it is also a forward-thinking approach, considering clinical integration and long-term uses to ensure sustainability of the solutions.

The webinar covered different proposed AI frameworks, such as the EU AI Act, which regulates the use of AI to ensure AI can be considered trustworthy. In addition, the Act gives guidelines on how AI can be evaluated for certain risks and what parameters it must meet to demonstrate its safety in use. The webinar also covered the EU’s digital strategy guidelines, a set of seven key requirements that AI systems should meet in order to be deemed trustworthy, as shown below.

Although iToBoS will align with all seven requirements, the webinar focused on three principles: human agency and oversight, privacy and data governance, and transparency. The webinar also included a discussion of the principles of informed consent and when it should be deployed in the clinical AI space, explaining how iToBoS partners gathered informed consent through a dynamic eConsent process and the benefits of this type of approach.

Following this discussion, the group debated how clinicians should be educated about AI applications and machine learning, especially considering the rapid growth of emerging technologies in the healthcare domain.

A related emerging issue is genetic data and genetic data sharing, which underlies many AI health innovations, and the possible consequences it may have in relation to security risks. Although the health data Regulation and open-sharing roadmap give guidance specific to genetic data sharing, researchers agree that these initiatives don’t provide adequate security for genomic data sharing. The webinar facilitator raised this issue and agreed that clarification was needed to support clinicians and researchers sharing the data fuelling AI technologies safely.

Why is genetic data so sensitive? Besides the fact a sequence of 30 to 80 single nucleotide polymorphisms (SNPs) could uniquely identify an individual, genetic data provides sensitive information about genetic conditions and predispositions to certain diseases. If breached, this information could stigmatize participants and be used against them to deny employment or insurance opportunities, even if these pre-dispositions never materialize. Genetic data not only provides information about the sequenced individual, but also about their relatives, raising complex ethical questions about consented participants’ obligations towards their family members’ present and future health.  

In addition, genetic data is still evolving and may hold hidden information that could be identified in the future, potentially leading to incidental findings and the identification of variants not of current significance. iToBoS has taken a number of steps to mitigate against these risks. These include an “opt-in” consent for genetic testing and offering participants the opportunity to speak with a genetic councillor before the biological sample is taken. In addition, raw genetic data will not be used in the project’s AI developments. Only pre-specified genotypes determined to be of importance with concern to melanoma will be returned to participants, and the AI developers will only receive the participants’ polygenic risk score.

Finally, the facilitator demonstrated the project’s approach to prioritizing transparency and explainability in the development stages. The group discussed ways the clinician-patient relationship may change to a machine-clinician-patient relationship. For instance, the iToBoS image detection tool will highlight moles of concern for further investigations, indicating what qualities the algorithm flagged as potentially cancerous. The clinician will understand how the AI output was reached and take this into account when making clinical decisions, thus including the tool in the decision-making process and, by extension, the relationship with the patient.

Once the talk was completed a Q&A was held with the members and the meeting was closed. The webinar can be found on the MPNE website.