Close Menu
Healthtost
  • News
  • Mental Health
  • Men’s Health
  • Women’s Health
  • Skin Care
  • Sexual Health
  • Pregnancy
  • Nutrition
  • Fitness
What's Hot

7 Summ Salt Swaps for a healthy heart

June 9, 2025

AI converts sights for visual impairments

June 9, 2025

30 minutes Kettlebell Core Workouts to strengthen your abdomen

June 9, 2025
Facebook X (Twitter) Instagram
  • About Us
  • Contact Us
  • Privacy Policy
  • Terms and Conditions
  • Disclaimer
Facebook X (Twitter) Instagram
Healthtost
SUBSCRIBE
  • News

    AI converts sights for visual impairments

    June 9, 2025

    AI model detects brain tumors with high precision using epigenetic fingerprints

    June 8, 2025

    The new study warns long -term risks from germ transplants

    June 8, 2025

    Self-collection tests could be a solution to increase access to cervical cancer control

    June 7, 2025

    Preeclampsia associated with long -term brave disbelief and worsening the effects of stroke

    June 7, 2025
  • Mental Health

    Heart attack or panic attack? Why do young people call ambulances for non -managed stress

    June 7, 2025

    My journey, my development, my truth – uninterrupted

    June 6, 2025

    Why I am fighting for mental health change

    June 3, 2025

    Girls with painful periods are twice as high as their peers to have symptoms of anxiety or depression

    June 2, 2025

    Does psychiatric drug kill creativity? Rejecting Van Gogh’s myth

    May 29, 2025
  • Men’s Health

    30 minutes Kettlebell Core Workouts to strengthen your abdomen

    June 9, 2025

    Scientists identify genetic indications that connect air pollution to neurodegeneration

    June 7, 2025

    Do you want a stronger back? This Powerlifter’s secret weapon is a secret weapon

    June 7, 2025

    Chains, bands and greater profits: Guide to deal with resistance

    June 6, 2025

    Phil Stutz, True Magic & Healing Mankind: Useful tools for today’s World – Part 2: Universe 1 and Universe 2

    June 6, 2025
  • Women’s Health

    Making the connection between collagen and recovery from exercise

    June 8, 2025

    Alice Connors for purpose and progress

    June 4, 2025

    8 teenagers of vitamins must actually get

    June 4, 2025

    Reasons for frequent urination

    June 3, 2025

    Life with myalgian encephalomyelitis/chronic fatigue syndrome

    June 3, 2025
  • Skin Care

    Semaglutide for Weight Loss – Summer Safety and Side Effects Guide

    June 9, 2025

    10 signs it’s time to see an acne expert

    June 8, 2025

    11 Important facts for Botox Botox hood

    June 7, 2025

    Liposcopy: Is it right for you?

    June 7, 2025

    Ideas for father’s day and beyond

    June 4, 2025
  • Sexual Health

    Teenagers and sexual education during the Covid-19 pandemic and beyond

    June 9, 2025

    The odd rise of cases of syphilis to heterosexual men in the UK

    June 8, 2025

    The Role of Certified LGBTQ Certified Sexual Therapist – Sexual Health Alliance

    June 7, 2025

    How Pride paved the way for sexual well -being

    June 5, 2025

    Best male masturbation positions Female

    June 4, 2025
  • Pregnancy

    Love in Melbourne Australia – Tiffany Rose Maternity Blog UK

    June 8, 2025

    How to remain calm with high blood pressure during pregnancy?

    June 7, 2025

    Pink Stork’s flagship product – as shown in “Empowered by Meg Ryan”

    June 7, 2025

    You don’t have to know everything – only that before birth

    June 6, 2025

    Top 100 Names of Girls 2024

    June 5, 2025
  • Nutrition

    7 Summ Salt Swaps for a healthy heart

    June 9, 2025

    The busy mom’s driver for intestinal-brain connection

    June 7, 2025

    Healthy Banana Bread (Child Approved) Sarah Remat

    June 6, 2025

    The secret to the loss of persistent belly fat by registered dieticians

    June 5, 2025

    Why are you always tired and what to do

    June 5, 2025
  • Fitness

    Creamy all the toast Avocado bagel (easy, salty breakfast!)

    June 8, 2025

    Men’s Health Month: Dealing longevity vacuum

    June 7, 2025

    Best 12 biceps exercises ranked: build larger and stronger weapons

    June 6, 2025

    Próximamente el 10 de junio: 25 minutes of train de Joel Freeman

    June 5, 2025

    The best weight counter to add to your home gym in your 2025

    June 5, 2025
Healthtost
Home»News»The impressive diagnostic skills of GPT-4 were demonstrated
News

The impressive diagnostic skills of GPT-4 were demonstrated

healthtostBy healthtostApril 21, 2024No Comments5 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Reddit WhatsApp Email
The Impressive Diagnostic Skills Of Gpt 4 Were Demonstrated
Share
Facebook Twitter LinkedIn Pinterest WhatsApp Email

In a recent study published in the journal PLOS Digital Healthresearchers assessed and compared the clinical knowledge and diagnostic reasoning capabilities of large language models (LLMs) with those of human ophthalmology experts.

Study: Large language models approximate expert-level clinical knowledge and reasoning in ophthalmology: A cross-sectional study. Image credit: ozrimoz / Shutterstock

Record

Generative Pre-trained Transformers (GPTs), GPT-3.5 and GPT-4, are advanced language models trained on massive Internet-based datasets. They power ChatGPT, a conversational artificial intelligence (AI) noted for its medical application success. Despite previous models struggling in specialized medical tests, the GPT-4 shows significant advances. Concerns remain about data “contamination” and the clinical relevance of test scores. Further research is needed to validate the clinical applicability and safety of language models in real medical settings and to address existing limitations in their expert knowledge and reasoning abilities.

About the study

The questions for the Part 2 examination of the Fellowship of the Royal College of Ophthalmologists (FRCOphth) have been extracted from a specialist manual that is not widely available online, minimizing the likelihood of these questions appearing in LLM training data. A total of 360 multiple-choice questions spanning six chapters were extracted and a pool of 90 questions was isolated for a mock examination used to compare the performance of LLMs and MDs. Two researchers aligned these questions with the categories established by the Royal College of Ophthalmologists and classified each question according to Bloom’s levels of cognitive processes. Questions with non-text elements that were unsuitable for LLM entry were excluded.

Exam questions were entered into versions of ChatGPT (GPT-3.5 and GPT-4) to collect responses, repeating the process up to three times per question where necessary. Once other models such as Bard and HuggingChat became available, similar tests were conducted. Correct answers, as defined by the textbook, were noted for comparison.

Five specialist ophthalmologists, three intern ophthalmologists and two general practitioners independently completed the virtual examination to assess the practical application of the models. Their answers were then compared with the answers of the LLMs. After testing, these ophthalmologists rated the LLMs’ responses using a Likert scale to rate accuracy and relevance, without knowing which model provided which response.

The statistical design of this study was powerful enough to detect significant performance differences between LLMs and human doctors, aiming to test the null hypothesis that both would perform similarly. Various statistical tests, including chi-squared and paired t-tests, were applied to compare performance and assess the consistency and reliability of LLM responses versus human responses.

Study results

Of the 360 ​​questions contained in the FRCOphth Part 2 exam manual, 347 were selected for use, including 87 from the mock exam chapter. The exceptions were mostly questions with pictures or tables, which were unsuitable for input into LLM interfaces.

Performance comparisons revealed that GPT-4 significantly outperformed GPT-3.5, with a correct response rate of 61.7% versus 48.41%. This advance in GPT-4 capabilities was consistent across different types of questions and subjects, as outlined by the Royal College of Ophthalmologists. Detailed results and statistical analyzes further confirmed the strong performance of GPT-4, making it a competitive tool even among other LLMs and human doctors, especially junior doctors and trainees.

Exam features and detailed performance data.  Topic and question type distributions are presented along with the scores achieved by LLMs (GPT-3.5, GPT-4, LLaMA and PalM 2), Ophthalmologists (E1-E5), Ophthalmologists (T1-T3) and non-ophthalmologists (T1-T3). specialized young doctors (J1- J2).  Median scores do not necessarily add up to the overall median score, as fractional scores are impossible.Exam features and detailed performance data. Topic and question type distributions are presented along with the scores achieved by LLMs (GPT-3.5, GPT-4, LLaMA and PalM 2), Ophthalmologists (E1-E5), Ophthalmologists (T1-T3) and non-ophthalmologists (T1-T3). specialized young doctors (J1- J2). Median scores do not necessarily add up to the overall median score, as fractional scores are impossible.

In the specially adapted 87-question mock exam, the GPT-4 not only outperformed among LLMs, but also scored comparably to ophthalmologists and significantly better than juniors and interns. Performance across different groups of participants showed that while specialist ophthalmologists maintained the highest accuracy, trainees approached these levels, far outperforming junior non-ophthalmology specialists.

Statistical tests also highlighted that agreement between responses given by different LLMs and human participants was generally low to moderate, indicating variation in reasoning and application of knowledge between groups. This was particularly evident when the differences in knowledge between the models and human doctors were compared.

A detailed examination of the mock questions against the actual exam standards showed that the mock setup closely mirrored the actual FRCOphth Part 2 Written Exam in difficulty and structure, as agreed upon by the ophthalmologists involved. This alignment ensured that the assessment of LLMs and human responses was based on a realistic and clinically relevant framework.

In addition, qualitative feedback from ophthalmologists confirmed a strong preference for GPT-4 over GPT-3.5, correlating with quantitative performance data. The higher accuracy and relevance scores for the GPT-4 highlighted its potential utility in clinical settings, particularly in ophthalmology.

Finally, an analysis of the cases where all LLMs failed to provide the correct answer showed no consistent patterns related to the complexity or subject matter of the questions.

demonstrated diagnostic GPT4 impressive skills
bhanuprakash.cg
healthtost
  • Website

Related Posts

AI converts sights for visual impairments

June 9, 2025

AI model detects brain tumors with high precision using epigenetic fingerprints

June 8, 2025

The new study warns long -term risks from germ transplants

June 8, 2025

Leave A Reply Cancel Reply

Don't Miss
Nutrition

7 Summ Salt Swaps for a healthy heart

By healthtostJune 9, 20250

(Say that ten times fast!) Over the years, many clients have told me that summer…

AI converts sights for visual impairments

June 9, 2025

30 minutes Kettlebell Core Workouts to strengthen your abdomen

June 9, 2025

Semaglutide for Weight Loss – Summer Safety and Side Effects Guide

June 9, 2025
Stay In Touch
  • Facebook
  • Twitter
  • Pinterest
  • Instagram
  • YouTube
  • Vimeo
TAGS
Baby benefits body brain cancer care Day Diet disease exercise finds Fitness food Guide health healthy heart Life Loss Men mental Natural Nutrition Patients Pregnancy protein research reveals Review risk routine sex sexual Skin study Therapy Tips Top Training Treatment Understanding ways weight women Workout
About Us
About Us

Welcome to HealthTost, your trusted source for breaking health news, expert insights, and wellness inspiration. At HealthTost, we are committed to delivering accurate, timely, and empowering information to help you make informed decisions about your health and well-being.

Latest Articles

7 Summ Salt Swaps for a healthy heart

June 9, 2025

AI converts sights for visual impairments

June 9, 2025

30 minutes Kettlebell Core Workouts to strengthen your abdomen

June 9, 2025
New Comments
    Facebook X (Twitter) Instagram Pinterest
    • About Us
    • Contact Us
    • Privacy Policy
    • Terms and Conditions
    • Disclaimer
    © 2025 HealthTost. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.