Artificial intelligence can turn medicine into a multitude of ways, including its promise to act as a reliable diagnostic in busy clinical doctors.
In the last two years, privately owned AI models, also known as a closed source models, have been distinguished in resolving hard medical cases that require complex clinical reasoning. Specifically, these AI models of closed sources have exceeded open source, so -called because their source code is available to the public and can be modified and modified by anyone.
Has the AI open source was trapped?
The answer seems to be yes, at least when it comes to such an AI open source model, according to the findings of a new NIH -funded study, led by researchers at the Harvard Medical School and was done in collaboration with Clinicians in Harvard associated with the Beth -Medical Center.
The results, published on March 14 Jama Health ForumShow that an AI open source challenger tool called Llama 3.1 405B is performed at the same level as the GPT-4, a top-of-the-line closed code model. In their analysis, researchers compared the performance of the two models in 92 cases of mystical The Journal of Medicine of New England Weekly rubrine diagnostic provocative clinical scenarios.
The findings indicate that AI open source tools are becoming more and more competitive and could offer a valuable alternative to privately owned models.
From our own knowledge, this is the first time that an AI open source model has matched the performance of the GPT-4 in such provocative cases evaluated by doctors. It is really amazing that the lama models fell so fast with the top -notch model. Patients, care providers and hospitals will win from this competition. ”
Arjun Manrai, Senior Writer, Assistant Professor of Biomedical Informatics, Blavatnik Institute in HMS
The advantages and disadvantages of AI open source systems and a closed source
The AI AI AI and the closed source differ in various important ways. First, open source models can be downloaded and executed on private computers of a hospital, maintaining home data at home. On the contrary, closed sources models operate on external servers, demanding users to transmit private data externally.
“The open source model is likely to be more attractive to many information officers, hospital managers and doctors, as there is something fundamentally different about the data left by the hospital for another entity, even a credible,” said a leader of the Study, the leader of the Study Inflayics.
Second, IT medical and professionals can modify open source models to tackle unique clinical and research needs, while closed source tools are generally more difficult to adapt.
“This is the key,” Buckley said. “You can use local data to adjust these models, either in basic ways or in sophisticated ways, so that they are tailored to the needs of your own doctors, researchers and patients.”
Thirdly, AI developers closed sources, such as Openai and Google, host their own models and provide traditional customer support, while open source models place responsibility for regulating and maintaining models to users. And at least so far, closed sources have proven to be easier to integrate with electronic health records and hospital IT infrastructure.
Open Code AI against AI closed source: A score card to resolve provocative clinical cases
Both AI algorithms, such as AI algorithms, trained in huge sets of data including medical textbooks, peer research, clinical decision support tools and patient anonymous data, such as case studies, test results, scans and confirmed diagnoses. By examining these material mountains in Hyperspeed, algorithms learn patterns. For example, what do they look like with cancerous and benign tumors in the pathology? What are the first signs of heart failure? How do you distinguish between a normal and inflammatory colon in CT scanning? When presented with a new clinical scenario, AI models compare incoming information with the content that has been assimilated during training and suggest possible diagnoses.
In their analysis, the researchers looked at the Llama in 70 provocative cases of Nejm clinics used previously to evaluate GPT-4 performance and described in a previous study led by Adam Rodman, HMS Assistant Professor of Medicine at Beth Israel Deaconess and co-author. In the new study, the researchers added 22 new cases published after the end of the Llama training period to protect the possibility that the Lama may have unintentionally encountered some of the 70 published cases during his basic education.
The open source model showed a genuine depth: Llama made a correct diagnosis in 70 % of cases, compared to 64 % for GPT-4. He also ranked the right choice as a first sentence of 41 percent of the year, compared to 37 percent for GPT-4. For the subset of 22 newer cases, the open source model scored even higher, making the correct call 73 percent of the time and identifying the final diagnosis as a top sentence of 45 percent of the year.
“As a doctor, I have seen a lot of the focus on powerful large linguistic models that focus around privately owned models that we cannot run locally,” Rodman said. “Our study suggests that open source models can be just as strong, giving doctors and health systems much greater control over how these technologies are used.”
Each year, about 795,000 patients in the United States die or suffer from permanent disability due to a diagnostic error, according to a 2023 report.
In addition to direct damage to patients, diagnostic errors and delays can give a serious financial burden on the healthcare system. Inaccurate or delayed diagnoses can lead to unnecessary tests, inappropriate treatment and, in some cases, serious complications that become more difficult – and more expensive – to manage over time.
“Used wisely and embedded in the current health infrastructure, AI tools could be invaluable copilots for busy clinical doctors and serve as reliable diagnostic assistants to enhance both accuracy and diagnosis.” “But it remains vital for doctors to help drive these efforts to make sure AI works for them.”
Source:
Magazine report:
Buckley, Ta, et al. (2025). Comparison of open source borders and privately owned large linguistic models for complex diagnoses. Jama Health Forum. doi.org/10.1001/jamahealthfor.2025.0040.
