Connected research, organized for discovery.

This study evaluated three generative AI models (GPT-4o, Claude-3.5-Sonnet, and Gemini-1.5-Pro) for lung adenocarcinoma histopathological assessment using 310 TCGA-LUAD slides and 87 local patient slides for validation, finding that Claude-3.5-Sonnet performed best with high grading accuracy (AUC=0.82) and moderate stability (ICC=0.59); a machine learning prognostic model using Claude-3.5-Sonnet’s extracted features achieved good performance (C-index=0.72) and effectively stratified patients into risk groups (HR=6.44), demonstrating the potential value of GenAI in pathological diagnosis and prognostic prediction for lung adenocarcinoma management.

This retrospective study evaluated the diagnostic performance of three generative AI models (GPT-4-turbo, Gemini-pro-vision, and Claude-3-opus) in interpreting chest CT scans from 404 patients with lung neoplasms (n=184) and non-malignant conditions (n=210), with external validation using TCGA (n=106) and MIDRC (n=110) datasets. Following standardized CT processing, diagnostic accuracy was assessed using chi-square tests, ROC analyses, and Likert scale scoring across varying clinical scenarios. While Gemini and Claude outperformed GPT in single-image diagnostics, all models showed decreased accuracy when additional CT slices or clinical histories were incorporated. ROC analysis revealed limited but improvable performance in simplified prompting environments or when integrated with machine learning methods. Feature analysis indicated that models primarily relied on morphology and margins for malignancy predictions but struggled with critical imaging features and occasionally fabricated data. These findings demonstrate that while generative AI shows variable potential for pulmonary CT diagnosis, significant limitations in processing complex multimodal information present substantial challenges for clinical integration, emphasizing the need for continued development to enhance model robustness and reliability before healthcare adoption.

Large Language Models (LLMs) show promise in medical applications, yet their performance in analyzing chest CT reports—the gold standard for thoracic disease diagnosis—remains understudied. This retrospective analysis evaluated five leading LLMs (Claude-3.5-Sonnet, GPT-4, GPT-3.5-Turbo, Gemini-Pro, Qwen-Max) on 13,489 chest CT reports covering 13 common thoracic conditions using both multiple-choice and open-ended questioning methodologies. Performance was quantified using Subjective Answer Accuracy Rate (SAAR), Reference Answer Accuracy Rate (RAAR), and ROC curve analysis, with GPT-3.5-Turbo additionally undergoing fine-tuning on 200 high-performing cases. GPT-4 achieved the highest RAAR of 75.1% in multiple-choice questioning, followed by Qwen-Max (66.0%) and Claude-3.5 (63.5%), significantly outperforming GPT-3.5-Turbo (41.8%) and Gemini-Pro (40.8%), with multiple-choice questioning consistently yielding better results than open-ended approaches across all models. Fine-tuning substantially improved GPT-3.5-Turbo’s initially suboptimal performance, and model effectiveness varied notably across different diseases and organ conditions. These findings demonstrate that general-purpose LLMs can effectively interpret chest CT reports when optimally configured, offering evidence-based guidance for integrating AI tools into surgical workflows to enhance preoperative planning, risk stratification, and diagnostic efficiency in thoracic procedures.

This study systematically evaluates three state-of-the-art multimodal generative AI models (GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro) for pathological grading and prognostic prediction in clear cell renal cell carcinoma (ccRCC). Using 499 TCGA slides and 349 external samples, the study applies a standardized feature extraction protocol and robust validation, comparing GenAI models with classical deep learning (ResNet-50, DenseNet-121, ABMIL) and image analysis tools. Claude-3.5-Sonnet achieved highest stability (ICC=0.76) and micro-average AUC=0.87, outperforming classical methods. The best machine learning prognostic model attained an average C-index of 0.739, and key features included stage, calcification, sarcomatoid differentiation, and vascular networks. The work demonstrates that advanced GenAI enhances accuracy and consistency in ccRCC pathology and holds strong clinical promise, especially for resource-limited settings

Objective This study aims to systematically investigate the potential harms of Large Language Models (LLMs) in the peer review process. Background LLMs are increasingly used in academic processes, including peer review. While they can address challenges like reviewer scarcity and review efficiency, concerns about fairness, transparency and potential biases in LLM-generated reviews have not been thoroughly investigated. Methods Claude 2.0 was used to generate peer review reports, rejection recommendations, citation requests and refutations for 20 original, unmodified cancer biology manuscripts obtained from eLife’s new publishing model. Artificial intelligence (AI) detection tools (zeroGPT and GPTzero) assessed whether the reviews were identifiable as LLM-generated.All LLM-generated outputs were evaluated for reasonableness by two expert on a five-point Likert scale. Results LLM-generated reviews were somewhat consistent with human reviews but lacked depth, especially in detailed critique. The model proved highly proficient at generating convincing rejection comments and could create plausible citation requests, including requests for unrelated references. AI detectors struggled to identify LLM-generated reviews, with 82.8% of responses classified as human-written by GPTzero. Conclusions LLMs can be readily misused to undermine the peer review process by generating biased, manipulative, and difficult-to-detect content, posing a significant threat to academic integrity. Guidelines and detection tools are needed to ensure LLMs enhance rather than harm the peer review process.

Large language models (LLMs) have shown substantial applications and development potential in bioinformatics, particularly for complex biological data analysis. This review systematically examines LLM development and applications in bioinformatics, with emphasis on protein and nucleic acid structure prediction, omics analysis, drug design and screening, and biomedical literature mining. The review also highlights LLM strengths in end-to-end learning and knowledge transfer, as well as current challenges such as model interpretability and data bias. It explores future LLM potentials in cross-modal learning and interdisciplinary innovation, aiming to position LLMs as essential bioinformatics research tools and guide further advances in the biomedical field

Recent advancements in artificial intelligence (AI) like ChatGPT have expanded possibilities for patient education, yet its impact on perioperative anxiety in total knee arthroplasty (TKA) patients remains unexplored. In this single-blind, randomized controlled pilot study from April to July 2023, 60 patients were randomly allocated using sealed envelopes to either ChatGPT-assisted or traditional surgeon-led informed consent groups. In the ChatGPT group, physicians used ChatGPT 4.0 to provide standardized, comprehensive responses to patient queries during the consent process, while maintaining their role in interpreting and contextualizing the information. Outcomes were measured using Hospital Anxiety and Depression Scales (HADS), Perioperative Apprehension Scale-7 (PAS-7), Visual Analogue Scales for Anxiety and Pain (VAS-A, VAS-P), Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), and satisfaction questionnaires. Of 55 patients completing the study, the ChatGPT group showed significantly lower anxiety scores after informed consent (HADS-A: 10.48 ± 3.84 vs 12.75 ± 4.12, P = .04, Power = .67; PAS-7: 12.44 ± 3.70 vs 14.64 ± 2.11, P = .01, Power = .85; VAS-A: 5.40 ± 1.89 vs 6.71 ± 2.27, P = .02, Power = .75) and on the fifth postoperative day (HADS-A: 8.33 ± 3.20 vs 10.71 ± 3.83, P = .01, Power = .79; VAS-A: 3.41 ± 1.58 vs 4.64 ± 1.70, P = .008, Power = .85). The ChatGPT group also reported higher satisfaction with preoperative education (4.22 ± 0.51 vs 3.43 ± 0.84, P<.001, Power = .99) and overall hospitalization experience (4.11 ± 0.65 vs 3.46 ± 0.69, P = .001, Power = .97). No significant differences were found in depression scores, knee function, or pain levels. ChatGPT-assisted informed consent effectively reduced perioperative anxiety and improved patient satisfaction in TKA patients. While these preliminary findings are promising, larger studies are needed to validate these results and explore broader applications of AI in preoperative patient education.

This correspondence summarizes the current progress and future directions of large language models (LLMs) in drug development. LLMs have transformed key steps in drug development—including target identification, molecular design and optimization, drug repurposing, preclinical studies, and clinical trials—by improving efficiency, reducing costs, and increasing success rates. The paper reviews current representative applications such as Llama-Gram, 3DSMILES-GPT, GPCR LLM, and others, as well as innovative methodological frameworks. It also analyzes key challenges regarding data quality, reliability, explainability, high computing requirements, and regulatory/ethical hurdles, and offers recommendations for addressing technical and social limitations to unlock the full potential of LLMs in pharma

While drug combinations are increasingly important in disease treatment, predicting their therapeutic interactions remains challenging. This review systematically analyzes computational methods for predicting drug combination effects through multi-omics data integration. We comprehensively assess key algorithms including DrugComboRanker and AuDNNsynergy, and evaluate integration approaches encompassing kernel regression and graph networks. The review elucidates artificial intelligence applications in predicting drug synergistic and antagonistic effects.

The commentary discusses the study by Chang et al. on the use of ChatGPT-4 for providing colonoscopy follow-up recommendations based on clinical data. The authors express concerns regarding the randomness inherent in large language models (LLMs) like ChatGPT-4, which can generate inconsistent or contradictory responses to the same input. They emphasize the importance of evaluating the consistency of AI models through repeated queries to capture variability and ensure reliable outputs for clinical decision-making. Additionally, the commentary critiques the method of converting continuous or categorical variables into numerical ones by taking the mean of a range. This simplification may introduce inaccuracies and fail to reflect the clinical flexibility needed in some situations. The authors suggest that more accurate handling of variable transformations and addressing the randomness of LLM responses are crucial for improving the clinical applicability and reliability of AI systems. In conclusion, the commentary advocates for further research to ensure that AI models like ChatGPT-4 provide consistent and precise recommendations before being integrated into clinical practice.

This study evaluated three Gen-AI models (GPT-4-turbo, Gemini-pro-vision, and Claude-3-opus) for lung tumor recognition in CT images, finding Gemini performed best in single-image analysis (92.21% accuracy), followed closely by Claude (91.49%), while GPT lagged (65.22%); performance declined with increased complexity for all models, though Claude showed the most stability; feature analysis revealed strengths in recognizing tumor morphology and margins but significant errors in complex cases; after optimization through regression techniques, performance improved substantially (AUC ~0.89), suggesting promising potential for Gen-AI in simplified lung CT analysis while highlighting current limitations for complex clinical applications.

Biosafety laboratories are crucial for safe research with pathogenic microorganisms, enabling significant advances in microbiology and epidemiology to address emerging infectious diseases. However, laboratory work carries inherent risks without stringent biosafety protocols and training. This study explores whether ChatGPT and other generative AI models can effectively answer biosafety-related questions and enhance laboratory management. Using a dataset of 62 text-based and 8 image-based questions, the performance of several large language models (LLMs) was evaluated. Models such as Gemini Pro, Claude-3, Claude-2, and GPT-4 achieved high accuracy in text-based tasks, while image-based performance was slightly lower. The study discusses the transformative potential of AI-driven systems for real-time monitoring, personalized training, workflow documentation, and predictive maintenance in biosafety labs. Despite their promise, current AI limitations include bias, limited specialized data, and real-time constraints, along with ethical concerns regarding data security and transparency. Recommendations are provided for overcoming technical and ethical challenges to optimize the integration of generative AI in biosafety laboratory management.

The rapid advancements in large language models (LLMs) such as ChatGPT have raised concerns about their potential impact on academic integrity. While initial concerns focused on ChatGPT’s writing capabilities, recent updates have integrated DALL-E 3’s image generation features, extending the risks to visual evidence in biomedical research. Our tests revealed ChatGPT’s nearly barrier-free image generation feature can be used to generate experimental result images, such as blood smears, Western Blot, immunofluorescence and so on. Although the current ability of ChatGPT to generate experimental images is limited, the risk of misuse is evident. This development underscores the need for immediate action. We suggest that AI providers restrict the generation of experimental image, develop tools to detect AI-generated images, and consider adding “invisible watermarks” to the generated images. By implementing these measures, we can better ensure the responsible use of AI technology in academic research and maintain the integrity of scientific evidence.

The commentary emphasizes the potential of artificial intelligence (AI) chatbots in providing empathetic and readable responses to cancer-related questions but raises significant concerns regarding their accuracy and consistency. While AI models are effective in certain aspects like engagement and communication, the authors stress that inaccuracies in the information provided could pose serious risks to patient safety. The inherent randomness of large language models (LLMs) can lead to inconsistent responses, further complicating their reliability. The authors argue that for AI to be safely integrated into clinical settings, models must prioritize accuracy, consistency, and safety, ensuring that they consistently deliver reliable information to patients.

The commentary explores the potential of large language models (LLMs) in combating medical disinformation, acknowledging both the risks and opportunities they present. While agreeing with concerns about LLMs enabling the spread of false medical information, the authors highlight that the problem predates AI technology. They argue that rather than focusing solely on restricting LLMs, efforts should be made to empower users to assess the reliability of online health information. The commentary demonstrates how well-trained LLMs can be powerful tools in identifying and correcting inaccurate medical claims. The authors’ experiment with multiple popular LLMs showed that most responses flagged misinformation, particularly in areas like vaccine safety, by providing evidence-based explanations. These models’ ability to reference the latest authoritative information, such as CDC data, reinforces their potential to challenge health misinformation, including emerging threats like novel infectious diseases. The commentary stresses that the positive potential of LLMs in addressing medical disinformation should not be overlooked.

This paper proposes a comprehensive framework for standardized evaluation of generative AI in medicine, addressing the current lack of standardized assessment methods. The authors recommend a three-pronged approach: establishing standardized scoring criteria with multiple complementary evaluation methods (including Reference Answer Accuracy Rate, Subjective Answer Accuracy Rate, and Strict Accuracy Rate), implementing rigorous evaluation processes (including pre-review alignment, multi-reviewer scoring, and independent audits), and utilizing statistical analysis to quantify scoring differences and refine evaluation methods. The recommendations aim to enhance reliability and consistency in assessing generative AI’s capabilities in healthcare applications, acknowledging both the technology’s potential and the need for careful validation before clinical implementation.

The introduction of ChatGPT-4V’s ‘Chat with images’ feature represents the beginning of the era of large multimodal models (LMMs), which allows ChatGPT to process and answer questions based on uploaded images. This advancement has the potential to transform how surgical teams utilize radiographic data, as radiological interpretation is crucial for surgical planning and postoperative care. However, a comprehensive evaluation of ChatGPT-4V’s capabilities in interpret radiological images and formulating treatment plans remains to be explored. Three types of questions were collected: (1) 87 USMLE-style questions, submitting only the question stems and images without providing options to assess ChatGPT’s diagnostic capability. For questions involving treatment plan formulations, a five-point Likert scale was used to assess ChatGPT’s proposed treatment plan. The 87 questions were then adapted by removing detailed patient history to assess its contribution to diagnosis. The diagnostic performance of ChatGPT-4V was also tested when only medical history was provided. (2) We randomly selected 100 chest radiography from the ChestX-ray8 database to test the ability of ChatGPT-4V to identify abnormal chest radiography. (3) Cases from the ‘Diagnose Please’ section in the Radiology journal were collected to evaluate the performance of ChatGPT-4V in diagnosing complex cases. Three responses were collected for each question. ChatGPT-4V achieved a diagnostic accuracy of 77.01% for USMLE-style questions. The average score of ChatGPT-4V’s treatment plans was 3.97 (Interquartile Range: 3.33-4.67). Removing detailed patient history dropped the diagnostic accuracy to 19.54% (P<0.0001). ChatGPT-4V achieved an AUC of 0.768 (95% CI: 0.684-0.851) in detecting abnormalities in chest radiography, but could not specify the exact disease due to the lack of detailed patient history. For cases from ‘Diagnose Please’ ChatGPT provided diagnoses consistent with or very similar to the reference answers. ChatGPT-4V demonstrated an impressive ability to combine patient history with radiological images to make diagnoses and directly design treatment plans based on images, suggesting its potential for future application in clinical practice.

Prostate cancer (PCa) is a common malignancy in males. The pathology review of PCa is crucial for clinical decision-making, but traditional pathology review is labor intensive and subjective to some extent. Digital pathology and whole-slide imaging enable the application of artificial intelligence (AI) in pathology. This review highlights the success of AI in detecting and grading PCa, predicting patient outcomes, and identifying molecular subtypes. We propose that AI-based methods could collaborate with pathologists to reduce workload and assist clinicians in formulating treatment recommendations. We also introduce the general process and challenges in developing AI pathology models for PCa. Importantly, we summarize publicly available datasets and open-source codes to facilitate the utilization of existing data and the comparison of the performance of different models to improve future studies.

This correspondence discusses the assessment of generative AI in the field of medicine, specifically focusing on ChatGPT’s capabilities in passing the American Heart Association (AHA) Basic Life Support (BLS) and Advanced Cardiovascular Life Support (ACLS) exams. The authors critique earlier studies that attempted to evaluate ChatGPT-3.5’s performance on these exams, noting that the lack of repeated testing may have underestimated its abilities. The authors’ subsequent study, which involved multiple rounds of questioning, found that ChatGPT-3.5 was capable of passing both exams. However, earlier limitations such as the inability to process image-based questions were overcome with the release of ChatGPT-4V, which was tested on complete AHA exams. While acknowledging the importance of exams in assessing knowledge, the authors argue that evaluating AI’s clinical potential should extend beyond these exams. Just as medical students progress through various stages of evaluation—beginning with written exams and advancing to case-based assessments and clinical practice—the authors suggest that generative AI should also be assessed through simulated clinical scenarios that reflect real-world applications. They emphasize the need for comprehensive evaluations of AI tools in healthcare, moving towards practical, real-world testing to better understand their potential impact in clinical practice.

This study highlights the significant language and cultural biases inherent in large language models (LLMs), particularly in the context of Traditional Chinese Medicine (TCM). The research compares the performance of eight prominent LLMs—four developed by Chinese companies and four by Western companies—on the National Medical Licensing Examination for TCM. The results revealed that Chinese-developed models, such as Qwen-max and GLM-4, significantly outperformed their Western counterparts, such as ChatGPT-3.5 and ChatGPT-4, in answering TCM-related questions. The Western models, with their training predominantly on English-language data, struggled with the cultural nuances and specific terminology used in TCM. The study emphasizes the need for localized models that are trained on culturally relevant data to ensure better accuracy and applicability in specific fields, like TCM. It suggests that AI models tailored to local languages and medical practices can offer more precise solutions and meet the unique needs of different populations. Additionally, these localized models could contribute to preserving cultural knowledge, improving data security, and addressing specific healthcare needs. In conclusion, the research advocates for the development of AI systems that integrate multilingual and culturally diverse training to enhance their effectiveness in global healthcare settings.

This study evaluated the capabilities of the newly released ChatGPT-4V, a large language model with visual recognition abilities, in interpreting electrocardiogram waveforms and answering related multiple-choice questions for assisting with cardiovascular care.

The increasing interest in the potential applications of generative artificial intelligence (AI) models like ChatGPT in health care has prompted numerous studies to explore its performance in various medical contexts. However, evaluating ChatGPT poses unique challenges due to the inherent randomness in its responses. Unlike traditional AI models, ChatGPT generates different responses for the same input, making it imperative to assess its stability through repetition. This commentary highlights the importance of including repetition in the evaluation of ChatGPT to ensure the reliability of conclusions drawn from its performance. Similar to biological experiments, which often require multiple repetitions for validity, we argue that assessing generative AI models like ChatGPT demands a similar approach. Failure to acknowledge the impact of repetition can lead to biased conclusions and undermine the credibility of research findings. We urge researchers to incorporate appropriate repetition in their studies from the outset and transparently report their methods to enhance the robustness and reproducibility of findings in this rapidly evolving field.

Ensuring the authenticity and reliability of scientific research is a critical responsibility in academic medicine, as image manipulation severely undermines scientific integrity and may misdirect resources and further investigations. Recent data indicate that approximately 1 in 25 biomedical articles contains inappropriate duplicate images, prompting a rising number of retractions. Advances in artificial intelligence (AI), such as the large multimodal model ChatGPT-4 Vision (ChatGPT-4 V), offer novel tools for detecting image manipulation. In this study, we assessed ChatGPT-4 V’s capability to identify manipulated images by testing it on representative samples from 12 biomedical articles suspected of containing image duplication or fraudulent modifications. The model performed well in identifying simple forms of image duplication and splicing, but its effectiveness declined with more complex manipulations such as rotation and flipping. While the integration of such AI systems holds promise for enhancing research oversight, important challenges remain, notably the potential for false positives and negatives and the limited representativeness of training data for certain research fields. Close collaboration between medical experts and AI developers is needed to optimize these tools’ reliability and applicability. Our preliminary findings underscore the potential of multimodal AI in protecting scientific integrity and highlight the continuing essential role of human expertise in the research validation process.

ChatGPT-4V model with image interpretation tested for distinguishing kidney & prostate tumors from normal tissue.

The study by Fijačko et al. tested ChatGPT’s ability to pass the BLS and ACLS exams of AHA, but found that ChatGPT failed both exams. A limitation of their study was using ChatGPT to generate only one response, which may have introduced bias. When generating three responses per question, ChatGPT can pass BLS exam with an overall accuracy of 84%. When incorrectly answered questions were rewritten as open-ended questions, ChatGPT’s accuracy rate increased to 96% and 92.1% for the BLS and ACLS exams, respectively, allowing ChatGPT to pass both exams with outstanding results.

This correspondence explores the best approach to assessing generative AI, particularly in the context of medical applications like ChatGPT. The authors discuss previous studies evaluating ChatGPT’s ability to pass the American Heart Association (AHA) BLS and ACLS exams, noting that earlier assessments underestimated its potential due to methodological limitations, such as the lack of repeated testing and exclusion of image-based questions. With the release of ChatGPT-4V, these limitations were overcome, allowing for more comprehensive testing. The authors argue that evaluating AI’s capabilities should not be confined to exams alone. Similar to how medical students progress from basic knowledge assessments to real-world clinical practice, AI should also be assessed in simulated clinical scenarios to better understand its application in healthcare. The authors advocate for comprehensive evaluations of generative AI that move beyond theoretical exams to real-world clinical simulations, enabling a deeper understanding of its potential impact in clinical settings.