Prediction of American Society of Anesthesiologists Physical Status Classification from preoperative clinical text narratives using natural language processing

Chung, Philip; Fong, Christine T.; Walters, Andrew M.; Yetisgen, Meliha; O’Reilly-Shah, Vikas N.

doi:10.1186/s12871-023-02248-0

Table 2 Area under receiver operator characteristic for all models

From: Prediction of American Society of Anesthesiologists Physical Status Classification from preoperative clinical text narratives using natural language processing

A. Macro-average AUROC
	Baseline	Diagnosis	Procedure	HPI	PMSH	ROS	Meds	Note	Note512
Random Classifier	0.500	---	---	---	---	---	---	---	---
Age & Meds	0.709	---	---	---	---	---	---	---	---
Random Forest	---	0.741	0.751	0.788	0.695	0.778	0.781	0.820	0.802
Support Vector Machine	---	0.714	0.717	0.789	0.697	0.787	0.768	0.850	0.829
fastText	---	0.757	0.758	0.791	0.720	0.793	0.789	0.865	0.844
BioClinicalBERT	---	0.767	0.755	0.814	0.737	0.806	0.784	0.843	0.845
B. Class-specific AUROC
	Baseline	Diagnosis	Procedure	HPI	PMSH	ROS	Meds	Note	Note512
Random Classifier
I	0.500	---	---	---	---	---	---	---	---
II	0.500	---	---	---	---	---	---	---	---
III	0.500	---	---	---	---	---	---	---	---
IV-V	0.500	---	---	---	---	---	---	---	---
Age & Meds
I	0.863	---	---	---	---	---	---	---	---
II	0.638	---	---	---	---	---	---	---	---
III	0.668	---	---	---	---	---	---	---	---
IV-V	0.668	---	---	---	---	---	---	---	---
Random Forest
I	---	0.790	0.810	0.864	0.810	0.869	0.861	0.898	0.886
II	---	0.708	0.713	0.744	0.636	0.729	0.738	0.783	0.759
III	---	0.660	0.674	0.708	0.644	0.708	0.718	0.747	0.719
IV-V	---	0.804	0.806	0.835	0.691	0.803	0.807	0.854	0.844
Support Vector Machine
I	---	0.776	0.793	0.874	0.827	0.904	0.869	0.938	0.924
II	---	0.653	0.633	0.738	0.592	0.691	0.680	0.806	0.775
III	---	0.639	0.650	0.709	0.655	0.728	0.702	0.775	0.750
IV-V	---	0.789	0.794	0.836	0.714	0.826	0.821	0.881	0.865
fastText
I	---	0.815	0.820	0.870	0.833	0.889	0.863	0.943	0.930
II	---	0.724	0.718	0.755	0.675	0.771	0.755	0.833	0.809
III	---	0.684	0.685	0.720	0.668	0.729	0.724	0.798	0.771
IV-V	---	0.805	0.811	0.819	0.702	0.782	0.815	0.884	0.867
BioClinicalBERT
I	---	0.838	0.816	0.901	0.851	0.902	0.861	0.917	0.922
II	---	0.711	0.707	0.768	0.674	0.748	0.737	0.806	0.804
III	---	0.688	0.681	0.741	0.682	0.752	0.719	0.776	0.779
IV-V	---	0.830	0.818	0.848	0.741	0.823	0.818	0.874	0.874

(A) Macro-average AUROC and (B) class-specific AUROC for each model architecture and task on the held-out test set compared to baseline models. Random Classifier serves as a negative control baseline. Age & Meds classifier serves as a simple clinical baseline. Supplemental Table 5 is a copy of this table with all standard errors reported

Back to article page

ISSN: 1471-2253

Contact us

Submission enquiries: bmcanesthesiology@biomedcentral.com
General enquiries: ORSupport@springernature.com

BMC Anesthesiology

Contact us