TwinLiverNet: Predicting TACE Treatment Outcome from CT scans for Hepatocellular Carcinoma using Deep Capsule Networks

Predicting response to treatment plays a key role to assist radiologists in hepato-cellular carcinoma (HCC) therapy planning. The most widely used treatment for unresectable HCC is the trans-arterial chemoembolization (TACE). A complete radiological response after the first TACE is a reliable predictor of treatment favourable outcome. However, visual inspection of contrast-enhanced CT scans is time-consuming, error prone and too operator-dependent. Thus, in this paper we propose TwinLiverNet: a deep neural network that is able to predict TACE treatment outcome through learning visual cue from CT scans. TwinLiverNet, specifically, integrates 3D convolutions and capsule networks and is designed to process simultaneously late arterial and delayed phases from contrast-enhanced CTs. Experimental results carried out on a dataset consisting of 126 HCC lesions show that TwinLiverNet reaches an average accuracy of 82% in predicting complete response to TACE treatment. Furthermore, combining multiple CT phases (specifically, late arterial and delayed ones) yields a performance increase of over 12 percent points. Finally, the introduction of capsule layers into the model avoids the model to overfit, while enhancing accuracy.Clinical relevance— TwinLiverNet supports radiologists in visual inspection of CT scans to assess TACE treatment outcome, while reducing inter-operator variability.