Drug-resistant focal epilepsy is associated with abnormalities in the brain in both grey matter (GM) and superficial white matter (SWM). However, it is unknown if both types of abnormalities are important in supporting seizures. Here, we test if surgical removal of GM and/or SWM abnormalities relates to post-surgical seizure outcome in people with temporal lobe epilepsy (TLE). We analyzed structural imaging data from 143 TLE patients (pre-op dMRI and pre-op T1-weighted MRI) and 97 healthy controls. We calculated GM volume abnormalities and SWM mean diffusivity abnormalities and evaluated if their surgical removal distinguished seizure outcome groups post-surgically. At a group level, GM and SWM abnormalities were most common in the ipsilateral temporal lobe and hippocampus in people with TLE. Analyzing both modalities together, compared to in isolation, improved surgical outcome discrimination (GM AUC = 0.68, p < 0.01, WM AUC = 0.65, p < 0.01; Union AUC = 0.72, p < 0.01, Concordance AUC = 0.64, p = 0.04). Additionally, 100% of people who had all concordant abnormal regions resected had ILAE$_{1,2}$ outcomes. These findings suggest that regions identified as abnormal from both diffusion-weighted and T1-weighted MRIs are involved in the epileptogenic network and that resection of both types of abnormalities may enhance the chances of living without disabling seizures.