Analyzing electrocardiography (ECG) data is essential for diagnosing and monitoring various heart diseases. The clinical adoption of automated methods requires accurate confidence measurements, which are largely absent from existing classification methods. In this paper, we present a robust Gaussian Process classification hyperparameter training model (MuyGPs) for discerning normal heartbeat signals from the signals affected by different arrhythmias and myocardial infarction. We compare the performance of MuyGPs with traditional Gaussian process classifier as well as conventional machine learning models, such as, Random Forest, Extra Trees, k-Nearest Neighbors and Convolutional Neural Network. Comparing these models reveals MuyGPs as the most performant model for making confident predictions on individual patient ECGs. Furthermore, we explore the posterior distribution obtained from the Gaussian process to interpret the prediction and quantify uncertainty. In addition, we provide a guideline on obtaining the prediction confidence of the machine learning models and quantitatively compare the uncertainty measures of these models. Particularly, we identify a class of less-accurate (ambiguous) signals for further diagnosis by an expert.