Cardiac magnetic resonance (CMR) is used extensively in the diagnosis and management of cardiovascular disease. Deep learningmethods have proven to deliver segmentation results comparable tohuman experts in CMR imaging, but there have been no convincingresults for the problem of end-to-end segmentation and diagnosisfrom CMR. This is in part due to a lack of sufficiently large datasetsrequired to train robust diagnosis models. In this paper, we proposea learning method to train diagnosis models, where our approach isdesigned to work with relatively small datasets. In particular, the optimisation loss is based on multi-task learning that jointly trains forthe tasks of segmentation and diagnosis classification. We hypothesize that segmentation has a regularizing effect on the learning offeatures relevant for diagnosis. Using the 100 training and 50 testingsamples available from the Automated Cardiac Diagnosis Challenge(ACDC) dataset, which has a balanced distribution of 5 cardiac diagnoses, we observe a reduction of the classification error from 32% to22%, and a faster convergence compared to a baseline without segmentation. To the best of our knowledge, this is the best diagnosisresults from CMR using an end-to-end diagnosis and segmentationlearning method.