Objective To determine the effect of cognitive training and aerobic exercise on the cognition function in elderly type 2 diabetes mellitus (T2DM) patients with mild cognitive impairment (MCI). Methods The elderly T2DM patients with MCI admitted in our hospital from May 2014 to July 2016 were enrolled in this study. They were randomly divided into control group (n=38), cognition training group (n=35) and aerobic exercise group (n=36). The patients from the control group were given conventional treatment (including diet control and oral hypoglycemic drugs). Besides the conventional treatment, those of cognition training group received computer cognitive function training (3 times/week, for 3 months), and those of the aerobic exercise group had fitness walking (≥3 times/week, for 3 months). Montreal cognitive assessment (MoCA) was carried out for the 3 groups before and after the treatment. Fasting blood glucose (FBG), glycosylated hemoglobin A1c (HbA1c), triglycerides (TG) and total cholesterol (TC) were also detected. The insulin resistance (IR) was estimated by the homeostasis model. Results After the treatment, the MoCA scores were significantly higher in the cognition training group and aerobic exercises group than the control group (P<0.05). The levels of FBG and HbA1c were decreased in the cognition training group after the treatment (P<0.05), and the values of FBG, HbA1c, TG and homeostasis model assessment IR (HOMA-IR) were obviously lower in the aerobic exercise group than the control group (P<0.01, P<0.01, P<0.05 and P<0.05, respectively). The total MoCA score was negatively correlated with FBG, HbA1c and HOMA-IR (r=-0.235,-0.358, and -0.263, respectively,P<0.01). Conclusion Cognition training and aerobic exercise improve the cognitive function in the elderly T2DM patients with MCI to some extent, which might be due to their effect on glucose metabolism.