Objective To investigate the influencing factors of glycosylated hemoglobin (HbA1c) control during long-term follow-up in the elderly patients with type 2 diabetes mellitus. Methods A total of 200 elderly male patients with type 2 diabetes mellitus were selected for the study, who received continuous glucose monitoring (CGM) in the Second Medical Center of Chinese PLA General Hospital from January 2017 to January 2011. They were divided into groups according to age (60 years≤age<70 years, n=57; 70 years≤age<80 years, n=51; age≥80 years, n=92), diabetes duration (diabetes<10 years, n=57; 10≤ diabetes <20 years, n=89; diabetes≥20 years, n=54), or baseline HbA1c level (HbA1c<6.5%, n=74; 6.5%≤HbA1c<7.5%, n=72; HbA1c≥7.5%, n=54). The groups were compared in mean HbA1c during long-term follow-up. Data analysis was performed using SPSS 13.0. Comparison between groups was performed with variance analysis or Chi-square test depending on the data types. The influencing factors of HbA1c control during follow-up were analyzed by logistic regression. Results All participants were followed up for an average of 12.5 years. There was no difference in mean HbA1c and adequate HbA1c control rate between different age groups during follow-up (P＞0.05). Compared with the diabetes duration<10 years group, the mean HbA1c increased (P<0.01) and the adequate HbA1c control rate decreased (P<0.05) in the diabetes duration≥20 years group. With the increasing baseline HbA1c, the mean HbA1c increased gradually (P<0.01), and adequate HbA1c control rate decreased gradually (P<0.01). The univariate logistic regression analysis showed that diabetes duration, baseline HbA1c, fasting plasma glucose, 2 h postprandial glucose, baseline glycemic variability, and insulin therapy were the influencing factors of HbA1c control during follow-up (P<0.01). Unconditional multivariate logistic regression analysis suggested that baseline HbA1c (OR=5.73,95%CI 3.02-10.88, P<0.001) was the most important influencing factor of HbA1c control during follow-up. Conclusion Diabetes duration, baseline HbA1c, fasting plasma glucose, 2 h postprandial glucose, baseline glycemic variability, and insulin therapy were the influencing factors of HbA1c control in elderly patients with type 2 diabetes during follow-up, baseline HbA1c being the most important one.