Objective To evaluate the success rate of glycosylated hemoglobin (HbA1c) control in the elderly patients with comorbid type 2 diabetes mellitus (T2DM) and its influencing factors in a Shanghai tertiary hospital. Methods According to clinical guidelines, a questionnaire was designed for a cross-sectional study to investigate 284 elderly patients with comorbid T2DM, who were admitted to the Internal Medicine Department of Huadong Hospital Affiliated to Fudan University from September to December 2020. Data analysis was performed using SPSS statistics 23.0. According to the data type, t-test or χ² test was used for comparison between groups. Logistic regression was performed to analyze the factors affecting the success rate of HBA1c control. Results The number of comorbidities was (2.04±0.88) in the 284 cases of patients, and metformin was the most widely used hypoglycemic drug. 61.6% patients (175/284) were treated with oral anti-diabetic only, and the success rate of HbA1c control was 68.0% (193/284). Logistic regression analysis showed that the course of T2DM, the number of comorbidities and the route of administration of hypoglycemic drugs were risk factors of success rate of HbA1c control (P<0.05). Conclusion The rate with target HbA1c level in elderly patients with comorbid T2DM needs to be further improved, warranting strengthened management of comorbidities, an individualized treatment plan and rational medication in clinical practice.