Considering the background languages, there are internal and external challenges for Minangkabau-Indonesian EFL students in their speaking ability. Even within one indigenous language, they basically speak in different dialects and accents. They need to accomodate these challenges into a proper english usage. This paper is aimed at investigating the speaking problems of the students and the factors affecting their speaking performance. The writer uses a questioner addressed to 60 students from STMIK Jayanusa Padang and IAIN Imam Bonjol Padang. The findings of this paper indicates that psychological factors are the major problems in learning spoken English. Among these are lack of self-confidence, lack of aptitude, lack of motivation, anxiety, and shyness. The study was expected to help students improve their performance in speaking and have more exposure to the language.