The upper bound finite element (FE) limit analysis is applied to stability problems of slopes using a nonlinear criterion. Generally speaking, the equivalent shear strength parameters of materials with a nonlinear failure criterion are expressed in terms of stress, which cannot be obtained directly by the upper bound method. Based on the Mohr-Coulomb criterion and FEM, the corresponding dual second-order cone programming (SOCP) problem of the upper bound limit analysis, which is considered as a static form, is formulated. By solving the dual problem, the distribution of stress of slopes can be obtained. Then, the equivalent shear strength parameters of materials can be determined iteratively so that the analysis of slope stability with a nonlinear failure criterion is able to be transformed into the traditional upper bound method. Finally, the results of two numerical examples are compared with the published solutions and demonstrate the accuracy and validity of the proposed method, by which the nonlinear failure criterion can be represented exactly.