Hemming is a manufacturing process of folding a panel onto itself or another sheet. Quality of hemming is characterized by geometry and formability. This paper presents a response surface study of three-dimensional (3D) curved-surface-curved-edge hemming of an aluminum alloy, AA6111-T4, using finite-element (FE) analysis. Solid elements and explicit FE solver are used for simulations of flanging, pre- and final hemming, and shell elements with implicit solver are deployed for springback prediction. A novel procedure called “solid-to-shell mapping” is developed to bridge the solid elements with the shell elements. Verified to be accurate and efficient, the model is utilized in a central composite design to quantitatively explore the relationships between certain key process variables and the hem dimensional quality and formability. The most significant variables are identified as: (i) prehemming angle on roll-in/roll-out; (ii) nominal surface curvature on sheet springback; and (iii) initial sheet strain and flanging die radius on the maximum hemline surface strain of the produced hem. These results provide insights for process parameter selections in designing and optimizing 3D hems under material formability constraints.