The residual stress profile in dissimilar metal sheets joined by a self-piercing rivet is simulated and compared to experimental measurements. Simulation of joining aluminum alloy 6111-T4 and steel HSLA340 sheets by self-piercing riveting (SPR) is performed using a two-dimensional axisymmetric model with an internal state variable (ISV) plasticity material model. Strain rate and temperature dependent deformation of the base materials is described by the ISV material model and calibrated with experimental data. Using the LS-DYNA simulation package, an element erosion technique is adopted in an explicit analysis of the separation of the upper sheet with maximum shear strain failure criterion. An explicit analysis with dynamic relaxation technique was then used for springback and cooling down analysis following the riveting simulation. The residual stress profile of SPR experimental joint with same configuration is characterized using neutron diffraction, and good agreement was found between the simulation and residual stress measurements.