Laser shock peening (LSP) is a surface treatment technique similar to conventional shot peening. The laser induced plasma causes plastic deformations and compressive residual stresses that are useful for developing improved properties in the fields of resistance to fatigue, wear or stress corrosion cracking. The actual distribution of residual stresses is extremely important while designing for improved fatigue life using laser shock peening, as fatigue cracks would initiate from the weakest point in the structure. In this paper, the variations in distribution of residual stresses due to laser shock peening are studied with a focus on two materials, annealed 1053 and hardened 52100 AISI steels. A 3D finite element model was developed to study the actual distributions of the residual stresses due to laser shock peening. The effect of hardness on the distribution of the residual stresses and the presence of tensile residual stresses in the surrounding regions of the impact is analyzed. Much larger variations in the residual stress distributions were observed in case of the 1053 steel as compared to hardened 52100 steel. A comprehensive analysis of the simulation results was performed in order to address and explain this behavior. It was observed that the extent of overlap would also affect the variations in the residual stress distributions. The tensile residual stresses present in the areas surrounding the shocked region were also analyzed based upon the extent of overlap and the hardness of the material. It was observed that the ratio of peak tensile to compressive residual stresses developed in 1053 steel was much higher as compared to that in the hardened 52100 steel.