The robotic riveting system requires a rivet robotic positioning process for rivet-in-hole insertions, which can be divided into two stages: rivet path-following and rivet spot-positioning. For the first stage, varying parameter-linear sliding surfaces are proposed to achieve robust rivet path-following against robot errors and external disturbances of the robotic riveting system. For the second stage, a second-order sliding surface is applied to attain accurate rivet spot-positioning within a finite time required by the riveting process. In order to improve the dynamic performance of the robot riveting system, the motion of robot end-effector between the two adjacent riveting spots has been properly designed. Overall, the proposed control scheme can guarantee not only the stability of the robot control system but also the robust rivet path-following and quick rivet spot-positioning in the presence of the robot errors and external disturbances of the robotic riveting system. The simulation and experimental results demonstrate the effectiveness of the proposed control scheme.