This paper presents a methodology for the optimal design of the measurement point layout for 3D workpiece localization in the presence of part surface errors and measurement errors. A number of frame-invariant norms of the infinitesimal rigid body displacement, two of which give Riemann metrics on the Euclidean group, are defined to quantify the localization accuracy required by manufacturing processes. Then, two types of indices, both frame invariant and scale invariant, are derived to characterize the sensitivities of the accuracy measures to the sampling errors at the measurement points. With a dense set of discrete points on the workpiece datum surfaces predefined as candidates for measurement, planning of probing points to accurately recover part location is modeled as a combinatorial problem focusing on minimizing the accuracy sensitivity index. It is shown that if the number of measurement points is large enough, there is no need to optimize their positions, and that if the systematic error component of the sampled geometric errors is not negligible as compared with the random error component, addition of measurement points offers no guarantee of a smaller upper bound of the localization error. A heuristic floating forward search algorithm is employed to efficiently find a near-optimal solution. Two relevant problems of sensor placement optimization for geometric imperfection diagnosis and fixture fault diagnosis are also briefly revisited in the same framework. Examples are given to illustrate the effectiveness of the proposed design criteria and algorithm.