A large amount of energy from power plants, vehicles, oil refining, and steel or glass making process is released to the atmosphere as waste heat. The thermoelectric generator (TEG) provides a way to reutilize this portion of energy by converting temperature differences into electricity using Seebeck phenomenon. Because the figures of merit zT of the thermoelectric materials are temperature-dependent, it is not feasible to achieve high efficiency of the thermoelectric conversion using only one single thermoelectric material in a wide temperature range. To address this challenge, the authors propose a method based on topology optimization to optimize the layouts of functional graded TEGs consisting of multiple materials. The multimaterial TEG is optimized using the solid isotropic material with penalization (SIMP) method. Instead of dummy materials, both the P-type and N-type electric conductors are optimally distributed with two different practical thermoelectric materials. Specifically, Bi2Te3 and Zn4Sb3 are selected for the P-type element while Bi2Te3 and CoSb3 are employed for the N-type element. Two optimization scenarios with relatively regular domains are first considered with one optimizing on both the P-type and N-type elements simultaneously, and the other one only on single P-type element. The maximum conversion efficiency could reach 9.61% and 12.34% respectively in the temperature range from 25 °C to 400 °C. CAD models are reconstructed based on the optimization results for numerical verification. A good agreement between the performance of the CAD model and optimization result is achieved, which demonstrates the effectiveness of the proposed method.