This paper describes the development of an on-line quality inspection algorithm for detecting the surface defect (“bleeds”) generated in continuous casting processes. The challenges of bleed detection in visual images lie in the low signal-to-noise ratio, the irregularity of bleed contour patterns, and significant false positives. In order to solve these problems, an engineering-driven rule-based detection (ERD) method is proposed. The ERD consists of three detection stages using the pixel features of bleeds, which are transferred from the physical features generated via engineering knowledge. The detailed strategy, ERD algorithm, parameter selection, and casting knowledge are presented. The real case study demonstrates that the developed algorithm is effective and applicable.