android - OpenCV speed traffic sign detection

Question

I have a problem detecting speed traffic signs with opencv 2.4 for Android. I do the following:

"capture frame -> convert it to HSV -> extract red areas -> detect signs with ellipse detection"

So far ellipse detection works perfect as long as picture is good quality. But as you see in pictures bellow, that red extraction does not work OK, because of poor quality of picture frames, by my opinion.

Converting original image to HSV:

Imgproc.cvtColor(this.source, this.source, Imgproc.COLOR_RGB2HSV, 3);

Extracting red colors:

Core.inRange(this.source, new Scalar(this.h,this.s,this.v), new Scalar(230,180,180), this.source);

So my question is is there another way of detecting traffic sign like this or extracting red areas out of it, which by the way can be very faint like in last picture ?

This is the original image:

This is converted to HSV, as you can see red areas look the same color as nearby trees. Thats how I'm suppose to know it's red but I can't.

Converted to HSV:

This is with red colors extracted. If colors would be correct I should get almost perfect circle/ellipse around sign, but it is incomplet due to false colors.

Result after extraction:

Ellipse method:

private void findEllipses(Mat input){
Mat thresholdOutput = new Mat();
int thresh = 150;

List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
MatOfInt4 hierarchy = new MatOfInt4();

Imgproc.threshold(source, thresholdOutput, thresh, 255, Imgproc.THRESH_BINARY);
//Imgproc.Canny(source, thresholdOutput, 50, 180);
Imgproc.findContours(source, contours, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
RotatedRect minEllipse[] = new RotatedRect[contours.size()];

for(int i=0; i<contours.size();i++){
    MatOfPoint2f temp=new MatOfPoint2f(contours.get(i).toArray());

    if(temp.size().height > minEllipseSize && temp.size().height < maxEllipseSize){
        double a = Imgproc.fitEllipse(temp).size.height;
        double b = Imgproc.fitEllipse(temp).size.width;
        if(Math.abs(a - b) < 10)
            minEllipse[i] = Imgproc.fitEllipse(temp);
    }
}
detectedObjects.clear();
for( int i = 0; i< contours.size(); i++ ){
    Scalar color = new Scalar(180, 255, 180);
    if(minEllipse[i] != null){
        detectedObjects.add(new DetectedObject(minEllipse[i].center));
        DetectedObject detectedObj = new DetectedObject(minEllipse[i].center);
        Core.ellipse(source, minEllipse[i], color, 2, 8);
    }
}

}

Problematic sign:

Answer 1

You can find a review of traffic signs detection methods here and here.

You'll see that there are 2 ways you can achieve this:

1. Color-based (like what you're doing now)
2. Shape-based

In my experience, I found that shape-based methods works pretty good, because the color may change a lot under different lighting conditions, camera quality, etc.

Since you need to detect speed traffic signs, which I assume are always circular, you can use an ellipse detector to find all circular objects in your image, and then apply some validation to determine if it's a traffic sign or not.

Why ellipse detection?

Well, since you're looking for perspective distorted circles, you are in fact looking for ellipses. Real-time ellipse detection is an interesting (although limited) research topic. I'll point you out to 2 papers with C++ source code available (which you can use in you app through native JNI calls):

1. L. Libuda, I. Grothues, K.-F. Kraiss, Ellipse detection in digital image data using geometric features, in: J. Braz, A. Ranchordas, H. Arajo, J. Jorge (Eds.), Advances in Computer Graphics and Computer Vision, volume 4 of Communications in Computer and Information Science, Springer Berlin Heidelberg, 2007, pp. 229-239. link, code

2. M. Fornaciari, A. Prati, R. Cucchiara, "A fast and effective ellipse detector for embedded vision applications", Pattern Recognition, 2014 link, code

---

UPDATE

I tried the method 2) without any preprocessing. You can see that at least the sign with the red border is detected very good: