Pool Computer Vision

Background

I love to play pool and I know nothing about computer vision. I figured that pool would be a good learning application because everything is pretty specific…the table is a rectangle, balls are circle and have specific colors and patterns. I know some python so I’m going to use the popular OpenCV platform.

Goal

Generate cool stats about my games such as cue ball velocity, collision angle, make %, etc in real time. I plan to do this by mounting a camera directly above the table in my house and streaming that video feed to my CV processor. The CV will output to a web frontend that will allow interaction with the user.

Preliminary research

I found a few examples online of people doing similar things. The most relevant example I found was this paper from UCSD (Go Tritons!).
I also found this less-detailed but still useful blog from a London startup.

Work blog

12/5/2020

I began with this incredible tutorial on how to create color histograms and identify peaks as commonly used colors in an image. This is a super important concept because I need to be able to pick out the color of the pool table. Here is a resulting RGB histogram: Notice 3 distinct peaks for each color of about 60-80 bins wide (R=45, G=50, B=80). When you punch that into any online RGB color calculator, you get a the color of the table felt.

However, I eventually found a solution on StackOverflow that did this job much more concisely. I wrote my own function that takes an image and a scale and returns the image zoomed in to that scale. Essentially, it crops and resizes. I used this function before I found the dominant color in the image just in case an image had a large background with another color that runs the risk of being identified as the dominant color.

I next needed to get a binary mask of just the pixels with that dominant color. The table color isn’t homogenous so I created a lower and upper bounds of the RGB color using a +/- 40 value. The mask looked like this:
Next step was to find the largest contour. I’m still a little unclear on what a contour actually is, but my current understanding is that it’s an edge between the two colors of the binary mask. The largest contour would then be the outline of the table. I then draw the smallest enclosing rectangle around that contour. I’m making a bit of an assumption here that the camera is directly perpendicular to the table, and that the image is a rectangle rather than another type of quadrilateral.