Welcome to Computer Vision
You're likely already familar with the concept of computer vision. The idea that we feed the computer some video/images and it can help us determine what is in the frame. This technology powers object identification and tracking throughout a medley of industries, with self-driving cars being a more famous of examples - with second place going to Jian Yang's SeeFood app.
You've also likely seen companies in other sports, Second Spectrum, Opta, Statsbomb, SportsVU, etc... But this work remains isolated to the NFL, NBA, Soccer, Baseball, and Hockey (read: sports with money). So to bring this technology to volleyball is much harder. The resources to put 18, 4k cameras in every collegiate arena to track players/ball in our sports simply does not exist.
So we need to solve this problem on the cheap - ideally where anyone with an iPhone and a tripod could get player tracking data + all the benefits that come with it, without spending millions.
There are a few foundational problems you need to tackle in order to solve computer vision in volleyball:
- Detect players in the frame
- Follow them around as they move
- Luckily this is a common problem, so there are many solutions out there
- Difficulty: pretty easy
- Detect the volleyball
- Follow the volleyball
- Difficulty: easy to medium
- Determine when a touch happens
- Figure out which player actually touched the ball
- Difficulty: medium to hard
- Use pose-estimation to teach the network what skills look like
- Difficulty: medium
- Use jersey numbers to ID the players
- Ugh, this is not easy
- Difficulty: hard
- Difficulty: umm, real real hard
- Has yet to be solved for single-camera multi-object tracking on this scale?
Make it Run at Real-Time
- Should be doable
- Would enable instant data / analysis
- In-match data
- Intriguing data for color commenators
- In-game sports betting
- Difficulty: medium to hard, but doable
Slow and steady over here...