This document details the Vision AI Setup Assistant tool, which has been developed to help users assess their camera setup for compatibility with the Vision AI feature, to ensure the best possible Vision AI detections during play. This tool will help with the positioning of cameras at both ends of the ground.
The tool is displayed on the Video Display window after ticking the Settings cog (top-right) -> Show Vision AI Setup Assistant option on the menu. Once activated, the Vision AI feature will begin detecting and tracking objects. If pitch corner detections are good, then a pitch map overlay will be displayed. The Vision AI Setup Assistant score panel will appear at the top-left of the Video Display panel:
Camera Configuration Scores
The Video Resolution score measures the suitability of the video resolution in use for Vision AI, which works best with high resolution video footage, preferably to the standard of 1080p (1920x1080). We do not recommend using with resolution lower than 1080p. When the Video Resolution score is low, indicated by orange or red colouring, it is strongly recommended to increase the video resolution and/or bitrate to improve results.
The Camera Stability score measures the amount of movement of the video frame. A stable camera is very important for achieving good ball tracking and trajectory plotting results, so any camera should be securely anchored to prevent motion, such as swaying in the wind. If the camera motion is minimal, then this score will be high, and Vision AI will be able to produce good ball tracking and trajectory plotting results when play starts. If the camera is unstable, for example swaying up-and-down and/or side-to-side in the wind, then the score will drop, and the camera will need to be further stabilised or protected from the wind.
Note that if used during play, this score will drop when players are moving, for example the bowler running in or fielders running across the field of view. However, if the Camera Stability score is monitored during periods of minimal activity, such as when the batter is taking strike, then the score will be more indicative of the basic stability of the footage.
The Zoom score measures the suitability of the camera’s zoom or its distance from the action. The Vision AI Setup Assistant looks for key object detections and structure sizes and compares these to known reference sizes. If the objects detected meet or exceed the reference values, then this score will be high and, provided the video quality is good, the results will be more accurate and reliable, e.g.:
When the video footage is not sufficiently zoomed, then this score will drop, e.g.:
If the Zoom score is consistently falling into the orange and red regions, then it is strongly recommended to increase the zoom level of the camera or, if possible, shift it closer to the action.
Pitch Field of View
The Pitch Field of View score measures the suitability of the vertical field of view of the pitch. For good detection results, the camera position needs to be sufficiently high for a good view of a large proportion of the pitch with minimal obstruction by the umpire or bowler, so that the ball and striker are clearly visible. This gives Vision AI the best conditions for ball detections, tracking and trajectory plotting, and the best results for determining the pitch map and ball arrival. This score is also closely tied to the Zoom score, so provided this is good, then the Pitch Field of View score will also be good when the camera is raised sufficiently that the pitch occupies up to half (50%) of the camera’s vertical field of view.
The following images show two camera setups, with the red arrows indicating the extent of the pitch, resulting in one with a good Pitch Field of View score (left) and the other with an average score (right). The pitch on the left clearly has more visible area while the pitch on the right is obscured by the umpire, which could significantly affect ball and striker detections:
In cases where it is not physically possible to raise the camera altitude sufficiently to see over the umpire, there will be a higher number of balls that cannot be processed by Vision AI.
The Pitch Geometry score measures how well the pitch corners have been detected and whether the resultant pitch area is as expected. Vision AI requires a camera view from behind the bowler and umpire looking down on the pitch towards the batter. From this perspective the pitch shape should be roughly trapezoidal for the best results. If a corner becomes obscured, for example by a player or umpire moving in front of it, then this score will drop during the period of obstruction.
If all the pitch corners are visible in the camera view but the Pitch Geometry score is still low, or lower than expected, then the four pitch corner Consistency scores (see below) should be checked. If one or more of these is low, then this suggests that Vision AI is struggling to detect that pitch corner. In this case, try to improve the detections by altering the video feed, e.g.:
- Changing the camera zoom
- Moving the camera closer to the action
- Slightly changing the camera’s angle relative to the pitch
- Increase the video resolution and/or contrast
In rare situations, consider re-applying crease line markings to make the corners clearer for detection.
Although a view from behind the umpire is generally best, Vision AI has some tolerance to less perfect pitch geometries and can still produce ball trajectories and pitch maps. The image below shows an example of an offline camera position with a lower than usual Pitch Field Of View score but a still viable Pitch Geometry score. Thus, if the camera cannot be raised sufficiently to see over the umpire and bowler, then it may be possible to shift the camera slightly to one side and still achieve reliable detections.
Bowler Approach Consistency
The Bowler Approach Consistency score measures how well Vision AI is detecting and tracking the bowler during their approach to the crease. This score can be used in a pre-match setting if a person can simulate the typical bowler approach. It can also be used for pre-existing footage so long as the clip starts early enough to include the bowler approach. In both cases Vision AI is checking for valid bowler approach detections in sequential frames, and that these detections are physically sufficiently close to the detection in the previous frame.
If the score is low, then Vision AI is struggling to consistently detect the bowler. The most likely reasons for poor bowler approach detections will be camera positioning, camera zoom or a particularly unorthodox bowler approach. The Bowler Approach Consistency score will only show values when bowler approach detections are occurring, and for a few frames afterwards. When there is no bowler approach, for example during a pre-match set up with no players present, the score will be greyed and show “n/a”, e.g.:
The Ball Consistency score measures how well Vision AI is detecting and tracking the ball. As with the Bowler Approach Consistency score, this score can be used in a pre-match setting if a ball can be bowled on the pitch adjacent to the match pitch, and it can also be used on pre-existing clips where the ball is present. Again, Vision AI is looking for sequential detections and physically logical tracking across the sequence of video frames, so low scores will indicate that Vision AI is struggling to detect and track the ball. This restriction will most likely be caused by poor video quality (insufficient resolution and/or low contrast and sharpness) or insufficient camera zoom. Improving these properties will help to improve the ball tracking, which is challenging due to the small size and motion of the ball, and as such benefit greatly from ensuring the video feed is the best possible. The Ball Consistency score will also revert to “n/a” during periods when there is no ball present.
The Top Left, Top Right, Bottom Right, and Bottom Left Consistency scores indicate how well the pitch area is being detected. With a stable camera setup and a good field of view of the pitch these scores will generally be high and stable. However, if any of these scores regularly drops noticeably, and there are no obstructions (such as a player or umpire walking across the pitch corner), then Vision AI may be struggling to consistently detect that pitch corner. In these cases, as above, it may be necessary to improve the camera zoom, position, angle, or resolution/contrast and/or consider re-applying crease line markings.
General Hardware and Video Considerations
The Vision AI Setup Assistant tool has been developed to provide meaningful Vision AI coding results on live and recorded video clips, across a range of setup scenarios on a significant portion of common laptop and desktop PC systems. As the tool heavily utilises deep neural network algorithms for much of its processing, if the intent is to use the tool either in real-time or near real-time, then it requires a laptop or desktop PC with suitable specifications. For the detailed technical requirements, please see our recommended minimum hardware specifications at https://support.nvplay.com/hc/en-gb/articles/1500003016681-System-and-Hardware-Requirements.
If Vision AI data already exists on the ball, i.e., if ball clips have already been processed by another machine and you are logging in as a Live Scorer or Reader, then there are no additional hardware requirements. For non-real-time usage and processing please see our article on post-match audit and cloud processing https://support.nvplay.com/hc/en-gb/articles/16082086055321-Vision-AI-Audit-May-2023-.
Generally, the NV Play Pro Cricket Scorer/Play-Cricket Scorer Pro Vision AI feature will provide the best results when the video footage is of a high quality (good contrast, low blur, even saturation, minimal noise) and high resolution. Accurate and reliable results have been provided from 1080p and similar video standards, whilst maintaining good frame rate and post-processing performance. Interlaced video, e.g., 1080i or similar, is known to be more challenging due to the blur and loss of contrast that can occur when this video is encoded. Although Vision AI has been specifically developed to handle a level of degradation in video quality and has several background processes designed to robustly handle intermittent detections and tracking, it is strongly recommended to provide the best possible video quality to ensure the most accurate and reliable results.