This document explains how to setup, configure, and run the new Vision AI Engine during a live match. The purpose of the Vision AI feature is to automatically create ball clips and to provide accurate and reliable high-performance coding on various supported data detections. This automation consequently lightens the coding load on the Video Analyst during live matches to allow them more time to analyze, rather than code, data and video during play. The automatic Vision AI coding currently includes:
- Ball Clip Capture (i.e., Start/Stop Ball Clip on all balls)
- Bowler Action (i.e., Around the Wicket when applicable)
- Keeper Stance (i.e., Keeper Up when applicable)
- Ball Pitch (i.e., Pitch Map/Line & Length)
- Bowler Wide on Crease (i.e., Events -> Wide on Crease Bowling when applicable)
- Ball Arrival (Beta)
- Ball Tracking After Impact (Beta) (i.e., the addition of ball trajectory graphics to ball clips)
The Vision AI coding requires a stationary camera at each end of the ground that is high and straight, so that the detections can be accurately applied to each ball clip and high-quality video input.
For more information on running Vision AI post-match, please see https://support.nvplay.com/hc/en-gb/articles/16082086055321-Vision-AI-Audit-May-2023-.
For more information on running Vision AI on a separate machine during a match, please see https://support.nvplay.com/hc/en-gb/articles/15741825488793-Vision-AI-Assistant-February-2023-.
Vision AI Setup
Vision AI users should check the Multi-Capture -> Vision AI section of our minimum hardware specifications to ensure their computer will handle the Vision AI processing. If users aren’t sure, they can run the Video -> Auto Detect Video Settings tool and apply any recommended changes, and then check the Tools -> Configuration -> Vision AI tab for confirmation that their computer supports Vision AI, e.g.:
After first logging-in, the Latest Vision AI Components will automatically begin downloading for installation:
Please note that this file is ~2Gb in size so users should ensure that they have a good internet connection, sufficient hard drive space, and do the download/install well in-advance of their first Vision AI match.
Once the download is complete, follow the prompts to install the components and restart NV Play Pro Cricket Scorer/Play-Cricket Scorer Pro.
Vision AI Configuration
On match day, the above Enable Vision AI Capture checkbox also needs to be ticked to activate the Vision AI coding for when the match begins. In addition, the other Tools -> Configuration -> Vision AI settings should be checked and selected.
Firstly, the two Ball Clip Capture settings are used to determine the start and end points of each ball clip, i.e., X number of seconds Offset from Ball Release and for X number of seconds total Clip Duration:
Thus, these settings will generate 10 second ball clips, starting 2 seconds before the bowler releases the ball and therefore finishing 8 seconds after that.
Next, these Detection Settings are enabled by default, but they can be disabled if required:
The two Quality Settings can be used to specify the level of computer resources to be committed to the Vision AI coding. The Live Vision AI Detection Quality determines this level when Vision AI codes each ball live, while the Post Processing Vision AI Detection Quality determines this level when Vision AI re-processes each ball after the initial detection. In both cases, there is a recommended setting (Medium and High, respectively), but these can be decreased or increased if necessary:
Second Pass Processing can be configured as:
- Disabled. Use this if you are satisfied with the initial detection quality and to reduce computer resource.
- Ball Start Timing Only. This will reprocess each ball clip to verify and adjust if needed the Offset from Ball Release point
- Fully Reprocess Ball. This will reprocess each ball clip to improve the initial detections and the Offset from Ball Release point if needed. This is the recommended setting for best quality detections.
Once all Configuration settings are confirmed, click OK to save and exit.
Vision AI Capture
The main layout will now show a Start Vision AI button below the Video Display:
When play is about to being, click this button to put NV Play Pro Cricket Scorer/Play-Cricket Scorer Pro into the Waiting for Capture mode, i.e., where it is waiting to detect the bowler coming into frame to deliver the first ball:
When Vision AI detects the bowler coming into frame to deliver the first ball, the main UI will become active, as if Start Ball Clip had been clicked when manual coding. The Vision AI coding will be added to each ball but will not end the ball automatically. The Video Analyst should add their coding data before ending the ball. After completing any Second Pass Processing, Vision AI will return to the Waiting for Capture mode, ready for the next ball. The Video Analyst can enter Edit mode and change any data, including Vision AI data, as usual.
The combined Vision AI and Video Analyst data will appear on each ball in the Ball by Ball panel, as usual, and the Vision AI column will show a tick mark for every ball that has had Vision AI applied. However, these ticks will be in different colours, with different tools tips available when hovering over them, according to the following coding/information:
- Light Grey = Vision AI processing has been run but could not detect any data
- Red = This ball contains the following Vision AI Assisted Coding, but with at least one data point of Low Confidence
- Black = This ball contains the following Vision AI Assisted Coding, with all data points of High Confidence, e.g.:
If a ball has no tick in the Vision AI column, this means that Start Vision AI had not been clicked beforehand, but this can be applied retrospectively by right-clicking on the ball and selecting Process Vision AI from the menu. If multiple balls require Vision AI processing, go to Video -> Vision AI -> Process Match Ball Clips, select the required options on the dialog, then click Start:
A ball may not have AI coding added for a few reasons, including:
- The ball is obscured by the bowler or umpire
- The lack of clear ball detections due to low contrast or video quality
- The bowler approach is not detected
In these cases, the analyst should review the balls to manually add any coding required.
NB: This feature requires the Vision AI Engine feature licence to be enabled on the user’s account.