Sony patents AI controller that completes partial inputs

Sony has filed a new patent for an AI-assisted controller input system designed to make gaming easier for players with disabilities or motor impairments, according to the patent text. The system detects incomplete or partial inputs and can produce the intended control action when a user is unable to press a button fully.

How the system works

The patent describes a model that examines how a player uses a controller, including timing, movement patterns and in-game conditions, to infer the user’s intended command. The document states, "A user may have difficulty pressing a left-side button, moving the left analog stick, pressing a trigger or other portions of a controller. A video game system can be configured to produce control outputs using an AI model based on controller inputs. The AI model is at least partially trained on control performance information and accessibility-related controller data." This means the model is trained on data about how controls are used and on accessibility-specific data, and it uses that training to interpret imperfect inputs.

The patent says the AI can adjust the degree of compensation for button presses based on the individual user, the type of controller and the game environment. It also describes the software’s ability to predict a player’s next movement from early inputs and the player’s prior play history, which the system can use to produce a final control output that matches the intended action.

Deployment and multiplayer controls

According to the filing, the system can run locally on a console or be provided via cloud services. In multiplayer sessions, players can choose to disable the AI assistant, and the system will present an indicator to show other players whether the assistant is active. The patent also notes that the approach is not limited to PlayStation-specific controllers and could be implemented for other controller types outside of Sony’s ecosystem.

The filing presents these capabilities as a technical approach described in the patent text. The company’s descriptions in the patent attribute functionality to the AI model and its training data, and do not include demonstrated shipping products or performance figures.