John J Bledsoe

US Patent:

20210390455, Dec 16, 2021

Filed:

Jun 10, 2021

Appl. No.:

17/344252

Inventors:

- Boston MA, US
Drew Roselli - Woodinville WA, US
Christopher Cozzi - Denver CO, US
Samuel Clark - Boston MA, US
John Bledsoe - Columbus OH, US
Mykola Novik - Chernihivs'ka Oblast, UA
Amar Mudrankit - San Jose CA, US
Lior Amar - Sunnyvale CA, US
Evan Chang - Sunnyvale CA, US
Scott Oglesby - Cupertino CA, US
Tristan Robert Spaulding - Arlington MA, US

International Classification:

G06N 20/00
G06N 5/04

Abstract:

The subject matter of this disclosure relates to systems and methods for monitoring and managing machine learning models and related data. Histogram structures can be used to aggregate streams of numerical data for storage and metric calculations. Drift in such data can be identified and monitored over time. When significant drift is detected and/or when model accuracy has deteriorated, models can be automatically refreshed with updated training data and/or replaced with one or more other models. A model controller is used to automate model monitoring and management activities across multiple prediction environments where models are deployed and prediction jobs are executed.