Data Science Platform Automation

SITUATION

Our client needed to implement a unified data science platform that would reduce the complexity, overhead costs, and resource bandwidth required to build individual machine learning solutions. They wanted to reduce friction between the machine learning model development process and the creation of the production services that rely on their data. The goal of the platform was to free up their data scientists to focus on their core skill sets, while empowering them to more quickly deploy their solutions into production.

SOLUTION

Our team identified inefficiencies between the client’s data science, application development, and DevOps teams to produce an improved deployment process. We partnered with the client to build a new machine learning platform that included a model execution engine, REST API, scoring repository, and feature engineering pipeline. Specifically, our solution provided:

• Fully automated model deployments and upgrades, while still allowing extensible APIs and customizations, with an easy-to-use web and command-line interface
• A standard REST API for all models with configurable performance profiles, built for performance tuning customized to typical machine learning needs
• An integrated data pipeline connecting Kafka and S3 sources to the machine learning model, capturing model scores for reuse and further analysis

RESULTS

Our team successfully delivered a new machine learning platform with enhanced tools for automation deployment, enabling data scientists to focus on their core responsibilities. The platform reduced the effort required to deploy and upgrade our client’s machine learning models from one week and multiple team members, to just a few hours using a fully automated process. By the end of the project, more than 50 new machine learning models had already been deployed into production.

Click here to read this in Spanish.