Yelp Unifies ML Model Training with Training Orchestrator
Quick Answer
Yelp's new Training Orchestrator replaces fragmented ML training scripts with a unified, configuration-driven framework, enhancing reproducibility and efficiency across teams.
Quick Take
The DAG-based model allows for local runs, improved testing, and streamlined orchestration, addressing common issues in large ML platforms.
Key Points
- Training Orchestrator uses Pydantic-based configurations for improved validation and efficiency.
- The framework allows for local runs and unit testing, enhancing development speed.
- Automatic logging of configurations as MLflow artifacts enables easy reruns.
- Yelp plans to implement model evaluation and lineage tracking in future updates.
- This centralization trend is seen across the industry, improving reproducibility and testability.
DeepSignal Analysis
What happened
Yelp has introduced the Training Orchestrator, an internal framework designed to unify machine learning model training. This system replaces fragmented Spark training scripts with a configuration-driven, Directed Acyclic Graph (DAG) model, which enhances reproducibility and efficiency across teams by addressing issues like duplicated code and inconsistent configurations.
Key evidence
- The Training Orchestrator replaces individual team Spark training scripts, addressing problems like duplicated code and inconsistent configurations.
- Yelp's previous training setup was monolithic and tightly coupled to Spark, causing slow iteration and poor testability.
- The new framework allows for local runs and unit testing, which improves the development process and reduces integration test times.
Why it matters
This development reflects a broader industry trend toward centralized machine learning infrastructure, which aims to improve reproducibility and efficiency. By addressing common issues faced by multiple teams, Yelp's Training Orchestrator could serve as a model for other organizations facing similar challenges. The ability to run tests locally and validate configurations quickly may lead to faster iterations and more reliable deployments.
Source Excerpt
Yelp has launched Training Orchestrator. This new internal framework replaces individual team Spark training scripts. Now, it uses a configuration-driven, DAG-based execution model.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from InfoQ AI, ML & Data Engineering
See more →Google Cloud Workbench Notebooks Extension Connects VS Code to Google Cloud's Jupyter Notebooks
The Google Cloud Workbench Notebooks extension for VS Code allows developers to seamlessly connect their local IDE to managed Jupyter notebook environments on Google Cloud, enhancing ML workflow efficiency. This integration eliminates context switching, enabling smooth transitions from local experimentation to high-performance cloud computing.
