Intermediate Representation Support for ML Engine
October 19, 2022
This case study emphasizes the role of MulticoreWare in creating and implementing a software layer to enable PyTorch backend support using the client’s existing software stack.
The Client
The client is one of the top chip manufacturers with a custom AI/ML accelerator for training and inference pipeline.
The Project
Enable PyTorch (An open-source machine learning framework that accelerates the path from research prototyping to production deployment) support for their custom hardware by extending the software stack in an optimised manner and provide an end-to-end test suite for the same.
Challenges
- Limited operator support in customer’s kernel Library
- Multiple abstraction layers in the customer’s software stack adds more complexity
Solutions Proposed
MulticoreWare’s engineers decided to use a combination of existing kernels to realize operators which were not supported in the kernel library of the customer.
Test suite with wide coverage to validate all operators for which support was added.
The MulticoreWare Advantage & Approach
MulticoreWare has 8+ years of experience working across layers of multiple clients’ ML Software stack. Be it kernel writing or kernel optimization or addition of ML Framework support or Model development & tuning, we have years of expertise on these areas.
In addition, we also have expertise from the much older Caffe framework to the current PyTorch. We quickly put together a team consisting of a solution architect, senior developers, and quality analysts to conduct a comprehensive assessment of the current technical ecosystem.
OUTCOME
80% of the OPs present in PyTorch framework were enabled to work in client’s hardware/software stack. These were validated using test suite for functional correctness.
