Amazon Web Services Inc. (AWS), has released its project to optimize machine learning models for better performance in the cloud and on resource constrained network edge devices.
The new functionality Neo, which AWS introduced last autumn as part of Amazon SageMaker service in the Amazon Web Services, allows machine learning models to be trained once and then deployed to different locations. This is what led to the donation to the community.
AWS announced Neo for SageMaker as a “deep-learning model compiler [that] allows customers to train models once and run them anywhere with up 2X performance improvement.”
AWS stated that the Neo-AI offering allows users and developers to tune machine learning models for different platforms, hardware, and software configurations. This manual process is further complicated by the insufficient storage and compute resources often found on edge devices, where models are deployed.
“Neo AI eliminates the time and effort required to tune machine learning models for deployment across multiple platforms by automatically optimizing TensorFlow and MXNet, PyTorch and ONNX models to perform at twice the speed of original models with no loss in accuracy,” AWS stated in a Jan. 23 blog posting.
AWS stated that models are converted into a common format to eliminate compatibility issues. A compact runtime consumes a fraction of the resources that a framework would normally use on the target platform. Neo-AI makes optimization easier and allows sophisticated models to run efficiently on resource-constrained devices. This can lead to innovation in areas like autonomous vehicles, home security, anomaly detection, and home security.
Neo-AI currently supports platforms from Intel and NVIDIA and ARM. Support for Xilinx and Cadence is expected to follow soon. AWS stated that its own development team and representatives from these organizations will contribute to the project’s direction. One component of the Neo-AI’s main component, a runtime is currently being deployed on devices from many vendors, including ADLINK, Lenovo Leopard Imaging, Panasonic, and others.
Neo-AI source code is available on GitHub
