Spotify’s Terraform module, which is used for creating Google Kubernetes Engine (GKE) clusters to run Kubeflow machine learning (ML) pipelines, has been open-sourced by the company. This would help Spotify to do 7x more experiments in developing end-to-end machine learning solutions and deploying these solutions to the market faster than ever before.
Spotify’s Discover Weekly service uses machine learning to provide music recommendations to users. The company initially used Scala language and other frameworks, most of which have been open-sourced by Spotify. The music streaming company faced several drawbacks for choosing these tools and frameworks as they didn’t scale well.
It started the “Paved Road” concept to address the data interface problems in their ML workflow. They started using TFRecord and tf.Example formats by Google’s TensorFlow Extended (TFX) and Tensorflow Data Validation (TFDV). However, these components and tools lacked the orchestration framework. Spotify then moved to Kubeflow Pipelines in which the clusters are configured using the Terraform module. Now, the company is using Kubeflow Pipelines for managing its entire machine learning lifecycle.
Open-sourcing the Terraform module would enable Spotify to develop and release machine learning-based services with faster time-to-market delivery.
If you want to leverage the power of emerging technologies like machine learning, artificial intelligence, big data, cloud computing, and IoT for the growth of your business, then contact our experts to discuss your needs.
4 Reasons to Say Yes to Swift For iOS App Development!
5 DIY Tips Before You Hire SEO Expert for Your Company
Amplify Your Success Graph by Connecting With the Right Custom Mobile Development Company
Most Popular Designing Tools and How They are Helpful for Businesses
5 Simple Tricks to Minimize Ad Budget for your Campaign
Copyright Disclaimer: Unauthorized use, publication, and/or duplication of our website and blog content without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Source Soft Solutions with appropriate and specific direction to the original content.