Embedding Recycling to Accelerate ML Inferencing

In the era of self-supervised deep learning, unstructured data in various modalities have become more useful than ever. However, the high computational costs of running task-specific deep-learning models over the same data often present a significant cost challenge. To address this issue, embedding recycling (ER) has emerged as a promising technique that enables the reuse of intermediate embedding representations for different tasks. By memorizing the output of intermediate layers of deep neural networks as embeddings, practitioners can reuse them for various tasks using task-specific layers conditioned on the task-specific input. 

This talk introduces the concept of recyclable embeddings and how Vespa, an efficient and scalable search and recommendation engine, can efficiently do ML inferencing over many data points with recyclable embeddings.

• Overview of deep-learned embeddings and the growing importance of embedding models for ML
• How to recycle embeddings for different tasks 
• Practical strategies and best practices for efficiently managing and operating deep-learned embeddings in real-world applications.

————————————————————————————————————————————————————

Jo Kristan Bergum – Distinguished Engineer | Yahoo

Jo Kristian works as a Distinguished Engineer at Yahoo, where he spends most of his time on the open-source serving engine, Vespa.ai.

Jo Kristian has 20 years of experience working on search and recommendation problems at scale.