How can I use inference.crop() with a list of segments for SpeechBrain/NeMo models?

[RakeshReddyKondeti]

Hello,

I am using the following approach (from the official examples) to extract embeddings for specific segments using the SpeechBrain ECAPA model:

import torch
from pyannote.audio.pipelines.speaker_verification import PretrainedSpeakerEmbedding
model = PretrainedSpeakerEmbedding(
    "speechbrain/spkrec-ecapa-voxceleb",
    device=torch.device("cuda"))

from pyannote.audio import Audio
from pyannote.core import Segment
audio = Audio(sample_rate=16000, mono="downmix")

# extract embedding for a speaker speaking between t=3s and t=6s
speaker1 = Segment(3., 6.)
waveform1, sample_rate = audio.crop("audio.wav", speaker1)
embedding1 = model(waveform1[None])

With pyannote models, I can use the Inference wrapper to easily extract embeddings for segments, like this:

from pyannote.audio import Inference
from pyannote.core import Segment

inference = Inference("pyannote/embedding")
embedding = inference.crop("audio.wav", Segment(3, 6))
# or for multiple segments:
embeddings = inference.crop("audio.wav", [Segment(1, 4), Segment(10, 15)])

However, the Inference wrapper expects a model that is a subclass of pyannote.audio.core.model.Model, and does not work with SpeechBrain or NeMo models (like ECAPA or Titanet), which are wrapped by PretrainedSpeakerEmbedding.

Question:

Is there a recommended way to pass a list of segments at once (like with inference.crop) and efficiently extract embeddings for all of them when using SpeechBrain or NeMo embedding models?

Thank you!

[hbredin]

Unfortunately, we are deprecating support for Speechbrain and NeMo models for this very reason: they are not pyannote-native models and cannot easily be used within this Inference class...