Membrane Multimedia Framework: GCloud Speech To Text

This package provides a Sink wrapping Google Cloud Speech To Text Streaming API client. Currently supports only audio streams in FLAC format.

The docs can be found at HexDocs.

Installation

The package can be installed by adding membrane_element_gcloud_speech_to_text to your list of dependencies in mix.exs:

def deps do
  [
    {:membrane_element_gcloud_speech_to_text, "~> 0.10.0"}
  ]
end

Configuration

To use the element you need a config/config.exs file with Google credentials:

use Mix.Config

config :goth, json: "a_path/to/google/credentials/creds.json" |> File.read!()

More info on how to configure credentials can be found in README of Goth library used for authentication.

Usage

The input stream for this element should be parsed, so most of the time it should be placed in pipeline right after FLACParser

Here's an example of pipeline streaming audio file to speech recognition API:

defmodule SpeechRecognition do
  use Membrane.Pipeline

  alias Google.Cloud.Speech.V1.StreamingRecognizeResponse

  @impl true
  def handle_init(_ctx, _options) do
    spec =
      child(%Membrane.File.Source{location: "sample.flac"})
      |> child(Membrane.FLAC.Parser)
      |> child(%Membrane.Element.GCloud.SpeechToText{
          interim_results: false,
          language_code: "en-GB",
          word_time_offsets: true
        })

    {[spec: links], %{}}
  end

  @impl true
  def handle_child_notification(%StreamingRecognizeResponse{} = response, _element, _ctx, state) do
    IO.inspect(response)
    {[], state}
  end

  @impl true
  def handle_child_notification(_notification, _element, _ctx, state) do
    {[], state}
  end
end

The pipeline also requires a config file and the following dependencies:

[
  {:membrane_core, "~> 1.0"},
  {:membrane_file_plugin, "~> 0.16.0"},
  {:membrane_flac_plugin, "~> 0.11.0"},
	{:membrane_element_gcloud_speech_to_text, "~> 0.10.0"}
]

Testing

Tests tagged :external are excluded by default since they contact the real API and require configuration of credentials. See Configuration

Fixture

A recording fragment in test/fixtures comes from an audiobook "The adventures of Sherlock Holmes (version 2)" available on LibriVox

Copyright and License

Copyright 2019, Software Mansion

Licensed under the Apache License, Version 2.0