Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AsyncTcpClient.from() fails to connect, nc from shell works #28

Open
sdetweil opened this issue Nov 17, 2024 · 4 comments
Open

AsyncTcpClient.from() fails to connect, nc from shell works #28

sdetweil opened this issue Nov 17, 2024 · 4 comments

Comments

@sdetweil
Copy link

earlier this year I created an asr, and built a test client, connect, transcribe, send audiochunks from 2 wav files,
check tanscript..
all good

trying to build my own service runner
testing each component as I go

snd,
mic
vad
wakeword
asr
tts

now I want to do the same test with the snd-external audio out to make sure connection to
local device works (and configured properly)

so I validated the audio output device (manually with aplay -D device_name wav_file) and started snd-external using those same parms

wyoming/wyoming-snd-external$ script/run  --program "aplay -r 22050 -c 1 -f S16_LE -t raw" --rate 22050 --width 2 --channels 1 --debug 
DEBUG:root:Namespace(program='aplay -r 22050 -c 1 -f S16_LE -t raw', rate=22050, width=2, channels=1, samples_per_chunk=1024, uri='tcp://0.0.0.0:10601', debug=True, log_format='%(levelname)s:%(name)s:%(message)s')
INFO:root:Ready
INFO:root:Server Ready

and replicated my wav client to use the snd events..

but it can't connect

the shell

nc localhost 10601

connects as expected
netstat shows the socket listening

netstat -a | grep 10601
tcp        0      0 0.0.0.0:10601           0.0.0.0:*               LISTEN 

here is my client (in the new wyoming-snd-external/test folder)

import wave
import asyncio

from timeit import default_timer as timer
from datetime import timedelta

import argparse
import logging

from wyoming.audio import AudioChunk, AudioStart, AudioStop, AudioChunkConverter
from wyoming.client import AsyncTcpClient
from wyoming.info import Describe
from wyoming.event import async_write_event, async_read_event

_LOGGER = logging.getLogger("wav2speaker_client")

# expected Audio recording parameters
RATE = 16000

# used in the read function to get a chunk of the audio file
CHUNK_SIZE = int(RATE / 10)  # 100ms

# connect to the server snd port and return the connection object
#@staticmethod
async def connect(uri:str) -> AsyncTcpClient:
    tcp = AsyncTcpClient.from_uri(uri)
    await tcp.connect()
    return tcp

# need to be async to use await on wyoming event functions
async def main() -> None:
    "send audio file to playback/snd instance."

    debug:bool  = False    
    
    # define the arguments and --help
    parser = argparse.ArgumentParser()
    parser.add_argument("server", help="uri of the server to connect to")
    parser.add_argument("wav_file", nargs="+", help="Path to WAV file(s) to transcribe")
    parser.add_argument("--debug", action="store_true", default=False, help="Log DEBUG messages")
    args = parser.parse_args()

    # do the debugging logging
    logging.basicConfig(level=logging.DEBUG if args.debug else logging.INFO)

    if args.debug:
        debug = args.debug

    if debug == True:
       _LOGGER.debug("server uri="+args.server)  
       
    # loop thru the wav files (may be one, space separated)
    for wave_file in args.wav_file:
        if debug:
            _LOGGER.debug("wave file="+wave_file)               
        snd_server_connection=None
        try:
 	    # connect to the snd server
            snd_server_connection=await connect(args.server)               
        except:
            print("unable to connect to snd at "+args.server)  // <=== get this message
            return
        # use each wav file in turn
        input_wav_file =  wave.open(wave_file, "r")

            
        with input_wav_file:

            # send the describe event
            await snd_server_connection.write_event( Describe().event())
            # read the info response
            info_event=await snd_server_connection.read_event()

            # Input from wave file the audio characteristics, may need to convert to  snd expected format
            rate = input_wav_file.getframerate()
            width = input_wav_file.getsampwidth()
            channels = input_wav_file.getnchannels()

            # send the transcribe event with language
            await snd_server_connection.write_event(AudioStart(rate,width,channels).event())
            # no response from AudioStart

            if debug:
                _LOGGER.debug("file rate="+str(rate)+" width="+str(width)+" channels="+str(channels))

            # get audio data from the file,      
            audio_bytes = input_wav_file.readframes(CHUNK_SIZE)
            
            # create a converter to insure the audio data is in the right format for the snd
            converter = AudioChunkConverter(
                rate=RATE, width=2, channels=1
            )
            
            # loop thru the audio buffer, 
            while audio_bytes:
                # convert to snd format as required (converter output format set before)
                chunk = converter.convert(
                    AudioChunk(rate, width, channels, audio_bytes)
                )

                # send this chunk to the snd 
                await snd_server_connection.write_event(chunk.event())
                # get more data from the wav file
                audio_bytes = input_wav_file.readframes(CHUNK_SIZE)
                
          
            # no more data, tell snd we are finished sending chunks
            # we will expect a full text output now
            await snd_server_connection.write_event(AudioStop().event())

        # done with this snd 
        await snd_server_connection.disconnect();

if __name__ == "__main__":
  # because we want to use await in the main function we need 
  # to launch as async task 
  asyncio.run(main())

launched like this
test/wav2speaker.py

#!/usr/bin/env bash
set -eo pipefail

# Directory of *this* script
this_dir="$( cd "$( dirname "$0" )" && pwd )"

# Base directory of repo
base_dir="$(realpath "${this_dir}/..")"

# Path to virtual environment
: "${venv:=${base_dir}/.venv}"

if [ -d "${venv}" ]; then
    source "${venv}/bin/activate"
fi
cd test
python3 "${base_dir}/test/wav2speaker.py" "$@

and
test/run_client_wav2speaker.sh

#!/bin/bash
test/wav2speaker.sh --debug tcp://localhost:10601 test.wav test1.wav

test

~/wyoming/wyoming-snd-external$ test/run_client_wav2speaker.sh 
DEBUG:wav2speaker_client:server uri=tcp://localhost:10601
DEBUG:wav2speaker_client:wave file=test.wav
unable to connect to snd at tcp://localhost:10601

the test client for my asr works. asr local or in docker

I just did setup for the 1st time in the wyoming-snd-external folder
so MAY have newer code then asr

@sdetweil
Copy link
Author

sdetweil commented Nov 17, 2024

didnt try nc -l, will later

using nc -l locahost 10601
the client cannot connect there either..
so must be some setup issue.. there was never a client in the config..

@sdetweil
Copy link
Author

so... requirements.txt has wyoming version 1.2, oops

@sdetweil
Copy link
Author

SO... wyoming-snd-external doesn't implement Describe (which my client used) (maybe cause Describe didn't exist in 1.2?)
but now it connects, sends and the service plays the audio on the speaker.. so it is configured properly

@sdetweil
Copy link
Author

can someone transfer this to the snd-external repo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant