STT Service: Third-party STT errors seem to be silently retried with no user-visible failure.

[raina-rudra]

Problem statement

When running our Speech-to-Text (STT) pipelines in Pipecat, we observed that errors from third-party services are not properly surfaced. Instead of reporting the failure, Pipecat continuously retries the connection without surfacing a meaningful error message or stopping the pipeline. This makes it difficult to identify the root cause and implement any graceful degradation or fallback strategies.

We used the foundational example 15a-switch-languages.py modifying it to use Hindi instead of Spanish. Our Deepgram setup is self-hosted and does not support Hindi.

2025-04-09 19:37:35.842 | INFO     | pipecat.services.stt_service:_update_settings:70 - Updating STT settings: {'channels': 1, 'encoding': 'linear16', 'interim_results': True, 'language': 'en', 'model': 'nova-2', 'punctuate': True, 'profanity_filter': True, 'smart_format': True, 'vad_events': False, 'sample_rate': 16000}
2025-04-09 19:37:35.842 | INFO     | pipecat.services.stt_service:_update_settings:73 - Updating STT setting language to: [hi]
2025-04-09 19:37:35.842 | INFO     | pipecat.services.deepgram.stt:set_language:108 - Switching STT language to: [hi]
2025-04-09 19:37:35.842 | DEBUG    | pipecat.services.deepgram.stt:_disconnect:155 - Disconnecting from Deepgram
2025-04-09 19:37:35.845 | DEBUG    | pipecat.services.openai.base_llm:_stream_chat_completions:156 - OpenAILLMService#0: Generating chat [[{"role": "system", "content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities. Respond to what the user said in a creative and helpful way. Your output should not include non-alphanumeric characters. You can speak the following languages: 'English' and 'Hindi'."}, {"role": "system", "content": "Please introduce yourself to the user and let them know the languages you speak. Your initial responses should be in English."}, {"role": "assistant", "content": "Hello. My name is LLM, and I am here to assist you in any way I can during our chat. I can speak English as well as Hindi, so feel free to communicate with me in either language. If there's anything specific you'd like to know or discuss, just let me know."}, {"role": "user", "content": "To me in Hindi, please."}, {"role": "assistant", "tool_calls": [{"id": "call_1QsrXrgNt6YeR8UXoV4nfXFN", "function": {"name": "switch_language", "arguments": "{\"language\": \"Hindi\"}"}, "type": "function"}]}, {"role": "tool", "content": "{\"voice\": \"Your answers from now on should be in Hindi.\"}", "tool_call_id": "call_1QsrXrgNt6YeR8UXoV4nfXFN"}]]
tasks cancelled error: 
tasks cancelled error: 
2025-04-09 19:37:36.345 | DEBUG    | pipecat.services.deepgram.stt:_connect:131 - Connecting to Deepgram
2025-04-09 19:37:36.563 | DEBUG    | pipecat.services.cartesia.tts:run_tts:278 - CartesiaTTSService#1: Generating TTS [नमस्ते।]
2025-04-09 19:37:36.644 | DEBUG    | pipecat.services.cartesia.tts:run_tts:278 - CartesiaTTSService#1: Generating TTS [ अब से मैं हिंदी में बात करूँगा।]
2025-04-09 19:37:36.746 | DEBUG    | pipecat.transports.base_output:_bot_started_speaking:225 - Bot started speaking
2025-04-09 19:37:36.819 | DEBUG    | pipecat.services.cartesia.tts:run_tts:278 - CartesiaTTSService#1: Generating TTS [ क्या आपकी कोई विशेष आवश्यकता है जिस पर मैं मदद कर सकता हूँ?]
WebSocketException in AbstractAsyncWebSocketClient.start: server rejected WebSocket connection: HTTP 400
AsyncListenWebSocketClient.start failed
2025-04-09 19:37:37.481 | ERROR    | pipecat.services.deepgram.stt:_connect:151 - DeepgramSTTService#0: unable to connect to Deepgram
2025-04-09 19:38:13.494 | DEBUG    | pipecat.transports.base_output:_bot_stopped_speaking:232 - Bot stopped speaking
{"timestamp":"2025-04-09T14:08:31.907783Z","level":"ERROR","fields":{"message":"no subscription for consumer: ConsumerId(\"7729052a-a437-4c01-9508-718efdc625e6\")"},"target":"daily_core::call_manager::events::from_sfu::soup_consumer_closed"}
2025-04-09 19:38:31.907 | INFO     | pipecat.transports.services.daily:_on_participant_left:1327 - Participant left 89895104-6647-4eef-a8e5-4d823474f26f
{"timestamp":"2025-04-09T14:08:31.960635Z","level":"ERROR","fields":{"message":"Failed to close consumer: ConsumerNoLongerExists(ConsumerId(\"7729052a-a437-4c01-9508-718efdc625e6\"))"},"target":"daily_core::call_manager::events::subscription::common"}
^C2025-04-09 19:38:44.508 | WARNING  | pipecat.pipeline.runner:_sig_cancel:75 - Interruption detected. Canceling runner PipelineRunner#0
2025-04-09 19:38:44.509 | DEBUG    | pipecat.pipeline.runner:cancel:62 - Canceling runner PipelineRunner#0
2025-04-09 19:38:44.510 | DEBUG    | pipecat.pipeline.task:cancel:288 - Canceling pipeline task PipelineTask#0
2025-04-09 19:38:44.512 | DEBUG    | pipecat.services.deepgram.stt:_disconnect:155 - Disconnecting from Deepgram
2025-04-09 19:38:44.516 | DEBUG    | pipecat.services.cartesia.tts:_disconnect_websocket:215 - Disconnecting from Cartesia
2025-04-09 19:38:44.517 | DEBUG    | pipecat.services.cartesia.tts:_disconnect_websocket:215 - Disconnecting from Cartesia
2025-04-09 19:38:44.555 | INFO     | pipecat.transports.services.daily:leave:540 - Leaving https://skit.daily.co/pipecat-test
2025-04-09 19:38:44.573 | INFO     | pipecat.transports.services.daily:leave:548 - Left https://skit.daily.co/pipecat-test
2025-04-09 19:38:44.576 | DEBUG    | pipecat.pipeline.runner:run:55 - Runner PipelineRunner#0 finished running PipelineTask#0

Possible Solution:

Is it possible to use ErrorFrame or any thing similar to propagate the error to outer layers so we can either terminate or handle the issue in our own way.