Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check if ASR grammars exist before access #24

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

danslavetskiy
Copy link
Contributor

@danslavetskiy danslavetskiy commented Feb 21, 2025

Hi Team,

unfortunately #20 didn't fix the issue and we still have crashes on production. The main reason is that there are situations when grammars get destroyed and then other functions assume that they exist and locking here does not solve the problem. So i've quickly added some checks to see if this really helps and it does..

i'm new to the project and don't know how to do it better:) maybe the issue is layer upper - in the code that calls mod_unimrcp.

@@ -1459,7 +1459,7 @@ static const char *speech_channel_type_to_string(speech_channel_type_t type)
static switch_status_t speech_channel_set_param(speech_channel_t *schannel, const char *param, const char *val)
{
switch_mutex_lock(schannel->mutex);
if (!zstr(param) && val != NULL) {
if (!zstr(param) && val != NULL && schannel->params) {
Copy link
Contributor Author

@danslavetskiy danslavetskiy Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get's destroyed in speech_channel_destroy

switch_core_hash_destroy(&schannel->params);

then used in

switch_core_hash_insert(schannel->params, p, v);

below core dump

#0  0x00007f7539767602 in hash (h=0x0, k=0x7f722c0a91a0) at ./src/include/private/switch_hashtable_private.h:53
#1  switch_hashtable_insert_destructor (h=h@entry=0x0, k=k@entry=0x7f722c0a91a0, v=v@entry=0x7f72001ea0d0, flags=flags@entry=(HASHTABLE_FLAG_FREE_KEY | HASHTABLE_DUP_CHECK), destructor=destructor@entry=0x0) at src/switch_hashtable.c:197
#2  0x00007f7539663944 in switch_core_hash_insert_destructor (hash=0x0, key=key@entry=0x7f72001ea0b8 "speech-language", data=data@entry=0x7f72001ea0d0, destructor=destructor@entry=0x0) at src/switch_core_hash.c:148
#3  0x00007f753520cc70 in speech_channel_set_param (schannel=0x7f722c17d608, param=<optimized out>, val=0x7f722c08cba2 "pl-PL") at mod_unimrcp.c:1471
#4  0x00007f7539659965 in switch_core_asr_load_grammar (ah=0x7f722c17d560, 
    grammar=grammar@entry=0x7f72001e9f2a "{ speech-language=pl-PL, session-id=60ea7a18-3fd3-4ba3-bd57-695a77712c8f, ai.lekta.asr.id=dictation, ai.lekta.asr.trace-id=60ea7a18-3fd3-4ba3-bd57-695a77712c8f, ai.lekta.asr.external-id=42ed1d9d-6445-"..., name=name@entry=0x7f75399a4de7 "") at src/switch_core_asr.c:151
....

if ((status = grammar_create(&g, name, type, data, schannel->memory_pool)) == SWITCH_STATUS_SUCCESS) {
recognizer_data_t *r = (recognizer_data_t *) schannel->data;
recognizer_data_t *r = (recognizer_data_t *) schannel->data;
if (r->grammars && (status = grammar_create(&g, name, type, data, schannel->memory_pool)) == SWITCH_STATUS_SUCCESS) {
Copy link
Contributor Author

@danslavetskiy danslavetskiy Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grammars get destroyed in recog_asr_close

switch_core_hash_destroy(&r->grammars);

then used in

switch_core_hash_insert(r->grammars, g->name, g);

Thread 1 (Thread 0x7fdd40baf6c0 (LWP 34013)):
#0  0x00007fdd985c1602 in hash (h=0x0, k=0x7fdb50191690) at ./src/include/private/switch_hashtable_private.h:53
#1  switch_hashtable_insert_destructor (h=h@entry=0x0, k=k@entry=0x7fdb50191690, v=v@entry=0x7fda900866a0, flags=flags@entry=(HASHTABLE_FLAG_FREE_KEY | HASHTABLE_DUP_CHECK), destructor=destructor@entry=0x0) at src/switch_hashtable.c:197
#2  0x00007fdd984bd944 in switch_core_hash_insert_destructor (hash=0x0, key=<optimized out>, data=data@entry=0x7fda900866a0, destructor=destructor@entry=0x0) at src/switch_core_hash.c:148
#3  0x00007fdd66a320c7 in recog_channel_load_grammar (data=0x7fdb5004a870 "session:de3cd22c-cbb2-4e89-99ad-df98865999a2", type=GRAMMAR_TYPE_URI, name=0x7fda90086678 "de3cd22c-cbb2-4e89-99ad-df98865999a2", schannel=0x7fdb500b0328) at mod_unimrcp.c:2433
#4  recog_asr_load_grammar (ah=<optimized out>, grammar=<optimized out>, name=0x7fda90086678 "de3cd22c-cbb2-4e89-99ad-df98865999a2") at mod_unimrcp.c:3313
...

so we check if exists in below cases

same for enabled_grammars


recognizer_data_t *r = (recognizer_data_t *) schannel->data;
switch_log_printf(SWITCH_CHANNEL_UUID_LOG(schannel->session_uuid), SWITCH_LOG_DEBUG, "(%s) Disabling all grammars\n", schannel->name);

switch_mutex_lock(schannel->mutex);
switch_core_hash_destroy(&r->enabled_grammars);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread 1 (Thread 0x7f923fe546c0 (LWP 34592)):
#0  __GI_abort () at ./stdlib/abort.c:107
#1  0x00007f931fa92395 in __assert_fail_base (fmt=0x7f931fc06a90 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7f932001b7a8 "hash != ((void *)0) && *hash != ((void *)0)", file=file@entry=0x7f932001b76a "src/switch_core_hash.c", line=line@entry=51, function=function@entry=0x7f932001b8f0 <__PRETTY_FUNCTION__.6> "switch_core_hash_destroy") at ./assert/assert.c:92
#2  0x00007f931faa0eb2 in __GI___assert_fail (assertion=assertion@entry=0x7f932001b7a8 "hash != ((void *)0) && *hash != ((void *)0)", file=file@entry=0x7f932001b76a "src/switch_core_hash.c", line=line@entry=51, function=function@entry=0x7f932001b8f0 <__PRETTY_FUNCTION__.6> "switch_core_hash_destroy") at ./assert/assert.c:101
#3  0x00007f931fcd661a in switch_core_hash_destroy (hash=hash@entry=0x7f90e4061820) at src/switch_core_hash.c:51
#4  0x00007f92ef409135 in recog_channel_disable_all_grammars (schannel=schannel@entry=0x7f90e4140498) at mod_unimrcp.c:2543
#5  0x00007f92ef410c77 in recog_asr_load_grammar (ah=<optimized out>, grammar=<optimized out>, name=0x7f8fb4149e18 "f88801fb-f8f2-456f-b147-35319489e7c3") at mod_unimrcp.c:3321

if (r->enabled_grammars) {
switch_core_hash_destroy(&r->enabled_grammars);
switch_core_hash_init(&r->enabled_grammars);
status = SWITCH_STATUS_SUCCESS;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so that we omit recog_channel_enable_grammar and recog_channel_start in

if (recog_channel_disable_all_grammars(schannel) != SWITCH_STATUS_SUCCESS) {

@danslavetskiy danslavetskiy changed the title ASR grammar conditions Check if ASR grammars exist before access Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants