guildMemberSpeaking event no longer working?

[tacodan]

Please describe the problem you are having in as much detail as possible:

All of a sudden the "guildMemberSpeaking" stopped working. The bot was running did not crash, no errors, all of a sudden the event stop working. It will however emit once when a user starts talking but never will again for that user.

The following is an example of my entire test bot to show the issue:

const path = require('path');
SERVER_DIST = (path.basename(__filename) == "bot.js" ? "prod" : "beta");
CONFIG = require('./config.js').loadVar(SERVER_DIST);

const Discord = require('discord.js');
const client = new Discord.Client();

client.on('error', console.error);

client.on('ready', () => {
	console.log(`Logged in as ${client.user.tag}!`);
	let voiceChan = client.channels.get(CONFIG.ids.mainVoiceChanID);
	voiceChan.join().then(connection => {
		console.log('Connected to voice channel.');
	}).catch(err => console.log(err)); 
});

client.on('guildMemberSpeaking', (member, speaking) => {
	if(speaking) {
		console.log(member.displayName + ' started talking.');    
	}
	else {
		console.log(member.displayName + ' stopped talking');
	}
});

client.login(CONFIG.discordApi.token);

Further details:

• discord.js version: 11.5.1
• Node.js version: v10.15.3
• Operating system: CentOS release 6.10 (Final)
• Priority this issue should have – please be realistic and elaborate if possible: We use this event to track voice activity in a voice channel, this is very important to this bots usage.

• I have also tested the issue on latest master, commit hash:

[dragonbane0]

Can confirm this started happening for us yesterday as well and really killed us. In our case it happens with the connection.on('speaking') event in the exact same way as described. I assume a discord API change?

[TreZc0]

This is becoming a rather severe issue for us, is there any fix on the horizon?

[1-max-1]

The exact same issue happened for me as well. At first I thought it was a bug in my code, but now I am nearly certain it isn't. Unusual. A fix would be greatly appreciated. :-)

[njohns67]

Same issue here. The connection.on("speaking") event seems to fire twice immediately and then never again. I've been debugging for hours trying to figure out what I did wrong before finally finding this thread.

[monbrey]

Is everyone here encountering this issue only on stable? Has anyone had the same issue on the master branch of djs?

[njohns67]

@monbrey I can't try it with the master branch atm but I can tomorrow. If you want to try it yourself I'm sure we'd all appreciate it. If not I'll let you know tomorrow

Edit: It doesn't seem that the master branch is even stable enough to test with which seems peculiar. I get a Syntax Error: Unexpected token * at src/structures/interfaces/Collector.js:203
Possibly could be some installation issue on my part.

[monbrey]

Syntax Error: Unexpected token * at src/structures/interfaces/Collector.js:203

This suggests your version of Node is outdated - I believe support for async iterators was added in Node 10 which the master branch requires.

[Moebits]

Same problem on master as well. Seems like there was an api change

[tacodan]

I know this feature isn't officially supported as it has been pointed out to me on discord but this feature is really important to my bot. I'd really appreciate it of someone could possibly look into it and determine if maybe there is a work around. I understand this might not be possible but either way I'd like to say in advance and that your efforts are greatly appreciated.

[DevYukine]

As you already pointed out this feature is not "official" supported by discord because recieving voice is completely undocumented and everything undocumented is subject to changes at any point without any notice.

Opinion Ahead!

This is my own opnion and does not reflect any position of the Library or maintainer/developer of it.

To me this is more like a legacy feature now, because it used to be (in-official) supported but discord decided against documenting it and i think with that d.js should not support it anymore. We should maybe deprecate this and remove it in the next major release because it might break at any point without a reliable way to fix that beside testing/reverse engineering against the api (which is forbidden btw :^) ).

@tacodan sorry that this might break the "really important" feature but i feel like this is just legacy & maintaining it is a pain at this point.

[dragonbane0]

That's unfortunate to hear. Our bot literally relies on being able to capture the isSpeaking event to properly show on our streaming layouts who is currently speaking or not. This is a major convenience feature for the viewers and currently is broken in the sense that everyone is showcased as speaking all the time. So this being supported was a major draw for discord.js.

However I understand if maintaining something that requires constant reverse engineering since discord wastes no effort to document and officially support it is too much to ask for.

If it could be looked into in case it is something simple it would be appreciated, else I try it with a fork I guess. If it gets removed entirely I hope at least getting the members who are inside a voice channel is still gonna work

[amishshah]

It seems that no speaking events are received after the first one from the voice gateway. I've asked for some more information on this 👍

[tacodan]

It seems that no speaking events are received after the first one from the voice gateway. I've asked for some more information on this 👍

Thank you very much for your time looking into this amishshah. :)

[dragonbane0]

Co-signed. Thank you very much amishshah if you can figure something out. As simple as it is, it really allowed all sorts of cool stuff to know when someone speaks. Really strange discord's documentation includes some voice stuff, but not this particular field

EDIT: Looking into it a bit, it really does seem there is no isSpeaking event any longer after the first one. Yet the Discord client is still able to discern whether someone is speaking or not, so I guess maybe they determine the isSpeaking event solely based on whether new voice packages arrive for that user in a certain timespan or something?

[dragonbane0]

This is really weird. If you open discord in the browser where the voice connection seemingly uses the webrtc protocol instead of UDP, there are still isSpeaking events coming through for 0 and 1. Feels like a discord bug tbh, but the client can handle it, so it doesn't rely on it to determine whether someone is speaking I guess

[njohns67]

As a side issue, has anyone else had issues with connection.createPCMStream() after connecting to a voice channel? When on("speaking") broke I tried moving my voice receive code outside that block since it runs off the same connnection returned from joining the voice channel but it doesn't seem to be working.

[dragonbane0]

Capturing voice only works if you send some audio first since a while. The latest master fixed this by making the bot play some silence upon joining a voice channel. Also in the latest stable release, due the onSpeaking event never coming through with a false value, the streams you create with createPCMStream() might not get destroyed properly when the user stops speaking

[njohns67]

Thanks I forgot to include my code to play a sound. I'll keep trying to workaround for the on.("speaking") bug

Edit: It seems you were right; since the stream is supposed to open and close when the user is speaking it doesn't work properly, or I couldn't get it to.

[jhgg]

We (discord) now only send speaking event once per WS connection - as our clients dont need this event beyond doing initial ssrc => user_id association.

There is a work-around that involves synthesizing the speaking events from UDP packet flow (which is what the native client does.)

See this example psuedo-ish code for more details:

const speakingMembers = new Map();
const SPEAKING_DELAY = 250;

onRtpPacket(packet) {
    const userId = someSsrcToUserIdMapping.get(packet.ssrc);
    if (userId == null) return;
    const currentSpeakingTimeoutRef = speakingMembers.get(userId);
    if (currentSpeakingTimeoutRef == null) {
        this.emit('speaking', userId);
    } else {
        clearTimeout(currentSpeakingTimeoutRef);
    }
    const newSpeakingTimeoutRef = setTimeout(() => { this.emit('stopSpeaking', userId);  speakingMembers.delete(userId); }, SPEAKING_DELAY);
    speakingMembers.set(userId, newSpeakingTimeoutRef); 
}

[tacodan]

Randomly without restarting my bot, changing any code or updating to a different version of d.js it all of a sudden is now working again. Maybe discord reverted their breaking change?

1 of my 2 test servers it's working on..... so still odd.

Is this working for anyone else?

[bodenbao]

The VoiceConnection.on listen event is still broken. Just confirmed this.

[dragonbane0]

I couldn't wait for this to get fixed, since we actively use this feature in a relatively big production. So I investigated a bit and indeed inspecting the UDP packages seems to be the way to go. Here is my spin on a quick fix I made 2 days ago, which surprisingly resembles jhgg's version pretty much.

This is designed for the latest stable version of discord.js, but it can be easily translated I'm sure.

src/client/voice/receiver/VoiceReceiver.js after L30 add:
this.speakingTimeouts = new Map();

Before L52 add:

if (this.speakingTimeouts.get(ssrc)) {
  clearTimeout(this.speakingTimeouts.get(ssrc));
  this.speakingTimeouts.delete(ssrc);
}
else {
  this.voiceConnection.onSpeaking({user_id: user.id, ssrc: ssrc, speaking: true});
}   
let speakingTimer = setTimeout(() => {
  try {
    this.voiceConnection.onSpeaking({ user_id: user.id, ssrc: ssrc, speaking: false });
    this.speakingTimeouts.delete(ssrc);
  }
  catch (ex) {
    console.log("Connection already closed");
  }
}, 50);

this.speakingTimeouts.set(ssrc, speakingTimer);

This gives you the connection.on('speaking') and client.on('guildMemberSpeaking') events back in a pretty much perfectly working state, as well as fix the streams not ending when someone stops speaking if you record them.

Remember you need to play some silence first in order for the bot to receive voice packages properly.

[Xilophinum]

The guildMemberSpeaking event is somehow working again, even though jhgg confirmed its only supposed to be firing only once. This makes it hard because it seems that I am making the event fire twice now if I implement any changes, or breaks it again all together, because I assume it is cancelling itself out. If this is only temporary, and is supposed to break again, what is the most ideal way to handle double events on master v12 until we find out? I am trying to handle the events inside the packetHandler push function, since it seems the most logical place to get the user and ssrc. Sorry to piggyback off the post, but this seems like the best place to secure an answer for myself and others trying to fix their bots.

[njohns67]

@dragonbane0 This does seem to properly emit the on("speaking") event but I'm running into some bugs that I didn't have previously. It seems the event fires twice in a row, causing a pcmStream error if you are trying to use the incoming voice data.

newMember.voiceChannel.join().catch(console.error).then(async function(connection) {
        connection.on("disconnect", () => console.log("Disconnecting dispatcher"))
        console.log("Connected")
        if(first){
           class Silence extends Readable {
               _read(){
                   this.push(Buffer.from([0xF8, 0xFF, 0xFE]))
               }
            }
           connection.playOpusStream(new Silence())
            first = false
        }
        const receiver = connection.createReceiver()
        connection.on('speaking', (user, speaking) => {
            console.log("Speaking")
            if (!speaking){
                return
            }
            const audioStream = receiver.createPCMStream(user)
           //More code to do stuff with the audioStream
        })
    })
})

This is my code minus any extraneous bits for readability's sake. Currently my console log looks like:

$ node bot.js
$ speaking
$ speaking
$  if (this.pcmStreams.get(user.id)) throw new Error('There is already an existing stream for that user.');
                                      ^

Error: There is already an existing stream for that user.

As you can see, it emits the event twice before crashing. Through testing I've found that it only emits twice the first time you speak. I'm guessing the first time is the event properly being emitted (as it was before everything broke) and the second time is your code emitting the event with subsequent emits being your code as well.

I was able to somewhat workaround this by modifing VoiceReceiver.js L161 to return the existing pcmStream instead of throwing an error, but this seems like a bandaid. It also cause the voice data to be corrupted.

[SpaceEEC]

This has been fixed in both relevant branches now:

• master -> Start/Stop speaking events on UDP packets #3578 (linked above somewhere)
• 11.5-dev -> fix(Voice*): fix speaking event and voice receive #3749

[jackred]

EDIT it works fine now
EDIT it's not :). I fixed it on v12 by inserting a silence frame when joining the channel

Using 11.5-dev (c955fd00c7ef5835e022db45ac16d8fe24689455)

The event guildMemberSpeaking on client and speaking are working, but intermittently.

When I make the bot join the vocal channel, sometimes the events work perfectly, sending true/false every time I start/stop speaking, but most of the time it just do nothing.

I tried on different server, using different bot, and trying alone or with someone. I couldn't find a denominator between the working / not working part.
I thought it was a problem with how I closed the bot, as I originally found some sort of connection with the event working when using a new bot or a new channel, but after more test it doesn't seem like it.

using:

function handleVocalMessage(userOrMember, speaking){
  console.log(userOrMember.displayName || userOrMember.username, "is talking?", speaking);
}

function joinChannel(channel) {
  channel.join().then(d => {
    console.log(`INFO! joined ${d.channel.name}`);
    d.on('speaking', handleVocalMessage);
  });
}

client.on('guildMemberSpeaking', handleVocalMessage);

(this debug only used one of the 2 events)

[KilMer56]

Hey, for me it's still not working on the master branch. In fact, with the simple following code :

connection.on('speaking', (user, speaking) => { console.log(speaking); });

The console is only displaying 'true' once and then never work again.

[foxt]

Seems to be busted again. Connection.on('speaking',(user,speaking) => {console.log("speaking")}) don't output anything, same with the client's guildMemberSpeaking and debug logging only outputs the one speaking event when the user first speaks after joining the VC.

[sillyfrog]

@theLMGN , would you please try running the https://github.com/sillyfrog/discord.js-listenexample example? I have just rebuilt and tested again (so it's against current master), and things are working OK. I would suggest it's most likely dependencies or similar causing issues.

[SunburntRock89]

@sillyfrog The script hasn't been updated and doesn't follow the new ChannelManager class change and so doesn't work with current master. It hasn't been updated since December, maybe you forgot to commit?

[SunburntRock89]

Followup - changing it to grab the channel from master, it actually does work. There seems to be some kind of link between playing a sound and having the speaking event fire. Strange.

[dragonbane0]

Yes you need to "speak" before Discord gives you audio. This has been a change on their side and can not be avoided

[sillyfrog]

@SunburntRock89 Thanks, it did indeed need some updates (including pulling from the correct repo)! That's now done should anyone else need it. Cheers.

[AlbertMarashi]

This still doesn't work in V12? Buggy as hell

[SunburntRock89]

What's your issue @DominusVilicus
To receive audio, you need to at least once, so you need to play at least a little bit of audio into the channel. It can't be fixed. If you have another issue feel free to ask.

[amishshah]

Is anyone on this thread still experiencing this issue on the master branch of discord.js?

[MaxmaxmaximusGitHub]

@amishshah

speakingTimeout is number, not the object =)

you need like this:

    let speakingTimeout = this.speakingTimeouts.get(ssrc);

    if (typeof speakingTimeout !== 'undefined') {
      this.receiver.connection.client.clearTimeout(speakingTimeout)
    }

    // Ensure at least the speaking bit is set.
    // As the object is by reference, it's only needed once per client re-connect.
    if (userStat.speaking === 0) {
      userStat.speaking = 1;
    }
    this.connection.onSpeaking({user_id: userStat.userID, ssrc: ssrc, speaking: userStat.speaking});

    speakingTimeout = this.receiver.connection.client.setTimeout(() => {

      try {
        this.connection.onSpeaking({user_id: userStat.userID, ssrc: ssrc, speaking: 0});
        this.receiver.connection.client.clearTimeout(speakingTimeout);
        this.speakingTimeouts.delete(ssrc);
      } catch {
        // Connection already closed, ignore
      }

    }, DISCORD_SPEAKING_DELAY);

    this.speakingTimeouts.set(ssrc, speakingTimeout);

[sillyfrog]

@MaxmaxmaximusGitHub how are you running this? What version of Node etc? That method should return a timer object (not a number).

[MaxmaxmaximusGitHub]

@sillyfrog

master, and 12 branches,
electron with node 12

returning id of timeout, not object

[sillyfrog]

@MaxmaxmaximusGitHub I'm guessing that electron is messing with the setTimeout implementation to make it more browser like (I know basically nothing about electron, so may well be off the mark). The node docs show it should return a <timeout> object: https://nodejs.org/api/timers.html#timers_settimeout_callback_delay_args
Running my example just now (in a fresh Node 12 docker image), adding in a log line:

@@ -103,6 +103,7 @@
       }, DISCORD_SPEAKING_DELAY);
       this.speakingTimeouts.set(ssrc, speakingTimeout);
     } else {
+      console.log("speakingTimeout:", speakingTimeout);
       speakingTimeout.refresh();
     }

I get this output (a lot as it's on every single packet):

speakingTimeout: Timeout {
  _idleTimeout: 250,
  _idlePrev: [TimersList],
  _idleNext: [TimersList],
  _idleStart: 7728,
  _onTimeout: [Function],
  _timerArgs: undefined,
  _repeat: null,
  _destroyed: false,
  [Symbol(refed)]: true,
  [Symbol(asyncId)]: 858,
  [Symbol(triggerId)]: 95
}

Hopefully that helps at least point you in the right direction. Cheers.

[d4sein]

I'm having a similar issue, the event actually fires every time I talk and stop talking, but I can't receive any stream of audio from it.

    case 'join':
      if (ctx.member.voice.channel) {

        const connection = await ctx.member.voice.channel.join()
        connection.play(new Silence(), { type: 'opus' })

        connection.on('speaking', (user, speaking) => {
          let audioStream = connection.receiver.createStream(user, { mode: 'pcm' })
  
          // Transforms the audio stream into something Dialogflow understands
          ffmpeg(audioStream)
            .inputFormat('s32le')
            .audioFrequency(44100)
            .audioChannels(1)
            .audioCodec('pcm_s16le')
            .format('s16le')
            .on('error', console.error)
            .pipe(fs.createWriteStream(`${user.id}.wav`))
          
          if (!speaking) return
        })
      }
      break

It works If I do the same thing outside of connection.on('speaking', ...).
client.on('guildMemberSpeaking') also doesn't work.

EDIT: sillyfrog's example works just fine, my bad for being lazy, lol. The only problem is that it always fires twice, I'm trying to find a workaround for that.

[MegaNoam]

Is there anything new with the issue? @amishshah

[amishshah]

Hi there,

We're working on a new implementation of Discord's Voice API that has better playback quality and is more reliable than what we currently support in Discord.js v12 - check it out at https://github.com/discordjs/voice!

The new library solves many of the issues that users are facing, and as part of this, we're dropping built-in support for voice in our next major release. We have a PR (#5402) that adds native support for our new voice library - once this PR is merged, this issue will be closed.

You can still use our new voice library before that PR lands - just take a look at our music bot example to see how to get started upgrading your voice code. By using the boilerplate music player in the example, you can make it even easier to upgrade your code.

Note that the PR above only reduces some of the boilerplate code you'd otherwise have to write - you do not have to wait for the PR to be merged to start using the new voice library.

---

If you have any questions about this, feel free to:

• Make an issue if you have found a bug in the new voice library
• Use GitHub Discussions or join our Discord server (we have a new channel, #djs-new-voice, specifically for this!) to ask general questions about the library, give feedback on the library, and get support with upgrading to it

---

Specific to this issue:

This event relies on the voice receive feature, which is undocumented by Discord and prone to breaking quite a lot. As it's an undocumented feature, we cannot guarantee support for it.