V3.7 (lavalink-devs#804)

* implement rest routes for ws ops

* remove private val from SocketContext field

* implement new player & session rest api according to new implementation proposal

* fix playlistInfo fields being top level in load result

* chore: cleanup stats task

* allow websocket on /v3/weboscket

* fix plugins & info endpoint, add identifier handling in player update endpoint, better request logging, split handlers into more files

* fix route planner & decode track endpoint

* json marshal tweaks

* reorder update player checks

* fix error when pause not provided & capitalization of voice properties

* fix error while deserializing player update

* fix filters

* feat: add /v3/stats endpoint

* chore: convert track end marker handler to kotlin

* chore: add @nullable annotation to IPlayer#getTrack

* chore: convert LavalinkPlayer to Kotlin

* chore: cleanup some stuff/kotlin conversion

All have been tested

* chore: remove gson dependency

* chore: remove unused class

* chore: convert config classes to kotlin

* chore: convert AudioLossCounter to kotlin

* reformat stats stuff, fix inclusion & nullability of json fields

* chore: deprecate ISocketContext#sendMessage(JSONObject)

* fix stopping track, added request logging & cleanup

* remove nullable modifier

* use custom decoder for nullable optional fields

* revert kotlin version changes

* add ws command deprecation warning

* make filters not nullable

* cleanup unneeded stuff

* fix wrong json field name in PlayerUpdateDeserializer

* bump kotlin version again due to issues

* add error for partial voice state update

* include message in lavalink rest error responses & error data class

* automatically register HandlerInterceptor for plugins to intercept rest endpoints

* use custom interface for endpoint interception

* deprecate org json methods in AudioFilterExtension

* add deprecation notice to old methods in AudioFilterExtension

* apply old loadtracks formatting to /v3/loadtracks endpoint too

* allow customization of request logging

* set empty after suffix

* remove debug logs

* formatting & more checks

* validate User-Id header & cleanup sessionId generation

* some more cleanup

* make generateUniqueSessionId beautiful (ty gino)

* replace multi if with when

* proposed api changes for v3.7.0 (lavalink-devs#750)

* proposed api changes for v3.6

* add missing deprecation notices in old rest routes, add changes summary and readd deprecated payloads in details view

* fix typo & add time units to IMPLEMENTATION.md

* Update IMPLEMENTATION.md

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>

* refactor ws op & event headers & move header description into one block

* fix weird wording

* fix broken jump links & rework track loading endpopint

* deprecate version endpoint and add info endpoint

* tweak info example values

* update version in info endpoint & versioning policy

* Update IMPLEMENTATION.md

Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>

* rework event docs with markdown tables

* make detail closed by default

* fix missing detail close tag

* update rest docs to use tables

* update events docs

* add missing details close tag

* rewrite more rest docs to tables & fixed some issues in events & co

* fix anchor

* fix some smaller issues

* fix search result json

* convert missing docs to tables except route planner api & fixed some inconsistencies

* update severity docs

* cleanup some minor stuff

* fix capitalization

* convert missing ip router planning stuff

* add missing lines

* add stats endpoint

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* fix filterBand and fix tables

* add equalizer band frequencies to docs

* Update IMPLEMENTATION.md

* Various changes to the API draft (lavalink-devs#764)

* Various changes to the API draft

* Expanded the change list
* Added future changes in v4 and v5
* Moved WS handshake to /v3/websocket
* Changed structure of Track
* Removed Voice Event, replaced with existing Voice State
* Clarify various things
* And more

* Apply suggestions from code review

Co-authored-by: ToπSenpai <15636011+TopiSenpai@users.noreply.github.com>

Co-authored-by: ToπSenpai <15636011+TopiSenpai@users.noreply.github.com>

* Clarify buildTime

* Fix formatting

* Bump versions

* Update IMPLEMENTATION.md

Co-authored-by: Oliver Wilkes <oliverwilkes2006@icloud.com>

* apply review changes & fix some more stuff

* update max volume

* add note about session resuming

* readd track field to track json with deprecation notice

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* update track events

* update volume range in player update endpoint

* Update IMPLEMENTATION.md

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* fix outdated example json

* update git object

* Apply suggestions from code review

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>

* Fix a grammar isssue

* undeprecate /version

* rename track to encodedTrack in track events

* update significant changes

* update version object

* Update IMPLEMENTATION.md

* Update IMPLEMENTATION.md

* Update IMPLEMENTATION.md

* update player update filters docs

* fix typo

* Update IMPLEMENTATION.md

* Apply suggestions from code review

Co-authored-by: Oliver Wilkes <oliverwilkes2006@icloud.com>

* fix wording

* Update IMPLEMENTATION.md

* add migration guide & deprecation notices

* Update IMPLEMENTATION.md

* update PATCH session endpoint

* update track event docs to match the current behavior

* fix some fields being optional

* update nullability of frameStats in stats object

* revert track & encodedTrack in track events being nullable

* update stats endpoint & make filters not nullable

* document error responses and reformat

* update stacktrace notice

* document data types more specific

* document intercepting existing REST endpoints

* update plugin docs with new RestInterceptor interface

* update common pitfalls and update player docs

* remove optional mofiier from ready op resumed field

* update ws connection header docs

* Apply suggestions from code review

Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>

* add RestInterceptor import to PLUGINS.md

* revert more specific types

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>
Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>
Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>
Co-authored-by: Oliver Wilkes <oliverwilkes2006@icloud.com>

* Update LavalinkServer/src/main/java/lavalink/server/info/AppInfo.kt

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>

* cleanup

* fix error from suggestion

* Update LavalinkServer/src/main/java/lavalink/server/io/WebSocketHandler.kt

Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>

* chore: update kotlin version (lavalink-devs#761)

* chore: update kotlin version to latest

* chore: improve handling of websocket extensions

* docs(plugins): add note about kotlin version

Co-authored-by: ToπSenpai <15636011+TopiSenpai@users.noreply.github.com>

* add filters to info endpoint & some rewording

* return enabled filters in info endpoint

* validate & return error for disabled filters

* add missing deprecation annotations

* Apply suggestions from code review

Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>

* get rid of unnecessary mapping annotation values

* cleanup load result in protocol & LavalinkServer module

* rename LoadResult.searchResult to LoadResult.searchResultLoaded

* update protocol package to include lavalink api version

* fix broken imports due to package renaming

* protocol Track.uri should be nullable

* fix plugins & info endpoint

* ignore voice in update player endpoint if it didn't change

* don't ignore voice when voice gateway is closed

* rename oldConnection to oldConn

* separate filter configs from protocol module for easier api versioning

* throw error if decoded track is not able to be decoded

* reformat protocol decodeTrack function

* return 400 for player update with identifier resulting in playlist/search result or no matches

* update update player identifier docs

* add missing session id getter to plugin api

* Update protocol/src/main/java/dev/arbjerg/lavalink/protocol/v3/stats.kt

Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>

* apply review suggestions

* fix framestats type

Co-authored-by: melike2d <real.melike2d@gmail.com>
Co-authored-by: Devoxin <15076404+Devoxin@users.noreply.github.com>
Co-authored-by: Freya Arbjerg <freya@arbjerg.dev>
Co-authored-by: aikaterna <20862007+aikaterna@users.noreply.github.com>
Co-authored-by: Oliver Wilkes <oliverwilkes2006@icloud.com>

.gitignore

@@ -15,6 +15,8 @@ build/*
 /Testbot/out/
 /plugin-api/build/
 /plugin-api/out/
+/protocol/build/
+/protocol/out/
 gradle.properties
 application.yml
 LavalinkServer/plugins

LavalinkServer/application.yml.example

@@ -69,6 +69,15 @@ logging:
     root: INFO
     lavalink: INFO
 
+  request:
+    enabled: true
+    includeClientInfo: true
+    includeHeaders: false
+    includeQueryString: true
+    includePayload: true
+    maxPayloadLength: 10000
+
+
   logback:
     rollingpolicy:
       max-file-size: 1GB

LavalinkServer/build.gradle.kts

@@ -34,7 +34,10 @@ configurations {
 }
 
 dependencies {
-    implementation(projects.pluginApi)
+    implementation(projects.protocol)
+    implementation(projects.pluginApi) {
+        exclude(group = "org.springframework.boot", module = "spring-boot-starter-tomcat")
+    }
 
     implementation(libs.bundles.metrics)
     implementation(libs.bundles.spring) {
@@ -59,7 +62,6 @@ dependencies {
     implementation(libs.sentry.logback)
     implementation(libs.oshi)
     implementation(libs.json)
-    implementation(libs.gson)
 
     compileOnly(libs.spotbugs)
 

LavalinkServer/src/main/java/lavalink/server/Launcher.kt

@@ -37,16 +37,16 @@ import org.springframework.boot.context.event.ApplicationFailedEvent
 import org.springframework.boot.context.event.ApplicationReadyEvent
 import org.springframework.context.ApplicationListener
 import org.springframework.context.ConfigurableApplicationContext
-import org.springframework.context.annotation.ComponentScan
-import org.springframework.context.annotation.FilterType
+import org.springframework.context.annotation.*
 import org.springframework.core.io.DefaultResourceLoader
 import java.io.File
 import java.time.Instant
 import java.time.ZoneId
 import java.time.format.DateTimeFormatter
 import java.util.*
 
-@Suppress("SpringBootApplicationSetup", "SpringComponentScan")
+
+@Suppress("SpringComponentScan")
 @SpringBootApplication
 @ComponentScan(
     value = ["\${componentScan}"],
@@ -68,29 +68,29 @@ object Launcher {
             .withZone(ZoneId.of("UTC"))
         val buildTime = dtf.format(Instant.ofEpochMilli(appInfo.buildTime))
         val commitTime = dtf.format(Instant.ofEpochMilli(gitRepoState.commitTime * 1000))
-
-        val version = appInfo.version.takeUnless { it.startsWith("@") } ?: "Unknown"
+        val version = appInfo.versionBuild.takeUnless { it.startsWith("@") }
+            ?: "Unknown"
 
         return buildString {
             if (vanity) {
-                appendln()
-                appendln()
-                appendln(getVanity())
+                appendLine()
+                appendLine()
+                appendLine(getVanity())
             }
             if (!gitRepoState.isLoaded) {
-                appendln()
-                appendln("$indentation*** Unable to find or load Git metadata ***")
+                appendLine()
+                appendLine("$indentation*** Unable to find or load Git metadata ***")
             }
-            appendln()
-            append("${indentation}Version:        "); appendln(version)
+            appendLine()
+            append("${indentation}Version:        "); appendLine(version)
             if (gitRepoState.isLoaded) {
-                append("${indentation}Build time:     "); appendln(buildTime)
-                append("${indentation}Branch          "); appendln(gitRepoState.branch)
-                append("${indentation}Commit:         "); appendln(gitRepoState.commitIdAbbrev)
-                append("${indentation}Commit time:    "); appendln(commitTime)
+                append("${indentation}Build time:     "); appendLine(buildTime)
+                append("${indentation}Branch          "); appendLine(gitRepoState.branch)
+                append("${indentation}Commit:         "); appendLine(gitRepoState.commitIdAbbrev)
+                append("${indentation}Commit time:    "); appendLine(commitTime)
             }
-            append("${indentation}JVM:            "); appendln(System.getProperty("java.version"))
-            append("${indentation}Lavaplayer      "); appendln(PlayerLibrary.VERSION)
+            append("${indentation}JVM:            "); appendLine(System.getProperty("java.version"))
+            append("${indentation}Lavaplayer      "); appendLine(PlayerLibrary.VERSION)
         }
     }
 
@@ -163,12 +163,18 @@ object Launcher {
             .resourceLoader(DefaultResourceLoader(pluginManager.classLoader))
             .listeners(
                 ApplicationListener { event: Any ->
-                    if (event is ApplicationEnvironmentPreparedEvent) {
-                        log.info(getVersionInfo())
-                    } else if (event is ApplicationReadyEvent) {
-                        log.info("Lavalink is ready to accept connections.")
-                    } else if (event is ApplicationFailedEvent) {
-                        log.error("Application failed", event.exception)
+                    when (event) {
+                        is ApplicationEnvironmentPreparedEvent -> {
+                            log.info(getVersionInfo())
+                        }
+
+                        is ApplicationReadyEvent -> {
+                            log.info("Lavalink is ready to accept connections.")
+                        }
+
+                        is ApplicationFailedEvent -> {
+                            log.error("Application failed", event.exception)
+                        }
                     }
                 }
             ).parent(parent)

LavalinkServer/src/main/java/lavalink/server/bootstrap/PluginManager.kt

@@ -55,7 +55,7 @@ class PluginManager(config: PluginsConfig) {
             Declaration(fragments[0], fragments[1], fragments[2], repository)
         }
 
-        declarations.forEach declarationLoop@ { declaration ->
+        declarations.forEach declarationLoop@{ declaration ->
             pluginJars.forEach { jar ->
                 if (declaration.name == jar.name) {
                     if (declaration.version == jar.version) {
@@ -102,7 +102,10 @@ class PluginManager(config: PluginsConfig) {
 
         if (jarsToLoad.isEmpty()) return emptyList()
 
-        val cl = URLClassLoader.newInstance(jarsToLoad.map { URL("jar:file:${it.absolutePath}!/") }.toTypedArray(), javaClass.classLoader)
+        val cl = URLClassLoader.newInstance(
+            jarsToLoad.map { URL("jar:file:${it.absolutePath}!/") }.toTypedArray(),
+            javaClass.classLoader
+        )
         classLoader = cl
 
         val manifests = mutableListOf<PluginManifest>()

LavalinkServer/src/main/java/lavalink/server/config/AudioPlayerConfiguration.kt

@@ -3,7 +3,6 @@ package lavalink.server.config
 import com.sedmelluq.discord.lavaplayer.container.MediaContainerProbe
 import com.sedmelluq.discord.lavaplayer.container.MediaContainerRegistry
 import com.sedmelluq.discord.lavaplayer.player.AudioConfiguration
-import com.sedmelluq.discord.lavaplayer.player.AudioConfiguration.ResamplingQuality
 import com.sedmelluq.discord.lavaplayer.player.AudioPlayerManager
 import com.sedmelluq.discord.lavaplayer.player.DefaultAudioPlayerManager
 import com.sedmelluq.discord.lavaplayer.source.AudioSourceManager
@@ -15,7 +14,6 @@ import com.sedmelluq.discord.lavaplayer.source.soundcloud.*
 import com.sedmelluq.discord.lavaplayer.source.twitch.TwitchStreamAudioSourceManager
 import com.sedmelluq.discord.lavaplayer.source.vimeo.VimeoAudioSourceManager
 import com.sedmelluq.discord.lavaplayer.source.youtube.YoutubeAudioSourceManager
-import com.sedmelluq.discord.lavaplayer.source.youtube.YoutubeHttpContextFilter
 import com.sedmelluq.lava.extensions.youtuberotator.YoutubeIpRotatorSetup
 import com.sedmelluq.lava.extensions.youtuberotator.planner.*
 import com.sedmelluq.lava.extensions.youtuberotator.tools.ip.Ipv4Block
@@ -30,6 +28,7 @@ import org.slf4j.LoggerFactory
 import org.springframework.context.annotation.Bean
 import org.springframework.context.annotation.Configuration
 import java.net.InetAddress
+import java.util.*
 import java.util.concurrent.TimeUnit
 import java.util.function.Predicate
 
@@ -59,45 +58,45 @@ class AudioPlayerConfiguration {
         val defaultFrameBufferDuration = audioPlayerManager.frameBufferDuration
         serverConfig.frameBufferDurationMs?.let {
             if (it < 200) { // At the time of writing, LP enforces a minimum of 200ms.
-                log.warn("Buffer size of {}ms is illegal. Defaulting to {}", it, defaultFrameBufferDuration)
+                log.warn("Buffer size of ${it}ms is illegal. Defaulting to ${defaultFrameBufferDuration}ms")
             }
 
             val bufferDuration = it.takeIf { it >= 200 } ?: defaultFrameBufferDuration
-            log.debug("Setting frame buffer duration to {}", bufferDuration)
+            log.debug("Setting frame buffer duration to ${bufferDuration}ms")
             audioPlayerManager.frameBufferDuration = bufferDuration
         }
 
         val defaultOpusEncodingQuality = AudioConfiguration.OPUS_QUALITY_MAX
         audioPlayerManager.configuration.let {
             serverConfig.opusEncodingQuality?.let { opusQuality ->
                 if (opusQuality !in 0..10) {
-                    log.warn("Opus encoding quality {} is not within the range of 0 to 10. Defaulting to {}", opusQuality, defaultOpusEncodingQuality)
+                    log.warn("Opus encoding quality $opusQuality is not within the range of 0 to 10. Defaulting to $defaultOpusEncodingQuality")
                 }
 
                 val qualitySetting = opusQuality.takeIf { it in 0..10 } ?: defaultOpusEncodingQuality
-                log.debug("Setting opusEncodingQuality to {}", qualitySetting)
+                log.debug("Setting opusEncodingQuality to $qualitySetting")
                 it.opusEncodingQuality = qualitySetting
             }
 
             serverConfig.resamplingQuality?.let { resamplingQuality ->
-                log.debug("Setting resamplingQuality to {}", resamplingQuality)
+                log.debug("Setting resamplingQuality to $resamplingQuality")
                 it.resamplingQuality = resamplingQuality
             }
         }
 
         val defaultTrackStuckThresholdMs = TimeUnit.NANOSECONDS.toMillis(audioPlayerManager.trackStuckThresholdNanos)
         serverConfig.trackStuckThresholdMs?.let {
             if (it < 100) {
-                log.warn("Track Stuck Threshold of {}ms is too small. Defaulting to {}ms", it, defaultTrackStuckThresholdMs)
+                log.warn("Track Stuck Threshold of ${it}ms is too small. Defaulting to ${defaultTrackStuckThresholdMs}ms")
             }
 
             val trackStuckThresholdMs: Long = it.takeIf { it >= 100 } ?: defaultTrackStuckThresholdMs
-            log.debug("Setting Track Stuck Threshold to {}ms", trackStuckThresholdMs)
+            log.debug("Setting Track Stuck Threshold to ${trackStuckThresholdMs}ms")
             audioPlayerManager.setTrackStuckThreshold(trackStuckThresholdMs)
         }
 
         serverConfig.useSeekGhosting?.let { seekGhosting ->
-            log.debug("Setting useSeekGhosting to {}", seekGhosting)
+            log.debug("Setting useSeekGhosting to $seekGhosting")
             audioPlayerManager.setUseSeekGhosting(seekGhosting)
         }
 
@@ -129,6 +128,7 @@ class AudioPlayerConfiguration {
                     retryLimit < 0 -> YoutubeIpRotatorSetup(routePlanner).forSource(youtube).setup()
                     retryLimit == 0 -> YoutubeIpRotatorSetup(routePlanner).forSource(youtube)
                         .withRetryLimit(Int.MAX_VALUE).setup()
+
                     else -> YoutubeIpRotatorSetup(routePlanner).forSource(youtube).withRetryLimit(retryLimit).setup()
 
                 }
@@ -161,7 +161,7 @@ class AudioPlayerConfiguration {
 
         audioSourceManagers.forEach {
             audioPlayerManager.registerSourceManager(it)
-            log.info("Registered {} provided from a plugin", it)
+            log.info("Registered $it provided from a plugin")
         }
 
         audioPlayerManager.configuration.isFilterHotSwapEnabled = true
@@ -224,7 +224,7 @@ class AudioPlayerConfiguration {
             }
         }
 
-        return when (rateLimitConfig.strategy.toLowerCase().trim()) {
+        return when (rateLimitConfig.strategy.lowercase(Locale.getDefault()).trim()) {
             "rotateonban" -> RotatingIpRoutePlanner(ipBlocks, filter, rateLimitConfig.searchTriggersFail)
             "loadbalance" -> BalancingIpRoutePlanner(ipBlocks, filter, rateLimitConfig.searchTriggersFail)
             "nanoswitch" -> NanoIpRoutePlanner(ipBlocks, rateLimitConfig.searchTriggersFail)

LavalinkServer/src/main/java/lavalink/server/config/AudioSourcesConfig.kt

@@ -0,0 +1,20 @@
+package lavalink.server.config
+
+import org.springframework.boot.context.properties.ConfigurationProperties
+import org.springframework.stereotype.Component
+
+/**
+ * Created by napster on 05.03.18.
+ */
+@ConfigurationProperties(prefix = "lavalink.server.sources")
+@Component
+data class AudioSourcesConfig(
+    var isYoutube: Boolean = true,
+    var isBandcamp: Boolean = true,
+    var isSoundcloud: Boolean = true,
+    var isTwitch: Boolean = true,
+    var isVimeo: Boolean = true,
+    var isMixer: Boolean = true,
+    var isHttp: Boolean = true,
+    var isLocal: Boolean = false,
+)

LavalinkServer/src/main/java/lavalink/server/config/HttpConfig.kt

@@ -1,8 +1,8 @@
 package lavalink.server.config
 
 data class HttpConfig(
-        var proxyHost: String = "",
-        var proxyPort: Int = 3128,
-        var proxyUser: String = "",
-        var proxyPassword: String = ""
+    var proxyHost: String = "",
+    var proxyPort: Int = 3128,
+    var proxyUser: String = "",
+    var proxyPassword: String = ""
 )