Skip to content

BREAKING CHANGE: remove btool and replace it with splunk.rest #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions docs/release_7_0_0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Removed usage of btool command from solnlib

As of version 7.0.0, the `btool` command has been removed from solnlib. Configuration stanzas and keys should now be accessed via the REST API.
Additionally, the `splunkenv` module can only be used in environments where Splunk is installed, as it relies on Splunk-specific methods for making internal calls.

## Session key is now mandatory in some of the functions

The affected functions now require a valid `session_key` to operate correctly. While solnlib attempts to retrieve the `session_key` automatically,
there are scenarios—such as within a modular input script—where this is not possible. In such cases, you must explicitly provide the `session_key`
to ensure proper authorization. Affected functions are:

* `get_splunk_host_info`
* `get_splunkd_access_info`
* `get_scheme_from_hec_settings`
* `get_splunkd_uri`
* `get_conf_key_value`
* `get_conf_stanza`

## Changed arguments in `get_conf_key_value` and `get_conf_stanza`

As of version 7.0.0, the following changes have been made to the function:

`get_conf_key_value` now requires 4 mandatory arguments:

* `conf_name`
* `stanza`
* `key`
* `app_name` (new)

`get_conf_stanza` now requires 3 mandatory arguments:

* `conf_name`
* `stanza`
* `app_name` (new)

Both functions also accept the following optional arguments:

* `session_key` - Used for authentication. If not provided, a 401 Unauthorized error may occur depending on the context.
* `users` - Limits results returned by the configuration endpoint. Defaults to `nobody`.
* `raw_output` - If set to `True`, the full decoded JSON response is returned.
This should be enabled when `app_name` is set to the global context `(/-/)`, as the Splunk REST API may return multiple entries in that case.

## The `get_session_key` function has been removed from solnlib

This function relied on reading the `scheme`, `host` and `port` using the deprecated btool utility, which is no longer supported.

1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ plugins:
nav:
- Home: index.md
- Release 6.0.0: release_6_0_0.md
- Release 7.0.0: release_7_0_0.md
- References:
- modular_input:
- "checkpointer.py": modular_input/checkpointer.md
Expand Down
18 changes: 18 additions & 0 deletions solnlib/_settings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#
# Copyright 2025 Splunk Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""This module provide settings that can be used to disable/switch features."""

use_btool = False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we would require this setting to be passed from add-on developers, or provide a way to them to override this setting if they want to use btool.

Copy link
Contributor Author

@sgoral-splunk sgoral-splunk Jun 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea here was to make some kind of backdoor. It's not a feature that user can choose which apprach use, he will be forced to use api but in case of any issue on production env, admins will be able to temporary switch this flag.

Copy link
Contributor

@hetangmodi-crest hetangmodi-crest Jun 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test where we switch this flag to True and then ensure the test suite passes, thus ensuring the same behaviour when some developers make this change in prod or pre-prod env?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added some test for that e.g. tests.unit.test_splunkenv.test_splunkd_uri_backdoor

57 changes: 0 additions & 57 deletions solnlib/credentials.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,19 @@

"""This module contains Splunk credential related interfaces."""

import json
import re
import warnings
from typing import Dict, List

from splunklib import binding, client

from . import splunk_rest_client as rest_client
from .net_utils import validate_scheme_host_port
from .splunkenv import get_splunkd_access_info
from .utils import retry

__all__ = [
"CredentialException",
"CredentialNotExistException",
"CredentialManager",
"get_session_key",
]


Expand Down Expand Up @@ -339,56 +335,3 @@ def _get_all_passwords(self) -> List[Dict[str, str]]:
)
passwords = self._storage_passwords.list(count=-1)
return self._get_clear_passwords(passwords)


@retry(exceptions=[binding.HTTPError])
def get_session_key(
username: str,
password: str,
scheme: str = None,
host: str = None,
port: int = None,
**context: dict,
) -> str:
"""Get splunkd access token.

Arguments:
username: The Splunk account username, which is used to authenticate the Splunk instance.
password: The Splunk account password.
scheme: (optional) The access scheme, default is None.
host: (optional) The host name, default is None.
port: (optional) The port number, default is None.
context: Other configurations for Splunk rest client.

Returns:
Splunk session key.

Raises:
CredentialException: If username/password are invalid.
ValueError: if scheme, host or port are invalid.

Examples:
>>> get_session_key('user', 'password')
"""
validate_scheme_host_port(scheme, host, port)

if any([scheme is None, host is None, port is None]):
scheme, host, port = get_splunkd_access_info()

uri = "{scheme}://{host}:{port}/{endpoint}".format(
scheme=scheme, host=host, port=port, endpoint="services/auth/login"
)
_rest_client = rest_client.SplunkRestClient(
None, "-", "nobody", scheme, host, port, **context
)
try:
response = _rest_client.http.post(
uri, username=username, password=password, output_mode="json"
)
except binding.HTTPError as e:
if e.status != 401:
raise

raise CredentialException("Invalid username/password.")

return json.loads(response.body.read())["sessionKey"]
20 changes: 18 additions & 2 deletions solnlib/modular_input/event_writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import threading
import time
import traceback
import warnings
from abc import ABCMeta, abstractmethod
from random import randint
from typing import List, Union
Expand All @@ -39,6 +40,16 @@
__all__ = ["ClassicEventWriter", "HECEventWriter"]


deprecation_msg = (
"Function 'create_from_token' is deprecated and incompatible with 'global_settings_schema=True'. "
"Use 'create_from_token_with_session_key' instead."
)


class FunctionDeprecated(Exception):
pass


class EventWriter(metaclass=ABCMeta):
"""Base class of event writer."""

Expand Down Expand Up @@ -233,13 +244,13 @@ def __init__(
scheme, host, hec_port = utils.extract_http_scheme_host_port(hec_uri)
else:
if not all([scheme, host, port]):
scheme, host, port = get_splunkd_access_info()
scheme, host, port = get_splunkd_access_info(self._session_key)
hec_port, hec_token = self._get_hec_config(
hec_input_name, session_key, scheme, host, port, **context
)

if global_settings_schema:
scheme = get_scheme_from_hec_settings()
scheme = get_scheme_from_hec_settings(self._session_key)

if not context.get("pool_connections"):
context["pool_connections"] = 10
Expand Down Expand Up @@ -272,6 +283,11 @@ def create_from_token(
Created HECEventWriter.
"""

warnings.warn(deprecation_msg, DeprecationWarning, stacklevel=2)

if global_settings_schema:
raise FunctionDeprecated(deprecation_msg)

return HECEventWriter(
None,
None,
Expand Down
2 changes: 1 addition & 1 deletion solnlib/server_info.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ def __init__(
"""
is_localhost = False
if not all([scheme, host, port]) and os.environ.get("SPLUNK_HOME"):
scheme, host, port = get_splunkd_access_info()
scheme, host, port = get_splunkd_access_info(session_key)
is_localhost = (
host == "localhost" or host == "127.0.0.1" or host in ("::1", "[::1]")
)
Expand Down
2 changes: 1 addition & 1 deletion solnlib/splunk_rest_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ def __init__(
"""
# Only do splunkd URI discovery in SPLUNK env (SPLUNK_HOME is set).
if not all([scheme, host, port]) and os.environ.get("SPLUNK_HOME"):
scheme, host, port = get_splunkd_access_info()
scheme, host, port = get_splunkd_access_info(session_key)
if os.environ.get("SPLUNK_HOME") is None:
if not all([scheme, host, port]):
raise ValueError(
Expand Down
Loading
Loading