Build clickhouse-backup from sources as non-privileged user

[rkzell]

Hi , I am new to GitHub.
We are using ClickHouse and want to implement a simple solution for backup and restore. I want to checkout clickhouse-backup, but I have issues setting it up.

• I want to install it as non-privileged user
• The working directory must not be /var/lib/clickhouse, but any other directory
  I have seen the instructions, but it is not clear to me, how to build from sources (and previously adapt the configuration).
  What are the prerequisites?
  The documented command (GO111MODULE=on go get github.com/AlexAkulov/clickhouse-backup/cmd/clickhouse-backup )is not working for me? What am I missing?
  BTW, it is not an option to use the docker image (docker is not installed).
  Sorry for the silly questions.

[Slach]

I want to install it as non-privileged user

no problem

he working directory must not be /var/lib/clickhouse, but any other directory

you need install clickhouse-backup to the same host where clickhouse-server already installed
and /var/lib/clickhouse came from clickhouse-server settings
look to SELECT * FROM system.disks and try to understand what settings use in your clickhouse-server

moreover, user which you will use to run clickhouse-backup need read and write access to /var/lib/clickhouse/

The documented command (GO111MODULE=on go get github.com/AlexAkulov/clickhouse-backup/cmd/clickhouse-backup )is not working for me? What am I missing?

Did you install golang?
Could you provide full error message?

[rkzell]

you need install clickhouse-backup to the same host where clickhouse-server already installed
and /var/lib/clickhouse came from clickhouse-server settings
look to SELECT * FROM system.disks and try to understand what settings use in your clickhouse-server

Indeed, clickhouse-server has been installed on the same server. I am fully aware of the path, which is custom-specific, let's say it is /local0/data/clickhouse. Where is clickhouse-backup getting the path from? I have not found an appropriate variable in the config. Which one should I use?

Did you install golang?
Could you provide full error message?

Thanks, golang seems to be a prerequisites (where is it mentioned in the documentation?) I'll try to install it. No need to send you an error message ('go' could not be found).

[Slach]

Thanks, golang seems to be a prerequisites (where is it mentioned in the documentation?) I'll try to install it. No need to send you an error message ('go' could not be found).

go get means golang is required for build from sources

Why you tried install from source?

Why you not download and install precompiled binaries as described first in Installation
just download and unpack tar.gz for your platform from https://github.com/AlexAkulov/clickhouse-backup/releases/

[rkzell]

Why you not download and install precompiled binaries as described first in Installation
just download and unpack tar.gz for your platform from https://github.com/AlexAkulov/clickhouse-backup/releases/

I did that now. You are right. I was only embarrassed because the extracted tarball was containing only two files.

Indeed, I few adaptions to the config.yaml and it worked out OK, backup as well as restore. And it was also detecting the correct working directory by just reading the config files from clickhouse server. All good.

Additional question: Currently, that password must be given in cleartext in the config file of clickhouse-backup, which is a no-go in a production environment. For the clickhouse-server, we only have an encrypted password in our users.xml by using "password_sha256_hex". Is this possible with clickhouse-backup too?

[Slach]

password_sha256 Not implemented now, it's not possible cause we need pass unencrypted password to client sdk

Usually, you can define backup user in /etc/clickhouse-server/users.d/backup.xml
and setup <yandex><users><backup><password/><network><ip>127.0.0.1/8</ip></backup></yandex>
and safe use backup user without password which will have ip connection restriction

[rkzell]

password_sha256 Not implemented now, it's not possible cause we need pass unencrypted password to client sdk

Sounds like this is not even planned? Correct?

Usually, you can define backup user in /etc/clickhouse-server/users.d/backup.xml
and setup 127.0.0.1/8
and safe use backup user without password which will have ip connection restriction

Alright, our clickhouse-server is using a dedicated users.xml (we are referring to this file in clickhouse-config.xml). Is clickhouse-backup picking up a "backup.xml" automatically or could we setup this up within the existing clickhouse config directory in the existing users.xml? Correct me , if I am wrong: The syntax in your example accepts login to clickhouse as user "backup" in the given network?

[Slach]

Sounds like this is not even planned? Correct?

no, its just not possible, we need pass plaintext password to client SDK
it's no way to pass hash, cause it will sequrity equivalent of plain password

[Slach]

. Where is clickhouse-backup getting the path from?

SELECT * FROM system.disks

I have not found an appropriate variable in the config. Which one should I use?

Did you read https://github.com/AlexAkulov/clickhouse-backup/blob/master/ReadMe.md#default-config?
You can define environment variable CLICKHOUSE_DISK_MAPPING=default:/local0/data/clickhouse

or define it in config

clickhouse:
  disk_mapping:
    default: /local0/data/clickhouse

use it only if you need different disk mapping

[rkzell]

Thank you so much, great support!
All open questions are answered!