You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -70,23 +70,25 @@ Sourcebot supports indexing and searching through public and private repositorie
70
70
cd sourcebot_workspace
71
71
```
72
72
73
-
2. Create a new config following the [configuration schema](./schemas/index.json) to specify which repositories Sourcebot should index. For example, to index [llama.cpp](https://github.com/ggerganov/llama.cpp):
73
+
2. Create a new config following the [configuration schema](./schemas/v2/index.json) to specify which repositories Sourcebot should index. For example, let's index llama.cpp:
(For more examples, see [example-config.json](./example-config.json). For additional usage information, see the [configuration schema](./schemas/index.json)).
90
+
>[!NOTE]
91
+
> Sourcebot can also index all repos owned by a organization, user, group, etc., instead of listing them individually. For examples, see the [configs](./configs) directory. For additional usage information, see the [configuration schema](./schemas/v2/index.json).
90
92
91
93
3. Run Sourcebot and point it to the new config you created with the `-e CONFIG_PATH` flag:
92
94
@@ -106,31 +108,8 @@ Sourcebot supports indexing and searching through public and private repositorie
106
108
</details>
107
109
<br>
108
110
109
-
You should see a `.sourcebot` folder in your current directory. This folder stores a cache of the repositories zoekt has indexed. The `HEAD` commit of a repository is re-indexed [every hour](https://github.com/sourcebot-dev/zoekt/blob/11b7713f1fb511073c502c41cea413d616f7761f/cmd/zoekt-indexserver/main.go#L86). Indexing private repos? See [Providing an access token](#providing-an-access-token).
110
-
111
-
>[!WARNING]
112
-
> Depending on the size of your repo(s), SourceBot could take a couple of minutes to finish indexing. SourceBot doesn't currently support displaying indexing progress in real-time, so please be patient while it finishes. You can track the progress manually by investigating the `.sourcebot` cache in your workspace.
113
-
114
-
<details>
115
-
<summary><img src="https://gitlab.com/favicon.ico" width="16" height="16" /> Using GitLab?</summary>
116
-
117
-
_tl;dr: A `GITLAB_TOKEN` is required to index GitLab repositories (both private & public). See [Providing an access token](#providing-an-access-token)._
118
-
119
-
Currently, the GitLab indexer is restricted to only indexing repositories that the associated `GITLAB_TOKEN` has access to. For example, if the token has access to `foo`, `bar`, and `baz` repositories, the following config will index all three:
See [Providing an access token](#providing-an-access-token).
133
-
</details>
111
+
You should see a `.sourcebot` folder in your current directory. This folder stores a cache of the repositories zoekt has indexed. The `HEAD` commit of a repository is re-indexed [every hour](./packages/backend/src/constants.ts). Indexing private repos? See [Providing an access token](#providing-an-access-token).
112
+
134
113
</br>
135
114
136
115
## Providing an access token
@@ -145,31 +124,92 @@ This will depend on the code hosting platform you're using:
145
124
</picture> GitHub
146
125
</summary>
147
126
148
-
In order to index private repositories, you'll need to generate a GitHub Personal Access Token (PAT) and pass it to Sourcebot. Create a new PAT [here](https://github.com/settings/tokens/new) and make sure you select the `repo` scope:
127
+
In order to index private repositories, you'll need to generate a GitHub Personal Access Token (PAT). Create a new PAT [here](https://github.com/settings/tokens/new) and make sure you select the `repo` scope:
149
128
150
129

151
130
152
-
You'll need to pass this PAT each time you run Sourcebot by setting the `GITHUB_TOKEN` environment variable:
131
+
Next, update your configuration with the `token` field:
> An access token is <b>required</b> to index GitLab repositories (both private & public) since the GitLab indexer needs the token to determine which repositories to index. See [example-config.json](./example-config.json) for example usage.
164
-
165
173
Generate a GitLab Personal Access Token (PAT) [here](https://gitlab.com/-/user_settings/personal_access_tokens) and make sure you select the `read_api` scope:
166
174
167
175

168
176
169
-
You'll need to pass this PAT each time you run Sourcebot by setting the `GITLAB_TOKEN` environment variable:
177
+
Next, update your configuration with the `token` field:
2. Set the `GITLAB_HOSTNAME` environment variable to your deployment's hostname. For example:
230
-
231
-
<pre>
232
-
docker run -e <b>GITLAB_HOSTNAME=gitlab.example.com</b> /* additional args */ ghcr.io/sourcebot-dev/sourcebot:latest
233
-
</pre>
234
-
235
-
</details>
236
-
237
-
</div>
221
+
If you're using a self-hosted GitLab or GitHub instance with a custom domain, you can specify the domain in your config file. See [configs/self-hosted.json](configs/self-hosted.json) for examples.
238
222
239
223
## Build from source
240
224
>[!NOTE]
@@ -266,49 +250,14 @@ If you're using a self-hosted GitLab or GitHub instance with a custom domain, th
266
250
267
251
5. Create a `config.json` file at the repository root. See [Configuring Sourcebot](#configuring-sourcebot) for more information.
268
252
269
-
6. (Optional) Depending on your `config.json`, you may need to pass an access token to Sourcebot:
First, generate a personal access token (PAT). See [Providing an access token](#providing-an-access-token).
282
-
283
-
Next, Create a text file named `.github-token` **in your home directory** and paste the token in it. The file should look like:
284
-
```sh
285
-
ghp_...
286
-
```
287
-
zoekt will [read this file](https://github.com/sourcebot-dev/zoekt/blob/6a5753692b46e669f851ab23211e756a3677185d/cmd/zoekt-mirror-github/main.go#L60) to authenticate with GitHub.
First, generate a personal access token (PAT). See [Providing an access token](#providing-an-access-token).
295
-
296
-
Next, Create a text file named `.gitlab-token` **in your home directory** and paste the token in it. The file should look like:
297
-
```sh
298
-
glpat-...
299
-
```
300
-
zoekt will [read this file](https://github.com/sourcebot-dev/zoekt/blob/11b7713f1fb511073c502c41cea413d616f7761f/cmd/zoekt-mirror-gitlab/main.go#L43) to authenticate with GitLab.
301
-
</details>
302
-
</div>
303
-
304
-
7. Start Sourcebot with the command:
253
+
6. Start Sourcebot with the command:
305
254
```sh
306
255
yarn dev
307
256
```
308
257
309
258
A `.sourcebot` directory will be created and zoekt will begin to index the repositories found given `config.json`.
0 commit comments