Skip to content

Commit

Permalink
Merge remote-tracking branch 'grafana/master' into anno-panel
Browse files Browse the repository at this point in the history
* grafana/master: (73 commits)
  docs: configuring custom headers in the dataproxy (grafana#17367)
  Explore: Queries the datasource once per run query and uses DataStreamObserver (grafana#17263)
  Feature: Adds redux action logging toggle from url params (grafana#17368)
  Build: Adds e2e tests back to master workflow with better error messages and with artifacts (grafana#17374)
  Explore: Handle datasources with long names better in ds picker (grafana#17393)
  Annotations: Improve annotation option tooltips (grafana#17384)
  InfluxDB: Fixes single quotes are not escaped (grafana#17398)
  Chore: Bump axios to 0.19.0 (grafana#17403)
  Alerting: golint fixes for alerting (grafana#17246)
  Batch disable users (grafana#17254)
  Chore: Remove unused properties in explore (grafana#17359)
  MySQL/Postgres/MSSQL: Add parsing for day, weeks and year intervals in macros (grafana#13086)
  Security: Prevent csv formula injection attack  (grafana#17363)
  LDAP: remove unused function (grafana#17351)
  Enterprise: remove gofakeit dep (grafana#17344)
  Explore: Update time range before running queries (grafana#17349)
  Build(package.json): improve npm commands (grafana#17022)
  Chore: upgrade webpack analyser (grafana#17340)
  NewDataSourcePage: Add Grafana Cloud link (grafana#17324)
  CloudWatch: Avoid exception while accessing results (grafana#17283)
  ...
  • Loading branch information
ryantxu committed Jun 3, 2019
2 parents 9e3fb06 + fbf37eb commit d8c791d
Show file tree
Hide file tree
Showing 302 changed files with 4,752 additions and 4,827 deletions.
17 changes: 13 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ jobs:
end-to-end-test:
docker:
- image: circleci/node:8-browsers
- image: grafana/grafana:master
- image: grafana/grafana-dev:master-$CIRCLE_SHA1
steps:
- run: dockerize -wait tcp://127.0.0.1:3000 -timeout 120s
- checkout
Expand All @@ -91,6 +91,12 @@ jobs:
name: run end-to-end tests
command: 'env BASE_URL=http://127.0.0.1:3000 yarn e2e-tests'
no_output_timeout: 5m
- store_artifacts:
path: public/e2e-test/screenShots/theTruth
destination: expected-screenshots
- store_artifacts:
path: public/e2e-test/screenShots/theOutput
destination: output-screenshots

codespell:
docker:
Expand Down Expand Up @@ -629,7 +635,7 @@ workflows:
- mysql-integration-test
- postgres-integration-test
- build-oss-msi
filters: *filter-only-master
filters: *filter-only-master
- grafana-docker-master:
requires:
- build-all
Expand Down Expand Up @@ -662,7 +668,10 @@ workflows:
- mysql-integration-test
- postgres-integration-test
filters: *filter-only-master

- end-to-end-test:
requires:
- grafana-docker-master
filters: *filter-only-master
release:
jobs:
- build-all:
Expand Down Expand Up @@ -723,7 +732,7 @@ workflows:
- backend-lint
- mysql-integration-test
- postgres-integration-test
filters: *filter-only-master
filters: *filter-only-release

build-branches-and-prs:
jobs:
Expand Down
32 changes: 31 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,33 @@
# 6.2.0 (unreleased)
# 6.3.0 (unreleased)

# 6.2.1 (2019-05-27)

### Features / Enhancements
* **CLI**: Add command to migrate all datasources to use encrypted password fields . [#17118](https://github.com/grafana/grafana/pull/17118), [@aocenas](https://github.com/aocenas)
* **Gauge/BarGauge**: Improvements to auto value font size . [#17292](https://github.com/grafana/grafana/pull/17292), [@torkelo](https://github.com/torkelo)

### Bug Fixes
* **Auth Proxy**: Resolve database is locked errors. [#17274](https://github.com/grafana/grafana/pull/17274), [@marefr](https://github.com/marefr)
* **Database**: Retry transaction if sqlite returns database is locked error. [#17276](https://github.com/grafana/grafana/pull/17276), [@marefr](https://github.com/marefr)
* **Explore**: Fixes so clicking in a Prometheus Table the query is filtered by clicked value. [#17083](https://github.com/grafana/grafana/pull/17083), [@hugohaggmark](https://github.com/hugohaggmark)
* **Singlestat**: Fixes issue with value placement and line wraps. [#17249](https://github.com/grafana/grafana/pull/17249), [@torkelo](https://github.com/torkelo)
* **Tech**: Update jQuery to 3.4.1 to fix issue on iOS 10 based browers as well as Chrome 53.x . [#17290](https://github.com/grafana/grafana/pull/17290), [@timbutler](https://github.com/timbutler)

# 6.2.0 (2019-05-22)

### Bug Fixes
* **BarGauge**: Fix for negative min values. [#17192](https://github.com/grafana/grafana/pull/17192), [@torkelo](https://github.com/torkelo)
* **Gauge/BarGauge**: Fix for issues editing min & max options. [#17174](https://github.com/grafana/grafana/pull/17174)
* **Search**: Make only folder name only open search with current folder filter. [#17226](https://github.com/grafana/grafana/pull/17226)
* **AzureMonitor**: Revert to clearing chained dropdowns. [#17212](https://github.com/grafana/grafana/pull/17212)

### Breaking Changes
* **Plugins**: Data source plugins that process hidden queries need to add a "hiddenQueries: true" attribute in plugin.json. [#17124](https://github.com/grafana/grafana/pull/17124), [@ryantxu](https://github.com/ryantxu)

### Removal of old deprecated package repository

5 months ago we deprecated our old package cloud repository and [replaced it](https://grafana.com/blog/2019/01/05/moving-to-packages.grafana.com/) with our own. We will remove the old depreciated
repo on July 1st. Make sure you have switched to the new repo by then. The new repository has all our old releases so you are not required to upgrade just to switch package repository.

# 6.2.0-beta2 (2019-05-15)

Expand Down Expand Up @@ -56,6 +85,7 @@
* **Provisioning**: Add API endpoint to reload provisioning configs. [#16579](https://github.com/grafana/grafana/pull/16579), [@aocenas](https://github.com/aocenas)
* **Provisioning**: Do not allow deletion of provisioned dashboards. [#16211](https://github.com/grafana/grafana/pull/16211), [@aocenas](https://github.com/aocenas)
* **Provisioning**: Interpolate env vars in provisioning files. [#16499](https://github.com/grafana/grafana/pull/16499), [@aocenas](https://github.com/aocenas)
* **Provisioning**: Support FolderUid in Dashboard Provisioning Config. [#16559](https://github.com/grafana/grafana/pull/16559), [@swtch1](https://github.com/swtch1)
* **Security**: Add new setting allow_embedding. [#16853](https://github.com/grafana/grafana/pull/16853), [@marefr](https://github.com/marefr)
* **Security**: Store datasource passwords encrypted in secureJsonData. [#16175](https://github.com/grafana/grafana/pull/16175), [@aocenas](https://github.com/aocenas)
* **UX**: Improve Grafana usage for smaller screens. [#16783](https://github.com/grafana/grafana/pull/16783), [@torkelo](https://github.com/torkelo)
Expand Down
30 changes: 29 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
-include local/Makefile

.PHONY: all deps-go deps-js deps build-go build-server build-cli build-js build build-docker-dev build-docker-full lint-go test-go test-js test run clean gosec revive
.PHONY: all deps-go deps-js deps build-go build-server build-cli build-js build build-docker-dev build-docker-full lint-go test-go test-js test run clean gosec revive devenv devenv-down revive-alerting

GO := GO111MODULE=on go
GO_FILES := ./pkg/...
Expand Down Expand Up @@ -84,6 +84,34 @@ revive: scripts/go/bin/revive
-config ./scripts/go/configs/revive.toml \
$(GO_FILES)

revive-alerting: scripts/go/bin/revive
@scripts/go/bin/revive \
-formatter stylish \
./pkg/services/alerting/...

# create docker-compose file with provided sources and start them
# example: make devenv sources=postgres,openldap
ifeq ($(sources),)
devenv:
@printf 'You have to define sources for this command \nexample: make devenv sources=postgres,openldap\n'
else
devenv: devenv-down
$(eval targets := $(shell echo '$(sources)' | tr "," " "))

@cd devenv; \
./create_docker_compose.sh $(targets) || \
(rm -rf docker-compose.yaml; exit 1)

@cd devenv; \
docker-compose up -d
endif

# drop down the envs
devenv-down:
@cd devenv; \
test -f docker-compose.yaml && \
docker-compose down || exit 0;

# TODO recheck the rules and leave only necessary exclusions
gosec: scripts/go/bin/gosec
@scripts/go/bin/gosec -quiet \
Expand Down
3 changes: 3 additions & 0 deletions conf/defaults.ini
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ enforce_domain = false
# The full public facing url
root_url = %(protocol)s://%(domain)s:%(http_port)s/

# Serve Grafana from subpath specified in `root_url` setting. By default it is set to `false` for compatibility reasons.
serve_from_sub_path = false

# Log web requests
router_logging = false

Expand Down
3 changes: 3 additions & 0 deletions conf/sample.ini
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,9 @@
# If you use reverse proxy and sub path specify full url (with sub path)
;root_url = http://localhost:3000

# Serve Grafana from subpath specified in `root_url` setting. By default it is set to `false` for compatibility reasons.
;serve_from_sub_path = false

# Log web requests
;router_logging = false

Expand Down
14 changes: 13 additions & 1 deletion devenv/datasources.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@

apiVersion: 1

datasources:
- name: gdev-graphite
type: graphite
access: proxy
url: http://localhost:8080
url: http://localhost:8180
jsonData:
graphiteVersion: "1.1"

Expand Down Expand Up @@ -142,6 +143,17 @@ datasources:
timeField: "@timestamp"
esVersion: 70

- name: gdev-elasticsearch-v7-metricbeat
type: elasticsearch
access: proxy
database: "[metricbeat-]YYYY.MM.DD"
url: http://localhost:12200
jsonData:
interval: Daily
timeField: "@timestamp"
esVersion: 70
timeInterval: "10s"

- name: gdev-mysql
type: mysql
url: localhost:3306
Expand Down
16 changes: 16 additions & 0 deletions devenv/docker/blocks/elastic7/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,19 @@
- ./docker/blocks/elastic7/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
- /var/log:/var/log:ro
- ../data/log:/var/log/grafana:ro

metricbeat7:
image: docker.elastic.co/beats/metricbeat-oss:7.0.0
network_mode: host
command: metricbeat -e -strict.perms=false
user: root
volumes:
- ./docker/blocks/elastic7/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro
- /var/run/docker.sock:/var/run/docker.sock:ro

kibana7:
image: docker.elastic.co/kibana/kibana-oss:7.0.0
ports:
- "5601:5601"
environment:
ELASTICSEARCH_HOSTS: http://elasticsearch7:9200
38 changes: 38 additions & 0 deletions devenv/docker/blocks/elastic7/metricbeat.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
metricbeat.config:
modules:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false

metricbeat.autodiscover:
providers:
- type: docker
hints.enabled: true

metricbeat.modules:
- module: docker
metricsets:
- "container"
- "cpu"
- "diskio"
- "healthcheck"
- "info"
#- "image"
- "memory"
- "network"
hosts: ["unix:///var/run/docker.sock"]
period: 10s
enabled: true

processors:
- add_cloud_metadata: ~

output.elasticsearch:
hosts: ["localhost:12200"]
index: "metricbeat-%{+yyyy.MM.dd}"

setup.template.name: "metricbeat"
setup.template.pattern: "metricbeat-*"
setup.template.settings:
index.number_of_shards: 1
index.number_of_replicas: 1
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ EXPOSE 389

VOLUME ["/etc/ldap", "/var/lib/ldap"]

COPY modules/ /etc/ldap.dist/modules
COPY prepopulate/ /etc/ldap.dist/prepopulate
COPY admins-ldap-server/modules/ /etc/ldap.dist/modules
COPY admins-ldap-server/prepopulate/ /etc/ldap.dist/prepopulate

COPY ../entrypoint.sh /entrypoint.sh
COPY ../prepopulate.sh /prepopulate.sh
COPY ./entrypoint.sh /entrypoint.sh
COPY ./prepopulate.sh /prepopulate.sh

ENTRYPOINT ["/entrypoint.sh"]

Expand Down
8 changes: 6 additions & 2 deletions devenv/docker/blocks/multiple-openldap/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
admins-openldap:
build: docker/blocks/multiple-openldap/admins-ldap-server
build:
context: docker/blocks/multiple-openldap
dockerfile: ./admins-ldap-server.Dockerfile
environment:
SLAPD_PASSWORD: grafana
SLAPD_DOMAIN: grafana.org
Expand All @@ -8,7 +10,9 @@
- "389:389"

openldap:
build: docker/blocks/multiple-openldap/ldap-server
build:
context: docker/blocks/multiple-openldap
dockerfile: ./ldap-server.Dockerfile
environment:
SLAPD_PASSWORD: grafana
SLAPD_DOMAIN: grafana.org
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ EXPOSE 389

VOLUME ["/etc/ldap", "/var/lib/ldap"]

COPY modules/ /etc/ldap.dist/modules
COPY prepopulate/ /etc/ldap.dist/prepopulate
COPY ldap-server/modules/ /etc/ldap.dist/modules
COPY ldap-server/prepopulate/ /etc/ldap.dist/prepopulate

COPY ../entrypoint.sh /entrypoint.sh
COPY ../prepopulate.sh /prepopulate.sh
COPY ./entrypoint.sh /entrypoint.sh
COPY ./prepopulate.sh /prepopulate.sh

ENTRYPOINT ["/entrypoint.sh"]

Expand Down
10 changes: 10 additions & 0 deletions devenv/docker/loadtest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ Runs load tests and checks using [k6](https://k6.io/).

Docker

To run the auth proxy test you'll need to setup nginx proxy from docker block and
enable auth proxy together with configuring Grafana for auth proxy.

## Run

Run load test for 15 minutes using 2 virtual users and targeting http://localhost:3000.
Expand All @@ -32,6 +35,13 @@ Run load test for 10 virtual users:
$ ./run.sh -v 10
```

Run auth proxy test:

```bash
$ ./run.sh -c auth_proxy_test
```


Example output:

```bash
Expand Down
56 changes: 56 additions & 0 deletions devenv/docker/loadtest/auth_proxy_test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import { sleep, check, group } from 'k6';
import { createBasicAuthClient } from './modules/client.js';

export let options = {
noCookiesReset: true
};

let endpoint = __ENV.URL || 'http://localhost:10080/grafana';
const client = createBasicAuthClient(endpoint, 'user1', 'grafana');
client.withOrgId(1);

export const setup = () => {
const adminClient = createBasicAuthClient(endpoint, 'admin', 'admin');
let res = adminClient.datasources.getByName('gdev-prometheus');
if (res.status !== 200) {
throw new Error('Expected 200 response status when creating datasource');
}

return {
datasourceId: res.json().id,
};
}

export default (data) => {
group("auth proxy test", () => {
group("batch proxy requests", () => {
const d = new Date();
const batchCount = 300;
const requests = [];
const query = encodeURI('topk(5, max(scrape_duration_seconds) by (job))');
const start = (d.getTime() / 1000) - 3600;
const end = (d.getTime() / 1000);
const step = 20;

requests.push({ method: 'GET', url: '/api/annotations?dashboardId=8&from=1558670300607&to=1558691900607' });

for (let n = 0; n < batchCount; n++) {
requests.push({
method: 'GET',
url: `/api/datasources/proxy/${data.datasourceId}/api/v1/query_range?query=${query}&start=${start}&end=${end}&step=${step}`,
});
}

let responses = client.batch(requests);
for (let n = 0; n < batchCount; n++) {
check(responses[n], {
'response status is 200': (r) => r.status === 200,
});
}
});
});

sleep(5)
}

export const teardown = (data) => {}
Loading

0 comments on commit d8c791d

Please sign in to comment.