1.3.0 Release (#1563)

* updating changelog and versions for 1.3.0 release

Signed-off-by: Edward Welch <edward.welch@grafana.com>

* fixing incorrect quote

Signed-off-by: Edward Welch <edward.welch@grafana.com>

CHANGELOG.md

@@ -1,14 +1,161 @@
-## master / unreleased
+## 1.3.0 (2019-01-16)
 
-### Notable Changes
-* [1486](https://github.com/grafana/loki/pull/1486) **pracucci**: Deprecated `-distributor.limiter-reload-period` flag / distributor's `limiter_reload_period` config option.
-* [1492](https://github.com/grafana/loki/pull/1492) **sandlis**: replaced GRAFANA_* with LOKI_* in logcli env vars, set default server url for logcli to http://localhost:3100.
+### What's New?? ###
 
-### Features
+With 1.3.0 we are excited to announce several improvements focusing on performance! 
+
+First and most significant is the Query Frontend:
+
+* [1442](https://github.com/grafana/loki/pull/1442) **cyriltovena**: Loki Query Frontend
+
+The query frontend allows for sharding queries by time and dispatching them in parallel to multiple queriers, giving true horizontal scaling ability for queries.  Take a look at the [jsonnet changes](https://github.com/grafana/loki/pull/1442/files?file-filters%5B%5D=.libsonnet) to see how we are deploying this in our production setup.  Keep an eye out for a blog post with more information on how the frontend works and more information on this exciting new feature.
+
+In our quest to improve query performance, we discovered that gzip, while good for compression ratio, is not the best for speed.  So we introduced the ability to select from several different compression algorithms:
+
+* [1411](https://github.com/grafana/loki/pull/1411) **cyriltovena**: Adds configurable compression algorithms for chunks
+
+We are currently testing out LZ4 and snappy, LZ4 seemed like a good fit however we found that it didn't always compress the same data to the same output which was causing some troubles for another important improvement:
+
+* [1438](https://github.com/grafana/loki/pull/1438) **pstibrany**: pkg/ingester: added sync period flags
+
+Extending on the work done by @bboreham on Cortex, @pstibrany added a few new flags and code to synchronize chunks between ingesters, which reduces the number of chunks persisted to object stores and therefore also reduces the number of chunks loaded on queries and the amount of de-duplication work which needs to be done.  
+
+As mentioned above, LZ4 was in some cases compressing the same data with a different result which was interfering with this change, we are still investigating the cause of this issue (It may be in how we implemented something, or may be in the compression code itself).  For now we have switched to snappy which has seen a reduction in data written to the object store from almost 3x the source data (with a replication factor of 3) to about 1.5x, saving a lot of duplicated log storage!
+
+Another valuable change related to chunks:
+
+* [1406](https://github.com/grafana/loki/pull/1406) **slim-bean**: allow configuring a target chunk size in compressed bytes
+
+With this change you can set a `chunk_target_size` and Loki will attempt to fill a chunk to approx that size before flushing (previously a chunk size was a hard coded 10 blocks where the default block size is 262144 bytes).  Larger chunks are beneficial for a few reasons, mainly on reducing API calls to your object store when performing queries, but also in reducing overhead in a few places, especially when processing very high volume log streams.
+
+Another big improvement is the introduction of accurate rate limiting when running microservices:
+
+* [1486](https://github.com/grafana/loki/pull/1486) **pracucci**: Add ingestion rate global limit support
+
+Previously the rate limit was applied at each distributor, however with traffic split over many distributors the limit would need to be adjusted accordingly.  This meant that scaling up distributors required changing the limit.  Now this information is communicated between distributors such that the limit should be applied accurately regardless of the number of distributors.
+
+And last but not least on the notable changes list is a new feature for Promtail:
+
+* [1275](https://github.com/grafana/loki/pull/1275) **bastjan**: pkg/promtail: IETF Syslog (RFC5424) Support
+
+With this change Promtail can receive syslogs via TCP!  Thanks to @bastjan for all the hard work on this submission!
+
+### Important things to note:
+
+* [1519](https://github.com/grafana/loki/pull/1519) Changes a core behavior in Loki regarding logs with duplicate content AND duplicate timestamps, previously Loki would store logs with duplicate timestamps and content, moving forward logs with duplicate content AND timestamps will be silently ignored.  Mainly this change is to prevent duplicates that appear when a batch is retried (the first entry in the list would be inserted again, now it will be ignored).  Logs with the same timestamp and different content will still be accepted.
+* [1486](https://github.com/grafana/loki/pull/1486) Deprecated `-distributor.limiter-reload-period` flag / distributor's `limiter_reload_period` config option.
+
+### All Changes
+
+Once again we can't thank our community and contributors enough for the significant work that everyone is adding to Loki, the entire list of changes is long!!
+
+#### Loki
+* [1526](https://github.com/grafana/loki/pull/1526) **codesome**: Support <selector> <range> <filters> for aggregation
+* [1522](https://github.com/grafana/loki/pull/1522) **cyriltovena**: Adds support for the old query string regexp in the frontend.
+* [1519](https://github.com/grafana/loki/pull/1519) **rfratto**: pkg/chunkenc: ignore duplicate lines pushed to a stream
+* [1511](https://github.com/grafana/loki/pull/1511) **sandlis**: querier: fix panic in tailer when max tail duration exceeds
+* [1499](https://github.com/grafana/loki/pull/1499) **slim-bean**: Fix a panic in chunk prefetch
+* [1495](https://github.com/grafana/loki/pull/1495) **slim-bean**: Prefetch chunks while processing
+* [1496](https://github.com/grafana/loki/pull/1496) **cyriltovena**: Add duplicates info and remove timing informations.
+* [1490](https://github.com/grafana/loki/pull/1490) **owen-d**: Fix/deadlock frontend queue
+* [1489](https://github.com/grafana/loki/pull/1489) **owen-d**: unifies reverse iterators
+* [1488](https://github.com/grafana/loki/pull/1488) **cyriltovena**: Fixes response json encoding and add regression tests.
+* [1486](https://github.com/grafana/loki/pull/1486) **pracucci**: Add ingestion rate global limit support* [1493](https://github.com/grafana/loki/pull/1493) **pracucci**: Added max streams per user global limit
+* [1480](https://github.com/grafana/loki/pull/1480) **cyriltovena**: Close iterator properly and check nil before releasing buffers.
+* [1473](https://github.com/grafana/loki/pull/1473) **rfratto**: pkg/querier: don't query all ingesters
+* [1470](https://github.com/grafana/loki/pull/1470) **cyriltovena**: Validates limit parameter.
+* [1448](https://github.com/grafana/loki/pull/1448) **cyriltovena**: Improving storage benchmark
+* [1445](https://github.com/grafana/loki/pull/1445) **cyriltovena**: Add decompression tracing instrumentation.
+* [1442](https://github.com/grafana/loki/pull/1442) **cyriltovena**: Loki Query Frontend
+* [1438](https://github.com/grafana/loki/pull/1438) **pstibrany**: pkg/ingester: added sync period flags
+* [1433](https://github.com/grafana/loki/pull/1433) **zendern**: Using strict parsing for yaml configs
+* [1425](https://github.com/grafana/loki/pull/1425) **pstibrany**: pkg/ingester: Added possibility to disable transfers.
+* [1423](https://github.com/grafana/loki/pull/1423) **pstibrany**: pkg/chunkenc: Fix BenchmarkRead to focus on reading chunks, not converting bytes to string
+* [1421](https://github.com/grafana/loki/pull/1421) **pstibrany**: pkg/chunkenc: change default LZ4 buffer size to 64k.
+* [1420](https://github.com/grafana/loki/pull/1420) **cyriltovena**: Sets the chunk encoding correctly when creating chunk from bytes.
+* [1419](https://github.com/grafana/loki/pull/1419) **owen-d**: Enables Series API in loki
+* [1413](https://github.com/grafana/loki/pull/1413) **pstibrany**: RangeQuery benchmark optimizations
+* [1411](https://github.com/grafana/loki/pull/1411) **cyriltovena**: Adds configurable compression algorithms for chunks
+* [1409](https://github.com/grafana/loki/pull/1409) **slim-bean**: change the chunk size histogram to allow for bigger buckets
+* [1408](https://github.com/grafana/loki/pull/1408) **slim-bean**: forgot to register the new metric for counting blocks per chunk
+* [1406](https://github.com/grafana/loki/pull/1406) **slim-bean**: allow configuring a target chunk size in compressed bytes
+* [1405](https://github.com/grafana/loki/pull/1405) **pstibrany**: Convert string to bytes once only when doing string filtering.
+* [1396](https://github.com/grafana/loki/pull/1396) **pstibrany**: pkg/cfg: print help only when requested, and print it on stdout
+* [1383](https://github.com/grafana/loki/pull/1383) **beornf**: Read websocket close in tail handler
+* [1071](https://github.com/grafana/loki/pull/1071) **rfratto**: pkg/ingester: limit total number of errors a stream can return on push
+* [1545](https://github.com/grafana/loki/pull/1545) **joe-elliott**: Critical n => m conversions
+* [1541](https://github.com/grafana/loki/pull/1541) **owen-d**: legacy endpoint 400s metric queries
+
+#### Promtail
+* [1515](https://github.com/grafana/loki/pull/1515) **slim-bean**: Promtail: Improve position and size metrics
+* [1485](https://github.com/grafana/loki/pull/1485) **p37ruh4**: Fileglob parsing fixes
+* [1472](https://github.com/grafana/loki/pull/1472) **owen-d**: positions.ignore-corruptions
+* [1453](https://github.com/grafana/loki/pull/1453) **chancez**: pkg/promtail: Initialize counters to 0 when creating client
+* [1436](https://github.com/grafana/loki/pull/1436) **rfratto**: promtail: add support for passing through journal entries as JSON
+* [1426](https://github.com/grafana/loki/pull/1426) **wphan**: Support microsecond timestamp format
+* [1416](https://github.com/grafana/loki/pull/1416) **pstibrany**: pkg/promtail/client: missing URL in client returns error
+* [1275](https://github.com/grafana/loki/pull/1275) **bastjan**: pkg/promtail: IETF Syslog (RFC5424) Support
+
+#### Fluent Bit
+* [1455](https://github.com/grafana/loki/pull/1455) **JensErat**: fluent-bit-plugin: re-enable failing JSON marshaller tests; pass error instead of logging and ignoring
+* [1294](https://github.com/grafana/loki/pull/1294) **JensErat**: fluent-bit: multi-instance support
+* [1514](https://github.com/grafana/loki/pull/1514) **shane-axiom**: fluent-plugin-grafana-loki: Add `fluentd_thread` label when `flush_thread_count` > 1
+
+#### FluentD
+* [1500](https://github.com/grafana/loki/pull/1500) **cyriltovena**: Bump fluentd plugin to 1.2.6.
+* [1475](https://github.com/grafana/loki/pull/1475) **Horkyze**: fluentd-plugin: call gsub for strings only
+
+#### Docker Driver
+* [1414](https://github.com/grafana/loki/pull/1414) **cyriltovena**: Adds tenant-id for docker driver.
+
+#### Logcli
+* [1492](https://github.com/grafana/loki/pull/1492) **sandlis**: logcli: replaced GRAFANA_* with LOKI_* in logcli env vars, set default server url for logcli to localhost
+
+#### Helm
+* [1534](https://github.com/grafana/loki/pull/1534) **olivierboudet**: helm : fix fluent-bit parser configuration syntax
+* [1506](https://github.com/grafana/loki/pull/1506) **terjesannum**: helm: add podsecuritypolicy for fluent-bit
+* [1431](https://github.com/grafana/loki/pull/1431) **eugene100**: Helm: fix issue with config.clients
+* [1430](https://github.com/grafana/loki/pull/1430) **olivierboudet**: helm : allow to define custom parsers to use with fluentbit.io/parser annotation
+* [1418](https://github.com/grafana/loki/pull/1418) **evalsocket**: Helm chart url added in helm.md
+* [1336](https://github.com/grafana/loki/pull/1336) **terjesannum**: helm: support adding init containers to the loki pod
+* [1530](https://github.com/grafana/loki/pull/1530) **WeiBanjo**: Allow extra command line args for external labels like hostname
+
+#### Jsonnet
+* [1518](https://github.com/grafana/loki/pull/1518) **benjaminhuo**: Fix error 'Field does not exist: jaeger_mixin' in tk show
+* [1501](https://github.com/grafana/loki/pull/1501) **anarcher**: jsonnet: fix common/defaultPorts parameters
+* [1497](https://github.com/grafana/loki/pull/1497) **cyriltovena**: Update Loki mixin to include frontend QPS and latency.
+* [1478](https://github.com/grafana/loki/pull/1478) **cyriltovena**: Fixes the typo in the result cache config of the Loki ksonnet lib.
+* [1543](https://github.com/grafana/loki/pull/1543) **sh0rez**: fix(ksonnet): use apps/v1
+
+#### Docs
+* [1531](https://github.com/grafana/loki/pull/1531) **fitzoh**: Documentation: Add note on using Loki with Amazon ECS
+* [1521](https://github.com/grafana/loki/pull/1521) **rfratto**: docs: Document timestamp ordering rules
+* [1516](https://github.com/grafana/loki/pull/1516) **rfratto**: Link to release docs in README.md, not master docs
+* [1508](https://github.com/grafana/loki/pull/1508) **cyriltovena**: Fixes bad json in Loki API documentation.
+* [1505](https://github.com/grafana/loki/pull/1505) **sandlis**: doc: fix sample yaml in docs for installing promtail to k8s
+* [1481](https://github.com/grafana/loki/pull/1481) **terjesannum**: docs: fix broken promtail link
+* [1474](https://github.com/grafana/loki/pull/1474) **Eraac**: <doc>: information about max_look_back_period
+* [1471](https://github.com/grafana/loki/pull/1471) **cyriltovena**: Update README.md
+* [1466](https://github.com/grafana/loki/pull/1466) **Eraac**: <documentation>: Update IAM requirement
+* [1441](https://github.com/grafana/loki/pull/1441) **vtereso**: <Docs>: README spelling fix
+* [1437](https://github.com/grafana/loki/pull/1437) **daixiang0**: fix all misspell
+* [1432](https://github.com/grafana/loki/pull/1432) **joe-elliott**: Removed unsupported encodings from docs
+* [1399](https://github.com/grafana/loki/pull/1399) **vishesh92**: Docs: Add configuration docs for redis
+* [1394](https://github.com/grafana/loki/pull/1394) **chancez**: Documentation: Fix example AWS storage configuration
+* [1227](https://github.com/grafana/loki/pull/1227) **daixiang0**: Add docker install doc
+* [1560](https://github.com/grafana/loki/pull/1560) **robshep**: Promtail Docs: Update output.md
+* [1546](https://github.com/grafana/loki/pull/1546) **mattmendick**: Removing third-party link
+* [1539](https://github.com/grafana/loki/pull/1539) **j18e**: docs: fix syntax error in pipeline example
+
+#### Build
+* [1494](https://github.com/grafana/loki/pull/1494) **pracucci**: Fixed TOUCH_PROTOS in all DroneCI pipelines
+* [1479](https://github.com/grafana/loki/pull/1479) **owen-d**: TOUCH_PROTOS build arg for dockerfile
+* [1476](https://github.com/grafana/loki/pull/1476) **owen-d**: initiates docker daemon for circle windows builds
+* [1469](https://github.com/grafana/loki/pull/1469) **rfratto**: Makefile: re-enable journal scraping on ARM
+
+#### New Members!
+* [1415](https://github.com/grafana/loki/pull/1415) **cyriltovena**: Add Joe as member of the team.
 
-* [1493](https://github.com/grafana/loki/pull/1493) **pracucci**: pkg/ingester: added a per-cluster limit on the maximum number of series per-user, configured via the `max_global_streams_per_user` config option.
-* [FEATURE] promtail positions file corruptions can be ignored with the `positions.ignore-invalid-yaml` flag. In the case the positions yaml is corrupted an empty positions config will be used and should later overwrite the malformed yaml.
-* [1486](https://github.com/grafana/loki/pull/1486) **pracucci**: Added `global` ingestion rate limiter strategy support.
 
 # 1.2.0 (2019-12-09)
 

README.md

@@ -29,35 +29,35 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
 
 ## Getting started
 
-The [Installation docs](https://github.com/grafana/loki/tree/v1.2.0/docs/installation/README.md) have instructions on how
+The [Installation docs](https://github.com/grafana/loki/tree/v1.3.0/docs/installation/README.md) have instructions on how
 to install Loki via Docker images, Helm charts, Jsonnet, or from source.
 
 You may also be interested in [installing
-Promtail](https://github.com/grafana/loki/tree/v1.2.0/docs/clients/promtail/installation.md) to send logs to Loki.
+Promtail](https://github.com/grafana/loki/tree/v1.3.0/docs/clients/promtail/installation.md) to send logs to Loki.
 
 Once you have Promtail, Loki, and Grafana running, continue with our [Getting
-Started Guide](https://github.com/grafana/loki/tree/v1.2.0/docs/getting-started/README.md) to get up and running with
+Started Guide](https://github.com/grafana/loki/tree/v1.3.0/docs/getting-started/README.md) to get up and running with
 querying logs.
 
 ### Documentation
 
 Documentation for the latest release can be found
-[here](https://github.com/grafana/loki/tree/v1.2.0/docs/README.md). Documentation for
+[here](https://github.com/grafana/loki/tree/v1.3.0/docs/README.md). Documentation for
 the unreleased master branch can be found inside the
 [`docs/` folder](./docs/README.md).
 
 
 Some key documents to read:
 
-- [API documentation](https://github.com/grafana/loki/tree/v1.2.0/docs/api.md) for alternative ways of getting logs into Loki.
-- [Operations](https://github.com/grafana/loki/tree/v1.2.0/docs/operations) for important aspects of running Loki.
-- [Promtail](https://github.com/grafana/loki/tree/v1.2.0/docs/clients/promtail) is an agent which can tail your log files and push them to Loki.
-- [Pipelines](https://github.com/grafana/loki/tree/v1.2.0/docs/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
-- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.2.0/docs/clients/docker-driver) is a docker plugin to send logs directly to Loki from Docker containers.
-- [LogCLI](https://github.com/grafana/loki/tree/v1.2.0/docs/getting-started/logcli.md) on how to query your logs without Grafana.
-- [Loki Canary](https://github.com/grafana/loki/tree/v1.2.0/docs/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
-- [Troubleshooting](https://github.com/grafana/loki/tree/v1.2.0/docs/getting-started/troubleshooting.md) for help around frequent error messages.
-- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.2.0/docs/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
+- [API documentation](https://github.com/grafana/loki/tree/v1.3.0/docs/api.md) for alternative ways of getting logs into Loki.
+- [Operations](https://github.com/grafana/loki/tree/v1.3.0/docs/operations) for important aspects of running Loki.
+- [Promtail](https://github.com/grafana/loki/tree/v1.3.0/docs/clients/promtail) is an agent which can tail your log files and push them to Loki.
+- [Pipelines](https://github.com/grafana/loki/tree/v1.3.0/docs/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
+- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.3.0/docs/clients/docker-driver) is a docker plugin to send logs directly to Loki from Docker containers.
+- [LogCLI](https://github.com/grafana/loki/tree/v1.3.0/docs/getting-started/logcli.md) on how to query your logs without Grafana.
+- [Loki Canary](https://github.com/grafana/loki/tree/v1.3.0/docs/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
+- [Troubleshooting](https://github.com/grafana/loki/tree/v1.3.0/docs/getting-started/troubleshooting.md) for help around frequent error messages.
+- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.3.0/docs/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
 
 ## Getting Help
 

docs/clients/promtail/installation.md

@@ -12,7 +12,7 @@ Every release includes binaries for Promtail which can be found on the
 
 ```bash
 # modify tag to most recent version
-$ docker pull grafana/promtail:v1.2.0
+$ docker pull grafana/promtail:v1.3.0
 ```
 
 ## Helm