diff --git a/CHANGELOG.md b/CHANGELOG.md index 40b3143f5bc4..ebab3563ae6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,286 @@ -## Unreleased (Master) +## 1.6.0 (2020-08-13) + +It's the second thursday of the eighth month of the year which means it's time for another Loki Release!! + +Before we highlight important features and changes, congratulations to [@adityacs](https://github.com/adityacs), who is the newest member of the Loki team! +Aditya has been regularly contributing to the Loki project for the past year, with each contribution better than the last. +Many of the items on the following list were thanks to his hard work. Thank you, Aditya, and welcome to the team! + +I think we might have set a new record with 189 PR's in this release! + +### Important Notes + +**Please Note** There are several changes in this release which might require your attention! + +* The NET_BIND_SERVICE capability was removed from the Loki process in the docker image, it's no longer possible to run Loki with the supplied image on a port less than 1024 +* If you run microservices, there is an important rollout sequence to prevent query errors. +* Scrape configs have changed for Promtail in both Helm and Ksonnet affecting two labels: `instance` -> `pod` and `container_name` -> `container`. +* Almost all of the Loki Canary metrics were renamed. +* A few command line flags where changed (although they are likely not commonly used) +* If you use ksonnet and run on GCS and Bigtable you may see an error in your config as a default value was removed. +* If you are using boltdb-shipper, you will likekly need to add a new schema_config entry. + +Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/operations/upgrade.md#160) for detailed information on all these changes. + +### Notable Features and Fixes + +#### Query language enhancements + +* [2150](https://github.com/grafana/loki/pull/2150) introduces `bytes_rate`, which calculates the per second byte rate of a log stream, and `bytes_over_time`, which returns the byte size of a log stream. +* [2182](https://github.com/grafana/loki/pull/2182) introduces a long list of comparison operators, which will let you write queries like `count_over_time({foo="bar"}[1m]) > 10`. Check out the PR for a more detailed description. + +#### Loki performance improvements + +* [2216](https://github.com/grafana/loki/pull/2216), [2218](https://github.com/grafana/loki/pull/2218), and [2219](https://github.com/grafana/loki/pull/2219) all improve how memory is allocated and reused for queries. +* [2239](https://github.com/grafana/loki/pull/2239) is a huge improvement for certain cases in which a query covers a large number of streams that all overlap in time. Overlapping data is now internally cached while Loki works to sort all the streams into the proper time order. +* [2293](https://github.com/grafana/loki/pull/2293) was a big refactor to how Loki internally processes log queries vs. metric queries, creating separate code paths to further optimize metric queries. Metric query performance is now 2 to 10 times faster. + +If you are using the query-frontend: + +* [2441](https://github.com/grafana/loki/pull/2441) improves how label queries can be split and queried in parallel +* [2123](https://github.com/grafana/loki/pull/2123) allows queries to the `series` API to be split by time and parallelized; and last but most significant +* [1927](https://github.com/grafana/loki/pull/1927) allows for a much larger range of queries to be sharded and performed in parallel. Query sharding is a topic in itself, but as a rough summary, this type of sharding is not time dependent and leverages how data is already stored by Loki to be able to split queries up into 16 separate pieces to be queried at the same time. + +#### Promtail + +* [2296](https://github.com/grafana/loki/pull/2296) allows Promtail to expose the Loki Push API. With this, you can push from any client to Promtail as if it were Loki, and Promtail can then forward those logs to another Promtail or to Loki. There are some good use cases for this with the Loki Docker Logging Driver; if you want an easier way to configure pipelines or expose metrics collection, point your Docker drivers at a Promtail instance. +* [2282](https://github.com/grafana/loki/pull/2282) contains an example Amazon Lambda where you can use a fan-in approach and ingestion timestamping in Promtail to work around `out of order` issues with multiple Lambdas processing the same log stream. This is one way to get logs from a high-cardinality source without adding a high-cardinality label. +* [2060](https://github.com/grafana/loki/pull/2060) introduces the `Replace` stage, which lets you find and replace or remove text inside a log line. Combined with [2422](https://github.com/grafana/loki/pull/2422) and [2480](https://github.com/grafana/loki/pull/2480), you can now find and replace sensitive data in a log line like a password or email address and replace it with ****, or hash the value to prevent readability, while still being able to trace the value through your logs. Last on the list of pipeline additions, +* [2496](https://github.com/grafana/loki/pull/2496) adds a `Drop` pipeline stage, which lets you drop log lines based on several criteria options including regex matching content, line length, or the age of the log line. The last two are useful to prevent sending to Loki logs that you know would be rejected based on configured limits in the Loki server. + +#### Logstash output plugin + +* [1822](https://github.com/grafana/loki/pull/1822) added a Logstash output plugin for Loki. If you have an existing Logstash install, you can now use this plugin to send your logs to Loki to make it easier to try out, or use Loki alongside an existing logging installation. + +#### Loki Canary + +* [2344](https://github.com/grafana/loki/pull/2344) improved the canaries capabilities for checking for data integrity, including spot checking for logs over a longer time window and running metric queries to verify count_over_time accuracy. + +#### Logcli + +* [2470](https://github.com/grafana/loki/pull/2470) allows you to color code your log lines based on their stream labels for a nice visual indicator of streams. +* [2497](https://github.com/grafana/loki/pull/2497) expands on the series API query to Loki with the`--analyze-labels` flag, which can show you a detailed breakdown of your label key and value combinations. This is very useful for finding improper label usage in Loki or labels with high cardinality. +* [2482](https://github.com/grafana/loki/pull/2482), in which LogCLI will automatically batch requests to Loki to allow making queries with a `--limit=` far larger than the server side limit defined in Loki. LogCLI will dispatch the request in a series of queries configured by the `--batch=` parameter (which defaults to 1000) until the requested limit is reached! + +#### Misc + +* [2453](https://github.com/grafana/loki/pull/2453) improves the error messages when a query times out, as `Context Deadline Exceeded` wasn’t the most intuitive. +* [2336](https://github.com/grafana/loki/pull/2336) provides two new flags that will print the entire Loki config object at startup. Be warned there are a lot of config options, and many won’t apply to your setup (such as storage configs you aren’t using), but this can be a really useful tool when troubleshooting. Sticking with the theme of best for last, +* [2224](https://github.com/grafana/loki/pull/2224) and [2288](https://github.com/grafana/loki/pull/2288) improve support for running Loki with a shared Ring using memberlist while not requiring Consul or Etcd. We need to follow up soon with some better documentation or a blog post on this! + + +### Dependencies + +* Go Version: 1.14.2 +* Cortex Version: 7014ff11ed70d9d59ad29d0a95e73999c436c47c + +### All Changes + +#### Loki +* [2484](https://github.com/grafana/loki/pull/2484) **slim-bean**: Loki: fix batch iterator error when all chunks overlap and chunk time ranges are greater than query time range +* [2483](https://github.com/grafana/loki/pull/2483) **sandeepsukhani**: download boltdb files parallelly during reads +* [2472](https://github.com/grafana/loki/pull/2472) **owen-d**: series endpoint uses normal splits +* [2466](https://github.com/grafana/loki/pull/2466) **owen-d**: BatchIter edge cases +* [2463](https://github.com/grafana/loki/pull/2463) **sandeepsukhani**: revendor cortex to latest master +* [2457](https://github.com/grafana/loki/pull/2457) **adityacs**: Fix panic in cassandra storage while registering metrics +* [2453](https://github.com/grafana/loki/pull/2453) **slim-bean**: Loki: Improve error messages on query timeout or cancel +* [2450](https://github.com/grafana/loki/pull/2450) **adityacs**: Fixes panic in runtime_config +* [2449](https://github.com/grafana/loki/pull/2449) **jvrplmlmn**: Replace usage of sync/atomic with uber-go/atomic +* [2441](https://github.com/grafana/loki/pull/2441) **cyriltovena**: Split label names queries in the frontend. +* [2427](https://github.com/grafana/loki/pull/2427) **owen-d**: Revendor cortex +* [2392](https://github.com/grafana/loki/pull/2392) **owen-d**: avoid mutating config while parsing -config.file +* [2346](https://github.com/grafana/loki/pull/2346) **cyriltovena**: Fixes LogQL grouping +* [2336](https://github.com/grafana/loki/pull/2336) **slim-bean**: Loki: add -print-config-stderr flag to dump loki's runtime config to stderr +* [2330](https://github.com/grafana/loki/pull/2330) **slim-bean**: Loki: Use a new context to update the ring state after a failed chunk transfer +* [2328](https://github.com/grafana/loki/pull/2328) **slim-bean**: Loki: Transfer one chunk at a time per series during chunk transfers +* [2327](https://github.com/grafana/loki/pull/2327) **adityacs**: Fix data race in ingester +* [2323](https://github.com/grafana/loki/pull/2323) **cyriltovena**: Improve object key parsing for boltdb shipper. +* [2306](https://github.com/grafana/loki/pull/2306) **cyriltovena**: Fixes buffered iterator skipping very long lines. +* [2302](https://github.com/grafana/loki/pull/2302) **cyriltovena**: Improve entry deduplication. +* [2294](https://github.com/grafana/loki/pull/2294) **cyriltovena**: Remove NET_BIND_SERVICE capability requirement. +* [2293](https://github.com/grafana/loki/pull/2293) **cyriltovena**: Improve metric queries by computing samples at the edges. +* [2288](https://github.com/grafana/loki/pull/2288) **periklis**: Add support for memberlist dns-based discovery +* [2268](https://github.com/grafana/loki/pull/2268) **owen-d**: lock fix for flaky test +* [2266](https://github.com/grafana/loki/pull/2266) **cyriltovena**: Update to latest cortex. +* [2264](https://github.com/grafana/loki/pull/2264) **adityacs**: Fix ingester results for series query +* [2261](https://github.com/grafana/loki/pull/2261) **sandeepsukhani**: create smaller unique files from boltdb shipper and other code refactorings +* [2254](https://github.com/grafana/loki/pull/2254) **slim-bean**: Loki: Series API will return all series with no match or empty matcher +* [2252](https://github.com/grafana/loki/pull/2252) **owen-d**: avoids further time splitting in querysharding mware +* [2250](https://github.com/grafana/loki/pull/2250) **slim-bean**: Loki: Remove redundant log warning +* [2249](https://github.com/grafana/loki/pull/2249) **owen-d**: avoids recording stats in the sharded engine +* [2248](https://github.com/grafana/loki/pull/2248) **cyriltovena**: Add performance profile flags for logcli. +* [2239](https://github.com/grafana/loki/pull/2239) **cyriltovena**: Cache overlapping blocks +* [2224](https://github.com/grafana/loki/pull/2224) **periklis**: Replace memberlist service in favor of cortex provided service +* [2223](https://github.com/grafana/loki/pull/2223) **adityacs**: Add Error method for step evaluators +* [2219](https://github.com/grafana/loki/pull/2219) **cyriltovena**: Reuse slice for the range vector allocations. +* [2218](https://github.com/grafana/loki/pull/2218) **cyriltovena**: Reuse buffer for hash computation in the engine. +* [2216](https://github.com/grafana/loki/pull/2216) **cyriltovena**: Improve point allocations for each steps in the logql engine. +* [2211](https://github.com/grafana/loki/pull/2211) **sandeepsukhani**: query tee proxy with support for comparison of responses +* [2206](https://github.com/grafana/loki/pull/2206) **sandeepsukhani**: disable index dedupe when rf > 1 and current or upcoming index type is boltdb-shipper +* [2204](https://github.com/grafana/loki/pull/2204) **owen-d**: bumps cortex & fixes conflicts +* [2191](https://github.com/grafana/loki/pull/2191) **periklis**: Add flag to disable tracing activation +* [2189](https://github.com/grafana/loki/pull/2189) **owen-d**: Fix vector-scalar comparisons +* [2182](https://github.com/grafana/loki/pull/2182) **owen-d**: Logql comparison ops +* [2178](https://github.com/grafana/loki/pull/2178) **cyriltovena**: Fixes path prefix in the querier. +* [2166](https://github.com/grafana/loki/pull/2166) **sandeepsukhani**: enforce requirment for periodic config for index tables to be 24h when using boltdb shipper +* [2161](https://github.com/grafana/loki/pull/2161) **cyriltovena**: Fix error message for max tail connections. +* [2156](https://github.com/grafana/loki/pull/2156) **sandeepsukhani**: boltdb shipper download failure handling and some refactorings +* [2150](https://github.com/grafana/loki/pull/2150) **cyriltovena**: Bytes aggregations +* [2136](https://github.com/grafana/loki/pull/2136) **cyriltovena**: Fixes Iterator boundaries +* [2123](https://github.com/grafana/loki/pull/2123) **adityacs**: Fix Series API slowness +* [1927](https://github.com/grafana/loki/pull/1927) **owen-d**: Feature/querysharding ii +* [2032](https://github.com/grafana/loki/pull/2032) **tivvit**: Added support for tail to query frontend + +#### Promtail +* [2496](https://github.com/grafana/loki/pull/2496) **slim-bean**: Promtail: Drop stage +* [2475](https://github.com/grafana/loki/pull/2475) **slim-bean**: Promtail: force the log level on any Loki Push API target servers to match Promtail's log level. +* [2474](https://github.com/grafana/loki/pull/2474) **slim-bean**: Promtail: use --client.external-labels for all clients +* [2471](https://github.com/grafana/loki/pull/2471) **owen-d**: Fix/promtail yaml config +* [2464](https://github.com/grafana/loki/pull/2464) **slim-bean**: Promtail: Bug: loki push api, clone labels before handling +* [2438](https://github.com/grafana/loki/pull/2438) **rfratto**: pkg/promtail: propagate a logger rather than using util.Logger globally +* [2432](https://github.com/grafana/loki/pull/2432) **pyr0hu**: Promtail: Allow empty replace values for replace stage +* [2422](https://github.com/grafana/loki/pull/2422) **wardbekker**: Template: Added a sha256 template function for obfuscating / anonymize PII data in e.g. the replace stage +* [2414](https://github.com/grafana/loki/pull/2414) **rfratto**: Add RegisterFlagsWithPrefix to config structs +* [2386](https://github.com/grafana/loki/pull/2386) **cyriltovena**: Add regex function to promtail template stage. +* [2345](https://github.com/grafana/loki/pull/2345) **adityacs**: Refactor Promtail target manager code +* [2301](https://github.com/grafana/loki/pull/2301) **flixr**: Promtail: support unix timestamps with fractional seconds +* [2296](https://github.com/grafana/loki/pull/2296) **slim-bean**: Promtail: Loki Push API +* [2282](https://github.com/grafana/loki/pull/2282) **owen-d**: Lambda-Promtail +* [2242](https://github.com/grafana/loki/pull/2242) **carlpett**: Set user agent on outgoing http requests +* [2196](https://github.com/grafana/loki/pull/2196) **cyriltovena**: Adds default -config.file for the promtail docker images. +* [2127](https://github.com/grafana/loki/pull/2127) **bastjan**: Update go-syslog to accept non-UTF8 encoding in syslog message +* [2111](https://github.com/grafana/loki/pull/2111) **adityacs**: Fix Promtail journal seeking known position +* [2105](https://github.com/grafana/loki/pull/2105) **fatpat**: promtail: Add Entry variable to template +* [1118](https://github.com/grafana/loki/pull/1118) **shuttie**: promtail: fix high CPU usage on large kubernetes clusters. +* [2060](https://github.com/grafana/loki/pull/2060) **adityacs**: Feature: Replace stage in pipeline +* [2087](https://github.com/grafana/loki/pull/2087) **adityacs**: Set JournalTarget Priority value to keyword + +#### Logcli +* [2497](https://github.com/grafana/loki/pull/2497) **slim-bean**: logcli: adds --analyize-labels to logcli series command and changes how labels are provided to the command +* [2482](https://github.com/grafana/loki/pull/2482) **slim-bean**: Logcli: automatically batch requests +* [2470](https://github.com/grafana/loki/pull/2470) **adityacs**: colored labels output for logcli +* [2235](https://github.com/grafana/loki/pull/2235) **pstibrany**: logcli: Remove single newline from the raw line before printing. +* [2126](https://github.com/grafana/loki/pull/2126) **cyriltovena**: Validate local storage config for the logcli +* [2083](https://github.com/grafana/loki/pull/2083) **adityacs**: Support querying labels on time range in logcli + +#### Docs +* [2473](https://github.com/grafana/loki/pull/2473) **owen-d**: fixes lambda-promtail relative doc link +* [2454](https://github.com/grafana/loki/pull/2454) **oddlittlebird**: Create CODEOWNERS +* [2439](https://github.com/grafana/loki/pull/2439) **till**: Docs: updated "Upgrading" for docker driver +* [2437](https://github.com/grafana/loki/pull/2437) **wardbekker**: DOCS: clarified globbing behaviour of __path__ of the doublestar library +* [2431](https://github.com/grafana/loki/pull/2431) **endu**: fix dead link +* [2425](https://github.com/grafana/loki/pull/2425) **RichiH**: Change conduct contact email address +* [2420](https://github.com/grafana/loki/pull/2420) **petuhovskiy**: Fix docker driver doc +* [2418](https://github.com/grafana/loki/pull/2418) **cyriltovena**: Add logstash to clients page with FrontMatter. +* [2402](https://github.com/grafana/loki/pull/2402) **cyriltovena**: More fixes for the website. +* [2400](https://github.com/grafana/loki/pull/2400) **tontongg**: Fix URL to LogQL documentation +* [2398](https://github.com/grafana/loki/pull/2398) **robbymilo**: Docs - update links, readme +* [2397](https://github.com/grafana/loki/pull/2397) **coderanger**: 📝 Note that entry_parser is deprecated. +* [2396](https://github.com/grafana/loki/pull/2396) **dnsmichi**: Docs: Fix Fluentd title (visible in menu) +* [2391](https://github.com/grafana/loki/pull/2391) **cyriltovena**: Update fluentd docs and fixes links for the website. +* [2390](https://github.com/grafana/loki/pull/2390) **cyriltovena**: Fluent bit docs +* [2389](https://github.com/grafana/loki/pull/2389) **cyriltovena**: Docker driver doc +* [2385](https://github.com/grafana/loki/pull/2385) **abowloflrf**: Update logo link in README.md +* [2378](https://github.com/grafana/loki/pull/2378) **robbymilo**: Sync docs to website +* [2360](https://github.com/grafana/loki/pull/2360) **owen-d**: Makes timestamp parsing docs clearer +* [2358](https://github.com/grafana/loki/pull/2358) **rille111**: Documentation: Add example for having separate pvc for loki, using helm +* [2357](https://github.com/grafana/loki/pull/2357) **owen-d**: Storage backend examples +* [2338](https://github.com/grafana/loki/pull/2338) **cyriltovena**: Add a complete tutorial on how to ship logs from AWS EKS. +* [2335](https://github.com/grafana/loki/pull/2335) **cyriltovena**: Improve documentation of the metric stage. +* [2331](https://github.com/grafana/loki/pull/2331) **cyriltovena**: Add a tutorial to forward AWS ECS logs to Loki. +* [2321](https://github.com/grafana/loki/pull/2321) **cyriltovena**: Tutorial to run Promtail on AWS EC2 +* [2318](https://github.com/grafana/loki/pull/2318) **adityacs**: Configuration documentation improvements +* [2317](https://github.com/grafana/loki/pull/2317) **owen-d**: remove DynamoDB chunk store doc +* [2308](https://github.com/grafana/loki/pull/2308) **wardbekker**: Added a link to the replace parsing stage +* [2305](https://github.com/grafana/loki/pull/2305) **rafaelpissolatto**: Fix schema_config store value +* [2285](https://github.com/grafana/loki/pull/2285) **adityacs**: Fix local.md doc +* [2284](https://github.com/grafana/loki/pull/2284) **owen-d**: Update local.md +* [2279](https://github.com/grafana/loki/pull/2279) **Fra-nk**: Documentation: Refine LogQL documentation +* [2273](https://github.com/grafana/loki/pull/2273) **RichiH**: Fix typo +* [2247](https://github.com/grafana/loki/pull/2247) **carlpett**: docs: Fix missing quotes +* [2233](https://github.com/grafana/loki/pull/2233) **vyzigold**: docs: Add readmes to individual helm charts +* [2220](https://github.com/grafana/loki/pull/2220) **oddlittlebird**: Docs: Local install edits +* [2217](https://github.com/grafana/loki/pull/2217) **fredr**: docs: BoltDB typo +* [2215](https://github.com/grafana/loki/pull/2215) **fredr**: docs: Correct loki address for docker-compose +* [2172](https://github.com/grafana/loki/pull/2172) **cyriltovena**: Update old link for pipeline stages. +* [2163](https://github.com/grafana/loki/pull/2163) **slim-bean**: docs: fix an error in the example log line and byte counter metrics +* [2160](https://github.com/grafana/loki/pull/2160) **slim-bean**: Fix some errors in the upgrade guide to 1.5.0 and add some missing notes discovered by users. +* [2152](https://github.com/grafana/loki/pull/2152) **eamonryan**: Fix typo in promtail ClusterRole +* [2139](https://github.com/grafana/loki/pull/2139) **adityacs**: Fix configuration docs +* [2137](https://github.com/grafana/loki/pull/2137) **RichiH**: Propose new governance +* [2136](https://github.com/grafana/loki/pull/2136) **cyriltovena**: Fixes Iterator boundaries +* [2125](https://github.com/grafana/loki/pull/2125) **theMercedes**: Update logql.md +* [2112](https://github.com/grafana/loki/pull/2112) **nileshcs**: Documentation: Outdated fluentd image name, UID details, link update +* [2092](https://github.com/grafana/loki/pull/2092) **i-takizawa**: docs: make visible + +#### Build +* [2467](https://github.com/grafana/loki/pull/2467) **slim-bean**: Update Loki build image + +#### Ksonnet +* [2460](https://github.com/grafana/loki/pull/2460) **Duologic**: refactor: use $.core.v1.envVar +* [2452](https://github.com/grafana/loki/pull/2452) **slim-bean**: ksonnet: Reduce querier parallelism to a more sane default value and remove the default setting for storage_backend +* [2377](https://github.com/grafana/loki/pull/2377) **Duologic**: refactor: moved jaeger-agent-mixin +* [2373](https://github.com/grafana/loki/pull/2373) **slim-bean**: Ksonnet: Add a Pod Disruption Budget to Loki Ingesters +* [2185](https://github.com/grafana/loki/pull/2185) **cyriltovena**: Refactor mixin routes and add series API. +* [2162](https://github.com/grafana/loki/pull/2162) **slim-bean**: ksonnet: Fix up datasources and variables in Loki Operational +* [2091](https://github.com/grafana/loki/pull/2091) **beorn7**: Keep scrape config in line with the new Prometheus scrape config + +#### Docker logging driver +* [2435](https://github.com/grafana/loki/pull/2435) **cyriltovena**: Add more precisions on the docker driver installed on the daemon. +* [2343](https://github.com/grafana/loki/pull/2343) **jdfalk**: loki-docker-driver: Change "ignoring empty line" to debug logging +* [2295](https://github.com/grafana/loki/pull/2295) **cyriltovena**: Remove mount in the docker driver. +* [2199](https://github.com/grafana/loki/pull/2199) **cyriltovena**: Docker driver relabeling +* [2116](https://github.com/grafana/loki/pull/2116) **cyriltovena**: Allows to change the log driver mode and buffer size. + +#### Logstash output plugin +* [2415](https://github.com/grafana/loki/pull/2415) **cyriltovena**: Set service values via --set for logstash. +* [2410](https://github.com/grafana/loki/pull/2410) **adityacs**: logstash code refactor and doc improvements +* [1822](https://github.com/grafana/loki/pull/1822) **adityacs**: Loki Logstash Plugin + +#### Loki canary +* [2413](https://github.com/grafana/loki/pull/2413) **slim-bean**: Loki-Canary: Backoff retries on query failures, add histograms for query performance. +* [2369](https://github.com/grafana/loki/pull/2369) **slim-bean**: Loki Canary: One more round of improvements to query for missing websocket entries up to max-wait +* [2350](https://github.com/grafana/loki/pull/2350) **slim-bean**: Canary tweaks +* [2344](https://github.com/grafana/loki/pull/2344) **slim-bean**: Loki-Canary: Add query spot checking and metric count checking +* [2259](https://github.com/grafana/loki/pull/2259) **ombre8**: Canary: make stream configurable + +#### Fluentd +* [2407](https://github.com/grafana/loki/pull/2407) **cyriltovena**: bump fluentd version to release a new gem. +* [2399](https://github.com/grafana/loki/pull/2399) **tarokkk**: fluentd: Make fluentd version requirements permissive +* [2179](https://github.com/grafana/loki/pull/2179) **takanabe**: Improve fluentd plugin development experience +* [2171](https://github.com/grafana/loki/pull/2171) **takanabe**: Add server TLS certificate verification + +#### Fluent Bit +* [2375](https://github.com/grafana/loki/pull/2375) **cyriltovena**: Fixes the fluentbit batchwait backward compatiblity. +* [2367](https://github.com/grafana/loki/pull/2367) **dojci**: fluent-bit: Add more loki client configuration options +* [2365](https://github.com/grafana/loki/pull/2365) **dojci**: fluent-bit: Fix fluent-bit exit callback when buffering is enabled +* [2290](https://github.com/grafana/loki/pull/2290) **cyriltovena**: Fixes the lint issue merged to master. +* [2286](https://github.com/grafana/loki/pull/2286) **adityacs**: Fix fluent-bit newline and tab characters +* [2142](https://github.com/grafana/loki/pull/2142) **dojci**: Add FIFO queue persistent buffering for fluent bit output plugin +* [2089](https://github.com/grafana/loki/pull/2089) **FrederikNS**: Allow configuring more options for output configuration + +#### Helm +* [2406](https://github.com/grafana/loki/pull/2406) **steven-sheehy**: Helm: Fix regression in chart name +* [2379](https://github.com/grafana/loki/pull/2379) **StevenReitsma**: production/helm: Add emptyDir volume type to promtail PSP +* [2366](https://github.com/grafana/loki/pull/2366) **StevenReitsma**: production/helm: Add projected and downwardAPI volume types to PodSecurityPolicy (#2355) +* [2258](https://github.com/grafana/loki/pull/2258) **Synehan**: helm: add annotations to service monitor +* [2241](https://github.com/grafana/loki/pull/2241) **chauffer**: Kubernetes manifests: Remove namespace from cluster-wide resources +* [2238](https://github.com/grafana/loki/pull/2238) **vhrosales**: helm: Add loadBalancerIP option to loki chart +* [2205](https://github.com/grafana/loki/pull/2205) **joschi36**: BUG: add missing namespace in ingress object +* [2197](https://github.com/grafana/loki/pull/2197) **cyriltovena**: Render loki datasources even if Grafana is disabled. +* [2141](https://github.com/grafana/loki/pull/2141) **cyriltovena**: Adds the ability to have a pull secrets for Promtail. +* [2099](https://github.com/grafana/loki/pull/2099) **allout58**: helm/loki-stack: Support Prometheus on a sub-path in Grafana config +* [2086](https://github.com/grafana/loki/pull/2086) **osela**: helm/loki-stack: render loki datasource only if grafana is enabled +* [2091](https://github.com/grafana/loki/pull/2091) **beorn7**: Keep scrape config in line with the new Prometheus scrape config + +#### Build +* [2371](https://github.com/grafana/loki/pull/2371) **cyriltovena**: Fixes helm publish that needs now to add repo. +* [2341](https://github.com/grafana/loki/pull/2341) **slim-bean**: Build: Fix CI helm test +* [2309](https://github.com/grafana/loki/pull/2309) **cyriltovena**: Test again arm32 on internal ci. +* [2307](https://github.com/grafana/loki/pull/2307) **cyriltovena**: Removes arm32 for now as we're migrating the CI. +* [2287](https://github.com/grafana/loki/pull/2287) **wardbekker**: Change the Grafana image to latest +* [2212](https://github.com/grafana/loki/pull/2212) **roidelapluie**: Remove unhelpful/problematic term in circleci.yml -* [2282](https://github.com/grafana/loki/pull/2282) **owen-d**: introduces a [lambda-promtail](https://github.com/grafana/loki/blob/master/docs/clients/lambda-promtail/README.md) workflow for shipping Cloudwatch logs to Loki. ## 1.5.0 (2020-05-20) @@ -20,7 +300,7 @@ there were substantial breaking config changes in Cortex 1.0 which standardized **The Loki docker image user has changed to no longer be root** -Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/operations/upgrade.md#150) for more detailed information on these changes. +Check the [upgrade guide](https://github.com/grafana/loki/blob/master/docs/sources/operations/upgrade.md#150) for more detailed information on these changes. ### Notable Features and Fixes @@ -274,7 +554,7 @@ Over 130 PR's merged for this release, from 40 different contributors!! We cont Before we get into new features, version 1.4.0 brings with it the first (that we are aware of) upgrade dependency. -We have created a dedicated page for upgrading Loki in the [operations section of the docs](https://github.com/grafana/loki/blob/master/docs/operations/upgrade.md#140) +We have created a dedicated page for upgrading Loki in the [operations section of the docs](https://github.com/grafana/loki/blob/master/docs/sources/operations/upgrade.md#140) The docker image tag naming was changed, the starting in 1.4.0 docker images no longer have the `v` prefix: `grafana/loki:1.4.0` diff --git a/docs/sources/configuration/_index.md b/docs/sources/configuration/_index.md index 72b72d1cfb25..1d333b5f4dbd 100644 --- a/docs/sources/configuration/_index.md +++ b/docs/sources/configuration/_index.md @@ -1101,11 +1101,16 @@ redis: [enable_tls: | default = false] fifocache: - # Number of entries to cache in-memory. - # CLI flag: -.fifocache.size - [size: | default = 0] + # Maximum memory size of the cache in bytes. A unit suffix (KB, MB, GB) may be + # applied. + # CLI flag: -.fifocache.max-size-bytes + [max_size_bytes: | default = ""] - # The expiry duration for the in-memory cache. + # Maximum number of entries in the cache. + # CLI flag: -.fifocache.max-size-items + [max_size_items: | default = 0] + + # The expiry duration for the cache. # CLI flag: -.fifocache.duration [validity: | default = 0s] ``` diff --git a/docs/sources/operations/upgrade.md b/docs/sources/operations/upgrade.md index a2f47ad2f2e0..2e731807c2ec 100644 --- a/docs/sources/operations/upgrade.md +++ b/docs/sources/operations/upgrade.md @@ -12,23 +12,102 @@ On this page we will document any upgrade issues/gotchas/considerations we are a ## Master / Unreleased -#### Documentation changes: +## 1.6.0 -Configuration document has been re-orderd a bit and for all the config, corresponding `CLI` flag is provided. +### IMPORTANT: Ksonnet Port Change and Removal of NET_BIND_SERVICE Capability from docker image -S3 config now supports exapnded config. Example can be found here [s3_expanded_config](../configuration/examples.md#s3-expanded-config) +In 1.5.0 we changed the Loki user to not run as root which created problems binding to port 80. +To address this we updated the docker image to add the NET_BIND_SERVICE capability to the loki process +which allowed Loki to bind to port 80 as a non root user, so long as the underlying system allowed that +linux capability. -### New Ingester GRPC API special rollout procedure in microservices mode +This has proved to be a problem for many reasons and in PR [2294](https://github.com/grafana/loki/pull/2294/files) +the capability was removed. -A new ingester GRPC API has been added allowing to speed up metric queries, to ensure a rollout without query errors make sure you upgrade all ingesters first. +It is now no longer possible for the Loki to be started with a port less than 1024 with the published docker image. + +The default for Helm has always been port 3100, and Helm users should be unaffect unless they changed the default. + +**Ksonnet users however should closely check their configuration, in PR 2294 the loki port was changed from 80 to 3100** + + +### IMPORTANT: If you run Loki in microservices mode, special rollout instructions + +A new ingester GRPC API has been added allowing to speed up metric queries, to ensure a rollout without query errors **make sure you upgrade all ingesters first.** Once this is done you can then proceed with the rest of the deployment, this is to ensure that queriers won't look for an API not yet available. If you roll out everything at once, queriers with this new code will attempt to query ingesters which may not have the new method on the API and queries will fail. This will only affect reads(queries) and not writes and only for the duration of the rollout. +### IMPORTANT: Scrape config changes to both Helm and Ksonnet will affect labels created by Promtail + +PR [2091](https://github.com/grafana/loki/pull/2091) Makes several changes to the promtail scrape config: + +```` +This is triggered by https://github.com/grafana/jsonnet-libs/pull/261 + +The above PR changes the instance label to be actually unique within +a scrape config. It also adds a pod and a container target label +so that metrics can easily be joined with metrics from cAdvisor, KSM, +and the Kubelet. + +This commit adds the same to the Loki scrape config. It also removes +the container_name label. It is the same as the container label +and was already added to Loki previously. However, the +container_name label is deprecated and has disappeared in K8s 1.16, +so that it will soon become useless for direct joining. +```` + +TL;DR + +The following label have been changed in both the Helm and Ksonnet Promtail scrape configs: + +`instance` -> `pod` +`container_name` -> `container` + + +### Experimental boltdb-shipper changes + +PR [2166](https://github.com/grafana/loki/pull/2166) now forces the index to have a period of exactly `24h`: + +Loki will fail to start with an error if the active schema or upcoming schema are not set to a period of `24h` + +You can add a new schema config like this: + +```yaml +schema_config: + configs: + - from: 2020-01-01 <----- This is your current entry, date will be different + store: boltdb-shipper + object_store: aws + schema: v11 + index: + prefix: index_ + period: 168h + - from: [INSERT FUTURE DATE HERE] <----- Add another entry, set a future date + store: boltdb-shipper + object_store: aws + schema: v11 + index: + prefix: index_ + period: 24h <--- This must be 24h +``` +If you are not on `schema: v11` this would be a good oportunity to make that change _in the new schema config_ also. + +**NOTE** If the current time in your timezone is after midnight UTC already, set the date one additional day forward. + +There was also a significant overhaul to how boltdb-shipper internals, this should not be visible to a user but as this +feature is experimental and under development bug are possible! + +The most noticeable change if you look in the storage, Loki no longer updates an existing file and instead creates a +new index file every 15mins, this is an important move to make sure objects in the object store are immutable and +will simplify future operations like compaction and deletion. + ### Breaking CLI flags changes +The following CLI flags where changed to improve consistency, they are not expected to be widely used + ```diff - querier.query_timeout + querier.query-timeout