Use log4j pattern syntax #57433

mshustov · 2020-02-12T09:37:47Z

Summary

switch to log4j syntax {level} --> %level. Note: we support only long patterns atm (%date, not %d, since the latter is a bit cryptic)
rename timestamp --> date, context --> logger to follow log4j terms. But still keep meta, since map is referenced to MapMessage in log4j
remove file from pre-defined appenders. I don't want to introduce a default for the log file name, since it creates another maintainability problem.
document conversion patterns for Pattern layout
document how to reproduce Legacy platform setup with Logging config

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

Dev docs

Logging output of the New platform plugins can use adjusted via new config.

file name is required. let users to setup everything

mshustov · 2020-02-12T13:31:11Z

src/core/server/logging/layouts/conversions/date.ts

@@ -66,21 +67,27 @@ function validateTimezone(timezone: string) {
 }

 function validate(rawString: string) {
-  for (const matched of rawString.matchAll(timestampRegExp)) {


TIL matchAll is not polyfilled in runtime. that was a surprise, since it works in tests

What was the issue outside of the scope of tests?

I contacted @elastic/kibana-operations to clarify this. The test and runtime envs should be identical

should be fixed in #57467

mshustov · 2020-02-12T13:32:50Z

src/core/server/logging/logging_config.ts

@@ -140,13 +140,6 @@ export class LoggingConfig {
        layout: { kind: 'pattern', highlight: true },
      } as AppenderConfigType,
    ],
-    [
-      'file',


Removed since it requires defining a file name. I don't think we should set a default for it.

elasticmachine · 2020-02-12T14:25:13Z

Pinging @elastic/kibana-platform (Team:Platform)

src/core/server/logging/README.md

pgayvallet · 2020-02-12T15:07:18Z

src/core/server/logging/layouts/conversions/date.ts

@@ -66,21 +67,27 @@ function validateTimezone(timezone: string) {
 }

 function validate(rawString: string) {
-  for (const matched of rawString.matchAll(timestampRegExp)) {


What was the issue outside of the scope of tests?

This reverts commit 9f491d4.

mshustov · 2020-02-13T11:30:22Z

@elasticmachine merge upstream

kibanamachine · 2020-02-13T13:10:59Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 28d8c82

History

💛 Build #26186 was flaky 2b0fa2d
💔 Build #26169 failed c8bc7e4
💔 Build #26116 failed bf3215a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* address comments * use log4j-like syntax in layout pattern * %timestamp --> %date to match log4j conversion pattern * %context --> %logger to match log4j pattern * remove file from pre-defined appenders. file name is required. let users to setup everything * matchAll is not polyfilled in runtime * document available patterns and migration path * document BWC requirements * Revert "matchAll is not polyfilled in runtime" This reverts commit 9f491d4. * address comments Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: (22 commits) Use log4j pattern syntax (elastic#57433) [ML] Categorization field example endpoint tests (elastic#57471) [Lens] Filter out pinned filters from saved object of Lens (elastic#57197) Lens client side shim cleanup (elastic#56976) [Maps] do not show border color for icon in legend when border width is zero (elastic#57501) refactors 'data-providers' tests (elastic#57474) add `absolute` option to `getUrlForApp` (elastic#57193) [Telemetry] Migrate public to NP (elastic#56285) address flaky test where instances might have different start… (elastic#57506) fix(NA): support legacy plugins path in plugins (elastic#57472) build immutable bundles for new platform plugins (elastic#53976) [SIEM] [Detection Engine] Reject if duplicate rule_id in request payload (elastic#57057) Add autocomplete="off" for input type="password" to appease the scanners (elastic#56922) Use default spaces suffix for signals index if spaces disabled (elastic#57244) [Alerting] Create alert design cleanup (elastic#56929) Management Api - add to migration guide (elastic#56892) fixing maps (elastic#56706) [Maps] Autocomplete for custom color palettes and custom icon palettes (elastic#56446) [Alerting] make actionGroup name's i18n-able (elastic#57404) fixed flaky test (elastic#57490) ... # Conflicts: # src/legacy/core_plugins/telemetry/public/components/__snapshots__/telemetry_form.test.js.snap # src/plugins/telemetry/public/components/telemetry_management_section.tsx

* address comments * use log4j-like syntax in layout pattern * %timestamp --> %date to match log4j conversion pattern * %context --> %logger to match log4j pattern * remove file from pre-defined appenders. file name is required. let users to setup everything * matchAll is not polyfilled in runtime * document available patterns and migration path * document BWC requirements * Revert "matchAll is not polyfilled in runtime" This reverts commit 9f491d4. * address comments Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* master: (22 commits) skip flaky suite (elastic#50018) skip settings tests (elastic#57608) skip failing suite (elastic#44631) [SIEM] [Case] Initial UI (elastic#57283) handle viewing sample dashboards on default dist (elastic#57510) Fix detection of "system requests" in plugins (elastic#57149) [ML] New Platform server shim: update job service schema (elastic#57614) skip flaky suite (elastic#44631) [APM] Update monospace font family variable (elastic#57555) skip flaky test (elastic#57377) Skip save query tests (elastic#57589) [Maps] allow simultaneous opening of multiple tooltips (elastic#57226) [Uptime] Fix/host connected components (elastic#56969) [logs][metrics][docs] Update screenshots for 7.6 (elastic#57254) [ML] New Platform server shim: update job service routes to use new platform router (elastic#57403) [Maps] Fix document source top hits split by scripted field (elastic#57481) Use log4j pattern syntax (elastic#57433) [ML] Categorization field example endpoint tests (elastic#57471) [Lens] Filter out pinned filters from saved object of Lens (elastic#57197) Lens client side shim cleanup (elastic#56976) ...

mshustov added 8 commits February 12, 2020 09:22

address comments

2b6ddda

use log4j-like syntax in layout pattern

864daaf

%timestamp --> %date to match log4j conversion pattern

94a89b8

%context --> %logger to match log4j pattern

bf3215a

remove file from pre-defined appenders.

96f1edb

file name is required. let users to setup everything

matchAll is not polyfilled in runtime

9f491d4

document available patterns and migration path

20ecaa8

Merge branch 'master' into use-log4j-syntax

c8bc7e4

mshustov commented Feb 12, 2020

View reviewed changes

mshustov mentioned this pull request Feb 12, 2020

Logging service uses new config format #41956

Closed

17 tasks

document BWC requirements

2b0fa2d

mshustov marked this pull request as ready for review February 12, 2020 14:25

mshustov requested a review from a team as a code owner February 12, 2020 14:25

mshustov added Feature:New Platform Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc labels Feb 12, 2020

mshustov added v7.7.0 release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. labels Feb 12, 2020

pgayvallet approved these changes Feb 12, 2020

View reviewed changes

mshustov added 3 commits February 13, 2020 08:57

Merge branch 'master' into use-log4j-syntax

d98db45

Revert "matchAll is not polyfilled in runtime"

d9fa26b

This reverts commit 9f491d4.

address comments

56cdef0

mshustov added the v8.0.0 label Feb 13, 2020

Merge branch 'master' into use-log4j-syntax

28d8c82

mshustov merged commit 9c1f5ae into elastic:master Feb 13, 2020

mshustov deleted the use-log4j-syntax branch February 13, 2020 13:28

mshustov mentioned this pull request Feb 13, 2020

[7.x] Use log4j pattern syntax (#57433) #57568

Merged

mshustov added the backported label Feb 21, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use log4j pattern syntax #57433

Use log4j pattern syntax #57433

mshustov commented Feb 12, 2020 •

edited

Loading

mshustov Feb 12, 2020 •

edited

Loading

pgayvallet Feb 12, 2020

mshustov Feb 12, 2020

mshustov Feb 12, 2020 •

edited

Loading

mshustov Feb 12, 2020

elasticmachine commented Feb 12, 2020

pgayvallet Feb 12, 2020

mshustov commented Feb 13, 2020

kibanamachine commented Feb 13, 2020

Use log4j pattern syntax #57433

Use log4j pattern syntax #57433

Conversation

mshustov commented Feb 12, 2020 • edited Loading

Summary

Checklist

For maintainers

Dev docs

mshustov Feb 12, 2020 • edited Loading

Choose a reason for hiding this comment

pgayvallet Feb 12, 2020

Choose a reason for hiding this comment

mshustov Feb 12, 2020

Choose a reason for hiding this comment

mshustov Feb 12, 2020 • edited Loading

Choose a reason for hiding this comment

mshustov Feb 12, 2020

Choose a reason for hiding this comment

elasticmachine commented Feb 12, 2020

pgayvallet Feb 12, 2020

Choose a reason for hiding this comment

mshustov commented Feb 13, 2020

kibanamachine commented Feb 13, 2020

💚 Build Succeeded

History

mshustov commented Feb 12, 2020 •

edited

Loading

mshustov Feb 12, 2020 •

edited

Loading

mshustov Feb 12, 2020 •

edited

Loading