Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New logger subsystem (Bunyan) #835

Merged
merged 44 commits into from
Jul 25, 2018
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
f4aa98e
code: extracted detox-common pkg, switched to bunyan logger
noomorph Jul 4, 2018
a10cba1
code: reduced log levels for wss
noomorph Jul 17, 2018
6803b7f
fix: instrumentationProcess.on bug
noomorph Jul 17, 2018
e101e9e
feat: detox.init() terminates tests if it fails
noomorph Jul 17, 2018
d5555b6
code: switched link to the fork of bunyan-debug-stream
noomorph Jul 17, 2018
bd3ca0c
code: removed remnants of previous experiments
noomorph Jul 17, 2018
d6feea8
fix: wss loglevel resolves to trace
noomorph Jul 17, 2018
126287c
fix: a more polite handling of detox.init() error
noomorph Jul 17, 2018
f995be5
test: removed real logging in the middle of test
noomorph Jul 17, 2018
2d67a57
build: fixing the CI build
noomorph Jul 17, 2018
6f0235a
feat: reduced extra verbosity of spawn logs
noomorph Jul 17, 2018
d594c00
fix: returned 100% coverage and tweaked spawn logs
noomorph Jul 18, 2018
5956648
fix: removed useless errors from the logs
noomorph Jul 18, 2018
46d0097
test: fixed test expectations
noomorph Jul 18, 2018
d9cc156
feat: added plain file logging too
noomorph Jul 18, 2018
2c80ffe
fix: bugfixes and unit test fixes
noomorph Jul 18, 2018
7c00301
feat: appending process.pid to log filename for multiple test workers…
noomorph Jul 19, 2018
b83e569
fix: expectation in index.test.js
noomorph Jul 19, 2018
c3af893
fix: updated artifact directory snapshots for iOS
noomorph Jul 23, 2018
abf36b7
code: more logging inside Artifact.js and SimulatorLogRecording.js also
noomorph Jul 23, 2018
113be22
build: temporary elevated log verbosity level
noomorph Jul 23, 2018
af185cc
code: added beforeEach and afterEach trace logging
noomorph Jul 23, 2018
9fccc1a
code: moved most of calls into .emit method of artifacts manager
noomorph Jul 23, 2018
2e5d458
code: minor fixes according to the review
noomorph Jul 23, 2018
8d107b6
test: updated snapshots
noomorph Jul 23, 2018
6cdc26e
wip: merging detox-server and detox-common into detox package
noomorph Jul 23, 2018
cf0d4c1
feat: printing environment variables before jest test command
noomorph Jul 24, 2018
aaa70dc
feat: added timeout for unlock screen command
noomorph Jul 24, 2018
0e6ba3e
build: force exit test suites that failed on beforeAll
noomorph Jul 24, 2018
c450657
test: updated unit tests
noomorph Jul 24, 2018
bc7203c
hack: trying to debug CI error
noomorph Jul 24, 2018
0b7142e
hack: more precise log
noomorph Jul 24, 2018
d308f5c
hack: ignoring exception
noomorph Jul 24, 2018
0abb2bf
fix: logging exec success even if there is no output
noomorph Jul 24, 2018
20819bf
code: switched AAPT to use utils/exec.js as well
noomorph Jul 24, 2018
e8c474c
code: switched back to the official bunyan-debug-stream package
noomorph Jul 24, 2018
c8ad84d
hack: using retries on CI
noomorph Jul 24, 2018
d9ddfe2
code: minor cleanups
noomorph Jul 24, 2018
c15572c
fix: made no-color argument really work
noomorph Jul 24, 2018
455e0ec
feat: adding deprecated warning for the old log level names
noomorph Jul 25, 2018
ffd9b7d
code: relaxed retries on adb.unlockScreen and added a TODO
noomorph Jul 25, 2018
7521d61
Merge remote-tracking branch 'origin/master' into noomorph/powerlogs
noomorph Jul 25, 2018
cd886ae
revert: too verbose logs for CI
noomorph Jul 25, 2018
1d6173f
docs: updated detox server mentions
noomorph Jul 25, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: adding deprecated warning for the old log level names
  • Loading branch information
noomorph committed Jul 25, 2018
commit 455e0ecc19cba3ea1eada3cfdde9a1d55fe2e692
4 changes: 3 additions & 1 deletion detox/local-cli/detox-run-server.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ const logger = require('../src/utils/logger');

program
.name('detox run-server')
.description('Starts a standalone Detox server')
.description('Start a standalone Detox server')
.option(
'-p, --port [port]',
'Port number',
'8099'
)
.option('-l, --loglevel [value]',
'Log level: fatal, error, warn, info, debug, trace', 'info')
.option(
'--no-color',
'Disable colorful logs',
Expand Down
24 changes: 21 additions & 3 deletions detox/src/utils/logger.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,26 @@ function adaptOlderLogLevelName(level) {
case 'verbose':
return 'debug';

case 'wss':
case 'silly':
case 'wss':
return 'trace';

default:
return 'info';
}
}

function isLogLevelNameDeprecated(level) {
switch (level) {
case 'verbose':
case 'silly':
case 'wss':
return true;
default:
return false;
}
}

function createPlainBunyanStream({ logPath, level }) {
const options = {
showDate: false,
Expand Down Expand Up @@ -63,7 +74,8 @@ function createPlainBunyanStream({ logPath, level }) {
}

function init() {
const level = adaptOlderLogLevelName(argparse.getArgValue('loglevel'));
const levelFromArg = argparse.getArgValue('loglevel');
const level = adaptOlderLogLevelName(levelFromArg);
const logBaseFilename = path.join(argparse.getArgValue('artifacts-location') || '', `detox_pid_${process.pid}`);
const shouldRecordLogs = ['failing', 'all'].indexOf(argparse.getArgValue('record-logs')) >= 0;

Expand All @@ -86,10 +98,16 @@ function init() {
}));
}

return bunyan.createLogger({
const logger = bunyan.createLogger({
name: 'detox',
streams: bunyanStreams,
});

if (isLogLevelNameDeprecated(levelFromArg)) {
logger.warn(`--loglevel ${levelFromArg} is deprecated and will be removed in detox@9.0.0, use --loglevel ${level} instead`);
}

return logger;
}

module.exports = init();
21 changes: 12 additions & 9 deletions docs/APIRef.DetoxCLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@ Initiating your test suite
| --- | --- |
| -h, --help | output usage information |
| -o, --runner-config \<config\> | Test runner config file, defaults to 'e2e/mocha.opts' for mocha and 'e2e/config.json' for jest |
| -l, --loglevel [value] | info, debug, verbose, silly, wss |
| -l, --loglevel [value] | Log level: fatal, error, warn, info, debug, trace |
| --no-color | Disable colors in log output |
| -c, -configuration \<device config\> | Select a device configuration from your defined configurations,if not supplied, and there's only one configuration, detox will default to it |
| -r, --reuse | Reuse existing installed app (do not delete and re-tall) for a faster run. |
| -u, --cleanup | Shutdown simulator when test is over, useful for CI ipts, to make sure detox exists cleanly with no residue |
Expand All @@ -55,12 +56,11 @@ Initiating your test suite
| --record-logs [failing/all/none] | Save logs during each test to artifacts directory. Pass "failing" to save logs of failing tests only. The default value is **none**. |
| --take-screenshots [failing/all/none] | Save screenshots before and after each test to artifacts directory. Pass "failing" to save screenshots of failing tests only. The default value is **none**. |
| --record-videos [failing/all/none] | Save screen recordings of each test to artifacts directory. Pass "failing" to save recordings of failing tests only. The default value is **none**. |
| -p, --platform [ios/android] | Run platform specific tests. Runs tests with invert grep on `:platform:`, e.g test with substring `:ios:` in its name will not run when passing `--platform android` |
| -p, --platform [ios/android] | Run platform specific tests. Runs tests with invert grep on `:platform:`, e.g test with substring `:ios:` in its name will not run when passing `--platform android` |
| -H, --headless | [Android Only] Launch Emulator in headless mode. Useful when running on CI. |
| -w, --workers | [iOS Only] Specifies number of workers the test runner should spawn, requires a test runner with parallel execution support (Detox CLI currently supports Jest) |
|&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;||


> NOTE: such log levels as `verbose`, `silly`, `wss` are deprecated since detox@8.1.0 and will be removed in 9.0.0.

### build
Run a command defined in 'configuration.build'
Expand All @@ -71,20 +71,23 @@ Run a command defined in 'configuration.build'
| --- | --- |
| -h, --help | output usage information |
| -c, --configuration \<device config\> | Select a device configuration from your defined configurations,if not supplied, and there's only one configuration, detox will default to it |
|&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;&#8195;||


### run-server
Start a standalone detox server

Start a standalone Detox server

`detox run-server [options]`

| Option | Description |
| Option | Description |
| --- | --- |
| -h, --help | output usage information |

| -p, --port [port] | Port number (default: 8099) |
| -l, --loglevel [value] | Log level: fatal, error, warn, info, debug, trace |
| --no-color | Disable colorful logs |
| -h, --help | output usage information |

### init

Scaffolds initial E2E test folder structure for a specific test runner

`detox init -r <test-runner-name>`
Expand Down
2 changes: 1 addition & 1 deletion docs/More.AndroidSupportStatus.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ Type 'android.attached' could be used to connect to any of already attached devi
3. Location - Coming soon

## Debugging
1. `--loglevel verbose` can give you pretty good insight on what going on.
1. `--loglevel trace` can give you pretty good insight on what going on.
2. `--debug-synchronization [ms]`, our tool to identify synchronization issues works on Android too.

## Cross platform support
Expand Down
8 changes: 4 additions & 4 deletions docs/Troubleshooting.Flakiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,20 @@ Assume you have a suite of 100 tests and each test is flaky in 0.5% of execution

It's important to identify the various sources of flakiness in Detox tests.

* Control of the device / simulator - in order to run your tests, Detox must communicate with a simulator and instruct it to install the app, restart it, etc. Simulators don't always behave and controlling them might occasionally fail.<br> Detox's underlying simulator control is [`AppleSimulatorUtils`](https://github.com/wix/AppleSimulatorUtils), it is a tool that supports both basic and advanced simulator and device interaction options, it uses some core simulator features which are not always stable and may need time to "warm up" (booting, shutting down, etc.). Detox is set to have a few retries on any of these actions before failing. It will also print all the `exec` commands when using verbose log level.
* Control of the device / simulator - in order to run your tests, Detox must communicate with a simulator and instruct it to install the app, restart it, etc. Simulators don't always behave and controlling them might occasionally fail.<br> Detox's underlying simulator control is [`AppleSimulatorUtils`](https://github.com/wix/AppleSimulatorUtils), it is a tool that supports both basic and advanced simulator and device interaction options, it uses some core simulator features which are not always stable and may need time to "warm up" (booting, shutting down, etc.). Detox is set to have a few retries on any of these actions before failing. It will also print all the `exec` commands when using `debug` log level, and with `trace` level it will print everything.

* Asynchronous operations inside your app - every time an E2E test runs, operations might take place in a different order inside your app. This makes E2E tests nondeterministic. Consider an HTTP request made to a server, this request may take a variable time to complete due to external concerns like network congestion and server load.<br>Detox takes this into account by monitoring all asynchronous operations that take place in your app from the inside. Detox knows which network requests are currently in-flight. Detox knows how busy the React Native bridge is. Tests are automatically synchronized to the app and only move forward when the app is idle.

### 3. Get More Data About the Problem

In order to identify the source of flakiness you're suffering from you need more data. If you catch a failing test that should be passing, you need to record as much information as possible in order to investigate.

* Enable verbose mode in Detox. This will output a lot of information about what happening during the test.<br>
* Enable `trace` mode in Detox. This will output a lot of information about what happening during the test.<br>
1. `exec` commands
2. All communication going over the websocket, both from tester and testee

To enable verbose mode run your tests in verbose log mode:
To enable `trace` mode run your tests in trace log mode:

```sh
detox test --loglevel verbose
detox test --loglevel trace
```
10 changes: 5 additions & 5 deletions docs/Troubleshooting.RunningTests.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
id: Troubleshooting.RunningTests
title: Failing Tests
---
* [Enable verbose mode](#enable-verbose-mode)
* [Enable trace mode](#enable-trace-mode)
* [Syntax Error: Unxpected Token](#syntax-error-unxpected-token)
* [Can't find my component even though I added testID to its props](#cant-find-my-component-even-though-i-added-testid-to-its-props)
* [Test tries to find my component before it's created](#test-tries-to-find-my-component-before-its-created)
Expand All @@ -15,12 +15,12 @@ title: Failing Tests

<br>

### Enable verbose mode
### Enable trace mode

It's a good idea to get as much information as possible about what's going on. We can enable verbose mode during tests by running our tests with:
It's a good idea to get as much information as possible about what's going on. We can enable trace mode during tests by running our tests with:

```
detox test --loglevel verbose
detox test --loglevel trace
```


Expand Down Expand Up @@ -210,7 +210,7 @@ Include the following information in your issue to increase the chances of resol

1. Versions of all dependencies - iOS version you're working on, simulator model, React Native version, Detox version, etc

2. The verbose log of the test (see above)
2. The trace log of the test (see above)

3. Source code of your test scenario

Expand Down