Skip to content

Commit

Permalink
support other app UITests
Browse files Browse the repository at this point in the history
  • Loading branch information
@codeskyblue
codeskyblue committed Mar 1, 2021
1 parent fc5ac0e commit bef6c49
Show file tree
Hide file tree
Showing 6 changed files with 145 additions and 34 deletions.
45 changes: 45 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,51 @@ c = wda.Client("http://localhost:8200")
print(c.info)
```

### 运行XCTest UITest
这个不是Unit Tests，而是UITests。具体可以看这里的解释说明 <https://fbidb.io/docs/test-execution>

以这个项目为例: https://github.com/FeiHuang93/XCTest-Demo
应用分为执行测试的应用 testXCTestUITests 和 被测应用 testXCTest

执行方法

```bash
$ tidevice xctest --bundle-id philhuang.testXCTestUITests.xctrunner --target-bundle-id philhuang.testXCTest
# ... 省略一部分不重要的信息 ...
[I 210301 15:37:07 _device:887] logProcess: 2021-03-01 15:37:07.924620+0800 testXCTestUITests-Runner[81644:13765443] Running tests...
[I 210301 15:37:07 _device:984] Test runner ready detected
[I 210301 15:37:07 _device:976] Start execute test plan with IDE version: 29
[I 210301 15:37:07 _device:887] logProcess: Test Suite 'All tests' started at 2021-03-01 15:37:08.009
XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: Test Suite 'testXCTestUITests.xctest' started at 2021-03-01 15:37:08.010
XCTestOutputBarrierTest Suite 'testXCTestUITests' started at 2021-03-01 15:37:08.010
[I 210301 15:37:07 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: Test Case '-[testXCTestUITests testExample]' started.
XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Start Test at 2021-03-01 15:37:08.010
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Set Up
[I 210301 15:37:07 _device:887] logProcess: 2021-03-01 15:37:08.010828+0800 testXCTestUITests-Runner[81644:13765443] testExample start
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Open philhuang.testXCTest
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Launch philhuang.testXCTest
[I 210301 15:37:08 _device:887] logProcess: t = 0.04s Wait for accessibility to load
[I 210301 15:37:08 _device:887] logProcess: t = 0.04s Setting up automation session
[I 210301 15:37:08 _device:887] logProcess: t = 0.10s Wait for philhuang.testXCTest to idle
[I 210301 15:37:09 _device:887] logProcess: t = 1.13s Tear Down
[I 210301 15:37:09 _device:887] logProcess: Test Case '-[testXCTestUITests testExample]' passed (1.337 seconds).
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'testXCTestUITests' passed at 2021-03-01 15:37:09.349.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.339) seconds
XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'testXCTestUITests.xctest' passed at 2021-03-01 15:37:09.350.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.340) seconds
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'All tests' passed at 2021-03-01 15:37:09.352.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.343) seconds
XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:1059] xctrunner quited
```

### 挂载开发者镜像
这个步骤其实不太需要，因为如果tidevice的命令需要开发者镜像的时候，会自动去挂载的

Expand Down
43 changes: 43 additions & 0 deletions README_EN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,49 @@ c = wda.Client("http://localhost:8200")
print(c.info)
```

### Run UITests
Demo <https://github.com/FeiHuang93/XCTest-Demo>

- `philhuang.testXCTestUITests.xctrunner` is the test to launch
- `philhuang.testXCTest` is the app to test

```bash
$ tidevice xctest --bundle-id philhuang.testXCTestUITests.xctrunner --target-bundle-id philhuang.testXCTest
# ... ignore some not important part ...
[I 210301 15:37:07 _device:887] logProcess: 2021-03-01 15:37:07.924620+0800 testXCTestUITests-Runner[81644:13765443] Running tests...
[I 210301 15:37:07 _device:984] Test runner ready detected
[I 210301 15:37:07 _device:976] Start execute test plan with IDE version: 29
[I 210301 15:37:07 _device:887] logProcess: Test Suite 'All tests' started at 2021-03-01 15:37:08.009
XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: Test Suite 'testXCTestUITests.xctest' started at 2021-03-01 15:37:08.010
XCTestOutputBarrierTest Suite 'testXCTestUITests' started at 2021-03-01 15:37:08.010
[I 210301 15:37:07 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: Test Case '-[testXCTestUITests testExample]' started.
XCTestOutputBarrier
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Start Test at 2021-03-01 15:37:08.010
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Set Up
[I 210301 15:37:07 _device:887] logProcess: 2021-03-01 15:37:08.010828+0800 testXCTestUITests-Runner[81644:13765443] testExample start
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Open philhuang.testXCTest
[I 210301 15:37:07 _device:887] logProcess: t = 0.00s Launch philhuang.testXCTest
[I 210301 15:37:08 _device:887] logProcess: t = 0.04s Wait for accessibility to load
[I 210301 15:37:08 _device:887] logProcess: t = 0.04s Setting up automation session
[I 210301 15:37:08 _device:887] logProcess: t = 0.10s Wait for philhuang.testXCTest to idle
[I 210301 15:37:09 _device:887] logProcess: t = 1.13s Tear Down
[I 210301 15:37:09 _device:887] logProcess: Test Case '-[testXCTestUITests testExample]' passed (1.337 seconds).
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'testXCTestUITests' passed at 2021-03-01 15:37:09.349.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.339) seconds
XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'testXCTestUITests.xctest' passed at 2021-03-01 15:37:09.350.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.340) seconds
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: Test Suite 'All tests' passed at 2021-03-01 15:37:09.352.
Executed 1 test, with 0 failures (0 unexpected) in 1.337 (1.343) seconds
XCTestOutputBarrier
[I 210301 15:37:09 _device:887] logProcess: XCTestOutputBarrier
[I 210301 15:37:09 _device:1059] xctrunner quited
```

### Mount DeveloperDiskImage
```bash
# Find in /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport/
Expand Down
19 changes: 12 additions & 7 deletions tidevice/__main__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ def cmd_list(args: argparse.Namespace):
print(udid, name)
result.append(dict(udid=udid, name=name))
if args.json:
print(json.dumps(result, indent=4))
print(json.dumps(result, indent=4, ensure_ascii=False))


def cmd_device_info(args: argparse.Namespace):
Expand Down Expand Up @@ -207,7 +207,10 @@ def cmd_xctest(args: argparse.Namespace):
env[key] = val
if env:
logger.info("Launch env: %s", env)
d.xctest(args.bundle_id, logger=setup_logger(level=logging.INFO), env=env)
d.xctest(args.bundle_id,
target_bundle_id=args.target_bundle_id,
logger=setup_logger(level=logging.INFO),
env=env)


def cmd_screenshot(args: argparse.Namespace):
Expand Down Expand Up @@ -290,7 +293,7 @@ def cmd_relay(args: argparse.Namespace):
def cmd_wdaproxy(args: argparse.Namespace):
""" start xctest and relay """
d = _udid2device(args.udid)

serv = WDAService(d, args.bundle_id)
p = None
if args.port:
Expand All @@ -299,7 +302,7 @@ def cmd_wdaproxy(args: argparse.Namespace):
str(args.port), '8100'
]
p = subprocess.Popen(cmds, stdout=sys.stdout, stderr=sys.stderr)

try:
serv.start()
while serv._service.running:
Expand All @@ -312,7 +315,7 @@ def cmd_wdaproxy(args: argparse.Namespace):
def cmd_syslog(args: argparse.Namespace):
d = _udid2device(args.udid)
s = d.start_service("com.apple.syslog_relay")
# print("SS")
# print("SS")
try:
while True:
text = s.recv().decode('utf-8')
Expand Down Expand Up @@ -382,9 +385,11 @@ def cmd_test(args: argparse.Namespace):
dict(action=cmd_xctest,
command="xctest",
flags=[
dict(args=['-B', '--bundle_id'],
dict(args=['-B', '--bundle_id', '--bundle-id'],
default="com.facebook.*.xctrunner",
help="test application bundle id"),
help="bundle id of the test to launch"),
dict(args=['--target-bundle-id'],
help='bundle id of the target app [optional]'),
dict(args=['-I', '--install-wda'],
action='store_true',
help='install webdriveragent app'),
Expand Down
30 changes: 23 additions & 7 deletions tidevice/_device.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -768,6 +768,7 @@ def _launch_app_runner(self,
bundle_id: str,
session_identifier: uuid.UUID,
env: dict = {},
target_app_bundle_id: str = None,
logger: logging.Logger = logging,
quit_event: threading.Event = None) -> int: # pid

Expand All @@ -787,6 +788,7 @@ def _launch_app_runner(self,
xctest_content = bplist.objc_encode(bplist.XCTestConfiguration({
"testBundleURL": bplist.NSURL(None, f"file://{app_info['Path']}/PlugIns/{target_name}.xctest"),
"sessionIdentifier": session_identifier,
"targetApplicationBundleID": target_app_bundle_id,
})) # yapf: disable

fsync = self.app_sync(bundle_id)
Expand All @@ -809,7 +811,7 @@ def _launch_app_runner(self,

xctestconfiguration_path = app_container + xctest_path # "/tmp/WebDriverAgentRunner-" + str(session_identifier).upper() + ".xctestconfiguration"
logger.debug("AppPath: %s", app_path)
logger.debug("AppContainer: %s", app_container)
logger.info("AppContainer: %s", app_container)
app_env = {
'CA_ASSERT_MAIN_THREAD_TRANSACTIONS': '0',
'CA_DEBUG_TRANSACTIONS': '0',
Expand All @@ -825,6 +827,8 @@ def _launch_app_runner(self,
# '__XPC_DYLD_LIBRARY_PATH': '/tmp/derivedDataPath/Build/Products/Release-iphoneos',
'MJPEG_SERVER_PORT': '',
'USE_PORT': '',
# maybe no needed
'LLVM_PROFILE_FILE': app_container + "/tmp/%p.profraw", # %p means pid
} # yapf: disable
app_env.update(env)

Expand Down Expand Up @@ -879,11 +883,17 @@ def _callback(m: DTXMessage):
if m.flags == 0x02:
method, args = m.result
if method == 'outputReceived:fromProcess:atTime:':
logger.debug("Output: %s", args[0].strip())
# logger.info("Output: %s", args[0].strip())
logger.info("logProcess: %s", args[0].rstrip())
# In low iOS versions, 'Using singleton test manager' may not be printed... mark wda launch status = True if server url has been printed
if "ServerURLHere" in args[0]:
logger.info("WebDriverAgent start successfully")

def _log_message_callback(m: DTXMessage):
identifier, args = m.result
logger.info("logConsole: %s", args)

conn.register_callback("_XCT_logDebugMessage:", _log_message_callback)
conn.register_callback(Event.NOTIFICATION, _callback)
if quit_event:
conn.register_callback(Event.FINISHED, lambda _: quit_event.set())
Expand All @@ -908,14 +918,18 @@ def _fnmatch_find_bundle_id(self, bundle_id: str) -> str:
key=lambda v: v != 'com.facebook.wda.irmarunner.xctrunner')
return bundle_ids[0]

def xctest(self, bundle_id="com.facebook.*.xctrunner", logger=None, env: dict={}):
def xctest(self, fuzzy_bundle_id="com.facebook.*.xctrunner", target_bundle_id=None, logger=None, env: dict={}):
"""
Launch xctrunner and wait until quit
Args:
target_bundle_id (str): optional, launch WDA-UITests will not need it
env: launch env
"""
if not logger:
logger = setup_logger(level=logging.INFO)

bundle_id = self._fnmatch_find_bundle_id(bundle_id)
bundle_id = self._fnmatch_find_bundle_id(fuzzy_bundle_id)
logger.info("BundleID: %s", bundle_id)

logger.info("DeviceIdentifier: %s", self.udid)
Expand Down Expand Up @@ -988,8 +1002,10 @@ def _show_log_message(m: DTXMessage):

# launch test app
# index: 1540

pid = self._launch_app_runner(bundle_id, session_identifier, env=env, logger=logger)
xclogger = setup_logger(name='xctest')
pid = self._launch_app_runner(bundle_id, session_identifier,
target_app_bundle_id=target_bundle_id,
env=env, logger=xclogger)

# xcode call the following commented method, twice
# but it seems can be ignored
Expand Down Expand Up @@ -1040,7 +1056,7 @@ def _show_log_message(m: DTXMessage):
# on windows threading.Event.wait can't handle ctrl-c
while not quit_event.wait(.1):
pass
logger.warning("xctrunner quited")
logger.info("xctrunner quited")


Device = BaseDevice
40 changes: 21 additions & 19 deletions tidevice/_instruments.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -528,9 +528,9 @@ def _handle_dtx_message(self, m: DTXMessage) -> bool:
if identifier == '_requestChannelWithCode:identifier:':
return self._reply_null(m)

if identifier == "_XCT_logDebugMessage:":
logger.debug("logDebugMessage: %s", args)
self._reply_null(m)
# if identifier == "_XCT_logDebugMessage:":
# logger.debug("logDebugMessage: %s", args)
# self._reply_null(m)

# logger.warning("Callback: identifier: %s", m.result) # TODO
if self._call_handlers(identifier, m):
Expand All @@ -557,7 +557,7 @@ def _drain(self):
try:
self._drain_single_message()
except MuxError as e:
logger.warning("unexpected error: %s", e)
# logger.warning("unexpected error: %s", e)
break
except:
if not self._stop_event.is_set():
Expand Down Expand Up @@ -585,26 +585,28 @@ def _drain_single_message(self):
channel_id=mheader.channel,
flags=flags,
result=result)
logger.debug("DTXMessage: %s", dtxm.result)

logger.debug("DTXMessage: expects_reply:%d flags:%d %s", mheader.expects_reply, dtxm.flags, dtxm.result)

if mheader.conversation_index == 1: # reply from server
self._reply_queues[mheader.message_id].put(dtxm)
elif mheader.conversation_index == 0:
# handle request
if mheader.expects_reply == 0: # notification from server
if not self._call_handlers(Event.NOTIFICATION, dtxm):
if dtxm.flags == 0x02 and dtxm.result[
0] == '_notifyOfPublishedCapabilities:':
# 公共方法消息，直接忽略
return
logger.debug(
"Ignore notification from server: %d, 0x%x, %s",
dtxm.message_id, dtxm.flags, dtxm.result)
return

handled = self._handle_dtx_message(dtxm)
if not handled:
logger.debug("server request not handled: %s", dtxm.result)
self._reply_null(dtxm)
if self._call_handlers(Event.NOTIFICATION, dtxm):
return
if dtxm.flags == 0x02 and dtxm.result[
0] == '_notifyOfPublishedCapabilities:':
# 公共方法消息，直接忽略
return
logger.debug(
"Ignore notification from server: %d, 0x%x, %s",
dtxm.message_id, dtxm.flags, dtxm.result)
else:
handled = self._handle_dtx_message(dtxm)
if not handled:
logger.debug("server request not handled: %s", dtxm.result)
self._reply_null(dtxm)
elif mheader.conversation_index == 2:
# usally NSError message
pass
Expand Down
2 changes: 1 addition & 1 deletion tidevice/bplist.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ class XCTestConfiguration(NSBaseObject):
'targetApplicationArguments': [], # maybe useless
'targetApplicationBundleID': None,
'targetApplicationEnvironment': None,
'targetApplicationPath': None,
'targetApplicationPath': "/whatever-it-does-not-matter/but-should-not-be-empty",
'testApplicationDependencies': {},
'testApplicationUserOverrides': None,
'testBundleRelativePath': None,
Expand Down

0 comments on commit bef6c49

Please sign in to comment.