Closed
Description
This test failed on my [PR build]. The reproduction line is:
05:17:22 2> REPRODUCE WITH: ./gradlew :x-pack:plugin:security:unitTest -Dtests.seed=D84FAE5E21AE11F0 -Dtests.class=org.elasticsearch.xpack.security.authc.ApiKeyIntegTests -Dtests.method="testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey" -Dtests.security.manager=true -Dtests.locale=es-GT -Dtests.timezone=America/Boise -Dcompiler.java=11 -Druntime.java=8
Here are some logs:
05:17:22 1> [2019-02-05T03:17:13,251][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] before test
05:17:22 1> [2019-02-05T03:17:13,251][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] [ApiKeyIntegTests#testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey]: setting up test
05:17:22 1> [2019-02-05T03:17:13,262][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/nodes/liveness" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:47336" origin.type="transport" request.id="8vfpOGzNSbaSlgyMNIYKtw" request.name="LivenessRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,265][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="indices:admin/delete" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="["*","-*"]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38398" origin.type="transport" request.id="IKekdxmqSWKLhIG7skdfyQ" request.name="DeleteIndexRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,265][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="indices:admin/delete" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="["*","-*"]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38398" origin.type="transport" request.id="IKekdxmqSWKLhIG7skdfyQ" request.name="DeleteIndexRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,274][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="indices:admin/template/put" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="["*"]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38400" origin.type="transport" request.id="JYMRBd9tRqWv2kTADv9YPA" request.name="PutIndexTemplateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,274][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="indices:admin/template/put" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="["*"]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38400" origin.type="transport" request.id="JYMRBd9tRqWv2kTADv9YPA" request.name="PutIndexTemplateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,277][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node_sm0] adding template [random_index_template] for index patterns [*]
05:17:22 1> [2019-02-05T03:17:13,299][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] [ApiKeyIntegTests#testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey]: all set up test
05:17:22 1> [2019-02-05T03:17:13,300][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/nodes/info" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38402" origin.type="transport" request.id="J41YgF6HRzuv-fNsQAN2EQ" request.name="NodesInfoRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,300][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/nodes/info" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38402" origin.type="transport" request.id="J41YgF6HRzuv-fNsQAN2EQ" request.name="NodesInfoRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,300][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/nodes/info[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:40373" origin.type="local_node" request.id="J41YgF6HRzuv-fNsQAN2EQ" request.name="NodeInfoRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,301][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/nodes/info[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35280" origin.type="transport" request.id="J41YgF6HRzuv-fNsQAN2EQ" request.name="NodeInfoRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,377][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/state" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38404" origin.type="transport" request.id="5TrULZfDSJGHN-Jvq9QUHw" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,378][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/state" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38404" origin.type="transport" request.id="5TrULZfDSJGHN-Jvq9QUHw" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,469][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/state" event.action="access_granted" event.type="transport" indices="[]" origin.address="127.0.0.1:36016" origin.type="transport" request.id="gG-TqB2rTSWNJampsC_GYQ" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,469][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/state" event.action="access_granted" event.type="transport" indices="[]" origin.address="127.0.0.1:36016" origin.type="transport" request.id="gG-TqB2rTSWNJampsC_GYQ" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:13,689][INFO ][o.e.c.m.MetaDataCreateIndexService] [node_sm0] [.security-6] creating index, cause [api], templates [random_index_template], shards [1]/[0], mappings [doc]
05:17:22 1> [2019-02-05T03:17:13,689][INFO ][o.e.c.r.a.AllocationService] [node_sm0] updating number_of_replicas to [1] for indices [.security-6]
05:17:22 1> [2019-02-05T03:17:13,713][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35346" origin.type="transport" request.id="wJ8H2gPmRLitONf82Sxg7Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,714][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38224" origin.type="transport" request.id="wJ8H2gPmRLitONf82Sxg7Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,714][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:39965" origin.type="local_node" request.id="qaDccyxyTvGmVDSw7EiRqw" request.name="ClearRealmCacheRequest" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,716][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38006" origin.type="transport" request.id="FTizYvvpRDOY4UF8YekK0Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,716][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:37940" origin.type="transport" request.id="qaDccyxyTvGmVDSw7EiRqw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,716][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:39965" origin.type="local_node" request.id="qaDccyxyTvGmVDSw7EiRqw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,717][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:40373" origin.type="local_node" request.id="D8CKn1aNRkaMWnDTjjKhhA" request.name="ClearRealmCacheRequest" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,717][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35180" origin.type="transport" request.id="FTizYvvpRDOY4UF8YekK0Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,718][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35288" origin.type="transport" request.id="D8CKn1aNRkaMWnDTjjKhhA" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,720][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:40373" origin.type="local_node" request.id="D8CKn1aNRkaMWnDTjjKhhA" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,721][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35160" origin.type="transport" request.id="oDMDWmf1RYWbdar2T9Lp1Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,721][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38100" origin.type="transport" request.id="oDMDWmf1RYWbdar2T9Lp1Q" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,743][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38318" origin.type="transport" request.id="EJrVpjo0SnWt6WMk2oChwA" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,743][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35318" origin.type="transport" request.id="EJrVpjo0SnWt6WMk2oChwA" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,763][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35216" origin.type="transport" request.id="83AcT2a1S6W3Nk5heXTc6g" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,767][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:37926" origin.type="transport" request.id="83AcT2a1S6W3Nk5heXTc6g" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:13,853][INFO ][o.e.c.r.a.AllocationService] [node_sm0] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.security-6][0]] ...]).
05:17:22 1> [2019-02-05T03:17:14,067][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/nodes/liveness" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:50076" origin.type="transport" request.id="awlsePjsRh6JYOSnDowkmw" request.name="LivenessRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:14,334][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/nodes/liveness" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:47686" origin.type="transport" request.id="nniQNuG4Smqd3qssDvgvlw" request.name="LivenessRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,074][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/nodes/liveness" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:50444" origin.type="transport" request.id="oYXL7SybRjui-2yqcvLhwQ" request.name="LivenessRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,133][INFO ][o.e.c.m.MetaDataDeleteIndexService] [node_sm0] [.security-6/vbuoEL2pT72weFdTR28x0Q] deleting index
05:17:22 1> [2019-02-05T03:17:15,144][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:40373" origin.type="local_node" request.id="VYU_AsQ6ShS7-8-1dhts8w" request.name="ClearRealmCacheRequest" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,145][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:39965" origin.type="local_node" request.id="bjFxNzf8TiaXo1NpkwUjVQ" request.name="ClearRealmCacheRequest" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,145][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:40373" origin.type="local_node" request.id="VYU_AsQ6ShS7-8-1dhts8w" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,146][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:39965" origin.type="local_node" request.id="bjFxNzf8TiaXo1NpkwUjVQ" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,147][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:37940" origin.type="transport" request.id="bjFxNzf8TiaXo1NpkwUjVQ" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,151][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35280" origin.type="transport" request.id="VYU_AsQ6ShS7-8-1dhts8w" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,151][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35180" origin.type="transport" request.id="tGBILR6YTn268uprSMgq4w" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,152][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38006" origin.type="transport" request.id="tGBILR6YTn268uprSMgq4w" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,160][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35160" origin.type="transport" request.id="zfSVabD1SrG5mUu6iFYyrw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,160][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38100" origin.type="transport" request.id="zfSVabD1SrG5mUu6iFYyrw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,163][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38318" origin.type="transport" request.id="ndN_SKbySCi9cvoqaT3WJw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,164][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35318" origin.type="transport" request.id="ndN_SKbySCi9cvoqaT3WJw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,169][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38216" origin.type="transport" request.id="94EHdriEQWaufxIw7xjObw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,169][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35338" origin.type="transport" request.id="94EHdriEQWaufxIw7xjObw" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,180][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:37922" origin.type="transport" request.id="WimfARS8RmmI7eTGi-gBFQ" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,181][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] [ApiKeyIntegTests#testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey]: cleaning up after test
05:17:22 1> [2019-02-05T03:17:15,186][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:admin/xpack/security/realm/cache/clear[n]" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:35210" origin.type="transport" request.id="WimfARS8RmmI7eTGi-gBFQ" request.name="Node" user.name="_xpack_security" user.realm="__attach" user.roles="["superuser"]"
05:17:22 1> [2019-02-05T03:17:15,300][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/health" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38406" origin.type="transport" request.id="wHuqWeTXT6a0P09VsPqkdg" request.name="ClusterHealthRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,300][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/health" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38406" origin.type="transport" request.id="wHuqWeTXT6a0P09VsPqkdg" request.name="ClusterHealthRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,367][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/state" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38408" origin.type="transport" request.id="ZsXLndPcTfa99k6bmTzIeQ" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,367][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sc6] action="cluster:monitor/state" event.action="access_granted" event.type="transport" host.ip="127.0.0.1" host.name="127.0.0.1" indices="[]" node.id="yCxkPJjtTryFjOKuvn-d-A" node.name="node_sc6" origin.address="127.0.0.1:38408" origin.type="transport" request.id="ZsXLndPcTfa99k6bmTzIeQ" request.name="ClusterStateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,419][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="cluster:monitor/nodes/liveness" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:48158" origin.type="transport" request.id="gBMRGW0lTxiozC3s7e77bw" request.name="LivenessRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,681][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] action="cluster:monitor/state" event.action="access_granted" event.type="transport" indices="[]" origin.address="127.0.0.1:39965" origin.type="local_node" request.id="V01OGnZRQ1O4fXoyQU6QVg" request.name="ClusterStateRequest" user.name="test_user" user.realm="file" user.roles="["user"]"
05:17:22 1> [2019-02-05T03:17:15,791][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="indices:admin/template/delete" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:36018" origin.type="transport" request.id="qHKReoZ7Tii2A2bIXNETqA" request.name="DeleteIndexTemplateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,791][INFO ][o.e.x.s.a.l.LoggingAuditTrail] [node_sm1] action="indices:admin/template/delete" event.action="access_granted" event.type="transport" origin.address="127.0.0.1:36018" origin.type="transport" request.id="qHKReoZ7Tii2A2bIXNETqA" request.name="DeleteIndexTemplateRequest" user.name="test_trans_client_user" user.realm="file" user.roles="["transport_client","user"]"
05:17:22 1> [2019-02-05T03:17:15,792][INFO ][o.e.c.m.MetaDataIndexTemplateService] [node_sm0] removing template [random_index_template]
05:17:22 1> [2019-02-05T03:17:15,821][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] [ApiKeyIntegTests#testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey]: cleaned up after test
05:17:22 1> [2019-02-05T03:17:15,821][INFO ][o.e.x.s.a.ApiKeyIntegTests] [testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey] after test
05:17:22 FAILURE 2.92s J5 | ApiKeyIntegTests.testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey <<< FAILURES!
05:17:22 > Throwable #1: java.lang.AssertionError:
05:17:22 > Expected: <1>
05:17:22 > but: was <0>
05:17:22 > at __randomizedtesting.SeedInfo.seed([D84FAE5E21AE11F0:82015FEC9C3DD7A3]:0)
05:17:22 > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
05:17:22 > at org.elasticsearch.xpack.security.authc.ApiKeyIntegTests.verifyInvalidateResponse(ApiKeyIntegTests.java:278)
05:17:22 > at org.elasticsearch.xpack.security.authc.ApiKeyIntegTests.testGetAndInvalidateApiKeysWithExpiredAndInvalidatedApiKey(ApiKeyIntegTests.java:261)
05:17:22 > at java.lang.Thread.run(Thread.java:748)