[7.x] Improve short-url redirect validation (#84366)

[jportner]

Backports the following commits to 7.x:

• Improve short-url redirect validation (Improve short-url redirect validation #84366)

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request
• Commit: 409ec7e
• Flaky suites:
  • xpack-kibana-ciGroup6

---

Test Failures

X-Pack API Integration Tests.x-pack/test/api_integration/apis/security_solution/kpi_network·ts.apis SecuritySolution Endpoints Kpi Network With filebeat Make sure that we get KpiNetwork networkEvents data

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: apis
[00:00:00]           └-> "before all" hook
[00:05:10]           └-: SecuritySolution Endpoints
[00:05:10]             └-> "before all" hook
[00:05:14]             └-: Kpi Network
[00:05:14]               └-> "before all" hook
[00:05:14]               └-: With filebeat
[00:05:14]                 └-> "before all" hook
[00:05:14]                 └-> "before all" hook
[00:05:14]                   │ info [filebeat/default] Loading "mappings.json"
[00:05:14]                   │ info [filebeat/default] Loading "data.json.gz"
[00:05:14]                   │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1606406027810917752] [filebeat-7.0.0-iot-2019.06] creating index, cause [api], templates [], shards [1]/[0]
[00:05:14]                   │ info [filebeat/default] Created index "filebeat-7.0.0-iot-2019.06"
[00:05:14]                   │ debg [filebeat/default] "filebeat-7.0.0-iot-2019.06" settings {"index":{"lifecycle":{"name":"filebeat-7.0.0","rollover_alias":"filebeat-7.0.0"},"mapping":{"total_fields":{"limit":"10000"}},"number_of_replicas":"0","number_of_shards":"1","query":{"default_field":["tags","message","agent.version","agent.name","agent.type","agent.id","agent.ephemeral_id","client.address","client.mac","client.domain","client.geo.continent_name","client.geo.country_name","client.geo.region_name","client.geo.city_name","client.geo.country_iso_code","client.geo.region_iso_code","client.geo.name","cloud.provider","cloud.availability_zone","cloud.region","cloud.instance.id","cloud.instance.name","cloud.machine.type","cloud.account.id","container.runtime","container.id","container.image.name","container.image.tag","container.name","destination.address","destination.mac","destination.domain","destination.geo.continent_name","destination.geo.country_name","destination.geo.region_name","destination.geo.city_name","destination.geo.country_iso_code","destination.geo.region_iso_code","destination.geo.name","ecs.version","error.id","error.message","error.code","event.id","event.kind","event.category","event.action","event.outcome","event.type","event.module","event.dataset","event.hash","event.timezone","file.path","file.target_path","file.extension","file.type","file.device","file.inode","file.uid","file.owner","file.gid","file.group","file.mode","group.id","group.name","host.hostname","host.name","host.id","host.mac","host.type","host.architecture","host.os.platform","host.os.name","host.os.full","host.os.family","host.os.version","host.os.kernel","host.geo.continent_name","host.geo.country_name","host.geo.region_name","host.geo.city_name","host.geo.country_iso_code","host.geo.region_iso_code","host.geo.name","http.request.method","http.request.body.content","http.request.referrer","http.response.body.content","http.version","log.level","network.name","network.type","network.iana_number","network.transport","network.application","network.protocol","network.direction","network.community_id","observer.mac","observer.hostname","observer.vendor","observer.version","observer.serial_number","observer.type","observer.os.platform","observer.os.name","observer.os.full","observer.os.family","observer.os.version","observer.os.kernel","observer.geo.continent_name","observer.geo.country_name","observer.geo.region_name","observer.geo.city_name","observer.geo.country_iso_code","observer.geo.region_iso_code","observer.geo.name","organization.name","organization.id","os.platform","os.name","os.full","os.family","os.version","os.kernel","process.name","process.args","process.executable","process.title","process.working_directory","server.address","server.mac","server.domain","server.geo.continent_name","server.geo.country_name","server.geo.region_name","server.geo.city_name","server.geo.country_iso_code","server.geo.region_iso_code","server.geo.name","service.id","service.name","service.type","service.state","service.version","service.ephemeral_id","source.address","source.mac","source.domain","source.geo.continent_name","source.geo.country_name","source.geo.region_name","source.geo.city_name","source.geo.country_iso_code","source.geo.region_iso_code","source.geo.name","url.original","url.full","url.scheme","url.domain","url.path","url.query","url.fragment","url.username","url.password","user.id","user.name","user.full_name","user.email","user.hash","user.group.id","user.group.name","user_agent.original","user_agent.name","user_agent.version","user_agent.device.name","user_agent.os.platform","user_agent.os.name","user_agent.os.full","user_agent.os.family","user_agent.os.version","user_agent.os.kernel","agent.hostname","error.type","cloud.project.id","kubernetes.pod.name","kubernetes.pod.uid","kubernetes.namespace","kubernetes.node.name","kubernetes.container.name","kubernetes.container.image","log.file.path","log.source.address","stream","input.type","syslog.severity_label","syslog.facility_label","process.program","log.flags","user_agent.os.full_name","fileset.name","apache.access.ssl.protocol","apache.access.ssl.cipher","apache.error.module","user.terminal","user.audit.id","user.audit.name","user.audit.group.id","user.audit.group.name","user.effective.id","user.effective.name","user.effective.group.id","user.effective.group.name","user.filesystem.id","user.filesystem.name","user.filesystem.group.id","user.filesystem.group.name","user.owner.id","user.owner.name","user.owner.group.id","user.owner.group.name","user.saved.id","user.saved.name","user.saved.group.id","user.saved.group.name","auditd.log.old_auid","auditd.log.new_auid","auditd.log.old_ses","auditd.log.new_ses","auditd.log.items","auditd.log.item","auditd.log.tty","auditd.log.a0","elasticsearch.component","elasticsearch.cluster.uuid","elasticsearch.cluster.name","elasticsearch.node.id","elasticsearch.node.name","elasticsearch.index.name","elasticsearch.index.id","elasticsearch.shard.id","elasticsearch.audit.layer","elasticsearch.audit.origin.type","elasticsearch.audit.realm","elasticsearch.audit.user.realm","elasticsearch.audit.user.roles","elasticsearch.audit.action","elasticsearch.audit.url.params","elasticsearch.audit.indices","elasticsearch.audit.request.id","elasticsearch.audit.request.name","elasticsearch.gc.phase.name","elasticsearch.gc.tags","elasticsearch.slowlog.logger","elasticsearch.slowlog.took","elasticsearch.slowlog.types","elasticsearch.slowlog.stats","elasticsearch.slowlog.search_type","elasticsearch.slowlog.source_query","elasticsearch.slowlog.extra_source","elasticsearch.slowlog.total_hits","elasticsearch.slowlog.total_shards","elasticsearch.slowlog.routing","elasticsearch.slowlog.id","elasticsearch.slowlog.type","haproxy.frontend_name","haproxy.backend_name","haproxy.server_name","haproxy.bind_name","haproxy.error_message","haproxy.source","haproxy.termination_state","haproxy.mode","haproxy.http.response.captured_cookie","haproxy.http.response.captured_headers","haproxy.http.request.captured_cookie","haproxy.http.request.captured_headers","haproxy.http.request.raw_request_line","icinga.debug.facility","icinga.main.facility","icinga.startup.facility","iis.access.site_name","iis.access.server_name","iis.access.cookie","iis.error.reason_phrase","iis.error.queue_name","iptables.fragment_flags","iptables.input_device","iptables.output_device","iptables.tcp.flags","iptables.ubiquiti.input_zone","iptables.ubiquiti.output_zone","iptables.ubiquiti.rule_number","iptables.ubiquiti.rule_set","kafka.log.component","kafka.log.class","kafka.log.trace.class","kafka.log.trace.message","kibana.log.tags","kibana.log.state","logstash.log.module","logstash.log.thread","text","logstash.slowlog.module","logstash.slowlog.thread","text","logstash.slowlog.event","text","logstash.slowlog.plugin_name","logstash.slowlog.plugin_type","logstash.slowlog.plugin_params","text","mongodb.log.component","mongodb.log.context","mysql.slowlog.query","mysql.slowlog.schema","mysql.slowlog.current_user","mysql.slowlog.last_errno","mysql.slowlog.killed","mysql.slowlog.log_slow_rate_type","mysql.slowlog.log_slow_rate_limit","mysql.slowlog.innodb.trx_id","netflow.type","netflow.exporter.address","netflow.source_mac_address","netflow.post_destination_mac_address","netflow.destination_mac_address","netflow.post_source_mac_address","netflow.interface_name","netflow.interface_description","netflow.sampler_name","netflow.application_description","netflow.application_name","netflow.class_name","netflow.wlan_ssid","netflow.vr_fname","netflow.metro_evc_id","netflow.nat_pool_name","netflow.p2p_technology","netflow.tunnel_technology","netflow.encrypted_technology","netflow.observation_domain_name","netflow.selector_name","netflow.information_element_description","netflow.information_element_name","netflow.virtual_station_interface_name","netflow.virtual_station_name","netflow.sta_mac_address","netflow.wtp_mac_address","netflow.user_name","netflow.application_category_name","netflow.application_sub_category_name","netflow.application_group_name","netflow.dot1q_customer_source_mac_address","netflow.dot1q_customer_destination_mac_address","netflow.mib_context_name","netflow.mib_object_name","netflow.mib_object_description","netflow.mib_object_syntax","netflow.mib_module_name","netflow.mobile_imsi","netflow.mobile_msisdn","netflow.http_request_method","netflow.http_request_host","netflow.http_request_target","netflow.http_message_version","netflow.http_user_agent","netflow.http_content_type","netflow.http_reason_phrase","osquery.result.name","osquery.result.action","osquery.result.host_identifier","osquery.result.calendar_time","postgresql.log.timestamp","postgresql.log.database","postgresql.log.query","redis.log.role","redis.slowlog.cmd","redis.slowlog.key","redis.slowlog.args","santa.action","santa.decision","santa.reason","santa.mode","santa.disk.volume","santa.disk.bus","santa.disk.serial","santa.disk.bsdname","santa.disk.model","santa.disk.fs","santa.disk.mount","certificate.common_name","certificate.sha256","hash.sha256","suricata.eve.event_type","suricata.eve.app_proto_orig","suricata.eve.tcp.tcp_flags","suricata.eve.tcp.tcp_flags_tc","suricata.eve.tcp.state","suricata.eve.tcp.tcp_flags_ts","suricata.eve.fileinfo.sha1","suricata.eve.fileinfo.state","suricata.eve.fileinfo.sha256","suricata.eve.fileinfo.md5","suricata.eve.dns.type","suricata.eve.dns.rrtype","suricata.eve.dns.rrname","suricata.eve.dns.rdata","suricata.eve.dns.rcode","suricata.eve.flow_id","suricata.eve.email.status","suricata.eve.http.redirect","suricata.eve.http.protocol","suricata.eve.http.http_content_type","suricata.eve.in_iface","suricata.eve.alert.category","suricata.eve.alert.signature","suricata.eve.ssh.client.proto_version","suricata.eve.ssh.client.software_version","suricata.eve.ssh.server.proto_version","suricata.eve.ssh.server.software_version","suricata.eve.tls.issuerdn","suricata.eve.tls.sni","suricata.eve.tls.version","suricata.eve.tls.fingerprint","suricata.eve.tls.serial","suricata.eve.tls.subject","suricata.eve.app_proto_ts","suricata.eve.flow.state","suricata.eve.flow.reason","suricata.eve.app_proto_tc","suricata.eve.smtp.rcpt_to","suricata.eve.smtp.mail_from","suricata.eve.smtp.helo","suricata.eve.app_proto_expected","system.auth.ssh.method","system.auth.ssh.signature","system.auth.sudo.error","system.auth.sudo.tty","system.auth.sudo.pwd","system.auth.sudo.user","system.auth.sudo.command","system.auth.useradd.home","system.auth.useradd.shell","traefik.access.user_identifier","traefik.access.frontend_name","traefik.access.backend_url","zeek.session_id","zeek.connection.state","zeek.connection.history","zeek.connection.orig_l2_addr","zeek.resp_l2_addr","zeek.vlan","zeek.inner_vlan","zeek.dns.query","zeek.dns.qclass_name","zeek.dns.qtype_name","zeek.dns.rcode_name","zeek.dns.answers","zeek.http.status_msg","zeek.http.info_msg","zeek.http.filename","zeek.http.tags","zeek.http.proxied","zeek.http.client_header_names","zeek.http.server_header_names","zeek.http.orig_fuids","zeek.http.orig_mime_types","zeek.http.orig_filenames","zeek.http.resp_fuids","zeek.http.resp_mime_types","zeek.http.resp_filenames","zeek.files.fuid","zeek.files.session_ids","zeek.files.source","zeek.files.analyzers","zeek.files.mime_type","zeek.files.filename","zeek.files.parent_fuid","zeek.files.md5","zeek.files.sha1","zeek.files.sha256","zeek.files.extracted","zeek.ssl.version","zeek.ssl.cipher","zeek.ssl.curve","zeek.ssl.server_name","zeek.ssl.next_protocol","zeek.ssl.cert_chain","zeek.ssl.cert_chain_fuids","zeek.ssl.client_cert_chain","zeek.ssl.client_cert_chain_fuids","zeek.ssl.issuer","zeek.ssl.client_issuer","zeek.ssl.validation_status","zeek.ssl.subject","zeek.ssl.client_subject","zeek.ssl.last_alert","fields.*"]},"refresh_interval":"5s"}}
[00:05:15]                   │ info [filebeat/default] Indexed 6157 docs into "filebeat-7.0.0-iot-2019.06"
[00:05:15]                 └-> Make sure that we get KpiNetwork uniqueFlows data
[00:05:15]                   └-> "before each" hook: global before each
[00:05:15]                   └- ✓ pass  (27ms) "apis SecuritySolution Endpoints Kpi Network With filebeat Make sure that we get KpiNetwork uniqueFlows data"
[00:05:15]                 └-> Make sure that we get KpiNetwork networkEvents data
[00:05:15]                   └-> "before each" hook: global before each
[00:05:15]                   └- ✓ pass  (22ms) "apis SecuritySolution Endpoints Kpi Network With filebeat Make sure that we get KpiNetwork networkEvents data"
[00:05:15]                 └-> Make sure that we get KpiNetwork DNS data
[00:05:15]                   └-> "before each" hook: global before each
[00:05:15]                   └- ✓ pass  (20ms) "apis SecuritySolution Endpoints Kpi Network With filebeat Make sure that we get KpiNetwork DNS data"
[00:05:15]                 └-> Make sure that we get KpiNetwork networkEvents data
[00:05:15]                   └-> "before each" hook: global before each
[00:05:15]                   │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1606406027810917752] [.async-search] creating index, cause [api], templates [], shards [1]/[1]
[00:05:15]                   │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1606406027810917752] updating number_of_replicas to [0] for indices [.async-search]
[00:05:16]                   └- ✖ fail: apis SecuritySolution Endpoints Kpi Network With filebeat Make sure that we get KpiNetwork networkEvents data
[00:05:16]                   │       Error: expected 0 to sort of equal 6157
[00:05:16]                   │       + expected - actual
[00:05:16]                   │ 
[00:05:16]                   │       -0
[00:05:16]                   │       +6157
[00:05:16]                   │       
[00:05:16]                   │       at Assertion.assert (/dev/shm/workspace/parallel/19/kibana/packages/kbn-expect/expect.js:100:11)
[00:05:16]                   │       at Assertion.eql (/dev/shm/workspace/parallel/19/kibana/packages/kbn-expect/expect.js:244:8)
[00:05:16]                   │       at Context.<anonymous> (test/api_integration/apis/security_solution/kpi_network.ts:144:45)
[00:05:16]                   │       at Object.apply (/dev/shm/workspace/parallel/19/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:84:16)
[00:05:16]                   │ 
[00:05:16]                   │ 

Stack Trace

Error: expected 0 to sort of equal 6157
    at Assertion.assert (/dev/shm/workspace/parallel/19/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/19/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.<anonymous> (test/api_integration/apis/security_solution/kpi_network.ts:144:45)
    at Object.apply (/dev/shm/workspace/parallel/19/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:84:16) {
  actual: '0',
  expected: '6157',
  showDiff: true
}

---

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[jportner]

Flaky test looks completely unrelated, merging!