Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Labels adjustments #181371

Merged
merged 9 commits into from
Apr 24, 2024
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -475,7 +475,8 @@ ${JSON.stringify(argv, null, 2)}
!process.env.KIBANA_MKI_USE_LATEST_COMMIT ||
process.env.KIBANA_MKI_USE_LATEST_COMMIT !== '1'
) {
cypressConfigFile.env.grepTags = '@serverlessQA --@skipInServerless';
cypressConfigFile.env.grepTags =
'@serverlessQA --@skipInServerless --@skipInServerlessMKI ';
}
const tier: string = argv.tier;
const endpointAddon: boolean = argv.endpointAddon;
Expand Down
1 change: 1 addition & 0 deletions x-pack/test/security_solution_cypress/cypress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ Note that we use tags in order to select which tests we want to execute:
- `@serverlessQA` includes a test in the Serverless test suite for the Kibana release process of serverless. You need to explicitly add this tag to any test you want yo run in CI for the second quality gate. These tests should be stable, otherviswe they will be blocking the release pipeline. They should be alsy critical enough, so that when they fail, there's a high chance of an SDH or blocker issue to be reported.
- `@ess` includes a test in the normal, non-Serverless test suite. You need to explicitly add this tag to any test you want to run against a non-Serverless environment.
- `@skipInEss` excludes a test from the non-Serverless test suite. The test will not be executed as part for the PR process. All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
- `@skipInServerlessMKI` excludes a test from the execution on any MKI environment (even if it's tagged as `@serverless` or `@serverlessQA`). Could indicate many things, e.g. "the test is flaky in Serverless MKI", "the test has been temporarily excluded, see the comment above why". All the skipped tests should have a link to a ticket describing the reason why the test got skipped.
- `@skipInServerless` excludes a test from the Serverless test suite and Serverless QA environment for both, periodic pipeline and second quality gate (even if it's tagged as `@serverless`). Could indicate many things, e.g. "the test is flaky in Serverless", "the test is Flaky in any type of environment", "the test has been temporarily excluded, see the comment above why". All the skipped tests should have a link to a ticket describing the reason why the test got skipped.

Please, before opening a PR with a new test, make sure that the test fails. If you never see your test fail you don’t know if your test is actually testing the right thing, or testing anything at all.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export default defineCypressConfig({
env: {
grepFilterSpecs: true,
grepOmitFiltered: true,
grepTags: '@serverless --@skipInServerless',
grepTags: '@serverless --@skipInServerless --@skipInServerlessMKI',
},
execTimeout: 300000,
pageLoadTimeout: 300000,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { CREATE_RULE_URL } from '../../../../urls/navigation';
describe(
'Detection rules, Alert Suppression for Essentials tier',
{
tags: ['@serverless'],
tags: ['@serverless', '@skipServerlessMKI'],
env: {
ftrConfig: {
productTypes: [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ const SUPPRESS_BY_FIELDS = ['agent.type'];
describe(
'Detection Rule Creation - EQL Rules - With Alert Suppression',
{
tags: ['@ess', '@serverless', '@skipInServerless'],
tags: ['@ess', '@serverless', '@skipServerlessMKI'],
// alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
env: {
ftrConfig: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import {
describe(
'Detection Rule Creation - EQL Rules - With Alert Suppression',
{
tags: ['@ess', '@serverless', '@skipInServerless'],
tags: ['@ess', '@serverless', '@skipServerlessMKI'],
// alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
env: {
ftrConfig: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ const SUPPRESS_BY_FIELDS = ['agent.type'];
describe(
'Detection Rule Creation - EQL Rules - With Alert Suppression - Serverless Essentials License',
{
tags: ['@serverless', '@skipInServerless'],
tags: ['@serverless', '@skipServerlessMKI'],
// alertSuppressionForNonSequenceEqlRuleEnabled feature flag is also enabled in a global config
env: {
ftrConfig: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ import { openRuleManagementPageViaBreadcrumbs } from '../../../../tasks/rules_ma
describe(
'New Terms rules',
{
tags: ['@ess', '@serverless'],
tags: ['@ess', '@serverless', '@skipServerlessMKI'],
env: {
// alertSuppressionForNewTermsRuleEnabled feature flag is also enabled in a global config
kbnServerArgs: [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ const rule = getNewTermsRule();
describe(
'Detection rules, New terms, Edit',
{
tags: ['@ess', '@serverless'],
tags: ['@ess', '@serverless', '@skipServerlessMKI'],
env: {
// alertSuppressionForNewTermsRuleEnabled feature flag is also enabled in a global config
kbnServerArgs: [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ import { RULES_MANAGEMENT_URL } from '../../../../urls/rules_management';
describe(
'Value list items',
{
tags: ['@ess', '@serverless', '@skipInServerless'],
tags: ['@ess', '@serverless', '@skipServerlessMKI'],
env: {
ftrConfig: {
kbnServerArgs: [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ const loginPageAsWriteAuthorizedUser = (url: string) => {
// https://github.com/elastic/kibana/issues/179965
describe(
'Detection rules, Prebuilt Rules Installation and Update - Authorization/RBAC',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
beforeEach(() => {
preventPrebuiltRulesPackageInstallation();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ import { visitRulesManagementTable } from '../../../../tasks/rules_management';
// https://github.com/elastic/kibana/issues/179970
describe(
'Detection rules, Prebuilt Rules Installation and Update - Error handling',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
beforeEach(() => {
preventPrebuiltRulesPackageInstallation();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ import { deleteAlertsAndRules } from '../../../../tasks/api_calls/common';

describe(
'Detection rules, Prebuilt Rules Installation and Update workflow',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
() => {
describe('Installation of prebuilt rules', () => {
const RULE_1 = createRuleAssetSavedObject({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ const rules = Array.from(Array(5)).map((_, i) => {
});

// https://github.com/elastic/kibana/issues/179973
describe('Prebuilt rules', { tags: ['@ess', '@serverless', '@skipInServerless'] }, () => {
describe('Prebuilt rules', { tags: ['@ess', '@serverless', '@skipInServerlessMKI'] }, () => {
beforeEach(() => {
login();
deleteAlertsAndRules();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ describe(
// https://github.com/elastic/kibana/issues/179967
it(
'should NOT display install or update notifications when latest rules are installed',
{ tags: ['@skipInServerless'] },
{ tags: ['@skipInServerlessMKI'] },
() => {
visitRulesManagementTable();
createAndInstallMockedPrebuiltRules([RULE_1]);
Expand All @@ -72,7 +72,7 @@ describe(
});

// https://github.com/elastic/kibana/issues/179968
describe('Notifications', { tags: ['@skipInServerless'] }, () => {
describe('Notifications', { tags: ['@skipInServerlessMKI'] }, () => {
beforeEach(() => {
installPrebuiltRuleAssets([RULE_1]);
});
Expand Down
Loading