Fix audit 2 findings

[aschemmel-tech]

In incubation phase the planned processes (areas safety mangement, change management, requirements, architecture and verification management) for S-CORE were audited (two sessions). The findings (no deviation / major) of the respective audit report need to be fixed before the first development release (incl. confirmation by the auditors). For better tracking those are copied here (in brackets the person planned to fix):

Safety Management:

• Action_15: The related page for safety culture / project communication cannot be found on the Eclipse web site. Cybersecurity is not yet addressed. The topic needs to be re-addressed. - see create safety management process description #329 - https://eclipse-score.github.io/score/main/platform_management_plan/safety_management.html#approach Cybersecurity: Improvement: Document (Cyber)-Security process #709
• Action_18: The related page for safety anomlaies cannot be found on the Eclipse web site. The topic needs to be re-addressed. see create safety management process description #329 - https://eclipse-score.github.io/score/main/process/process_areas/safety_management/guidance/guideline_safety_management.html
• Finding: The role descriptions in “SCORE Management Roles“ are not detailed enough. The roles and their responsibility shall be clearly and precisely described. - see https://eclipse-score.github.io/score/main/process/process_areas/safety_management/roles.html and https://eclipse-score.github.io/score/main/process/roles/index.html
• Finding: Not all roles and responsibilities are configured yet in with the eclipse environment yet. The linkage of roles to work products and process is missing - done for safety management roles, workproducts and workflows: create safety management process description #329 - https://eclipse-score.github.io/score/main/process/process_areas/safety_management/workflows.html
• Action_20: It need to be clarified which part of ISO 26262-4 §7 are judged to be applicable. (The related ISO Requirements need then to be added to the audit checklist below.) - tailored completely, commented in create safety management process description #329 - https://eclipse-score.github.io/score/main/platform_management_plan/safety_management.html (Tailoring)
• Action_21: The related page for Platform Safety Plan cannot be found on the Eclipse web site. The topic needs to be re-addressed. - see process: define safety_management plan #399 - https://eclipse-score.github.io/score/main/platform_management_plan/safety_management.html
• Action_22: A Safety Plan describes the objectives of the planning activity (to plan, to monitor and to measure), but it remains unclear how the objectives are reached - should be covered by issue planning, see PIM - Process Implementation Community
• Action: The tool evaluation is still in progress. (subject of later audit) - taken over to Improvement: Fix audit 4 findings #1085
• Action_5: it needs to be checked if the classification is ISO PAS 8926 compliant. - we think it is: https://eclipse-score.github.io/score/main/process/process_areas/safety_management/guidance/guideline_component_classification.html
• Action_6: A qualification process based on the software component’s classification is not yet described. - see module safety plan, now also referenced in the safety management guideline - start from https://eclipse-score.github.io/score/main/process/process_areas/safety_management/guidance/guideline_safety_management.html
• Action_7: The related page for the classification of software components cannot be found on the Eclipse web site. The topic needs to be re-addressed. - see create safety management process description #329 - ttps://eclipse-score.github.io/score/main/process/process_areas/safety_management/guidance/guideline_component_classification.html

Change Mgt:

• Action_47: Evidence missing that "A contributor cannot approve his own change request" applies to all roles. - for github there is only the committer role: show it's not allowed for a PR example

Requirements (prio 1):

• Action_8: Tool is not yet implemented to create the hash attribute - ticket see docs: Add versioned links docs-as-code#314
• Finding: It’s not clear when a requirement has been created or updated. It’s recommended to surface these attributes in Eclipse. - when a requirement is created it is in a branch/pull request. When it is "updated" after it was inspected it changes from valid(inspected) to valid - see https://eclipse-score.github.io/score/main/process/general_concepts/score_review_concept.html
• Finding: The content of “process/process_model/guidance/requirements/index.html#concept-description” cannot be found at “https://eclipse-score.github.io/score - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/requirements_concept.html
• Action_9: Requirements related to tools need to be audited. (subject of later audit)
• Action_10: AoU (Assumption of Use) requirements need to be audited. (subject of later audit)
• Action_11: The AoUs shall be automatically populated into the Safety Manuals. - https://eclipse-score.github.io/score/main/process/process_areas/safety_management/guidance/template_safety_manual.html (but currently no AoU in the repository)
• Action_12: It need to be clarified if the Safety Manuals for the Feature level are missing in the upper picture. - refers to https://eclipse-score.github.io/score/main/_images/wp_traceability_model.drawio.svg : feature level included in Platform Safety Manual (see also https://eclipse-score.github.io/score/main/process/process_areas/safety_management/workproducts.html)
• Finding: The requirements formulation template should include subjects for tool requirements - now called process requirements see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/guidance/requirements_templates.html and also example in "formulation template"
• Recommendation: A (tool supported) process is needed for the ASIL inheritance for decomposed requirements in future if ASIL B is not the maximum ASIL anymore. Hint: even in today’s safety concepts ASIL decomposition is sometimes used to reach ASIL B. (not relevant now)
• Finding: The requirement “GD_REQ_linkage_safety” shall be changed. The requirements that at least one parent requirement shall have the same ASIL level the downstream requirement shall be deleted. It might be the case that the platform supports higher ASILs than needed by a project. - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/guidance/requirements_process_reqs.html#gd_req__req__linkage_safety
• Finding: The Checklist question 2 of the requirements review checklist needs to be reformulated. The ISO terms of the characteristics are not suitable for a checklist. The checklist shall ask questions which then lead to the compliance of the requirement with being unambiguous, comprehensible, etc. - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/guidance/requirements_inspection_checklist.html
• Finding: The chapter Requirements Guidelines need to be changed. The statement that “The ID consists of an 8 digit hex value.” is not valid anymore. - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/guidance/requirements_guideline.html
• Finding: The requirements inspection checklist question 5 need to be reclassified. The need for timing requirements shall be asked for all ASIL / QM levels. Potentially the ASIL classification column can be deleted. - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/guidance/requirements_inspection_checklist.html
• Finding: The formal inspection review process needs to be audited. see https://eclipse-score.github.io/score/main/process/general_concepts/score_review_concept.html
• Finding: The Configuration Management of the requirements needs to be audited. Baselines of a complete set of requirements shall be described, not only the version management of single requirements. - see https://eclipse-score.github.io/score/main/process/process_areas/requirements_engineering/requirements_concept.html#requirements-versioning

Architecture:

• Deviation_10: The use of UML shall be specified in order to limit the definition of UML to a useful subset. - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/architecture_concept.html#specification-of-the-architectural-design
• Action_29: It’s not specified at which level dynamic views are expected. (Trivial views (e.g., function call) may be excluded. - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/guidance/architecture_guideline.html
• Action_30: It should be specified that negative cases in dynamics views should be documented. - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/architecture_concept.html#dynamic-view
• Deviation_11: The traceability concept description shall take the traceability of the requirement to the design into account. - see for example https://eclipse-score.github.io/score/main/_images/score_traceability_model_feat_overview.drawio.svg
• Action_31: The ASIL allocation of the architectural elements shall be visible in the diagrams - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/architecture_concept.html#comp_arc_sta__archdes_component_2
• Finding: The language interface (Rust, C++) should be visible within the interfaces. (JochenH)
• Action_33: Criteria shall be defined, in which cases dynamic views are required on Feature (and Component) level. (same as 2nd finding above) - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/guidance/architecture_guideline.html
• Action_34: It shall be defined what a sub-component is. - sub-components are removed from concept, see https://eclipse-score.github.io/score/main/process/general_concepts/score_building_blocks_concept.html and https://eclipse-score.github.io/score/main/_images/score_traceability_model_cmp_overview_2.drawio.svg
• Action_35: The caption of Fig. 19 “Definition of the static model architecture” should be changed as the architecture is based on components (and not on modules). - Fig. 19 is now the "feature 1" example, modules are only the top-level components in a repository, see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/architecture_concept.html#definition-of-architectural-viewpoints
• Deviation_12: Traceability from/ to the architecture to/ from the code is not given yet. - Architecture: see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/architecture_concept.html#definition-of-architectural-viewpoints examples
• Action_36: Formal Inspection Reviews shall be defined for the Software Design. - see https://eclipse-score.github.io/score/main/process/general_concepts/score_review_concept.html and implementation process (VolkerH)
• Action_37: The complexity criteria (e.g. number of interfaces, size of interfaces) shall be specified (ARC_03_03). - see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/guidance/architecture_inspection_checklist.html
• Action_38: Review checklist questions shall not only be answered by yes / no. - see https://eclipse-score.github.io/score/main/process/general_concepts/score_review_concept.html
• Recommendation_3: Checklists consist of a ASIL/ QM classification which should be removed. see https://eclipse-score.github.io/score/main/process/process_areas/architecture_design/guidance/architecture_inspection_checklist.html
• Action_39: There shall be a thorough description of how to work with checklists (../process/process_model/guidance/architecture/checklists/architecture_inspection_checklist.html) - see https://eclipse-score.github.io/score/main/process/general_concepts/score_review_concept.html
• Action_40: Checklist shall be listed in the directory (document management) (solved, doc indexing tool issue)

Verification Mgt:

• Action_48: An argument shall be provided which covers the ISO 26262 requirement that all test levels are executed in a target near environment. - see https://eclipse-score.github.io/score/main/platform_management_plan/software_verification.html#test-execution-environment-and-reference-hardware
• Action_49: The methods of ISO 26262 shall be explained: E.g. what is meant with “static code analysis”? (PhilippA) - The methods of ISO 26262 shall be explained #497
• Action_50: The Coverage target values as listed by table 37 need to described in more detail. E.g. what is meant by “Coverage of detailed/architectural design” (Sequence diagram coverage?) (PhilippA) - The Coverage target values as listed by table 37 need to described in more detail #499
• Action_51: It needs to be clarified where the reference system is located physically (affecting e.g. access rights). - process: add reference HW information to verification plan #666
• Action_53: The tool Bazel needs to be evaluated as it may not be able to see the dependencies of changes in requirements, code and tests and may skip some tests. Define the counter measurements. - as the Verification plan defines that all tests will be executed at least for a release and this must be checked by looking in the respective Verification Reports at release, this failure case of Bazel is irrelevant, see https://eclipse-score.github.io/score/main/platform_management_plan/software_verification.html#test-selection-and-regression-testing
• Action_54: The test shall have an attribute which level the test case belongs to. Second it‘s not clear which test belongs to which level; this shall be fixed. Third, the application of the test methods must be recognizable (e.g. boundary value analysis, equivalence class definition, interface-based testing, ...). (PhilippA) - The application of the test methods must be recognizable #498
• Deviation_19: The Verification Plan does not define the required review activities (for reuse of OSS existing tests). - see https://eclipse-score.github.io/score/main/platform_management_plan/software_verification.html#pre-existing-test-cases
• Action_55: The independence of the verification team is not yet described in the Verification Plan - see e.g. https://eclipse-score.github.io/score/main/process/process_areas/verification/workflows.html#wf__verification__unit_test
• Action_55: It remains unclear how much Verification reports will be created (one per module?) - see https://eclipse-score.github.io/score/main/process/process_areas/verification/workflows.html#wf__verification__mod_ver_report

Notes:

• Findings shall be resolved (and presented in follow-up audit)
• Recommendations may be resolved, if not comment here in the ticket
• Actions are usually "reminders" for the auditors in follow-up audit because the topic could not be presented already (gap in process)

[aschemmel-tech]

copied this from eclipse-score/process_definition#2 (as this is in wrong repo at the moment)