refactor code structure #378

frjcomp · 2025-11-20T16:05:08Z

No description provided.

- Moved pkg/gitlab/runners/list.go into list/ subfolder and integrated processor.go - Renamed processor_test.go to list_test.go with all tests passing - Moved pkg/gitlab/cicd/yaml.go into yaml/ subfolder for consistency - Removed duplicate/redundant files - Fixed golangci-lint issue: changed if/else chain to switch statement - All tests passing, build successful

- Update gl cicd yaml command to read gitlab/token flags at runtime - Update gl runners list command to read gitlab/token flags at runtime - Update gl runners exploit command to read gitlab/token flags at runtime - Fix issue where flags passed at command creation were empty strings - All e2e tests now pass successfully

…rs for tests

frjcomp · 2025-11-22T13:46:13Z

@copilot i want you to analyze how we can integrate /workspaces/pipeleak/src/pipeleak/pkg/gitlab/scan with /workspaces/pipeleak/src/pipeleak/pkg/scan and in /workspaces/pipeleak/src/pipeleak/pkg/scanner like the other scan commands do. Then refactor gitlab scan to remove duplicate logic.

Copilot · 2025-11-22T13:46:20Z

@frjcomp I've opened a new pull request, #381, to work on those changes. Once the pull request is ready, I'll request review from you.

* Initial plan * refactor: integrate GitLab scan with common scan packages Co-authored-by: frjcomp <107982661+frjcomp@users.noreply.github.com> * fix: use 'note' field instead of 'message' for dotenv findings Co-authored-by: frjcomp <107982661+frjcomp@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: frjcomp <107982661+frjcomp@users.noreply.github.com>

Copilot

Pull request overview

This PR performs a comprehensive refactoring to improve code structure by separating concerns between CLI command definitions (cmd/) and business logic (pkg/). The refactoring follows a clear pattern of moving command implementations from pkg/ packages into corresponding cmd/ packages, while business logic remains in or moves to pkg/ packages with renamed exported functions (e.g., FetchVariables → RunFetchVariables).

Key changes include:

Separation of CLI command definitions from business logic across GitLab, Gitea, and documentation commands
Reorganization of runner, renovate, and other GitLab subcommands into proper cmd/ and pkg/ hierarchies
Creation of new shared packages for artifact processing, log scanning, and result reporting
Removal of obsolete internal processor packages and unused command wrapper files
Addition of comprehensive test coverage for new code structures

Reviewed changes

Copilot reviewed 55 out of 57 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/pipeleak/tests/e2e/gitlab_commands_test.go`	Updated test to use new `--project` flag instead of `--repo`
`src/pipeleak/pkg/gitlab/vuln/vuln.go`	Moved command definition to cmd/, business logic stays in pkg/ with `RunCheckVulns`
`src/pipeleak/pkg/gitlab/variables/variables.go`	Refactored to export `RunFetchVariables` function for cmd/ layer
`src/pipeleak/pkg/gitlab/variables/scanner.go`	Removed unused wrapper file
`src/pipeleak/pkg/gitlab/shodan/shodan.go`	Exported `RunShodan` function, removed command definition
`src/pipeleak/pkg/gitlab/shodan/scanner.go`	Removed unused wrapper file
`src/pipeleak/pkg/gitlab/secureFiles/secure_files.go`	Exported `RunFetchSecureFiles` function
`src/pipeleak/pkg/gitlab/schedule/schedule.go`	Exported `RunFetchSchedules` function
`src/pipeleak/pkg/gitlab/schedule/scanner.go`	Removed unused wrapper file
`src/pipeleak/pkg/gitlab/scan/scanner.go`	Updated import paths from cmd/gitlab/util to pkg/gitlab/util
`src/pipeleak/pkg/gitlab/scan/queue.go`	Refactored to use new scan packages (artifact, logline, result)
`src/pipeleak/pkg/gitlab/scan/pipeline.go`	Updated to use new runner.InitScanner function
`src/pipeleak/pkg/gitlab/scan/internal/processor/*.go`	Removed obsolete internal processor files
`src/pipeleak/pkg/gitlab/runners/`	Reorganized into list/ and exploit/ subdirectories with proper separation
`src/pipeleak/pkg/gitlab/renovate/`	Restructured with enum/, autodiscovery/, and privesc/ subdirectories
`src/pipeleak/pkg/gitlab/register/register.go`	Removed, moved to cmd/ layer
`src/pipeleak/pkg/gitlab/enum/enum.go`	Exported `RunEnum` function
`src/pipeleak/pkg/gitlab/cicd/`	Restructured with yaml/ subdirectory
`src/pipeleak/pkg/gitea/enum/enum.go`	Removed command definition, kept business logic
`src/pipeleak/pkg/docs/generator.go`	Created new documentation generator package
`src/pipeleak/main.go`	Removed unused TerminalRestoringWriter code
`src/pipeleak/cmd/gitlab/*/`	Added command layer files that delegate to pkg/ functions
`src/pipeleak/cmd/gitlab/util/util.go`	Removed, functionality moved to pkg/gitlab/util
`src/pipeleak/cmd/gitlab/nist/nist.go`	Removed unused wrapper file
`src/pipeleak/cmd/docs/docs.go`	Simplified to use new pkg/docs package
`.github/copilot-instructions.md`	Updated project structure documentation

Copilot · 2025-11-22T18:44:57Z

src/pipeleak/pkg/gitlab/runners/list/list.go

+	groupRunners := listGroupRunners(git)
+	runnerMap := MergeRunnerMaps(projectRunners, groupRunners)
+
+	log.Info().Msg("Listing avaialable runenrs: Runners are only shown once, even when available by multiple source e,g, group or project")


Spelling errors in log message: "avaialable" should be "available" and "runenrs" should be "runners". Also, "e,g" should be "e.g."

src/pipeleak/cmd/gitlab/runners/exploit/exploit.go

src/pipeleak/cmd/gitlab/renovate/enum/enum.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

frjcomp · 2025-11-22T18:51:57Z

@copilot You are Copilot working on CompassSecurity/pipeleak (Go). Your goal: find packages under src/pipeleak that lack meaningful unit tests and add only meaningful *_test.go files next to those packages.

Do this concisely:

Scope: src/pipeleak only. Do NOT edit e2e tests or files outside src/pipeleak.
Test criteria: add tests that exercise real logic — error paths, edge cases, parsing, filtering, archive handling, API translation. Do NOT add trivial getters/setters tests.
Conventions: Go table-driven tests, use testify/assert, t.Parallel() where safe, t.TempDir() for IO, mock external clients via interfaces (fakes in _test.go), deterministic and fast.
Implementation: for each package produce (1) a short list of target functions/methods and why a test is needed, then (2) add *_test.go files covering normal, edge, and error cases. Use minimal production-code changes only if absolutely needed for testability (add tiny adapter interfaces).
Verification: run golangci-lint and go test ./... (excluding tests/e2e). Include test results in PR/commit message.
Deliverable: added test files, one-line rationale per file, and commands+results.

Copilot · 2025-11-22T18:52:05Z

@frjcomp I've opened a new pull request, #382, to work on those changes. Once the pull request is ready, I'll request review from you.

frjcomp added 6 commits November 20, 2025 16:04

refactor code structure

d9948a6

remove duplicates

aa89843

removed dead code

6f92181

chore: remove unused terminal restoration code; retain HitEvent helpe…

3d9d721

…rs for tests

Copilot AI mentioned this pull request Nov 22, 2025

Refactor GitLab scan to use common scan packages #381

Merged

frjcomp requested a review from Copilot November 22, 2025 18:41

Copilot started reviewing on behalf of frjcomp November 22, 2025 18:41 View session

Copilot finished reviewing on behalf of frjcomp November 22, 2025 18:43

Copilot AI reviewed Nov 22, 2025

View reviewed changes

frjcomp and others added 3 commits November 22, 2025 18:50

fix typo

872451f

Update src/pipeleak/cmd/gitlab/renovate/enum/enum.go

ddc9a7d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/pipeleak/cmd/gitlab/runners/exploit/exploit.go

846d661

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI mentioned this pull request Nov 22, 2025

[WIP] Refactor GitLab scan to integrate with other scan commands #382

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor code structure #378

refactor code structure #378

Uh oh!

frjcomp commented Nov 20, 2025

Uh oh!

frjcomp commented Nov 22, 2025

Uh oh!

Copilot AI commented Nov 22, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 22, 2025

Uh oh!

Uh oh!

Uh oh!

frjcomp commented Nov 22, 2025

Uh oh!

Copilot AI commented Nov 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

refactor code structure #378

Are you sure you want to change the base?

refactor code structure #378

Uh oh!

Conversation

frjcomp commented Nov 20, 2025

Uh oh!

frjcomp commented Nov 22, 2025

Uh oh!

Copilot AI commented Nov 22, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

frjcomp commented Nov 22, 2025

Uh oh!

Copilot AI commented Nov 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants