Extend GitProxy to Handle All GitHub Communications and Reviews

[monfla000]

Is your feature request related to a problem? Please describe.
Financial institutions and enterprises with strict security requirements need complete control and visibility over all GitHub interactions, including both git protocol operations and web-based communications. While git-proxy handles some git protocol operations, a comprehensive solution is needed that covers all possible interaction channels with GitHub. This includes git protocol operations (push/pull/fetch), API communications, issue management, pull request reviews, and general GitHub interactions. Currently, the fragmented coverage of these channels creates potential security gaps, makes it difficult to maintain a complete audit trail, and risks sensitive information being accidentally exposed.

Describe the solution you'd like
Implement a comprehensive GitHub proxy layer that intercepts and manages all forms of GitHub interaction from within an organization:

1. Git Protocol and Web/API Communication Coverage:

   • SSH protocol communications
   • HTTPS protocol communications
   • REST API calls to GitHub
   • GraphQL API communications

2. Feature Coverage:

   • Issues Management:
     • Creation and modification of issues
     • Comment management
   • Pull Request Operations:
     • PR creation and updates
     • Review comments and discussions
     • Review state management (approve/reject/request changes)

3. Security Features:

   • Content scanning pipeline for all communications (both git and web)
   • Pattern matching for sensitive data (API keys, internal IPs, etc.)
   • Custom regex rules for organization-specific sensitive data
   • Automatic redaction of identified sensitive information
   • Block/Allow lists for specific types of content
   • Pre-push hooks for git operations
   • Commit content scanning

4. Audit Capabilities:

   • Detailed logging of all GitHub interactions (git and web)
   • Who-did-what-when tracking across all channels
   • Full text search across historical communications
   • Git operation audit trail
   • Export capabilities for compliance reporting
   • Commit history analysis

5. Review Workflow:

   • Multi-level approval process for different types of operations
   • Automated review rules based on content patterns
   • Emergency override capabilities with proper authentication
   • Pre-push review requirements

6. Administration:

   • Role-based access control for all proxy features
   • Configuration management through code
   • Health monitoring and alerts
   • Performance metrics and analytics

Additional context
This feature would be particularly valuable for:

• Financial institutions requiring strict compliance with regulations
• Organizations handling sensitive intellectual property
• Enterprises with strict data loss prevention requirements

The implementation should provide a unified interface for managing all GitHub interactions while being minimally intrusive to developer workflows.

[coopernetes]

Thanks for creating this issue!

At RBC, we have an internal app that does this by proxying GitHub's CLI that we've namely dubbed "github-proxy" ;) it only has rudimentary support for creating PRs & issues but we've been trying to tackle the same gaps. It uses the same Express-based middleware as GitProxy.

There are some sharp edges in handling these interactions and there are many aspects of interacting with GitHub (not even talking about other providers at this point) such as posting screenshots in issues, discussion participation, etc. that the CLI doesn't necessarily have full support for.

I think it makes sense to bring this functionality into this project. I would say it likely requires some "re-invention" in terms of mimicking GitHub's own interface and applying similar filtering/policy enforcement on the content that leaves from that app bound for GitHub.

I'd also caution on trying to tackle this before some of the tech debt is addressed. See #425 for details.

Also, just to point out - I don't think it's reasonable to reinvent a full-blown web browser proxy since there's plenty of commercial vendors that play in this space which are more feature complete.

[monfla000]

Thanks @coopernetes for sharing your insights. Is the internal app you mentioned something that can be ported to GitProxy or should we consider new implementation of the features? Also, which of the tech debt items do you consider essential to deliver above functionality at least in parts?

[JamieSlome]

[coopernetes]

@monfla000 I will raise a PR and incorporate it into FINOS Git Proxy directly. It uses the same machinery so it can be integrated directly. The actual code is fairly small and trivial.

[monfla000]

@monfla000 I will raise a PR and incorporate it into FINOS Git Proxy directly. It uses the same machinery so it can be integrated directly. The actual code is fairly small and trivial.

Sounds great. We have been quite active on the topic since a couple of weeks now and aim to push our contributions out as well as soon as possible.

[coopernetes]

@jescalada @kriswest unless there's any objections, I would like to be assigned this one. I have a working HTTP API proxy implementation that should satisfy some of these basic requirements that I would like to integrate into GitProxy. It uses the same sort of machinery (express-http-proxy) that the git-based proxying does.

[kriswest]

Happy for you to take this @coopernetes - but will add I'd love to see it done in such a way that it can be generalized to handle Gitlab and other SCM providers if at all possible! Particularly as far as the issue and pull merge request workflows on Gitlab.

[coopernetes]

Happy for you to take this @coopernetes - but will add I'd love to see it done in such a way that it can be generalized to handle Gitlab and other SCM providers if at all possible! Particularly as far as the issue and pull merge request workflows on Gitlab.

New UI elements can generalize some things like issue/bug creation and PR/MR create/update, adding comments, etc. However, in our initial implementation, we are hijacking the GitHub Enterprise Server (self hosted, on premises) support in GitHub's official CLI to make an internal, dummy URL look like a GHES endpoint but it is transformed & proxied upstream to api.github.com for the purposes of those operations. GitLab has their own CLI so this should be compatible with their platform as well. At a HL:

flowchart LR
    A[gh CLI] -->|API Request<br/>/api/v3<br/>/api/graphql| B[github-proxy]
    B -->|Translates & Proxies to<br/>api.github.com<br/>api.github.com/graphql| C[api.github.com]
    C -->|Response| B
    B -->|Proxied Response| A
    
    style A fill:#2088FF,stroke:#0366d6,stroke-width:2px,color:#fff
    style B fill:#6f42c1,stroke:#5a32a3,stroke-width:2px,color:#fff
    style C fill:#28a745,stroke:#22863a,stroke-width:2px,color:#fff

Loading

I believe this will always be highly specific to the git hosting provider and there's really no way around it.

Also, this matches the UX on git-proxy itself (proxying git). It does force that UX however by relying on command line tools. I understand that using a web browser to manage contributions may be preferred and a more natural way to contribute to open source repositories for many developers. Once I have added the proxy server support, we can add the corresponding frontend features.