Skip to content

[integration][Github] Implement basic Github support #1551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

melodyogonna
Copy link

@melodyogonna melodyogonna commented Apr 10, 2025

User description

Description

What - A basic Github integration, focusing only on Repositories, Pull Requests, Issues, and Teams.

Why - I have to do it as an assessement.

How - Users can configure the organizations they want to ingest data from, as well as a few other selectors, depending on resource kind, to help with filtering.

Type of change

Please leave one option from the following and delete the rest:

  • New Integration (non-breaking change which adds a new integration)

All tests should be run against the port production environment(using a testing org).

Core testing checklist

  • Integration able to create all default resources from scratch
  • Resync finishes successfully
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Scheduled resync able to abort existing resync and start a new one
  • Tested with at least 2 integrations from scratch
  • Tested with Kafka and Polling event listeners
  • Tested deletion of entities that don't pass the selector

Integration testing checklist

  • Integration able to create all default resources from scratch
  • Resync able to create entities
  • Resync able to update entities
  • Resync able to detect and delete entities
  • Resync finishes successfully
  • If new resource kind is added or updated in the integration, add example raw data, mapping and expected result to the examples folder in the integration directory.
  • If resource kind is updated, run the integration with the example data and check if the expected result is achieved
  • If new resource kind is added or updated, validate that live-events for that resource are working as expected
  • Docs PR link here

Preflight checklist

  • Handled rate limiting
  • Handled pagination
  • Implemented the code in async
  • Support Multi account

Screenshots

Include screenshots from your environment showing how the resources of the integration will look.
Screen Shot 2025-04-10 at 07 54 22
Screen Shot 2025-04-10 at 07 54 46
Screen Shot 2025-04-10 at 07 55 04

API Documentation

Provide links to the API documentation used for this integration.


PR Type

Enhancement, Documentation, Tests


Description

  • Implemented a new GitHub integration for Port Ocean.

    • Supports repositories, pull requests, issues, teams, and workflows.
    • Includes rate-limiting and pagination handling in the GitHub client.
  • Added configuration and resource mapping for GitHub entities.

    • Defined selectors for filtering resources like repositories, PRs, and issues.
    • Introduced blueprints for GitHub resources in .port/resources/blueprints.json.
  • Provided documentation and examples for the integration.

    • Added README, CONTRIBUTING, and CHANGELOG files.
    • Included .env.example for environment variable setup.
  • Configured testing and development tools.

    • Added pyproject.toml with dependencies and configurations.
    • Included VSCode launch configurations and Makefile.

Changes walkthrough 📝

Relevant files
Enhancement
10 files
debug.py
Add debug entry point for GitHub integration                         
+4/-0     
__init__.py
Initialize GitHub client module                                                   
+3/-0     
client.py
Implement GitHub API client with resource fetching             
+209/-0 
integration.py
Define integration configuration and resource selectors   
+111/-0 
main.py
Add resync handlers for GitHub resources                                 
+83/-0   
port.py
Define GitHub resource types for Port                                       
+9/-0     
__init__.py
Initialize webhook handlers for GitHub events                       
+4/-0     
issues.py
Add webhook handler for GitHub issues                                       
+17/-0   
pull_request.py
Add webhook handler for GitHub pull requests                         
+17/-0   
blueprints.json
Define blueprints for GitHub resources                                     
+272/-0 
Configuration changes
7 files
launch.json
Add VSCode launch configuration for GitHub integration     
+11/-0   
port-app-config.yml
Add Port app configuration for GitHub integration               
+107/-0 
spec.yaml
Define integration specifications for GitHub                         
+17/-0   
Makefile
Add Makefile for GitHub integration                                           
+1/-0     
poetry.toml
Configure Poetry for dependency management                             
+3/-0     
pyproject.toml
Add project configuration and dependencies                             
+114/-0 
sonar-project.properties
Add SonarQube project configuration                                           
+2/-0     
Documentation
4 files
.env.example
Provide example environment variables for GitHub integration
+6/-0     
CHANGELOG.md
Add changelog for GitHub integration                                         
+8/-0     
CONTRIBUTING.md
Add contributing guidelines for GitHub integration             
+7/-0     
README.md
Add README for GitHub integration                                               
+7/-0     
Additional files
1 files
__init__.py [link]   

Need help?
  • Type /help how to ... in the comments thread for any questions about Qodo Merge usage.
  • Check out the documentation for more information.
  • Copy link
    Contributor

    PR Reviewer Guide 🔍

    Here are some key observations to aid the review process:

    ⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
    🧪 No relevant tests
    🔒 Security concerns

    Sensitive information exposure:
    The integration handles GitHub tokens which are sensitive credentials. While the token is marked as sensitive in the spec.yaml configuration, there's no validation to ensure tokens aren't accidentally logged. Consider adding additional safeguards to prevent token exposure in logs, especially in error handling sections.

    ⚡ Recommended focus areas for review

    Error Handling

    The error handling in the _make_request method has a formatting issue. The f-string on line 77 is not being properly logged or returned, making the error message incomplete.

    logger.error(f"Error occured while fetching {e.request.url}")
    f"status code: {e.response.status_code} - {e.response.text}"
    raise
    Missing Parameter

    The _make_request method accepts a json parameter but doesn't pass it to the http_client.request call, which would cause any POST requests with JSON bodies to fail.

    async def _make_request(
        self,
        url: str,
        method: str = "GET",
        params: dict[str, Any] | None = None,
        json: Any | None = None,
    ) -> httpx.Response:
        async with self.rate_limitter:
            try:
                return await self._http_client.request(method, url, params=params)
    Typo in Mapping

    There's a typo in the issues entity mapping for labels where it uses ".nam" instead of ".name", which would cause labels to not be properly mapped.

    labels: "[.labels[].nam]"
    status: ".state" # merged, closed, opened

    Copy link
    Contributor

    qodo-merge-pro bot commented Apr 10, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Fix unused parameters
    Suggestion Impact:The commit implemented part of the suggestion by adding the json parameter to the request method call, which was the first issue mentioned in the suggestion. However, it did not fix the error logging string literal issue.

    code diff:

    -                return await self._http_client.request(method, url, params=params)
    +                return await self._http_client.request(
    +                    method, url, params=params, json=json
    +                )

    The json parameter is defined in the function signature but not used in the
    actual request. Also, there's a string literal that's not being used in the
    error logging.

    integrations/github_melody/github/client.py [65-78]

     async def _make_request(
         self,
         url: str,
         method: str = "GET",
         params: dict[str, Any] | None = None,
         json: Any | None = None,
     ) -> httpx.Response:
         async with self.rate_limitter:
             try:
    -            return await self._http_client.request(method, url, params=params)
    +            return await self._http_client.request(method, url, params=params, json=json)
             except httpx.HTTPStatusError as e:
    -            logger.error(f"Error occured while fetching {e.request.url}")
    -            f"status code: {e.response.status_code} - {e.response.text}"
    +            logger.error(f"Error occured while fetching {e.request.url} - status code: {e.response.status_code} - {e.response.text}")
                 raise

    [Suggestion has been applied]

    Suggestion importance[1-10]: 9

    __

    Why: The suggestion fixes two critical issues: the json parameter is defined but not used in the request, and there's a string literal that's not properly included in the error logging. These issues would cause webhook creation to fail and produce incomplete error logs.

    High
    Fix property mapping typo
    Suggestion Impact:The commit directly implemented the suggestion by changing the typo in the labels property mapping from '[.labels[].nam]' to '[.labels[].name]', which will allow labels to be properly extracted from GitHub issues.

    code diff:

    -            labels: "[.labels[].nam]"
    +            labels: "[.labels[].name]"

    There's a typo in the labels property mapping. It's using .nam instead of .name
    which will cause labels to not be properly extracted from GitHub issues.

    integrations/github_melody/.port/resources/port-app-config.yml [62]

    -labels: "[.labels[].nam]"
    +labels: "[.labels[].name]"

    [Suggestion has been applied]

    Suggestion importance[1-10]: 8

    __

    Why: The suggestion correctly identifies a typo in the labels property mapping (".nam" instead of ".name"), which would cause GitHub issue labels to not be properly extracted. This is a critical data mapping issue.

    Medium
    • Update

    Copy link

    This pull request is automatically being deployed by Amplify Hosting (learn more).

    Access this pull request here: https://pr-1551.d1ftd8v2gowp8w.amplifyapp.com

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    1 participant