Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial commit of feature-complete v1 version #1

Merged
merged 2 commits into from
Jul 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: pip
directory: /
schedule:
interval: monthly
- package-ecosystem: github-actions
directory: /
schedule:
interval: monthly
31 changes: 31 additions & 0 deletions .github/workflows/test.yml
Copy link
Collaborator

@danielallspice danielallspice Jul 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename test.yml to semantic meaning. What is this workflow doing? Name the file that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. It's named as test.py in generate-bom and cofactr-cogs also - we will prolly want to rename those as well. It's a ruff format/lint check workflow - we should rename it as that.

Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Lint and test

on:
push:
branches: [main]
pull_request:
branches: ["**"]

jobs:
test:
name: Test
runs-on: ubuntu-22.04
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-test.txt
- name: Check formatting
run: ruff format --diff .
- name: Lint with ruff
run: ruff check --target-version=py310 .
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Git Ignore File

**.DS_Store
9 changes: 9 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM python:3.12-bookworm

COPY entrypoint.py /entrypoint.py
COPY report_template /report_template
COPY requirements.txt /requirements.txt

RUN pip install -r /requirements.txt

ENTRYPOINT [ "/entrypoint.py" ]
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# DigiKey-Reports
An actions repository for DigiKey API integration and report generation given an input BOM
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be worth linking to the generate BOM action to show how the BOM input can be generated.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea. Let's add that to the README.


This action uses the DigiKey API. See the [DigiKey API docs](https://developer.digikey.com/products) for more information.

## Usage

Add the following step to your actions:

```yaml
- name: Generate BOM report using DigiKey
uses: https://hub.allspice.io/Actions/digikey-report@v1
with:
bom_file: bom.csv
digikey_client_id: ${{ secrets.DIGIKEY_CLIENT_ID }}
digikey_client_secret: ${{ secrets.DIGIKEY_CLIENT_SECRET }}
```
18 changes: 18 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
name: "Generate BOM report from querying Digi-Key API"
description: >
Generate a BOM report from querying Digi-Key's API.
inputs:
bom_file:
description: "Path to the BOM CSV file"
required: true
runs:
using: "docker"
image: "Dockerfile"
args:
- ${{ inputs.bom_file }}
- "--output_path"
- "${{ github.workspace }}"
env:
ALLSPICE_AUTH_TOKEN: ${{ github.token }}
DIGIKEY_CLIENT_ID: ${{ inputs.digikey_client_id }}
DIGIKEY_CLIENT_SECRET: ${{ inputs.digikey_client_secret }}
267 changes: 267 additions & 0 deletions entrypoint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,267 @@
#! /usr/bin/env python3

from jinja2 import Environment, FileSystemLoader
from argparse import ArgumentParser
from contextlib import ExitStack
import requests
import zipfile
import shutil
import json
import csv
import os

DIGIKEY_API_URL_BASE = "https://api.digikey.com"
DIGIKEY_API_AUTH_ENDPOINT = DIGIKEY_API_URL_BASE + "/v1/oauth2/token"
DIGIKEY_API_V4_KEYWORD_SEARCH_ENDPOINT = (
DIGIKEY_API_URL_BASE + "/products/v4/search/keyword"
)


################################################################################
class ComponentData:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a @dataclass, which means you don't have to define __init__ here and then mutate the object later.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point, I'll add the decorator.

def __init__(self):
self.associated_refdes = ""
self.part_description = None
self.mfr_name = None
self.mfr_part_number = None
self.photo_url = None
self.datasheet_url = None
self.product_url = None
self.qty_available = None
self.lifecycle_status = None
self.eol_status = None
self.discontinued_status = None
self.pricing = None
self.package_case = None
self.supplier_device_package = None
self.operating_temp = None
self.xy_size = None
self.height = None
self.thickness = None


################################################################################
def get_access_token(url, client_id, client_secret):
# Populate request header
headers = {
"Content-Type": "application/x-www-form-urlencoded",
}
# Post the request and get the response
response = requests.post(
url,
data={"grant_type": "client_credentials"},
headers=headers,
auth=(client_id, client_secret),
)
# Populate the access token return value
access_token = (
response.json()["access_token"] if response.status_code == 200 else None
)
# Return response status code and access token
return (response.status_code, access_token)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this response code being used anywhere. Is there a reason we're returning it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, the response code is being used in the API query endpoint but not in the auth. I'll add the check to auth to exit if auth fails.



################################################################################
def query_digikey_v4_API_keyword_search(
url,
client_id,
access_token,
locale_site,
locale_language,
locale_currency,
customer_id,
keyword,
):
# Populate request header
headers = {
"charset": "utf-8",
"Content-Type": "application/json",
"Accept": "application/json",
"Authorization": "Bearer " + access_token,
"X-DIGIKEY-Client-Id": client_id,
"X-DIGIKEY-Locale-Site": locale_site,
"X-DIGIKEY-Locale-Language": locale_language,
"X-DIGIKEY-Locale-Currency": locale_currency,
"X-DIGIKEY-Customer-Id": customer_id,
}
# Populate request data
request_data = {
"Keywords": keyword,
}
# Post the request and get the response
response = requests.post(
url,
data=str(json.dumps(request_data)),
headers=headers,
)
# Populate the search result return value
keyword_search_json = (
response.json() if response.status_code == 200 else response.text
)
# Return response status code and search result
return (response.status_code, keyword_search_json)


################################################################################
def extract_data_from_digikey_search_response(keyword_search_json):
# Initialize a component data object to store the extracted info
part_data = ComponentData()
# Extract product data from exact match
product_data = (
(keyword_search_json["ExactMatches"])[0]
if keyword_search_json["ExactMatches"]
else None
)
# If the product data matched with valid results, process the line item
if product_data:
# Get the product description
part_data.part_description = product_data["Description"]["ProductDescription"]
# Get the manufacturer name
part_data.mfr_name = product_data["Manufacturer"]["Name"]
# Get the manufacturer part number
part_data.mfr_part_number = product_data["ManufacturerProductNumber"]
# Save the photo from the photo URL
part_data.photo_url = product_data["PhotoUrl"]
# Get the datasheet URL
part_data.datasheet_url = product_data["DatasheetUrl"]
# Get the product URL
part_data.product_url = product_data["ProductUrl"]
# Get the in stock quantity
part_data.qty_available = product_data["QuantityAvailable"]
# Get the part lifecycle, EOL, and discontinued status
part_data.lifecycle_status = product_data["ProductStatus"]["Status"]
part_data.eol_status = product_data["EndOfLife"]
part_data.discontinued_status = product_data["Discontinued"]
# Get the pricing information
part_data.pricing = product_data["ProductVariations"]
# Remove Digi-Reel and rename as DR/CT if both Digi-Reel and Cut-Tape exist
pricing_variations = [
variation["PackageType"]["Name"] for variation in part_data.pricing
]
cut_tape_idx = digi_reel_idx = [
i
for i in range(0, len(pricing_variations))
if "Cut Tape" in pricing_variations[i]
]
digi_reel_idx = [
i
for i in range(0, len(pricing_variations))
if "Digi-Reel" in pricing_variations[i]
]
if cut_tape_idx and digi_reel_idx:
part_data.pricing[cut_tape_idx[0]]["PackageType"]["Name"] = (
"Cut Tape (CT) & Digi-Reel®"
)
del part_data.pricing[digi_reel_idx[0]]
# Initialize part parameter variables
part_data.package_case = part_data.supplier_device_package = (
part_data.operating_temp
) = part_data.xy_size = part_data.height = part_data.thickness = None
# Get product parameter information
for parameter in product_data["Parameters"]:
if "ParameterText" in parameter.keys():
# Get the package / case
if parameter["ParameterText"] == "Package / Case":
part_data.package_case = parameter["ValueText"]
# Get the supplier device package
if parameter["ParameterText"] == "Supplier Device Package":
part_data.supplier_device_package = parameter["ValueText"]
# Get the operating temperature range
if parameter["ParameterText"] == "Operating Temperature":
part_data.operating_temp = parameter["ValueText"]
# Get the package XY dimensions
if "Size" in parameter["ParameterText"]:
part_data.xy_size = parameter["ValueText"]
# Get the package height or thickness
if "Height" in parameter["ParameterText"]:
part_data.height = parameter["ValueText"]
if "Thickness" in parameter["ParameterText"]:
part_data.thickness = parameter["ValueText"]

# Return the extracted part data
return part_data


################################################################################
if __name__ == "__main__":
# Initialize argument parser
parser = ArgumentParser()
parser.add_argument("bom_file", help="Path to the BOM file")
parser.add_argument(
"--output_path", help="Path to the directory to output report to"
)
args = parser.parse_args()

# Read the BOM file into list
with open(args.bom_file, newline="") as bomfile:
# Comma delimited file with " as quote character to be included
bomreader = csv.reader(bomfile, delimiter=",", quotechar='"')
# Save as a list
bom_line_items = list(bomreader)
# Save the index of the designator field
refdes_col_idx = (bom_line_items[0]).index("Designator")
# Skip the header
del bom_line_items[0]

# Authenticate with DigiKey
digikey_client_id = os.environ.get("DIGIKEY_CLIENT_ID")
digikey_client_secret = os.environ.get("DIGIKEY_CLIENT_SECRET")
(response_code, access_token) = get_access_token(
DIGIKEY_API_AUTH_ENDPOINT, digikey_client_id, digikey_client_secret
)

# Initialize list of BOM item part data
bom_items_digikey_data = []

# Fetch information for all parts in the BOM
for line_item in bom_line_items:
print("- Fetching info for " + line_item[0])
# Search for parts in DigiKey by Manufacturer Part Number as keyword
(response_code, keyword_search_json) = query_digikey_v4_API_keyword_search(
DIGIKEY_API_V4_KEYWORD_SEARCH_ENDPOINT,
digikey_client_id,
access_token,
"US",
"en",
"USD",
"0",
line_item[0],
)
# Process a successful response
if response_code == 200:
# Extract the part data from the keyword search response
part_data = extract_data_from_digikey_search_response(keyword_search_json)
# Add the associated reference designators
part_data.associated_refdes = line_item[refdes_col_idx]
# Add the extracted data to the list of BOM items part data
bom_items_digikey_data.append(part_data)
# Print out the details of an unsuccessful response
else:
print(response_code, keyword_search_json)

# Load Jinja with output HTML template
template_env = Environment(loader=FileSystemLoader("/report_template/"))
template = template_env.get_template("index.html")
# Populuate the context data
context = {"bom": bom_items_digikey_data}
# Create report output folder if it doesn't exist
try:
os.makedirs("/component_report")
except FileExistsError:
pass
# Unzip the JS/CSS assets
shutil.unpack_archive("/report_template/assets.zip", "/component_report")
# Write HTML output file
with ExitStack() as stack:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need an exit stack here, you can directly use:

with open(...) as report_file:

report_file = stack.enter_context(
open("/component_report/index.html", mode="w", encoding="utf-8")
)
print("- Outputting report")
report_file.write(template.render(context))
# Zip the component report as a git workspace artifact
with zipfile.ZipFile(
args.output_path + "/component_report.zip", "w", zipfile.ZIP_DEFLATED
) as zipper:
for root, dirs, files in os.walk("/component_report"):
for file in files:
zipper.write(os.path.join(root, file))
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[tool.ruff]
exclude = []
Binary file added report_template/assets.zip
Binary file not shown.
5 changes: 5 additions & 0 deletions report_template/assets/bootstrap/css/bootstrap.min.css
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to keep these files here and in the zip? I think just the zip should be enough?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, let's can the assets folders and keep only the zip

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions report_template/assets/bootstrap/js/bootstrap.min.js

Large diffs are not rendered by default.

Loading