-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial commit of feature-complete v1 version #1
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
version: 2 | ||
updates: | ||
- package-ecosystem: pip | ||
directory: / | ||
schedule: | ||
interval: monthly | ||
- package-ecosystem: github-actions | ||
directory: / | ||
schedule: | ||
interval: monthly |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
name: Lint and test | ||
|
||
on: | ||
push: | ||
branches: [main] | ||
pull_request: | ||
branches: ["**"] | ||
|
||
jobs: | ||
test: | ||
name: Test | ||
runs-on: ubuntu-22.04 | ||
strategy: | ||
matrix: | ||
python-version: ["3.10", "3.11", "3.12"] | ||
|
||
steps: | ||
- uses: actions/checkout@v4 | ||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
- name: Install dependencies | ||
run: | | ||
python -m pip install --upgrade pip | ||
pip install -r requirements.txt | ||
pip install -r requirements-test.txt | ||
- name: Check formatting | ||
run: ruff format --diff . | ||
- name: Lint with ruff | ||
run: ruff check --target-version=py310 . |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Git Ignore File | ||
|
||
**.DS_Store |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
FROM python:3.12-bookworm | ||
|
||
COPY entrypoint.py /entrypoint.py | ||
COPY report_template /report_template | ||
COPY requirements.txt /requirements.txt | ||
|
||
RUN pip install -r /requirements.txt | ||
|
||
ENTRYPOINT [ "/entrypoint.py" ] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# DigiKey-Reports | ||
An actions repository for DigiKey API integration and report generation given an input BOM | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It might be worth linking to the generate BOM action to show how the BOM input can be generated. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a good idea. Let's add that to the README. |
||
|
||
This action uses the DigiKey API. See the [DigiKey API docs](https://developer.digikey.com/products) for more information. | ||
|
||
## Usage | ||
|
||
Add the following step to your actions: | ||
|
||
```yaml | ||
- name: Generate BOM report using DigiKey | ||
uses: https://hub.allspice.io/Actions/digikey-report@v1 | ||
with: | ||
bom_file: bom.csv | ||
digikey_client_id: ${{ secrets.DIGIKEY_CLIENT_ID }} | ||
digikey_client_secret: ${{ secrets.DIGIKEY_CLIENT_SECRET }} | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
name: "Generate BOM report from querying Digi-Key API" | ||
description: > | ||
Generate a BOM report from querying Digi-Key's API. | ||
inputs: | ||
bom_file: | ||
description: "Path to the BOM CSV file" | ||
required: true | ||
runs: | ||
using: "docker" | ||
image: "Dockerfile" | ||
args: | ||
- ${{ inputs.bom_file }} | ||
- "--output_path" | ||
- "${{ github.workspace }}" | ||
env: | ||
ALLSPICE_AUTH_TOKEN: ${{ github.token }} | ||
DIGIKEY_CLIENT_ID: ${{ inputs.digikey_client_id }} | ||
DIGIKEY_CLIENT_SECRET: ${{ inputs.digikey_client_secret }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,267 @@ | ||
#! /usr/bin/env python3 | ||
|
||
from jinja2 import Environment, FileSystemLoader | ||
from argparse import ArgumentParser | ||
from contextlib import ExitStack | ||
import requests | ||
import zipfile | ||
import shutil | ||
import json | ||
import csv | ||
import os | ||
|
||
DIGIKEY_API_URL_BASE = "https://api.digikey.com" | ||
DIGIKEY_API_AUTH_ENDPOINT = DIGIKEY_API_URL_BASE + "/v1/oauth2/token" | ||
DIGIKEY_API_V4_KEYWORD_SEARCH_ENDPOINT = ( | ||
DIGIKEY_API_URL_BASE + "/products/v4/search/keyword" | ||
) | ||
|
||
|
||
################################################################################ | ||
class ComponentData: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This could be a There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's a good point, I'll add the decorator. |
||
def __init__(self): | ||
self.associated_refdes = "" | ||
self.part_description = None | ||
self.mfr_name = None | ||
self.mfr_part_number = None | ||
self.photo_url = None | ||
self.datasheet_url = None | ||
self.product_url = None | ||
self.qty_available = None | ||
self.lifecycle_status = None | ||
self.eol_status = None | ||
self.discontinued_status = None | ||
self.pricing = None | ||
self.package_case = None | ||
self.supplier_device_package = None | ||
self.operating_temp = None | ||
self.xy_size = None | ||
self.height = None | ||
self.thickness = None | ||
|
||
|
||
################################################################################ | ||
def get_access_token(url, client_id, client_secret): | ||
# Populate request header | ||
headers = { | ||
"Content-Type": "application/x-www-form-urlencoded", | ||
} | ||
# Post the request and get the response | ||
response = requests.post( | ||
url, | ||
data={"grant_type": "client_credentials"}, | ||
headers=headers, | ||
auth=(client_id, client_secret), | ||
) | ||
# Populate the access token return value | ||
access_token = ( | ||
response.json()["access_token"] if response.status_code == 200 else None | ||
) | ||
# Return response status code and access token | ||
return (response.status_code, access_token) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see this response code being used anywhere. Is there a reason we're returning it? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch, the response code is being used in the API query endpoint but not in the auth. I'll add the check to auth to exit if auth fails. |
||
|
||
|
||
################################################################################ | ||
def query_digikey_v4_API_keyword_search( | ||
url, | ||
client_id, | ||
access_token, | ||
locale_site, | ||
locale_language, | ||
locale_currency, | ||
customer_id, | ||
keyword, | ||
): | ||
# Populate request header | ||
headers = { | ||
"charset": "utf-8", | ||
"Content-Type": "application/json", | ||
"Accept": "application/json", | ||
"Authorization": "Bearer " + access_token, | ||
"X-DIGIKEY-Client-Id": client_id, | ||
"X-DIGIKEY-Locale-Site": locale_site, | ||
"X-DIGIKEY-Locale-Language": locale_language, | ||
"X-DIGIKEY-Locale-Currency": locale_currency, | ||
"X-DIGIKEY-Customer-Id": customer_id, | ||
} | ||
# Populate request data | ||
request_data = { | ||
"Keywords": keyword, | ||
} | ||
# Post the request and get the response | ||
response = requests.post( | ||
url, | ||
data=str(json.dumps(request_data)), | ||
headers=headers, | ||
) | ||
# Populate the search result return value | ||
keyword_search_json = ( | ||
response.json() if response.status_code == 200 else response.text | ||
) | ||
# Return response status code and search result | ||
return (response.status_code, keyword_search_json) | ||
|
||
|
||
################################################################################ | ||
def extract_data_from_digikey_search_response(keyword_search_json): | ||
# Initialize a component data object to store the extracted info | ||
part_data = ComponentData() | ||
# Extract product data from exact match | ||
product_data = ( | ||
(keyword_search_json["ExactMatches"])[0] | ||
if keyword_search_json["ExactMatches"] | ||
else None | ||
) | ||
# If the product data matched with valid results, process the line item | ||
if product_data: | ||
# Get the product description | ||
part_data.part_description = product_data["Description"]["ProductDescription"] | ||
# Get the manufacturer name | ||
part_data.mfr_name = product_data["Manufacturer"]["Name"] | ||
# Get the manufacturer part number | ||
part_data.mfr_part_number = product_data["ManufacturerProductNumber"] | ||
# Save the photo from the photo URL | ||
part_data.photo_url = product_data["PhotoUrl"] | ||
# Get the datasheet URL | ||
part_data.datasheet_url = product_data["DatasheetUrl"] | ||
# Get the product URL | ||
part_data.product_url = product_data["ProductUrl"] | ||
# Get the in stock quantity | ||
part_data.qty_available = product_data["QuantityAvailable"] | ||
# Get the part lifecycle, EOL, and discontinued status | ||
part_data.lifecycle_status = product_data["ProductStatus"]["Status"] | ||
part_data.eol_status = product_data["EndOfLife"] | ||
part_data.discontinued_status = product_data["Discontinued"] | ||
# Get the pricing information | ||
part_data.pricing = product_data["ProductVariations"] | ||
# Remove Digi-Reel and rename as DR/CT if both Digi-Reel and Cut-Tape exist | ||
pricing_variations = [ | ||
variation["PackageType"]["Name"] for variation in part_data.pricing | ||
] | ||
cut_tape_idx = digi_reel_idx = [ | ||
i | ||
for i in range(0, len(pricing_variations)) | ||
if "Cut Tape" in pricing_variations[i] | ||
] | ||
digi_reel_idx = [ | ||
i | ||
for i in range(0, len(pricing_variations)) | ||
if "Digi-Reel" in pricing_variations[i] | ||
] | ||
if cut_tape_idx and digi_reel_idx: | ||
part_data.pricing[cut_tape_idx[0]]["PackageType"]["Name"] = ( | ||
"Cut Tape (CT) & Digi-Reel®" | ||
) | ||
del part_data.pricing[digi_reel_idx[0]] | ||
# Initialize part parameter variables | ||
part_data.package_case = part_data.supplier_device_package = ( | ||
part_data.operating_temp | ||
) = part_data.xy_size = part_data.height = part_data.thickness = None | ||
# Get product parameter information | ||
for parameter in product_data["Parameters"]: | ||
if "ParameterText" in parameter.keys(): | ||
# Get the package / case | ||
if parameter["ParameterText"] == "Package / Case": | ||
part_data.package_case = parameter["ValueText"] | ||
# Get the supplier device package | ||
if parameter["ParameterText"] == "Supplier Device Package": | ||
part_data.supplier_device_package = parameter["ValueText"] | ||
# Get the operating temperature range | ||
if parameter["ParameterText"] == "Operating Temperature": | ||
part_data.operating_temp = parameter["ValueText"] | ||
# Get the package XY dimensions | ||
if "Size" in parameter["ParameterText"]: | ||
part_data.xy_size = parameter["ValueText"] | ||
# Get the package height or thickness | ||
if "Height" in parameter["ParameterText"]: | ||
part_data.height = parameter["ValueText"] | ||
if "Thickness" in parameter["ParameterText"]: | ||
part_data.thickness = parameter["ValueText"] | ||
|
||
# Return the extracted part data | ||
return part_data | ||
|
||
|
||
################################################################################ | ||
if __name__ == "__main__": | ||
# Initialize argument parser | ||
parser = ArgumentParser() | ||
parser.add_argument("bom_file", help="Path to the BOM file") | ||
parser.add_argument( | ||
"--output_path", help="Path to the directory to output report to" | ||
) | ||
args = parser.parse_args() | ||
|
||
# Read the BOM file into list | ||
with open(args.bom_file, newline="") as bomfile: | ||
# Comma delimited file with " as quote character to be included | ||
bomreader = csv.reader(bomfile, delimiter=",", quotechar='"') | ||
# Save as a list | ||
bom_line_items = list(bomreader) | ||
# Save the index of the designator field | ||
refdes_col_idx = (bom_line_items[0]).index("Designator") | ||
# Skip the header | ||
del bom_line_items[0] | ||
|
||
# Authenticate with DigiKey | ||
digikey_client_id = os.environ.get("DIGIKEY_CLIENT_ID") | ||
digikey_client_secret = os.environ.get("DIGIKEY_CLIENT_SECRET") | ||
(response_code, access_token) = get_access_token( | ||
DIGIKEY_API_AUTH_ENDPOINT, digikey_client_id, digikey_client_secret | ||
) | ||
|
||
# Initialize list of BOM item part data | ||
bom_items_digikey_data = [] | ||
|
||
# Fetch information for all parts in the BOM | ||
for line_item in bom_line_items: | ||
print("- Fetching info for " + line_item[0]) | ||
# Search for parts in DigiKey by Manufacturer Part Number as keyword | ||
(response_code, keyword_search_json) = query_digikey_v4_API_keyword_search( | ||
DIGIKEY_API_V4_KEYWORD_SEARCH_ENDPOINT, | ||
digikey_client_id, | ||
access_token, | ||
"US", | ||
"en", | ||
"USD", | ||
"0", | ||
line_item[0], | ||
) | ||
# Process a successful response | ||
if response_code == 200: | ||
# Extract the part data from the keyword search response | ||
part_data = extract_data_from_digikey_search_response(keyword_search_json) | ||
# Add the associated reference designators | ||
part_data.associated_refdes = line_item[refdes_col_idx] | ||
# Add the extracted data to the list of BOM items part data | ||
bom_items_digikey_data.append(part_data) | ||
# Print out the details of an unsuccessful response | ||
else: | ||
print(response_code, keyword_search_json) | ||
|
||
# Load Jinja with output HTML template | ||
template_env = Environment(loader=FileSystemLoader("/report_template/")) | ||
template = template_env.get_template("index.html") | ||
# Populuate the context data | ||
context = {"bom": bom_items_digikey_data} | ||
# Create report output folder if it doesn't exist | ||
try: | ||
os.makedirs("/component_report") | ||
except FileExistsError: | ||
pass | ||
# Unzip the JS/CSS assets | ||
shutil.unpack_archive("/report_template/assets.zip", "/component_report") | ||
# Write HTML output file | ||
with ExitStack() as stack: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You don't need an exit stack here, you can directly use: with open(...) as report_file: |
||
report_file = stack.enter_context( | ||
open("/component_report/index.html", mode="w", encoding="utf-8") | ||
) | ||
print("- Outputting report") | ||
report_file.write(template.render(context)) | ||
# Zip the component report as a git workspace artifact | ||
with zipfile.ZipFile( | ||
args.output_path + "/component_report.zip", "w", zipfile.ZIP_DEFLATED | ||
) as zipper: | ||
for root, dirs, files in os.walk("/component_report"): | ||
for file in files: | ||
zipper.write(os.path.join(root, file)) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
[tool.ruff] | ||
exclude = [] |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to keep these files here and in the zip? I think just the zip should be enough? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree, let's can the assets folders and keep only the zip |
Large diffs are not rendered by default.
Large diffs are not rendered by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rename test.yml to semantic meaning. What is this workflow doing? Name the file that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. It's named as test.py in generate-bom and cofactr-cogs also - we will prolly want to rename those as well. It's a ruff format/lint check workflow - we should rename it as that.