Skip to content

Commit f6699d7

Browse files
TG1999TG1999
committed
Address review comments
Signed-off-by: Tushar Goel <tushar.goel.dav@gmail.com>
1 parent 8843ab3 commit f6699d7

File tree

2 files changed

+67
-89
lines changed

2 files changed

+67
-89
lines changed

‎etc/scripts/d2d/README.rst‎

Lines changed: 59 additions & 70 deletions
Original file line numberDiff line numberDiff line change
@@ -1,86 +1,75 @@
1-
==============================================================================
2-
Run ScanCode.io Pipelines in Docker (D2D Runner)
3-
==============================================================================
1+
Run ScanCode.io Mapping Script
2+
================================
43

5-
This script helps execute **ScanCode.io** pipelines in isolated Docker containers,
6-
using a local PostgreSQL database and a working directory named ``./d2d``.
4+
This script executes the ``map_deploy_to_develop`` mapping workflow from
5+
ScanCode.io inside a Docker container. It optionally spins up a temporary
6+
PostgreSQL instance when needed. The script copies the specified input files to
7+
a working directory, runs the mapping, writes the output to a file, and cleans
8+
up afterward.
79

8-
-------------------------------------------------------------------------------
9-
Prerequisites
10-
-------------------------------------------------------------------------------
10+
Usage
11+
-----
1112

12-
1. **Python 3.8+** must be installed
13-
2. **Docker** must be installed and accessible via ``sudo`` or user group
13+
.. code-block:: bash
1414
15-
-------------------------------------------------------------------------------
16-
Environment Variables
17-
-------------------------------------------------------------------------------
15+
./run_mapping.sh <from-path> <to-path> [options] <output-file> <spin-db> [db-port]
1816
19-
.. list-table::
20-
:widths: 25 75
21-
:header-rows: 1
17+
Arguments
18+
---------
2219

23-
* - Variable
24-
- Description
25-
* - ``SCANCODE_DB_PASS``
26-
- Database password (default: ``scancode``)
27-
* - ``SCANCODE_DB_USER``
28-
- Database user (default: ``scancode``)
20+
+-----------------+-------------------------------------------------------------+
21+
| Argument | Description |
22+
+=================+=============================================================+
23+
| ``from-path`` | Path to the base deployment/scan file |
24+
+-----------------+-------------------------------------------------------------+
25+
| ``to-path`` | Path to the target deployment/scan file |
26+
+-----------------+-------------------------------------------------------------+
27+
| ``options`` | D2D pipeline parameters (can be empty ``""``) |
28+
+-----------------+-------------------------------------------------------------+
29+
| ``output-file`` | File where ScanCode.io output will be written |
30+
+-----------------+-------------------------------------------------------------+
31+
| ``spin-db`` | ``true`` = spin temp DB container, ``false`` = skip |
32+
+-----------------+-------------------------------------------------------------+
33+
| ``db-port`` | Port to bind Postgres (default: ``5432``) |
34+
+-----------------+-------------------------------------------------------------+
2935

30-
-------------------------------------------------------------------------------
31-
Usage Example
32-
-------------------------------------------------------------------------------
3336

34-
.. code-block:: bash
37+
Example
38+
-------
3539

36-
sudo su -
37-
python3 etc/scripts/run_d2d_scio.py \
38-
--input-file ./path/from/from-intbitset.tar.gz:from \
39-
--input-file ./path/to/to-intbitset.whl:to \
40-
--option Python \
41-
--output res1.json
42-
43-
-------------------------------------------------------------------------------
44-
Parameters
45-
-------------------------------------------------------------------------------
46-
47-
.. list-table::
48-
:widths: 25 75
49-
:header-rows: 1
50-
51-
* - Parameter
52-
- Description
53-
* - ``--input-file <path:tag>``
54-
- Required twice: one tagged ``:from``, one tagged ``:to``
55-
* - ``--option <name>``
56-
- Optional; e.g., ``Python``, ``Java``, ``Javascript``, ``Scala``, ``Kotlin``
57-
* - ``--output <file.json>``
58-
- Required; JSON output file for results
59-
60-
-------------------------------------------------------------------------------
61-
Internal Steps
62-
-------------------------------------------------------------------------------
63-
64-
1. Creates or uses the ``./d2d`` directory
65-
2. Copies ``from`` and ``to`` files into it
66-
3. Spins up a temporary **Postgres 13** container
67-
4. Waits for database readiness
68-
5. Runs **ScanCode.io** pipeline (``map_deploy_to_develop``)
69-
6. Saves pipeline output to the specified JSON file
70-
7. Cleans up containers automatically
71-
72-
-------------------------------------------------------------------------------
73-
Cleanup
74-
-------------------------------------------------------------------------------
75-
76-
Containers are auto-removed, but you can verify active containers with:
40+
Run mapping without database:
7741

7842
.. code-block:: bash
7943
80-
docker ps -a | grep scancode
44+
./run_mapping.sh ./from.tar.gz ./to.whl "" results.txt false
8145
82-
If manual cleanup is needed:
46+
Run mapping with database on a custom port:
8347

8448
.. code-block:: bash
8549
86-
docker rm -f <container_id>
50+
./run_mapping.sh ./from.tar.gz ./to.whl "Python,Java" output.txt true 5555
51+
52+
Script Actions
53+
--------------
54+
55+
1. Validates required arguments
56+
2. Starts PostgreSQL in Docker (if ``spin-db=true``)
57+
3. Creates a temporary working directory: ``./d2d``
58+
4. Copies input files into working directory
59+
5. Runs ScanCode.io mapping step:
60+
61+
.. code-block:: text
62+
63+
run map_deploy_to_develop:<D2D_OPTIONS> \
64+
"/code/<from-file>:from,/code/<to-file>:to"
65+
66+
6. Writes mapping output into ``output-file``
67+
7. Cleans up temp directory
68+
8. Stops DB container if it was started
69+
70+
Dependencies
71+
------------
72+
73+
* Bash
74+
* Docker
75+
* Local filesystem permissions for creating ``./d2d`` and writing output

‎etc/scripts/d2d/run_d2d.sh‎

Lines changed: 8 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -60,25 +60,14 @@ TO_FILENAME=$(basename "$TO_PATH")
6060

6161
echo "Running ScanCode.io mapping..."
6262

63-
if [ -z "$D2D_OPTIONS" ]; then
64-
docker run --rm \
65-
-v "$(pwd)/$WORKDIR":/code \
66-
--network host \
67-
-e SCANCODEIO_NO_AUTO_DB=1 \
68-
ghcr.io/aboutcode-org/scancode.io:latest \
69-
run map_deploy_to_develop \
70-
"/code/${FROM_FILENAME}:from,/code/${TO_FILENAME}:to" \
71-
> "$OUTPUT_FILE"
72-
else
73-
docker run --rm \
74-
-v "$(pwd)/$WORKDIR":/code \
75-
--network host \
76-
-e SCANCODEIO_NO_AUTO_DB=1 \
77-
ghcr.io/aboutcode-org/scancode.io:latest \
78-
run map_deploy_to_develop:"$D2D_OPTIONS" \
79-
"/code/${FROM_FILENAME}:from,/code/${TO_FILENAME}:to" \
80-
> "$OUTPUT_FILE"
81-
fi
63+
docker run --rm \
64+
-v "$(pwd)/$WORKDIR":/code \
65+
--network host \
66+
-e SCANCODEIO_NO_AUTO_DB=1 \
67+
ghcr.io/aboutcode-org/scancode.io:latest \
68+
run map_deploy_to_develop:"$D2D_OPTIONS" \
69+
"/code/${FROM_FILENAME}:from,/code/${TO_FILENAME}:to" \
70+
> "$OUTPUT_FILE"
8271

8372
echo "Output saved to $OUTPUT_FILE"
8473

0 commit comments

Comments
 (0)