Skip to content

Commit

Permalink
Written readme for experiment
Browse files Browse the repository at this point in the history
  • Loading branch information
LennartJKlein committed Dec 14, 2017
1 parent f1d6e0d commit af77668
Showing 1 changed file with 92 additions and 0 deletions.
92 changes: 92 additions & 0 deletions experiments/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Correlation between score and length of a netlist
_Published on: 14-12-2017_

## Summary
We conducted this experiment to get a better understanding on the returned scores of the solutions. We used a set of predefined conditions (similarly to the [original case](http://heuristieken.nl/wiki/index.php?title=Chips_%26_Circuits)). In this experiment the average score of multiple lengths of netlists were compared.
This experiment showed that the score is always linearly (independent of the amount of gates, or their location, on the board) within the scope of a netlist length from 2 to 50.

See the ‘discussion’ paragraph for future improvements and extensions on this experiment.

## Background
Our program is able to solve the [6 predefined netlists](http://heuristieken.nl/resources/CC_netlists2.txt) with a length of 30 up to 70 connections.
These netlists were designed for 2 predefined boards (consisting of 25 and 50 gates).

![Board 1 - 25 gates](http://heuristieken.nl/wiki/images/7/77/Print1.gif "Board 1 - 25 gates")
![Board 2 - 50 gates](http://heuristieken.nl/wiki/images/7/77/Print2.gif "Board 2 - 50 gates")

The program we wrote for this case returns a non-deterministic solution (due to some semi-random pruning). To get a better understanding on the returned scores of the solutions, we conducted this experiment.
## Hypothesis
The score is measured by the total length of all the drawn paths on the board. The more paths, the higher the score.

H0 = We expect the score of the solution to grow linear when the length of a netlist (the amount of connections that has to be made on the board) increases.
## Method

The experiment consists of testing different lengths of netlists on different boards.
To achieve that, we did the following:
1. For each amount of gates (30, 35, 40, 45, 50 … to 95), generate a board with the gates placed randomly.
2. For each generated board, generate 50 random netlists of lengths 2, 3, 4, 5, 6 … to 50
3. _Run our program_ for every generated board:
1. Get the average result of solving a netlist with length 2 a hundred times.
2. Get the average result of solving a netlist with length 3 a hundred times.
3. Get the average result of solving a netlist with length 4 a hundred times.
4. …
5. Plot the average *score* for every length of netlist

### Programmatically
* Use the runXX.py files to automatically run the __main__.py in the concerned folder.
* a _runXX.py_ file will loop through the files within a map, and will loop through the maps

### Conditions
* A board is set at a width of 18, a height of 17, a depth of 8 (1 + 7 layers).
* A gate cannot have more than 5 connections.
* The maximum ‘no improve’ of a solution is set at 5.
* Only the average score of the 50 randomly generated netlists (of the same length) will be used as results for the experiment

## Results
Scores at 30 gates
![Scores 30 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run30_score.png "Scores 30 gates")
Scores at 35 gates
![Scores 35 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run35_score.png "Scores 35 gates")
Scores at 40 gates
![Scores 40 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run40_score.png "Scores 40 gates")
Scores at 45 gates
![Scores 45 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run45_score.png "Scores 45 gates")
Scores at 50 gates
![Scores 50 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run50_score.png "Scores 50 gates")
Scores at 55 gates
![Scores 55 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run55_score.png "Scores 55 gates")
Scores at 60 gates
![Scores 60 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run60_score.png "Scores 60 gates")
Scores at 65 gates
![Scores 65 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run65_score.png "Scores 65 gates")
Scores at 70 gates
![Scores 70 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run70_score.png "Scores 70 gates")
Scores at 75 gates
![Scores 75 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run75_score.png "Scores 75 gates")
Scores at 80 gates
![Scores 80 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run80_score.png "Scores 80 gates")
Scores at 85 gates
![Scores 85 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run85_score.png "Scores 85 gates")
Scores at 90 gates
![Scores 90 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run90_score.png "Scores 90 gates")
Scores at 95 gates
![Scores 95 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run95_score.png "Scores 95 gates")

In the graphs above you can see the *average* scores for different lengths of netlists on different amount of gates. Every graph has a linear increase.

## Conclusions

Within the scope of our experiment and by the data we were able to produce, we can say that the score function increases linearly with the length of a netlist. There are two possible explanations for this phenomenon:
1. The written program always produces the best score for a netlist (which has the shortest possible paths, so the score is only depending on the amount of paths that is drawn)
2. The score function has a simple nature. If it contained more variables, it could be that the score function behaves differently.

## Discussion

After completing this experiment, we found some ways to improve and extend this experiment in the future:
1. Increase the scope of the experiment by generating netlists with a length of 50 and up. This can be used to verify or disprove the credibility of this experiment.
2. Find a way to shorten the runtime of the program (runXX.py) to generate results faster.
3. Save more data of every solution than purely the score of the board. For example:
- if all connections were made
- runtime of each solution
- amount of layers needed
- a heatmap of busy junctions on the board

0 comments on commit af77668

Please sign in to comment.