diff --git a/experiments/README.md b/experiments/README.md new file mode 100644 index 000000000..da147d82d --- /dev/null +++ b/experiments/README.md @@ -0,0 +1,92 @@ +# Correlation between score and length of a netlist +_Published on: 14-12-2017_ + +## Summary +We conducted this experiment to get a better understanding on the returned scores of the solutions. We used a set of predefined conditions (similarly to the [original case](http://heuristieken.nl/wiki/index.php?title=Chips_%26_Circuits)). In this experiment the average score of multiple lengths of netlists were compared. +This experiment showed that the score is always linearly (independent of the amount of gates, or their location, on the board) within the scope of a netlist length from 2 to 50. + +See the ‘discussion’ paragraph for future improvements and extensions on this experiment. + +## Background +Our program is able to solve the [6 predefined netlists](http://heuristieken.nl/resources/CC_netlists2.txt) with a length of 30 up to 70 connections. +These netlists were designed for 2 predefined boards (consisting of 25 and 50 gates). + +![Board 1 - 25 gates](http://heuristieken.nl/wiki/images/7/77/Print1.gif "Board 1 - 25 gates") +![Board 2 - 50 gates](http://heuristieken.nl/wiki/images/7/77/Print2.gif "Board 2 - 50 gates") + +The program we wrote for this case returns a non-deterministic solution (due to some semi-random pruning). To get a better understanding on the returned scores of the solutions, we conducted this experiment. +## Hypothesis +The score is measured by the total length of all the drawn paths on the board. The more paths, the higher the score. + +H0 = We expect the score of the solution to grow linear when the length of a netlist (the amount of connections that has to be made on the board) increases. +## Method + +The experiment consists of testing different lengths of netlists on different boards. +To achieve that, we did the following: +1. For each amount of gates (30, 35, 40, 45, 50 … to 95), generate a board with the gates placed randomly. +2. For each generated board, generate 50 random netlists of lengths 2, 3, 4, 5, 6 … to 50 +3. _Run our program_ for every generated board: + 1. Get the average result of solving a netlist with length 2 a hundred times. + 2. Get the average result of solving a netlist with length 3 a hundred times. + 3. Get the average result of solving a netlist with length 4 a hundred times. + 4. … + 5. Plot the average *score* for every length of netlist + +### Programmatically +* Use the runXX.py files to automatically run the __main__.py in the concerned folder. +* a _runXX.py_ file will loop through the files within a map, and will loop through the maps + +### Conditions +* A board is set at a width of 18, a height of 17, a depth of 8 (1 + 7 layers). +* A gate cannot have more than 5 connections. +* The maximum ‘no improve’ of a solution is set at 5. +* Only the average score of the 50 randomly generated netlists (of the same length) will be used as results for the experiment + +## Results +Scores at 30 gates +![Scores 30 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run30_score.png "Scores 30 gates") +Scores at 35 gates +![Scores 35 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run35_score.png "Scores 35 gates") +Scores at 40 gates +![Scores 40 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run40_score.png "Scores 40 gates") +Scores at 45 gates +![Scores 45 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run45_score.png "Scores 45 gates") +Scores at 50 gates +![Scores 50 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run50_score.png "Scores 50 gates") +Scores at 55 gates +![Scores 55 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run55_score.png "Scores 55 gates") +Scores at 60 gates +![Scores 60 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run60_score.png "Scores 60 gates") +Scores at 65 gates +![Scores 65 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run65_score.png "Scores 65 gates") +Scores at 70 gates +![Scores 70 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run70_score.png "Scores 70 gates") +Scores at 75 gates +![Scores 75 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run75_score.png "Scores 75 gates") +Scores at 80 gates +![Scores 80 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run80_score.png "Scores 80 gates") +Scores at 85 gates +![Scores 85 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run85_score.png "Scores 85 gates") +Scores at 90 gates +![Scores 90 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run90_score.png "Scores 90 gates") +Scores at 95 gates +![Scores 95 gates](https://github.com/LennartJKlein/pathfinder/blob/exp-heuristic-value/experiments/netlist-length/img/run95_score.png "Scores 95 gates") + +In the graphs above you can see the *average* scores for different lengths of netlists on different amount of gates. Every graph has a linear increase. + +## Conclusions + +Within the scope of our experiment and by the data we were able to produce, we can say that the score function increases linearly with the length of a netlist. There are two possible explanations for this phenomenon: +1. The written program always produces the best score for a netlist (which has the shortest possible paths, so the score is only depending on the amount of paths that is drawn) +2. The score function has a simple nature. If it contained more variables, it could be that the score function behaves differently. + +## Discussion + +After completing this experiment, we found some ways to improve and extend this experiment in the future: +1. Increase the scope of the experiment by generating netlists with a length of 50 and up. This can be used to verify or disprove the credibility of this experiment. +2. Find a way to shorten the runtime of the program (runXX.py) to generate results faster. +3. Save more data of every solution than purely the score of the board. For example: + - if all connections were made + - runtime of each solution + - amount of layers needed + - a heatmap of busy junctions on the board \ No newline at end of file