Merge pull request #2479 from verilog-to-routing/noc_qor_doc_issue

Noc QoR measurement

Author: vaughnbetz

.gitignore

@@ -38,6 +38,19 @@ vtr_flow/benchmarks/titan_blif/titan23
 vtr_flow/benchmarks/titan_blif/titan_new
 
 
+#
+# NoC MLP benchmarks
+#
+#   We ignore blif and vqm files because of thier large size.
+#   We also ignore symbolic links to traffic flow and blif files.
+#
+vtr_flow/benchmarks/noc/Large_Designs/MLP/**/*.vqm
+vtr_flow/benchmarks/noc/Large_Designs/MLP/**/*.blif
+vtr_flow/benchmarks/noc/Large_Designs/MLP/blif_files/*
+vtr_flow/benchmarks/noc/Large_Designs/MLP/traffic_flow_files/*
+MLP_Benchmark_Netlist_Files_blif.tar.gz
+MLP_Benchmark_Netlist_Files_vqm_blif.tar.gz
+
 #
 # ISPD benchmarks
 #

CMakeLists.txt

@@ -341,6 +341,14 @@ add_custom_target(get_titan_benchmarks
     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
     COMMENT "Downloading (~1GB) and extracting Titan benchmarks (~10GB) into VTR source tree.")
 
+#
+# NoC MLP Benchmarks
+#
+add_custom_target(get_noc_mlp_benchmarks
+        COMMAND ./vtr_flow/scripts/download_noc_mlp.py --vtr_flow_dir ./vtr_flow
+        WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
+        COMMENT "Downloading (~100MB) and extracting NoC MLP benchmarks (~3.2GB) into VTR source tree.")
+
 #
 # ISPD Benchmarks
 #

README.developers.md

@@ -4,7 +4,7 @@ Overview
 The VTR documentation is generated using sphinx, a python based documentation generator.
 
 The documentation itself is written in re-structured text (files ending in .rst), which
-is a lightwieght mark-up language for text documents.
+is a lightweight mark-up language for text documents.
 
 Currently VTR's documenation is automatically built by https://readthedocs.org/projects/vtr/ and is served at:
 
@@ -36,7 +36,7 @@ from the main documentation directory (i.e. <vtr_root>/doc).
 
 This will produce the output html in the _build directory.
 
-You can then view the resulting documention with the web-browser of your choice.
+You can then view the resulting documentation with the web-browser of your choice.
 For instance:
 
     $ firefox _build/html/index.html

doc/README

@@ -12,15 +12,17 @@ Benchmark Structure:
 		|---<Benchmark>.flows - Is the NoC traffic flows file associated with the given benchmark
 					(A benchmark can have multiple traffic flows files)
 		|---verilog	      - Contains design files needed to generate the netlist file for the benchmark
-	|---shared_verilog	      - Contains design files needed by all benchmarks to generate thier netlist files
+	|---shared_verilog	      - Contains design files needed by all benchmarks to generate their netlist files
+	|---blif_files	          - Contains symbolic links to all .blif files that exist in this directory
+    |---flow_files	          - Contains symbolic links to all .flow files that exist in this directory
 
 Running the benchmarks:
 	Pre-requisite
 		- Ensure VPR is built (refer to 'https://docs.verilogtorouting.org/en/latest/' for build instructions)
 		- Set 'VTR_ROOT' as environment variable pointing to the location of the VTR source tree
 		- Ensure python version 3.6.9 or higher is installed
 		- Copy over the netlist files from 'https://drive.google.com/drive/folders/135QhmfgUaGnK2ZEfbfEXtdm1BfS7YoG7?usp=sharing'.
-	          The file structure in the previous link is similiar to structure found in '$VTR_ROOT/vtr_flow/benchmarks/noc/Large_Designs/MLP'.
+	          The file structure in the previous link is similar to structure found in '$VTR_ROOT/vtr_flow/benchmarks/noc/Large_Designs/MLP'.
 		  Place the netlist files in the appropriate folder locations.
 
 	Running single instance:
@@ -48,7 +50,7 @@ Running the benchmarks:
 		  -vpr_executable $VTR_ROOT/build/vpr/vpr --device EP4SE820 -flow_file $VTR_ROOT/vtr_flow/benchmarks/noc/Large_Designs/MLP/MLP_1/mlp_1.flows \
 		  -noc_routing_algorithm xy_routing -number_of_seeds 5 -number_of_threads 1 -route
 
-	        - The above command will generate an output file in the run directory that contains all the place and route metrics. This is a txt file with a name which matches the
+	        - The above command will generate an output file in the run directory that contains all the place and route metrics. This is a txt file with a name which matches
 		  the flows file provided. So for the command shown above the output file is 'mlp_1.txt'
 
 	Special benchmarks:
@@ -64,8 +66,14 @@ Running the benchmarks:
 			  of the NoC routers needs to be locked. A 
 			- To run a single instance of this benchmark, pass in the following command line parameter and its value to the command shown above:
 			  '--fix_clusters $VTR_ROOT/vtr_flow/benchmarks/noc/Large_Designs/MLP/MLP_2_phase_optimization/MLP_2_phase_optimization_step_2/MLP_two_phase_optimization_step_two_constraints.place'
-			- To run the benchmarkusing the automated script just pass in the following command line parameter and its value to the script command above:
+			- To run the benchmarking the automated script just pass in the following command line parameter and its value to the script command above:
 			  '-fix_clusters $VTR_ROOT/vtr_flow/benchmarks/noc/Large_Designs/MLP/MLP_2_phase_optimization/MLP_2_phase_optimization_step_2/MLP_two_phase_optimization_step_two_constraints.place'
+
+	Running VTR tasks:
+        - All synthetic benchmarks can be run as VTR tasks. Example tasks are provided in vtr_flow/tasks/noc_qor
+        - Instructions on how to run VTR tasks to measure QoR for NoC benchmarks in available in VTR Developer Guide.
+        - Link to VTR Developer Guide: https://docs.verilogtorouting.org/en/latest/README.developers/#example-noc-benchmarks-qor-measurements
+
 	Expected run time:
 		- These benchmarks are quite large so the maximum expected run time for a single run is a few hours
 		- To speed up the run time with multiple VPR runs the thread count can be increased from 1. Set thread count equal to number seeds for fastest run time.

doc/src/vtr/benchmarks.rst

@@ -8,7 +8,9 @@ Benchmark Structure:
 		|---<Benchmark>.flows - Is the NoC traffic flows file associated with the given benchmark
 					(A benchmark can have multiple traffic flows files)
 		|---verilog	      - Contains design files needed to generate the netlist file for the benchmark
-	|---shared_verilog	      - Contains design files needed by all benchmarks to generate thier netlist files
+	|---shared_verilog	      - Contains design files needed by all benchmarks to generate their netlist files
+	|---blif_files	          - Contains symbolic links to all .blif files that exist in this directory
+	|---flow_files	          - Contains symbolic links to all .flow files that exist in this directory
 
 Running the benchmarks:
 	Pre-requisite
@@ -42,7 +44,12 @@ Running the benchmarks:
 		  -noc_routing_algorithm xy_routing -noc_swap_percentage 40 -number_of_seeds 5 -number_of_threads 1
 
 	        - The above command will generate an output file in the run directory that contains all the place and route metrics. This is a txt file with a name which matches the
-		  the flows file provided. So for the command shown above the outout file is 'complex_2_noc_1D_chain.txt'
+		    flows file provided. So for the command shown above the output file is 'complex_2_noc_1D_chain.txt'
+
+    Running VTR tasks:
+        - All synthetic benchmarks can be run as VTR tasks. Example tasks are provided in vtr_flow/tasks/noc_qor
+        - Instructions on how to run VTR tasks to measure QoR for NoC benchmarks in available in VTR Developer Guide.
+        - Link to VTR Developer Guide: https://docs.verilogtorouting.org/en/latest/README.developers/#example-noc-benchmarks-qor-measurements
 
 	Expected run time:
 		- These benchmarks are quite small so the maximum expected run time for a single run is ~30 minutes

vtr_flow/benchmarks/noc/Large_Designs/MLP/Readme.txt

@@ -0,0 +1 @@
+.././complex_16_noc_1D_chain/complex_16_noc_1D_chain.blif

vtr_flow/benchmarks/noc/Synthetic_Designs/Readme.txt

@@ -0,0 +1 @@
+.././complex_2_noc_1D_chain/complex_2_noc_1D_chain.blif

vtr_flow/benchmarks/noc/Synthetic_Designs/blif_files/complex_16_noc_1D_chain.blif

@@ -0,0 +1 @@
+.././complex_32_noc_clique/complex_32_noc_clique.blif