CS 259 Accelerating Convolutional Neural Network

Fall 2024 CS 259 Lab 1Accelerating Convolutional Neural Network (CNN) on FPGAs using Merlin Compiler

Due October 9 11:59pm

DescriptionYour task is to accelerate the computation of two layers in a convolutional neural network(CNN) using a high-level synthesis (HLS) tool on an FPGA. We encourage you to start withusing the Merlin Compiler. For an input image with 228 × 228 pixels and 256 channels, youare going to calculate the tensor after going through a 2D convolution layer and a 2D maxpooling layer. The convolution layer has 256 filters of shape 256 × 5 × 5, uses the ReLUactivation relu(x) = max{x, 0} with a bias value for each output channel. The 2Dmaxpooling layer operates on 2 × 2 non-overlapping windows. You will need to implement this

function using HLS:void CnnKernel(const float* input, const float* weight, const float* bias, float*output)

where input is the input image of size [256][228][228], weight stores the weights of theconvolution filters of size [256][256][5][5], bias stores the offset values of size [256] thatwill be added to the output channels, and output should be written to by you as definedabove to store the result of maxpool(relu(conv2d(input, weight) + bias)). The outputsize is [256][112][112].How-ToFPGA accelerator compilation typically involves three (3) stages: high-level synthesis (HLS),

bitstream generation, and onboard execution. The last two stages can take days tocomplete. Therefore, in this lab, we only focus on the first stage: HLS. Your performance willonly be assessed using the estimation in the HLS reports, which is usually accurate.However, you are welcome to try out the last two steps if you are interested.

Connecting to the Server: Method 1

In this method, you won’t be able to run Merlin directly from your /home directory, so you’ll need copy files back and forth.Build and Run Baseline with Software SimulationWe have prepared the starter kit for you. Please run:makThis command will perform a software simulation of the provided starter FPGA HLS kernel. Itshould show “PASS”. You need to use FPGA Developer AMI in this lab unless you are using a computer with Xilinx Vitis HLS installation. However, you are still suggested to develop codeand run software simulation locally to test the correctness. You can move to AWS once youenter the tuning stageUnderstand the automatic Merlin’s optimizationBefore modifying the kernel and adding pragmas, synthesize the CNN kernel with Merlin anddescribe in your report the automatic optimizations made by Merlin andhow this reduceslatency.Modify the HLS CNN KernelIf you have successfully built and run the baseline HLS CNN kernel, you can now optimizethe code to design your CNN kernel. Your task is to 代 写CS 259 Accelerating Convolutional Neural Network  implement a fast, parallel version of thCNN kernel on FPGA. You should start with the provided starter kit. You should edit cnnkrnl.cpp for this task. When editing, please use the given types input_t, weight_t, bias_t,

and output_t for the corresponding data, and compute_t for your intermediate values.You can use them as if they are float numbers.

Parallelism should be exploited by using Merlin pragmas and tiling. You are encouraged to

focus on Merlin pragmas (#pragma ACCEL parallel, #pragma ACCEL pipeline and #pragmaACCEL tile). You can explicitly modify the code (tiling, loop permutation, …)but make surethe code modified is correctIn the starter kit, we simply wrap a sequential CNN code with #pragma ACCEL kernel, andMerlin automatically performs data caching, memory coalescing, pipelining andparallelization, which yield about 10 GFLOPs.Although the skeleton kernel is provided, you are also free to create your own by removingthe header file inclusion of “lib/cnn-krnl.h” and implement the basic kernel from scratch.However, this would require specific expertise in Xilinx FPGA architecture and is not

recommended for this course.Test Your HLS CNN Kernel with Software Simulation

To perform software emulation of your FPGA implementation of CNN kernel:makeIf you see something similar to the following message, your implementation is incorrect.Found 2120190 errorsFAILSince the software simulation step uses the CPU to emulate the hardware behavior, it only

erves as correctness test and its execution time doesn’t reflect that of actual hardware. Your

estimated execution time should be retrieved using the command below:

make estimate

This command will print out the estimated latency and resource usage of your kernel:

+---------------------------+------------------------+----------+----------+---------+--------+-------+------+

| Kernel | Cycles | LUT | FF | BRAM | DSP | URAM |Detail|

+---------------------------+------------------------+----------+----------+---------+--------+-------+------+

|CnnKernel (cnn-krnl.cpp:12)|4179564052 (16718.256ms)|49558 (4%)|49381 (2%)|810 (18%)|202 (2%)|25 (2%)|- |

+---------------------------+------------------------+----------+----------+---------+--------+-------+------+

The time highlighted in yellow is the estimated execution time of your FPGA kernel. You can

get the performance by “kNum*kNum*kImSize*kImSize*kKernel*kKernel*2/latency”, or164.4/latency (in s) to get the performance in GFLOPS.

IMPORTANT: Please make sure that all your loops have fixed loop bounds. If any of the loop bounds are variable, a performance estimation will not be shown and you will receive no

performance grade. IMPORTANT: The “make estimate” command should finish in 30 minutes, or in two hours

with highly-complex optimizations. Our recommendation is to halt your estimation usingCtrl-C when the time exceeds 30 minutes, except for your last step (after you reach ~100GOPS). More than 12 hours in the estimation will result in zero for the performance score. As your kernel design becomes more complex, thesoftware simulation and the estimation

will start to take a significantly longer time.IMPORTANT: As you apply more optimizations, your resource usage will also increase.Ideally, you should keep applying optimization until your kernel occupies about 80% of theseesources. The remaining 20% should be reserved for the interfaces DRAM/PCI-e controller)

and the downstream flows. Please make sure that resource utilization is less than 80% for allFPGA resources. If any of the resources are over this limit, you will receive no performance

grade.

IMPORTANT: You can check the HLS report by opening merlin.rpt with a text editor. Thisfile will be generated with the command make estimate. You must submit this file with your final submission. You should not modify this file in your submission, and it will be all verified

after submission due. Any modification to this file in your submission constitutes academic

misconduct and will be reported.

Advanced Tips for HLS

Kernel Profiling: If you want to “profile” your kernel, you can open merlin.rpt with a text

editor and scroll down to Performance Estimate. You can see the trip count, accumulated

ycles and cycles per call, as well as pipeline initiation interval and parallel factor for each

loop in the table. For resource usage, you can go to Resource Estimate. No loop levelinformation is available, though. If you want to check the resource usage of a code region,you can wrap it with a function then run again.

Kernel after transformation: If you want to see the kernel after being transformed by Merlinyou can look for that in .merlin_prj/run/implement/exec/hls/kernel.Annotation for Profiling: If you find the loops in your report hard to read, you can name theloops you are interested in using a goto label. For example, this_loop: for (int i = 0;i < n; i++);

Debugging Pipelining: If you are not sure about why you cannot achieve a specific initiation

interval as you expected, you can open the file below and read the logs. HLS usually gives out

a reason.

.merlin_prj/run/implement/exec/hls/_x/logs/CnnKernel/CnnKernel/vitis_hls.log

Long Synthesis Time In Pipelining: You will experience long HLS synthesis time (for

generating the estimation) if you pipeline a loop with a large loop body. Besides, please note

that as all loops inside a pipeline will be unrolled, it may be automatically a large loop body.

In this case, you may want to exchange the order of pipelining and unrolling and see if the time

can get improved.

Use Functions for Shorter Synthesis Time: If you experience long synthesis time, you may try

wrapping some loops into a function and specify #pragma HLS inline off inside the

function body. However, this may lead to inaccurate dependency analysis or memory port

analysis and cause lower performance sometimes. There might be some workarounds, or

not. For example, if you have access to A[k + i][j] inside the function, passing A + k to

the function and accessing A’[i][j] can allow HLS to understand the array partitioning

better than passing A. You need to do experiments.

General Tips

• When you develop on AWS, to resume a session in case you lose your connection, you

can run screen after login. You can recover your session with screen -DRR. You should

stop your AWS instance if you are going to come back and resume your work in a few

hours or days. Your data will be preserved but you will be charged for the EBS storage

for $0.10 per GB per month (with default settings). You should terminate your instance

if you are not going to come back and resume your work. Data on the instance will be

lost.

• You are recommended to use private repositories provided by GitHub to backup your

code. Never put your code in a public repo to avoid potential plagiarism. To check in

your code to a private GitHub repo, create a repo first.

git branch -m upstream

git checkout -b main # skip these two lines if you are reusing the folder in Lab 1

... // your modifications

git add cnn-krnl.cpp merlin.rpt

git commit -m "lab1: first version" # change commit message accordingly

# please replace the URL with your own URL

git remote add origin [email protected]:YourGitHubUserName/your-repo-name.gitgit push -u origin main

• You are recommended to git add and git commit often so that you can keep track of

the history and revert whenever necessary.

• Make sure your code produces correct results!

(Optional) Modify the HLS CNN Kernel using Vitis Pragmas

You are encouraged to use mainly Merlin pragmas. If needed, you can use Vitis pragmas for

finer-grained control and optimization. The list of pragmas in Vitis can be found here. You can simply write Vitis pragmas and Merlin pragmas in the same file (cnn-krnl.cpp), but note

that, to apply an HLS pragma to a loop, you need to put the pragma inside the loop body

instead of before it.

Submission

You need to report the estimated performance results of your FPGA-based implementation on

a Xilinx Ultrascale+ VU9P FPGA (the FPGA we are using, specified in the makefile). Pleaseexpress your performance in GFLOPS and the speedup compared with the starter-kit version.Your report should also include:

• Please run the input C file through the Merlin Compiler, identify the codetransformation and HLS pragmas that Merlin added, and discuss why.

• Please explain the parallelization and optimization strategies you have applied foreach loop in the CNN program (convolution, max pooling, etc) in this lab. Include thepragmas (if any) or code segments you have added to achieve your strategy.

• Please incrementally evaluate each parallelization/optimization that you have appliedand explain why it improves the performance.

• Please report the FPGA resources (LUT/FF/DSP/BRAM) usages, in terms of resource

count and percentage of the total. Which resource has been used most, in terms of

percentage?

• Optional: The challenges you faced, and how you overcame them.
• (Bonus +5pts): Analyze your code and check if the DSP/BRAM resource usage

matches your expectation. Only the adders, multipliers, and size of arrays need to be

considered. Please attach related code segments to your report and show how you

computed the expected number. Provide a discussion on possible reasons if theydiffer significantly.You also need to submit your optimized kernel code. Do not modify code in the lib directory.Please submit on Gradescope. Your final submission should contain and only contain these

files individually:

├ cnn-krnl.cpp

merlin.rpt

└ lab1-report.pdfFile lab1-report.pdf must be in PDF format.Grading PolicyYour submission will only be graded if it complies with the formatting requirements.

Missing reports/code or compilation errors will result in 0 for the corresponding category(ies).

Correctness (40%)Please check the correctness using the command “make”.Performance (40%)Your performance will be evaluated based on the estimation report generated using the

command “make estimate”. The performance point will be added only if you have the

rrect result, so please prioritize the correctness over performance. Your performance will

e evaluated based on the ranges of throughput (GOPS). Ranges A+ and A++ will be defined

after all the submissions are graded:

• Range A++, better than Range A+ performance: 40 points + 20 points (bonus)
• Range A+, better than Range A performance: 40 points + 10 points (bonus)
• Range A GFLOPS [200, 280]: 40 points
• Range B GFLOPS [120, 200): 30 points
• Range C GFLOPS [60, 120): 20 points
• Range D GFLOPS [30, 60): 10 points
• Lower than range F [0, 30): 0 pointsReport (20%)

Points may be deducted if your report misses any of the sections described above.

Academic IntegrityAll work is to be done individually, and any sources of help are to be explicitly cited. You mustnot modify the HLS report merlin.rpt in your submission. Any instance of academic

ishonesty will be promptly reported to the Office of the Dean of Students. Academicdishonesty includes, but is not limited to, cheating, fabrication, plagiarism, copying code from

other students or from the internet, modifying the software-generated report, or facilitatingacademic misconduct. We’ll use automated software to identify similar sections betweendifferent student programming assignments, against previous students’ code, or against

Internet sources. We’ll run HLS on all submissions and compare the reproduced HLS report with the submitted report. Students are not allowed to post the lab solutions on publicwebsites (including GitHub). Please note that any version of your submission must be yourown work and will be compared with sources for plagiarism detection.

Late policy: Late submission will be accepted for 24 hours with a 10% penalty. No lateubmission will be accepted after that (you lost all points after the late submission time).