An issue about the clock frequency

[lsq314]

Dear all,

It's a very very weird question that I met. I have successfully tried the "basic" design in this repo. Then, I try my kernels connecting to the networklayer kernel. So, the block design looks like cmac + networklayer + mykernels. I tried multiple clock settings: 1) all 100Mhz, 2) all 200Mhz, 3) cmac+networklayer: 300Mhz, mykernels: 200Mhz. 4) cmac: 300Mhz, networklayer + mykernels: 200Mhz. The xclbin files are successfully generated.

When I tested on u250 boards, as long as the IP is not in 300Mhz and I assign a value to a register of the IP (e.g., networklayer.set_ip, cmac.link_status(), mykernels.start()), I got the bus error and the weird thing: the FPGA chip reset (Yes, whole chip resets, no bitstream in the device using "xcbutil examine -d").

Is there any hint about this question? Thank you very much for any help and comments.

Best,
lsq

[mariodruiz]

Hi @lsq314,

Can you elaborate on the bus error?

This could indicate that your IP is not responding properly to the AXI4-Lite transactions. Have you tried to narrow down the the exact point the issue happens?

Mario

[lsq314]

Hi @mariodruiz

I am using PYNQ, so I only get the bus error info. Meanwhile, the networklayer IP in this project (if I change the clock to 200Mhz) also has the same issue.

Could you please provide some hints about how to narrow it down? Since I mainly focus on the architecture design, my experience is poor in this part.

Many thanks for your help.

Best,
lsq

[lsq314]

Hi @mariodruiz

I attached some info got from dmesg here. Don't know whether it helps.

Best,
lsq

[mariodruiz]

I don't think the frequency has to do with it. How are you writing your IP? What language? Have you fully verified it in simulation?

[lsq314]

Hi,

I wrote my ip using hls and pass the co-simulation.

I don’t know the reason. I mentioned the frequency because when I change the clock frequency of the networklayer ip in this repo from 200mhz to 300mhz (the setting 4 ->3 described in the first part of this issue). The error didn’t appear. By the way, I mean I can execute networklayer.setipaddr. When I call mykernel.start, the error also appears.

[mariodruiz]

Is there any scenario where your IP works when integrated on the design?

I think the problem could be when you define different frequencies for the IP, I would say to use always the same frequency for all of them.

[lsq314]

Hi @mariodruiz

Thanks for your reply. Because my kernels reply on the network part, they have not worked so far. Actually, as you said, I tried to set all of them to 200Mhz by modifying the 100G-network-stack and related ap_clk of cmac and networklayer. But it didn't work.

For now, I am trying to make it work if all of them run in 300Mhz. But I am afraid that if I target a bigger design in the future, it may be hard to achieve 300Mhz.

[mariodruiz]

There is no need to modify the logic in any of the kernels. You can define the frequency of the user logic with flags.

Uncomment this line and set your desired frequency and recompile.

xup_vitis_network_example/Makefile

Line 60 in d5147d8

#CONFIGFLAGS += --kernel_frequency 280

[lsq314]

Hi @mariodruiz

Yeah, I tried adding options like "--clock.default_freqhz" or "--clock.defaultid" or "--freqHz" in the makefile (btw, I use Vitis v2022.1). But it didn't work. So I further modified the clock settings in cmac and networklayer to see whether they would work.

[mariodruiz]

further modified the clock settings in cmac and networklayer to see whether they would work

How exactly did you do this?

Is your HLS code available anywhere?

[lsq314]

Hi @mariodruiz

Specifically, I modify "CONFIG.FREQ_HZ" and "set ap_clk" part in the file "bd_network_layer.tcl". And similar items in bd_cmac.tcl.

My HLS code is not available at the moment. However, I wonder whether this project (krnl_mm2s + krnl_s2mm + networklayer + cmac) can run at 200Mhz. I don't think my HLS kernels have a fundamental difference with krnl_mm2s. For my issue on the real board, my kernel didn't begin.

[mariodruiz]

Specifically, I modify "CONFIG.FREQ_HZ" and "set ap_clk" part in the file "bd_network_layer.tcl". And similar items in bd_cmac.tcl.

This does not have an effect in the frequency. The --kernel_frequency flag does.

I wonder whether this project (krnl_mm2s + krnl_s2mm + networklayer + cmac) can run at 200Mhz.

Yes, it can. There are projects that include AIE-128 and they run. https://github.com/OCT-FPGA/udp-network-demo

From what you're saying the problems only occur when you integrate your HLS IP.

[lsq314]

Oh, many thanks for your info. I will try the --kernel_frequency flag and refer to your shared project.

[lsq314]

Hi @mariodruiz

I tried to use the --kernel_frequency flag. But after adding it in the Makefile, I opened the Vivado project, and the clock frequency of all the kernels is 300Mhz (checking the ap_clk value in the block design). Could you please try running this? This issue really confuses me.

Steps to Reproduce (I am using Vitis v2022.1):

1. git clone https://github.com/Xilinx/xup_vitis_network_example.git --recursive
2. Modify the Makefile file: Uncomment line 60 and change 280 to 200.
3. make all DEVICE=/path-to-xilinx_u250_gen3x16_xdma_3_1_202020_1.xpfm DESIGN=basic INTERFACE=0

Many thanks for your time and help.

Best,
Lsq

[lsq314]

I rechecked the attached figures. ChatGPT told me that the "AXI Firewall tripped" thing should not occur. But I don't know why it appears.