[dvsim] Various questions

[lmg260a]

Description

https://opentitan.org/book/hw/top_earlgrey/index.html says to execute this command, and it executes successfully.
NOTE: the HTML does not say which directory to run in, it would be useful if the user did not have to figure it out.

cd $(REPO_TOP)/hw/top_earlgrey; ../..//util/topgen.py -t data/top_earlgrey.hjson -o . -v

https://opentitan.org/book/hw/top_earlgrey/dv/index.html says to execute this command to simulate the UART.
cd $(REPO_TOP);./util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_uart_tx_rx

However, the following errors occur:
INFO: [dvsim] [proj_root]: /opentitan/Ubuntu1/opentitan
INFO: [SimCfg] [scratch_path]: [chip] [/opentitan/Ubuntu1/opentitan/scratch/master/chip_earlgrey_asic-sim-vcs]
ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "xcelium_ci_0" not found!
ERROR: [Modes] Test "chip_sw_aes_force_prng_reseed" added to regression "V2" not found!
ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "V2" not found!
ERROR: [Modes] Test "chip_sw_aes_prng_reseed" added to regression "V2" not found!
INFO: [StatusPrinter] [ legend ]: [Q: queued, D: dispatched, P: passed, F: failed, K: killed, T: total]
INFO: [StatusPrinter] 00:00:00 [ build ]: [Q: 0, D: 1, P: 0, F: 0, K: 0, T: 1] 0% chip:default
WARNING: [Deploy] chip:default: Job runtime not found in the log. Using dvsim-maintained job_runtime instead.
ERROR: [Scheduler] [00:00:01]: [build]: [status] [chip:default: F]
INFO: [StatusPrinter] 00:00:01 [ build ]: [Q: 0, D: 0, P: 0, F: 1, K: 0, T: 1] 100%
INFO: [StatusPrinter] 00:00:01 [ run ]: [Q: 0, D: 0, P: 0, F: 0, K: 5, T: 5] 100%
INFO: [FlowCfg] [results]: [chip]:

I am guessing that there are some intermediate steps required for generating the rest of the RTL, but it is not obvious to me what command(s) are required.

[andreaskurth]

Hi @lmg260a, thanks for reporting this issue.

https://opentitan.org/book/hw/top_earlgrey/index.html says to execute this command, and it executes successfully.
NOTE: the HTML does not say which directory to run in, it would be useful if the user did not have to figure it out.

cd $(REPO_TOP)/hw/top_earlgrey; ../..//util/topgen.py -t data/top_earlgrey.hjson -o . -v

We will update the documentation in PR #22251.

https://opentitan.org/book/hw/top_earlgrey/dv/index.html says to execute this command to simulate the UART.
cd $(REPO_TOP);./util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_uart_tx_rx

This command starts dvsim, which isn't a simulator by itself but interfaces different EDA tools that elaborate the RTL and run the simulation. The default simulator for Earlgrey top-level test is VCS. Do you have VCS installed and is it available on your $PATH?

However, the following errors occur:
INFO: [dvsim] [proj_root]: /opentitan/Ubuntu1/opentitan
INFO: [SimCfg] [scratch_path]: [chip] [/opentitan/Ubuntu1/opentitan/scratch/master/chip_earlgrey_asic-sim-vcs]
ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "xcelium_ci_0" not found!
ERROR: [Modes] Test "chip_sw_aes_force_prng_reseed" added to regression "V2" not found!
ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "V2" not found!
ERROR: [Modes] Test "chip_sw_aes_prng_reseed" added to regression "V2" not found!
INFO: [StatusPrinter] [ legend ]: [Q: queued, D: dispatched, P: passed, F: failed, K: killed, T: total]
INFO: [StatusPrinter] 00:00:00 [ build ]: [Q: 0, D: 1, P: 0, F: 0, K: 0, T: 1] 0% chip:default
WARNING: [Deploy] chip:default: Job runtime not found in the log. Using dvsim-maintained job_runtime instead.
ERROR: [Scheduler] [00:00:01]: [build]: [status] [chip:default: F]

The errors about "test X added to regression Y not found" aren't relevant for your problem (though they can be puzzling). The real problem is indicated by the last line I quoted: the build failed. Can you paste the build.log, the path of which should be printed on the lines just below the ones you pasted?

[lmg260a]

I don't have VCS - I do have Questasim, but it runs on CentOS, not Ubuntu. When I try to run build-only, I do get the qrun command, but when I try to run it, it errors out with this.

** Error: (qrun-7) Failed to open -f file "/opentitan/Ubuntu1/opentitan/scratch/master/uart-sim-questa/default/sim-vcs/lowrisc_dv_uart_sim_0.1.scr" in read mode.
No such file or directory. (errno = ENOENT)

I saw two problems:

1. it says "uart-sim-questa" ... "sim-vcs". So there's something that's being set to VCS when I've selected Questa. I've no idea where to find that.
2. I can't find any .scr file anywhere, for any simulator.

So what I did was try to do dry-run with VCS, so I could at least look at the VCS .scr file. But I don't get one generated from it either.

So I suspect that there's some command that I need to do, or change, so I get a .scr file for Questa or VCS or Verilator.

BTW - I do have all of this running in a Docker container running Ubuntu, and I've got it scripted to do all the installs in a Dockerfile and Makefile, in case that would be useful to you. Then other users will get up and running faster.

[andreaskurth]

While there is a Questa configuration for dvsim, we don't currently actively use it in the project and we thus don't officially support it.

When you run a top-level test (i.e., when running util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson [...]), the files for building and running the test should end up in in scratch/<branch-name>/chip_earlgrey_asic-sim-<vcs/xcelium/questa>/default/sim-<vcs/xcelium/questa>/. There you should find a file named lowrisc_dv_chip_sim_0.1.scr, which contains the list of include directories, defines, and files to be passed to the simulator for elaboration (flist). That flist gets generated by fusesoc, and I think the commands for that are here. This will show up in build.log as [make]: gen_sv_flist followed by the command on the next line. If you don't have the flist, please check the build log for errors.

[lmg260a]

Thanks - this was my build command; it didn't report pass or fail.
I got it from https://opentitan.org/book/hw/top_earlgrey/index.html

cd opentitan/hw/top_earlgrey; ../..//util/topgen.py -t data/top_earlgrey.hjson -o . -v

The last few lines were:
INFO: sram_ctrl_ret_aon.regs_tl --> ['peri.tl_sram_ctrl_ret_aon__regs']
INFO: sram_ctrl_ret_aon.ram_tl --> ['peri.tl_sram_ctrl_ret_aon__ram']
INFO: aon_timer_aon.tl --> ['peri.tl_aon_timer_aon']
INFO: sysrst_ctrl_aon.tl --> ['peri.tl_sysrst_ctrl_aon']
INFO: adc_ctrl_aon.tl --> ['peri.tl_adc_ctrl_aon']

I saw no errors, and only two warnings:
WARNING: main contains generated key reset_connections
WARNING: peri contains generated key reset_connections

When I attemped to run the sim, I got:

ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "xcelium_ci_0" not found!
ERROR: [Modes] Test "chip_sw_aes_prng_reseed" added to regression "V2" not found!
ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "V2" not found!
ERROR: [Modes] Test "chip_sw_aes_force_prng_reseed" added to regression "V2" not found!

So I can't tell what happened with the chip-level build. My guess is that it abruptly died.

[lmg260a]

when I attempt to run with Verilator on just the UART:
./util/dvsim/dvsim.py hw/ip/uart/dv/uart_sim_cfg.hjson -i uart_smoke --fixed-seed 123 --build-only --tool=verilator

I get
INFO: [dvsim] [proj_root]: /opentitan/Ubuntu1/opentitan
ERROR: [CfgFactory] '/opentitan/Ubuntu1/opentitan/hw/dv/tools/dvsim/verilator.hjson': Value for key 'run_dir' is '{scratch_path}/{run_dir_name}/out', but we already had a conflicting value of '{scratch_path}/{run_dir_name}/latest'.

But since it doesn't say what values are the problem, I don't know how to fix it.

[andreaskurth]

Thanks - this was my build command; it didn't report pass or fail. I got it from https://opentitan.org/book/hw/top_earlgrey/index.html

cd opentitan/hw/top_earlgrey; ../..//util/topgen.py -t data/top_earlgrey.hjson -o . -v

That command only needs to be run to generate the hw/top_earlgrey/rtl/top_earlgrey.sv module. It's checked into the repo, so as long as you don't change its sources, you don't have to run it.

When I attemped to run the sim, I got:

ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "xcelium_ci_0" not found! ERROR: [Modes] Test "chip_sw_aes_prng_reseed" added to regression "V2" not found! ERROR: [Modes] Test "chip_plic_all_irqs" added to regression "V2" not found! ERROR: [Modes] Test "chip_sw_aes_force_prng_reseed" added to regression "V2" not found!

So I can't tell what happened with the chip-level build. My guess is that it abruptly died.

As mentioned before, those errors aren't relevant for your problem. Do you have a build.log in your /opentitan/Ubuntu1/opentitan/scratch/master/uart-sim-questa/default/sim-vcs/ directory? If not, can you run find /opentitan/Ubuntu1/opentitan/scratch/master -name build.log? Please upload the newest build.log here.

[lmg260a]

I found opentitan/scratch/master/chip_earlgrey_asic-sim-vcs/default/build.log
File is attached.

build.log

[andreaskurth]

Your build.log shows this:

[make]: gen_sv_flist
cd /opentitan/Ubuntu1/opentitan/scratch/master/chip_earlgrey_asic-sim-vcs/default && fusesoc --cores-root /opentitan/Ubuntu1/opentitan run --flag=fileset_top --target=sim --build-root=/opentitan/Ubuntu1/opentitan/scratch/master/chip_earlgrey_asic-sim-vcs/default --setup lowrisc:dv:chip_sim:0.1
/bin/bash: line 1: fusesoc: command not found
make[1]: *** [/opentitan/Ubuntu1/opentitan/hw/dv/tools/dvsim/sim.mk:30: gen_sv_flist] Error 127

Please follow our Getting Started guide to prepare a development environment in which you can work with OpenTitan.

[lmg260a]

I walked through the Getting Started guide and implemented each step - but I missed adding export PATH=~/.local/bin:$PATH" to the standard project environment; I just did it in a single shell. So I've fixed that - thanks!

[lmg260a]

I reran
./util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson --dry-run --tool verilator -i chip_sw_uart_tx_rx

and get
INFO: [dvsim] [proj_root]: /opentitan/U2/opentitan
ERROR: [CfgFactory] '/opentitan/U2/opentitan/hw/dv/tools/dvsim/verilator.hjson': Value for key 'run_dir' is '{scratch_path}/{run_dir_name}/out', but we already had a conflicting value of '{scratch_path}/{run_dir_name}/latest'.

I don't see any new files in /scratch so this is all I have for debug.

[andreaskurth]

We only use and support dvsim with VCS and Xcelium for simulation at this time. Verilator can be used to run some tests that aren't based on UVM environments; see Verilator setup for more information.

[lmg260a]

I have Verilator working for the smoketest, how do I tell which tests will run in Verilator and which ones won't.
Is this error message really telling me that a test isn't supported for Verilator?

ERROR: [CfgFactory] '/opentitan/Ubuntu1/opentitan/hw/dv/tools/dvsim/verilator.hjson': Value for key 'run_dir' is '{scratch_path}/{run_dir_name}/out', but we already had a conflicting value of '{scratch_path}/{run_dir_name}/latest'.

[andreaskurth]

I have Verilator working for the smoketest

Good!

how do I tell which tests will run in Verilator and which ones won't.

bazel query 'attr(name, "sim_verilator$", "//sw/device/tests/...")' lists all tests that will run in Verilator.

Is this error message really telling me that a test isn't supported for Verilator?

ERROR: [CfgFactory] '/opentitan/Ubuntu1/opentitan/hw/dv/tools/dvsim/verilator.hjson': Value for key 'run_dir' is '{scratch_path}/{run_dir_name}/out', but we already had a conflicting value of '{scratch_path}/{run_dir_name}/latest'.

I think this rather tells us that dvsim currently doesn't work well with Verilator (independent of the test). We don't actively use and support simulations with Verilator through dvsim at this point.

[lmg260a]

I'm trying to get Questa to run, what do you recommend is the best approach to do that?
I don't have VCS, and I don't know dvsim or the infrastructure. But I do know Questa.
I tried using --dry-run on VCS and then compiling the same files into Questa, but I got a lot of missing RTL modules reported.
Is there possibly an area where RTL modules get compiled once and then referenced by VCS?