ESP8266 support for websvf #2
Comments
|
HI
Thanks for fitting up the code - would you also like to try it?
I initially started it on ESP8266 JTAGing to lattice XP2.
It worked from SPIFFS but unreliable, often dropping the wifi
connection or rebooting.
Later the streaming version was developed on ESP32 which buffers packetized
SVF stream coming from wifi and on-the fly JTAGs it after suffictient
data arrive
for one complete SVF command. Such web-based JTAG upload has some
chance to work for 8266.
It's up to SVF generating application to generate SVF file with multiple short
commands nstead of one long single SVF command, because esp8266/esp32
can't buffer e.g. 4MB SVF content.
approximately speaking, one SVF cmd shoudn't be longer than 1-2K)
i
Lattice has suitable option for such SVF but altera and xilinx don't.
They generate all
in single command. Such SVF must be first postprocessed by 3rd party script or
application before seding to esp.
…On 7/8/18, udif ***@***.***> wrote:
I just left PR #1 to start this effort.
My ultimate target (I would love to hear your feedback) is to be able to
reuse old ESP8266-01 for JTAG programming via WiFi. I don't want to store
any bitstream file on the SPIFFS, just send it to the FPGA on-the-fly. If I
need to slow it a bit to get it to work reliably with WiFi, so be it.
Is this feasible with the single core ESP8266?
As I want to be able to reuse tons of old modules lying around, I really
hope to be able to squeeze it into 512KB SPI memories.
Currently websvf compiles to 373,160 bytes , hopefully leaving room for a
small SPIFFS just for the minimal read-only web files.
As I haven't used ESP8266 in the few years, I need to do some catch-up on
SPIFFS and other ESP8266 stuff, so I haven't ran this code yet other than
getting it to compile.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#2
|
|
I spent some time reading the Xilinx XAPP503 dealing with SVF and XSVF and I see several approaches):
The preprocessing is easy for the XSDRTDO* commands and can simply interleave the TDIvalue and TDOexpected bits, saving us from the need to save the entire TDI vector before shifting it in. I also understand that XSETSDRMASK and XSDRINC commands are deprecated and not used even by ISE (Let alone Vivado). These tools from OpenOCD might help (svf2xsvf.py and xsvfdump.py) I will take a deeper look into libXSVF code and see how it works. |
|
Using the OpenOCD contrib xsvfcdump.py I checked one of my XSVF files generated by ISE 14.7 and it seems that the maximum SDR length in the file is 1024 (128 bytes). |
|
HI
If your desired target is xilinx device with xsvf (and not svf) format, it's not
so easy
It would be best if there's some open source svf2xsvf application so we
can fine tune output format suitable for small esp32/8266 buffering
I've modified original cliffordwolf's svf parser to accept packetized
data stream. Data from wifi come in form of packets, normally sequential
from start to end of file. Each packet is passed as whole to the modified
svf parser. When it receives sufficient data that enclose one complete svf
command then it will execute it over jtag.
Something similar has to be done with XSVF parser - currently I think it is not
completely suitable because it originaly is done as old code, to
receive char-by-char
stream. Looking at how it is done for SVF code can similar be done for XSVF.
…On 7/9/18, udif ***@***.***> wrote:
I spent some time reading the Xilinx XAPP503 dealing with SVF and XSVF and I
see several approaches):
1. Allow only XSVF commands with a limited record length, and stop the
uploader if such a command is detected.
2. Start with an SVF file, and use the Xilinx SDF2XSVF utility to generate
an XSVF file with a limited record length
3. Preprocess the XSVF file, and do all the magic in the uploader, or in a
separate preprocessor.
Using the uploader is OK when you do command line processing, but if you
want to upload an XSVF by a drag-and-drop operation into a web page I don't
know if its possible (can you do the local preprocessing using Javascript on
the browser, or is the file always uploaded as-is?)
The preprocessing is easy for the XSDRTDO* commands and can simply
interleave the TDIvalue and TDOexpected bits, saving us from the need to
save the entire TDI vector before shifting it in.
The XTDOMASK command is more problematic since its value has no length limit
and needs to be used on all future TDO operations. The only way to cancel
the buffering is to add an explicit TDO mask to every XSDRTDO* command
preprocessed, potentially increasing the bitstream length.
I also understand that XSETSDRMASK and XSDRINC commands are deprecated and
not used even by ISE (Let alone Vivado).
These tools from OpenOCD might help (svf2xsvf.py and xsvfdump.py)
https://github.com/arduino/OpenOCD/tree/master/contrib/xsvf_tools
This one is based on libXSVF:
https://github.com/makestuff/dump-xsvf
I will take a deeper look into libXSVF code and see how it works.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2 (comment)
|
|
128 bytes is small enough. Actualy 1024 bytes would be still ok.
Idea is not to buffer too many ethernet packets of 1492 bytes
before JTAGing because JTAG processing by bitbang will accupy
ESP CPU which may miss CPU cycles required to keepalive wifi
connection
…On 7/9/18, The EMARD ***@***.***> wrote:
HI
If your desired target is xilinx device with xsvf (and not svf) format, it's
not
so easy
It would be best if there's some open source svf2xsvf application so we
can fine tune output format suitable for small esp32/8266 buffering
I've modified original cliffordwolf's svf parser to accept packetized
data stream. Data from wifi come in form of packets, normally sequential
from start to end of file. Each packet is passed as whole to the modified
svf parser. When it receives sufficient data that enclose one complete svf
command then it will execute it over jtag.
Something similar has to be done with XSVF parser - currently I think it is
not
completely suitable because it originaly is done as old code, to
receive char-by-char
stream. Looking at how it is done for SVF code can similar be done for
XSVF.
On 7/9/18, udif ***@***.***> wrote:
> I spent some time reading the Xilinx XAPP503 dealing with SVF and XSVF and
> I
> see several approaches):
>
> 1. Allow only XSVF commands with a limited record length, and stop the
> uploader if such a command is detected.
> 2. Start with an SVF file, and use the Xilinx SDF2XSVF utility to
> generate
> an XSVF file with a limited record length
> 3. Preprocess the XSVF file, and do all the magic in the uploader, or in
> a
> separate preprocessor.
> Using the uploader is OK when you do command line processing, but if you
> want to upload an XSVF by a drag-and-drop operation into a web page I
> don't
> know if its possible (can you do the local preprocessing using Javascript
> on
> the browser, or is the file always uploaded as-is?)
>
> The preprocessing is easy for the XSDRTDO* commands and can simply
> interleave the TDIvalue and TDOexpected bits, saving us from the need to
> save the entire TDI vector before shifting it in.
> The XTDOMASK command is more problematic since its value has no length
> limit
> and needs to be used on all future TDO operations. The only way to cancel
> the buffering is to add an explicit TDO mask to every XSDRTDO* command
> preprocessed, potentially increasing the bitstream length.
>
> I also understand that XSETSDRMASK and XSDRINC commands are deprecated
> and
> not used even by ISE (Let alone Vivado).
>
> These tools from OpenOCD might help (svf2xsvf.py and xsvfdump.py)
> https://github.com/arduino/OpenOCD/tree/master/contrib/xsvf_tools
> This one is based on libXSVF:
> https://github.com/makestuff/dump-xsvf
>
> I will take a deeper look into libXSVF code and see how it works.
>
>
> --
> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub:
> #2 (comment)
|
|
Here is my progress for the last few days: I also spent some time working on the (X)SVF code. I cannot simply use SVF because ISE 14.7 generates SVF files with a single SDR command for the entire FPGA configuration bitsream. Only the XSVF output uses smaller data blocks (1024 bytes). Also , there is a log of code duplication between the packetized and original SVF drivers. For code, see https://github.com/udif/LibXSVF-ESP/tree/add_tests |
Done. Liberally turned some auto vars into static. |
|
HI
Glad to hear about this great progress with XSVF!
If you'd like to clean up the code please do so!
I might like to check after this new changes how the code
work for my ESP32-lattice boards. I think I have small spartan-6
board and nodemcu's around so I could assemble a setup to test XSVF too
ESP01 has very few pin count, great if you can fit JTAG there :)
for this I prefer ose of ESP12E in breakout modules or nodemcu 0.9/1.0
What do you think, can ESP SPI or some other DMA be used as accelerator for
JTAG, in many long JTAG traansfer signals look similar to SPI transfer when
TMS is not changing.
…On 7/17/18, udif ***@***.***> wrote:
> Also , there is a log of code duplication between the packetized and
> original SVF drivers.
> I will try to factor that out to reduce code size and simplify it. The new
> xsvftool-test will let me regress any changes so I won't break the code.
>
Done. Liberally turned some auto vars into static.
I assume that after each upload I may need to call ESP.restart() so that
each upload gets a fresh copy of all the static vars.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2 (comment)
|
|
Quick update: Regarding your comment on the DMA engine, I'm familiar with this. The issues with using DMA is that it uses a fixed pinout requiring me to either pull wires directly from the chip and not the existing 4 IOs on the ESP01 connector. |
|
HI
Yes the function has to be repeatedy called any number times,
in each time supplying one ethernet packet from the data stream.
I think also my svf edition is full of static vars, ugly and non-reentrant
too. Hopfully not too buggy also.
I think ESP doesn't need to reenter at all (no recursion, no linux, no threads)
all it has to do is process one packet as fast as possible with buffering,
parsing and JTAG-ing and return control to outside loop for uninterrupted
wifi connection. Also this ESP realloc is suspicious to me as it probably
affects stability and causes ESP reboots; frequency of calling realloc should
be reduced.
…On 7/22/18, udif ***@***.***> wrote:
Quick update:
I've heavily modified libxsvf_xsvf() on my branch to work with streaming
data.
The getbyte() function is now allowed to return -2 to indicate data not yet
available.
In this case libxsvf_xsvf() will also return -2, and you are expected to
subsequently call it again with the same parameters once new data is
available.
The price I had to pay for this change was to make the code ugly and
non-reentrant, full of static vars.
I definitely want to clean it further, as there is lots of duplication that
can be moved into the READ_xx() macros.
It is also possible to move all the static vars to a context struct, giving
us back the reentrancy.
All of this was tested on the host (linux) only.
I now need to integrate this with the esp8266 example and test with real
hardware.
Regarding your comment on the DMA engine, I'm familiar with this. The issues
with using DMA is that it uses a fixed pinout requiring me to either pull
wires directly from the chip and not the existing 4 IOs on the ESP01
connector.
As for whether the performance increase will be worth it (if at all - DMA
setup also takes time) remains to be seen. This will definitely wait until I
have something simple working with bit banging.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2 (comment)
|
|
I haven't implied that ESP needs reentrant code. In the meantime, I did some cleanup on libxsvf_xsvf() and moved the retry logic into trhe READ_xxx() macros, making the code much simpler. |
|
Yes indeed, original code was much cleaner.
Looking in this direction, moving this static
vars to a struct like libxsvf_host might
make it reentrant again
…On 7/22/18, udif ***@***.***> wrote:
I haven't implied that ESP needs reentrant code.
Given the low memory capacity and the fact that we a single task run
bare-metal program, non reentrant code is OK.
However, what I meant was that the original code in LibXSVF was written to
be much cleaner -
All variables are local, and all the state is encapsulated in the
libxsvf_host structure.
This is OOP done in C, with libxsvf_host as the class, and the function
pointers instead of virtual functions.
You could run 2 parallel JTAG interface using the same library and different
libxsvf_host structures.
Ofcourse, this is irrelevant for the ESP.
In the meantime, I did some cleanup on libxsvf_xsvf() and moved the retry
logic into trhe READ_xxx() macros, making the code much simpler.
--
You are receiving this because you commented.
Reply to this email directly or view it on GitHub:
#2 (comment)
|
I just left PR #1 to start this effort.
My ultimate target (I would love to hear your feedback) is to be able to reuse old ESP8266-01 for JTAG programming via WiFi. I don't want to store any bitstream file on the SPIFFS, just send it to the FPGA on-the-fly. If I need to slow it a bit to get it to work reliably with WiFi, so be it.
Is this feasible with the single core ESP8266?
As I want to be able to reuse tons of old modules lying around, I really hope to be able to squeeze it into 512KB SPI memories.
Currently websvf compiles to 373,160 bytes , hopefully leaving room for a small SPIFFS just for the minimal read-only web files.
As I haven't used ESP8266 in the few years, I need to do some catch-up on SPIFFS and other ESP8266 stuff, so I haven't ran this code yet other than getting it to compile.
The text was updated successfully, but these errors were encountered: