You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nasher is currently single-threaded. While it is still fast because of the underlying tools, multithreading would increase the speed at which large modules could be processed.
The text was updated successfully, but these errors were encountered:
I had an initial exploration of this. I stuck with the unpack module since I knew it best. A few thoughts:
gffConvert appears to be the most appropriate process to parallelise, being repeatedly called over many files.
This requires gffConvert to be {.gcsafe.}. This traces through gffToJson/JsonToGff -> fatal -> error -> display* -> displayLine ->display Category*
Both of those procs with * above reference the cli object, which is a global and therefore blocking {.gcsafe.}. I temporarily replaced fatal and info (which also has a similar proc path) with echo just to test.
Spawning many gffConverts worked, and put the CPU to 100% and begins to convert at a high pace, however unfortunately something always snags. Most often I got was parseJson errors, I imagine related to the buffer being overwritten by another thread?
That's just my run of experimentation at the moment.
Nasher is currently single-threaded. While it is still fast because of the underlying tools, multithreading would increase the speed at which large modules could be processed.
The text was updated successfully, but these errors were encountered: