Suggestion for prodigal gene-calling parallelization

[brymerr921]

Hi anvi'o developers,

I was wondering if you could modify the steps that use prodigal so that they break the input file into many roughly-equal-sized pieces (one per CPU requested) and run it in an embarrasingly parallel fashion. This will allow users to make much better use of multi-CPU systems and speed up gene-calling steps all in one go! Thanks so much.

Best,
Bryan

[ekiefl]

Hey @brymerr921, do you think that gene calling is a bottleneck in speed? I predicted gene calls for approximately 350,000,000 million genes in less than 6 hours the other week.

EDIT: it turns out I am exaggerating, that was with external gene calls being processed. Do you have any data on how long this step is taking?

[meren]

Hey @brymerr921, probably it is too late for your needs, but I still wanted to close this by mentioning that this is now addressed in master thanks to @mooreryan's recent PRs #1437, #1468, and #1445.