Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decrease Markov network memory footprint #413

Open
ForNeVeR opened this issue Nov 23, 2016 · 2 comments
Open

Decrease Markov network memory footprint #413

ForNeVeR opened this issue Nov 23, 2016 · 2 comments

Comments

@ForNeVeR
Copy link
Member

I believe that currently Markov network images are consuming more memory that it's needed for them. We need to profile Horta memory and decrease network memory footprint if that's possible.

@ForNeVeR
Copy link
Member Author

Please first of all check #134 when implementing.

@ForNeVeR
Copy link
Member Author

ForNeVeR commented Dec 4, 2016

Some optimization points (things to consider):

  1. Intern strings (words) after parsing the file.
  2. Intern phrase vectors inside Platonus library.
  3. Make a single parser actor (so it won't eat too much memory while parsing multiple networks at once).
  4. Progressively load strings from the database (don't load them all at once).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant