Designing for JVM heap size limitations

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Designing for JVM heap size limitations

Ravi Mundoli
Dear JGAP community

At our company, we've recently started playing around with JGAP to see if we can use it to solve a supply chain management problem (specifically, inventory optimization). We've designed a genetic algorithm implemented in JGAP which is showing good (i.e. close to optimal) results on a "toy" data set. Over the last couple of days, we've tried to run it against some production sized test data, and have started running into OutOfMemoryError issues in the JVM, even after max-ing the heap allocated. 

To be sure, we anticipated that we'd run into memory issues on our problem, but assumed that others would've faced similar issues before and solved it, so we weren't that worried. But a search of this mailing list and the website doesn't throw up too much by way of information. So we have a couple of questions/thoughts that we thought we'd throw out there to see if this list can help us. BTW, our chromosome at the moment is one big (200,000 elements) MapGene array, which we're modify to a custom, domain specific one shortly. Obviously managing a population of 1000 such chromosomes is going to cause problems.

1. Is there an out of the box or recommended way of handling this issue? Some best practices or utilities or whatever?

2. If the answer to #1 is "no", then our plan is to:

(i) create the initial population one by one or in some tractable batches and marshal the chromosomes to persist them in some scratch location in the file system
(ii) evolve the population on our own i.e. read 2 chromosomes (or some tractable number) at a time into memory, mutate/crossover, and marshal and persist again
(iii) run the fitness function evaluator on the now evolved population in the scratch file system location

etc. etc.

Is this feasible/advisable? If we override the JGAP evolve method and do all the above, in effect do we end up not even using JGAP for anything meaningful?

We'd really appreciate any inputs before we start down this path.

Best regards


Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization.
jgap-users mailing list
[hidden email]