Hi,
I'm testing java 1.6 and have a problem with the HXTT CSV driver. Previously when doing a union all on big tables the memory consumption of the application was fairly stable (java 1.5.x). When using java 1.6 the memory jumps up to 1.5GB usage and the union all becomes very slow of course. Is there a version that can be used with java 1.6 that doesn't have this problem?
Thanks,
Frank
|
Hi,
Seems after all that it's not really related to the java version. Once the files reach a certain size, all memory is consumed and things slow to a crawl. At least for a "union all" that does not seem necessary because it's merely an glorified append of files. Can you please have a look?
Thanks,
Frank
|
> At least for a "union all" that does not seem necessary because it's merely
> an glorified append of files.
Yeah. Sometimes UNIION ALL will use where clause, ORDER BY clause and ResultSet.absolute() , so that HXTT does a simple design which maybe consume some large memory to do union all sometimes. Don't worry:)
Please show your frequent UNION ALL sql sample, and we will optimize it to avoid large memory issue. I guess that should be solved in 3 days:)
|
Hi,
It's really a very simple query that is being performed. Apparently when the file size reaches a certain size things start to slow down.
Example:
select * from A
union all
select * from B
Frank
|
Main
webmaster@hxtt.com