|
|
Heng Xing Tian Tai Lab of Xi'an City (abbr, HXTT)
HXTT ACCESS
|
java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-06 00:00:00 |
I am running out of heap space in my vm. I allocated 512 megs of heap space but the application still dies with 'java.lang.OutOfMemoryError: Java heap space' error after 1074 records out of 11051 total records are copied into the access database. The data that is copied is copied well. These records are deffiently on the the large size they contain 19 cloumns, 2 of which are CLOB fields and one is a Long VARCHAR. I'm not sure if you have a solution for this or if you have ever run into this. Any help would be greatly apperciated. The purpose of the system that I am trying to write is to do data extract for clients to be able to see a little more user friendly version of thier data currently stored in a db2 database. I am currently testing this on a windows XP box writting into an Access 2000 database from DB2 with java version 1.5.01 giving it the command line options -Xmx512m. I can work on paging this data into the access database a few hundred rows at a time but would prefer a different fix instead of adding the hack. Thank you very much for you time.
|
Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-06 00:00:00 |
I am running out of heap space in my vm. I allocated 512 megs of heap space but the application still dies with 'java.lang.OutOfMemoryError: Java heap space' error after 1074 records out of 11051 total records are copied into the access database. The data that is copied is copied well. These records are deffiently on the the large size they contain 19 cloumns, 2 of which are CLOB fields and one is a Long VARCHAR. I'm not sure if you have a solution for this or if you have ever run into this. Any help would be greatly apperciated. The purpose of the system that I am trying to write is to do data extract for clients to be able to see a little more user friendly version of thier data currently stored in a db2 database. I am currently testing this on a windows XP box writting into an Access 2000 database from DB2 with java version 1.5.01 giving it the command line options -Xmx512m. I can work on paging this data into the access database a few hundred rows at a time but would prefer a different fix instead of adding the hack. Thank you very much for you time.
|
Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Support |
2005-09-06 00:00:00 |
|
Are you writing data from DB2 to MS Access? Pease try Compact /Repair database in MS Access. Then try again. If your issue exists still, let us see your insert code. Thanks.
|
Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-07 00:00:00 |
I already tried the compare and compact and it didn't help here is code that I'm using
//create statment from DB2 connection;
Statement statement = conn.createStatement();
//get record set from DB2 query
ResultSet rs = statement.executeQuery(SQL);
String createTableSQL ="create table if not exists Aging_Level1 ?;";
//create prepared statement from Access connection
pstmt = accessConn.prepareStatement(createTableSQL);
pstmt.setObject(1,rs); //insert a resultSet into a new table abcd
pstmt.executeUpdate();
pstmt.close();
pstmt = null;
rs.close();
rs = null;
|
Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Support |
2005-09-08 00:00:00 |
>These records are deffiently on the the large size they contain 19 cloumns,
> 2 of which are CLOB fields and one is a Long VARCHAR. I'm not sure if you
> have a solution for this or if you have ever run into this. Any help would
> be greatly apperciated.
Try:
//get record set from DB2 query
//ResultSet rs = statement.executeQuery(SQL);
Modify your SQL temporarily to:
select colmun1,coulmn2,... from yourDB2Table, which column list doesn't contain CLOB and LONGVARCHAR column. If you succeeded in inserting all rows, please let us know. We will check our code for CREATE TABLE of MS ACCESS with CLOB and LOGVARCHAR column soon. Thanks for your patience.
|
Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Support |
2005-09-08 00:00:00 |
|
BTW, Aging_Level1 table is created by HXTT Access, or MS Access?
|
Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berrry |
2005-09-08 00:00:00 |
|
Aging_Level1 is created by HXTT Access
|
Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Support |
2005-09-08 00:00:00 |
|
Check your Access package whether is the latest package which has a date later than 2005-09-04. If not, please download the latest package and try again. BTW, you should drop Aging_Level1 table in MS Access first. HXTT Access will support DROP TABLE soon too.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-08 00:00:00 |
|
the Release date on the website says 09/07/2005. If this not the correct date or if there is anthor place to get the package date let me know.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Support |
2005-09-08 00:00:00 |
|
Yeah. It's the latest package. You can try it.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-08 00:00:00 |
even if i remove the 2 clob fields and the longvarchar i still get the same error and hxtt acces still insert the same number of rows before it dies.
The table signature in access is
Number(Long Integer),
Text(8),
Text(1),
Text(25),
Text(8),
Text(32),
Text(32),
Text(64),
Text(8),
Number(Double),
Text(8),
Text(32),
Date/Time,
Number(Long Integer),
Number(Long Integer),
Text(6)
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-08 00:00:00 |
|
We will try to recur your issue. Thanks for your information. After fix your issue, we will notify you. If we need more details, we will notify you too. Thanks.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-08 00:00:00 |
|
Something else that I discovered after putting the table signature in is that 2 fields don't actually keep the same lengths as what is in the DB2 database for text fields. The forst text field that is Text(8) in access is actualy char(16) in DB2. The text(25) is actualy a varchar(50). This is not a huge deal because all of the data is in the fields even though some of the data in the fields is longer then access allows. For example the smallest length charater string in the first Text(8) field is 14 chars all 14 characters are in the access database even though access says the max length is only 8. Not sure if this is related to the heap space issue or not but thought I'ld let you know.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Access |
2005-09-08 00:00:00 |
|
We will check too. Maybe it's resulted by DB2 uses ASCII length (one byte) as CHAR length, but ACCESS uses UNICODE length(two bytes) as CHAR length.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-11 00:00:00 |
>The forst text field that is Text(8) in access is actualy char(16) in DB2. The text(25) is actualy a varchar(50).
It's normal if you find that in MS Access 2000(or 2003) since it show the column length as UNICODE length. HXTT Access create a char(16) in fact.
>even if i remove the 2 clob fields and the longvarchar i still get the same
>error and hxtt acces still insert the same number of rows before it dies.
Try to catch java.lang.OutOfMemoryError ,and e.printStackTrace(). Maybe the error is thrown by DB2 JDBC driver. You can try stmt.setFetchDirection(10); for your DB2 connection. I have tested your code from HXTT DBF connection to HXTT Access connection for inserting 100,000 rows. The memory is maintained at a lower constant from 10MB to 15MB. BTW, please download the latest package of HXTT Access.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-11 00:00:00 |
|
BTW, please check the column length of Text(1) column in DB2. It's char(1) or char(2)?
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-12 00:00:00 |
The Text(1) column is actually a char(3) column in DB2.
I attempted to use stmt.setFetchDirection(10) but I got an error of illegal fetch direction. I thought that you might have meant stmt.setFetchSize(10) so I tried that and it had no effect on the OutOfMemoryError. I also tried to catch the OutOfMemoryError and do a stacktrace but this is not possible because once that error is thrown the JVM dumps and will not process any more code, including exception handling.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-12 00:00:00 |
>The Text(1) column is actually a char(3) column in DB2
Supported now. HXTT Access will use char(4) for MS Access compatibility.
>I thought that you might have meant stmt.setFetchSize(10) so I tried that
>and it had no effect on the OutOfMemoryError.
Sorry for the paste error. Recurred and fixed that OutOfMemoryError now:)
You can download the latest package after 2 hours:)
Thanks for your help.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-13 00:00:00 |
The outofmemorterror does not occur now. Thank you very much for the fix.
I am currently having an issue though with the table create lengths of text fields. You have mentioned the difference in length was due to the conversion from ASCII to Unicode. And in most the cases the smaller unicode charter lengths created in Access are not causing any issues because even though access says that the field is text(25) it still allows 50 characters. But I have run into a problem with on of my tables. I use the following code to create the table in access:
String SQL = "Select Author_name, Contact_id, PRG_id, case_dt, " +
"FUNDSRC_ID, SRVCUNIT_ID, srvc_units, srvc_hrs, " +
"trvl_hrs, " +
"admin_hrs, HV, collateral, PC, OFFC, OSA, QAMAILED, " +
"SRVC_CODE, ENTRYPOINT, UNIVERSALID as universalid, " +
"CASE_ID, SERVICETYPE, ORIGIN from CEventsList";
tableName = "CEventsList";
statement = conn.createStatement();
rs = statement.executeQuery(SQL);
String createTableSQL ="create table if not exists " + tableName + " ?;";
pstmt = accessConn.prepareStatement(createTableSQL);
pstmt.setObject(1,rs); //insert a resultSet into a new table <>
pstmt.executeUpdate();
Where the DB2 table Definition is(CEVentsList is a view acctually but this is what the table definition would be if it were a table):
create table CEVENTSLIST(
AUTHOR_NAME VARCHAR(64),
CONTACT_ID INTEGER(10),
PRG_ID INTEGER(10),
CASE_DT DATE,
FUNDSRC_ID VARCHAR(256),
SRVCUNIT_ID VARCHAR(256),
SRVC_UNITS DECIMAL(31,3),
SRVC_HRS DECIMAL(31,3),
TRVL_HRS DECIMAL(31,3),
ADMIN_HRS DECIMAL(31,3),
HV DECIMAL(31,3),
COLLATERAL DECIMAL(31,3),
PC DECIMAL(31,3),
OFFC DECIMAL(31,3),
OSA DECIMAL(31,3),
QAMAILED VARCHAR(256),
SRVC_CODE VARCHAR(256),
ENTRYPOINT VARCHAR(8) not null,
UNIVERSALID CHAR(50) not null,
CASE_ID CHAR(16),
SERVICETYPE VARCHAR(6) not null,
ORIGIN VARCHAR(6) not null);
The Access table definition created by hxtt access is:
create table CEVETSList(
Author_Name Text(32),
Contact_id Number(Long Int),
PRG_Id Number(Long Int),
CASE_Dt Date/Time,
fundsrc_id Text(128),
srvcunit_id Text(128),
srvc_units Number(Decimal),
srvc_hrs Number(Decimal),
trvl_hrs Number(Decimal),
admin_hrs Number(Decimal),
HV Number(Decimal),
Collateral Number(Decimal),
PC Number(Decimal),
OFFC Number(Decimal),
OSA Number(Decimal),
QAMailed Text(128),
SRVC_Code Text(128),
EntryPoint Text(4),
UniversalId Text(25),
CAse_Id Text(8),
ServiceType Text(3),
Origin Text(3));
Most of these columns the smaller text size is not an issue. But Entry Point is trimmed 6 chars when there are 8 char long entry's, ServiceType and Origin are trimmed to 4 chars when there are 6 char long entries.
Any help or advice would be greatly appreciated.
|
character length truncation |
Kevin Berry |
2005-09-13 00:00:00 |
I have located a simpler example of the perivous problem. I have noticed that all text fields in access are half the length of the corrisponding fields in DB2. I have found that the truncation occurs in a smaller refrence table. The access deffienation is:
Create table referadTo(
REFERREDTO number(long integer),
Desc TExt(24),
County Text(16),
State Text(1) )
The DB2 database definition is:
create table REFERREDTO(
REFERREDTO INTEGER(10) PRIMARY KEY not null,
DESC VARCHAR(48),
COUNTY VARCHAR(32),
STATE CHAR(2))
The the state field gets truncated to 1 character anf any field that has a text length longer then the access field length can not be edited. I get an error text is to long. Here is all of the data in the DB2 table so that you can recreate.
insert into refferalTo values
(0,' None','Monroe','NY'),
(1,'Alzheimer`s Support Group','Monroe','NY'),
(2,'EAPP','Monroe','NY'),
(3,'EISEP','Monroe','NY'),
(4,'Eldersource','Monroe','NY'),
(5,'Family Services of Rochester','Monroe','NY'),
(6,'Future Care Planning Services','Monroe','NY'),
(7,'JCC Fitness and Wellness Classes','Monroe','NY'),
(8,'JFS Transportation','Monroe','NY'),
(9,'Kinship Care','Monroe','NY'),
(10,'LS Financial Services','Monroe','NY'),
(11,'LS Title V','Monroe','NY'),
(12,'Ombudsman','Monroe','NY'),
(13,'Senior Center - Centro de Oro','Monroe','NY'),
(14,'Senior Center - Charles Settlement House','Monroe','NY'),
(15,'Senior Center - Just Friends','Monroe','NY'),
(16,'Senior Center-LS','Monroe','NY'),
(17,'Senior Center- Young at Heart','Monroe','NY'),
(18,'STAR','Monroe','NY'),
(19,'Unity Social Adult Day Care','Monroe','NY'),
(20,'SWAN','Monroe','NY'),21','Urban League','Monroe','NY'),
(22,'Mercy Center','Monroe','NY'),23','VNS MOW','Monroe','NY'),
(24,'Other','Monroe','NY'),
(25,'Senior Center','Monroe','NY'),
(26,'Geriatric Addictions Program','Monroe','NY'),
(27,'Safety and Security','Monroe','NY);
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-14 00:00:00 |
>Most of these columns the smaller text size is not an issue. But Entry Point is trimmed 6 chars when there are 8 char long entry's, ServiceType and Origin are trimmed to 4 chars when there are 6 char long entries.
Please download the latest package which won't trim the tail character of DB2 resultSet again. Now HXTT Access will map VARCHAR(n) to Text(n), but VARCHAR(256) will be mapped to Text(255). Text(255) can store 255 UNICODE characters, or 508 ANSI characters, so that your data won't be trimmed again.
BTW, you should drop the existent table referadTo first manually. HXTT ACCESS will support DROP TABLE soon.
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
HXTT Supports |
2005-09-14 00:00:00 |
|
:)
|
Re:Re:Re:Re:Re:Re:Re:Re:java.lang.OutOfMemoryError: Java heap space |
Kevin Berry |
2005-09-14 00:00:00 |
|
Thank you very much. I think that your product is excelent and has saved us a lot of time. As a fellow developer I understand that this is a working release and I am very happy with the follow up and attention that you have given me.
|
|
|
Main
webmaster@hxtt.com