Main   Products   Offshore Outsourcing   Customers   Partners   ContactUs  
JDBC Databases
  HXTT Access v5.2
  HXTT Cobol v2.1
  HXTT DBF v5.2
  HXTT Excel v4.2
  HXTT Paradox v5.2
  HXTT Text(CSV) v5.2
 
  Buy Now
  Support
  Download
  Document
  FAQ
  HXTT XML v1.2
Offshore Outsourcing
Oracle Data Import/Export
DB2 Data Import/Export
Sybase Data Import/Export
Free Resources
  Firewall Tunneling
  Search Indexing Robot
  Conditional Compilation
  Password Recovery for MS Access
  Password Recovery for Corel Paradox
  Checksum Tool for MD5
  Character Set Converter
  Pyramid - Poker of ZYH
   
   
   
Hongxin Technology & Trade Ltd. of Xiangtan City (abbr, HXTT)

HXTT Text(CSV)
Performance issue when the CSV file grows
Carmen Rodriguez
2016-02-17 09:41:59.0
Dear colleagues,

We are developing a standalone application that is batch inserting record in CSV tables. It uses HXTT CSV and Hibernate (with the com.hxtt.support.hibernate.HxttTextDialect). The data model is complex with many relationships between entities. We have observed that when the CSV files starts growing (around 10000 lines) the performance decreases a lot.

After 40 minutes of execution, the time taken to perform a flush() is multiplied by 10.

Could you help us with this issue?

Kind regards,
Carmen
Re:Performance issue when the CSV file grows
HXTT Support
2016-02-18 05:47:40.0
CSV file need to parse first. You can use delayedClose=120 for connection property.
delayedClose: Indicates the delayed seconds for close transaction. That option is used to avoid frequent close/open table operations for following sqls. Automatic temporary index is disabled when delayedClose<=60s. You can use 0~120 seconds. Default: 3.

Search Key   Search by Last 50 Questions




Google
 

Address: 9 Station Rd., Xiangtan City, Hunan Province, P.R. China
Postcode: 411100
Phone: (86)731-58225727
Fax: (86)731-58225727
Email: webmaster@hxtt.com
Copyright © 1999-2011 Hongxin Technology & Trade Ltd. | All Rights Reserved. | Privacy | Legal | Sitemap