Denny | 2015-02-03 05:51:34.0 | |
| Dear hxtt,
i have encountered a problem within multithreading environment with csv driver, a simple select does not give correct result anymore (double values, missing values) Problems: 1. multi threaded query gives wrong result (367454 vs 280202) 2. single threaded query much more slower than older version (2437 ms vs 40861ms) 3. multi threaded query totally failed using old driver i attach test code below & my tests result using new and older driver version thank you for assistance regards A. RESULTS using New Driver (1,447,440 bytes) SINGLE THREADED rowcount = 367454, UNIQUE = 367454 time=40861ms MULTI THREADED rowcount = 367323, UNIQUE = 280202 time=58761ms rowcount = 367293, UNIQUE = 281081 time=58770ms B. RESULTS using Old Driver (1,324,231 bytes) SINGLE THREADED rowcount = 367454, UNIQUE = 367454 time=2437ms MULTI THREADED java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at com.hxtt.concurrent.o.a(Unknown Source) at com.hxtt.sql.text.b.aN(Unknown Source) at com.hxtt.sql.text.b.a(Unknown Source) at com.hxtt.sql.text.l.a(Unknown Source) at com.hxtt.sql.a7.a(Unknown Source) at com.hxtt.sql.c9.a(Unknown Source) at com.hxtt.sql.c9.a(Unknown Source) at com.hxtt.sql.fg.x(Unknown Source) at com.hxtt.sql.d2.a(Unknown Source) at com.hxtt.sql.d2.a(Unknown Source) at com.hxtt.sql.dg.new(Unknown Source) at com.hxtt.sql.dg.next(Unknown Source) at Test.p1(Test.java:152) at Test.access$000(Test.java:7) at Test$1.run(Test.java:125) at java.lang.Thread.run(Thread.java:662) rowcount = 366899, UNIQUE = 335508 time=1766ms test code: public static void main(String [] args) throws Throwable{ System.out.println("SINGLE THREADED"); p1(); System.out.println("MULTI THREADED"); new Thread(new Runnable() { public void run() { p1(); } }).start();; p1(); } private static void p1() { try { long t = System.currentTimeMillis(); Driver d = (Driver)Class.forName("com.hxtt.sql.text.TextDriver").newInstance(); Connection connection = d.connect("jdbc:CSV:///c:\\afolder?_CSV_Header=true;_CSV_Separator=\\u007C;csvfileExtension=csv;_CSV_Quoter=;_CSV_EOL=\\u000A",null); HashSet v = new HashSet(); int n=0; try { PreparedStatement PS = connection.prepareStatement("select * from test_case_mt"); try { ResultSet RS = PS.executeQuery(); try { int i=0; while (RS.next()) { i++; //System.out.println(i+". "+RS.getString(1)); v.add(RS.getString(1)); n++; } } finally { RS.close(); } } finally { PS.close(); } } finally { connection.close(); } System.out.println("rowcount = "+n+", UNIQUE = "+v.size()+" time="+(System.currentTimeMillis()-t)+"ms"); } catch (Throwable e) { e.printStackTrace(); } } | ||
HXTT Support | 2015-02-04 22:35:44.0 | |
| >B. RESULTS using Old Driver (1,324,231 bytes)
What's the released date? >1. multi threaded query gives wrong result (367454 vs 280202) Try soloMode=false connection propery. | ||
Denny | 2015-02-04 23:41:53.0 | |
| tried with soloMode, it gives correct result (both old & new driver)
however, performance still bad with the new driver almost 10x slower i dont have the release date of old driver, and its not listed at download page anymore but the files inside the jar has the date of 3 jan 2013 i have emailed it to webmaster@hxtt.com thank you regards A. RESULTS using New Driver (1,447,440 bytes) : SINGLE THREADED rowcount = 367454, UNIQUE = 367454 time=26531ms MULTI THREADED rowcount = 367454, UNIQUE = 367454 time=50089ms rowcount = 367454, UNIQUE = 367454 time=50108ms B. RESULTS using Old Driver (1,324,231 bytes) SINGLE THREADED rowcount = 367454, UNIQUE = 367454 time=2413ms MULTI THREADED rowcount = 367454, UNIQUE = 367454 time=4039ms rowcount = 367454, UNIQUE = 367454 time=4047ms | ||
Denny | 2015-02-10 11:48:00.0 | |
| hello,
are you going to fix the performance issue above ? im currently falling back to the old version because the new version is too slow regards | ||
HXTT Support | 2015-02-16 00:07:15.0 | |
| Supported. Please download the latest package. Both of soloMode=true and soloMode=false can be used. | ||