• Welcome, Guest. Please login.
 
September 17, 2019, 11:01:04 am

News:

Welcome to the SQLitening support forums!


Addendum to 'a BIG than you'

Started by Peter Presland, December 11, 2008, 04:39:27 pm

Previous topic - Next topic

Peter Presland

I have a trial installation of SQLite Expert Pro (It has a text file import facility). I tied importing the same text file and aborted after an hour. It had still only reached 1.5 million. I presume therefore that it does not use the "Begin Immediate Transaction" syntax when importing text files. That together with displaying the escalating 'No of records imported' number, the display overhead of which I presume gobbles up a good few milliseconds per record too.

Maybe my usage is a bit extreme but, nonetheless I reckon they'd do themselves a favour by addressing that import speed issue - unless of course there is a User tweekable parameter that I've missed somewhere - which is very likely.

Fred Meier

I don't have the Pro version so can't do an Import, but the Personal version has Transactions on the main menu.  Does Begin Transaction followed by your Import and then followed by Commit (or Rollback) run faster?

Peter Presland

December 12, 2008, 02:03:31 pm #2 Last Edit: December 12, 2008, 02:13:47 pm by Peter Presland
Fred

If I menu select 'Begin transaction' then try to initiate an import, an error 'Transaction already in progress' is returned and the import is not initiated. Also selecting the Import/Export menu first disables all other menus. So it does not appear possible on present version.

Also, there is no 'abort' (or any other options) available once an import has been initiated until complete, so the utility does not appear to have very large data sets in mind (yet?). In fact data manager reports the program as 'not responding' whilst an import is in progress so there's not even a 'doevents' in the import loop.

Pat Dooley

The latest version or so of SQLite Expert addresses the slow import speed of CSV files (check out the History link at web site).  So I downloaded it and gave it a test.  It is way, way faster than before.  But still not as fast as SQLitening.
300K records in SQLite Expert took 2 minutes - I hate to even imagine how long this would have taken before.
SQLitening with the BEGIN/END TRANSACTION took 20 seconds.

Peter Presland

December 18, 2008, 05:40:15 am #4 Last Edit: December 18, 2008, 06:49:53 am by Peter Presland
Thanks Pat

I've already had a couple of email exchanges with Bogdan Ureche, the author, about it. He is very responsive to issues and suggestions. I've purchased the pro version now and am finding it very useful indeed.