• Welcome, Guest. Please login.
 
September 17, 2019, 10:17:57 am

News:

Welcome to the SQLitening support forums!


Storing large byte array

Started by Bern Ertl, January 08, 2013, 12:31:08 pm

Previous topic - Next topic

Bern Ertl

I've got a routine that creates a two dimensional byte array and fills it with data.  Both dimensions to the array can vary from one calculation to the next.  I want to store the calculated array in my database until I need to calculate a new one.  The byte array could get large (say 25,000,000 records) in practical use. 

I was thinking it would be best to treat the array as a single blob of binary data.  Is this the correct approach?X = 5000
Y = 5000
sBuffer = STRING$( X * Y, $NUL)
DIM bLG( X, Y) AS BYTE AT STRPTR( sBuffer)

... <fill array with data> ...

slExeBind slBuildInsertOrUpdate( "T1",  "?", "bLG", "RowID=1"), slBuildBindDat( sBuffer, "C")

Bern Ertl

Rats.  Just realized I need to do this in a Procs function.  How do you handle the update with binding (and compression?) using SQLite (not SQLitening)?  Hmmm..

Fred Meier

Starting with version 1.4, July 2010, you can use SQLitening rather than calling SQLite direct in database Procs.
Below are Release Notes from that release:
Quote#===================<[ Version 1.4  July 12, 2010 ]>===================
1. SQLitening.Dll is now thread-safe.  You can now have multiple threads
    accessing your SQLite database in both local and remote mode.  This
    will allow you to have multiple connections per client to the server,
    one for each thread.  This can greatly increase response time for
    situations where you can take advantage of multiple threads.  A new
    example program (ExampleD.Bas) is included using multiple theads.
    This new feature also allows database Procs to use SQLitening.Dll
    rather than have to call SQlite direct (makes coding Procs much easier). 
    There is a new Proc (SQLiteningProcB.Bas) included using this new feature.

Bern Ertl


Bern Ertl

Hmm... The Procs functions are passed an rhDab handle to the database which is used by the slm... macros.  If I try to include SQLitening.Inc and use the sl... functions, I'm not able to pass the rhDab handle to them. 

Do you have an example of using the slmBindBlob macro to insert a record?

Bern Ertl

NM... found it in the ProcsA source.

Bern Ertl

OK.  I'm able to insert the record as a bound blob using the slm... macros.  Is there a way to compress the data using SQLitening's compression?

cj

I saw this unanswered and wondered if one of these might help within a proc?

MACRO CompressEncrypt(sData) = slBuildBindDat(sData,"CN") 'compress/encrypt
MACRO Compress(sData) = slBuildBindDat(sData,"C") 'compress   
MACRO Encrypt(sData) = slBuildBindDat(sData,"N") 'encrypt