SQLitening Support Forum

SQLitening Support => You've got Questions? We've got Answers! => Topic started by: Bern Ertl on January 08, 2013, 12:31:08 pm

Title: Storing large byte array
Post by: Bern Ertl on January 08, 2013, 12:31:08 pm
I've got a routine that creates a two dimensional byte array and fills it with data.  Both dimensions to the array can vary from one calculation to the next.  I want to store the calculated array in my database until I need to calculate a new one.  The byte array could get large (say 25,000,000 records) in practical use. 

I was thinking it would be best to treat the array as a single blob of binary data.  Is this the correct approach?X = 5000
Y = 5000
sBuffer = STRING$( X * Y, $NUL)
DIM bLG( X, Y) AS BYTE AT STRPTR( sBuffer)

... <fill array with data> ...

slExeBind slBuildInsertOrUpdate( "T1",  "?", "bLG", "RowID=1"), slBuildBindDat( sBuffer, "C")
Title: Re: Storing large byte array
Post by: Bern Ertl on January 08, 2013, 12:50:30 pm
Rats.  Just realized I need to do this in a Procs function.  How do you handle the update with binding (and compression?) using SQLite (not SQLitening)?  Hmmm..
Title: Re: Storing large byte array
Post by: Fred Meier on January 08, 2013, 01:42:58 pm
Starting with version 1.4, July 2010, you can use SQLitening rather than calling SQLite direct in database Procs.
Below are Release Notes from that release:
Quote#===================<[ Version 1.4  July 12, 2010 ]>===================
1. SQLitening.Dll is now thread-safe.  You can now have multiple threads
    accessing your SQLite database in both local and remote mode.  This
    will allow you to have multiple connections per client to the server,
    one for each thread.  This can greatly increase response time for
    situations where you can take advantage of multiple threads.  A new
    example program (ExampleD.Bas) is included using multiple theads.
    This new feature also allows database Procs to use SQLitening.Dll
    rather than have to call SQlite direct (makes coding Procs much easier). 
    There is a new Proc (SQLiteningProcB.Bas) included using this new feature.
Title: Re: Storing large byte array
Post by: Bern Ertl on January 08, 2013, 02:23:12 pm
Excellent.  Thanks Fred.
Title: Re: Storing large byte array
Post by: Bern Ertl on January 09, 2013, 12:43:06 pm
Hmm... The Procs functions are passed an rhDab handle to the database which is used by the slm... macros.  If I try to include SQLitening.Inc and use the sl... functions, I'm not able to pass the rhDab handle to them. 

Do you have an example of using the slmBindBlob macro to insert a record?
Title: Re: Storing large byte array
Post by: Bern Ertl on January 09, 2013, 12:46:53 pm
NM... found it in the ProcsA source.
Title: Re: Storing large byte array
Post by: Bern Ertl on January 09, 2013, 01:04:50 pm
OK.  I'm able to insert the record as a bound blob using the slm... macros.  Is there a way to compress the data using SQLitening's compression?
Title: Storing large byte array using a PROC
Post by: cj on October 12, 2014, 11:01:29 am
I saw this unanswered and wondered if one of these might help within a proc?

MACRO CompressEncrypt(sData) = slBuildBindDat(sData,"CN") 'compress/encrypt
MACRO Compress(sData) = slBuildBindDat(sData,"C") 'compress   
MACRO Encrypt(sData) = slBuildBindDat(sData,"N") 'encrypt