[BDB 11gR2 Beta] Using FTS2 with the BDB SQLite compatibility library code [#18385]
Sandra Whitman
sandra.whitman at oracle.com
Wed Mar 3 07:35:19 PST 2010
Thanks,
I have made some of those changes, but not all. I will do that. Right
now I am trying to add in the fts2 manual build. I'll let you know how
that goes.
One thing Emily told me last evening is that although the SQLite
documentation version 3.5 is not compatible with fts2, from:
http://www.mail-archive.com/sqlite-users@sqlite.org/msg31627.html,
fts2 has a design flaw that would cause corruptions of fts2 index when
running VACUUM against db. However if the VACUUM is not run, it should
be fine.
Thanks,
Sandra
Greg Burd wrote:
> Sandra,
>
> sqlite-poison.patch fails because it changes the btree.c code which is not a shared part of the BDB version. We use that file to map from SQLite's btree into Berkeley DB's. It looks like the patch is used not used.
> http://src.chromium.org/cgi-bin/gitweb.cgi?p=chromium.git&a=search&h=HEAD&st=grep&s=sqlite3Poison
>
> safe-tolower.patch is very straight forward, it changes three lines of code. It looks like it can easily be applied by hand.
>
> preload-cache.patch can be replaced by an empty function:
> int sqlite3Preload(sqlite3 *db) { return SQLITE_OK; }
> and a prototype in sqlite.h.in
> int sqlite3Preload(sqlite3 *);
>
> misc.patch has a bunch of small configuration settings and other minor changes, all easily integrated some already in our tree. You can skip removing the ifdef around sqlite3PcacheGetCachesize() because it's not used in our (empty) version of the sqlite3Preload() call (above).
>
>
>
> Liam, are these instructions enough for you to give it a try or do you need more help from us?
>
> -greg
>
>
>
>> -----Original Message-----
>> From: Sandra Whitman
>> Sent: Wednesday, March 03, 2010 9:51 AM
>> To: Greg Burd
>> Cc: bdb-beta-feedback at oss.oracle.com
>> Subject: Re: [BDB 11gR2 Beta] Using FTS2 with the BDB SQLite
>> compatibility library code [#18385]
>>
>> Hi Greg,
>>
>>
>> I applied the 6 Chromium SQLite patches to our 5.0.11 source this
>> morning. I did get rejects/filures for:
>> misc.patch
>> preload-cache.patch
>> safe-tolower.patch
>> sqlite-poison.patch
>>
>>
>> I will try a manual fts2 build on top of this, but I'll have to get the
>> patch failures straightened out.
>>
>>
>> Thanks,
>> Sandra
>>
>> Hexxeh wrote:
>>
>>> Hi,
>>>
>>> The patches that are applied to SQLite are as follows:
>>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/sqlite-
>> poison.patch;h=38f81a0c52fa2ff59b0e536f2418b4e003f97644;hb=HEAD
>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/preload-
>> cache.patch;h=8dc5778a03a29aa14d3416c83da61db3ccfdfa7c;hb=HEAD
>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/fts2.patch;h=
>> 1f02161ee29aba22edad93ad7bba7c244e2dc2a1;hb=HEAD
>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/icu-
>> regexp.patch;h=cb758ce062a3dc6a0112ace3cdd136c18c7f8274;hb=HEAD
>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/misc.patch;h=
>> e2f0c5525053a958bdd59019ab89d028ec201a97;hb=HEAD
>>
>>> http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/safe-
>> tolower.patch;h=c42bb065b7302ad1897be7c50dab59a3eb10c54e;hb=HEAD
>>
>>> You can see notes on what they changed and from which version these
>>> patches are made
>>> here: http://src.chromium.org/cgi-
>>>
>> bin/gitweb.cgi?p=chromium.git;a=blob;f=third_party/sqlite/README.chromi
>> um;h=6953e64e7bf0c462c10d0c7b84ff0052638f8e46;hb=HEAD
>>
>>> Thanks,
>>> Liam
>>>
>>> On 2 March 2010 22:44, Sandra Whitman <sandra.whitman at oracle.com
>>> <mailto:sandra.whitman at oracle.com>> wrote:
>>>
>>> Hi Liam,
>>>
>>> Can you send us the patches/source code changes made for the
>>> Chromium web browser, along with any particular directions for
>>> installing them? I think we need to apply these as a starting
>>>
>> point.
>>
>>> Thanks,
>>> Sandra
>>>
>>>
>>> Hexxeh wrote:
>>>
>>> Yep, that's right. There's an option (just like with Firefox)
>>> to use the system version of SQLite which after a little of
>>>
>> of
>>
>>> fiddling works as you'd expect.
>>>
>>> Liam
>>>
>>> On 2 March 2010 20:54, Greg Burd <GREG.BURD at oracle.com
>>> <mailto:GREG.BURD at oracle.com> <mailto:GREG.BURD at oracle.com
>>> <mailto:GREG.BURD at oracle.com>>> wrote:
>>>
>>> Sandra,
>>>
>>> Chromium OS is a Linux variant built by Google (sometimes
>>> called
>>> the "Google OS") which boots up and then uses the Google
>>> browser,
>>> Chromium (aka Chrome), for everything. Essentially, this
>>>
>> is
>>
>>> almost identical to what we've done with Firefox porting.
>>> Liam is
>>> replacing the Chromium web browser's use of SQLite with
>>>
>> BDB
>>
>>> SQL.
>>> That then runs on this stripped down version of Linux and
>>> runs on
>>> a 2GB USB.
>>>
>>> When Liam looked into the way that Chromium web browser
>>>
>> used
>>
>>> SQLite he found that it:
>>> a) modified it, patching bugs, adding external functions
>>> using the
>>> SQLite extensions API
>>> b) used FTS2 (not FTS3) for full-text search
>>> c) linked this statically against the browser
>>>
>>> For us to help Liam we will need to help him:
>>> a) determine what patches to apply and how to apply them
>>>
>> to BDB
>>
>>> SQL (if necessary)
>>> b) integrate the Google custom functions
>>> c) integrate FTS2 for full-text search
>>>
>>> I believe he's already found a way to link Chromium
>>>
>> against a
>>
>>> dynamic sqlite3.so library rather than statically
>>> compile/link the
>>> amalgamated SQLite source as it does today.
>>>
>>> Did I get that right Liam?
>>>
>>> -greg
>>>
>>> > -----Original Message-----
>>> > From: Sandra Whitman
>>> > Sent: Tuesday, March 02, 2010 3:04 PM
>>> > To: Hexxeh
>>> > Cc: bdb-beta-feedback at oss.oracle.com
>>> <mailto:bdb-beta-feedback at oss.oracle.com>
>>> <mailto:bdb-beta-feedback at oss.oracle.com
>>> <mailto:bdb-beta-feedback at oss.oracle.com>>
>>> > Subject: Re: [BDB 11gR2 Beta] Using FTS2 with the BDB
>>>
>> SQLite
>>
>>> > compatibility library code
>>> >
>>> > Hi Liam,
>>> >
>>> > I am not familiar with the Chromium OS, but if it is
>>> UNIX-like
>>> then any
>>> > arguments which can be passed to a standard SQLite
>>>
>> configure
>>
>>> script can
>>> > also be passed to the Berkeley DB configure script. For
>>> example, I
>>> > typically build:
>>> >
>>> > ../dist/configure CPPFLAGS="-DSQLITE_ENABLE_FTS3"
>>> --enable-sql
>>> > or
>>> > ../dist/configure CPPFLAGS="-DSQLITE_ENABLE_FTS3"
>>> --enable-sql_compat
>>> >
>>> > As Dave mentioned, let me know the problem you are
>>>
>> having
>>
>>> and I can
>>> > check further.
>>> >
>>> > Thanks,
>>> > Sandra
>>> >
>>> > Hexxeh wrote:
>>> > > Hi all,
>>> > >
>>> > > I'm trying to integrate the new BDB SQLite compatible
>>> library into
>>> > > Chromium/ChromiumOS, and I'm encountering a few issues
>>> that I hope
>>> > one
>>> > > of you may have the answer to.
>>> > >
>>> > > One requirement of Chromium is the FTS2 extension, has
>>> anyone
>>> managed
>>> > > to successfully use this with the library? If so, how
>>> did you
>>> > > accomplish this?
>>> > >
>>> > > Thanks,
>>> > > Liam McLoughlin
>>> > >
>>>
>>> ------------------------------------------------------------
>>>
>> ---------
>>
>>> > ---
>>> > >
>>> > > _______________________________________________
>>> > > BDB-BETA-FEEDBACK mailing list
>>> > > BDB-BETA-FEEDBACK at oss.oracle.com
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com>
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com>>
>>>
>>> > > http://oss.oracle.com/mailman/listinfo/bdb-beta-
>>>
>> feedback
>>
>>> >
>>> > _______________________________________________
>>> > BDB-BETA-FEEDBACK mailing list
>>> > BDB-BETA-FEEDBACK at oss.oracle.com
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com>
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com>>
>>>
>>> > http://oss.oracle.com/mailman/listinfo/bdb-beta-feedback
>>>
>>>
>>> -------------------------------------------------------------
>>>
>> -----------
>>
>>> _______________________________________________
>>> BDB-BETA-FEEDBACK mailing list
>>> BDB-BETA-FEEDBACK at oss.oracle.com
>>> <mailto:BDB-BETA-FEEDBACK at oss.oracle.com>
>>> http://oss.oracle.com/mailman/listinfo/bdb-beta-feedback
>>>
>>>
>>> ---------------------------------------------------------------------
>>>
>> ---
>>
>>> _______________________________________________
>>> BDB-BETA-FEEDBACK mailing list
>>> BDB-BETA-FEEDBACK at oss.oracle.com
>>> http://oss.oracle.com/mailman/listinfo/bdb-beta-feedback
>>>
More information about the BDB-BETA-FEEDBACK
mailing list