mdb:mdb.git
4 years agonew date LMDB_0.9.14
Howard Chu [Sat, 20 Sep 2014 06:24:32 +0000 (07:24 +0100)]
new date

4 years agoITS#7943 reinit txn flags
Howard Chu [Fri, 19 Sep 2014 00:45:43 +0000 (01:45 +0100)]
ITS#7943 reinit txn flags

When reusing the main write txn

4 years agoPlug leak in 4d02c741b120786df1b87ee9ed49c1d3f9bc7522
Howard Chu [Thu, 18 Sep 2014 00:22:02 +0000 (01:22 +0100)]
Plug leak in 4d02c741b120786df1b87ee9ed49c1d3f9bc7522

Free writer txn on env_close

4 years agoSilence some clang warnings
Howard Chu [Tue, 16 Sep 2014 20:54:07 +0000 (21:54 +0100)]
Silence some clang warnings

4 years agoRelease 0.9.14
Howard Chu [Mon, 15 Sep 2014 18:48:32 +0000 (19:48 +0100)]
Release 0.9.14

4 years agoChange retry to num times 60. Testing shows that on a known dataset, this has the...
Quanah Gibson-Mount [Sat, 13 Sep 2014 21:21:17 +0000 (16:21 -0500)]
Change retry to num times 60.  Testing shows that on a known dataset, this has the same growth behavior as 2.4.39, while num times 20 resulted in significant growth.

4 years agoMore for me_pgoldest
Howard Chu [Tue, 9 Sep 2014 18:44:23 +0000 (19:44 +0100)]
More for me_pgoldest

Leave it in main env struct, it doesn't need to be rolled back
with nested txns.

4 years agoRemember oldest reader txnid
Howard Chu [Tue, 9 Sep 2014 18:40:05 +0000 (19:40 +0100)]
Remember oldest reader txnid

Don't walk the readers table unless we really need to.

4 years agoUse a single write txn
Howard Chu [Thu, 4 Sep 2014 09:40:11 +0000 (10:40 +0100)]
Use a single write txn

4 years agoITS#7913 cleanup
Howard Chu [Sun, 24 Aug 2014 20:18:03 +0000 (21:18 +0100)]
ITS#7913 cleanup

we already checked for data == NULL, drop unnecessary if

4 years agoMore for MIPS
Howard Chu [Fri, 22 Aug 2014 00:40:20 +0000 (17:40 -0700)]
More for MIPS

Move CACHEFLUSH, avoid unnecessary if()

4 years agoPartially revert ada26306992babeccad0c5443fbd76e28dd24cd6
Howard Chu [Mon, 18 Aug 2014 03:44:53 +0000 (20:44 -0700)]
Partially revert ada26306992babeccad0c5443fbd76e28dd24cd6

Of course, MacOSX only has posix_memalign, and only in recent versions.
(Older versions could use valloc instead. That precludes ever changing
the LMDB pagesize to be different from the OS pagesize.) Not all that
crucial since it only affects mdb_env_copyfd1.

4 years agoMicrooptimize loose pages
Hallvard Furuseth [Sat, 16 Aug 2014 23:50:32 +0000 (01:50 +0200)]
Microoptimize loose pages

4 years agoFix loose pages in mdb_freelist_save().
Hallvard Furuseth [Sat, 16 Aug 2014 22:29:12 +0000 (00:29 +0200)]
Fix loose pages in mdb_freelist_save().

Leaving them in dirty_list caused breakage.  Instead merge them
into me_pghead at the end, when no more pages will be allocated.

4 years agoSplit out mdb_midl_xmerge, add MDB_IDL_ALLOCLEN
Hallvard Furuseth [Sat, 16 Aug 2014 09:34:44 +0000 (11:34 +0200)]
Split out mdb_midl_xmerge, add MDB_IDL_ALLOCLEN

4 years agoITS#7917
Howard Chu [Mon, 11 Aug 2014 18:01:20 +0000 (11:01 -0700)]
ITS#7917

4 years agoITS#7917 fix mdb_dbi_open
Howard Chu [Mon, 11 Aug 2014 18:00:11 +0000 (11:00 -0700)]
ITS#7917 fix mdb_dbi_open

Don't let dummy go out of scope

4 years agoJust use memalign, not posix_memalign
Howard Chu [Thu, 7 Aug 2014 18:06:53 +0000 (11:06 -0700)]
Just use memalign, not posix_memalign

Everything has it. Solaris doesn't have posix_memalign.

4 years agoMore for MIPS
Howard Chu [Mon, 4 Aug 2014 18:40:24 +0000 (11:40 -0700)]
More for MIPS

Do a cacheflush after writing data pages, separate from meta's cacheflush

4 years agoITS#7793 more for branch key update
Howard Chu [Mon, 4 Aug 2014 16:26:35 +0000 (09:26 -0700)]
ITS#7793 more for branch key update

4 years agoITS#7793 doc update, again
Howard Chu [Mon, 4 Aug 2014 12:13:29 +0000 (05:13 -0700)]
ITS#7793 doc update, again

4 years agoITS#7793 update branch key if needed
Howard Chu [Mon, 4 Aug 2014 11:57:53 +0000 (04:57 -0700)]
ITS#7793 update branch key if needed

4 years agoMore for MIPS
Howard Chu [Sat, 2 Aug 2014 17:33:35 +0000 (10:33 -0700)]
More for MIPS

Paranoia for 3630066843b7ca6b2cd12911d3e2fe3314cd4549 do the
cacheflush before setting mti_txnid.

4 years agoITS#7913 mdb_cursor_get(*_DUP) cleanup
Howard Chu [Thu, 31 Jul 2014 16:59:52 +0000 (09:59 -0700)]
ITS#7913 mdb_cursor_get(*_DUP) cleanup

FIRST_DUP/LAST_DUP should succeed even on non-duplicate values
PREV_DUP should reset cursor's EOF flag.

4 years agoFix MIPS cache coherency on Linux
Howard Chu [Sun, 27 Jul 2014 01:16:02 +0000 (18:16 -0700)]
Fix MIPS cache coherency on Linux

MIPS chips require manual control of on-chip caches. The cacheflush
syscall being used here only exists on MIPS Linux, other OSs will
require revisiting.

4 years agoUpdate date
Howard Chu [Thu, 24 Jul 2014 11:15:45 +0000 (04:15 -0700)]
Update date

4 years agomore for mdb_cursor_count
Howard Chu [Thu, 24 Jul 2014 11:13:55 +0000 (04:13 -0700)]
more for mdb_cursor_count

return MDB_NOTFOUND if the cursor is empty

4 years agoFix mdb_copy & co when MDB pagesize < OS pagesize
Hallvard Furuseth [Wed, 9 Jul 2014 19:38:45 +0000 (21:38 +0200)]
Fix mdb_copy & co when MDB pagesize < OS pagesize

4 years agoITS#7789 doc updates
Howard Chu [Thu, 24 Jul 2014 00:31:34 +0000 (17:31 -0700)]
ITS#7789 doc updates

4 years agoITS#7789 Fix mdb_env_write_meta().
Hallvard Furuseth [Thu, 24 Jul 2014 00:24:31 +0000 (17:24 -0700)]
ITS#7789 Fix mdb_env_write_meta().

Always write mapsize = max(other metapage:mapsize, env:mapsize).

4 years agoRevert ITS#7789 commits, let's try this again.
Howard Chu [Thu, 24 Jul 2014 00:10:41 +0000 (17:10 -0700)]
Revert ITS#7789 commits, let's try this again.

Revert "ITS#7789 Fix resize vs MDB_NOMETASYNC, and a comment."
Revert "ITS#7789 more"
Revert "Revert "ITS#7789 update RESIZED errmsg text""
Revert "ITS#7789 update RESIZED errmsg text"
Revert "ITS#7789 persist mapsize changes"

This reverts commit 02285aca58f5629547263ba09e7dd685dcf6b4b5.
This reverts commit 38e71c68de2ad61fb0fd72fd40214fd9a94479b9.
This reverts commit a5e4eecb1e59e9645113bb4c63d06a23a2c4bdf9.
This reverts commit d8e18551c3f03d5205f4dc8d1711e494d3814b17.
This reverts commit 5926e54bba7c976e0837221efb20cbff384207b8.

4 years agoFix env_open mode bits
Howard Chu [Tue, 22 Jul 2014 01:20:31 +0000 (18:20 -0700)]
Fix env_open mode bits

4 years agoFix mdb_cursor_count, reject uninit'd cursor
Howard Chu [Mon, 21 Jul 2014 15:33:59 +0000 (08:33 -0700)]
Fix mdb_cursor_count, reject uninit'd cursor

4 years agoSilence prototype warnings
Howard Chu [Mon, 21 Jul 2014 15:11:33 +0000 (08:11 -0700)]
Silence prototype warnings

4 years agoTypo
Howard Chu [Mon, 14 Jul 2014 16:53:53 +0000 (09:53 -0700)]
Typo

4 years agoITS#7896 fix for mdb_env_map
Howard Chu [Mon, 14 Jul 2014 16:51:16 +0000 (09:51 -0700)]
ITS#7896 fix for mdb_env_map

Don't pass non-NULL map address unless MDB_FIXEDMAP was used.

4 years agoMore for cursor EOF
Howard Chu [Wed, 9 Jul 2014 22:03:54 +0000 (15:03 -0700)]
More for cursor EOF

Must also set in sub-cursor if deleting entire subDB

4 years agoZero-init copy2 writebuf
Howard Chu [Wed, 9 Jul 2014 19:05:37 +0000 (12:05 -0700)]
Zero-init copy2 writebuf

4 years agoMDB_MAXKEYSIZE = 0 (dynamic value) when MDB_DEVEL
Hallvard Furuseth [Wed, 9 Jul 2014 18:18:08 +0000 (20:18 +0200)]
MDB_MAXKEYSIZE = 0 (dynamic value) when MDB_DEVEL

4 years agoMDB_DEVEL doxygen fix
Hallvard Furuseth [Wed, 9 Jul 2014 18:17:13 +0000 (20:17 +0200)]
MDB_DEVEL doxygen fix

4 years agoITS#7789 Fix resize vs MDB_NOMETASYNC, and a comment.
Hallvard Furuseth [Wed, 9 Jul 2014 18:16:33 +0000 (20:16 +0200)]
ITS#7789 Fix resize vs MDB_NOMETASYNC, and a comment.

4 years agoWindows fixes
Howard Chu [Wed, 9 Jul 2014 17:22:25 +0000 (10:22 -0700)]
Windows fixes

Always set the filesize when opening for writes. Otherwise
can't use backups from mdb_copy.

4 years agoWindows compat
Howard Chu [Wed, 9 Jul 2014 16:31:49 +0000 (09:31 -0700)]
Windows compat

We mainly use Win32 functions, so cannot just use C-runtime strerror
to return error messages. We have to use Win32-specific messages.
Unfortunately, we document the API to return C-runtime error codes,
so we can't just switch to all Win32 error codes.

4 years agoFix loading of printable dump
Howard Chu [Wed, 9 Jul 2014 16:16:03 +0000 (09:16 -0700)]
Fix loading of printable dump

Off-by-one in end check, would complain on valid input.

4 years agoITS#7789 more
Howard Chu [Wed, 9 Jul 2014 13:54:32 +0000 (06:54 -0700)]
ITS#7789 more

persist size changes from env_open() as well. This was the
original behavior.

4 years agoRevert "ITS#7789 update RESIZED errmsg text"
Howard Chu [Wed, 9 Jul 2014 13:17:54 +0000 (06:17 -0700)]
Revert "ITS#7789 update RESIZED errmsg text"

This reverts commit d8e18551c3f03d5205f4dc8d1711e494d3814b17.
And partially reverts mapsize check. Only mapsize increases are relevant.

4 years agoITS#7825 more
Howard Chu [Wed, 9 Jul 2014 12:13:10 +0000 (05:13 -0700)]
ITS#7825 more

Set read txn's dbiseqs to env's. Set child txn's dbiseqs to parent's.
Simplify DBI_CHANGED test, no need to check md_name. No-op dbi_close
of already closed handle.

4 years agoUpdates #7789, #7825
Howard Chu [Wed, 9 Jul 2014 04:40:11 +0000 (21:40 -0700)]
Updates #7789, #7825

4 years agoITS#7825 more
Howard Chu [Wed, 9 Jul 2014 04:31:48 +0000 (21:31 -0700)]
ITS#7825 more

only update dbiseq in dbi_open on write txns

4 years agoITS#7825 tweak prev commit
Howard Chu [Wed, 9 Jul 2014 04:22:07 +0000 (21:22 -0700)]
ITS#7825 tweak prev commit

dbiseqs are only checked in write txns so omit from read txns

4 years agoITS#7825 add MDB_BAD_DBI error code
Howard Chu [Wed, 9 Jul 2014 04:02:40 +0000 (21:02 -0700)]
ITS#7825 add MDB_BAD_DBI error code

Use DBI sequence numbers to detect DBIs being closed while in use.

4 years agoITS#7789 update RESIZED errmsg text
Howard Chu [Wed, 9 Jul 2014 03:05:31 +0000 (20:05 -0700)]
ITS#7789 update RESIZED errmsg text

4 years agoITS#7789 persist mapsize changes
Howard Chu [Wed, 9 Jul 2014 02:06:45 +0000 (19:06 -0700)]
ITS#7789 persist mapsize changes

Write decreases too, not just increases. Check for any size change
that was not requested by this process.

4 years agoUpdate LMDB version date/info
Howard Chu [Tue, 8 Jul 2014 23:27:26 +0000 (16:27 -0700)]
Update LMDB version date/info

4 years agoITS#7892 Fix MacOSX section name
Howard Chu [Tue, 8 Jul 2014 23:27:03 +0000 (16:27 -0700)]
ITS#7892 Fix MacOSX section name

4 years agoITS#7713 cleanups
Hallvard Furuseth [Tue, 8 Jul 2014 22:01:18 +0000 (15:01 -0700)]
ITS#7713 cleanups

'#ifdef MDB_DEVEL' -> '#if MDB_DEVEL'
Fix comments, use actual PAGEBASE macro as originally intended.
Actually enable 64K pages. Set MDB_DATA_VERSION for testing only.

4 years agoFixes for loose pages
Hallvard Furuseth [Tue, 8 Jul 2014 18:52:53 +0000 (11:52 -0700)]
Fixes for loose pages

mdb_txn_commit(child): Give loose pages to parent.
Use a pointer beyond the page header instead of mp_next, so
we will not need to save/restore mp_pgno. This avoids a crash
caused by references to mp_pgno.

4 years agoRe-fix cursor EOF
Howard Chu [Tue, 8 Jul 2014 18:54:24 +0000 (11:54 -0700)]
Re-fix cursor EOF

Fix in 534dcc721287cfe1051191a5cd7d3df6cf1dc409 was in wrong place

4 years agoITS#7713 handle 65536 byte page size
Howard Chu [Mon, 7 Jul 2014 16:12:22 +0000 (09:12 -0700)]
ITS#7713 handle 65536 byte page size

Fix mp_lower/mp_upper to exclude PAGEHDRSZ, which means our
max mp_upper is now pagesize - PAGEHDRSZ and won't overflow.
Currently must define MDB_DEVEL to use. Will be default for
LMDB 1.0/OpenLDAP 2.5

4 years agoSet xcursor EOF after deleting last entry
Howard Chu [Tue, 8 Jul 2014 00:13:51 +0000 (17:13 -0700)]
Set xcursor EOF after deleting last entry

4 years agoCompacting copy doesn't need to snapshot metas
Howard Chu [Mon, 7 Jul 2014 00:47:25 +0000 (17:47 -0700)]
Compacting copy doesn't need to snapshot metas

4 years agoWindows fixes
Howard Chu [Sun, 6 Jul 2014 14:49:38 +0000 (07:49 -0700)]
Windows fixes

4 years agoPlay games with object file layout
Howard Chu [Sat, 5 Jul 2014 15:48:47 +0000 (08:48 -0700)]
Play games with object file layout

Use gcc section attribute to keep less frequently used functions
away from main code.

4 years agoRationalize mdb_env_copy2 API
Howard Chu [Sat, 5 Jul 2014 14:49:10 +0000 (07:49 -0700)]
Rationalize mdb_env_copy2 API

4 years agoMore tweaks to copyfd2
Howard Chu [Thu, 3 Jul 2014 21:26:14 +0000 (14:26 -0700)]
More tweaks to copyfd2

Make sure the writer thread starts and stops when we expect it to.

4 years agoMore copyfd2 tweaks
Howard Chu [Thu, 3 Jul 2014 17:49:05 +0000 (10:49 -0700)]
More copyfd2 tweaks

4 years agoFix envinfo to return fixed map address
Howard Chu [Thu, 3 Jul 2014 15:31:57 +0000 (08:31 -0700)]
Fix envinfo to return fixed map address

4 years agoWindows compat
Howard Chu [Thu, 3 Jul 2014 14:49:22 +0000 (07:49 -0700)]
Windows compat

4 years agoFix subDB/ovpage copying
Howard Chu [Wed, 2 Jul 2014 18:44:06 +0000 (11:44 -0700)]
Fix subDB/ovpage copying

4 years agoLarger WBUF still better on large DB
Howard Chu [Wed, 2 Jul 2014 15:06:19 +0000 (08:06 -0700)]
Larger WBUF still better on large DB

4 years agoDon't check LEAF2 pages for subnodes
Howard Chu [Wed, 2 Jul 2014 14:45:12 +0000 (07:45 -0700)]
Don't check LEAF2 pages for subnodes

4 years agoAdd mdb_env_copy2()
Howard Chu [Wed, 2 Jul 2014 04:45:43 +0000 (21:45 -0700)]
Add mdb_env_copy2()

And mdb_env_copyfd2(). Perform compaction on the copy. Trims out
freed pages and renumbers data pages in sequential order. This is
more CPU-intensive since it copies and modifies data pages.

4 years agodump/load need mapsize etc.
Howard Chu [Wed, 2 Jul 2014 01:53:47 +0000 (18:53 -0700)]
dump/load need mapsize etc.

4 years agoAll diagnostics should go to stderr
Howard Chu [Tue, 1 Jul 2014 12:11:43 +0000 (05:11 -0700)]
All diagnostics should go to stderr

4 years agoMore gcov setup tweaks
Howard Chu [Mon, 30 Jun 2014 11:42:41 +0000 (04:42 -0700)]
More gcov setup tweaks

4 years agoITS#7886 fix mdb_copy write size
Howard Chu [Mon, 30 Jun 2014 11:41:50 +0000 (04:41 -0700)]
ITS#7886 fix mdb_copy write size

Don't try to write past end of file

4 years agoFix mdb_page_loose() vs. mdb_page_flush().
Hallvard Furuseth [Sat, 28 Jun 2014 11:38:03 +0000 (13:38 +0200)]
Fix mdb_page_loose() vs. mdb_page_flush().

Do not overwrite loose pages' mp_pgno with mp_next.
mdb_page_flush() will use mp_pgno to rebuild dirty_list.

4 years agoTrap signals
Howard Chu [Fri, 27 Jun 2014 17:53:11 +0000 (10:53 -0700)]
Trap signals

Try to exit cleanly to avoid leaving stale readers. Not
a critical issue since mdb_reader_check can take care of
them, but still cleaner.

4 years agoSetup for gcov/coverage testing
Howard Chu [Thu, 26 Jun 2014 12:19:47 +0000 (05:19 -0700)]
Setup for gcov/coverage testing

4 years agoMore for mdb_page_loose
Howard Chu [Tue, 24 Jun 2014 18:43:13 +0000 (11:43 -0700)]
More for mdb_page_loose

Require that the page is dirty in the current txn,
not a parent txn.

4 years agoMore MDB -> LMDB renames
Howard Chu [Tue, 24 Jun 2014 11:42:44 +0000 (04:42 -0700)]
More MDB -> LMDB renames

4 years agoDoc updates
Howard Chu [Tue, 24 Jun 2014 11:16:19 +0000 (04:16 -0700)]
Doc updates

Rename MDB -> LMDB
Integrate tool manpages

4 years agoITS#7879 Windows build compatibility
Howard Chu [Mon, 23 Jun 2014 14:39:06 +0000 (07:39 -0700)]
ITS#7879 Windows build compatibility

Just use srand/rand, we don't care about the quality
of the random numbers, we just want some data.

4 years agoFlesh out mdb_cmp_cint for BigEndians
Howard Chu [Mon, 23 Jun 2014 14:36:38 +0000 (07:36 -0700)]
Flesh out mdb_cmp_cint for BigEndians

4 years agoMore for page_loosen
Howard Chu [Sun, 22 Jun 2014 23:39:14 +0000 (16:39 -0700)]
More for page_loosen

Don't try this when working on the FreeDB. Again,
too much hassle to unkink the recursions...

4 years agoMore cleanup for page_merge, page_loosen
Howard Chu [Sat, 21 Jun 2014 22:53:41 +0000 (15:53 -0700)]
More cleanup for page_merge, page_loosen

Only loosen src page if it's actually dirty.
Refactor page refs in page_merge.

4 years agoSilence ptr arithmetic warnings
Howard Chu [Sat, 21 Jun 2014 15:34:45 +0000 (08:34 -0700)]
Silence ptr arithmetic warnings

4 years agoHandle loose pages
Howard Chu [Sat, 21 Jun 2014 10:30:34 +0000 (03:30 -0700)]
Handle loose pages

Pages that were dirtied and deleted in the same txn should be
reused, instead of consuming freeDB pages.

4 years agoFix mdb_page_merge
Howard Chu [Sat, 21 Jun 2014 06:40:44 +0000 (23:40 -0700)]
Fix mdb_page_merge

Similar to prev commit for mdb_node_move, the call of
mdb_page_serch_lowest() was leaving csrc unusable.

4 years agoFix mdb_node_move
Howard Chu [Sat, 21 Jun 2014 00:56:04 +0000 (17:56 -0700)]
Fix mdb_node_move

Was leaving stale info in dst cursor

4 years agoFix lineno count
Howard Chu [Fri, 20 Jun 2014 16:03:41 +0000 (09:03 -0700)]
Fix lineno count

4 years agoFix multiple subDBs
Howard Chu [Fri, 20 Jun 2014 16:00:33 +0000 (09:00 -0700)]
Fix multiple subDBs

4 years agoFix unhex
Howard Chu [Fri, 20 Jun 2014 15:56:52 +0000 (08:56 -0700)]
Fix unhex

4 years agoAdd mdb_load
Howard Chu [Fri, 20 Jun 2014 15:49:59 +0000 (08:49 -0700)]
Add mdb_load

4 years agoAdd mdb_dump, update copyrights
Howard Chu [Fri, 20 Jun 2014 12:33:31 +0000 (05:33 -0700)]
Add mdb_dump, update copyrights

4 years agoBump to 0.9.13 LMDB_0.9.13
Howard Chu [Wed, 18 Jun 2014 21:25:46 +0000 (14:25 -0700)]
Bump to 0.9.13

4 years agoBump to 0.9.13
Howard Chu [Wed, 18 Jun 2014 21:07:15 +0000 (14:07 -0700)]
Bump to 0.9.13

4 years agoITS#7793 Re-fix MDB_CURRENT doc: Match current item
Hallvard Furuseth [Wed, 18 Jun 2014 21:01:14 +0000 (23:01 +0200)]
ITS#7793 Re-fix MDB_CURRENT doc: Match current item

4 years agoFix MDB_GET_MULTIPLE/MDB_NEXT_MULTIPLE doc.
Hallvard Furuseth [Wed, 18 Jun 2014 21:01:14 +0000 (23:01 +0200)]
Fix MDB_GET_MULTIPLE/MDB_NEXT_MULTIPLE doc.

4 years agoTweak ovpage search
Howard Chu [Wed, 18 Jun 2014 20:37:04 +0000 (13:37 -0700)]
Tweak ovpage search

Use num*20, chosen from empirical testing