aboutsummaryrefslogtreecommitdiff
path: root/page.go (follow)
Commit message (Collapse)AuthorAgeFilesLines
* Move code to src/ and tests/EuAndreh2024-10-251-197/+0
|
* Clean up after #636Josh Bleecher Snyder2016-12-231-1/+1
| | | | | freelist.lenall duplicated freelist.count. freelist.copyall and mergepgids docs had typos.
* Don't allocate huge slices to merge pgids in freelist.writeJosh Bleecher Snyder2016-12-201-6/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using a large (50gb) database with a read-write-delete heavy load, nearly 100% of allocated space came from freelists. 1/3 came from freelist.release, 1/3 from freelist.write, and 1/3 came from tx.allocate to make space for freelist.write. In the case of freelist.write, the newly allocated giant slice gets copied to the space prepared by tx.allocate and then discarded. To avoid this, add func mergepgids that accepts a destination slice, and use it in freelist.write. This has a mild negative impact on the existing benchmarks, but cuts allocated space in my real world db by over 30%. name old time/op new time/op delta _FreelistRelease10K-8 18.7µs ±10% 18.2µs ± 4% ~ (p=0.548 n=5+5) _FreelistRelease100K-8 233µs ± 5% 258µs ±20% ~ (p=0.151 n=5+5) _FreelistRelease1000K-8 3.34ms ± 8% 3.13ms ± 8% ~ (p=0.151 n=5+5) _FreelistRelease10000K-8 32.3ms ± 1% 32.2ms ± 7% ~ (p=0.690 n=5+5) DBBatchAutomatic-8 2.18ms ± 3% 2.19ms ± 4% ~ (p=0.421 n=5+5) DBBatchSingle-8 140ms ± 6% 140ms ± 4% ~ (p=0.841 n=5+5) DBBatchManual10x100-8 4.41ms ± 2% 4.37ms ± 3% ~ (p=0.548 n=5+5) name old alloc/op new alloc/op delta _FreelistRelease10K-8 82.5kB ± 0% 82.5kB ± 0% ~ (all samples are equal) _FreelistRelease100K-8 805kB ± 0% 805kB ± 0% ~ (all samples are equal) _FreelistRelease1000K-8 8.05MB ± 0% 8.05MB ± 0% ~ (all samples are equal) _FreelistRelease10000K-8 80.4MB ± 0% 80.4MB ± 0% ~ (p=1.000 n=5+5) DBBatchAutomatic-8 384kB ± 0% 384kB ± 0% ~ (p=0.095 n=5+5) DBBatchSingle-8 17.2MB ± 1% 17.2MB ± 1% ~ (p=0.310 n=5+5) DBBatchManual10x100-8 908kB ± 0% 902kB ± 1% ~ (p=0.730 n=4+5) name old allocs/op new allocs/op delta _FreelistRelease10K-8 5.00 ± 0% 5.00 ± 0% ~ (all samples are equal) _FreelistRelease100K-8 5.00 ± 0% 5.00 ± 0% ~ (all samples are equal) _FreelistRelease1000K-8 5.00 ± 0% 5.00 ± 0% ~ (all samples are equal) _FreelistRelease10000K-8 5.00 ± 0% 5.00 ± 0% ~ (all samples are equal) DBBatchAutomatic-8 10.2k ± 0% 10.2k ± 0% +0.07% (p=0.032 n=5+5) DBBatchSingle-8 58.6k ± 0% 59.6k ± 0% +1.70% (p=0.008 n=5+5) DBBatchManual10x100-8 6.02k ± 0% 6.03k ± 0% +0.17% (p=0.029 n=4+4)
* fix Go 1.7 pointer reference bugBen Johnson2016-08-181-0/+6
| | | | | | | | | | | | This commit fixes a bug where page end-of-header pointers were being converted to byte slices even when the pointer did not point to allocated memory. This occurs with pages that have a `page.count` of zero. Note: This was not an issue in Go 1.6 but the new Go 1.7 SSA backend handles `nil` checks differently. See https://github.com/golang/go/issues/16772
* set slice capacityBen Johnson2016-03-251-2/+2
| | | | | | | | | | | | | This commit sets the capacity on slices returned from `Bucket.Get()` to match the slice length. Previously the capacity would be the size of the mmap max size. This does not cause any backwards compatibility issues, however, it does allow users to `append()` to the returned slice since that will cause Go to realloc a new slice on the heap. Fixes #544
* Merge sorted pgids rather than resorting everythingMartin Kobetic2015-06-161-0/+38
|
* fix `slice bounds out of range`/maxAllocSize bugssasha-s2015-04-161-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when accessing the node data we used to use cast to *[maxAllocSize]byte, which breaks if we try to go across maxAllocSize boundary. This leads to occasional panics. Sample stacktrace: ``` panic: runtime error: slice bounds out of range goroutine 1 [running]: github.com/boltdb/bolt.(*node).write(0xc208010f50, 0xc27452a000) $GOPATH/src/github.com/boltdb/bolt/node.go:228 +0x5a5 github.com/boltdb/bolt.(*node).spill(0xc208010f50, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/node.go:364 +0x506 github.com/boltdb/bolt.(*node).spill(0xc208010700, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d github.com/boltdb/bolt.(*node).spill(0xc208010620, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/node.go:336 +0x12d github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae880, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/bucket.go:535 +0x1c4 github.com/boltdb/bolt.(*Bucket).spill(0xc22b6ae840, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2 github.com/boltdb/bolt.(*Bucket).spill(0xc22f4e2018, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/bucket.go:502 +0xac2 github.com/boltdb/bolt.(*Tx).Commit(0xc22f4e2000, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/tx.go:150 +0x1ee github.com/boltdb/bolt.(*DB).Update(0xc2080e4000, 0xc24d077508, 0x0, 0x0) $GOPATH/src/github.com/boltdb/bolt/db.go:483 +0x169 ``` It usually happens when working with large (50M/100M) values. One way to reproduce it is to change maxAllocSize in bolt_amd64.go to 70000 and run the tests. TestBucket_Put_Large crashes.
* Update max alloc size on 386 arch.Ben Johnson2015-02-271-1/+0
|
* Increase max array size to 2GB.Ben Johnson2015-02-261-1/+1
| | | | | | | | | | | This commit changes the maxAllocSize from 256GB to 2GB to handle large values. It was previously 0xFFFFFFF and I tried adding one more "F" but it caused an "array too large" error. I played around with the value some more and found that 0x7FFFFFFF (2GB) is the highest allowed value. This does not affect how the data is stored. It is simply used for type converting pointers to array pointers in order to utilize zero copy from the mmap.
* Fix i686 build error.Ben Johnson2014-06-181-4/+4
|
* Increase max nodes per page.Ben Johnson2014-06-101-5/+4
| | | | | | | | This commit changes the maxNodesPerPage constant so that it is essentially unlimited. Previously, a single large transaction could create more nodes than the mock array could handle. Fixes #188.
* Fix freelist allocate().Ben Johnson2014-05-191-1/+1
|
* Fix freelist allocation direction.Ben Johnson2014-05-191-0/+6
| | | | | | This commit fixes the freelist so that it frees from the beginning of the data file instead of the end. It also adds a fast path for pages which can be allocated from the first free pages and it includes read transaction stats.
* Add nested buckets.Ben Johnson2014-04-111-3/+4
| | | | | | | This commit adds the ability to create buckets inside of other buckets. It also replaces the buckets page with a root bucket. Fixes #56.
* Add 'bolt pages'.Ben Johnson2014-03-211-0/+8
|
* Allow reads of unflushed nodes.Ben Johnson2014-03-011-6/+0
| | | | | This commit allows cursors to read updated values from within the RWTransaction.
* API Documentation.Ben Johnson2014-02-131-10/+10
|
* Add freelist.Ben Johnson2014-02-101-5/+0
|
* Refactor node lookup.Ben Johnson2014-02-071-0/+6
|
* Rename sys ☞ buckets.Ben Johnson2014-02-051-3/+3
|
* Add RWTransaction.Put().Ben Johnson2014-02-011-12/+73
|
* Add RWTransaction.write().Ben Johnson2014-01-301-1/+8
|
* Fix leaf/branch deserialization.Ben Johnson2014-01-301-0/+10
|
* Add freelist page type.Ben Johnson2014-01-291-3/+9
|
* Refactor meta.copy() and page.init().Ben Johnson2014-01-291-23/+7
|
* Add tpage.read() test.Ben Johnson2014-01-281-1/+1
|
* lpageBen Johnson2014-01-271-2/+1
|
* Intermediate.Ben Johnson2014-01-271-12/+15
|
* Initialize transaction/rwtransaction.Ben Johnson2014-01-261-1/+0
|
* TODOBen Johnson2014-01-241-108/+14
|
* TODOBen Johnson2014-01-241-2/+1
|
* Intermediate commit.Ben Johnson2014-01-211-7/+6
|
* Refactoring to RWCursor, RWTxn, and branch/leaf nodes and pages.Ben Johnson2014-01-171-11/+61
|
* Add system buckets.Ben Johnson2014-01-141-1/+1
|
* Begin Transaction.Cursor().Ben Johnson2014-01-131-5/+7
|
* Finish open coverage.Ben Johnson2014-01-131-2/+2
|
* Initial db.open.Ben Johnson2014-01-111-21/+27
|
* DB.Open(), pages, and meta.Ben Johnson2014-01-101-18/+43
|
* Move all C code into repo.Ben Johnson2014-01-091-0/+26
|
* Basic types.Ben Johnson2014-01-081-0/+44
generated by cgit v1.2.3 (git 2.50.1) at 2025-08-14 12:26:12 +0000