aboutsummaryrefslogtreecommitdiff
path: root/sqlite3.go (follow)
Commit message (Collapse)AuthorAgeFilesLines
...
| * fix all goconst issuesMario Trangoni2018-04-171-2/+8
| |
| * fix all unconvert issuesMario Trangoni2018-04-171-7/+7
| |
* | Merge pull request #530 from navytux/y/no-go-if-notneededmattn2018-02-181-19/+23
|\ \ | | | | | | Don't spawn interrupt goroutine if we know that context cannot be canceled
| * | Don't spawn interrupt goroutine if we know that context cannot be canceledKirill Smelkov2018-02-171-19/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For a Go-only project the following code pattern go func() { select { case <-ctx.Done(): // call some cancel case <-done: // work finished ok } }() // do some work close(done) works good and fast - without high scheduling overhead because scheduler usually puts spawned goroutine into run queue on the same OS thread and so after done is closed control is passed to spawned goroutine without OS context switch. However in the presence of Cgo calls in "do some work" the situation can become different - Cgo calls are treated by go runtime similarly to system calls with the effect that goroutines spawned on original OS thread tend to be migrated by scheduler to be executed on another OS thread. This in turn can bring high overhead for communicating on "done", which ultimately can result in full context switch: if the spawned goroutine had chance to run, already checked done and ctx to be not ready, and went into sleep via wait on futex - showing as something like below in strace for one read query (note futex calls): 27867 00:38:39.782146 stat(".../neo.sqlite-journal", 0x7f83809c4a20) = -1 ENOENT (No such file or directory) 27867 00:38:39.782165 pread64(3, "\0\0\0\33\0\0\10\235\0\0\10]\0\0\0\27", 16, 24) = 16 27871 00:38:39.782179 <... pselect6 resumed> ) = 0 (Timeout) 27868 00:38:39.782187 <... pselect6 resumed> ) = 0 (Timeout) 27871 00:38:39.782193 futex(0xc4200f8538, FUTEX_WAIT, 0, NULL <unfinished ...> 27868 00:38:39.782199 futex(0xc420013138, FUTEX_WAIT, 0, NULL <unfinished ...> 27867 00:38:39.782205 stat(".../neo.sqlite-wal", 0x7f83809c4a20) = -1 ENOENT (No such file or directory) 27867 00:38:39.782224 fstat(3, {st_mode=S_IFREG|0644, st_size=9031680, ...}) = 0 27867 00:38:39.782247 futex(0xc420013138, FUTEX_WAKE, 1 <unfinished ...> 27868 00:38:39.782259 <... futex resumed> ) = 0 27867 00:38:39.782265 <... futex resumed> ) = 1 27868 00:38:39.782270 pselect6(0, NULL, NULL, NULL, {tv_sec=0, tv_nsec=3000}, NULL <unfinished ...> 27867 00:38:39.782279 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 27867 00:38:39.782315 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0 27868 00:38:39.782336 <... pselect6 resumed> ) = 0 (Timeout) 27867 00:38:39.782342 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741826, l_len=510} <unfinished ...> 27868 00:38:39.782348 futex(0xc4200f8538, FUTEX_WAKE, 1 <unfinished ...> 27867 00:38:39.782355 <... fcntl resumed> ) = 0 27871 00:38:39.782360 <... futex resumed> ) = 0 27868 00:38:39.782367 <... futex resumed> ) = 1 27871 00:38:39.782372 futex(0xc4200f8138, FUTEX_WAKE, 1 <unfinished ...> 27868 00:38:39.782377 pselect6(0, NULL, NULL, NULL, {tv_sec=0, tv_nsec=3000}, NULL <unfinished ...> 27871 00:38:39.782384 <... futex resumed> ) = 1 27870 00:38:39.782389 <... futex resumed> ) = 0 27867 00:38:39.782394 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1} <unfinished ...> 27870 00:38:39.782400 pselect6(0, NULL, NULL, NULL, {tv_sec=0, tv_nsec=3000}, NULL <unfinished ...> 27867 00:38:39.782408 <... fcntl resumed> ) = 0 Below link shows that go scheduler itself might be significantly improved for cases when there are several Cgo calls made for a request in a server: https://github.com/golang/go/issues/21827#issuecomment-329092317 in particular CGo-4 case should be closely related to this sqlite3 go package, because for one query many CGo calls are made to SQLite. However until there are proper scheduler fixes, let's make what could be made to improve time to do queries: If we know that the context under which a query is executed will never be canceled - we know we can safely skip spawning the interrupt goroutine and this was avoid ping-pong on done in between different OS threads. This brings the following speedup on my notebook with go1.10: name old req/s new req/s delta Exec 254k ± 1% 379k ± 1% +48.89% (p=0.000 n=10+10) Query 90.6k ± 2% 96.4k ± 1% +6.37% (p=0.000 n=10+10) Params 81.5k ± 1% 87.0k ± 1% +6.83% (p=0.000 n=10+10) Stmt 122k ± 2% 129k ± 1% +6.07% (p=0.000 n=10+9) Rows 2.98k ± 1% 3.06k ± 1% +2.77% (p=0.000 n=9+10) StmtRows 3.10k ± 1% 3.13k ± 1% +1.12% (p=0.000 n=9+10) name old time/op new time/op delta CustomFunctions-4 10.6µs ± 1% 10.1µs ± 1% -5.01% (p=0.000 n=10+10)
* | | only enable pread/pwrite for linux. fixes #533 and fixes #532Bas van Beek2018-02-171-1/+2
| | |
* | | Let SQLite use pread/pwriteKirill Smelkov2018-02-161-1/+1
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With current settings SQLite was using lseek/read syscalls to read data, e.g.: 20:43:17.640660 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 20:43:17.640683 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0 20:43:17.640705 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741826, l_len=510}) = 0 20:43:17.640725 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0 20:43:17.640744 stat(".../neo.sqlite-journal", 0x7ffef2c91080) = -1 ENOENT (No such file or directory) 20:43:17.640764 lseek(3, 24, SEEK_SET) = 24 20:43:17.640779 read(3, "\0\0\0\33\0\0\10\235\0\0\10]\0\0\0\27", 16) = 16 20:43:17.640795 stat(".../neo.sqlite-wal", 0x7ffef2c91080) = -1 ENOENT (No such file or directory) but if we allow it to use pread it will be only 1 system call instead of 2 and reading this way can also be done in parallel because there is no global to file seeking: 20:48:42.668466 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0 20:48:42.668501 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0 20:48:42.668522 fcntl(3, F_SETLK, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=1073741826, l_len=510}) = 0 20:48:42.668542 fcntl(3, F_SETLK, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=1073741824, l_len=1}) = 0 20:48:42.668561 stat(".../neo.sqlite-journal", 0x7ffdbc1f22c0) = -1 ENOENT (No such file or directory) 20:48:42.668580 pread64(3, "\0\0\0\33\0\0\10\235\0\0\10]\0\0\0\27", 16, 24) = 16 20:48:42.668597 stat(".../neo.sqlite-wal", 0x7ffdbc1f22c0) = -1 ENOENT (No such file or directory) (if needed this enablement can be done per OS)
* | Merge pull request #525 from mattn/add-usleepmattn2018-02-071-1/+1
|\ \ | | | | | | add -DHAVE_USLEEP=1
| * | add -DHAVE_USLEEP=1Yasuhiro Matsumoto2018-02-071-1/+1
| | | | | | | | | | | | fixes #211
* | | Add static_mock.go to allow building with CGO_ENABLED=0James C Kimble2018-01-311-0/+2
|/ /
* | Merge pull request #462 from faruzzy/mastermattn2018-01-121-2/+1
|\ \ | | | | | | Updated "context" import since it has become a standard library
| * | Updated "context" import since it has become a standard library after go 1.7 ↵Roland Pangu2017-09-051-2/+1
| | | | | | | | | | | | https://golang.org/doc/go1.7#context
* | | Fix race in ExecContextNiklas Janlert2017-11-211-2/+6
| | | | | | | | | | | | | | | When the context is cancelled, an interrupt should only be made if the operation is still ongoing.
* | | Merge pull request #479 from kenshaw/move-registeraggregatormattn2017-11-141-0/+127
|\ \ \ | | | | | | | | Move RegisterAggregator implementation
| * | | Move RegisterAggregator implementationKenneth Shaw2017-11-051-0/+127
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The SQLiteConn.RegisterAggregator implementation was defined in sqlite3_trace.go file, which is guarded with a build constraint. This change simply moves RegisterAggregator to the main sqlite3.go file, and moves accompanying unit tests. The rationale for this move is that it was not possible for downstream using packages to use RegisterAggregator without also specifying (and notifying the user) the 'trace' build tag.
* | | fix to be able to build with GOTAGS=libsqlite3Tetsuya Morimoto2017-11-051-0/+2
| | |
* | | update to call _sqlite3_limit as a wrapper instead of sqlite3_limitTetsuya Morimoto2017-11-051-2/+26
| | |
* | | support sqlite3_limit to get/set run time limitTetsuya Morimoto2017-11-051-0/+30
|/ /
* | Merge branch 'master' into mastermattn2017-08-301-16/+136
|\ \
| * | fix raceYasuhiro Matsumoto2017-08-301-10/+10
| | |
| * | fix raceYasuhiro Matsumoto2017-08-301-1/+0
| | |
| * | fix lockYasuhiro Matsumoto2017-08-301-2/+24
| | |
| * | fixes #458Yasuhiro Matsumoto2017-08-281-0/+1
| |/
| * Fix to pass TestNilAndEmptyBytesGreg Holt2017-08-211-2/+3
| |
| * remove mutexYasuhiro Matsumoto2017-08-021-5/+4
| |
| * fix possibly double Close.Yasuhiro Matsumoto2017-08-021-5/+9
| | | | | | | | fixes #448
| * Add connection option for recursive triggersRoss Light2017-07-091-0/+26
| | | | | | | | | | Similar to foreign keys, the recursive triggers PRAGMA affects the interpretation of all statements on a connection.
| * Merge pull request #2 from mattn/masterJason Abbott2017-07-061-1/+1
| |\ | | | | | | Merge lastest from mattn
| | * SQLITE_THREADSAFE=1Yasuhiro Matsumoto2017-07-051-1/+1
| | | | | | | | | | | | fixes #274
| * | Incorporate original PR 271 from https://github.com/brokensandalsJason Abbott2017-07-031-0/+54
| |/
| * Don't convert Unix times to nanoseconds when querying datetime fields. Fixes ↵deepilla2017-06-301-2/+3
| | | | | | | | #430.
| * Fix for cgo panic, issue #428: https://github.com/mattn/go-sqlite3/issues/428Evgeniy Makeev2017-06-201-7/+4
| |
| * Sync database-close and statement-closePhilip O'Toole2017-06-171-1/+14
| | | | | | | | Potential fix for issue #426.
| * Use global variable for better performance.Xu Xinran2017-06-141-3/+4
| |
| * Treat []byte{} as empty bytes instead of NULL.Xu Xinran2017-06-141-2/+4
| |
* | Add support for collation sequences implemented in Go.David Anderson2017-06-081-0/+25
|/ | | | | This allows Go programs to register custom comparison functions with sqlite, and ORDER BY that comparator.
* Merge pull request #407 from zombiezen/foreignkeysmattn2017-04-021-3/+43
|\ | | | | Add _foreign_keys connection parameter
| * Add _foreign_keys connection parameterRoss Light2017-04-011-3/+43
| | | | | | | | | | Fixes #377 Updates #255
* | Avoid leaking db if setting busy timeout failsRoss Light2017-04-011-0/+1
|/
* Removed ambitious conn.Close()Marko Kungla2017-03-241-1/+0
|
* close connection when got errors in OpenYasuhiro Matsumoto2017-03-241-0/+3
|
* fix breaking compatibility.Yasuhiro Matsumoto2017-03-211-6/+6
| | | | | | revert cf4bd560f1588d96c502b4c3407fe1a10cef4a28 close #394
* fix buildYasuhiro Matsumoto2017-03-201-2/+2
|
* fix buildYasuhiro Matsumoto2017-03-201-3/+3
|
* fix buildYasuhiro Matsumoto2017-03-201-1/+1
|
* return nil when last error is SQLITE_OKYasuhiro Matsumoto2017-03-201-1/+5
|
* refactoringYasuhiro Matsumoto2017-03-051-2/+2
|
* Add Go API for virtual tablesConor Branagan2017-03-041-1/+1
| | | | | | | | | | | | | | See https://www.sqlite.org/vtab.html for more details. This work was started from https://github.com/gwenn/gosqlite/blob/master/vtab.{c,go} and adds: - Porting the API to go-sqlite3 APIs. - Support for >= Go 1.6 without requiring the `cgocheck` flag to be changed. - Filling out the unfinished callback functions for the `Vtable` struct. - A simple `Context` API layer for ease of use when adding modules. Tests are included.
* rename functionYasuhiro Matsumoto2017-03-051-2/+2
|
* workaround for a compilerYasuhiro Matsumoto2017-03-011-0/+1
| | | | | | Apple LLVM version 7.0.2 (clang-700.1.81) Close #386
* use variable dbYasuhiro Matsumoto2017-02-161-1/+1
|
generated by cgit v1.2.3 (git 2.49.0) at 2025-07-21 18:50:03 +0000