History log of /4.0.0/forestdb/src/blockcache.cc (Results 1 - 25 of 76)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: v1.2, v1.1, v1.0
# 5f7821ba 11-Aug-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-16038 Support the lazy cleanup of a file's buffer cache blocks

If a given file is compacted ans no longer accessed, then its cache
blocks in the buffer cache should be cleaned up and

MB-16038 Support the lazy cleanup of a file's buffer cache blocks

If a given file is compacted ans no longer accessed, then its cache
blocks in the buffer cache should be cleaned up and moved to the free list.

As this operation can takes up to several seconds depending on the number
of cache blocks belonging to the file, it is better to support the lazy
cleanup of these cache blocks to avoid the long blocking period in the
application.

Change-Id: I3d607f17e189b63c592298c822672b9178356b0a

show more ...


# 52ade7a4 10-Aug-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-16018: Prevent free of zombie file still in use

Only allow free of those zombie files which are no longer
still in use.
Test case to catch this bug.

Change-Id: I71f325295

MB-16018: Prevent free of zombie file still in use

Only allow free of those zombie files which are no longer
still in use.
Test case to catch this bug.

Change-Id: I71f325295b7f092e3b576fd577ffb13633706e03

show more ...


# 4656e0ae 22-Jun-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-15348 Use reader-writer spinlock for the buffer cache file list on Windows

Change-Id: If1096b551d7048dd12dca6a9aaabea9e39b8ad3e


# b31f83a4 18-Jun-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-15348 Use file access timestamp to select a victime file for cache eviction.

From the performance tests, we observed the contention on the lock that is used
to maintain the global fil

MB-15348 Use file access timestamp to select a victime file for cache eviction.

From the performance tests, we observed the contention on the lock that is used
to maintain the global file LRU list. This change reduces the lock conention
by using a file access timestamp to select a victim file for cache eviction.

In addition, this change uses a reader-writer spinlock to synchronize accesses
to the file list array in the buffer cache. Currently, the reader-writer spinlock
is only supported in non-Windows environments.

Our reader-writer spinlock implementation is based on the approach proposed in

https://jfdube.wordpress.com/2014/01/12/optimizing-the-recursive-read-write-spinlock/

Change-Id: I61b6916735d715b19572d484253e180f57d9b724

show more ...


# b924f687 17-Jun-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-15348 Remove unused spinlock from file buffer cache structure.

Change-Id: I20c2b528f28c89ccc1832f396bd57b1a24dd1ee4


# ddbd458d 15-Jun-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Make ForestDB::atomic::get() atomic

ThreadSanitizer reports that ForestDB::atomic::read() is not atomic.

Change-Id: Ia45970cd9f5972192423abb8736ac5766a74a779


# 7e3e5567 08-May-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-14808: fix crash in _bcache_evict race with bcache_remove_file

In _bcache_evict() fileA picks fileB as victim, and just then fileB
is closed. Crash happens in _bcache_evict as it acce

MB-14808: fix crash in _bcache_evict race with bcache_remove_file

In _bcache_evict() fileA picks fileB as victim, and just then fileB
is closed. Crash happens in _bcache_evict as it accesses freed memory

Use ref_counters and file_zombies list to delay deletions if file is
in use. Garbage collect file_zombies list on open, close and shutdown

Change-Id: If4f3015bfcd2db720408df334c981ba59ca99efb

show more ...


# 4404c491 30-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

Update freelist_count on bcache_shutdown

This prevents possible out-of-bound computations

Change-Id: I1dee48325e7746a0c3beb4a7bf48fc45259655b3


# 2b79205f 17-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Fix some compile warnings related to type comparisons.

Change-Id: Ia929193c9f14978193e7b0baa9a63acd55ee1153


# 5aa14abc 11-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

Compiler warnings and minor bug fixes

Change-Id: I753118f2f79760d43b421e63aca512449da89574


# 6ef65b54 01-Apr-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-14060 Insert new docs by normal writer during compaction into old file

- New docs inserted by normal writer during compaction are stored in
the old file.

- After the first ph

MB-14060 Insert new docs by normal writer during compaction into old file

- New docs inserted by normal writer during compaction are stored in
the old file.

- After the first phase of compaction (moving all docs in HB+trie at
the time when the compaction begins) is done, the second phase of
compaction (moving all delta docs inserted during the first phase)
is performed.

- The second phase is repeated until there is no more delta.

- Now 'handle->new_file' is never set and not used at all. Those
unused codes will be removed in a separate refactoring commit.

Change-Id: I034c074767264a245edcb8b89182a49086e99f02

show more ...


# 14f97ccf 22-Mar-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Enlarge sorting window size for compaction

- Enlarge the size of sorting window for document offsets.

- Use larger batch size (up to 4 MB, or 4096 docs) to maximize
write bandwi

Enlarge sorting window size for compaction

- Enlarge the size of sorting window for document offsets.

- Use larger batch size (up to 4 MB, or 4096 docs) to maximize
write bandwidth.

Change-Id: I9a1354a4c23ca3eb26141ff8535202f57ea74775

show more ...


# b6be7a1d 01-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

fdb_assert() to trace & dump all forestdb handles if SIGABRT

cmake -D_TRACK_HANDLES=1 /path/to/forestdb
to turn on handle track and dump on assert
This change will help track open fi

fdb_assert() to trace & dump all forestdb handles if SIGABRT

cmake -D_TRACK_HANDLES=1 /path/to/forestdb
to turn on handle track and dump on assert
This change will help track open file handles and dump their content
on asserts() only if the macro is enabled at compile-time
To help debug golang environments where there are no useful symbols
available in core dumps
Change-Id: I7ce97ed713452c3dbe2c7eb2925d913a2c0c902b

show more ...


# f53a893e 24-Mar-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-13955: avoid looping on random picks for bcache_evicts

For the second chance algorithm if we loop on the pick from a
random shard, the loop could end up in an infinite loop.
Inste

MB-13955: avoid looping on random picks for bcache_evicts

For the second chance algorithm if we loop on the pick from a
random shard, the loop could end up in an infinite loop.
Instead serially examine all shards if empty.
Change-Id: If9635c9dd330a1cb5e95a2299a5d33f4ce7de40d

show more ...


# 0c128414 12-Mar-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Revert "Use file scanning style compaction"

This reverts commit 8462df0c0ca9915416400a046cdf8bd85b14c024.

Conflicts:
src/forestdb.cc

Change-Id: Iff4ca130e699a39

Revert "Use file scanning style compaction"

This reverts commit 8462df0c0ca9915416400a046cdf8bd85b14c024.

Conflicts:
src/forestdb.cc

Change-Id: Iff4ca130e699a397843fc95e230d48c484628b68

show more ...


# 8462df0c 05-Mar-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Use file scanning style compaction

- The current compaction scans docs using HB+trie iterator, so docs
are moved into new the file in a key order. However, since docs are
randomly sc

Use file scanning style compaction

- The current compaction scans docs using HB+trie iterator, so docs
are moved into new the file in a key order. However, since docs are
randomly scattered on the DB file, the overhead from random reads
becomes a bottleneck of the compaction throughput.

- Instead, we can largely improve the overall throughput if we
sequentially scan docs in a byte offset order, from the beginning of
the file.

- Use more large batch size (up to 32 MB, or 65536 docs) to maximize
write bandwidth.

- Verified that the performance of compaction for workloads smaller
than the RAM size is also improved.

Change-Id: Ief90ed3ce1dbc28e19f44dd7338976fd9b40225c

show more ...


# 0745bc38 24-Feb-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-13574 Use atomic variable for frequently accessed filemgr attributes

- The elimination of locking the 'file->lock' during accessing the
frequently accessed shared filemgr values such

MB-13574 Use atomic variable for frequently accessed filemgr attributes

- The elimination of locking the 'file->lock' during accessing the
frequently accessed shared filemgr values such as 'pos' and
'header.bid' largely reduces the lock contention overhead.

Change-Id: I97672f829fb53659094565fd8ee59b705f42165a

show more ...


# 69467020 17-Feb-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-13101 Partition the buffer cache to reduce the lock contention.

This change partitions each file's buffer cache to reduce the lock
contention among multiple readers and a wrtier. The

MB-13101 Partition the buffer cache to reduce the lock contention.

This change partitions each file's buffer cache to reduce the lock
contention among multiple readers and a wrtier. The number of
partitions is configurable when a file is created.

Change-Id: I7cf3c0c0f8a794c92465a1253f75ab504713c530

show more ...


# 064f53e0 22-Jan-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Make block_cache internal functions static.

Change-Id: I72ea24ade6bb3b7bea420fbf233e9d13f167b5bb


# 8d63ff2d 17-Jan-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-13117 Fix a typo in bcache_write_partial()

- Since 'buf' is not page aligned, out-of-bounds memory access can
occur. We have to use item->addr instead.

Change-Id: I6a10bc6472

MB-13117 Fix a typo in bcache_write_partial()

- Since 'buf' is not page aligned, out-of-bounds memory access can
occur. We have to use item->addr instead.

Change-Id: I6a10bc647210410a57a3e6ce476fd2e322058d7a

show more ...


# e4da0646 13-Jan-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-12923 Exclude unreachable functions or code paths from lcov coverage

Change-Id: I877f1aa917ad7f11911895d0b187f9271e46866f


# 65a4da43 27-Dec-2014 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Improve dirty block eviction performance

- Block cache tries to avoid evicting dirty B+tree nodes because
they are likely to be updated later. This largely reduces the
write amplific

Improve dirty block eviction performance

- Block cache tries to avoid evicting dirty B+tree nodes because
they are likely to be updated later. This largely reduces the
write amplification during bulk load when working set is larger
than the RAM size (using 200M docs, W.amplification during bulk
load is reduced up to 5x).

- If O_DIRECT flag is not set, we don't need to use additional buffer
for flushing dirty blocks. Dirty blocks are directly flushed by using
pwrite(), and they will be buffered by OS's page cache. It reduces the
memcpy overhead (and also malloc overhead).

Change-Id: I86e958b29be38981fd56ba07430d8cf7cc41d847

show more ...


# 1a3969cd 04-Dec-2014 Chiyoung Seo <chiyoung.seo@gmail.com>

Remove unused debugging functions from block cache module.

Change-Id: I274f526fa8c38bbe0743f9d8dd166644fe7e014f


# cb0f0747 04-Dec-2014 Sundar Sridharan <sundar.sridharan@gmail.com>

Handle Disk Write errors gracefully

Propogate filemgr_* write and read errors upwards
to callers so disk errors do not cause asserts.

Change-Id: Iae9991f16d08595f73fa200004cdde8

Handle Disk Write errors gracefully

Propogate filemgr_* write and read errors upwards
to callers so disk errors do not cause asserts.

Change-Id: Iae9991f16d08595f73fa200004cdde8f2137ad25

show more ...


# a40256a8 04-Dec-2014 Chiyoung Seo <chiyoung.seo@gmail.com>

Revert "MB-12612 Anomalous Test Framework + Graceful DiskOutOfSpace handling"

This reverts commit 69488222297a65d82d6faca806111221db80a5ce.


1234