History log of /4.0.0/forestdb/src/filemgr.cc (Results 1 - 25 of 146)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: v1.2, v1.1, v1.0
# 41a273e3 13-Aug-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-16060: Do not free buffer cache blocks twice

If the blocks are freed by filemgr_remove_all_buffer_blocks() then
there it is not necessary to call the same set of functions again
e

MB-16060: Do not free buffer cache blocks twice

If the blocks are freed by filemgr_remove_all_buffer_blocks() then
there it is not necessary to call the same set of functions again
even if the file was picked as victim by another thread.
This prevents assertion in _fname_try_free, blockcache.cc, line 794

Change-Id: I9016097fd400718ff01e6194af5896c98cac8573

show more ...


# 3155b82d 13-Aug-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-16038 Use intentional sleep instead of grabbing mutex for throttling writer

- Since it is hard to control writer speed using mutex, we use sleep instead.

Change-Id: I51a6af7077fe

MB-16038 Use intentional sleep instead of grabbing mutex for throttling writer

- Since it is hard to control writer speed using mutex, we use sleep instead.

Change-Id: I51a6af7077fe56e74b5e0976993f36e25b6abd1c

show more ...


# 17f72b63 09-Jul-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

[BP] MB-16038 Reduce the filemgr lock contention in fdb_open()

This commit removes the unnecessary critical section synchronization
in fdb_open().

Change-Id: I08fd00f7c6ae0ea86e

[BP] MB-16038 Reduce the filemgr lock contention in fdb_open()

This commit removes the unnecessary critical section synchronization
in fdb_open().

Change-Id: I08fd00f7c6ae0ea86e490f38fd5502a1cf9b5308

Conflicts:
src/forestdb.cc

show more ...


# 5f7821ba 11-Aug-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-16038 Support the lazy cleanup of a file's buffer cache blocks

If a given file is compacted ans no longer accessed, then its cache
blocks in the buffer cache should be cleaned up and

MB-16038 Support the lazy cleanup of a file's buffer cache blocks

If a given file is compacted ans no longer accessed, then its cache
blocks in the buffer cache should be cleaned up and moved to the free list.

As this operation can takes up to several seconds depending on the number
of cache blocks belonging to the file, it is better to support the lazy
cleanup of these cache blocks to avoid the long blocking period in the
application.

Change-Id: I3d607f17e189b63c592298c822672b9178356b0a

show more ...


# d6eb040e 05-Aug-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-15908 Support lazy deletion of old DB file

- To avoid API blocking by high latency of file removal, we support
lazy deletion of old DB file after compaction.

- Once the ref_c

MB-15908 Support lazy deletion of old DB file

- To avoid API blocking by high latency of file removal, we support
lazy deletion of old DB file after compaction.

- Once the ref_count of an old file becomes zero, then it is registered
to the file list of background compaction daemon, and then the daemon
compactor threads periodically check and remove the pending files.

Change-Id: Ie0705a681a05df8ccd68e87ba87576cb81f92156

show more ...


# e5d8350a 24-Jul-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Avoid grabbing destroyed spin lock during shutdown on Windows

- Since spin lock in Windows is not a primitive type, crash can occur
when multiple threads are racing; the first thread des

Avoid grabbing destroyed spin lock during shutdown on Windows

- Since spin lock in Windows is not a primitive type, crash can occur
when multiple threads are racing; the first thread destroys spin lock
and then the second thread attempts to grab the spin lock.

- To avoid this, we need to use atomic CAS.

Change-Id: I0f97d7082ba73d95be4eb074af5eed423a05a3c3

show more ...


# adab8ed5 23-Jul-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Avoid duplicated shutdown

- When more than one thread is going to shut down ForestDB, we have
to ignore the second shutdown call if ForestDB is already shut down
by the first call.

Avoid duplicated shutdown

- When more than one thread is going to shut down ForestDB, we have
to ignore the second shutdown call if ForestDB is already shut down
by the first call.

Change-Id: I7ce5068379dfba8a39ceee49c7a25d8ff9b22d66

show more ...


# a44d9d75 19-Jun-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-15396: supress false log HDR mismatch warning when bid=0

Change-Id: Id573dc734af413f723406563a5e7f636d5ee3860


# ddbd458d 15-Jun-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Make ForestDB::atomic::get() atomic

ThreadSanitizer reports that ForestDB::atomic::read() is not atomic.

Change-Id: Ia45970cd9f5972192423abb8736ac5766a74a779


# e8ecc3cf 01-Jun-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-15191: fail fdb_shutdown if open handles are present

This prevents crashes on shutdown race with other api

Change-Id: I9f6f0e63d7aac72ac557eeeaf9a34fcaaf39b95d


# c6c3d274 20-May-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-14060 Use async I/O to move data blocks to a new file during the compaction

This change uses libaio in Linux to read data blocks from the old file
and move them to the new file during

MB-14060 Use async I/O to move data blocks to a new file during the compaction

This change uses libaio in Linux to read data blocks from the old file
and move them to the new file during the compaction.

We also plan to add the asynchronous I/O support for other OSs
(e.g., Windows, OSx).

Change-Id: I2ba7462291b17171fde06fe5887db8cd0cf76c39

show more ...


# 254d62e8 22-May-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-15092: Validate FILEMGR_MAGIC to ensure valid DB header

Sometimes while scanning blocks in move_delta phase 3 of compaction,
there can be say Btree blocks in the last byte that make i

MB-15092: Validate FILEMGR_MAGIC to ensure valid DB header

Sometimes while scanning blocks in move_delta phase 3 of compaction,
there can be say Btree blocks in the last byte that make it appear
like a DB header. To avoid a false identification, also validate
with FILEMGR_MAGIC, otherwise we can falsely refuse to move certain
blocks in phase 3 of compaction.

Change-Id: Ifd625426d5a13160e9de4ce632eb32c2ff7231ec

show more ...


# 4d2e5d98 04-May-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-14799: init several atomic variables before use

Change-Id: I624d970d1b2f8cfb0e6270006af28ff877151500


# c30e7c9d 30-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-14060 Interleave compactor and normal writer through the old file's lock.

From the write-heavy tests, we observed that the compactor occasionally can't
catch up with the normal writer

MB-14060 Interleave compactor and normal writer through the old file's lock.

From the write-heavy tests, we observed that the compactor occasionally can't
catch up with the normal writer during the second phase of the compaction.

As a short-term solution, this change forces the compactor and writer to
interleave through the old file's lock, so that the compactor can catch up
with the writer in write-heavy use cases. We plan to address this issue
without affecting the writer performance in the future.

Change-Id: I69b25a7a7f5833089c77960e6c3e68355a66dc3f

show more ...


# 272e741f 28-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-14253: Operational Stats for ForestDB CRUD apis

Operational stats at coarse granularity added for
set, gets, commits, compacts, iterator_gets, iterator_moves
These are in-memory &

MB-14253: Operational Stats for ForestDB CRUD apis

Operational stats at coarse granularity added for
set, gets, commits, compacts, iterator_gets, iterator_moves
These are in-memory & migrated over on compaction to new file.
Snapshots can still access old_file's stats.

Change-Id: Ibbc490600dc65402b2133143edba2213b6417583

show more ...


# 5030839b 30-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

MB-14592: Add fdb_get_buffer_cache_used()

Returns global buffer cache space used across all files.
User will need the value of the buffercache set at the first
fdb_open call issued i

MB-14592: Add fdb_get_buffer_cache_used()

Returns global buffer cache space used across all files.
User will need the value of the buffercache set at the first
fdb_open call issued in order to compute amount of free space

Change-Id: I3cb236867542ad233abd6738ddf065b761784ae5

show more ...


# 200ec8f5 27-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-14060 Scan the file to catch up the remaining blocks in fdb_compact_upto API.

This change avoids traversing the main index for each commit header written after
the compaction marker,

MB-14060 Scan the file to catch up the remaining blocks in fdb_compact_upto API.

This change avoids traversing the main index for each commit header written after
the compaction marker, but instead scans the file to copy the remaining data
blocks and commit headers to the new file.

Change-Id: Ibbd2aea70eecf2510a14174a9e70e8cf2baecd80

show more ...


# ea2a4512 21-Apr-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Fix a bug in snapshot on a file containing uncommitted data

- If there is uncommitted data beyond the last DB header and snapshot
is requested with the most recent seq number, opening sn

Fix a bug in snapshot on a file containing uncommitted data

- If there is uncommitted data beyond the last DB header and snapshot
is requested with the most recent seq number, opening snapshot is
denied as the seq number is identified as dirty.

- However, if there is more than one KV store in a DB file and
uncommitted data is stored in one KV store only, then this denial
does not work correctly.

- To address this issue, compare the requested seq number with the
last committed seq number of the target KV store.

Change-Id: If2e1a0a708aff14fa8882ab4cb04a5dd8e4cd658

show more ...


# 7088dbef 16-Apr-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

MB-14415 Postpone renaming of in-place compacted file if old file exists

- If in-place compacted file is renamed to its original file name
when its old file is still opened, then the oth

MB-14415 Postpone renaming of in-place compacted file if old file exists

- If in-place compacted file is renamed to its original file name
when its old file is still opened, then the other threads referring
to the old file will open wrong & empty file, and various kinds of
failures will occur.

Change-Id: I849d36025047fa0851a97909fc34dcde569eaeb1

show more ...


# 899a7f61 14-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Remove mmaped implementation of WAL index.

As incoming writes are appended into the old file during compaction,
the mmaped implemenation of WAL index is no longer needed.

Change

Remove mmaped implementation of WAL index.

As incoming writes are appended into the old file during compaction,
the mmaped implemenation of WAL index is no longer needed.

Change-Id: Ie45991bcf4473faf38b2894be400a595d374ec52

show more ...


# 6ad4926c 14-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

MB-14463 Return FDB_RESULT_NO_DB_HEADERS if no commit headers are found.

If a given database file doesn't have any commit headers, it would be
better to return an appropriate error code

MB-14463 Return FDB_RESULT_NO_DB_HEADERS if no commit headers are found.

If a given database file doesn't have any commit headers, it would be
better to return an appropriate error code to indicate no headers in the
file. If necessary, the client can simply remove the file in this case.

Change-Id: I50ca7c154a072d0f63ea1878a72e5cc0a3c48b8c

Conflicts:
src/fdb_errors.cc
tests/functional/fdb_functional_test.cc

show more ...


# c279f70e 13-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Fix compiler warnings caused by number formattings.

Change-Id: I9d0314959d41edf9058457a688904daeaacb962b


# 18054ff6 08-Apr-2015 Chiyoung Seo <chiyoung.seo@gmail.com>

Add more loggings to erroneous cases in a file manager.

Change-Id: I13c0b5888f29f6cd5e5ec580a78a886df44b4beb


# 5aa14abc 11-Apr-2015 Sundar Sridharan <sundar.sridharan@gmail.com>

Compiler warnings and minor bug fixes

Change-Id: I753118f2f79760d43b421e63aca512449da89574


# 843dd51b 22-Mar-2015 Jung-Sang Ahn <jungsang.ahn@gmail.com>

Use separate WAL shard for compactor

- To speed up compaction, all WAL entries inserted by compactor are
stored in a separate WAL shard.

Change-Id: If43bc820dcf982d19463d5f3ad3e

Use separate WAL shard for compactor

- To speed up compaction, all WAL entries inserted by compactor are
stored in a separate WAL shard.

Change-Id: If43bc820dcf982d19463d5f3ad3e09d8b7e2b63f

show more ...


123456