Fix introduction of duplicates into _changes feed

When a document is updated the new update_seq is assigned as part of the
rev_tree merging in couch_db_updater:merge_rev_trees/7 based on

Fix introduction of duplicates into _changes feed

When a document is updated the new update_seq is assigned as part of the
rev_tree merging in couch_db_updater:merge_rev_trees/7 based on the
condition of whether the new rev_tree is equal to the old tree. This
equality is done as a simple Erlang term comparison. If the trees are
not equal a new update_seq is assigned to the #full_doc_info{} record
that is stored in fulldocinfo_by_id_btree.

During replication it is possible that a document update merges into the
rev_tree internally without creating a leaf. If the terminal node of the
replicated path happens to land on a node with a value of ?REV_MISSING
the new document information will be preferred and replace the
?REV_MISSING value.

This preference ends up causing the rev_tree comparison to evaluate to
false which ends up giving this document a new update_seq. Up until this
point everything is fine. We write a bit of extra data (that will be
cleared during compaction) because of a previous bug where we decided to
be cautious and avoid losing data due to a broken rev_tree merging
aglorithm. It is also important to note that this is the place were we
calculate the update_seq to remove from the docinfo_by_seq_tree.

After this point we get back to couch_db_udpater:update_docs_int/5 where
we eventually call couch_db_updater:new_index_entries/3 which creates
the new entries for the fulldocinfo_by_id_tree and docinfo_by_seq_btree.
At this point we end up creating a #doc_info{} record based on the
leaves in the rev_tree. As you recall, the update that caused the new
update_seq was not a leaf, at this point we create a #doc_info{} record
with an incorrect high_seq member pointing to the update_seq we are
about to remove from the docinfo_by_seq_tree (remember we calculated the
seq to remove before we consulted the leaves).

The end result is that we remove the same update_seq we insert. This
sets us up for the real bug. The next time we go to update this document
the same procedure is applied. But what happens is at the point we
calculate the seq to remove from docinfo_by_seq_tree, we calculate the
wrong value. Thus when the update continues we remove an update_seq that
doesn't exist in the tree and insert our new seq. But, the seq from the
previous update is still in the tree. Thus, our docinfo_by_seq_tree now
contains two entries pointing at the same document.

At this point, we run into the observed behavior of this bug that ends
up causing duplicate entries in views which then ends up throwing errors
when the view is compaction. These errors are also particularly nasty
because they bubble up the the couch_view gen_server which crashes and
spiders out crashing every couch_view_group process. That part probably
isn't important though.

There's a simple test include with the patch to illustrate the behavior
and maintain an assertion that it stays fixed.

Fixes COUCHDB-1265

Change-Id: Icc962bd55c6e031de935c003aac76cebbbeed0c1
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1164350 13f79535-47bb-0310-9956-ffa450edef68
Reviewed-on: http://review.couchbase.org/9659
Tested-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Damien Katz <damien@couchbase.com>
Reviewed-by: Filipe David Borba Manana <fdmanana@gmail.com>

show more ...

COUCHDB-1163 - fix internal state of documents affected by COUCHDB-885

(originally committed by Robert Newson)

Change-Id: Ibddb931308496d100e494b61f91feb1e44cd7b80
git-svn-id: h

COUCHDB-1163 - fix internal state of documents affected by COUCHDB-885

(originally committed by Robert Newson)

Change-Id: Ibddb931308496d100e494b61f91feb1e44cd7b80
git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1128107 13f79535-47bb-0310-9956-ffa450edef68
Reviewed-on: http://review.couchbase.org/6681
Reviewed-by: Jan Lehnardt <jan@apache.org>
Tested-by: Jan Lehnardt <jan@apache.org>

show more ...

Track used space for database and view index files

The database and view group info URIs now expose a new field, named "data_size",
which corresponds to the number of bytes used by the c

Track used space for database and view index files

The database and view group info URIs now expose a new field, named "data_size",
which corresponds to the number of bytes used by the current data snapshot.
Users can now use this value, compare it to the "disk_size" value (total file size)
and decide whether or not to trigger a compaction based on this comparison.
This new value is an approximation and therefore not 100% accurate (but close enough).

Closes COUCHDB-1132.

Special thanks to Adam Kocoloski and Robert Dionne for they're good work on this
feature as well.



git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1095477 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Fix spurious declarations of new merge conflicts

This patch also adds extra tests of the key tree merging logic as well
as edoc-formatted documentation for the module and a few of the me

Fix spurious declarations of new merge conflicts

This patch also adds extra tests of the key tree merging logic as well
as edoc-formatted documentation for the module and a few of the merge
functions. Closes COUCHDB-902.

Thanks Paul Davis, Bob Dionne, Klaus Trainer.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1065471 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Stem revision trees after merging a path, COUCHDB-968

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1053508 13f79535-47bb-0310-9956-ffa450edef68

Skip recursive path merging, COUCHDB-968

This patch ensures that we only ever merge a linear path into the tree.
It relies on the stemming code to collapse paths that could have been

Skip recursive path merging, COUCHDB-968

This patch ensures that we only ever merge a linear path into the tree.
It relies on the stemming code to collapse paths that could have been
merged together by a recursive use of merge_one.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1053507 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Rename doc_to_tree -> to_path and move to couch_doc

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1043453 13f79535-47bb-0310-9956-ffa450edef68

Change key_tree merge to take path as 2nd arg, add type specs

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1043452 13f79535-47bb-0310-9956-ffa450edef68

Prefer values from old tree when merging, COUCHDB-968

This commit represents a substantial refactor of the key tree merging
logic, some of which is not strictly necessary for the resolut

Prefer values from old tree when merging, COUCHDB-968

This commit represents a substantial refactor of the key tree merging
logic, some of which is not strictly necessary for the resolution of
COUCHDB-968.

Two etap test cases checking the ability to merge in a non-linear tree
are removed because the functionality is no longer supported. CouchDB
only ever merged a linear revision history into an existing revision
tree.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@1043450 13f79535-47bb-0310-9956-ffa450edef68

show more ...

trailing whitespace pedantry

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@950545 13f79535-47bb-0310-9956-ffa450edef68

code improvements from tidier. Patch by Kostis Sagonas. COUCHDB-570

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@882903 13f79535-47bb-0310-9956-ffa450edef68

Deterministic revids, MD5 checking of documents, added tracking of rev when an attachment is edited to allow attachment level replication.

git-svn-id: https://svn.apache.org/repos/asf/couchd

Deterministic revids, MD5 checking of documents, added tracking of rev when an attachment is edited to allow attachment level replication.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@795232 13f79535-47bb-0310-9956-ffa450edef68

show more ...

wheeeeeeee

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@794057 13f79535-47bb-0310-9956-ffa450edef68

trimmed trailing whitespace

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@791350 13f79535-47bb-0310-9956-ffa450edef68

last three dialyzer recommended cleanups

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@791136 13f79535-47bb-0310-9956-ffa450edef68

removed some dead code found by Dialyzer, more to come


git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@788979 13f79535-47bb-0310-9956-ffa450edef68

Forgot to remove old test/0 exports.



git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@783604 13f79535-47bb-0310-9956-ffa450edef68

Integrating more ETAP tests from Bob Dionne.



git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@783600 13f79535-47bb-0310-9956-ffa450edef68

First cut at _changes api. Update the by_id and by_seq indexes to contain update seq numbers and pointers to bodies on disk, for use in the _changes api. This is a new file version, but the code can

First cut at _changes api. Update the by_id and by_seq indexes to contain update seq numbers and pointers to bodies on disk, for use in the _changes api. This is a new file version, but the code can continue to serve the old 0.9 version without problems, though certain features in the _changes api will not be able to work. Upgrade to new file version (from 1 to 2) by compacting the file. Also fixed bugs with how the stats api tracks open databases.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@772406 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Fix for sporatic replication error during tests.

git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@754970 13f79535-47bb-0310-9956-ffa450edef68

Commit Damien's rep_security branch to trunk.
Changes bulk_docs conflict checking.
Breaks file format, see mailing list for data upgrade procedure, or
http://wiki.apache.org/couchdb/Break

Commit Damien's rep_security branch to trunk.
Changes bulk_docs conflict checking.
Breaks file format, see mailing list for data upgrade procedure, or
http://wiki.apache.org/couchdb/Breaking_changes


git-svn-id: https://svn.apache.org/repos/asf/couchdb/trunk@753448 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Check in of initial validation and authorization work. This work is incomplete, as there is not yet any way of restricting who can update the design docs.

git-svn-id: https://svn.apache.org/

Check in of initial validation and authorization work. This work is incomplete, as there is not yet any way of restricting who can update the design docs.

git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@713132 13f79535-47bb-0310-9956-ffa450edef68

show more ...

Check-in of document purge functionality.

git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@694430 13f79535-47bb-0310-9956-ffa450edef68

removed individual committer copyright statements as per policy

git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@680156 13f79535-47bb-0310-9956-ffa450edef68

Incremental reduce first checkin. Warning! Disk format change.

git-svn-id: https://svn.apache.org/repos/asf/incubator/couchdb/trunk@656861 13f79535-47bb-0310-9956-ffa450edef68