History log of /3.1.0/couchdb/ (Results 1 - 25 of 3732)
Revision (<<< Hide revision tags) (Show revision tags >>>)Date Author Comments
(<<< Hide modified files)
(Show modified files >>>)
Revision tags: v4.6.0, v4.5.0
23f7cfbf25-Jun-2015 Volker Mische <vmx@couchbase.com>

MB-15274 Reset file properly

View compaction creates a new file. Such a file always starts with a header. This
header doesn't point to any data, as there wasn't any data written yet. Tho

MB-15274 Reset file properly

View compaction creates a new file. Such a file always starts with a header. This
header doesn't point to any data, as there wasn't any data written yet. Though this
header contains enough information so that the view-engine can start up properly
with a file that contains only such a header. This is important as data corruption
on files can always happen and you don't want to end up in an undefined/not working
state.

The first header in a file that was created due to compaction is based on the header
of the view group when the compaction took place. Some information is reset as the
header e.g. can't point to any trees as the file doesn't contain any data yet.

In the current code, not enough information was reset. The update sequence numbers
of the partitions were still the ones of the view group. In case of a data corruption
of the view file, where the only valid header is a the first one in the file, this
lead to a view that didn't update itself properly anymore if the node was restarted.

On a node restart the view-engine reads the view file from the back until it finds
a valid header. Due to a corruption it only finds the first one in the file. It uses
that header for the view group. When a `stale=false` request comes in, the sequence
numbers of the header are compared to the ones the partitions currently hold. They
will match, so no update will be performed and the data of the view file will be
returned. But as the view file only contains the header, no data will be returned.
The view will be stuck until new updates come in. But even then things will break, as
it is expected that the view contains all the data up to the sequence numbers, but
it doesn't.

The solution is to reset the sequence numbers of the header before it is written at
the beginning of the file. In case of the corruption mentioned above, the index
will be recreated in case a `stale=false` request comes in, as the sequence numbers
are all `0`.

Change-Id: I53460bb7d7ec4cdf1b029fb0c0b1cd1660f338e0
Reviewed-on: http://review.couchbase.org/52562
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-on: http://review.couchbase.org/52595

show more ...

2b185bd225-Jun-2015 Volker Mische <vmx@couchbase.com>

MB-15274 Get proper partition list

Due to the view-engine no longer controlling the database files, the get
sequence numbers command in the fake DCP server was broken. Now the directory

MB-15274 Get proper partition list

Due to the view-engine no longer controlling the database files, the get
sequence numbers command in the fake DCP server was broken. Now the directory
containing the database files is scanned directly.

Change-Id: I274350cca1457462161ee6ff276859b1a8013b92
Reviewed-on: http://review.couchbase.org/52561
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-on: http://review.couchbase.org/52594
Tested-by: Volker Mische <volker.mische@gmail.com>

show more ...

d58f1c1b25-Jun-2015 Volker Mische <vmx@couchbase.com>

MB-15274 On shutdown, shutdown the fake DCP server as well

When shutting down the view engine, make sure that also the fake DCP server
is shutdowned properly, else it can't be restarted.

MB-15274 On shutdown, shutdown the fake DCP server as well

When shutting down the view engine, make sure that also the fake DCP server
is shutdowned properly, else it can't be restarted.

Change-Id: I1353169d16753f2623c161d66432d78aa6692d5f
Reviewed-on: http://review.couchbase.org/52560
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Nimish Gupta <nimish@couchbase.com>
Reviewed-on: http://review.couchbase.org/52593

show more ...

8b5ee65920-May-2015 Nimish Gupta <nimish@couchbase.com>

MB-14068 Remove the duplicate partitions

In this fix, we are removing the duplicate partitions to avoid
the partition version list to grow very large. Ideally there should not
be dup

MB-14068 Remove the duplicate partitions

In this fix, we are removing the duplicate partitions to avoid
the partition version list to grow very large. Ideally there should not
be duplicate partition, but we have seen duplicate partition versions
in production and we were not able to find or reproduce the issue.

Change-Id: Ic0a320a2176dccd5c1b96fdb35ff3e6f73e5ba47
Reviewed-on: http://review.couchbase.org/51278
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-on: http://review.couchbase.org/51893
Tested-by: Volker Mische <volker.mische@gmail.com>

show more ...

37c1780d21-May-2015 Nimish Gupta <nimish@couchbase.com>

MB-15076 Fix the last sequence number for updater

We are using an outdated last sequence list in flush writes.
In this fix, we use the updated last sequence list.This regression
was

MB-15076 Fix the last sequence number for updater

We are using an outdated last sequence list in flush writes.
In this fix, we use the updated last sequence list.This regression
was introduced with commit 43deb764259b0372a5b28517f1d13c905bd3f3d2.

Change-Id: Ie9e98a4506deb5d13e864749e358f80a6df3300a
Reviewed-on: http://review.couchbase.org/51316
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-on: http://review.couchbase.org/51892

show more ...

b2b22e5028-Mar-2015 harsha <hhs.couchbase@gmail.com>

MB-13160 Update ID B-tree before de-duplication

During incremental indexing, mutations are batched and
written to temporary file and B-tree is updated whenever
a multiple snapshot is

MB-13160 Update ID B-tree before de-duplication

During incremental indexing, mutations are batched and
written to temporary file and B-tree is updated whenever
a multiple snapshot is seen, or is a final batch or the
temporary file size exceeds the size limit. This is done
to optimize the disk access.

This resulted in updater writing duplicate row entries
into ID B-tree. For example, consider
..., A, B, C, D, E, #, A, E, C, #
This is a sequence of document mutations,
the `#` is a snapshot marker. Let batch size be 4.
batch-1 process: A, B, C, D which is a partial snapshot.
batch-2 process: E, #, A, E which is part of multiple snapshot.
After de-duplication of batch-2 we end up with A, (last update of) E
Force flushing at this point, as there were multiple snapshots
results in Id B-tree updated with A, B, C, D, A, E. A being duplicated.

Solution is to update B-trees from temporary files before doing
de-duplication, whenever a multiple snapshot is seen.

Add unit test for this scenario from Volker.

Change-Id: I7a36aab51193e37ab72e356280abbb704f7a8ac2
Reviewed-on: http://review.couchbase.org/48897
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Nimish Gupta <nimish@couchbase.com>
Reviewed-on: http://review.couchbase.org/51891
Tested-by: Volker Mische <volker.mische@gmail.com>

show more ...

841044cb16-Apr-2015 Nimish Gupta <nimish@couchbase.com>

MB-14528 Use get all sequence numbers DCP command

It implements a new dcp command for getting all the
sequence numbers.
Change-Id: I0d10322ff58c2489e5672854c4e2ab951374c9e3
Revie

MB-14528 Use get all sequence numbers DCP command

It implements a new dcp command for getting all the
sequence numbers.
Change-Id: I0d10322ff58c2489e5672854c4e2ab951374c9e3
Reviewed-on: http://review.couchbase.org/49852
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-on: http://review.couchbase.org/51855

show more ...

3291e22913-Apr-2015 Nimish Gupta <nimish@couchbase.com>

MB-14466 Mixing of data in views

We were applying a wrong rereduce function in views when we collect and merge the
query result and apply the rereduce. In that rereduce function, we crea

MB-14466 Mixing of data in views

We were applying a wrong rereduce function in views when we collect and merge the
query result and apply the rereduce. In that rereduce function, we create a reduce
context and store it in the erlang process table, but we were not deleting that context
processing the query. So if the next query comes and mochiweb uses the same
erlang process, the next query will find the context and use it. If both are
queries are from different views, the reduce context applied to second query
is wrong and the result will be wrong. To reproduce this issue, create following
2 view functions in different design doc on a 3 node cluster with 1000 documents:
view1:
map:
function (doc, meta) {
emit(["A", "B"], ["couch", "base"]);
}
reduce:
function(keys, values, rereduce) {
return ["couch", "base"];
}

view2:
map:
function (doc, meta) {
emit(["A1", "B1"], ["couch1", "base1"]);
}
reduce:
function(keys, values, rereduce) {
return ["couch1", "base1"];
}
Now when we query view1 and view2, we can see that view2 result
has value from view1.
Result:
{"rows":[
{"key":["A1","B1"],"value":["couch","base"]}
]
}
In this fix, now we delete the context after query is finished.

Change-Id: I37f7488136c5023e97b35fec7ca554411ebf9eb5
Reviewed-on: http://review.couchbase.org/49589
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: Volker Mische <volker.mische@gmail.com>
Reviewed-on: http://review.couchbase.org/51854

show more ...

c3612fee23-Feb-2015 Nimish Gupta <nimish@couchbase.com>

MB-12607 Remove the extra call to updater

We were calling the updater twice in get_set_view function to update
the stat.This was causing multiple stats call to ep-engine which was
im

MB-12607 Remove the extra call to updater

We were calling the updater twice in get_set_view function to update
the stat.This was causing multiple stats call to ep-engine which was
impacting query performance in stale=false case.This change removes the
extra updater call while keeping the stat intact. After this change,
the rate almost doubled (from 29.6 to 57 with 1M items) when there were
no mutations on a single server node.

Change-Id: I7bcbc767e17ee38a8488df79e85a87d506a90d8d
Reviewed-on: http://review.couchbase.org/47190
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-on: http://review.couchbase.org/51853
Tested-by: Volker Mische <volker.mische@gmail.com>

show more ...

19551d2b26-Nov-2014 harsha <hhs.couchbase@gmail.com>

Revert "MB-11917 Async couch_view_parser implementation"

This reverts commit 3bf0b23892a11299ff5cc25e3d1ebf83e3beec9f.
Reverting async couch_view_parser implementation as it is
resul

Revert "MB-11917 Async couch_view_parser implementation"

This reverts commit 3bf0b23892a11299ff5cc25e3d1ebf83e3beec9f.
Reverting async couch_view_parser implementation as it is
resulting in lexical error and also not helping in resolving
erlang scheduler collapse issue.

Change-Id: I94f1cc5c0eb78bcbd95e321b0ed4d43870cc439b
Reviewed-on: http://review.couchbase.org/43633
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Nimish Gupta <nimish@couchbase.com>

show more ...

44d4d9b226-Nov-2014 harsha <hhs.couchbase@gmail.com>

Revert "MB-12697 Serialize parse_json requests chunks per query context"

This reverts commit cede9fe0761aa36eec2488103485c7b985e08f77.

Change-Id: Ieee9ccf680b1e627c4d6e0d30377ca8600

Revert "MB-12697 Serialize parse_json requests chunks per query context"

This reverts commit cede9fe0761aa36eec2488103485c7b985e08f77.

Change-Id: Ieee9ccf680b1e627c4d6e0d30377ca860040f7de
Reviewed-on: http://review.couchbase.org/43632
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Nimish Gupta <nimish@couchbase.com>

show more ...

cede9fe021-Nov-2014 harsha <hhs.couchbase@gmail.com>

MB-12697 Serialize parse_json requests chunks per query context

The order of processing json parsing request chunks
was assigned randomly to an execution thread which
resulted in lex

MB-12697 Serialize parse_json requests chunks per query context

The order of processing json parsing request chunks
was assigned randomly to an execution thread which
resulted in lexical error while processing json
parsing request during view querying. It is now
serialized, such that the request chunks per query
is processed in the same order of their arrival by
queueing every chunk to same execution thread.
However different query requests are assigned
randomly to a thread in the thread pool.

Change-Id: I16c89f7d4d422e9521ffc2a98d1d035b40e9b0fa
Reviewed-on: http://review.couchbase.org/43494
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Reviewed-by: Sriram Melkote <siri@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>

show more ...

6f2f1af218-Nov-2014 Artem Stemkovski <artem@couchbase.com>

do not return url in the "from" field of view_merger error record

because it can contain user:pasword@ and overall contains too much
internal info for user to know

return host:p

do not return url in the "from" field of view_merger error record

because it can contain user:pasword@ and overall contains too much
internal info for user to know

return host:port instead so the user knows which node had a problem

Change-Id: Ia7d401ba70161cf79afabe899b0e24e6331c30a2
Reviewed-on: http://review.couchbase.org/43382
Tested-by: Artem Stemkovski <artem@couchbase.com>
Reviewed-by: Aliaksey Kandratsenka <alkondratenko@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>

show more ...

3bf0b23805-Nov-2014 harsha <hhs.couchbase@gmail.com>

MB-11917 Async couch_view_parser implementation

Scheduler collapse in Erlang R16 is observed due to changes made
in R16 version with regard to erlang thread scheduler in general
and

MB-11917 Async couch_view_parser implementation

Scheduler collapse in Erlang R16 is observed due to changes made
in R16 version with regard to erlang thread scheduler in general
and to scheduling NIF threads in particular. couch_view_parser is
a json stream parser NIF and takes variable amount of time to parse
incoming data depending on the incoming stream length. This resulted
in reporting improper reduction count to erlang scheduler, which in
turn caused scheduler collapse. This NIF is now implemented in
asynchronous fashion, where NIF queues the incoming tasks and returns
to erlang world immediately.

Change-Id: I9e87fe63b78b10a31aba70aba31913d60331b5d7
Reviewed-on: http://review.couchbase.org/42821
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

7f6f0e2810-Nov-2014 Nimish Gupta <nimish@couchbase.com>

MB-12572 Sync the view group after upgrade

During offline upgrade of couchbase, if seqs are empty([]) in the
group, initial build will happen after upgrade and that will set
partitio

MB-12572 Sync the view group after upgrade

During offline upgrade of couchbase, if seqs are empty([]) in the
group, initial build will happen after upgrade and that will set
partition version list to nil.We should sync the group after upgrading
the header, so that partition version list will not be nil after
initial build.

Change-Id: I3eb5114d69dc2c8087310b59d28ff3e01e6ef5e2
Reviewed-on: http://review.couchbase.org/43048
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

e34cf8e207-Nov-2014 Volker Mische <vmx@couchbase.com>

MB-12048: Add test for upgrading header from 2.x to 3.x

There was a unit test missing that the upgrading from a Couchbase 2.x
to a 3.x view file really works.

Change-Id: I819f7a

MB-12048: Add test for upgrading header from 2.x to 3.x

There was a unit test missing that the upgrading from a Couchbase 2.x
to a 3.x view file really works.

Change-Id: I819f7a1594e73765223871da5f1652826f1b926c
Reviewed-on: http://review.couchbase.org/42933
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Nimish Gupta <nimish@couchbase.com>
Tested-by: Volker Mische <volker.mische@gmail.com>

show more ...

232747b406-Nov-2014 Volker Mische <vmx@couchbase.com>

Make test logging more verbose

With the `--verbose` set the test logging output is more verbose.
The output on the screen is still the same it only changes what
gets written to the l

Make test logging more verbose

With the `--verbose` set the test logging output is more verbose.
The output on the screen is still the same it only changes what
gets written to the log file.

Change-Id: Iedaf4612d406450a2fc25e6066e325aa137915cf
Reviewed-on: http://review.couchbase.org/42893
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Dave Rigby <daver@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>

show more ...

529883ba12-Aug-2014 harsha <hhs.couchbase@gmail.com>

MB-11755 Receive memcached messages in bulk

Receive worker in DCP client handled memcached messages
individually, reading header first and any extra bytes
as indicated in header. Thi

MB-11755 Receive memcached messages in bulk

Receive worker in DCP client handled memcached messages
individually, reading header first and any extra bytes
as indicated in header. This resulted in making several
thousands of tcp receive calls per design doc and
resulted in timeouts while handling stat responses from
memcached. This behavior is modified to receive all the
available data for memcached from socket and process each
individual response from the stored binary, which results
in far few tcp receive calls. Socket connection is made
active and buffer size is set to 64K, same as TCP window
size. More than 2x increase in speed is observed for
accesses to stat responses from memcached.

Change-Id: I5b25f3ed15b6825b8e88fac6e1eb4d1e5dbe3c5f
Reviewed-on: http://review.couchbase.org/40570
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Harsha H S <hhs.couchbase@gmail.com>
Tested-by: Harsha H S <hhs.couchbase@gmail.com>

show more ...

1cf5cbe331-Oct-2014 Volker Mische <vmx@couchbase.com>

MB-12251 Don't log stacktrace on query parse errors

Change-Id: I437f736a9e268244450de0538e93a102b2c2d38e
Reviewed-on: http://review.couchbase.org/42672
Reviewed-by: Volker Mische <vo

MB-12251 Don't log stacktrace on query parse errors

Change-Id: I437f736a9e268244450de0538e93a102b2c2d38e
Reviewed-on: http://review.couchbase.org/42672
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>

show more ...

c34007e315-Oct-2014 Volker Mische <vmx@couchbase.com>

MB-12357 Make customs errors in updater possible

When there was an error in the updater it is not possible to send
a custom error. So if you throw a custom error:

throw({cus

MB-12357 Make customs errors in updater possible

When there was an error in the updater it is not possible to send
a custom error. So if you throw a custom error:

throw({custom_error, <<"this is the reason">>)

Such an error was always wrappen into another error tuple, hence
ending up as:

{error, {custom_error, <<"this is the reason">>}}

This lead to a user visible response of:

{"error":"error","reason":"{custom_error,<<\"this is the reason\">>}"}

This is not user-friendy or what you would expect. With this commit
the response is:

{"error":"custom_error","reason":"this is the reason"}

Change-Id: If68ffb9d5b8d70de338a9441ed74edb2aa28f33a
Reviewed-on: http://review.couchbase.org/42163
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

f0c69b1602-Oct-2014 Volker Mische <vmx@couchbase.com>

Make geocouch's dialyzer happy

geocouch uses some of the couchdb functions, hence some type specs
need to be adapted in order to make the dialyzer run on geocouch
happy.

Cha

Make geocouch's dialyzer happy

geocouch uses some of the couchdb functions, hence some type specs
need to be adapted in order to make the dialyzer run on geocouch
happy.

Change-Id: Iaea98815ac8670d7222c181f6d11d406ff5295a9
Reviewed-on: http://review.couchbase.org/41951
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>

show more ...

2897407824-Sep-2014 Volker Mische <vmx@couchbase.com>

MB-12112 Add option to add search paths to unit test run script

It's now possible to add additional search paths for beam files via
the "-f" option. It expects a format that can be used

MB-12112 Add option to add search paths to unit test run script

It's now possible to add additional search paths for beam files via
the "-f" option. It expects a format that can be used in ERL_FLAGS.

This makes it possible for geocouch to add additional modules needed
to run the unit tests. It's about test helpers residing in the the
`test` subdirectory of the geocouch apps.

Change-Id: I339ff146a4920db0d59a8eafb02cf910e2dc02a1
Reviewed-on: http://review.couchbase.org/41762
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>

show more ...

82a9257002-Oct-2014 Volker Mische <vmx@couchbase.com>

Check whether PLT file is outdated or not

Erlang's dialyzer creates a PLT file for static analysis. Creating
such a file takes a long time, hence it's preserved between builds
on the

Check whether PLT file is outdated or not

Erlang's dialyzer creates a PLT file for static analysis. Creating
such a file takes a long time, hence it's preserved between builds
on the build bots.

It can happen that the PLT file is outdated and needs to be
regenerated. This commits adds the automated regeneration.

Change-Id: I1194c8db44a662821e8f55d435ee2e34d4c61707
Reviewed-on: http://review.couchbase.org/41814
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Tested-by: buildbot <build@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>

show more ...

efc2e92823-Sep-2014 Volker Mische <vmx@couchbase.com>

MB-12112 Simplify unit test runner

The unit test runner `runtest.py` needed the source as well as the
build directory of couchdb. This didn't work well as ERL_LIBS picks
the first ap

MB-12112 Simplify unit test runner

The unit test runner `runtest.py` needed the source as well as the
build directory of couchdb. This didn't work well as ERL_LIBS picks
the first application it encounters without checking if there might
be one which contains missing files later in the path.

The build system was changes so that the build directory also includes
the header files, hence only the information about the build path is
needed.

The `source` parameter got renamed to `libsdirs` and now takes a
directory as input that will be added to the ERL_LIBS environment
variable. This enables geocouch to use the same test runner.

All the subdirectories from the supplied couchdb build directory are
now added to erl_flags. This way you don't need to supply as many
modules as previously.

Change-Id: I07d5c40071ea85342ef72d2e64acf9feda0fb25c
Reviewed-on: http://review.couchbase.org/41581
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: Volker Mische <volker.mische@gmail.com>
Tested-by: buildbot <build@couchbase.com>

show more ...

3824c9a002-Sep-2014 Volker Mische <vmx@couchbase.com>

MB-12112: Streamline CMake files

With an improved ERL_BUILD_OTP macro less code is needed within
the individual CMakeLists.txt files.

Change-Id: I9b03b82194dc787715730d3adb2bc59

MB-12112: Streamline CMake files

With an improved ERL_BUILD_OTP macro less code is needed within
the individual CMakeLists.txt files.

Change-Id: I9b03b82194dc787715730d3adb2bc599af162d0a
Reviewed-on: http://review.couchbase.org/41147
Reviewed-by: Sarath Lakshman <sarath@couchbase.com>
Reviewed-by: Chris Hillery <ceej@couchbase.com>
Reviewed-by: Trond Norbye <trond.norbye@gmail.com>
Tested-by: Trond Norbye <trond.norbye@gmail.com>

show more ...

12345678910>>...150