Mixing
DataStax java.lang.AssertionError: Invalid offset
We are running a 3 node cluster for testing. So this is not a production architecture that we have as a plan.
What I have observed is that this in the debug.log. Also after this, DataStax is not able to do compaction of the table, in which the issue is unless I restart the node
java.lang.AssertionError: Invalid offset: 3814, buf: java.nio.DirectByteBuffer[pos=0 lim=3514 cap=4096], bh: ChunkCache$Buffer(/var/lib/cassandra/data/databasename/tablename-4d572062e99311e8ae3c232aeb4f8023/aa-70906-bti-Partitions.db@135168)
I tried searching for this error but could not find the answer to it anywhere
Most surprising part is that there is no such file as
aa-70906-bti
in the folder /var/lib/cassandra/dataname/tablename
I see files with higher number like 72096 or so
cassandra datastax-enterprise
asked Nov 19 '18 at 14:12
Tahseen
315220
|
show 3 more comments
We are running a 3 node cluster for testing. So this is not a production architecture that we have as a plan.
What I have observed is that this in the debug.log. Also after this, DataStax is not able to do compaction of the table, in which the issue is unless I restart the node
java.lang.AssertionError: Invalid offset: 3814, buf: java.nio.DirectByteBuffer[pos=0 lim=3514 cap=4096], bh: ChunkCache$Buffer(/var/lib/cassandra/data/databasename/tablename-4d572062e99311e8ae3c232aeb4f8023/aa-70906-bti-Partitions.db@135168)
I tried searching for this error but could not find the answer to it anywhere
Most surprising part is that there is no such file as
aa-70906-bti
in the folder /var/lib/cassandra/dataname/tablename
I see files with higher number like 72096 or so
cassandra datastax-enterprise
asked Nov 19 '18 at 14:12
Tahseen
315220
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18
|
show 3 more comments
We are running a 3 node cluster for testing. So this is not a production architecture that we have as a plan.
What I have observed is that this in the debug.log. Also after this, DataStax is not able to do compaction of the table, in which the issue is unless I restart the node
java.lang.AssertionError: Invalid offset: 3814, buf: java.nio.DirectByteBuffer[pos=0 lim=3514 cap=4096], bh: ChunkCache$Buffer(/var/lib/cassandra/data/databasename/tablename-4d572062e99311e8ae3c232aeb4f8023/aa-70906-bti-Partitions.db@135168)
I tried searching for this error but could not find the answer to it anywhere
Most surprising part is that there is no such file as
aa-70906-bti
in the folder /var/lib/cassandra/dataname/tablename
I see files with higher number like 72096 or so
cassandra datastax-enterprise
asked Nov 19 '18 at 14:12
Tahseen
315220
We are running a 3 node cluster for testing. So this is not a production architecture that we have as a plan.
What I have observed is that this in the debug.log. Also after this, DataStax is not able to do compaction of the table, in which the issue is unless I restart the node
java.lang.AssertionError: Invalid offset: 3814, buf: java.nio.DirectByteBuffer[pos=0 lim=3514 cap=4096], bh: ChunkCache$Buffer(/var/lib/cassandra/data/databasename/tablename-4d572062e99311e8ae3c232aeb4f8023/aa-70906-bti-Partitions.db@135168)
I tried searching for this error but could not find the answer to it anywhere
Most surprising part is that there is no such file as
aa-70906-bti
in the folder /var/lib/cassandra/dataname/tablename
I see files with higher number like 72096 or so
cassandra datastax-enterprise
cassandra datastax-enterprise
asked Nov 19 '18 at 14:12
Tahseen
315220
asked Nov 19 '18 at 14:12
Tahseen
315220
edited Nov 19 '18 at 14:25
asked Nov 19 '18 at 14:12
Tahseen
315220
asked Nov 19 '18 at 14:12
Tahseen
315220
asked Nov 19 '18 at 14:12
Tahseen
315220
315220
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18
|
show 3 more comments
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18
|
show 3 more comments
2 Answers
2
active
oldest
votes
Ok, it looks like that you hit a known bug that was fixed in the DSE 6.0.4 with offset calculation:
Fix wrong offset in size calculation in trie builder. (DB-2477)
Please upgrade to 6.0.4 to fix this issue.
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
add a comment |
Thank you for the report, this was a bug in the interaction of two different kinds of early open for levelled compaction. The bug has been fixed for DSE 6.0.5.
Meanwhile, if you need to work around the problem, set sstable_preemptive_open_interval_in_mb in cassandra.yaml to something large, e.g. 100000. This will disable early open on incomplete files, but enable it for completed sstables and work around this problem, but still get almost all of the performance advantages of early open.
answered Nov 30 '18 at 8:28
Branimir Lambov
1
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53376481%2fdatastax-java-lang-assertionerror-invalid-offset%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Ok, it looks like that you hit a known bug that was fixed in the DSE 6.0.4 with offset calculation:
Fix wrong offset in size calculation in trie builder. (DB-2477)
Please upgrade to 6.0.4 to fix this issue.
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
add a comment |
Ok, it looks like that you hit a known bug that was fixed in the DSE 6.0.4 with offset calculation:
Fix wrong offset in size calculation in trie builder. (DB-2477)
Please upgrade to 6.0.4 to fix this issue.
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
add a comment |
Ok, it looks like that you hit a known bug that was fixed in the DSE 6.0.4 with offset calculation:
Fix wrong offset in size calculation in trie builder. (DB-2477)
Please upgrade to 6.0.4 to fix this issue.
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
Ok, it looks like that you hit a known bug that was fixed in the DSE 6.0.4 with offset calculation:
Fix wrong offset in size calculation in trie builder. (DB-2477)
Please upgrade to 6.0.4 to fix this issue.
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
answered Nov 19 '18 at 16:07
Alex Ott
27.1k35072
27.1k35072
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
add a comment |
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
version is already 6.0.4
– Tahseen
Nov 19 '18 at 17:06
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
Ok... let me show this to dev team
– Alex Ott
Nov 19 '18 at 17:07
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
I can share debug log if required
– Tahseen
Nov 19 '18 at 17:08
add a comment |
Thank you for the report, this was a bug in the interaction of two different kinds of early open for levelled compaction. The bug has been fixed for DSE 6.0.5.
Meanwhile, if you need to work around the problem, set sstable_preemptive_open_interval_in_mb in cassandra.yaml to something large, e.g. 100000. This will disable early open on incomplete files, but enable it for completed sstables and work around this problem, but still get almost all of the performance advantages of early open.
answered Nov 30 '18 at 8:28
Branimir Lambov
1
add a comment |
Thank you for the report, this was a bug in the interaction of two different kinds of early open for levelled compaction. The bug has been fixed for DSE 6.0.5.
Meanwhile, if you need to work around the problem, set sstable_preemptive_open_interval_in_mb in cassandra.yaml to something large, e.g. 100000. This will disable early open on incomplete files, but enable it for completed sstables and work around this problem, but still get almost all of the performance advantages of early open.
answered Nov 30 '18 at 8:28
Branimir Lambov
1
add a comment |
Thank you for the report, this was a bug in the interaction of two different kinds of early open for levelled compaction. The bug has been fixed for DSE 6.0.5.
Meanwhile, if you need to work around the problem, set sstable_preemptive_open_interval_in_mb in cassandra.yaml to something large, e.g. 100000. This will disable early open on incomplete files, but enable it for completed sstables and work around this problem, but still get almost all of the performance advantages of early open.
answered Nov 30 '18 at 8:28
Branimir Lambov
1
Thank you for the report, this was a bug in the interaction of two different kinds of early open for levelled compaction. The bug has been fixed for DSE 6.0.5.
Meanwhile, if you need to work around the problem, set sstable_preemptive_open_interval_in_mb in cassandra.yaml to something large, e.g. 100000. This will disable early open on incomplete files, but enable it for completed sstables and work around this problem, but still get almost all of the performance advantages of early open.
answered Nov 30 '18 at 8:28
Branimir Lambov
1
answered Nov 30 '18 at 8:28
Branimir Lambov
1
answered Nov 30 '18 at 8:28
Branimir Lambov
1
answered Nov 30 '18 at 8:28
Branimir Lambov
1
1
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53376481%2fdatastax-java-lang-assertionerror-invalid-offset%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
what version of DSE are you using?
– Alex Ott
Nov 19 '18 at 16:00
DSE 6.0.4 version
– Tahseen
Nov 19 '18 at 17:07
can you paste log somewhere, like, pastebin?
– Alex Ott
Nov 19 '18 at 17:51
@AlexOtt here you go pastebin.com/xW89v0T9
– Tahseen
Nov 19 '18 at 19:17
@AlexOtt let me know if you know, I can share more and can give all debug files
– Tahseen
Nov 19 '18 at 19:18