Mixing
What Datastore/Database runs on top of S3?
What Datastore/Database runs on top of Amazon S3 or S3-compatible storage?
I understand that S3 is an Object Storage and thus not a database, but a database must have something to store data into, thus, my question is if there is a Database or Datastore that saves its data on an Amazon S3 or S3-compatible storage instead of a local file system.
database amazon-s3
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
add a comment |
What Datastore/Database runs on top of Amazon S3 or S3-compatible storage?
I understand that S3 is an Object Storage and thus not a database, but a database must have something to store data into, thus, my question is if there is a Database or Datastore that saves its data on an Amazon S3 or S3-compatible storage instead of a local file system.
database amazon-s3
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
add a comment |
What Datastore/Database runs on top of Amazon S3 or S3-compatible storage?
I understand that S3 is an Object Storage and thus not a database, but a database must have something to store data into, thus, my question is if there is a Database or Datastore that saves its data on an Amazon S3 or S3-compatible storage instead of a local file system.
database amazon-s3
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
What Datastore/Database runs on top of Amazon S3 or S3-compatible storage?
I understand that S3 is an Object Storage and thus not a database, but a database must have something to store data into, thus, my question is if there is a Database or Datastore that saves its data on an Amazon S3 or S3-compatible storage instead of a local file system.
database amazon-s3
database amazon-s3
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
asked Nov 19 '18 at 22:53
xybrekxybrek
12.4k47191370
12.4k47191370
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Here are some databases and database-like products that use S3 (or can use S3).
- Amazon Athena
- S3 Select
- Apache HBase
- Redshift
Also, if you want some theory, here’s a MIT paper about Building a Database on S3.
This is by no means exhaustive, but it’s probably a good place to start.
Update
Here are some more that aren't AWS owned software.
- Cassandra
Hadoop—this isn't a database, but S3 already provides you with key-value storage, and Hadoop can provide you with querying.- s3-db
Ultimately, you need to consider what sort of query functionality you need and what sort of consistency you can tolerate.
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53383801%2fwhat-datastore-database-runs-on-top-of-s3%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Here are some databases and database-like products that use S3 (or can use S3).
- Amazon Athena
- S3 Select
- Apache HBase
- Redshift
Also, if you want some theory, here’s a MIT paper about Building a Database on S3.
This is by no means exhaustive, but it’s probably a good place to start.
Update
Here are some more that aren't AWS owned software.
- Cassandra
Hadoop—this isn't a database, but S3 already provides you with key-value storage, and Hadoop can provide you with querying.- s3-db
Ultimately, you need to consider what sort of query functionality you need and what sort of consistency you can tolerate.
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
add a comment |
Here are some databases and database-like products that use S3 (or can use S3).
- Amazon Athena
- S3 Select
- Apache HBase
- Redshift
Also, if you want some theory, here’s a MIT paper about Building a Database on S3.
This is by no means exhaustive, but it’s probably a good place to start.
Update
Here are some more that aren't AWS owned software.
- Cassandra
Hadoop—this isn't a database, but S3 already provides you with key-value storage, and Hadoop can provide you with querying.- s3-db
Ultimately, you need to consider what sort of query functionality you need and what sort of consistency you can tolerate.
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
add a comment |
Here are some databases and database-like products that use S3 (or can use S3).
- Amazon Athena
- S3 Select
- Apache HBase
- Redshift
Also, if you want some theory, here’s a MIT paper about Building a Database on S3.
This is by no means exhaustive, but it’s probably a good place to start.
Update
Here are some more that aren't AWS owned software.
- Cassandra
Hadoop—this isn't a database, but S3 already provides you with key-value storage, and Hadoop can provide you with querying.- s3-db
Ultimately, you need to consider what sort of query functionality you need and what sort of consistency you can tolerate.
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
Here are some databases and database-like products that use S3 (or can use S3).
- Amazon Athena
- S3 Select
- Apache HBase
- Redshift
Also, if you want some theory, here’s a MIT paper about Building a Database on S3.
This is by no means exhaustive, but it’s probably a good place to start.
Update
Here are some more that aren't AWS owned software.
- Cassandra
Hadoop—this isn't a database, but S3 already provides you with key-value storage, and Hadoop can provide you with querying.- s3-db
Ultimately, you need to consider what sort of query functionality you need and what sort of consistency you can tolerate.
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
edited Nov 26 '18 at 22:55
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
answered Nov 25 '18 at 3:39
Matthew PopeMatthew Pope
1,3921612
1,3921612
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
add a comment |
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
So it means all "DB" based on S3 is part of AWS platform services? There's not open source DB that does the same?
– xybrek
Nov 26 '18 at 20:49
1
1
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
HBase is an open source project. I don't know if the HBase to S3 connector is open source, but you don't pay extra for it and you can't run HBase on S3 without using S3 anyway. If there's something specific you're looking for, you should make your question more specific.
– Matthew Pope
Nov 26 '18 at 21:44
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
I mean, I am using Wasabi S3-compatible, accessed through key-pair is that possible with such DB?
– xybrek
Nov 26 '18 at 22:24
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53383801%2fwhat-datastore-database-runs-on-top-of-s3%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
