Mixing
spring data elastisearch change indexName dynamically
I'm trying to use spring data elastisearch save some data. I need to create same index for different client. Ex. If I have index my-index, I need create my-index-A, my-index-B for client A and B. But annotation @Document works only with static indexName or with spEL which is not thread safe.
My question is, if I create index and search manually (ElasticsearchTemplate.createIndex(), NativeSearchQueryBuilder().withIndices()), and delete this line on entity class.
@Document(indexName = "my-index-A")
The entity can still receive its values? In another words, the annotation
@Id
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String aid;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String entityId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userName;
Still works?
spring elasticsearch spring-data spring-data-elasticsearch
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
add a comment |
I'm trying to use spring data elastisearch save some data. I need to create same index for different client. Ex. If I have index my-index, I need create my-index-A, my-index-B for client A and B. But annotation @Document works only with static indexName or with spEL which is not thread safe.
My question is, if I create index and search manually (ElasticsearchTemplate.createIndex(), NativeSearchQueryBuilder().withIndices()), and delete this line on entity class.
@Document(indexName = "my-index-A")
The entity can still receive its values? In another words, the annotation
@Id
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String aid;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String entityId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userName;
Still works?
spring elasticsearch spring-data spring-data-elasticsearch
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
add a comment |
I'm trying to use spring data elastisearch save some data. I need to create same index for different client. Ex. If I have index my-index, I need create my-index-A, my-index-B for client A and B. But annotation @Document works only with static indexName or with spEL which is not thread safe.
My question is, if I create index and search manually (ElasticsearchTemplate.createIndex(), NativeSearchQueryBuilder().withIndices()), and delete this line on entity class.
@Document(indexName = "my-index-A")
The entity can still receive its values? In another words, the annotation
@Id
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String aid;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String entityId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userName;
Still works?
spring elasticsearch spring-data spring-data-elasticsearch
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
I'm trying to use spring data elastisearch save some data. I need to create same index for different client. Ex. If I have index my-index, I need create my-index-A, my-index-B for client A and B. But annotation @Document works only with static indexName or with spEL which is not thread safe.
My question is, if I create index and search manually (ElasticsearchTemplate.createIndex(), NativeSearchQueryBuilder().withIndices()), and delete this line on entity class.
@Document(indexName = "my-index-A")
The entity can still receive its values? In another words, the annotation
@Id
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String aid;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String entityId;
@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
private String userName;
Still works?
spring elasticsearch spring-data spring-data-elasticsearch
spring elasticsearch spring-data spring-data-elasticsearch
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
asked Nov 20 '18 at 9:19
Chao JiangChao Jiang
223
223
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
TL;DR
Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.
Explanation:
If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.
Solution
I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.
Proof
Writing with
ElasticEntity foo = new ElasticEntity();
foo.setAid("foo-a-id");
foo.setEntityId("foo-entity-id");
foo.setUserName("foo-user-name");
foo.setUserId("foo-user-id");
IndexQuery fooIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-foo")
.withObject(foo)
.build();
String fooId = template.index(fooIdxQuery);
and
ElasticEntity bar = new ElasticEntity();
bar.setAid("bar-a-id");
bar.setEntityId("bar-entity-id");
bar.setUserName("bar-user-name");
bar.setUserId("bar-user-id");
IndexQuery barIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-bar")
.withObject(bar)
.build();
String barId = template.index(barIdxQuery);
should store the objects in differnet indices.
Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 10,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
{
"_index" : "idx-bar",
"_type" : "elasticentity",
"_id" : "bar-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "bar-a-id",
"userId" : "bar-user-id",
"entityId" : "bar-entity-id",
"userName" : "bar-user-name"
}
},
{
"_index" : "idx-foo",
"_type" : "elasticentity",
"_id" : "foo-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "foo-a-id",
"userId" : "foo-user-id",
"entityId" : "foo-entity-id",
"userName" : "foo-user-name"
}
}
]
}
}
As you can see, the index name and the _id is correct in the response.
Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withIndices("idx-foo", "idx-bar")
.build();
List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());
The mapping works too as the logger yields fully populated classes in the result:
[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]
Hope this helped!
answered Nov 20 '18 at 23:19
ibexitibexit
760413
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53389760%2fspring-data-elastisearch-change-indexname-dynamically%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
TL;DR
Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.
Explanation:
If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.
Solution
I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.
Proof
Writing with
ElasticEntity foo = new ElasticEntity();
foo.setAid("foo-a-id");
foo.setEntityId("foo-entity-id");
foo.setUserName("foo-user-name");
foo.setUserId("foo-user-id");
IndexQuery fooIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-foo")
.withObject(foo)
.build();
String fooId = template.index(fooIdxQuery);
and
ElasticEntity bar = new ElasticEntity();
bar.setAid("bar-a-id");
bar.setEntityId("bar-entity-id");
bar.setUserName("bar-user-name");
bar.setUserId("bar-user-id");
IndexQuery barIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-bar")
.withObject(bar)
.build();
String barId = template.index(barIdxQuery);
should store the objects in differnet indices.
Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 10,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
{
"_index" : "idx-bar",
"_type" : "elasticentity",
"_id" : "bar-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "bar-a-id",
"userId" : "bar-user-id",
"entityId" : "bar-entity-id",
"userName" : "bar-user-name"
}
},
{
"_index" : "idx-foo",
"_type" : "elasticentity",
"_id" : "foo-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "foo-a-id",
"userId" : "foo-user-id",
"entityId" : "foo-entity-id",
"userName" : "foo-user-name"
}
}
]
}
}
As you can see, the index name and the _id is correct in the response.
Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withIndices("idx-foo", "idx-bar")
.build();
List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());
The mapping works too as the logger yields fully populated classes in the result:
[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]
Hope this helped!
answered Nov 20 '18 at 23:19
ibexitibexit
760413
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
add a comment |
TL;DR
Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.
Explanation:
If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.
Solution
I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.
Proof
Writing with
ElasticEntity foo = new ElasticEntity();
foo.setAid("foo-a-id");
foo.setEntityId("foo-entity-id");
foo.setUserName("foo-user-name");
foo.setUserId("foo-user-id");
IndexQuery fooIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-foo")
.withObject(foo)
.build();
String fooId = template.index(fooIdxQuery);
and
ElasticEntity bar = new ElasticEntity();
bar.setAid("bar-a-id");
bar.setEntityId("bar-entity-id");
bar.setUserName("bar-user-name");
bar.setUserId("bar-user-id");
IndexQuery barIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-bar")
.withObject(bar)
.build();
String barId = template.index(barIdxQuery);
should store the objects in differnet indices.
Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 10,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
{
"_index" : "idx-bar",
"_type" : "elasticentity",
"_id" : "bar-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "bar-a-id",
"userId" : "bar-user-id",
"entityId" : "bar-entity-id",
"userName" : "bar-user-name"
}
},
{
"_index" : "idx-foo",
"_type" : "elasticentity",
"_id" : "foo-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "foo-a-id",
"userId" : "foo-user-id",
"entityId" : "foo-entity-id",
"userName" : "foo-user-name"
}
}
]
}
}
As you can see, the index name and the _id is correct in the response.
Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withIndices("idx-foo", "idx-bar")
.build();
List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());
The mapping works too as the logger yields fully populated classes in the result:
[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]
Hope this helped!
answered Nov 20 '18 at 23:19
ibexitibexit
760413
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
add a comment |
TL;DR
Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.
Explanation:
If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.
Solution
I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.
Proof
Writing with
ElasticEntity foo = new ElasticEntity();
foo.setAid("foo-a-id");
foo.setEntityId("foo-entity-id");
foo.setUserName("foo-user-name");
foo.setUserId("foo-user-id");
IndexQuery fooIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-foo")
.withObject(foo)
.build();
String fooId = template.index(fooIdxQuery);
and
ElasticEntity bar = new ElasticEntity();
bar.setAid("bar-a-id");
bar.setEntityId("bar-entity-id");
bar.setUserName("bar-user-name");
bar.setUserId("bar-user-id");
IndexQuery barIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-bar")
.withObject(bar)
.build();
String barId = template.index(barIdxQuery);
should store the objects in differnet indices.
Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 10,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
{
"_index" : "idx-bar",
"_type" : "elasticentity",
"_id" : "bar-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "bar-a-id",
"userId" : "bar-user-id",
"entityId" : "bar-entity-id",
"userName" : "bar-user-name"
}
},
{
"_index" : "idx-foo",
"_type" : "elasticentity",
"_id" : "foo-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "foo-a-id",
"userId" : "foo-user-id",
"entityId" : "foo-entity-id",
"userName" : "foo-user-name"
}
}
]
}
}
As you can see, the index name and the _id is correct in the response.
Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withIndices("idx-foo", "idx-bar")
.build();
List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());
The mapping works too as the logger yields fully populated classes in the result:
[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]
Hope this helped!
answered Nov 20 '18 at 23:19
ibexitibexit
760413
TL;DR
Spring-Data-Elasticseach won´t work anymore if you remove the @Document annotation from your class.
Explanation:
If you remove @Document from your class, several elasticsearch operations will fail when reading or writing (when determining index name, type and id) as ElasticsearchTemplate.getPersistentEntityFor(Class clazz) relies heavily on this annotation.
Solution
I have managed to successfully read/write with different indices using one annotated class with a dummy annotation @Document(indexName = "dummy", createIndex = false) and explicitly setting the index name for all read/write operations using elasticsearchTemplate.
Proof
Writing with
ElasticEntity foo = new ElasticEntity();
foo.setAid("foo-a-id");
foo.setEntityId("foo-entity-id");
foo.setUserName("foo-user-name");
foo.setUserId("foo-user-id");
IndexQuery fooIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-foo")
.withObject(foo)
.build();
String fooId = template.index(fooIdxQuery);
and
ElasticEntity bar = new ElasticEntity();
bar.setAid("bar-a-id");
bar.setEntityId("bar-entity-id");
bar.setUserName("bar-user-name");
bar.setUserId("bar-user-id");
IndexQuery barIdxQuery = new IndexQueryBuilder()
.withIndexName("idx-bar")
.withObject(bar)
.build();
String barId = template.index(barIdxQuery);
should store the objects in differnet indices.
Double checking with curl http://localhost:9200/idx-*/_search?pretty gives:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 10,
"successful" : 10,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [
{
"_index" : "idx-bar",
"_type" : "elasticentity",
"_id" : "bar-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "bar-a-id",
"userId" : "bar-user-id",
"entityId" : "bar-entity-id",
"userName" : "bar-user-name"
}
},
{
"_index" : "idx-foo",
"_type" : "elasticentity",
"_id" : "foo-a-id",
"_score" : 1.0,
"_source" : {
"aid" : "foo-a-id",
"userId" : "foo-user-id",
"entityId" : "foo-entity-id",
"userName" : "foo-user-name"
}
}
]
}
}
As you can see, the index name and the _id is correct in the response.
Reading works too using following code (you´ll need to change the query to your needs and set the indices to the current client)
SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(matchAllQuery())
.withIndices("idx-foo", "idx-bar")
.build();
List<ElasticEntity> elasticEntities = template.queryForList(searchQuery, ElasticEntity.class);
logger.trace(elasticEntities.toString());
The mapping works too as the logger yields fully populated classes in the result:
[ElasticEntity(aid=bar-a-id, userId=bar-user-id, entityId=bar-entity-id, userName=bar-user-name), ElasticEntity(aid=foo-a-id, userId=foo-user-id, entityId=foo-entity-id, userName=foo-user-name)]
Hope this helped!
answered Nov 20 '18 at 23:19
ibexitibexit
760413
edited Nov 20 '18 at 23:35
answered Nov 20 '18 at 23:19
ibexitibexit
760413
answered Nov 20 '18 at 23:19
ibexitibexit
760413
answered Nov 20 '18 at 23:19
ibexitibexit
760413
760413
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
add a comment |
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
This works. Thanks!
– Chao Jiang
Nov 27 '18 at 10:14
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53389760%2fspring-data-elastisearch-change-indexname-dynamically%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
