How to declare mapping for nested fields in Elasticsearch to allow for storing different types?ElasticSearch automatic nested mappingElasticsearch mapping - different data types in same fieldHow to highlight nested fields in ElasticsearchElasticsearch mapping : how to store a not-indexed field that could be of different typesWhat are the advantages of mapping a field to a type in Elasticsearch?Aggregating nested fields of varying datatypes in ElasticsearchElasticsearch - dynamic field template mappingdynamically searching in multi/all elasticsearch nested fieldElasticsearch Mapping With Nested DatatypeMapping for a nested map in elasticsearch
The Binomial Elks Club
How to ask my office to remove the pride decorations without appearing anti-LGBTQ?
Alternator dying so junk car?
How to delete certain lists from a nested list?
Is there a way to get more home Xbox switches?
What are the arguments for California’s nonpartisan blanket (jungle) primaries?
How to determine the optimal threshold to achieve the highest accuracy
Sankhara meditation
What happens when I team swap while I have Pokemon inside a gym?
Animal Shelter Management C++
Unix chat server making communication between terminals possible
Does the Intel 8085 CPU use real memory addresses?
Is the Gritty Realism variant incompatible with dungeon-based adventures?
Is there an English equivalent for "Les carottes sont cuites", while keeping the vegetable reference?
Is this Android phone Android 9.0 or Android 6.0?
Why did Steve Rogers choose this character in Endgame?
(Piano) is the purpose of sheet music to be played along to? Or a guide for learning and reference during playing?
What problems was on a lunar module of Apollo 11?
Strategy to pay off revolving debt while building reserve savings fund?
Cine footage fron Saturn V launch's
Is it OK to use personal email ID for faculty job applications or should we use (current) institute's ID
When does Fisher's "go get more data" approach make sense?
Did Voldemort kill his father before finding out about Horcruxes?
Alphanumeric Line and Curve Counting
How to declare mapping for nested fields in Elasticsearch to allow for storing different types?
ElasticSearch automatic nested mappingElasticsearch mapping - different data types in same fieldHow to highlight nested fields in ElasticsearchElasticsearch mapping : how to store a not-indexed field that could be of different typesWhat are the advantages of mapping a field to a type in Elasticsearch?Aggregating nested fields of varying datatypes in ElasticsearchElasticsearch - dynamic field template mappingdynamically searching in multi/all elasticsearch nested fieldElasticsearch Mapping With Nested DatatypeMapping for a nested map in elasticsearch
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
In essence, I want my mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
When I try to add a document where some fields have different types of values, I get an error like this:
"type": "illegal_argument_exception",
"reason": "mapper [data.customData.value] of different type, current_type [long], merged_type [text]"
This can easily be solved by mapping the field value to text (or create it dynamically by first inserting a document with only text). However, I would like to avoid having a schema. Perhaps having all of the fields nested in customData to be set to text? How do I do that?
I had the problem earlier, but then it started working after accidentally managing to get a dynamical mapping that worked (since everything was regarded as text. I was later made aware of this problem since I needed to change the mapping to allow for nested types.
Documents with this kind of data are troublesome to store successfully:
"customData": [
"value": "some_text",
"key": "some_text"
,
"value": 0,
"key": "some_text"
]
A part of the mapping that works:
"my_index":
"aliases": ,
"mappings":
"_doc":
"properties":
"data":
"properties":
"customData":
"properties":
"key":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"value":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"some_list":
"type": "nested",
"properties":
"some_field":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
In essence, I want the mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
"mappings":
"_doc":
"properties":
"data":
"type": "object"
,
"somee_list":
"type": "nested"
So what would be the best approach to go about this problem?
elasticsearch types nested mapping asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
add a comment |
In essence, I want my mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
When I try to add a document where some fields have different types of values, I get an error like this:
"type": "illegal_argument_exception",
"reason": "mapper [data.customData.value] of different type, current_type [long], merged_type [text]"
This can easily be solved by mapping the field value to text (or create it dynamically by first inserting a document with only text). However, I would like to avoid having a schema. Perhaps having all of the fields nested in customData to be set to text? How do I do that?
I had the problem earlier, but then it started working after accidentally managing to get a dynamical mapping that worked (since everything was regarded as text. I was later made aware of this problem since I needed to change the mapping to allow for nested types.
Documents with this kind of data are troublesome to store successfully:
"customData": [
"value": "some_text",
"key": "some_text"
,
"value": 0,
"key": "some_text"
]
A part of the mapping that works:
"my_index":
"aliases": ,
"mappings":
"_doc":
"properties":
"data":
"properties":
"customData":
"properties":
"key":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"value":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"some_list":
"type": "nested",
"properties":
"some_field":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
In essence, I want the mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
"mappings":
"_doc":
"properties":
"data":
"type": "object"
,
"somee_list":
"type": "nested"
So what would be the best approach to go about this problem?
elasticsearch types nested mapping asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02
add a comment |
In essence, I want my mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
When I try to add a document where some fields have different types of values, I get an error like this:
"type": "illegal_argument_exception",
"reason": "mapper [data.customData.value] of different type, current_type [long], merged_type [text]"
This can easily be solved by mapping the field value to text (or create it dynamically by first inserting a document with only text). However, I would like to avoid having a schema. Perhaps having all of the fields nested in customData to be set to text? How do I do that?
I had the problem earlier, but then it started working after accidentally managing to get a dynamical mapping that worked (since everything was regarded as text. I was later made aware of this problem since I needed to change the mapping to allow for nested types.
Documents with this kind of data are troublesome to store successfully:
"customData": [
"value": "some_text",
"key": "some_text"
,
"value": 0,
"key": "some_text"
]
A part of the mapping that works:
"my_index":
"aliases": ,
"mappings":
"_doc":
"properties":
"data":
"properties":
"customData":
"properties":
"key":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"value":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"some_list":
"type": "nested",
"properties":
"some_field":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
In essence, I want the mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
"mappings":
"_doc":
"properties":
"data":
"type": "object"
,
"somee_list":
"type": "nested"
So what would be the best approach to go about this problem?
elasticsearch types nested mapping asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
In essence, I want my mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
When I try to add a document where some fields have different types of values, I get an error like this:
"type": "illegal_argument_exception",
"reason": "mapper [data.customData.value] of different type, current_type [long], merged_type [text]"
This can easily be solved by mapping the field value to text (or create it dynamically by first inserting a document with only text). However, I would like to avoid having a schema. Perhaps having all of the fields nested in customData to be set to text? How do I do that?
I had the problem earlier, but then it started working after accidentally managing to get a dynamical mapping that worked (since everything was regarded as text. I was later made aware of this problem since I needed to change the mapping to allow for nested types.
Documents with this kind of data are troublesome to store successfully:
"customData": [
"value": "some_text",
"key": "some_text"
,
"value": 0,
"key": "some_text"
]
A part of the mapping that works:
"my_index":
"aliases": ,
"mappings":
"_doc":
"properties":
"data":
"properties":
"customData":
"properties":
"key":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"value":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
,
"some_list":
"type": "nested",
"properties":
"some_field":
"type": "text",
"fields":
"keyword":
"type": "keyword",
"ignore_above": 256
In essence, I want the mapping to be as schemaless as possible, but allow for nested types and being able to store data that may have different types:
"mappings":
"_doc":
"properties":
"data":
"type": "object"
,
"somee_list":
"type": "nested"
So what would be the best approach to go about this problem?
elasticsearch types nested mapping elasticsearch types nested mapping asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
asked Mar 26 at 8:12
HellenbergHellenberg
33 bronze badges
33 bronze badges
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02
add a comment |
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55352479%2fhow-to-declare-mapping-for-nested-fields-in-elasticsearch-to-allow-for-storing-d%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.
Is this question similar to what you get asked at work? Learn more about asking and sharing private information with your coworkers using Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55352479%2fhow-to-declare-mapping-for-nested-fields-in-elasticsearch-to-allow-for-storing-d%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Store data as a text will take a lot of memory but will be searchable. You only works with words and want to have a kind of string dictionnary or you also want to work with geolocation, range, date fields?
– LeBigCat
Mar 26 at 9:42
At the moment the requirements for the searching capabilities are quite vague. At the moment it seems as if possible, the following comparators should be supported: ">=", ">", "<=", "<", "!=", "=". By indexing as keyword (instead of text as I first thought of) I should be able to allow for "!=" and "=". Is there any way to keep the types and try to handle cases where types do not match?
– Hellenberg
Mar 26 at 12:32
Elasticsearch index data according to their type. What s why it is so fast. Es philosophie is to build your own search engine. If you want somehting generic, try something like this: "key": keyword (and use as keyword something to know his type like "int_age", "value_key": keyword "type_key": keyword (= one es type like long, datetime, text, location, int, keyword....) "value_long": long "value_date": date "value_text": test "value_location" : location ...
– LeBigCat
Mar 26 at 15:27
So you could store the key value type you want, if you use a java - .net or another high level client, you should be able to build generic querry easily according to the type of the field.
– LeBigCat
Mar 26 at 15:28
That is a good idea! I should probably do that if given the time. Until then I think that I will treat every field as text or keyword. However, I do not manage to find a way to do that. What would be the best approach for only having one property-type for all objects in customData?
– Hellenberg
Apr 3 at 10:02