ES automatically creates mapping for each new object #178
Description
I have an object stored like so in FB:
"users" : {
"0" : {
"bio" : "Consectetur ut consectetur eiusmod sunt ipsum. Laboris velit ad et magna tempor deserunt. Sint commodo voluptate aliquip...",
"citystate" : "Driftwood,Wisconsin",
"email" : "suttonschmidt@avit.com",
"events" : [ {
"date" : "2017-02-11",
"eventname" : "do",
"id" : 12
}, {
"date" : "2014-01-08",
"eventname" : "esse",
"id" : 14
}, {
"date" : "2017-03-24",
"eventname" : "commodo",
"id" : 71
}, {
"date" : "2017-10-18",
"eventname" : "ad",
"id" : 12
}, {
"date" : "2014-12-10",
"eventname" : "id",
"id" : 4
}, {
"date" : "2014-08-11",
"eventname" : "dolor",
"id" : 46
}, {
"date" : "2014-07-17",
"eventname" : "nulla",
"id" : 21
} ],
"phone" : "(811) 459-2045",
"picture" : "https://loremflickr.com/320/240/music",
"tags" : [ "occaecat", "tempor", "ex", "adipisicing", "nulla", "ea", "culpa" ],
"username" : "eu"
}...
When this object is initially created wholesale, firebase renders the "events" property as an array, because each key in the array is an int. However, if I were to add an event to the events "array" via a POST to myfirebaseinstance.com/users/0/events.json, the unique key autogenerated by firebase causes ES to now index each child as its own object in the mapping.
So when I created my mapping in ES, I had
"events" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
}
but once that POST goes through to add an event to the user's events array, FB stops rendering it as an array and now renders it as an object because the unique key (ex: "-K-ASDAIS"). Now that this is rendered as an object instead of an array, ES updates the mapping to accommodate, and now my mapping went from looking like the above to looking something like this:
"events" : {
"properties" : {
"-Kz_wVObvfW3ZE2vkzgF" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"-L-xyscEkIxgrH7JbuWK" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"-L-xz1RF_y_6D0MYu-12" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"0" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"1" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"2" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"3" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"4" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"id" : {
"type" : "long"
}
}
},
"5" : {
"properties" : {
"date" : {
"type" : "date"
},
"eventname" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
Obviously its not scalable to have ES update my mapping every time I add an event to a user's event's array. Also, it breaks my Android client's serialization. The client is expecting an array wheras now it's getting an object.
Is there any way around this? Am I missing something obvious? Or should I not have used Flashlight/Firebase/ES for this particular use case? I'll admit I didn't know about FB's lack of array support until tonight, so part of this is on me. I'm just hoping I can find a workaround rather than having to reimplement my backend from scratch.
Activity