FR: FieldValue arrayUnion/arrayRemove for array with objects

[emadalam]

[REQUIRED] Describe your environment

• Operating System version: macOS 10.14.3
• Browser version: Google Chrome 74.0.3729.169
• Firebase SDK version: 6.0.2
• Firebase Product: database

[REQUIRED] Describe the problem

Steps to reproduce:

firebase.firestore.FieldValue.arrayUnion and firebase.firestore.FieldValue.arrayRemove are great if the array is made up of primitive values. However to update a field value which is made up of array of objects, it's impossible to perform the operation without querying the actual data and performing the union/without operation on the client.

While this is alright if we already have queried the data, but in cases where we haven't queried the data to begin with, it's an unnecessary overhead to make this request. What's worse is when we haven't subscribed to the document updates and are trying to manipulate the array with the stale values, it's going to cause more harm than good, defeating the purpose of allowing array operations with arrayUnion and arrayRemove.

Relevant Code:

// existing document data
const data = {
  uid: 'xxxx'
  key: 'value',
  uploadedDocuments: [{
    uid: 'yyyy',
    name: 'something.png',
    url: '....'
  }, {
    uid: 'zzzz',
    name: 'something-else.png',
    url: '....'
  }]
}

// update document but it will add new object to the list
firebase.firestore
  .doc('myCollection/xxxx')
  .update({
    uploadedDocuments:
      firebase.firestore.FieldValue.arrayUnion({
        uid: 'yyyy',
        name: 'something.png',
        url: '....'
      })
  })

What is desirable is an option to set uniqueBy parameter or something similar.

firebase.firestore.doc('myCollection/xxxx')
  .update({
    uploadedDocuments:
      firebase.firestore.FieldValue.arrayUnion({
        uid: 'yyyy',
        name: 'something.png',
        url: '....'
      }, firebase.firestore.FieldValue.uniqueBy('uid'))
  })

// signature
firebase.firestore.FieldValue.arrayUnion(
  ...values,
  firebase.firestore.FieldValue.uniqueBy(key)
)

[schmidt-sebastian]

@emadalam Thanks for filing this issue! Since array-union currently treats all elements as distinct, using a nested key-value object might be a better fit for you. If you can key your data by user ID, you could potentially do the following:

firestore.doc('myCollection/xxxx').update(`uploadedDocuments/${uid}`, {name: ...} );

We will keep your feature request in mind as we shape our APIs, but you might be able to adjust your data model to take advantage of our existing features.

As a further comment, if you do have to rely on Arrays and need to download, modify and set your data manually, please take a look at our Transaction API. If you perform these operations in a transaction, then you will not encounter issues with stale data.

[emadalam]

@schmidt-sebastian Thanks for your suggestion. Though it was just an example model. I do understand that I can model all my array needs into a hash map with unique keys, but that essentially means I'm not using arrays anymore.

Furthermore it would be tricky to remove items from the hash map without performing a complete update/set operation, while in case of firebase.firestore.FieldValue.arrayRemove it would be much simpler and convenient if it would support uniqueness identifier.

I'd keep an eye on this issue and see if the feature request gets implemented in near future 🙏

[schmidt-sebastian]

@emadalam You can remove a single key from a map as such:

firestore.doc('myCollection/xxxx').update(`uploadedDocuments/${uid}`,FieldValue.delete());

[emadalam]

@schmidt-sebastian Thanks for the input, however all the approaches are under the assumption that we model the data as hash map. The original question with array values, still remains.

Until the feature request is added, I'm using the below version for now that makes use of transaction api to update the array. I still don't like the fact that I need to fetch data and run a transaction just to perform an update operation, but I guess as of now there's no better alternate that exists 🤷‍♂If you know any better/faster alternate to perform the same, feel free to mention here 💪

import isPlainObject from 'lodash.isplainobject'
import get from 'lodash.get'
import partialRight from 'lodash.partialright'
import unionBy from 'lodash.unionby'
import reject from 'lodash.reject'
import firebase, { db } from 'configs/firebase'

export async function arrayUnion({ documentRef, path, uniqueBy, data }) {
  if (!uniqueBy || !isPlainObject(data)) {
    return documentRef.update({
      [path]: firebase.firestore.FieldValue.arrayUnion(data),
    })
  }

  const updateFn = partialRight(unionBy, uniqueBy)
  return updateArray({ documentRef, path, data, updateFn })
}

export async function arrayRemove({ documentRef, path, uniqueBy, data }) {
  if (!uniqueBy || !isPlainObject(data)) {
    return documentRef.update({
      [path]: firebase.firestore.FieldValue.arrayRemove(data),
    })
  }

  const updateFn = values => reject(values, { [uniqueBy]: data[uniqueBy] })
  return updateArray({ documentRef, path, data, updateFn })
}

export async function updateArray({
  documentRef,
  path,
  data,
  updateFn = firebase.firestore.FieldValue.arrayUnion,
}) {
  if (!documentRef || !path) return

  return db.runTransaction(async transaction => {
    const doc = await transaction.get(documentRef)
    if (!doc.exists) return

    const dataArray = Array.isArray(data) ? data : [data]
    const existingValues = get(doc.data(), path)
    const newValues = updateFn(existingValues, dataArray)
    await transaction.update(documentRef, { [path]: newValues })
  })
}

Usage:

arrayUnion({
  documentRef: db.doc('myCollection/xxxx'),
  path: 'uploadedDocuments',
  uniqueBy: 'uid',
  data: {
    uid: 'yyyy',
    name: 'something.png',
    url: '....',
  },
})

arrayRemove({
  documentRef: db.doc('myCollection/xxxx'),
  path: 'uploadedDocuments',
  uniqueBy: 'uid',
  data: {
    uid: 'yyyy',
    name: 'something.png',
    url: '....',
  },
})

[schmidt-sebastian]

@emadalam Unfortunately, there is no succinct way to manipulate array data based on partial equality. We will keep this issue updated as we evolve our APIs. If you do need to use arrays in your documents, updating them inside a Transaction (as shown in your code sample) is the best way forward at this point.

[goleary]

I'd like to register a vote for this functionality.

[napindc]

Very much needed!

[xnio94]

i encounter the same problem but in my case, i already downloaded the data because i'm subscribing to a stream, the problem is that I'm not sure how to combine this with the Transaction API
in other words, i don't know how to make a transaction without redownloading the data because i'm already subscribing to it.
so the solution I'm considering is to store the data JSON-serialized, and instead of dealing with an array of objects I'll deal with an array of JSON strings, I know it's ugly and may generate problems in the future but until now it seems the most reasonable solution for me
i hope firebase.firestore.FieldValue.arrayUnion and firebase.firestore.FieldValue.arrayRemove will support non-primitives values soon

[viktorlarsson]

@emadalam Did you get this to work? Trying out your code, but i'm a bit unsure if partialRight does what.. I want it do to. Take a simple checklist for example:

{id: 1, checked: true},
{id: 2, checked: false}
]

const data = {
   id: 2, 
  checked: true
}

arrayUnion({
      documentRef: firestore()
        .collection(`collection/checklists`)
        .doc(checklist.id),
      path: "checklistitems",
      uniqueBy: "id",
      data
    })

Seems to just leave the array in the same state.

[emadalam]

@viktorlarsson The code works just fine for the use case. However from what I see for your specific use case you'd need to change the updateFn of the arrayUnion function to account for merging the values.

const updateFn = (values, updates) =>
  _(values)
    .keyBy(uniqueBy)
    .merge(_.keyBy(updates, uniqueBy))
    .values()
    .value()

---

export async function arrayUnion({ documentRef, path, uniqueBy, data }) {
  if (!uniqueBy || !isPlainObject(data)) {
    return documentRef.update({
      [path]: firebase.firestore.FieldValue.arrayUnion(data),
    })
  }

  const updateFn = (values, updates) =>
    _(values)
      .keyBy(uniqueBy)
      .merge(_.keyBy(updates, uniqueBy))
      .values()
      .value()
  return updateArray({ documentRef, path, data, updateFn })
}

[viktorlarsson]

Works like a charm, thanks @emadalam !