Add Singleton support for Functions to ensure only one function running at a time

[mathewc]

We should discuss whether we want to bring this functionality forward for Functions. However, it will be relatively simple for us to do - we just need to expose new properties via function.json for function level singletons. See here for singleton doc.

In addition to Function level singleton support, we might also consider supporting Listener level Singleton as well.

When we do this, we should make sure it works across multiple languages (not just C#)

[alrod]

Customer wants this feature:
https://social.msdn.microsoft.com/Forums/en-US/4f21e730-e4f3-4f5f-a408-56e23283fb92/run-azure-function-as-singleton-listener-with-node-js?forum=AzureFunctions

[vladkosarev]

We would love to have that. Let us specify singleton in different scopes just like in web jobs.

[mathewc]

Thanks @vladkosarev. This issue has been on the back burner, since not many people have been asking for it yet. What specifically is your scenario?

[vladkosarev]

Being able to process messages from multiple queues in series as long as they affect the same entity. So if we have two queues and both of them will affect Order/1 (which is encoded in queue messages) we want the two different functions to kick in in series and process messages from those two queues one by one instead of in parallel. The idea here is having serialized access to data storage for a particular entity.

[ajukraine]

@vladkosarev sounds like actor model. I would love to see how efficient it's with using scoped singleton and Azure functions. Another option is to use Service Fabric, but it's huge piece of infrastructure to be handled. Prefer to start with something as lightweight as Azure functions

[vladkosarev]

That is exactly what I'm trying to achieve. Actor-like model using Azure functions. Now that they announced that functions will support serverless framework this might not be as important but I'd still like this ability in the 'raw'. Service Fabric is great but it's still not serverless. I want to pay for compute resources not for VMs. Consumption functions + actors on top is my path to nirvana. Obviously you can't have in memory state, etc but it would still be a good start to the path of properly infinitely scalable architecture.

[WonderPanda]

@lindydonna Hey I'm just wondering if there are any updates in regards to when we might expect support for singleton functionality with Azure Functions. Based on the conversation here it seemed like it might be low hanging fruit. It would be extremely helpful to be able to specify singleton behavior in function.json

[tmakin]

I have use case where this would very helpful. I have long running background processing tasks which are triggered via a storage queue. In order to prevent spikes in the database load, I need to ensure that that there is not more than one queue item being processed at a time.

My current solution is to use a timer trigger with a short interval to manually poll the queue, but a singleton flag for queue triggers would be a much tidier option.

[ericleigh007]

Yes, I didn't know what this was from the title. Perhaps rename the report to something more descriptive.

My case is just about the same. Would like to guarantee only one queue function running at a time. NOTE with the current time limitations we cannot just wait.

-thanks Donna
-e

[alohaninja]

Seems like you could support locking on your own - you just need shared storage backing it - SQL Azure, Azure Blob/Table/Queue, Redis, etc. Would be great just to add a [SingletonAttribute] to our Azure Functions like webjobs has. host.json has configurable options for singletons, but I don't know if it supports Azure functions (AF) versus Web Jobs. Could just add an environment key which has the storage connection string etc, and assign it in our host.json.

[alexjebens]

@lindydonna any updates on a timeline for this feature?

I have a couple of Projects where I would like to switch from WebJobs to Azure Functions as well as some new Projects that need serial processing for queues, which require this functionality.

From my understanding of the way this works for WebJobs is a lock blob with a lease in the storage accounts. Azure Functions appear to already use this mechanism for the Timer-Trigger.

[alexjebens]

@alohaninja Supporting locking on our own is not trivial. e.g. In a Queue-Trigger Function you could only throw an exception so that the message is put back in the queue, this may however lead to the message being marked as poison and therefore lost if you cannot process it in time. Additionaly the Function will still be invoked leading to extra costs.

According to this issue there is currently no support for the Singleton Attribute in Azure Functions and the host.json options are therefore moot.

Possible Workarounds:

• Timer-Trigger Functions appear to run as Singletons. They produce a blob in the storage account under "locks/" and create a lease. This requires implementing the input on your own.
• ServiceBus-Trigger Functions with serviceBus.maxConcurrentCalls to 1 in host.json. This is however a global setting and I would like use this on a per function basis.

@lindydonna is it possible to confirm that Timer-Trigger Functions run as singletons?