[-]: fix race conditions in 'fileMetricReader', fixes #1198

[gemy26]

This PR fixes #1198

• Add sync.Mutex to fileMetricReader struct.
• used mutex locks with all operations (Create/Update/Delete).
• Add concurrent test cases.

[0xgouda]

No need to duplicate mu.Lock() mu.Unlock() everywhere we only need it around the critical section in the shared fmr.WriteMetrics()

Also, we should use RWLock instead, and lock the critical section in getMetrics() while reading.

Also, please do the same for yaml sources handler

[gemy26]

Thanks for reply,
If we only lock inside the read and write functions, we can still have a race condition. in case of multiple goroutines may call CreateMetric (or any other operations) at the same time Each goroutine would : GetMetrics() -> modify -> WriteMetrics(). Even if GetMetrics and WriteMetrics are protected, the sequence is not atomic. Two goroutines could read the same initial state, apply different changes, and then one write would overwrite the other, causing data loss.
Did i miss something ?

[0xgouda]

Oh yeah, you are right, sorry for that, I got a bit confused.

But we still should use RWLocks.

[gemy26]

I applied the changes and fixed sources yaml also I implemented getMetricsNoLock/getSourcesNoLock to be able to use RWLock and avoid deadlock in write operations.

Without it: CreateMetric acquires Lock -> Calls GetMetrics() which tries to acquire RLock() = Deadlock.

Please let me know if there's a better approach.