Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add modification_time field to filestat input plugin #3305

Merged
merged 8 commits into from
Nov 7, 2017
Merged

add modification_time field to filestat input plugin #3305

merged 8 commits into from
Nov 7, 2017

Conversation

puckpuck
Copy link
Contributor

@puckpuck puckpuck commented Oct 4, 2017

Required for all PRs:

  • Signed CLA.
  • Associated README.md updated.
  • Has appropriate unit tests.

Adds a modification_time field to filestat. Fixes #2116

I'm not particularly fond of the way unit tests has to work for this. Because the modification time will change based on when you fetch from git, there is no easy way to unit test this. My solution was to create a function which is leveraged by each unit test to set the expected modification time to that of the actual file. If this approach isn't acceptable, I'm happy to change it to something else.

@danielnelson
Copy link
Contributor

Sounds like we need to add an interface and mock globpath.

I also wonder if we should use a different format for the time, other options I can think of are unixnano 1507152973123456789i or as a decimal float 1507152973.123456789. It is just a file so precision is likely not that important, so maybe the current unix time is fine. @desa What do you think?

if fileInfo == nil {
return 0
} else {
return fileInfo.ModTime().Unix()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about providing precision greater than 1s?

@danielnelson
Copy link
Contributor

@phemmer Do you think we should use UnixNano format here? Floats are easier to read IMO but not as accurate?

@phemmer
Copy link
Contributor

phemmer commented Oct 4, 2017

That would make sense to me (using UnixNano()).

@danielnelson
Copy link
Contributor

Let's make that change too then @puckpuck

@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 5, 2017

updated to use UnixNano() ... let me know if you need anything else

"exists": int64(1),
"size_bytes": int64(0),
"exists": int64(1),
"modification_time": int64(getModificationTime(fs, tags1["file"])),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just use acc.HasInt64Field( to assert that the field is set, then you can remove the getModificationTime function. This will be good enough for now in my opinion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping you wouldn't ask for something like that 😉

means I need to modify each test case to assert all fields individually, because acc.AssertContainsTaggedFields(...) will fail now. Hence why I did the getModificationTime function... required fewer changes to implement.

Test each field individually and modification_time is an int64 when exists is 1
@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 8, 2017

Updated to test each field individually and modification_time is an int64 if the file exists. I think this looks kinda ugly, happy to revert it back, change something else, or leave it as is.

@danielnelson
Copy link
Contributor

It is too bad from a naming perspective that AssertContainsTaggedFields requires the fields to be exactly equal, I might rename this function sometime soon.

In the meantime I added HasPoint and converted filestat_test.go, sorry about all the conflicts but I think this will allow the tests to be reasonable.

@puckpuck
Copy link
Contributor Author

puckpuck commented Oct 10, 2017

This is helping, but still have the issue that I don't have a way to test an accumulator that it contains an int64 field, for a specific set of tags.

Given this test set:

		dir + "log1.log",
		dir + "log2.log",
		"/non/existant/file",
	}

	acc := testutil.Accumulator{}
	acc.GatherError(fs.Gather)

The first 2 files should have a modification_time but the 3rd should not, however acc.HasInt64Field("filestat", "modification_time") will always return true because at least 1 of the 3 metrics have it. I almost need a HasInt64FieldForTags function

Not wanting to make this become such a monumental task for testing, I decided to break it out into 2 more unit test. Each will test for a single file only, first one should have the field, the second should not.

@danielnelson danielnelson added this to the 1.5.0 milestone Nov 7, 2017
@danielnelson danielnelson added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Nov 7, 2017
@danielnelson danielnelson merged commit dcff769 into influxdata:master Nov 7, 2017
@puckpuck puckpuck deleted the filestat-modtime branch February 2, 2018 15:14
aromeyer pushed a commit to aromeyer/telegraf that referenced this pull request May 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants