Allow empty items

[OutOfTouch]

Hi,

Great clean library that works except for one flaw, the option to remove white space needs to be optional so that empty values between delimiters match up to the headers, without that it is impossible to know which data value goes with which header column, assuming the first line in csv file is the header column names.

The line to remove the white space in this method should be made optional with a bool parameter.

private static ICollection<string> ProcessLine(string original_line, string? separator)
     {
         List<string> wordsInLine = original_line
             .Split(separator?.ToCharArray())
             .Select(str => str.Trim()).ToList();

         wordsInLine.RemoveAll(string.IsNullOrWhiteSpace);   // Probably not needed, but just in case

         return wordsInLine;
     }

[eduherminio]

Hi @OutOfTouch, thanks for trying the library.
Could you post an example of a file you're trying to parse and the code you're using to achieve it, to make sure I fully understand your use case?
If it's a .csv-styed file, please include a line where the library works and one where it doesn't.

[OutOfTouch]

@eduherminio I can just give you an example, say the first line in the csv file is your header column names and next line is your data lines.
Here is an example header line that I just made up, using pipe for the separator:
FirstName|LastName|Age|EyeColor
Here is an example data line that I just made up:
John|Doe||brown
You can see the matching age value is empty and if we remove the white space from the data line there is no way to know that the value brown goes with the header EyeColor because we will have 1 less element in our data line than our header line.

[eduherminio]

First let me tell you that, even if I'm going against my interests here, you should consider using one of the specialized libraries out there for parsing csv files if you're doing any critical job in terms of performance and/or reliability.

Having said that, I also find parsing csv files as a perfectly fine use case I want to support (and I didn't consider before).

I'll try to find an acceptable solution for it, however it requires more than the modification you suggested, since without further changes ParsedFile.NextLine<T> and ParsedLine.ToList<T> would fail when trying to parse that empty element if T is not string.

And parsing is kinda the point of this library (otherwise, you could just do File.ReadAllLines("<your_file_path>").Select(str => str.Split('|'));). That's why I also asked you how you were using FileParser.

[OutOfTouch]

@eduherminio I am using the library to read a csv file line by line that is pipe delimited so that I can write the values to a database. You are correct if the Type was not a string than it would fail on those statements where the value T is not a string. The problem is sometimes we are parsing a file from a third party and we can't completely control what that external system does that is creating the csv file.
I am digging more in your code and not sure how I would handle this scenario.
Age|Test1Score|Test2Score
35||88
I don't think you would need to change ParsedFile.NextLine<T> or ParsedLine.ToList<T>,
I am thinking that I would check if the element can be of Type T with a helper method and if not return the default value for the Type. This would require changing public static T Convert<T>(string str, TypeConverter? typeConverter = null), I will dig some more. Let me know if I missed something, thank you for discussing this with me. Or does this concept destroy why you created FileParser?

[eduherminio]

I am digging more in your code and not sure how I would handle this scenario. Age|Test1Score|Test2Score 35||88

Well, Test1Score needs to be the default value of the type you want to parse in that case

I am thinking that I would check if the element can be of Type T with a helper method and if not return the default value for the Type. This would require changing public static T Convert<T>(string str, TypeConverter? typeConverter = null)

That's the overall idea I was playing with earlier, but it turns out not to me trivial to return the default value of T in T Convert<T> because it messes with nullability of the returned value, which would require more dedication than playing with it for a few minutes.

Or does this concept destroy why you created FileParser?

Nah, It'd be nice to support the csv use case in a way that doesn't disturb the actual behavior of the library, I trust I'll find a way.
Not immediately, since I'm focused on something else right now, but I'll keep you posted.

thank you for discussing this with me

No worries, thanks for engaging!

[eduherminio]

@OutOfTouch It's taken me a while to find time to look into this, but I finally did.
It'll be released soon as part of v3.0.0.

[OutOfTouch]

@OutOfTouch It's taken me a while to find time to look into this, but I finally did. It'll be released soon as part of v3.0.0.

Thank you very much.