LINQ to TEXT and LINQ to CSV

re: LINQ to TEXT and LINQ to CSV

Jeffrey L. Whitledge — Wed, 01 Oct 2008 00:29:43 GMT

If I'm reading that code correctly, quoted strings in your CSV files cannot contain backslashes unless they are escaping quotes?

I find it very interesting that you decided against a functional approach, because when I wrote a CSV parsing method back in the nineties using VB4, I decided that the easiest way to do it would be a recursive function that parsed the first value, then passed the rest of the string to itself. It was not efficient, but it was easy to write. That was back before all the functional programming stuff appeared on my radar screen.

re: LINQ to TEXT and LINQ to CSV

EricWhite — Wed, 01 Oct 2008 20:45:05 GMT

Hi Jeffrey,

You are absolutely right, quoted strings could not contain backslashes unless escaping quotes. I've updated the code so that backslashes can escape any character, including backslash.

Regarding writing this using the functional approach, your idea is interesting. However, given that I only take CSV files under duress, and then only for a quick and dirty solution to data transformation (generally, I immediately convert to an XML document), this is probably something I'm not going to do in the near future. :-)

But this raises the point - til now, I think that pattern matching of certain types is one of the areas where it is harder to use LINQ. I'm still thinking about this area - what would a regex replacement look like using LINQ? What are the appropriate LINQ patterns to use when you have items in a collection that depend on their context, which in turn depends on an enclosing context? Grammars and recursive descent parsers are the cannonical way to do this - to write lexical parsers, for instance. But what about just looking for patterns in a string?

I've been thinking about approaches to specify patterns to match, and then having a method that yields up matches, but the implementation of the method that yields matches is an area that I haven't finalized. I have ideas about how to do this with queries, but haven't fully developed them. This is going to require more research. (I'm sure that some of the FP experts out there have solved this problem; I just don't know the solution.)

-Eric

re: LINQ to TEXT and LINQ to CSV

Francis — Sun, 12 Oct 2008 20:10:26 GMT

Other sample in codeproject web site

re: LINQ to TEXT and LINQ to CSV

MarkJ — Thu, 26 Mar 2009 14:57:44 GMT

"Stop rolling your own CSV parser"? And use a free one instead.

http://www.secretgeek.net/csv_trouble.asp

re: LINQ to TEXT and LINQ to CSV

jim b — Thu, 14 May 2009 23:39:43 GMT

Great post, It helped out immensely. One question, would it be possible to make the

return new {

Person = split[0],

Quote = split[1]

};

dynamic in nature, ie use a configuration file to control the property and the location in the string to get the value from.

thanks,

jim

re: LINQ to TEXT and LINQ to CSV

EricWhite — Fri, 15 May 2009 00:15:16 GMT

Hi Jim, it certainly would be possible - with .NET 3.5, one way to do this is using a Dictionary<T> per row. With .NET 4.0, there will be dynamic types that will enable this in a very clean way.

-Eric

re: LINQ to TEXT and LINQ to CSV

jbutler — Fri, 15 May 2009 17:56:31 GMT

thanks for the tip, we are using 3.5 now and i am anxiously awaiting 4.0 for quite a few things now :)

re: LINQ to TEXT and LINQ to CSV

Attila — Fri, 26 Jun 2009 16:48:51 GMT

Hi Eric!

Thanks for this post it's quite useful.

There is one thing that I encountered when I used this for the first time, there is no whitespace (or at least space) support in the code.

Because I'm writing much code, I did my sample file like this:

Alias,Display Name,Email

alias1, "test, me", asd@asd.com

For the 2nd row you'll parse it into 4 parts:

alias1

test

asd@asd.com

which is "wrong".

In my opinion, (white)spaces are allowed around a quoted string. For regular data I'm not sure, but if someone wants to use spaces in the data itself, then enclose the string in quotes like: " spaces around me ".

So for the regular data spaces are allowed too, but they must be trimmed, and quoted strings should be trimmed only outside of the quotes.

It was quite easy to add support spaces as I described above.

I added this as the last check in the AtBeginningOfToken state:

if (sourceCharArray[i] == ' ')

{

continue;

}

and this as the FIRST check in the ExpectingComma state:

if (sourceCharArray[i] == ' ')

{

continue;

}

The following enhancements could make it more generic:

- Add support for parameterized quote identifier(s)

- Add support for parameterized separator

- Add support for number parsing, based on number format (I know it's not definitely a split function, but can be :-))

Thanks,

Attila