file - Dealing with Windows line-endings in Python

Question

I've got a 700MB XML file coming from a Windows provider.

As one might expect, the line endings are ' ' (or ^M in vi). What is the most efficient way to deal with this situation aside from getting the supplier to send over ' ' :-)

1. Use os.linesep
2. Use rstrip() (requiring opening the file ... which seems crazy)
3. Using Universal newline support is not standard on my Mac Snow Leopard - so isn't an option.

I'm open to anything that requires Python 2.6+ but it needs to work on Snow Leopard and Ubuntu 9.10 with minimal external requirements. I don't mind a small performance penalty but I am looking for the standard best way to deal with this.

----edit----

The line endings are in the middle of the tag descriptors, otherwise they wouldn't be such a problem. I know this is bad form and that they shouldn't be sending this to me, but this is how I have the file and the vendor is mostly incompetent.

Answer 1

Why are the DOS line-endings a problem? Most things can deal with them just fine, including XML parsers. If you really want to get rid of them, open the file in universal line-endings mode:

open(filename, 'rU')

Python will convert all line-endings to UNIX line-endings for you. If you really can't use that (which I find a little surprising), there's no way to get Python to do the work for you. You will have to open the file regardless, though, so your objection to #2 seems a little odd.