string - Max size of a file Python can open?

Question

I opened an 8 MB file in Python, because I wanted to batch change various types of file names. I went through and loaded the file into a string and used the string method replace to replace everything. I then noticed that only half of the file was being replaced; as if Python wasn't fully opening the file.

Is there some kind of string size limit or max file size limit that I must play within the bounds of in Python?

Refer to the code in Python search and replace not replacing properly.

I have changed to the suggested code. The buffer is an 8 MB HTML file that is over 150k lines. The replacement code works perfectly; it's just that it's not replacing everything. Or for example one error that is a pain is:

When I'm attempting to replace the string ff10 to FF-10, it'll be changed to FF-010.

Answer 1

No, there is no reachable maximum on the size of a file Python can open. 8 MB is tiny in modern terms. You made a mistake somewhere.

People regularly load gigabytes of data into memory. Depending on your computer's RAM, whether it's 64- or 32- bit OS and processor, the practical maximum for you may be anywhere from 1 GB up before you get a MemoryError.

As a test, I just loaded a 350 MB file into a string. It took only a few seconds. I then wrote it back out to a file. That took a little longer. I then hashed the file. The two are identical.

Python has no problems with large strings, until you hit the limit of your RAM, operating system, or processor.

You say you "went through and loaded the file into a string" -- that sounds like the first place you could have made a mistake. To load a file into a string, you just do fileobject.read(). If you did it some other way, that could be the problem.