c - Read text from a file and realloc when needed

Question

I want to read text from a text file line by line and do some processing on these lines. I can do all processing, but I can't do grow memory with malloc-realloc. I gave limited memory first, if my text file's lines characters are in this limit everything is ok. If I use large files like 10,000 chars per line it only reads until my limit. I don't exactly understand how to use realloc(). what can i do about this code?

 void stat(char* fileptr)
{

  FILE *fp;
  char *linebuffer;
  int line=0;
  int sum=0;
  int max=0;
  int min=0;
  int maxlinelen=512;
  int i=0,j=0;
  int maxlen=512;
  int curlinelen[maxlen];

  linebuffer=(char*) malloc(maxlinelen * sizeof(char));
  if(linebuffer==NULL)
    {
      printf("Error occurred allocating memory for linebuffer");
      exit(1);
    }


  if((fp=fopen(fileptr,"r"))!=NULL)
  {
    while((fgets(linebuffer,maxlinelen,fp))!=NULL)
      {
    if(strlen(linebuffer)==maxlinelen)
      {
        maxlinelen*=2;
        linebuffer=realloc(linebuffer,maxlinelen * sizeof(char));
        if(linebuffer==NULL)
          {
        printf("Error occurred reallocating space for linebuffer");
        exit(1);
          }
      }
    line++;

        sum=sum+strlen(linebuffer);
    curlinelen[i]=strlen(linebuffer);
    i++;

      }
  }
  min=curlinelen[0];
  max=curlinelen[0];
  for(j=0;j<line;j++)
    {
      if(curlinelen[j]<min)
    {
      min=curlinelen[j];
    }
      if(curlinelen[j]>max)
    {
      max=curlinelen[j];
    }
    }


 printf("No. of lines        =%d
",line);
 printf("Maximum line length =%d
",max);
 printf("Minimum line length =%d
",min);       
 printf("Average line length =%8.2f
",(float)sum/(float)line);

 fclose(fp);
}

Answer 1

fgets(linebuffer,maxlinelen,fp)

reads and stores at most maxlinelen - 1 characters in linebuffer and 0-terminates it. Thus

if(strlen(linebuffer)==maxlinelen)

is never satisfied, strlen(linebuffer) can be at most maxlinelen - 1. Change the condition, and you will see that maxlinelen increases if the file contains long lines (unless realloc fails).

Your current code will however count the partial line read in as an entire line then, and read the next chunk of the line as a new line. To grow the buffer until the entire line fits in, you must continue reading from the file before collecting the line length and incrementing the line count. But we must check whether a full line (including the newline at the end) was read in case fgets reads the maximal allowed number of chars before enlarging the buffer, or we'd concatenate the following line and count two (or in freak cases even more) lines as one.

while((fgets(linebuffer,maxlinelen,fp))!=NULL)
  {
  while((strlen(linebuffer) == maxlinelen-1) && (linebuffer[maxlinelen-2] != '
'))
  {
    maxlinelen*=2;
    linebuffer=realloc(linebuffer,maxlinelen * sizeof(char));
    if(linebuffer==NULL)
    {
        printf("Error occurred reallocating space for linebuffer");
        exit(1);
    }
    fgets(linebuffer + (maxlinelen/2 - 1), maxlinelen/2 + 1, fp);
  }

would be a (rather inefficient, due to the strlen calls) way to do that.