OW 2.0 C: Substitute for fopen()?

[Harry Potter]

Hi! I have and use Open Watcom C/C++ and am using DMC to program file compression. Recently, I tried to convert the DMC code to OW C and ran into two problems: 1. some ANSI functions were not found as the library that contains them couldn't be found, but when I moved the program to the hard drive, it worked. And 2. the fopen() function didn't work properly: fread() reported zero bytes read, and the program crashed a few seconds later. I really want OW's optimizations, as the compression code is too slow. What functions do I use to replace the ANSI file functions?

[Charlie Gibbs]

Post by Harry Potter
Hi! I have and use Open Watcom C/C++ and am using DMC to program
file compression. Recently, I tried to convert the DMC code to OW C
and ran into two problems: 1. some ANSI functions were not found as
the library that contains them couldn't be found, but when I moved
the program to the hard drive, it worked.

I'm guessing that you haven't given the linker a proper path to
your libraries. Check the commands or makefile that you use to
compile the program; post them here if you're stumped.

Post by Harry Potter
And 2. the fopen() function didn't work properly: fread() reported
zero bytes read, and the program crashed a few seconds later.
I really want OW's optimizations, as the compression code is too
slow. What functions do I use to replace the ANSI file functions?

Don't be too quick to replace fopen() and friends. You might not
be using them properly. Post a snippet of the code that calls
fopen() and fread() (preferably a single chunk that includes both)
so we can see whether anything looks strange.

Also, be sure to check return codes. If fopen() fails, it will
return NULL, and a subsequent fread() will crash. If either
function fails, check errno to find out why.

[Harry Potter]

Here's the erroneous code:
------------------------------
fIn=fopen (argv[1], "r");
vz.InEnd=fread(InBuffer, 16*1024*1024, 1, fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
------------------------------
How do I specify the drive and path of the needed libraries in OWC?

[Charlie Gibbs]

Post by Harry Potter
------------------------------
fIn=fopen (argv[1], "r");

if(fIn == NULL) {
printf("%s open error - errno = %d\n", argv[1], errno);
exit(1);
}

Post by Harry Potter
vz.InEnd=fread(InBuffer, 16*1024*1024, 1, fIn);

if(vz.InEnd <= 0) {
printf("%s read error - errno = %d\n", argv[1], errno);
exit(1);
}

You have the second and third arguments backwards. The second
argument is the size of each element you're reading, while the
third is the maximum number of elements you're reading. Your
fread() statement should look like this:

vz.InEnd=fread(InBuffer, 1, 16*1024*1024, fIn);

This tells fread() to read one byte at a time up to a
maximum of 16MB. (Don't worry about inefficiency, fread()
is pretty smart at optimizing these things.) The value
returned in vz.InEnd then becomes the number of bytes read.
Your original statement can only return two values: 1 if
you successfully read 16 megabytes of data, or zero if the
file is smaller than 16MB or you encountered an error.

Also, make sure that InBuffer is at least 16 megabytes long.
Otherwise, you'll overrun it and bad things will happen.

Do you really need to read the whole 16 megabytes into memory
at once? If your algorithm permits processing in smaller
chunks, you won't have to wait as long before it can start
chewing away at the data.

Post by Harry Potter
fclose (fIn); fIn=0;

fIn = NULL; /* Consistent with documentation */

Post by Harry Potter
printf ("Size: %d\n", vz.InEnd);
------------------------------
How do I specify the drive and path of the needed libraries in OWC?

I'm afraid I'm not familiar with OWC. Check your documentation.
Worst case, you can try to run the compiler and linker with no
command-line parameters, or with garbage. If you're lucky, they
might spit out command templates. Otherwise, try including a
parameter like -h or --help.

[Harry Potter]

Gibbs, I thank you for your response. I know the ftell() function returns the current pos. in a file, but how do I use it? I don't need a 16MB buffer, but the code is in its alpha stages, so I will keep it for now.

[Harry Potter]

I did what you said, but I'm getting the wrong file size. :( Following is the current code:
---------------------------
fIn=fopen (argv[1], "r");
fread(InBuffer, 1, (long) 16*1024*1024, fIn);
vz.InEnd=ftell (fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
---------------------------

[Harry Potter]

I found the problem: I was opening binary files in ASCII mode. In the second argument of fread(), I'm supposed to use "rb-".

[Charlie Gibbs]

Post by Harry Potter
I found the problem: I was opening binary files in ASCII mode.
In the second argument of fread(), I'm supposed to use "rb-".

Ah yes, I forgot to warn you about that. (Too much Linux
programming, I guess.) My rule of thumb is that fseek() and
ftell() are unusable unless a file is opened in binary mode.
(One exception: fseek(fp,0L,0) will get you back to the start
of a file regardless of which mode you're in.)

Now that you've cleaned up your fread(), how's it working?

BTW if you don't want to allocate 16MB of memory if you
might only use a fraction of it, use fseek(fp,0L,2) followed
by ftell(fp); this will give you the size of the file in
bytes (provided you're open in binary mode :-), and you
can pass this to malloc() to dynamically allocate a buffer
which, as Goldilocks says, is not too large, not too small,
but just right.

[Harry Potter]

The following is the current code, and it _still_ gives me 0 bytes:
-------------------------
fIn=fopen (argv[1], "rb");
fread(InBuffer, 1, 16*1024*1024, fIn);
vz.InEnd=ftell (fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
-------------------------

[Charlie Gibbs]

Post by Harry Potter
-------------------------
fIn=fopen (argv[1], "rb");
fread(InBuffer, 1, 16*1024*1024, fIn);
vz.InEnd=ftell (fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
-------------------------

I'm not sure what to say. I wrote a test program that I compiled
both under gcc on Linux and Borland C++ Builder 5.5 on Windows XP.
When run with the command

foo foo.c

it displays the following lines:

fread() returned 857
ftell() returned 857

857 is the size in bytes of the source module, foo.c.
Under Linux it displays 816 rather than 857 because
Linux newlines are LF rather than CRLF. None of the
error checks in the code were triggered.

Here's the complete program:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

int main(int argc, char **argv)
{
FILE *fp;
char *buf;
long templong;
int i;

if(argc != 2) {
printf("Usage: %s <filespec>\n", argv[0]);
exit(1);
}
fp = fopen(argv[1], "rb");
if(fp == NULL) {
printf("Can't open %s - errno = %d\n", argv[1], errno);
exit(1);
}
buf = malloc(16777216L);
if(buf == NULL) {
printf("malloc() failed - errno = %d\n", errno);
fclose(fp);
exit(1);
}
i = fread(buf, 1, 16777216L, fp);
if(i <= 0) {
printf("fread() returned %d - errno = %d\n", i, errno);
free(buf);
fclose(fp);
exit(1);
}
printf("fread() returned %d\n", i);
templong = ftell(fp);
printf("ftell() returned %ld\n", templong);
fclose(fp);
free(buf);
exit(0);
return(0);
}

[Harry Potter]

The following is the current code. It still has the same problem:
------------------------------------
fIn=fopen (argv[1], "rb");
fread(InBuffer, 1, 16*1024*1024, fIn);
vz.InEnd=ftell (fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
------------------------------------

[Charlie Gibbs]

Post by Harry Potter
------------------------------------
fIn=fopen (argv[1], "rb");
fread(InBuffer, 1, 16*1024*1024, fIn);
vz.InEnd=ftell (fIn);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);
------------------------------------

You could try determining the file size before actually reading
it. Plus, if you already know the file size, you can ask fread()
for exactly that number of bytes.

fIn=fopen (argv[1], "rb");
fseek(fIn, 0L, 2); /* Jump to end of file. */
vz.InEnd=ftell (fIn);
fseek(fIn, 0L, 0); /* Go back to start of file. */
if(fread(InBuffer, vz.InEnd, 1, fIn) <= 0)
printf("Read error %d\n", errno);
fclose (fIn); fIn=0;
printf ("Size: %d\n", vz.InEnd);

Note that I swapped the fread() arguments again. Now that you
know exactly how many bytes to read, you can ask fread() to
get them in a single chunk, which might be more efficient.

[Harry Potter]

Your code worked! Thank you! :D Converting to OW didn't help in speed, though. :(