fopen - open in exclusive read mode ?

[R.Wieser]

Hello all,

I've got a small program which rewrites the contents of a textfile. I'm
using "fopen" for both the source as well as the target file, reading lines
from the first and writing them to the second.

The problem is that when both have the same name the file is destroyed. :-((

To fix that I've been looking for an "exclusive read" open mode (so that
opening the targetfile would error-out). The thing is that Windows doesn't
seem to offer it :

https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen

It does offer an "x" modifier, but that one only works on writing.

I've also found a reference to "flock", but that doesn't seem to be part of
the CRTDLL.DLL on XPsp3 .

tl;dr:
How do I use "fopen" (or a similar crtdll function) to get an exclusive read
lock ?

Regards,
Rudy Wieser

[Charlie Gibbs]

Post by R.Wieser
Hello all,
I've got a small program which rewrites the contents of a textfile. I'm
using "fopen" for both the source as well as the target file, reading lines
from the first and writing them to the second.
The problem is that when both have the same name the file is destroyed. :-((
To fix that I've been looking for an "exclusive read" open mode (so that
opening the targetfile would error-out). The thing is that Windows doesn't
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen
It does offer an "x" modifier, but that one only works on writing.
I've also found a reference to "flock", but that doesn't seem to be part of
the CRTDLL.DLL on XPsp3 .
How do I use "fopen" (or a similar crtdll function) to get an exclusive read
lock ?

Your tl;dr restatement sounds like an example of the XY problem.

How about just checking whether the names of the source and
target files are the same? Yes, I know that such code could
be fooled, e.g. by calling them "myfile" and ".\myfile", so
you might need to make the code rather smart.

Better still (IMHO - it works for me) is to write the output
to a work file, then copy the work file to the target file
when you're done. Warning: do _not_ attempt to simply delete
the target file and rename the work file as the target file;
due to quirks in the Windows file system, this will sometimes
fail and you'll lose your data.

BTDTGTS (been there, done that, got the scars)

[R.Wieser]

Charlie,

Post by Charlie Gibbs
How about just checking whether the names of the
source and target files are the same?

Yeah, that passed my find too ...

Post by Charlie Gibbs
Yes I know that such code could be fooled, e.g. by
calling them "myfile" and ".\myfile"

.. including that (and how "PathAppend", "PathCombine" or "GetFullPathName"
could resolve "." and ".." parts), but than realised that that won't help if
the the source and target are not accessed from within the same program
(like when its started twice), and it does nothing for a hardlink with a
different name.

Just thought of another problem : Supplying one filename in Windows long,
and the other in DOS 8.3 format.

Its also a but-fuggely hackkish quite literal "work around" if you ask me -
although I've, with the absense of something better, already implemented it.
:-)

Hence my question, seeing if I can drop that work-around in favour of
something thats not as easy to fool.

Post by Charlie Gibbs
Warning: do _not_ attempt to simply delete the target file
and rename the work file as the target file;

:-) Rename the targetfile to temp, rename the workfile to target file,
delete the temp file - and only if no errors pop up anywhere in between.

Regards,
Rudy Wieser

[Charlie Gibbs]

Post by R.Wieser
Charlie,

Post by Charlie Gibbs
How about just checking whether the names of the
source and target files are the same?

Yeah, that passed my find too ...

<snip>

Post by R.Wieser

Post by Charlie Gibbs
Warning: do _not_ attempt to simply delete the target file
and rename the work file as the target file;

:-) Rename the targetfile to temp, rename the workfile to target file,
delete the temp file - and only if no errors pop up anywhere in between.

That's getting complicated, which is why I just copy the work file back.
In addition, it works even if the two file names come out the same by any
syntactical trick, in which case you effectively update the file in place.
In fact, I honed the technique developing routines to do just that.

Just to provide a bit of background, I discovered that the reason the
delete/rename technique sometimes fails is that when you tell Windows
to delete a file, it doesn't necessarily do it right away, but queues
the request and does it whenever it feels like getting around to it.
So your program thinks the file is gone, but it might not be yet.
If this happens, a subsequent rename might fail because the original
file is still there.

The odds of this happening aren't that great - well within Microsoft's
"sort of works, most of the time" reliability criteria. But I've written
programs that are running every day in about a thousand customer sites;
in such an environment, a failure rate of even 0.01% results in enough
anguished telephone calls to make the support people unhappy (in addition
to the customers themselves, of course).

This problem is unique to Windows; delete/rename is bombproof under Linux
(although I can think of some unlikely situations which might create a
race condition that will cause a failure under any OS).

[R.Wieser]

Charlie,

Post by Charlie Gibbs

Post by R.Wieser
:-) Rename the targetfile to temp, rename the workfile to target file,
delete the temp file - and only if no errors pop up anywhere in between.

That's getting complicated, which is why I just copy the work file back.

Hmm .. You might be right and I should revisit my above method (it
origionates from a time where copy actions where slow - and needed to be
written by yourself)

Post by Charlie Gibbs
Just to provide a bit of background,

[snip]

Thanks for the explanation. Certainly helps to remember your "don't"
suggestion.

Post by Charlie Gibbs
But I've written programs that are running every day in
about a thousand customer sites;

... and with an ammount like that those "special case" flaws will make their
appearance soner or later.

So, I've got two options: work around the lack of file locking by using a
method that doesn't need it, or drop the crtdll functions in favour of the
kernel32 ones (which do offer locking).

There is something I did not mention though : in my case not being able to
rewrite the sourcefile would have been an added plus (the sourcefile is a
kind of template).

Regards,
Rudy Wieser

[Charlie Gibbs]

Post by R.Wieser
There is something I did not mention though : in my case not being able to
rewrite the sourcefile would have been an added plus (the sourcefile is a
kind of template).

Could you mark the source file read-only?

[R.Wieser]

Charlie,

Post by Charlie Gibbs
Could you mark the source file read-only?

Yes I could. I also thought of that, but kept looking because it is "a kind
of" template. While the program must never attempt to change the template
itself, I would still like to be able to easily update the template if
called for.

By the way, JJ suggested using "_fsopen". I just did, and it allows for
and honors a read-and-write lock on the sourcefile - causing an attempt to
open the same file as the target to fail. In other words, it solves my
problem in, as far as I can tell, the best way possible.

Thanks for the suggestion though.

Regards,
Rudy Wieser

[JJ]

Post by R.Wieser
Hello all,
I've got a small program which rewrites the contents of a textfile. I'm
using "fopen" for both the source as well as the target file, reading lines
from the first and writing them to the second.
The problem is that when both have the same name the file is destroyed. :-((
To fix that I've been looking for an "exclusive read" open mode (so that
opening the targetfile would error-out). The thing is that Windows doesn't
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen
It does offer an "x" modifier, but that one only works on writing.
I've also found a reference to "flock", but that doesn't seem to be part of
the CRTDLL.DLL on XPsp3 .
How do I use "fopen" (or a similar crtdll function) to get an exclusive read
lock ?
Regards,
Rudy Wieser

Not possible using `fopen()`. Use `fsopen()` instead. `fopen()` is simply
not share aware.

`fopen()` uses `_SH_DENYNO` sharing flag no matter what. That flag
translates to Windows' `FILE_SHARE_READ` and `FILE_SHARE_WRITE`.

[JJ]

`fopen()` is simply not share aware.

Uh, that's worded correctly.
I meant it always share the file.

[R.Wieser]

JJ,

Post by JJ
Not possible using `fopen()`. Use `fsopen()` instead.

Thank you. I almost overread that "f*S*open" though ... :-|


And in the mean time I found something interresting (and not in a good way)
: I was trying to see how "_open" would work. As it turns out it happily
opens both the input as well as the output (same file) and (ofcourse)
trashes the file (checked by doing a "_read" before and after opening the
file in write mode.

The odd thing is that running the same program twice doesn't work: the
trashed file cannot be opened anymore. Worse still, a warm or cold boot did
not alter that. <whut ?>

Mind you, as long as I did not actually read from the file everything worked
OK (both filehandles ok).

Although I could not delete the file from the commandline, Windows file
explorer had no problem with removing it.

Used test program.(snipped debugging output for brevity)
- - - - - - - - - - - -
call _open,offset @@TXT_File,O_RDONLY
mov [@@hSrcFile],eax
and eax,eax
jl @@Testing9

lea ebx,[@@sBuffer]
call _read,[@@hSrcFile],ebx,MAX_PATH

call _open,offset @@TXT_File,O_CREAT or O_EXCL
mov [@@hTrgFile],eax
and eax,eax
jl @@Testing8

lea ebx,[@@sBuffer]
call _read,[@@hSrcFile],ebx,MAX_PATH

call _close,[@@hTrgFile]
@@Testing8:
call _close,[@@hSrcFile]
@@Testing9:
- - - - - - - - - - - -

Regards,
Rudy Wieser

[JJ]

Post by R.Wieser
The odd thing is that running the same program twice doesn't work: the
trashed file cannot be opened anymore. Worse still, a warm or cold boot did
not alter that. <whut ?>
Mind you, as long as I did not actually read from the file everything worked
OK (both filehandles ok).
Although I could not delete the file from the commandline, Windows file
explorer had no problem with removing it.

That is odd.

The fact that the problem persist after reboot suggest that there was (or
still is) file system inconsistency that caused file system glitches, and it
happens to affect that specific file. Try CHKDSK-ing the drive.

[R.Wieser]

JJ,

Post by JJ
That is odd.

:-) Which is why I mentioned it.

Post by JJ
The fact that the problem persist after reboot suggest that there
was (or still is) file system inconsistency that caused file system
glitches, and it happens to affect that specific file.

I don't think so. I checked a few times to make sure it wasn't just a
fluke, and its repeatable using different files (new created ones).

Post by JJ
Try CHKDSK-ing the drive.

Done. NTFS disk (aren't those self-repairing?), all three stages completed
and no mentioning of any problems.

Regards,
Rudy Wieser

[R.Wieser]

JJ,

As far as I can tell "_open" in combination with _O_CREAT simply sets the
read-only file-attribute on the file. I'm a bit miffeled to why anything
is mentioned in the MS docs to that function though.

I should have realised earlier that the RO flag could be a factor when the
"access denied" survived a cold boot.

Regards,
Rudy Wieser

[holyghost]

Post by JJ

Post by R.Wieser
Hello all,
I've got a small program which rewrites the contents of a textfile. I'm
using "fopen" for both the source as well as the target file, reading lines
from the first and writing them to the second.
The problem is that when both have the same name the file is destroyed. :-((
To fix that I've been looking for an "exclusive read" open mode (so that
opening the targetfile would error-out). The thing is that Windows doesn't
https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-wfopen
It does offer an "x" modifier, but that one only works on writing.
I've also found a reference to "flock", but that doesn't seem to be part of
the CRTDLL.DLL on XPsp3 .
How do I use "fopen" (or a similar crtdll function) to get an exclusive read
lock ?
Regards,
Rudy Wieser

Not possible using `fopen()`. Use `fsopen()` instead. `fopen()` is simply
not share aware.
`fopen()` uses `_SH_DENYNO` sharing flag no matter what. That flag
translates to Windows' `FILE_SHARE_READ` and `FILE_SHARE_WRITE`.

AFAIK you don't need a read lock for file access (see a database course.)

holly

[R.Wieser]

holyghost,

Post by holyghost
AFAIK you don't need a read lock for file access (see a database course.)

:-) A database locks the record when writing to it, so that when reading
there is no way that you will get half the old and half the new record.

But I suggest you read my problem specification first. Its not about the
read and write actions themselves.

Regards,
Rudy Wieser