NtReplyWaitReceivePortEx timeout problems

Don't you hate it when you post a question and than you find the answer mere
minutes later ?

Post by R.Wieser
So, my question is: does anyone know how the "timeout" argument to the
NtReplyWaitReceivePortEx function works ? Is it perhaps a record that
needs a special setup ? If so, what ? Anything else ?

It turns out it was a double problem.

1) Some further google-hunting showed that there is also an
"NtCreateWaitablePort" function. Although I could not find any description
the "Waitable" in its name suggest that would need it.[1]

2) A quick test with "NtCreateWaitablePort" and "NtWaitForSingleObject"
showed that the latter still failed the same way. However,
"WaitForSingleObject" did actually wait the set time. So, after
disassembling it it turns out that the timeout argument for
"NtWaitForSingleObject" is in 100 nSec increments, and needs to be the
*negative* of the desired value.

[1]Indeed, "NtCreatePort" with the correct timeout value for
"NtWaitForSingleObject" also still failed.

Doing the same for the "NtReplyWaitReceivePortEx" shows it works there too.

In short: Victory! :-)

Post by R.Wieser
Secondary question : what is the function of NtListenPort ?
NtReplyWaitReceivePort seems to do something quite similar ...

I could still use an answer to this one though ...

Regards,
Rudy Wieser

Apd

2020-12-06 03:02:33 UTC

Post by R.Wieser
Don't you hate it when you post a question and than you find the answer
mere minutes later ?

Have you seen the book "Windows NT/2000 Native API Reference" by Gary
Nebbett? I found a PDF of it years ago.

It turns out it was a double problem.
1) Some further google-hunting showed that there is also an
"NtCreateWaitablePort" function. Although I could not find any
description the "Waitable" in its name suggest that would need it.[1]
2) A quick test with "NtCreateWaitablePort" and "NtWaitForSingleObject"
showed that the latter still failed the same way. However,
"WaitForSingleObject" did actually wait the set time. So, after
disassembling it it turns out that the timeout argument for
"NtWaitForSingleObject" is in 100 nSec increments, and needs to be the
*negative* of the desired value.

Timeout
"Optionally points to a value that specifies the absolute or relative
time at which the wait is to be timed out.A negative value specifies
an interval relative to the current time.The value is expressed in
units of 100 nanoseconds.Absolute times track any changes in the
system time; relative times are not affected by system time changes.
If Timeout is a null pointer, the wait will not timeout".

Post by R.Wieser
[1]Indeed, "NtCreatePort" with the correct timeout value for
"NtWaitForSingleObject" also still failed.
Doing the same for the "NtReplyWaitReceivePortEx" shows it works there too.
In short: Victory! :-)

Post by R.Wieser
Secondary question : what is the function of NtListenPort ?
NtReplyWaitReceivePort seems to do something quite similar ...

I could still use an answer to this one though ...

Substitute Nt for Zw (most of the native functions are doubly nameed -
makes no difference from user mode).

ZwListenPort
Listens on a port for a connection request message.

NTSYSAPI
NTSTATUSNTAPI
ZwListenPort(
IN HANDLE PortHandle,
OUT PPORT_MESSAGE Message
);

PortHandle
A handle to a port object.The handle need not grant any specific
access.

Message
Points to a caller-allocated buffer or variable that receives the
connect message sent to the port.

Return Value
Returns STATUS_SUCCESS or an error status, such as
STATUS_INVALID_HANDLE.

Remarks
The message type of the received message is
LPC_CONNECTION_REQUEST.The message data is the connect data
specified in the call to ZwConnectPort.

ZwReplyWaitReceivePort
Optionally sends a reply message to a port and waits for a message.

NTSYSAPI
NTSTATUS
NTAPI
ZwReplyWaitReceivePort(
IN HANDLE PortHandle,
OUT PULONG PortIdentifier OPTIONAL,
IN PPORT_MESSAGE ReplyMessage OPTIONAL,
OUT PPORT_MESSAGE Message
);

PortHandle
A handle to either a port object or a waitable port object.The
handle need not grant any specific access.

PortIdentifier
Optionally points to a variable that receives a numeric identifier
associated with the port.

ReplyMessage
Optionally points to a caller-allocated buffer or variable that
specifies the reply mes- sage to send to the port.

Message
Points to a caller-allocated buffer or variable that receives the
message sent to the port.

Return Value
Returns STATUS_SUCCESS or an error status, such as
STATUS_INVALID_HANDLE or STATUS_REPLY_MESSAGE_MISMATCH.

R.Wieser

2020-12-06 06:51:05 UTC

Apd,

Post by Apd
Have you seen the book "Windows NT/2000 Native API Reference"
by Gary Nebbett? I found a PDF of it years ago.

Nope. And a google search just now shows its rather unavailable in /any/
form (pdf, text, other). Any chance you can share your copy of it ?

Post by Apd
Timeout
"Optionally points to a value that

...

I wish I could have found that info. But no. The best I got was this :
http://undocumented.ntinternals.net/helpcontents.html

Post by Apd
Substitute Nt for Zw (most of the native functions are doubly
nameed - makes no difference from user mode).

I noticed that when extracting function names from a DLL. Didn't think of
/remember searching for the "Zw" name though. :-\

Post by Apd
ZwListenPort

...

Post by Apd
Remarks
The message type of the received message is
LPC_CONNECTION_REQUEST.The message data is the
connect data specified in the call to ZwConnectPort.

Ah, so that is the whole difference: where NtReplyWaitReceivePort waits for
/any/ message, NtListenPort filters out a specific one. Good to know.
Thanks.

Regards,
Rudy Wieser

Apd

2020-12-06 12:16:59 UTC

Post by R.Wieser
Apd,

Post by Apd
Have you seen the book "Windows NT/2000 Native API Reference"
by Gary Nebbett? I found a PDF of it years ago.

Nope. And a google search just now shows its rather unavailable in /any/
form (pdf, text, other). Any chance you can share your copy of it ?

Sure:
http://s000.tinyupload.com/?file_id=11764490474007274119

The introduction, contents and index are missing.

Post by Apd
Timeout
"Optionally points to a value that

...
http://undocumented.ntinternals.net/helpcontents.html

Yes, I've seen that before. Nebbett's book is better.

Post by Apd
Substitute Nt for Zw (most of the native functions are doubly
nameed - makes no difference from user mode).

I noticed that when extracting function names from a DLL. Didn't think of
/remember searching for the "Zw" name though. :-\

It's a weird way of doing things and the difference is in the way kernel
code handles calls from user mode. I don't know why both types are
available from user mode.

Post by Apd
ZwListenPort

...

Post by Apd
Remarks
The message type of the received message is
LPC_CONNECTION_REQUEST.The message data is the
connect data specified in the call to ZwConnectPort.

Ah, so that is the whole difference: where NtReplyWaitReceivePort waits for
/any/ message, NtListenPort filters out a specific one. Good to know.
Thanks.

I guess so. I've never used these functions. Isn't the win32
WaitForSingleObject good enough?

R.Wieser

2020-12-06 13:27:59 UTC

Apd,

[Snip]
The introduction, contents and index are missing.

Lol. My first thought was "with even the contents gone, what, if anything,
will be left ?".

Luckily when I opened the PDF there was still enough to read there ...

[Snip]

Yes, I've seen that before. Nebbett's book is better.

I can imagine. Multiple Nt* functions in the link I posted have zero
information about what certain arguments should look like and are ment for,
let alone a description of any dependancies of the function itself (not that
MS itself fares much better in the latter case).

Didn't think of /remember searching for the "Zw" name
though. :-\

It's a weird way of doing things and the difference is in the way
kernel code handles calls from user mode.

The code itself determines in which mode it called its called and than
responds differently ? Thats ... interresting.

I don't know why both types are available from user mode.

If most of the code is the same with only the pre- and postamble different
(setting up for kernel or usermode call) I can see it as a good decision
(de-duplicating code).

That both the Nt* and Zw* names still exists is than probably a legacy thing
(where both origionally pointed to different routines).

I guess so. I've never used these functions. Isn't the
win32 WaitForSingleObject good enough?

:-) I only used that kernel32 function on a hunch to test.

The target was to get the NtReplyWaitReceivePortEx function to work (I'm
trying to write some local IPC), but thought it would be a good idea to
isolate the "wait for" functionality to make things easier to check. I
first tried the NtWaitForSingleObject, but after that one failed too I
wondered if maybe the "higher level" function in Kernel32 might be doing
something extra that might make it work. And it did.

Regards,
Rudy Wieser

Apd

2020-12-06 16:37:49 UTC

Post by Apd
The introduction, contents and index are missing.

Lol. My first thought was "with even the contents gone, what, if anything,
will be left ?".

Yep! Of course, I meant "table of contents".

Post by R.Wieser
Luckily when I opened the PDF there was still enough to read there ...

I hope you find it useful. There's plenty of info despite the missing
introduction.

Didn't think of /remember searching for the "Zw" name
though. :-\

It's a weird way of doing things and the difference is in the way
kernel code handles calls from user mode.

The code itself determines in which mode it called its called and than
responds differently ? Thats ... interresting.

<https://docs.microsoft.com/en-us/windows-hardware/drivers/kernel/using-nt-and-zw-versions-of-the-native-system-services-routines>

See also the link to "previous mode".

The ntoskrnl exports of the Zw routines go via the system service
table whereas the Nt ones go straight to the code that implements the
service. My understanding from other sources is that drivers should
use Zw to be safe but the article above suggests Nt. Who knows? I
don't write kernel code but I sometimes analyse it.

Post by Apd
I don't know why both types are available from user mode.

If most of the code is the same with only the pre- and postamble different
(setting up for kernel or usermode call) I can see it as a good decision
(de-duplicating code).
That both the Nt* and Zw* names still exists is than probably a legacy thing
(where both origionally pointed to different routines).

That would make sense but I don't think it's the case. Being in a user
module (ntdll), I don't see the need for different routines like that.
I wonder why the Zw names are needed at all in ntdll since a number of
them are exported from ntoskrnl.exe. Those prefixes didn't exist in
pre-NT versions of Windows.

Somewhere at Microsoft I read that user code should use Nt and that Zw
was not supported. Despite that, both varieties of a name point to the
same entry point in ntdll and all go through the system service table
via INT 2E for W2k and SYSENTER for XP.

Post by Apd
I guess so. I've never used these functions. Isn't the
win32 WaitForSingleObject good enough?

:-) I only used that kernel32 function on a hunch to test.
The target was to get the NtReplyWaitReceivePortEx function to work (I'm
trying to write some local IPC), but thought it would be a good idea to
isolate the "wait for" functionality to make things easier to check. I
first tried the NtWaitForSingleObject, but after that one failed too I
wondered if maybe the "higher level" function in Kernel32 might be doing
something extra that might make it work. And it did.

So you need to go down to the native ntdll level to get things to
work. Not sure I completely understand why the win32 API won't do what
you want, but Ok!

R.Wieser

2020-12-06 17:47:35 UTC

Apd,

Post by Apd
My understanding from other sources is that drivers
should use Zw

That is what I read too. Knowing that both call the same NTDLL function I
could only imagine that its either legacy, or that the name prefix is ment
as a "warning, you are writing kernel-mode stuff".

Post by Apd
Those prefixes didn't exist in pre-NT versions of Windows.

That sure makes mince-meat of my "legacy" idea.

There ofcourse is also the possibility that they wanted to make the way free
to, in the future, make those functions different. Though that is nothing
more than guesswork - for which I've not seen anything to support it.

Post by Apd
So you need to go down to the native ntdll level to get things
to work.

Not quite. At some point in my looking thru IPC methods read that there is
also a Local IPC (or LPC) method. From there I landed on a webpage showing
some example code, which used the Nt* functions.

Post by Apd
Not sure I completely understand why the win32 API won't
do what you want, but Ok!

While searching for information on LPC and subsequently on the different
NtCreatePort family of functions I have not seen any mentioning of the
existence of Win32 API counterparts. Than again, I did not really look
for it either ...

A bit of disclosure: My programming language of choice is Assembly. Which
might explain why I was (and am) not bothered by using low-level functions.
:-)

Regards,
Rudy Wieser

Apd

2020-12-07 16:21:14 UTC

Post by Apd
My understanding from other sources is that drivers
should use Zw

That is what I read too. Knowing that both call the same NTDLL
function

Drivers don't - they call Zw funcs exported from ntoskrnl.exe of which
there are fewer than in ntdll. They mainly use other kernel APIs.

Post by R.Wieser
I could only imagine that its either legacy, or that the name prefix
is ment as a "warning, you are writing kernel-mode stuff".

Warning: Here be dragons!

Post by Apd
So you need to go down to the native ntdll level to get things
to work.

Not quite. At some point in my looking thru IPC methods read that
there is also a Local IPC (or LPC) method. From there I landed on a
webpage showing some example code, which used the Nt* functions.

Having now read the start of the section dealing with ports, I can see
why it uses native funcs.

Post by R.Wieser
While searching for information on LPC and subsequently on the
different NtCreatePort family of functions I have not seen any
mentioning of the existence of Win32 API counterparts

I see the native API book says: "Port objects are not directly exposed
via the Win32 API".

Post by R.Wieser
A bit of disclosure: My programming language of choice is Assembly.
Which might explain why I was (and am) not bothered by using low-level
functions. :-)

I remember you saying you prefer ASM. It's one thing knowing that but
another finding out how the half-documented NT API works.

R.Wieser

2020-12-07 16:49:37 UTC

Apd,

Post by Apd
Drivers don't - they call Zw funcs exported from ntoskrnl.exe

Doesn't that end in the same difference ?

Post by R.Wieser
I could only imagine that its either legacy, or that the name
prefix is ment as a "warning, you are writing kernel-mode
stuff".

Warning: Here be dragons!

:-) Yep, that.

Post by Apd
Having now read the start of the section dealing with ports, I can
see why it uses native funcs.

Can you tell a bit about that ? I've got no idea about why that is, and
I'm intending two wrap code using those NT port functions up in an API DLL.
Is there perhaps a reason why I should not be doing that ?

Post by Apd
It's one thing knowing that but another finding out how the
half-documented NT API works.

Indeed. I'm currently trying to figure out which datagram functions need to
be used in which order to send and receive a stream of data.

And in that direction having contradicting documentation doesn't really help
either :Like the second argument to NtAcceptConnectPort. The PDF I got from
you named it "PortIdentifier". The info on the undocumented.ntinternals.net
website describes it as "AlternativeReceivePortHandle". Without a usage
description I've decided to just name it "UserValue".

Regards,
Rudy Wieser

Apd

2020-12-07 18:41:49 UTC

Post by R.Wieser
Apd,

Post by Apd
Drivers don't - they call Zw funcs exported from ntoskrnl.exe

Doesn't that end in the same difference ?

Um... Yes. What I don't undersand is the need for ntdll to export them
since user mode doesn't need them at all, and kernel mode gets them
from ntoskrnl.

Post by Apd
Having now read the start of the section dealing with ports, I can
see why it uses native funcs.

Can you tell a bit about that ? I've got no idea about why that is,
and I'm intending two wrap code using those NT port functions up in an
API DLL. Is there perhaps a reason why I should not be doing that ?

Sorry, all I know is what it says about there being no Win32
equivalents. How about using rpcrt4.dll? At least you can see how it
uses those funcs. There's also example usage in Appendix D.4.

Post by Apd
It's one thing knowing that but another finding out how the
half-documented NT API works.

Indeed. I'm currently trying to figure out which datagram functions need to
be used in which order to send and receive a stream of data.
And in that direction having contradicting documentation doesn't really help
either :Like the second argument to NtAcceptConnectPort. The PDF I got from
you named it "PortIdentifier". The info on the undocumented.ntinternals.net
website describes it as "AlternativeReceivePortHandle". Without a usage
description I've decided to just name it "UserValue".

Might also be worth checking how/if these native routines are
implemented in ReactOS.

R.Wieser

2020-12-07 21:00:18 UTC

Apd,

Post by Apd
What I don't undersand is the need for ntdll to export them
since user mode doesn't need them at all, and kernel mode
gets them from ntoskrnl.

Ehrmmm ... no, I can't think of such a reason either. On the other hand
.... Didn't you say that ntdll has a few more Nt* functions than ntoskrnl ?
Is that perhaps why they "shortcutted" the other functions into ntdll too ?

Post by Apd
Sorry, all I know is what it says about there being no
Win32 equivalents.

Ah. You ment that "I can see why it uses native funcs" as "because there
are no Win32 equivalents". I thought that you maybe had some further
insight into it.

Post by Apd
Might also be worth checking how/if these native routines
are implemented in ReactOS.

Might be worth anther look. Though I did look into the ReactOS code for the
timeout problem of the NtReplyWaitReceivePortEx function (as in my initial
question), and got nowhere fast. The ReactOS function did some "is the
argument actually readable" checking, but I could not see where it was
actually used for its purpose. A page-wide search for the argument name
turned up zilch.

Worse: At some point I even found some diff-ing removing the timeout
functionality from that function - presumably because of problems with its
implementation.

Regards,
Rudy Wieser

Apd

2020-12-08 14:39:27 UTC

Post by R.Wieser
Apd,

Post by Apd
What I don't undersand is the need for ntdll to export them
since user mode doesn't need them at all, and kernel mode
gets them from ntoskrnl.

To me it seems strange that the kernel needs any Nt or Zw funcs, after
all it has its own subsystems. For example, the IO subsystem has
IoCreateFile so why the need for ZwCreateFile? Perhaps some don't have
a place in those subsystems or it's convenient to have the strict
parameter verification done when user data is involved. This may
explain why it needs fewer of those user-mode interface functions,
which is how I see the Nt* routines.

R.Wieser

2020-12-08 08:03:10 UTC

Apd,

Post by Apd
How about using rpcrt4.dll?

Somehow I forgot to mention that I did take a look at the functions the DLL
exposed, and must say that those names do not really hint at what they are
for.

I see five groups and (some ungroupable ones at the begin), and cannot even
figure out which of those groups I should be starting with, or which
function of each group initialises ... whatever it is those groups do. :-|

I'll probably do a google for them though (can't stand not knowing what they
do :-) ). Even though their names indicate remote, not local IPC ...

Regards,
Rudy Wieser

Apd

2020-12-08 14:40:06 UTC

Post by R.Wieser
Apd,

Post by Apd
How about using rpcrt4.dll?

Somehow I forgot to mention that I did take a look at the functions the DLL
exposed, and must say that those names do not really hint at what they are
for.

That could be a problem. Otherwise, it might help if you could see how
they used the NT port functions. Don't forget the example in the API
book.

How about another book?

Undocumented Windows NT
Chapter 8 LPC
http://index-of.es/Programming/Programming%20-%20Undocumented%20Windows%20NT/part_II_8.pdf

All the other chapters can be found there, too.

R.Wieser

2020-12-08 18:49:49 UTC

Apd,

Don't forget the example in the API book.

The API book ? I take it you mean the PDF link you just provided ? The
first one did have some code, but no example that I could see.

How about another book?
Undocumented Windows NT
Chapter 8 LPC

Fun. It looks like thats same as the webpage I found containing the
example code I started with :

https://web.archive.org/web/20080612141354/http://www.windowsitlibrary.com/Content/356/08/toc.html

All the other chapters can be found there, too.

I don't know why, but pretty-much my first thought after downloading the PDF
was to see what would happen if I would remove the PDF filename part, to see
if I would drop into the folder and what would be there. :-)

Thanks.

Regards,
Rudy Wieser

Apd

2020-12-08 23:43:59 UTC

Post by R.Wieser
Apd,

Don't forget the example in the API book.

The API book ? I take it you mean the PDF link you just provided ?

Yes.

Post by R.Wieser
The first one did have some code, but no example that I could see.

The debug example 4 in the Appendix. Search the pdf for
ZwReplyWaitReceivePort.

Undocumented Windows NT
Chapter 8 LPC

Fun. It looks like thats same as the webpage I found containing the
https://web.archive.org/web/20080612141354/http://www.windowsitlibrary.com/Content/356/08/toc.html

Ah, you've seen it.

All the other chapters can be found there, too.

I don't know why, but pretty-much my first thought after downloading the
PDF was to see what would happen if I would remove the PDF filename
part, to see if I would drop into the folder and what would be there.
:-)

That's how I found it!

R.Wieser

2020-12-09 07:36:47 UTC

Apd,

Post by R.Wieser
The first one did have some code, but no example that I could see.

The debug example 4 in the Appendix. Search the pdf for
ZwReplyWaitReceivePort.

Ah yes. Thank you. I must have browsed it to fast to notice it :-\

I certainly can use everything there is. I'm having a blast of a time
trying to figure out what message types the different "request" and "reply"
functions send and wish to accept[1] (trying to find working client/server
combinations). I also got a fair share of STATUS_INVALID_PARAMETER
(0xC000000D) errors, until I figured out that the (optional) reply message
needed to have its Client and Server ID's filled in.

[1] I just tried to have the *server* close the connection, but in response
the client (using NtRequestWaitReplyPort) just threw an error
(STATUS_LPC_REPLY_LOST, 0xC0000253). I can understand why that that
happened, but as there does not seem to be an NtRequestWaitReceivePort
function available ...

Post by R.Wieser
was to see what would happen if I would remove the PDF filename part, to
see if I would drop into the folder and what would be there. :-)

That's how I found it!

I downloaded the lot for further reference. Did you notice that chaper 7 is
missing ? Not that I think I will ever need it (adding services to the
kernel) though.

Regards,
Rudy Wieser

Apd

2020-12-09 13:00:08 UTC

Post by R.Wieser
I certainly can use everything there is. I'm having a blast of a time
trying to figure out what message types the different "request" and "reply"
functions send and wish to accept[1] (trying to find working client/server
combinations). I also got a fair share of STATUS_INVALID_PARAMETER
(0xC000000D) errors, until I figured out that the (optional) reply message
needed to have its Client and Server ID's filled in.
[1] I just tried to have the *server* close the connection, but in response
the client (using NtRequestWaitReplyPort) just threw an error
(STATUS_LPC_REPLY_LOST, 0xC0000253). I can understand why that that
happened, but as there does not seem to be an NtRequestWaitReceivePort
function available ...

Maybe these:
ReplyWaitReceivePort ReplyWaitReceivePortEx

These two seem to be paired:
ReplyWaitReplyPort ReplyWaitReceivePort

I really don't know. Just throwing out some suggestions.

Post by R.Wieser
I downloaded the lot for further reference. Did you notice that chaper 7 is
missing ? Not that I think I will ever need it (adding services to the
kernel) though.

It's at the Wayback Machine web archive link you gave earlier if you
follow the book table of contents. I already have that short chapter
as a single html doc from when I was trying to understand how the
system service table was used.

R.Wieser

2020-12-09 18:28:00 UTC

Apd,

Post by Apd
ReplyWaitReceivePort ReplyWaitReceivePortEx

The problem is that the client needs to initiate the communication - which I
assume needs a "Request" function.

Than again, I'm still trying to figure out the pairings ... It might work
differently than I now assume.

So, thanks for the suggestion. I definitily must take a look at it.

Post by Apd
It's at the Wayback Machine web archive link you gave earlier if you
follow the book table of contents.

Sigh ... I did not even think of doing that. :-\

Regards,
Rudy Wieser

R.Wieser

2020-12-10 07:46:49 UTC

Apd,