x-dotnet-pub-seq-no header encoding

[zlepper]

We recently updated to v7 of the library. However we are now encountering a strange issue, I think, related to the x-dotnet-pub-seq-no header.
We publish messages using rabbitmq-dotnet-client. Some of those messages are picked up by amqprs. This has started failing since the upgrade to v7, where amqprs will fail with an error like this:

socket will be closed due to failure of reading frame, cause: framing error: serde error: len = 8, content = [00, 00, 00, 00, 00, 00, 03, 8A]

By troubleshooting using wireshark i have arrived at the conclusion that this is when it is processing the x-dotnet-pub-seq-no header:

It seems like even Wireshark can't process it as a valid string. Looking at the code in the dotnet client here:

rabbitmq-dotnet-client/projects/RabbitMQ.Client/Impl/Channel.BasicPublish.cs

Lines 220 to 222 in 85fd42f

	byte[] publishSequenceNumberBytes = new byte[8];
	NetworkOrderSerializer.WriteUInt64(ref publishSequenceNumberBytes.GetStart(), publishSequenceNumber);
	headers[Constants.PublishSequenceNumberHeader] = publishSequenceNumberBytes;

it seems like the dotnet client is actually writing out a uint64, not a string, so maybe the actual header is encoded wrongly?

I have tried to read my way thorugh the amqp specification, however I don't see anything that points at the dotnet client, is going something wrong specifically. However considering that the problem is new with v7, and that even wireshark cannot process the header correctly, I thought to start the discussion here first, before I go to the amqprs library to make my case.

The raw message dump from wireshark can be found here: https://gist.github.com/zlepper/bfbb489acfc2ef5ce95ef0f6e0d9e3cb

[lukebakken]

it seems like the dotnet client is actually writing out a uint64

Yes, that is the intent. This number is used to correlate a basic.publish with the corresponding basic.ack from the server (publisher confirmation).

This is where that header's ulong value is encoded:

https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/main/projects/RabbitMQ.Client/Impl/Channel.BasicPublish.cs#L220-L222

Notice that it is converted into bytes in this section of the code, which means it'll be encoded here:

https://github.com/rabbitmq/rabbitmq-dotnet-client/blob/main/projects/RabbitMQ.Client/Impl/WireFormatting.Write.cs#L144-L146

Hm, so it's encoding using S which the spec says is a long string. I made the assumption that x- headers would be ignored by client libraries, but that doesn't seem to be the case.

OK, now I remember why I'm encoding it this way - ulong (unsigned 64-bit int) is not a supported type in AMQP 0.9.1. Since that's the type we use for publisher confirmation identifiers in this client, we encode it as a byte array and use an x-dotnet- header to indicate its use by this client library.

So, the alternatives are:

• Use long instead of ulong for publish sequence number.
• Convert to ASCII and encode as an actual string.

Probably best to change it to a long.

cc @danielmarbach for his thoughts

I see you've already asked here: gftea/amqprs#158

[michaelklishin]

@lukebakken the Channel.BasicPublish code linked to uses an unsigned 64-bit integer, which is not uint. I'm just pointing out the discrepancy in the discussion. The Rust library should be able to deserialize both 32 and 64-bit uints in any case.

[lukebakken]

Oh wait, good point. I misunderstood the code. I'll edit my comment above.

[michaelklishin]

@zlepper amqprs must handle this header like any other one, that is, deserialize it and put it into the headers map (or however the headers are represented there). Headers are entirely optional (and so are most message properties), this client has every right to use an x- header for its own needs.

[michaelklishin]

In every discussion of AMQP 0-9-1 data types, headers and tables (it's a data type in AMQP 0-9-1) in general we must mention this AMQP 0-9-1 errata document.

[lukebakken]

@zlepper - #1806

Thanks for bringing this issue up.

[lukebakken]

Some more context: gftea/amqprs#158 (comment)