Message Interceptors #13760
Merged
Message Interceptors #13760
+473
−163
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ansd
force-pushed
the
message-interceptors
branch
from
0a167d3 to
1565eab
Compare
gomoripeti
reviewed
Apr 16, 2025
ansd
force-pushed
the
message-interceptors
branch
from
40a1659 to
12bf6ec
Compare
gomoripeti
reviewed
Apr 16, 2025
ansd
force-pushed
the
message-interceptors
branch
from
12bf6ec to
fe0401a
Compare
ansd
force-pushed
the
message-interceptors
branch
from
eb5ec2a to
d0b528d
Compare
ansd
added a commit
to rabbitmq/rabbitmq-website
that referenced
this pull request
Apr 18, 2025
ansd
added a commit
to rabbitmq/rabbitmq-website
that referenced
this pull request
Apr 22, 2025
This commit enables users to provide custom message interceptor modules, i.e. modules to process incoming and outgoing messages. The `rabbit_message_interceptor` behaviour defines a `intercept/4` callback, for those modules to implement. Co-authored-by: Péter Gömöri <[email protected]>
1. Force the config for timestamp and routing node message interceptors to be configured with the overwrite boolean() to avoid defining multiple default values throughout the code. 2. Add type specs 3. Extend existing test case for new MQTT client ID interceptor 4. routing node and timestamp should only set the annotation for incoming_message_interceptors group 5. Fix `rabbitmq.conf`. Prior to this commit there were several issue: a.) Setting the right configuration was too user unfriendly, e.g. the user has to set ``` message_interceptor.incoming.rabbit_mqtt_message_interceptor_client_id.annotation_key = x-opt-mqtt-client-id ``` just to enable the MQTT message interceptor. b.) The code that parses was too difficult to understand c.) MQTT plugin was setting the env for app rabbit, which is an anti-pattern d.) disabling a plugin (e.g. MQTT), left its message interceptors still in place This is now all fixed, the user sets the rabbitmq.conf as follows: ``` message_interceptors.incoming.set_header_timestamp.overwrite = true message_interceptors.incoming.set_header_routing_node.overwrite = false mqtt.message_interceptors.incoming.set_client_id_annotation.enabled = true ``` Note that the first two lines use the same format as for RabbitMQ 4.0 for backwards compatiblity. The last line (MQTT) follows a similar pattern.
It's a tradeoff between building the map for each incoming and outgoing message (now that there are also outgoing interceptors) vs increased memory usage for the MQTT proc state. Connecting with MQTT 5.0 and client ID "xxxxxxxx", the number of words are 201 before this commit vs 235 after this commit as determined by: ``` S = sys:get_state(MQTTConnectionPid), erts_debug:size(S). ``` Therefore, this commit requires 34 word * 8 bytes = 272 bytes more per MQTT connection, that is 272 MB more for 1,000,000 MQTT connections.
[skip ci]
Intercept outgoing message just before conversion to target protocol as this will give most flexibility to 3rd party plugins.
ansd
force-pushed
the
message-interceptors
branch
from
80b0db8 to
69c2855
Compare
kjnilsson
approved these changes
Apr 23, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR builds on top of #13641 contributed by @LoisSotoLopez and @gomoripeti.
This PR allows writing 3rd party plugins to intercept both incoming and outgoing messages for AMQP 1.0, AMQP 0.9.1, and MQTT.
This PR also ships two new optional built-in interceptors:
Solves #10050
Solves #10051
Docs PR including exact description:
rabbitmq/rabbitmq-website#2246
Docs are live:
https://www.rabbitmq.com/docs/next/message-interceptors