Skip to content

Conversation

@ChanningHe
Copy link

Problem

When RDMA NICs (e.g., enp8s14, enp8s20) are bridge members, rdma -j link returns the physical interface names, but IP addresses are configured on the bridge interface (e.g., br0). The current code only queries the physical interface names, resulting in no available addresses in the dropdown.

Note: RDMA functionality works correctly when NICs are bridged - the RDMA stack operates below the bridge layer.
bridge-rdma

About the relationships=False change

I noticed the original code (line 366) accesses i['alias_interface'].get('int_interface') when querying network aliases:

for i in await self.middleware.call('datastore.query', 'network.alias', filters):
    if i['alias_interface'].get('int_interface') in rdma_netdevs:
        choices[i['alias_vip']] = f'{i["alias_address"]}/{i["alias_address_b"]}'

This relies on the default relationships=True behavior to automatically join the related network_interfaces table. While this probably works fine in most cases, I encountered a KeyError: 'int_interface' when testing in my environment (specifically when enabling RDMA or configuring ports).

I'm not entirely familiar with how the datastore relationship joins are structured throughout the codebase, but it seems the joined data structure might not always include the expected fields, or there could be edge cases where the join doesn't behave as expected.

My approach was to:

  1. First query the interface IDs for RDMA-related interfaces
  2. Then query aliases and filter by alias_interface_id (which is always present as it's the actual foreign key column)

This way we're working with explicit fields that are guaranteed to exist in the database schema rather than depending on how the ORM constructs the joined result.

However, I'm open to suggestions if there's a better or more idiomatic way to handle this in the middlewared codebase. If the original approach should work and I'm hitting an edge case, I'm happy to investigate further or adjust the implementation.

Testing

Tested with Mellanox ConnectX NICs as bridge members:

  • ✅ Address dropdown now shows IPs configured on bridge
  • ✅ NVMeoF over RDMA working
  • ✅ Non-bridged RDMA configs work as before
image

Copy link
Contributor

@themylogin themylogin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve on the database usage logic, it is correct and efficient. Let's wait for @bmeagherix approval on the business logic.

@themylogin themylogin requested a review from bmeagherix October 31, 2025 09:40
@yocalebo
Copy link
Contributor

yocalebo commented Nov 5, 2025

May I ask what is the use-case for this?

@ChanningHe
Copy link
Author

ChanningHe commented Nov 6, 2025

May I ask what is the use-case for this?

Some setups need to bridge the NIC (for VM or LXC) while TrueNAS host also running RDMA as a storage node for other machines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants