Mailing List Archive

Updated InfiniBand/RDMA merge plans for 2.6.24
Since 2.6.23 still isn't out, and I've managed to reduce my patch
review backlog a bit, it's probably a good idea to give another update
about what I have queued for 2.6.24 already and what I hope to get to
before the merge window opens.

Core:

- My user_mad P_Key index support patch. Merged this, although I
still owe Sasha a patch to update libraries to use this.

- A fix to the user_mad 32-bit big-endian userspace 64/32 problem
with the method_mask when registering agents. Merged.

- Sean's QoS changes. Merged.

- Sean's IB CM MRA interface changes. I merged these -- what the
heck, if it breaks we can back them out.

ULPs:

- Pradeep's IPoIB CM support for devices that don't have SRQs. Sean
started reviewing but I didn't see any updated patches.

- Moni's IPoIB bonding support. Seems like we found a clean set of
changes, and these will go in via another (Jeff Garzik's?) tree.

- Rolf's IPoIB MGID scope changes. No review progress here.

- Eli and Michael's IPoIB stateless offload (checksum offload, LSO,
LRO, etc). Not much review progress here; I'll try to chip away at
the series and see what we can get into 2.6.24.

- Or's IPoIB/userspace multicast coexistence stuff. I think we've
converged on this; I'll merge this once a final version of the
patch appears.

HW specific:

- I already merged patches to enable MSI-X by default for mthca and
mlx4. I hope there aren't too many systems that get hosed if a
MSI-X interrupt is generated.

- Jack and Michael's mlx4 FMR support. Merged. I guess the fix for
running in Xen domU may need to wait for 2.6.25, but I'll see what
I can do.

- ehca patch queue. Merged everything I think.

- Steve's mthca router mode support. No one looked at it, seems like
it's at risk of missing the window.

- Arthur's mthca doorbell alignment fixes. I still need to check
various approaches; I'll definitely merge something for 2.6.24.

- Michael's mlx4 WQE shrinking patch. May miss the window and go for
2.6.25, I'll see if I can get to it.

Here are a few topics that I believe will not be ready in time for the
2.6.24 window and will need to wait for 2.6.25:

- Multiple CQ event vector support. I haven't seen any discussions
about how ULPs or userspace apps should decide which vector to use,
and hence no progress has been made since we deferred this during
the 2.6.23 merge window.

- XRC. Given the length of the backlog above and the fact that a
first draft of this code has not been posted yet, I don't see any
way that we could have something this major ready in time.

BOILERPLATE
===========

I keep patches in a git tree, available from

git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git

There are several branches of interest in this tree:

for-2.6.23 - changes queued for merging into the current kernel release
for-2.6.24 - changes queued for the next merge window
for-linus - changes I have asked Linus to pull upstream
for-mm - pulled by Andrew for inclusion in -mm

I frequently rewrite history and rebase my tree, so the best way to
track it is to keep a clone of Linus's tree around and then pull a
fresh copy of my tree with

git clone --reference /path/to/linus/tree \
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git

If you would like me to merge a patch, please send it to me as soon as
it is ready. Do NOT wait for the merge window to open; if your change
is not strictly a fix and you send it to me after the merge window
opens, then it will likely have to wait for the next merge window.
Please let me know if your patch is a fix that should go into the
current release or if it can wait for the next merge window; if it is
a fix, please describe the severity of the issue your are fixing, so I
can make a good judgement about which release it should go into.

Including a good changelog entry that explains what you are changing,
why you are changing it, and how your change accomplishes your goal
will greatly increase the chance of your patch being merged promptly.
Getting an independent review and a Reviewed-by: line also helps a lot.

The files Documentation/SubmittingPatches and Documentation/SubmitChecklist
in kernel source tree also have a lot of good advice that makes it
easier for me to handle your patches.

DETAILS
=======

Here is the complete list of patches I have in my for-2.6.24 branch:

Ali Ayoub (1):
IB/sa: Error handling thinko fix

Anton Blanchard (3):
IB/fmr_pool: Clean up some error messages in fmr_pool.c
IB/ehca: Make output clearer by removing some debug messages
IB/ehca: Export module parameters in sysfs

Dotan Barak (1):
mlx4_core: Use enum value GO_BIT_TIMEOUT_MSECS

Eli Cohen (2):
IPoIB: Fix typo to end statement with ';' instead of ','
IPoIB: Fix error path memory leak

Hoang-Nam Nguyen (4):
IB/ehca: Use remap_4k_pfn() to map firmware contexts to user space
IB/ehca: Fix large page HW cap defines
IB/ehca: Fix mem leak of firmware ctrlblock in ehca_create_srq()
IB/ehca: Adjust 64-bit alignment of create QP response for userspace

Jack Morgenstein (5):
IB/mlx4: Display misc device information under /sys/class/infiniband/
mlx4_core: Support ICM tables in coherent memory
mlx4_core: Write MTTs from CPU instead with of WRITE_MTT FW command
IB/mlx4: Implement FMRs
mlx4_core: Increase max number of QPs per multicast group to 56

Joachim Fenkes (11):
IB/ehca: Refactor hvcall tracing
IB/ehca: Print return codes as signed decimal integers
IB/ehca: ehca_gen_warn() should always print
IB/ehca: Add check for max #SGE to create_qp()
IB/ehca: Path migration support
IB/ehca: Serialize MR alloc and MR free hvCalls
IB/ehca: Replace get_paca()->paca_index by the more portable raw_smp_processor_id()
IB/ehca: Bump version number and change its format
IB/umem: Add hugetlb flag to struct ib_umem
IB/ehca: Only use MR large pages for hugetlb regions
IB/ehca: Return srq_attr->max_sge in ehca_query_srq()

Michael S. Tsirkin (2):
mlx4_core: Enable MSI-X by default
IB/mthca: Enable MSI-X by default

Peter Oruba (1):
IB/mthca: Use PCI-X/PCI-Express read control interfaces

Ralph Campbell (1):
IB/core: Fix handling of multicast response failures

Roland Dreier (14):
IPoIB: Make sure no receives are handled when stopping device
IB: find_first_zero_bit() takes unsigned pointer
mlx4_core: Don't free special QPs in QP number bitmap
IB/mlx4: Use __set_data_seg() in mlx4_ib_post_recv()
IB/ehca: Include <linux/mutex.h> from ehca_classes.h
IB/mlx4: Fix up SRQ limit_watermark endianness
IB/iser: Remove unnecessary includes
mlx4_core: Change capability decoding: SRC->XRC
IB/umad: Add P_Key index support
IB/umad: Fix bit ordering and 32-on-64 problems on big endian systems
IB/uverbs: Make ib_uverbs_release_event_file() static
mlx4_core: Reserve the correct number of MTT segments
mlx4_core: Fix meaning of dev->caps.reserved_mtts
IB/mthca: Increase max number of QPs per multicast group to 56

Satyam Sharma (1):
IB/ehca: Misc cpuinit section annotations and #ifdef cleanups

Sean Hefty (7):
IPoIB: Specify Traffic Class with path record queries for QoS support
IB/sa: Add new QoS fields to path record
RDMA/cma: Add ability to specify type of service
RDMA/ucma: Allow user space to set service type
IB/srp: Add QoS support through service ID
IB/cm: Modify interface to send MRAs in response to duplicate messages
RDMA/cma: Queue IB CM MRAs to avoid unnecessary remote retries

Stefan Roscher (2):
IB/ehca: Small QP userspace support
IB/ehca: Support more than 4k QPs for userspace and kernelspace

Steve Wise (2):
RDMA/cxgb3: Make the iw_cxgb3 module parameters writable
RDMA/cma: Use neigh_event_send() to start neighbour discovery
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
>
> ULPs:
>
> - Pradeep's IPoIB CM support for devices that don't have SRQs. Sean
> started reviewing but I didn't see any updated patches.
>
Roland, I submitted an updated patch incorporating some of Sean's comments within
a day or two. Rest of comments pertained to restructuring the code and adding
some additional module parameters.

This would require more discussions since some of these had been already discussed
previously. We had decided upon this code structure after a lot of discussions and
incorporating these would be undoing some of that.

We can discuss and revisit the comments after the merge.

Pradeep

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
No mention about the iwarp port space issue?

Here is the status of the current proposed patch:

- needs another round of changes based on Sean's feedback

- Arkady raised issues about the pain this puts on admins

- it forces services like nfs-rdma, which already separates the nfs-rdma
server by port number, to needlessly use a separate subnet for the rdma
service.

I'm at a loss as to how to proceed.

Any ideas?

Steve.


Roland Dreier wrote:
> Since 2.6.23 still isn't out, and I've managed to reduce my patch
> review backlog a bit, it's probably a good idea to give another update
> about what I have queued for 2.6.24 already and what I hope to get to
> before the merge window opens.
>
> Core:
>
> - My user_mad P_Key index support patch. Merged this, although I
> still owe Sasha a patch to update libraries to use this.
>
> - A fix to the user_mad 32-bit big-endian userspace 64/32 problem
> with the method_mask when registering agents. Merged.
>
> - Sean's QoS changes. Merged.
>
> - Sean's IB CM MRA interface changes. I merged these -- what the
> heck, if it breaks we can back them out.
>
> ULPs:
>
> - Pradeep's IPoIB CM support for devices that don't have SRQs. Sean
> started reviewing but I didn't see any updated patches.
>
> - Moni's IPoIB bonding support. Seems like we found a clean set of
> changes, and these will go in via another (Jeff Garzik's?) tree.
>
> - Rolf's IPoIB MGID scope changes. No review progress here.
>
> - Eli and Michael's IPoIB stateless offload (checksum offload, LSO,
> LRO, etc). Not much review progress here; I'll try to chip away at
> the series and see what we can get into 2.6.24.
>
> - Or's IPoIB/userspace multicast coexistence stuff. I think we've
> converged on this; I'll merge this once a final version of the
> patch appears.
>
> HW specific:
>
> - I already merged patches to enable MSI-X by default for mthca and
> mlx4. I hope there aren't too many systems that get hosed if a
> MSI-X interrupt is generated.
>
> - Jack and Michael's mlx4 FMR support. Merged. I guess the fix for
> running in Xen domU may need to wait for 2.6.25, but I'll see what
> I can do.
>
> - ehca patch queue. Merged everything I think.
>
> - Steve's mthca router mode support. No one looked at it, seems like
> it's at risk of missing the window.
>
> - Arthur's mthca doorbell alignment fixes. I still need to check
> various approaches; I'll definitely merge something for 2.6.24.
>
> - Michael's mlx4 WQE shrinking patch. May miss the window and go for
> 2.6.25, I'll see if I can get to it.
>
> Here are a few topics that I believe will not be ready in time for the
> 2.6.24 window and will need to wait for 2.6.25:
>
> - Multiple CQ event vector support. I haven't seen any discussions
> about how ULPs or userspace apps should decide which vector to use,
> and hence no progress has been made since we deferred this during
> the 2.6.23 merge window.
>
> - XRC. Given the length of the backlog above and the fact that a
> first draft of this code has not been posted yet, I don't see any
> way that we could have something this major ready in time.
>
> BOILERPLATE
> ===========
>
> I keep patches in a git tree, available from
>
> git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git
>
> There are several branches of interest in this tree:
>
> for-2.6.23 - changes queued for merging into the current kernel release
> for-2.6.24 - changes queued for the next merge window
> for-linus - changes I have asked Linus to pull upstream
> for-mm - pulled by Andrew for inclusion in -mm
>
> I frequently rewrite history and rebase my tree, so the best way to
> track it is to keep a clone of Linus's tree around and then pull a
> fresh copy of my tree with
>
> git clone --reference /path/to/linus/tree \
> git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband.git
>
> If you would like me to merge a patch, please send it to me as soon as
> it is ready. Do NOT wait for the merge window to open; if your change
> is not strictly a fix and you send it to me after the merge window
> opens, then it will likely have to wait for the next merge window.
> Please let me know if your patch is a fix that should go into the
> current release or if it can wait for the next merge window; if it is
> a fix, please describe the severity of the issue your are fixing, so I
> can make a good judgement about which release it should go into.
>
> Including a good changelog entry that explains what you are changing,
> why you are changing it, and how your change accomplishes your goal
> will greatly increase the chance of your patch being merged promptly.
> Getting an independent review and a Reviewed-by: line also helps a lot.
>
> The files Documentation/SubmittingPatches and Documentation/SubmitChecklist
> in kernel source tree also have a lot of good advice that makes it
> easier for me to handle your patches.
>
> DETAILS
> =======
>
> Here is the complete list of patches I have in my for-2.6.24 branch:
>
> Ali Ayoub (1):
> IB/sa: Error handling thinko fix
>
> Anton Blanchard (3):
> IB/fmr_pool: Clean up some error messages in fmr_pool.c
> IB/ehca: Make output clearer by removing some debug messages
> IB/ehca: Export module parameters in sysfs
>
> Dotan Barak (1):
> mlx4_core: Use enum value GO_BIT_TIMEOUT_MSECS
>
> Eli Cohen (2):
> IPoIB: Fix typo to end statement with ';' instead of ','
> IPoIB: Fix error path memory leak
>
> Hoang-Nam Nguyen (4):
> IB/ehca: Use remap_4k_pfn() to map firmware contexts to user space
> IB/ehca: Fix large page HW cap defines
> IB/ehca: Fix mem leak of firmware ctrlblock in ehca_create_srq()
> IB/ehca: Adjust 64-bit alignment of create QP response for userspace
>
> Jack Morgenstein (5):
> IB/mlx4: Display misc device information under /sys/class/infiniband/
> mlx4_core: Support ICM tables in coherent memory
> mlx4_core: Write MTTs from CPU instead with of WRITE_MTT FW command
> IB/mlx4: Implement FMRs
> mlx4_core: Increase max number of QPs per multicast group to 56
>
> Joachim Fenkes (11):
> IB/ehca: Refactor hvcall tracing
> IB/ehca: Print return codes as signed decimal integers
> IB/ehca: ehca_gen_warn() should always print
> IB/ehca: Add check for max #SGE to create_qp()
> IB/ehca: Path migration support
> IB/ehca: Serialize MR alloc and MR free hvCalls
> IB/ehca: Replace get_paca()->paca_index by the more portable raw_smp_processor_id()
> IB/ehca: Bump version number and change its format
> IB/umem: Add hugetlb flag to struct ib_umem
> IB/ehca: Only use MR large pages for hugetlb regions
> IB/ehca: Return srq_attr->max_sge in ehca_query_srq()
>
> Michael S. Tsirkin (2):
> mlx4_core: Enable MSI-X by default
> IB/mthca: Enable MSI-X by default
>
> Peter Oruba (1):
> IB/mthca: Use PCI-X/PCI-Express read control interfaces
>
> Ralph Campbell (1):
> IB/core: Fix handling of multicast response failures
>
> Roland Dreier (14):
> IPoIB: Make sure no receives are handled when stopping device
> IB: find_first_zero_bit() takes unsigned pointer
> mlx4_core: Don't free special QPs in QP number bitmap
> IB/mlx4: Use __set_data_seg() in mlx4_ib_post_recv()
> IB/ehca: Include <linux/mutex.h> from ehca_classes.h
> IB/mlx4: Fix up SRQ limit_watermark endianness
> IB/iser: Remove unnecessary includes
> mlx4_core: Change capability decoding: SRC->XRC
> IB/umad: Add P_Key index support
> IB/umad: Fix bit ordering and 32-on-64 problems on big endian systems
> IB/uverbs: Make ib_uverbs_release_event_file() static
> mlx4_core: Reserve the correct number of MTT segments
> mlx4_core: Fix meaning of dev->caps.reserved_mtts
> IB/mthca: Increase max number of QPs per multicast group to 56
>
> Satyam Sharma (1):
> IB/ehca: Misc cpuinit section annotations and #ifdef cleanups
>
> Sean Hefty (7):
> IPoIB: Specify Traffic Class with path record queries for QoS support
> IB/sa: Add new QoS fields to path record
> RDMA/cma: Add ability to specify type of service
> RDMA/ucma: Allow user space to set service type
> IB/srp: Add QoS support through service ID
> IB/cm: Modify interface to send MRAs in response to duplicate messages
> RDMA/cma: Queue IB CM MRAs to avoid unnecessary remote retries
>
> Stefan Roscher (2):
> IB/ehca: Small QP userspace support
> IB/ehca: Support more than 4k QPs for userspace and kernelspace
>
> Steve Wise (2):
> RDMA/cxgb3: Make the iw_cxgb3 module parameters writable
> RDMA/cma: Use neigh_event_send() to start neighbour discovery
> _______________________________________________
> general mailing list
> general@lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
On Saturday 06 October 2007 01:18, Roland Dreier wrote:
>  - XRC.  Given the length of the backlog above and the fact that a
>    first draft of this code has not been posted yet, I don't see any
>    way that we could have something this major ready in time.
>
I posted the first draft patch set to the OpenFabrics list on September 18:

[ofa-general] [PATCH 0 of 5] XRC implementation patches (libibverbs, libmlx4, core, mlx4) Jack Morgenstein
[ofa-general] [PATCH 1 of 5] libibverbs: XRC implementation Jack Morgenstein
[ofa-general] [PATCH 2 of 5] libmlx4: XRC implementation Jack Morgenstein
[ofa-general] [PATCH 3 of 5] core: XRC implementation for fd = -1 when opening an xrc domain Jack Morgenstein
[ofa-general] [PATCH 4 of 5] core: XRC implementation -- add support for working with file descriptors Jack Morgenstein
[ofa-general] [PATCH 5 of 5] mlx4: XRC implementation Jack Morgenstein

The above patch set implements XRC for userspace verbs layer and below.
The Kernel-space verbs implementation (1 more patch) is finished,
but as yet untested.

- Jack
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
> >  - XRC.  Given the length of the backlog above and the fact that a
> >    first draft of this code has not been posted yet, I don't see any
> >    way that we could have something this major ready in time.
> >
> I posted the first draft patch set to the OpenFabrics list on September 18:

Sorry, I didn't update that text. But my backlog is still too big to
get XRC into 2.6.24.

- R.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
> No mention about the iwarp port space issue?

I don't think we're at a stage where I'm prepared to merge something--
we all agree the latest patch has serious drawbacks, and it commits us
to a suboptimal interface that is userspace-visible.

> I'm at a loss as to how to proceed.

Could we try to do some cleanups to the net core to make the alias
stuff less painful? eg is there any sane way to make it possible for
a device that creates 'eth0' to also create an 'iw0' alias without an
assigning an address?

- R.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
Roland Dreier wrote:
> > No mention about the iwarp port space issue?
>
> I don't think we're at a stage where I'm prepared to merge something--
> we all agree the latest patch has serious drawbacks, and it commits us
> to a suboptimal interface that is userspace-visible.
>

Fair enough.

> > I'm at a loss as to how to proceed.
>
> Could we try to do some cleanups to the net core to make the alias
> stuff less painful? eg is there any sane way to make it possible for
> a device that creates 'eth0' to also create an 'iw0' alias without an
> assigning an address?
>

Well, "alias" interfaces really don't exist. ethX:iw is really just
adding a address record (struct in_ifaddr) to ethX. So in the current
core design, adding an alias without an address is really adding the
alias with address 0.0.0.0. And I think the core net code assumes if an
in_ifaddr struct exists for a device, then its IP address is indeed valid.

So I think the changes wouldn't be small to enhance the design to add a
concept of an alias interface.

I'll look into this more though.

Steve.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
> Roland, I submitted an updated patch incorporating some of Sean's comments within
> a day or two. Rest of comments pertained to restructuring the code and adding
> some additional module parameters.
>
> This would require more discussions since some of these had been already discussed
> previously. We had decided upon this code structure after a lot of discussions and
> incorporating these would be undoing some of that.

Can you give a link to your current final version of the patch?

Sean, what's your opinion of where we stand?

Since module parameters create a userspace-visible interface that we
are stuck with for a long time, we definitely have to get at least
that much right before merging.

- R.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
> Can you give a link to your current final version of the patch?
>
> Sean, what's your opinion of where we stand?

Let me look back over the last version that was sent and reply back
later today or tomorrow. Several of my initial comments were on code
structure.

> Since module parameters create a userspace-visible interface that we
> are stuck with for a long time, we definitely have to get at least
> that much right before merging.

I was taking a slightly different view of the design. It would be nice
to agree on whether SRQ should be separated from the QP type before
merging upstream, even if the implementation doesn't immediately support
all available options.

- Sean
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [ofa-general] Updated InfiniBand/RDMA merge plans for 2.6.24 [ In reply to ]
Roland Dreier wrote:
> > Roland, I submitted an updated patch incorporating some of Sean's comments within
> > a day or two. Rest of comments pertained to restructuring the code and adding
> > some additional module parameters.
> >
> > This would require more discussions since some of these had been already discussed
> > previously. We had decided upon this code structure after a lot of discussions and
> > incorporating these would be undoing some of that.
>
> Can you give a link to your current final version of the patch?
>
Roland,

This is the link to the last one that I submitted on 09/18.

http://lists.openfabrics.org/pipermail/general/2007-September/040917.html

Pradeep

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/