Mailing List Archive: KEXEC: allocate crash note buffers at boot time v5

KEXEC: allocate crash note buffers at boot time v5

Dec 2, 2011, 7:19 AM

Post #1 of 7 (192 views)

Here is v5.

I have tested quite carefully the impact of changing the error
conditions in kexec_get_cpu(). dom0 is happy to accept ERANGE as well
as EINVAL, and ERANGE is a more sensible response. Moreover, this new
scheme (ignoring for a moment ENOMEM) allows for all nr_cpu_ids to give
note ranges to dom0 whether they are up or not at the moment. It is
better to have more notes provided with some of them potentially not
filled in, than to miss a crashed CPU register state because it was
offline when the crash kernel was loaded.

All this testing has been done via backport to 4.1.2. Minimal changes
were necessary, such as the lack of xzalloc_array() and NR_CPUS instead
of nr_cpu_ids, but it was otherwise functionally identical.

I am not sure whether to recomend this for backport into 4.1-testing or
not. It does make the crashkernel more likely to get crash notes,
although in reality, it was only special circumstances which caused
problems. If you consider it worth backporting, I can provide my
already-backported patch.

--
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: KEXEC: allocate crash note buffers at boot time v5 [ In reply to ]

JBeulich at suse

Dec 2, 2011, 8:04 AM

Post #2 of 7 (191 views)

Permalink

(Andrew, my previous reply dropped all Cc-s, but I'm re-sending not only
because of that - I also adjusted the first part of the response.)

>>> On 02.12.11 at 16:19, Andrew Cooper <andrew.cooper3@citrix.com> wrote:

I just had another look at the Dom0 side of things, and I fail to see why
you think moving this out of per-CPU data is necessary: All Dom0 does
with the provided info is set up the resource tree. The data doesn't get
stored for any post-boot use. What am I overlooking?

>+ /* In the case of still not having enough memory to allocate buffer room,
>+ * returning a range of 0,0 is still valid. */
>+ range->start = __pa((unsigned long)crash_notes[nr].start);
>+ range->size = crash_notes[nr].size;

Comment and implementation don't match - __pa(NULL) is not zero
(and the cast to 'unsigned long' is pointless afaict).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: KEXEC: allocate crash note buffers at boot time v [ In reply to ]

andrew.cooper3 at citrix

Dec 2, 2011, 8:10 AM

Post #3 of 7 (191 views)

Permalink

On 02/12/11 16:04, Jan Beulich wrote:
> (Andrew, my previous reply dropped all Cc-s, but I'm re-sending not only
> because of that - I also adjusted the first part of the response.)
>
>>>> On 02.12.11 at 16:19, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> I just had another look at the Dom0 side of things, and I fail to see why
> you think moving this out of per-CPU data is necessary: All Dom0 does
> with the provided info is set up the resource tree. The data doesn't get
> stored for any post-boot use. What am I overlooking?

(for the benifit of the list)

/sbin/kexec opens /proc/iomem and parses the strings looking for "Crash
Note" and pulls the ranges out that way. (It also assumes that the
lowest crash note is pcpu0 and so on, which is also something I will
work around in later notes changes).

>> + /* In the case of still not having enough memory to allocate buffer room,
>> + * returning a range of 0,0 is still valid. */
>> + range->start = __pa((unsigned long)crash_notes[nr].start);
>> + range->size = crash_notes[nr].size;
> Comment and implementation don't match - __pa(NULL) is not zero
> (and the cast to 'unsigned long' is pointless afaict).
> Jan

Very silly mistake on my behalf. v6 is attached with that fixed.

--
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

Re: KEXEC: allocate crash note buffers at boot time v5 [ In reply to ]

JBeulich at suse

Dec 2, 2011, 8:11 AM

Post #4 of 7 (188 views)

Permalink

>>> On 02.12.11 at 16:59, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 02/12/11 15:43, Jan Beulich wrote:
>> I just had another look at the Dom0 side of things, and I fail to see why
>> you think boot time allocation is necessary: All Dom0 does with the
>> provided info is set up the resource tree. The data doesn't get stored
>> for any post-boot use. What am I overlooking?
>
> /sbin/kexec opens /proc/iomem and looks for "Crash note" and interprets
> the range values. This is how it grabs the locations to pack into its
> magic binary package.

So how does the hotplug scenario then get handled on native? I can't
imagine they expect things to remain stable across CPU unplug and
re-activation.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: KEXEC: allocate crash note buffers at boot time v5 [ In reply to ]

andrew.cooper3 at citrix

Dec 2, 2011, 8:27 AM

Post #5 of 7 (192 views)

Permalink

On 02/12/11 16:11, Jan Beulich wrote:
>>>> On 02.12.11 at 16:59, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> On 02/12/11 15:43, Jan Beulich wrote:
>>> I just had another look at the Dom0 side of things, and I fail to see why
>>> you think boot time allocation is necessary: All Dom0 does with the
>>> provided info is set up the resource tree. The data doesn't get stored
>>> for any post-boot use. What am I overlooking?
>> /sbin/kexec opens /proc/iomem and looks for "Crash note" and interprets
>> the range values. This is how it grabs the locations to pack into its
>> magic binary package.
> So how does the hotplug scenario then get handled on native? I can't
> imagine they expect things to remain stable across CPU unplug and
> re-activation.
>
> Jan

I am not how (or even if) the hotplug condition is handled on native. I
guess it depends on what is put into the resource tree on boot. With my
patch, Xen will give crash areas for all pcpus up to nr_cpu_ids, which
covers all the cases. The worst that will happen is that some crash
notes do not get written if certain cpus are offline at the time of a crash.

--
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: KEXEC: allocate crash note buffers at boot time v5 [ In reply to ]

JBeulich at suse

Dec 2, 2011, 8:38 AM

Post #6 of 7 (188 views)

Permalink

>>> On 02.12.11 at 17:27, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> On 02/12/11 16:11, Jan Beulich wrote:
>>>>> On 02.12.11 at 16:59, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>>> On 02/12/11 15:43, Jan Beulich wrote:
>>>> I just had another look at the Dom0 side of things, and I fail to see why
>>>> you think boot time allocation is necessary: All Dom0 does with the
>>>> provided info is set up the resource tree. The data doesn't get stored
>>>> for any post-boot use. What am I overlooking?
>>> /sbin/kexec opens /proc/iomem and looks for "Crash note" and interprets
>>> the range values. This is how it grabs the locations to pack into its
>>> magic binary package.
>> So how does the hotplug scenario then get handled on native? I can't
>> imagine they expect things to remain stable across CPU unplug and
>> re-activation.
>
> I am not how (or even if) the hotplug condition is handled on native. I
> guess it depends on what is put into the resource tree on boot. With my

I don't think native kexec depends on stuff being made visible in
/proc/iomem - grep-ing for "Crash note" in 2.6.18 as well as a current
tree hits exclusively the Xen instance.

Native uses a per-CPU allocation (i.e. address and contents can't be
expected to survive offlining of a CPU).

> patch, Xen will give crash areas for all pcpus up to nr_cpu_ids, which
> covers all the cases. The worst that will happen is that some crash
> notes do not get written if certain cpus are offline at the time of a crash.

And did you check that nothing in the producer or consumer chain gets
confused by this new behavior?

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: KEXEC: allocate crash note buffers at boot time v5 [ In reply to ]

andrew.cooper3 at citrix

Dec 2, 2011, 8:55 AM

Post #7 of 7 (183 views)

Permalink

On 02/12/11 16:38, Jan Beulich wrote:
>> patch, Xen will give crash areas for all pcpus up to nr_cpu_ids, which
>> covers all the cases. The worst that will happen is that some crash
>> notes do not get written if certain cpus are offline at the time of a crash.
> And did you check that nothing in the producer or consumer chain gets
> confused by this new behavior?
>
> Jan
>

/proc/vmcore reported by the kdump kernel is fine, even with extra notes
which have 0 contents (after I deliberately caused kexec_get_cpu to
report 1 more cpu than was present for testing exactly this)

The producer/consumer chain will not change. There was a case
previously where a CPU which was present at boot and subsequently
offlined would leave crash notes with 0's in them. Therefore, if a tool
couldn't deal with that case, it wont be able to deal with this case.
If a tool could deal with that case, it can deal with the new case.

I would hazard a guess that most of the time, we will boot on all CPUs
and crash with all of those CPUs still up, so it is more likely that
there will be no 0'd notes.

(When I say a 0'd note, I mean a note with correct header, type and
name, but 0's in the desc)

--
Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer
T: +44 (0)1223 225 900, http://www.citrix.com

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Mailing List Archive

Attached Files:

Attached Files: