Mailing List Archive

livepatch-build-tools regression
Hello,

Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives
the following error during LP upload:

(XEN) livepatch: lp: Unknown symbol: .LC7

Bisecting identified the first bad commit as:

commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"

Base version of Xen used for this experiment is d13dfb02aafab
The patch file used for LP is attached.

--
Thanks,
Sergey
Re: livepatch-build-tools regression [ In reply to ]
> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> Hello,
>
> Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives
> the following error during LP upload:
>
> (XEN) livepatch: lp: Unknown symbol: .LC7
>
> Bisecting identified the first bad commit as:
>
> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"
>
> Base version of Xen used for this experiment is d13dfb02aafab
> The patch file used for LP is attached.
>
> --
> Thanks,
> Sergey
> <0001-live-patch.patch>

Could you give this a try?

https://patchwork.kernel.org/patch/11228191/
https://patchwork.kernel.org/patch/11228189/

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
On 18/11/2019 16:47, Wieczorkiewicz, Pawel wrote:
>
>
>> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>
>> Hello,
>>
>> Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives
>> the following error during LP upload:
>>
>> (XEN) livepatch: lp: Unknown symbol: .LC7
>>
>> Bisecting identified the first bad commit as:
>>
>> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"
>>
>> Base version of Xen used for this experiment is d13dfb02aafab
>> The patch file used for LP is attached.
>>
>> --
>> Thanks,
>> Sergey
>> <0001-live-patch.patch>
>
> Could you give this a try?
>
> https://patchwork.kernel.org/patch/11228191/
> https://patchwork.kernel.org/patch/11228189/

Unfortunately, those patches didn't resolve the issue for me.

Forgot to add, my gcc version is

gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0

--
Thanks,
Sergey

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
> On 18. Nov 2019, at 18:09, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> On 18/11/2019 16:47, Wieczorkiewicz, Pawel wrote:
>>
>>
>>> On 18. Nov 2019, at 17:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>>
>>> Hello,
>>>
>>> Trying to build a simple version of XSA-304 Live-Patch for 4.13 gives
>>> the following error during LP upload:
>>>
>>> (XEN) livepatch: lp: Unknown symbol: .LC7
>>>
>>> Bisecting identified the first bad commit as:
>>>
>>> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"
>>>
>>> Base version of Xen used for this experiment is d13dfb02aafab
>>> The patch file used for LP is attached.
>>>
>>> --
>>> Thanks,
>>> Sergey
>>> <0001-live-patch.patch>
>>
>> Could you give this a try?
>>
>> https://patchwork.kernel.org/patch/11228191/
>> https://patchwork.kernel.org/patch/11228189/
>
> Unfortunately, those patches didn't resolve the issue for me.
>
> Forgot to add, my gcc version is
>
> gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0

Could you build the lp with debug (-d) and provide me with the create-diff-object.log file?

>
> --
> Thanks,
> Sergey

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote:
>>
>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file?
>>
>
> I've attached the log. Btw, I think I provided all the necessary information
> for others to repeat my experiment.
>

Sorry for another request, but I do not seem to be able to reproduce this locally.
Could you send me the livepatch module binary that fails to upload?

> --
> Thanks,
> Sergey
> <create-diff-object.log.xz>

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote:
>
>
>> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>
>> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote:
>>>
>>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file?
>>>
>>
>> I've attached the log. Btw, I think I provided all the necessary information
>> for others to repeat my experiment.
>>
>
> Sorry for another request, but I do not seem to be able to reproduce this locally.
> Could you send me the livepatch module binary that fails to upload?

That's interesting. I've attached the binary that my system produces.
What version of gcc do you use?

--
Thanks,
Sergey
Re: livepatch-build-tools regression [ In reply to ]
> On 20. Nov 2019, at 12:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote:
>>
>>
>>> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>>
>>> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote:
>>>>
>>>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file?
>>>>
>>>
>>> I've attached the log. Btw, I think I provided all the necessary information
>>> for others to repeat my experiment.
>>>
>>
>> Sorry for another request, but I do not seem to be able to reproduce this locally.
>> Could you send me the livepatch module binary that fails to upload?
>
> That's interesting. I've attached the binary that my system produces.
> What version of gcc do you use?

The version used was: gcc (GCC) 7.2.1 20170915

But I have finally managed to reproduce the issue with:
1. gcc (Ubuntu 6.5.0-2ubuntu1~18.04) 6.5.0 20181026
2. gcc-7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0

I think it is not related to the commit:
commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"

I managed to reproduce it also with earlier version commit:
"0c10457 Remove section alignment requirement"

But this time a different symbol causes the failure:

(XEN) livepatch: 0001-live-patch: Unknown symbol: hvm.c#lastpage.22856

>
> --
> Thanks,
> Sergey
> <0001-live-patch-stripped.livepatch>

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
It looks like gcc plays the usual dirty tricks with local variables renaming:

- xen-syms
7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857
- livepatch
289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856

Then, symbols resolution by name fails..

Can you please try to build the livepatch module with additional option '—prelink' and give it a try ?

> On 26. Nov 2019, at 18:51, Wieczorkiewicz, Pawel <wipawel@amazon.de> wrote:
>
>
>
>> On 20. Nov 2019, at 12:42, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>
>> On 19/11/2019 17:21, Wieczorkiewicz, Pawel wrote:
>>>
>>>
>>>> On 18. Nov 2019, at 18:41, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>>>
>>>> On 18/11/2019 17:28, Wieczorkiewicz, Pawel wrote:
>>>>>
>>>>> Could you build the lp with debug (-d) and provide me with the create-diff-object.log file?
>>>>>
>>>>
>>>> I've attached the log. Btw, I think I provided all the necessary information
>>>> for others to repeat my experiment.
>>>>
>>>
>>> Sorry for another request, but I do not seem to be able to reproduce this locally.
>>> Could you send me the livepatch module binary that fails to upload?
>>
>> That's interesting. I've attached the binary that my system produces.
>> What version of gcc do you use?
>
> The version used was: gcc (GCC) 7.2.1 20170915
>
> But I have finally managed to reproduce the issue with:
> 1. gcc (Ubuntu 6.5.0-2ubuntu1~18.04) 6.5.0 20181026
> 2. gcc-7 (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
>
> I think it is not related to the commit:
> commit 854a7ca60e35 "create-diff-object: Do not include all .rodata sections"
>
> I managed to reproduce it also with earlier version commit:
> "0c10457 Remove section alignment requirement"
>
> But this time a different symbol causes the failure:
>
> (XEN) livepatch: 0001-live-patch: Unknown symbol: hvm.c#lastpage.22856
>
>>
>> --
>> Thanks,
>> Sergey
>> <0001-live-patch-stripped.livepatch>
>
> Best Regards,
> Pawel Wieczorkiewicz

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote:
> It looks like gcc plays the usual dirty tricks with local variables renaming:
>
> - xen-syms
> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857
> - livepatch
> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856
>
> Then, symbols resolution by name fails..
>
> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ?

My LP loading error is:

(XEN) livepatch: lp: Unknown symbol: .LC7

When I pass --prelink to livepatch-build, it complains in a similar way:

livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c)

--
Thanks,
Sergey

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote:
>> It looks like gcc plays the usual dirty tricks with local variables renaming:
>>
>> - xen-syms
>> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857
>> - livepatch
>> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856
>>
>> Then, symbols resolution by name fails..
>>
>> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ?
>
> My LP loading error is:
>
> (XEN) livepatch: lp: Unknown symbol: .LC7
>
> When I pass --prelink to livepatch-build, it complains in a similar way:
>
> livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c)
>

Could you give this testing patch a try?

diff --git a/create-diff-object.c b/create-diff-object.c
index 8d63940..10807d2 100644
--- a/create-diff-object.c
+++ b/create-diff-object.c
@@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist)
list_for_each_entry(sym, symlist, list) {
if (sym->twin)
kpatch_compare_correlated_symbol(sym);
- else
+ else {
sym->status = NEW;
+ sym->include = 1;
+ }

log_debug("symbol %s is %s\n", sym->name, status_str(sym->status));
}

> --
> Thanks,
> Sergey

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
On 27/11/2019 15:22, Wieczorkiewicz, Pawel wrote:
>
>
>> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>
>> On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote:
>>> It looks like gcc plays the usual dirty tricks with local variables renaming:
>>>
>>> - xen-syms
>>> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857
>>> - livepatch
>>> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856
>>>
>>> Then, symbols resolution by name fails..
>>>
>>> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ?
>>
>> My LP loading error is:
>>
>> (XEN) livepatch: lp: Unknown symbol: .LC7
>>
>> When I pass --prelink to livepatch-build, it complains in a similar way:
>>
>> livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c)
>>
>
> Could you give this testing patch a try?
>
> diff --git a/create-diff-object.c b/create-diff-object.c
> index 8d63940..10807d2 100644
> --- a/create-diff-object.c
> +++ b/create-diff-object.c
> @@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist)
> list_for_each_entry(sym, symlist, list) {
> if (sym->twin)
> kpatch_compare_correlated_symbol(sym);
> - else
> + else {
> sym->status = NEW;
> + sym->include = 1;
> + }
>
> log_debug("symbol %s is %s\n", sym->name, status_str(sym->status));
> }
>

Looks like this change fixed the issue for me!
One thing to notice is that the size of a stripped LP binary increased
from 45K to 60K.

--
Thanks,
Sergey

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: livepatch-build-tools regression [ In reply to ]
> On 27. Nov 2019, at 16:56, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>
> On 27/11/2019 15:22, Wieczorkiewicz, Pawel wrote:
>>
>>
>>> On 27. Nov 2019, at 12:16, Sergey Dyasli <sergey.dyasli@citrix.com> wrote:
>>>
>>> On 26/11/2019 18:37, Wieczorkiewicz, Pawel wrote:
>>>> It looks like gcc plays the usual dirty tricks with local variables renaming:
>>>>
>>>> - xen-syms
>>>> 7529: ffff82d0805fed50 8 OBJECT LOCAL DEFAULT 4230 lastpage.22857
>>>> - livepatch
>>>> 289: 0000000000000000 8 OBJECT GLOBAL DEFAULT UND hvm.c#lastpage.22856
>>>>
>>>> Then, symbols resolution by name fails..
>>>>
>>>> Can you please try to build the livepatch module with additional option '—prelink' and give it a try ?
>>>
>>> My LP loading error is:
>>>
>>> (XEN) livepatch: lp: Unknown symbol: .LC7
>>>
>>> When I pass --prelink to livepatch-build, it complains in a similar way:
>>>
>>> livepatch-build-tools/prelink: ERROR: output.o: livepatch_resolve_symbols: 80: lookup_local_symbol .LC7 (p2m.c)
>>>
>>
>> Could you give this testing patch a try?
>>
>> diff --git a/create-diff-object.c b/create-diff-object.c
>> index 8d63940..10807d2 100644
>> --- a/create-diff-object.c
>> +++ b/create-diff-object.c
>> @@ -839,8 +839,10 @@ static void kpatch_compare_symbols(struct list_head *symlist)
>> list_for_each_entry(sym, symlist, list) {
>> if (sym->twin)
>> kpatch_compare_correlated_symbol(sym);
>> - else
>> + else {
>> sym->status = NEW;
>> + sym->include = 1;
>> + }
>>
>> log_debug("symbol %s is %s\n", sym->name, status_str(sym->status));
>> }
>>
>
> Looks like this change fixed the issue for me!
> One thing to notice is that the size of a stripped LP binary increased
> from 45K to 60K.
>

Yes, this was not supposed to be a proper fix. I was merely trying to establish if we were looking at the same issue.
I did fix it now the proper way, though. I shall send the patches soon.

> --
> Thanks,
> Sergey

Best Regards,
Pawel Wieczorkiewicz






Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Ralf Herbrich
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel