taskwarrior (REQUIRES KTERM)

[nasser]

Originally Posted by knc1

Use:
LD_DEBUG=libs task
to start the program and the loader will show you what the problem is.

knc1,

I tried it. The output varies depending on whether I use the 'ld-linux' or 'export LD_LIBRARY' method to pre-load the required libraries.

I'm still trying to sort out and understand the output.

With the 'ld-linux' method, the output show a 'Segmentation fault' at the end, after a 'trying libpthread.so' message.

With the 'export LD_LIBRARY' method, the output goes into an endless loop! After I did a Ctrl-C and scrolled back, I could only see a series of 'searching', 'finding' and 'trying' messages.

[nasser]

There is a synchronisation feature in the recent releases of taskwarrior. It allows the same user with multiple devices to sync the data across all of them.

I already had tw running and synced on my ubuntu netbook and Android mobile. With tw now able to run on my pw3, it too joined in!

You need to set up your own "task server", which all the devices can access. However, Jack Desert has set up a free server, which anybody can use... unless, of course, your data is too confidential..!

Full instructions here:
https://freecinc.com

[knc1]

Originally Posted by nasser

knc1,

I tried it. The output varies depending on whether I use the 'ld-linux' or 'export LD_LIBRARY' method to pre-load the required libraries.

I'm still trying to sort out and understand the output.

With the 'ld-linux' method, the output show a 'Segmentation fault' at the end, after a 'trying libpthread.so' message.

With the 'export LD_LIBRARY' method, the output goes into an endless loop! After I did a Ctrl-C and scrolled back, I could only see a series of 'searching', 'finding' and 'trying' messages.

That is interesting.
I haven't seen or heard of that happening, the dynamic loader is a pretty solid piece of software.
To put it into a loop deserves some sort of award (or a message to Ulrich Drepper).

It should have output the complete pathname to each library it was going to try using. That so you can check the specific file that is giving the trouble.

That information is cached, and only output when first encountered.
I.E: you have to capture the head of the output, not the tail.

And -
LD_DEBUG=help
IIRC, that will output a full set of the options (you can use more than one at a time) - maybe =libs is not the most helpful one in this case.

[nasser]

Have now got all the files ready for a packaged release of taskwarrior and tasksh ports on PW3 and KT3. Should possibly work on PW2, KT2, KV & KOA too.

...But still trying to make them work on PW1!

Removed all the files and started from scratch.
Did a ldconfig cache refresh too.
Still the same 'GLIBC' errors.
Apparently, /lib/libc.so.6 - which is the PW1 version does not have GLIBC versions 2.15, 2.16, 2.17 and 2.18 -- which is required by other libraries in the package.

knc1 noted this:

Originally Posted by knc1

Your problem is not the system library this time.

Dynamic Section:
  RUNPATH              $ORIGIN/tasklib
  NEEDED               libgnutls-deb0.so.28 ; provided
  NEEDED               libuuid.so.1 ; provided
  NEEDED               libstdc++.so.6  <<-- check this one, it isn't shipped with the package.
  NEEDED               libm.so.6 ; should be alright at 2.12.1
  NEEDED               libgcc_s.so.1 <<-- check this one if it was built with a newer compiler than what is on the PW1
  NEEDED               libc.so.6 ; should be alright at 2.12.1


Version References:
  required from libuuid.so.1:
    0x09da27b0 0x00 12 UUID_1.0
  required from libgnutls-deb0.so.28:
    0x097dcb90 0x00 18 GNUTLS_DEBIAN_0_2_10
    0x07d5bcb0 0x00 10 GNUTLS_DEBIAN_0_3_1_0
    0x0d97dbb4 0x00 08 GNUTLS_DEBIAN_0_1_4
  required from libgcc_s.so.1:
    0x0b792654 0x00 19 GCC_3.4
    0x0b792655 0x00 06 GCC_3.5
  required from libm.so.6:
    0x0d696914 0x00 05 GLIBC_2.4
  required from libstdc++.so.6:
    0x0297f870 0x00 17 GLIBCXX_3.4.20
    0x0bafd178 0x00 16 CXXABI_1.3.8
    0x0bafd171 0x00 15 CXXABI_1.3.1
    0x0297f868 0x00 14 GLIBCXX_3.4.18
    0x0297f861 0x00 13 GLIBCXX_3.4.11
    0x0bafd175 0x00 11 CXXABI_1.3.5
    0x02297f89 0x00 09 GLIBCXX_3.4.9
    0x0849afa3 0x00 07 CXXABI_ARM_1.3.3
    0x08922974 0x00 04 GLIBCXX_3.4
    0x056bafd3 0x00 03 CXXABI_1.3
  required from libc.so.6:
    0x0d696914 0x00 02 GLIBC_2.4

The paperwhite has 2.12.1 installed on it.
12 > 4
That should not be the problem.

In addition to the notes above, I did not check for anything it might be dl opening.

PW-1, 5.6.1.1 has:
libstdc++.so.6.0.14
pre-installed
6 > 3 so that should be alright.

Alas, My PW1 is still on 5.3.6 Firmware version.
Running

ldd --version

gave

ldd (Sourcery G++ Lite 2009q3-67) 2.10.1

3 < 6... (the second digit of the FW version)
But 10 is still > 4 (the second digit of the glibc version?)
And libstdc++.so.x. is absent in /lib.. but it is able to use the latest one which I copied over into ./tasklib/

So.. libc.so.6 is the problem. Though a later version is available in the ./tasklib/ directory, it is not used, for some reason..

I stopped updating my PW1 FW after 5.3.6., since I think later versions broke Collections Manager.

Any comments appreciated!

[knc1]

Well now, that is new information to me, that you are not using the current firmware.

If you do switch to the current firmware, you can use LibrarianSync for your collections.
And of course, Amazon has since CM introduced maintaining your collection in the "manage devices and content" web page of your account.

= = = =

PS: The LDD script is just looking at the system cache - which I don't recall you mentioning that you had updated, that you where using LD_LIBRARY_* features.

Use readelf and/or objdump (+ grep as required) to find out what the binaries contain and require.
Not what Amazon's cache thinks you want.

[nasser]

knc1,

Thank you for your numerous tips in this and other recent threads!
Based on those , I've listed the following information.

Just to remind, I built taskwarrior on a Debian-Jessie-Armel VM, copied it over to my PW1, but was unable to run it due to GLIBC version errors. The executable worked OK on my PW3( 5.7.4 with GLIBC 2.19) and KT3(5.8.0 with GLIBC 2.19).

Kindle Models and associated Framework and GLIBC versions
Model -- FW version ------------ (e)GLIBC version
KT1 ---- 5.0.0 to 5.3.7.3 ------ 2.12.1
PW1 ---- 5.2.0 to 5.6.1.1 ------ 2.12.1
[But my PW1 has 5.3.6 and 2.10.1]
PW2 ---- 5.4.2 to 5.6.2.1 ------ 2.12.1
PW2 ---- 5.6.5 to 5.7.4 -------- 2.19
PW3 ---- 5.6.1.1 to 5.8.2.1 ---- 2.19
PW3 ---- 5.8.5 to 5.8.9.0.2 ---- 2.20
[My KT3 has 5.8.0 and 2.19]

Requirement of GLIBC versions by libraries for running on PW1
libgnutls-deb0.so.28: 2.15, 2.17
stdc++.so.6: 2.17, 2.18
libp11-kit.so.0: 2.15, 2.16
So... minimum 2.18

Possible Options:
1. Update Framework version of PW1 from 5.3.6 to 5.6.1.1? but as the above table indicates, this does not push the GLIBC version of the PW1 beyond 2.12.1
2. Compile taskwarrior again on the VM and this time, link older libraries to it?
3. Compile taskwarrior again on the VM and statically link the older libraries? But this not recommended by the experts.
4. Use patchelf to hack the executable headers? Seems risky...

What would you suggest?

[knc1]

None of the above.

From your list, you only need to ship a more recent libstdc++ with task warrior.

Note:
libstdc++ is not shipped with the c++ compiler nor with the system library, it is a separate item.

Hmm...
Sort of the same situation you will find with some other languages.
Their 'extension' libraries are a separate install.
Although it is also true that it is a requirement for anything more complex than "Hello World" written in c++.
You have some history and tradition at work here, calling it an "extension" to the language.

- - - -

O.T:
On one of my test VM images, I included Ada.
Just for the challenge of supporting it.

The VM image building and testing has hit a speed bump - Suse Studio has run into server problems this past week.
I actually have started to move the whole project over to their build farm (a cluster of 1200 machines).

[nasser]

Originally Posted by knc1

None of the above.

From your list, you only need to ship a more recent libstdc++ with task warrior.
...
...

I think I haven't explained more clearly:

libgnutls-deb0.so.28, stdc++.so.6, libp11-kit.so.0 - All these files were already copied over to the PW1 from the Debian-Jessie-Armel VM image. The older versions on the PW1, if at all available, weren't "found".

But the executable still didn't work because higher versions of GLIBC were not found in PW1's /lib/libc.so.6 by these 3 library files:

• libgnutls-deb0.so.28: needed GLIBC versions 2.15, 2.17
• stdc++.so.6: : needed GLIBC versions 2.17, 2.18
• libp11-kit.so.0: : needed GLIBC versions 2.15, 2.16

So, we need to either:

• Provide PW1 with a later version of libc.so.6/GLIBC... or
• Link the executable with earlier versions of the above 3 library files - which can work with GLIBC version 2.10.1(the version available on PW1 with framework version 5.3.6).

EDIT: I've mentioned libstdc++.so.6 above as "only" stdc++.so.6. Hope that didn't cause any confusion!

[knc1]

What symbols are those?

I only recognize memcpy (which can be fixed in the build process).

I.E: You don't have to provide the entire updated library, just satisfy the linkage targets.

[nasser]

Originally Posted by knc1

What symbols are those?

I only recognize memcpy (which can be fixed in the build process).

I.E: You don't have to provide the entire updated library, just satisfy the linkage targets.

knc1,

Here's the entire message dump when trying to run it on PW1:

[root@kindle tw251]# ./taskwarrior.sh 
./task: /lib/libc.so.6: version `GLIBC_2.15' not found (required by tasklib/libgnutls-deb0.so.28)
./task: /lib/libc.so.6: version `GLIBC_2.17' not found (required by tasklib/libgnutls-deb0.so.28)
./task: /lib/libc.so.6: version `GLIBC_2.18' not found (required by tasklib/libstdc++.so.6)
./task: /lib/libc.so.6: version `GLIBC_2.17' not found (required by tasklib/libstdc++.so.6)
./task: /lib/libc.so.6: version `GLIBC_2.16' not found (required by tasklib/libp11-kit.so.0)
./task: /lib/libc.so.6: version `GLIBC_2.15' not found (required by tasklib/libp11-kit.so.0)

[knc1]

So use your tools to list the symbols being requested.
Your choice of objdump, readelf, nm (and probably others that slip my mind at the moment).

[knc1]

Originally Posted by nasser

knc1,

Here's the entire message dump when trying to run it on PW1:

[root@kindle tw251]# ./taskwarrior.sh 
./task: /lib/libc.so.6: version `GLIBC_2.15' not found (required by tasklib/libgnutls-deb0.so.28)
./task: /lib/libc.so.6: version `GLIBC_2.17' not found (required by tasklib/libgnutls-deb0.so.28)
./task: /lib/libc.so.6: version `GLIBC_2.18' not found (required by tasklib/libstdc++.so.6)
./task: /lib/libc.so.6: version `GLIBC_2.17' not found (required by tasklib/libstdc++.so.6)
./task: /lib/libc.so.6: version `GLIBC_2.16' not found (required by tasklib/libp11-kit.so.0)
./task: /lib/libc.so.6: version `GLIBC_2.15' not found (required by tasklib/libp11-kit.so.0)

Originally Posted by knc1

So use your tools to list the symbols being requested.
Your choice of objdump, readelf, nm (and probably others that slip my mind at the moment).

OK, a worked example:

knc1:tasklib> objdump -T libgnutls-deb0.so.28 | grep -E 'GLIBC_2.15|GLIBC_2.17'
00000000      DF *UND*    00000000  GLIBC_2.17  clock_gettime
00000000      DF *UND*    00000000  GLIBC_2.15  __fdelt_chk

and then something like:

knc1:tasklib> man clock_gettime

And find among other things that the system call was introduced in Kernel-2.6
So the kernel being run is recent enough, only thing needed is the symbol definition.

I.E: A small code file, whose *.o is included in the link command of the build.

Here is an example cribbed from the web for a different symbol (Cliff's notes version):

$ objdump -p myprog
...
Version References:
  required from libc.so.6:
    0x09691972 0x00 05 GLIBC_2.3
    0x09691a75 0x00 03 GLIBC_2.2.5

$ objdump -T myprog | fgrep GLIBC_2.3
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.3   realpath

// Check depended upon library for symbol versions:

$ objdump -T /lib/libc.so.6 | grep -w realpath
0000000000105d90 g    DF .text  0000000000000021 (GLIBC_2.2.5) realpath
000000000003e7b0 g    DF .text  00000000000004bf  GLIBC_2.3   realpath

// Request the earlier symbol version in your application:

#include <limits.h>
#include <stdlib.h>

__asm__(".symver realpath,realpath@GLIBC_2.2.5");

int main () {
    realpath ("foo", "bar");
}

// Check re-compiled application:

$ objdump -p myprog
...
Version References:
  required from libc.so.6:
    0x09691a75 0x00 02 GLIBC_2.2.5

$ objdump -T myprog | grep realpath
0000000000000000      DF *UND*  0000000000000000  GLIBC_2.2.5 realpath

Now the above presumed you wanted to change your source code.
But the same could be put into its own source code, compiled to an -FPIC shared *.o.
Now, how to get it included in the build without modifying the source code, or even the build script / makefile (or whatever the application uses).

Look in /usr/lib and you should find a symbolic link from the soname to the file which actually provides the code, as in (only an example from my desktop, not the names you need to look for):

knc1:lib> ls -l | grep libgnu
lrwxrwxrwx  1 root root        20 Jun 28 06:04 libgnutls.so.30 -> libgnutls.so.30.14.5
-rwxr-xr-x  1 root root   1361328 Jun 28 06:04 libgnutls.so.30.14.5

Let's assume your files are clock_gettime.c and clock_gettime.o or *.so
Replace the symbolic link using the soname with a (text) file of the same name and extension.
Your text file needs to contain:

GROUP( clock_gettime.so, libgnutls.so.30.14.5)

Note: I may have typo'd that in some strange and/or wonderful way - I don't have the reference materials in front of me. Just consider it an illustration of the procedure.
Note: the 'GROUP' command also alters the search order of the linker.

Edit: I did.
It is gnutls that is requesting a sysmbol missing from libc.so
So the name of the new provider needs to go in the libc.so group command file.
What I wrote above is as if the application was requesting a symbol missing from libgnutls.
Too tired to fix it tonight, but you should have enough clues now to follow up on a fix for taskwarrior on older systems.

The arguments to the 'GROUP' may be absolute or relative paths to the name of the file(s) that provide the symbols associated with the soname.

Display the contents of your: /usr/lib/libc.so
for a worked/working example (it is a text file with linker commands, not a binary).

= = = =

Note:
Building on a FAT-32 system (which does not support symbolic links) requires a whole lot of these little linker command files to associate the soname with the name of the file providing the binary (without using symbolic links).

I think twobob did it (several times) but I am not sure if even you found his post/files, they would help now - probably too old by now anyway.

[nasser]

Thanks, knc1!
While I was trying out, you already posted an example!

My output:

$ readelf -s libgnutls-deb0.so.28|grep GLIBC_2.1
    51: 00000000     0 FUNC    GLOBAL DEFAULT  UND clock_gettime@GLIBC_2.17 (13)
    91: 00000000     0 FUNC    GLOBAL DEFAULT  UND __fdelt_chk@GLIBC_2.15 (14)
$ readelf -s libstdc++.so.6|grep GLIBC_2.1
    16: 00000000     0 FUNC    GLOBAL DEFAULT  UND clock_gettime@GLIBC_2.17 (38)
    70: 00000000     0 FUNC    GLOBAL DEFAULT  UND __cxa_thread_atexit_impl@GLIBC_2.18 (40)
$ readelf -s libp11-kit.so.0|grep GLIBC_2.1
    71: 00000000     0 FUNC    GLOBAL DEFAULT  UND getauxval@GLIBC_2.16 (6)
$ objdump -T libgnutls-deb0.so.28|grep GLIBC_2.1
00000000      DF *UND*	00000000  GLIBC_2.17  clock_gettime
00000000      DF *UND*	00000000  GLIBC_2.15  __fdelt_chk
$ objdump -T libstdc++.so.6|grep GLIBC_2.1
00000000      DF *UND*	00000000  GLIBC_2.17  clock_gettime
00000000      DF *UND*	00000000  GLIBC_2.18  __cxa_thread_atexit_impl
$ objdump -T libp11-kit.so.0|grep GLIBC_2.1
00000000      DF *UND*	00000000  GLIBC_2.16  getauxval

So, the functions being requested are:

GLIBC_2.15: __fdelt_chk
GLIBC_2.16: getauxval
GLIBC_2.17: clock_gettime
GLIBC_2.18: __cxa_thread_atexit_impl

Actually, libp11-kit.so.0 should also be needing GLIBC_2.15, as per the runtime error messages, but in the above, it shows that only GLIBC_2.16 is being requested. Don't know why this difference.

Now, I'll try to check further as indicated by you...

[knc1]

Originally Posted by nasser

Thanks, knc1!
While I was trying out, you already posted an example!

My output:

$ readelf -s libgnutls-deb0.so.28|grep GLIBC_2.1
    51: 00000000     0 FUNC    GLOBAL DEFAULT  UND clock_gettime@GLIBC_2.17 (13)
    91: 00000000     0 FUNC    GLOBAL DEFAULT  UND __fdelt_chk@GLIBC_2.15 (14)
$ readelf -s libstdc++.so.6|grep GLIBC_2.1
    16: 00000000     0 FUNC    GLOBAL DEFAULT  UND clock_gettime@GLIBC_2.17 (38)
    70: 00000000     0 FUNC    GLOBAL DEFAULT  UND __cxa_thread_atexit_impl@GLIBC_2.18 (40)
$ readelf -s libp11-kit.so.0|grep GLIBC_2.1
    71: 00000000     0 FUNC    GLOBAL DEFAULT  UND getauxval@GLIBC_2.16 (6)
$ objdump -T libgnutls-deb0.so.28|grep GLIBC_2.1
00000000      DF *UND*    00000000  GLIBC_2.17  clock_gettime
00000000      DF *UND*    00000000  GLIBC_2.15  __fdelt_chk
$ objdump -T libstdc++.so.6|grep GLIBC_2.1
00000000      DF *UND*    00000000  GLIBC_2.17  clock_gettime
00000000      DF *UND*    00000000  GLIBC_2.18  __cxa_thread_atexit_impl
$ objdump -T libp11-kit.so.0|grep GLIBC_2.1
00000000      DF *UND*    00000000  GLIBC_2.16  getauxval

So, the functions being requested are:

GLIBC_2.15: __fdelt_chk
GLIBC_2.16: getauxval
GLIBC_2.17: clock_gettime
GLIBC_2.18: __cxa_thread_atexit_impl

Actually, libp11-kit.so.0 should also be needing GLIBC_2.15, as per the runtime error messages, but in the above, it shows that only GLIBC_2.16 is being requested. Don't know why this difference.

Now, I'll try to check further as indicated by you...

I wrote most of the above from memory at the end of a long and tiring day.
The general, overall concept(s) should be correct, all of the details may be wrong or incorrectly described.

What I had intended to do was extract (and compiled as individual libraries) the changes from one system library version to the next.
Then do a sequence of individual, partial, linking steps. Those steps would have statically linked in ONLY the 'shim' code.

In your case, only 4 library functions (out of the zillion or more that would still be dynamically linked in at runtime).

In my example that I was working on, before SuseStudio went belly-up last week, was memcpy. (The support and maintenance of which has fallen into a black hole during the MicroFocus purchase of Novell (who owns (owned) Suse).

In that case, the way to get applications to link that require memcpy-GLIBC_2.14 or more recent is to write a one line source file that provides a function called mempy that is versioned as GLIBC_2.14 that in fact call memmov with the same arguments it was passed and returns the result returned by memmov.
(In fact, that is what memcpy_2.14 does, calls memmov).

I don't know what the 'backport action' is required for your four symbols.
But that only means I haven't looked (yet).

One thing that puzzles me is:
Symbols that begin with "_" are suppose to be "private" and symbols that begin with "__" are suppose to be "protected".
The 'C' language does not implement those classifications, but they are suppose to be implemented by agreed upon practice.

So I am surprised to see GNU tls (thread local storage) calling any "__" symbol of GNU libc (the system library).

But the folks at GNU development wrote them both and also wrote the 'rules' of how symbols that begin with one or more underscores are intended to be used.
So I guess they can get away with doing whatever they please.

= = = = =

And none of the above deals with symbols missing from libraries that are DLopened.
Which is probably where your error at run time but not at load time is coming from.

To find those, it will probably be required to 'instrument' the DL** (dynamic load) set of library entries.
The GNU toolchain has the ability to instrument any symbol reference. Known as a "wrap" function, passed to the linker from the compiler command line as -Wl,wrap .... option.
And in fact, it tells the linker to add a double pair of underscores to the external symbol name.
Which only deepens the mystery in my mind described above about symbol naming in actual practice.

But then I am old and easily befuddled.

[nasser]

Thanks for your feedback, knc1!
Over the last 4 years, I have always found your posts a great learning experience.. even if you do make us do some work on our own!