* soname handling has been re-written from scratch and improved substantially.

       It now identifies every soname versioned library in elf executables
       along with their full path.  It also properly handles executables
       built with *--rpath* loader options.

       Previous versions relied on makepkg soname output
       which, unfortunately, only lists sonames if they are also listed as a PKGBUILD dependency.
       We need every soname versioned library to ensure we do the right thing
       and rebuild when needed. So it was a mistake to rely on this.

       Can also specify how to handle version comparisons similar to the way
       package version comparisons are done (e.g. soname > major)

     * Old options now deprecated

        * (*--mpk-xxx*)
        * (*--soname-build*) : use *--soname-comp* instead

Author: Gene C

Docs/conf.py

@@ -9,7 +9,7 @@
 project = "mkpkg"
 copyright = '2023, Gene C'
 author = 'Gene C'
-release = '5.0.0'
+release = '6.0.0'
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

README.rst

@@ -12,8 +12,26 @@ Tool to rebuild Arch packages based on dependency triggers.
 New / Interesting
 ==================
 
- * Version comparisons now use pyalpm instead of packaging
-   packaging.version barfs on systemd package version 255rc2.1
+ * Major update: soname handling has been re-written from scratch and improved substantially. 
+
+   It now identifies every soname versioned library in elf executables
+   along with their full path.  It also properly handles executables 
+   built with *--rpath* loader options.
+
+   Previous versions relied on makepkg soname output
+   which, unfortunately, only lists sonames if they are also listed as a PKGBUILD dependency.
+   We need every soname versioned library to ensure we do the right thing
+   and rebuild when needed. So it was a mistake to rely on this.
+
+   Can also specify how to handle version comparisons similar to the way 
+   package version comparisons are done (e.g. soname > major)
+
+   If you're interested, the soname info is saved into the file *.mkp_dep_soname*
+
+ * Old options now deprecated
+   
+    * (*--mpk-xxx*)
+    * (*--soname-build*) : use *--soname-comp* instead
 
 #################
 mkpkg application
@@ -205,7 +223,7 @@ With the trigger conditions in the PKGBUID, then simply call mkpkg instead of ma
 Options for mkpkg are those before any double dash *--*. Any options following *--*
 are passed through to *makepkg* [#]_.
 
-.. [#] The older style options using *--mkp-* are deprecated but are supported to ensure backward compatibility. They will be removed at some point in the future.
+.. [#] The older style options using *--mkp-* are now deprecated.
 
 Options
 =======
@@ -224,9 +242,26 @@ The options currently supported by mkpkg are:
 
    Attempts to update saved metadata files. Faster, if imperfect, alternative to rebuild.
    If there is no saved metadata, and build is up to date, will try refresh the build info.
-   Files updated are *.mkp\_dep\_vers* and  *.mkp_dep_soname*. The soname data can only be updated
-   if the .PKGINFO file is still in the *pkg* directory.  Forcing a rebuild is the 
-   slower alternative but is guaranteed to have information needed.
+   Files updated are *.mkp\_dep\_vers* and  *.mkp_dep_soname*. 
+
+   Note that *sonames* are found by examining any executables in the *pkg* directory.
+   If the *pkg* directory is empty, the refresh will not find any sonames.
+   
+ * (**so-comp, --soname-comp**)
+
+   How to handle automatic soname changes. Default value is *last* - which uses the entire soname version
+   when comparing to what's available.
+
+    * *never* : soname dependencies are ignored
+
+    * *newer* : if soname is newer then reubild (time based)
+
+    * *keep* : if soname library is still available, then dont rebuild even if newer version(s) are available
+
+    * *vcomp* : rebuild if soname version is greater than the *vcomp* version. *vcomp* is one of *major*, *minor*, *patch*, *extra* or *last* - same as for regular depenencies.
+
+    * *neverever* : Developer option - will not rebuild even if the soname library is no longer available.
+
 
  * (*--*)  
 
@@ -236,9 +271,9 @@ The options currently supported by mkpkg are:
 
 Configs are looked for in first in /etc/mkpkg/config and then in
 ~/.config/mkpkg/config. Config files are in TOML format. 
-e.g. to change the default soname rebuild option::
+e.g. to change the default soname rebuild compare option from default of *last*::
 
-        soname_build = "newer"
+        soname_comp = "newer"
 
 How mkpkg works
 ===============
@@ -488,6 +523,31 @@ Created by Gene C. and licensed under the terms of the MIT license.
 Some history
 ============
 
+Version 4.1.0
+-------------
+
+ * soname rewrite
+   
+   New argument for how soname changes are treated : *-so-comp, --soname-comp*. 
+
+   Can be *<compare>*, *newer*,  *never* or key how to compare the soname versions. 
+   The comparison types are the same as for package dependencies described above.
+   Default is *last* which means the entire soname version will be compared to 
+   whats available and rebuild will be triggered if a later version now available.
+
+   *<compare>* e.g. *>major* or *>minor*' or *last* etc. 
+   If the last built soname was 5.1, and now available is 5.2 then
+   *minor* and *last* will trigger rebuild while *major* would not. *newer* triggers if the
+   last modify time of the library is newer.
+
+   Previous version used sonmaes produced by makepkg - however this only generates
+   sonames if they are listed as dependencies. We want to get every soname - so 
+   we started over from scratch. By using our own soname generate we catch
+   every soname and its absolute path - this enables us to correctly treat soname
+   changes. This approach will also correctly deal with any *rpath* loader flags
+   causing executable to use shared library from path(s) specified at compile time.
+
+
 Version 4.1.0
 -------------
 
@@ -498,17 +558,13 @@ Version 4.1.0
     compatibility the older *--mkp-* style arguments are honored, but the newer simpler
     ones are preferred. e.g. *-v, --verb* for verbose. Help availble via *-h*. 
 
- * New argument for how soname changes are treated : *-so-bld, --soname-build*. 
-
-   Can be *missing*, *newer* or *never*. Default is missing - rebuild if soname 
-   no longer available.  
 
  * Config file now available.
 
    Configs are looked for in /etc/mkpkg/config then ~/.config/mkpkg/config. It should
    be in TOML format. e.g. to change the default soname rebuild option::
 
-        soname_build = "newer"
+        soname_comp = "newer"
 
 Version 4.0.0
 -------------

packaging/PKGBUILD

@@ -10,14 +10,14 @@ pkgname='mkpkg'
 pkgdesc='Tool to rebuild packages based on dependency triggers'
 _gitname='Arch-mkpkg'
 
-pkgver=5.0.0
+pkgver=6.0.0
 pkgrel=1
 url="https://github.com/gene-git/Arch-mkpkg"
 
 arch=(any)
 license=(MIT)
 # tomli only needed for python < 3.11
-depends=('python>3.9' 'pyalpm' 'python-tomli' 'python-tomli-w')
+depends=('python>3.9' 'pyalpm' 'python-tomli' 'python-tomli-w' 'python-pyelftools')
 
 # To build docs uncommont sphinx/texlive
 makedepends=('git' 'python-build' 'python-installer' 'python-wheel' 'python-hatch' 'rsync'

src/mkpkg/__about__.py

@@ -3,4 +3,4 @@
 """
 Project mkpkg
 """
-__version__ = "5.0.0"
+__version__ = "6.0.0"

src/mkpkg/lib/build.py

@@ -14,8 +14,8 @@
 from .tools import check_package_exists
 from .build_makepkg import build_w_makepkg
 from .dep_vers import get_pkg_dep_vers_now
-from .soname import read_last_pkg_dep_soname
-from .soname import soname_rebuild_needed
+from .soname_deps import read_soname_deps
+from .soname_rebuild import soname_rebuild_needed
 
 def _build_if_needed(pkg_vers_changed, soname_build, pkg_file_info, mkpkg):
     """
@@ -140,7 +140,7 @@ def build(mkpkg):
     #
     # read the last soname data (if exists)
     #
-    read_last_pkg_dep_soname(mkpkg)
+    read_soname_deps(mkpkg)
     soname_build = soname_rebuild_needed(mkpkg)
 
     #

src/mkpkg/lib/check_deps.py

@@ -31,8 +31,7 @@ def _most_recent_package_date(mkpkg):
         mod_time = os.path.getmtime(pkgfile)
         pkg_dtime = datetime.datetime.fromtimestamp(mod_time)
         if dtime :
-            if pkg_dtime > dtime:
-                dtime = pkg_dtime
+            dtime = max(dtime, pkg_dtime)
         else:
             dtime = pkg_dtime
     return dtime

src/mkpkg/lib/class_config.py

@@ -16,8 +16,8 @@ def __init__(self):
         self.verb = False
         self.force = False
         self.refresh = False
-        self.use_makedepends = False    # deprecated
-        self.soname_build = 'missing'
+
+        self.soname_comp = 'last' # was previously soname_build
         self.makepkg_args = None
 
         #
@@ -40,28 +40,7 @@ def __init__(self):
             for (opt,val) in conf.items():
                 setattr(self, opt, val)
 
-        # Command line args
-        opts = [
-                [('-v', '--verb', '--mkp-verb'), {'action' : 'store_true',
-                  'help' : 'More verbose output',
-                  }
-                ],
-                [('-f', '--force', '--mkp-force'), {'action' : 'store_true',
-                    'help' : 'Bump package release and rebuild'}
-                ],
-                [('-r', '--refresh', '--mkp-refresh'), {'action' : 'store_true',
-                    'help' : 'update saved metadata files.'}
-                ],
-                [('-so-bld', '--mkp-soname-build'),
-                 {'help' : f'Rebuild if soname missing, newer, never (default {self.soname_build})'}
-                ],
-                [('--mkp-use_makedepends'), {'action' : 'store_true',
-                    'help' : 'Use makedepends array of no _mkpkg_xxx set (deprecated))'}
-                ],
-                [('makepkg'), {'nargs' : '*',
-                    'help' : 'All remaining args after -- passed to makepkg'}
-                ],
-               ]
+        opts = get_available_opts(self)
         par = argparse.ArgumentParser(description='mkpkg')
         for opt in opts:
             (opts), kwargs = opt
@@ -85,3 +64,35 @@ def __init__(self):
                 del opt_dict['makepkg']
             for (opt, val) in opt_dict.items():
                 setattr(self, opt, val)
+
+def get_available_opts(conf) -> [[tuple, dict]]:
+    """
+    Available options
+    """
+    # Command line args
+    opts = []
+    act = 'action'
+    act_on = 'store_true'
+
+    ohelp = 'More verbose output'
+    opt = [('-v', '--verb'), {'help' : ohelp, act : act_on}]
+    opts.append(opt)
+
+    ohelp = 'Bump package release and rebuild'
+    opt = [('-f', '--force'), {'help' : ohelp, act : act_on}]
+    opts.append(opt)
+
+    ohelp = 'Update saved metadata files.'
+    opt = [('-r', '--refresh'), {'help' : ohelp, act : act_on}]
+    opts.append(opt)
+
+    default = conf.soname_comp
+    ohelp = 'When soname rebuilds: never, newer, keep or major/minor/last etc (default {default})'
+    opt = [('-so-comp', '--soname-comp'), {'help' : ohelp, act : act_on, 'default' : default}]
+    opts.append(opt)
+
+    ohelp = 'All remaining args after -- passed to makepkg'
+    opt = [('makepkg'), {'help' : ohelp, 'nargs' : '*'}]
+    opts.append(opt)
+
+    return opts

src/mkpkg/lib/class_mkpkg.py

@@ -16,8 +16,8 @@ class MkPkg
 from .tools import print_summary
 from .build import build
 from .dep_vers import write_current_pkg_dep_vers
-from .soname import write_current_pkg_dep_soname
-from .soname import pkginfo_soname_dep_info
+from .soname_deps import (write_soname_deps )
+from .soname import (get_current_soname_info )
 
 class MkPkg:
     """ MkPkg wrapper class """
@@ -44,11 +44,14 @@ def __init__(self):
         self.verb = self.conf.verb              # don't show normal makepkg output
         self.force = self.conf.force            # run makepkg even if not necessary
         self.refresh = self.conf.refresh        # refresh .mkpkg_dep_soname .mkpkg_dep_vers
-        self.soname_build = self.conf.soname_build
-        self.argv = self.conf.makepkg_args      # passed down to makepkg
-        self.use_makedepends = self.conf.use_makedepends  # deprecated
 
+        # soname_build ~ 'never', 'newer', <compare-how>
+        #  These compare using greater than: 'major' or 'minor' or 'last' etc
+        self.soname_comp = self.conf.soname_comp
         self.soname_info = {}
+        self.avail_soname_info = {}
+
+        self.argv = self.conf.makepkg_args      # passed down to makepkg
 
         self.cwd = os.getcwd()
         self.mymsg = GcMsg()
@@ -77,6 +80,6 @@ def build(self):
             # of any depenencies (including sonames)
             #
             write_current_pkg_dep_vers(self)
-            self.soname_info = pkginfo_soname_dep_info(self)
-            write_current_pkg_dep_soname(self)
+            self.soname_info = get_current_soname_info('pkg')
+            write_soname_deps(self)
         print_summary(self)

src/mkpkg/lib/dep_vers.py

@@ -158,7 +158,7 @@ def get_depends_versions(mkpkg, depends_vers):
             pkg_vers = dep_vers_now[pkg]
             last_vers = _get_pkg_dep_vers_last(mkpkg, pkg)
 
-            info = check_version_trigger(mkpkg, oper, vers_trigger, pkg_vers, last_vers)
+            info = check_version_trigger(mkpkg.msg, oper, vers_trigger, pkg_vers, last_vers)
             (trigger, pvers_comp, lvers_comp) = info
 
             this_one = [pkg, oper, vers_trigger, pvers_comp, lvers_comp, trigger]

src/mkpkg/lib/elf_utils.py

@@ -0,0 +1,62 @@
+# SPDX-License-Identifier: MIT
+# Copyright (c) 2022,2023 Gene C
+"""
+soname tools
+"""
+import os
+from elftools.elf.elffile import ELFFile
+from elftools.common.exceptions import ELFError
+from .run_prog import run_prog
+
+def file_is_elf(filename):
+    """
+    Return True if filename is elf executable
+    """
+    is_elf = False
+    if not os.path.exists(filename):
+        return is_elf
+
+    with open(filename, 'rb') as fob:
+        try:
+            elffile = ELFFile(fob)
+            header = elffile.header
+            e_ident = header.get('e_ident')
+            if e_ident:
+                ei_class = e_ident.get('EI_CLASS')
+                if ei_class and ei_class.startswith('ELFCLASS'):
+                    is_elf = True
+        except ELFError :
+            pass
+    return is_elf
+
+def sonames_in_elf_file(elf_file):
+    """
+    Extract list of sonames
+      eaech item is tuple (soname, vers, path)
+      e.g. (libssl.so, 56, /usr/lib/libressl.so.56)
+    """
+    sonames = []
+    if not os.path.exists(elf_file):
+        return sonames
+
+    pargs = ['/usr/bin/ldd', elf_file]
+    [retc, output, _errors] = run_prog(pargs)
+    if retc == 0 and output:
+        rows = output.splitlines()
+        for row in rows:
+            if 'linux-vdso.so' in row or  'ld-linux' in row:
+                continue
+
+            srow = row.strip().split()
+            if len(srow) < 3:
+                continue
+            soname = srow[0]
+            libname = srow[2]
+            vsplit = soname.split('.so.')
+            if len(vsplit) < 2:
+                continue
+
+            #item = (soname, vers, libname)
+            if libname not in sonames:
+                sonames.append(libname)
+    return sonames