Import Debian version 2.1.18+parrot1

apt (2.1.18+parrot1) rolling-testing; urgency=medium
.
  * import new Debian release.
.
apt (2.1.18) unstable; urgency=high
.
  * pkgcachegen: Avoid write to old cache for Version::Extra (Closes: #980037)
  * Adjust apt-mark test for dpkg 1.20.7
.
apt (2.1.17) unstable; urgency=medium
.
  [ Américo Monteiro ]
  * Portuguese manpages translation update (Closes: #979725)
.
  [ Julian Andres Klode ]
  * kernels: Fix std::out_of_range if no kernels to protect
  * Call ischroot with -t
.
apt (2.1.16) unstable; urgency=medium
.
  [ Faidon Liambotis ]
  * Various fixes to http and connect method
    - basehttp: also consider Access when a Server's URI
    - connect: convert a C-style string to std::string
    - connect: use ServiceNameOrPort, not Port, as the cache key
.
  [ Julian Andres Klode ]
  * patterns: Add dependency patterns ?depends, ?conflicts, etc.
    Note that the -broken- variants are not implemented yet.
  * Rewrite of the kernel autoremoval code:
    - Determine autoremovable kernels at run-time (LP: #1615381), this fixes the
      issue where apt could consider a running kernel autoremovable
    - Automatically remove unused kernels on apt {full,dist}-upgrade.
      This helps ensuring that we don't run out of /boot space.
    - Only keep up to 3 (not 4) kernels.
      Ubuntu boot partitions were sized for 3 kernels, not 4.
  * Bump codenames to bullseye/hirsute and adjust -security codename for
    bullseye (Closes: #969932)
  * Ignore failures from immediate configuration. This does not change the
    actual installation ordering - we never passed the return code to the
    caller and installation went underway anyway if it could be ordered at a
    later stage, this just removes spurious after-the-fact errors.
    (Closes: #973305, #188161, #211075, #649588) (LP: #1871268)
  * Add support for Phased-Update-Percentage, previously used only by
    update-manager.
  * Implement update --error-on=any so that scripts can reliably check for
    transient failures as well. (Closes: #594813)
.
  [ Demi M. Obenour ]
  * test/integration/framework: Be compatible with Bash
.
  [ Vangelis Skarmoutsos ]
  * Greek program translation update
.
apt (2.1.15) unstable; urgency=medium
.
  [ Julian Andres Klode ]
  * Unroll pkgCache::sHash 8 time, break up dependency
  * Do not require libxxhash-dev for including pkgcachegen.h (Closes: #978171)
.
  [ David Kalnischkies ]
  * Proper URI encoding for config requests to our test webserver
  * Keep URIs encoded in the acquire system
  * Implement encoded URI handling in all methods
  * Don't re-encode encoded URIs in pkgAcqFile
.
  [ Helge Kreutzmann ]
  * German program translation update (Closes: #977938)
.
apt (2.1.14) unstable; urgency=medium
.
  * test: fixup for hash table size increase (changed output order)
  * Use XXH3 for cache, hash table hashing
.
apt (2.1.13) unstable; urgency=medium
.
  [ Debian Janitor ]
  * Apply multi-arch hints.
    + apt-doc, libapt-pkg-doc: Add Multi-Arch: foreign.
.
  [ Jordi Mallach ]
  * Fix typo in Catalan translation.
.
  [ David Kalnischkies ]
  * Prepare rred binary for external usage
  * Support reading compressed patches in rred direct call modes
  * Support compressed output from rred similar to apt-helper cat-file
.
  [ Julian Andres Klode ]
  * gitignore: Add /build and /obj-* build dirs
  * gitignore: Add .*.swp files
  * HexDigest: Silence -Wstringop-overflow
  * patterns: Terminate short pattern by ~ and !
  * SECURITY UPDATE: Integer overflow in parsing (LP: #1899193)
    - apt-pkg/contrib/arfile.cc: add extra checks.
    - apt-pkg/contrib/tarfile.cc: limit tar item sizes to 128 GiB
    - apt-pkg/deb/debfile.cc: limit control file sizes to 64 MiB
    - test/*: add tests.
    - CVE-2020-27350
  * Additional hardening:
    - apt-pkg/contrib/tarfile.cc: Limit size of long names and links to 1 MiB
  * Raise APT::Cache-HashtableSize to 196613
.
apt (2.1.12) unstable; urgency=medium
.
  [ Julian Andres Klode ]
  * pkgnames: Correctly set the default for AllNames to false (LP: #1876495)
  * pkgnames: Do not exclude virtual packages with --all-names
  * Remove expired domain that became nsfw from debian/changelog
  * Do not immediately configure m-a: same packages in lockstep (LP: #1871268)
.
  [ Américo Monteiro ]
  * Portuguese manpages translation update (Closes: #968414)
.
  [ David Kalnischkies ]
  * Rename CMake find_package helpers to avoid developer warnings
  * Install translated apt-patterns(7) man pages
  * Remove ancient versions support from apts postinst
  * Update libapt-pkg6.0 symbols file
  * Refresh lintian-overrides of apt and libapt-pkg-doc

CMake/FindBerkeleyDB.cmake → CMake/FindBerkeley.cmake

 # - Try to find Berkeley DB
 # Once done this will define
 #
-#  BERKELEY_DB_FOUND - system has Berkeley DB
-#  BERKELEY_DB_INCLUDE_DIRS - the Berkeley DB include directory
-#  BERKELEY_DB_LIBRARIES - Link these to use Berkeley DB
-#  BERKELEY_DB_DEFINITIONS - Compiler switches required for using Berkeley DB
+#  BERKELEY_FOUND - system has Berkeley DB
+#  BERKELEY_INCLUDE_DIRS - the Berkeley DB include directory
+#  BERKELEY_LIBRARIES - Link these to use Berkeley DB
+#  BERKELEY_DEFINITIONS - Compiler switches required for using Berkeley DB
 
 # Copyright (c) 2006, Alexander Dymo, <adymo@kdevelop.org>
 # Copyright (c) 2016, Julian Andres Klode <jak@debian.org>
@@ -35,7 +35,7 @@
 
 # We need NO_DEFAULT_PATH here, otherwise CMake helpfully picks up the wrong
 # db.h on BSD systems instead of the Berkeley DB one.
-find_path(BERKELEY_DB_INCLUDE_DIRS db.h
+find_path(BERKELEY_INCLUDE_DIRS db.h
   ${CMAKE_INSTALL_FULL_INCLUDEDIR}/db5
   /usr/local/include/db5
   /usr/include/db5
@@ -51,9 +51,9 @@ find_path(BERKELEY_DB_INCLUDE_DIRS db.h
   NO_DEFAULT_PATH
 )
 
-find_library(BERKELEY_DB_LIBRARIES NAMES db db-5)
+find_library(BERKELEY_LIBRARIES NAMES db db-5)
 
 include(FindPackageHandleStandardArgs)
-find_package_handle_standard_args(Berkeley "Could not find Berkeley DB >= 4.1" BERKELEY_DB_INCLUDE_DIRS BERKELEY_DB_LIBRARIES)
-# show the BERKELEY_DB_INCLUDE_DIRS and BERKELEY_DB_LIBRARIES variables only in the advanced view
-mark_as_advanced(BERKELEY_DB_INCLUDE_DIRS BERKELEY_DB_LIBRARIES)
+find_package_handle_standard_args(Berkeley "Could not find Berkeley DB >= 4.1" BERKELEY_INCLUDE_DIRS BERKELEY_LIBRARIES)
+# show the BERKELEY_INCLUDE_DIRS and BERKELEY_LIBRARIES variables only in the advanced view
+mark_as_advanced(BERKELEY_INCLUDE_DIRS BERKELEY_LIBRARIES)

CMake/FindXXHASH.cmake

+# - Try to find XXHASH
+# Once done, this will define
+#
+#  XXHASH_FOUND - system has XXHASH
+#  XXHASH_INCLUDE_DIRS - the XXHASH include directories
+#  XXHASH_LIBRARIES - the XXHASH library
+find_package(PkgConfig)
+
+pkg_check_modules(XXHASH_PKGCONF libxxhash)
+
+find_path(XXHASH_INCLUDE_DIRS
+  NAMES xxhash.h
+  PATHS ${XXHASH_PKGCONF_INCLUDE_DIRS}
+)
+
+
+find_library(XXHASH_LIBRARIES
+  NAMES xxhash
+  PATHS ${XXHASH_PKGCONF_LIBRARY_DIRS}
+)
+
+include(FindPackageHandleStandardArgs)
+find_package_handle_standard_args(XXHASH DEFAULT_MSG XXHASH_INCLUDE_DIRS XXHASH_LIBRARIES)
+
+mark_as_advanced(XXHASH_INCLUDE_DIRS XXHASH_LIBRARIES)

CMakeLists.txt

@@ -80,8 +80,8 @@ add_optional_compile_options(Wsuggest-override)
 add_optional_compile_options(Werror=suggest-override)
 add_optional_compile_options(Werror=return-type)
 # apt-ftparchive dependencies
-find_package(BerkeleyDB REQUIRED)
-if (BERKELEY_DB_FOUND)
+find_package(Berkeley REQUIRED)
+if (BERKELEY_FOUND)
   set(HAVE_BDB 1)
 endif()
 
@@ -113,7 +113,7 @@ if (LZ4_FOUND)
   set(HAVE_LZ4 1)
 endif()
 
-find_package(Zstd)
+find_package(ZSTD)
 if (ZSTD_FOUND)
   set(HAVE_ZSTD 1)
 endif()
@@ -129,12 +129,13 @@ if (SYSTEMD_FOUND)
   set(HAVE_SYSTEMD 1)
 endif()
 
-find_package(Seccomp)
+find_package(SECCOMP)
 if (SECCOMP_FOUND)
   set(HAVE_SECCOMP 1)
 endif()
 
-find_package(Gcrypt REQUIRED)
+find_package(GCRYPT REQUIRED)
+find_package(XXHASH REQUIRED)
 
 # Mount()ing and stat()ing and friends
 check_symbol_exists(statfs sys/vfs.h HAVE_VFS_H)
@@ -204,7 +205,7 @@ check_cxx_target(HAVE_FMV_SSE42_AND_CRC32DI "sse4.2" "__builtin_ia32_crc32di(0,i
 # Configure some variables like package, version and architecture.
 set(PACKAGE ${PROJECT_NAME})
 set(PACKAGE_MAIL "APT Development Team <deity@lists.debian.org>")
-set(PACKAGE_VERSION "2.1.11")
+set(PACKAGE_VERSION "2.1.18")
 string(REGEX MATCH "^[0-9.]+" PROJECT_VERSION ${PACKAGE_VERSION})
 
 if (NOT DEFINED DPKG_DATADIR)

apt-pkg/CMakeLists.txt

@@ -49,6 +49,7 @@ target_include_directories(apt-pkg
                                    $<$<BOOL:${SYSTEMD_FOUND}>:${SYSTEMD_INCLUDE_DIRS}>
                                    ${ICONV_INCLUDE_DIRS}
                                    $<$<BOOL:${GCRYPT_FOUND}>:${GCRYPT_INCLUDE_DIRS}>
+                                   $<$<BOOL:${XXHASH_FOUND}>:${XXHASH_INCLUDE_DIRS}>
 )
 
 target_link_libraries(apt-pkg
@@ -63,6 +64,7 @@ target_link_libraries(apt-pkg
                              $<$<BOOL:${SYSTEMD_FOUND}>:${SYSTEMD_LIBRARIES}>
                              ${ICONV_LIBRARIES}
                              $<$<BOOL:${GCRYPT_FOUND}>:${GCRYPT_LIBRARIES}>
+                             $<$<BOOL:${XXHASH_FOUND}>:${XXHASH_LIBRARIES}>
 )
 set_target_properties(apt-pkg PROPERTIES VERSION ${MAJOR}.${MINOR})
 set_target_properties(apt-pkg PROPERTIES SOVERSION ${MAJOR})

apt-pkg/acquire-item.cc

@@ -1380,7 +1380,7 @@ string pkgAcqMetaBase::Custom600Headers() const
 void pkgAcqMetaBase::QueueForSignatureVerify(pkgAcqTransactionItem * const I, std::string const &File, std::string const &Signature)
 {
    AuthPass = true;
-   I->Desc.URI = "gpgv:" + Signature;
+   I->Desc.URI = "gpgv:" + pkgAcquire::URIEncode(Signature);
    I->DestFile = File;
    QueueURI(I->Desc);
    I->SetActiveSubprocess("gpgv");
@@ -1415,7 +1415,7 @@ bool pkgAcqMetaBase::CheckDownloadDone(pkgAcqTransactionItem * const I, const st
    if (FileName != I->DestFile && RealFileExists(I->DestFile) == false)
    {
       I->Local = true;
-      I->Desc.URI = "copy:" + FileName;
+      I->Desc.URI = "copy:" + pkgAcquire::URIEncode(FileName);
       I->QueueURI(I->Desc);
       return false;
    }
@@ -2888,9 +2888,10 @@ bool pkgAcqIndexDiffs::QueueNextDiff()					/*{{{*/
    }
 
    // queue the right diff
-   Desc.URI = Target.URI + ".diff/" + available_patches[0].file + ".gz";
+   auto const BaseFileURI = Target.URI + ".diff/" + pkgAcquire::URIEncode(available_patches[0].file);
+   Desc.URI = BaseFileURI + ".gz";
    Desc.Description = Target.Description + " " + available_patches[0].file + string(".pdiff");
-   DestFile = GetKeepCompressedFileName(GetPartialFileNameFromURI(Target.URI + ".diff/" + available_patches[0].file), Target);
+   DestFile = GetKeepCompressedFileName(GetPartialFileNameFromURI(BaseFileURI), Target);
 
    if(Debug)
       std::clog << "pkgAcqIndexDiffs::QueueNextDiff(): " << Desc.URI << std::endl;
@@ -2923,7 +2924,7 @@ void pkgAcqIndexDiffs::Done(string const &Message, HashStringList const &Hashes,
 	    std::clog << "Sending to rred method: " << UnpatchedFile << std::endl;
 	 State = StateApplyDiff;
 	 Local = true;
-	 Desc.URI = "rred:" + UnpatchedFile;
+	 Desc.URI = "rred:" + pkgAcquire::URIEncode(UnpatchedFile);
 	 QueueURI(Desc);
 	 SetActiveSubprocess("rred");
 	 return;
@@ -2979,9 +2980,9 @@ pkgAcqIndexMergeDiffs::pkgAcqIndexMergeDiffs(pkgAcquire *const Owner,
 
    Desc.Owner = this;
    Desc.ShortDesc = Target.ShortDesc;
-   Desc.URI = Target.URI + ".diff/" + patch.file + ".gz";
+   Desc.URI = Target.URI + ".diff/" + pkgAcquire::URIEncode(patch.file) + ".gz";
    Desc.Description = Target.Description + " " + patch.file + ".pdiff";
-   DestFile = GetPartialFileNameFromURI(Target.URI + ".diff/" + patch.file + ".gz");
+   DestFile = GetPartialFileNameFromURI(Desc.URI);
 
    if(Debug)
       std::clog << "pkgAcqIndexMergeDiffs: " << Desc.URI << std::endl;
@@ -3068,7 +3069,7 @@ void pkgAcqIndexMergeDiffs::Done(string const &Message, HashStringList const &Ha
 	    std::clog << "Sending to rred method: " << UnpatchedFile << std::endl;
 	 State = StateApplyDiff;
 	 Local = true;
-	 Desc.URI = "rred:" + UnpatchedFile;
+	 Desc.URI = "rred:" + pkgAcquire::URIEncode(UnpatchedFile);
 	 QueueURI(Desc);
 	 SetActiveSubprocess("rred");
 	 return;
@@ -3270,7 +3271,7 @@ void pkgAcqIndex::StageDownloadDone(string const &Message)
       if (symlink(Filename.c_str(), DestFile.c_str()) != 0)
 	 _error->WarningE("pkgAcqIndex::StageDownloadDone", "Symlinking file %s to %s failed", Filename.c_str(), DestFile.c_str());
       Stage = STAGE_DECOMPRESS_AND_VERIFY;
-      Desc.URI = "store:" + DestFile;
+      Desc.URI = "store:" + pkgAcquire::URIEncode(DestFile);
       QueueURI(Desc);
       SetActiveSubprocess(::URI(Desc.URI).Access);
       return;
@@ -3299,9 +3300,9 @@ void pkgAcqIndex::StageDownloadDone(string const &Message)
    Stage = STAGE_DECOMPRESS_AND_VERIFY;
    DestFile = GetKeepCompressedFileName(GetPartialFileNameFromURI(Target.URI), Target);
    if (Filename != DestFile && flExtension(Filename) == flExtension(DestFile))
-      Desc.URI = "copy:" + Filename;
+      Desc.URI = "copy:" + pkgAcquire::URIEncode(Filename);
    else
-      Desc.URI = "store:" + Filename;
+      Desc.URI = "store:" + pkgAcquire::URIEncode(Filename);
    if (DestFile == Filename)
    {
       if (CurrentCompressionExtension == "uncompressed")
@@ -3627,7 +3628,7 @@ void pkgAcqChangelog::Init(std::string const &DestDir, std::string const &DestFi
 	 DestFile = SrcName + ".changelog";
       else
 	 DestFile = DestFilename;
-      Desc.URI = "changelog:/" + DestFile;
+      Desc.URI = "changelog:/" + pkgAcquire::URIEncode(DestFile);
       return;
    }
 
@@ -3792,13 +3793,13 @@ std::string pkgAcqChangelog::URI(std::string const &Template,
       return "";
 
    // the path is: COMPONENT/SRC/SRCNAME/SRCNAME_SRCVER, e.g. main/a/apt/apt_1.1 or contrib/liba/libapt/libapt_2.0
-   std::string Src = SrcName;
-   std::string path = APT::String::Startswith(SrcName, "lib") ? Src.substr(0, 4) : Src.substr(0,1);
-   path.append("/").append(Src).append("/");
-   path.append(Src).append("_").append(StripEpoch(SrcVersion));
+   std::string const Src{SrcName};
+   std::string path = pkgAcquire::URIEncode(APT::String::Startswith(SrcName, "lib") ? Src.substr(0, 4) : Src.substr(0,1));
+   path.append("/").append(pkgAcquire::URIEncode(Src)).append("/");
+   path.append(pkgAcquire::URIEncode(Src)).append("_").append(pkgAcquire::URIEncode(StripEpoch(SrcVersion)));
    // we omit component for releases without one (= flat-style repositories)
    if (Component != NULL && strlen(Component) != 0)
-      path = std::string(Component) + "/" + path;
+      path = pkgAcquire::URIEncode(Component) + "/" + path;
 
    return SubstVar(Template, "@CHANGEPATH@", path);
 }
@@ -3858,8 +3859,12 @@ pkgAcqFile::pkgAcqFile(pkgAcquire *const Owner, string const &URI, HashStringLis
    else
       DestFile = flNotDir(URI);
 
+   ::URI url{URI};
+   if (url.Path.find(' ') != std::string::npos || url.Path.find('%') == std::string::npos)
+      url.Path = pkgAcquire::URIEncode(url.Path);
+
    // Create the item
-   Desc.URI = URI;
+   Desc.URI = std::string(url);
    Desc.Description = Dsc;
    Desc.Owner = this;
 
@@ -3901,7 +3906,7 @@ void pkgAcqFile::Done(string const &Message,HashStringList const &CalcHashes,
       if (_config->FindB("Acquire::Source-Symlinks",true) == false ||
 	  Cnf->Removable == true)
       {
-	 Desc.URI = "copy:" + FileName;
+	 Desc.URI = "copy:" + pkgAcquire::URIEncode(FileName);
 	 QueueURI(Desc);
 	 return;
       }

apt-pkg/acquire-method.cc

@@ -75,6 +75,9 @@ pkgAcqMethod::pkgAcqMethod(const char *Ver,unsigned long Flags)
    if ((Flags & AuxRequests) == AuxRequests)
       try_emplace(fields, "AuxRequests", "true");
 
+   if ((Flags & SendURIEncoded) == SendURIEncoded)
+      try_emplace(fields, "Send-URI-Encoded", "true");
+
    SendMessage("100 Capabilities", std::move(fields));
 
    SetNonBlock(STDIN_FILENO,true);

apt-pkg/acquire-method.h

@@ -112,7 +112,8 @@ class APT_PUBLIC pkgAcqMethod
       LocalOnly = (1 << 3),
       NeedsCleanup = (1 << 4),
       Removable = (1 << 5),
-      AuxRequests = (1 << 6)
+      AuxRequests = (1 << 6),
+      SendURIEncoded = (1 << 7),
    };
 
    void Log(const char *Format,...);

apt-pkg/acquire-worker.cc

@@ -307,8 +307,17 @@ bool pkgAcquire::Worker::RunMessages()
                break;
             }
 
-	    std::string const NewURI = LookupTag(Message,"New-URI",URI.c_str());
-            Itm->URI = NewURI;
+	    std::string const GotNewURI = LookupTag(Message,"New-URI",URI.c_str());
+	    if (Config->GetSendURIEncoded())
+	       Itm->URI = GotNewURI;
+	    else
+	    {
+	       ::URI tmpuri{GotNewURI};
+	       tmpuri.Path = pkgAcquire::URIEncode(tmpuri.Path);
+	       Itm->URI = tmpuri;
+	    }
+	    auto NewURI = Itm->URI;
+
 	    auto const AltUris = VectorizeString(LookupTag(Message, "Alternate-URIs"), '\n');
 
 	    ItemDone();
@@ -323,18 +332,37 @@ bool pkgAcquire::Worker::RunMessages()
 	    Itm = nullptr;
 	    for (auto const &Owner: ItmOwners)
 	    {
-	       for (auto alt = AltUris.crbegin(); alt != AltUris.crend(); ++alt)
-		  Owner->PushAlternativeURI(std::string(*alt), {}, false);
-
 	       pkgAcquire::ItemDesc &desc = Owner->GetItemDesc();
+
 	       // for a simplified retry a method might redirect without URI change
 	       // see also IsRedirectionLoop implementation
-	       if (desc.URI != NewURI)
+	       bool simpleRetry = false;
+	       if (Config->GetSendURIEncoded())
 	       {
-		  auto newuri = NewURI;
-		  if (Owner->IsGoodAlternativeURI(newuri) == false && Owner->PopAlternativeURI(newuri) == false)
-		     newuri.clear();
-		  if (newuri.empty() || Owner->IsRedirectionLoop(newuri))
+		  for (auto alt = AltUris.crbegin(); alt != AltUris.crend(); ++alt)
+		     Owner->PushAlternativeURI(std::string(*alt), {}, false);
+		  if (desc.URI == GotNewURI)
+		     simpleRetry = true;
+	       }
+	       else
+	       {
+		  for (auto alt = AltUris.crbegin(); alt != AltUris.crend(); ++alt)
+		  {
+		     ::URI tmpuri{*alt};
+		     tmpuri.Path = pkgAcquire::URIEncode(tmpuri.Path);
+		     Owner->PushAlternativeURI(std::string(tmpuri), {}, false);
+		  }
+		  ::URI tmpuri{desc.URI};
+		  tmpuri.Path = DeQuoteString(tmpuri.Path);
+		  if (GotNewURI == std::string(tmpuri))
+		     simpleRetry = true;
+	       }
+
+	       if (not simpleRetry)
+	       {
+		  if (Owner->IsGoodAlternativeURI(NewURI) == false && Owner->PopAlternativeURI(NewURI) == false)
+		     NewURI.clear();
+		  if (NewURI.empty() || Owner->IsRedirectionLoop(NewURI))
 		  {
 		     std::string msg = Message;
 		     msg.append("\nFailReason: RedirectionLoop");
@@ -665,12 +693,18 @@ bool pkgAcquire::Worker::Capabilities(string Message)
    Config->NeedsCleanup = StringToBool(LookupTag(Message,"Needs-Cleanup"),false);
    Config->Removable = StringToBool(LookupTag(Message,"Removable"),false);
    Config->SetAuxRequests(StringToBool(LookupTag(Message, "AuxRequests"), false));
+   if (_config->FindB("Acquire::Send-URI-Encoded", true))
+      Config->SetSendURIEncoded(StringToBool(LookupTag(Message, "Send-URI-Encoded"), false));
 
    // Some debug text
    if (Debug == true)
    {
       clog << "Configured access method " << Config->Access << endl;
-      clog << "Version:" << Config->Version << " SingleInstance:" << Config->SingleInstance << " Pipeline:" << Config->Pipeline << " SendConfig:" << Config->SendConfig << " LocalOnly: " << Config->LocalOnly << " NeedsCleanup: " << Config->NeedsCleanup << " Removable: " << Config->Removable << " AuxRequests: " << Config->GetAuxRequests() << endl;
+      clog << "Version:" << Config->Version << " SingleInstance:" << Config->SingleInstance
+	   << " Pipeline:" << Config->Pipeline << " SendConfig:" << Config->SendConfig
+	   << " LocalOnly: " << Config->LocalOnly << " NeedsCleanup: " << Config->NeedsCleanup
+	   << " Removable: " << Config->Removable << " AuxRequests: " << Config->GetAuxRequests()
+	   << " SendURIEncoded: " << Config->GetSendURIEncoded() << '\n';
    }
 
    return true;
@@ -737,6 +771,8 @@ bool pkgAcquire::Worker::SendConfiguration()
       configuration tree */
    std::ostringstream Message;
    Message << "601 Configuration\n";
+   if (not _config->Exists("Acquire::Send-URI-Encoded"))
+      Message << "Config-Item: Acquire::Send-URI-Encoded=1\n";
    _config->Dump(Message, NULL, "Config-Item: %F=%V\n", false);
    Message << '\n';
 
@@ -763,10 +799,16 @@ bool pkgAcquire::Worker::QueueItem(pkgAcquire::Queue::QItem *Item)
 
    string Message = "600 URI Acquire\n";
    Message.reserve(300);
-   Message += "URI: " + Item->URI;
+   URI URL(Item->URI);
+   if (Config->GetSendURIEncoded())
+      Message += "URI: " + Item->URI;
+   else
+   {
+      URL.Path = DeQuoteString(URL.Path);
+      Message += "URI: " + std::string(URL);
+   }
    Message += "\nFilename: " + Item->Owner->DestFile;
 
-   URI URL(Item->URI);
    // FIXME: We should not hard code proxy protocols here.
    if (URL.Access == "http" || URL.Access == "https")
    {

apt-pkg/acquire.cc

@@ -51,6 +51,12 @@
 
 using namespace std;
 
+std::string pkgAcquire::URIEncode(std::string const &part)		/*{{{*/
+{
+   // The "+" is encoded as a workaround for an S3 bug (LP#1003633 and LP#1086997)
+   return QuoteString(part, _config->Find("Acquire::URIEncode", "+~ ").c_str());
+}
+									/*}}}*/
 // Acquire::pkgAcquire - Constructor					/*{{{*/
 // ---------------------------------------------------------------------
 /* We grab some runtime state from the configuration space */
@@ -662,10 +668,11 @@ static void CheckDropPrivsMustBeDisabled(pkgAcquire const &Fetcher)
 	 if (_config->Exists(conf) == true)
 	    continue;
 
-	 if (IsAccessibleBySandboxUser(source.Path, false) == false)
+	 auto const filepath = DeQuoteString(source.Path);
+	 if (not IsAccessibleBySandboxUser(filepath, false))
 	 {
 	    _error->NoticeE("pkgAcquire::Run", _("Download is performed unsandboxed as root as file '%s' couldn't be accessed by user '%s'."),
-		  source.Path.c_str(), SandboxUser.c_str());
+			    filepath.c_str(), SandboxUser.c_str());
 	    _config->CndSet("Binary::file::APT::Sandbox::User", "root");
 	    _config->CndSet("Binary::copy::APT::Sandbox::User", "root");
 	 }
@@ -880,6 +887,7 @@ class pkgAcquire::MethodConfig::Private
 {
    public:
    bool AuxRequests = false;
+   bool SendURIEncoded = false;
 };
 pkgAcquire::MethodConfig::MethodConfig() : d(new Private()), Next(0), SingleInstance(false),
 					   Pipeline(false), SendConfig(false), LocalOnly(false), NeedsCleanup(false),
@@ -897,6 +905,17 @@ void pkgAcquire::MethodConfig::SetAuxRequests(bool const value) /*{{{*/
    d->AuxRequests = value;
 }
 									/*}}}*/
+bool pkgAcquire::MethodConfig::GetSendURIEncoded() const		/*{{{*/
+{
+   return d->SendURIEncoded;
+}
+									/*}}}*/
+void pkgAcquire::MethodConfig::SetSendURIEncoded(bool const value)	/*{{{*/
+{
+   d->SendURIEncoded = value;
+}
+									/*}}}*/
+
 // Queue::Queue - Constructor						/*{{{*/
 // ---------------------------------------------------------------------
 /* */
@@ -1073,10 +1092,25 @@ bool pkgAcquire::Queue::Shutdown(bool Final)
 /* */
 pkgAcquire::Queue::QItem *pkgAcquire::Queue::FindItem(string URI,pkgAcquire::Worker *Owner)
 {
-   for (QItem *I = Items; I != 0; I = I->Next)
-      if (I->URI == URI && I->Worker == Owner)
-	 return I;
-   return 0;
+   if (Owner->Config->GetSendURIEncoded())
+   {
+      for (QItem *I = Items; I != nullptr; I = I->Next)
+	 if (I->URI == URI && I->Worker == Owner)
+	    return I;
+   }
+   else
+   {
+      for (QItem *I = Items; I != nullptr; I = I->Next)
+      {
+	 if (I->Worker != Owner)
+	    continue;
+	 ::URI tmpuri{I->URI};
+	 tmpuri.Path = DeQuoteString(tmpuri.Path);
+	 if (URI == std::string(tmpuri))
+	    return I;
+      }
+   }
+   return nullptr;
 }
 									/*}}}*/
 // Queue::ItemDone - Item has been completed				/*{{{*/

apt-pkg/acquire.h

@@ -363,6 +363,8 @@ class APT_PUBLIC pkgAcquire
     */
    virtual ~pkgAcquire();
 
+   APT_HIDDEN static std::string URIEncode(std::string const &part);
+
    private:
    APT_HIDDEN void Initialize();
 };
@@ -680,6 +682,8 @@ struct APT_PUBLIC pkgAcquire::MethodConfig
 
    APT_HIDDEN bool GetAuxRequests() const;
    APT_HIDDEN void SetAuxRequests(bool const value);
+   APT_HIDDEN bool GetSendURIEncoded() const;
+   APT_HIDDEN void SetSendURIEncoded(bool const value);
 
    virtual ~MethodConfig();
 };

apt-pkg/algorithms.cc

@@ -16,20 +16,32 @@
 #include <config.h>
 
 #include <apt-pkg/algorithms.h>
+#include <apt-pkg/cachefilter.h>
 #include <apt-pkg/configuration.h>
 #include <apt-pkg/depcache.h>
 #include <apt-pkg/dpkgpm.h>
 #include <apt-pkg/edsp.h>
 #include <apt-pkg/error.h>
+#include <apt-pkg/macros.h>
 #include <apt-pkg/packagemanager.h>
 #include <apt-pkg/pkgcache.h>
+#include <apt-pkg/string_view.h>
+#include <apt-pkg/strutl.h>
+#include <apt-pkg/version.h>
+
 #include <apt-pkg/prettyprinters.h>
 
 #include <cstdlib>
 #include <iostream>
+#include <map>
+#include <regex>
+#include <set>
+#include <sstream>
 #include <string>
 #include <utility>
+#include <vector>
 #include <string.h>
+#include <sys/utsname.h>
 
 #include <apti18n.h>
 									/*}}}*/
@@ -1442,3 +1454,179 @@ void pkgPrioSortList(pkgCache &Cache,pkgCache::Version **List)
    std::sort(List,List+Count,PrioComp(Cache));
 }
 									/*}}}*/
+
+namespace APT
+{
+
+namespace KernelAutoRemoveHelper
+{
+
+// \brief Returns the uname from a kernel package name, or "" for non-kernel packages.
+std::string getUname(std::string const &packageName)
+{
+
+   static const constexpr char *const prefixes[] = {
+      "linux-image-",
+      "kfreebsd-image-",
+      "gnumach-image-",
+   };
+
+   for (auto prefix : prefixes)
+   {
+      if (likely(not APT::String::Startswith(packageName, prefix)))
+	 continue;
+      if (unlikely(APT::String::Endswith(packageName, "-dbgsym")))
+	 continue;
+      if (unlikely(APT::String::Endswith(packageName, "-dbg")))
+	 continue;
+
+      auto aUname = packageName.substr(strlen(prefix));
+
+      // aUname must start with [0-9]+\.
+      if (aUname.length() < 2)
+	 continue;
+      if (strchr("0123456789", aUname[0]) == nullptr)
+	 continue;
+      auto dot = aUname.find_first_not_of("0123456789");
+      if (dot == aUname.npos || aUname[dot] != '.')
+	 continue;
+
+      return aUname;
+   }
+
+   return "";
+}
+std::string GetProtectedKernelsRegex(pkgCache *cache, bool ReturnRemove)
+{
+   if (_config->FindB("APT::Protect-Kernels", true) == false)
+      return "";
+
+   struct CompareKernel
+   {
+      pkgCache *cache;
+      bool operator()(const std::string &a, const std::string &b) const
+      {
+	 return cache->VS->CmpVersion(a, b) < 0;
+      }
+   };
+   bool Debug = _config->FindB("Debug::pkgAutoRemove", false);
+   // kernel version -> list of unames
+   std::map<std::string, std::vector<std::string>, CompareKernel> version2unames(CompareKernel{cache});
+   // needs to be initialized to 0s, might not be set up.
+   utsname uts{};
+   std::string bootedVersion;
+   std::string lastInstalledVersion;
+
+   std::string lastInstalledUname = _config->Find("APT::LastInstalledKernel");
+
+   // Get currently booted version, but only when not on reproducible build.
+   if (getenv("SOURCE_DATE_EPOCH") == 0)
+   {
+      if (uname(&uts) != 0)
+	 abort();
+   }
+
+   auto VirtualKernelPkg = cache->FindPkg("$kernel", "any");
+   if (VirtualKernelPkg.end())
+      return "";