UTF-8 issue during reposync
Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan
Hi Stefan. Could you please run spacewalk-repo-sync for this channel with --fail or -f command line parameters appended? It should write a traceback on the error. And what is the URL for the repo? Victor On Thu, 2021-12-23 at 08:00 +0100, Stefan Bluhm wrote: Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan
Hello Victor, URL used is [ http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os | http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os ] Let me try with -f. Will report back in a while. Am also trying to take the code apart now. Best wishes, Stefan Von: "Victor Zhestkov" <Victor.Zhestkov@suse.com> An: "opensuse org" <opensuse.org@bluhm-de.com>, "uyuni-users" <uyuni-users@opensuse.org> Gesendet: Donnerstag, 23. Dezember 2021 08:16:03 Betreff: Re: UTF-8 issue during reposync Hi Stefan. Could you please run spacewalk-repo-sync for this channel with --fail or -f command line parameters appended? It should write a traceback on the error. And what is the URL for the repo? Victor On Thu, 2021-12-23 at 08:00 +0100, Stefan Bluhm wrote: Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan
Actually was quicker than expected. Also includes a custom line with the file path. Looks interesting... 08:22:03 Package batch #382 of 480 completed... 08:22:03 Package batch #383 of 480 completed... 08:22:03 Package batch #384 of 480 completed... 08:22:03 Package batch #385 of 480 completed... 08:22:03 Package batch #386 of 480 completed... packages/1/ffc/autogen/5.18-5.el7/x86_64/ffc02110cf6bd3a97180ae4c8ce7af9f261fa2d23a489a22ad09fb9902ede70e/autogen-5.18-5.el7.x86_64.rpm packages/1/f03/compat-libcap1/1.10-7.el7/x86_64/f03a3843b53fc9eba490760caefdd79679b1043b67bfccc784e5d326395fc887/compat-libcap1-1.10-7.el7.x86_64.rpm packages/1/275/expat-devel/2.1.0-12.el7/i686/275fe3cbc73d662f72c3244222c2329caa68e4aaf7d107ef009657dd06fb6e92/expat-devel-2.1.0-12.el7.i686.rpm packages/1/745/compat-libcap1/1.10-7.el7/i686/7453914fde0bec5a7849548d54f0f922da541493b2e9f2e6a1be19f57956364b/compat-libcap1-1.10-7.el7.i686.rpm packages/1/c73/expat/2.1.0-12.el7/x86_64/c73d74dee6c977bda30983111a290f054a8301797f32a20b550b4242679f057e/expat-2.1.0-12.el7.x86_64.rpm 08:22:03 Package batch #389 of 480 completed... packages/1/18e/glib2-devel/2.56.1-7.el7/i686/18e10901a5adadad23b729b4ae13c28a6cc19e238237d69bc1bbd087aba4e5bf/glib2-devel-2.56.1-7.el7.i686.rpm packages/1/6de/glib2/2.56.1-7.el7/x86_64/6dea72243379ae62c699ac7311cd5e8020d21bce162640046e8c4edc7bb270c3/glib2-2.56.1-7.el7.x86_64.rpm packages/1/199/gtk2-devel-docs/2.24.31-1.el7/x86_64/19964d905f10899d0454ae9f1b5bb3efe69e2f331920b756ad10ab081cbb027f/gtk2-devel-docs-2.24.31-1.el7.x86_64.rpm packages/1/0d5/gtk2-devel/2.24.31-1.el7/x86_64/0d5303d89e65fa5a6231ed9740f1aa64621bfeff5bdfec01ccc3588c5bc74553/gtk2-devel-2.24.31-1.el7.x86_64.rpm packages/1/045/intltool/0.50.2-7.el7/noarch/0451aa0d15dda258f0e852888d618dac270718042a215f10f58d2999ec1194e5/intltool-0.50.2-7.el7.noarch.rpm packages/1/458/iok/2.1.3-6.el7/x86_64/458f2b61c6f95964027ed7cff1e52a2b92949a5de234887058efc75a47686420/iok-2.1.3-6.el7.x86_64.rpm packages/1/69b/kdepimlibs-kxmlrpcclient/4.10.5-4.el7/i686/69b2339a003256e48508d16902874b3e247d8f6cbb1f55239cb383ff22aab510/kdepimlibs-kxmlrpcclient-4.10.5-4.el7.i686.rpm packages/1/a36/kdepimlibs-devel/4.10.5-4.el7/x86_64/a36a2a72fcb2335541220a75e09b9d74be7f877fb3475380cdf756e608426b15/kdepimlibs-devel-4.10.5-4.el7.x86_64.rpm 08:22:03 Package batch #390 of 480 completed... 08:22:03 Package batch #391 of 480 completed... 08:22:03 Package batch #392 of 480 completed... 08:22:03 Package batch #393 of 480 completed... 08:22:03 Package batch #394 of 480 completed... 08:22:03 Package batch #395 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #396 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #397 of 480 completed... 08:22:04 Package batch #398 of 480 completed... 08:22:04 Package batch #399 of 480 completed... 08:22:04 Unexpected error: <class 'UnicodeEncodeError'> 08:22:04 multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1293, in import_package_batch importer.run() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/importLib.py", line 777, in run self.fix() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/packageImport.py", line 337, in fix self.backend.processChangeLog(self.changelog_data) File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/backend.py", line 171, in processChangeLog h.execute_values(sql, values, fetch=False) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 166, in execute_values return self._execute_wrapper(self._execute_values, sql, argslist, template, page_size, fetch) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 299, in _execute_wrapper retval = function(*p, **kw) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 353, in _execute_values results = psycopg2.extras.execute_values(self._real_cursor, sql, argslist, template=template, page_size=page_size, fetch=fetch) File "/usr/lib64/python3.6/site-packages/psycopg2/extras.py", line 1267, in execute_values parts.append(cur.mogrify(template, args)) UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 598, in sync ret = self.import_packages(plugin, data['id'], url, is_non_local_repo) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1113, in import_packages affected_channels_batch, failed_packages_batch, all_packages, processed_batch = result.get() File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get raise self._value UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:04 Total time: 0:00:32 Von: "Victor Zhestkov" <Victor.Zhestkov@suse.com> An: "opensuse org" <opensuse.org@bluhm-de.com>, "uyuni-users" <uyuni-users@opensuse.org> Gesendet: Donnerstag, 23. Dezember 2021 08:16:03 Betreff: Re: UTF-8 issue during reposync Hi Stefan. Could you please run spacewalk-repo-sync for this channel with --fail or -f command line parameters appended? It should write a traceback on the error. And what is the URL for the repo? Victor On Thu, 2021-12-23 at 08:00 +0100, Stefan Bluhm wrote: Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan
Stefan, feel free to open an issue for it. Will try to take care of it. Victor On Thu, 2021-12-23 at 08:26 +0100, Stefan Bluhm wrote: Actually was quicker than expected. Also includes a custom line with the file path. Looks interesting... 08:22:03 Package batch #382 of 480 completed... 08:22:03 Package batch #383 of 480 completed... 08:22:03 Package batch #384 of 480 completed... 08:22:03 Package batch #385 of 480 completed... 08:22:03 Package batch #386 of 480 completed... packages/1/ffc/autogen/5.18-5.el7/x86_64/ffc02110cf6bd3a97180ae4c8ce7af9f261fa2d23a489a22ad09fb9902ede70e/autogen-5.18-5.el7.x86_64.rpm packages/1/f03/compat-libcap1/1.10-7.el7/x86_64/f03a3843b53fc9eba490760caefdd79679b1043b67bfccc784e5d326395fc887/compat-libcap1-1.10-7.el7.x86_64.rpm packages/1/275/expat-devel/2.1.0-12.el7/i686/275fe3cbc73d662f72c3244222c2329caa68e4aaf7d107ef009657dd06fb6e92/expat-devel-2.1.0-12.el7.i686.rpm packages/1/745/compat-libcap1/1.10-7.el7/i686/7453914fde0bec5a7849548d54f0f922da541493b2e9f2e6a1be19f57956364b/compat-libcap1-1.10-7.el7.i686.rpm packages/1/c73/expat/2.1.0-12.el7/x86_64/c73d74dee6c977bda30983111a290f054a8301797f32a20b550b4242679f057e/expat-2.1.0-12.el7.x86_64.rpm 08:22:03 Package batch #389 of 480 completed... packages/1/18e/glib2-devel/2.56.1-7.el7/i686/18e10901a5adadad23b729b4ae13c28a6cc19e238237d69bc1bbd087aba4e5bf/glib2-devel-2.56.1-7.el7.i686.rpm packages/1/6de/glib2/2.56.1-7.el7/x86_64/6dea72243379ae62c699ac7311cd5e8020d21bce162640046e8c4edc7bb270c3/glib2-2.56.1-7.el7.x86_64.rpm packages/1/199/gtk2-devel-docs/2.24.31-1.el7/x86_64/19964d905f10899d0454ae9f1b5bb3efe69e2f331920b756ad10ab081cbb027f/gtk2-devel-docs-2.24.31-1.el7.x86_64.rpm packages/1/0d5/gtk2-devel/2.24.31-1.el7/x86_64/0d5303d89e65fa5a6231ed9740f1aa64621bfeff5bdfec01ccc3588c5bc74553/gtk2-devel-2.24.31-1.el7.x86_64.rpm packages/1/045/intltool/0.50.2-7.el7/noarch/0451aa0d15dda258f0e852888d618dac270718042a215f10f58d2999ec1194e5/intltool-0.50.2-7.el7.noarch.rpm packages/1/458/iok/2.1.3-6.el7/x86_64/458f2b61c6f95964027ed7cff1e52a2b92949a5de234887058efc75a47686420/iok-2.1.3-6.el7.x86_64.rpm packages/1/69b/kdepimlibs-kxmlrpcclient/4.10.5-4.el7/i686/69b2339a003256e48508d16902874b3e247d8f6cbb1f55239cb383ff22aab510/kdepimlibs-kxmlrpcclient-4.10.5-4.el7.i686.rpm packages/1/a36/kdepimlibs-devel/4.10.5-4.el7/x86_64/a36a2a72fcb2335541220a75e09b9d74be7f877fb3475380cdf756e608426b15/kdepimlibs-devel-4.10.5-4.el7.x86_64.rpm 08:22:03 Package batch #390 of 480 completed... 08:22:03 Package batch #391 of 480 completed... 08:22:03 Package batch #392 of 480 completed... 08:22:03 Package batch #393 of 480 completed... 08:22:03 Package batch #394 of 480 completed... 08:22:03 Package batch #395 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #396 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #397 of 480 completed... 08:22:04 Package batch #398 of 480 completed... 08:22:04 Package batch #399 of 480 completed... 08:22:04 Unexpected error: <class 'UnicodeEncodeError'> 08:22:04 multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1293, in import_package_batch importer.run() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/importLib.py", line 777, in run self.fix() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/packageImport.py", line 337, in fix self.backend.processChangeLog(self.changelog_data) File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/backend.py", line 171, in processChangeLog h.execute_values(sql, values, fetch=False) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 166, in execute_values return self._execute_wrapper(self._execute_values, sql, argslist, template, page_size, fetch) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 299, in _execute_wrapper retval = function(*p, **kw) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 353, in _execute_values results = psycopg2.extras.execute_values(self._real_cursor, sql, argslist, template=template, page_size=page_size, fetch=fetch) File "/usr/lib64/python3.6/site-packages/psycopg2/extras.py", line 1267, in execute_values parts.append(cur.mogrify(template, args)) UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 598, in sync ret = self.import_packages(plugin, data['id'], url, is_non_local_repo) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1113, in import_packages affected_channels_batch, failed_packages_batch, all_packages, processed_batch = result.get() File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get raise self._value UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:04 Total time: 0:00:32 ________________________________ Von: "Victor Zhestkov" <Victor.Zhestkov@suse.com> An: "opensuse org" <opensuse.org@bluhm-de.com>, "uyuni-users" <uyuni-users@opensuse.org> Gesendet: Donnerstag, 23. Dezember 2021 08:16:03 Betreff: Re: UTF-8 issue during reposync Hi Stefan. Could you please run spacewalk-repo-sync for this channel with --fail or -f command line parameters appended? It should write a traceback on the error. And what is the URL for the repo? Victor On Thu, 2021-12-23 at 08:00 +0100, Stefan Bluhm wrote: Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan
Thank you Victor, will try to dig deeper and open the issue later (as this is still easily reproducible for me). Best wishes, Stefan Von: "Victor Zhestkov" <Victor.Zhestkov@suse.com> An: "opensuse org" <opensuse.org@bluhm-de.com>, "uyuni-users" <uyuni-users@opensuse.org> Gesendet: Donnerstag, 23. Dezember 2021 08:37:16 Betreff: Re: UTF-8 issue during reposync Stefan, feel free to open an issue for it. Will try to take care of it. Victor On Thu, 2021-12-23 at 08:26 +0100, Stefan Bluhm wrote: Actually was quicker than expected. Also includes a custom line with the file path. Looks interesting... 08:22:03 Package batch #382 of 480 completed... 08:22:03 Package batch #383 of 480 completed... 08:22:03 Package batch #384 of 480 completed... 08:22:03 Package batch #385 of 480 completed... 08:22:03 Package batch #386 of 480 completed... packages/1/ffc/autogen/5.18-5.el7/x86_64/ffc02110cf6bd3a97180ae4c8ce7af9f261fa2d23a489a22ad09fb9902ede70e/autogen-5.18-5.el7.x86_64.rpm packages/1/f03/compat-libcap1/1.10-7.el7/x86_64/f03a3843b53fc9eba490760caefdd79679b1043b67bfccc784e5d326395fc887/compat-libcap1-1.10-7.el7.x86_64.rpm packages/1/275/expat-devel/2.1.0-12.el7/i686/275fe3cbc73d662f72c3244222c2329caa68e4aaf7d107ef009657dd06fb6e92/expat-devel-2.1.0-12.el7.i686.rpm packages/1/745/compat-libcap1/1.10-7.el7/i686/7453914fde0bec5a7849548d54f0f922da541493b2e9f2e6a1be19f57956364b/compat-libcap1-1.10-7.el7.i686.rpm packages/1/c73/expat/2.1.0-12.el7/x86_64/c73d74dee6c977bda30983111a290f054a8301797f32a20b550b4242679f057e/expat-2.1.0-12.el7.x86_64.rpm 08:22:03 Package batch #389 of 480 completed... packages/1/18e/glib2-devel/2.56.1-7.el7/i686/18e10901a5adadad23b729b4ae13c28a6cc19e238237d69bc1bbd087aba4e5bf/glib2-devel-2.56.1-7.el7.i686.rpm packages/1/6de/glib2/2.56.1-7.el7/x86_64/6dea72243379ae62c699ac7311cd5e8020d21bce162640046e8c4edc7bb270c3/glib2-2.56.1-7.el7.x86_64.rpm packages/1/199/gtk2-devel-docs/2.24.31-1.el7/x86_64/19964d905f10899d0454ae9f1b5bb3efe69e2f331920b756ad10ab081cbb027f/gtk2-devel-docs-2.24.31-1.el7.x86_64.rpm packages/1/0d5/gtk2-devel/2.24.31-1.el7/x86_64/0d5303d89e65fa5a6231ed9740f1aa64621bfeff5bdfec01ccc3588c5bc74553/gtk2-devel-2.24.31-1.el7.x86_64.rpm packages/1/045/intltool/0.50.2-7.el7/noarch/0451aa0d15dda258f0e852888d618dac270718042a215f10f58d2999ec1194e5/intltool-0.50.2-7.el7.noarch.rpm packages/1/458/iok/2.1.3-6.el7/x86_64/458f2b61c6f95964027ed7cff1e52a2b92949a5de234887058efc75a47686420/iok-2.1.3-6.el7.x86_64.rpm packages/1/69b/kdepimlibs-kxmlrpcclient/4.10.5-4.el7/i686/69b2339a003256e48508d16902874b3e247d8f6cbb1f55239cb383ff22aab510/kdepimlibs-kxmlrpcclient-4.10.5-4.el7.i686.rpm packages/1/a36/kdepimlibs-devel/4.10.5-4.el7/x86_64/a36a2a72fcb2335541220a75e09b9d74be7f877fb3475380cdf756e608426b15/kdepimlibs-devel-4.10.5-4.el7.x86_64.rpm 08:22:03 Package batch #390 of 480 completed... 08:22:03 Package batch #391 of 480 completed... 08:22:03 Package batch #392 of 480 completed... 08:22:03 Package batch #393 of 480 completed... 08:22:03 Package batch #394 of 480 completed... 08:22:03 Package batch #395 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #396 of 480 completed... 08:22:03 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:03 Package batch #397 of 480 completed... 08:22:04 Package batch #398 of 480 completed... 08:22:04 Package batch #399 of 480 completed... 08:22:04 Unexpected error: <class 'UnicodeEncodeError'> 08:22:04 multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1293, in import_package_batch importer.run() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/importLib.py", line 777, in run self.fix() File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/packageImport.py", line 337, in fix self.backend.processChangeLog(self.changelog_data) File "/usr/lib/python3.6/site-packages/spacewalk/server/importlib/backend.py", line 171, in processChangeLog h.execute_values(sql, values, fetch=False) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 166, in execute_values return self._execute_wrapper(self._execute_values, sql, argslist, template, page_size, fetch) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 299, in _execute_wrapper retval = function(*p, **kw) File "/usr/lib/python3.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 353, in _execute_values results = psycopg2.extras.execute_values(self._real_cursor, sql, argslist, template=template, page_size=page_size, fetch=fetch) File "/usr/lib64/python3.6/site-packages/psycopg2/extras.py", line 1267, in execute_values parts.append(cur.mogrify(template, args)) UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 598, in sync ret = self.import_packages(plugin, data['id'], url, is_non_local_repo) File "/usr/lib/python3.6/site-packages/spacewalk/satellite_tools/reposync.py", line 1113, in import_packages affected_channels_batch, failed_packages_batch, all_packages, processed_batch = result.get() File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get raise self._value UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 08:22:04 Total time: 0:00:32 Von: "Victor Zhestkov" <Victor.Zhestkov@suse.com> An: "opensuse org" <opensuse.org@bluhm-de.com>, "uyuni-users" <uyuni-users@opensuse.org> Gesendet: Donnerstag, 23. Dezember 2021 08:16:03 Betreff: Re: UTF-8 issue during reposync Hi Stefan. Could you please run spacewalk-repo-sync for this channel with --fail or -f command line parameters appended? It should write a traceback on the error. And what is the URL for the repo? Victor On Thu, 2021-12-23 at 08:00 +0100, Stefan Bluhm wrote: BQ_BEGIN Hello, whilst trying to do the initial CentOS 7 import into Uyuni 12.2021 on AlmaLinux 8, I am hitting the below reproducible error: 2021/12/23 07:52:50 +02:00 Package batch #471 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #472 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #473 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #474 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #475 of 480 completed... 2021/12/23 07:52:50 +02:00 Package batch #476 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #477 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #478 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #479 of 480 completed... 2021/12/23 07:52:51 +02:00 Package batch #480 of 480 completed... 2021/12/23 07:53:41 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:41 +02:00 Package batch #388 of 480 completed... 2021/12/23 07:53:45 +02:00 'utf-8' codec can't encode character '\udcf8' in position 19: surrogates not allowed 2021/12/23 07:53:45 +02:00 Package batch #387 of 480 completed... 2021/12/23 07:53:45 +02:00 Importing packages finished. 2021/12/23 07:53:45 +02:00 2021/12/23 07:53:45 +02:00 Linking packages to the channel. 2021/12/23 07:54:13 +02:00 Unexpected error: <class 'spacewalk.server.importlib.importLib.InvalidPackageError'> Do you have any experience what this means? Is it a DB/File System/Package or code issue (bug)? In the meantime, I will see if I can spot the error package/code location. Thank you and best wishes, Stefan BQ_END
participants (2)
-
Stefan Bluhm -
Victor Zhestkov