April 2018 - openSUSE Commits - openSUSE Mailing Lists

commit salt for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package salt for openSUSE:Factory checked in at 2018-04-30 22:58:17 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/salt (Old) and /work/SRC/openSUSE:Factory/.salt.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "salt" Mon Apr 30 22:58:17 2018 rev:75 rq:602524 version:2018.3.0 Changes: -------- --- /work/SRC/openSUSE:Factory/salt/salt.changes 2018-04-16 12:48:11.246861936 +0200 +++ /work/SRC/openSUSE:Factory/.salt.new/salt.changes 2018-04-30 22:59:33.495104654 +0200 @@ -1,0 +2,41 @@ +Wed Apr 25 14:50:36 UTC 2018 - Pablo Suárez Hernández <psuarezhernandez(a)suse.com> + +- Fix minion scheduler to return a 'retcode' attribute (bsc#1089112) +- Fix for logging during network interface querying (bsc#1087581) +- Fix rhel packages requires both net-tools and iproute (bsc#1087055) + +- Added: + * initialize-__context__-retcode-for-functions-handled.patch + +- Modified: + * fix-for-errno-0-resolver-error-0-no-error-bsc-108758.patch + +------------------------------------------------------------------- +Wed Apr 18 17:09:41 UTC 2018 - Pablo Suárez Hernández <psuarezhernandez(a)suse.com> + +- Fix patchinstall on yum module. Bad comparison (bsc#1087278) + +- Added: + * provide-kwargs-to-pkg_resource.parse_targets-require.patch + +------------------------------------------------------------------- +Wed Apr 18 16:55:28 UTC 2018 - Pablo Suárez Hernández <psuarezhernandez(a)suse.com> + +- Strip trailing commas on Linux user's GECOS fields (bsc#1089362) +- Fallback to PyMySQL (bsc#1087891) +- Improved test for fqdns +- Update SaltSSH patch +- Fix for [Errno 0] Resolver Error 0 (no error) (bsc#1087581) + * Lintfix: PEP8 ident + * Use proper levels of the error handling, use proper log formatting. + * Fix unit test for reversed fqdns return data + +- Added: + * strip-trailing-commas-on-linux-user-gecos-fields.patch + * fall-back-to-pymysql.patch + * fix-for-errno-0-resolver-error-0-no-error-bsc-108758.patch + +- Modified: + * add-saltssh-multi-version-support-across-python-inte.patch + +------------------------------------------------------------------- New: ---- fall-back-to-pymysql.patch fix-for-errno-0-resolver-error-0-no-error-bsc-108758.patch initialize-__context__-retcode-for-functions-handled.patch provide-kwargs-to-pkg_resource.parse_targets-require.patch strip-trailing-commas-on-linux-user-gecos-fields.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ salt.spec ++++++ --- /var/tmp/diff_new_pack.IdNY7V/_old 2018-04-30 22:59:34.455069625 +0200 +++ /var/tmp/diff_new_pack.IdNY7V/_new 2018-04-30 22:59:34.459069478 +0200 @@ -86,6 +86,17 @@ Patch13: explore-module.run-response-to-catch-the-result-in-d.patch # PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/46684 Patch14: add-saltssh-multi-version-support-across-python-inte.patch +# PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/46635 +Patch15: fix-for-errno-0-resolver-error-0-no-error-bsc-108758.patch +# PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/46890 +Patch16: fall-back-to-pymysql.patch +# PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/47149 +Patch17: strip-trailing-commas-on-linux-user-gecos-fields.patch +# PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/47155 +Patch18: provide-kwargs-to-pkg_resource.parse_targets-require.patch +# PATCH-FIX_UPSTREAM https://github.com/saltstack/salt/pull/47270 +Patch19: initialize-__context__-retcode-for-functions-handled.patch + # BuildRoot: %{_tmppath}/%{name}-%{version}-build BuildRoot: %{_tmppath}/%{name}-%{version}-build @@ -116,12 +127,12 @@ %if 0%{?suse_version} >= 1500 Requires: iproute2 %else +%if 0%{?suse_version} Requires: net-tools -%endif - -%if 0%{?rhel} +%else Requires: iproute %endif +%endif %if %{with systemd} BuildRequires: systemd @@ -558,6 +569,11 @@ %patch12 -p1 %patch13 -p1 %patch14 -p1 +%patch15 -p1 +%patch16 -p1 +%patch17 -p1 +%patch18 -p1 +%patch19 -p1 %build %if 0%{?build_py2} ++++++ _lastrevision ++++++ --- /var/tmp/diff_new_pack.IdNY7V/_old 2018-04-30 22:59:34.503067873 +0200 +++ /var/tmp/diff_new_pack.IdNY7V/_new 2018-04-30 22:59:34.503067873 +0200 @@ -1 +1 @@ -f43b8fb2425e3371decf3cde040c70ed15de375d \ No newline at end of file +057b96d7f072dc69b61ca9e3b8d04a890003e58f \ No newline at end of file ++++++ _service ++++++ --- /var/tmp/diff_new_pack.IdNY7V/_old 2018-04-30 22:59:34.523067143 +0200 +++ /var/tmp/diff_new_pack.IdNY7V/_new 2018-04-30 22:59:34.523067143 +0200 @@ -3,7 +3,7 @@ <param name="url">https://github.com/openSUSE/salt-packaging.git</param> <param name="subdir">salt</param> <param name="filename">package</param> - <param name="revision">oxygen-rc1</param> + <param name="revision">2018.3.0</param> <param name="scm">git</param> </service> <service name="extract_file" mode="disabled"> ++++++ add-saltssh-multi-version-support-across-python-inte.patch ++++++ ++++ 603 lines (skipped) ++++ between /work/SRC/openSUSE:Factory/salt/add-saltssh-multi-version-support-across-python-inte.patch ++++ and /work/SRC/openSUSE:Factory/.salt.new/add-saltssh-multi-version-support-across-python-inte.patch ++++++ fall-back-to-pymysql.patch ++++++ >From f7ba683153e11be401a5971ba029d0a3964b1ecb Mon Sep 17 00:00:00 2001 From: Maximilian Meister <mmeister(a)suse.de> Date: Thu, 5 Apr 2018 13:23:23 +0200 Subject: [PATCH] fall back to PyMySQL same is already done in modules (see #26803) Signed-off-by: Maximilian Meister <mmeister(a)suse.de> --- salt/auth/mysql.py | 25 ++++++++++++++++++++++--- salt/cache/mysql_cache.py | 28 +++++++++++++++++++--------- salt/modules/mysql.py | 22 ++++++++++------------ salt/pillar/mysql.py | 21 ++++++++++++++++----- salt/returners/mysql.py | 29 +++++++++++++++++++++-------- tests/unit/pillar/test_mysql.py | 2 +- 6 files changed, 89 insertions(+), 38 deletions(-) diff --git a/salt/auth/mysql.py b/salt/auth/mysql.py index 8bc18a4101..86d00a4373 100644 --- a/salt/auth/mysql.py +++ b/salt/auth/mysql.py @@ -55,10 +55,29 @@ import logging log = logging.getLogger(__name__) try: + # Trying to import MySQLdb import MySQLdb - HAS_MYSQL = True + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.connections import OperationalError except ImportError: - HAS_MYSQL = False + try: + # MySQLdb import failed, try to import PyMySQL + import pymysql + pymysql.install_as_MySQLdb() + import MySQLdb + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.err import OperationalError + except ImportError: + MySQLdb = None + + +def __virtual__(): + ''' + Confirm that a python mysql client is installed. + ''' + return bool(MySQLdb), 'No python mysql client installed.' if MySQLdb is None else '' def __get_connection_info(): @@ -95,7 +114,7 @@ def auth(username, password): _info['username'], _info['password'], _info['database']) - except MySQLdb.OperationalError as e: + except OperationalError as e: log.error(e) return False diff --git a/salt/cache/mysql_cache.py b/salt/cache/mysql_cache.py index 9d6aa17987..8b0a942310 100644 --- a/salt/cache/mysql_cache.py +++ b/salt/cache/mysql_cache.py @@ -46,11 +46,24 @@ value to ``mysql``: from __future__ import absolute_import, print_function, unicode_literals from time import sleep import logging + try: + # Trying to import MySQLdb import MySQLdb - HAS_MYSQL = True + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.connections import OperationalError except ImportError: - HAS_MYSQL = False + try: + # MySQLdb import failed, try to import PyMySQL + import pymysql + pymysql.install_as_MySQLdb() + import MySQLdb + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.err import OperationalError + except ImportError: + MySQLdb = None from salt.exceptions import SaltCacheError @@ -71,12 +84,9 @@ __func_alias__ = {'ls': 'list'} def __virtual__(): ''' - Confirm that python-mysql package is installed. + Confirm that a python mysql client is installed. ''' - if not HAS_MYSQL: - return (False, "Please install python-mysql package to use mysql data " - "cache driver") - return __virtualname__ + return bool(MySQLdb), 'No python mysql client installed.' if MySQLdb is None else '' def run_query(conn, query, retries=3): @@ -84,13 +94,13 @@ def run_query(conn, query, retries=3): Get a cursor and run a query. Reconnect up to `retries` times if needed. Returns: cursor, affected rows counter - Raises: SaltCacheError, AttributeError, MySQLdb.OperationalError + Raises: SaltCacheError, AttributeError, OperationalError ''' try: cur = conn.cursor() out = cur.execute(query) return cur, out - except (AttributeError, MySQLdb.OperationalError) as e: + except (AttributeError, OperationalError) as e: if retries == 0: raise # reconnect creating new client diff --git a/salt/modules/mysql.py b/salt/modules/mysql.py index 0625b02a96..8b17e461ea 100644 --- a/salt/modules/mysql.py +++ b/salt/modules/mysql.py @@ -51,13 +51,14 @@ import salt.utils.stringutils from salt.ext import six # pylint: disable=import-error from salt.ext.six.moves import range, zip # pylint: disable=no-name-in-module,redefined-builtin + try: - # Try to import MySQLdb + # Trying to import MySQLdb import MySQLdb import MySQLdb.cursors import MySQLdb.converters from MySQLdb.constants import FIELD_TYPE, FLAG - HAS_MYSQLDB = True + from MySQLdb.connections import OperationalError except ImportError: try: # MySQLdb import failed, try to import PyMySQL @@ -67,10 +68,9 @@ except ImportError: import MySQLdb.cursors import MySQLdb.converters from MySQLdb.constants import FIELD_TYPE, FLAG - HAS_MYSQLDB = True + from MySQLdb.err import OperationalError except ImportError: - # No MySQL Connector installed, return False - HAS_MYSQLDB = False + MySQLdb = None log = logging.getLogger(__name__) @@ -195,11 +195,9 @@ And theses could be mixed, in a like query value with args: 'f\_o\%%o`b\'a"r' def __virtual__(): ''' - Only load this module if the mysql libraries exist + Confirm that a python mysql client is installed. ''' - if HAS_MYSQLDB: - return True - return (False, 'The mysql execution module cannot be loaded: neither MySQLdb nor PyMySQL is available.') + return bool(MySQLdb), 'No python mysql client installed.' if MySQLdb is None else '' def __check_table(name, table, **connection_args): @@ -331,7 +329,7 @@ def _connect(**kwargs): connargs.pop('passwd') try: dbc = MySQLdb.connect(**connargs) - except MySQLdb.OperationalError as exc: + except OperationalError as exc: err = 'MySQL Error {0}: {1}'.format(*exc) __context__['mysql.error'] = err log.error(err) @@ -647,7 +645,7 @@ def query(database, query, **connection_args): log.debug('Using db: %s to run query %s', database, query) try: affected = _execute(cur, query) - except MySQLdb.OperationalError as exc: + except OperationalError as exc: err = 'MySQL Error {0}: {1}'.format(*exc) __context__['mysql.error'] = err log.error(err) @@ -772,7 +770,7 @@ def status(**connection_args): qry = 'SHOW STATUS' try: _execute(cur, qry) - except MySQLdb.OperationalError as exc: + except OperationalError as exc: err = 'MySQL Error {0}: {1}'.format(*exc) __context__['mysql.error'] = err log.error(err) diff --git a/salt/pillar/mysql.py b/salt/pillar/mysql.py index 8029e5c197..d3f9619ad5 100644 --- a/salt/pillar/mysql.py +++ b/salt/pillar/mysql.py @@ -59,16 +59,27 @@ log = logging.getLogger(__name__) # Import third party libs try: + # Trying to import MySQLdb import MySQLdb - HAS_MYSQL = True + import MySQLdb.cursors + import MySQLdb.converters except ImportError: - HAS_MYSQL = False + try: + # MySQLdb import failed, try to import PyMySQL + import pymysql + pymysql.install_as_MySQLdb() + import MySQLdb + import MySQLdb.cursors + import MySQLdb.converters + except ImportError: + MySQLdb = None def __virtual__(): - if not HAS_MYSQL: - return False - return True + ''' + Confirm that a python mysql client is installed. + ''' + return bool(MySQLdb), 'No python mysql client installed.' if MySQLdb is None else '' class MySQLExtPillar(SqlBaseExtPillar): diff --git a/salt/returners/mysql.py b/salt/returners/mysql.py index af6698142b..85892cb06c 100644 --- a/salt/returners/mysql.py +++ b/salt/returners/mysql.py @@ -155,11 +155,24 @@ import salt.exceptions # Import 3rd-party libs from salt.ext import six + try: + # Trying to import MySQLdb import MySQLdb - HAS_MYSQL = True + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.connections import OperationalError except ImportError: - HAS_MYSQL = False + try: + # MySQLdb import failed, try to import PyMySQL + import pymysql + pymysql.install_as_MySQLdb() + import MySQLdb + import MySQLdb.cursors + import MySQLdb.converters + from MySQLdb.err import OperationalError + except ImportError: + MySQLdb = None log = logging.getLogger(__name__) @@ -168,10 +181,10 @@ __virtualname__ = 'mysql' def __virtual__(): - if not HAS_MYSQL: - return False, 'Could not import mysql returner; ' \ - 'mysql python client is not installed.' - return True + ''' + Confirm that a python mysql client is installed. + ''' + return bool(MySQLdb), 'No python mysql client installed.' if MySQLdb is None else '' def _get_options(ret=None): @@ -228,7 +241,7 @@ def _get_serv(ret=None, commit=False): conn = __context__['mysql_returner_conn'] conn.ping() connect = False - except MySQLdb.connections.OperationalError as exc: + except OperationalError as exc: log.debug('OperationalError on ping: %s', exc) if connect: @@ -254,7 +267,7 @@ def _get_serv(ret=None, commit=False): __context__['mysql_returner_conn'] = conn except TypeError: pass - except MySQLdb.connections.OperationalError as exc: + except OperationalError as exc: raise salt.exceptions.SaltMasterError('MySQL returner could not connect to database: {exc}'.format(exc=exc)) cursor = conn.cursor() diff --git a/tests/unit/pillar/test_mysql.py b/tests/unit/pillar/test_mysql.py index 8d49ac24e2..b72988673d 100644 --- a/tests/unit/pillar/test_mysql.py +++ b/tests/unit/pillar/test_mysql.py @@ -12,7 +12,7 @@ import salt.pillar.mysql as mysql @skipIf(NO_MOCK, NO_MOCK_REASON) -@skipIf(not mysql.HAS_MYSQL, 'MySQL-python module not installed') +(a)skipIf(mysql.MySQLdb is None, 'MySQL-python module not installed') class MysqlPillarTestCase(TestCase): maxDiff = None -- 2.13.6 ++++++ fix-for-errno-0-resolver-error-0-no-error-bsc-108758.patch ++++++ >From b9cc71639d4e918ef14635124f6991917150de46 Mon Sep 17 00:00:00 2001 From: Bo Maryniuk <bo(a)suse.de> Date: Wed, 21 Mar 2018 11:10:23 +0100 Subject: [PATCH] Fix for [Errno 0] Resolver Error 0 (no error) (bsc#1087581) * Lintfix: PEP8 ident * Use proper levels of the error handling, use proper log formatting. * Fix unit test for reversed fqdns return data --- salt/grains/core.py | 19 ++++++++++++------- tests/unit/grains/test_core.py | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 7 deletions(-) diff --git a/salt/grains/core.py b/salt/grains/core.py index 17a7d9819a..cd9ba1f29c 100644 --- a/salt/grains/core.py +++ b/salt/grains/core.py @@ -1900,16 +1900,21 @@ def fqdns(): fqdns = set() addresses = salt.utils.network.ip_addrs(include_loopback=False, - interface_data=_INTERFACES) + interface_data=_INTERFACES) addresses.extend(salt.utils.network.ip_addrs6(include_loopback=False, - interface_data=_INTERFACES)) - + interface_data=_INTERFACES)) + err_message = 'Exception during resolving address: %s' for ip in addresses: try: - fqdns.add(socket.gethostbyaddr(ip)[0]) - except (socket.error, socket.herror, - socket.gaierror, socket.timeout) as e: - log.info("Exception during resolving address: " + str(e)) + fqdns.add(socket.getfqdn(socket.gethostbyaddr(ip)[0])) + except socket.herror as err: + if err.errno == 0: + # No FQDN for this IP address, so we don't need to know this all the time. + log.debug("Unable to resolve address %s: %s", ip, err) + else: + log.error(err_message, err) + except (socket.error, socket.gaierror, socket.timeout) as err: + log.error(err_message, err) grains['fqdns'] = list(fqdns) return grains diff --git a/tests/unit/grains/test_core.py b/tests/unit/grains/test_core.py index 47c9cdd35b..c604df6c57 100644 --- a/tests/unit/grains/test_core.py +++ b/tests/unit/grains/test_core.py @@ -784,3 +784,35 @@ SwapTotal: 4789244 kB''' []}} with patch.object(salt.utils.dns, 'parse_resolv', MagicMock(return_value=resolv_mock)): assert core.dns() == ret + + def _run_dns_test(self, resolv_mock, ret): + with patch.object(salt.utils, 'is_windows', + MagicMock(return_value=False)): + with patch.dict(core.__opts__, {'ipv6': False}): + with patch.object(salt.utils.dns, 'parse_resolv', + MagicMock(return_value=resolv_mock)): + get_dns = core.dns() + self.assertEqual(get_dns, ret) + + @skipIf(not salt.utils.platform.is_linux(), 'System is not Linux') + @patch.object(salt.utils, 'is_windows', MagicMock(return_value=False)) + @patch('salt.utils.network.ip_addrs', MagicMock(return_value=['1.2.3.4', '5.6.7.8'])) + @patch('salt.utils.network.ip_addrs6', + MagicMock(return_value=['fe80::a8b2:93ff:fe00:0', 'fe80::a8b2:93ff:dead:beef'])) + @patch('salt.utils.network.socket.getfqdn', MagicMock(side_effect=lambda v: v)) # Just pass-through + def test_fqdns_return(self): + ''' + test the return for a dns grain. test for issue: + https://github.com/saltstack/salt/issues/41230 + ''' + reverse_resolv_mock = [('foo.bar.baz', [], ['1.2.3.4']), + ('rinzler.evil-corp.com', [], ['5.6.7.8']), + ('foo.bar.baz', [], ['fe80::a8b2:93ff:fe00:0']), + ('bluesniff.foo.bar', [], ['fe80::a8b2:93ff:dead:beef'])] + ret = {'fqdns': ['bluesniff.foo.bar', 'foo.bar.baz', 'rinzler.evil-corp.com']} + with patch.object(socket, 'gethostbyaddr', side_effect=reverse_resolv_mock): + fqdns = core.fqdns() + self.assertIn('fqdns', fqdns) + self.assertEqual(len(fqdns['fqdns']), len(ret['fqdns'])) + self.assertEqual(set(fqdns['fqdns']), set(ret['fqdns'])) + -- 2.13.6 ++++++ initialize-__context__-retcode-for-functions-handled.patch ++++++ >From c374feb62af75dfe18e8c81fb9cb556d678487ce Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pablo=20Su=C3=A1rez=20Hern=C3=A1ndez?= <psuarezhernandez(a)suse.com> Date: Tue, 24 Apr 2018 13:50:49 +0100 Subject: [PATCH] Initialize __context__ retcode for functions handled via schedule util module --- salt/utils/schedule.py | 1 + 1 file changed, 1 insertion(+) diff --git a/salt/utils/schedule.py b/salt/utils/schedule.py index de057477a3..6cb3ce0ef8 100644 --- a/salt/utils/schedule.py +++ b/salt/utils/schedule.py @@ -701,6 +701,7 @@ class Schedule(object): for global_key, value in six.iteritems(func_globals): self.functions[mod_name].__globals__[global_key] = value + self.functions.pack['__context__']['retcode'] = 0 ret['return'] = self.functions[func](*args, **kwargs) if not self.standalone: -- 2.15.1 ++++++ provide-kwargs-to-pkg_resource.parse_targets-require.patch ++++++ >From f7af1739a5795de6f98cfe2856372c755711e6dc Mon Sep 17 00:00:00 2001 From: Michael Calmer <mc(a)suse.de> Date: Wed, 18 Apr 2018 17:19:18 +0200 Subject: [PATCH] provide kwargs to pkg_resource.parse_targets required to detect advisory type fix invalid string compare --- salt/modules/yumpkg.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/salt/modules/yumpkg.py b/salt/modules/yumpkg.py index 39abb77fbc..9eb27e7701 100644 --- a/salt/modules/yumpkg.py +++ b/salt/modules/yumpkg.py @@ -1322,7 +1322,7 @@ def install(name=None, try: pkg_params, pkg_type = __salt__['pkg_resource.parse_targets']( - name, pkgs, sources, saltenv=saltenv, normalize=normalize + name, pkgs, sources, saltenv=saltenv, normalize=normalize, **kwargs ) except MinionError as exc: raise CommandExecutionError(exc) @@ -1620,7 +1620,7 @@ def install(name=None, if _yum() == 'dnf': cmd.extend(['--best', '--allowerasing']) _add_common_args(cmd) - cmd.append('install' if pkg_type is not 'advisory' else 'update') + cmd.append('install' if pkg_type != 'advisory' else 'update') cmd.extend(targets) out = __salt__['cmd.run_all']( cmd, -- 2.15.1 ++++++ strip-trailing-commas-on-linux-user-gecos-fields.patch ++++++ >From f9fb3639bb3c44babd92d9499bdde83a0a81d6ab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Pablo=20Su=C3=A1rez=20Hern=C3=A1ndez?= <psuarezhernandez(a)suse.com> Date: Wed, 18 Apr 2018 12:05:35 +0100 Subject: [PATCH] Strip trailing commas on Linux user GECOS fields Add unit tests for GECOS fields --- salt/modules/useradd.py | 2 +- tests/unit/modules/test_useradd.py | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/salt/modules/useradd.py b/salt/modules/useradd.py index 545fe2a6f1..a61ba0e960 100644 --- a/salt/modules/useradd.py +++ b/salt/modules/useradd.py @@ -81,7 +81,7 @@ def _build_gecos(gecos_dict): return '{0},{1},{2},{3}'.format(gecos_dict.get('fullname', ''), gecos_dict.get('roomnumber', ''), gecos_dict.get('workphone', ''), - gecos_dict.get('homephone', '')) + gecos_dict.get('homephone', '')).rstrip(',') def _update_gecos(name, key, value, root=None): diff --git a/tests/unit/modules/test_useradd.py b/tests/unit/modules/test_useradd.py index eb983685bb..fa30a0df71 100644 --- a/tests/unit/modules/test_useradd.py +++ b/tests/unit/modules/test_useradd.py @@ -393,3 +393,21 @@ class UserAddTestCase(TestCase, LoaderModuleMockMixin): mock = MagicMock(side_effect=[{'name': ''}, False, {'name': ''}]) with patch.object(useradd, 'info', mock): self.assertFalse(useradd.rename('salt', 'salt')) + + def test_build_gecos_field(self): + ''' + Test if gecos fields are built correctly (removing trailing commas) + ''' + test_gecos = {'fullname': 'Testing', + 'roomnumber': 1234, + 'workphone': 22222, + 'homephone': 99999} + expected_gecos_fields = 'Testing,1234,22222,99999' + self.assertEqual(useradd._build_gecos(test_gecos), expected_gecos_fields) + test_gecos.pop('roomnumber') + test_gecos.pop('workphone') + expected_gecos_fields = 'Testing,,,99999' + self.assertEqual(useradd._build_gecos(test_gecos), expected_gecos_fields) + test_gecos.pop('homephone') + expected_gecos_fields = 'Testing' + self.assertEqual(useradd._build_gecos(test_gecos), expected_gecos_fields) -- 2.15.1

1 0

commit lua-lua-mpris for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package lua-lua-mpris for openSUSE:Factory checked in at 2018-04-30 22:58:05 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/lua-lua-mpris (Old) and /work/SRC/openSUSE:Factory/.lua-lua-mpris.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "lua-lua-mpris" Mon Apr 30 22:58:05 2018 rev:6 rq:602503 version:0.0+git20170918.28b1c75 Changes: -------- --- /work/SRC/openSUSE:Factory/lua-lua-mpris/lua-lua-mpris.changes 2018-04-29 19:37:34.029931371 +0200 +++ /work/SRC/openSUSE:Factory/.lua-lua-mpris.new/lua-lua-mpris.changes 2018-04-30 22:59:28.455288555 +0200 @@ -1,0 +2,7 @@ +Mon Apr 30 09:46:02 UTC 2018 - alarrosa(a)suse.com + +- Since mpv uses lua 5.1, we better build mpv-plugin-mpris only + for the lua51 flavor instead of building it only for lua53. Otherwise + the package is useless. + +------------------------------------------------------------------- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ lua-lua-mpris.spec ++++++ --- /var/tmp/diff_new_pack.n3Lpo5/_old 2018-04-30 22:59:29.027267684 +0200 +++ /var/tmp/diff_new_pack.n3Lpo5/_new 2018-04-30 22:59:29.031267538 +0200 @@ -31,7 +31,7 @@ BuildRequires: pkgconfig(dbus-1) Requires: %{flavor} Requires: %{flavor}-lua-dbus -%if "%{flavor}" == "lua53" +%if "%{flavor}" == "lua51" Provides: lua-%{mod_name} = %{version} Obsoletes: lua-%{mod_name} < %{version} %endif @@ -68,7 +68,7 @@ %install mkdir -p %{buildroot}%{lua_noarchdir}/lua-mpris cp -Ra applet.lua client.lua init.lua %{buildroot}%{lua_noarchdir}/lua-mpris/ -%if "%{flavor}" == "lua53" +%if "%{flavor}" == "lua51" mkdir -p %{buildroot}%{_datadir}/mpv-plugin-mpris cp mpv.lua %{buildroot}%{_datadir}/mpv-plugin-mpris/ %endif @@ -84,7 +84,7 @@ %{lua_noarchdir}/lua-mpris # Only produce during one flavor to avoid duplicate binary. -%if "%{flavor}" == "lua53" +%if "%{flavor}" == "lua51" %files -n mpv-plugin-mpris %defattr(-,root,root) %doc LICENSE

1 0

commit hawk2 for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package hawk2 for openSUSE:Factory checked in at 2018-04-30 22:58:01 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/hawk2 (Old) and /work/SRC/openSUSE:Factory/.hawk2.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "hawk2" Mon Apr 30 22:58:01 2018 rev:50 rq:602495 version:2.1.0+git.1525082027.af7afec4 Changes: -------- --- /work/SRC/openSUSE:Factory/hawk2/hawk2.changes 2018-03-29 11:59:44.489428808 +0200 +++ /work/SRC/openSUSE:Factory/.hawk2.new/hawk2.changes 2018-04-30 22:59:27.399327087 +0200 @@ -1,0 +2,7 @@ +Mon Apr 30 10:38:34 UTC 2018 - opensuse-packaging(a)opensuse.org + +- Update to version 2.1.0+git.1525082027.af7afec4: + * Low: Update copyright year in branding + * Medium: Implement Hawk's cluster API (fate#323437) + +------------------------------------------------------------------- Old: ---- hawk2-2.1.0+git.1522277768.8e5a2060.tar.bz2 New: ---- hawk2-2.1.0+git.1525082027.af7afec4.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ hawk2.spec ++++++ --- /var/tmp/diff_new_pack.FJLWoA/_old 2018-04-30 22:59:28.215297312 +0200 +++ /var/tmp/diff_new_pack.FJLWoA/_new 2018-04-30 22:59:28.223297021 +0200 @@ -51,7 +51,7 @@ Summary: HA Web Konsole License: GPL-2.0-only Group: %{pkg_group} -Version: 2.1.0+git.1522277768.8e5a2060 +Version: 2.1.0+git.1525082027.af7afec4 Release: 0 Url: http://www.clusterlabs.org/wiki/Hawk Source: %{name}-%{version}.tar.bz2 ++++++ _servicedata ++++++ --- /var/tmp/diff_new_pack.FJLWoA/_old 2018-04-30 22:59:28.275295123 +0200 +++ /var/tmp/diff_new_pack.FJLWoA/_new 2018-04-30 22:59:28.275295123 +0200 @@ -1,4 +1,4 @@ <servicedata> <service name="tar_scm"> <param name="url">git://github.com/ClusterLabs/hawk.git</param> - <param name="changesrevision">8e5a2060c93c8cce8d486e170708c3cddbf1f0e1</param></service></servicedata> \ No newline at end of file + <param name="changesrevision">af7afec456f4b2671c1f30a9d705e24e6e06c83f</param></service></servicedata> \ No newline at end of file ++++++ hawk2-2.1.0+git.1522277768.8e5a2060.tar.bz2 -> hawk2-2.1.0+git.1525082027.af7afec4.tar.bz2 ++++++ /work/SRC/openSUSE:Factory/hawk2/hawk2-2.1.0+git.1522277768.8e5a2060.tar.bz2 /work/SRC/openSUSE:Factory/.hawk2.new/hawk2-2.1.0+git.1525082027.af7afec4.tar.bz2 differ: char 11, line 1

1 0

commit uget for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package uget for openSUSE:Factory checked in at 2018-04-30 22:57:53 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/uget (Old) and /work/SRC/openSUSE:Factory/.uget.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "uget" Mon Apr 30 22:57:53 2018 rev:42 rq:602480 version:2.2.1 Changes: -------- --- /work/SRC/openSUSE:Factory/uget/uget.changes 2018-01-22 16:18:33.565819209 +0100 +++ /work/SRC/openSUSE:Factory/.uget.new/uget.changes 2018-04-30 22:59:25.791385760 +0200 @@ -1,0 +2,12 @@ +Sat Apr 28 19:32:52 UTC 2018 - enzokiel(a)kabelmail.de.de + +- Update to version 2.2.1: + + reduce memory usage. + + mega plug-in: completed size should not be '-1' if file size + > 2G on a 32-bit system. + + adjust speed limit independently without enabling global speed + limit. + + Fix: Can't get 1080p video from YouTube. + + update translation files. + +------------------------------------------------------------------- Old: ---- uget-2.2.0.tar.gz New: ---- uget-2.2.1.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ uget.spec ++++++ --- /var/tmp/diff_new_pack.WPyqdR/_old 2018-04-30 22:59:26.463361240 +0200 +++ /var/tmp/diff_new_pack.WPyqdR/_new 2018-04-30 22:59:26.471360948 +0200 @@ -17,7 +17,7 @@ Name: uget -Version: 2.2.0 +Version: 2.2.1 Release: 0 Summary: Easy-to-use GTK+ Download Manager License: LGPL-2.1+ ++++++ uget-2.2.0.tar.gz -> uget-2.2.1.tar.gz ++++++ ++++ 100395 lines of diff (skipped)

1 0

commit eolie for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package eolie for openSUSE:Factory checked in at 2018-04-30 22:57:47 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/eolie (Old) and /work/SRC/openSUSE:Factory/.eolie.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "eolie" Mon Apr 30 22:57:47 2018 rev:2 rq:602478 version:0.9.31 Changes: -------- --- /work/SRC/openSUSE:Factory/eolie/eolie.changes 2018-04-24 15:34:21.699547188 +0200 +++ /work/SRC/openSUSE:Factory/.eolie.new/eolie.changes 2018-04-30 22:59:24.463434217 +0200 @@ -1,0 +2,9 @@ +Thu Apr 26 18:30:57 UTC 2018 - antoine.belvire(a)opensuse.org + +- Update to version 0.9.31: + * Fix a crash when closing a website. +- Update to version 0.9.30: + * Fix webkit crashes. + * Fix WebView popover. + +------------------------------------------------------------------- Old: ---- eolie-0.9.29.tar.xz New: ---- eolie-0.9.31.tar.xz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ eolie.spec ++++++ --- /var/tmp/diff_new_pack.gh6VKp/_old 2018-04-30 22:59:25.103410864 +0200 +++ /var/tmp/diff_new_pack.gh6VKp/_new 2018-04-30 22:59:25.107410719 +0200 @@ -15,11 +15,12 @@ # Please submit bugfixes or comments via http://bugs.opensuse.org/ # + # Filter out unwanted Requires %global __requires_exclude typelib\$Unity\$ Name: eolie -Version: 0.9.29 +Version: 0.9.31 Release: 0 Summary: Web browser for GNOME License: GPL-3.0-or-later ++++++ _service ++++++ --- /var/tmp/diff_new_pack.gh6VKp/_old 2018-04-30 22:59:25.135409697 +0200 +++ /var/tmp/diff_new_pack.gh6VKp/_new 2018-04-30 22:59:25.135409697 +0200 @@ -1,6 +1,6 @@ <services> <service mode="disabled" name="tar_scm"> - <param name="revision">0.9.29</param> + <param name="revision">0.9.31</param> <param name="scm">git</param> <param name="url">https://gitlab.gnome.org/gnumdk/eolie.git</param> <param name="versionformat">@PARENT_TAG@</param> ++++++ eolie-0.9.29.tar.xz -> eolie-0.9.31.tar.xz ++++++ ++++ 7969 lines of diff (skipped)

1 0

commit nautilus-share for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package nautilus-share for openSUSE:Factory checked in at 2018-04-30 22:57:41 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/nautilus-share (Old) and /work/SRC/openSUSE:Factory/.nautilus-share.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "nautilus-share" Mon Apr 30 22:57:41 2018 rev:29 rq:602477 version:0.7.3 Changes: -------- --- /work/SRC/openSUSE:Factory/nautilus-share/nautilus-share.changes 2011-11-28 12:54:53.000000000 +0100 +++ /work/SRC/openSUSE:Factory/.nautilus-share.new/nautilus-share.changes 2018-04-30 22:59:23.475470267 +0200 @@ -1,0 +2,6 @@ +Thu Apr 26 16:35:55 UTC 2018 - badshah400(a)gmail.com + +- Drop Requires: gnome-icon-theme; the appropriate icon is + provided by adwaita-icon-theme (pulled in by gtk3) already. + +------------------------------------------------------------------- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ nautilus-share.spec ++++++ --- /var/tmp/diff_new_pack.gpexBY/_old 2018-04-30 22:59:23.995451294 +0200 +++ /var/tmp/diff_new_pack.gpexBY/_new 2018-04-30 22:59:23.999451148 +0200 @@ -1,7 +1,7 @@ # # spec file for package nautilus-share # -# Copyright (c) 2011 SUSE LINUX Products GmbH, Nuernberg, Germany. +# Copyright (c) 2018 SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -20,17 +20,16 @@ Name: nautilus-share Version: 0.7.3 -Release: 1 -License: GPL-2.0+ +Release: 0 Summary: A Quick and Easy Way to Share Folders in Nautilus Via Samba -Url: http://git.gnome.org/nautilus-share +License: GPL-2.0-or-later Group: Productivity/Networking/Samba +Url: http://git.gnome.org/nautilus-share Source: http://download.gnome.org/sources/nautilus-share/0.7/%{name}-%{version}.tar… BuildRequires: intltool BuildRequires: libtool BuildRequires: nautilus-devel BuildRequires: translation-update-upstream -Requires: gnome-icon-theme Requires: samba >= 3.0.23 Recommends: %{name}-lang BuildRoot: %{_tmppath}/%{name}-%{version}-build

1 0

commit transactional-update for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package transactional-update for openSUSE:Factory checked in at 2018-04-30 22:57:27 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/transactional-update (Old) and /work/SRC/openSUSE:Factory/.transactional-update.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "transactional-update" Mon Apr 30 22:57:27 2018 rev:27 rq:602459 version:2.1 Changes: -------- --- /work/SRC/openSUSE:Factory/transactional-update/transactional-update.changes 2018-04-25 10:02:00.753387693 +0200 +++ /work/SRC/openSUSE:Factory/.transactional-update.new/transactional-update.changes 2018-04-30 22:59:21.403545871 +0200 @@ -1,0 +2,6 @@ +Thu Apr 26 15:14:36 CEST 2018 - kukuk(a)suse.de + +- Update to version 2.1 + - Cleanup the code cleaning up /etc + +------------------------------------------------------------------- Old: ---- transactional-update-2.0.tar.bz2 New: ---- transactional-update-2.1.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ transactional-update.spec ++++++ --- /var/tmp/diff_new_pack.0yxsJw/_old 2018-04-30 22:59:21.943526168 +0200 +++ /var/tmp/diff_new_pack.0yxsJw/_new 2018-04-30 22:59:21.947526022 +0200 @@ -17,7 +17,7 @@ Name: transactional-update -Version: 2.0 +Version: 2.1 Release: 0 Summary: Transactional Updates with btrfs and snapshots License: GPL-2.0-or-later ++++++ transactional-update-2.0.tar.bz2 -> transactional-update-2.1.tar.bz2 ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/NEWS new/transactional-update-2.1/NEWS --- old/transactional-update-2.0/NEWS 2018-04-23 10:03:28.000000000 +0200 +++ new/transactional-update-2.1/NEWS 2018-04-26 15:00:29.000000000 +0200 @@ -2,6 +2,9 @@ Copyright (C) 2016, 2017, 2018 Thorsten Kukuk +Version 2.1 +* Cleanup /etc cleanup + Version 2.0 * Create missing directories from rpm database during boot * Merge /etc overlay with root subvolume during update diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/configure new/transactional-update-2.1/configure --- old/transactional-update-2.0/configure 2018-04-23 10:24:56.000000000 +0200 +++ new/transactional-update-2.1/configure 2018-04-26 15:13:56.000000000 +0200 @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.69 for transactional-update 2.0. +# Generated by GNU Autoconf 2.69 for transactional-update 2.1. # # # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc. @@ -576,8 +576,8 @@ # Identity of this package. PACKAGE_NAME='transactional-update' PACKAGE_TARNAME='transactional-update' -PACKAGE_VERSION='2.0' -PACKAGE_STRING='transactional-update 2.0' +PACKAGE_VERSION='2.1' +PACKAGE_STRING='transactional-update 2.1' PACKAGE_BUGREPORT='' PACKAGE_URL='' @@ -1261,7 +1261,7 @@ # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures transactional-update 2.0 to adapt to many kinds of systems. +\`configure' configures transactional-update 2.1 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1328,7 +1328,7 @@ if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of transactional-update 2.0:";; + short | recursive ) echo "Configuration of transactional-update 2.1:";; esac cat <<\_ACEOF @@ -1438,7 +1438,7 @@ test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -transactional-update configure 2.0 +transactional-update configure 2.1 generated by GNU Autoconf 2.69 Copyright (C) 2012 Free Software Foundation, Inc. @@ -1531,7 +1531,7 @@ This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by transactional-update $as_me 2.0, which was +It was created by transactional-update $as_me 2.1, which was generated by GNU Autoconf 2.69. Invocation command line was $ $0 $@ @@ -2394,7 +2394,7 @@ # Define the identity of the package. PACKAGE='transactional-update' - VERSION='2.0' + VERSION='2.1' cat >>confdefs.h <<_ACEOF @@ -5070,7 +5070,7 @@ # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by transactional-update $as_me 2.0, which was +This file was extended by transactional-update $as_me 2.1, which was generated by GNU Autoconf 2.69. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -5127,7 +5127,7 @@ cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -transactional-update config.status 2.0 +transactional-update config.status 2.1 configured by $0, generated by GNU Autoconf 2.69, with options \\"\$ac_cs_config\\" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/configure.ac new/transactional-update-2.1/configure.ac --- old/transactional-update-2.0/configure.ac 2018-04-23 09:57:43.000000000 +0200 +++ new/transactional-update-2.1/configure.ac 2018-04-26 15:13:51.000000000 +0200 @@ -1,5 +1,5 @@ dnl Process this file with autoconf to produce a configure script. -AC_INIT(transactional-update, 2.0) +AC_INIT(transactional-update, 2.1) AM_INIT_AUTOMAKE AC_CONFIG_SRCDIR([sbin/transactional-update.in]) AC_PREFIX_DEFAULT(/usr) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/dracut/transactional-update-etc-cleaner.service new/transactional-update-2.1/dracut/transactional-update-etc-cleaner.service --- old/transactional-update-2.0/dracut/transactional-update-etc-cleaner.service 2018-04-20 18:34:52.000000000 +0200 +++ new/transactional-update-2.1/dracut/transactional-update-etc-cleaner.service 2018-04-26 14:59:47.000000000 +0200 @@ -4,7 +4,7 @@ After=dracut-pre-pivot.service Before=initrd-cleanup.service DefaultDependencies=no -ConditionPathExists=/sysroot/var/lib/overlay/etc/transactional-update.newsnapshot +ConditionPathExists=/sysroot/var/lib/overlay/transactional-update.newsnapshot [Service] Type=oneshot diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/dracut/transactional-update-etc-cleaner.sh new/transactional-update-2.1/dracut/transactional-update-etc-cleaner.sh --- old/transactional-update-2.0/dracut/transactional-update-etc-cleaner.sh 2018-04-20 18:34:52.000000000 +0200 +++ new/transactional-update-2.1/dracut/transactional-update-etc-cleaner.sh 2018-04-26 14:59:47.000000000 +0200 @@ -19,7 +19,7 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. ETC_OVERLAY="${NEWROOT}/var/lib/overlay/etc" -TU_FLAGFILE="${ETC_OVERLAY}/transactional-update.newsnapshot" +TU_FLAGFILE="${NEWROOT}/var/lib/overlay/transactional-update.newsnapshot" # Import common dracut variables . /dracut-state.sh 2>/dev/null @@ -46,7 +46,8 @@ } # Remove files from overlay with safety checks -# etc directory mustn't be deleted completely, as it's still mounted and the kernel won't be able to recover from that (different inode?) +# etc directory mustn't be deleted completely, as it's still mounted and the +# kernel won't be able to recover from that (different inode?) clean_overlay() { local dir="${1:-.}" local file @@ -98,7 +99,6 @@ fi UMOUNT_DOT_SNAPSHOTS=1 fi - CURRENT_SNAPSHOT_ID="`btrfs subvolume get-default /${NEWROOT} | sed 's#.*/.snapshots/$.*$/snapshot#\1#g'`" PREV_SNAPSHOT_ID="`sed -n 's#.*<pre_num>$[^>]\+$</pre_num>#\1#p' ${NEWROOT}/.snapshots/${CURRENT_SNAPSHOT_ID}/info.xml`" PREV_SNAPSHOT_DIR="`btrfs subvolume list /${NEWROOT} | sed -n 's#.*$/.snapshots/'${PREV_SNAPSHOT_ID}'/snapshot$#\1#p'`" if [ -n "${PREV_SNAPSHOT_DIR}" ] && ! findmnt "${NEWROOT}/${PREV_SNAPSHOT_DIR}" >/dev/null; then @@ -121,13 +121,18 @@ } if [ -e "${ETC_OVERLAY}" -a -e "${TU_FLAGFILE}" ]; then - # Previous snapshot may not be available; just delete all overlay contents in this case - rm "${TU_FLAGFILE}" - if prepare_environment; then - clean_overlay - else - remove_overlay + CURRENT_SNAPSHOT_ID="`findmnt /${NEWROOT} | sed -n 's#.*\[/@/\.snapshots/$[[:digit:]]\+$/snapshot\].*#\1#p'`" + . "${TU_FLAGFILE}" + + if [ "${CURRENT_SNAPSHOT_ID}" = "${EXPECTED_SNAPSHOT_ID}" ]; then + # Previous snapshot may not be available; just delete all overlay contents in this case + rm "${TU_FLAGFILE}" + if prepare_environment; then + clean_overlay + else + remove_overlay + fi + mount -o remount "${NEWROOT}/etc" + release_environment fi - mount -o remount "${NEWROOT}/etc" - release_environment fi diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/transactional-update-2.0/sbin/transactional-update.in new/transactional-update-2.1/sbin/transactional-update.in --- old/transactional-update-2.0/sbin/transactional-update.in 2018-04-20 20:40:09.000000000 +0200 +++ new/transactional-update-2.1/sbin/transactional-update.in 2018-04-30 09:44:56.000000000 +0200 @@ -45,7 +45,7 @@ SYSTEMCONFFILE="@prefix@@sysconfdir@/transactional-update.conf" LOGFILE="/var/log/transactional-update.log" STATE_FILE="/var/lib/misc/transactional-update.state" -NEW_SNAPSHOT_FLAG="/var/lib/overlay/etc/transactional-update.newsnapshot" +NEW_SNAPSHOT_FLAG="/var/lib/overlay/transactional-update.newsnapshot" SELF_PATH="`dirname $0`" PACKAGE_UPDATES=0 ZYPPER_AUTO_IMPORT_KEYS=0 @@ -454,7 +454,7 @@ fi fi -if [ ${ZYPPER_AUTO_IMPORT_KEYS} -eq 1 ]; then +if [ -n "${ZYPPER_ARG}" -a ${ZYPPER_AUTO_IMPORT_KEYS} -eq 1 ]; then ZYPPER_ARG="--gpg-auto-import-keys ${ZYPPER_ARG}" fi @@ -570,6 +570,13 @@ -o ${RUN_SHELL} -eq 1 -o ${REWRITE_BOOTLOADER} -eq 1 \ -o ${DO_REGISTRATION} -eq 1 ]; then + # If the current snapshot and the default snapshot differ + # there have been changes that we are now discarding. + if [ "${DEFAULT_SNAPSHOT_ID}" -ne "${CURRENT_SNAPSHOT_ID}" ]; then + log_error: "WARNING: Default snapshot differs from current snapshot." + log_error: "WARNING: Any changes within the previous snapshot will be discarded!" + fi + # Check if there are updates at all. Don't check if we do # a registration, as this can change the zypper result. if [ -n "${ZYPPER_ARG}" -a -n "${ZYPPER_NONINTERACTIVE}" \ @@ -629,7 +636,7 @@ # Copy the contents of /etc into the backup snapshot log_info "Copying /etc state into backup snapshot" - rsync --archive --xattrs --acls --quiet --exclude "${NEW_SNAPSHOT_FLAG}" --delete-excluded /etc ${BACKUP_SNAPSHOT_DIR} + rsync --archive --xattrs --acls --quiet --delete-excluded /etc ${BACKUP_SNAPSHOT_DIR} if [ $? -ne 0 ]; then log_error "ERROR: copying of /etc into backup snapshot failed!" quit 1; @@ -725,7 +732,7 @@ # Copy the contents of /etc log_info "Copying /etc state into snapshot" - rsync --archive --xattrs --acls --quiet --exclude "${NEW_SNAPSHOT_FLAG}" --delete-excluded /etc ${SNAPSHOT_DIR} + rsync --archive --xattrs --acls --quiet --delete-excluded /etc ${SNAPSHOT_DIR} if [ $? -ne 0 ]; then log_error "ERROR: copying of /etc into snapshot failed!" quit 1; @@ -897,7 +904,9 @@ add_unique_id ${CURRENT_SNAPSHOT_ID} save_state_file ${SNAPSHOT_ID} # Create flag file for overlay purging - touch "${NEW_SNAPSHOT_FLAG}" + if [ ${RO_ROOT} = "true" ]; then + echo "EXPECTED_SNAPSHOT_ID=${SNAPSHOT_ID}" > "${NEW_SNAPSHOT_FLAG}" + fi # Reset in-progress flag snapper ${SNAPPER_NO_DBUS} modify -u "transactional-update-in-progress=" ${SNAPSHOT_ID} fi

1 0

commit etc-update for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package etc-update for openSUSE:Factory checked in at 2018-04-30 22:57:35 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/etc-update (Old) and /work/SRC/openSUSE:Factory/.etc-update.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "etc-update" Mon Apr 30 22:57:35 2018 rev:6 rq:602474 version:2.3.31 Changes: -------- --- /work/SRC/openSUSE:Factory/etc-update/etc-update.changes 2018-04-27 16:08:36.051878450 +0200 +++ /work/SRC/openSUSE:Factory/.etc-update.new/etc-update.changes 2018-04-30 22:59:22.311512740 +0200 @@ -1,0 +2,7 @@ +Mon Apr 30 09:29:28 UTC 2018 - tchvatal(a)suse.com + +- Refresh etc-update-opensuse.patch to match for rpmnew files too. + Now it matched only rpmsave and rpmorig + * etc-update-opensuse.patch + +------------------------------------------------------------------- ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ etc-update-opensuse.patch ++++++ --- /var/tmp/diff_new_pack.GSgd61/_old 2018-04-30 22:59:22.943489680 +0200 +++ /var/tmp/diff_new_pack.GSgd61/_new 2018-04-30 22:59:22.943489680 +0200 @@ -2,7 +2,7 @@ =================================================================== --- portage-portage-2.3.31.orig/bin/etc-update +++ portage-portage-2.3.31/bin/etc-update -@@ -32,11 +32,12 @@ get_config() { +@@ -32,10 +32,11 @@ get_config() { "${PORTAGE_CONFIGROOT}"etc/etc-update.conf) } @@ -11,51 +11,47 @@ case $OS_RELEASE_ID in - suse|opensuse) OS_FAMILY='suse' NEW_EXT='rpmnew';; -- arch|manjaro|antergos) OS_FAMILY='arch' NEW_EXT='pacnew';; + suse|opensuse|opensuse-leap|opensuse-tumbleweed) OS_FAMILY='rpm' ;; + fedora|rhel) OS_FAMILY='rpm' ;; -+ arch|manjaro|antergos) OS_FAMILY='arch' ;; + arch|manjaro|antergos) OS_FAMILY='arch' NEW_EXT='pacnew';; *) OS_FAMILY='gentoo' ;; esac - -@@ -53,18 +54,32 @@ if [[ $OS_FAMILY == 'gentoo' ]]; then +@@ -53,7 +54,7 @@ if [[ $OS_FAMILY == 'gentoo' ]]; then get_live_file() { echo "${rpath}/${rfile:10}" } -elif [[ $OS_FAMILY == 'suse' ]] || [[ $OS_FAMILY == 'arch' ]]; then +elif [[ $OS_FAMILY == 'arch' ]]; then -+ get_basename() { -+ printf -- '%s\n' "${1%.pacnew}" -+ } -+ get_basename_find_opt() { -+ printf -- '%s\n' "${1}.pacnew" -+ } -+ get_scan_regexp() { -+ echo "s:$^.*/$$.*$$\.pacnew$:\1\2\3$b\1$b\3$b\2:" -+ } -+ get_live_file() { -+ printf -- '%s\n' "${cfg_file%.pacnew}" -+ } -+# In rpm we have rpmsave rpmorig and rpmnew -+elif [[ $OS_FAMILY == 'rpm' ]]; then get_basename() { -- printf -- '%s\n' "${1%.${NEW_EXT}}" -+ printf -- '%s\n' "${1}" |sed -e 's/\.rpmsave$//' -e 's/\.rpmnew$//' -e 's/\.rpmorig$//' + printf -- '%s\n' "${1%.${NEW_EXT}}" } - get_basename_find_opt() { -- printf -- '%s\n' "${1}.${NEW_EXT}" -+ printf -- '%s\n' "${1}.rpm????" +@@ -61,11 +62,25 @@ elif [[ $OS_FAMILY == 'suse' ]] || [[ $O + printf -- '%s\n' "${1}.${NEW_EXT}" } get_scan_regexp() { - echo "s:$^.*/$$.*$$\.rpmnew$:\1\2\3$b\1$b\3$b\2:" -+ echo "s:$^.*/$$.*$$\.\(rpmnew|rpmsave|rpmorig$\):\1\2\3$b\1$b\3$b\2:" ++ echo "s:$^.*/$$.*$$\.${NEW_EXT}$:\1\2\3$b\1$b\3$b\2:" } get_live_file() { -- printf -- '%s\n' "${cfg_file%.${NEW_EXT}}" -+ printf -- '%s\n' "${cfg_file}" |sed -e 's/\.rpmsave$//' -e 's/\.rpmnew$//' -e 's/\.rpmorig$//' + printf -- '%s\n' "${cfg_file%.${NEW_EXT}}" } ++# In rpm we have rpmsave, rpmorig, and rpmnew. ++elif [[ $OS_FAMILY == 'rpm' ]]; then ++ get_basename() { ++ printf -- '%s\n' "${1}" |sed -e 's/\.rpmsave$//' -e 's/\.rpmnew$//' -e 's/\.rpmorig$//' ++ } ++ get_basename_find_opt() { ++ printf -- '%s\n' "${1}.rpm???*" ++ } ++ get_scan_regexp() { ++ echo "s:$^.*/$$.*$$\.\(rpmnew|rpmsave|rpmorig$\):\1\2\3$b\1$b\3$b\2:" ++ } ++ get_live_file() { ++ printf -- '%s\n' "${cfg_file}" |sed -e 's/\.rpmsave$//' -e 's/\.rpmnew$//' -e 's/\.rpmorig$//' ++ } fi + cmd_var_is_valid() { @@ -557,7 +572,7 @@ do_merge() { local file="${1}"

1 0

commit youtube-dl for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package youtube-dl for openSUSE:Factory checked in at 2018-04-30 22:57:20 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/youtube-dl (Old) and /work/SRC/openSUSE:Factory/.youtube-dl.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "youtube-dl" Mon Apr 30 22:57:20 2018 rev:74 rq:602414 version:2018.04.25 Changes: -------- --- /work/SRC/openSUSE:Factory/youtube-dl/python-youtube-dl.changes 2018-04-17 11:18:15.780314514 +0200 +++ /work/SRC/openSUSE:Factory/.youtube-dl.new/python-youtube-dl.changes 2018-04-30 22:59:19.239624832 +0200 @@ -1,0 +2,27 @@ +Wed Apr 25 07:48:22 UTC 2018 - seb95.scou(a)gmail.com + +- Update to new upstream release 2018.04.25 + * Fix match_str for boolean meta fields + * Add support for pandoc 2 and disable smart extension (#16251) + * Fix typo in media extension compatibility checker (#16215) + * Recognize IPv6 stream URLs (#16136, #16137, #16205, #16246, + #16250) + * Extract is_live according to status (#16259) + * Relax URL regular expression (#16258) + * Remove extractor (#16256) + * Fix extraction (#16254) + * Add ability to authenticate with cookies + * Implement lazy playlist extraction (#10184) + * Add support for TV channel live streams (#15279, #15809) + * Fix video extraction (#15931) + * Fix extraction (#15227) + * Add support for nickjr.nl (#16230) + * Fix metadata extraction + * Add support for generic embeds (#16134, #16154) + * Extract new azure URLs (#16223) + * Fix extraction (#16217) + * Improve embeds detection (#16201) + * Fix extraction (#16119) + * Skip DRM asset types (#16104) + +------------------------------------------------------------------- youtube-dl.changes: same change Old: ---- youtube-dl-2018.04.16.tar.gz youtube-dl-2018.04.16.tar.gz.sig New: ---- youtube-dl-2018.04.25.tar.gz youtube-dl-2018.04.25.tar.gz.sig ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-youtube-dl.spec ++++++ --- /var/tmp/diff_new_pack.NFRUak/_old 2018-04-30 22:59:20.795568056 +0200 +++ /var/tmp/diff_new_pack.NFRUak/_new 2018-04-30 22:59:20.803567764 +0200 @@ -19,7 +19,7 @@ %define modname youtube-dl %{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-youtube-dl -Version: 2018.04.16 +Version: 2018.04.25 Release: 0 Summary: A python module for downloading from video sites for offline watching License: SUSE-Public-Domain AND CC-BY-SA-3.0 ++++++ youtube-dl.spec ++++++ --- /var/tmp/diff_new_pack.NFRUak/_old 2018-04-30 22:59:20.823567035 +0200 +++ /var/tmp/diff_new_pack.NFRUak/_new 2018-04-30 22:59:20.827566889 +0200 @@ -17,7 +17,7 @@ Name: youtube-dl -Version: 2018.04.16 +Version: 2018.04.25 Release: 0 Summary: A tool for downloading from video sites for offline watching License: SUSE-Public-Domain AND CC-BY-SA-3.0 ++++++ youtube-dl-2018.04.16.tar.gz -> youtube-dl-2018.04.25.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/AUTHORS new/youtube-dl/AUTHORS --- old/youtube-dl/AUTHORS 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/AUTHORS 2018-04-24 20:11:43.000000000 +0200 @@ -236,3 +236,6 @@ Petr Novák Leonardo Taccari Martin Weinelt +Surya Oktafendri +TingPing +Alexandre Macabies diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/ChangeLog new/youtube-dl/ChangeLog --- old/youtube-dl/ChangeLog 2018-04-15 20:09:12.000000000 +0200 +++ new/youtube-dl/ChangeLog 2018-04-24 20:12:36.000000000 +0200 @@ -1,3 +1,32 @@ +version 2018.04.25 + +Core +* [utils] Fix match_str for boolean meta fields ++ [Makefile] Add support for pandoc 2 and disable smart extension (#16251) +* [YoutubeDL] Fix typo in media extension compatibility checker (#16215) + +Extractors ++ [openload] Recognize IPv6 stream URLs (#16136, #16137, #16205, #16246, + #16250) ++ [twitch] Extract is_live according to status (#16259) +* [pornflip] Relax URL regular expression (#16258) +- [etonline] Remove extractor (#16256) +* [breakcom] Fix extraction (#16254) ++ [youtube] Add ability to authenticate with cookies +* [youtube:feed] Implement lazy playlist extraction (#10184) ++ [svt] Add support for TV channel live streams (#15279, #15809) +* [ccma] Fix video extraction (#15931) +* [rentv] Fix extraction (#15227) ++ [nick] Add support for nickjr.nl (#16230) +* [extremetube] Fix metadata extraction ++ [keezmovies] Add support for generic embeds (#16134, #16154) +* [nexx] Extract new azure URLs (#16223) +* [cbssports] Fix extraction (#16217) +* [kaltura] Improve embeds detection (#16201) +* [instagram:user] Fix extraction (#16119) +* [cbs] Skip DRM asset types (#16104) + + version 2018.04.16 Extractors diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/Makefile new/youtube-dl/Makefile --- old/youtube-dl/Makefile 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/Makefile 2018-04-24 20:11:43.000000000 +0200 @@ -14,6 +14,9 @@ # set SYSCONFDIR to /etc if PREFIX=/usr or PREFIX=/usr/local SYSCONFDIR = $(shell if [ $(PREFIX) = /usr -o $(PREFIX) = /usr/local ]; then echo /etc; else echo $(PREFIX)/etc; fi) +# set markdown input format to "markdown-smart" for pandoc version 2 and to "markdown" for pandoc prior to version 2 +MARKDOWN = $(shell if [ `pandoc -v | head -n1 | cut -d" " -f2 | head -c1` = "2" ]; then echo markdown-smart; else echo markdown; fi) + install: youtube-dl youtube-dl.1 youtube-dl.bash-completion youtube-dl.zsh youtube-dl.fish install -d $(DESTDIR)$(BINDIR) install -m 755 youtube-dl $(DESTDIR)$(BINDIR) @@ -82,11 +85,11 @@ $(PYTHON) devscripts/make_supportedsites.py docs/supportedsites.md README.txt: README.md - pandoc -f markdown -t plain README.md -o README.txt + pandoc -f $(MARKDOWN) -t plain README.md -o README.txt youtube-dl.1: README.md $(PYTHON) devscripts/prepare_manpage.py youtube-dl.1.temp.md - pandoc -s -f markdown -t man youtube-dl.1.temp.md -o youtube-dl.1 + pandoc -s -f $(MARKDOWN) -t man youtube-dl.1.temp.md -o youtube-dl.1 rm -f youtube-dl.1.temp.md youtube-dl.bash-completion: youtube_dl/*.py youtube_dl/*/*.py devscripts/bash-completion.in diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/README.txt new/youtube-dl/README.txt --- old/youtube-dl/README.txt 2018-04-15 20:09:46.000000000 +0200 +++ new/youtube-dl/README.txt 2018-04-24 20:12:56.000000000 +0200 @@ -1353,12 +1353,12 @@ 1. Fork this repository 2. Check out the source code with: - git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git + git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git 3. Start a new git branch with - cd youtube-dl - git checkout -b yourextractor + cd youtube-dl + git checkout -b yourextractor 4. Start with this simple template and save it to youtube_dl/extractor/yourextractor.py: @@ -1421,10 +1421,10 @@ 9. When the tests pass, add the new files and commit them and push the result, like this: - $ git add youtube_dl/extractor/extractors.py - $ git add youtube_dl/extractor/yourextractor.py - $ git commit -m '[yourextractor] Add new extractor' - $ git push origin yourextractor + $ git add youtube_dl/extractor/extractors.py + $ git add youtube_dl/extractor/yourextractor.py + $ git commit -m '[yourextractor] Add new extractor' + $ git push origin yourextractor 10. Finally, create a pull request. We'll then review and merge it. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/docs/supportedsites.md new/youtube-dl/docs/supportedsites.md --- old/youtube-dl/docs/supportedsites.md 2018-04-15 20:09:18.000000000 +0200 +++ new/youtube-dl/docs/supportedsites.md 2018-04-24 20:12:40.000000000 +0200 @@ -257,7 +257,6 @@ - **ESPN** - **ESPNArticle** - **EsriVideo** - - **ETOnline** - **Europa** - **EveryonesMixtape** - **ExpoTV** diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/test/test_subtitles.py new/youtube-dl/test/test_subtitles.py --- old/youtube-dl/test/test_subtitles.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/test/test_subtitles.py 2018-04-24 20:11:37.000000000 +0200 @@ -232,7 +232,7 @@ class TestMTVSubtitles(BaseTestSubtitles): - url = 'http://www.cc.com/video-clips/kllhuv/stand-up-greg-fitzsimmons--uncensored-…' + url = 'http://www.cc.com/video-clips/p63lk0/adam-devine-s-house-party-chasing-whit…' IE = ComedyCentralIE def getInfoDict(self): @@ -243,7 +243,7 @@ self.DL.params['allsubtitles'] = True subtitles = self.getSubtitles() self.assertEqual(set(subtitles.keys()), set(['en'])) - self.assertEqual(md5(subtitles['en']), 'b9f6ca22a6acf597ec76f61749765e65') + self.assertEqual(md5(subtitles['en']), '78206b8d8a0cfa9da64dc026eea48961') class TestNRKSubtitles(BaseTestSubtitles): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/test/test_utils.py new/youtube-dl/test/test_utils.py --- old/youtube-dl/test/test_utils.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/test/test_utils.py 2018-04-24 20:11:43.000000000 +0200 @@ -1072,6 +1072,18 @@ self.assertFalse(match_str( 'like_count > 100 & dislike_count <? 50 & description', {'like_count': 190, 'dislike_count': 10})) + self.assertTrue(match_str('is_live', {'is_live': True})) + self.assertFalse(match_str('is_live', {'is_live': False})) + self.assertFalse(match_str('is_live', {'is_live': None})) + self.assertFalse(match_str('is_live', {})) + self.assertFalse(match_str('!is_live', {'is_live': True})) + self.assertTrue(match_str('!is_live', {'is_live': False})) + self.assertTrue(match_str('!is_live', {'is_live': None})) + self.assertTrue(match_str('!is_live', {})) + self.assertTrue(match_str('title', {'title': 'abc'})) + self.assertTrue(match_str('title', {'title': ''})) + self.assertFalse(match_str('!title', {'title': 'abc'})) + self.assertFalse(match_str('!title', {'title': ''})) def test_parse_dfxp_time_expr(self): self.assertEqual(parse_dfxp_time_expr(None), None) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/test/test_youtube_lists.py new/youtube-dl/test/test_youtube_lists.py --- old/youtube-dl/test/test_youtube_lists.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/test/test_youtube_lists.py 2018-04-24 20:11:37.000000000 +0200 @@ -61,7 +61,7 @@ dl = FakeYDL() dl.params['extract_flat'] = True ie = YoutubePlaylistIE(dl) - result = ie.extract('https://www.youtube.com/playlist?list=PLwiyx1dc3P2JR9N8gQaQN_BCvlSlap7re') + result = ie.extract('https://www.youtube.com/playlist?list=PL-KKIb8rvtMSrAO9YFbeM6UQrAqoFTUWv') self.assertIsPlaylist(result) for entry in result['entries']: self.assertTrue(entry.get('title')) Binary files old/youtube-dl/youtube-dl and new/youtube-dl/youtube-dl differ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube-dl.1 new/youtube-dl/youtube-dl.1 --- old/youtube-dl/youtube-dl.1 2018-04-15 20:09:47.000000000 +0200 +++ new/youtube-dl/youtube-dl.1 2018-04-24 20:12:57.000000000 +0200 @@ -1,4 +1,7 @@ +.\" Automatically generated by Pandoc 2.1.3 +.\" .TH "YOUTUBE\-DL" "1" "" "" "" +.hy .SH NAME .PP youtube\-dl \- download videos from youtube.com or other video platforms @@ -997,7 +1000,7 @@ .PP To activate authentication with the \f[C]\&.netrc\f[] file you should pass \f[C]\-\-netrc\f[] to youtube\-dl or place it in the configuration -file (#configuration). +file. .PP On Windows you may also need to setup the \f[C]%HOME%\f[] environment variable manually. @@ -1013,7 +1016,7 @@ The \f[C]\-o\f[] option allows users to indicate a template for the output file names. .PP -\f[B]tl;dr:\f[] navigate me to examples (#output-template-examples). +\f[B]tl;dr:\f[] navigate me to examples. .PP The basic usage is not to set any template arguments when downloading a single file, like in @@ -1293,7 +1296,7 @@ an expression that describes format or formats you would like to download. .PP -\f[B]tl;dr:\f[] navigate me to examples (#format-selection-examples). +\f[B]tl;dr:\f[] navigate me to examples. .PP The simplest case is requesting a specific format, for example with \f[C]\-f\ 22\f[] you can download the format with format code equal to @@ -1418,8 +1421,8 @@ youtube\-dl 2015.04.26), i.e. you want to download the best available quality media served as a single file, you should explicitly specify your choice with \f[C]\-f\ best\f[]. -You may want to add it to the configuration file (#configuration) in -order not to type it every time you run youtube\-dl. +You may want to add it to the configuration file in order not to type it +every time you run youtube\-dl. .SS Format selection examples .PP Note that on Windows you may need to use double quotes instead of @@ -1629,12 +1632,12 @@ .PP YouTube has switched to a new video info format in July 2011 which is not supported by old versions of youtube\-dl. -See above (#how-do-i-update-youtube-dl) for how to update youtube\-dl. +See above for how to update youtube\-dl. .SS ERROR: unable to download video .PP YouTube requires an additional signature since September 2012 which is not supported by old versions of youtube\-dl. -See above (#how-do-i-update-youtube-dl) for how to update youtube\-dl. +See above for how to update youtube\-dl. .SS Video URL contains an ampersand and I\[aq]m getting some strange output \f[C][1]\ 2839\f[] or \f[C]\[aq]v\[aq]\ is\ not\ recognized\ as\ an\ internal\ or\ external\ command\f[] @@ -1665,15 +1668,15 @@ .PP In February 2015, the new YouTube player contained a character sequence in a string that was misinterpreted by old versions of youtube\-dl. -See above (#how-do-i-update-youtube-dl) for how to update youtube\-dl. +See above for how to update youtube\-dl. .SS HTTP Error 429: Too Many Requests or 402: Payment Required .PP These two error codes indicate that the service is blocking your IP address because of overuse. Contact the service and ask them to unblock your IP address, or \- if you have acquired a whitelisted IP address already \- use the -\f[C]\-\-proxy\f[] or \f[C]\-\-source\-address\f[] -options (#network-options) to select another IP address. +\f[C]\-\-proxy\f[] or \f[C]\-\-source\-address\f[] options to select +another IP address. .SS SyntaxError: Non\-ASCII character .PP The error @@ -1721,10 +1724,10 @@ matter what directory you\[aq]re in. .SS How do I put downloads into a specific folder? .PP -Use the \f[C]\-o\f[] to specify an output template (#output-template), -for example \f[C]\-o\ "/home/user/videos/%(title)s\-%(id)s.%(ext)s"\f[]. +Use the \f[C]\-o\f[] to specify an output template, for example +\f[C]\-o\ "/home/user/videos/%(title)s\-%(id)s.%(ext)s"\f[]. If you want this for all of your downloads, put the option into your -configuration file (#configuration). +configuration file. .SS How do I download a video starting with a \f[C]\-\f[]? .PP Either prepend \f[C]https://www.youtube.com/watch?v=\f[] or separate the @@ -1872,7 +1875,7 @@ Unfortunately, the youtube\-dl project has grown too large to use personal email as an effective communication channel. .PP -Please read the bug reporting instructions (#bugs) below. +Please read the bug reporting instructions below. A lot of bugs lack all the necessary information. If you can, offer proxy, VPN, or shell access to the youtube\-dl developers. @@ -1918,15 +1921,14 @@ .SH Why do I need to go through that much red tape when filing bugs? .PP Before we had the issue template, despite our extensive bug reporting -instructions (#bugs), about 80% of the issue reports we got were -useless, for instance because people used ancient versions hundreds of -releases old, because of simple syntactic errors (not in youtube\-dl but -in general shell usage), because the problem was already reported -multiple times before, because people did not actually read an error -message, even if it said "please install ffmpeg", because people did not -mention the URL they were trying to download and many more simple, -easy\-to\-avoid problems, many of whom were totally unrelated to -youtube\-dl. +instructions, about 80% of the issue reports we got were useless, for +instance because people used ancient versions hundreds of releases old, +because of simple syntactic errors (not in youtube\-dl but in general +shell usage), because the problem was already reported multiple times +before, because people did not actually read an error message, even if +it said "please install ffmpeg", because people did not mention the URL +they were trying to download and many more simple, easy\-to\-avoid +problems, many of whom were totally unrelated to youtube\-dl. .PP youtube\-dl is an open\-source project manned by too few volunteers, so we\[aq]d rather spend time fixing bugs where we are certain none of @@ -1963,8 +1965,8 @@ \f[] .fi .PP -See item 6 of new extractor tutorial (#adding-support-for-a-new-site) -for how to run extractor specific test cases. +See item 6 of new extractor tutorial for how to run extractor specific +test cases. .PP If you want to create a build of youtube\-dl yourself, you\[aq]ll need .IP \[bu] 2 @@ -1996,7 +1998,7 @@ .IP .nf \f[C] -git\ clone\ git\@github.com:YOUR_GITHUB_USERNAME/youtube\-dl.git +\ git\ clone\ git\@github.com:YOUR_GITHUB_USERNAME/youtube\-dl.git \f[] .fi .RE @@ -2006,8 +2008,8 @@ .IP .nf \f[C] -cd\ youtube\-dl -git\ checkout\ \-b\ yourextractor +\ cd\ youtube\-dl +\ git\ checkout\ \-b\ yourextractor \f[] .fi .RE @@ -2082,9 +2084,8 @@ return (https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/common.p…). Add tests and code for as many as you want. .IP " 8." 4 -Make sure your code follows youtube\-dl coding -conventions (#youtube-dl-coding-conventions) and check the code with -flake8 (https://pypi.python.org/pypi/flake8). +Make sure your code follows youtube\-dl coding conventions and check the +code with flake8 (https://pypi.python.org/pypi/flake8). Also make sure your code works under all Python (https://www.python.org/) versions claimed supported by youtube\-dl, namely 2.6, 2.7, and 3.2+. @@ -2096,10 +2097,10 @@ .IP .nf \f[C] -$\ git\ add\ youtube_dl/extractor/extractors.py -$\ git\ add\ youtube_dl/extractor/yourextractor.py -$\ git\ commit\ \-m\ \[aq][yourextractor]\ Add\ new\ extractor\[aq] -$\ git\ push\ origin\ yourextractor +\ $\ git\ add\ youtube_dl/extractor/extractors.py +\ $\ git\ add\ youtube_dl/extractor/yourextractor.py +\ $\ git\ commit\ \-m\ \[aq][yourextractor]\ Add\ new\ extractor\[aq] +\ $\ git\ push\ origin\ yourextractor \f[] .fi .RE @@ -2404,7 +2405,7 @@ (webchat (https://webchat.freenode.net/?randomnick=1&channels=youtube-dl)). .PP \f[B]Please include the full output of youtube\-dl when run with -\f[C]\-v\f[]\f[], i.e. +\f[BC]\-v\f[B]\f[], i.e. \f[B]add\f[] \f[C]\-v\f[] flag to \f[B]your command line\f[], copy the \f[B]whole\f[] output and post it in the issue body wrapped in ``` for better formatting. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/YoutubeDL.py new/youtube-dl/youtube_dl/YoutubeDL.py --- old/youtube-dl/youtube_dl/YoutubeDL.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/YoutubeDL.py 2018-04-24 20:11:37.000000000 +0200 @@ -1853,7 +1853,7 @@ def compatible_formats(formats): video, audio = formats # Check extension - video_ext, audio_ext = audio.get('ext'), video.get('ext') + video_ext, audio_ext = video.get('ext'), audio.get('ext') if video_ext and audio_ext: COMPATIBLE_EXTS = ( ('mp3', 'mp4', 'm4a', 'm4p', 'm4b', 'm4r', 'm4v', 'ismv', 'isma'), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/breakcom.py new/youtube-dl/youtube_dl/extractor/breakcom.py --- old/youtube-dl/youtube_dl/extractor/breakcom.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/breakcom.py 2018-04-24 20:11:37.000000000 +0200 @@ -3,15 +3,13 @@ import re from .common import InfoExtractor +from .youtube import YoutubeIE from ..compat import compat_str -from ..utils import ( - int_or_none, - parse_age_limit, -) +from ..utils import int_or_none class BreakIE(InfoExtractor): - _VALID_URL = r'https?://(?:www\.)?(?P<site>break|screenjunkies)\.com/video/(?P<display_id>[^/]+?)(?:-(?P<id>\d+))?(?:[/?#&]|$)' + _VALID_URL = r'https?://(?:www\.)?break\.com/video/(?P<display_id>[^/]+?)(?:-(?P<id>\d+))?(?:[/?#&]|$)' _TESTS = [{ 'url': 'http://www.break.com/video/when-girls-act-like-guys-2468056', 'info_dict': { @@ -19,125 +17,73 @@ 'ext': 'mp4', 'title': 'When Girls Act Like D-Bags', 'age_limit': 13, - } - }, { - 'url': 'http://www.screenjunkies.com/video/best-quentin-tarantino-movie-2841915', - 'md5': '5c2b686bec3d43de42bde9ec047536b0', - 'info_dict': { - 'id': '2841915', - 'display_id': 'best-quentin-tarantino-movie', - 'ext': 'mp4', - 'title': 'Best Quentin Tarantino Movie', - 'thumbnail': r're:^https?://.*\.jpg', - 'duration': 3671, - 'age_limit': 13, - 'tags': list, }, }, { - 'url': 'http://www.screenjunkies.com/video/honest-trailers-the-dark-knight', + # youtube embed + 'url': 'http://www.break.com/video/someone-forgot-boat-brakes-work', 'info_dict': { - 'id': '2348808', - 'display_id': 'honest-trailers-the-dark-knight', + 'id': 'RrrDLdeL2HQ', 'ext': 'mp4', - 'title': 'Honest Trailers - The Dark Knight', - 'thumbnail': r're:^https?://.*\.(?:jpg|png)', - 'age_limit': 10, - 'tags': list, - }, - }, { - # requires subscription but worked around - 'url': 'http://www.screenjunkies.com/video/knocking-dead-ep-1-the-show-so-far-30032…', - 'info_dict': { - 'id': '3003285', - 'display_id': 'knocking-dead-ep-1-the-show-so-far', - 'ext': 'mp4', - 'title': 'State of The Dead Recap: Knocking Dead Pilot', - 'thumbnail': r're:^https?://.*\.jpg', - 'duration': 3307, - 'age_limit': 13, - 'tags': list, + 'title': 'Whale Watching Boat Crashing Into San Diego Dock', + 'description': 'md5:afc1b2772f0a8468be51dd80eb021069', + 'upload_date': '20160331', + 'uploader': 'Steve Holden', + 'uploader_id': 'sdholden07', }, + 'params': { + 'skip_download': True, + } }, { 'url': 'http://www.break.com/video/ugc/baby-flex-2773063', 'only_matching': True, }] - _DEFAULT_BITRATES = (48, 150, 320, 496, 864, 2240, 3264) - def _real_extract(self, url): - site, display_id, video_id = re.match(self._VALID_URL, url).groups() + display_id, video_id = re.match(self._VALID_URL, url).groups() - if not video_id: - webpage = self._download_webpage(url, display_id) - video_id = self._search_regex( - (r'src=["\']/embed/(\d+)', r'data-video-content-id=["\'](\d+)'), - webpage, 'video id') - - webpage = self._download_webpage( - 'http://www.%s.com/embed/%s' % (site, video_id), - display_id, 'Downloading video embed page') - embed_vars = self._parse_json( - self._search_regex( - r'(?s)embedVars\s*=\s*({.+?})\s*</script>', webpage, 'embed vars'), - display_id) + webpage = self._download_webpage(url, display_id) - youtube_id = embed_vars.get('youtubeId') - if youtube_id: - return self.url_result(youtube_id, 'Youtube') + youtube_url = YoutubeIE._extract_url(webpage) + if youtube_url: + return self.url_result(youtube_url, ie=YoutubeIE.ie_key()) - title = embed_vars['contentName'] + content = self._parse_json( + self._search_regex( + r'(?s)content["\']\s*:\s*(\[.+?\])\s*[,\n]', webpage, + 'content'), + display_id) formats = [] - bitrates = [] - for f in embed_vars.get('media', []): - if not f.get('uri') or f.get('mediaPurpose') != 'play': + for video in content: + video_url = video.get('url') + if not video_url or not isinstance(video_url, compat_str): continue - bitrate = int_or_none(f.get('bitRate')) - if bitrate: - bitrates.append(bitrate) + bitrate = int_or_none(self._search_regex( + r'(\d+)_kbps', video_url, 'tbr', default=None)) formats.append({ - 'url': f['uri'], + 'url': video_url, 'format_id': 'http-%d' % bitrate if bitrate else 'http', - 'width': int_or_none(f.get('width')), - 'height': int_or_none(f.get('height')), 'tbr': bitrate, - 'format': 'mp4', }) - - if not bitrates: - # When subscriptionLevel > 0, i.e. plus subscription is required - # media list will be empty. However, hds and hls uris are still - # available. We can grab them assuming bitrates to be default. - bitrates = self._DEFAULT_BITRATES - - auth_token = embed_vars.get('AuthToken') - - def construct_manifest_url(base_url, ext): - pieces = [base_url] - pieces.extend([compat_str(b) for b in bitrates]) - pieces.append('_kbps.mp4.%s?%s' % (ext, auth_token)) - return ','.join(pieces) - - if bitrates and auth_token: - hds_url = embed_vars.get('hdsUri') - if hds_url: - formats.extend(self._extract_f4m_formats( - construct_manifest_url(hds_url, 'f4m'), - display_id, f4m_id='hds', fatal=False)) - hls_url = embed_vars.get('hlsUri') - if hls_url: - formats.extend(self._extract_m3u8_formats( - construct_manifest_url(hls_url, 'm3u8'), - display_id, 'mp4', entry_protocol='m3u8_native', m3u8_id='hls', fatal=False)) self._sort_formats(formats) + title = self._search_regex( + (r'title["\']\s*:\s*(["\'])(?P<value>(?:(?!\1).)+)\1', + r'<h1[^>]*>(?P<value>[^<]+)'), webpage, 'title', group='value') + + def get(key, name): + return int_or_none(self._search_regex( + r'%s["\']\s*:\s*["\'](\d+)' % key, webpage, name, + default=None)) + + age_limit = get('ratings', 'age limit') + video_id = video_id or get('pid', 'video id') or display_id + return { 'id': video_id, 'display_id': display_id, 'title': title, - 'thumbnail': embed_vars.get('thumbUri'), - 'duration': int_or_none(embed_vars.get('videoLengthInSeconds')) or None, - 'age_limit': parse_age_limit(embed_vars.get('audienceRating')), - 'tags': embed_vars.get('tags', '').split(','), + 'thumbnail': self._og_search_thumbnail(webpage), + 'age_limit': age_limit, 'formats': formats, } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/cbs.py new/youtube-dl/youtube_dl/extractor/cbs.py --- old/youtube-dl/youtube_dl/extractor/cbs.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/cbs.py 2018-04-24 20:11:37.000000000 +0200 @@ -65,7 +65,7 @@ last_e = None for item in items_data.findall('.//item'): asset_type = xpath_text(item, 'assetType') - if not asset_type or asset_type in asset_types: + if not asset_type or asset_type in asset_types or asset_type in ('HLS_FPS', 'DASH_CENC'): continue asset_types.append(asset_type) query = { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/cbssports.py new/youtube-dl/youtube_dl/extractor/cbssports.py --- old/youtube-dl/youtube_dl/extractor/cbssports.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/cbssports.py 2018-04-24 20:11:37.000000000 +0200 @@ -4,28 +4,35 @@ class CBSSportsIE(CBSBaseIE): - _VALID_URL = r'https?://(?:www\.)?cbssports\.com/video/player/[^/]+/(?P<id>\d+)' + _VALID_URL = r'https?://(?:www\.)?cbssports\.com/[^/]+/(?:video|news)/(?P<id>[^/?#&]+)' _TESTS = [{ - 'url': 'http://www.cbssports.com/video/player/videos/708337219968/0/ben-simmons-the…', + 'url': 'https://www.cbssports.com/nba/video/donovan-mitchell-flashes-star-potential…', 'info_dict': { - 'id': '708337219968', + 'id': '1214315075735', 'ext': 'mp4', - 'title': 'Ben Simmons the next LeBron? Not so fast', - 'description': 'md5:854294f627921baba1f4b9a990d87197', - 'timestamp': 1466293740, - 'upload_date': '20160618', + 'title': 'Donovan Mitchell flashes star potential in Game 2 victory over Thunder', + 'description': 'md5:df6f48622612c2d6bd2e295ddef58def', + 'timestamp': 1524111457, + 'upload_date': '20180419', 'uploader': 'CBSI-NEW', }, 'params': { # m3u8 download 'skip_download': True, } + }, { + 'url': 'https://www.cbssports.com/nba/news/nba-playoffs-2018-watch-76ers-vs-heat-ga…', + 'only_matching': True, }] def _extract_video_info(self, filter_query, video_id): return self._extract_feed_info('dJ5BDC', 'VxxJg8Ymh8sE', filter_query, video_id) def _real_extract(self, url): - video_id = self._match_id(url) + display_id = self._match_id(url) + webpage = self._download_webpage(url, display_id) + video_id = self._search_regex( + [r'(?:=|%26)pcid%3D(\d+)', r'embedVideo(?:Container)?_(\d+)'], + webpage, 'video id') return self._extract_video_info('byId=%s' % video_id, video_id) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/ccma.py new/youtube-dl/youtube_dl/extractor/ccma.py --- old/youtube-dl/youtube_dl/extractor/ccma.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/ccma.py 2018-04-24 20:11:37.000000000 +0200 @@ -4,11 +4,13 @@ import re from .common import InfoExtractor +from ..compat import compat_str from ..utils import ( + clean_html, int_or_none, parse_duration, parse_iso8601, - clean_html, + parse_resolution, ) @@ -40,34 +42,42 @@ def _real_extract(self, url): media_type, media_id = re.match(self._VALID_URL, url).groups() - media_data = {} - formats = [] - profiles = ['pc'] if media_type == 'audio' else ['mobil', 'pc'] - for i, profile in enumerate(profiles): - md = self._download_json('http://dinamics.ccma.cat/pvideo/media.jsp', media_id, query={ + + media = self._download_json( + 'http://dinamics.ccma.cat/pvideo/media.jsp', media_id, query={ 'media': media_type, 'idint': media_id, - 'profile': profile, - }, fatal=False) - if md: - media_data = md - media_url = media_data.get('media', {}).get('url') - if media_url: - formats.append({ - 'format_id': profile, - 'url': media_url, - 'quality': i, - }) + }) + + formats = [] + media_url = media['media']['url'] + if isinstance(media_url, list): + for format_ in media_url: + format_url = format_.get('file') + if not format_url or not isinstance(format_url, compat_str): + continue + label = format_.get('label') + f = parse_resolution(label) + f.update({ + 'url': format_url, + 'format_id': label, + }) + formats.append(f) + else: + formats.append({ + 'url': media_url, + 'vcodec': 'none' if media_type == 'audio' else None, + }) self._sort_formats(formats) - informacio = media_data['informacio'] + informacio = media['informacio'] title = informacio['titol'] durada = informacio.get('durada', {}) duration = int_or_none(durada.get('milisegons'), 1000) or parse_duration(durada.get('text')) timestamp = parse_iso8601(informacio.get('data_emissio', {}).get('utc')) subtitles = {} - subtitols = media_data.get('subtitols', {}) + subtitols = media.get('subtitols', {}) if subtitols: sub_url = subtitols.get('url') if sub_url: @@ -77,7 +87,7 @@ }) thumbnails = [] - imatges = media_data.get('imatges', {}) + imatges = media.get('imatges', {}) if imatges: thumbnail_url = imatges.get('url') if thumbnail_url: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/etonline.py new/youtube-dl/youtube_dl/extractor/etonline.py --- old/youtube-dl/youtube_dl/extractor/etonline.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/etonline.py 1970-01-01 01:00:00.000000000 +0100 @@ -1,39 +0,0 @@ -# coding: utf-8 -from __future__ import unicode_literals - -import re - -from .common import InfoExtractor - - -class ETOnlineIE(InfoExtractor): - _VALID_URL = r'https?://(?:www\.)?etonline\.com/(?:[^/]+/)*(?P<id>[^/?#&]+)' - _TESTS = [{ - 'url': 'http://www.etonline.com/tv/211130_dove_cameron_liv_and_maddie_emotional_epi…', - 'info_dict': { - 'id': '211130_dove_cameron_liv_and_maddie_emotional_episode_series_finale', - 'title': 'md5:a21ec7d3872ed98335cbd2a046f34ee6', - 'description': 'md5:8b94484063f463cca709617c79618ccd', - }, - 'playlist_count': 2, - }, { - 'url': 'http://www.etonline.com/media/video/here_are_the_stars_who_love_bringing_th…', - 'only_matching': True, - }] - BRIGHTCOVE_URL_TEMPLATE = 'http://players.brightcove.net/1242911076001/default_default/index.html?vide…' - - def _real_extract(self, url): - playlist_id = self._match_id(url) - - webpage = self._download_webpage(url, playlist_id) - - entries = [ - self.url_result( - self.BRIGHTCOVE_URL_TEMPLATE % video_id, 'BrightcoveNew', video_id) - for video_id in re.findall( - r'site\.brightcove\s*\([^,]+,\s*["\'](title_\d+)', webpage)] - - return self.playlist_result( - entries, playlist_id, - self._og_search_title(webpage, fatal=False), - self._og_search_description(webpage)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/extractors.py new/youtube-dl/youtube_dl/extractor/extractors.py --- old/youtube-dl/youtube_dl/extractor/extractors.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/extractors.py 2018-04-24 20:11:43.000000000 +0200 @@ -326,7 +326,6 @@ FiveThirtyEightIE, ) from .esri import EsriVideoIE -from .etonline import ETOnlineIE from .europa import EuropaIE from .everyonesmixtape import EveryonesMixtapeIE from .expotv import ExpoTVIE diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/extremetube.py new/youtube-dl/youtube_dl/extractor/extremetube.py --- old/youtube-dl/youtube_dl/extractor/extremetube.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/extremetube.py 2018-04-24 20:11:37.000000000 +0200 @@ -8,12 +8,12 @@ _VALID_URL = r'https?://(?:www\.)?extremetube\.com/(?:[^/]+/)?video/(?P<id>[^/#?&]+)' _TESTS = [{ 'url': 'http://www.extremetube.com/video/music-video-14-british-euro-brit-european-…', - 'md5': '1fb9228f5e3332ec8c057d6ac36f33e0', + 'md5': '92feaafa4b58e82f261e5419f39c60cb', 'info_dict': { 'id': 'music-video-14-british-euro-brit-european-cumshots-swallow-652431', 'ext': 'mp4', 'title': 'Music Video 14 british euro brit european cumshots swallow', - 'uploader': 'unknown', + 'uploader': 'anonim', 'view_count': int, 'age_limit': 18, } @@ -36,10 +36,10 @@ r'<h1[^>]+title="([^"]+)"[^>]*>', webpage, 'title') uploader = self._html_search_regex( - r'Uploaded by:\s*</strong>\s*(.+?)\s*</div>', + r'Uploaded by:\s*</[^>]+>\s*<a[^>]+>(.+?)</a>', webpage, 'uploader', fatal=False) view_count = str_to_int(self._search_regex( - r'Views:\s*</strong>\s*<span>([\d,\.]+)</span>', + r'Views:\s*</[^>]+>\s*<[^>]+>([\d,\.]+)</', webpage, 'view count', fatal=False)) info.update({ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/generic.py new/youtube-dl/youtube_dl/extractor/generic.py --- old/youtube-dl/youtube_dl/extractor/generic.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/generic.py 2018-04-24 20:11:37.000000000 +0200 @@ -1220,7 +1220,7 @@ 'title': '35871', 'timestamp': 1355743100, 'upload_date': '20121217', - 'uploader_id': 'batchUser', + 'uploader_id': 'cplapp(a)learn360.com', }, 'add_ie': ['Kaltura'], }, @@ -1271,6 +1271,22 @@ }, 'add_ie': ['Kaltura'], }, + { + # meta twitter:player + 'url': 'http://thechive.com/2017/12/08/all-i-want-for-christmas-is-more-twerk/', + 'info_dict': { + 'id': '0_01b42zps', + 'ext': 'mp4', + 'title': 'Main Twerk (Video)', + 'upload_date': '20171208', + 'uploader_id': 'sebastian.salinas(a)thechive.com', + 'timestamp': 1512713057, + }, + 'params': { + 'skip_download': True, + }, + 'add_ie': ['Kaltura'], + }, # referrer protected EaglePlatform embed { 'url': 'https://tvrain.ru/lite/teleshow/kak_vse_nachinalos/namin-418921/', diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/instagram.py new/youtube-dl/youtube_dl/extractor/instagram.py --- old/youtube-dl/youtube_dl/extractor/instagram.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/instagram.py 2018-04-24 20:11:37.000000000 +0200 @@ -6,11 +6,16 @@ import re from .common import InfoExtractor -from ..compat import compat_str +from ..compat import ( + compat_str, + compat_HTTPError, +) from ..utils import ( + ExtractorError, get_element_by_attribute, int_or_none, lowercase_escape, + std_headers, try_get, ) @@ -239,6 +244,8 @@ } } + _gis_tmpl = None + def _entries(self, data): def get_count(suffix): return int_or_none(try_get( @@ -254,19 +261,39 @@ for page_num in itertools.count(1): variables = json.dumps({ 'id': uploader_id, - 'first': 100, + 'first': 12, 'after': cursor, }) - s = '%s:%s:%s' % (rhx_gis, csrf_token, variables) - media = self._download_json( - 'https://www.instagram.com/graphql/query/', uploader_id, - 'Downloading JSON page %d' % page_num, headers={ - 'X-Requested-With': 'XMLHttpRequest', - 'X-Instagram-GIS': hashlib.md5(s.encode('utf-8')).hexdigest(), - }, query={ - 'query_hash': '472f257a40c653c64c666ce877d59d2b', - 'variables': variables, - })['data']['user']['edge_owner_to_timeline_media'] + + if self._gis_tmpl: + gis_tmpls = [self._gis_tmpl] + else: + gis_tmpls = [ + '%s' % rhx_gis, + '', + '%s:%s' % (rhx_gis, csrf_token), + '%s:%s:%s' % (rhx_gis, csrf_token, std_headers['User-Agent']), + ] + + for gis_tmpl in gis_tmpls: + try: + media = self._download_json( + 'https://www.instagram.com/graphql/query/', uploader_id, + 'Downloading JSON page %d' % page_num, headers={ + 'X-Requested-With': 'XMLHttpRequest', + 'X-Instagram-GIS': hashlib.md5( + ('%s:%s' % (gis_tmpl, variables)).encode('utf-8')).hexdigest(), + }, query={ + 'query_hash': '42323d64886122307be10013ad2dcc44', + 'variables': variables, + })['data']['user']['edge_owner_to_timeline_media'] + self._gis_tmpl = gis_tmpl + break + except ExtractorError as e: + if isinstance(e.cause, compat_HTTPError) and e.cause.code == 403: + if gis_tmpl != gis_tmpls[-1]: + continue + raise edges = media.get('edges') if not edges or not isinstance(edges, list): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/kaltura.py new/youtube-dl/youtube_dl/extractor/kaltura.py --- old/youtube-dl/youtube_dl/extractor/kaltura.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/kaltura.py 2018-04-24 20:11:37.000000000 +0200 @@ -135,10 +135,10 @@ ''', webpage) or re.search( r'''(?xs) - <iframe[^>]+src=(?P<q1>["']) - (?:https?:)?//(?:www\.)?kaltura\.com/(?:(?!(?P=q1)).)*\b(?:p|partner_id)/(?P<partner_id>\d+) + <(?:iframe[^>]+src|meta[^>]+\bcontent)=(?P<q1>["']) + (?:https?:)?//(?:(?:www|cdnapi)\.)?kaltura\.com/(?:(?!(?P=q1)).)*\b(?:p|partner_id)/(?P<partner_id>\d+) (?:(?!(?P=q1)).)* - [?&]entry_id=(?P<id>(?:(?!(?P=q1))[^&])+) + [?&;]entry_id=(?P<id>(?:(?!(?P=q1))[^&])+) (?P=q1) ''', webpage) ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/keezmovies.py new/youtube-dl/youtube_dl/extractor/keezmovies.py --- old/youtube-dl/youtube_dl/extractor/keezmovies.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/keezmovies.py 2018-04-24 20:11:37.000000000 +0200 @@ -20,23 +20,23 @@ class KeezMoviesIE(InfoExtractor): _VALID_URL = r'https?://(?:www\.)?keezmovies\.com/video/(?:(?P<display_id>[^/]+)-)?(?P<id>\d+)' _TESTS = [{ - 'url': 'http://www.keezmovies.com/video/petite-asian-lady-mai-playing-in-bathtub-12…', - 'md5': '1c1e75d22ffa53320f45eeb07bc4cdc0', + 'url': 'https://www.keezmovies.com/video/arab-wife-want-it-so-bad-i-see-she-thirsty…', + 'md5': '2ac69cdb882055f71d82db4311732a1a', 'info_dict': { - 'id': '1214711', - 'display_id': 'petite-asian-lady-mai-playing-in-bathtub', + 'id': '18070681', + 'display_id': 'arab-wife-want-it-so-bad-i-see-she-thirsty-and-has-tiny-money', 'ext': 'mp4', - 'title': 'Petite Asian Lady Mai Playing In Bathtub', - 'thumbnail': r're:^https?://.*\.jpg$', + 'title': 'Arab wife want it so bad I see she thirsty and has tiny money.', + 'thumbnail': None, 'view_count': int, 'age_limit': 18, } }, { - 'url': 'http://www.keezmovies.com/video/1214711', + 'url': 'http://www.keezmovies.com/video/18070681', 'only_matching': True, }] - def _extract_info(self, url): + def _extract_info(self, url, fatal=True): mobj = re.match(self._VALID_URL, url) video_id = mobj.group('id') display_id = (mobj.group('display_id') @@ -55,7 +55,7 @@ encrypted = False def extract_format(format_url, height=None): - if not isinstance(format_url, compat_str) or not format_url.startswith('http'): + if not isinstance(format_url, compat_str) or not format_url.startswith(('http', '//')): return if format_url in format_urls: return @@ -105,7 +105,11 @@ raise ExtractorError( 'Video %s is no longer available' % video_id, expected=True) - self._sort_formats(formats) + try: + self._sort_formats(formats) + except ExtractorError: + if fatal: + raise if not title: title = self._html_search_regex( @@ -122,7 +126,9 @@ } def _real_extract(self, url): - webpage, info = self._extract_info(url) + webpage, info = self._extract_info(url, fatal=False) + if not info['formats']: + return self.url_result(url, 'Generic') info['view_count'] = str_to_int(self._search_regex( r'<b>([\d,.]+)</b> Views?', webpage, 'view count', fatal=False)) return info diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/mofosex.py new/youtube-dl/youtube_dl/extractor/mofosex.py --- old/youtube-dl/youtube_dl/extractor/mofosex.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/mofosex.py 2018-04-24 20:11:37.000000000 +0200 @@ -12,7 +12,7 @@ _VALID_URL = r'https?://(?:www\.)?mofosex\.com/videos/(?P<id>\d+)/(?P<display_id>[^/?#&.]+)\.html' _TESTS = [{ 'url': 'http://www.mofosex.com/videos/318131/amateur-teen-playing-and-masturbating-…', - 'md5': '39a15853632b7b2e5679f92f69b78e91', + 'md5': '558fcdafbb63a87c019218d6e49daf8a', 'info_dict': { 'id': '318131', 'display_id': 'amateur-teen-playing-and-masturbating-318131', diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/nexx.py new/youtube-dl/youtube_dl/extractor/nexx.py --- old/youtube-dl/youtube_dl/extractor/nexx.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/nexx.py 2018-04-24 20:11:37.000000000 +0200 @@ -230,15 +230,18 @@ azure_locator = stream_data['azureLocator'] - AZURE_URL = 'http://nx%s%02d.akamaized.net/' - - def get_cdn_shield_base(shield_type='', prefix='-p'): + def get_cdn_shield_base(shield_type='', static=False): for secure in ('', 's'): cdn_shield = stream_data.get('cdnShield%sHTTP%s' % (shield_type, secure.upper())) if cdn_shield: return 'http%s://%s' % (secure, cdn_shield) else: - return AZURE_URL % (prefix, int(stream_data['azureAccount'].replace('nexxplayplus', ''))) + if 'fb' in stream_data['azureAccount']: + prefix = 'df' if static else 'f' + else: + prefix = 'd' if static else 'p' + account = int(stream_data['azureAccount'].replace('nexxplayplus', '').replace('nexxplayfb', '')) + return 'http://nx-%s%02d.akamaized.net/' % (prefix, account) azure_stream_base = get_cdn_shield_base() is_ml = ',' in language @@ -260,7 +263,7 @@ formats.extend(self._extract_ism_formats( azure_manifest_url % '', video_id, ism_id='%s-mss' % cdn, fatal=False)) - azure_progressive_base = get_cdn_shield_base('Prog', '-d') + azure_progressive_base = get_cdn_shield_base('Prog', True) azure_file_distribution = stream_data.get('azureFileDistribution') if azure_file_distribution: fds = azure_file_distribution.split(',') diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/nick.py new/youtube-dl/youtube_dl/extractor/nick.py --- old/youtube-dl/youtube_dl/extractor/nick.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/nick.py 2018-04-24 20:11:37.000000000 +0200 @@ -81,13 +81,23 @@ class NickBrIE(MTVServicesInfoExtractor): IE_NAME = 'nickelodeon:br' - _VALID_URL = r'https?://(?P<domain>(?:www\.)?nickjr|mundonick\.uol)\.com\.br/(?:programas/)?[^/]+/videos/(?:episodios/)?(?P<id>[^/?#.]+)' + _VALID_URL = r'''(?x) + https?:// + (?: + (?P<domain>(?:www\.)?nickjr|mundonick\.uol)\.com\.br| + (?:www\.)?nickjr\.nl + ) + /(?:programas/)?[^/]+/videos/(?:episodios/)?(?P<id>[^/?\#.]+) + ''' _TESTS = [{ 'url': 'http://www.nickjr.com.br/patrulha-canina/videos/210-labirinto-de-pipoca/', 'only_matching': True, }, { 'url': 'http://mundonick.uol.com.br/programas/the-loud-house/videos/muitas-irmas/7l…', 'only_matching': True, + }, { + 'url': 'http://www.nickjr.nl/paw-patrol/videos/311-ge-wol-dig-om-terug-te-zijn/', + 'only_matching': True, }] def _real_extract(self, url): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/openload.py new/youtube-dl/youtube_dl/extractor/openload.py --- old/youtube-dl/youtube_dl/extractor/openload.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/openload.py 2018-04-24 20:11:43.000000000 +0200 @@ -340,7 +340,10 @@ get_element_by_id('streamurj', webpage) or self._search_regex( (r'>\s*([\w-]+~\d{10,}~\d+\.\d+\.0\.0~[\w-]+)\s*<', - r'>\s*([\w~-]+~\d+\.\d+\.\d+\.\d+~[\w~-]+)'), webpage, + r'>\s*([\w~-]+~\d+\.\d+\.\d+\.\d+~[\w~-]+)', + r'>\s*([\w-]+~\d{10,}~(?:[a-f\d]+:){2}:~[\w-]+)\s*<', + r'>\s*([\w~-]+~[a-f0-9:]+~[\w~-]+)\s*<', + r'>\s*([\w~-]+~[a-f0-9:]+~[\w~-]+)'), webpage, 'stream URL')) video_url = 'https://openload.co/stream/%s?mime=true' % decoded_id diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/pornflip.py new/youtube-dl/youtube_dl/extractor/pornflip.py --- old/youtube-dl/youtube_dl/extractor/pornflip.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/pornflip.py 2018-04-24 20:11:43.000000000 +0200 @@ -14,7 +14,7 @@ class PornFlipIE(InfoExtractor): - _VALID_URL = r'https?://(?:www\.)?pornflip\.com/(?:v|embed)/(?P<id>[0-9A-Za-z-]{11})' + _VALID_URL = r'https?://(?:www\.)?pornflip\.com/(?:v|embed)/(?P<id>[^/?#&]+)' _TESTS = [{ 'url': 'https://www.pornflip.com/v/wz7DfNhMmep', 'md5': '98c46639849145ae1fd77af532a9278c', @@ -40,6 +40,9 @@ }, { 'url': 'https://www.pornflip.com/embed/EkRD6-vS2-s', 'only_matching': True, + }, { + 'url': 'https://www.pornflip.com/v/NG9q6Pb_iK8', + 'only_matching': True, }] def _real_extract(self, url): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/rentv.py new/youtube-dl/youtube_dl/extractor/rentv.py --- old/youtube-dl/youtube_dl/extractor/rentv.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/rentv.py 2018-04-24 20:11:37.000000000 +0200 @@ -3,6 +3,10 @@ from .common import InfoExtractor from ..compat import compat_str +from ..utils import ( + determine_ext, + int_or_none, +) class RENTVIE(InfoExtractor): @@ -13,7 +17,9 @@ 'info_dict': { 'id': '118577', 'ext': 'mp4', - 'title': 'Документальный спецпроект: "Промывка мозгов. Технологии XXI века"' + 'title': 'Документальный спецпроект: "Промывка мозгов. Технологии XXI века"', + 'timestamp': 1472230800, + 'upload_date': '20160826', } }, { 'url': 'http://ren.tv/player/118577', @@ -26,9 +32,33 @@ def _real_extract(self, url): video_id = self._match_id(url) webpage = self._download_webpage('http://ren.tv/player/' + video_id, video_id) - jw_config = self._parse_json(self._search_regex( - r'config\s*=\s*({.+});', webpage, 'jw config'), video_id) - return self._parse_jwplayer_data(jw_config, video_id, m3u8_id='hls') + config = self._parse_json(self._search_regex( + r'config\s*=\s*({.+})\s*;', webpage, 'config'), video_id) + title = config['title'] + formats = [] + for video in config['src']: + src = video.get('src') + if not src or not isinstance(src, compat_str): + continue + ext = determine_ext(src) + if ext == 'm3u8': + formats.extend(self._extract_m3u8_formats( + src, video_id, 'mp4', entry_protocol='m3u8_native', + m3u8_id='hls', fatal=False)) + else: + formats.append({ + 'url': src, + }) + self._sort_formats(formats) + return { + 'id': video_id, + 'title': title, + 'description': config.get('description'), + 'thumbnail': config.get('image'), + 'duration': int_or_none(config.get('duration')), + 'timestamp': int_or_none(config.get('date')), + 'formats': formats, + } class RENTVArticleIE(InfoExtractor): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/svt.py new/youtube-dl/youtube_dl/extractor/svt.py --- old/youtube-dl/youtube_dl/extractor/svt.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/svt.py 2018-04-24 20:11:37.000000000 +0200 @@ -22,6 +22,8 @@ _GEO_COUNTRIES = ['SE'] def _extract_video(self, video_info, video_id): + is_live = dict_get(video_info, ('live', 'simulcast'), default=False) + m3u8_protocol = 'm3u8' if is_live else 'm3u8_native' formats = [] for vr in video_info['videoReferences']: player_type = vr.get('playerType') or vr.get('format') @@ -30,7 +32,7 @@ if ext == 'm3u8': formats.extend(self._extract_m3u8_formats( vurl, video_id, - ext='mp4', entry_protocol='m3u8_native', + ext='mp4', entry_protocol=m3u8_protocol, m3u8_id=player_type, fatal=False)) elif ext == 'f4m': formats.extend(self._extract_f4m_formats( @@ -90,6 +92,7 @@ 'season_number': season_number, 'episode': episode, 'episode_number': episode_number, + 'is_live': is_live, } @@ -134,7 +137,7 @@ class SVTPlayIE(SVTPlayBaseIE): IE_DESC = 'SVT Play and Öppet arkiv' - _VALID_URL = r'https?://(?:www\.)?(?:svtplay|oppetarkiv)\.se/(?:video|klipp)/(?P<id>[0-9]+)' + _VALID_URL = r'https?://(?:www\.)?(?:svtplay|oppetarkiv)\.se/(?:video|klipp|kanaler)/(?P<id>[^/?#&]+)' _TESTS = [{ 'url': 'http://www.svtplay.se/video/5996901/flygplan-till-haile-selassie/flygplan-t…', 'md5': '2b6704fe4a28801e1a098bbf3c5ac611', @@ -158,6 +161,9 @@ }, { 'url': 'http://www.svtplay.se/klipp/9023742/stopptid-om-bjorn-borg', 'only_matching': True, + }, { + 'url': 'https://www.svtplay.se/kanaler/svt1', + 'only_matching': True, }] def _real_extract(self, url): @@ -173,6 +179,10 @@ thumbnail = self._og_search_thumbnail(webpage) + def adjust_title(info): + if info['is_live']: + info['title'] = self._live_title(info['title']) + if data: video_info = try_get( data, lambda x: x['context']['dispatcher']['stores']['VideoTitlePageStore']['data']['video'], @@ -183,6 +193,7 @@ 'title': data['context']['dispatcher']['stores']['MetaStore']['title'], 'thumbnail': thumbnail, }) + adjust_title(info_dict) return info_dict video_id = self._search_regex( @@ -198,6 +209,7 @@ info_dict['title'] = re.sub( r'\s*\|\s*.+?$', '', info_dict.get('episode') or self._og_search_title(webpage)) + adjust_title(info_dict) return info_dict diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/twitch.py new/youtube-dl/youtube_dl/extractor/twitch.py --- old/youtube-dl/youtube_dl/extractor/twitch.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/twitch.py 2018-04-24 20:11:43.000000000 +0200 @@ -168,6 +168,13 @@ return self.playlist_result(entries, info['id'], info['title']) def _extract_info(self, info): + status = info.get('status') + if status == 'recording': + is_live = True + elif status == 'recorded': + is_live = False + else: + is_live = None return { 'id': info['_id'], 'title': info.get('title') or 'Untitled Broadcast', @@ -178,6 +185,7 @@ 'uploader_id': info.get('channel', {}).get('name'), 'timestamp': parse_iso8601(info.get('recorded_at')), 'view_count': int_or_none(info.get('views')), + 'is_live': is_live, } def _real_extract(self, url): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/extractor/youtube.py new/youtube-dl/youtube_dl/extractor/youtube.py --- old/youtube-dl/youtube_dl/extractor/youtube.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/extractor/youtube.py 2018-04-24 20:11:37.000000000 +0200 @@ -87,7 +87,7 @@ (username, password) = self._get_login_info() # No authentication to be performed if username is None: - if self._LOGIN_REQUIRED: + if self._LOGIN_REQUIRED and self._downloader.params.get('cookiefile') is None: raise ExtractorError('No login info available, needed for using %s.' % self.IE_NAME, expected=True) return True @@ -2699,10 +2699,7 @@ def _real_initialize(self): self._login() - def _real_extract(self, url): - page = self._download_webpage( - 'https://www.youtube.com/feed/%s' % self._FEED_NAME, self._PLAYLIST_TITLE) - + def _entries(self, page): # The extraction process is the same as for playlists, but the regex # for the video ids doesn't contain an index ids = [] @@ -2713,12 +2710,15 @@ # 'recommended' feed has infinite 'load more' and each new portion spins # the same videos in (sometimes) slightly different order, so we'll check # for unicity and break when portion has no new videos - new_ids = filter(lambda video_id: video_id not in ids, orderedSet(matches)) + new_ids = list(filter(lambda video_id: video_id not in ids, orderedSet(matches))) if not new_ids: break ids.extend(new_ids) + for entry in self._ids_to_results(new_ids): + yield entry + mobj = re.search(r'data-uix-load-more-href="/?(?P<more>[^"]+)"', more_widget_html) if not mobj: break @@ -2730,8 +2730,12 @@ content_html = more['content_html'] more_widget_html = more['load_more_widget_html'] + def _real_extract(self, url): + page = self._download_webpage( + 'https://www.youtube.com/feed/%s' % self._FEED_NAME, + self._PLAYLIST_TITLE) return self.playlist_result( - self._ids_to_results(ids), playlist_title=self._PLAYLIST_TITLE) + self._entries(page), playlist_title=self._PLAYLIST_TITLE) class YoutubeWatchLaterIE(YoutubePlaylistIE): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/utils.py new/youtube-dl/youtube_dl/utils.py --- old/youtube-dl/youtube_dl/utils.py 2018-04-09 00:29:45.000000000 +0200 +++ new/youtube-dl/youtube_dl/utils.py 2018-04-24 20:11:43.000000000 +0200 @@ -2574,8 +2574,8 @@ return op(actual_value, comparison_value) UNARY_OPERATORS = { - '': lambda v: v is not None, - '!': lambda v: v is None, + '': lambda v: (v is True) if isinstance(v, bool) else (v is not None), + '!': lambda v: (v is False) if isinstance(v, bool) else (v is None), } operator_rex = re.compile(r'''(?x)\s* (?P<op>%s)\s*(?P<key>[a-z_]+) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/youtube-dl/youtube_dl/version.py new/youtube-dl/youtube_dl/version.py --- old/youtube-dl/youtube_dl/version.py 2018-04-15 20:09:12.000000000 +0200 +++ new/youtube-dl/youtube_dl/version.py 2018-04-24 20:12:36.000000000 +0200 @@ -1,3 +1,3 @@ from __future__ import unicode_literals -__version__ = '2018.04.16' +__version__ = '2018.04.25'

1 0

commit cantata for openSUSE:Factory
by root＠hilbert.suse.de 30 Apr '18

30 Apr '18

Hello community, here is the log from the commit of package cantata for openSUSE:Factory checked in at 2018-04-30 22:57:15 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/cantata (Old) and /work/SRC/openSUSE:Factory/.cantata.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "cantata" Mon Apr 30 22:57:15 2018 rev:8 rq:602401 version:2.3.0 Changes: -------- --- /work/SRC/openSUSE:Factory/cantata/cantata.changes 2018-03-28 10:32:48.237975885 +0200 +++ /work/SRC/openSUSE:Factory/.cantata.new/cantata.changes 2018-04-30 22:59:15.535759985 +0200 @@ -1,0 +2,56 @@ +Sun Apr 29 21:32:58 UTC 2018 - cornelis(a)solcon.nl + +- Replace fix-no-return-in-nonvoid-function.diff with patch from + upstream + +------------------------------------------------------------------- +Sat Apr 28 20:16:49 UTC 2018 - cornelis(a)solcon.nl + +- For now disable remote device support. The dbus service that comes + with it needs a security audit. +- Remove cantata.rpmlintrc: it dealt with the dbus service rpmlint + error in a way not allowed in factory. + +------------------------------------------------------------------- +Sat Apr 28 16:36:41 UTC 2018 - enzokiel(a)kabelmail.de + + - Added fix-no-return-in-nonvoid-function.diff and + cantata.rpmlintrc. + +------------------------------------------------------------------- +Fri Apr 27 21:19:16 UTC 2018 - cornelis(a)solcon.nl + +- Use %license for LICENSE for newer versions. + +------------------------------------------------------------------- +Fri Apr 27 20:08:51 UTC 2018 - cornelis(a)solcon.nl + +- Update to 2.3.0: + * Only disable system tray support if org.kde.StatusNotifierWatcher + is not registered when running under Gnome. + * Add ability to change grid cover size - Ctrl+ / Ctrl- + * Avahi support (automatic mpd discovery) + * Make serving of files to MPD via HTTP configurable. + * Improved transcode options. + * If dynamic playlists helper does not start, show link to wiki + page detailing perl dependencies. + * Add "Add Local Files" to playqueue context menu. + * Remember last loaded/saved playlist name - and use this as the + default name when saving. + * Add option to show bitrate, sample rate, and format in toolbar. + * Add support for forked-DAAP - thanks to Wolfgang Scherer. + * Add checkbox to indicate that server auto-updates when files + changed. Thanks to Wolfgang Scherer. + * Make it possible to filter on year (or range of years) in library + and playqueue search fields. + * Enable remote (sshfs, samba) device support by default for Linux builds. + * Re-add option to show menumbar for Linux builds if not run under GNOME. + * Work-around Cantata preventing logout under GNOME/KDE if set to + minimise to system tray and main window is open. + * Fix reading embedded covers from OGG files. + * Fetch missing covers from iTunes. + * And more, see https://github.com/CDrummond/cantata/releases +- Deleted now included patches: fix-prevent-logout.patch and + fix-qt-510.patch + +------------------------------------------------------------------- Old: ---- cantata-2.2.0.tar.gz fix-prevent-logout.patch fix-qt-510.patch New: ---- cantata-2.3.0.tar.gz fix-no-return-in-nonvoid-function.diff ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ cantata.spec ++++++ --- /var/tmp/diff_new_pack.z2ijFB/_old 2018-04-30 22:59:16.383729043 +0200 +++ /var/tmp/diff_new_pack.z2ijFB/_new 2018-04-30 22:59:16.387728897 +0200 @@ -17,7 +17,7 @@ Name: cantata -Version: 2.2.0 +Version: 2.3.0 Release: 0 Summary: Client for the Music Player Daemon (MPD) License: GPL-3.0-only @@ -27,10 +27,8 @@ # PATCH-FIX-OPENSUSE fix-translations-with-qt5.diff -- Make sure that it finds the right lrelease and lconvert binaries with Qt5 Patch1: fix-translations-with-qt5.diff Patch2: install-libsolidlite.diff -# PATCH-FIX-UPSTREAM fix-qt-510.pach -- fix crash with Qt 5.10 (boo#1077468) -Patch3: fix-qt-510.patch -# PATCH-FIX-UPSTREAM: https://github.com/CDrummond/cantata/issues/1183 -Patch4: fix-prevent-logout.patch +# PATCH-FIX-UPSTREAM +Patch3: fix-no-return-in-nonvoid-function.diff #Source0: https://github.com/CDrummond/cantata/archive/v%{version}.tar.gz BuildRequires: fdupes BuildRequires: media-player-info @@ -104,10 +102,11 @@ %patch1 -p0 %patch2 -p0 %patch3 -p1 -%patch4 -p1 %build -%cmake +# TODO: an dbus service is used to access remote device through sshfs or sambar +# This needs an audit (https://en.opensuse.org/openSUSE:Package_security_guidelines#DBus_Services) +%cmake -DENABLE_REMOTE_DEVICES=OFF %make_jobs %install @@ -135,7 +134,12 @@ %exclude %{_datadir}/%{name}/translations/ %{_datadir}/applications/%{name}.desktop %{_datadir}/icons/hicolor/ -%doc AUTHORS ChangeLog LICENSE README README.md TODO +%doc AUTHORS ChangeLog README README.md TODO +%if 0%{?suse_version} < 1500 +%doc LICENSE +%else +%license LICENSE +%endif %files lang -f %{name}.lang %defattr(-,root,root) ++++++ cantata-2.2.0.tar.gz -> cantata-2.3.0.tar.gz ++++++ ++++ 126610 lines of diff (skipped) ++++++ fix-no-return-in-nonvoid-function.diff ++++++ >From bb421602604c30768195905201047e31331130d1 Mon Sep 17 00:00:00 2001 From: Craig Drummond <craig.p.drummond(a)gmail.com> Date: Sun, 29 Apr 2018 22:27:27 +0100 Subject: [PATCH] Ensure always return a value for ApiKeys::isLimitReached Closes #1255 --- gui/apikeys.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/gui/apikeys.cpp b/gui/apikeys.cpp index 5603398f8..d3e77e54d 100644 --- a/gui/apikeys.cpp +++ b/gui/apikeys.cpp @@ -132,6 +132,7 @@ bool ApiKeys::isLimitReached(Service srv) } return true; } + return false; } bool ApiKeys::isLimitReached(const QNetworkReply *job, Service srv)

1 0