Hello community,
here is the log from the commit of package python-distributed for openSUSE:Factory checked in at 2018-09-18 11:49:09
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-distributed (Old)
and /work/SRC/openSUSE:Factory/.python-distributed.new (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-distributed"
Tue Sep 18 11:49:09 2018 rev:7 rq:636249 version:1.23.2
Changes:
--------
--- /work/SRC/openSUSE:Factory/python-distributed/python-distributed.changes 2018-09-11 17:17:51.055359160 +0200
+++ /work/SRC/openSUSE:Factory/.python-distributed.new/python-distributed.changes 2018-09-18 11:49:57.299505417 +0200
@@ -1,0 +2,12 @@
+Mon Sep 17 14:55:33 UTC 2018 - Arun Persaud
+
+- update to version 1.23.2:
+ * Discard dependent rather than remove (#2250) Matthew Rocklin
+ * Use dask_sphinx_theme Matthew Rocklin
+ * Drop the Bokeh index page (#2241) John Kirkham
+ * Revert change to keep link relative (#2242) Matthew Rocklin
+ * docs: Fix broken AWS link in setup.rst file (#2240) Vladyslav
+ Moisieienkov
+ * Return cancelled futures in as_completed (#2233) Chris White
+
+-------------------------------------------------------------------
Old:
----
distributed-1.23.1.tar.gz
New:
----
distributed-1.23.2.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-distributed.spec ++++++
--- /var/tmp/diff_new_pack.BnybPN/_old 2018-09-18 11:50:08.067494352 +0200
+++ /var/tmp/diff_new_pack.BnybPN/_new 2018-09-18 11:50:08.071494348 +0200
@@ -12,7 +12,7 @@
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.
-# Please submit bugfixes or comments via http://bugs.opensuse.org/
+# Please submit bugfixes or comments via https://bugs.opensuse.org/
#
@@ -20,7 +20,7 @@
# Test requires network connection
%bcond_with test
Name: python-distributed
-Version: 1.23.1
+Version: 1.23.2
Release: 0
Summary: Library for distributed computing with Python
License: BSD-3-Clause
++++++ distributed-1.23.1.tar.gz -> distributed-1.23.2.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/PKG-INFO new/distributed-1.23.2/PKG-INFO
--- old/distributed-1.23.1/PKG-INFO 2018-09-06 14:11:46.000000000 +0200
+++ new/distributed-1.23.2/PKG-INFO 2018-09-17 15:57:35.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: distributed
-Version: 1.23.1
+Version: 1.23.2
Summary: Distributed scheduler for Dask
Home-page: https://distributed.readthedocs.io/en/latest/
Author: Matthew Rocklin
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/_version.py new/distributed-1.23.2/distributed/_version.py
--- old/distributed-1.23.1/distributed/_version.py 2018-09-06 14:11:46.000000000 +0200
+++ new/distributed-1.23.2/distributed/_version.py 2018-09-17 15:57:35.000000000 +0200
@@ -8,11 +8,11 @@
version_json = '''
{
- "date": "2018-09-06T08:10:35-0400",
+ "date": "2018-09-17T09:56:32-0400",
"dirty": false,
"error": null,
- "full-revisionid": "8bd288a7b7e087202ed74d67bc7a438221d0b2bf",
- "version": "1.23.1"
+ "full-revisionid": "57aeb70b96396021b2b996a49e6625e0a547609f",
+ "version": "1.23.2"
}
''' # END VERSION_JSON
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/bokeh/core.py new/distributed-1.23.2/distributed/bokeh/core.py
--- old/distributed-1.23.1/distributed/bokeh/core.py 2018-08-30 15:08:21.000000000 +0200
+++ new/distributed-1.23.2/distributed/bokeh/core.py 2018-09-07 19:16:37.000000000 +0200
@@ -32,6 +32,7 @@
port=port, address=ip,
check_unused_sessions_milliseconds=500,
allow_websocket_origin=["*"],
+ use_index=False,
**self.server_kwargs)
self.server.start()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/bokeh/templates/base.html new/distributed-1.23.2/distributed/bokeh/templates/base.html
--- old/distributed-1.23.1/distributed/bokeh/templates/base.html 2018-08-30 19:13:22.000000000 +0200
+++ new/distributed-1.23.2/distributed/bokeh/templates/base.html 2018-09-07 18:22:57.000000000 +0200
@@ -26,7 +26,7 @@
</li>
{% for page in pages %}
<li>
- <a href="/{{ page }}">{{ page|title }}</a>
+ <a href="{{ page }}">{{ page|title }}</a>
</li>
{% endfor %}
<li>
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/bokeh/tests/test_scheduler_bokeh.py new/distributed-1.23.2/distributed/bokeh/tests/test_scheduler_bokeh.py
--- old/distributed-1.23.1/distributed/bokeh/tests/test_scheduler_bokeh.py 2018-08-30 15:08:21.000000000 +0200
+++ new/distributed-1.23.2/distributed/bokeh/tests/test_scheduler_bokeh.py 2018-09-07 18:22:57.000000000 +0200
@@ -1,6 +1,7 @@
from __future__ import print_function, division, absolute_import
import json
+import re
import sys
from time import sleep
@@ -48,7 +49,9 @@
'individual-nprocessing',
'individual-profile']:
response = yield http_client.fetch('http://localhost:%d/%s' % (port, suffix))
- assert 'bokeh' in response.body.decode().lower()
+ body = response.body.decode()
+ assert 'bokeh' in body.lower()
+ assert not re.search("href=./", body) # no absolute links
response = yield http_client.fetch('http://localhost:%d/individual-plots.json' % port)
response = json.loads(response.body.decode())
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/bokeh/tests/test_scheduler_bokeh_html.py new/distributed-1.23.2/distributed/bokeh/tests/test_scheduler_bokeh_html.py
--- old/distributed-1.23.1/distributed/bokeh/tests/test_scheduler_bokeh_html.py 2018-08-30 15:00:17.000000000 +0200
+++ new/distributed-1.23.2/distributed/bokeh/tests/test_scheduler_bokeh_html.py 2018-09-07 18:22:57.000000000 +0200
@@ -1,6 +1,7 @@
from __future__ import print_function, division, absolute_import
import json
+import re
import xml.etree.ElementTree
import pytest
@@ -40,6 +41,7 @@
json.loads(body)
else:
assert xml.etree.ElementTree.fromstring(body) is not None
+ assert not re.search("href=./", body) # no absolute links
@gen_cluster(client=True,
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/client.py new/distributed-1.23.2/distributed/client.py
--- old/distributed-1.23.1/distributed/client.py 2018-09-06 14:05:25.000000000 +0200
+++ new/distributed-1.23.2/distributed/client.py 2018-09-14 17:34:05.000000000 +0200
@@ -77,12 +77,14 @@
def _get_global_client():
- for k in sorted(_global_clients, reverse=True):
+ L = sorted(list(_global_clients), reverse=True)
+ for k in L:
c = _global_clients[k]
if c.status != 'closed':
return c
else:
del _global_clients[k]
+ del L
return None
@@ -3567,6 +3569,18 @@
Additionally, you can also add more futures to this object during
computation with the ``.add`` method
+ Parameters
+ ----------
+ futures: Collection of futures
+ A list of Future objects to be iterated over in the order in which they
+ complete
+ with_results: bool (False)
+ Whether to wait and include results of futures as well;
+ in this case `as_completed` yields a tuple of (future, result)
+ raise_errors: bool (True)
+ Whether we should raise when the result of a future raises an exception;
+ only affects behavior when `with_results=True`.
+
Examples
--------
>>> x, y, z = client.map(inc, [1, 2, 3]) # doctest: +SKIP
@@ -3602,7 +3616,7 @@
3
"""
- def __init__(self, futures=None, loop=None, with_results=False):
+ def __init__(self, futures=None, loop=None, with_results=False, raise_errors=True):
if futures is None:
futures = []
self.futures = defaultdict(lambda: 0)
@@ -3612,6 +3626,7 @@
self.condition = Condition()
self.thread_condition = threading.Condition()
self.with_results = with_results
+ self.raise_errors = raise_errors
if futures:
self.update(futures)
@@ -3626,12 +3641,9 @@
try:
yield _wait(future)
except CancelledError:
- del self.futures[future]
- if not self.futures:
- self._notify()
- return
+ pass
if self.with_results:
- result = yield future._result()
+ result = yield future._result(raiseit=False)
with self.lock:
self.futures[future] -= 1
if not self.futures[future]:
@@ -3680,13 +3692,21 @@
def __aiter__(self):
return self
+ def _get_and_raise(self):
+ res = self.queue.get()
+ if self.with_results:
+ future, result = res
+ if self.raise_errors and future.status == 'error':
+ six.reraise(*result)
+ return res
+
def __next__(self):
while self.queue.empty():
if self.is_empty():
raise StopIteration()
with self.thread_condition:
self.thread_condition.wait(timeout=0.100)
- return self.queue.get()
+ return self._get_and_raise()
@gen.coroutine
def __anext__(self):
@@ -3697,7 +3717,7 @@
raise StopAsyncIteration
yield self.condition.wait()
- raise gen.Return(self.queue.get())
+ raise gen.Return(self._get_and_raise())
next = __next__
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/tests/py3_test_client.py new/distributed-1.23.2/distributed/tests/py3_test_client.py
--- old/distributed-1.23.1/distributed/tests/py3_test_client.py 2018-07-20 17:53:29.000000000 +0200
+++ new/distributed-1.23.2/distributed/tests/py3_test_client.py 2018-09-07 13:50:12.000000000 +0200
@@ -79,7 +79,7 @@
yield f()
- assert L == [x]
+ assert L == [x, y]
def test_async_with(loop):
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/tests/test_as_completed.py new/distributed-1.23.2/distributed/tests/test_as_completed.py
--- old/distributed-1.23.1/distributed/tests/test_as_completed.py 2018-07-20 17:53:29.000000000 +0200
+++ new/distributed-1.23.2/distributed/tests/test_as_completed.py 2018-09-07 13:50:12.000000000 +0200
@@ -1,4 +1,5 @@
from collections import Iterator
+from concurrent.futures._base import CancelledError
from operator import add
import random
from time import sleep
@@ -9,7 +10,7 @@
from distributed import Client
from distributed.client import _as_completed, as_completed, _first_completed
from distributed.compatibility import Empty, StopAsyncIteration, Queue
-from distributed.utils_test import cluster, gen_cluster, inc
+from distributed.utils_test import cluster, gen_cluster, inc, throws
from distributed.utils_test import loop # noqa: F401
@@ -118,26 +119,30 @@
def test_as_completed_cancel(loop):
with cluster() as (s, [a, b]):
with Client(s['address'], loop=loop) as c:
- x = c.submit(sleep, 1)
+ x = c.submit(inc, 1)
y = c.submit(inc, 1)
ac = as_completed([x, y])
x.cancel()
- assert next(ac) is y
+ assert next(ac) is x or y
+ assert next(ac) is y or x
with pytest.raises(Empty):
ac.queue.get(timeout=0.1)
- assert list(as_completed([x, y, x])) == [y]
+ res = list(as_completed([x, y, x]))
+ assert len(res) == 3
+ assert set(res) == {x, y}
+ assert res.count(x) == 2
def test_as_completed_cancel_last(loop):
with cluster() as (s, [a, b]):
with Client(s['address'], loop=loop) as c:
- w = c.submit(sleep, 0.3)
+ w = c.submit(inc, 0.3)
x = c.submit(inc, 1)
- y = c.submit(sleep, 0.3)
+ y = c.submit(inc, 0.3)
@gen.coroutine
def _():
@@ -148,9 +153,9 @@
loop.add_callback(_)
ac = as_completed([x, y])
- result = list(ac)
+ result = set(ac)
- assert result == [x]
+ assert result == {x, y}
@gen_cluster(client=True)
@@ -168,3 +173,107 @@
with pytest.raises(StopAsyncIteration):
yield seq.__anext__()
+
+
+@gen_cluster(client=True)
+def test_as_completed_error_async(c, s, a, b):
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 1)
+
+ ac = as_completed([x, y])
+ first = yield ac.__anext__()
+ second = yield ac.__anext__()
+ result = {first, second}
+
+ assert result == {x, y}
+ assert x.status == 'error'
+ assert y.status == 'finished'
+
+
+def test_as_completed_error(loop):
+ with cluster() as (s, [a, b]):
+ with Client(s['address'], loop=loop) as c:
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 1)
+
+ ac = as_completed([x, y])
+ result = set(ac)
+
+ assert result == {x, y}
+ assert x.status == 'error'
+ assert y.status == 'finished'
+
+
+def test_as_completed_with_results(loop):
+ with cluster() as (s, [a, b]):
+ with Client(s['address'], loop=loop) as c:
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 5)
+ z = c.submit(inc, 1)
+
+ ac = as_completed([x, y, z], with_results=True)
+ y.cancel()
+ with pytest.raises(RuntimeError) as exc:
+ res = list(ac)
+ assert str(exc.value) == 'hello!'
+
+
+@gen_cluster(client=True)
+def test_as_completed_with_results_async(c, s, a, b):
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 5)
+ z = c.submit(inc, 1)
+
+ ac = as_completed([x, y, z], with_results=True)
+ y.cancel()
+ with pytest.raises(RuntimeError) as exc:
+ first = yield ac.__anext__()
+ second = yield ac.__anext__()
+ third = yield ac.__anext__()
+ assert str(exc.value) == 'hello!'
+
+
+def test_as_completed_with_results_no_raise(loop):
+ with cluster() as (s, [a, b]):
+ with Client(s['address'], loop=loop) as c:
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 5)
+ z = c.submit(inc, 1)
+
+ ac = as_completed([x, y, z], with_results=True, raise_errors=False)
+ y.cancel()
+ res = list(ac)
+
+ dd = {r[0]: r[1:] for r in res}
+ assert set(dd.keys()) == {y, x, z}
+ assert x.status == 'error'
+ assert y.status == 'cancelled'
+ assert z.status == 'finished'
+
+ assert isinstance(dd[y][0], CancelledError)
+ assert isinstance(dd[x][0][1], RuntimeError)
+ assert dd[z][0] == 2
+
+
+@gen_cluster(client=True)
+def test_as_completed_with_results_no_raise_async(c, s, a, b):
+ x = c.submit(throws, 1)
+ y = c.submit(inc, 5)
+ z = c.submit(inc, 1)
+
+ ac = as_completed([x, y, z], with_results=True, raise_errors=False)
+ y.cancel()
+ first = yield ac.__anext__()
+ second = yield ac.__anext__()
+ third = yield ac.__anext__()
+ res = [first, second, third]
+
+ dd = {r[0]: r[1:] for r in res}
+ assert set(dd.keys()) == {y, x, z}
+ assert x.status == 'error'
+ assert y.status == 'cancelled'
+ assert z.status == 'finished'
+
+ assert isinstance(dd[y][0], CancelledError)
+ assert isinstance(dd[x][0][1], RuntimeError)
+ assert dd[z][0] == 2
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/utils_test.py new/distributed-1.23.2/distributed/utils_test.py
--- old/distributed-1.23.1/distributed/utils_test.py 2018-08-30 16:15:06.000000000 +0200
+++ new/distributed-1.23.2/distributed/utils_test.py 2018-09-07 19:16:37.000000000 +0200
@@ -142,7 +142,8 @@
if PY2: # no forkserver, so no extra procs
for child in psutil.Process().children(recursive=True):
- child.terminate()
+ with ignoring(psutil.NoSuchProcess):
+ child.terminate()
_global_clients.clear()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed/worker.py new/distributed-1.23.2/distributed/worker.py
--- old/distributed-1.23.1/distributed/worker.py 2018-08-30 19:13:22.000000000 +0200
+++ new/distributed-1.23.2/distributed/worker.py 2018-09-15 00:16:39.000000000 +0200
@@ -2121,7 +2121,7 @@
for dep in self.dependencies.pop(key, ()):
if dep in self.dependents:
- self.dependents[dep].remove(key)
+ self.dependents[dep].discard(key)
if not self.dependents[dep] and self.dep_state[dep] in ('waiting', 'flight'):
self.release_dep(dep)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/distributed.egg-info/PKG-INFO new/distributed-1.23.2/distributed.egg-info/PKG-INFO
--- old/distributed-1.23.1/distributed.egg-info/PKG-INFO 2018-09-06 14:11:46.000000000 +0200
+++ new/distributed-1.23.2/distributed.egg-info/PKG-INFO 2018-09-17 15:57:35.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: distributed
-Version: 1.23.1
+Version: 1.23.2
Summary: Distributed scheduler for Dask
Home-page: https://distributed.readthedocs.io/en/latest/
Author: Matthew Rocklin
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/docs/source/changelog.rst new/distributed-1.23.2/docs/source/changelog.rst
--- old/distributed-1.23.1/docs/source/changelog.rst 2018-09-06 14:10:03.000000000 +0200
+++ new/distributed-1.23.2/docs/source/changelog.rst 2018-09-17 15:56:19.000000000 +0200
@@ -1,10 +1,16 @@
Changelog
=========
-1.23.2 - YYYY-MM-DD
+1.23.2 - 2018-09-17
-------------------
--
+- Discard dependent rather than remove (#2250) `Matthew Rocklin`_
+- Use dask_sphinx_theme `Matthew Rocklin`_
+- Drop the Bokeh index page (#2241) `John Kirkham`_
+- Revert change to keep link relative (#2242) `Matthew Rocklin`_
+- docs: Fix broken AWS link in setup.rst file (#2240) `Vladyslav Moisieienkov`_
+- Return cancelled futures in as_completed (#2233) `Chris White`_
+
1.23.1 - 2018-09-06
-------------------
@@ -776,3 +782,5 @@
.. _`Anderson Banihirwe`: https://github.com/andersy005
.. _`Yu Feng`: https://github.com/rainwoodman
.. _`Guillaume EB`: https://github.com/guillaumeeb
+.. _`Vladyslav Moisieienkov`: https://github.com/VMois
+.. _`Chris White`: https://github.com/cicdw
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/distributed-1.23.1/docs/source/setup.rst new/distributed-1.23.2/docs/source/setup.rst
--- old/distributed-1.23.1/docs/source/setup.rst 2018-07-20 17:53:29.000000000 +0200
+++ new/distributed-1.23.2/docs/source/setup.rst 2018-09-07 13:50:12.000000000 +0200
@@ -200,7 +200,7 @@
See `Cloud Deployments`_ for the latest information on deploying to Amazon
cloud.
-.. _cloud deployments: _https://dask.pydata.org/en/latest/setup/cloud.html
+.. _`Cloud Deployments`: https://dask.pydata.org/en/latest/setup/cloud.html
Using Google Cloud