Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-hypothesmith for openSUSE:Factory checked in at 2022-10-21 16:19:04 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-hypothesmith (Old) and /work/SRC/openSUSE:Factory/.python-hypothesmith.new.2275 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Package is "python-hypothesmith" Fri Oct 21 16:19:04 2022 rev:5 rq:1030202 version:0.2.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-hypothesmith/python-hypothesmith.changes 2021-04-01 14:15:51.739853099 +0200 +++ /work/SRC/openSUSE:Factory/.python-hypothesmith.new.2275/python-hypothesmith.changes 2022-10-21 16:19:05.926122430 +0200 @@ -1,0 +2,10 @@ +Thu Oct 20 13:24:58 UTC 2022 - Torsten Gruner <simmphonie@opensuse.org> + +- Update to v0.2.0 + * More efficient generation of Python identifiers + * Workaround for CPython parser bug in 3.9.8 (#16) Yes, yet another one - still, + that's why hypothesmith exists... +- Release v0.1.9 + * Fixed rare internal error when from_node() generated misplaced except: clauses + +------------------------------------------------------------------- Old: ---- hypothesmith-0.1.8.tar.gz New: ---- hypothesmith-0.2.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-hypothesmith.spec ++++++ --- /var/tmp/diff_new_pack.WlP540/_old 2022-10-21 16:19:06.590123674 +0200 +++ /var/tmp/diff_new_pack.WlP540/_new 2022-10-21 16:19:06.594123682 +0200 @@ -1,7 +1,7 @@ # # spec file for package python-hypothesmith-test # -# Copyright (c) 2021 SUSE LLC +# Copyright (c) 2022 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -27,7 +27,7 @@ %bcond_with test %endif Name: python-hypothesmith%{psuffix} -Version: 0.1.8 +Version: 0.2.0 Release: 0 Summary: Hypothesis strategies for generating Python programs, something like CSmith License: MPL-2.0 @@ -39,12 +39,12 @@ BuildRequires: python-rpm-macros Requires: python-base >= 3.6 Requires: python-hypothesis >= 5.46 -Requires: python-lark-parser >= 0.7.2 +Requires: python-lark >= 0.7.2 Requires: python-libcst >= 0.3.8 %if %{with test} BuildRequires: %{python_module black} BuildRequires: %{python_module hypothesis >= 5.46} -BuildRequires: %{python_module lark-parser >= 0.7.2} +BuildRequires: %{python_module lark >= 0.7.2} BuildRequires: %{python_module libcst >= 0.3.8} BuildRequires: %{python_module parso} BuildRequires: %{python_module pytest-xdist} ++++++ hypothesmith-0.1.8.tar.gz -> hypothesmith-0.2.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/CHANGELOG.md new/hypothesmith-0.2.0/CHANGELOG.md --- old/hypothesmith-0.1.8/CHANGELOG.md 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/CHANGELOG.md 2021-11-27 09:58:08.000000000 +0100 @@ -1,5 +1,13 @@ # Changelog +### 0.2.0 - 2021-11-27 +- More efficient generation of Python identifiers +- Workaround for [CPython parser bug in 3.9.8](https://bugs.python.org/issue45738) (#16) + Yes, *yet another* one - still, that's why `hypothesmith` exists... + +### 0.1.9 - 2021-08-19 +- Fixed rare internal error when `from_node()` generated misplaced `except:` clauses + ### 0.1.8 - 2021-01-10 - Now compatible with Hypothesis 5.46 and later - Increased diversity of examples generated by `from_node()` diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/PKG-INFO new/hypothesmith-0.2.0/PKG-INFO --- old/hypothesmith-0.1.8/PKG-INFO 2021-01-10 13:36:42.288650000 +0100 +++ new/hypothesmith-0.2.0/PKG-INFO 2021-11-27 09:58:16.368064200 +0100 @@ -1,75 +1,12 @@ Metadata-Version: 2.1 Name: hypothesmith -Version: 0.1.8 +Version: 0.2.0 Summary: Hypothesis strategies for generating Python programs, something like CSmith Home-page: https://github.com/Zac-HD/hypothesmith Author: Zac Hatfield-Dodds Author-email: zac@hypothesis.works License: MPL 2.0 Project-URL: Funding, https://github.com/sponsors/Zac-HD -Description: # hypothesmith - Hypothesis strategies for generating Python programs, something like CSmith. - - This is definitely pre-alpha, but if you want to play with it feel free! - You can even keep the shiny pieces when - not if - it breaks. - - Get it today with [`pip install hypothesmith`](https://pypi.org/project/hypothesmith/), - or by cloning [the GitHub repo](https://github.com/Zac-HD/hypothesmith). - - You can run the tests, such as they are, with `tox` on Python 3.6 or later. - Use `tox -va` to see what environments are available. - - ## Usage - This package provides two Hypothesis strategies for generating Python source code. - - The generated code will always be syntatically valid, and is useful for testing - parsers, linters, auto-formatters, and other tools that operate on source code. - - > DO NOT EXECUTE CODE GENERATED BY THESE STRATEGIES. - > - > It could do literally anything that running Python code is able to do, - > including changing, deleting, or uploading important data. Arbitrary - > code can be useful, but "arbitrary code execution" can be very, very bad. - - #### `hypothesmith.from_grammar(start="file_input", *, auto_target=True)` - - Generates syntactically-valid Python source code based on the grammar. - - Valid values for ``start`` are ``"single_input"``, ``"file_input"``, or - ``"eval_input"``; respectively a single interactive statement, a module or - sequence of commands read from a file, and input for the eval() function. - - If ``auto_target`` is ``True``, this strategy uses ``hypothesis.target()`` - internally to drive towards larger and more complex examples. We recommend - leaving this enabled, as the grammar is quite complex and only simple examples - tend to be generated otherwise. - - #### `hypothesmith.from_node(node=libcst.Module, *, auto_target=True)` - - Generates syntactically-valid Python source code based on the node types - defined by the [`LibCST`](https://libcst.readthedocs.io/en/latest/) project. - - You can pass any subtype of `libcst.CSTNode`. Alternatively, you can use - Hypothesis' built-in `from_type(node_type).map(lambda n: libcst.Module([n]).code`, - after Hypothesmith has registered the required strategies. However, this does - not include automatic targeting and limitations of LibCST may lead to invalid - code being generated. - - ## Notable bugs found with Hypothesmith - - [BPO-40661, a segfault in the new parser](https://bugs.python.org/issue40661), - was given maximum priority and blocked the planned release of CPython 3.9 beta1. - - [BPO-38953](https://bugs.python.org/issue38953) `tokenize` -> `untokenize` roundtrip bugs. - - [BPO-42218](https://bugs.python.org/issue42218) mishandled error case in new PEG parser. - - [`lib2to3` errors on \r in comment](https://github.com/psf/black/issues/970) - - [Black fails on files ending in a backslash](https://github.com/psf/black/issues/1012) - - [At least three round-trip bugs in LibCST](https://github.com/Instagram/LibCST#acknowledgements) - (search commits for "hypothesis") - - [Invalid code generated by LibCST](https://github.com/Instagram/LibCST/issues/287) - - ### Changelog - - Patch notes [can be found in `CHANGELOG.md`](https://github.com/Zac-HD/hypothesmith/blob/master/CHANGELOG.md). - Keywords: python testing fuzzing property-based-testing Platform: UNKNOWN Classifier: Development Status :: 4 - Beta @@ -85,3 +22,69 @@ Classifier: Topic :: Software Development :: Testing Requires-Python: >=3.6 Description-Content-Type: text/markdown +License-File: LICENSE + +# hypothesmith +Hypothesis strategies for generating Python programs, something like CSmith. + +This is definitely pre-alpha, but if you want to play with it feel free! +You can even keep the shiny pieces when - not if - it breaks. + +Get it today with [`pip install hypothesmith`](https://pypi.org/project/hypothesmith/), +or by cloning [the GitHub repo](https://github.com/Zac-HD/hypothesmith). + +You can run the tests, such as they are, with `tox` on Python 3.6 or later. +Use `tox -va` to see what environments are available. + +## Usage +This package provides two Hypothesis strategies for generating Python source code. + +The generated code will always be syntatically valid, and is useful for testing +parsers, linters, auto-formatters, and other tools that operate on source code. + +> DO NOT EXECUTE CODE GENERATED BY THESE STRATEGIES. +> +> It could do literally anything that running Python code is able to do, +> including changing, deleting, or uploading important data. Arbitrary +> code can be useful, but "arbitrary code execution" can be very, very bad. + +#### `hypothesmith.from_grammar(start="file_input", *, auto_target=True)` + +Generates syntactically-valid Python source code based on the grammar. + +Valid values for ``start`` are ``"single_input"``, ``"file_input"``, or +``"eval_input"``; respectively a single interactive statement, a module or +sequence of commands read from a file, and input for the eval() function. + +If ``auto_target`` is ``True``, this strategy uses ``hypothesis.target()`` +internally to drive towards larger and more complex examples. We recommend +leaving this enabled, as the grammar is quite complex and only simple examples +tend to be generated otherwise. + +#### `hypothesmith.from_node(node=libcst.Module, *, auto_target=True)` + +Generates syntactically-valid Python source code based on the node types +defined by the [`LibCST`](https://libcst.readthedocs.io/en/latest/) project. + +You can pass any subtype of `libcst.CSTNode`. Alternatively, you can use +Hypothesis' built-in `from_type(node_type).map(lambda n: libcst.Module([n]).code`, +after Hypothesmith has registered the required strategies. However, this does +not include automatic targeting and limitations of LibCST may lead to invalid +code being generated. + +## Notable bugs found with Hypothesmith +- [BPO-40661, a segfault in the new parser](https://bugs.python.org/issue40661), + was given maximum priority and blocked the planned release of CPython 3.9 beta1. +- [BPO-38953](https://bugs.python.org/issue38953) `tokenize` -> `untokenize` roundtrip bugs. +- [BPO-42218](https://bugs.python.org/issue42218) mishandled error case in new PEG parser. +- [`lib2to3` errors on \r in comment](https://github.com/psf/black/issues/970) +- [Black fails on files ending in a backslash](https://github.com/psf/black/issues/1012) +- [At least three round-trip bugs in LibCST](https://github.com/Instagram/LibCST#acknowledgements) + ([search commits for "hypothesis"](https://github.com/Instagram/LibCST/search?q=Fix+hypothesis&type=commits)) +- [Invalid code generated by LibCST](https://github.com/Instagram/LibCST/issues/287) + +### Changelog + +Patch notes [can be found in `CHANGELOG.md`](https://github.com/Zac-HD/hypothesmith/blob/master/CHANGELOG.md). + + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/README.md new/hypothesmith-0.2.0/README.md --- old/hypothesmith-0.1.8/README.md 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/README.md 2021-11-27 09:58:08.000000000 +0100 @@ -54,7 +54,7 @@ - [`lib2to3` errors on \r in comment](https://github.com/psf/black/issues/970) - [Black fails on files ending in a backslash](https://github.com/psf/black/issues/1012) - [At least three round-trip bugs in LibCST](https://github.com/Instagram/LibCST#acknowledgements) - (search commits for "hypothesis") + ([search commits for "hypothesis"](https://github.com/Instagram/LibCST/search?q=Fix+hypothesis&type=commits)) - [Invalid code generated by LibCST](https://github.com/Instagram/LibCST/issues/287) ### Changelog diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/deps/check.txt new/hypothesmith-0.2.0/deps/check.txt --- old/hypothesmith-0.1.8/deps/check.txt 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/deps/check.txt 2021-11-27 09:58:08.000000000 +0100 @@ -1,37 +1,24 @@ # -# This file is autogenerated by pip-compile +# This file is autogenerated by pip-compile with python 3.7 # To update, run: # # pip-compile --output-file=deps/check.txt deps/check.in # -appdirs==1.4.4 - # via black -attrs==20.3.0 +attrs==21.2.0 # via flake8-bugbear autoflake==1.4 # via shed -bandit==1.7.0 +bandit==1.7.1 # via flake8-bandit -black==20.8b1 +black==21.11b1 # via shed -click==7.1.2 - # via - # black - # pybetter - # pyemojify +click==8.0.3 + # via black colorama==0.4.4 - # via bandit -flake8-bandit==2.1.2 - # via -r deps/check.in -flake8-bugbear==20.11.1 - # via -r deps/check.in -flake8-comprehensions==3.3.1 - # via -r deps/check.in -flake8-docstrings==1.5.0 - # via -r deps/check.in -flake8-polyfill==1.0.2 - # via flake8-bandit -flake8==3.8.4 + # via + # bandit + # click +flake8==4.0.1 # via # -r deps/check.in # flake8-bandit @@ -39,82 +26,90 @@ # flake8-comprehensions # flake8-docstrings # flake8-polyfill -gitdb==4.0.5 +flake8-bandit==2.1.2 + # via -r deps/check.in +flake8-bugbear==21.9.2 + # via -r deps/check.in +flake8-comprehensions==3.7.0 + # via -r deps/check.in +flake8-docstrings==1.6.0 + # via -r deps/check.in +flake8-polyfill==1.0.2 + # via flake8-bandit +gitdb==4.0.9 # via gitpython -gitpython==3.1.12 +gitpython==3.1.24 # via bandit -importlib-metadata==3.3.0 +importlib-metadata==4.2.0 # via + # click # flake8 # flake8-comprehensions # stevedore -isort==5.7.0 +isort==5.10.1 + # via shed +libcst==0.3.23 # via shed -libcst==0.3.16 - # via pybetter mccabe==0.6.1 # via flake8 +mypy==0.910 + # via -r deps/check.in mypy-extensions==0.4.3 # via # black # mypy # typing-inspect -mypy==0.790 - # via -r deps/check.in -pathspec==0.8.1 +pathspec==0.9.0 # via black -pbr==5.5.1 +pbr==5.8.0 # via stevedore -pybetter==0.3.5 - # via shed -pycodestyle==2.6.0 +platformdirs==2.4.0 + # via black +pycodestyle==2.8.0 # via # flake8 # flake8-bandit -pydocstyle==5.1.1 +pydocstyle==6.1.1 # via flake8-docstrings -pyemojify==0.2.0 - # via pybetter -pyflakes==2.2.0 +pyflakes==2.4.0 # via # autoflake # flake8 -pygments==2.7.3 - # via pybetter -pyupgrade==2.7.4 +pyupgrade==2.29.1 # via shed -pyyaml==5.3.1 +pyyaml==6.0 # via # bandit # libcst -regex==2020.11.13 +regex==2021.11.10 # via black -shed==0.2.5 +shed==0.5.3 # via -r deps/check.in -six==1.15.0 - # via bandit -smmap==3.0.4 +smmap==5.0.0 # via gitdb -snowballstemmer==2.0.0 +snowballstemmer==2.2.0 # via pydocstyle -stevedore==3.3.0 +stevedore==3.5.0 # via bandit -tokenize-rt==4.0.0 +tokenize-rt==4.2.1 # via pyupgrade toml==0.10.2 + # via mypy +tomli==1.2.2 # via black -typed-ast==1.4.2 +typed-ast==1.4.3 # via # black # mypy -typing-extensions==3.7.4.3 +typing-extensions==4.0.0 # via # black + # gitpython # importlib-metadata # libcst # mypy # typing-inspect -typing-inspect==0.6.0 +typing-inspect==0.7.1 # via libcst -zipp==3.4.0 +zipp==3.6.0 # via importlib-metadata diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/deps/test.txt new/hypothesmith-0.2.0/deps/test.txt --- old/hypothesmith-0.1.8/deps/test.txt 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/deps/test.txt 2021-11-27 09:58:08.000000000 +0100 @@ -1,90 +1,93 @@ # -# This file is autogenerated by pip-compile +# This file is autogenerated by pip-compile with python 3.7 # To update, run: # # pip-compile --output-file=deps/test.txt deps/test.in setup.py # -apipkg==1.5 - # via execnet -appdirs==1.4.4 - # via black atomicwrites==1.4.0 # via pytest -attrs==20.3.0 +attrs==21.2.0 # via # hypothesis # pytest -black==20.8b1 +black==21.11b1 # via -r deps/test.in -click==7.1.2 +click==8.0.3 # via black colorama==0.4.4 - # via pytest -coverage==5.3.1 + # via + # click + # pytest +coverage[toml]==6.2 # via pytest-cov -execnet==1.7.1 +execnet==1.9.0 # via pytest-xdist -hypothesis==6.0.0 +hypothesis==6.27.2 # via hypothesmith (setup.py) -importlib-metadata==3.3.0 +importlib-metadata==4.8.2 # via + # click # pluggy # pytest iniconfig==1.1.1 # via pytest -lark-parser==0.11.1 +lark-parser==0.12.0 # via hypothesmith (setup.py) -libcst==0.3.16 +libcst==0.3.23 # via hypothesmith (setup.py) mypy-extensions==0.4.3 # via # black # typing-inspect -packaging==20.8 +packaging==21.3 # via pytest -parso==0.8.1 +parso==0.8.2 # via -r deps/test.in -pathspec==0.8.1 +pathspec==0.9.0 # via black -pluggy==0.13.1 +platformdirs==2.4.0 + # via black +pluggy==1.0.0 # via pytest -py==1.10.0 +py==1.11.0 # via # pytest # pytest-forked -pyparsing==2.4.7 +pyparsing==3.0.6 # via packaging -pytest-cov==2.10.1 - # via -r deps/test.in -pytest-forked==1.3.0 - # via pytest-xdist -pytest-xdist==2.2.0 - # via -r deps/test.in -pytest==6.2.1 +pytest==6.2.5 # via # -r deps/test.in # pytest-cov # pytest-forked # pytest-xdist -pyyaml==5.3.1 +pytest-cov==3.0.0 + # via -r deps/test.in +pytest-forked==1.3.0 + # via pytest-xdist +pytest-xdist==2.4.0 + # via -r deps/test.in +pyyaml==6.0 # via libcst -regex==2020.11.13 +regex==2021.11.10 # via black -sortedcontainers==2.3.0 +sortedcontainers==2.4.0 # via hypothesis toml==0.10.2 + # via pytest +tomli==1.2.2 # via # black - # pytest -typed-ast==1.4.2 + # coverage +typed-ast==1.5.0 # via black -typing-extensions==3.7.4.3 +typing-extensions==4.0.0 # via # black # importlib-metadata # libcst # typing-inspect -typing-inspect==0.6.0 +typing-inspect==0.7.1 # via libcst -zipp==3.4.0 +zipp==3.6.0 # via importlib-metadata diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/src/hypothesmith/__init__.py new/hypothesmith-0.2.0/src/hypothesmith/__init__.py --- old/hypothesmith-0.1.8/src/hypothesmith/__init__.py 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/src/hypothesmith/__init__.py 2021-11-27 09:58:08.000000000 +0100 @@ -3,5 +3,5 @@ from hypothesmith.cst import from_node from hypothesmith.syntactic import from_grammar -__version__ = "0.1.8" +__version__ = "0.2.0" __all__ = ["from_grammar", "from_node"] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/src/hypothesmith/cst.py new/hypothesmith-0.2.0/src/hypothesmith/cst.py --- old/hypothesmith-0.1.8/src/hypothesmith/cst.py 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/src/hypothesmith/cst.py 2021-11-27 09:58:08.000000000 +0100 @@ -76,9 +76,12 @@ [libcst.AsName, st.from_type(libcst.Name)], [libcst.AnnAssign, infer, infer, infer], [libcst.Assign, nonempty_seq(libcst.AssignTarget)], + [libcst.Await, infer, st.just(()), st.just(()), nonempty_whitespace], + [libcst.Attribute, infer, infer, infer], [libcst.Comparison, infer, nonempty_seq(libcst.ComparisonTarget)], [libcst.Decorator, st.from_type(libcst.Name) | st.from_type(libcst.Attribute)], [libcst.EmptyLine, infer, infer, infer], + [libcst.ExceptHandler, infer, infer, infer, infer, nonempty_whitespace, infer], [libcst.Global, nonempty_seq(libcst.NameItem)], [libcst.Import, nonempty_seq(libcst.ImportAlias)], [ @@ -132,6 +135,7 @@ # If a Try node has no `except` clause, it *must* have a `finally` clause and # *must not* have an `else` clause. With one or more except clauses, it may # have an else and/or a finally, or neither. +# The .map() ensures that any bare-`except:` clauses are ordered last. st.register_type_strategy( libcst.Try, st.builds(libcst.Try, finalbody=st.from_type(libcst.Finally)) @@ -142,7 +146,7 @@ st.deferred(lambda: st.from_type(libcst.ExceptHandler)), min_size=1, unique_by=lambda caught: caught.type, - ), + ).map(lambda xs: sorted(xs, key=lambda x: x.type is None)), orelse=infer, finalbody=infer, ), @@ -210,6 +214,10 @@ st.register_type_strategy(libcst.BooleanOperation, boolean_op_with_whitespace()) +st.register_type_strategy( + libcst.ComparisonTarget, builds_filtering(libcst.ComparisonTarget) +) + # TODO: this works pretty well, but it's also a pretty poor trick for performance. # Instead of filtering, we should continue expanding the specific node diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/src/hypothesmith/syntactic.py new/hypothesmith-0.2.0/src/hypothesmith/syntactic.py --- old/hypothesmith-0.1.8/src/hypothesmith/syntactic.py 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/src/hypothesmith/syntactic.py 2021-11-27 09:58:08.000000000 +0100 @@ -2,7 +2,6 @@ import ast import dis -import re import sys import urllib.request from functools import lru_cache @@ -10,6 +9,7 @@ from hypothesis import assume, strategies as st from hypothesis.extra.lark import LarkStrategy +from hypothesis.internal.charmap import _union_intervals from lark import Lark from lark.indenter import Indenter @@ -35,19 +35,25 @@ } +def _chars_to_regex_set(s: list) -> str: + spans = tuple((ord(c), ord(c)) for c in s) + return "".join( + chr(a) if a == b else f"{chr(a)}-{chr(b)}" + for a, b in _union_intervals(spans, spans) + ) + + @lru_cache() def identifiers() -> st.SearchStrategy[str]: - _lead = [] - _subs = [] + lead = [] + subs = [] for c in map(chr, range(sys.maxunicode + 1)): - if not utf8_encodable(c): - continue - if c.isidentifier(): - _lead.append(c) # e.g. "a" - if ("_" + c).isidentifier(): - _subs.append(c) # e.g. "1" - pattern = "[{}][{}]*".format(re.escape("".join(_lead)), re.escape("".join(_subs))) - return st.from_regex(pattern, fullmatch=True).filter(str.isidentifier) + if utf8_encodable(c) and ("_" + c).isidentifier(): + subs.append(c) # e.g. "1" + if c.isidentifier(): + lead.append(c) # e.g. "a" + pattern = f"[{_chars_to_regex_set(lead)}][{_chars_to_regex_set(subs)}]*" + return st.from_regex(pattern, fullmatch=True) class PythonIndenter(Indenter): @@ -121,9 +127,15 @@ # of bizzare errors in CPython, especially with the new PEG parser, # and so we maintain this extra clause to ensure that we get a decent # error message out of it. - if isinstance(err, SystemError) and sys.version_info[:3] == (3, 9, 0): + if isinstance(err, SystemError) and ( + sys.version_info[:3] == (3, 9, 0) + or sys.version_info[:3] >= (3, 9, 8) + and str(err) == "Negative size passed to PyUnicode_New" + ): # We've triggered https://bugs.python.org/issue42218 - it's been # fixed upstream, so we'll treat it as if it were a SyntaxError. + # Or the new https://bugs.python.org/issue45738 which makes me + # wish CPython would start running proptests in CI already. assume(False) source_code = ascii("".join(draw_state.result[count:])) raise type(err)( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/src/hypothesmith.egg-info/PKG-INFO new/hypothesmith-0.2.0/src/hypothesmith.egg-info/PKG-INFO --- old/hypothesmith-0.1.8/src/hypothesmith.egg-info/PKG-INFO 2021-01-10 13:36:41.000000000 +0100 +++ new/hypothesmith-0.2.0/src/hypothesmith.egg-info/PKG-INFO 2021-11-27 09:58:16.000000000 +0100 @@ -1,75 +1,12 @@ Metadata-Version: 2.1 Name: hypothesmith -Version: 0.1.8 +Version: 0.2.0 Summary: Hypothesis strategies for generating Python programs, something like CSmith Home-page: https://github.com/Zac-HD/hypothesmith Author: Zac Hatfield-Dodds Author-email: zac@hypothesis.works License: MPL 2.0 Project-URL: Funding, https://github.com/sponsors/Zac-HD -Description: # hypothesmith - Hypothesis strategies for generating Python programs, something like CSmith. - - This is definitely pre-alpha, but if you want to play with it feel free! - You can even keep the shiny pieces when - not if - it breaks. - - Get it today with [`pip install hypothesmith`](https://pypi.org/project/hypothesmith/), - or by cloning [the GitHub repo](https://github.com/Zac-HD/hypothesmith). - - You can run the tests, such as they are, with `tox` on Python 3.6 or later. - Use `tox -va` to see what environments are available. - - ## Usage - This package provides two Hypothesis strategies for generating Python source code. - - The generated code will always be syntatically valid, and is useful for testing - parsers, linters, auto-formatters, and other tools that operate on source code. - - > DO NOT EXECUTE CODE GENERATED BY THESE STRATEGIES. - > - > It could do literally anything that running Python code is able to do, - > including changing, deleting, or uploading important data. Arbitrary - > code can be useful, but "arbitrary code execution" can be very, very bad. - - #### `hypothesmith.from_grammar(start="file_input", *, auto_target=True)` - - Generates syntactically-valid Python source code based on the grammar. - - Valid values for ``start`` are ``"single_input"``, ``"file_input"``, or - ``"eval_input"``; respectively a single interactive statement, a module or - sequence of commands read from a file, and input for the eval() function. - - If ``auto_target`` is ``True``, this strategy uses ``hypothesis.target()`` - internally to drive towards larger and more complex examples. We recommend - leaving this enabled, as the grammar is quite complex and only simple examples - tend to be generated otherwise. - - #### `hypothesmith.from_node(node=libcst.Module, *, auto_target=True)` - - Generates syntactically-valid Python source code based on the node types - defined by the [`LibCST`](https://libcst.readthedocs.io/en/latest/) project. - - You can pass any subtype of `libcst.CSTNode`. Alternatively, you can use - Hypothesis' built-in `from_type(node_type).map(lambda n: libcst.Module([n]).code`, - after Hypothesmith has registered the required strategies. However, this does - not include automatic targeting and limitations of LibCST may lead to invalid - code being generated. - - ## Notable bugs found with Hypothesmith - - [BPO-40661, a segfault in the new parser](https://bugs.python.org/issue40661), - was given maximum priority and blocked the planned release of CPython 3.9 beta1. - - [BPO-38953](https://bugs.python.org/issue38953) `tokenize` -> `untokenize` roundtrip bugs. - - [BPO-42218](https://bugs.python.org/issue42218) mishandled error case in new PEG parser. - - [`lib2to3` errors on \r in comment](https://github.com/psf/black/issues/970) - - [Black fails on files ending in a backslash](https://github.com/psf/black/issues/1012) - - [At least three round-trip bugs in LibCST](https://github.com/Instagram/LibCST#acknowledgements) - (search commits for "hypothesis") - - [Invalid code generated by LibCST](https://github.com/Instagram/LibCST/issues/287) - - ### Changelog - - Patch notes [can be found in `CHANGELOG.md`](https://github.com/Zac-HD/hypothesmith/blob/master/CHANGELOG.md). - Keywords: python testing fuzzing property-based-testing Platform: UNKNOWN Classifier: Development Status :: 4 - Beta @@ -85,3 +22,69 @@ Classifier: Topic :: Software Development :: Testing Requires-Python: >=3.6 Description-Content-Type: text/markdown +License-File: LICENSE + +# hypothesmith +Hypothesis strategies for generating Python programs, something like CSmith. + +This is definitely pre-alpha, but if you want to play with it feel free! +You can even keep the shiny pieces when - not if - it breaks. + +Get it today with [`pip install hypothesmith`](https://pypi.org/project/hypothesmith/), +or by cloning [the GitHub repo](https://github.com/Zac-HD/hypothesmith). + +You can run the tests, such as they are, with `tox` on Python 3.6 or later. +Use `tox -va` to see what environments are available. + +## Usage +This package provides two Hypothesis strategies for generating Python source code. + +The generated code will always be syntatically valid, and is useful for testing +parsers, linters, auto-formatters, and other tools that operate on source code. + +> DO NOT EXECUTE CODE GENERATED BY THESE STRATEGIES. +> +> It could do literally anything that running Python code is able to do, +> including changing, deleting, or uploading important data. Arbitrary +> code can be useful, but "arbitrary code execution" can be very, very bad. + +#### `hypothesmith.from_grammar(start="file_input", *, auto_target=True)` + +Generates syntactically-valid Python source code based on the grammar. + +Valid values for ``start`` are ``"single_input"``, ``"file_input"``, or +``"eval_input"``; respectively a single interactive statement, a module or +sequence of commands read from a file, and input for the eval() function. + +If ``auto_target`` is ``True``, this strategy uses ``hypothesis.target()`` +internally to drive towards larger and more complex examples. We recommend +leaving this enabled, as the grammar is quite complex and only simple examples +tend to be generated otherwise. + +#### `hypothesmith.from_node(node=libcst.Module, *, auto_target=True)` + +Generates syntactically-valid Python source code based on the node types +defined by the [`LibCST`](https://libcst.readthedocs.io/en/latest/) project. + +You can pass any subtype of `libcst.CSTNode`. Alternatively, you can use +Hypothesis' built-in `from_type(node_type).map(lambda n: libcst.Module([n]).code`, +after Hypothesmith has registered the required strategies. However, this does +not include automatic targeting and limitations of LibCST may lead to invalid +code being generated. + +## Notable bugs found with Hypothesmith +- [BPO-40661, a segfault in the new parser](https://bugs.python.org/issue40661), + was given maximum priority and blocked the planned release of CPython 3.9 beta1. +- [BPO-38953](https://bugs.python.org/issue38953) `tokenize` -> `untokenize` roundtrip bugs. +- [BPO-42218](https://bugs.python.org/issue42218) mishandled error case in new PEG parser. +- [`lib2to3` errors on \r in comment](https://github.com/psf/black/issues/970) +- [Black fails on files ending in a backslash](https://github.com/psf/black/issues/1012) +- [At least three round-trip bugs in LibCST](https://github.com/Instagram/LibCST#acknowledgements) + ([search commits for "hypothesis"](https://github.com/Instagram/LibCST/search?q=Fix+hypothesis&type=commits)) +- [Invalid code generated by LibCST](https://github.com/Instagram/LibCST/issues/287) + +### Changelog + +Patch notes [can be found in `CHANGELOG.md`](https://github.com/Zac-HD/hypothesmith/blob/master/CHANGELOG.md). + + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/src/hypothesmith.egg-info/SOURCES.txt new/hypothesmith-0.2.0/src/hypothesmith.egg-info/SOURCES.txt --- old/hypothesmith-0.1.8/src/hypothesmith.egg-info/SOURCES.txt 2021-01-10 13:36:42.000000000 +0100 +++ new/hypothesmith-0.2.0/src/hypothesmith.egg-info/SOURCES.txt 2021-11-27 09:58:16.000000000 +0100 @@ -20,6 +20,7 @@ src/hypothesmith.egg-info/not-zip-safe src/hypothesmith.egg-info/requires.txt src/hypothesmith.egg-info/top_level.txt +tests/__init__.py tests/conftest.py tests/test_cst.py tests/test_syntactic.py diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/tests/test_syntactic.py new/hypothesmith-0.2.0/tests/test_syntactic.py --- old/hypothesmith-0.1.8/tests/test_syntactic.py 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/tests/test_syntactic.py 2021-11-27 09:58:08.000000000 +0100 @@ -103,6 +103,6 @@ @given(source_code=hypothesmith.from_grammar()) -def test_parso_from_node(source_code): +def test_parso_from_grammar(source_code): result = parso.parse(source_code).get_code() assert source_code == result diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/hypothesmith-0.1.8/tox.ini new/hypothesmith-0.2.0/tox.ini --- old/hypothesmith-0.1.8/tox.ini 2021-01-10 13:36:24.000000000 +0100 +++ new/hypothesmith-0.2.0/tox.ini 2021-11-27 09:58:08.000000000 +0100 @@ -12,7 +12,7 @@ commands = shed flake8 - mypy --config-file=tox.ini . src/ + mypy --config-file=tox.ini . [testenv:test] description = Runs pytest with posargs - `tox -e test -- -v` == `pytest -v` @@ -49,17 +49,8 @@ ignore = D1,E501,W503,S101,S310 exclude = .*,__pycache__ -[isort] -known_first_party = hypothesmith -default_section = THIRDPARTY -multi_line_output = 3 -include_trailing_comma = True -force_grid_wrap = 0 -combine_as_imports = True -line_length = 88 - [mypy] -python_version = 3.6 +python_version = 3.8 platform = linux disallow_untyped_calls = True disallow_untyped_defs = True @@ -71,5 +62,5 @@ warn_unused_ignores = True warn_unused_configs = True warn_redundant_casts = True -[mypy-test_hypothesmith] +[mypy-tests.*] ignore_errors = True