Mailinglist Archive: opensuse-commit (1903 mails)

< Previous Next >
commit python-remoto for openSUSE:Factory
Hello community,

here is the log from the commit of package python-remoto for openSUSE:Factory
checked in at 2019-04-30 12:56:01
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-remoto (Old)
and /work/SRC/openSUSE:Factory/.python-remoto.new.5536 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Package is "python-remoto"

Tue Apr 30 12:56:01 2019 rev:4 rq:697807 version:1.1.2

Changes:
--------
--- /work/SRC/openSUSE:Factory/python-remoto/python-remoto.changes
2019-02-11 21:23:59.219125701 +0100
+++ /work/SRC/openSUSE:Factory/.python-remoto.new.5536/python-remoto.changes
2019-04-30 12:56:03.970224003 +0200
@@ -1,0 +2,27 @@
+Thu Apr 25 09:25:33 UTC 2019 - pgajdos@xxxxxxxx
+
+- version update to 1.1.2
+ * Try a few different executables (not only python) to check for
+ a working one, in order of preference, starting with python3 and
+ ultimately falling back to the connection interpreter
+ * Fix an issue with remote Python interpreters that might not be
+ python, like in distros that use python3 or similar.
+ * Allow to specify --context to kubernetes connections
+ * When a remote exception happens using the JsonModuleExecute,
+ include both stderr and stdout.
+ * Create other connection backends aside from ssh and local:
+ kubernetes, podman, docker, and openshift.
+ * Adds new remote function/module execution model for non-native
+ (for execnet) backends, so that modules will work in backends
+ like kubernetes.
+ * Create a helper (remoto.connection.get()) for retrieving connection
+ backends based on strings
+ * Increase the test coverage.
+ * Allow using localhost, 127.0.0.1, and 127.0.1.1 to detect local
+ connections (before the full hostname was required, as returned by
+ socket.gethostname())
+ * No longer require creating logging loggers to pass in to connection
+ classes, it will create a basic one when undefined.
+- turn the test suite on
+
+-------------------------------------------------------------------

Old:
----
remoto-0.0.35.tar.gz

New:
----
remoto-1.1.2.tar.gz

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Other differences:
------------------
++++++ python-remoto.spec ++++++
--- /var/tmp/diff_new_pack.Tzr1lB/_old 2019-04-30 12:56:05.190223179 +0200
+++ /var/tmp/diff_new_pack.Tzr1lB/_new 2019-04-30 12:56:05.190223179 +0200
@@ -1,7 +1,7 @@
#
# spec file for package python-remoto
#
-# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany.
+# Copyright (c) 2019 SUSE LINUX GmbH, Nuernberg, Germany.
#
# All modifications and additions to the file contributed by third parties
# remain the property of their copyright owners, unless otherwise agreed
@@ -12,34 +12,33 @@
# license that conforms to the Open Source Definition (Version 1.9)
# published by the Open Source Initiative.

-# Please submit bugfixes or comments via http://bugs.opensuse.org/
+# Please submit bugfixes or comments via https://bugs.opensuse.org/
#

+
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
-%bcond_without test
Name: python-remoto
-Version: 0.0.35
+Version: 1.1.2
Release: 0
Summary: Remote command executor using ssh and Python in the remote end
License: MIT
Group: Development/Languages/Python
-Url: https://pypi.python.org/pypi/remoto/%{version}
-Source0:
https://files.pythonhosted.org/packages/01/6c/2be61c4afdfdfc5b18565def7ef40029d7bdabe9437734f02521f30c592a/remoto-%{version}.tar.gz
-BuildRequires: python-devel
+Url: https://github.com/alfredodeza/remoto
+Source0:
https://files.pythonhosted.org/packages/source/r/remoto/remoto-%{version}.tar.gz
BuildRequires: %{python_module execnet}
BuildRequires: %{python_module setuptools}
BuildRequires: %{python_module virtualenv}
BuildRequires: fdupes
+BuildRequires: python-devel
BuildRequires: python-rpm-macros
Requires: python-execnet
Requires: python-setuptools
BuildRoot: %{_tmppath}/%{name}-%{version}-build
BuildArch: noarch
-%if %{with test}
-BuildRequires: %{python_module tox >= 1.2}
+# SECTION build requirements
BuildRequires: %{python_module mock >= 1.0b1}
BuildRequires: %{python_module pytest >= 2.1.3}
-%endif
+# /SECTION
%python_subpackages

%description
@@ -60,11 +59,8 @@
%python_install
%python_expand %fdupes %{buildroot}%{$python_sitelib}

-%if %{with test}
%check
-export REMOTO_NO_VENDOR=no
-%python_exec setup.py test
-%endif
+%pytest

%files %{python_files}
%defattr(-,root,root,-)

++++++ remoto-0.0.35.tar.gz -> remoto-1.1.2.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/PKG-INFO new/remoto-1.1.2/PKG-INFO
--- old/remoto-0.0.35/PKG-INFO 2019-01-08 17:24:34.000000000 +0100
+++ new/remoto-1.1.2/PKG-INFO 2019-03-13 16:27:34.000000000 +0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: remoto
-Version: 0.0.35
+Version: 1.1.2
Summary: Execute remote commands or processes.
Home-page: http://github.com/alfredodeza/remoto
Author: Alfredo Deza
@@ -8,14 +8,15 @@
License: MIT
Description: remoto
======
- A very simplistic remote-command-executor using ``ssh`` and Python in
the
- remote end.
+ A very simplistic remote-command-executor using connections to hosts
(``ssh``,
+ local, containers, and several others are supported) and Python in the
remote
+ end.

All the heavy lifting is done by execnet, while this minimal API
provides the
bare minimum to handle easy logging and connections from the remote
end.

``remoto`` is a bit opinionated as it was conceived to replace helpers
and
- remote utilities for ``ceph-deploy`` a tool to run remote commands to
configure
+ remote utilities for ``ceph-deploy``, a tool to run remote commands to
configure
and setup the distributed file system Ceph.


@@ -31,10 +32,7 @@

This is how it would look with a basic logger passed in::

- >>> import logging
- >>> logging.basicConfig(level=logging.DEBUG)
- >>> logger = logging.getLogger('hostname')
- >>> conn = remoto.Connection('hostname', logger=logger)
+ >>> conn = remoto.Connection('hostname')
>>> run(conn, ['ls', '-a'])
INFO:hostname:Running command: ls -a
DEBUG:hostname:.
@@ -43,10 +41,8 @@
DEBUG:hostname:.bash_logout
DEBUG:hostname:.bash_profile
DEBUG:hostname:.bashrc
- DEBUG:hostname:.gem
DEBUG:hostname:.lesshst
DEBUG:hostname:.pki
- DEBUG:hostname:.puppet
DEBUG:hostname:.ssh
DEBUG:hostname:.vim
DEBUG:hostname:.viminfo
@@ -67,9 +63,9 @@
one is with ``process.run``::

>>> from remoto.process import run
- >>> from remoto import Connection
- >>> logger = logging.getLogger('myhost')
- >>> conn = Connection('myhost', logger=logger)
+ >>> from remoto import connection
+ >>> Connection = connection.get('ssh')
+ >>> conn = Connection('myhost')
>>> run(conn, ['whoami'])
INFO:myhost:Running command: whoami
DEBUG:myhost:root
@@ -97,13 +93,43 @@

Remote Functions
================
-
- To execute remote functions (ideally) you would need to define them in
a module
- and add the following to the end of that module::
-
- if __name__ == '__channelexec__':
- for item in channel:
- channel.send(eval(item))
+ There are two supported ways to execute functions on the remote side.
The
+ library that ``remoto`` uses to connect (``execnet``) only supports a
few
+ backends *natively*, and ``remoto`` has extended this ability for
other backend
+ connections like kubernetes.
+
+ The remote function capabilities are provided by
``LegacyModuleExecute`` and
+ ``JsonModuleExecute``. By default, both ``ssh`` and ``local``
connection will
+ use the legacy execution class, and everything else will use the
``legacy``
+ class. The ``ssh`` and ``local`` connections can still be forced to
use the new
+ module execution by setting::
+
+ conn.remote_import_system = 'json'
+
+
+ ``json``
+ --------
+ The default module for ``docker``, ``kubernetes``, ``podman``, and
+ ``openshift``. It does not require any magic on the module to be
executed,
+ however it is worth noting that the library *will* add the following
bit of
+ magic when sending the module to the remote end for execution::
+
+
+ if __name__ == '__main__':
+ import json, traceback
+ obj = {'return': None, 'exception': None}
+ try:
+ obj['return'] = function_name(*a)
+ except Exception:
+ obj['exception'] = traceback.format_exc()
+ try:
+ print(json.dumps(obj).decode('utf-8'))
+ except AttributeError:
+ print(json.dumps(obj))
+
+ This allows the system to execute ``function_name`` (replaced by the
real
+ function to be executed with its arguments), grab any results,
serialize them
+ with ``json`` and send them back for local processing.


If you had a function in a module named ``foo`` that looks like this::
@@ -135,8 +161,22 @@
dictionaries. Also safe to use are ints and strings.


- Automatic detection for remote connections
- ------------------------------------------
+ ``legacy``
+ ----------
+ When using the ``legacy`` execution model (the default for ``local``
and
+ ``ssh`` connections), modules are required to add the following to the
end of
+ that module::
+
+ if __name__ == '__channelexec__':
+ for item in channel:
+ channel.send(eval(item))
+
+ This piece of code is fully compatible with the ``json`` execution
model, and
+ would not cause conflicts.
+
+
+ Automatic detection for ssh connections
+ ---------------------------------------
There is automatic detection for the need to connect remotely (via
SSH) or not
that it is infered by the hostname of the current host (vs. the host
that is
connecting to).
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/README.rst new/remoto-1.1.2/README.rst
--- old/remoto-0.0.35/README.rst 2016-03-22 14:00:23.000000000 +0100
+++ new/remoto-1.1.2/README.rst 2019-03-01 16:16:01.000000000 +0100
@@ -1,13 +1,14 @@
remoto
======
-A very simplistic remote-command-executor using ``ssh`` and Python in the
-remote end.
+A very simplistic remote-command-executor using connections to hosts (``ssh``,
+local, containers, and several others are supported) and Python in the remote
+end.

All the heavy lifting is done by execnet, while this minimal API provides the
bare minimum to handle easy logging and connections from the remote end.

``remoto`` is a bit opinionated as it was conceived to replace helpers and
-remote utilities for ``ceph-deploy`` a tool to run remote commands to configure
+remote utilities for ``ceph-deploy``, a tool to run remote commands to
configure
and setup the distributed file system Ceph.


@@ -23,10 +24,7 @@

This is how it would look with a basic logger passed in::

- >>> import logging
- >>> logging.basicConfig(level=logging.DEBUG)
- >>> logger = logging.getLogger('hostname')
- >>> conn = remoto.Connection('hostname', logger=logger)
+ >>> conn = remoto.Connection('hostname')
>>> run(conn, ['ls', '-a'])
INFO:hostname:Running command: ls -a
DEBUG:hostname:.
@@ -35,10 +33,8 @@
DEBUG:hostname:.bash_logout
DEBUG:hostname:.bash_profile
DEBUG:hostname:.bashrc
- DEBUG:hostname:.gem
DEBUG:hostname:.lesshst
DEBUG:hostname:.pki
- DEBUG:hostname:.puppet
DEBUG:hostname:.ssh
DEBUG:hostname:.vim
DEBUG:hostname:.viminfo
@@ -59,9 +55,9 @@
one is with ``process.run``::

>>> from remoto.process import run
- >>> from remoto import Connection
- >>> logger = logging.getLogger('myhost')
- >>> conn = Connection('myhost', logger=logger)
+ >>> from remoto import connection
+ >>> Connection = connection.get('ssh')
+ >>> conn = Connection('myhost')
>>> run(conn, ['whoami'])
INFO:myhost:Running command: whoami
DEBUG:myhost:root
@@ -89,13 +85,43 @@

Remote Functions
================
-
-To execute remote functions (ideally) you would need to define them in a module
-and add the following to the end of that module::
-
- if __name__ == '__channelexec__':
- for item in channel:
- channel.send(eval(item))
+There are two supported ways to execute functions on the remote side. The
+library that ``remoto`` uses to connect (``execnet``) only supports a few
+backends *natively*, and ``remoto`` has extended this ability for other backend
+connections like kubernetes.
+
+The remote function capabilities are provided by ``LegacyModuleExecute`` and
+``JsonModuleExecute``. By default, both ``ssh`` and ``local`` connection will
+use the legacy execution class, and everything else will use the ``legacy``
+class. The ``ssh`` and ``local`` connections can still be forced to use the new
+module execution by setting::
+
+ conn.remote_import_system = 'json'
+
+
+``json``
+--------
+The default module for ``docker``, ``kubernetes``, ``podman``, and
+``openshift``. It does not require any magic on the module to be executed,
+however it is worth noting that the library *will* add the following bit of
+magic when sending the module to the remote end for execution::
+
+
+ if __name__ == '__main__':
+ import json, traceback
+ obj = {'return': None, 'exception': None}
+ try:
+ obj['return'] = function_name(*a)
+ except Exception:
+ obj['exception'] = traceback.format_exc()
+ try:
+ print(json.dumps(obj).decode('utf-8'))
+ except AttributeError:
+ print(json.dumps(obj))
+
+This allows the system to execute ``function_name`` (replaced by the real
+function to be executed with its arguments), grab any results, serialize them
+with ``json`` and send them back for local processing.


If you had a function in a module named ``foo`` that looks like this::
@@ -127,8 +153,22 @@
dictionaries. Also safe to use are ints and strings.


-Automatic detection for remote connections
-------------------------------------------
+``legacy``
+----------
+When using the ``legacy`` execution model (the default for ``local`` and
+``ssh`` connections), modules are required to add the following to the end of
+that module::
+
+ if __name__ == '__channelexec__':
+ for item in channel:
+ channel.send(eval(item))
+
+This piece of code is fully compatible with the ``json`` execution model, and
+would not cause conflicts.
+
+
+Automatic detection for ssh connections
+---------------------------------------
There is automatic detection for the need to connect remotely (via SSH) or not
that it is infered by the hostname of the current host (vs. the host that is
connecting to).
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/__init__.py
new/remoto-1.1.2/remoto/__init__.py
--- old/remoto-0.0.35/remoto/__init__.py 2019-01-08 17:21:16.000000000
+0100
+++ new/remoto-1.1.2/remoto/__init__.py 2019-03-13 16:26:39.000000000 +0100
@@ -4,4 +4,4 @@
from . import connection


-__version__ = '0.0.35'
+__version__ = '1.1.2'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/__init__.py
new/remoto-1.1.2/remoto/backends/__init__.py
--- old/remoto-0.0.35/remoto/backends/__init__.py 1970-01-01
01:00:00.000000000 +0100
+++ new/remoto-1.1.2/remoto/backends/__init__.py 2019-03-13
16:24:25.000000000 +0100
@@ -0,0 +1,316 @@
+import inspect
+import json
+import socket
+import sys
+import execnet
+import logging
+from remoto.process import check
+
+
+class BaseConnection(object):
+ """
+ Base class for Connection objects. Provides a generic interface to execnet
+ for setting up the connection
+ """
+ executable = ''
+ remote_import_system = 'legacy'
+
+ def __init__(self, hostname, logger=None, sudo=False, threads=1,
eager=True,
+ detect_sudo=False, interpreter=None, ssh_options=None):
+ self.sudo = sudo
+ self.hostname = hostname
+ self.ssh_options = ssh_options
+ self.logger = logger or basic_remote_logger()
+ self.remote_module = None
+ self.channel = None
+ self.global_timeout = None # wait for ever
+
+ self.interpreter = interpreter or 'python%s' % sys.version_info[0]
+
+ if eager:
+ try:
+ if detect_sudo:
+ self.sudo = self._detect_sudo()
+ self.gateway = self._make_gateway(hostname)
+ except OSError:
+ self.logger.error(
+ "Can't communicate with remote host, possibly because "
+ "%s is not installed there" % self.interpreter
+ )
+ raise
+
+ def _make_gateway(self, hostname):
+ gateway = execnet.makegateway(
+ self._make_connection_string(hostname)
+ )
+ gateway.reconfigure(py2str_as_py3str=False, py3str_as_py2str=False)
+ return gateway
+
+ def _detect_sudo(self, _execnet=None):
+ """
+ ``sudo`` detection has to create a different connection to the remote
+ host so that we can reliably ensure that ``getuser()`` will return the
+ right information.
+
+ After getting the user info it closes the connection and returns
+ a boolean
+ """
+ exc = _execnet or execnet
+ gw = exc.makegateway(
+ self._make_connection_string(self.hostname, use_sudo=False)
+ )
+
+ channel = gw.remote_exec(
+ 'import getpass; channel.send(getpass.getuser())'
+ )
+
+ result = channel.receive()
+ gw.exit()
+
+ if result == 'root':
+ return False
+ self.logger.debug('connection detected need for sudo')
+ return True
+
+ def _make_connection_string(self, hostname, _needs_ssh=None,
use_sudo=None):
+ _needs_ssh = _needs_ssh or needs_ssh
+ interpreter = self.interpreter
+ if use_sudo is not None:
+ if use_sudo:
+ interpreter = 'sudo ' + interpreter
+ elif self.sudo:
+ interpreter = 'sudo ' + interpreter
+ if _needs_ssh(hostname):
+ if self.ssh_options:
+ return 'ssh=%s %s//python=%s' % (
+ self.ssh_options, hostname, interpreter
+ )
+ else:
+ return 'ssh=%s//python=%s' % (hostname, interpreter)
+ return 'popen//python=%s' % interpreter
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, exc_type, exc_val, exc_tb):
+ self.exit()
+ return False
+
+ def cmd(self, cmd):
+ """
+ In the base connection class, this method just returns the ``cmd``
+ as-is. Other implementations will end up doing transformations to the
+ command by prefixing it with other flags needed. See
+ :class:`KubernetesConnection` for an example
+ """
+ return cmd
+
+ def execute(self, function, **kw):
+ return self.gateway.remote_exec(function, **kw)
+
+ def exit(self):
+ self.gateway.exit()
+
+ def import_module(self, module):
+ """
+ Allows remote execution of a local module. Depending on the
+ ``remote_import_system`` attribute it may use execnet's implementation
+ or remoto's own based on JSON.
+
+ .. note:: It is not possible to use execnet's remote execution model on
+ connections that aren't SSH or Local.
+ """
+ if self.remote_import_system is not None:
+ if self.remote_import_system == 'json':
+ self.remote_module = JsonModuleExecute(self, module,
self.logger)
+ else:
+ self.remote_module = LegacyModuleExecute(self.gateway, module,
self.logger)
+ else:
+ self.remote_module = LegacyModuleExecute(self.gateway, module,
self.logger)
+ return self.remote_module
+
+
+class LegacyModuleExecute(object):
+ """
+ This (now legacy) class, is the way ``execnet`` does its remote module
+ execution: it sends it over a channel, and does a send/receive for
+ exchanging information. This only works when there is native support in
+ execnet for a given connection. This currently means it would only work for
+ ssh and local (Popen) connections, and will not work for anything like
+ kubernetes or containers.
+ """
+
+ def __init__(self, gateway, module, logger=None):
+ self.channel = gateway.remote_exec(module)
+ self.module = module
+ self.logger = logger
+
+ def __getattr__(self, name):
+ if not hasattr(self.module, name):
+ msg = "module %s does not have attribute %s" % (str(self.module),
name)
+ raise AttributeError(msg)
+ docstring = self._get_func_doc(getattr(self.module, name))
+
+ def wrapper(*args):
+ arguments = self._convert_args(args)
+ if docstring:
+ self.logger.debug(docstring)
+ self.channel.send("%s(%s)" % (name, arguments))
+ try:
+ return self.channel.receive()
+ except Exception as error:
+ # Error will come as a string of a traceback, remove everything
+ # up to the actual exception since we do get garbage otherwise
+ # that points to non-existent lines in the compiled code
+ exc_line = str(error)
+ for tb_line in reversed(str(error).split('\n')):
+ if tb_line:
+ exc_line = tb_line
+ break
+ raise RuntimeError(exc_line)
+
+ return wrapper
+
+ def _get_func_doc(self, func):
+ try:
+ return getattr(func, 'func_doc').strip()
+ except AttributeError:
+ return ''
+
+ def _convert_args(self, args):
+ if args:
+ if len(args) > 1:
+ arguments = str(args).rstrip(')').lstrip('(')
+ else:
+ arguments = str(args).rstrip(',)').lstrip('(')
+ else:
+ arguments = ''
+ return arguments
+
+
+dump_template = """
+if __name__ == '__main__':
+ import json, traceback
+ obj = {'return': None, 'exception': None}
+ try:
+ obj['return'] = %s%s
+ except Exception:
+ obj['exception'] = traceback.format_exc()
+ try:
+ print(json.dumps(obj).decode('utf-8'))
+ except AttributeError:
+ print(json.dumps(obj))
+"""
+
+
+class JsonModuleExecute(object):
+ """
+ This remote execution class allows to ship Python code over to the remote
+ node, load it via ``stdin`` and call any function with arguments. The
+ resulting response is dumped over JSON so that it can get printed to
+ ``stdout``, then captured locally, loaded into regular Python and returned.
+
+ If the remote end generates an exception with a traceback, that is captured
+ as well and raised accordingly.
+ """
+
+ def __init__(self, conn, module, logger=None):
+ self.conn = conn
+ self.module = module
+ self._module_source = inspect.getsource(module)
+ self.logger = logger
+ self.python_executable = None
+
+ def __getattr__(self, name):
+ if not hasattr(self.module, name):
+ msg = "module %s does not have attribute %s" % (str(self.module),
name)
+ raise AttributeError(msg)
+ docstring = self._get_func_doc(getattr(self.module, name))
+
+ def wrapper(*args):
+ if docstring:
+ self.logger.debug(docstring)
+ if len(args):
+ source = self._module_source + dump_template % (name,
repr(args))
+ else:
+ source = self._module_source + dump_template % (name, '()')
+
+ # check python interpreter
+ if self.python_executable is None:
+ self.python_executable = get_python_executable(self.conn)
+
+ out, err, code = check(self.conn, [self.python_executable],
stdin=source.encode('utf-8'))
+ if not out:
+ if not err:
+ err = [
+ 'Traceback (most recent call last):',
+ ' File "<stdin>", in <module>',
+ 'Exception: error calling "%s"' % name
+ ]
+ if code:
+ raise Exception('Unexpected remote exception: \n%s\n%s' %
('\n'.join(out), '\n'.join(err)))
+ # at this point, there was no stdout, and the exit code was 0,
+ # we must return so that we don't fail trying to serialize back
+ # the JSON
+ return
+ response = json.loads(out[0])
+ if response['exception']:
+ raise Exception(response['exception'])
+ return response['return']
+
+ return wrapper
+
+ def _get_func_doc(self, func):
+ try:
+ return getattr(func, 'func_doc').strip()
+ except AttributeError:
+ return ''
+
+
+def basic_remote_logger():
+ logging.basicConfig()
+ logger = logging.getLogger(socket.gethostname())
+ logger.setLevel(logging.DEBUG)
+ return logger
+
+
+def needs_ssh(hostname, _socket=None):
+ """
+ Obtains remote hostname of the socket and cuts off the domain part
+ of its FQDN.
+ """
+ if hostname.lower() in ['localhost', '127.0.0.1', '127.0.1.1']:
+ return False
+ _socket = _socket or socket
+ fqdn = _socket.getfqdn()
+ if hostname == fqdn:
+ return False
+ local_hostname = _socket.gethostname()
+ local_short_hostname = local_hostname.split('.')[0]
+ if local_hostname == hostname or local_short_hostname == hostname:
+ return False
+ return True
+
+
+def get_python_executable(conn):
+ """
+ Try to determine the remote Python version so that it can be used
+ when executing. Avoids the problem of different Python versions, or distros
+ that do not use ``python`` but do ``python3``
+ """
+ # executables in order of preference:
+ executables = ['python3', 'python', 'python2.7']
+ for executable in executables:
+ conn.logger.debug('trying to determine remote python executable with
%s' % executable)
+ out, err, code = check(conn, ['which', executable])
+ if code:
+ conn.logger.warning('skipping %s, was not found in path' %
executable)
+ else:
+ try:
+ return out[0].strip()
+ except IndexError:
+ conn.logger.warning('could not parse stdout: %s' % out)
+
+ # if all fails, we just return whatever the main connection had
+ conn.logger.info('Falling back to using interpreter: %s' %
conn.interpreter)
+ return conn.interpreter
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/docker.py
new/remoto-1.1.2/remoto/backends/docker.py
--- old/remoto-0.0.35/remoto/backends/docker.py 1970-01-01 01:00:00.000000000
+0100
+++ new/remoto-1.1.2/remoto/backends/docker.py 2019-03-01 16:16:01.000000000
+0100
@@ -0,0 +1,45 @@
+from . import BaseConnection
+
+
+class DockerConnection(BaseConnection):
+ """
+ This connection class allows to (optionally) define a remote hostname
+ to connect that holds a given container::
+
+ >>> conn = DockerConnection(hostname='srv-1', container_id='asdf-lkjh')
+
+ Either ``container_id`` or ``container_name`` can be provided to connect to
+ a given container.
+
+ .. note:: ``hostname`` defaults to 'localhost' when undefined
+ """
+
+ executable = 'docker'
+ remote_import_system = 'json'
+
+ def __init__(self, hostname=None, container_id=None, container_name=None,
user=None, **kw):
+ self.hostname = hostname or 'localhost'
+ self.identifier = container_id or container_name
+ if not self.identifier:
+ raise TypeError('Either container_id or container_name must be
provided')
+ self.user = user
+ super(DockerConnection, self).__init__(hostname=self.hostname, **kw)
+
+ def command_template(self):
+ if self.user:
+ prefix = [
+ self.executable, 'exec', '-i',
+ '-u', self.user,
+ self.identifier, '/bin/sh', '-c'
+ ]
+ else:
+ prefix = [
+ self.executable, 'exec', '-i',
+ self.identifier, '/bin/sh', '-c'
+ ]
+ return prefix
+
+ def cmd(self, cmd):
+ tmpl = self.command_template()
+ tmpl.append(' '.join(cmd))
+ return tmpl
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/kubernetes.py
new/remoto-1.1.2/remoto/backends/kubernetes.py
--- old/remoto-0.0.35/remoto/backends/kubernetes.py 1970-01-01
01:00:00.000000000 +0100
+++ new/remoto-1.1.2/remoto/backends/kubernetes.py 2019-03-01
16:16:01.000000000 +0100
@@ -0,0 +1,36 @@
+from . import BaseConnection
+
+
+class KubernetesConnection(BaseConnection):
+
+ executable = 'kubectl'
+ remote_import_system = 'json'
+
+ def __init__(self, pod_name, namespace=None, context=None, **kw):
+ self.namespace = namespace
+ self.context = context
+ self.pod_name = pod_name
+ super(KubernetesConnection, self).__init__(hostname='localhost', **kw)
+
+ def command_template(self):
+ base_command = [self.executable]
+ if self.context:
+ base_command.extend(['--context', self.context])
+
+ base_command.extend(['exec', '-i'])
+
+ if self.namespace:
+ base_command.extend(['-n', self.namespace])
+
+ base_command.extend([
+ self.pod_name,
+ '--',
+ '/bin/sh',
+ '-c'
+ ])
+ return base_command
+
+ def cmd(self, cmd):
+ tmpl = self.command_template()
+ tmpl.append(' '.join(cmd))
+ return tmpl
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/local.py
new/remoto-1.1.2/remoto/backends/local.py
--- old/remoto-0.0.35/remoto/backends/local.py 1970-01-01 01:00:00.000000000
+0100
+++ new/remoto-1.1.2/remoto/backends/local.py 2019-03-01 16:16:01.000000000
+0100
@@ -0,0 +1,23 @@
+from . import BaseConnection
+import socket
+
+
+class LocalConnection(BaseConnection):
+
+ def __init__(self, **kw):
+ # hostname gets ignored, and forced to be localhost always
+ kw.pop('hostname', None)
+ super(LocalConnection, self).__init__(
+ hostname='localhost',
+ detect_sudo=False,
+ **kw
+ )
+
+ def _make_connection_string(self, hostname, _needs_ssh=None,
use_sudo=None):
+ interpreter = self.interpreter
+ if use_sudo is not None:
+ if use_sudo:
+ interpreter = 'sudo ' + interpreter
+ elif self.sudo:
+ interpreter = 'sudo ' + interpreter
+ return 'popen//python=%s' % interpreter
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/openshift.py
new/remoto-1.1.2/remoto/backends/openshift.py
--- old/remoto-0.0.35/remoto/backends/openshift.py 1970-01-01
01:00:00.000000000 +0100
+++ new/remoto-1.1.2/remoto/backends/openshift.py 2019-03-01
16:16:01.000000000 +0100
@@ -0,0 +1,6 @@
+from .kubernetes import KubernetesConnection
+
+
+class OpenshiftConnection(KubernetesConnection):
+
+ executable = 'oc'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/podman.py
new/remoto-1.1.2/remoto/backends/podman.py
--- old/remoto-0.0.35/remoto/backends/podman.py 1970-01-01 01:00:00.000000000
+0100
+++ new/remoto-1.1.2/remoto/backends/podman.py 2019-03-01 16:16:01.000000000
+0100
@@ -0,0 +1,6 @@
+from .docker import DockerConnection
+
+
+class PodmanConnection(DockerConnection):
+
+ executable = 'podman'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/backends/ssh.py
new/remoto-1.1.2/remoto/backends/ssh.py
--- old/remoto-0.0.35/remoto/backends/ssh.py 1970-01-01 01:00:00.000000000
+0100
+++ new/remoto-1.1.2/remoto/backends/ssh.py 2019-03-01 16:16:01.000000000
+0100
@@ -0,0 +1 @@
+from . import BaseConnection as SshConnection
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/connection.py
new/remoto-1.1.2/remoto/connection.py
--- old/remoto-0.0.35/remoto/connection.py 2018-12-21 18:38:02.000000000
+0100
+++ new/remoto-1.1.2/remoto/connection.py 2019-03-01 16:16:01.000000000
+0100
@@ -1,185 +1,48 @@
-import socket
-import sys
-import execnet
-
-
-#
-# Connection Object
-#
-
-class Connection(object):
-
- def __init__(self, hostname, logger=None, sudo=False, threads=1,
eager=True,
- detect_sudo=False, interpreter=None, ssh_options=None):
- self.sudo = sudo
- self.hostname = hostname
- self.ssh_options = ssh_options
- self.logger = logger or FakeRemoteLogger()
- self.remote_module = None
- self.channel = None
- self.global_timeout = None # wait for ever
-
- self.interpreter = interpreter or 'python%s' % sys.version_info[0]
-
- if eager:
- try:
- if detect_sudo:
- self.sudo = self._detect_sudo()
- self.gateway = self._make_gateway(hostname)
- except OSError:
- self.logger.error(
- "Can't communicate with remote host, possibly because "
- "%s is not installed there" % self.interpreter
- )
- raise
-
- def _make_gateway(self, hostname):
- gateway = execnet.makegateway(
- self._make_connection_string(hostname)
- )
- gateway.reconfigure(py2str_as_py3str=False, py3str_as_py2str=False)
- return gateway
-
- def _detect_sudo(self, _execnet=None):
- """
- ``sudo`` detection has to create a different connection to the remote
- host so that we can reliably ensure that ``getuser()`` will return the
- right information.
-
- After getting the user info it closes the connection and returns
- a boolean
- """
- exc = _execnet or execnet
- gw = exc.makegateway(
- self._make_connection_string(self.hostname, use_sudo=False)
- )
-
- channel = gw.remote_exec(
- 'import getpass; channel.send(getpass.getuser())'
- )
-
- result = channel.receive()
- gw.exit()
-
- if result == 'root':
- return False
- self.logger.debug('connection detected need for sudo')
- return True
-
- def _make_connection_string(self, hostname, _needs_ssh=None,
use_sudo=None):
- _needs_ssh = _needs_ssh or needs_ssh
- interpreter = self.interpreter
- if use_sudo is not None:
- if use_sudo:
- interpreter = 'sudo ' + interpreter
- elif self.sudo:
- interpreter = 'sudo ' + interpreter
- if _needs_ssh(hostname):
- if self.ssh_options:
- return 'ssh=%s %s//python=%s' % (
- self.ssh_options, hostname, interpreter)
- else:
- return 'ssh=%s//python=%s' % (hostname, interpreter)
- return 'popen//python=%s' % interpreter
-
- def __enter__(self):
- return self
-
- def __exit__(self, exc_type, exc_val, exc_tb):
- self.exit()
- return False
-
- def execute(self, function, **kw):
- return self.gateway.remote_exec(function, **kw)
-
- def exit(self):
- self.gateway.exit()
-
- def import_module(self, module):
- self.remote_module = ModuleExecute(self.gateway, module, self.logger)
- return self.remote_module
-
-
-class ModuleExecute(object):
-
- def __init__(self, gateway, module, logger=None):
- self.channel = gateway.remote_exec(module)
- self.module = module
- self.logger = logger
-
- def __getattr__(self, name):
- if not hasattr(self.module, name):
- msg = "module %s does not have attribute %s" % (str(self.module),
name)
- raise AttributeError(msg)
- docstring = self._get_func_doc(getattr(self.module, name))
-
- def wrapper(*args):
- arguments = self._convert_args(args)
- if docstring:
- self.logger.debug(docstring)
- self.channel.send("%s(%s)" % (name, arguments))
- try:
- return self.channel.receive()
- except Exception as error:
- # Error will come as a string of a traceback, remove everything
- # up to the actual exception since we do get garbage otherwise
- # that points to non-existent lines in the compiled code
- exc_line = str(error)
- for tb_line in reversed(str(error).split('\n')):
- if tb_line:
- exc_line = tb_line
- break
- raise RuntimeError(exc_line)
-
- return wrapper
-
- def _get_func_doc(self, func):
- try:
- return getattr(func, 'func_doc').strip()
- except AttributeError:
- return ''
-
- def _convert_args(self, args):
- if args:
- if len(args) > 1:
- arguments = str(args).rstrip(')').lstrip('(')
- else:
- arguments = str(args).rstrip(',)').lstrip('(')
- else:
- arguments = ''
- return arguments
-
-
-#
-# FIXME this is getting ridiculous
-#
-
-class FakeRemoteLogger:
-
- def error(self, *a, **kw):
- pass
-
- def debug(self, *a, **kw):
- pass
+import logging
+# compatibility for older clients that rely on the previous ``Connection``
class
+from remoto.backends import BaseConnection as Connection # noqa
+from remoto.backends import ssh, openshift, kubernetes, local, podman, docker

- def info(self, *a, **kw):
- pass

- def warning(self, *a, **kw):
- pass
+logger = logging.getLogger('remoto')


-def needs_ssh(hostname, _socket=None):
+def get(name, fallback='ssh'):
"""
- Obtains remote hostname of the socket and cuts off the domain part
- of its FQDN.
+ Retrieve the matching backend class from a string. If no backend can be
+ matched, it raises an error.
+
+ >>> get('ssh')
+ <class 'remoto.backends.BaseConnection'>
+ >>> get()
+ <class 'remoto.backends.BaseConnection'>
+ >>> get('non-existent')
+ <class 'remoto.backends.BaseConnection'>
+ >>> get('non-existent', 'openshift')
+ <class 'remoto.backends.openshift.OpenshiftConnection'>
"""
- _socket = _socket or socket
- fqdn = _socket.getfqdn()
- if hostname == fqdn:
- return False
- local_hostname = _socket.gethostname()
- local_short_hostname = local_hostname.split('.')[0]
- if local_hostname == hostname or local_short_hostname == hostname:
- return False
- return True
+ mapping = {
+ 'ssh': ssh.SshConnection,
+ 'oc': openshift.OpenshiftConnection,
+ 'openshift': openshift.OpenshiftConnection,
+ 'kubernetes': kubernetes.KubernetesConnection,
+ 'k8s': kubernetes.KubernetesConnection,
+ 'local': local.LocalConnection,
+ 'popen': local.LocalConnection,
+ 'localhost': local.LocalConnection,
+ 'docker': docker.DockerConnection,
+ 'podman': podman.PodmanConnection,
+ }
+ if not name:
+ # fallsback to just plain local/ssh
+ name = 'ssh'
+
+ name = name.strip().lower()
+ connection_class = mapping.get(name)
+ if not connection_class:
+ logger.warning('no connection backend found for: "%s"' % name)
+ if fallback:
+ logger.info('falling back to "%s"' % fallback)
+ # this assumes that ``fallback`` is a valid mapping name
+ return mapping.get(fallback)
+ return connection_class
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/file_sync.py
new/remoto-1.1.2/remoto/file_sync.py
--- old/remoto-0.0.35/remoto/file_sync.py 2017-07-05 21:27:28.000000000
+0200
+++ new/remoto-1.1.2/remoto/file_sync.py 2019-03-01 16:16:01.000000000
+0100
@@ -1,5 +1,6 @@
import execnet
-from .connection import Connection, FakeRemoteLogger
+from remoto.backends import basic_remote_logger
+from remoto.backends import BaseConnection as Connection


class _RSync(execnet.RSync):
@@ -26,7 +27,7 @@
same. This deviates from what execnet does because it has the flexibility
to push to different locations.
"""
- logger = logger or FakeRemoteLogger()
+ logger = logger or basic_remote_logger()
sync = _RSync(source, logger=logger)

# setup_targets
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/process.py
new/remoto-1.1.2/remoto/process.py
--- old/remoto-0.0.35/remoto/process.py 2018-12-21 19:00:03.000000000 +0100
+++ new/remoto-1.1.2/remoto/process.py 2019-03-12 20:12:05.000000000 +0100
@@ -115,6 +115,8 @@
# the path without wiping out everything
kw = extend_env(conn, kw)

+ command = conn.cmd(command)
+
timeout = timeout or conn.global_timeout
conn.logger.info('Running command: %s' % ' '.join(admin_command(conn.sudo,
command)))
result = conn.execute(_remote_run, cmd=command, **kw)
@@ -138,12 +140,27 @@

def _remote_check(channel, cmd, **kw):
import subprocess
-
+ stdin = kw.pop('stdin', None)
process = subprocess.Popen(
- cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kw
+ cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
stdin=subprocess.PIPE, **kw
)
- stdout = process.stdout.read().splitlines()
- stderr = process.stderr.read().splitlines()
+
+ if stdin:
+ if not isinstance(stdin, bytes):
+ stdin.encode('utf-8', errors='ignore')
+ stdout_stream, stderr_stream = process.communicate(stdin)
+ else:
+ stdout_stream = process.stdout.read()
+ stderr_stream = process.stderr.read()
+
+ try:
+ stdout_stream = stdout_stream.decode('utf-8')
+ stderr_stream = stderr_stream.decode('utf-8')
+ except AttributeError:
+ pass
+
+ stdout = stdout_stream.splitlines()
+ stderr = stderr_stream.splitlines()
channel.send((stdout, stderr, process.wait()))


@@ -155,6 +172,8 @@
This helper function *does not* provide any logging as it is the caller's
responsibility to do so.
"""
+ command = conn.cmd(command)
+
stop_on_error = kw.pop('stop_on_error', True)
timeout = timeout or conn.global_timeout
if not kw.get('env'):
@@ -192,4 +211,3 @@
if exit:
conn.exit()
return response
-
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/tests/conftest.py
new/remoto-1.1.2/remoto/tests/conftest.py
--- old/remoto-0.0.35/remoto/tests/conftest.py 1970-01-01 01:00:00.000000000
+0100
+++ new/remoto-1.1.2/remoto/tests/conftest.py 2019-03-01 16:16:01.000000000
+0100
@@ -0,0 +1,25 @@
+import pytest
+
+
+class Capture(object):
+
+ def __init__(self, *a, **kw):
+ self.a = a
+ self.kw = kw
+ self.calls = []
+ self.return_values = kw.get('return_values', False)
+ self.always_returns = kw.get('always_returns', False)
+
+ def __call__(self, *a, **kw):
+ self.calls.append({'args': a, 'kwargs': kw})
+ if self.always_returns:
+ return self.always_returns
+ if self.return_values:
+ return self.return_values.pop()
+
+
+class Factory(object):
+
+ def __init__(self, **kw):
+ for k, v in kw.items():
+ setattr(self, k, v)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/tests/fake_module.py
new/remoto-1.1.2/remoto/tests/fake_module.py
--- old/remoto-0.0.35/remoto/tests/fake_module.py 2015-08-07
14:50:36.000000000 +0200
+++ new/remoto-1.1.2/remoto/tests/fake_module.py 2019-03-01
16:16:01.000000000 +0100
@@ -2,6 +2,24 @@
this is just a stub module to use to test the `import_module` functionality in
remoto
"""
+import sys
+

def function(conn):
return True
+
+
+def fails():
+ raise Exception('failure from fails() function')
+
+
+def unexpected_fail():
+ sys.exit(1)
+
+
+def noop():
+ sys.exit(0)
+
+
+def passes():
+ pass
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto/tests/test_connection.py
new/remoto-1.1.2/remoto/tests/test_connection.py
--- old/remoto-0.0.35/remoto/tests/test_connection.py 2018-12-21
18:38:02.000000000 +0100
+++ new/remoto-1.1.2/remoto/tests/test_connection.py 2019-03-01
16:16:01.000000000 +0100
@@ -1,148 +1,29 @@
-import sys
-from mock import Mock, patch
-from py.test import raises
-from remoto import connection
-from . import fake_module
+import pytest
+from remoto.connection import get


-class FakeSocket(object):
+base_names = [
+ 'ssh', 'oc', 'openshift', 'kubernetes', 'k8s', 'local', 'popen',
'localhost', 'docker', 'podman',
+]

- def __init__(self, gethostname, getfqdn=None):
- self.gethostname = lambda: gethostname
- self.getfqdn = lambda: getfqdn or gethostname
+capitalized_names = [n.capitalize() for n in base_names]

+spaced_names = [" %s " % n for n in base_names]

-class TestNeedsSsh(object):
+valid_names = base_names + capitalized_names + spaced_names

- def test_short_hostname_matches(self):
- socket = FakeSocket('foo.example.org')
- assert connection.needs_ssh('foo', socket) is False

- def test_long_hostname_matches(self):
- socket = FakeSocket('foo.example.org')
- assert connection.needs_ssh('foo.example.org', socket) is False
+class TestGet(object):

- def test_hostname_does_not_match(self):
- socket = FakeSocket('foo')
- assert connection.needs_ssh('meh', socket) is True
+ @pytest.mark.parametrize('name', valid_names)
+ def test_valid_names(self, name):
+ conn_class = get(name)
+ assert conn_class.__name__.endswith('Connection')

- def test_fqdn_hostname_matches_short_hostname(self):
- socket = FakeSocket('foo', getfqdn='foo.example.org')
- assert connection.needs_ssh('foo.example.org', socket) is False
+ def test_fallback(self):
+ conn_class = get('non-existent')
+ assert conn_class.__name__ == 'BaseConnection'

-
-class FakeGateway(object):
-
- def remote_exec(self, module):
- pass
-
-
-class TestRemoteModule(object):
-
- def setup(self):
- self.conn = connection.Connection('localhost', sudo=True, eager=False)
- self.conn.gateway = FakeGateway()
-
- def test_importing_it_sets_it_as_remote_module(self):
- self.conn.import_module(fake_module)
- assert fake_module == self.conn.remote_module.module
-
- def test_importing_it_returns_the_module_too(self):
- remote_foo = self.conn.import_module(fake_module)
- assert remote_foo.module == fake_module
-
-
-class TestModuleExecuteArgs(object):
-
- def setup(self):
- self.remote_module = connection.ModuleExecute(FakeGateway(), None)
-
- def test_single_argument(self):
- assert self.remote_module._convert_args(('foo',)) == "'foo'"
-
- def test_more_than_one_argument(self):
- args = ('foo', 'bar', 1)
- assert self.remote_module._convert_args(args) == "'foo', 'bar', 1"
-
- def test_dictionary_as_argument(self):
- args = ({'some key': 1},)
- assert self.remote_module._convert_args(args) == "{'some key': 1}"
-
-
-class TestModuleExecuteGetAttr(object):
-
- def setup(self):
- self.remote_module = connection.ModuleExecute(FakeGateway(), None)
-
- def test_raise_attribute_error(self):
- with raises(AttributeError) as err:
- self.remote_module.foo()
- assert err.value.args[0] == 'module None does not have attribute foo'
-
-
-class TestMakeConnectionString(object):
-
- def test_makes_sudo_python_no_ssh(self):
- conn = connection.Connection('localhost', sudo=True, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: False)
- assert conn_string == 'popen//python=sudo python'
-
- def test_makes_sudo_python_with_ssh(self):
- conn = connection.Connection('localhost', sudo=True, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: True)
- assert conn_string == 'ssh=localhost//python=sudo python'
-
- def test_makes_sudo_python_with_ssh_options(self):
- conn = connection.Connection('localhost', sudo=True, eager=False,
interpreter='python', ssh_options='-F vagrant_ssh_config')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: True)
- assert conn_string == 'ssh=-F vagrant_ssh_config
localhost//python=sudo python'
-
- def test_makes_python_no_ssh(self):
- conn = connection.Connection('localhost', sudo=False, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: False)
- assert conn_string == 'popen//python=python'
-
- def test_makes_python_with_ssh(self):
- conn = connection.Connection('localhost', sudo=False, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: True)
- assert conn_string == 'ssh=localhost//python=python'
-
- def test_makes_sudo_python_with_forced_sudo(self):
- conn = connection.Connection('localhost', sudo=True, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: False, use_sudo=True)
- assert conn_string == 'popen//python=sudo python'
-
- def test_does_not_make_sudo_python_with_forced_sudo(self):
- conn = connection.Connection('localhost', sudo=True, eager=False,
interpreter='python')
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: False, use_sudo=False)
- assert conn_string == 'popen//python=python'
-
- def test_detects_python3(self):
- with patch.object(sys, 'version_info', (3, 5, 1)):
- conn = connection.Connection('localhost', sudo=True, eager=False)
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: False)
- assert conn_string == 'popen//python=sudo python3'
-
- def test_detects_python2(self):
- with patch.object(sys, 'version_info', (2, 7, 11)):
- conn = connection.Connection('localhost', sudo=False, eager=False)
- conn_string = conn._make_connection_string('localhost',
_needs_ssh=lambda x: True)
- assert conn_string == 'ssh=localhost//python=python2'
-
-class TestDetectSudo(object):
-
- def setup(self):
- self.execnet = Mock()
- self.execnet.return_value = self.execnet
- self.execnet.makegateway.return_value = self.execnet
- self.execnet.remote_exec.return_value = self.execnet
-
- def test_does_not_need_sudo(self):
- self.execnet.receive.return_value = 'root'
- conn = connection.Connection('localhost', sudo=True, eager=False)
- assert conn._detect_sudo(_execnet=self.execnet) is False
-
- def test_does_need_sudo(self):
- self.execnet.receive.return_value = 'alfredo'
- conn = connection.Connection('localhost', sudo=True, eager=False)
- assert conn._detect_sudo(_execnet=self.execnet) is True
+ def test_custom_fallback(self):
+ conn_class = get('non-existent', 'openshift')
+ assert conn_class.__name__ == 'OpenshiftConnection'
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto.egg-info/PKG-INFO
new/remoto-1.1.2/remoto.egg-info/PKG-INFO
--- old/remoto-0.0.35/remoto.egg-info/PKG-INFO 2019-01-08 17:24:34.000000000
+0100
+++ new/remoto-1.1.2/remoto.egg-info/PKG-INFO 2019-03-13 16:27:34.000000000
+0100
@@ -1,6 +1,6 @@
Metadata-Version: 1.1
Name: remoto
-Version: 0.0.35
+Version: 1.1.2
Summary: Execute remote commands or processes.
Home-page: http://github.com/alfredodeza/remoto
Author: Alfredo Deza
@@ -8,14 +8,15 @@
License: MIT
Description: remoto
======
- A very simplistic remote-command-executor using ``ssh`` and Python in
the
- remote end.
+ A very simplistic remote-command-executor using connections to hosts
(``ssh``,
+ local, containers, and several others are supported) and Python in the
remote
+ end.

All the heavy lifting is done by execnet, while this minimal API
provides the
bare minimum to handle easy logging and connections from the remote
end.

``remoto`` is a bit opinionated as it was conceived to replace helpers
and
- remote utilities for ``ceph-deploy`` a tool to run remote commands to
configure
+ remote utilities for ``ceph-deploy``, a tool to run remote commands to
configure
and setup the distributed file system Ceph.


@@ -31,10 +32,7 @@

This is how it would look with a basic logger passed in::

- >>> import logging
- >>> logging.basicConfig(level=logging.DEBUG)
- >>> logger = logging.getLogger('hostname')
- >>> conn = remoto.Connection('hostname', logger=logger)
+ >>> conn = remoto.Connection('hostname')
>>> run(conn, ['ls', '-a'])
INFO:hostname:Running command: ls -a
DEBUG:hostname:.
@@ -43,10 +41,8 @@
DEBUG:hostname:.bash_logout
DEBUG:hostname:.bash_profile
DEBUG:hostname:.bashrc
- DEBUG:hostname:.gem
DEBUG:hostname:.lesshst
DEBUG:hostname:.pki
- DEBUG:hostname:.puppet
DEBUG:hostname:.ssh
DEBUG:hostname:.vim
DEBUG:hostname:.viminfo
@@ -67,9 +63,9 @@
one is with ``process.run``::

>>> from remoto.process import run
- >>> from remoto import Connection
- >>> logger = logging.getLogger('myhost')
- >>> conn = Connection('myhost', logger=logger)
+ >>> from remoto import connection
+ >>> Connection = connection.get('ssh')
+ >>> conn = Connection('myhost')
>>> run(conn, ['whoami'])
INFO:myhost:Running command: whoami
DEBUG:myhost:root
@@ -97,13 +93,43 @@

Remote Functions
================
-
- To execute remote functions (ideally) you would need to define them in
a module
- and add the following to the end of that module::
-
- if __name__ == '__channelexec__':
- for item in channel:
- channel.send(eval(item))
+ There are two supported ways to execute functions on the remote side.
The
+ library that ``remoto`` uses to connect (``execnet``) only supports a
few
+ backends *natively*, and ``remoto`` has extended this ability for
other backend
+ connections like kubernetes.
+
+ The remote function capabilities are provided by
``LegacyModuleExecute`` and
+ ``JsonModuleExecute``. By default, both ``ssh`` and ``local``
connection will
+ use the legacy execution class, and everything else will use the
``legacy``
+ class. The ``ssh`` and ``local`` connections can still be forced to
use the new
+ module execution by setting::
+
+ conn.remote_import_system = 'json'
+
+
+ ``json``
+ --------
+ The default module for ``docker``, ``kubernetes``, ``podman``, and
+ ``openshift``. It does not require any magic on the module to be
executed,
+ however it is worth noting that the library *will* add the following
bit of
+ magic when sending the module to the remote end for execution::
+
+
+ if __name__ == '__main__':
+ import json, traceback
+ obj = {'return': None, 'exception': None}
+ try:
+ obj['return'] = function_name(*a)
+ except Exception:
+ obj['exception'] = traceback.format_exc()
+ try:
+ print(json.dumps(obj).decode('utf-8'))
+ except AttributeError:
+ print(json.dumps(obj))
+
+ This allows the system to execute ``function_name`` (replaced by the
real
+ function to be executed with its arguments), grab any results,
serialize them
+ with ``json`` and send them back for local processing.


If you had a function in a module named ``foo`` that looks like this::
@@ -135,8 +161,22 @@
dictionaries. Also safe to use are ints and strings.


- Automatic detection for remote connections
- ------------------------------------------
+ ``legacy``
+ ----------
+ When using the ``legacy`` execution model (the default for ``local``
and
+ ``ssh`` connections), modules are required to add the following to the
end of
+ that module::
+
+ if __name__ == '__channelexec__':
+ for item in channel:
+ channel.send(eval(item))
+
+ This piece of code is fully compatible with the ``json`` execution
model, and
+ would not cause conflicts.
+
+
+ Automatic detection for ssh connections
+ ---------------------------------------
There is automatic detection for the need to connect remotely (via
SSH) or not
that it is infered by the hostname of the current host (vs. the host
that is
connecting to).
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/remoto-0.0.35/remoto.egg-info/SOURCES.txt
new/remoto-1.1.2/remoto.egg-info/SOURCES.txt
--- old/remoto-0.0.35/remoto.egg-info/SOURCES.txt 2019-01-08
17:24:34.000000000 +0100
+++ new/remoto-1.1.2/remoto.egg-info/SOURCES.txt 2019-03-13
16:27:34.000000000 +0100
@@ -15,7 +15,15 @@
remoto.egg-info/not-zip-safe
remoto.egg-info/requires.txt
remoto.egg-info/top_level.txt
+remoto/backends/__init__.py
+remoto/backends/docker.py
+remoto/backends/kubernetes.py
+remoto/backends/local.py
+remoto/backends/openshift.py
+remoto/backends/podman.py
+remoto/backends/ssh.py
remoto/tests/__init__.py
+remoto/tests/conftest.py
remoto/tests/fake_module.py
remoto/tests/test_connection.py
remoto/tests/test_log.py


< Previous Next >
This Thread
  • No further messages