Hi, I wrote some new source services so I could automate producing spec files for a multibuild of emacs 28+ using different compiler versions for the native-compilation feature, since only one version of libgccjit is allowed to be installed at a time. The source services are sed,pattern-service, and run-services. They are available on github under the MIT license: https://github.com/owinebar/obs-service-sed https://github.com/owinebar/obs-service-pattern-service https://github.com/owinebar/obs-service-run-services
They each have the possibility of being used to consume unlimited resources, so I have included system options to restrict the resources consumed using ulimit and (for the latter two) restricting the services that may be generated/invoked and (for pattern-service) output files that may not be generated, e.g. _services, so that recursive invocation is prevented. There is a detailed example in obs-service-pattern-service showing how the 3 can be used together to produce the emacs multibuild specs and file. I wrote a sed script that takes a sed script augmented by the project conf keywords Substitute:, Substitute=: (equivalent to substitute with the name ending in =), and Ignore to transform spec files. I still need to write malicious test cases to verify the restrictions actually work, e.g. a sed script that accumulates an infinite string in the hold space or just loops forever. I'd like these installed on the build server so a webhook trigger can be used to generate the multibuild automatically. Obviously these could be done manually in a shell script, but trying to limit the malicious possibilities of running arbitrary shell scripts is not something I have time for. sed is an inherently restricted language (when run with --sandbox) which is much more amenable to being limited.
Thanks, Lynn
On Dienstag, 31. Januar 2023, 06:35:57 CET Lynn Winebarger wrote:
Hi, I wrote some new source services so I could automate producing spec files for a multibuild of emacs 28+ using different compiler versions for the native-compilation feature, since only one version of libgccjit is allowed to be installed at a time. The source services are sed,pattern-service, and run-services. They are available on github under the MIT license: https://github.com/owinebar/obs-service-sed https://github.com/owinebar/obs-service-pattern-service https://github.com/owinebar/obs-service-run-services
They each have the possibility of being used to consume unlimited resources, so I have included system options to restrict the resources consumed using ulimit and (for the latter two) restricting the services that may be generated/invoked and (for pattern-service) output files that may not be generated, e.g. _services, so that recursive invocation is prevented. There is a detailed example in obs-service-pattern-service showing how the 3 can be used together to produce the emacs multibuild specs and file. I wrote a sed script that takes a sed script augmented by the project conf keywords Substitute:, Substitute=: (equivalent to substitute with the name ending in =), and Ignore to transform spec files. I still need to write malicious test cases to verify the restrictions actually work, e.g. a sed script that accumulates an infinite string in the hold space or just loops forever. I'd like these installed on the build server so a webhook trigger can be used to generate the multibuild automatically.
A prerequirement of that would be that a security review of these service has been done.
The policy here is that input (sources) should not allow to run random executions (eg. not call a Makefile from the soures) to keep the service safe on a workstation.
Obviously these could be done manually in a shell script, but trying to limit the malicious possibilities of running arbitrary shell scripts is not something I have time for. sed is an inherently restricted language (when run with --sandbox) which is much more amenable to being limited.
esp. this part sounds a bit dangerous to me.
However, if you think it is safe, please request a security review via bugzilla for them. Please assign it to the security-team@suse.de
thanks adrian
On Tue, Jan 31, 2023 at 2:17 AM Adrian Schröter adrian@suse.de wrote:
A prerequirement of that would be that a security review of these service has been done.
The policy here is that input (sources) should not allow to run random executions (eg. not call a Makefile from the soures) to keep the service safe on a workstation.
I understand.
Obviously these could be done manually in a shell script, but trying to limit the malicious possibilities of running arbitrary shell scripts is not something I have time for. sed is an inherently restricted language (when run with --sandbox) which is much more amenable to being limited.
esp. this part sounds a bit dangerous to me.
I must have expressed myself poorly. As an example, the gcc packages have a "pre_checkin.sh" script the maintainers run before "osc commit" that uses sed on two template spec file to generate a number of spec files for a multibuild (or multiple projects in older versions).
With some effort, that script could be replaced by a fixed number of pattern-service and sed source services. In place of iteration, pattern-service executes over a table of fixed size. In place of conditionals, each distinct case would require a different <service> element in _services for a pattern-service that handles the table of settings that would have activated that conditional, or otherwise specify the variation explicitly in the table.
For the emacs example, I have two variants of the sed script template, one for gcc and one for clang. I was able to handle the associated condition by including the template file name as one of the parameters in the table driving the template expansion. But another approach would have been to have one pattern-service for the gcc cases and another pattern-service for the clang cases.
It's not trivial - pattern-service/run-services/sed requires explicitly listing out all the cases and variations in logic. As a programmer, it's ugly. But it also allows the system to verify the finiteness of the task.
Even with the sandbox, the sed programming language is still Turing complete, but it only sees the strings of the input files and computes some output based on those. There are configuration options to specify the maximum cpu time and memory allowed to be used by the process. It's easy enough to write a non-terminating sed program by accident, but that's part of what the resource limits are for.
However, if you think it is safe, please request a security review via bugzilla for them. Please assign it to the security-team@suse.de
I still need to write some more tests to make sure the resource limiters actually work in practice, but then will request the security review. I've done what I can, but I would definitely appreciate feedback from expert review.
Thanks, Lynn
buildservice@lists.opensuse.org
-
Adrian Schröter -
Lynn Winebarger