On 01/27/2015 11:48 AM, Josef Reidinger wrote:

On Thu, 22 Jan 2015 15:09:01 +0100 Ladislav Slezak wrote:

...

Hi all,

I'd like to discuss one Travis feature here.

Currently all Travis steps described in .travis.yml are executed regardless whether something fails or not.

The problem is that if any step in the middle fails then it's quite hard to find the issue as the logs are usually very long. For example the yast2 log has over 4500 lines [1].

Is possible to have it less verbose? all traversing dirs looks useless for me.

Well, what about just to listing the tests which succeed without any additional data and only including the details for failed tests in the log? Jiri

...

The question is whether we want to switch to the fail fast principle [2], so the Travis build would stop at the first failure and you could easily find it at the end of the log. (Travis does not support failing fast out of box, but there is a workaround for this [3].)

Summary:

Current state (always run all steps)

+ can find multiple issues in a single run

this is quite good with rubocop, as it is good to find problem in style and test together.

...

- too long log for finding the issues

I think in general we should find way how to reduce log length

...

- may report false positives for dependent steps (e.g. if compilation fails then obviously the tests fail too)

not much often in yast I think

...

- longer build times

Travis ones, so not a problem for us :)

...

Fail fast

+ easy to find the failure + failed builds finish faster - you can only see/fix one issue at a time, more iterations for fixing all issues is needed (e.g. you fix a rubocop style issue, but then the testsuite does not pass, after fixing it the install step fails because of a missing change in Makefile... each fix means a new commit and a new build)

Also setting set -x can have some side effects that is not expected. in general we should try to have short log if it succeed and long if failed. maybe it is just small tunning of commands?

...

What do you think about it? Should we change the Travis behavior?

In general I am fine with current behavior, just succeed command should not have so much log lines.

Josef

-- Regards, Jiri Srain Project Manager --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.com Lihovarska 1060/12 tel: +420 284 084 659 190 00 Praha 9 fax: +420 284 084 001 Czech Republic http://www.suse.com -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

On 27.1.2015 13:25, Ladislav Slezak wrote:

Dne 27.1.2015 v 12:31 Jiri Srain napsal(a):

...

On 01/27/2015 11:48 AM, Josef Reidinger wrote: [...]

...

Is possible to have it less verbose? all traversing dirs looks useless for me. The result is really interesting, the log size has been decreased from ~4600 lines [1] to ~2500 [2], which is a huge difference (almost half of the original size!).

That's a very good progress, but still not enough. If it sounds too bad, sorry for that. It's definitely a job well done and we all appreciate it! :)

...

Well, what about just to listing the tests which succeed without any additional data and only including the details for failed tests in the log?

Um, that would be nice, unfortunately it cannot be implemented easily :-(

But still sounds like the best solution.

The problem is that Travis displays STDOUT + STDERR of the executed script, you would need to either change the make/rake behavior (huh...) or add some output filtering (that would be non-trivial and probably very sensitive to output format changes in rake/make so not really recommended...).

Just an idea... don't throw stones at me :) for every single test do: 1. Redirect stderr to stdout 2. Save stdout to a temporary file 3. Check the exit code, if it passed, show: X -> passed 4. If it failed, show all the details and x -> failed 5. Remove the tempfile Would it work? Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org