Mailinglist Archive: opensuse-project (120 mails)

< Previous Next >
[opensuse-project] Re: Fwd: Suse Bug Reporter - GSoC midterm report
  • From: Mihnea Dobrescu-Balaur <mihneadb@xxxxxxxxx>
  • Date: Thu, 14 Jul 2011 16:08:08 +0300
  • Message-id: <CAGmmvXLhtH4Fum=rmCnMRqFNgV94CodFnCg3nWqu4MogU10yKA@mail.gmail.com>
On Thu, Jul 14, 2011 at 4:04 PM, Michal Vyskocil <mvyskocil@xxxxxxx> wrote:
On Thu, Jul 14, 2011 at 02:59:09PM +0300, Mihnea Dobrescu-Balaur wrote:
On Thu, Jul 14, 2011 at 1:19 PM, Michal Vyskocil <mvyskocil@xxxxxxx> wrote:
On Mon, Jul 11, 2011 at 04:41:34PM +0300, Mihnea Dobrescu-Balaur wrote:
Hello Michal,

This is the report I've sent to the mailing list.


Mihnea


---------- Forwarded message ----------
From: Mihnea Dobrescu-Balaur <mihneadb@xxxxxxxxx>
Date: Mon, Jul 11, 2011 at 4:39 PM
Subject: Suse Bug Reporter - GSoC midterm report
To: opensuse-project@xxxxxxxxxxxx


Hello everyone,

The expected/requested functionalities for the project are done. I'm
talking about the CLI version of the tool.
Its main functionalities are splitted in four categories at the moment:
* aid - helps the user to determine which is the „suspected” package
that causes a problem
* gather - scans the system for relevant data to provide in a bug
report or to someone that helps, via IRC let's say
* query - search the Bugzilla instance for bug reports that fit given
keywords and sorts them by relevance
* submit - gets input from the user and submits the bug report to Bugzilla

Underlying nice stuff:
* duplicate search - this feature was very wanted from what I could
tell. I managed to implement it using a sorting by relevance of bug
reports. From what I could see in the tests I've done, it works great.
Also, it supports "excluded words" functionality.
* get assignee + maintainers - I use data from the rpm and then get
the relevant information via OBS, using it's python bindings, not the
shell tool
* check account - verifies that the user has a valid account and if
so, it stores the user's credentials in a hidden file, base64
encrypted and pickled to make them not human readable; if the data is
stored, the user isn't prompted again for username & password when
interacting with bugzilla
* globbing : when searching for a package, you can also do something
like search for ' kde* ' and it will show you what packages match
* searching local rpm database: it also looks in 'provides',  not only in
'name'

There are also some other things that I've implemented by necessity,
but I think the ones listed above are the ones that deserve to be
mentioned in this report.

Some sample usage for submitting is here [0].

You can find the repo here: [1]

There is more to the tool that I can think of right now, so if you got
any questions, fire away!

I intend to do also a GUI version but I don't know what to use. Maybe
if libYUI comes along nicely I could use it. What do you think?

I would postpone the gui version to the next GSoc and focus on the
command line client first :).


Also, I'll research what I can do to have maybe the bug reporter pop
up when something crashes. I'm thinking of listening for specific
signals, like SIGSEGV. Any suggestions are welcome!

That's trivial - read man 5 core - there's a special file called
/proc/sys/kernel/core_pattern, which can be used for defining the
default name of cores. You might add the /cores/core.%e.%p, which will
created cores like /cores/core.mutt.5687. And you might check this dir
using inotify.

An alternative approach is add the '|
/usr/bin/susre-bug-reporter-crash-handler' and then the core will be
sent to the standard input of such program. This has only one caveat,
when your handler crasher, it will trigger and another handler run,
which will very probably crashes as well, so the new handler will be
executed, .... and you will end with the DOS on your system.

Ubuntu's Apport and Fedora's ABRT use this approach, so you might want to
check
https://wiki.ubuntu.com/Apport
http://kerneltrap.org/node/14010
http://justlearningtoblog.wordpress.com/2010/07/23/enable-core-dump-files-in-fedora/

or simply google
http://www.google.com/search?q=apport+core_pattern&ie=UTF-8&oe=UTF-8
http://www.google.com/search?hl=cs&q=abrt+core_patern&aq=f&aqi=&aql=&oq=


BTW: listening of SIGSEGV isn't so trivial, because only parent
processes can do it reliable, so the core_pattern approach is much
simpler. The only one think to do is where to setup - it's not the
problem, after your program is packaged, the init script started on boot
will do it.


I want to implement the crash-watch-feature via core_pattern and
inotify. I guess in the C daemon that I will write (something like
this [0]) I can have a system call to the bug reporter giving it the
path to the core dump file and having that I can detect which package
was it and attach the dump to the bug report. What do you think?


You might consider systemd as very usefull thing for you. Given the fact
systemd will be very probably the default init system in 12.1, you have
to be aware there's an another option. Systemd's path activation. [1] The
approach is really simple, you need only two files

reporterd.service with

ExecStart=/usr/lib/suse-bug-reporter/reporterd

and

reporterd.path with

DirectoryNotEmpty=/crashes/

Once the path unit is enabled, systemd will spawn your daemon defined in
ExecStart line, which might handle the crash file.


Of course, because the user part of systemd is still not finished, this
will need to have two daemons - the one started by systemd and the one
running under user and have some D-BUS communication between them.

[1] http://0pointer.de/public/systemd-man/systemd.path.html


I'll have to read. I don't know much about services and stuff like
that. I'd like to start with the described method first, and then
adapt it to systemd, if you think it's ok.


Speaking of attach - python-bugzilla hasn't got the attach
functionality implemented. I'm not sure I know how to implement it
myself. I'm thinking something like - using the bugzilla XMLRPC and
doing it via http somehow - documentation is here [1]. I don't know if
this would be too 'dirty' and if there is a better way. Please let me
know, I'm new to this kind of programming. :)a

Ther aren't? I thought thery are - as far I can see some methods for the
work with attachments in the git

https://fedorahosted.org/python-bugzilla/browser/bugzilla/base.py#L634

so this is supposed to be working - if not let's try to fix it somehow.


That function does (attachid, mailresults) =
self._attachfile(id,**kwargs), and _attachfile raises
NotImplementedYet :). I checked before telling you.


One more thing - regarding what files / logs to attach when reporting
against certain packages (i.e. Yast log for reporting against yast),
I'm thinking of implementing some sort of application hooks (as
Ubuntu's Apport calls it). That means a file for each app with a
string that represents what to attach. Over this, I'd write a function
that scans the folder for such files and checks if the package name in
the report matches one of the respective files. If so, it attaches the
corresponding log. If you can think of a better infrastructure for
this, please let me know!

As I said you before. Create some gather.d/ directory, where every package
might emit own gather code. Your gathering code will just do
something like (it needs to be very safe and robust code, because your
code should not fail just because someone else post the wrong code). And
yes, your own modules should be installed here as well.

gather_info = dict()

for path in glob.glob("PATH/gather.d/*.py"): do
  try:
      module = __import__(path)
  except ImportError:
      log(error, "Cannot import module ``s''" % (path)

  try:
      gather_info[os.path.basename(path)] = module.gather()
  except Exception, e:
      log(error, "module ``%s'' failed with\n%s" % (module, e))


OK.


Mihnea
--
To unsubscribe, e-mail: opensuse-project+unsubscribe@xxxxxxxxxxxx
For additional commands, e-mail: opensuse-project+help@xxxxxxxxxxxx

< Previous Next >