Hey guys, we deloyed an private OBS instance via the official image (uname -r: '4.12.14-lp151.28.7-default') to build our packages, recently, after several reboots of instance and services individually, I found the OBS server is unable to delete any packages now. When I click to delete the package the website would end up 500 error and I can see some error in `src_server.log`. this is the steps to rerpduce: 1. create a package `test_package2` in `Test_Scm` project, and I could see the logs in src_server.log: ``` 2020-03-05 11:28:11: [24352] PUT (172.16.1.81) /source/Test_Scm/test_package2/_meta?user=Admin 2020-03-05 11:29:03: [24404] GET (172.16.1.81) /source/Test_Scm/test_package2? 2020-03-05 11:29:03: [24405] GET (172.16.1.81) /source/Test_Scm/test_package2?expand=1 2020-03-05 11:29:04: [24411] GET (172.16.1.81) /build/Test_Scm/_result?package=test_package2&view=status ``` also there are some internal files created in projects folder: ``` host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # ls -al drwxr-xr-x 2 obsrun obsrun 4096 Mar 5 11:28 . drwxr-xr-x 528 obsrun obsrun 69632 Mar 5 10:17 .. -rw-r--r-- 1 obsrun obsrun 561 Mar 4 07:42 kernel.mrev -rw-r--r-- 1 obsrun obsrun 2276 Mar 5 02:56 kernel.rev -rw-r--r-- 1 obsrun obsrun 108 Mar 4 07:42 kernel.xml -rw-r--r-- 1 obsrun obsrun 337 Mar 5 11:13 lxc.mrev.del -rw-r--r-- 1 obsrun obsrun 823 Mar 5 11:13 lxc.rev.del -rw-r--r-- 1 obsrun obsrun 56 Mar 4 05:51 _project.mrev -rw-r--r-- 1 obsrun obsrun 56 Mar 5 11:28 test_package2.mrev -rw-r--r-- 1 obsrun obsrun 122 Mar 5 11:28 test_package2.xml ``` the content of each file is: ``` host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # cat test_package2.mrev 1||7846b0687fb026c496f38090ee9d3b19||1583407691|Admin|| host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # cat test_package2.xml <package name="test_package2" project="Test_Scm"> <title>test_package2</title> <description></description> </package> ``` 2. then I try to delete the package I can see the internal error on website but log in src_server.log seems normal: ``` 2020-03-05 11:34:16: [25728] DELETE (172.16.1.81) /source/Test_Scm/test_package2?user=Admin&comment file /srv/obs/events/lastevents changed! ``` 3. but when I refresh the package on website, and try to add files in that package the webpage would tell me: ``` Error while creating '_service' file: package 'test_package2' does not exist. ``` and we could find the abnormal log in src_server.log file: ``` 2020-03-05 11:38:09: [27070] PUT (172.16.1.81) /source/Test_Scm/test_package2/_service?user=Admin 172.16.1.81 [27070]: 404 package 'test_package2' does not exist 2020-03-05 11:38:09: [27071] GET (172.16.1.81) /source/Test_Scm/test_package2? 172.16.1.81 [27071]: 404 package 'test_package2' does not exist 2020-03-05 11:38:09: [27072] GET (172.16.1.81) /source/Test_Scm/test_package2? 172.16.1.81 [27072]: 404 package 'test_package2' does not exist ``` and the project file would now be like: ``` host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # ls -al total 116 drwxr-xr-x 2 obsrun obsrun 4096 Mar 5 11:34 . drwxr-xr-x 528 obsrun obsrun 69632 Mar 5 10:17 .. -rw-r--r-- 1 obsrun obsrun 561 Mar 4 07:42 kernel.mrev -rw-r--r-- 1 obsrun obsrun 2276 Mar 5 02:56 kernel.rev -rw-r--r-- 1 obsrun obsrun 108 Mar 4 07:42 kernel.xml -rw-r--r-- 1 obsrun obsrun 337 Mar 5 11:13 lxc.mrev.del -rw-r--r-- 1 obsrun obsrun 823 Mar 5 11:13 lxc.rev.del -rw-r--r-- 1 obsrun obsrun 56 Mar 4 05:51 _project.mrev -rw-r--r-- 1 obsrun obsrun 113 Mar 5 11:34 test_package2.mrev.del -rw-r--r-- 1 obsrun obsrun 57 Mar 5 11:34 test_package2.rev.del host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # cat test_package2.mrev.del 1||7846b0687fb026c496f38090ee9d3b19||1583407691|Admin|| 2||7846b0687fb026c496f38090ee9d3b19||1583408056|Admin|1| host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # cat test_package2.rev.del 1||d41d8cd98f00b204e9800998ecf8427e||1583408056|Admin|1| host-172-16-1-89:/srv/obs/projects/Test_Scm.pkg # ``` does anyone can tell me what happened and what how can I fix this? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org