*** brinzhang has joined #openstack-qa | 00:12 | |
*** brinzhang_ has joined #openstack-qa | 00:18 | |
*** brinzhang has quit IRC | 00:21 | |
*** brinzhang0 has joined #openstack-qa | 00:21 | |
*** brinzhang_ has quit IRC | 00:24 | |
*** eandersson has quit IRC | 00:35 | |
*** eandersson has joined #openstack-qa | 00:36 | |
*** hongbin has quit IRC | 00:47 | |
*** brinzhang0 has quit IRC | 00:48 | |
*** brinzhang0 has joined #openstack-qa | 00:49 | |
*** hongbin has joined #openstack-qa | 00:50 | |
*** Liang__ has joined #openstack-qa | 01:11 | |
*** brinzhang_ has joined #openstack-qa | 01:49 | |
*** brinzhang0 has quit IRC | 01:51 | |
*** rcernin has quit IRC | 02:28 | |
*** Liang__ has quit IRC | 02:46 | |
*** rcernin has joined #openstack-qa | 02:58 | |
*** ricolin has joined #openstack-qa | 04:14 | |
*** slaweq has joined #openstack-qa | 04:15 | |
*** slaweq has quit IRC | 04:20 | |
*** vishalmanchanda has joined #openstack-qa | 04:33 | |
*** redrobot has quit IRC | 04:36 | |
*** abdysn has joined #openstack-qa | 04:57 | |
*** Liang__ has joined #openstack-qa | 04:59 | |
*** gcheresh has joined #openstack-qa | 05:05 | |
*** hongbin has quit IRC | 05:12 | |
*** Liang__ is now known as LiangFang | 05:15 | |
*** redrobot has joined #openstack-qa | 05:20 | |
*** udesale has joined #openstack-qa | 06:19 | |
*** ccamposr has joined #openstack-qa | 06:21 | |
*** ccamposr__ has quit IRC | 06:24 | |
*** vishakha has joined #openstack-qa | 06:29 | |
*** ccamposr__ has joined #openstack-qa | 06:44 | |
*** ccamposr has quit IRC | 06:46 | |
*** gcheresh has quit IRC | 06:57 | |
*** LiangFang has quit IRC | 07:02 | |
*** Liang__ has joined #openstack-qa | 07:03 | |
*** Liang__ is now known as LiangFang | 07:07 | |
*** iurygregory has joined #openstack-qa | 07:10 | |
openstackgerrit | Lukas Piwowarski proposed openstack/tempest master: Fix the argument check for account-generator https://review.opendev.org/733127 | 07:12 |
---|---|---|
*** rpittau|afk is now known as rpittau | 07:14 | |
*** ralonsoh has joined #openstack-qa | 07:15 | |
*** slaweq has joined #openstack-qa | 07:17 | |
*** rcernin has quit IRC | 07:21 | |
openstackgerrit | Lukas Piwowarski proposed openstack/tempest master: Fix the argument check for account-generator https://review.opendev.org/733127 | 07:34 |
*** tosky has joined #openstack-qa | 07:39 | |
*** gcheresh has joined #openstack-qa | 07:46 | |
*** jpena|off is now known as jpena | 07:56 | |
*** lucasagomes has joined #openstack-qa | 07:59 | |
openstackgerrit | Jens Harbott (frickler) proposed openstack/devstack master: DNM: Drop tempest blacklist for focal https://review.opendev.org/734029 | 08:01 |
*** rishabhhpe has joined #openstack-qa | 08:04 | |
*** tkajinam has quit IRC | 08:07 | |
*** nightmare_unreal has joined #openstack-qa | 08:08 | |
*** ttx has quit IRC | 08:19 | |
*** ttx has joined #openstack-qa | 08:19 | |
AJaeger | QA cores, please review these updates for the docstheme: coverage2sql https://review.opendev.org/729135; devstack-plugin-nfs https://review.opendev.org/732562 ; os-testr https://review.opendev.org/732558 ; | 08:24 |
*** gcheresh has quit IRC | 08:27 | |
abdysn | Hi all, where can i open bugs for devstack? thanks | 08:31 |
openstackgerrit | Merged openstack/devstack-plugin-open-cas master: Correctly set nova-cpu.conf and tempest flag https://review.opendev.org/731973 | 08:48 |
tosky | abdysn: on launchpad: https://docs.openstack.org/devstack/latest/hacking.html#general | 08:52 |
abdysn | tosky, thanks | 08:52 |
*** gfidente has joined #openstack-qa | 08:59 | |
abdysn | https://bugs.launchpad.net/devstack/+bug/1882504, hi all, could take a look at this? | 09:01 |
openstack | Launchpad bug 1882504 in devstack "fedora30 worldmap.py fails" [Undecided,New] | 09:01 |
tosky | adriant: fedora 30 is out of support | 09:11 |
tosky | adriant: ups, sorry | 09:11 |
tosky | abdysn: as I was saying: fedora 30 is out of support | 09:11 |
tosky | abdysn: also, that's just the last part of the logs, and they don't show the real issue, which happens much earlier | 09:12 |
abdysn | tosky, oh, isee | 09:12 |
abdysn | tosky, since it is out of support, i guess no need to take alook | 09:15 |
tosky | but please check the real error, it's a good exercise | 09:16 |
abdysn | tosky, though on the error issue, i debugged it a bit, pgrep seams able to grep the process but not killall | 09:16 |
tosky | I added a comment, but basically when worlddump.py fails, the real failure usually happens before that | 09:16 |
abdysn | tosky, tried to switch to pkill on frickler suggestion, this point passed, but another error started to pop up when some service tried to start | 09:17 |
*** ricolin has quit IRC | 09:18 | |
abdysn | on a side note, i tried to use fedora31, but the mariadb service would fail to start | 09:19 |
abdysn | with no apparent error at that, it would say that terminated by signal and just fails to start | 09:20 |
tosky | I probably miss the issue which lead to the usage of pkill; the point anyway is that a bug report like that doesn't make it easy to understand what happened | 09:22 |
abdysn | tosky, ok, will try to attach the full log | 09:23 |
tosky | and fedora 32? I guess the system you use generally works fine when you do other operations? Do you use VMs? Are those big enough? | 09:23 |
abdysn | tosky, did not try fedora 32, since our pxe does not have it yet. our ci used to use fedora 29 to be honest, and it was working until the end of support. | 09:25 |
frickler | yoctozepto: I'm doing a bit of bug cleanup, do you still want to proceed with https://bugs.launchpad.net/devstack/+bug/1860287 ? | 09:39 |
openstack | Launchpad bug 1860287 in devstack "post-config set after WSGI services are already started" [Undecided,New] | 09:39 |
yoctozepto | frickler: yeah, but it got very low prio (priorities shift) | 09:40 |
*** rishabhhpe has quit IRC | 09:43 | |
*** rishabhhpe has joined #openstack-qa | 09:44 | |
*** LiangFang has quit IRC | 09:49 | |
*** iokiwi3 has quit IRC | 09:56 | |
*** adriant has quit IRC | 09:56 | |
*** adriant has joined #openstack-qa | 09:57 | |
*** slaweq has quit IRC | 09:57 | |
*** slaweq has joined #openstack-qa | 09:58 | |
openstackgerrit | Jens Harbott (frickler) proposed openstack/devstack master: Fix typos in networking document https://review.opendev.org/734053 | 10:00 |
abdysn | tosky, added the full log, if you are kind to take a look, could not find any other error other than this http://paste.openstack.org/show/794455/. which by the way is the error i saw when i used pkill, and which i see when i use fedora31 | 10:09 |
abdysn | but saw it for mariadb in case of fedora31 | 10:10 |
tosky | abdysn: so nothing in the system logs? Maybe too much memory used? How big are those systems? | 10:14 |
abdysn | tosky, i started from a fresh setup | 10:15 |
*** rpittau is now known as rpittau|bbl | 10:15 | |
abdysn | when you say how big you mean disk space? | 10:15 |
abdysn | there is 34G available | 10:17 |
tosky | [Mon Jun 8 11:17:11 2020] Out of memory: Kill process 103139 ((cheduler)) score 618 or sacrifice child | 10:19 |
tosky | memory | 10:19 |
tosky | RAM | 10:19 |
abdysn | oh | 10:19 |
tosky | see http://13.74.249.42/186/SPDK-Daily/logs/dmesg.log.gz | 10:19 |
abdysn | let me check | 10:19 |
abdysn | it should have more than enough | 10:19 |
tosky | even earlier: [Mon Jun 8 11:17:11 2020] mysqld invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0 | 10:19 |
abdysn | cat /proc/meminfo | grep -i free | 10:20 |
abdysn | MemFree: 47500572 kB | 10:20 |
abdysn | SwapFree: 530336 kB | 10:20 |
abdysn | CmaFree: 0 kB | 10:20 |
abdysn | HugePages_Free: 10069 | 10:20 |
tosky | uhm | 10:21 |
*** yamamoto has quit IRC | 10:22 | |
tosky | need to go away, but I anyway: the oomkiller is invoked despite having so much free memory, so something strange is going on | 10:23 |
abdysn | would spdk affect it in anyway | 10:23 |
abdysn | ? | 10:23 |
abdysn | tosky, anyway thanks, will try other things on the setup | 10:24 |
abdysn | to make sure it is an issue with the setup or the devstack. | 10:24 |
tosky | I guess you can monitor the status of the processes during the execution and see which ones are taking so much memory | 10:25 |
abdysn | tosky, ok, will try that, if find anything will update you. thanks | 10:27 |
*** gcheresh has joined #openstack-qa | 10:30 | |
*** nightmare_unreal has quit IRC | 10:33 | |
*** gfidente has quit IRC | 10:37 | |
*** rishabhhpe has quit IRC | 10:41 | |
*** rishabhhpe has joined #openstack-qa | 10:42 | |
*** yamamoto has joined #openstack-qa | 10:57 | |
*** yamamoto has quit IRC | 10:59 | |
*** yamamoto has joined #openstack-qa | 10:59 | |
*** gfidente has joined #openstack-qa | 11:21 | |
abdysn | tosky, so, i did a reboot for the machine, then tried to start a job that is passing on another machine on this machine, but there a process called ksocket that keeps eating the cpu and ram | 11:30 |
abdysn | tosky, so it seams the setup is broken, for another reason, not sure though | 11:30 |
abdysn | the process keeps poping up even if i am running the stack | 11:31 |
abdysn | so maybe it is not related to devstack | 11:31 |
abdysn | i am not running* | 11:32 |
*** nightmare_unreal has joined #openstack-qa | 11:32 | |
*** jpena is now known as jpena|lunch | 11:32 | |
*** rfolco has joined #openstack-qa | 11:37 | |
*** Luzi has joined #openstack-qa | 11:53 | |
*** rfolco is now known as rfolco|rover | 11:54 | |
*** raildo has joined #openstack-qa | 11:57 | |
tosky | abdysn: I agree with you that it's most likely not a devstack issue | 12:06 |
*** rpittau|bbl is now known as rpittau | 12:13 | |
*** yamamoto has quit IRC | 12:18 | |
*** rh-jelabarre has joined #openstack-qa | 12:18 | |
*** afazekas has quit IRC | 12:40 | |
*** udesale_ has joined #openstack-qa | 12:49 | |
*** yamamoto has joined #openstack-qa | 12:52 | |
*** udesale has quit IRC | 12:52 | |
*** paras333 has joined #openstack-qa | 12:52 | |
*** brinzhang0 has joined #openstack-qa | 12:54 | |
*** brinzhang_ has quit IRC | 12:57 | |
*** yamamoto has quit IRC | 12:57 | |
*** artom has joined #openstack-qa | 12:59 | |
*** jpena|lunch is now known as jpena | 13:03 | |
*** nweinber has joined #openstack-qa | 13:09 | |
*** rishabhhpe has quit IRC | 13:15 | |
*** brinzhang_ has joined #openstack-qa | 13:16 | |
*** brinzhang has joined #openstack-qa | 13:18 | |
*** brinzhang0 has quit IRC | 13:19 | |
*** iurygregory is now known as iurygregory|afk | 13:20 | |
*** brinzhang_ has quit IRC | 13:20 | |
*** brinzhang_ has joined #openstack-qa | 13:21 | |
*** brinzhang has quit IRC | 13:24 | |
*** jparker has joined #openstack-qa | 13:37 | |
*** Luzi has quit IRC | 13:41 | |
*** yamamoto has joined #openstack-qa | 13:44 | |
*** brinzhang0 has joined #openstack-qa | 13:47 | |
*** yamamoto has quit IRC | 13:50 | |
*** brinzhang_ has quit IRC | 13:50 | |
*** gcheresh has quit IRC | 13:51 | |
*** gcheresh has joined #openstack-qa | 14:01 | |
*** amodi has joined #openstack-qa | 14:05 | |
*** gcheresh has quit IRC | 14:12 | |
*** yamamoto has joined #openstack-qa | 14:24 | |
*** yamamoto has quit IRC | 14:25 | |
*** yamamoto has joined #openstack-qa | 14:25 | |
*** Liang__ has joined #openstack-qa | 14:26 | |
*** iurygregory|afk is now known as iurygregory | 14:36 | |
openstackgerrit | Merged openstack/devstack master: Fix typos in networking document https://review.opendev.org/734053 | 14:38 |
*** abdysn has quit IRC | 14:43 | |
*** gcheresh has joined #openstack-qa | 15:01 | |
openstackgerrit | Lukas Piwowarski proposed openstack/tempest master: WIP: Create tests for check_uuid https://review.opendev.org/734120 | 15:15 |
*** vishakha has quit IRC | 15:16 | |
*** larainema has joined #openstack-qa | 15:22 | |
*** iurygregory has quit IRC | 15:29 | |
*** iurygregory has joined #openstack-qa | 15:29 | |
*** brinzhang0 has quit IRC | 15:33 | |
*** gcheresh has quit IRC | 15:38 | |
*** Liang__ has quit IRC | 15:38 | |
*** lucasagomes has quit IRC | 16:01 | |
*** rpittau is now known as rpittau|afk | 16:08 | |
*** udesale_ has quit IRC | 16:08 | |
*** dtantsur is now known as dtantsur|afk | 16:10 | |
*** gcheresh has joined #openstack-qa | 16:13 | |
*** gcheresh has quit IRC | 16:20 | |
*** nightmare_unreal has quit IRC | 16:43 | |
*** ccamposr has joined #openstack-qa | 16:52 | |
*** ccamposr__ has quit IRC | 16:54 | |
*** jpena is now known as jpena|off | 17:20 | |
*** vishalmanchanda has quit IRC | 17:20 | |
*** artom has quit IRC | 17:26 | |
*** artom has joined #openstack-qa | 17:45 | |
*** artom has quit IRC | 17:54 | |
*** artom has joined #openstack-qa | 18:08 | |
*** ralonsoh has quit IRC | 18:15 | |
*** dustinc has joined #openstack-qa | 18:16 | |
*** gfidente is now known as gfidente|afk | 18:32 | |
AJaeger | QA cores, please review these updates for the docstheme: coverage2sql https://review.opendev.org/729135; devstack-plugin-nfs https://review.opendev.org/732562 ; os-testr https://review.opendev.org/732558 ; | 19:58 |
AJaeger | and also these reviews: https://review.opendev.org/725536 (tempest); https://review.opendev.org/726577 (grenade) | 19:58 |
*** nweinber has quit IRC | 20:02 | |
*** gcheresh has joined #openstack-qa | 20:13 | |
openstackgerrit | Hervé Beraud proposed openstack/bashate master: drop mock from lower-constraints and requirements https://review.opendev.org/734220 | 20:22 |
*** gcheresh has quit IRC | 20:39 | |
*** gcheresh has joined #openstack-qa | 20:39 | |
*** gcheresh has quit IRC | 21:17 | |
*** raildo has quit IRC | 21:18 | |
*** slaweq has quit IRC | 21:45 | |
*** slaweq has joined #openstack-qa | 21:47 | |
*** paras333 has quit IRC | 21:56 | |
*** rcernin has joined #openstack-qa | 22:17 | |
*** tkajinam has joined #openstack-qa | 22:49 | |
*** paras333 has joined #openstack-qa | 23:07 | |
*** dmellado_ has joined #openstack-qa | 23:10 | |
*** dmellado has quit IRC | 23:11 | |
openstackgerrit | Hervé Beraud proposed openstack/hacking master: drop mock from lower-constraints and requirements https://review.opendev.org/734346 | 23:11 |
*** slaweq has quit IRC | 23:11 | |
*** dmellado_ is now known as dmellado | 23:11 | |
*** tosky has quit IRC | 23:13 | |
*** slaweq has joined #openstack-qa | 23:17 | |
openstackgerrit | Merged openstack/coverage2sql master: Switch to newer openstackdocstheme and reno versions https://review.opendev.org/729135 | 23:19 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!