Monday, 2020-06-08

*** brinzhang has joined #openstack-qa00:12
*** brinzhang_ has joined #openstack-qa00:18
*** brinzhang has quit IRC00:21
*** brinzhang0 has joined #openstack-qa00:21
*** brinzhang_ has quit IRC00:24
*** eandersson has quit IRC00:35
*** eandersson has joined #openstack-qa00:36
*** hongbin has quit IRC00:47
*** brinzhang0 has quit IRC00:48
*** brinzhang0 has joined #openstack-qa00:49
*** hongbin has joined #openstack-qa00:50
*** Liang__ has joined #openstack-qa01:11
*** brinzhang_ has joined #openstack-qa01:49
*** brinzhang0 has quit IRC01:51
*** rcernin has quit IRC02:28
*** Liang__ has quit IRC02:46
*** rcernin has joined #openstack-qa02:58
*** ricolin has joined #openstack-qa04:14
*** slaweq has joined #openstack-qa04:15
*** slaweq has quit IRC04:20
*** vishalmanchanda has joined #openstack-qa04:33
*** redrobot has quit IRC04:36
*** abdysn has joined #openstack-qa04:57
*** Liang__ has joined #openstack-qa04:59
*** gcheresh has joined #openstack-qa05:05
*** hongbin has quit IRC05:12
*** Liang__ is now known as LiangFang05:15
*** redrobot has joined #openstack-qa05:20
*** udesale has joined #openstack-qa06:19
*** ccamposr has joined #openstack-qa06:21
*** ccamposr__ has quit IRC06:24
*** vishakha has joined #openstack-qa06:29
*** ccamposr__ has joined #openstack-qa06:44
*** ccamposr has quit IRC06:46
*** gcheresh has quit IRC06:57
*** LiangFang has quit IRC07:02
*** Liang__ has joined #openstack-qa07:03
*** Liang__ is now known as LiangFang07:07
*** iurygregory has joined #openstack-qa07:10
openstackgerritLukas Piwowarski proposed openstack/tempest master: Fix the argument check for account-generator  https://review.opendev.org/73312707:12
*** rpittau|afk is now known as rpittau07:14
*** ralonsoh has joined #openstack-qa07:15
*** slaweq has joined #openstack-qa07:17
*** rcernin has quit IRC07:21
openstackgerritLukas Piwowarski proposed openstack/tempest master: Fix the argument check for account-generator  https://review.opendev.org/73312707:34
*** tosky has joined #openstack-qa07:39
*** gcheresh has joined #openstack-qa07:46
*** jpena|off is now known as jpena07:56
*** lucasagomes has joined #openstack-qa07:59
openstackgerritJens Harbott (frickler) proposed openstack/devstack master: DNM: Drop tempest blacklist for focal  https://review.opendev.org/73402908:01
*** rishabhhpe has joined #openstack-qa08:04
*** tkajinam has quit IRC08:07
*** nightmare_unreal has joined #openstack-qa08:08
*** ttx has quit IRC08:19
*** ttx has joined #openstack-qa08:19
AJaegerQA cores, please review these updates for the docstheme: coverage2sql https://review.opendev.org/729135; devstack-plugin-nfs https://review.opendev.org/732562 ; os-testr https://review.opendev.org/732558 ;08:24
*** gcheresh has quit IRC08:27
abdysnHi all, where can i open bugs for devstack? thanks08:31
openstackgerritMerged openstack/devstack-plugin-open-cas master: Correctly set nova-cpu.conf and tempest flag  https://review.opendev.org/73197308:48
toskyabdysn: on launchpad: https://docs.openstack.org/devstack/latest/hacking.html#general08:52
abdysntosky, thanks08:52
*** gfidente has joined #openstack-qa08:59
abdysnhttps://bugs.launchpad.net/devstack/+bug/1882504, hi all, could take a look at this?09:01
openstackLaunchpad bug 1882504 in devstack "fedora30 worldmap.py fails" [Undecided,New]09:01
toskyadriant: fedora 30 is out of support09:11
toskyadriant: ups, sorry09:11
toskyabdysn: as I was saying: fedora 30 is out of support09:11
toskyabdysn: also, that's just the last part of the logs, and they don't show the real issue, which happens much earlier09:12
abdysntosky, oh, isee09:12
abdysntosky, since it is out of support, i guess no need to take alook09:15
toskybut please check the real error, it's a good exercise09:16
abdysntosky, though on the error issue, i debugged it a bit, pgrep seams able to grep the process but not killall09:16
toskyI added a comment, but basically when worlddump.py fails, the real failure usually happens before that09:16
abdysntosky, tried to switch to pkill on frickler suggestion, this point passed, but another error started to pop up when some service tried to start09:17
*** ricolin has quit IRC09:18
abdysnon a side note, i tried to use fedora31, but the mariadb service would fail to start09:19
abdysnwith no apparent error at that, it would say that terminated by signal and just fails to start09:20
toskyI probably miss the issue which lead to the usage of pkill; the point anyway is that a bug report like that doesn't make it easy to understand what happened09:22
abdysntosky, ok, will try to attach the full log09:23
toskyand fedora 32? I guess the system you use generally works fine when you do other operations? Do you use VMs? Are those big enough?09:23
abdysntosky, did not try fedora 32, since our pxe does not have it yet. our ci used to use fedora 29 to be honest, and it was working until the end of support.09:25
frickleryoctozepto: I'm doing a bit of bug cleanup, do you still want to proceed with https://bugs.launchpad.net/devstack/+bug/1860287 ?09:39
openstackLaunchpad bug 1860287 in devstack "post-config set after WSGI services are already started" [Undecided,New]09:39
yoctozeptofrickler: yeah, but it got very low prio (priorities shift)09:40
*** rishabhhpe has quit IRC09:43
*** rishabhhpe has joined #openstack-qa09:44
*** LiangFang has quit IRC09:49
*** iokiwi3 has quit IRC09:56
*** adriant has quit IRC09:56
*** adriant has joined #openstack-qa09:57
*** slaweq has quit IRC09:57
*** slaweq has joined #openstack-qa09:58
openstackgerritJens Harbott (frickler) proposed openstack/devstack master: Fix typos in networking document  https://review.opendev.org/73405310:00
abdysntosky, added the full log, if you are kind to take a look, could not find any other error other than this http://paste.openstack.org/show/794455/. which by the way is the error i saw when i used pkill, and which i see when i use fedora3110:09
abdysnbut saw it for mariadb in case of fedora3110:10
toskyabdysn: so nothing in the system logs? Maybe too much memory used? How big are those systems?10:14
abdysntosky, i started from a fresh setup10:15
*** rpittau is now known as rpittau|bbl10:15
abdysnwhen you say how big you mean disk space?10:15
abdysnthere is 34G available10:17
tosky[Mon Jun  8 11:17:11 2020] Out of memory: Kill process 103139 ((cheduler)) score 618 or sacrifice child10:19
toskymemory10:19
toskyRAM10:19
abdysnoh10:19
toskysee http://13.74.249.42/186/SPDK-Daily/logs/dmesg.log.gz10:19
abdysnlet me check10:19
abdysnit should have more than enough10:19
toskyeven earlier: [Mon Jun  8 11:17:11 2020] mysqld invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=010:19
abdysncat /proc/meminfo | grep -i free10:20
abdysnMemFree:        47500572 kB10:20
abdysnSwapFree:         530336 kB10:20
abdysnCmaFree:               0 kB10:20
abdysnHugePages_Free:    1006910:20
toskyuhm10:21
*** yamamoto has quit IRC10:22
toskyneed to go away, but I anyway: the oomkiller is invoked despite having so much free memory, so something strange is going on10:23
abdysnwould spdk affect it in anyway10:23
abdysn?10:23
abdysntosky, anyway thanks, will try other things on the setup10:24
abdysnto make sure it is an issue with the setup or the devstack.10:24
toskyI guess you can monitor the status of the processes during the execution and see which ones are taking so much memory10:25
abdysntosky, ok, will try that, if find anything will update you. thanks10:27
*** gcheresh has joined #openstack-qa10:30
*** nightmare_unreal has quit IRC10:33
*** gfidente has quit IRC10:37
*** rishabhhpe has quit IRC10:41
*** rishabhhpe has joined #openstack-qa10:42
*** yamamoto has joined #openstack-qa10:57
*** yamamoto has quit IRC10:59
*** yamamoto has joined #openstack-qa10:59
*** gfidente has joined #openstack-qa11:21
abdysntosky, so, i did a reboot for the machine, then tried to start a job that is passing on another machine on this machine, but there a process called ksocket that keeps eating the cpu and ram11:30
abdysntosky, so it seams the setup is broken, for another reason, not sure though11:30
abdysnthe process keeps poping up even if i am running the stack11:31
abdysnso maybe it is not related to devstack11:31
abdysni am not running*11:32
*** nightmare_unreal has joined #openstack-qa11:32
*** jpena is now known as jpena|lunch11:32
*** rfolco has joined #openstack-qa11:37
*** Luzi has joined #openstack-qa11:53
*** rfolco is now known as rfolco|rover11:54
*** raildo has joined #openstack-qa11:57
toskyabdysn: I agree with you that it's most likely not a devstack issue12:06
*** rpittau|bbl is now known as rpittau12:13
*** yamamoto has quit IRC12:18
*** rh-jelabarre has joined #openstack-qa12:18
*** afazekas has quit IRC12:40
*** udesale_ has joined #openstack-qa12:49
*** yamamoto has joined #openstack-qa12:52
*** udesale has quit IRC12:52
*** paras333 has joined #openstack-qa12:52
*** brinzhang0 has joined #openstack-qa12:54
*** brinzhang_ has quit IRC12:57
*** yamamoto has quit IRC12:57
*** artom has joined #openstack-qa12:59
*** jpena|lunch is now known as jpena13:03
*** nweinber has joined #openstack-qa13:09
*** rishabhhpe has quit IRC13:15
*** brinzhang_ has joined #openstack-qa13:16
*** brinzhang has joined #openstack-qa13:18
*** brinzhang0 has quit IRC13:19
*** iurygregory is now known as iurygregory|afk13:20
*** brinzhang_ has quit IRC13:20
*** brinzhang_ has joined #openstack-qa13:21
*** brinzhang has quit IRC13:24
*** jparker has joined #openstack-qa13:37
*** Luzi has quit IRC13:41
*** yamamoto has joined #openstack-qa13:44
*** brinzhang0 has joined #openstack-qa13:47
*** yamamoto has quit IRC13:50
*** brinzhang_ has quit IRC13:50
*** gcheresh has quit IRC13:51
*** gcheresh has joined #openstack-qa14:01
*** amodi has joined #openstack-qa14:05
*** gcheresh has quit IRC14:12
*** yamamoto has joined #openstack-qa14:24
*** yamamoto has quit IRC14:25
*** yamamoto has joined #openstack-qa14:25
*** Liang__ has joined #openstack-qa14:26
*** iurygregory|afk is now known as iurygregory14:36
openstackgerritMerged openstack/devstack master: Fix typos in networking document  https://review.opendev.org/73405314:38
*** abdysn has quit IRC14:43
*** gcheresh has joined #openstack-qa15:01
openstackgerritLukas Piwowarski proposed openstack/tempest master: WIP: Create tests for check_uuid  https://review.opendev.org/73412015:15
*** vishakha has quit IRC15:16
*** larainema has joined #openstack-qa15:22
*** iurygregory has quit IRC15:29
*** iurygregory has joined #openstack-qa15:29
*** brinzhang0 has quit IRC15:33
*** gcheresh has quit IRC15:38
*** Liang__ has quit IRC15:38
*** lucasagomes has quit IRC16:01
*** rpittau is now known as rpittau|afk16:08
*** udesale_ has quit IRC16:08
*** dtantsur is now known as dtantsur|afk16:10
*** gcheresh has joined #openstack-qa16:13
*** gcheresh has quit IRC16:20
*** nightmare_unreal has quit IRC16:43
*** ccamposr has joined #openstack-qa16:52
*** ccamposr__ has quit IRC16:54
*** jpena is now known as jpena|off17:20
*** vishalmanchanda has quit IRC17:20
*** artom has quit IRC17:26
*** artom has joined #openstack-qa17:45
*** artom has quit IRC17:54
*** artom has joined #openstack-qa18:08
*** ralonsoh has quit IRC18:15
*** dustinc has joined #openstack-qa18:16
*** gfidente is now known as gfidente|afk18:32
AJaegerQA cores, please review these updates for the docstheme: coverage2sql https://review.opendev.org/729135; devstack-plugin-nfs https://review.opendev.org/732562 ; os-testr https://review.opendev.org/732558 ;19:58
AJaegerand also these reviews: https://review.opendev.org/725536 (tempest); https://review.opendev.org/726577 (grenade)19:58
*** nweinber has quit IRC20:02
*** gcheresh has joined #openstack-qa20:13
openstackgerritHervĂ© Beraud proposed openstack/bashate master: drop mock from lower-constraints and requirements  https://review.opendev.org/73422020:22
*** gcheresh has quit IRC20:39
*** gcheresh has joined #openstack-qa20:39
*** gcheresh has quit IRC21:17
*** raildo has quit IRC21:18
*** slaweq has quit IRC21:45
*** slaweq has joined #openstack-qa21:47
*** paras333 has quit IRC21:56
*** rcernin has joined #openstack-qa22:17
*** tkajinam has joined #openstack-qa22:49
*** paras333 has joined #openstack-qa23:07
*** dmellado_ has joined #openstack-qa23:10
*** dmellado has quit IRC23:11
openstackgerritHervĂ© Beraud proposed openstack/hacking master: drop mock from lower-constraints and requirements  https://review.opendev.org/73434623:11
*** slaweq has quit IRC23:11
*** dmellado_ is now known as dmellado23:11
*** tosky has quit IRC23:13
*** slaweq has joined #openstack-qa23:17
openstackgerritMerged openstack/coverage2sql master: Switch to newer openstackdocstheme and reno versions  https://review.opendev.org/72913523:19

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!