Tuesday, 2017-03-21

*** hongbin has quit IRC00:00
*** aeng has quit IRC00:02
openstackgerritJamie Lennox proposed openstack-infra/zuul feature/zuulv3: Allow loading additional variables file for site config  https://review.openstack.org/44773400:02
clarkbkevinbenton: for 204 it doesn't appear that zuul got the approval message. Zuul should post a comment removing the jenkins +1 when approval is made there00:03
clarkbkevinbenton: and that roughly happens when the other one was approved so I think similar issue00:03
*** dingyichen has joined #openstack-infra00:03
clarkbwas zuul off at that time?00:03
kevinbentonclarkb: it may have been, i think it was last friday00:03
clarkbkevinbenton: in any case I thnk the easy fix is to reapprove them, just add your approval vote to each00:03
*** rook-ugh is now known as rook00:03
kevinbentonclarkb: ack00:03
clarkbrechecking doesn't work because gerrit doesn't emit vote number data on events when that number doesn't change :/00:03
clarkbso jenkins +1 happening again emits a pretty useless event that zuul can't action on (its somethign that newer gerrit should fix for us)00:04
*** ZZelle has quit IRC00:04
kevinbentonclarkb: perfect. that did the trick00:05
kevinbentonclarkb: we have a new gerrit in the cards?00:05
*** ijw has joined #openstack-infra00:05
clarkbkevinbenton: we are slowly working on it and by we I mostly mean zaro00:05
clarkbwhich reminds me thats on my list of things to catch up on and help with00:05
kevinbentonclarkb: how much does gertty beat the crap out of gerrit?00:07
clarkbkevinbenton: I don't think too badly, its uses a single connection mostly, just like your web browser00:07
clarkbthe initial cache update can hit it pretty hard but once thats done it should be pretty cheap00:07
kevinbentonclarkb: i can't help but wonder if me running gertty makes me responsible for 50% of the queries :P00:07
kevinbenton(particularily being subscribed to both nova and neutron)00:07
*** wolverin_ has quit IRC00:09
*** mat128 has joined #openstack-infra00:10
*** mat128 has quit IRC00:11
*** mat128 has joined #openstack-infra00:12
*** mtanino has joined #openstack-infra00:14
*** wolverineav has joined #openstack-infra00:18
*** aeng has joined #openstack-infra00:19
*** tosky has quit IRC00:20
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773900:21
*** baoli has joined #openstack-infra00:22
*** wolverineav has quit IRC00:22
*** sflanigan_afk has joined #openstack-infra00:23
*** masber has joined #openstack-infra00:24
*** baoli has quit IRC00:26
*** armax has quit IRC00:26
*** amotoki has joined #openstack-infra00:26
*** ym200 has quit IRC00:27
*** ym200 has joined #openstack-infra00:28
*** funzo has quit IRC00:30
*** vhosakot_ has joined #openstack-infra00:31
*** xarses_ has joined #openstack-infra00:32
*** aeng has quit IRC00:33
*** vhosakot has quit IRC00:33
*** andreww has joined #openstack-infra00:33
*** xarses_ has quit IRC00:34
*** xarses_ has joined #openstack-infra00:34
*** gcb_ has quit IRC00:35
*** zul has quit IRC00:36
*** andreww has quit IRC00:38
*** wolverineav has joined #openstack-infra00:38
*** xarses_ has quit IRC00:39
*** andreww has joined #openstack-infra00:39
*** mtanino has quit IRC00:39
*** andreww has quit IRC00:39
*** andreww has joined #openstack-infra00:40
*** andreww has quit IRC00:40
*** gyee_ has quit IRC00:40
*** zul has joined #openstack-infra00:41
*** andreww has joined #openstack-infra00:41
*** mat128 has quit IRC00:41
*** gongysh has quit IRC00:42
*** wolverineav has quit IRC00:42
*** neiljerram has quit IRC00:44
fungi#status log LP bugs for monasca migrated to openstack/monasca-api in StoryBoard, defcore to openstack/defcore, refstack to openstack/refstack00:44
openstackstatusfungi: finished logging00:44
*** ym200 has quit IRC00:45
*** aeng has joined #openstack-infra00:45
*** ym200 has joined #openstack-infra00:45
*** ym200 has quit IRC00:46
*** ym200 has joined #openstack-infra00:46
*** ym200 has quit IRC00:48
*** ym200 has joined #openstack-infra00:49
*** adrian_otto has quit IRC00:50
*** sbezverk has joined #openstack-infra00:52
sbezverkHello, I need to disable echo "0" > /proc/sys/net/bridge/bridge-nf-call-iptables on vm where we runb one specific job. It seems I am not allowed to do that. any guidance you can provide to disable it?00:53
*** mat128 has joined #openstack-infra00:54
fungisbezverk: a link to the error in a job log would help. there could be a number of reasons depending on how and when you're trying to call that00:55
sbezverkfungi: sure thing, let me get it..00:56
sbezverkfungi: here you go http://logs.openstack.org/50/441550/94/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/39ea7d3/console.html#_2017-03-21_00_41_26_83845900:56
*** ym200 has quit IRC00:58
*** ym200 has joined #openstack-infra00:59
clarkbsbezverk: you would just remove the line from whatever code runs that? also fwiw sudo echo "0" > /some/file doesn't write the 0 to the file with root perms00:59
fungibut also it says that doesn't exist... are you sure it's valid with centos 7?00:59
fungiclarkb: is right about the redirect with sudo though00:59
sbezverkfungi: it is working perfectly fine on my local machine00:59
fungii usually do something like sudo sh -c 'echo 0 > /proc/sys/...'01:00
fungior you can sudo tee or a bunch of other alternatives01:00
sbezverkfungi: I can try that, thank you!01:00
clarkbsbezverk: when you say you need to disable echo "0" > /foo that means you want to stop executing that code or do you mean that you need to write a 0 there?01:01
sbezverkclarkb: writing 0 to that location will disable iptables filtering linux bridge01:02
clarkbsbezverk: and taht is what you want to do?01:02
clarkbor do you want to stop disabling the setting of that value?01:03
*** mriedem has quit IRC01:03
*** stakeda has joined #openstack-infra01:04
sbezverkclarkb: I want to disable iptables filtering on linux bridges.. on my system echo 0 to blah produces desired effect01:04
clarkbgotcha, just making sure I parsed your earlier question properly01:04
sbezverkclarkb: fungi: still the same :( http://logs.openstack.org/50/441550/96/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/fdd23b2/console.html#_2017-03-21_01_04_38_10639101:05
*** krtaylor has joined #openstack-infra01:06
clarkbsbezverk: ya thats the separate issue of the file not existing on that centos host01:06
sbezverkit seems that folder is not available inside vm01:06
fungisbezverk: is that in a chroot without /proc mounted maybe? or dependent on a particular lkm being modprobed?01:06
clarkbI can confirm its there on a random centos7 host booted in rax01:07
sbezverkfungi: sorry I cannot answer that, it is vm we get to run gate job..01:07
clarkbhttp://logs.openstack.org/50/441550/96/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/fdd23b2/console.html#_2017-03-21_01_04_32_401723 another thing I notice01:08
fungisbezverk: sure, but since its a kolla job i don't rule out the possibility that your jobs might do things in a chroot or container on that vm, so i asked01:08
*** Julien-zte has joined #openstack-infra01:08
* clarkb checks on osic where the job failed01:08
clarkbits there on random centos7 host I grabebd (I copy pasted the filepath from the job even01:10
sbezverkclarkb: I see the same error on all job, the error you mentioned above we saw before, just did not have a chance top fix it01:12
*** mriedem has joined #openstack-infra01:12
*** yamamoto has joined #openstack-infra01:12
sbezverkfungi: at that moment no fancy stuff is done, basic env prepping01:12
clarkbthat function needs indentation...01:13
fungidoes adding an ls of that path in the job work, i wonder01:13
sbezverkfungi: check this oout http://logs.openstack.org/50/441550/97/check/gate-kolla-kubernetes-deploy-centos-binary-2-iscsi-nv/3ee338d/console.html#_2017-03-21_01_12_20_24530601:14
sbezverkfungi: ls shows that bridge and further down is not there01:15
clarkbya I am wondering if you are just missing a required kernel module or similar01:15
clarkband whatever job is running on the hosts I am checking have that loaded01:15
*** jamielennox is now known as jamielennox|away01:16
*** liujiong has joined #openstack-infra01:16
sbezverkclarkb: it is really strange, it allows us to create bridges, which normally would mean that kernel module for bridging is loaded, no?01:17
clarkbbridge                107106  1 br_netfilter is loaded where I checked01:18
clarkbsbezverk: I don't know01:18
clarkbbut its something to check since procfs is populated by the kernel modules01:18
clarkbits possible you need br_netfilter ?01:19
sbezverkclarkb: will try to add it01:19
clarkbso brctl works because bridge is there but br_netfilter isn't or something like that01:19
clarkbsbezverk: I would lsmod and check for those two modules01:19
clarkbI would test unloading them but I am actually currently using them for local VMs01:20
clarkband finally while throwing ideas out there maybe just edit your iptables rules instaed?01:21
openstackgerritBrian Moss proposed openstack-infra/infra-manual master: Adding instructions for rebasing  https://review.openstack.org/44734101:24
*** pleia2_ is now known as pleia201:25
sbezverkclarkb: it looks lile loading br_netfilter did the trick, thank you very much.01:25
*** jamielennox|away is now known as jamielennox01:28
*** Apoorva has quit IRC01:28
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773901:31
*** Sukhdev_ has quit IRC01:31
*** mozart89 has joined #openstack-infra01:31
*** funzo has joined #openstack-infra01:31
*** ym200 has quit IRC01:33
*** ym200 has joined #openstack-infra01:34
*** ym200 has quit IRC01:34
*** ym200 has joined #openstack-infra01:34
*** yanyanhu has joined #openstack-infra01:34
*** dimtruck is now known as zz_dimtruck01:35
*** ym200 has quit IRC01:35
*** kaisers1 has joined #openstack-infra01:35
*** funzo has quit IRC01:35
*** ym200 has joined #openstack-infra01:36
*** rfolco has joined #openstack-infra01:36
*** ym200 has quit IRC01:36
*** kaisers has quit IRC01:37
*** claudiub has quit IRC01:37
*** ym200 has joined #openstack-infra01:38
*** rhallisey has quit IRC01:39
*** ym200 has quit IRC01:40
*** ym200 has joined #openstack-infra01:41
*** ym200 has quit IRC01:42
*** ym200 has joined #openstack-infra01:42
*** ym200 has quit IRC01:42
*** skelso has joined #openstack-infra01:46
*** skelso has quit IRC01:48
*** JDub has joined #openstack-infra01:48
*** gongysh has joined #openstack-infra01:50
*** yanyanhu has quit IRC01:51
*** knangia has quit IRC01:51
*** skelso has joined #openstack-infra01:53
*** JDub has quit IRC01:56
*** yanyanhu has joined #openstack-infra01:57
*** zhurong has joined #openstack-infra01:57
*** pramodrj07 has joined #openstack-infra01:59
*** pramodrj07 has quit IRC02:01
*** MasterOfBugs has quit IRC02:02
*** ijw has quit IRC02:06
*** tphummel has joined #openstack-infra02:08
*** mriedem has quit IRC02:14
*** adrian_otto has joined #openstack-infra02:15
*** gouthamr has joined #openstack-infra02:15
*** EricGonczer_ has quit IRC02:19
*** vhosakot_ has quit IRC02:22
*** vhosakot has joined #openstack-infra02:23
*** vhosakot has quit IRC02:27
*** zz_dimtruck is now known as dimtruck02:32
*** funzo has joined #openstack-infra02:32
*** vhosakot has joined #openstack-infra02:33
*** funzo has quit IRC02:37
*** yamamoto has quit IRC02:41
*** winggundamth has joined #openstack-infra02:42
*** hongbin has joined #openstack-infra02:47
*** yamamoto has joined #openstack-infra02:48
*** yanyanhu has quit IRC02:48
*** _Cyclone_ has quit IRC02:49
*** yanyanhu has joined #openstack-infra02:53
*** masber has quit IRC02:54
*** nicolasbock has quit IRC03:02
*** zhurong has quit IRC03:05
*** adrian_otto has quit IRC03:06
*** rlandy has quit IRC03:06
*** rfolco has quit IRC03:06
*** rfolco has joined #openstack-infra03:13
*** rfolco has quit IRC03:13
*** masber has joined #openstack-infra03:15
dmsimardIf any core happens to be around, we need to start mirroring Ubuntu Cloud Archive Pike repositories: https://review.openstack.org/#/c/447478/03:15
*** gcb has joined #openstack-infra03:15
*** gildub has quit IRC03:15
*** spzala has joined #openstack-infra03:19
*** dimtruck is now known as zz_dimtruck03:21
*** zhurong has joined #openstack-infra03:22
*** armax has joined #openstack-infra03:24
*** skelso has quit IRC03:24
*** skelso has joined #openstack-infra03:25
*** yamamoto has quit IRC03:25
*** dingyichen has quit IRC03:25
openstackgerritManjeet Singh Bhatia proposed openstack-infra/reviewday master: Append a new section to neutron dashboard  https://review.openstack.org/43911403:27
openstackgerritManjeet Singh Bhatia proposed openstack-infra/reviewday master: Append a new section to neutron dashboard  https://review.openstack.org/43911403:29
*** dave-mccowan has quit IRC03:29
*** dingyichen has joined #openstack-infra03:29
*** adrian_otto has joined #openstack-infra03:32
*** funzo has joined #openstack-infra03:33
*** funzo has quit IRC03:37
*** links has joined #openstack-infra03:44
*** yamahata has quit IRC03:47
*** sree has joined #openstack-infra03:50
*** mat128 has quit IRC03:57
*** hongbin has quit IRC04:02
*** stakeda has quit IRC04:03
*** aeng has quit IRC04:04
*** spzala has quit IRC04:05
*** adrian_otto has quit IRC04:06
*** adrian_otto has joined #openstack-infra04:07
*** ihrachys has joined #openstack-infra04:07
*** gildub has joined #openstack-infra04:09
*** yamamoto has joined #openstack-infra04:09
*** skelso has quit IRC04:11
*** jamielennox is now known as jamielennox|away04:12
*** zhurong has quit IRC04:12
*** gildub has quit IRC04:13
*** shu-mutou-AWAY is now known as shu-mutou04:15
*** d0ugal has quit IRC04:16
*** d0ugal has joined #openstack-infra04:17
*** aeng has joined #openstack-infra04:21
*** psachin has joined #openstack-infra04:27
*** gildub has joined #openstack-infra04:29
*** adrian_otto has quit IRC04:29
*** funzo has joined #openstack-infra04:34
*** tonytan4ever has quit IRC04:37
*** aeng has quit IRC04:38
*** funzo has quit IRC04:39
*** ramishra has joined #openstack-infra04:45
*** saibarspeis has joined #openstack-infra04:46
*** spzala has joined #openstack-infra04:47
*** spzala has quit IRC04:51
*** anilvenkata has joined #openstack-infra04:52
*** jaypipes has quit IRC04:57
*** jaypipes has joined #openstack-infra04:57
*** aeng has joined #openstack-infra04:59
*** yamahata has joined #openstack-infra05:00
*** zhurong has joined #openstack-infra05:00
openstackgerritMerged openstack-infra/system-config master: Add support for mirroring the new Pike UCA repos  https://review.openstack.org/44747805:01
*** tphummel has quit IRC05:01
*** gouthamr has quit IRC05:03
*** aeng has quit IRC05:06
*** chuckC_ has joined #openstack-infra05:06
*** arxcruz has joined #openstack-infra05:08
*** spzala has joined #openstack-infra05:11
*** spzala has quit IRC05:15
*** esikache1 has joined #openstack-infra05:16
*** dhajare has joined #openstack-infra05:20
*** aeng has joined #openstack-infra05:24
*** masahito has joined #openstack-infra05:25
*** cshastri has joined #openstack-infra05:25
*** dhajare has quit IRC05:25
*** prateek has joined #openstack-infra05:27
*** aeng has quit IRC05:30
*** ihrachys has quit IRC05:32
*** funzo has joined #openstack-infra05:36
*** nadya has joined #openstack-infra05:36
*** nadya has quit IRC05:39
*** zhurong has quit IRC05:40
*** funzo has quit IRC05:40
*** vikrant has joined #openstack-infra05:41
*** dhajare has joined #openstack-infra05:42
*** masahito has quit IRC05:43
*** andreww has quit IRC05:43
*** rcernin has joined #openstack-infra05:43
*** aeng has joined #openstack-infra05:43
*** Julien-zte has quit IRC05:48
*** markvoelker has quit IRC05:53
*** mozart89|2 has joined #openstack-infra05:53
*** Julien-zte has joined #openstack-infra05:56
*** claudiub has joined #openstack-infra05:57
*** rcernin has quit IRC05:57
*** knangia has joined #openstack-infra06:02
*** Wei_Liu has quit IRC06:07
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Create dir on subnode for copying deploy.env  https://review.openstack.org/44784906:07
*** Wei_Liu has joined #openstack-infra06:07
*** Julien-zte has quit IRC06:09
*** dhajare has quit IRC06:12
*** vsaienko has joined #openstack-infra06:13
*** adriant has quit IRC06:14
*** vsaienko has quit IRC06:18
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Support report by periodic multinode jobs  https://review.openstack.org/44786006:19
*** esikache1 has quit IRC06:23
*** esikachev has joined #openstack-infra06:23
*** dhajare has joined #openstack-infra06:25
openstackgerrittoan.t.pham proposed openstack-infra/jenkins-job-builder master: Add folder create for cloudbee folder jobs  https://review.openstack.org/43879506:26
*** bhavik1 has joined #openstack-infra06:29
*** jbadiapa has joined #openstack-infra06:30
*** kornicameister has joined #openstack-infra06:31
*** jaosorior has joined #openstack-infra06:31
*** yolanda has quit IRC06:32
*** alexchadin has joined #openstack-infra06:32
*** kornicameister has quit IRC06:35
*** Wei_Liu1 has joined #openstack-infra06:35
*** Wei_Liu has quit IRC06:36
*** Wei_Liu1 is now known as Wei_Liu06:36
*** funzo has joined #openstack-infra06:36
*** funzo has quit IRC06:41
*** psachin_ has joined #openstack-infra06:44
*** psachin has quit IRC06:46
*** yolanda has joined #openstack-infra06:46
*** mozart89|2 has quit IRC06:49
*** markvoelker has joined #openstack-infra06:54
*** Julien-zte has joined #openstack-infra06:54
*** Julien-z_ has joined #openstack-infra06:57
*** markvoelker has quit IRC06:58
*** Julien-zte has quit IRC06:59
*** amotoki_ has joined #openstack-infra07:00
*** tuanluong has joined #openstack-infra07:00
*** david-lyle has quit IRC07:02
*** amotoki has quit IRC07:03
*** vhosakot has quit IRC07:03
*** Julien-z_ has quit IRC07:07
*** pgadiya has joined #openstack-infra07:08
*** pgadiya has quit IRC07:08
*** andreas_s has joined #openstack-infra07:08
*** pgadiya has joined #openstack-infra07:09
*** amotoki has joined #openstack-infra07:18
*** Wei_Liu has quit IRC07:18
*** spzala has joined #openstack-infra07:18
*** aviau has quit IRC07:20
*** aviau has joined #openstack-infra07:20
*** amotoki_ has quit IRC07:21
*** Wei_Liu has joined #openstack-infra07:22
*** e0ne has joined #openstack-infra07:22
*** jtomasek has quit IRC07:23
*** spzala has quit IRC07:23
*** mozart89 has quit IRC07:24
*** mozart89 has joined #openstack-infra07:25
*** Julien-zte has joined #openstack-infra07:26
*** ym200 has joined #openstack-infra07:27
*** ym200 has quit IRC07:32
*** Wei_Liu1 has joined #openstack-infra07:34
*** tesseract has joined #openstack-infra07:35
*** Wei_Liu has quit IRC07:35
*** Wei_Liu1 is now known as Wei_Liu07:35
*** funzo has joined #openstack-infra07:37
jiaohaolin1hey guys, the wiki seems broken07:41
jiaohaolin1https://wiki.openstack.org/07:41
*** funzo has quit IRC07:42
jiaohaolin1Sorry! This site is experiencing technical difficulties.Try waiting a few minutes and reloading.(Cannot access the database)07:42
*** e0ne has quit IRC07:43
*** spzala has joined #openstack-infra07:43
*** ssbarnea has joined #openstack-infra07:44
*** ym200 has joined #openstack-infra07:45
*** psachin_ has quit IRC07:47
*** spzala has quit IRC07:48
*** Wei_Liu1 has joined #openstack-infra07:48
*** Wei_Liu has quit IRC07:49
*** Wei_Liu1 is now known as Wei_Liu07:49
*** ralonsoh has joined #openstack-infra07:49
*** spzala has joined #openstack-infra07:50
*** ym200 has quit IRC07:50
*** nadya has joined #openstack-infra07:50
*** matrohon has joined #openstack-infra07:51
*** markus_z has joined #openstack-infra07:52
*** spzala has quit IRC07:54
*** psachin_ has joined #openstack-infra07:59
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-infra08:00
*** sree_ has joined #openstack-infra08:01
*** sree_ is now known as Guest2928408:01
*** spzala has joined #openstack-infra08:02
*** ym200 has joined #openstack-infra08:03
*** sree has quit IRC08:04
*** martinkopec has joined #openstack-infra08:04
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Use StoryBoard for python-monascaclient  https://review.openstack.org/44793208:05
*** florianf has joined #openstack-infra08:05
*** gibi has joined #openstack-infra08:06
*** shardy has joined #openstack-infra08:07
*** spzala has quit IRC08:07
*** ym200 has quit IRC08:08
openstackgerritvenkata anil proposed openstack-infra/devstack-gate master: Set agent modes for DVR+HA multinode job  https://review.openstack.org/38382708:08
*** matrohon has quit IRC08:09
*** pcaruana has joined #openstack-infra08:11
*** shu-mutou is now known as shu-mutou-AWAY-u08:11
*** shu-mutou-AWAY-u is now known as shu-mutou-AWAY08:12
*** ym200 has joined #openstack-infra08:24
*** shardy has quit IRC08:24
openstackgerritMerged openstack-infra/project-config master: Add releasenotes job for kuryr  https://review.openstack.org/44067608:26
yolanda#status alert Wiki is broken with database problems, we are working to resolve it08:28
openstackstatusyolanda: sending alert08:28
*** ym200 has quit IRC08:28
openstackgerritMerged openstack-infra/project-config master: Make Zaqar Swift gate voting  https://review.openstack.org/44771908:28
*** e0ne has joined #openstack-infra08:30
-openstackstatus- NOTICE: Wiki is broken with database problems, we are working to resolve it08:31
*** knangia has quit IRC08:31
*** ChanServ changes topic to "Wiki is broken with database problems, we are working to resolve it"08:31
openstackgerritThomas Herve proposed openstack-infra/project-config master: Disable swift on heat python3 gate (again)  https://review.openstack.org/44794708:31
*** matrohon has joined #openstack-infra08:32
*** dingyichen has quit IRC08:33
*** jpich has joined #openstack-infra08:34
*** zhurong has joined #openstack-infra08:35
*** matrohon has quit IRC08:36
yolanda#success Wiki problems have been fixed, it's up and running08:36
openstackstatusyolanda: Added success to Success page08:36
yolanda#status ok Wiki problems have been fixed, it's up and running08:37
openstackstatusyolanda: sending ok08:37
*** funzo has joined #openstack-infra08:38
*** panda|off is now known as panda|pto08:38
*** spzala has joined #openstack-infra08:38
AJaegerthanks, yolanda !08:38
*** ChanServ changes topic to "Discussion of OpenStack Developer and Community Infrastructure | docs http://docs.openstack.org/infra/ | bugs https://storyboard.openstack.org/ | source https://git.openstack.org/cgit/openstack-infra/ | channel logs http://eavesdrop.openstack.org/irclogs/%23openstack-infra/"08:39
-openstackstatus- NOTICE: Wiki problems have been fixed, it's up and running08:39
*** dizquierdo has joined #openstack-infra08:40
yolandai wonder if something had changed recently, mysql was struggling, could not start due to apparmor conflicts08:40
*** amoralej|off is now known as amoralej08:41
*** spzala has quit IRC08:42
openstackstatusyolanda: finished sending ok08:43
*** funzo has quit IRC08:43
openstackgerritMerged openstack-infra/project-config master: Swith Heat amqp job to Ubuntu  https://review.openstack.org/44692908:44
openstackgerritMerged openstack-infra/project-config master: Add repo for VM/Baremetal working group  https://review.openstack.org/44670308:45
*** hashar has joined #openstack-infra08:45
openstackgerritMerged openstack-infra/project-config master: Add monitorstack project for OpenStack-Ansible  https://review.openstack.org/44432608:46
*** ym200 has joined #openstack-infra08:47
openstackgerritMerged openstack-infra/project-config master: Standalone Docker Machine driver for OpenStack  https://review.openstack.org/44658608:48
openstackgerritMerged openstack-infra/project-config master: End Project Gating for python-k8sclient  https://review.openstack.org/44763508:49
openstackgerritMerged openstack-infra/project-config master: Run separate jobs for py2 and py3 d-g unittests  https://review.openstack.org/44764108:50
openstackgerritMerged openstack-infra/project-config master: layout/tripleo: remove panko from scenario002  https://review.openstack.org/44453508:50
openstackgerritMerged openstack-infra/project-config master: layout/tripleo: run upgrade job in tripleo-common  https://review.openstack.org/44561008:51
openstackgerritMerged openstack-infra/project-config master: Remove Gnocchi doc publishing on docs.o.o  https://review.openstack.org/44745008:52
*** ym200 has quit IRC08:52
openstackgerritMerged openstack-infra/project-config master: Fix tempest-dsvm-neutron-multinode-full job VNC console settings  https://review.openstack.org/44416408:54
*** markvoelker has joined #openstack-infra08:55
openstackgerritKevin Benton proposed openstack-infra/irc-meetings master: Adjust time of neutron meeting and neutron-drivers  https://review.openstack.org/44796108:59
*** rossella_s has joined #openstack-infra09:00
*** markvoelker has quit IRC09:00
*** pgadiya has quit IRC09:01
*** openstackgerrit has quit IRC09:03
*** pgadiya has joined #openstack-infra09:04
*** ym200 has joined #openstack-infra09:11
*** mhickey has joined #openstack-infra09:12
*** dtantsur|afk is now known as dtantsur09:13
*** ym200 has quit IRC09:16
*** openstackgerrit has joined #openstack-infra09:17
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/44796709:17
openstackgerritMerged openstack-infra/irc-meetings master: Adjust time of neutron meeting and neutron-drivers  https://review.openstack.org/44796109:18
jpichAJaeger: Good morning! I'm still looking into the tripleo-ui translation job issues. I'm moderately confident the patch that merged yesterday should fix it, however it doesn't look like the post job ran afterwards, looking at http://status.openstack.org/openstack-health/#/job/tripleo-ui-upstream-translation-update ... I thought there should be a new entry there whenever a patch merges?09:19
*** nadya has quit IRC09:21
*** spzala has joined #openstack-infra09:26
*** matrohon has joined #openstack-infra09:29
*** ym200 has joined #openstack-infra09:29
*** spzala has quit IRC09:31
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: test containers with single-nic-vlan  https://review.openstack.org/44797309:32
*** ym200 has quit IRC09:33
AJaegerjpich: that looks indeed strange, let me check...09:34
jpichAJaeger: Thanks!09:35
*** ociuhandu has joined #openstack-infra09:35
AJaegerjpich: hash of last commit of repo gives you this log file: http://logs.openstack.org/ad/ad6314526b80f656b4807ed0f537f3a089c9b87c/ - so let's look at http://logs.openstack.org/ad/ad6314526b80f656b4807ed0f537f3a089c9b87c/post/tripleo-ui-upstream-translation-update/7c973ec/09:36
AJaeger"ValueError: No JSON object could be decoded"09:36
AJaegerbut that is in our CI scripts.09:36
AJaegerianychoi: ^09:36
*** derekh has joined #openstack-infra09:36
jpichOh, I see09:37
*** spzala has joined #openstack-infra09:38
*** funzo has joined #openstack-infra09:39
*** nadya has joined #openstack-infra09:42
*** jtomasek has joined #openstack-infra09:42
*** spzala has quit IRC09:43
*** funzo has quit IRC09:43
*** flepied has quit IRC09:48
jpichAJaeger, ianychoi: I wonder if that error may have been transient, it seems to work fine now when I try the script locally09:49
*** ym200 has joined #openstack-infra09:50
*** bhavik1 has quit IRC09:52
AJaegerjpich: run "git pull" in the repo and it will fail. I know what's broken...09:52
jpichOh, I hadn't refreshed that repo for a while indeed09:54
*** tosky has joined #openstack-infra09:54
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Fix query-zanata  https://review.openstack.org/44798209:54
AJaegerjpich, ianychoi ^09:54
*** ym200 has quit IRC09:54
AJaegeryolanda: could you review the change above to unbreak translation sync, please? ^09:54
yolandasure09:55
jpichAJaeger: I will pull and test as well, thank you for looking into this! You seem to have an empty comment line in there FWIW :)09:55
*** markvoelker has joined #openstack-infra09:56
*** s-shiono_ has quit IRC09:56
AJaegerthanks, fixing09:57
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Fix query-zanata  https://review.openstack.org/44798209:58
AJaegerjpich: so, health at no data since the script failed far earlier.09:59
*** gildub has quit IRC09:59
*** markvoelker has quit IRC10:00
*** pgadiya has quit IRC10:02
*** zhurong has quit IRC10:05
*** neiljerram has joined #openstack-infra10:06
*** ym200 has joined #openstack-infra10:07
AJaegeryolanda: could you +2 again, please? Just fixed layout. Thanks!10:08
*** nicolasbock has joined #openstack-infra10:08
*** liujiong_lj has joined #openstack-infra10:09
yolandadone10:09
AJaegerthanks10:09
* AJaeger will self +2A change 447982 since currently pushing to zanata is completely broken10:09
*** liujiong has quit IRC10:09
*** Julien-zte has quit IRC10:10
*** yanyanhu has quit IRC10:11
AJaegerjpich: once this is merged, it needs an hour to be acitive - so you need to get something else merged to triplo-ui afterwards to see that your fix worsk. Thanks for alerting us of the issue!10:12
*** ym200 has quit IRC10:12
*** liujiong_lj has quit IRC10:12
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/44796710:13
jpichAJaeger: And thank you for fixing it! I'll see if there's something simple I can encourage people to merge :)10:13
*** jaosorior is now known as jaosorior_brb10:14
*** spzala has joined #openstack-infra10:15
*** pgadiya has joined #openstack-infra10:16
*** amotoki has quit IRC10:17
*** spzala has quit IRC10:19
*** Julien-zte has joined #openstack-infra10:22
*** flepied has joined #openstack-infra10:23
*** zhurong has joined #openstack-infra10:24
openstackgerritMerged openstack-infra/project-config master: Fix query-zanata  https://review.openstack.org/44798210:24
*** gcb has quit IRC10:26
*** ccamacho has joined #openstack-infra10:27
*** gcb has joined #openstack-infra10:28
*** jamielennox|away is now known as jamielennox10:30
*** ym200 has joined #openstack-infra10:31
*** skelso has joined #openstack-infra10:32
*** yamahata has quit IRC10:32
*** winggundamth has quit IRC10:34
*** ym200 has quit IRC10:35
*** gcb has quit IRC10:37
*** kaisers1 has quit IRC10:38
*** alexchadin has quit IRC10:39
*** funzo has joined #openstack-infra10:39
*** e0ne has quit IRC10:42
*** jkilpatr has quit IRC10:44
*** e0ne has joined #openstack-infra10:44
*** funzo has quit IRC10:44
*** shardy has joined #openstack-infra10:45
*** gildub has joined #openstack-infra10:47
*** wolverineav has joined #openstack-infra10:48
*** ym200 has joined #openstack-infra10:49
*** spzala has joined #openstack-infra10:51
*** ym200 has quit IRC10:53
*** wolverineav has quit IRC10:53
*** kaisers has joined #openstack-infra10:53
*** alexchadin has joined #openstack-infra10:54
*** esikachev has quit IRC10:54
*** spzala has quit IRC10:56
*** e0ne has quit IRC10:56
*** gcb has joined #openstack-infra10:56
*** faizy has joined #openstack-infra10:56
*** gcb has quit IRC10:57
*** gcb has joined #openstack-infra10:57
*** faizy has quit IRC10:59
*** faizy has joined #openstack-infra11:00
*** alexchadin has quit IRC11:01
*** alexchadin has joined #openstack-infra11:01
*** jkilpatr has joined #openstack-infra11:02
*** e0ne has joined #openstack-infra11:03
*** wolverineav has joined #openstack-infra11:04
*** matrohon has quit IRC11:05
*** jamielennox is now known as jamielennox|away11:05
*** gcb has quit IRC11:06
*** alexchadin has quit IRC11:06
*** ym200 has joined #openstack-infra11:07
*** wolverineav has quit IRC11:09
*** sdague__ is now known as sdague11:09
*** skelso has quit IRC11:11
*** ym200 has quit IRC11:12
*** jamielennox|away is now known as jamielennox11:12
ianychoiAJaeger, thanks for fixing https://review.openstack.org/#/c/447982/ -> It was because of my last patch: https://review.openstack.org/#/c/447178/11:12
openstackgerritfumihiko kakuma proposed openstack-infra/project-config master: Add neutron job with ryu master  https://review.openstack.org/44526211:13
*** Wei_Liu has quit IRC11:16
*** wolverineav has joined #openstack-infra11:16
*** gcb has joined #openstack-infra11:20
*** wolverineav has quit IRC11:21
*** faizy_ has joined #openstack-infra11:24
*** dizquierdo has quit IRC11:25
*** dizquierdo has joined #openstack-infra11:25
*** faizy has quit IRC11:25
*** ym200 has joined #openstack-infra11:26
*** ym200 has quit IRC11:30
*** jamespage has quit IRC11:31
*** jamespage has joined #openstack-infra11:31
*** gongysh has quit IRC11:33
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760211:34
*** askb has quit IRC11:35
*** amotoki has joined #openstack-infra11:36
*** ldnunes has joined #openstack-infra11:39
*** dizquierdo is now known as dizquierdo_afk11:40
*** ym200 has joined #openstack-infra11:41
*** ym200 has quit IRC11:41
*** kornicameister has joined #openstack-infra11:42
*** ym200 has joined #openstack-infra11:43
*** Julien-zte has quit IRC11:44
AJaegerianychoi: yeah, figured that one out11:45
openstackgerritAttila Darazs proposed openstack-infra/project-config master: Add tripleo-quickstart* repos to the tripleo group  https://review.openstack.org/44802411:47
*** faizy_ has quit IRC11:48
*** rfolco has joined #openstack-infra11:49
*** lucasagomes is now known as lucas-brb11:50
*** yolanda has quit IRC11:52
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create zuul_workspace_root job variable  https://review.openstack.org/44144111:53
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Organize playbooks folder  https://review.openstack.org/44154711:53
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info to bootstrap role  https://review.openstack.org/44161711:53
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add revoke-sudo role and update tox jobs  https://review.openstack.org/44146711:53
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create tox-tarball job  https://review.openstack.org/44160911:53
*** yolanda has joined #openstack-infra11:54
*** pgadiya has quit IRC11:57
*** aysyd has joined #openstack-infra11:57
*** vsaienko has joined #openstack-infra11:57
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add ansible-lint to tox (pep8)  https://review.openstack.org/43827611:58
*** dizquierdo_afk is now known as dizquierdo11:59
*** stevejims has joined #openstack-infra11:59
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704712:01
pabelangerAJaeger: So, if you have some time: 447089 might be good for you to review12:01
pabelangerAJaeger: since you just spend time removing all the tox-db jobs :D12:01
*** skelso has joined #openstack-infra12:01
stevejimshi, anyone able to answer a question about launchpad -> storyboard migration?12:01
*** radez` is now known as radez12:01
AJaegerpabelanger: why add them back again?12:01
pabelangerAJaeger: hopefully the review will make sense, but created a base playbook to setup the database in ansible12:02
*** alexchadin has joined #openstack-infra12:03
pabelangerAJaeger: it something I think we'll bikeshed today12:03
AJaegerpabelanger: WE don't need that IMHO.12:03
*** tuanluong has quit IRC12:03
AJaegerwe have extra-test-setup macro for this in project-config and I suggest to support that logic instead.12:04
*** ym200 has quit IRC12:05
*** ym200 has joined #openstack-infra12:06
pabelangerright, there is going to be a few way to do this in zuulv312:08
*** pgadiya has joined #openstack-infra12:08
pabelangerthe first question, do we as openstack-infra want to support a playbook to setup database things for jobs12:09
pabelangerbecause, it would be possible for a project to have there own .zuul.yaml with creates a tox-db-py27 job as I just did^12:09
pabelangerthis moves the tools/test.sh script to a playbook too12:10
*** rossella_s has quit IRC12:11
AJaegeryes, that's all possible. Part of the move away from db jobs was that we document to developer how to setup their own systems. A playbook would do that as well but would mean another language to learn. A shell script is far simpler here...12:11
pabelangeryes, but in zuulv3, we'll expect jobs to also write playbooks in ansible12:12
pabelangerlet me do this12:12
AJaegerLet's discuss another time...12:12
*** rossella_s has joined #openstack-infra12:12
pabelangerI'll port the extra-test-setup logic into ansible now12:12
pabelangerand we can discuss12:12
AJaegerthanks. That'S something we should do anyhow.12:12
AJaegerok, until later...12:12
openstackgerritMerged openstack-infra/shade master: add separate releasenotes build  https://review.openstack.org/44665812:14
*** rhallisey has joined #openstack-infra12:19
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Switch trunk/cbs/buildlogs to use https  https://review.openstack.org/44804112:20
*** gcb has quit IRC12:20
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create extra-test-setup role  https://review.openstack.org/44804212:20
*** gcb has joined #openstack-infra12:21
pabelangerAJaeger: ^JJB way for ansible12:23
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: Switch trunk/cbs/buildlogs to use https  https://review.openstack.org/44804712:24
openstackgerritMerged openstack-infra/shade master: Convert test_identity_roles to requests mock  https://review.openstack.org/44567312:26
*** EricGonczer_ has joined #openstack-infra12:27
*** Guest29284 has quit IRC12:27
*** sree has joined #openstack-infra12:28
*** sree_ has joined #openstack-infra12:30
*** gildub has quit IRC12:30
*** sree_ is now known as Guest7697812:30
*** Guest76978 has quit IRC12:31
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix test_node_assignment_at_quota  https://review.openstack.org/44763012:32
*** sree has quit IRC12:32
*** sree has joined #openstack-infra12:32
*** pradk has joined #openstack-infra12:32
*** dave-mccowan has joined #openstack-infra12:36
*** sree has quit IRC12:37
*** kgiusti has joined #openstack-infra12:37
*** ym200 has quit IRC12:38
*** rlandy has joined #openstack-infra12:38
*** jaosorior_brb is now known as jaosorior12:38
*** ym200 has joined #openstack-infra12:39
*** zhurong has quit IRC12:39
markus_zfrickler: AJaeger: yolanda: The VNC issue isn't fully resolved: https://bugs.launchpad.net/nova/+bug/166946812:41
openstackLaunchpad bug 1669468 in OpenStack Compute (nova) "tempest.api.compute.servers.test_novnc.NoVNCConsoleTestJSON.test_novnc fails intermittently in neutron multinode nv job" [Medium,In progress] - Assigned to Markus Zoeller (markus_z) (mzoeller)12:41
*** funzo has joined #openstack-infra12:41
markus_zThe values are correct, but the order of localrc variables isn't correct. I added it as comment to the bug report.12:42
AJaegermarkus_z: best discuss with sdague12:42
markus_zI'm clueless how to influence the order of localrc variables.12:42
*** matrohon has joined #openstack-infra12:42
fricklermarkus_z: hmm, I read something about neutron also needing to set local.conf stuff late, let me dig a bit12:42
sdaguemarkus_z: what's going on?12:43
markus_zsdague: This change https://review.openstack.org/#/c/444164/6 tried to solve intermittent VNC issues in multinode jobs.12:43
markus_zSomehow the localrc gets build with "VNCSERVER_PROXYCLIENT_ADDRESS=$HOST_IP" first, and then "HOST_IP=158.69.78.119". which doesn't work.12:44
*** gordc has joined #openstack-infra12:45
markus_z"doesn't work" means is sets an empty value. The bug report contains more info.12:45
*** trown|outtypewww is now known as trown12:45
markus_zhttp://logs.openstack.org/76/400876/31/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/63d2bdd/logs/subnode-2/local.conf.txt.gz12:46
*** funzo has quit IRC12:46
sdaguemarkus_z: it's only the subnode that has this issue, right?12:46
markus_zsdague: yes, the controller node sets it up correctly.12:47
*** EricGonczer_ has quit IRC12:47
sdaguemarkus_z: it's because of this line https://github.com/openstack-infra/devstack-gate/blob/ed2ed82338286f96cf7c017de30634757ea2d185/devstack-vm-gate.sh#L20412:48
*** markvoelker has joined #openstack-infra12:48
*** EricGonczer_ has joined #openstack-infra12:48
*** dprince has joined #openstack-infra12:48
markus_zsdague: Ah, it appends it always at the end?12:49
*** dave-mccowan has quit IRC12:49
*** rhallisey has quit IRC12:49
*** Julien-zte has joined #openstack-infra12:50
sdaguemarkus_z: yes12:51
*** rhallisey has joined #openstack-infra12:51
fricklersdague: so if we would add a dummy line earlier, would it replace that instead of adding at the end?12:51
sdagueno12:52
sdaguethe local section opperates in append only mode because it's really executable shell12:52
sdaguelet me try something...12:52
markus_zsdague: sure? I think it does: https://github.com/openstack-dev/devstack/blob/dec121114c3ea6f9e515a452700e5015d1e34704/inc/ini-config#L43612:52
markus_zIt does an inline "sed": https://github.com/openstack-dev/devstack/blob/dec121114c3ea6f9e515a452700e5015d1e34704/inc/ini-config#L33812:53
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Add libffi development headers to bindep  https://review.openstack.org/44807612:53
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix test_node_assignment_at_quota  https://review.openstack.org/44763012:55
*** mriedem has joined #openstack-infra12:56
*** kencjohnston_ has quit IRC12:56
*** cshastri has quit IRC12:56
openstackgerritSean Dague proposed openstack-infra/devstack-gate master: Set HOST_IP for subnode first  https://review.openstack.org/44807812:57
*** markmcd has quit IRC12:57
sdaguemarkus_z: we aren't using that12:57
*** shardy has quit IRC12:57
*** dave-mccowan has joined #openstack-infra12:57
sdaguemarkus_z: https://github.com/openstack-infra/devstack-gate/blob/873e8e5ea0b1b0a1f1544586c08741dee0f672d0/devstack-vm-gate.sh#L40-L4412:57
openstackgerritDavid Moreau Simard proposed openstack-infra/tripleo-ci master: Do not merge: Test python-heat-agents package build  https://review.openstack.org/43917812:58
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing novajoin authtoken  https://review.openstack.org/44634812:58
*** markmcd has joined #openstack-infra12:58
*** jcoufal has joined #openstack-infra12:58
markus_zsdague: Ah, didn't know that, thanks.12:59
fungiyolanda: what did you end up doing to clear the issue with the wiki?13:00
fungiyolanda: as for package upgrades, the following got upgraded today on that server at 06:36 utc: libc-bin libc-bin libc-bin libc-dev-bin libc6 libc6-dev libfreetype6 libgnutls-openssl27 libgnutls26 man-db multiarch-support13:01
fungier, i guess i got a couple repeats in there13:01
*** gouthamr has joined #openstack-infra13:02
*** gongysh has joined #openstack-infra13:02
*** ym200 has quit IRC13:03
fungiyolanda: you said something about mysql being a problem an not starting, but we don't use a local mysql for that service (it's all in a trove db)13:03
*** shardy has joined #openstack-infra13:03
*** rosmaita_ is now known as rosmaita13:04
*** ym200 has joined #openstack-infra13:04
*** kornicameister has quit IRC13:04
*** eharney has joined #openstack-infra13:05
yolandafungi, oh... so thee was a mysql service running there, but when i restarted it didn't come back. So i fixed and it came up cleanly, that's why i thought it was the failure. But i also did a restart of apache, so maybe that was the real failure...13:05
*** kornicameister has joined #openstack-infra13:05
fungiagreed, apache seems far more likely to have been the culprit13:05
*** baoli has joined #openstack-infra13:06
*** baoli has quit IRC13:06
*** markmcd has quit IRC13:06
fungiwell, anyway, thanks for getting things going again13:06
*** Goneri has joined #openstack-infra13:06
*** gongysh has quit IRC13:07
sdaguemarkus_z: ok, can you see if that  https://review.openstack.org/448078 fixes things?13:07
*** baoli has joined #openstack-infra13:08
fricklersdague: can one have a nova patch depend on your change to test it or will it need to be merged first?13:08
markus_zsdague: I have to wait until its multinode job is executed, right? Or is there another way to test it?13:08
yolandafungi yes, looks like, although apache error logs did not show any problem13:08
*** baoli has quit IRC13:09
sdaguemarkus_z: yeh, probably just wait and keep an eye on it13:09
*** baoli has joined #openstack-infra13:09
markus_zsdague: yep, will do that.13:09
*** eroux has joined #openstack-infra13:10
*** ym200 has quit IRC13:10
*** mdrabe has joined #openstack-infra13:11
*** EricGonczer_ has quit IRC13:11
*** markmcd has joined #openstack-infra13:11
*** mat128 has joined #openstack-infra13:11
*** alexchadin has quit IRC13:11
*** ym200 has joined #openstack-infra13:12
*** gongysh has joined #openstack-infra13:12
*** funzo has joined #openstack-infra13:13
*** lucas-brb is now known as lucasagomes13:14
*** gongysh has quit IRC13:15
*** ym200 has quit IRC13:16
*** ym200 has joined #openstack-infra13:17
*** eroux has quit IRC13:19
*** ramishra has quit IRC13:19
*** rhallisey has quit IRC13:20
*** anilvenkata has quit IRC13:20
*** dtantsur is now known as dtantsur|bbl13:20
*** amoralej is now known as amoralej|lunch13:20
*** rhallisey has joined #openstack-infra13:21
*** wolverineav has joined #openstack-infra13:23
*** links has quit IRC13:24
*** jamesdenton has joined #openstack-infra13:25
*** prateek has quit IRC13:25
*** ihrachys has joined #openstack-infra13:26
*** wolverin_ has joined #openstack-infra13:26
*** wolverineav has quit IRC13:28
*** jtomasek has quit IRC13:29
*** pgadiya has quit IRC13:30
*** faizy_ has joined #openstack-infra13:31
*** sekelso has joined #openstack-infra13:33
*** faizy__ has joined #openstack-infra13:33
*** skelso has quit IRC13:35
*** mdrabe has quit IRC13:36
*** faizy_ has quit IRC13:36
*** mdrabe has joined #openstack-infra13:37
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44752213:38
*** sekelso has quit IRC13:38
openstackgerritMikhail S Medvedev proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809913:39
openstackgerritMonty Taylor proposed openstack/os-client-config master: Add designateclient to constructors list  https://review.openstack.org/44810013:39
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: DONT REVIEW: test consistent repo  https://review.openstack.org/44752213:39
*** adrian_otto has joined #openstack-infra13:40
*** eroux has joined #openstack-infra13:42
*** ym200 has quit IRC13:42
*** jtomasek has joined #openstack-infra13:43
*** ym200 has joined #openstack-infra13:43
*** rwsu has quit IRC13:45
*** ym200 has quit IRC13:47
*** ym200 has joined #openstack-infra13:48
*** dizquierdo has quit IRC13:49
openstackgerritDavid Moreau Simard proposed openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html"  https://review.openstack.org/44810813:51
*** faizy__ has quit IRC13:51
*** tonytan4ever has joined #openstack-infra13:51
*** amoralej|lunch is now known as amoralej13:53
*** mat128 has quit IRC13:55
*** spzala has joined #openstack-infra13:55
*** tosky has quit IRC13:55
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Add libffi development headers to bindep  https://review.openstack.org/44807613:56
*** bh526r has joined #openstack-infra13:57
*** adrian_otto has quit IRC13:58
*** kornicameister has quit IRC13:58
*** ZZelle_ has left #openstack-infra13:59
*** mtanino has joined #openstack-infra13:59
*** ZZelle has joined #openstack-infra13:59
*** ZZelle is now known as Guest9367214:00
*** tosky has joined #openstack-infra14:01
*** Guest93672 is now known as ZZelle__14:01
openstackgerritMerged openstack-infra/project-config master: Enable Images v1 API for glanceclient tests  https://review.openstack.org/44662714:01
*** tphummel has joined #openstack-infra14:04
openstackgerritMonty Taylor proposed openstack-infra/shade master: Get rid of magnumclient dependency  https://review.openstack.org/44757814:04
*** ZZelle__ is now known as ZZelle_14:04
*** rbrndt has joined #openstack-infra14:07
*** xarses has quit IRC14:08
*** saibarspeis has quit IRC14:12
openstackgerritRoman Podoliaka proposed openstack-infra/project-config master: placement-osc-plugin: an OSC plugin for Placement service  https://review.openstack.org/44811514:14
fricklersdague: markus_z: http://logs.openstack.org/78/448078/1/check/gate-tempest-dsvm-neutron-multinode-full-ubuntu-xenial-nv/3e60914/logs/subnode-2/local.conf.txt.gz lgtm14:17
*** chlong has joined #openstack-infra14:18
*** knangia has joined #openstack-infra14:19
*** zz_dimtruck is now known as dimtruck14:20
*** jkilpatr has quit IRC14:21
*** burgerk has joined #openstack-infra14:22
*** sekelso has joined #openstack-infra14:22
*** nadya has quit IRC14:24
*** chlong has quit IRC14:24
*** EricGonczer_ has joined #openstack-infra14:24
sc`AJaeger: thanks for -wf that patch. i was in the midst of my morning tea14:29
*** sekelso has quit IRC14:29
AJaeger;)14:31
*** alexchadin has joined #openstack-infra14:31
*** admcleod has joined #openstack-infra14:33
*** hashar has quit IRC14:33
*** eharney has quit IRC14:33
*** fguillot has joined #openstack-infra14:34
*** jkilpatr has joined #openstack-infra14:34
*** eranrom has quit IRC14:35
*** hashar has joined #openstack-infra14:35
*** vikrant has quit IRC14:36
*** eranrom has joined #openstack-infra14:36
markus_zfrickler: yep, that's it +114:37
*** diltram has quit IRC14:37
*** e0ne has quit IRC14:37
*** pblaho has quit IRC14:39
ym200i am at X version of  mitaka release how  can i go to X + 114:39
ym200can someone suggest14:39
bretonttx: could you please confirm that a mail to openstack-dev-owner@ with subject "Issues with sending" was received?14:40
ym200say i am at 13.X.X14:40
ym200& if i wanna go to 13.y.y14:40
*** eranrom has quit IRC14:41
*** eranrom has joined #openstack-infra14:41
sc`frickler: AJaeger: i had node-release in there before submitting the change to gerrit, but tox kept failing due to ignorance. does node still need to exist in that case?14:41
*** esikachev has joined #openstack-infra14:41
*** mat128 has joined #openstack-infra14:42
*** beslemon has joined #openstack-infra14:43
*** marst has quit IRC14:43
*** marst has joined #openstack-infra14:43
sc`i've been reading every bit of yaml and code i could find to better understand things. i've only been working with this part of things for a few weeks, so there are deficiencies in understanding14:44
*** ociuhandu has quit IRC14:44
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test new ovb with images building  https://review.openstack.org/44522814:45
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: New ovb playbook and settings  https://review.openstack.org/43156714:45
*** eharney has joined #openstack-infra14:47
*** andreas_s has quit IRC14:47
*** marst has quit IRC14:48
*** e0ne has joined #openstack-infra14:48
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Bring back postci log  https://review.openstack.org/44813514:49
*** david-lyle has joined #openstack-infra14:49
*** esikache1 has joined #openstack-infra14:49
*** mdrabe has quit IRC14:50
*** hongbin has joined #openstack-infra14:50
AJaegersc`: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/projects.yaml#n3703 - check for example '{pipeline}-tempest-dsvm-multinode-full-{node-release}{suffix}':14:51
AJaegerand see how it's defined in devstack-gate.yaml as well14:52
ttxbreton: hmm, no14:52
bretonttx: :(14:52
*** esikachev has quit IRC14:52
bretonttx: i will send it to knelson at openstack.org and thierry at openstack.org then14:53
AJaegerym200: better ask on #openstack channel. This channel is for infrastructure for CI and development.14:53
*** psachin_ has quit IRC14:53
*** marst has joined #openstack-infra14:54
*** faizy has joined #openstack-infra14:57
*** armax has quit IRC14:57
*** mdrabe has joined #openstack-infra14:58
*** numans has joined #openstack-infra14:58
*** xarses has joined #openstack-infra14:59
sc`AJaeger: thanks for the pointers. one of these days, it might actually work15:00
AJaegerand thanks to frickler for noticing that it was wrong15:01
*** dave-mccowan has quit IRC15:02
sc`indeed :D15:02
*** esikache1 has quit IRC15:02
fungiym200: i think you found the wrong channel. this is where we maintain the community infrastructure (mailing lists, continuous integration, wiki, et cetera)15:03
*** vsaienko has quit IRC15:03
fungiym200: you might want to try asking general openstack-software-related questions in #openstack instead15:03
*** vsaienko has joined #openstack-infra15:07
*** ym200 has quit IRC15:09
*** david-lyle has quit IRC15:09
*** ociuhandu has joined #openstack-infra15:10
*** xarses_ has joined #openstack-infra15:10
*** faizy has quit IRC15:10
*** faizy has joined #openstack-infra15:11
*** ym200 has joined #openstack-infra15:11
*** claudiub|2 has joined #openstack-infra15:13
*** dtantsur|bbl is now known as dtantsur15:14
*** pblaho has joined #openstack-infra15:14
*** claudiub has quit IRC15:14
*** faizy_ has joined #openstack-infra15:14
*** david-lyle has joined #openstack-infra15:15
clarkbis lists unhappy? (just noticing I haven't had a -dev email in about 10 hours)15:16
clarkband we did poke at it yesterday15:16
clarkbjeblair: fungi ^15:16
*** baoli has quit IRC15:17
*** sekelso has joined #openstack-infra15:17
*** faizy__ has joined #openstack-infra15:17
fungiclarkb: i was thinking volume was lowish this morning15:17
fungii wonder if unattended upgrades kicked in and "restarted" services without us having reenabled them15:17
AJaegerlatest email "Tue Mar 21 06:13:49 UTC 2017"15:18
fungithat looks like the right timeframe15:18
*** faizy has quit IRC15:18
*** xarses_ has quit IRC15:18
fungiexim is listening on 25/tcp15:18
AJaegerclarkb: send a test mail ?15:19
*** xarses_ has joined #openstack-infra15:19
*** xarses_ has quit IRC15:19
numansHi Infra team, I have been trying to fix a CI failure for networking-ovn. I am not able to reproduce it locally. Is it possible for me to get access to the VM in the infra where the issue is seen - https://review.openstack.org/#/c/447756/15:19
*** martinkopec has quit IRC15:19
*** xarses_ has joined #openstack-infra15:19
fungioldest message in the deferral queue is 10 hours15:19
clarkbnumans: have you tried running http://logs.openstack.org/56/447756/2/check/gate-tempest-dsvm-networking-ovn-ovs-master/a10e1f8/logs/reproduce.sh ?15:20
jeblairclarkb: looking15:20
clarkbAJaeger: ya though I expect many others have already sent email through my early morning because seems like -dev never stops15:20
jeblairmailman isn't running15:20
*** faizy_ has quit IRC15:20
*** faizy__ has quit IRC15:21
fungithat'd do it15:21
numansclarkb, no. I havent tried it. thanks for pointing me out15:21
clarkbnumans: the comment at the beginning of the file should tell you what is required to get that going15:21
openstackgerritRoman Podoliaka proposed openstack-infra/project-config master: placement-osc-plugin: an OSC plugin for Placement service  https://review.openstack.org/44811515:21
*** faizy__ has joined #openstack-infra15:21
numansclarkb, on to it. thanks.15:21
clarkbjeblair: fungi we haev our snapshot now so we can probably return the cron and service enablement at this point (enabling puppet may do it automatically for us)15:21
*** florianf has quit IRC15:22
fungijeblair: dpkg.log indicates unattended upgrades upgraded mailman at 2017-03-21 06:44:29 utc15:22
jeblairwhat luck.  :)15:22
*** dave-mccowan has joined #openstack-infra15:22
fungiindeed :/15:22
jeblairit's started, i'm finishing cleanup15:23
fungithanks!15:23
AJaegernumans: that failure looks unrelated to your changes. I know that eventlet was updated today to 0.20.1 and it caused problems elsewhere. Double check eventlet versions in gate and your local setup.15:23
fungi2017-03-21 06:44:29 upgrade mailman 1:2.1.14-3ubuntu0.2 1:2.1.14-3ubuntu0.415:23
*** shardy is now known as coding_shardy15:23
*** faizy__ has quit IRC15:24
numansAJaeger, ack. I will do that. thanks15:24
jeblairclarkb, fungi: bup cron, exim/mailman init scripts, and puppet emergency file all restored15:24
clarkband I got a bunch of -dev mail so that looks happy15:24
*** faizy__ has joined #openstack-infra15:24
mordred\o/15:24
jeblair exim -bpc15:24
jeblair517515:24
jeblairif you want to watch the queue numbers, btw.15:24
*** vsaienko has quit IRC15:25
* fungi shakes fist at ubuntu package maintainers for not updating /usr/share/doc/mailman/changelog.Debian.gz on latest patch update15:25
*** vsaienko has joined #openstack-infra15:25
fungithough it shows at http://changelogs.ubuntu.com/changelogs/pool/main/m/mailman/mailman_2.1.14-3ubuntu0.4/changelog15:26
*** faizy__ has quit IRC15:26
clarkbone thing I checked was that http://lists.openstack.org was up and happy and it was. Guessing that that doesn't actually require daemon to serve content just update content?15:27
*** faizy__ has joined #openstack-infra15:27
*** florianf has joined #openstack-infra15:27
jeblairyeah, it's all cgi (or equivalent) i think15:27
fungiyeep15:27
sdagueclarkb / fungi / jeblair - https://review.openstack.org/#/c/448078/ - should fix an issue for subnode set15:28
sdaguedevstack-gate change15:28
fungianyway, nice to see ubuntu still applying security fixes to the version of mailman in precise, even if it took us by surprise15:28
clarkbfungi: maybe our conversation about it yesterday made them go "wow people still use that? OK lets fix it"15:29
clarkb:)15:29
*** matrohon has quit IRC15:29
*** esikache1 has joined #openstack-infra15:29
*** faizy__ has quit IRC15:29
*** faizy__ has joined #openstack-infra15:30
openstackgerritRoman Podoliaka proposed openstack-infra/project-config master: placement-osc-plugin: an OSC plugin for Placement service  https://review.openstack.org/44811515:32
fungisdague: looks like frickler left an inline question15:32
*** jrist has quit IRC15:33
*** hieulq_ has joined #openstack-infra15:33
clarkbI +2'd but didn't approve in case my responset o frickler was incorrect15:33
clarkbsdague: ^ so maybe take a look and self approve if happy with it ?15:34
sdagueclarkb: no, I think it's correct15:35
sdaguemy response was similar15:35
sdagueonce the f25 job fails and times out, it will get to the gate, that's the only thing holding it back15:35
clarkbI briefly looked at that last week and it appears that libvirt just doesn't work?15:36
clarkblots of things crash hard there15:36
sdagueso, the f25 job is basically always going to a full timeout15:36
*** andreww has joined #openstack-infra15:36
sdaguethat feels like we should move it to experimental to keep it from slowing things down15:36
openstackgerritThomas Goirand proposed openstack-infra/system-config master: Do not install from puppetlab for deb  https://review.openstack.org/44581315:36
clarkbya thats probably a decent way to balance wanting to get f25 working and everything else while it doesn't15:37
sdagueany idea if anyone is looking at it?15:37
clarkbthen switch it if/when it is useable15:37
clarkbsdague: typically ianw has been on that15:37
clarkbbut EmilienM more recently too I think15:37
clarkb(not sure if either is actively addressing the current fails though)15:37
fungimoving nonvoting jobs to experimental when they're 100% failures seems like the right move to me15:37
fungieasy to repro15:37
fungino need to waste resources and, more importantly, turn-around time on results for the other jobs15:38
EmilienMclarkb: what's up?15:38
jeblairsdague, clarkb: belated +2, thanks for the answers :)15:38
*** adrian_otto has joined #openstack-infra15:39
clarkbEmilienM: fedora 25 + devstack + tempest is not working15:39
*** xarses has quit IRC15:39
clarkbEmilienM: it sets up a cloud but once tempest starts running many things fail and break then eventually the job times out15:39
jeblairclarkb, fungi: queue is trending down now fram a high of 19 or 20k or so.  at 16.6k.15:39
*** annegentle has joined #openstack-infra15:39
EmilienMclarkb: what version of qemy?15:39
clarkbEmilienM: whatever comes with f2515:39
EmilienMqemu*15:40
*** jrist has joined #openstack-infra15:40
clarkbEmilienM: I haven't actually debugged it much beyond "yup its broken" and not something silly easy to fix15:40
EmilienMclarkb: I haven't followed devstack jobs recently15:40
openstackgerritStephen Finucane proposed openstack-dev/cookiecutter master: Remove MANIFEST.in  https://review.openstack.org/44816315:40
openstackgerritStephen Finucane proposed openstack-dev/cookiecutter master: Remove 'build_releasenotes'  https://review.openstack.org/44816415:40
openstackgerritStephen Finucane proposed openstack-dev/cookiecutter master: Bump pbr to 2.0, Sphinx to 1.5  https://review.openstack.org/44816515:40
EmilienMit sounds like fedora jobs don't get much love15:40
openstackgerritStephen Finucane proposed openstack-dev/cookiecutter master: Use Sphinx 1.5 warning-is-error  https://review.openstack.org/44816615:40
openstackgerritSean Dague proposed openstack-infra/project-config master: move fedora 25 to experimental on d-g and devstack  https://review.openstack.org/44816715:40
sdagueclarkb / jeblair https://review.openstack.org/448167 - that does the move15:41
jeblairpabelanger: fyi ^15:41
EmilienMI don't think we should keep fedora jobs15:42
EmilienMand rather keep gate-tempest-dsvm-neutron-full-centos-7-nv15:42
EmilienMcentos7 is much more reliable15:42
pabelangerjeblair: no objection15:42
Shrewsclarkb: i believe i have satisified your requirements for the test in https://review.openstack.org/44763015:43
sdaguepabelanger: if someone wants to fix it, it's cool to bring it back to the check queue, but it seems to be quite hard on the failure side15:43
clarkbShrews: cool will take a look15:43
ttxbreton: now I received both :)15:44
pabelangersdague: seems reasonable15:44
openstackgerritRamy Asselin proposed openstack-infra/system-config master: Jeepyb also needs the committer e-mail in gerrit  https://review.openstack.org/44816815:44
*** yamahata has joined #openstack-infra15:45
*** sekelso has quit IRC15:45
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704715:46
*** sekelso has joined #openstack-infra15:46
*** annegentle has quit IRC15:46
*** annegentle has joined #openstack-infra15:47
fungittx: a mishap related to our server upgrade testing for the listserv resulted in it queuing up inbound messages for 10-11 hours15:47
fungibreton: ^ sorry about that!~15:48
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704715:49
*** armax has joined #openstack-infra15:49
*** rkukura has quit IRC15:50
*** xarses_ has quit IRC15:50
* clarkb is going to pop out now for breakfast and other morning things. Back in a bit.15:51
openstackgerritMerged openstack-infra/nodepool feature/zuulv3: Fix test_node_assignment_at_quota  https://review.openstack.org/44763015:51
*** adrian_otto has quit IRC15:52
clarkbsdague: oh before I go, thoughts on https://review.openstack.org/#/c/447119/ ? it seems to have some affect on total memory used according to dstat but I think I need more run numbers to say that with confidence15:52
*** vsaienko has quit IRC15:52
clarkbsdague: thats the enable KSM in our test hypervisors change15:52
*** hamzy has quit IRC15:53
sdagueok15:53
sdaguewhere are all the mem tracker patches atm?15:53
*** vsaienko has joined #openstack-infra15:54
*** david-lyle has quit IRC15:54
clarkbI'm not sure about mem tracker. Have relied on dstat more since for ksm at least I think it gives a better vuew (rss isnt affected by ksm aiui)15:56
*** dizquierdo has joined #openstack-infra15:56
openstackgerritGage Hugo proposed openstack-dev/pbr master: Add Changelog build handling for invalid chars  https://review.openstack.org/43989715:56
*** bhavik1 has joined #openstack-infra15:56
*** david-lyle has joined #openstack-infra16:01
sdagueyeh, that's fine, just the memtracker was going to answer the mlock question16:01
*** adrian_otto has joined #openstack-infra16:01
openstackgerritMerged openstack-infra/project-config master: move fedora 25 to experimental on d-g and devstack  https://review.openstack.org/44816716:01
sdagueand while it's fine to nibble around the edges here, until someone gets into the python process weight issue... there isn't a whole lot of progress to be made16:01
*** xarses has joined #openstack-infra16:03
*** xarses has quit IRC16:03
*** xarses has joined #openstack-infra16:04
fungiif anything, slimming kernel memory overhead and mysql may just be postponing the pain needed for various teams to actually dig deeper on improving openstack service footprints16:04
*** xarses has quit IRC16:04
*** xarses has joined #openstack-infra16:05
bkeroAre you suggesting our software may be heavy?16:05
fungiespecially given it's still early in teh cycle, and a bunch of projects put off implementation of major features last cycle to help with stabilization16:05
fungiexpecting this to get worse (leading up to pike feature freeze) before it gets better16:06
*** ym200 has quit IRC16:06
*** ym200 has joined #openstack-infra16:07
clarkbyup I completely agree16:07
*** isaacb has joined #openstack-infra16:07
*** isaacb_ has joined #openstack-infra16:07
clarkbbut at lwast this way I can tell then " we did our part on the infrastructure side, now we really need you to do yours "16:08
*** isaacb_ has left #openstack-infra16:08
*** ym200 has quit IRC16:09
*** vsaienko has quit IRC16:09
*** mozart89 has quit IRC16:09
*** ym200 has joined #openstack-infra16:09
*** spzala has quit IRC16:10
*** rossella_s has quit IRC16:11
*** vsaienko has joined #openstack-infra16:11
*** rossella_s has joined #openstack-infra16:12
*** amotoki has quit IRC16:13
fungithat won't stop most teams from just asking us to give them instances with more ram16:14
fungipeople, developers in particular though, have become accustomed to ever increasing inefficiency on the assumption it's easier to just keep throwing more and more hardware at the problem16:16
*** soliosg1 has quit IRC16:16
funginow that moore's law is dead, maybe we'll see that trend begin to reverse a little16:17
*** beslemon has quit IRC16:18
*** hieulq_ has quit IRC16:20
*** e0ne has quit IRC16:20
openstackgerritsebastian marcet proposed openstack-infra/openstackid-resources master: Updated feedback validation rules  https://review.openstack.org/44818416:21
openstackgerritMerged openstack-infra/openstackid-resources master: Updated feedback validation rules  https://review.openstack.org/44818416:25
*** leong has joined #openstack-infra16:26
*** dhajare has quit IRC16:26
*** ym200 has quit IRC16:27
leonghi, is there an etherpad api that we can use for etherpad.openstack.org?16:27
*** ym200 has joined #openstack-infra16:28
*** jaosorior has quit IRC16:29
jeblairfungi, clarkb: i'm thinking of ccing the openstack-dev list on my encryption email to get more eyes on it.  good idea?  how should i [tag] it?16:29
*** erikwilson has quit IRC16:29
openstackgerritBen Nemec proposed openstack-infra/tripleo-ci master: Only force http for repos that allow it  https://review.openstack.org/44818716:29
fungileong: you can try http://etherpad.org/doc/v1.5.1/#index_http_api16:30
*** Julien-zte has quit IRC16:30
*** Julien-zte has joined #openstack-infra16:30
fungileong: obviously any methods which require an api key won't work, but i don't know whether our apache proxy plumbs through enough of the url pattern to allow anonymous methods to still work16:31
*** Julien-zte has quit IRC16:31
fungijeblair: [infra][security] maybe?16:31
*** Julien-zte has joined #openstack-infra16:32
*** Julien-zte has quit IRC16:32
*** Julien-zte has joined #openstack-infra16:32
*** Julien-zte has quit IRC16:32
*** alexchadin has quit IRC16:33
*** Julien-zte has joined #openstack-infra16:33
*** soliosg has left #openstack-infra16:33
*** Julien-zte has quit IRC16:33
fungii skimmed official tags for the ml and those were the only two which jumped out at me as applicable16:34
jeblairfungi: cool, thanks.16:34
*** Julien-zte has joined #openstack-infra16:34
*** Julien-zte has quit IRC16:34
fungi[security] gets us (hopefully) some crypto-minded people paying attention who might not otherwise16:34
*** Julien-zte has joined #openstack-infra16:35
*** Julien-zte has quit IRC16:35
*** Julien-zte has joined #openstack-infra16:36
leongfungi: thanks, will give that a try...16:36
*** dizquierdo has quit IRC16:36
*** nadya has joined #openstack-infra16:36
fungileong: https://etherpad.openstack.org/api/ gives me {"currentVersion":"1.2.12"} so i guess it works, but you probably want to poke around http://etherpad.org/doc/ for the correct release before assuming the api methods listed are correct16:37
fungileong: this is the version you want i guess http://etherpad.org/doc/v1.2.12/#index_http_api16:38
*** isaacb has quit IRC16:38
*** stevejims has quit IRC16:38
*** markus_z has quit IRC16:39
*** faizy__ has quit IRC16:40
*** soliosg has joined #openstack-infra16:41
openstackgerritManjeet Singh Bhatia proposed openstack-infra/reviewday master: Append a new section to neutron dashboard  https://review.openstack.org/43911416:41
*** mozart89|2 has joined #openstack-infra16:42
*** sekelso has quit IRC16:43
*** jpich has quit IRC16:43
sbezverkfungi: good morning, need a bit of help. would you have a couple of minutes to chat?16:43
*** jcoufal has quit IRC16:43
*** Apoorva has joined #openstack-infra16:43
*** ym200 has quit IRC16:43
*** ym200 has joined #openstack-infra16:44
fungisbezverk: i can make time briefly, but need to go run some errands in just a bit. is this related to your kernel module issues earlier?16:45
*** esikachev has joined #openstack-infra16:46
openstackgerritManjeet Singh Bhatia proposed openstack-infra/reviewday master: Append a new section to neutron dashboard  https://review.openstack.org/43911416:47
sbezverkfungi: no that thing is fixed :) I am debugging VM I bring up using ironic and kolla-kubernetes (at the gate) and it looks ok, but I need to get from that VM even BIOS messages, do you know a way to do it? capturing console to the file has only OS realted messages and not whole thing16:47
*** bhavik1 has quit IRC16:48
*** ym200 has quit IRC16:48
fungisbezverk: no clue. i think the virtual "bios" used for these may be all over the map given the diversity of hypervisors we have at various providers16:49
*** Daviey_ is now known as Daviey16:49
*** rkukura has joined #openstack-infra16:49
*** esikache1 has quit IRC16:49
*** jcoufal has joined #openstack-infra16:49
fungisbezverk: or are you talking about a vm nested in the vm where the job itself runs?16:49
*** ym200 has joined #openstack-infra16:49
sbezverkfungi: right it is nested VM I defined with virsh inside of gate VM16:50
sbezverkwhcih I use like "baremetal" node16:50
fungisbezverk: no clue, i don't generally work with openstack at that level but you might try asking in #openstack-qa since the devstack maintainers may have some ideas (or maybe #openstack-nova even?)16:50
clarkbya danpb is probably a good one to ask16:51
Davieyfungi / sbezverk: I think the issue is that currently only the tty is being logged to a txt file.  With the qemu console being on a VNC connection.  So either, someone from infra could screenshot that.. Or sbezverk could change it to dump the qemu console (pre-kernel) to txt file.16:51
fungiright, this is probably a hypervisor internals question really16:51
*** jkilpatr_ has joined #openstack-infra16:51
fungiDaviey: not sure what screen would screenshot the nested vnc console (if vnc console support is even being enabled in that job)16:52
Davieyfungi: Qemu + VNC is default, no?16:52
Davieyunless -nographic is being added to the cmd line?16:53
fungiDaviey: in kolla? no idea16:53
Davieyfungi: this is a qemu problem... not really kolla specific i don't think16:53
fungidoes that cause the vnc session of the nested vm to be automatically displayed to the outer vm's virtual console?16:53
*** jkilpatr has quit IRC16:54
sbezverkfungi: in my local test bed exact the same config works fine with the same level of nesting16:54
*** pblaho has quit IRC16:54
fungior, i guess it causes vmc to listen on a local network socket provided by the hypervisor, right?16:54
fungivnc16:54
sbezverkfungi: when I connect to vnc console I see everything16:54
Davieyright16:54
*** jordanP has joined #openstack-infra16:54
Davieyvncviewer 127.0.0.1:016:54
*** trown is now known as trown|lunch16:55
*** jaosorior has joined #openstack-infra16:55
fungigot it. so the job could call vmcviewer with some option to dump a video stream to disk?16:55
sbezverkDaviey: so if vnc can capture those messages I do not see why log cannot16:55
fungiwhy do i keep typing vmc?16:55
*** rkukura has quit IRC16:55
*** rkukura has joined #openstack-infra16:56
Davieysbezverk: I *think* you are only logging once the kernel has started.. You need qemu to output everything it sees (rather than use vnc), which is independent of the kernel tty16:56
fungivnc is pixel-oriented so can't just write text-mode i guess16:56
fungior can it?16:56
Davieyfungi: because you are thinking about the weather?16:56
fungiDaviey: probably16:56
Daviey(visual meteorological conditions)16:56
*** nadya has quit IRC16:56
fungiit's cool and sunny here16:57
Davieysbezverk: if -nographic is added to the qemu process, then what would have been put on VNC will be in ncurses instead16:57
*** coding_shardy is now known as shardy16:57
fungioh, that's handy16:57
sbezverkDaviey: since I use virsh to define how do I define this mode there? do you know from the top of your head?16:58
Davieyumm16:59
*** ociuhandu has quit IRC16:59
*** dizquierdo has joined #openstack-infra17:00
Davieysbezverk: you might be able to use <console type='file'> ... ?17:02
clarkbsbezverk: it might also help to know what information you are hoping to find? is there something specific that could potentially be retrieved some other way?17:02
sbezverkDaviey: that is what I have17:03
sbezverkclarkb: the same code works in my test bed and itdoes not at the gate.  baremetal VM by some reason does not callback to ironic-api17:04
sbezverkclarkb: since all possibilities were eliminated from code perspective I need to see the console output of that vm to see where it gets stuck17:05
sbezverkclarkb: in my test bed for debugging I was using vnc where I could every aspect of that vm17:05
jrollsbezverk: link to the gate logs?17:05
*** yee379 has quit IRC17:06
* jroll can point out some debugging help17:06
*** yee379 has joined #openstack-infra17:06
clarkbsbezverk: and haev you tried using reproduce.sh17:06
clarkb(basically same set of suggestions as yesterday for the kernel module, start with reproducing the gate job and see what happens)17:06
sbezverkjroll: here you go http://logs.openstack.org/50/441550/100/experimental/gate-kolla-kubernetes-deploy-centos-source-3-ironic-nv/830318b/console.html17:07
sbezverkclarkb: never use reproduce before..17:07
jrolloh my, this is much different than ironic setup :(17:07
sbezverkjroll: it is ironic running on kubernetes cluster17:08
clarkboh there is no reproduce.sh17:08
jrollsbezverk: I mean in terms of logs and such17:08
clarkbin that case I would suggest starting with a bit mroe of the prior art that has come before you17:08
*** vsaienko has quit IRC17:09
jrollsbezverk: I'll just leave you with this, then, https://docs.openstack.org/developer/ironic/deploy/troubleshooting.html#retrieving-logs-from-the-deploy-ramdisk17:09
clarkbsbezverk: basically its a script that devstack-gate writes out that reproduces a devstack-gate run17:09
jrollsbezverk: also <console type='file'> can be helpful17:09
clarkbsbezverk: it is very handy for reproducing failures17:09
*** vhosakot has joined #openstack-infra17:09
*** ociuhandu has joined #openstack-infra17:10
*** claudiub|2 has quit IRC17:10
*** nadya has joined #openstack-infra17:11
*** nadya has quit IRC17:11
sbezverkjroll: I use it and it dumps ONLY OS messages, so when VM is in BIOS stage or pxe booting nothing shows up17:12
*** baoli has joined #openstack-infra17:12
*** vsaienko has joined #openstack-infra17:12
sbezverkclarkb: running devstack on kube cluster is even more challenge than figuring out my vm issue, that would be plan Z ;)17:12
jrollsbezverk: that's odd, ironic jobs get everything, like http://logs.openstack.org/04/439404/5/check/gate-tempest-dsvm-ironic-ipa-partition-bios-pxe_ipmitool-tinyipa-ubuntu-xenial/c8a1837/logs/ironic-bm-logs/node-0_console_2017-03-21-06:01:05.txt.gz17:13
clarkbsbezverk: you don't need to run devstack at all17:13
clarkbsbezverk: devstack-gate is its own thing (and possibly poorly named) but it knows how to do things like reproduce itself, test itself, setup all the git repos properly, etc17:13
clarkbits a bootstrapping framework, no devstack required17:14
sbezverkjroll: that looks exactly what I need17:14
sbezverkjroll: how do you collect it or define your vm?17:14
*** ociuhandu has quit IRC17:14
jrollsbezverk: https://github.com/openstack/ironic/tree/master/devstack/tools/ironic17:14
sbezverkclarkb: I see what you mean, sorry I misunderstood you17:14
jrollsbezverk: look at scripts/configure-vm.py and templates/vm.xml17:14
*** Hal has quit IRC17:15
*** Qiming has quit IRC17:15
sbezverkjroll" this looks interesting <bios useserial='yes'/>17:17
*** esikachev has quit IRC17:17
*** esikache1 has joined #openstack-infra17:17
sbezverkjroll: but I do not see how console is defined, it seems rendered17:17
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Remove ZooKeeperConnectionConfig class  https://review.openstack.org/44768317:18
jrollsbezverk: https://github.com/openstack/ironic/blob/master/devstack/lib/ironic#L1365-L137217:19
jrollsbezverk: and https://github.com/openstack/ironic/blob/master/devstack/tools/ironic/scripts/create-node.sh17:19
sbezverkjroll: nvm I found it, it is almost like I have with exception bios use serial17:19
jrollit's complex but that's about everything17:19
* jroll brb17:19
*** Qiming has joined #openstack-infra17:19
*** baoli has quit IRC17:19
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704717:20
*** tinwood_ is now known as tinwood17:22
*** vsaienko has quit IRC17:22
*** vsaienko has joined #openstack-infra17:24
*** pblaho has joined #openstack-infra17:24
*** baoli has joined #openstack-infra17:24
*** ociuhandu has joined #openstack-infra17:25
*** pbourke_ is now known as pbourke17:25
*** hamzy has joined #openstack-infra17:26
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Remove ZooKeeperConnectionConfig class  https://review.openstack.org/44768317:27
*** nadya has joined #openstack-infra17:27
*** spzala has joined #openstack-infra17:27
*** mhickey has quit IRC17:27
*** flwang has quit IRC17:27
*** hashar has quit IRC17:28
*** witek has quit IRC17:31
sbezverkjroll: thanks very much, that did the trick!!17:31
*** chuckC_ has quit IRC17:32
*** spzala has quit IRC17:32
*** rfolco has quit IRC17:32
jrollsbezverk: awesome :)17:33
*** annegentle has quit IRC17:33
hamzyheya I have a question about https://review.openstack.org/#/c/427255/ I keep running into timeout problems. For example, http://logs.openstack.org/55/427255/13/check/gate-openstack-ansible-openstack-ansible-aio-ubuntu-xenial/077687b/console.html "Request Time-out: Your browser didn't send a complete request in time." and http://logs.openstack.org/55/427255/13/check/gate-openstack-ansible-openstack-ansible-aio-ubuntu-xenial/d818461/console.html "WARNIN17:35
hamzyG  [tempest.lib.common.ssh] Failed to establish authenticated ssh connection to cirros@172.29.249.122"17:35
openstackgerritMerged openstack-infra/devstack-gate master: Set HOST_IP for subnode first  https://review.openstack.org/44807817:35
*** rwsu has joined #openstack-infra17:36
*** manjeets has joined #openstack-infra17:37
*** david-lyle_ has joined #openstack-infra17:37
manjeetshttps://review.openstack.org/#/c/400227/17:37
fungimordred: looks like we may have yet another gerrit acl manage-projects is failing to apply... http://lists.openstack.org/pipermail/openstack-dev/2017-March/114407.html (the change merged yesterday but still doesn't seem to match what's live in gerrit)17:37
manjeetsis waiting reviews and without it pep8 is broken17:38
* fungi heads out to run errands, back in time to run infra meeting at 19:00z17:38
*** david-lyle has quit IRC17:38
*** spzala has joined #openstack-infra17:38
*** ssbarnea has quit IRC17:38
*** lucasagomes is now known as lucas-afk17:38
pabelangerapparently forumtopics.o.o is a thing17:39
mordredfungi: BLERG17:39
pabelangerbut not in system-config17:39
*** flwang has joined #openstack-infra17:39
pabelanger:(17:39
fungipabelanger: it's a foundation instance, not in the openstackci tenant17:39
fungipabelanger: is it misbehaving in a way that needs to be silenced? i can look into it when i get back17:40
pabelangerfungi: no, was just curious what is was.17:40
*** yamahata has quit IRC17:40
*** spzala has quit IRC17:41
*** spzala has joined #openstack-infra17:41
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WIP: create a README_LOGS to document log files  https://review.openstack.org/44765317:42
openstackgerritMerged openstack-infra/reviewday master: setup.py has a tox error, making reviewday fail gating  https://review.openstack.org/40022717:43
*** jtomasek has quit IRC17:43
jordanPmordred, https://github.com/openstack-infra/shade/blob/master/shade/openstackcloud.py#L5648 means that when we call delete_server(..., wait=True) we will hit the nova api as hard as possible17:46
*** rbrndt has quit IRC17:46
*** florianf has quit IRC17:46
jordanPbecause by default self._SERVER_AGE = 017:47
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Handle the 'all' group in callback.  https://review.openstack.org/44674917:47
*** trown|lunch is now known as trown17:47
*** tonytan4ever has quit IRC17:47
mordredjordanP: yes - you are quite right17:48
*** beagles is now known as beagles_lunch17:48
*** florianf has joined #openstack-infra17:48
*** tonytan4ever has joined #openstack-infra17:48
mordredjordanP: there are several bad edgecases in the overlap of our cache vs. no-cache codepaths17:48
mordredjordanP: perhaps for now we should have that loop do wait=self._SERVER_AGE or 2 ... or something like that to keep deletes from ddos-ing?17:49
jordanPmordred, what's the best patch ? set wait=1 instead17:49
jordanPok, I'll do that17:49
jordanP2 seems very good17:49
mordredjordanP: a thing to double-check for is that it doesn't cause the unittests to balloon in time- if it does, we may also want to set SERVER_AGE in them if we aren't already17:50
mordredthanks!17:50
clarkbthe task manager delay will still be in effect if used though17:50
*** derekh has quit IRC17:50
*** leong has quit IRC17:51
jordanPmordred, time.sleep should be mocked in unittests, it's a good rule for unit testing17:51
mordredjordanP: ++17:51
jordanPbut yeah, I'll look at it17:51
clarkbso in theory there is a queueing mechanism set to request lower than dos threshold17:51
mordredclarkb: yah. if you're using task_manager you're likely a much more advanced user though who is also setting cache times17:51
mordredfor folks using shade on command line or in ansible - this could be unfriendly :)17:51
*** ralonsoh has quit IRC17:52
openstackgerritManjeet Singh Bhatia proposed openstack-infra/project-config master: Forcing tempest concurrency  https://review.openstack.org/44762817:53
*** adreznec has quit IRC17:55
*** annegentle has joined #openstack-infra17:56
*** florianf has quit IRC17:56
bretonttx: fungi: thanks for fixing the ML17:56
*** adreznec has joined #openstack-infra17:57
*** jaosorior has quit IRC17:57
clarkbttx: fungi EmilienM so if I put a forum session idea on the etherpad that was sent out I now need to transfer it to forumtopics.o.o?17:58
openstackgerritJordan Pittier proposed openstack-infra/shade master: delete_server: make sure we sleep a bit when waiting for srv deletion  https://review.openstack.org/44821517:58
*** dtantsur is now known as dtantsur|afk17:58
*** mtanino has quit IRC17:58
*** esikache1 has quit IRC17:58
EmilienMclarkb: yes17:59
*** rwsu has quit IRC17:59
*** spzala has quit IRC17:59
clarkbwhy'd we send out the etherpad with a call for suggestions then... /me goes to dig up etherpad and move bits around18:00
*** rwsu has joined #openstack-infra18:00
*** flepied has quit IRC18:00
*** spzala has joined #openstack-infra18:01
EmilienMclarkb: AFIK the f.o.o tool wasn't ready until yesterday or so18:03
*** spzala has quit IRC18:05
*** esikache1 has joined #openstack-infra18:05
*** spzala has joined #openstack-infra18:07
*** xyang1 has joined #openstack-infra18:08
openstackgerritManjeet Singh Bhatia proposed openstack-infra/devstack-gate master: Do not merge experimental  https://review.openstack.org/44821818:09
*** electrofelix has quit IRC18:09
*** esikache1 has quit IRC18:10
*** dprince has quit IRC18:10
*** spzala has quit IRC18:10
*** spzala has joined #openstack-infra18:10
*** sekelso has joined #openstack-infra18:11
mordredjordanP: I may be a dork - my first thought on reading that commit message, even though we'd just spoken, was "why are we deleting SRV records" :)18:11
*** ym200 has quit IRC18:11
*** mtanino has joined #openstack-infra18:11
jordanPmordred, yeah I was afraid of the max commit title length18:11
*** ym200 has joined #openstack-infra18:12
*** skelso has joined #openstack-infra18:12
openstackgerritJordan Pittier proposed openstack-infra/shade master: delete_server: make sure we sleep a bit when waiting for server deletion  https://review.openstack.org/44821518:12
*** ociuhandu has quit IRC18:14
*** skelso has quit IRC18:14
*** skelso has joined #openstack-infra18:14
*** sekelso has quit IRC18:15
openstackgerritJordan Pittier proposed openstack-infra/shade master: wait_for_server: ensure we sleep a bit when waiting for server  https://review.openstack.org/44822018:15
jordanPso there was another occurence of the very same issue. But I am not going to fix it everywhere, only where I need it for now18:16
jordanPstill working on my patch to shade, the one with a flaky functional test18:16
*** skelso has quit IRC18:16
*** shardy has quit IRC18:18
*** bh526r has quit IRC18:18
*** pvaneck has joined #openstack-infra18:20
*** rossella_s has quit IRC18:23
*** esikache1 has joined #openstack-infra18:24
*** weshay_sick is now known as weshay_brb18:25
*** yamahata has joined #openstack-infra18:27
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create extra-test-setup role  https://review.openstack.org/44804218:27
*** spzala has quit IRC18:27
*** esikache1 has quit IRC18:28
clarkbmnaser: pabelanger beacuse the dhcp vs not dhcp thing keeps coming up, did we determine what the issue was with eth1 config in rax?18:29
*** yamahata has quit IRC18:31
*** tesseract has quit IRC18:31
*** yamahata has joined #openstack-infra18:31
*** esikache1 has joined #openstack-infra18:31
pabelangerclarkb: so, talked to mnaser on the weekend, I think we can add logic to systemd service for glance to restart the service on failure18:32
pabelangeras it looks like chkconfig is bombing due to race18:32
pabelangernot glean18:32
*** stevejims has joined #openstack-infra18:33
*** ssbarnea has joined #openstack-infra18:33
clarkbso glean's units are running before chkconfig can properly configure them?18:33
*** flepied has joined #openstack-infra18:33
*** beagles_lunch is now known as beagles18:33
*** eroux has quit IRC18:34
*** spzala has joined #openstack-infra18:34
pabelangerclarkb: something something systemd?  If you look at a log, systemd will redirect to chkconfig because networking.service is not there?  And since both eth1 and eth0 run at the same time, eth1 looks to fail to symlink something18:34
pabelangerso, we could serialize systemd some how, not sure18:34
pabelangeror restart service on failure18:34
clarkbI see, I'd likely have to draw a graph of execution order to understand it myself18:35
*** baoli has quit IRC18:35
*** eroux has joined #openstack-infra18:36
*** baoli has joined #openstack-infra18:36
clarkbsystemd-analyze might help me there18:36
clarkbI may try generating a graph on a rax host later today18:36
*** rbrndt has joined #openstack-infra18:37
*** Sukhdev has joined #openstack-infra18:38
*** esikache1 has quit IRC18:38
*** spzala has quit IRC18:38
*** pcaruana has quit IRC18:39
*** ym200 has quit IRC18:40
*** tosky has quit IRC18:40
*** zara_the_lemur__ has joined #openstack-infra18:41
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create extra-test-setup role  https://review.openstack.org/44804218:41
*** weshay_brb is now known as weshay18:42
*** rwsu has quit IRC18:42
*** dhajare has joined #openstack-infra18:43
*** dprince has joined #openstack-infra18:43
*** david-lyle_ has quit IRC18:46
*** armax has quit IRC18:49
*** rkukura has quit IRC18:49
*** rockyg has joined #openstack-infra18:50
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Add dracut-regenerate elements  https://review.openstack.org/44704718:55
jheskethMorning18:58
*** mat128 has quit IRC18:59
*** ociuhandu has joined #openstack-infra18:59
*** rbrndt has quit IRC18:59
AJaegermorning, jhesketh19:00
jhesketho/19:00
*** nadya has quit IRC19:00
openstackgerritJordan Pittier proposed openstack-infra/shade master: attach_volume should always return a vol attachment.  https://review.openstack.org/43496419:01
*** ym200 has joined #openstack-infra19:01
fungiit's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour19:02
*** annegentle has quit IRC19:02
*** esikache1 has joined #openstack-infra19:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Fix hostname issue with nodepool integration job  https://review.openstack.org/44823919:08
*** dharinic is now known as dharinic|lunch19:09
*** dhajare has quit IRC19:09
*** rkukura has joined #openstack-infra19:11
*** rwsu has joined #openstack-infra19:12
*** bswartz has quit IRC19:13
*** chlong has joined #openstack-infra19:13
*** spzala has joined #openstack-infra19:13
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Fix hostname issue with nodepool integration job  https://review.openstack.org/44823919:14
*** spzala has quit IRC19:15
*** spzala has joined #openstack-infra19:15
*** beslemon has joined #openstack-infra19:15
*** rwsu has quit IRC19:17
*** tosky has joined #openstack-infra19:18
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773919:20
*** annegentle has joined #openstack-infra19:22
*** gyee has joined #openstack-infra19:26
*** jordanP has quit IRC19:28
*** rfolco has joined #openstack-infra19:29
*** dizquierdo has quit IRC19:29
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809919:31
*** MasterOfBugs has joined #openstack-infra19:36
*** ym200 has quit IRC19:36
*** ym200 has joined #openstack-infra19:38
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773919:39
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809919:39
*** stevejims has quit IRC19:41
*** amoralej is now known as amoralej|off19:44
*** rhallisey has quit IRC19:45
*** dharinic|lunch is now known as dharinic19:46
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs  https://review.openstack.org/44562519:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Better merge message for GitHub pull reqeusts  https://review.openstack.org/44564419:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Set filter according to PR/Change in URL  https://review.openstack.org/44678219:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github  https://review.openstack.org/44529219:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support  https://review.openstack.org/44611319:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit  https://review.openstack.org/44615019:48
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Query changes only via relevant sources  https://review.openstack.org/44825719:48
*** rhallisey has joined #openstack-infra19:48
*** MasterOfBugs has quit IRC19:52
*** rockyg has quit IRC19:52
*** hashar has joined #openstack-infra19:53
*** esikache1 has quit IRC19:54
*** spzala has quit IRC19:54
*** rockyg has joined #openstack-infra19:55
*** mat128 has joined #openstack-infra19:57
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773919:59
*** claudiub|2 has joined #openstack-infra19:59
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809920:00
*** dave-mccowan has quit IRC20:02
*** matrohon has joined #openstack-infra20:02
*** armax has joined #openstack-infra20:02
*** rbrndt has joined #openstack-infra20:03
*** rbrndt has quit IRC20:03
*** e0ne has joined #openstack-infra20:04
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create zuul_workspace_root job variable  https://review.openstack.org/44144120:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Organize playbooks folder  https://review.openstack.org/44154720:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info to bootstrap role  https://review.openstack.org/44161720:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add ansible-lint to tox (pep8)  https://review.openstack.org/43827620:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add revoke-sudo role and update tox jobs  https://review.openstack.org/44146720:06
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create tox-tarball job  https://review.openstack.org/44160920:07
*** zara_the_lemur__ has left #openstack-infra20:07
*** MasterOfBugs has joined #openstack-infra20:11
*** sshnaidm is now known as sshnaidm|afk20:16
*** pbourke has quit IRC20:16
*** vsaienko has quit IRC20:17
bretonwell, i've just got fifth mail "Thank you for registering for the OpenStack Summit" :)20:17
*** priteau has joined #openstack-infra20:18
bretonand it says that i don't have "have an OpenStack Community Account or OpenStack Foundation "20:19
bretonMembership20:19
bretonwhich is true20:19
bretonbut 5 times20:19
*** chlong has quit IRC20:21
pabelangerbreton: as did I20:21
*** rhallisey has quit IRC20:22
clarkbbreton: did you just register?20:23
clarkbpabelanger: ^ you too I guess20:23
pabelangerclarkb: no, I registered at PTG20:23
clarkbpabelanger: but emails just came in today?20:23
* clarkb trying to collect data so can pass it along20:23
pabelangerclarkb: right, 53 mins go20:23
clarkbpabelanger: and you got 5 of them too?20:24
pabelangerclarkb: just 4 currently20:24
pabelangerlast was 8min ago20:24
clarkbcool, will pass it along and see if we can sort it out20:24
clarkbwill let you know if I leanr anything20:24
*** clenimar has joined #openstack-infra20:26
bretonclarkb: same as pabelanger, registered at the PTG20:26
*** jkilpatr_ has quit IRC20:28
clarkbbreton: pabelanger I am passing info along, they aren't sure what is causing it just yet but are looking into it20:30
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773920:30
priteauHello. I am not sure who to ping, I just noticed https://groups.openstack.org is throwing a database access error20:30
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809920:31
*** esikache1 has joined #openstack-infra20:31
pabelangerchecking groups.o.o20:32
pabelangernot sure if that is a local DB or trove20:32
*** dave-mccowan has joined #openstack-infra20:32
clarkbpriteau: this is probably a good place20:32
clarkbfungi: ^ aware of any changes to groups?20:33
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: WIP: Add lvm management to diskimage-builder  https://review.openstack.org/44440320:34
fungino, though it may be a trove maintenance in rackspace? i'd check the openstackci tenant tickets if it weren't for the tc meeting20:34
fungigiven the wiki had trouble earlier today20:34
pabelangerya, looks like groups.o.o is using trove20:34
clarkbI am able to connect to the db instance from the groups server20:35
clarkband the groups db is there20:35
pabelangerclarkb: and just got 5th email20:37
clarkbwondering if apache/php has cached some bad ip addr info or something20:37
*** kgiusti has left #openstack-infra20:37
clarkband they moved the trove server?20:37
pabelangerdidn't yolanda restart apache this morning20:37
pabelangerwhich fixed up the wiki20:37
clarkbpabelanger: breton looks like they just spotted the issue with the emails and it should be fixed soon20:37
clarkbpabelanger: fungi ya based on my direct testing of mysql conectivity I think that may be our next step here20:38
clarkbsince it appears to all work and be happy if I connect to it20:38
pabelangerseems reasonable20:38
clarkbfungi: ^ any last minute objection before I do that?20:38
fungiclarkb: no objection, do the thing20:38
*** esikache1 has quit IRC20:38
clarkbits back20:39
clarkbpriteau: thank you for letting us know, can you check that the service is doing what you were hoping it would do now?20:39
priteauThank you clarkb & fungi!20:39
priteauIt looks good now here20:39
openstackgerritK Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks  https://review.openstack.org/44627520:41
*** gildub has joined #openstack-infra20:41
*** esikache1 has joined #openstack-infra20:42
openstackgerritMikhail S Medvedev proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773920:43
*** ym200 has quit IRC20:43
openstackgerritMikhail S Medvedev proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809920:44
*** jkilpatr has joined #openstack-infra20:44
*** ym200 has joined #openstack-infra20:44
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create zuul_workspace_root job variable  https://review.openstack.org/44144120:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Organize playbooks folder  https://review.openstack.org/44154720:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info to bootstrap role  https://review.openstack.org/44161720:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add ansible-lint to tox (pep8)  https://review.openstack.org/43827620:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Add revoke-sudo role and update tox jobs  https://review.openstack.org/44146720:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create tox-tarball job  https://review.openstack.org/44160920:45
openstackgerritPaul Belanger proposed openstack-infra/zuul feature/zuulv3: Create extra-test-setup role  https://review.openstack.org/44804220:45
*** esikache1 has quit IRC20:47
*** chlong has joined #openstack-infra20:48
pabelangerclarkb: fungi: neat, cacti.o.o has the same database issue is seems20:48
clarkbnice20:48
pabelangergoing to restart apache now20:49
clarkbkk20:49
pabelangerrestarted20:49
pabelangerand back20:49
fungipabelanger: i wonder how gerrit escaped this20:49
fungicacti has a trove db?20:49
clarkbfungi: I think java is better at noticing and reconnecting than php?20:49
pabelangerfungi: apparently it does, but it also has mysql running locally20:50
pabelangernot sure why that is20:50
pabelangerYa, setup for trove in /etc/cacti20:51
openstackgerritManjeet Singh Bhatia proposed openstack-infra/reviewday master: Append a new section to neutron dashboard  https://review.openstack.org/43911420:51
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: WIP: create a README_LOGS to document log files  https://review.openstack.org/44765320:51
*** e0ne has quit IRC20:53
*** jerryz has joined #openstack-infra20:53
*** trown is now known as trown|outtypewww20:54
*** priteau_ has joined #openstack-infra20:55
*** priteau has quit IRC20:59
*** beslemon_ has joined #openstack-infra21:01
*** beslemon has quit IRC21:02
*** eroux has quit IRC21:02
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773921:05
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809921:06
clarkbpabelanger: fungi guessing we half did a migration to trove and never removed the old local db?21:07
pabelangermaybe?21:07
pabelangerwill have to go back and check system-config21:07
openstackgerritEmilien Macchi proposed openstack-infra/project-config master: puppet/tripleo: stop testing Puppet 3 in Pike and beyond  https://review.openstack.org/44828021:08
*** Goneri has quit IRC21:09
*** aysyd has quit IRC21:09
*** Apoorva has quit IRC21:10
fungiclarkb: possible, yes21:10
*** dimtruck is now known as zz_dimtruck21:10
*** Apoorva has joined #openstack-infra21:10
*** esikache1 has joined #openstack-infra21:11
zigopabelanger: Hi there!21:12
zigopabelanger: I tried to hardcode /etc/puppet/modules in install_modules.sh of system-config, but then the script doesn't even attempt to download puppet modules.21:12
zigoSo I'm not sure what to do now... :/21:12
zigoAny iea?21:12
zigoidea21:12
pabelangerzigo: log file?21:13
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773921:13
pabelangerzigo: it will git clone the modules21:14
zigopabelanger: http://sid.gplhost.com/os/build-image.log.txt21:14
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809921:14
zigopabelanger: That's with the hardcoding of things...21:14
pabelangerzigo: I see them getting cloned into /usr/share/puppet/modules still21:15
*** cody-somerville has joined #openstack-infra21:15
*** esikache1 has quit IRC21:15
*** ldnunes has quit IRC21:16
zigopabelanger: I don't...21:16
pabelangerzigo: grep install_modules.sh21:16
zigoIt used to be "Cloning into", now it's printing that at all.21:16
pabelangerCloning into '/usr/share/puppet/modules/mysql_backup'...21:17
*** ym200 has quit IRC21:17
pabelangeram I looking at the wrong log?21:17
*** ym200 has joined #openstack-infra21:17
zigopabelanger: I can't see "Cloning into '/usr/share/puppet/modules/mysql_backup'"21:18
zigoDid you refresh your browser content ?21:18
pabelangeryup21:19
pabelangerdownloading file locally too21:19
zigopabelanger: It seems to me you got an old version or something. Grepping -i for "cloning" doesn't show that line for me.21:20
*** ihrachys has quit IRC21:21
pabelangerzigo: set -xe for install_roles.sh and see what is going on21:21
*** tonytan4ever has quit IRC21:21
pabelangerpossible issue with your new code21:21
zigopabelanger: Ok, will do, and will revert my patch of install_modules.sh21:22
*** tonytan4ever has joined #openstack-infra21:22
zigoOr shouldn't I ?21:22
pabelangerzigo: give me a few days, I'll try and spin up a sid locally21:23
*** david-lyle has joined #openstack-infra21:23
*** esikache1 has joined #openstack-infra21:24
zigopabelanger: Where is install_roles.sh ? I can't find it neither in project-config or system-config ...21:24
pabelangerzigo: oh, sorry. install_modules.sh21:24
pabelangerin system-config21:24
pabelangerzigo: the fix is to update /etc/puppet/puppet.conf and set the proper basemodule path21:24
pabelangeryou can likely use sed -i, like we do for templatedir21:25
pabelangerbut only do it for sid21:25
*** gildub has quit IRC21:25
zigopabelanger: I don't see any "basemodule path" in the default puppet package's /etc/puppet/puppet.conf21:25
pabelangerzigo: what do you have?21:26
zigoLet me purge and reinstall the puppet package, just to make sure.21:26
zigopabelanger: http://paste.openstack.org/show/603688/21:27
pabelangerzigo: have to run, but will look and comment21:27
zigoThanks.21:28
*** ym200 has quit IRC21:28
*** gildub has joined #openstack-infra21:32
*** zz_dimtruck is now known as dimtruck21:34
*** sree has joined #openstack-infra21:34
*** annegentle has quit IRC21:34
*** Jeffrey4l_ has quit IRC21:35
openstackgerritLuz Cazares proposed openstack-infra/project-config master: Add refstack py35 functional test gate job  https://review.openstack.org/44829221:36
*** annegentle has joined #openstack-infra21:37
*** neiljerram has quit IRC21:37
*** alex_xu has quit IRC21:37
*** ijw has joined #openstack-infra21:38
*** alex_xu has joined #openstack-infra21:38
*** gildub has quit IRC21:38
*** sree has quit IRC21:38
*** esikache1 has quit IRC21:41
*** tonytan4ever has quit IRC21:42
*** burgerk has quit IRC21:43
*** esikache1 has joined #openstack-infra21:43
openstackgerritThomas Goirand proposed openstack-infra/system-config master: Do not install from puppetlab for deb  https://review.openstack.org/44581321:46
*** Jeffrey4l_ has joined #openstack-infra21:48
*** gouthamr has quit IRC21:48
*** xarses has quit IRC21:49
*** xarses has joined #openstack-infra21:49
*** esikache1 has quit IRC21:51
*** marst has quit IRC21:51
*** dprince has quit IRC21:54
*** jcoufal has quit IRC21:55
*** fguillot has quit IRC21:56
*** ijw has quit IRC21:58
*** ijw has joined #openstack-infra21:59
*** matrohon has quit IRC22:00
*** mat128 has quit IRC22:00
*** annegentle has quit IRC22:00
*** ssbarnea has quit IRC22:03
*** mdrabe has quit IRC22:04
*** gordc has quit IRC22:06
*** ijw has quit IRC22:08
*** Qiming has quit IRC22:09
*** vhosakot has quit IRC22:09
*** Qiming has joined #openstack-infra22:12
*** beslemon_ has quit IRC22:13
*** EricGonczer_ has quit IRC22:16
*** baoli has quit IRC22:18
*** jamesdenton has quit IRC22:19
*** jaypipes has quit IRC22:22
*** chlong has quit IRC22:27
*** dimtruck is now known as zz_dimtruck22:29
*** xyang1 has quit IRC22:31
openstackgerritBen Kero proposed openstack-infra/tripleo-ci master: Split multinode_setup.sh out into its own script for use with quickstart  https://review.openstack.org/44830222:36
*** adrian_otto has quit IRC22:44
*** maratoid_ has joined #openstack-infra22:46
mwhahahadon't suppose anyone is around to merge https://review.openstack.org/#/c/447602/ as it's kinda blocking our newton gates22:51
*** adriant has joined #openstack-infra22:52
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773922:53
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809922:53
clarkbmwhahaha: done (-1'd for regex issue)22:53
*** eharney has quit IRC22:54
* mwhahaha sighs ಠ_ಠ22:54
openstackgerritAlex Schultz proposed openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760222:55
*** david-lyle has quit IRC22:57
*** _d34dh0r53_ has joined #openstack-infra23:00
*** _d34dh0r53_ has quit IRC23:00
*** Apoorva_ has joined #openstack-infra23:01
*** Apoorva has quit IRC23:03
*** cloudkiller has joined #openstack-infra23:04
*** _d34dh0r53_ has joined #openstack-infra23:04
*** eglute_s has joined #openstack-infra23:04
*** gildub has joined #openstack-infra23:06
*** harlowja has quit IRC23:06
openstackgerritJamie Lennox proposed openstack-infra/zuul feature/zuulv3: Re-add the ability to set username on zuul-executor  https://review.openstack.org/44630823:07
*** eglute_s has quit IRC23:07
*** cloudkiller has quit IRC23:07
*** _d34dh0r53_ has quit IRC23:07
*** mozart89|2 has quit IRC23:12
*** andreww has quit IRC23:13
*** xarses has quit IRC23:13
*** priteau_ has quit IRC23:15
*** priteau has joined #openstack-infra23:15
jeblairomg i hope we get to zuulv3 before that wraps around to 5 lines.23:16
openstackgerritMerged openstack-infra/project-config master: tripleo: exclude reno jobs before Ocata  https://review.openstack.org/44760223:20
clarkbmwhahaha: ^23:21
mwhahahaclarkb: thanks23:21
*** rwsu has joined #openstack-infra23:22
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773923:31
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809923:31
*** cloudkiller has joined #openstack-infra23:34
*** eglute_s has joined #openstack-infra23:34
*** priteau has quit IRC23:34
*** _d34dh0r53_ has joined #openstack-infra23:34
*** jamielennox is now known as jamielennox|away23:35
*** mtanino has quit IRC23:38
*** jamielennox|away is now known as jamielennox23:39
*** priteau has joined #openstack-infra23:41
*** priteau has quit IRC23:46
*** _d34dh0r53_ has quit IRC23:46
*** eglute_s has quit IRC23:46
*** cloudkiller has quit IRC23:46
*** cloudkiller has joined #openstack-infra23:47
*** _d34dh0r53_ has joined #openstack-infra23:47
*** eglute_s has joined #openstack-infra23:47
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Create PReP boot partition for PPC  https://review.openstack.org/44773923:49
*** irtermite has quit IRC23:50
*** eglute has quit IRC23:50
*** cloudnull has quit IRC23:50
*** d34dh0r53 has quit IRC23:50
*** cloudkiller is now known as cloudnull23:50
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] Test boot partition code  https://review.openstack.org/44809923:50
*** notmyname has quit IRC23:50
*** bswartz has joined #openstack-infra23:52
*** notmyname has joined #openstack-infra23:54
*** askb has joined #openstack-infra23:55
*** rockyg has quit IRC23:55
*** hashar has quit IRC23:56
openstackgerritMorgan Fainberg proposed openstack-infra/shade master: Convert test_role_assignments to requests mock  https://review.openstack.org/44831323:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!