Thursday, 2018-07-26

*** bobh has joined #openstack-infra00:03
*** rcernin has joined #openstack-infra00:03
*** bobh has quit IRC00:08
*** mist27 has joined #openstack-infra00:08
mist27Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/00:08
*** mist27 has quit IRC00:08
pabelangerclarkb: I'm around now00:10
clarkbpabelanger: do we want to go ahead with the restarts without corvus? I'm reasonably confident in the changes but have not reviewed all of them since the last restart00:10
pabelangerclarkb: which git sha are we running now?00:11
*** shrircha has quit IRC00:11
pabelangerlooking at git log now00:11
clarkbI'm not sure of the exact sha00:11
*** shrircha has joined #openstack-infra00:12
clarkbbut it included tobiash change to configs00:12
pabelangerclarkb: I propose we land https://review.openstack.org/583639/ then wait until morning, so we don't ask ourself it again :)00:12
clarkbI like this idea00:13
*** bobh has joined #openstack-infra00:14
pabelangerclarkb: I wouldn't mind maybe upgrading the kernel on ze01.o.o, and see if we see the same improvement that we see on ze11.o.o00:15
*** zhangfei has joined #openstack-infra00:15
pabelangerbut that means we a reboot of zuul00:15
pabelangerokay, I'll wait until we tag the next release then schedule a reboot00:16
*** wolverineav has joined #openstack-infra00:17
clarkbwhy do we have to reboot zuul?00:17
clarkbwould just be the executor00:17
pabelangeryah, sorry zuul-executor00:17
*** bobh has quit IRC00:18
*** zhangfei has quit IRC00:19
*** wolverineav has quit IRC00:21
*** bobh has joined #openstack-infra00:23
clarkbpabelanger: mordred I left a comment on 583639 but approvd it as its functional as is and I think a good improvement for this sort of thing00:23
*** theaetetus16 has joined #openstack-infra00:23
theaetetus16Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/00:24
*** theaetetus16 has quit IRC00:24
*** dingyichen has joined #openstack-infra00:24
pabelangerclarkb: re nodepool.yaml I'll check, but thought now builder always reloaded clouds.yaml00:25
*** austin-sun has quit IRC00:25
*** dtruong2 has joined #openstack-infra00:25
clarkbpabelanger: I ran into it not doing that recently and have to restart launchers (that is what precipitated the failed upgrade with the zk schema change recently)00:25
pabelangerIsee00:26
*** bobh has quit IRC00:27
*** jamesmcarthur has joined #openstack-infra00:28
*** owalsh_ is now known as owalsh00:32
*** akhilaki has quit IRC00:33
*** bobh has joined #openstack-infra00:34
*** dtruong2 has quit IRC00:36
*** rpioso is now known as rpioso|afk00:37
*** hamzy has joined #openstack-infra00:38
*** bobh has quit IRC00:39
*** yamamoto has quit IRC00:39
*** bobh has joined #openstack-infra00:45
*** jamesmcarthur has quit IRC00:46
*** bobh has quit IRC00:50
*** jamesmcarthur has joined #openstack-infra00:52
*** Guest87659 has joined #openstack-infra00:53
Guest87659Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/00:53
*** Guest87659 has quit IRC00:53
*** jamesmcarthur has quit IRC00:54
*** bdodd has joined #openstack-infra00:56
*** jamesmcarthur has joined #openstack-infra00:56
*** bobh has joined #openstack-infra00:58
*** iDanoo16 has joined #openstack-infra00:59
iDanoo16Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/00:59
*** iDanoo16 has quit IRC00:59
*** bobh has quit IRC01:02
*** yamamoto has joined #openstack-infra01:03
*** gyee has quit IRC01:08
*** bobh has joined #openstack-infra01:09
*** ddurst has quit IRC01:09
*** ddurst has joined #openstack-infra01:09
*** jamesmcarthur has quit IRC01:10
*** jamesmcarthur has joined #openstack-infra01:10
*** annp has quit IRC01:13
*** bobh has quit IRC01:13
*** annp has joined #openstack-infra01:14
*** bobh has joined #openstack-infra01:15
*** hongbin has joined #openstack-infra01:16
*** edmondsw has joined #openstack-infra01:17
*** jamesmcarthur has quit IRC01:18
*** bobh has quit IRC01:20
*** dtruong_ has quit IRC01:20
*** yamahata has quit IRC01:20
*** edmondsw has quit IRC01:21
*** yamamoto has quit IRC01:22
*** jamesmcarthur has joined #openstack-infra01:22
*** bobh has joined #openstack-infra01:26
*** bobh has quit IRC01:31
*** wolverineav has joined #openstack-infra01:31
*** wolverineav has quit IRC01:35
openstackgerritAustin Sun proposed openstack-infra/project-config master: Add Starlingx (openstack/stx-*) projects to #starlingx IRC channel  https://review.openstack.org/58591901:35
*** bobh has joined #openstack-infra01:38
*** bobh has quit IRC01:42
*** bobh has joined #openstack-infra01:43
*** markvoelker has joined #openstack-infra01:43
*** felipemonteiro has joined #openstack-infra01:43
*** jamesmcarthur has quit IRC01:44
*** markvoelker_ has quit IRC01:44
*** zhangfei has joined #openstack-infra01:45
*** markvoelker_ has joined #openstack-infra01:46
*** bobh has quit IRC01:47
*** markvoelker has quit IRC01:48
*** jamesmcarthur has joined #openstack-infra01:48
*** yamamoto has joined #openstack-infra01:50
*** jamesmcarthur has quit IRC01:53
*** bobh has joined #openstack-infra01:54
*** zhangfei has quit IRC01:57
openstackgerritAustin Sun proposed openstack-infra/project-config master: Add gerritbot for StarlingX (openstack/stx-*) projects to #starlingx IRC channel  https://review.openstack.org/58591901:58
*** bobh has quit IRC01:58
*** graphene has quit IRC02:01
*** graphene has joined #openstack-infra02:02
*** bobh has joined #openstack-infra02:02
*** wolverineav has joined #openstack-infra02:06
*** zhangfei has joined #openstack-infra02:10
*** wolverineav has quit IRC02:11
*** graphene has quit IRC02:12
*** graphene has joined #openstack-infra02:13
*** armaan has quit IRC02:13
openstackgerritMerged openstack-infra/system-config master: Add #starlingx channel to meetbot.  https://review.openstack.org/58189702:16
*** yamamoto has quit IRC02:26
*** gongysh has joined #openstack-infra02:27
*** jelly15 has joined #openstack-infra02:33
jelly15Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/02:33
jelly15or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/02:33
jelly15Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate02:33
*** jelly15 has quit IRC02:33
*** pixdamix18 has joined #openstack-infra02:33
pixdamix18Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/02:33
pixdamix18or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/02:33
pixdamix18Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate02:33
*** pixdamix18 has quit IRC02:33
*** felipemonteiro has quit IRC02:38
*** yamamoto has joined #openstack-infra02:39
*** roman_g has quit IRC02:43
*** bobh has quit IRC02:43
*** openstack has joined #openstack-infra02:48
*** ChanServ sets mode: +o openstack02:48
*** jamesmcarthur has quit IRC02:50
*** hongbin has quit IRC02:56
*** zhangfei has quit IRC03:01
*** dave-mcc_ has quit IRC03:02
*** edmondsw has joined #openstack-infra03:05
*** yamamoto has quit IRC03:07
*** yamahata has joined #openstack-infra03:08
gongyshhi, I am setup a multi node ci job03:09
*** edmondsw has quit IRC03:09
gongyshneed a two nic two node env.03:10
gongyshhow I can get second nic's name and nic's ip address?03:10
gongyshthe job is here: https://review.openstack.org/#/c/578170/03:11
*** yamamoto has joined #openstack-infra03:13
*** zhangfei has joined #openstack-infra03:14
*** ramishra has joined #openstack-infra03:15
*** Fusl21 has joined #openstack-infra03:16
Fusl21Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/03:16
*** Fusl21 has quit IRC03:16
clarkbgongysh: you can use the multinode bridge role to set up additonal interfaces for multinode jobs http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/multi-node-bridge03:16
clarkbbut default you should already get one if using the base devstack multinode job03:16
clarkb*by default03:16
gongyshbr-infra03:20
gongyshyes, I got the one.03:20
*** mriedem_afk has quit IRC03:20
clarkbyou should be able to run it again with different values to set up another interfave (and bridge)03:21
clarkbgongysh: why do you need multiple interfaces?03:22
gongyshclarkb, I need to set up a flat provider network and another for tenant network.03:22
openstackgerritMerged openstack-infra/zuul master: Emit a log line containing version info around startup  https://review.openstack.org/58363903:23
clarkbgongysh: neutron should manage the tenant network though right?03:23
gongyshclarkb, right03:23
clarkbit may work on the one interface in that case03:24
clarkbeither way you should be able to add a new bridge by running that role again with different args03:24
gongyshclarkb, how to do that?03:27
openstackgerritIan Wienand proposed openstack-infra/zuul master: doc: Move zuul variable references to a section  https://review.openstack.org/58477903:27
openstackgerritIan Wienand proposed openstack-infra/zuul master: Add variables to project  https://review.openstack.org/58423003:27
openstackgerritIan Wienand proposed openstack-infra/zuul master: Use definition list for job status  https://review.openstack.org/58478003:27
*** Sukhdev has joined #openstack-infra03:27
gongyshhttp://logs.openstack.org/70/578170/46/check/tacker-functional-devstack-multinode/0b5ddfc/controller/confs/devstack/ovsvsctl.txt03:29
clarkbgongysh: https://docs.ansible.com/ansible/2.5/user_guide/playbooks_reuse_roles.html#using-roles you use include_role I think03:29
gongyshclarkb, I can see that the br-infra is using second card's ip address as sub vxlan port.03:29
*** armax has quit IRC03:29
clarkbthat depends on the cloud but it will use the in cloud private address if that exists to avoid nat03:31
clarkbgongysh: if you use br-infra for provider network then bridge br-ex and br-infra for tenant networking will that work?03:34
clarkbbasically neutron sharing the provider network to run the overlays03:34
*** dtruong2 has joined #openstack-infra03:34
gongyshclarkb, br-infra for provider network flat.,  br-tun for tenant vxlan network03:35
clarkbsure, could you run them both over br-infra though03:35
clarkbthe tenant networks are another layer of vxlan so should be isolated frkm the provider flat networking03:36
clarkbbut could share the l2 to make sure packets go between the two nodes03:37
*** dtruong2 has quit IRC03:38
*** dtruong2 has joined #openstack-infra03:38
openstackgerritIan Wienand proposed openstack-infra/zuul master: Rename didAnyJobFail() to hasAnyJobFailed()  https://review.openstack.org/58513403:39
*** dtruong2 has quit IRC03:40
*** sins- has joined #openstack-infra03:43
sins-Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/03:43
*** sins- has quit IRC03:43
*** graphene has quit IRC03:47
*** graphene has joined #openstack-infra03:48
*** udesale has joined #openstack-infra03:48
*** slaweq has quit IRC03:50
*** lpetrut has joined #openstack-infra03:53
gongyshclarkb,  {{ hostvars[groups['switch'][0]]['nodepool']['private_ipv4'] }}03:56
gongyshso {{ hostvars[groups['switch'][0]]['nodepool']['public_ipv4'] }} is referring to the eth0's ipv4 address, right?03:56
clarkbgongysh: not necessarily. If that cloud uses floating IPs for public IPs it will the be floating IP address not the one on the interface03:57
clarkbgongysh: if you use the multinode node bridge role it should figure out which IP to use for you iirc03:58
*** slaweq has joined #openstack-infra03:58
clarkbgongysh: all you have to worry about is providing the overlay's ip address range03:58
*** eernst has joined #openstack-infra03:59
*** lpetrut has quit IRC04:03
gongyshclarkb, now the br-infra has Ip address. is there a way to let zuul not asign the IP address?04:05
gongyshI know, If I set up the br-infra, I can call the role with: bridge_configure_address == false.04:05
*** mschuppert has joined #openstack-infra04:05
clarkbhttps://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/multi-node-bridge/README.rst#n49 yes that is what you would do. If you are inheriting from the existing multinode job I don't think you can override that04:06
*** dtruong_ has joined #openstack-infra04:06
clarkbyou may be able to override it by setting it at the job level though. I am not sure04:06
*** wolverineav has joined #openstack-infra04:07
clarkbgongysh: one reason to set the IPs is that neutron by default sets up the floating IP range in that /23 so that instances are pingable from tempest04:07
clarkb(this makes all the nova VMs reachable from tempest without setting up routing (since it is all direct attached, a bit of a hack but simple)04:08
*** felipemonteiro has joined #openstack-infra04:09
gongyshclarkb, the default br-infra ip is 172.24.4.1/23 for switch node.04:11
*** janki has joined #openstack-infra04:11
clarkbyup and then fip range is 172.24.5.0/2404:11
clarkbwhich allows the switch node (where tempest is running) to talk to VMs via floating IPs even if they are on the other nodes04:12
*** wolverineav has quit IRC04:12
clarkb(beacuse linux adds routes for direct attached networks)04:12
*** rosmaita has quit IRC04:12
*** yamamoto has quit IRC04:13
clarkbyou should be able to run another l3 subnet range across that l2 if you need to (but not sure) for the flat network04:14
*** psachin`` has joined #openstack-infra04:14
gongysh172.24.5.1/24 on br-ex at switch node.04:16
gongyshno ip assigned at peer node's br-ex.04:16
*** jamesmcarthur has joined #openstack-infra04:17
clarkbgongysh: correct since the fips are all terminated on the swift node if that is the networking node04:17
clarkbI think this changes if you use dvr but by default this is how it is set up04:18
gongyshgot it. now I think I am clear about the multinode stuff.04:18
gongyshthanks04:19
clarkbs/swift/switch/04:19
gongyshclarkb, if you are interested, you can help me to review https://review.openstack.org/#/c/578170/04:20
clarkbgongysh: it is getting late here now, I'll try to take a look in the morning04:20
gongyshclarkb, ok04:21
*** jamesmcarthur has quit IRC04:21
gongyshmany thanks04:21
*** kjackal has joined #openstack-infra04:28
*** gongysh has quit IRC04:38
*** yamamoto has joined #openstack-infra04:38
*** felipemonteiro has quit IRC04:39
*** Sukhdev has quit IRC04:44
*** gongysh has joined #openstack-infra04:46
*** Sukhdev has joined #openstack-infra04:47
*** ykarel|away has joined #openstack-infra04:47
*** spsurya_ has quit IRC04:48
*** spsurya_ has joined #openstack-infra04:49
*** edmondsw has joined #openstack-infra04:53
*** gongysh has quit IRC04:54
*** armaan has joined #openstack-infra04:57
*** dtruong2 has joined #openstack-infra04:57
*** edmondsw has quit IRC04:58
*** armaan has quit IRC05:01
*** Whiskey27 has joined #openstack-infra05:03
Whiskey27Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/05:03
*** Whiskey27 has quit IRC05:03
*** eernst has quit IRC05:05
*** e0ne has joined #openstack-infra05:06
*** Tahvok has quit IRC05:06
*** spsurya_ is now known as spsurya05:07
*** quiquell has joined #openstack-infra05:08
*** Tahvok has joined #openstack-infra05:10
*** yamamoto has quit IRC05:13
*** yamamoto has joined #openstack-infra05:15
*** dtruong2 has quit IRC05:21
*** rwsu has quit IRC05:23
*** felipemonteiro has joined #openstack-infra05:24
*** dtruong2 has joined #openstack-infra05:24
*** gongysh has joined #openstack-infra05:24
*** dtruong2 has quit IRC05:28
*** dtruong2 has joined #openstack-infra05:30
*** dtruong2 has quit IRC05:30
*** Sukhdev has quit IRC05:31
*** yamamoto has quit IRC05:31
*** dtruong2 has joined #openstack-infra05:31
*** dtruong2 has quit IRC05:32
*** dtruong2 has joined #openstack-infra05:33
*** dtruong2 has quit IRC05:33
*** zhangfei has quit IRC05:33
*** bambams24 has joined #openstack-infra05:34
bambams24Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/05:34
*** bambams24 has quit IRC05:34
*** pgadiya has joined #openstack-infra05:35
*** pgadiya has quit IRC05:35
*** yamamoto has joined #openstack-infra05:36
*** agopi has quit IRC05:38
*** gongysh has quit IRC05:38
*** FiendKing04 has joined #openstack-infra05:44
FiendKing04Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/05:44
FiendKing04or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/05:44
FiendKing04Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate05:44
*** FiendKing04 has quit IRC05:44
*** zhangfei has joined #openstack-infra05:47
*** yaymuffins2 has joined #openstack-infra05:48
yaymuffins2Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/05:48
*** yaymuffins2 has quit IRC05:48
*** links has joined #openstack-infra05:48
*** cshastri has joined #openstack-infra05:50
*** dklyle has quit IRC05:52
*** dklyle has joined #openstack-infra05:53
*** Guest46157 has joined #openstack-infra05:58
*** wook_10 has joined #openstack-infra05:58
wook_10Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/05:58
*** wook_10 has quit IRC05:58
*** e0ne has quit IRC06:00
*** yamamoto has quit IRC06:01
*** yamamoto has joined #openstack-infra06:03
*** udesale has quit IRC06:04
*** udesale has joined #openstack-infra06:04
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Tolerate missing project  https://review.openstack.org/57987206:04
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Restore zuul_swift_upload.py script  https://review.openstack.org/58581406:07
*** wolverineav has joined #openstack-infra06:08
*** wolverineav has quit IRC06:12
*** yamamoto has quit IRC06:19
*** kjackal has quit IRC06:20
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Restore zuul_swift_upload.py script  https://review.openstack.org/58581406:20
openstackgerritZhao Chao proposed openstack-infra/project-config master: Aslo apply the py35 job for trove-dashboard queens  https://review.openstack.org/58596606:22
*** dhajare has joined #openstack-infra06:23
openstackgerritOpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/58596906:24
*** Douhet has quit IRC06:29
*** Douhet has joined #openstack-infra06:31
*** edmondsw has joined #openstack-infra06:41
*** pguimaraes has joined #openstack-infra06:42
*** Douhet has quit IRC06:43
*** Douhet has joined #openstack-infra06:44
*** edmondsw has quit IRC06:47
openstackgerritMerged openstack-infra/zuul master: doc: Move zuul variable references to a section  https://review.openstack.org/58477906:51
*** mcspud21 has joined #openstack-infra06:53
mcspud21Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/06:53
mcspud21or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/06:53
*** mcspud21 has quit IRC06:53
*** jpich has joined #openstack-infra06:58
*** florianf has joined #openstack-infra07:01
*** Douhet has quit IRC07:08
*** Douhet has joined #openstack-infra07:08
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Remove unecessary shebang and exec bit  https://review.openstack.org/58356307:12
*** gongysh has joined #openstack-infra07:12
*** kjackal has joined #openstack-infra07:14
openstackgerritSampath Priyankara (samP) proposed openstack-infra/project-config master: Add publish-to-pypi-horizon job to masakari-dashboard  https://review.openstack.org/58597507:22
*** slaweq has quit IRC07:23
*** slaweq has joined #openstack-infra07:23
*** felipemonteiro has quit IRC07:23
*** pcaruana has joined #openstack-infra07:25
*** ccamacho has joined #openstack-infra07:27
*** gfidente has joined #openstack-infra07:28
*** gfidente has joined #openstack-infra07:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Restructure return values section in docs  https://review.openstack.org/58598107:32
*** zoli|gone is now known as zoli07:35
*** tosky has joined #openstack-infra07:38
*** zhangfei has quit IRC07:42
*** quiquell is now known as quiquell|bbl07:42
*** dtantsur|afk is now known as dtantsur07:46
*** alexchadin has joined #openstack-infra07:48
*** quiquell|bbl has quit IRC07:49
*** zhangfei has joined #openstack-infra07:55
*** quiquell has joined #openstack-infra08:02
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Restructure return values section in docs  https://review.openstack.org/58598108:03
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support job pause  https://review.openstack.org/58538908:03
*** quiquell has quit IRC08:03
*** alexchad_ has joined #openstack-infra08:05
*** alexchadin has quit IRC08:07
*** wolverineav has joined #openstack-infra08:09
*** shardy has joined #openstack-infra08:10
*** AJaeger has quit IRC08:12
*** jiapei has joined #openstack-infra08:13
*** Sagan27 has joined #openstack-infra08:13
Sagan27Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/08:13
*** Sagan27 has quit IRC08:13
*** wolverineav has quit IRC08:13
*** Bhujay has joined #openstack-infra08:14
*** yamamoto has joined #openstack-infra08:15
*** kjackal has quit IRC08:15
*** e0ne has joined #openstack-infra08:17
*** Guest72310 has joined #openstack-infra08:18
Guest72310Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/08:18
Guest72310or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/08:18
*** Guest72310 has quit IRC08:18
*** yamamoto has quit IRC08:18
*** armaan has joined #openstack-infra08:19
*** AJaeger has joined #openstack-infra08:19
openstackgerritMerged openstack/diskimage-builder master: modprobe DIB_MODPROBE_BLACKLIST should be optional  https://review.openstack.org/58578408:25
openstackgerritMerged openstack/diskimage-builder master: Add DIB element to blacklist nouveau  https://review.openstack.org/57177708:25
*** zoli is now known as zoli|dentist08:25
openstackgerritMerged openstack-infra/zuul master: Remove unecessary shebang and exec bit  https://review.openstack.org/58356308:29
*** edmondsw has joined #openstack-infra08:30
*** kashyap has joined #openstack-infra08:32
kashyapMorning folks08:32
kashyapSeems like git.openstack.org isn't staler compared to GitHub?08:32
kashyapE.g.08:32
kashyapCompare these two:08:33
kashyaphttp://git.openstack.org/cgit/openstack/nova/tree/nova/conf/libvirt.py?id=6b601b7#n64208:33
kashyaphttps://github.com/openstack/nova/blob/master/nova/conf/libvirt.py#L66208:33
kashyapThe GitHub edition is correct (& up2date).08:33
*** edmondsw has quit IRC08:34
*** Bhujay has quit IRC08:37
*** Bhujay has joined #openstack-infra08:37
*** ykarel|away has quit IRC08:38
*** ykarel| has joined #openstack-infra08:38
*** pbourke has quit IRC08:39
*** pbourke has joined #openstack-infra08:41
*** tesseract has joined #openstack-infra08:42
*** KindOne1423 has joined #openstack-infra08:43
KindOne1423Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/08:43
*** KindOne1423 has quit IRC08:43
*** derekh has joined #openstack-infra08:43
*** udesale_ has joined #openstack-infra08:44
*** janki has quit IRC08:45
*** udesale has quit IRC08:46
openstackgerritMerged openstack-infra/project-config master: Normalize projects.yaml  https://review.openstack.org/58596908:48
*** sshnaidm|afk is now known as sshnaidm08:51
*** dingyichen has quit IRC08:54
AJaegerkashyap: git.openstack.org is our *master*, github is only a mirror.08:55
AJaegerkashyap: why do you have ?id=... in the URL08:56
kashyapAJaeger: Then, surely "master" is out of date?08:56
AJaegerhttp://git.openstack.org/cgit/openstack/nova/tree/nova/conf/libvirt.py#n662 looks correct08:56
kashyapOr is it a PEBKAC?08:56
kashyapAJaeger: Err08:56
kashyapAJaeger: Sorry, I was looking at a stale commit!08:56
AJaegerthat explains it ;)08:57
*** sambetts|afk has quit IRC08:57
kashyapAJaeger: Yeah, sorry for the noise.08:58
*** alexchadin has joined #openstack-infra08:59
*** sambetts_ has joined #openstack-infra08:59
*** aOssed- has joined #openstack-infra09:00
aOssed-Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/09:00
aOssed-or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/09:00
*** aOssed- has quit IRC09:00
*** alexchad_ has quit IRC09:01
*** dtruong_ has quit IRC09:02
*** florianf has quit IRC09:06
AJaegerkashyap: no worries09:08
kashyapNod09:09
*** zhangfei has quit IRC09:19
*** d0ugal has joined #openstack-infra09:21
*** armaan has quit IRC09:22
*** armaan has joined #openstack-infra09:23
*** ginopc has joined #openstack-infra09:29
*** Bhujay has quit IRC09:30
*** Bhujay has joined #openstack-infra09:30
*** zhangfei has joined #openstack-infra09:31
*** icywiz24 has joined #openstack-infra09:33
icywiz24Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/09:33
*** icywiz24 has quit IRC09:33
*** stakeda has quit IRC09:46
*** Bhujay has quit IRC09:48
*** Bhujay has joined #openstack-infra09:48
*** zoli|dentist is now known as zoli09:50
*** armaan has quit IRC09:50
*** armaan has joined #openstack-infra09:51
*** Bhujay has quit IRC09:51
*** d0ugal has quit IRC09:52
*** Bhujay has joined #openstack-infra09:52
*** Bhujay has quit IRC09:54
*** Bhujay has joined #openstack-infra09:55
*** Bhujay has quit IRC09:57
*** Bhujay has joined #openstack-infra09:58
*** armaan has quit IRC10:01
*** armaan has joined #openstack-infra10:02
*** quiquell has joined #openstack-infra10:03
*** janki has joined #openstack-infra10:06
*** udesale__ has joined #openstack-infra10:08
*** alexchadin has quit IRC10:08
*** apetrich has quit IRC10:09
*** wolverineav has joined #openstack-infra10:10
*** dtruong_ has joined #openstack-infra10:10
*** udesale_ has quit IRC10:10
*** sshnaidm is now known as sshnaidm|afk10:13
*** wolverineav has quit IRC10:14
*** jamesmcarthur has joined #openstack-infra10:18
*** edmondsw has joined #openstack-infra10:18
*** alexchadin has joined #openstack-infra10:18
*** Techman29 has joined #openstack-infra10:21
Techman29Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/10:21
*** Techman29 has quit IRC10:21
*** Bhujay has quit IRC10:22
*** jamesmcarthur has quit IRC10:22
*** Bhujay has joined #openstack-infra10:23
*** dx18 has joined #openstack-infra10:23
dx18Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/10:23
dx18or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/10:23
*** dx18 has quit IRC10:23
*** edmondsw has quit IRC10:23
openstackgerritJean-Philippe Evrard proposed openstack-infra/bindep master: Add opensuse-leap support  https://review.openstack.org/58603810:25
evrardjpAJaeger: could you have a look at this ^ and give me your opinion?10:25
openstackgerritJean-Philippe Evrard proposed openstack-infra/bindep master: Add opensuse-leap support  https://review.openstack.org/58603810:27
*** pelegreno______ has joined #openstack-infra10:28
pelegreno______Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/10:28
pelegreno______or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/10:28
*** pelegreno______ has quit IRC10:28
*** alexchadin has quit IRC10:29
*** yamamoto has joined #openstack-infra10:32
*** d0ugal has joined #openstack-infra10:32
*** d0ugal has quit IRC10:34
*** alexchadin has joined #openstack-infra10:38
AJaegerevrardjp: let's ask dirk as well10:40
*** dhajare has quit IRC10:40
evrardjpok -- for the tests I expected this -- it was for the idea I was looking for, but you didn't say anything against it, so I am good :)10:42
evrardjpwill add the fixtures and the rest.10:42
evrardjpwill add dirk on this too.10:42
AJaegerevrardjp: yes, change looks fine - just missing tests.10:42
AJaegerevrardjp: thanks!10:42
evrardjpok10:42
*** yamamoto has quit IRC10:45
*** edmondsw has joined #openstack-infra10:49
*** alexchad_ has joined #openstack-infra10:49
*** dhajare has joined #openstack-infra10:50
*** armaan has quit IRC10:50
*** armaan has joined #openstack-infra10:50
*** alexchadin has quit IRC10:51
*** Bhujay has quit IRC10:56
*** Bhujay has joined #openstack-infra10:57
*** Bhujay has quit IRC10:57
*** sdx2324 has joined #openstack-infra10:58
sdx2324Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/10:58
*** sdx2324 has quit IRC10:58
*** Bhujay has joined #openstack-infra10:58
*** linkmark has joined #openstack-infra10:58
*** Bhujay has quit IRC10:59
*** Bhujay has joined #openstack-infra10:59
*** pguimaraes has quit IRC11:01
*** rcernin has quit IRC11:04
*** quiquell is now known as quiquell|bbl11:05
*** dave-mccowan has joined #openstack-infra11:07
*** Bhujay has quit IRC11:11
*** Bhujay has joined #openstack-infra11:11
*** shardy is now known as shardy_lunch11:13
*** larainema has quit IRC11:14
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support complex data structures as secrets  https://review.openstack.org/58589211:24
*** udesale__ has quit IRC11:25
*** rosmaita has joined #openstack-infra11:26
*** Bhujay has quit IRC11:27
*** Bhujay has joined #openstack-infra11:27
*** 07IADFUJ8 has joined #openstack-infra11:28
*** tomaw12 has joined #openstack-infra11:28
07IADFUJ8Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/11:28
*** 07IADFUJ8 has quit IRC11:28
tomaw12Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/11:28
*** tomaw12 has quit IRC11:28
*** quiquell|bbl is now known as quiquell11:32
*** Bhujay has quit IRC11:33
*** graphene has quit IRC11:33
*** kjackal has joined #openstack-infra11:34
*** graphene has joined #openstack-infra11:35
*** zhangfei has quit IRC11:35
*** panda|rover|off is now known as panda|rover11:37
*** zoli is now known as zoli|lunch11:38
*** Bhujay has joined #openstack-infra11:38
*** gongysh has quit IRC11:44
*** agopi has joined #openstack-infra11:45
*** tpsilva has joined #openstack-infra11:46
*** boden has joined #openstack-infra11:49
*** graphene has quit IRC11:53
*** graphene has joined #openstack-infra11:54
*** graphene has quit IRC11:57
*** ccamacho1 has joined #openstack-infra11:58
*** ccamacho has quit IRC11:58
*** graphene has joined #openstack-infra11:58
*** gongysh has joined #openstack-infra11:59
*** Bhujay has quit IRC12:00
*** armaan has quit IRC12:03
*** armaan has joined #openstack-infra12:04
*** rfolco|ruck|off is now known as rfolco|ruck12:05
*** armaan has quit IRC12:07
*** armaan has joined #openstack-infra12:08
*** shardy_lunch is now known as shardy12:10
*** zoli|lunch is now known as zoli12:11
*** armaan has quit IRC12:11
*** tharkun22 has joined #openstack-infra12:11
tharkun22Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/12:11
*** tharkun22 has quit IRC12:11
*** armaan has joined #openstack-infra12:12
*** hamzy has quit IRC12:14
*** shardy has quit IRC12:16
*** jamesde__ has joined #openstack-infra12:16
*** rh-jelabarre has joined #openstack-infra12:18
*** wolverineav has joined #openstack-infra12:18
*** hamzy has joined #openstack-infra12:19
*** jamesden_ has quit IRC12:19
*** wolverineav has quit IRC12:20
*** wolverineav has joined #openstack-infra12:20
*** shardy has joined #openstack-infra12:20
*** pguimaraes has joined #openstack-infra12:26
*** psachin`` has quit IRC12:26
*** armaan has quit IRC12:27
*** armaan has joined #openstack-infra12:27
*** rlandy has joined #openstack-infra12:28
*** jcoufal has joined #openstack-infra12:33
*** jiapei has quit IRC12:33
*** yamamoto has joined #openstack-infra12:34
*** armaan has quit IRC12:35
*** armaan has joined #openstack-infra12:36
*** graphene has quit IRC12:36
*** openstackgerrit has quit IRC12:36
*** mriedem has joined #openstack-infra12:36
*** quiquell is now known as quiquell|lunch12:37
*** yamamoto has quit IRC12:38
*** yamamoto has joined #openstack-infra12:39
mordred infra-root: openstackgerrit got killed by Sigyn just now in #openstack-sdks ...12:43
mordred"12:36:45             <-- | openstackgerrit (~openstack@review01.openstack.org) has quit (Killed (Sigyn (Spam is off topic on freenode.)))"12:43
*** gongysh has quit IRC12:43
mordreddidn't we have it registered somewhere as official?12:44
mordredoh - I guess it got killed everywhere - I just saw it in openstack-sdks :)12:44
*** cshastri has quit IRC12:46
*** bobh has joined #openstack-infra12:46
*** armaan has quit IRC12:46
*** links has quit IRC12:46
*** armaan has joined #openstack-infra12:46
rcarrillocruz\o/12:47
rcarrillocruzthat spam is plaguing freenode lately12:47
*** sshnaidm|afk is now known as sshnaidm12:48
*** vivsoni_ has quit IRC12:51
*** bobh has quit IRC12:51
*** quiquell|lunch is now known as quiquell12:51
*** bobh has joined #openstack-infra12:54
*** Sukhdev has joined #openstack-infra12:56
*** bobh has quit IRC12:58
*** armaan has quit IRC13:00
*** kgiusti has joined #openstack-infra13:00
*** Sukhdev has quit IRC13:01
*** ykarel| has quit IRC13:01
*** ykarel| has joined #openstack-infra13:03
*** vivsoni has joined #openstack-infra13:03
*** bdodd has quit IRC13:05
*** bobh has joined #openstack-infra13:05
*** armaan has joined #openstack-infra13:05
*** armaan has quit IRC13:08
*** alexchad_ has quit IRC13:09
*** bobh has quit IRC13:09
*** ykarel| has quit IRC13:12
*** bobh has joined #openstack-infra13:15
*** neiloy has joined #openstack-infra13:17
*** dhajare has quit IRC13:17
*** shardy has quit IRC13:18
*** bobh has quit IRC13:19
*** shardy has joined #openstack-infra13:20
*** eharney has joined #openstack-infra13:21
*** yamamoto has quit IRC13:22
*** bdodd has joined #openstack-infra13:24
*** mriedem is now known as mriedem_afk13:25
evrardjprcarrillocruz: yeah it's so pleasant!13:25
evrardjp:p13:25
EmilienMI guess you probably know but in case: it seems like the bot isn't alive anymore13:29
*** jistr is now known as jistr|mtg13:29
*** kiennt26 has joined #openstack-infra13:34
*** gongysh has joined #openstack-infra13:35
AJaegerEmilienM: scroll up 3 lines, please13:38
EmilienMoops ok sorry13:38
AJaegeryou're welcome13:38
*** r-daneel has joined #openstack-infra13:40
*** alexchadin has joined #openstack-infra13:44
*** rook-- is now known as rook13:45
*** yamamoto has joined #openstack-infra13:45
*** Bhujay has joined #openstack-infra13:47
*** Bhujay has quit IRC13:47
*** annp_ has joined #openstack-infra13:50
*** hongbin has joined #openstack-infra13:52
*** mriedem_afk is now known as mriedem13:53
*** roman_g has joined #openstack-infra13:57
*** eharney has quit IRC14:00
*** Guest14217 is now known as zeus14:05
*** zeus has quit IRC14:06
*** zeus has joined #openstack-infra14:06
*** _ari_ has quit IRC14:07
*** rnoriega has quit IRC14:07
*** haleyb has quit IRC14:07
*** myoung has quit IRC14:07
*** pabelanger has quit IRC14:07
*** weshay_PTO has quit IRC14:07
*** kashyap has quit IRC14:07
*** kashyap has joined #openstack-infra14:07
*** _ari_ has joined #openstack-infra14:07
*** pabelanger has joined #openstack-infra14:07
*** rnoriega has joined #openstack-infra14:07
*** weshay has joined #openstack-infra14:08
*** haleyb has joined #openstack-infra14:08
*** armaan has joined #openstack-infra14:09
*** myoung has joined #openstack-infra14:12
Shrewspabelanger: clarkb: mordred: please check my puppeting, but i think https://review.openstack.org/586231 with https://review.openstack.org/586236 should fix logging file mixup14:13
Shrewstl;dr  launchers are getting logging config from puppet-nodepool, builders from system-config14:14
*** gongysh has quit IRC14:14
*** eharney has joined #openstack-infra14:14
*** armaan has quit IRC14:20
*** eernst has joined #openstack-infra14:24
*** jamesmcarthur has joined #openstack-infra14:24
*** gongysh has joined #openstack-infra14:26
*** psachin`` has joined #openstack-infra14:26
*** rpioso|afk is now known as rpioso14:27
*** gongysh has quit IRC14:33
*** florianf has joined #openstack-infra14:38
roman_gHello. Please, review/+2/merge: minor thing - adding statusbot to the #airshipit channel - https://review.openstack.org/#/c/581704/14:39
roman_gPending since July, 11th14:39
*** L23515 has joined #openstack-infra14:41
L23515Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/14:41
*** L23515 has quit IRC14:41
*** yamamoto has quit IRC14:43
*** gema has quit IRC14:44
*** yamamoto has joined #openstack-infra14:45
*** hongbin has quit IRC14:46
*** yamamoto has quit IRC14:49
*** Sukhdev has joined #openstack-infra14:50
*** greghaynes has quit IRC14:52
*** psachin`` has quit IRC14:54
*** jistr|mtg is now known as jistr14:56
*** gongysh has joined #openstack-infra14:57
*** tbarron has quit IRC15:01
*** gongysh has quit IRC15:01
*** kiennt26 has quit IRC15:08
*** tbarron has joined #openstack-infra15:08
*** alexchadin has quit IRC15:08
gouthamrhey folks, we have a problem on #openstack-meeting-alt - tbarron just got banned from freenode; and we were midway a meeting he started15:09
gouthamrhow does one of us gain control?15:09
toskygouthamr: tbarron is back, it seems15:10
gouthamryep, no one panic :) he's back.15:11
pabelangerlooking15:11
mordred\o/15:11
pabelangercool15:11
pabelangerI invited Sigyn there to help with spam, so maybe I should undo that15:12
toskypabelanger: spam is still ongoing15:12
toskytbarron: suggestion: use #chair <nickname> to also set someone else as chair15:12
pabelangertosky: yah, Sigyn will watch and kick potential spammer15:12
pabelangerguess it didn't like the massive ping15:13
*** Guest19794 is now known as kklimonda15:13
*** mgoddard has joined #openstack-infra15:13
*** kklimonda has quit IRC15:14
*** kklimonda has joined #openstack-infra15:14
*** janki has quit IRC15:15
*** greghaynes has joined #openstack-infra15:16
*** gema has joined #openstack-infra15:16
*** gema has quit IRC15:16
*** gema has joined #openstack-infra15:16
mgoddardhi mordred, any chance you could take a look at https://github.com/ansible/ansible/pull/34472?15:17
*** Sukhdev has quit IRC15:17
mordredmgoddard: yes - it's now open in my browser15:19
*** greghaynes has quit IRC15:19
mgoddardmordred: great, thanks15:19
*** jpeeler[m] has joined #openstack-infra15:19
*** annp_ has quit IRC15:27
tbarrontosky: yup15:28
*** pblaho has quit IRC15:35
*** greghaynes has joined #openstack-infra15:35
*** pcaruana has quit IRC15:38
*** jpeeler[m] has left #openstack-infra15:39
*** hamzy has quit IRC15:41
*** hamzy has joined #openstack-infra15:41
*** zoli is now known as zoli|gone15:46
*** hongbin has joined #openstack-infra15:49
*** psachin`` has joined #openstack-infra15:49
roman_gActually where is bot who reports on gerrit submissions?15:52
roman_gopenstackgerrit is gone15:53
clarkbroman_g: it got kicked by Sigyn. I think we may leave it in that state until the spam problem decreases then we can remove Sigyn and add our bots back?15:53
clarkbpabelanger: corvus can we whitelist users with Sigyn or is it all or nothing?15:53
roman_gclarkb: I don't mind if it would not be reporting here for every patch set. At max - only for merges.15:54
*** armax has joined #openstack-infra15:54
*** yamamoto has joined #openstack-infra15:55
*** hamerins has joined #openstack-infra15:55
pabelangerclarkb: not sure, docs say we can unkline after a few minutes15:55
clarkbroman_g: the problem (I think) is that sigyn sees repeated messages with little differentiation as spam particularly if they happen often across many channels15:55
clarkbroman_g: so even merge reports would fall victim to that15:55
*** erlon_ has joined #openstack-infra15:55
*** ykarel| has joined #openstack-infra15:57
roman_gaah. I see.15:57
AJaegerso, people should not resubmit the same changes so often? ;)15:57
*** edmondsw has quit IRC15:57
*** edmondsw has joined #openstack-infra15:57
clarkbroman_g: but we really only need sigyn around while the spam attack is happening. Hopefully that goes away soon and we can bring back openstackgerrit and it will be happy15:58
*** gfidente has quit IRC15:58
*** edmondsw has quit IRC15:59
*** rkukura has quit IRC15:59
*** gyee has joined #openstack-infra15:59
*** myoung is now known as myoung|lunch16:00
*** sshnaidm is now known as sshnaidm|afk16:00
roman_gthank you for the explanation, clarkb16:00
*** yamamoto has quit IRC16:01
tbarronYeah, Sigyn doesn't like courtesy pings for meetings :)16:01
tbarronjungleboyj: ^^^16:01
*** e0ne has quit IRC16:02
jungleboyjtbarron: What's up?16:02
tbarronjungleboyj: just be careful with courtesy pings16:03
tbarronjungleboyj: Sigyn (bot that helps with that annoyming spam) doesn't like them and may ban you as you start your next meeting16:03
jungleboyjOh, ok.  Good to know.16:04
jungleboyjtbarron:  Thanks!16:04
*** adriant has quit IRC16:05
* fungi isn't personally a fan of "courtesy pings" either, uses a reminder application to let him know when meetings and other events are about to start anyway16:05
*** adriant has joined #openstack-infra16:06
*** yamahata has quit IRC16:06
jungleboyjfungi:  You are so organized.  ;-)16:06
tbarronfungi: yeah, I don't have to ping myself, but we ring a cowbell for the community16:07
clarkbsigyn catches it because it is a common spammer behavior too16:07
tbarronclarkb: makes sense16:07
*** ginopc has quit IRC16:07
clarkb"everyone look at my spam!"16:07
clarkb:)16:08
fungiright, my point was it's annoying in both cases ;)16:08
tbarronI'm happy to have an excuse not to do the courtesy ping.16:08
clarkbhrm elastic-recheck hasn't regenerated graphs in a while. /me will check on that after caffeine16:08
clarkb(this has become the morning ritual, what is broken with the gate during feature freeze)16:08
tbarronBut it was requested by the meeting attendees.16:09
pabelangerclarkb: I've asked in #freenode-sigyn for a whitelist, but unsure since it is a lot of channels16:09
*** jpich has quit IRC16:09
clarkbpabelanger: ya I don't think its the worst thing to ride it out for another day, I'm assuming the spam attack won't last much longer16:09
*** florianf has quit IRC16:10
jungleboyjtbarron:  ++16:10
jungleboyjI do use them sometimes but for myself but I can set better reminders.  :-)16:10
hamerinszuul is failing to create the environment correctly for openstack-helm. where/who can i reach out to double check zuul environment16:12
clarkbhamerins: this is a good place. Can you provide logs and more specific details on what is wrong with the environment?16:12
hamerinsthe pods for the job are in initializing status.16:14
hamerinslogs :16:15
hamerins1. http://logs.openstack.org/70/585570/4/check/openstack-helm-dev-deploy-ceph-radosgw-ovs/9bfa295/job-output.txt.gz,16:15
hamerins2. http://logs.openstack.org/70/585570/4/check/openstack-helm-dev-deploy-nfs-lb/e1c4bb0/job-output.txt.gz16:15
hamerins3. http://logs.openstack.org/70/585570/4/check/openstack-helm-ironic-ubuntu/91b82b4/job-output.txt.gz16:15
hamerinssame check was successful yesterday morning.16:15
*** quiquell has quit IRC16:16
clarkbhamerins: where can we see the logs for the initializing pods? http://logs.openstack.org/70/585570/4/check/openstack-helm-dev-deploy-ceph-radosgw-ovs/9bfa295/primary/pod-logs/ceph/ceph-cephfs-provisioner-784c6f9d59-9vnx5/ ended up running and has same log data as the one that didn't16:18
*** pguimaraes has quit IRC16:18
clarkbhttp://logs.openstack.org/70/585570/4/check/openstack-helm-dev-deploy-ceph-radosgw-ovs/9bfa295/primary/pod-logs/ceph/ceph-cephfs-provisioner-784c6f9d59-44bdz/16:18
*** ykarel| has quit IRC16:18
*** openstackgerrit has joined #openstack-infra16:19
openstackgerritMerged openstack-infra/zuul master: Fix delegate_to for ansible synchronize  https://review.openstack.org/58465616:19
*** verdurin has quit IRC16:21
hamerinsclarkb: in the job-output.txt.gz  ceph-cephfs-provisioner is in PodInitializing. the playbook is exiting with container failed to start16:23
clarkbhamerins: yes, and the logs for the started cephfs provisioner are identical to the failed cephfs provisioner. So we don't really have much info to go off of on why it failed to start. Are there other logs that show why it failed to start or can we add logging for that?16:24
*** verdurin has joined #openstack-infra16:24
pabelangerclarkb: openstackgerrit is back16:26
pabelangershould be protected now16:26
clarkbpabelanger: thank you16:26
hamerinsclarkb: i suspect this have to do some thing with error on job openstack-helm-dev-deploy-nfs-lb. its failing to complete the TASK [build-images : Kubeadm-AIO image build path with proxy] ******************16:27
clarkbhamerins: I thinkthe errors are independent, in the one it looks like a network failure downloading a tarball from github (not super surprising considering that cloud has to NAT for ipv4 and github is ipv4 only), and the other job is a failure to start a container which already has an image (based on the fact that another copy of that container did start)16:29
*** andymccr_ has joined #openstack-infra16:29
clarkblogan-: ^ fyi there may be ipv4 NAT sadness in limestone (I can help look at that more directly soon)16:29
*** ykarel| has joined #openstack-infra16:30
clarkboh wait sorry this is packethost not limestone, not sure why I read it as limestone16:30
clarkbnow I feel like a derp16:30
clarkbin any case internet reliability (particularly github) has never been stellar16:31
clarkbI really need that cup of tea16:31
hamerins:)16:31
*** andymccr_ has quit IRC16:33
*** andymccr_ has joined #openstack-infra16:33
*** andymccr_ has quit IRC16:34
*** andymccr_ has joined #openstack-infra16:34
clarkbhamerins: but still, I think those failures are separate, one is relying on internet resource which tend to be flaky and the other is we don't know what container image failing to start but another copy of it did start16:34
clarkbhamerins: I think we need more logging for the failure to start of the container image16:34
*** adriant has quit IRC16:34
*** adriant has joined #openstack-infra16:35
*** psachin`` has quit IRC16:35
clarkband for the third it appear to be a multinode job using 172.24.6.0/24 for floating Ips but I think we may only have routes set up for 172.24.4.0/23 (that is our default setup I would have to check job logs more closely to see if you are expanding that network range at all)16:36
hamerinsclarkb: i have not changed anything regarding to network in the code commit.16:37
*** dtantsur is now known as dtantsur|afk16:38
*** andymccr_ has quit IRC16:38
*** andymccr_ has joined #openstack-infra16:38
clarkbhamerins: http://logs.openstack.org/70/585570/4/check/openstack-helm-ironic-ubuntu/91b82b4/job-output.txt.gz#_2018-07-26_01_09_00_975182 you can see there that the firewall rules are only added for 172.24.4.0/2316:38
clarkbso unless you are configuring 172.24.6.0/24 somewhere else I don't expect that to work on multinode16:38
*** tesseract has quit IRC16:39
*** andymccr_ has quit IRC16:39
hamerinsclarkb: seems like 172.24.6.0/24 is part of the ironic setup16:41
hamerinsopenstack-helm/tools/deployment/baremetal/005-setup-nodes.sh:83:OSH_IRONIC_PXE_ADDR="${OSH_IRONIC_PXE_ADDR:="172.24.6.1/24"}"16:41
*** yamahata has joined #openstack-infra16:41
clarkbhamerins: but what makes that routable (and valid for the firewall) from the primary node to whatever other node is running the "baremetal" host16:42
*** ykarel| has quit IRC16:42
clarkbhamerins: looks like you may be bootstrapping that yourselves and not using our multinode node bridge role16:44
clarkbosh-infra-deploy-docker.yaml is what updates the firewall rules above (but using that /2316:44
*** andymccr_ has joined #openstack-infra16:44
clarkbfwiw I think this is a third distinct failure case likely related to lack of ip routing from primary node (where ssh runs) to where the ironic fake baremetal instance is running either due to no route at all or firewall blocking the traffic16:45
clarkbthe connection refused message probably implies it is the firewall16:46
hamerinsclarkb: hmm, i have not done anything on the ironic code.16:46
*** bobh has joined #openstack-infra16:46
clarkbI don't think it is ironic, this is a level above that16:47
clarkbyou need ip connectivity between the nodes16:47
*** chandankumar is now known as chkumar|trekk16:47
clarkbmy hunch is that if the ip was 172.24.5.6 it would work16:48
clarkbbecause we allow that in the firewall16:48
clarkbbut we don't allow 172.24.6.616:48
*** rkukura has joined #openstack-infra16:50
hamerinsseems like the same check is working now for other code commit  logs: http://logs.openstack.org/52/581152/15/check/openstack-helm-ironic-ubuntu/ad6d4a7/job-output.txt.gz16:52
*** andymccr_ has quit IRC16:53
*** andymccr_ has joined #openstack-infra16:54
clarkbhamerins: I think that could happen if the ironic baremetal instance was on the primary node, but if it doesn't schedule there it wouldn't work?16:54
clarkb(multinode makes this complicated particularly for networking because you get away with a lot more on localhost)16:55
*** andymccr_ has quit IRC16:55
clarkbhamerins: can we check where that baremetal instance booted in the logs for each of those jobs?16:55
*** andymccr_ has joined #openstack-infra16:55
*** d0ugal has joined #openstack-infra16:55
*** d0ugal has quit IRC16:55
*** d0ugal has joined #openstack-infra16:55
hamerinshmm. i have re-trigged the zuul review and  its in queue.16:56
clarkbhamerins: http://logs.openstack.org/70/585570/4/check/openstack-helm-ironic-ubuntu/91b82b4/primary/helm/libvirt.txt is there something else that would tell us where each of those pods was scheduled?16:59
clarkboh the libvirt logs themselves capture the hostname it is running on17:00
clarkbeach pod runs on a different host in the 3 node node job17:00
clarkbso I think it may be 1/3 chance of working if yo uhappen to shcedule to the primary node17:00
*** derekh has quit IRC17:00
clarkb(still tryin gto figure out what the floating IP to libvirt pod mapping is)17:00
*** shardy has quit IRC17:01
hamerinshmm. i think this can help you identify the correct node http://logs.openstack.org/70/585570/4/check/openstack-helm-ironic-ubuntu/91b82b4/primary/system/ip-addr.txt17:02
clarkbhamerins: one thing I'm noticing is that you allocate 4GB of memory to the "baremetal" instances that is quite significant when the host only has 8GB of memory (could be something to tune down for general reliability)17:06
*** jamesmcarthur has quit IRC17:10
*** r-daneel_ has joined #openstack-infra17:10
*** r-daneel has quit IRC17:10
*** r-daneel_ is now known as r-daneel17:10
clarkbheat isn't recording where it got its stack resources from only if they failed or succeeded17:11
clarkblets see if the other services record this info17:11
*** Sukhdev has joined #openstack-infra17:13
clarkbhamerins: ya I'm not sure I can debug tis further without debug logs from services17:14
*** r-daneel has quit IRC17:14
clarkbhamerins: I can't find what "actual" resources are mapped to the openstack resources with the current level of debugging17:15
clarkb*level of logging17:15
clarkbalso we should probably compress the log files17:15
*** e0ne has joined #openstack-infra17:20
*** d0ugal has quit IRC17:21
*** r-daneel has joined #openstack-infra17:22
*** felipemonteiro has joined #openstack-infra17:22
*** felipemonteiro has quit IRC17:24
clarkbe-r graphing is running again, a process was holding the lock for ~16 hours so I killed it and that freed up everything queued up behind17:24
*** kjackal has quit IRC17:27
*** bobh_ has joined #openstack-infra17:28
*** jtomasek has quit IRC17:29
*** trown is now known as trown|lunch17:29
hamerinsclarkb: this completed just now. http://logs.openstack.org/52/581152/15/check/openstack-helm-ironic-ubuntu/ad6d4a7/17:30
clarkbhamerins: I think thats the one from before?17:30
hamerinsyes. the job is almost complete for this patch set.17:30
hamerinsclarkb: how many jobs are run parallely for zuul validation?17:31
*** bobh has quit IRC17:32
clarkbhamerins: anywhere from none to ~1k (depends on demand and available resources)17:32
chkumar|trekkpabelanger: we are seeing mirror timed out http://logs.openstack.org/13/586213/1/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/85287a1/job-output.txt.gz#_2018-07-26_17_14_28_70036217:33
chkumar|trekk mirror.us-west-1.packethost.openstack.org:8080; No route to host17:34
clarkbhrm did that mirror die again17:34
clarkbyes seems so ugh17:34
pabelangerlooking17:34
clarkbpabelanger: I think it may need a reboot again? I'm tempted to build a new one so that nova stops trying to be smart with it17:34
clarkbits like once a day nova checks and goes I don't know about you (yes you do silly nova) and it shelves it17:35
pabelangerclarkb: sure, lets reboot and I can help build the replacement17:35
pabelangerthen change dns17:35
clarkbok I'm rebooting it now17:35
*** kjackal has joined #openstack-infra17:35
pabelangerk, let me get coffee and I'll build mirror0217:35
*** myoung|lunch is now known as myoung17:35
*** felipemonteiro has joined #openstack-infra17:36
*** e0ne has quit IRC17:37
clarkbhamerins: as a first step I would try to add logging around the ceph pods so that we can see why they are failing and increasing the logging level for openstack services to debug and compress the log files17:37
clarkbhamerins: that should make all of this much more debuggable17:37
clarkbchkumar|trekk: it should be working again now17:38
chkumar|trekkclarkb: thanks :-)17:38
clarkbpabelanger: and maybe don't delete mirror01 so that john can try and debug why nova was confused about it17:39
pabelangerclarkb: wfm17:40
*** felipemonteiro has quit IRC17:41
*** jamesmcarthur has joined #openstack-infra17:45
*** edmondsw has joined #openstack-infra17:48
*** jamesmcarthur has quit IRC17:52
*** baoli has joined #openstack-infra17:52
*** jamesmcarthur has joined #openstack-infra17:56
*** DuncanT_ has left #openstack-infra17:58
*** jamesmcarthur has quit IRC18:00
*** dklyle has quit IRC18:01
clarkbpabelanger: looks like we are also mostly just building and failing to build instances in packethost again18:01
pabelangerclarkb: yah, having issues getting new mirror online18:01
pabelangerout of space again?18:01
clarkbtimeout waiting for server to come up again18:01
clarkbmaybe?18:01
pabelangeryah, I get timeout out mirror too18:02
pabelangerseems to have just started18:02
pabelanger45mins ago18:02
clarkbpabelanger: the new centos image there is raw fwiw18:03
clarkbso your change took effect18:03
clarkband it was uploaded 6 hours ago18:03
clarkbpabelanger: image list doesn't show us leaking images so its got to be on the converted cache side if it is image leaks18:05
clarkbpabelanger: doyou want to email john with this info and the update that we've switched to raw uploads? (I think you have his email address)18:05
pabelangersure18:06
clarkbpabelanger: looks like at least some of the in use nodes are centos. It is possible that maybe the raw image has it working for those? I dunno18:07
*** kgiusti has quit IRC18:08
clarkbbut there are new in use xenial nodes too and I don't think those are raw yet (image is ~20 hours old)18:09
clarkbya confirmed it is still qcow218:11
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Restructure return values section in docs  https://review.openstack.org/58598118:18
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support job pause  https://review.openstack.org/58538918:18
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support job pause  https://review.openstack.org/58538918:19
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Restructure return values section in docs  https://review.openstack.org/58598118:20
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support job pause  https://review.openstack.org/58538918:21
*** e0ne has joined #openstack-infra18:22
*** kjackal has quit IRC18:26
*** kgiusti has joined #openstack-infra18:27
clarkbmgagne: hey, what was that thing where IPs were being reused or arp was not in agreement over what nova/neutron were assigning to VMs in the inap cloud? I think maybe that is happening again18:40
clarkbmgagne: http://logs.openstack.org/40/568940/9/check/ironic-tempest-dsvm-functional-python3/9e5bf50/job-output.txt.gz#_2018-07-26_18_32_21_369151 after jobs has started it starts complaining that the ssh host key has changed out from under it and it appears that inap is doing this more than other clouds. Could be explained by arp not pointing to proper VM18:41
*** baoli has quit IRC18:41
*** mgoddard has quit IRC18:45
mgagneclarkb: could be a rogue instance18:53
mgagnelet me check, I have a couple of free minutes right now18:55
clarkbthank you18:55
*** tosky has quit IRC18:57
openstackgerritMerged openstack-infra/zuul master: Support complex data structures as secrets  https://review.openstack.org/58589218:57
*** anteaya has joined #openstack-infra18:57
openstackgerritMerged openstack-infra/nodepool master: Use pipelines for stats keys  https://review.openstack.org/58557718:57
pabelangerclarkb: sorry, stepped away, but also looks to be recovering18:59
pabelangertrying mirror again now18:59
mgagneclarkb: I could only find the IP address on one compute node and the associated instance still exist. So I'm not sure what to do. Maybe the rogue instance got deleted?19:00
clarkbmgagne: that is possible19:01
clarkbmgagne: let me check a couple other logs really quickly to see if they have different IPs19:01
*** hamzy has quit IRC19:02
mriedemi've noticed that 564440 keeps getting re-queued into the check queue,19:02
*** hamzy_ has joined #openstack-infra19:02
mriedemeven though it's been in the gate queue twice today19:02
mriedemi'm assuming b/c of rechecks?19:02
clarkbmgagne: what is the hostname of the instance that has the IP now?19:03
clarkbmgagne: 198.72.124.171 was the issue IP in a couple cases according to logstsah19:03
mgagneubuntu-xenial-inap-mtl01-000099466719:04
clarkbmriedem: yes it failed in the gate multiple times so when rechecked it has to go through check to get the +1 before it can go back to the gate19:04
mgagnenow it's on ubuntu-xenial-inap-mtl01-000099507019:06
mriedemyeah it's just weird to see it in both19:06
clarkbmgagne: ok that is currently a valid instnace19:06
clarkbmriedem: hrm ya that is odd19:07
mgagneI think I found other rogue instances19:07
mgagnelet me check those19:07
clarkbmriedem: I wonder if the recheck caught it at a specific time where it could enqueue to both19:07
*** ihar has quit IRC19:08
*** agopi is now known as agopi|brb19:11
mgagneclarkb: I deleted ~10 rogue instances19:12
clarkbmgagne: thank you for looking19:12
clarkband for cleaning those up19:13
mgagneyw19:13
pabelangerclarkb: hmm, I think we might only have a single FIP in mirror tenant19:14
clarkbpabelanger: is that a quota thing?19:14
pabelangerno, just not attaching a pubic ip19:14
pabelangerchecking to see how many we have how19:14
*** ianychoi_ has joined #openstack-infra19:15
clarkblooks like quota is 10 and it doens't thin kwe are using any of them19:15
clarkbyou cna try to manually make an fip (just delete it when done) to check if it works by hand19:15
*** pcaruana has joined #openstack-infra19:17
*** mdrabe has joined #openstack-infra19:18
*** ianychoi has quit IRC19:18
pabelanger$ openstack floating ip create openstackci-net19:18
pabelangerI get back19:18
pabelanger400 Bad floatingip request: Network 02f9b052-12d9-48a7-82cb-0c520b8bf894 is not a valid external network19:19
pabelangerchecking why19:19
clarkbpabelanger: that is the private ip network that you attach the fips to, I think external is where fips are sources19:19
pabelangerclarkb: that is the network that mirror01 is using now19:19
pabelangerlet me check where its FIPs is19:20
clarkbpabelanger: yes for its private ip19:20
openstackgerritNate Johnston proposed openstack/gertty master: Add zuul v3 style job to run pyflakes  https://review.openstack.org/58187219:20
clarkbthe fip is from external (I just confirmed by subnet showing its subnet and that is where the fip allocation range is)19:20
pabelangeropenstackci-net=192.168.1.11, 147.75.38.15819:20
pabelangerlooks confusing19:20
*** bobh_ has quit IRC19:21
*** r-daneel has quit IRC19:21
openstackgerritNate Johnston proposed openstack/gertty master: Add zuul v3 style job to run pyflakes  https://review.openstack.org/58187219:21
*** r-daneel has joined #openstack-infra19:22
clarkbpabelanger: network show external shows that this is the subnet subnet show 64fd0d1f-61a9-47e6-9211-940b161bebe9 and if you show that subnet you will see the fip allocation range19:22
*** jamesmcarthur has joined #openstack-infra19:28
pabelangerk, testing again19:29
clarkbok I'm going to step out a bit for lunch now, back in a bit19:31
clarkbthings look mostly under control19:31
pabelangerclarkb: oh, launch-node.py worked this time19:33
pabelangernot sure what changed19:33
*** kgiusti has quit IRC19:35
mriedemif i'm looking at https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/ceph.yaml#n19 and stats_counts.zuul.pipeline.check.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.FAILURE compared to http://graphite.openstack.org/ that's broken because stats_counts.zuul.pipeline.check.job no longer exists in the tree,19:39
mriedemdid something replace that?19:39
clarkbmriedem: things moved with zuulv3 I dont know if that moved19:40
clarkbcheck the neutron graphs as I believe they were updated to work19:41
mriedemok, i'm not sure how else to find and compare legacy-tempest-dsvm-full-devstack-plugin-ceph to tempest-full failure rates to see if we can make the former voting now19:41
corvusmriedem: yeah, a few things changed... what project does this run on?19:41
mriedemat least cinder/glance/nova/tempest in check queue non-voting19:42
mriedemoh i see https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/neutron.yaml19:42
mriedemmaybe i can model after that19:42
*** kgiusti has joined #openstack-infra19:42
corvusstats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_nova.master.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.SUCCESS19:43
mriedemyup19:43
corvusmriedem: there's one :)19:43
mriedemthanks, was just getting there19:43
*** trown|lunch is now known as trown19:43
*** ccamacho1 has quit IRC19:43
corvusmriedem: the biggest thing is that all the stats are in context of a project (admittedly, makes less sense for this, but more sense for "tox-py36").  second biggest thing is adding the tenant.19:43
*** bobh has joined #openstack-infra19:44
corvusmriedem: graphite will let you combine them, so you can replace nova with {nova,glance,...}19:44
corvusor even *, though that may be slow at that point in the hierarchy19:45
*** pcaruana has quit IRC19:45
corvusoh, also the branch is in there now.19:45
mriedemhttp://graphite.openstack.org/render/?width=1531&height=592&_salt=1532634445.928&from=00%3A00_20180428&until=23%3A59_20180726&target=stats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_nova.master.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.FAILURE&target=stats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_nova.master.job.tempest-full.FAILURE&title=cep19:47
mriedemvs%20tempest-full%20fail%20rates%20over%203%20months19:47
mriedemthat's what i needed, thanks19:47
*** felipemonteiro has joined #openstack-infra19:48
mriedemcorvus: so i can do stats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_{nova,cinder,glance,tempest}.master.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.FAILURE ?19:50
pabelangerclarkb: okay, mirror came up19:51
corvusi expect something like that to work, yeah, though my graphite may be a little rusty.  that may give you 4 lines, but if you put it in a function (like sumseries) it should add them up i think19:51
pabelangerclarkb: which virtualenv should I use for rackdns19:51
pabelangerto ensure it works via CLI again19:51
*** dklyle has joined #openstack-infra19:51
corvusmriedem: yeah, i guess my graphite isn't rusty.  that's exactly what happens.  sumSeries is the function to use if you want to combine them.19:52
corvusmriedem: eg: http://graphite.openstack.org/render/?width=586&height=308&_salt=1532634806.158&target=sumSeries(stats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_%7Bnova%2Ccinder%2Cglance%2Ctempest%7D.master.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.FAILURE)&from=-3months19:53
corvusmriedem: vs: http://graphite.openstack.org/render/?width=586&height=308&_salt=1532634806.158&target=stats_counts.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_{nova%2Ccinder%2Cglance%2Ctempest}.master.job.legacy-tempest-dsvm-full-devstack-plugin-ceph.FAILURE&from=-3months19:54
*** anteaya has quit IRC19:54
mriedemyup i was able to use the graphite UI to apply the sumseries to the 319:54
clarkbpabelanger: /home/clarkb/rackdns-venv19:55
clarkbon puppetmaster19:55
pabelangerkk19:55
clarkbif that works I think its only a cert issue19:55
corvusmriedem: should be able to do similar things with the branch if that's an issue19:55
mriedemyeah, just care about master right now19:55
pabelangerclarkb: worked19:56
mriedemand i need to fix https://git.openstack.org/cgit/openstack-infra/project-config/tree/grafana/ceph.yaml#n1919:56
pabelangermirror02.us-west-1.packethost.openstack.org online19:56
pabelangerclarkb: but no rdns19:56
openstackgerritPaul Belanger proposed openstack-infra/system-config master: Replace mirror01.us-west-1.packethost.openstack.org  https://review.openstack.org/58633719:58
*** yamamoto has joined #openstack-infra19:59
*** agopi|brb is now known as agopi20:00
*** chinna100 has joined #openstack-infra20:01
openstackgerritMerged openstack-infra/zuul master: Restructure return values section in docs  https://review.openstack.org/58598120:01
*** hashar has joined #openstack-infra20:02
*** yamamoto has quit IRC20:03
clarkbpabelanger: but forward worked?20:04
*** jamesmcarthur has quit IRC20:05
*** eharney has quit IRC20:05
pabelangerclarkb: yup!20:06
pabelangerclarkb: how did we setup rdns for mirror01? email john I guess20:09
hamerinsclarkb: zuul run completed without any issues this time.20:10
*** Swimingly has quit IRC20:13
*** e0ne has quit IRC20:13
pabelangerclarkb: okay, email sent. Are we ready to update dns for mirror.us-west-1.packethost.openstack.org to mirror02?20:13
pabelangerclarkb: I'll grab some food now, but will be back in a few hours to follow up on new mirror20:17
clarkbpabelanger: we havent set up rdns for all nodes outside rax, just fir vexxhost I think20:20
clarkbpabelanger: I'm ready to update dns cname when you are20:21
clarkbhamerins: ok, as I said, without more logging its really hard to dig into the failures. Can we add more logging so that it is there if/when it fails again?20:29
*** kgiusti has left #openstack-infra20:32
*** felipemonteiro has quit IRC20:33
*** felipemonteiro has joined #openstack-infra20:33
*** diablo_rojo has joined #openstack-infra20:34
*** mdrabe has quit IRC20:36
corvusclarkb: do you happen to recall the blockers for enabling the opendaylight connection?20:38
corvushttps://review.openstack.org/57008720:38
clarkbcorvus: the ssh host key verification doesn't work for the ed525119 key iirc20:38
clarkbso we had to hard set an rsa key or something20:39
clarkbbasically their gerrit advertises a host key that they don't handle properly so we have to have zuul check for a different key and I think it will do that if we set a known hosts entry for it as rsa20:39
clarkbI think we thought we needed to change zuuls use of paramiko but paramiko does the right thing if we give it the right hints20:42
clarkbiirc20:42
corvusi'm re-reading http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-06-07.log.html to refresh my memory20:42
*** kjackal has joined #openstack-infra20:47
corvusclarkb: yep, i think you got it.  your change https://review.openstack.org/573397 added known_hosts to the scheduler, so once we add odls rsa key to that (and all other places), it should work this time.20:48
clarkb++20:49
corvusi'm working on a revised change to add that20:49
*** mdrabe has joined #openstack-infra20:54
*** rh-jelabarre has quit IRC20:56
openstackgerritJames E. Blair proposed openstack-infra/system-config master: Add opendaylight connection to zuul  https://review.openstack.org/58634920:56
*** hamerins has quit IRC20:57
*** hamerins has joined #openstack-infra20:58
*** chinna100 has quit IRC20:58
*** r-daneel_ has joined #openstack-infra20:59
*** e0ne has joined #openstack-infra20:59
*** e0ne has quit IRC21:00
*** r-daneel has quit IRC21:00
*** r-daneel_ is now known as r-daneel21:00
clarkbcorvus: that looks right to me21:01
*** trown is now known as trown|outtypewww21:03
*** erlon_ has quit IRC21:05
mordredcorvus, clarkb: also lgtm - should I wait to +A?21:11
*** rfolco|ruck is now known as rfolco|off21:11
clarkbI don't think we need to wait, it requires a zuul restart to pick that change up right?21:12
corvusyeah.  lets go ahead and +A it so it'll be there for the next restart21:12
corvuswhich... hopefully... soon.  :)21:12
mordredthe next restart is going to be exciting21:13
corvusyeah, not restarting during crunch time is kind of a two-edged sword, eh?21:13
mordredyah21:13
*** chinna100 has joined #openstack-infra21:14
*** jcoufal has quit IRC21:15
*** r-daneel has quit IRC21:17
mordredheck - at the rate we're going, we're going to have project-vars, extra-vars, pause and complex-secrets21:18
*** mdrabe has quit IRC21:19
*** kjackal has quit IRC21:23
*** hashar has quit IRC21:23
bkeroMight as well get some -e @more-vars.yml -e even_more=vars21:31
*** hamerins has left #openstack-infra21:33
*** hamerins has joined #openstack-infra21:34
*** hamerins has left #openstack-infra21:34
openstackgerritMerged openstack-infra/zuul master: Add support for Ansible extra-vars flag  https://review.openstack.org/54647421:34
openstackgerritMerged openstack-infra/zuul master: Rename didAnyJobFail() to hasAnyJobFailed()  https://review.openstack.org/58513421:34
clarkbwe appear on the tail end of the daily demand though21:35
clarkbmight be able to do a restart soonish depending on smcginnis' plans21:35
smcginnisWhat is the change?21:39
clarkbsmcginnis: restarting zuul to pick up the weeks worth of changes we've put off sothat feature freeze can happen :)21:39
smcginnisCan we wait until tomorrow for that? There's still R3 and client release patches trickling in.21:40
clarkbsmcginnis: you expect that will happen through the end of pacific caost day? ( I guess tonyb will shepherd?)21:41
*** wolverineav has quit IRC21:42
smcginnisclarkb: Yeah, there always seems to be a few stragglers that have to get those last one or two patches in before they can release.21:43
*** edmondsw has quit IRC21:43
mordredsmcginnis: I feel like you're talking about me21:43
*** wolverineav has joined #openstack-infra21:43
smcginnisI'm not naming any names! :D21:43
tonybclarkb: Yeah I'll be watching it today21:43
mordredsmcginnis: like, maybe you're suggesting that I don't wait until the last day to start trying to land essential patches or something crazy like that ...21:43
smcginnisI know, I know... crazy talk.21:44
*** edmondsw has joined #openstack-infra21:45
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Add variables to project  https://review.openstack.org/58423021:47
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Use definition list for job status  https://review.openstack.org/58478021:47
*** wolverineav has quit IRC21:48
*** bandini has quit IRC21:48
*** bandini has joined #openstack-infra21:48
mordredclarkb: feel like reviewing a random patch? https://review.openstack.org/#/c/585411/21:48
*** edmondsw has quit IRC21:49
clarkbmordred: for the test in that the 'missing' service type is going to check the permissive code path ya?21:51
*** slaweq has quit IRC21:52
mordredclarkb: yah21:52
clarkbok approved21:52
mordredclarkb: there's a sort of silly matrix of testscenarios in there21:52
mordredclarkb: thanks!21:52
mordredclarkb: that repo is low-traffic enough that it doesn't get much in the way of drive-by reviews21:53
clarkbya after I asked you I found that official was set in the scenario too21:53
*** bobh has quit IRC21:58
openstackgerritMerged openstack-infra/system-config master: Add opendaylight connection to zuul  https://review.openstack.org/58634922:01
tonybI notice that the post pipeline is pretty long (132) ATM and the election publish-static job is been queued fro 8+ hours.  Is that an issue or do I just need to be more patient ;P22:04
*** benj_ has quit IRC22:06
*** benj_ has joined #openstack-infra22:06
clarkbtonyb: I think you just need to be patient :) queues were really long earlier in the day and we are just getting to the point where post is processing22:06
tonybclarkb: okay22:07
*** Sukhdev has quit IRC22:08
clarkbtonyb: http://grafana.openstack.org/d/T6vSHcSik/zuul-status?orgId=1 the node requests graph gives you a sense of the backlog22:09
tonybclarkb: Ahh cool.  I'll bookmark that for next time22:11
*** wolverineav has joined #openstack-infra22:13
*** mriedem is now known as mriedem_afk22:14
*** wolverineav has quit IRC22:16
*** wolverineav has joined #openstack-infra22:16
*** wolverineav has quit IRC22:16
*** rcernin has joined #openstack-infra22:19
*** shrircha_ has joined #openstack-infra22:19
*** shrircha has quit IRC22:23
pabelangerclarkb: back22:27
clarkbpabelanger: I've not udpated DNS in case there was anything else you wanted to do prior to that, but responded to John's questions and checked that the new server loads things over http22:27
pabelangerclarkb: ack, I'll do dns update now22:28
pabelangerclarkb: cname updated22:30
clarkbpabelanger: I'm seeing traffic to the ubuntu mirror on it now22:31
pabelangerclarkb: and some traffic already hitting server22:31
pabelanger++22:31
pabelangerclarkb: https://review.openstack.org/586337/22:31
pabelangercacti.o.o update22:32
openstackgerritMerged openstack-infra/zuul master: Docs: add a cross-ref in secrets  https://review.openstack.org/58330122:32
clarkbpabelanger: most images should be transitioned to raw now too. Looks like just waiting on fedoras and bionic?22:33
*** wolverineav has joined #openstack-infra22:35
pabelangercool22:35
pabelanger#status log mirror.us-west-1.packethost.openstack.org cname updated to mirror02.us-west-1.packethost.openstack.org22:45
*** pabelanger has quit IRC22:46
*** pabelanger has joined #openstack-infra22:46
pabelanger#status log mirror.us-west-1.packethost.openstack.org cname updated to mirror02.us-west-1.packethost.openstack.org22:46
openstackstatuspabelanger: finished logging22:46
*** mriedem_afk is now known as mriedem22:47
*** rpioso is now known as rpioso|afk22:48
*** caphrim007 has joined #openstack-infra22:48
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Report to gerrit over HTTP  https://review.openstack.org/57702722:51
openstackgerritMerged openstack-infra/zuul master: Add variables to project  https://review.openstack.org/58423022:54
*** bdodd has quit IRC22:56
*** caphrim007_ has joined #openstack-infra22:56
*** bdodd has joined #openstack-infra22:57
*** caphrim007 has quit IRC22:58
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Report to gerrit over HTTP  https://review.openstack.org/57702723:00
*** hongbin has quit IRC23:00
*** lbragstad_ has joined #openstack-infra23:01
openstackgerritMerged openstack-infra/zuul master: Use definition list for job status  https://review.openstack.org/58478023:01
*** lbragstad has quit IRC23:01
*** knikolla has quit IRC23:01
*** boden has quit IRC23:02
clarkbtonyb: we seem to have largely caught up now23:03
tonybclarkb: Oh cool that was quicker then I'd expected.23:04
*** mschuppert has quit IRC23:04
tonybclarkb: Hmm the election site hasn't updated I'll check the post jobs for errors after I've dismantled my daughters bed23:05
clarkbtonyb: let me know if I can help23:05
clarkbwith the zuul stuff, the bed is on wrong side of planet23:06
corvus20:04 <@mst> we're getting hit by a substantial botnet that isn't re-using IPs noticeably, so while we're playing whack-a-troll as fast as we can, it's imperfect at best23:07
corvusin case anyone was wondering ^23:07
tonybclarkb: :)23:09
clarkbcorvus: ya the number of IPs is impressive23:09
*** jamesmcarthur has joined #openstack-infra23:12
*** jamesmcarthur has quit IRC23:16
*** tpsilva has quit IRC23:19
*** wolverineav has quit IRC23:23
*** david-lyle has joined #openstack-infra23:24
*** dklyle has quit IRC23:26
*** knikolla has joined #openstack-infra23:37
tonybclarkb: ... no problem just more tonyb checks too soon and the things haven't actually finished :/23:42
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Report to gerrit over HTTP  https://review.openstack.org/57702723:43
*** codenix has joined #openstack-infra23:48
*** felipemonteiro has quit IRC23:50
*** stakeda has joined #openstack-infra23:50
*** david-lyle has quit IRC23:51
*** dklyle has joined #openstack-infra23:52
clarkbI thin kwe are fully on new mirror now23:53
clarkbon packethost23:53
*** bobh has joined #openstack-infra23:57
*** felipemonteiro has joined #openstack-infra23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!