| mnaser | :) | 00:00 |
|---|---|---|
| openstackgerrit | Merged openstack/diskimage-builder master: Fix typo in CMDLINE env var for bootloader https://review.openstack.org/448508 | 00:02 |
| *** eroux has quit IRC | 00:03 | |
| *** gongysh has joined #openstack-infra | 00:03 | |
| *** gcb has quit IRC | 00:05 | |
| *** mriedem has quit IRC | 00:08 | |
| *** esikachev has joined #openstack-infra | 00:18 | |
| *** masber has quit IRC | 00:21 | |
| *** baoli_ has joined #openstack-infra | 00:21 | |
| *** esikachev has quit IRC | 00:22 | |
| *** baoli has quit IRC | 00:25 | |
| *** gouthamr has joined #openstack-infra | 00:27 | |
| *** xarses has joined #openstack-infra | 00:27 | |
| *** aeng has quit IRC | 00:27 | |
| *** rbuzatu has joined #openstack-infra | 00:30 | |
| *** kgiusti has left #openstack-infra | 00:31 | |
| *** ssbarnea has quit IRC | 00:33 | |
| *** adrian_otto has joined #openstack-infra | 00:33 | |
| *** ihrachys has joined #openstack-infra | 00:41 | |
| *** zhurong has joined #openstack-infra | 00:42 | |
| *** aeng has joined #openstack-infra | 00:44 | |
| *** rbuzatu has quit IRC | 00:44 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Query changes only via relevant sources https://review.openstack.org/448257 | 00:45 |
| *** wolverineav has quit IRC | 00:50 | |
| *** knangia has quit IRC | 00:51 | |
| *** wolverineav has joined #openstack-infra | 00:52 | |
| *** Goneri has quit IRC | 00:53 | |
| *** Julien-zte has joined #openstack-infra | 00:55 | |
| *** wolverineav has quit IRC | 00:56 | |
| *** EricGonczer_ has joined #openstack-infra | 00:56 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: functests: skip qcow2 generically but add specific test https://review.openstack.org/448837 | 00:57 |
| *** adrian_otto has quit IRC | 00:59 | |
| *** lifeless_ is now known as lifeless | 01:00 | |
| *** aeng has quit IRC | 01:00 | |
| sdake | hey folks, I just wanted to double check, I believe the answer is no, however, are third party gates permitted to be voting? | 01:00 |
| openstackgerrit | Merged openstack/diskimage-builder master: Capture output in _exec_sudo https://review.openstack.org/448810 | 01:01 |
| *** baoli has joined #openstack-infra | 01:02 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Add default PPC block-device layout https://review.openstack.org/448811 | 01:03 |
| clarkb | sdake: they canvote but they cant gate. So they get +/-1 not +/-2 | 01:03 |
| sdake | clarkb thanks - that matches my understanding however I may have miscommunicated that - i'll have to sync up with lazyPower in the morning | 01:04 |
| *** liujiong has joined #openstack-infra | 01:04 | |
| sdake | clarkb I said they would be nonvoting - which to me means +1/-1 whereas voting is +2/-2 | 01:04 |
| sdake | clarkb but didn't expalin the +1/-1 +2/-2 delta | 01:05 |
| sdake | clarkb I guess I just got the terminology wrong - they can check but they can't gate | 01:05 |
| *** baoli_ has quit IRC | 01:05 | |
| clarkb | sdake: in our terminology voting means a job affects the vote eg +/-1 gating is any job that prevents merges on failure | 01:06 |
| sdake | clarkb got it | 01:06 |
| sdake | clarkb why does project-config have the -nv flag? | 01:06 |
| sdake | (non-voting) | 01:06 |
| sdake | should it not be ng (non-gating?) | 01:06 |
| sdake | clarkb fwiw not being difficult just curiosu if you know how that came about | 01:07 |
| clarkb | -nv doesnt say anything about gating, only voting | 01:07 |
| *** tphummel has joined #openstack-infra | 01:07 | |
| sdake | perhaps we are doing it wrong in kolla | 01:07 |
| sdake | all our non-gating jobs end in -nv | 01:07 |
| clarkb | (transitively you cant gate if you dont vote but its specofically not voting there) | 01:07 |
| sdake | all our gating jobs end in nothing :) | 01:08 |
| clarkb | yes because a non voting job cant gate | 01:08 |
| mnaser | gating jobs should always end in nothing (i think?) | 01:08 |
| mnaser | ^ | 01:08 |
| sdake | mnaser right - no suffix | 01:08 |
| *** armax has quit IRC | 01:08 | |
| clarkb | but thats derivative the first orser is non voting | 01:08 |
| mnaser | but we have -nv and non -nv in check pipeline tho | 01:08 |
| *** ihrachys has quit IRC | 01:08 | |
| mnaser | our build jobs have no -nv so they have to pass to get the +1 | 01:08 |
| mnaser | deploy jobs can fail and not get the -1 to move onto the gate | 01:09 |
| sdake | mnaser thanks that clarifies things | 01:09 |
| sdake | clarkb thanks as well :) | 01:09 |
| mnaser | ex: gate-kolla-dsvm-build-centos-source-centos-7 => if this fails, patch gets -1 | 01:09 |
| mnaser | however: gate-kolla-dsvm-deploy-centos-source-centos-7-nv => if this fails, but all with no -nv pass, patch doesnt get -1 | 01:09 |
| sdake | -nv means non-voting on the check stage | 01:10 |
| sdake | TIL :) | 01:10 |
| mnaser | sdake if you like looking at things - http://status.openstack.org/zuul/ | 01:10 |
| sdake | mnaser i've seen zuul a million times | 01:10 |
| mnaser | i usually throw my patches there and watch them chug along, so thats how i caught the (not voting) listed there | 01:10 |
| sdake | mnaser and have an intuitive understanding of how it all works | 01:10 |
| sdake | mnaser just want to use the correct terminology mostly with lazyPower as he wants to setup a kolla-kubernetes third party ci | 01:11 |
| mnaser | mhmm | 01:11 |
| sdake | mnaser and I think I got it wrong this morning in our meeting | 01:11 |
| sdake | not sure if he was in here afterwards - my understanding was he talked with infra but -ETOOTIREDTOLOOKATLOGS :) | 01:12 |
| sdake | on that note, nap time | 01:12 |
| sdake | bbiafhours | 01:12 |
| *** aeng has joined #openstack-infra | 01:13 | |
| *** esberglu has quit IRC | 01:13 | |
| *** gouthamr has quit IRC | 01:16 | |
| *** zhurong has quit IRC | 01:23 | |
| *** mat128 has joined #openstack-infra | 01:23 | |
| *** MasterOfBugs has quit IRC | 01:23 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: functests: skip qcow2 generically but add specific test https://review.openstack.org/448837 | 01:24 |
| *** yamahata has quit IRC | 01:27 | |
| *** rbuzatu has joined #openstack-infra | 01:27 | |
| *** masber has joined #openstack-infra | 01:29 | |
| *** wolverineav has joined #openstack-infra | 01:31 | |
| *** rbuzatu has quit IRC | 01:31 | |
| *** kaisers1 has joined #openstack-infra | 01:32 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 01:33 |
| *** kaisers has quit IRC | 01:34 | |
| *** Apoorva_ has joined #openstack-infra | 01:36 | |
| mnaser | hey infra.. logstash job queue is rising non stop to a peak of 100k | 01:37 |
| mnaser | is that.. okay? | 01:37 |
| *** dave-mccowan has quit IRC | 01:37 | |
| mnaser | it doesnt look like its ever peaked this high (without going down) | 01:38 |
| *** yanyanhu has joined #openstack-infra | 01:38 | |
| *** dave-mccowan has joined #openstack-infra | 01:39 | |
| *** tphummel has quit IRC | 01:40 | |
| *** Apoorva has quit IRC | 01:40 | |
| *** gongysh has quit IRC | 01:40 | |
| *** Apoorva_ has quit IRC | 01:40 | |
| *** zhurong has joined #openstack-infra | 01:43 | |
| *** yanyanhu has quit IRC | 01:43 | |
| *** manjeets has joined #openstack-infra | 01:46 | |
| *** yanyanhu has joined #openstack-infra | 01:46 | |
| *** ianychoi has quit IRC | 01:48 | |
| *** masber has quit IRC | 01:49 | |
| *** masber has joined #openstack-infra | 01:53 | |
| *** dharinic- has joined #openstack-infra | 01:53 | |
| *** ianychoi has joined #openstack-infra | 01:54 | |
| *** mat128 has quit IRC | 01:56 | |
| *** Sukhdev_ has quit IRC | 01:56 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack/diskimage-builder master: Use correct Ubuntu distro url on non-x86 arches https://review.openstack.org/448848 | 01:57 |
| *** esikachev has joined #openstack-infra | 02:03 | |
| *** esikachev has quit IRC | 02:08 | |
| *** dave-mcc_ has joined #openstack-infra | 02:08 | |
| *** dave-mccowan has quit IRC | 02:10 | |
| *** dave-mccowan has joined #openstack-infra | 02:13 | |
| *** dave-mcc_ has quit IRC | 02:16 | |
| *** rfolco has quit IRC | 02:19 | |
| *** esikachev has joined #openstack-infra | 02:21 | |
| openstackgerrit | Samuel Cassiba proposed openstack-infra/project-config master: Chef: adds basic multinode integration tests https://review.openstack.org/446294 | 02:21 |
| *** hongbin has joined #openstack-infra | 02:23 | |
| *** rbuzatu has joined #openstack-infra | 02:24 | |
| *** esikachev has quit IRC | 02:25 | |
| *** mozart89 has joined #openstack-infra | 02:26 | |
| *** ramishra has joined #openstack-infra | 02:30 | |
| *** beslemon has quit IRC | 02:33 | |
| *** esikachev has joined #openstack-infra | 02:34 | |
| *** rbuzatu_ has joined #openstack-infra | 02:35 | |
| *** gongysh has joined #openstack-infra | 02:36 | |
| *** rbuzatu__ has joined #openstack-infra | 02:37 | |
| *** dharinic- is now known as dharinic | 02:37 | |
| *** rbuzatu has quit IRC | 02:38 | |
| *** rbuzatu has joined #openstack-infra | 02:38 | |
| *** esikachev has quit IRC | 02:39 | |
| *** rbuzatu_ has quit IRC | 02:40 | |
| *** esikachev has joined #openstack-infra | 02:40 | |
| *** ihrachys has joined #openstack-infra | 02:41 | |
| *** rbuzatu__ has quit IRC | 02:41 | |
| *** rbuzatu has quit IRC | 02:43 | |
| *** eliqiao has joined #openstack-infra | 02:43 | |
| jiaohaolin1 | hey guys. is there any guide of CI environment? I have built a master node ,but slave node is still not work? | 02:45 |
| *** csomerville has joined #openstack-infra | 02:46 | |
| *** esikachev has quit IRC | 02:47 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: functests: skip qcow2 generically but add specific test https://review.openstack.org/448837 | 02:49 |
| *** cody-somerville has quit IRC | 02:50 | |
| *** wolverin_ has joined #openstack-infra | 02:51 | |
| *** csomerville has quit IRC | 02:52 | |
| *** wolverineav has quit IRC | 02:54 | |
| *** masber has quit IRC | 02:56 | |
| *** gouthamr has joined #openstack-infra | 02:59 | |
| *** tuanluong has joined #openstack-infra | 03:00 | |
| *** lihi has quit IRC | 03:06 | |
| *** nicolasbock has quit IRC | 03:06 | |
| *** lihi has joined #openstack-infra | 03:07 | |
| *** dimak has quit IRC | 03:13 | |
| *** dimak has joined #openstack-infra | 03:13 | |
| *** esikachev has joined #openstack-infra | 03:14 | |
| *** beslemon has joined #openstack-infra | 03:17 | |
| *** gouthamr has quit IRC | 03:19 | |
| *** esikachev has quit IRC | 03:21 | |
| *** baoli has quit IRC | 03:22 | |
| *** esikachev has joined #openstack-infra | 03:26 | |
| *** beslemon has quit IRC | 03:29 | |
| *** esikachev has quit IRC | 03:31 | |
| *** uberjay has quit IRC | 03:31 | |
| *** esikachev has joined #openstack-infra | 03:31 | |
| *** uberjay has joined #openstack-infra | 03:32 | |
| *** Sukhdev has joined #openstack-infra | 03:33 | |
| *** hongbin has quit IRC | 03:37 | |
| *** links has joined #openstack-infra | 03:37 | |
| *** amotoki has quit IRC | 03:37 | |
| *** rbuzatu has joined #openstack-infra | 03:39 | |
| *** esikachev has quit IRC | 03:40 | |
| *** wolverineav has joined #openstack-infra | 03:44 | |
| *** zhurong has quit IRC | 03:48 | |
| *** wolverin_ has quit IRC | 03:48 | |
| *** masber has joined #openstack-infra | 03:48 | |
| *** armax has joined #openstack-infra | 03:49 | |
| *** dave-mccowan has quit IRC | 03:50 | |
| *** amotoki has joined #openstack-infra | 03:50 | |
| *** sflanigan_afk has quit IRC | 03:52 | |
| *** knangia has joined #openstack-infra | 03:54 | |
| *** amotoki has quit IRC | 03:55 | |
| *** rbuzatu has quit IRC | 03:55 | |
| *** dhajare has joined #openstack-infra | 03:58 | |
| *** amotoki has joined #openstack-infra | 03:59 | |
| *** sflanigan has joined #openstack-infra | 04:01 | |
| *** anilvenkata has joined #openstack-infra | 04:07 | |
| *** sree has joined #openstack-infra | 04:08 | |
| *** esikachev has joined #openstack-infra | 04:09 | |
| *** tellesnobrega has quit IRC | 04:12 | |
| *** esikachev has quit IRC | 04:15 | |
| *** tellesnobrega has joined #openstack-infra | 04:18 | |
| *** amotoki has quit IRC | 04:19 | |
| *** wolverineav has quit IRC | 04:23 | |
| *** wolverineav has joined #openstack-infra | 04:24 | |
| *** wolverin_ has joined #openstack-infra | 04:26 | |
| *** psachin has joined #openstack-infra | 04:28 | |
| *** amotoki has joined #openstack-infra | 04:28 | |
| *** wolverineav has quit IRC | 04:28 | |
| *** dhajare has quit IRC | 04:29 | |
| *** wolverin_ has quit IRC | 04:30 | |
| *** yee379 has quit IRC | 04:30 | |
| *** yee37914 has joined #openstack-infra | 04:31 | |
| *** esikachev has joined #openstack-infra | 04:31 | |
| *** liujiong has quit IRC | 04:31 | |
| *** liujiong has joined #openstack-infra | 04:33 | |
| *** gongysh has quit IRC | 04:34 | |
| *** esikachev has quit IRC | 04:36 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/irc-meetings master: Revert "Adjust time of neutron meeting and neutron-drivers" https://review.openstack.org/448886 | 04:40 |
| *** sflanigan has quit IRC | 04:40 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/irc-meetings master: Revert "Adjust time of neutron meeting and neutron-drivers" https://review.openstack.org/448886 | 04:40 |
| *** masber has quit IRC | 04:40 | |
| *** sflanigan has joined #openstack-infra | 04:41 | |
| *** esikachev has joined #openstack-infra | 04:42 | |
| *** MasterOfBugs has joined #openstack-infra | 04:44 | |
| *** armax has quit IRC | 04:44 | |
| *** armax has joined #openstack-infra | 04:44 | |
| *** pgadiya has joined #openstack-infra | 04:45 | |
| *** esikachev has quit IRC | 04:46 | |
| *** wolverineav has joined #openstack-infra | 04:47 | |
| *** ihrachys has quit IRC | 04:47 | |
| *** yamahata has joined #openstack-infra | 04:48 | |
| *** wolverineav has quit IRC | 04:51 | |
| *** rbuzatu has joined #openstack-infra | 04:52 | |
| *** nmathew has joined #openstack-infra | 04:52 | |
| *** Jeffrey4l_ has quit IRC | 04:55 | |
| *** dhajare has joined #openstack-infra | 04:56 | |
| *** Jeffrey4l has joined #openstack-infra | 04:56 | |
| *** EricGonczer_ has quit IRC | 05:01 | |
| *** gongysh has joined #openstack-infra | 05:03 | |
| *** rbuzatu has quit IRC | 05:06 | |
| *** wolverineav has joined #openstack-infra | 05:08 | |
| *** markvoelker has quit IRC | 05:08 | |
| *** markvoelker has joined #openstack-infra | 05:09 | |
| *** wolverineav has quit IRC | 05:12 | |
| *** markvoelker has quit IRC | 05:13 | |
| *** gcb has joined #openstack-infra | 05:14 | |
| *** prateek has joined #openstack-infra | 05:18 | |
| *** rlandy|bbl is now known as rlandy | 05:23 | |
| *** adriant has quit IRC | 05:23 | |
| *** rlandy has quit IRC | 05:23 | |
| *** EricGonczer_ has joined #openstack-infra | 05:23 | |
| *** wolverineav has joined #openstack-infra | 05:23 | |
| *** baoli has joined #openstack-infra | 05:24 | |
| *** uberjay has quit IRC | 05:27 | |
| *** wolverineav has quit IRC | 05:28 | |
| *** baoli has quit IRC | 05:29 | |
| *** uberjay has joined #openstack-infra | 05:29 | |
| *** reed has quit IRC | 05:30 | |
| *** reed has joined #openstack-infra | 05:30 | |
| *** wolverineav has joined #openstack-infra | 05:36 | |
| *** wolverineav has quit IRC | 05:40 | |
| *** aunnam has joined #openstack-infra | 05:42 | |
| *** wolverineav has joined #openstack-infra | 05:46 | |
| *** wolverineav has quit IRC | 05:50 | |
| *** EricGonczer_ has quit IRC | 05:51 | |
| *** zhurong has joined #openstack-infra | 05:53 | |
| *** Sukhdev has quit IRC | 05:53 | |
| *** sflanigan has quit IRC | 05:54 | |
| *** sree_ has joined #openstack-infra | 05:54 | |
| *** jascott1 has quit IRC | 05:54 | |
| *** sree_ is now known as Guest90664 | 05:54 | |
| *** jascott1 has joined #openstack-infra | 05:55 | |
| *** wolverineav has joined #openstack-infra | 05:56 | |
| *** tjones has joined #openstack-infra | 05:56 | |
| *** tjones1 has quit IRC | 05:56 | |
| *** sree has quit IRC | 05:56 | |
| *** arif-ali has quit IRC | 05:56 | |
| *** arif-ali has joined #openstack-infra | 05:58 | |
| *** jascott1 has quit IRC | 05:59 | |
| *** wolverineav has quit IRC | 06:00 | |
| *** knangia has quit IRC | 06:01 | |
| *** armax has quit IRC | 06:03 | |
| *** rbuzatu has joined #openstack-infra | 06:03 | |
| *** armax has joined #openstack-infra | 06:03 | |
| *** esikachev has joined #openstack-infra | 06:04 | |
| *** sheeprine has quit IRC | 06:04 | |
| *** armax has quit IRC | 06:04 | |
| *** armax has joined #openstack-infra | 06:05 | |
| *** nadya has joined #openstack-infra | 06:05 | |
| *** armax has quit IRC | 06:05 | |
| *** wolverineav has joined #openstack-infra | 06:06 | |
| *** tonyb_ has joined #openstack-infra | 06:06 | |
| *** nadya has quit IRC | 06:08 | |
| *** askb_ has joined #openstack-infra | 06:08 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Refactor out Changeish https://review.openstack.org/448913 | 06:08 |
| *** slaweq__ has joined #openstack-infra | 06:08 | |
| *** sheeprine has joined #openstack-infra | 06:08 | |
| *** jokke__ has joined #openstack-infra | 06:09 | |
| *** markvoelker has joined #openstack-infra | 06:09 | |
| *** dutsmoc is now known as comstud | 06:09 | |
| *** tjones has quit IRC | 06:09 | |
| *** wolverineav has quit IRC | 06:10 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] Try symlinks for grub-install on PPC https://review.openstack.org/448914 | 06:10 |
| *** gcb has quit IRC | 06:11 | |
| *** gcb has joined #openstack-infra | 06:11 | |
| *** hemna_ has joined #openstack-infra | 06:12 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] Try symlinks for grub-install on PPC https://review.openstack.org/448914 | 06:12 |
| *** vgnbkr_ has joined #openstack-infra | 06:12 | |
| *** tonyb has quit IRC | 06:13 | |
| *** flaper87 has quit IRC | 06:13 | |
| *** vgnbkr has quit IRC | 06:13 | |
| *** askb has quit IRC | 06:13 | |
| *** slaweq has quit IRC | 06:13 | |
| *** purp has quit IRC | 06:13 | |
| *** hemna has quit IRC | 06:13 | |
| *** jokke_ has quit IRC | 06:13 | |
| *** vgnbkr_ is now known as vgnbkr | 06:13 | |
| *** purp has joined #openstack-infra | 06:13 | |
| *** markvoelker has quit IRC | 06:13 | |
| *** wolverineav has joined #openstack-infra | 06:16 | |
| *** isaacb has joined #openstack-infra | 06:17 | |
| *** rbuzatu has quit IRC | 06:18 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/infra-specs master: New spec for translation_check_site (Ansible AIO) https://review.openstack.org/440825 | 06:19 |
| *** wolverineav has quit IRC | 06:20 | |
| *** masber has joined #openstack-infra | 06:22 | |
| *** tonytan4ever has quit IRC | 06:26 | |
| *** wolverineav has joined #openstack-infra | 06:26 | |
| *** tonytan4ever has joined #openstack-infra | 06:26 | |
| *** jaosorior has joined #openstack-infra | 06:30 | |
| *** tonytan4ever has quit IRC | 06:31 | |
| *** wolverineav has quit IRC | 06:31 | |
| *** gildub has quit IRC | 06:33 | |
| *** pahuang has quit IRC | 06:36 | |
| *** wolverineav has joined #openstack-infra | 06:36 | |
| *** masber has quit IRC | 06:39 | |
| *** flepied has quit IRC | 06:40 | |
| *** wolverineav has quit IRC | 06:41 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Use stevedore for module config of block device https://review.openstack.org/447090 | 06:44 |
| *** wolverineav has joined #openstack-infra | 06:46 | |
| *** rwsu has joined #openstack-infra | 06:48 | |
| *** wolverineav has quit IRC | 06:50 | |
| *** amotoki_ has joined #openstack-infra | 06:52 | |
| *** gildub has joined #openstack-infra | 06:53 | |
| *** amotoki has quit IRC | 06:55 | |
| *** wolverineav has joined #openstack-infra | 06:56 | |
| *** yanyanhu_ has joined #openstack-infra | 07:00 | |
| *** yanyanhu has quit IRC | 07:00 | |
| *** wolverineav has quit IRC | 07:01 | |
| *** isq has quit IRC | 07:01 | |
| *** sbezverk_ has joined #openstack-infra | 07:01 | |
| *** kornicameister has joined #openstack-infra | 07:02 | |
| *** yanyanhu_ has quit IRC | 07:02 | |
| *** isq has joined #openstack-infra | 07:02 | |
| *** sheeprine has quit IRC | 07:02 | |
| *** abadger1999 has quit IRC | 07:02 | |
| *** jasondotstar has quit IRC | 07:03 | |
| *** kornicameister has quit IRC | 07:03 | |
| openstackgerrit | gecong proposed openstack/diskimage-builder master: Typo fix: curent => current https://review.openstack.org/448966 | 07:03 |
| *** tristanC has quit IRC | 07:03 | |
| *** reed has quit IRC | 07:03 | |
| *** dmellado has quit IRC | 07:03 | |
| *** rodrigods has quit IRC | 07:03 | |
| *** kornicameister has joined #openstack-infra | 07:03 | |
| *** rattboi has quit IRC | 07:04 | |
| *** reed has joined #openstack-infra | 07:04 | |
| *** kfox1111 has quit IRC | 07:04 | |
| *** kfox1111 has joined #openstack-infra | 07:04 | |
| *** abadger1999 has joined #openstack-infra | 07:05 | |
| *** tristanC has joined #openstack-infra | 07:05 | |
| *** rattboi has joined #openstack-infra | 07:05 | |
| *** sbezverk has quit IRC | 07:05 | |
| *** rodrigods has joined #openstack-infra | 07:06 | |
| *** yanyanhu_ has joined #openstack-infra | 07:06 | |
| *** wolverineav has joined #openstack-infra | 07:07 | |
| *** jasondotstar has joined #openstack-infra | 07:08 | |
| *** dmellado has joined #openstack-infra | 07:08 | |
| *** andreas_s has joined #openstack-infra | 07:08 | |
| *** xarses has quit IRC | 07:09 | |
| *** sheeprine has joined #openstack-infra | 07:09 | |
| *** wolverineav has quit IRC | 07:11 | |
| *** Hal has quit IRC | 07:11 | |
| *** isaacb has quit IRC | 07:13 | |
| *** rbuzatu has joined #openstack-infra | 07:14 | |
| *** florianf has joined #openstack-infra | 07:15 | |
| *** reed has quit IRC | 07:17 | |
| *** wolverineav has joined #openstack-infra | 07:17 | |
| *** reed has joined #openstack-infra | 07:19 | |
| *** rbuzatu has quit IRC | 07:19 | |
| *** aviau has quit IRC | 07:19 | |
| *** aviau has joined #openstack-infra | 07:20 | |
| *** ssbarnea has joined #openstack-infra | 07:20 | |
| *** wolverineav has quit IRC | 07:21 | |
| *** isaacb has joined #openstack-infra | 07:23 | |
| *** baoli has joined #openstack-infra | 07:25 | |
| *** wolverineav has joined #openstack-infra | 07:26 | |
| *** markus_z has joined #openstack-infra | 07:30 | |
| *** e0ne has joined #openstack-infra | 07:30 | |
| *** baoli has quit IRC | 07:31 | |
| *** gcb has quit IRC | 07:31 | |
| *** wolverineav has quit IRC | 07:31 | |
| *** ssbarnea_ has joined #openstack-infra | 07:31 | |
| *** ssbarnea has quit IRC | 07:34 | |
| *** gcb has joined #openstack-infra | 07:35 | |
| *** wolverineav has joined #openstack-infra | 07:37 | |
| *** rbuzatu has joined #openstack-infra | 07:38 | |
| *** flepied has joined #openstack-infra | 07:39 | |
| *** wolverineav has quit IRC | 07:41 | |
| *** askb_ has quit IRC | 07:43 | |
| *** amoralej|off is now known as amoralej | 07:43 | |
| *** e0ne has quit IRC | 07:44 | |
| *** tesseract has joined #openstack-infra | 07:44 | |
| *** wolverineav has joined #openstack-infra | 07:47 | |
| *** gcb has quit IRC | 07:50 | |
| *** jiaohaolin1 has left #openstack-infra | 07:50 | |
| *** wolverineav has quit IRC | 07:51 | |
| *** matrohon has joined #openstack-infra | 07:53 | |
| *** tobasco has quit IRC | 07:57 | |
| *** wolverineav has joined #openstack-infra | 07:57 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** tuanluong has quit IRC | 08:00 | |
| *** askb has joined #openstack-infra | 08:00 | |
| *** wolverineav has quit IRC | 08:01 | |
| *** gcb has joined #openstack-infra | 08:02 | |
| *** askb has quit IRC | 08:02 | |
| *** askb has joined #openstack-infra | 08:04 | |
| *** mozart89 has quit IRC | 08:05 | |
| *** shardy has joined #openstack-infra | 08:05 | |
| *** pcaruana has joined #openstack-infra | 08:06 | |
| *** flaper87 has joined #openstack-infra | 08:06 | |
| *** flaper87 has quit IRC | 08:07 | |
| *** flaper87 has joined #openstack-infra | 08:07 | |
| *** mozart89 has joined #openstack-infra | 08:08 | |
| *** markvoelker has joined #openstack-infra | 08:10 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/infra-specs master: New spec for translation_check_site (Ansible AIO) https://review.openstack.org/440825 | 08:10 |
| vsaienk01 | sdague: ianw please help to merge https://review.openstack.org/#/c/448983/ it should fix ironic gates that are broken at the moment | 08:10 |
| ianw | vsaienk01: we already have 448910 out there, i can quick merge that | 08:11 |
| vsaienk01 | ianw: thanks! someone beat me :) | 08:13 |
| *** markvoelker has quit IRC | 08:14 | |
| *** wolverineav has joined #openstack-infra | 08:16 | |
| *** kiltzman has quit IRC | 08:17 | |
| *** wolverin_ has joined #openstack-infra | 08:17 | |
| *** mkoderer has joined #openstack-infra | 08:17 | |
| *** Hal has joined #openstack-infra | 08:18 | |
| *** matrohon has quit IRC | 08:20 | |
| *** wolverineav has quit IRC | 08:21 | |
| *** wolverin_ has quit IRC | 08:21 | |
| *** kiltzman has joined #openstack-infra | 08:22 | |
| *** tonytan4ever has joined #openstack-infra | 08:27 | |
| *** martinkopec has joined #openstack-infra | 08:27 | |
| *** wolverineav has joined #openstack-infra | 08:27 | |
| *** wolverineav has quit IRC | 08:32 | |
| *** matrohon has joined #openstack-infra | 08:35 | |
| openstackgerrit | toan.t.pham proposed openstack-infra/jenkins-job-builder master: Add folder create for cloudbee folder jobs https://review.openstack.org/438795 | 08:37 |
| *** wolverineav has joined #openstack-infra | 08:37 | |
| *** wolverineav has quit IRC | 08:41 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: test containers with single-nic-vlan https://review.openstack.org/447973 | 08:43 |
| *** ralonsoh has joined #openstack-infra | 08:43 | |
| *** dingyichen has quit IRC | 08:44 | |
| *** nadya has joined #openstack-infra | 08:46 | |
| *** yamahata has quit IRC | 08:46 | |
| *** wolverineav has joined #openstack-infra | 08:47 | |
| *** jiaohaolin2 has joined #openstack-infra | 08:48 | |
| jiaohaolin2 | What's wrong with the git ? It's always time out when I clone | 08:50 |
| *** wolverineav has quit IRC | 08:52 | |
| *** Julien-zte has quit IRC | 08:54 | |
| *** jaosorior is now known as jaosorior_brb | 08:55 | |
| openstackgerrit | Merged openstack-infra/project-config master: Chef: adds basic multinode integration tests https://review.openstack.org/446294 | 08:57 |
| *** wolverineav has joined #openstack-infra | 08:57 | |
| openstackgerrit | Merged openstack-infra/project-config master: puppet: fix zuul layout for legacy unit jobs https://review.openstack.org/448772 | 08:58 |
| *** shardy has quit IRC | 08:58 | |
| *** shardy has joined #openstack-infra | 09:00 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Sushy to gerritbot https://review.openstack.org/448542 | 09:00 |
| *** makowals_ has quit IRC | 09:00 | |
| *** makowals has joined #openstack-infra | 09:01 | |
| *** wolverineav has quit IRC | 09:02 | |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** hashar has joined #openstack-infra | 09:03 | |
| *** openstackgerrit has joined #openstack-infra | 09:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Improve release note translation import criteria https://review.openstack.org/383622 | 09:04 |
| *** wolverineav has joined #openstack-infra | 09:07 | |
| *** lucas-afk is now known as lucasagomes | 09:08 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Refactor: block-device filesystem creation, mount and fstab https://review.openstack.org/444586 | 09:11 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/bindep master: bindep: depends: Add support for openSUSE {Leap,Tumbleweed} https://review.openstack.org/448800 | 09:11 |
| *** dtardivel has joined #openstack-infra | 09:11 | |
| *** david-lyle has quit IRC | 09:11 | |
| *** wolverineav has quit IRC | 09:12 | |
| *** jrist has quit IRC | 09:12 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Add lvm management to diskimage-builder https://review.openstack.org/444403 | 09:13 |
| *** david-lyle has joined #openstack-infra | 09:13 | |
| *** jrist has joined #openstack-infra | 09:13 | |
| *** alexchadin has joined #openstack-infra | 09:15 | |
| *** wolverineav has joined #openstack-infra | 09:18 | |
| *** rossella_s has joined #openstack-infra | 09:19 | |
| *** jordanP has joined #openstack-infra | 09:21 | |
| *** dtantsur|afk is now known as dtantsur | 09:21 | |
| *** wolverineav has quit IRC | 09:22 | |
| *** e0ne has joined #openstack-infra | 09:22 | |
| *** david-lyle_ has joined #openstack-infra | 09:27 | |
| *** david-lyle has quit IRC | 09:27 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix timeout for quickstart jobs https://review.openstack.org/449023 | 09:27 |
| *** rbuzatu_ has joined #openstack-infra | 09:27 | |
| *** tonytan4ever has quit IRC | 09:27 | |
| *** wolverineav has joined #openstack-infra | 09:28 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Fix timeout for quickstart jobs https://review.openstack.org/449023 | 09:28 |
| *** HeOS has joined #openstack-infra | 09:29 | |
| *** rbuzatu has quit IRC | 09:31 | |
| *** jiaohaolin2 has left #openstack-infra | 09:31 | |
| *** kaisers1 has quit IRC | 09:32 | |
| *** rbuzatu_ has quit IRC | 09:32 | |
| *** wolverineav has quit IRC | 09:32 | |
| *** wolverineav has joined #openstack-infra | 09:38 | |
| *** gongysh has quit IRC | 09:38 | |
| *** derekh has joined #openstack-infra | 09:38 | |
| *** electrofelix has joined #openstack-infra | 09:39 | |
| *** links has quit IRC | 09:39 | |
| *** ociuhandu has joined #openstack-infra | 09:40 | |
| *** amotoki_ has quit IRC | 09:40 | |
| *** wolverineav has quit IRC | 09:42 | |
| *** matrohon has quit IRC | 09:42 | |
| *** matrohon has joined #openstack-infra | 09:43 | |
| thomas_oneill | made a little progress since yesterday, bifrost will happily manipulate the network_data.json and this is present in the configdrive and being mounted as expected, but I'm still having the same problems with glean starting up as a service for each interface and failing because already mounted. I'm going to re-deploy and try again, but I was reading the code last night did wonder about this: | 09:46 |
| thomas_oneill | https://github.com/openstack-infra/glean/blob/master/glean/install.py#L105-L109 | 09:46 |
| *** kaisers1 has joined #openstack-infra | 09:47 | |
| thomas_oneill | what is the @.service thing all about? bleh I'm not great on systemd.. | 09:47 |
| *** wolverineav has joined #openstack-infra | 09:48 | |
| *** dizquierdo_ has joined #openstack-infra | 09:48 | |
| *** dizquierdo_ is now known as dizquierdo | 09:49 | |
| *** wolverineav has quit IRC | 09:52 | |
| thomas_oneill | ah that's the syntax for having it so you've got @eno1.service @eno2.service | 09:55 |
| *** links has joined #openstack-infra | 09:56 | |
| *** wolverineav has joined #openstack-infra | 09:58 | |
| *** amotoki has joined #openstack-infra | 09:59 | |
| *** MasterOfBugs has quit IRC | 09:59 | |
| *** gildub has quit IRC | 09:59 | |
| *** wolverineav has quit IRC | 10:02 | |
| *** wolverineav has joined #openstack-infra | 10:08 | |
| *** gcb has quit IRC | 10:08 | |
| *** flepied has quit IRC | 10:11 | |
| *** markvoelker has joined #openstack-infra | 10:11 | |
| *** links has quit IRC | 10:11 | |
| neiljerram | Morning all! | 10:12 |
| *** wolverineav has quit IRC | 10:12 | |
| *** nicolasbock has joined #openstack-infra | 10:12 | |
| neiljerram | I'm interested in creating a CI job for my project (networking-calico) that runs a DSVM/Tempest test with its _master_ code against stable/ocata of the rest of OpenStack. Could anyone point me to how to do that? | 10:13 |
| *** vsaienk01 has left #openstack-infra | 10:16 | |
| *** markvoelker has quit IRC | 10:16 | |
| *** wolverineav has joined #openstack-infra | 10:18 | |
| lennyb | HI, I see that nodepoold takes too much CPU 147.2% . are there any hints/ways to tune it? Thanks. | 10:22 |
| *** wolverineav has quit IRC | 10:22 | |
| *** liujiong has quit IRC | 10:22 | |
| *** links has joined #openstack-infra | 10:24 | |
| *** yanyanhu_ has quit IRC | 10:25 | |
| hashar | lennyb: potentially you can try generating a stack trace of its thread | 10:25 |
| hashar | if one has the yappi python module (Yet Another Python Profiler) sending SIGUSR2 enables profiling and the second SIGUSR2 dump the info to the debug log + stop profiling | 10:26 |
| hashar | (based on https://docs.openstack.org/infra/nodepool/operation.html#daemon-usage ) | 10:26 |
| *** flepied has joined #openstack-infra | 10:27 | |
| *** uberjay has quit IRC | 10:28 | |
| *** wolverineav has joined #openstack-infra | 10:28 | |
| lennyb | 10x hashar, I will check it | 10:30 |
| *** uberjay has joined #openstack-infra | 10:30 | |
| hashar | lennyb: strace has some profiling system as well | 10:30 |
| hashar | strace -c -p <pid of nodepool here> | 10:30 |
| hashar | might want '-f' as well to follow sub process but I don't think nodepool forks | 10:31 |
| hashar | gdb probably has a similar system | 10:31 |
| *** wolverineav has quit IRC | 10:32 | |
| *** links has quit IRC | 10:35 | |
| dims | any infra/gerrit cores around? please add me to the newly created empty docker-machine-openstack-core group https://review.openstack.org/#/admin/groups/1731,members | 10:35 |
| *** beslemon has joined #openstack-infra | 10:37 | |
| *** makowals has quit IRC | 10:38 | |
| *** wolverineav has joined #openstack-infra | 10:38 | |
| *** beslemon_ has joined #openstack-infra | 10:41 | |
| dims | mordred : ^ | 10:42 |
| *** beslemon has quit IRC | 10:42 | |
| *** wolverineav has quit IRC | 10:43 | |
| *** makowals has joined #openstack-infra | 10:43 | |
| *** amotoki has quit IRC | 10:46 | |
| *** bhavik1 has joined #openstack-infra | 10:47 | |
| *** wolverineav has joined #openstack-infra | 10:48 | |
| *** amotoki has joined #openstack-infra | 10:49 | |
| *** jkilpatr has quit IRC | 10:50 | |
| frickler | dims: yolanda might be. and usually citing the review that created the repo helps. ;) | 10:50 |
| dims | frickler : yolanda : ack thanks https://review.openstack.org/#/c/446586/ | 10:51 |
| *** links has joined #openstack-infra | 10:51 | |
| *** wolverineav has quit IRC | 10:53 | |
| *** EricGonczer_ has joined #openstack-infra | 10:54 | |
| *** mat128 has joined #openstack-infra | 10:57 | |
| *** rhallisey has joined #openstack-infra | 10:58 | |
| *** wolverineav has joined #openstack-infra | 10:58 | |
| *** pblaho has quit IRC | 11:00 | |
| *** jaosorior_brb is now known as jaosorior | 11:02 | |
| *** wolverineav has quit IRC | 11:02 | |
| *** jistr is now known as jistr|afk | 11:06 | |
| *** jkilpatr has joined #openstack-infra | 11:06 | |
| *** bhavik1 has quit IRC | 11:08 | |
| *** wolverineav has joined #openstack-infra | 11:09 | |
| *** Guest90664 has quit IRC | 11:09 | |
| *** alexchadin has quit IRC | 11:12 | |
| thomas_oneill | hey how do I report a glean bug? there's no glean or openstack-infra lajunchpad? | 11:13 |
| thomas_oneill | s/laj/la/ | 11:13 |
| *** esikachev has quit IRC | 11:13 | |
| *** wolverineav has quit IRC | 11:13 | |
| thomas_oneill | oh there is an /infra launchpad... there's nothing on it though | 11:14 |
| *** EricGonczer_ has quit IRC | 11:14 | |
| mat128 | thomas_oneill: the only bug I could find is the one referenced here: https://review.openstack.org/#/c/279818/ | 11:15 |
| mat128 | thomas_oneill: and it's a bug in DIB | 11:15 |
| mat128 | thomas_oneill: https://storyboard.openstack.org/#!/project/815 | 11:15 |
| mat128 | thomas_oneill: the bugs are here ^ | 11:16 |
| *** matrohon has quit IRC | 11:16 | |
| *** wolverineav has joined #openstack-infra | 11:19 | |
| radez | ianw: hey ianw, re: your comment on following up on docs on that dib patch, are you suggesting that I sould follow up and put more info in the docs or that the dib team should do that? | 11:19 |
| radez | just want to make sure that I understand your intentions :) | 11:19 |
| thomas_oneill | mat128: ah ty, wow this looks really nice compared to launchpad, cosmetically anyway | 11:19 |
| thomas_oneill | I don't have to squint at it at least.. | 11:20 |
| mat128 | thomas_oneill: :) | 11:20 |
| *** wolverineav has quit IRC | 11:23 | |
| *** Julien-zte has joined #openstack-infra | 11:24 | |
| thomas_oneill | and it supports markdown... nice.. | 11:24 |
| thomas_oneill | is there a plan to migrate other projects over to this thing? | 11:25 |
| *** reed has quit IRC | 11:26 | |
| Zara | thomas_oneill: indeed there is! | 11:27 |
| Zara | there should be a talk about it in boston | 11:27 |
| Zara | and there are various info things dotted around, will grab some links | 11:28 |
| *** pgadiya has quit IRC | 11:28 | |
| *** tonytan4ever has joined #openstack-infra | 11:28 | |
| *** wolverineav has joined #openstack-infra | 11:28 | |
| Zara | thomas_oneill: https://storyboard-blog.io/ summarises a bunch of stuff | 11:29 |
| thomas_oneill | okay well apparently I've written a story :D https://storyboard.openstack.org/#!/story/2000946 | 11:30 |
| Zara | and the spec for the migration is here: https://specs.openstack.org/openstack-infra/infra-specs/specs/task-tracker.html | 11:30 |
| Zara | cool, let us know in #storyboard if you have any questions about using it etc :) | 11:32 |
| *** wolverineav has quit IRC | 11:33 | |
| *** reed has joined #openstack-infra | 11:33 | |
| *** masber has joined #openstack-infra | 11:33 | |
| thomas_oneill | okay, cheers | 11:33 |
| *** andreykurilin_ has quit IRC | 11:36 | |
| *** andreykurilin has joined #openstack-infra | 11:36 | |
| *** dharinic has quit IRC | 11:36 | |
| *** aunnam has quit IRC | 11:36 | |
| *** manjeets has quit IRC | 11:36 | |
| *** rbuzatu has joined #openstack-infra | 11:37 | |
| *** ldnunes has joined #openstack-infra | 11:38 | |
| *** nadya has quit IRC | 11:39 | |
| *** wolverineav has joined #openstack-infra | 11:39 | |
| *** EricGonczer_ has joined #openstack-infra | 11:42 | |
| *** wolverineav has quit IRC | 11:43 | |
| *** amotoki has quit IRC | 11:45 | |
| *** rfolco has joined #openstack-infra | 11:48 | |
| *** wolverineav has joined #openstack-infra | 11:49 | |
| *** vi01et03_ has joined #openstack-infra | 11:50 | |
| *** nmathew has quit IRC | 11:50 | |
| ianw | dims: done | 11:52 |
| anilvenkata | any second core review for https://review.openstack.org/#/c/383827/ ? | 11:52 |
| *** vi01et03_ has quit IRC | 11:53 | |
| dims | thanks ianw | 11:53 |
| *** wolverineav has quit IRC | 11:53 | |
| anilvenkata | https://review.openstack.org/#/c/383827/ is already reviewed by neutron PTL and cores | 11:54 |
| anilvenkata | need another devstack core review | 11:54 |
| *** gcb has joined #openstack-infra | 11:55 | |
| *** dizquierdo has quit IRC | 11:55 | |
| *** aysyd has joined #openstack-infra | 11:55 | |
| ianw | anilvenkata: why does it fail the dvr multinode job? | 11:57 |
| ianw | radez: well, i'm never going to tell you not to contribute :) if you've got time to think about multi-arch support a bit more broadly that would be welcome. | 11:58 |
| *** dizquierdo has joined #openstack-infra | 11:59 | |
| *** wolverineav has joined #openstack-infra | 11:59 | |
| radez | ianw: sure, not suggesting that I'll ignore the docs, just making sure it wasn't a request for a direct action :) | 11:59 |
| *** Guest3223 is now known as dasm | 12:03 | |
| *** wolverineav has quit IRC | 12:03 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: temporary hack for heat-agents https://review.openstack.org/449102 | 12:04 |
| *** askb has quit IRC | 12:05 | |
| *** EricGonczer_ has quit IRC | 12:06 | |
| *** saibarspeis has joined #openstack-infra | 12:07 | |
| *** shardy is now known as shardy_lunch | 12:08 | |
| *** trown|outtypewww is now known as trown | 12:08 | |
| *** wolverineav has joined #openstack-infra | 12:09 | |
| frickler | anilvenkata: ianw: the failure in http://logs.openstack.org/27/383827/18/check/gate-tempest-dsvm-neutron-dvr-multinode-full-ubuntu-xenial-nv/865d1e4/ is to the vnc config issue we had, should be fixed by now. maybe recheck? | 12:12 |
| *** markvoelker has joined #openstack-infra | 12:13 | |
| *** anilvenkata has quit IRC | 12:13 | |
| frickler | while we're talking ds-gate, this one is also still waiting for reviews ... https://review.openstack.org/416985 :) | 12:13 |
| *** wolverineav has quit IRC | 12:14 | |
| thomas_oneill | sambetts_: hi, thought I'd ping you on this since you wrote the code that's barfing https://storyboard.openstack.org/#!/story/2000946 I suspect there may be something wrong with the way I'm specifying my network_data.json | 12:14 |
| thomas_oneill | I'm trying to make it so it doesn't try to do anything on eno1 and eno2, and only configures ens2f0 and maybe ens2f1 | 12:15 |
| *** EricGonczer_ has joined #openstack-infra | 12:15 | |
| thomas_oneill | in any case I don't understand why pop and not get on the dict, and it might be nice to throw a more meaningful error. | 12:16 |
| thomas_oneill | going to experiment with changing my network_data.json again now in any case (this didn't occur before I put the eno1 and eno2 links in without any networks... I should add that to the storyboard actually) | 12:16 |
| *** markvoelker has quit IRC | 12:17 | |
| *** wolverineav has joined #openstack-infra | 12:19 | |
| *** pblaho has joined #openstack-infra | 12:19 | |
| *** EricGonczer_ has quit IRC | 12:19 | |
| mordred | Zara: did we ever implement an IRC bot for storyboard? | 12:19 |
| Zara | mordred: hm, I don't think we did. | 12:21 |
| * Zara would definitely *like* one | 12:21 | |
| mordred | thomas_oneill: interesting. so - we coded that based on the spec here: https://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/metadata-service-network-info.html and also what we found in the clouds we were using | 12:22 |
| mordred | and it seemed that ethernet_mac_address was always there | 12:22 |
| *** zhurong has quit IRC | 12:22 | |
| mordred | that said - I could certainly see an argument to put in logic to make sure ethernet_mac_address is there | 12:23 |
| mordred | Zara: ++ | 12:23 |
| *** rbuzatu has quit IRC | 12:23 | |
| *** wolverineav has quit IRC | 12:23 | |
| thomas_oneill | mordred: ethernet_mac_address was most certainly there but it is popped, therefore it is not there after it is popped :P | 12:24 |
| mordred | oh! | 12:24 |
| thomas_oneill | if it was just retrieved with a get and not popped it would still be there the second time around though I haven't looked at the code logic long enough, maybe then it would just loop forever or something, I should read the code more | 12:24 |
| thomas_oneill | but yeah I did have a quick look in pdb and the behaviour is a successful first pop and then a key error on the 2nd hitting of the breakpoint if I break at the line of the error | 12:25 |
| mordred | well - wait - are you sure? it doesn't look like you have a mac_address for eno1 or eno2 in your sample you pasted | 12:25 |
| thomas_oneill | oh shit | 12:25 |
| thomas_oneill | so a mac address is required | 12:25 |
| thomas_oneill | that's annoying... I'm trying to automate lol | 12:25 |
| *** tosky has joined #openstack-infra | 12:25 | |
| mordred | sure. well - I mean, we can likely write a patch to support no mac_address | 12:25 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add check for valid zk attribute before disconnect https://review.openstack.org/449110 | 12:26 |
| mordred | most of the time this is info coming from nova for us, so there just always is one | 12:26 |
| mordred | but I have no problem supporting the no-mac case | 12:26 |
| thomas_oneill | ah yeah, I'm using bifrost/ironic so it's baremetal | 12:26 |
| mordred | cool! yes, we want you to be happy :) | 12:27 |
| *** jistr|afk is now known as jistr | 12:27 | |
| *** kornica has quit IRC | 12:28 | |
| *** priteau has joined #openstack-infra | 12:28 | |
| Zara | (heh, yeah, this the most recent story on an irc bot for storyboard. https://storyboard.openstack.org/#!/story/182) | 12:28 |
| *** tonytan4ever has quit IRC | 12:28 | |
| Zara | so we got as far as 'we should notify project channels with an irc bot' | 12:29 |
| *** wolverineav has joined #openstack-infra | 12:29 | |
| *** jbernard has quit IRC | 12:30 | |
| *** jbernard has joined #openstack-infra | 12:31 | |
| *** tonytan4ever has joined #openstack-infra | 12:33 | |
| *** wolverineav has quit IRC | 12:33 | |
| mordred | Zara: well, I mean, that's at least something | 12:34 |
| Zara | haha :) I think it just got put to one side since there were more urgent things, but I wouldn't expect it to be too difficult to set up... I have a vague recollection of looking at the launchpad bot a while ago, though maybe that was just for gerrit notifications | 12:37 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Do not require secure file for nodepoold https://review.openstack.org/449118 | 12:37 |
| Zara | anyway yeah I think it's just been an ENOTIME thing rather than anything technically complex | 12:37 |
| *** matrohon has joined #openstack-infra | 12:38 | |
| mordred | yah - I mean, we also have the "rework all the bots" project | 12:38 |
| mordred | which is also not going to be hard, but ENOTIME | 12:39 |
| *** dprince has joined #openstack-infra | 12:39 | |
| *** wolverineav has joined #openstack-infra | 12:39 | |
| *** kgiusti has joined #openstack-infra | 12:39 | |
| Zara | ohh yeahh I think we were vaguely thinking of hooking storyboard up to firehose to get events that way... | 12:40 |
| Zara | yaks, yaks everywhere | 12:40 |
| *** rlandy has joined #openstack-infra | 12:41 | |
| mordred | SO MANY YAKS | 12:42 |
| Zara | :D | 12:42 |
| *** wolverineav has quit IRC | 12:44 | |
| *** markvoelker has joined #openstack-infra | 12:46 | |
| *** bh526r has joined #openstack-infra | 12:46 | |
| *** mkoderer has quit IRC | 12:47 | |
| *** dprince has quit IRC | 12:49 | |
| *** wolverineav has joined #openstack-infra | 12:50 | |
| *** dprince has joined #openstack-infra | 12:51 | |
| *** psachin has quit IRC | 12:52 | |
| *** dave-mccowan has joined #openstack-infra | 12:54 | |
| *** wolverineav has quit IRC | 12:54 | |
| *** ssbarnea_ has quit IRC | 12:55 | |
| *** jascott1 has joined #openstack-infra | 12:56 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove SSH support from nodepool https://review.openstack.org/446683 | 12:56 |
| thomas_oneill | paste.openstack.org has a line/character limit or something? | 12:57 |
| *** shardy_lunch is now known as shardy | 12:57 | |
| *** nmathew has joined #openstack-infra | 12:57 | |
| *** nmathew has quit IRC | 12:57 | |
| thomas_oneill | trying to post this https://gist.github.com/thomasodot/54b14023a4b52d5e8b586690954e2c2a it only allows the first 776 lines, http://paste.openstack.org/show/603911/ | 12:58 |
| *** wolverineav has joined #openstack-infra | 13:00 | |
| *** jascott1 has quit IRC | 13:00 | |
| *** Goneri has joined #openstack-infra | 13:01 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Docs: Remove "jenkins" reference https://review.openstack.org/449129 | 13:02 |
| *** rodrigods has quit IRC | 13:03 | |
| *** rodrigods has joined #openstack-infra | 13:03 | |
| *** jcoufal has joined #openstack-infra | 13:03 | |
| *** wolverineav has quit IRC | 13:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: [sahara multinode] Specify empty config for subnode https://review.openstack.org/447215 | 13:04 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Docs: Clarify secure file usage https://review.openstack.org/449130 | 13:06 |
| *** amoralej is now known as amoralej|lunch | 13:07 | |
| thomas_oneill | I haven't got any cloud-init dib elements in my dib cli, only "ubuntu vm simple-init devuser", there's clearly cloud-init and dhclient stuff occurring at boot time on this node that's running this image though, does anyone know where this comes from? | 13:08 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: temporary hack for heat-agents https://review.openstack.org/449102 | 13:08 |
| *** wolverineav has joined #openstack-infra | 13:10 | |
| *** ssbarnea has joined #openstack-infra | 13:11 | |
| *** mriedem has joined #openstack-infra | 13:13 | |
| openstackgerrit | Jens Rosenboom proposed openstack-infra/project-config master: Fix VNC setup in neutron DVR multinode job https://review.openstack.org/449132 | 13:13 |
| *** baoli has joined #openstack-infra | 13:13 | |
| *** amotoki has joined #openstack-infra | 13:13 | |
| *** mdrabe has joined #openstack-infra | 13:13 | |
| *** wolverineav has quit IRC | 13:14 | |
| *** kornicameister has quit IRC | 13:14 | |
| AJaeger | fungi, pabelanger, once you're around, could either of you review two jeepyb changes for codesearch and regenerate the list of repos, please (last time this was not done automatically)? https://review.openstack.org/#/c/446646/ and https://review.openstack.org/#/c/446634 | 13:18 |
| *** gouthamr has joined #openstack-infra | 13:20 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Add separate ACL/group for castellan https://review.openstack.org/449138 | 13:21 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Docs: Remove cron references https://review.openstack.org/449140 | 13:21 |
| mordred | thomas_oneill: you want ubuntu-minimal as your base image, not ubuntu | 13:24 |
| thomas_oneill | oh I see, I'll try with that, I'm also currently trying with DIB_CLOUD_INIT_DATASOURCES set to ConfigDrive maybe that won't work though, will try the minimal image if/when this fails, ty! | 13:25 |
| *** priteau has quit IRC | 13:26 | |
| *** rbuzatu has joined #openstack-infra | 13:27 | |
| *** wolverineav has joined #openstack-infra | 13:29 | |
| *** sree has joined #openstack-infra | 13:29 | |
| *** wolverin_ has joined #openstack-infra | 13:30 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Docs: Correct availability-zones documentation. https://review.openstack.org/449147 | 13:30 |
| *** eharney has joined #openstack-infra | 13:31 | |
| *** wolverineav has quit IRC | 13:33 | |
| *** wolverin_ has quit IRC | 13:34 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Docs: Remove refs to removed nodepool commands https://review.openstack.org/449152 | 13:39 |
| thomas_oneill | yeah trying with minimal, the datasources env var didn't do the trick... time for lunch anyway :D | 13:41 |
| *** xarses has joined #openstack-infra | 13:42 | |
| *** krtaylor has joined #openstack-infra | 13:43 | |
| *** mat128 has quit IRC | 13:47 | |
| *** fguillot has joined #openstack-infra | 13:50 | |
| *** eharney has quit IRC | 13:55 | |
| *** rbuzatu has quit IRC | 13:56 | |
| *** dizquierdo has quit IRC | 13:56 | |
| *** rbuzatu has joined #openstack-infra | 13:57 | |
| *** saibarspeis has quit IRC | 13:59 | |
| *** annegentle has joined #openstack-infra | 14:00 | |
| sshnaidm | we have periodic jobs stuck in queue for 30 hours (http://status.openstack.org/zuul/), is there any way to reset it or to trigger them again..? | 14:01 |
| *** hongbin has joined #openstack-infra | 14:02 | |
| mordred | sdague: any chance you know of anything that has changed in the last day or so that would cause all of the nodepool dsvm jobs to start timing out on not being able to ssh in to guests? | 14:02 |
| sdague | mordred: nope | 14:02 |
| sdague | I can look at changelogs | 14:02 |
| mordred | sdague: cool - I shall do the same | 14:03 |
| sdague | there has been some tuning of the env | 14:03 |
| *** dhajare has quit IRC | 14:03 | |
| sdague | last thing that mattered in there was apache tuning | 14:05 |
| mordred | hrm | 14:05 |
| *** dhajare has joined #openstack-infra | 14:05 | |
| *** dhajare has quit IRC | 14:06 | |
| mordred | sdague: I don't see anything smoking in devstack, devstack-gate, nova or neutron :( | 14:07 |
| *** annegentle has quit IRC | 14:07 | |
| sdague | mordred: it consistent? | 14:07 |
| mordred | sdague: yes - every nodepool patch is now failing | 14:07 |
| *** jamesdenton has joined #openstack-infra | 14:07 | |
| * mordred goes to put in an auto-hold | 14:07 | |
| mordred | http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/ is an example of a failure, fwiw | 14:08 |
| sdague | mordred: ok, jobs with logs ? | 14:08 |
| mordred | http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-nodepool.txt.gz#_2017-03-23_13_06_38_000 | 14:09 |
| mordred | is the error we see | 14:09 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove puppetdb / puppetboard server https://review.openstack.org/449167 | 14:09 |
| mordred | k. I added a hold - will re-trigger so we can catch a node | 14:10 |
| mordred | Shrews: ^^ | 14:10 |
| *** eharney has joined #openstack-infra | 14:11 | |
| Shrews | *nod* | 14:11 |
| *** sree has quit IRC | 14:11 | |
| *** prateek has quit IRC | 14:11 | |
| mordred | sdague: is this bad: http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-n-cpu.txt.gz#_2017-03-23_12_50_20_978 ? | 14:11 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove SSH support from nodepool https://review.openstack.org/446683 | 14:13 |
| sdague | mordred: no... placement just isn't up yet | 14:14 |
| mordred | ok | 14:14 |
| sdague | it reports in http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-n-cpu.txt.gz#_2017-03-23_12_51_23_981 | 14:15 |
| Shrews | http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-q-agt.txt.gz#_2017-03-23_12_49_31_807 | 14:15 |
| Shrews | also an error | 14:15 |
| sdague | there is just a startup order bit, honestly, we should probably mask a few of those, but so it is | 14:15 |
| Shrews | but over my head | 14:15 |
| sdague | http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-n-cpu.txt.gz#_2017-03-23_12_56_23_055 is the build of the first instance here | 14:16 |
| sdague | http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-q-agt.txt.gz#_2017-03-23_12_56_22_190 | 14:17 |
| sdague | that is the relevant port create | 14:17 |
| *** armax has joined #openstack-infra | 14:18 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: temporary hack for heat-agents https://review.openstack.org/449102 | 14:20 |
| *** burgerk has joined #openstack-infra | 14:20 | |
| *** gordc has joined #openstack-infra | 14:22 | |
| *** amoralej|lunch is now known as amoralej | 14:22 | |
| dims | mordred : fungi : my id got added to this acl/group https://review.openstack.org/#/admin/groups/1731,members but i don't see +2 right yet. tried logout/login - doesn't help | 14:22 |
| sdague | mordred: so I don't know what that nodepool command really is doing | 14:22 |
| sdague | and why the list seems to still say building | 14:23 |
| sdague | but the vm was built for sure | 14:23 |
| frickler | sdague: mordred: it times out trying to ssh to the node http://logs.openstack.org/81/448781/2/check/gate-dsvm-nodepool-src/4cdf5b3/logs/screen-nodepool.txt.gz#_2017-03-23_13_06_38_000 | 14:23 |
| frickler | that repeats for every node launched. so either a networking issue or the built image doesn't run ssh properly | 14:24 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Add novaclient and keystoneauth debug logging https://review.openstack.org/449173 | 14:24 |
| mordred | sdague: ^^ I just added some more logging for just that reason | 14:25 |
| *** fguillot has quit IRC | 14:25 | |
| mordred | dims: the acls may not have been applied to the group properly - one sec | 14:25 |
| *** ihrachys has joined #openstack-infra | 14:26 | |
| *** gcb has quit IRC | 14:26 | |
| pabelanger | frickler: SSH should be fine, we just rebuilt production images for nodepool.o.o 1 hour ago. It would be using the same elements | 14:26 |
| dims | mordred : add me to this too? (-release got missed) https://review.openstack.org/#/admin/groups/1732,members | 14:26 |
| frickler | pabelanger: can you double-check for a trusty image just to be sure? | 14:27 |
| mordred | dims: the acls are there | 14:27 |
| mordred | dims: I have added you to release - can you re-check if you can see +2? | 14:28 |
| dims | ok let me try switching browsers mordred | 14:28 |
| sdague | mordred: it would be nice to do a console dump if a guest fails | 14:28 |
| sdague | like we do in tempest rus | 14:29 |
| sdague | runs | 14:29 |
| dims | mordred : thanks, seems to work now | 14:29 |
| mordred | woot | 14:29 |
| *** trown is now known as trown|afk | 14:29 | |
| sdague | so you can corelate if things like ip addresses all lined up like you expected | 14:29 |
| *** sshnaidm is now known as sshnaidm|mtg | 14:29 | |
| *** annegentle has joined #openstack-infra | 14:29 | |
| mordred | sdague: we've talked about that before - it's a bit tricky since we certainly _don't_ want to do that in normal nodepool operation, but it would be helpful in debugging in the gate - I should figure out how to plumb that through | 14:30 |
| pabelanger | frickler: nc 2001:4800:1ae1:18:f816:3eff:fe24:156e 19885 | 14:30 |
| pabelanger | frickler: that is trusty | 14:30 |
| pabelanger | frickler: and latest image build | 14:30 |
| pabelanger | however, I am still looking at DIB to just be 100% | 14:31 |
| mordred | pabelanger: this would be the image _in_ the job, not the nodepool image | 14:31 |
| frickler | pabelanger: o.k., thx, should be fine, then. | 14:31 |
| pabelanger | mordred: right, they should be using the same release of DIB. So, ssh _should_ work | 14:32 |
| sdague | mordred: right, if you fail do a nova console | 14:32 |
| pabelanger | but, looking into it to confirm | 14:32 |
| mordred | pabelanger: ah - good point | 14:33 |
| openstackgerrit | Mikhail S Medvedev proposed openstack/diskimage-builder master: Use correct Ubuntu distro url on non-x86 arches https://review.openstack.org/448848 | 14:34 |
| mordred | sdague: yah, totally - I'm just saying we fail booting nodes all the time in production and it's fine - I think we'd want some sort of config flag we can enable in the gate to do that extra api call / log | 14:34 |
| pabelanger | Looking at diskimage-builder I don't see any things that stands out. So, I think we are okay there | 14:34 |
| mordred | I'll see if I can work that out | 14:34 |
| pabelanger | mordred: that would be helpful to toggle per provider too, if you are looking into it | 14:35 |
| sdague | mordred: you don't use network injection do you? | 14:35 |
| mordred | sdague: what's network injection? | 14:35 |
| mordred | pabelanger: ++ | 14:35 |
| pabelanger | mordred: https://review.openstack.org/#/c/358730/ was my first attempt a while back. | 14:35 |
| *** rbuzatu_ has joined #openstack-infra | 14:36 | |
| *** pradk has joined #openstack-infra | 14:37 | |
| *** links has quit IRC | 14:38 | |
| *** rbuzatu has quit IRC | 14:39 | |
| jeblair | pabelanger, mordred: that change looks pretty good if we implement my suggestion to move it up to the ssh check rather than the ready script. i think mordred may be right about the option -- it's probably too much logging by default and we should keep it as an option. | 14:41 |
| mordred | yup. I'm working on it right now | 14:41 |
| *** fandi has joined #openstack-infra | 14:42 | |
| thomas_oneill | mordred: no joy with the minimal image element either :( same result I think, need to inspect the boot log again but it looks the same at first glance | 14:43 |
| *** esberglu has joined #openstack-infra | 14:44 | |
| *** gouthamr has quit IRC | 14:44 | |
| *** nadya has joined #openstack-infra | 14:45 | |
| *** adarazs has joined #openstack-infra | 14:47 | |
| openstackgerrit | Merged openstack-dev/cookiecutter master: Remove MANIFEST.in https://review.openstack.org/448163 | 14:47 |
| openstackgerrit | Merged openstack-dev/cookiecutter master: Remove 'build_releasenotes' https://review.openstack.org/448164 | 14:47 |
| openstackgerrit | Merged openstack-dev/cookiecutter master: Bump pbr to 2.0, Sphinx to 1.5 https://review.openstack.org/448165 | 14:47 |
| openstackgerrit | Merged openstack-dev/cookiecutter master: Use Sphinx 1.5 warning-is-error https://review.openstack.org/448166 | 14:47 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Fetch server console log if ready_script fails https://review.openstack.org/358730 | 14:48 |
| mordred | pabelanger, jeblair: ^^ how about that? | 14:48 |
| thomas_oneill | ah I need to set this https://github.com/openstack-infra/glean/blob/master/glean/init/glean.sh#L21 | 14:48 |
| thomas_oneill | how can I set this in the image or the config drive though? (the INTERFACES for the glean.sh) | 14:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Fetch server console log if ready_script fails https://review.openstack.org/358730 | 14:49 |
| mordred | jeblair: updated - added it to the provider config in the devstack plugin so that it's self-testing :) | 14:49 |
| thomas_oneill | ah it is simple-init that calls it so I should look at that | 14:49 |
| openstackgerrit | Merged openstack-infra/project-config master: Update barbican kmip gate to non-voting check from experimental https://review.openstack.org/446161 | 14:49 |
| *** hemna_ is now known as hemna | 14:50 | |
| sshnaidm|mtg | we have periodic jobs stuck in queue for 30 hours (http://status.openstack.org/zuul/), is there any way to reset it or to trigger them again..? | 14:51 |
| pabelanger | sshnaidm|mtg: no, they need to wait for nodes to be available | 14:55 |
| pabelanger | looks like pipelines are just backed up for tripleo | 14:56 |
| fungi | probably need run fewer tripleo-node-specific jobs or increase capacity in thr tripleo cloud | 14:57 |
| pabelanger | yup | 14:57 |
| jeblair | mordred: the exception handling looks good. i still think we should not run it on ready script failure though. | 14:58 |
| fungi | tripleo-check que length is ~80 changes deep | 14:58 |
| fungi | queue | 14:58 |
| jeblair | mordred: left comments | 14:58 |
| *** fandi has quit IRC | 14:58 | |
| pabelanger | fungi: I believe there was an outage this week, and it is just trying to catch up still | 14:58 |
| fungi | pabelanger: makes sense | 14:58 |
| fungi | so not only needs more capacity, but also redundant sites | 14:59 |
| pabelanger | they've made good process on moving some jobs to our public clouds | 14:59 |
| pabelanger | but there is still a dependency on OVB things (virtual baremetal) | 14:59 |
| thomas_oneill | https://github.com/openstack-infra/glean/blob/master/glean/install.py#L55-L61 - what's going on here? there isn't an actual binary called 'install' is there? | 15:00 |
| jeblair | thomas_oneill: yes there is | 15:00 |
| jeblair | thomas_oneill: https://linux.die.net/man/1/install | 15:01 |
| thomas_oneill | oh wow, why haven't I heard of this lolll ty | 15:01 |
| jeblair | thomas_oneill: it's commonly used in the autoconf world by 'make install' targets | 15:01 |
| *** xarses_ has joined #openstack-infra | 15:02 | |
| *** marst has joined #openstack-infra | 15:03 | |
| *** andreww has joined #openstack-infra | 15:03 | |
| *** adrian_otto has joined #openstack-infra | 15:03 | |
| *** tphummel has joined #openstack-infra | 15:04 | |
| *** jokke__ is now known as jokke_ | 15:05 | |
| *** xarses_ has quit IRC | 15:06 | |
| fungi | the predecessor of the gnu coreutils install command dates back to bsd 4.2 (according to the install(1) manpage on my openbsd systems) | 15:06 |
| *** xarses_ has joined #openstack-infra | 15:07 | |
| fungi | or 4.2bsd if you prefer | 15:07 |
| *** dimtruck is now known as zz_dimtruck | 15:07 | |
| mnaser | fungi mind pm for a quick second? | 15:07 |
| fungi | mnaser: go for it | 15:07 |
| *** jamesdenton has left #openstack-infra | 15:07 | |
| *** andreww has quit IRC | 15:08 | |
| thomas_oneill | so what's this %I thing and how can I set it https://github.com/openstack-infra/glean/blob/master/glean/init/glean%40.service#L11 | 15:08 |
| *** armax has quit IRC | 15:09 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Use shorter timeout for update step https://review.openstack.org/448778 | 15:11 |
| *** soliosg has joined #openstack-infra | 15:11 | |
| *** mozart89 has quit IRC | 15:13 | |
| *** matrohon has quit IRC | 15:14 | |
| *** matrohon has joined #openstack-infra | 15:15 | |
| clarkb | mordred: pabelanger looking at the nodepool job failures is it really trying to build and launch all of those iamges? | 15:15 |
| clarkb | its probably just timing out because qemu is slow if its trying to do ~5 real distro images | 15:15 |
| pabelanger | clarkb: it should only launch 1 for nodepool | 15:16 |
| pabelanger | 1 image that is | 15:16 |
| clarkb | ya looking more closedly it seems to repeatedly fail launching the trusty only | 15:16 |
| clarkb | but there are allocation requests for the others | 15:16 |
| *** zz_dimtruck is now known as dimtruck | 15:17 | |
| pabelanger | those should be paused | 15:17 |
| clarkb | that doesn't appear to reflect on the allocator at least | 15:17 |
| clarkb | also is the failure consistent? | 15:18 |
| *** nadya has quit IRC | 15:18 | |
| clarkb | reproduce.sh is probably where I would start if so | 15:18 |
| sdague | thomas_oneill: https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Specifiers | 15:18 |
| *** sshnaidm|mtg is now known as sshnaidm | 15:18 | |
| *** wolverineav has joined #openstack-infra | 15:19 | |
| *** tphummel has quit IRC | 15:21 | |
| *** nicolasbock has quit IRC | 15:32 | |
| thomas_oneill | ty | 15:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Fetch server console log if ssh connection files https://review.openstack.org/358730 | 15:34 |
| mordred | jeblair: there ya go - sorry, I'd missed your non-inline review before | 15:34 |
| mordred | also - the parent patch which adds novaclient and keystoneauth debug logging passed and is appropriately putting the openstack API call informatoin into the log file | 15:34 |
| mordred | which _should_ help us track request-id interactions a bit better | 15:35 |
| *** spzala has joined #openstack-infra | 15:36 | |
| *** martinkopec has quit IRC | 15:36 | |
| *** jascott1 has joined #openstack-infra | 15:37 | |
| openstackgerrit | Jens Rosenboom proposed openstack-infra/project-config master: Fix VNC setup in neutron DVR multinode jobs https://review.openstack.org/449132 | 15:37 |
| *** electrofelix has quit IRC | 15:38 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Switch trunk/cbs/buildlogs to use https https://review.openstack.org/448041 | 15:38 |
| strigazi | hi clarkb, there was some progress yesterday | 15:39 |
| strigazi | clarkb, this configuration worked for me: https://review.openstack.org/#/c/443002/16/magnum/tests/contrib/gate_hook.sh | 15:40 |
| *** electrofelix has joined #openstack-infra | 15:41 | |
| clarkb | strigazi: you shouldn't need to disable tempest on the subnode (beacuse its already disabled there) but that doesn't hurt | 15:42 |
| *** armax has joined #openstack-infra | 15:42 | |
| clarkb | strigazi: so I think that implies we do want to have a config for just the primary node then. Did you want to add that to devstack-gate? | 15:42 |
| clarkb | strigazi: I would call it DEVSTACK_PRIMARY_CONFIG to line up with DEVSTACK_SUBNODE_CONFIG | 15:43 |
| strigazi | clarkb without disabling tempest I was getting http://logs.openstack.org/02/443002/14/experimental/gate-functional-dsvm-magnum-k8s-multinode-ubuntu-xenial-nv/99e91e2/logs/subnode-2/devstacklog.txt.gz#_2017-03-23_10_25_28_835 | 15:44 |
| *** ttx has quit IRC | 15:44 | |
| clarkb | http://logs.openstack.org/02/443002/14/experimental/gate-functional-dsvm-magnum-k8s-multinode-ubuntu-xenial-nv/99e91e2/logs/subnode-2/local.conf.txt.gz huh its in the local.conf, it shouldn't be | 15:45 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Refactor: block-device filesystem creation, mount and fstab https://review.openstack.org/444586 | 15:45 |
| strigazi | clarkb this looks problematic: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/magnum.yaml#n15 | 15:46 |
| *** ttx has joined #openstack-infra | 15:46 | |
| *** knangia has joined #openstack-infra | 15:46 | |
| clarkb | ah yup | 15:47 |
| strigazi | clarkb It was like this before, I noticed when I refactored our jobs | 15:47 |
| *** jkilpatr_ has joined #openstack-infra | 15:47 | |
| strigazi | But I wanted to keep things the same | 15:47 |
| strigazi | now that we know about I can remove it | 15:47 |
| clarkb | strigazi: tempest is the default, and since there is no code to disable tempest being installed I think you can just remove that and instead just set DEVSTACK_GATE_TEMPEST=0 | 15:48 |
| *** dimtruck is now known as zz_dimtruck | 15:48 | |
| clarkb | there is a different flag to set if you don't want devstack gate to run tempest | 15:48 |
| clarkb | DEVSTACK_GATE_TEMPEST_NOTESTS | 15:49 |
| *** isaacb has quit IRC | 15:49 | |
| *** nicolasbock has joined #openstack-infra | 15:49 | |
| *** Swami has joined #openstack-infra | 15:49 | |
| clarkb | so set ^ to 1 and then remove the code there | 15:49 |
| *** jkilpatr has quit IRC | 15:49 | |
| *** vgadiraj has joined #openstack-infra | 15:50 | |
| strigazi | clarkb will tempest be installed? Just the module | 15:51 |
| clarkb | strigazi: yes if you set DEVSTACK_GATE_TEMPEST=1 and DEVSTACK_GATE_TEMPEST_NOTESTS=1 then tempest will be installed only on the primary node but temepst tests will not be run | 15:51 |
| strigazi | clarkb cool | 15:52 |
| strigazi | clarkb should I push to devstach-gate for DEVSTACK_PRIMARY_CONFIG then? | 15:53 |
| *** aunnam has joined #openstack-infra | 15:53 | |
| *** amoralej is now known as amoralej|brb | 15:54 | |
| *** jianghuaw has joined #openstack-infra | 15:57 | |
| *** dharinic- has joined #openstack-infra | 15:58 | |
| *** david-lyle_ is now known as david-lyle | 15:58 | |
| *** zz_dimtruck is now known as dimtruck | 16:00 | |
| mriedem | clarkb: i've had to recheck this same patch 6 times for ansible failures, and about to do it again for a 7th time https://review.openstack.org/#/c/447657/ - the tests always pass, it's failing on a "NO MORE HOSTS LEFT" in the ansible logs | 16:00 |
| mriedem | clarkb: is there some known issue going on? | 16:00 |
| mriedem | http://logs.openstack.org/57/447657/1/gate/gate-novaclient-dsvm-functional-neutron-ubuntu-xenial/b3aaf58/_zuul_ansible/ansible_log.txt | 16:00 |
| clarkb | strigazi: ya I think we should work on getting that sorted out so this is cleaner for you (you can do the hacky thing in the meantime though) | 16:01 |
| fungi | amusing... so wiki.o.o and groups.o.o seem fine today, but cacti.o.o is back to complaining about database connectivity now | 16:01 |
| clarkb | mriedem: I'm not aware of anything but can take a look in a few | 16:01 |
| *** dprince has quit IRC | 16:01 | |
| strigazi | clarkb I'll give it a go | 16:01 |
| mriedem | thanks | 16:01 |
| clarkb | fungi: maybe they unded our mysql config that extended timeouts? or are effectively doing that at the network layer under us? | 16:01 |
| pabelanger | fungi: wow | 16:02 |
| jeblair | mriedem: http://logs.openstack.org/57/447657/1/gate/gate-novaclient-dsvm-functional-neutron-ubuntu-xenial/b3aaf58/console.html#_2017-03-23_13_14_54_954517 | 16:02 |
| jeblair | mriedem: it looks like the tests timed out | 16:02 |
| mriedem | oh hrm | 16:03 |
| thomas_oneill | so there's an instantiated unit file glean@.service but where is the 'generator' ? | 16:04 |
| mriedem | ok well that makes more sense then :) thanks | 16:04 |
| thomas_oneill | apparently instantiated unit files are set up with systemd generators? I don't see anything to do with this in the glean repo though... | 16:04 |
| clarkb | thomas_oneill: the generator is udev iirc | 16:05 |
| fungi | tcpdump on eth1 of cacti.o.o sees tons of active 3306/tcp traffic to the rfc-1918 address belonging to the current dns entry for the database instance cacti's supposed to be using | 16:05 |
| thomas_oneill | oh the udev rules thing? | 16:05 |
| clarkb | thomas_oneill: for each network device that udev detects it fires off glean@interface.service | 16:05 |
| thomas_oneill | clarkb: but how does it know how to do that, I don't suppose you remember that as well? :P I'll look at the rules file again | 16:05 |
| clarkb | thomas_oneill: ya its all in the rules file | 16:05 |
| thomas_oneill | ah yeah it's in the rules file! | 16:06 |
| *** matrohon has quit IRC | 16:06 | |
| mriedem | weird, looks like novaclient functional job started timing out consistently around 3/22 | 16:07 |
| *** dprince has joined #openstack-infra | 16:07 | |
| *** jascott1_ has joined #openstack-infra | 16:09 | |
| fungi | once again, restarting the apache2 service on cacti.o.o cleared up db connectivity issues... i'm starting to agree this seems more like an ungraceful query socket timeout and less like a spontaneous ip address change | 16:10 |
| fungi | i'll check the applied configs | 16:10 |
| openstackgerrit | Jiri Stransky proposed openstack-infra/project-config master: Use tripleo-quickstart for container upgrades job https://review.openstack.org/449203 | 16:10 |
| *** rossella_s has quit IRC | 16:11 | |
| *** adarazs has left #openstack-infra | 16:11 | |
| *** rossella_s has joined #openstack-infra | 16:12 | |
| *** spzala has quit IRC | 16:15 | |
| mriedem | jeblair: note the gap between the last test in the console log and the timeout | 16:15 |
| mriedem | it's nearly an hour of nothing | 16:15 |
| *** eharney has quit IRC | 16:17 | |
| mriedem | i start seeing this shortly after we don't see anything else until the timeout | 16:17 |
| mriedem | http://logs.openstack.org/57/447657/1/gate/gate-novaclient-dsvm-functional-neutron-ubuntu-xenial/b3aaf58/logs/screen-n-api.txt.gz?level=TRACE#_2017-03-23_12_23_50_112 | 16:17 |
| mriedem | sdague: ^ | 16:17 |
| mriedem | =WARNING REPORT==== 23-Mar-2017::12:23:58 === memory resource limit alarm set on node 'rabbit@ubuntu-xenial-osic-cloud1-s3500-8037294'. | 16:18 |
| *** eharney has joined #openstack-infra | 16:19 | |
| *** markus_z has quit IRC | 16:20 | |
| fungi | #status log Cinder volumes static.openstack.org/main08, eavesdrop.openstack.org/main01 and review-dev.openstack.org/main01 will lose connectivity Friday March 31 03:00-09:00 UTC unless replaced by Wednesday March 29. | 16:20 |
| openstackstatus | fungi: finished logging | 16:20 |
| fungi | infra-root: ^ in case any of you want to try your hand at using pvmove to migrate active lvm2 extents to replacement block storage volumes (static.o.o may present a problem since we have no capacity to temporarily add a replacement block device) | 16:20 |
| *** ccamacho has quit IRC | 16:20 | |
| fungi | alternative ideas on static.o.o also welcome | 16:21 |
| *** mdrabe has quit IRC | 16:21 | |
| mordred | fungi: do they give a timeframe on how long it'll be down? | 16:21 |
| *** erikmwilson has joined #openstack-infra | 16:21 | |
| clarkb | I read that as march 21 and march 29 initially and was confused | 16:21 |
| mordred | fungi: might be easier at this point to just take an outage if it's short enough | 16:21 |
| clarkb | I think static is the one I'd worry about | 16:21 |
| mordred | yah | 16:21 |
| clarkb | since jobs get mad if they can't write to logs | 16:21 |
| pabelanger | I'd like to try pvmove again | 16:22 |
| fungi | mordred: "Expected disruption: Loss of connection to CBS volume for up to 6 hours." | 16:22 |
| pabelanger | so I can do eavesdrop.o.o | 16:22 |
| mordred | jeez | 16:22 |
| fungi | pabelanger: yeah, that one's probably easy enough. for review-dev i don't think we should bother | 16:22 |
| fungi | pabelanger: probably want to make sure the bup backup of eavesdrop is current beforehand, just to be on the safe side | 16:23 |
| clarkb | mordred: speaking of review dev, pabelanger posted the chagne to get upstream sync working again | 16:23 |
| clarkb | mordred: are you in a place to approve and watch that today/now? | 16:23 |
| clarkb | (people were holding off on you being around to do that) | 16:23 |
| mordred | fungi: well, we could expire an additional 1TB of data from static, then maybe shrink ourselves down to 11 instead of 12 - then do a pvmove dance? | 16:23 |
| *** jamesden_ has joined #openstack-infra | 16:23 | |
| clarkb | then I can get latest gerrit 2.13.6 tag branch for us and we can work on getting that building | 16:23 |
| mordred | fungi: it sounds pretty awful, but I don't have a ton of _better_ ideas | 16:24 |
| fungi | mordred: i don't think ext4 supports online shrinking | 16:24 |
| mordred | gah | 16:24 |
| mordred | foiled again | 16:24 |
| fungi | i may be behind on modern developments though | 16:24 |
| clarkb | fungi: thats correct only online growing | 16:24 |
| pabelanger | fungi: I can get started on eavesdrop.o.o now, do you have the process you did last time handy? We should maybe document it for our goto process for such emails (if possible) | 16:24 |
| yolanda | hi fungi, clarb, i have garyk , that is having problems trying to push a signed tag. I checked the acl and everything seems fine, but he is still getting error "remote rejected, prohibited by gerrit" | 16:24 |
| jeblair | fungi, mordred: even so, a 15 minute offline shrink is shorter than 6 hours of outage | 16:24 |
| fungi | and no idea how long an offline shrink on a fairly full 12tb filesystem would take | 16:24 |
| clarkb | yolanda: garyk did you sign the tag? | 16:24 |
| yolanda | is there anything else to consider, apart from having the pushSignedTag perm | 16:24 |
| yolanda | ? | 16:24 |
| fungi | jeblair: if it truly is 15 minutes, i agree | 16:25 |
| jeblair | fungi: hrm, it may require a full fsck though | 16:25 |
| fungi | yolanda: i pointed out to mordred yesterday, manage-projects seems not to have applied that acl update | 16:25 |
| clarkb | mordred: https://review.openstack.org/#/c/433663/ is the change | 16:25 |
| fungi | so probably still a hole in our cache handling there | 16:25 |
| mordred | fungi: I re-ran manage-projects this morning when dims had an acl issue and his seemed to be better | 16:25 |
| mordred | yolanda: which project? | 16:26 |
| *** matrohon has joined #openstack-infra | 16:26 | |
| yolanda | i think i saw it properly populated, networking-l2gw project | 16:26 |
| yolanda | oh, no, it's not applied | 16:26 |
| mordred | good- so that probably happened when I ran for dims today | 16:26 |
| mordred | no? | 16:27 |
| mordred | cool - looking | 16:27 |
| *** Julien-zte has quit IRC | 16:27 | |
| fungi | jeblair: in the past, fsck (granted with error fixes) on that volume took hours | 16:27 |
| *** ricardoas has left #openstack-infra | 16:27 | |
| yolanda | https://review.openstack.org/#/admin/projects/openstack/networking-l2gw,access | 16:27 |
| yolanda | and that's the change http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/acls/openstack/networking-l2gw.config | 16:27 |
| mordred | yolanda: how about now? | 16:28 |
| *** Julien-zte has joined #openstack-infra | 16:28 | |
| yolanda | no, i cannot see pushsignedtag | 16:28 |
| *** jamesden_ has quit IRC | 16:28 | |
| *** Julien-zte has quit IRC | 16:28 | |
| fungi | fun, we _just_ got a new rackspace ticket that the host for ethercalc01 is in distress. i guess we should check in on it once they bring the host back online again (ethercalc01 is currently unreachable for me) | 16:28 |
| *** jamesden_ has joined #openstack-infra | 16:28 | |
| *** Julien-zte has joined #openstack-infra | 16:29 | |
| *** Julien-zte has quit IRC | 16:29 | |
| yolanda | mordred, so seems he was having this problem since last week, if you applied manage-projects today is strange to don't see it populated | 16:29 |
| *** jascott1_ has quit IRC | 16:30 | |
| yolanda | maybe something with the acl syntax? although i see it fine | 16:30 |
| *** Julien-zte has joined #openstack-infra | 16:30 | |
| *** Julien-zte has quit IRC | 16:30 | |
| *** Julien-zte has joined #openstack-infra | 16:30 | |
| *** tonytan4ever has quit IRC | 16:31 | |
| *** Julien-zte has quit IRC | 16:31 | |
| *** jamesden_ has quit IRC | 16:31 | |
| mordred | no - I think I see a different issue - lemme look a little more | 16:31 |
| *** tonytan4ever has joined #openstack-infra | 16:31 | |
| fungi | there was no "sanity" config applied to the trove instance for cacti.o.o, so i fixed that and restarted its instance. it probably never got that config applied though. looks like groups.o.o is in the same boat. wiki.o.o on the other hand claims to have it applied already | 16:32 |
| jeblair | fungi: thanks | 16:32 |
| yolanda | mordred, i need to step out, be back in a pair of hours | 16:33 |
| fungi | applied the sanity config to and restarted the trove instance for groups.o.o now (probably caused a brief blip for the service) but seems up again | 16:34 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/system-config master: Highlight POST_FAILURE same as FAILURE on Gerrit page https://review.openstack.org/448786 | 16:38 |
| *** mdrabe has joined #openstack-infra | 16:39 | |
| *** spzala has joined #openstack-infra | 16:39 | |
| *** ricardoas has joined #openstack-infra | 16:39 | |
| *** rbuzatu_ has quit IRC | 16:40 | |
| *** jianghuaw-m has joined #openstack-infra | 16:40 | |
| clarkb | fungi: do we know if sanity was applied before? | 16:41 |
| clarkb | also ask.o.o is postgres right and not trove? guessing this doesn't expalin the outages we've seen there | 16:41 |
| *** amoralej|brb is now known as amoralej | 16:41 | |
| *** ramishra has quit IRC | 16:43 | |
| *** rhallisey has quit IRC | 16:44 | |
| *** Apoorva has joined #openstack-infra | 16:46 | |
| *** andreas_s has quit IRC | 16:47 | |
| *** jkilpatr_ has quit IRC | 16:47 | |
| *** adrian_otto1 has joined #openstack-infra | 16:47 | |
| *** beslemon_ has quit IRC | 16:48 | |
| *** jkilpatr has joined #openstack-infra | 16:48 | |
| frickler | the gerrit experience also seems to be getting bumpy again, not as bad as it could be, but maybe plan for a restart in a quiet hour before the weekend? | 16:49 |
| *** adrian_otto has quit IRC | 16:49 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Make Ironic UEFI job voting https://review.openstack.org/449222 | 16:49 |
| fungi | clarkb: i didn't think we saw it with ask.o.o did we? | 16:50 |
| fungi | as for whether it was applied to groups.o.o and cacti.o.o previously, i don't know but it seemed to _still_ be applied to the trove instance for wiki.o.o | 16:51 |
| *** rhallisey has joined #openstack-infra | 16:51 | |
| fungi | and that was one where we saw it hit (two days in a row, though not today) | 16:51 |
| pabelanger | fungi: okay, need some help on the last step. I have the new volume attached to eavesdrop.o.o, parted and pcreate. I can now see 2 volumes in pvs. Is it just a matter of doing pvmove /dev/xvdb1 /dev/xvdc1 now? | 16:51 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Make the UEFI job in Ironic voting https://review.openstack.org/449222 | 16:52 |
| pabelanger | fungi: or is there a step to vgextend in there | 16:52 |
| *** e0ne has quit IRC | 16:53 | |
| fungi | pabelanger: i think you have to vgextend the volume group onto the new physical volume before you can use pvmove | 16:53 |
| zaro | clarkb: gerrit 2.13.7 was just released. | 16:53 |
| fungi | and then vgreduce off the old volume before you can take it out of use | 16:53 |
| clarkb | fungi: well we see ask.o.o occasionally drop off but its not mysql or trove so unrelated | 16:53 |
| pabelanger | fungi: ack | 16:53 |
| fungi | need to head out and grab some lunch... bbiab | 16:53 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: New Project - OpenStack-OCI https://review.openstack.org/449224 | 16:54 |
| *** tphummel has joined #openstack-infra | 16:54 | |
| clarkb | zaro: cool we should get that too once we are syncing upstream repos again. I will branch off whatever is latest 2.13.x release | 16:54 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Make the UEFI job in Ironic voting https://review.openstack.org/449222 | 16:55 |
| clarkb | frickler: yes melody shows that GCing is happening at an elevated rate. We likely won't make it to a quiter time | 16:55 |
| *** rlandy is now known as rlandy|brb | 16:55 | |
| *** Apoorva_ has joined #openstack-infra | 16:56 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Disable tempest service https://review.openstack.org/449225 | 16:57 |
| strigazi | clarkb have a look | 16:57 |
| *** Apoorva has quit IRC | 17:00 | |
| *** jianghuaw-m has quit IRC | 17:00 | |
| clarkb | strigazi: reviewed, there was on thing that should be changed and then a suggestion | 17:00 |
| *** trown|afk is now known as trown | 17:01 | |
| clarkb | mriedem: so did the job timeout give you enough to work on or do you still need me to dig into something? | 17:01 |
| mriedem | clarkb: sdague has a patch up to make the tests actually timeout, so we'll see if that makes a difference | 17:02 |
| mriedem | if we have a test hanging somewhere | 17:02 |
| clarkb | mordred: fyi, I don't see your console log dump in nodepool working http://logs.openstack.org/30/358730/4/check/gate-dsvm-nodepool/98f01fc/logs/screen-nodepool.txt.gz#_2017-03-23_16_18_23_913 shows it failed but I don't see console logs in there. | 17:02 |
| clarkb | mordred: I think it may be due to a yaml thing, let me test really quiick then I can comment or just push a new patchset | 17:03 |
| clarkb | ya | 17:03 |
| jeblair | oh yeah, that should just be on providers | 17:04 |
| clarkb | ya the code does labels but the devstack config is for the provider | 17:05 |
| clarkb | they need to be brought into sync (I left a -1 with comment) | 17:05 |
| *** EricGonczer_ has joined #openstack-infra | 17:05 | |
| *** rbuzatu has joined #openstack-infra | 17:06 | |
| *** rbuzatu_ has joined #openstack-infra | 17:07 | |
| *** dtantsur is now known as dtantsur|afk | 17:07 | |
| *** migi is now known as migi_afk | 17:07 | |
| *** migi_afk is now known as migi | 17:07 | |
| clarkb | I'm fine with it being done per label as thats a bit finer grained control and expect that we'll have more failures based on broken images than broken clouds at this point? but provider works too | 17:09 |
| *** nicolasbock has quit IRC | 17:09 | |
| openstackgerrit | Spyros Trigazis proposed openstack-infra/project-config master: [magnum] Disable tempest service https://review.openstack.org/449225 | 17:09 |
| *** isaacb has joined #openstack-infra | 17:11 | |
| *** rbuzatu has quit IRC | 17:11 | |
| *** mat128 has joined #openstack-infra | 17:12 | |
| *** nicolasbock has joined #openstack-infra | 17:14 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Fetch server console log if ssh connection fails https://review.openstack.org/358730 | 17:14 |
| *** amotoki has quit IRC | 17:14 | |
| *** pcaruana has quit IRC | 17:14 | |
| mordred | clarkb, jeblair: updated the config to match labels so that we can maybe get some insight into the current issue | 17:14 |
| clarkb | mnaser: re yesterday's logstash queue question it appears the elasticsearch cluster is in a yellow state trying to recover indexes. In the past we haev seen this happen and it will continuously fail to recover those indexes until the master is restarted :( so I am going to go ahead and restart elasticsaerch03 now | 17:16 |
| jeblair | mordred, clarkb: ok. let's keep the config simple because this is radically changing in v3. | 17:16 |
| *** flepied has quit IRC | 17:16 | |
| mordred | yah | 17:18 |
| mordred | jeblair: ++ | 17:18 |
| clarkb | #status log elasticsearch03 es service restarted as it was master and cluster was unable to recover indexes slowing indexing. | 17:19 |
| pabelanger | okay, eavesdrop.o.o volume migrated | 17:20 |
| *** gyee has joined #openstack-infra | 17:20 | |
| *** rbuzatu_ has quit IRC | 17:20 | |
| pabelanger | original volume removed from server too | 17:20 |
| pabelanger | fungi: ^ | 17:20 |
| clarkb | mordred: can has eyeballs on https://review.openstack.org/#/c/433663/ ? | 17:20 |
| clarkb | mordred: not sure where you are on networking-l2gw and don't want those to conflict | 17:21 |
| AJaeger | pabelanger, mordred, fungi, could either of you review two jeepyb changes for codesearch and regenerate the list of repos, please (last time this was not done automatically)? https://review.openstack.org/#/c/446646/ and https://review.openstack.org/#/c/446634 | 17:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add docs to replace a cinder volume https://review.openstack.org/449232 | 17:23 |
| pabelanger | fungi: ^ and some documentation | 17:23 |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb master: If we're going to apply acls, we need a dir https://review.openstack.org/449236 | 17:28 |
| mordred | clarkb: ^^ | 17:28 |
| mordred | clarkb: I have +A'd the cron-job and also another jeepyb change (dedup) that was waiting for me to watch | 17:29 |
| clarkb | mordred: cool thanks | 17:29 |
| *** dprince has quit IRC | 17:29 | |
| clarkb | I shall review the dirs for acls change | 17:29 |
| *** rlandy|brb is now known as rlandy | 17:30 | |
| clarkb | just as soon as gerrit loads, we may need that restart sooner than later | 17:30 |
| *** jaosorior has quit IRC | 17:30 | |
| *** ociuhandu has quit IRC | 17:33 | |
| *** gyee has quit IRC | 17:33 | |
| *** gyee has joined #openstack-infra | 17:34 | |
| mnaser | clarkb ok cool, so it was a problem :-P | 17:36 |
| mnaser | i just didnt know if it was or not but figured id point a finger at it | 17:36 |
| clarkb | mordred: and reviewed the jeepyb dirs for acls change one small thing | 17:36 |
| clarkb | mnaser: its an annoying bug and I think one that will force us to upgrade ES sooner than later | 17:36 |
| clarkb | mnaser: basically after a while master seems to go out for lunch and never come back | 17:36 |
| *** jianghuaw-m has joined #openstack-infra | 17:36 | |
| mnaser | clarkb: should get elastic to give you some support, then they can use it as a "model" large scale deployment people can see at how big it can get | 17:38 |
| *** gyee has quit IRC | 17:39 | |
| *** rossella_s has quit IRC | 17:39 | |
| *** lucasagomes is now known as lucas-afk | 17:41 | |
| *** jianghuaw-m has quit IRC | 17:41 | |
| *** krtaylor has quit IRC | 17:42 | |
| *** erikmwilson has quit IRC | 17:42 | |
| thomas_oneill | so is it just me or is it pretty horrible design that glean installs itself as a unit for every interface on the host, before its even looked at the meta/network data? | 17:43 |
| *** derekh has quit IRC | 17:43 | |
| mordred | thomas_oneill: well - I'd say the horrible design is letting systemd take over linux and make everything painful to deal with | 17:44 |
| *** erikmwilson has joined #openstack-infra | 17:44 | |
| thomas_oneill | I tried adding KERNEL=="ens2f?", as part of the udev rules and restarted, hoping that it would only try running on interfaces beginning with ens2f | 17:44 |
| clarkb | thomas_oneill: its only one unut fwiw executed by udev for each real interface | 17:44 |
| mordred | the current assumption is that things without explicit static overrides _want_ dhcp - since on most of our clouds we just dhcp all the interfaces and win | 17:45 |
| mordred | but - you've got a fun new use-case - so we may have to change some things to accommodate it well | 17:45 |
| clarkb | what is the use case ? dont think I caught that | 17:45 |
| thomas_oneill | nothing too far out - just not actually configuring the interfaces | 17:46 |
| thomas_oneill | (baremetal node with some interfaces that I'm not wanting to use) | 17:46 |
| strigazi | clarkb I have updated https://review.openstack.org/#/c/449225/ | 17:47 |
| *** dprince has joined #openstack-infra | 17:49 | |
| mordred | clarkb: thomas_oneill is using bifrost to manage baremetal - bifrost uses the config-drive interface to tell glean what to do | 17:51 |
| clarkb | fwiw I am a fan of the single shot that upstart uses (its upstart that is one ahot right?) | 17:51 |
| *** adrian_otto1 has quit IRC | 17:52 | |
| clarkb | since we have had problems with glean firing for virtual neutron interfaces in the past | 17:52 |
| clarkb | (thats since fixed) | 17:52 |
| *** links has joined #openstack-infra | 17:52 | |
| *** rhallisey has quit IRC | 17:53 | |
| mordred | me too | 17:53 |
| *** sshnaidm is now known as sshnaidm|afk | 17:54 | |
| AJaeger | these graphs look really strange - straight lines: http://grafana.openstack.org/dashboard/db/nodepool - do we still have problems launching nodes? | 17:54 |
| *** rhallisey has joined #openstack-infra | 17:54 | |
| mordred | those definitely do not look pleasing | 17:54 |
| mordred | Shrews, pabelanger: ^^ | 17:55 |
| thomas_oneill | oh i see, it's these stanzas that are screwing me up eh https://github.com/openstack-infra/glean/blob/master/glean/cmd.py#L676-L689 | 17:55 |
| mordred | thomas_oneill: yes | 17:56 |
| *** adrian_otto has joined #openstack-infra | 17:56 | |
| mordred | thomas_oneill: so I _think_ we may want to just invent some way of you explicitly disabling them in your network_info.json | 17:56 |
| *** hashar has quit IRC | 17:58 | |
| *** links has quit IRC | 17:58 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:00 | |
| pabelanger | I am unable to ssh to nodepool.o.o | 18:01 |
| pabelanger | clarkb: mordred ^ | 18:01 |
| mordred | pabelanger: I am _also_ not able to ssh to nodepool.o.o | 18:02 |
| *** MasterOfBugs has joined #openstack-infra | 18:02 | |
| *** rbrndt has joined #openstack-infra | 18:02 | |
| mordred | pabelanger: I'm checking the web console | 18:02 |
| pabelanger | k, was looking for the info for that too | 18:03 |
| jeblair | it responds to pings but not ssh? | 18:03 |
| *** Goneri has quit IRC | 18:03 | |
| *** rbrndt has quit IRC | 18:03 | |
| *** ralonsoh has quit IRC | 18:04 | |
| mordred | yes | 18:04 |
| jeblair | i've tried ssh over v4 and v6; no go | 18:04 |
| mordred | the web console is responsive | 18:04 |
| mordred | but of course I do not have a login with a password :) | 18:04 |
| thomas_oneill | what's the deal with hosting the stuff like the openstack website and other services? is it all by like redhat or rackspace or something? | 18:05 |
| jeblair | mordred: nothing interesting on console output? | 18:05 |
| mordred | jeblair: just a login prompt | 18:05 |
| mordred | jeblair: checking from command line | 18:05 |
| *** isaacb has quit IRC | 18:05 | |
| *** ZZelle has joined #openstack-infra | 18:06 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add support for github enterprise https://review.openstack.org/449258 | 18:06 |
| mordred | oh - right. rackspace doens't give content in the nova text console | 18:06 |
| jeblair | mordred, pabelanger: reboot i guess? | 18:06 |
| pabelanger | 2017-03-23 16:42:54,731 seems to be the last time ansible run on nodepool.o.o, according to puppetmaster | 18:06 |
| mordred | jeblair: yah. let's try reboot | 18:07 |
| pabelanger | sure | 18:07 |
| mordred | rebooting | 18:07 |
| mordred | thomas_oneill: the majority of our control plane runs on rackspace, although we have services running in each of the public clouds we use for nodes in nodepool. the openstack.org website also runs in rackspace, but we do not run it | 18:08 |
| *** matrohon has quit IRC | 18:08 | |
| *** bramwelt1 is now known as bramwelt | 18:08 | |
| *** bramwelt has quit IRC | 18:08 | |
| *** bramwelt has joined #openstack-infra | 18:08 | |
| pabelanger | okay I can ssh again | 18:09 |
| openstackgerrit | Jens Rosenboom proposed openstack-infra/project-config master: Fix VNC setup in neutron DVR multinode jobs https://review.openstack.org/449132 | 18:09 |
| mordred | pabelanger: as can I | 18:09 |
| *** pvaneck has joined #openstack-infra | 18:09 | |
| *** jascott1_ has joined #openstack-infra | 18:10 | |
| jeblair | mordred, pabelanger: that's weird. it was logging snmp connections until the end. | 18:10 |
| pabelanger | I see that too | 18:10 |
| *** anilvenkata has joined #openstack-infra | 18:11 | |
| mordred | yup | 18:11 |
| jeblair | (though it was not getting data) | 18:11 |
| *** nadya has joined #openstack-infra | 18:11 | |
| *** beslemon has joined #openstack-infra | 18:12 | |
| fungi | pabelanger: thanks! you should #status log the updated state of eavesdrop.o.o too, so nobody in other timezones thinks it still needs doing | 18:12 |
| jeblair | i have zero ideas. | 18:13 |
| pabelanger | jeblair: mordred: I think nodepoold was still running, because debug.log has gearman things | 18:13 |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: New Project - OCI https://review.openstack.org/449224 | 18:13 |
| *** manjeets_ has joined #openstack-infra | 18:13 | |
| pabelanger | fungi: sure | 18:14 |
| *** adrian_otto has quit IRC | 18:16 | |
| *** adrian_otto has joined #openstack-infra | 18:16 | |
| *** Sukhdev has joined #openstack-infra | 18:17 | |
| clarkb | I was on it earlier this morning to delete a couple held nodes | 18:18 |
| clarkb | seemed to be ok then | 18:18 |
| pabelanger | ya, I don't see anything in the logs | 18:18 |
| bnemec | It looks like we stopped getting new nodes about an hour and a half ago | 18:19 |
| pabelanger | bnemec: yes we are looking into it | 18:19 |
| bnemec | Yeah, I realize that. Was just trying to provide a timeframe. | 18:20 |
| clarkb | it might have been a little laggy? but nothing crazy | 18:20 |
| clarkb | (also I tend to blame local networking for that most often) | 18:20 |
| *** jordanP has quit IRC | 18:21 | |
| clarkb | mordred: did you see my comments on https://review.openstack.org/#/c/449236/ ? | 18:24 |
| mordred | clarkb: aha! good catch | 18:25 |
| *** electrofelix has quit IRC | 18:26 | |
| *** esikachev has joined #openstack-infra | 18:26 | |
| *** nadya has quit IRC | 18:26 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb master: If we're going to apply acls, we need a dir https://review.openstack.org/449236 | 18:26 |
| pabelanger | nodepool services are still stopped, are we fine bringing them back online? Or does somebody else want to look longer? | 18:28 |
| *** nicolasbock has quit IRC | 18:28 | |
| pabelanger | I can't seem to see anything wrong | 18:28 |
| mordred | me either | 18:28 |
| mordred | fine be me to bring them back | 18:29 |
| pabelanger | k | 18:30 |
| pabelanger | starting things back up | 18:30 |
| *** esikachev has quit IRC | 18:30 | |
| *** rbrndt has joined #openstack-infra | 18:30 | |
| *** beslemon_ has joined #openstack-infra | 18:30 | |
| *** beslemon has quit IRC | 18:32 | |
| *** tosky has quit IRC | 18:33 | |
| *** nadya has joined #openstack-infra | 18:34 | |
| *** anilvenkata has quit IRC | 18:34 | |
| mnaser | is there a way to find a list of failures for a job | 18:37 |
| mnaser | rather than scrolling through reviews and finding -1s | 18:37 |
| clarkb | mnaser: logstash/elasticsaerch will give you that for roughly the last 10 days | 18:37 |
| *** nicolasbock has joined #openstack-infra | 18:37 | |
| mnaser | good enough for me | 18:37 |
| clarkb | but otherwise I don't think there is a good way to look it up other than crawling gerrit with http api | 18:37 |
| mnaser | ill get searching | 18:37 |
| clarkb | mnaser: if you expand the events you'll even get a link to the log files | 18:38 |
| *** eroux has joined #openstack-infra | 18:38 | |
| *** neiljerram has quit IRC | 18:38 | |
| openstackgerrit | Merged openstack-infra/jeepyb master: Deduplicate some more code https://review.openstack.org/435469 | 18:39 |
| openstackgerrit | Mike Perez proposed openstack-infra/shade master: Accept device_id option when updating ports https://review.openstack.org/449271 | 18:39 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Add more periodic tests for OpenStack-Ansible https://review.openstack.org/449272 | 18:39 |
| clarkb | looks like jobs are starting | 18:40 |
| *** jascott1_ has quit IRC | 18:42 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Add more periodic tests for OpenStack-Ansible https://review.openstack.org/449272 | 18:42 |
| samueldmq | hi all, is it possible to easily check how many million loc OpenStack has ? | 18:42 |
| samueldmq | and how much of that is Python ? | 18:42 |
| AJaeger | samueldmq: what is OpenStack here? All big tent projects? or everything hosted in OpenStack CI? | 18:43 |
| AJaeger | samueldmq: the openstack/openstack repo might help | 18:43 |
| samueldmq | AJaeger: everything accepted in the big tent would be okay | 18:43 |
| AJaeger | https://github.com/openstack/openstack/ - but I doubt that's complete | 18:44 |
| clarkb | samueldmq: AJaeger ya probably more complete to clone everything in governance/projects.yaml and then count it all | 18:44 |
| AJaeger | samueldmq: and leave out the deb-* projects to not count everything twice ;) | 18:45 |
| AJaeger | I agree with clarkb, don't see other short cuts | 18:45 |
| samueldmq | AJaeger: clarkb: gotcha, thanks! I will have a try | 18:45 |
| mnaser | please forgive me .. this seems so simple, i tossed: build_name:"kolla" in the logstash search | 18:46 |
| mnaser | and getting 0 results | 18:46 |
| mnaser | i want to partial match kolla in the build_name? | 18:46 |
| clarkb | mnaser: due to how lucene indexes words you have to provide the whole name or use * to wildcard | 18:46 |
| openstackgerrit | Merged openstack-infra/system-config master: Run track-upstream in a cronjob https://review.openstack.org/433663 | 18:46 |
| clarkb | however using * in quotes makes it literal I think. So you need to do build_name gate-kolla* ? | 18:46 |
| clarkb | something like that | 18:46 |
| mnaser | okay there we go, thanks clarkb | 18:47 |
| mnaser | build_name:gate-kolla* did the job | 18:47 |
| mnaser | the quotes were being taken literally yeah | 18:47 |
| inc0 | speaking of gates and kolla...is zuul ok?;) | 18:48 |
| openstackgerrit | Merged openstack-infra/jeepyb master: hound: ignore deb-* projects https://review.openstack.org/446634 | 18:48 |
| openstackgerrit | Merged openstack-infra/jeepyb master: Codesearch: Ignore dead repos https://review.openstack.org/446646 | 18:48 |
| mordred | inc0: _zuul_ is great - nodepool, on the other hand, had a sad | 18:48 |
| clarkb | inc0: yes zuul is fine. Nodepool did go out to lunch early but pabelanger et al have chased it down and put it back to work | 18:48 |
| inc0 | right, pardon my nomenclature | 18:48 |
| mordred | no - it's a good question | 18:48 |
| mordred | we're just answering more pedantically :) | 18:49 |
| inc0 | correcting other peoples language is always good thing | 18:49 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Add MQTT support to the gearman worker https://review.openstack.org/449279 | 18:49 |
| inc0 | at least that's what grammar nazi in me says | 18:49 |
| mtreinish | clarkb, fungi: ^^^ do you think we should do something similar for the lgostash worker? | 18:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Add more periodic tests for OpenStack-Ansible https://review.openstack.org/449272 | 18:50 |
| clarkb | mtreinish: we could but the volume is a lot higher (more files) | 18:50 |
| *** dprince has quit IRC | 18:51 | |
| *** esikachev has joined #openstack-infra | 18:51 | |
| mtreinish | clarkb: it is, but I don't think we've ever pushed the limit of the broker | 18:51 |
| mtreinish | clarkb: the issues we were having with logstash and mqtt before was because the logstash plugin didn't work at our volume | 18:52 |
| mtreinish | it kept crashing the daemons | 18:52 |
| clarkb | and this would be a layer up in the python worker ya? in that case should be fine | 18:52 |
| fungi | mtreinish: i think we ended up disabling it for the log worker because the volume of logs when funneled through was more than would fit through the network inteface | 18:52 |
| mtreinish | fungi: oh is that what was crashing the daemons? | 18:52 |
| mtreinish | heh, I didn't realize that's why | 18:52 |
| mtreinish | I assumed it was just a bad plugin | 18:53 |
| clarkb | also this would be quite a bit less since its per job not per log event | 18:53 |
| fungi | okay, so probably not _as_ bad | 18:53 |
| fungi | but still this is decompressed log data | 18:53 |
| mtreinish | well I can write the dual of that patch for the logstash gearman worker | 18:54 |
| mtreinish | if it doesn't work we can always turn it off | 18:54 |
| *** tosky has joined #openstack-infra | 18:54 | |
| mtreinish | fungi: I'll just put a status in the payload, I don't think we need to actually send the logs over the wire (which is what the logstash plugin was doing iirc) | 18:54 |
| fungi | right, so this should be way lower volume | 18:55 |
| fungi | seems fine | 18:55 |
| openstackgerrit | Mike Perez proposed openstack-infra/shade master: Accept device_id option when updating ports https://review.openstack.org/449271 | 18:55 |
| *** esikachev has quit IRC | 18:56 | |
| openstackgerrit | Anton Studenov proposed openstack-infra/project-config master: [Rally] Make gate-rally-tox-self-* voting https://review.openstack.org/449282 | 18:58 |
| clarkb | mordred: looking at https://review.openstack.org/#/c/433663/3/modules/openstack_project/manifests/gerrit.pp again beacuse wondering why it hadn't applied yet (because it just merged) and wonderif if that will run the cron job constantly? what does minute default to in puppet? | 18:59 |
| mordred | uh | 18:59 |
| bkero | Should there be a statusbot tweet about the worker shortage? | 18:59 |
| *** baoli has quit IRC | 19:00 | |
| fungi | bkero: i think the workers union was very specific that it's a strike | 19:00 |
| clarkb | the puppet docs just say "optional. if specified must be between 0 and 59 inclusive) | 19:00 |
| clarkb | which doesn't tell me what I get out of a cron created by puppet without minute specified | 19:00 |
| *** baoli has joined #openstack-infra | 19:01 | |
| clarkb | looks like the implication is * | 19:01 |
| clarkb | so ya I think that cron entry will run always | 19:01 |
| clarkb | we want minute => '42', or similar ? | 19:01 |
| yolanda | hi mordred, i still don't see the acl refreshed. Is there some issue in manage-projects? | 19:01 |
| mordred | yolanda: yes - I have a patch up to fix it | 19:02 |
| mordred | clarkb: ++ | 19:02 |
| yolanda | ah great | 19:02 |
| mordred | yolanda: https://review.openstack.org/#/c/449236/ if you wanna look | 19:02 |
| * clarkb really has to go eat something because starving | 19:02 | |
| mordred | clarkb: want me to update it? | 19:02 |
| clarkb | but maybe we should disable puppet on gerrit and update it? | 19:02 |
| clarkb | mordred: yes please also consider disabling puppet on review.o.o for now to prevent unwatned cpu spinning | 19:03 |
| yolanda | ah, so if the project is new, the acls are not created? | 19:03 |
| *** esikachev has joined #openstack-infra | 19:03 | |
| mordred | clarkb: disabled | 19:04 |
| mordred | yolanda: actually the opposite - it's not doing acl updates to existing projects because there was no git repo present | 19:04 |
| *** nadya has quit IRC | 19:04 | |
| *** nadya has joined #openstack-infra | 19:05 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add a minute argument to the track_upstream cron https://review.openstack.org/449287 | 19:07 |
| mordred | clarkb: ^^ | 19:07 |
| *** nadya has quit IRC | 19:09 | |
| openstackgerrit | Anthony Lin proposed openstack-infra/project-config master: Add New Test Type to allow handling of both "fast" and "slow" gate for Patrole https://review.openstack.org/449289 | 19:10 |
| *** Goneri has joined #openstack-infra | 19:11 | |
| *** nadya has joined #openstack-infra | 19:12 | |
| *** sree has joined #openstack-infra | 19:12 | |
| pabelanger | #status log eavesdrop.o.o cinder volume rotated to avoid rackspace outage on Friday March 31 03:00-09:00 UTC | 19:13 |
| openstackstatus | pabelanger: finished logging | 19:13 |
| *** krtaylor has joined #openstack-infra | 19:15 | |
| bnemec | Hmm, looks like our nodepool instances from before the restart are still hanging around unused. I assume nodepool will clean those up eventually? | 19:16 |
| *** nadya has quit IRC | 19:16 | |
| *** sree has quit IRC | 19:17 | |
| pabelanger | yes | 19:17 |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Refactor: block-device filesystem creation, mount and fstab https://review.openstack.org/444586 | 19:17 |
| *** baoli has quit IRC | 19:17 | |
| bnemec | Okay, thanks | 19:18 |
| *** shardy is now known as shardy_afk | 19:18 | |
| mnaser | fyi just got a random "cannot get response from upstream" in gerrit, f5 fixed it but there might be something | 19:20 |
| mnaser | 502 proxy error | 19:20 |
| mnaser | looks liek something is up :< | 19:21 |
| clarkb | mnaser: ya its the GCing it will need to be restarted I can do that after lunch | 19:21 |
| mnaser | no worries just wanted to put eyes on it | 19:21 |
| openstackgerrit | Jon Bernard proposed openstack-infra/project-config master: grafana: add ceph dashboard https://review.openstack.org/449293 | 19:23 |
| *** adrian_otto has quit IRC | 19:24 | |
| *** dprince has joined #openstack-infra | 19:25 | |
| *** sshnaidm|afk has quit IRC | 19:26 | |
| *** adrian_otto has joined #openstack-infra | 19:26 | |
| *** baoli has joined #openstack-infra | 19:26 | |
| *** vhosakot has joined #openstack-infra | 19:27 | |
| *** sshnaidm has joined #openstack-infra | 19:27 | |
| *** tesseract has quit IRC | 19:28 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Add MQTT support to the gearman worker https://review.openstack.org/449294 | 19:32 |
| mtreinish | clarkb, fungi: ^^^ | 19:32 |
| mtreinish | I wasn't sure if we wanted to put anything else in the payloads or not | 19:32 |
| *** matrohon has joined #openstack-infra | 19:32 | |
| dmsimard | clarkb, pabelanger: logs.o.o uses os-loganalyze without swift, right ? | 19:33 |
| sc` | are the private ips for vms predictable at all? | 19:33 |
| *** eharney has quit IRC | 19:33 | |
| clarkb | dmsimard: ya, we stopped using swift | 19:33 |
| clarkb | sc`: no | 19:33 |
| sc` | clarkb: didn't think so | 19:34 |
| clarkb | sc`: some dont even get private IPs | 19:34 |
| *** baoli has quit IRC | 19:34 | |
| *** adrian_otto has quit IRC | 19:34 | |
| mordred | it would be neat if none of them did, but alas, such is life | 19:34 |
| dmsimard | clarkb: it doesn't look like I can get it to work without giving it a swift config ? Leaving it to the default gibberish times out because it actually tries to use it or removing the swift block, it complains that there's no swift config | 19:35 |
| sc` | without reinventing a ton of wheels, i'm trying to get ip addresses in my environments file, or at least be able to point things to a predictable rmq without having to reinvent the orchestration wheel | 19:35 |
| pabelanger | bnemec: should see some improvement now. I just did a manually clean up of leaked nodes | 19:35 |
| sc` | chef's complexity, natch | 19:36 |
| mordred | sc`: well, we have a bunch of things in /etc/nodepool - do you know about those? | 19:36 |
| sc` | mordred: loosely. i may need to dynamically generate my environments file based on what nodepool says | 19:36 |
| clarkb | dmsimard: oh we may still have it configured to fall back on swif,t we probably want to clean that up and make it more optional | 19:36 |
| sc` | trying to avoid too many wheels getting reinvented before zuulv3 becomes viable | 19:36 |
| mordred | sc`: https://docs.openstack.org/infra/nodepool/scripts.html if it's helpful | 19:37 |
| mordred | sc`: and yah - it'll obviously be easier on you when we roll out the v3 | 19:37 |
| bnemec | pabelanger: Cool, thanks | 19:37 |
| sc` | yeah... been trying to conceptualize how i'd consume that with chef | 19:37 |
| mordred | sc`: carefully? :) | 19:37 |
| sc` | i was thinking like stabbing a drink box too many times | 19:38 |
| clarkb | mordred: ok I've eaten going to review your puppet cron change then hopefully restart gerrit before we turn on a bunch of the jeepyb things | 19:38 |
| mordred | yah. that sounds right | 19:38 |
| mordred | clarkb: ++ | 19:38 |
| *** trown is now known as trown|brb | 19:38 | |
| clarkb | sc`: the way we do it with ansible is to make the primary node a local connection and then list out the subnodes in a subnode group | 19:38 |
| dmsimard | clarkb: it looks like there is a "source" parameter (that you could set to "swift") but I can't see in the code what could be the other parameter -- there's a mention of "filesystems" | 19:38 |
| mordred | sc`: can you describe what the thing is you need to do - assuming I've never used chef before and only know how to say the words cookbook and recipe to sound like I konw what I'm talking about | 19:38 |
| *** bh526r has quit IRC | 19:38 | |
| clarkb | mordred: I might have to restart gerrit first as currently failing to review due to 502s | 19:39 |
| mordred | clarkb: nod | 19:39 |
| sc` | mordred: i need to get the same thing i do with chef-provisioning working in openstack ci, or as close as possible without reinventing every wheel along the way | 19:39 |
| sc` | chef-prov for me acts as somewhat of an orchestration piece | 19:39 |
| *** dougwig has joined #openstack-infra | 19:39 | |
| mordred | sc`: oh - no, sorry - I meant, you said something aobut an environment file - can you point me to an example of what that is or something? | 19:40 |
| sc` | i may have no choice but emit environments files dynamically. they're static json | 19:40 |
| clarkb | ya gerrit is not working very well going to restart it now | 19:40 |
| sc` | mordred: oh, sure. sec. | 19:40 |
| sc` | mordred: this is changing somewhat, but this is what we use locally for testing a la chefdk and vagrant: https://github.com/openstack/openstack-chef-repo/blob/master/environments/multi-node-ubuntu16.json | 19:41 |
| sc` | one per supported platform | 19:41 |
| sc` | it has to be static, as it's all interpreted as json | 19:41 |
| clarkb | mordred: there are actualyl a ton of upstream sync processes running, I don't think you disabled puppet early enough | 19:41 |
| clarkb | mordred: can you work on cleaning those up hwile I restart gerrit? | 19:41 |
| mordred | clarkb: yup | 19:42 |
| sc` | which leaves me in need of an orchestration piece. ansible seems like it's the path forward, but i really don't want to put a ton of work into it just for zuulv3 to come and obsolete 95+% of it | 19:42 |
| clarkb | mordred: you probably need to hand edit the cron to match your followup change | 19:42 |
| mordred | clarkb: done | 19:43 |
| mordred | clarkb: I just deleted the cron | 19:43 |
| clarkb | tyty | 19:43 |
| *** trown|brb is now known as trown | 19:44 | |
| clarkb | and system load is falling now. Maybe we also want a lock around that cron? | 19:44 |
| *** baoli has joined #openstack-infra | 19:44 | |
| mordred | clarkb: maybe so | 19:44 |
| clarkb | I approved it as is as that fixes the immediate problem | 19:45 |
| openstackgerrit | Anthony Lin proposed openstack-infra/project-config master: Add New Test Type to allow handling of both "fast" and "slow" gate for Patrole https://review.openstack.org/449289 | 19:45 |
| mordred | sc`: so - we do some things with ansible already in devstack-gate you might be able to copy without too much effort | 19:45 |
| mordred | sc`: or maybe just a silly thing to run to generate the environments file from what it finds in the /etc/nodepool files? (that might be gross though) | 19:46 |
| *** amoralej is now known as amoralej|off | 19:46 | |
| clarkb | zaro: openstack/2.13.7 branch has been created on openstack-infra/gerrit project | 19:46 |
| sc` | mordred: that's one place paths keep converging toward | 19:46 |
| openstackgerrit | Anthony Lin proposed openstack-infra/project-config master: Add New Test Type to allow handling of both "fast" and "slow" gate for Patrole https://review.openstack.org/449289 | 19:47 |
| clarkb | #status log review.openstack.org's gerrit process was restarted to address our friends Garbage Collector and Memory Leak | 19:47 |
| *** spzala has quit IRC | 19:49 | |
| sc` | murf. seems like the alternative is more cookbook changes | 19:49 |
| *** eharney has joined #openstack-infra | 19:49 | |
| AJaeger | fungi, thanks for approving the jeepyb changes like https://review.openstack.org/#/c/446646/. Could an infra-root please update the repos for codesearch? | 19:53 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config master: Make the scenario multinode job multinode https://review.openstack.org/449300 | 19:54 |
| *** vhosakot has quit IRC | 19:54 | |
| *** vhosakot_ has joined #openstack-infra | 19:55 | |
| andreaf | sdague, clarkb: ^^^ quick d-g patch if you have a sec | 19:55 |
| *** xyang1 has joined #openstack-infra | 19:57 | |
| *** skelso has joined #openstack-infra | 19:59 | |
| clarkb | andreaf: one small thing | 19:59 |
| *** dtardivel has quit IRC | 20:00 | |
| *** annegentle has quit IRC | 20:00 | |
| andreaf | clarkb: fix coming... thanks for the review | 20:00 |
| *** annegentle has joined #openstack-infra | 20:01 | |
| *** esikachev has quit IRC | 20:02 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config master: Make the scenario multinode job multinode https://review.openstack.org/449300 | 20:02 |
| andreaf | clarkb: ^^^ | 20:03 |
| clarkb | andreaf: you'll also need to upate the job if it isn't already using node-release | 20:04 |
| clarkb | andreaf: in jenkins/jobs/whereverjobis.yaml | 20:04 |
| andreaf | clarkb: heh sorry that was too quick | 20:05 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config master: Make the scenario multinode job multinode https://review.openstack.org/449300 | 20:06 |
| andreaf | clarkb: that should do it :) | 20:06 |
| clarkb | so many distractions. I'm hoping to delete the old zanata production server this afternoon. Can someone second set of eyeballs me that c2c0dff1-db14-4df9-afd1-c06f722d736d is the server to delete on rax dfw? | 20:10 |
| *** baoli has quit IRC | 20:12 | |
| *** asselin_ has quit IRC | 20:12 | |
| *** nadya has joined #openstack-infra | 20:12 | |
| *** askb has joined #openstack-infra | 20:17 | |
| *** nadya has quit IRC | 20:17 | |
| *** spzala has joined #openstack-infra | 20:17 | |
| *** jkilpatr has quit IRC | 20:21 | |
| *** esikachev has joined #openstack-infra | 20:23 | |
| mordred | clarkb: looking | 20:23 |
| *** spzala has quit IRC | 20:23 | |
| mordred | clarkb: I confirm that that is the uuid of the server named translate.openstack.org and the one we want to delete. translate01.openstack.org is the good one | 20:25 |
| clarkb | ianychoi: ^ can you let me know when you are around just in case? | 20:25 |
| clarkb | ianychoi: I will go ahead with the delete shortly | 20:26 |
| ianychoi | clarkb, looking messages now | 20:26 |
| ianychoi | clarkb, yep I am fine with your deletion on the old production of translate.o.o | 20:27 |
| *** esikachev has quit IRC | 20:27 | |
| clarkb | ianychoi: mordred ok deleting the uuid above now | 20:28 |
| ianychoi | clarkb, yep :) Just let me know after the deletion is finished | 20:29 |
| clarkb | and its gone | 20:29 |
| clarkb | ianychoi: ^ all done | 20:29 |
| ianychoi | clarkb, wow all upgrade stuffs have been done - thanks a lot! | 20:29 |
| *** hashar has joined #openstack-infra | 20:30 | |
| mnaser | is nodepool back to life or still? | 20:30 |
| *** eharney has quit IRC | 20:30 | |
| clarkb | mnaser: its back but likely slow cathing up | 20:30 |
| mnaser | ah gotcha | 20:30 |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate master: Capture any devstack unit logs https://review.openstack.org/449311 | 20:31 |
| clarkb | ianychoi: thank you for sticking with it ! | 20:32 |
| *** baoli has joined #openstack-infra | 20:32 | |
| openstackgerrit | Merged openstack-infra/jeepyb master: If we're going to apply acls, we need a dir https://review.openstack.org/449236 | 20:32 |
| ianychoi | clarkb, hoo.. I will take a rest for a few days and move on other infra stuffs regarding I18n :) | 20:34 |
| clarkb | zaro: reading gerrit 2.13 docs looks like the accounts index is something we'll need to add support for | 20:34 |
| ianychoi | clarkb, it would be also nice if you have time and then see https://review.openstack.org/#/c/440825/ for translation checksite. See u! | 20:34 |
| clarkb | zaro: we also need to add the new hooks plugin to our plugins | 20:34 |
| clarkb | ianychoi: ok I'll put it on my list | 20:35 |
| ianychoi | Thanks! | 20:35 |
| *** kgiusti has left #openstack-infra | 20:35 | |
| openstackgerrit | Anthony Lin proposed openstack-infra/project-config master: Add test types to patrole jobs https://review.openstack.org/449289 | 20:36 |
| *** xarses_ has quit IRC | 20:36 | |
| *** e0ne has joined #openstack-infra | 20:37 | |
| *** andreww has joined #openstack-infra | 20:37 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:39 | |
| *** esikachev has joined #openstack-infra | 20:40 | |
| *** rbuzatu has joined #openstack-infra | 20:42 | |
| mnaser | be fun if nodepool/zuul had a turbo button to catch up >:) | 20:42 |
| *** mtanino has joined #openstack-infra | 20:45 | |
| *** esikachev has quit IRC | 20:45 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Add MQTT support to the gearman worker https://review.openstack.org/449294 | 20:49 |
| *** flepied has joined #openstack-infra | 20:49 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Add MQTT support to the gearman worker https://review.openstack.org/449279 | 20:50 |
| *** rbuzatu has quit IRC | 20:50 | |
| fungi | pabelanger: how did you update the config on codesearch.o.o? just use the ad-hoc playbook? | 20:53 |
| *** jcoufal has quit IRC | 20:54 | |
| * fungi is trying to figure out the easy way to get the updated hound config applied | 20:54 | |
| *** jcoufal has joined #openstack-infra | 20:55 | |
| *** mtanino has quit IRC | 20:56 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add a minute argument to the track_upstream cron https://review.openstack.org/449287 | 20:56 |
| *** esikachev has joined #openstack-infra | 20:56 | |
| pabelanger | fungi: yes, kick.sh | 20:56 |
| *** jose-idar has joined #openstack-infra | 20:57 | |
| *** trown is now known as trown|outtypewww | 20:57 | |
| *** rbrndt has quit IRC | 20:58 | |
| *** yamahata has joined #openstack-infra | 20:58 | |
| *** adrian_otto has joined #openstack-infra | 20:59 | |
| *** dprince has quit IRC | 20:59 | |
| openstackgerrit | Eric Kao proposed openstack-infra/project-config master: Add periodic-newton/ocata jobs to openstack/congress https://review.openstack.org/449321 | 21:00 |
| *** esikachev has quit IRC | 21:01 | |
| *** jascott1_ has joined #openstack-infra | 21:01 | |
| *** jose-idar has left #openstack-infra | 21:02 | |
| *** esikachev has joined #openstack-infra | 21:02 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/project-config master: Run periodic tripleo job in periodic every 8 hours https://review.openstack.org/443964 | 21:05 |
| *** rfolco has quit IRC | 21:06 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Refactor: block-device filesystem creation, mount and fstab https://review.openstack.org/444586 | 21:07 |
| *** esikachev has quit IRC | 21:07 | |
| *** dimtruck is now known as zz_dimtruck | 21:07 | |
| *** krtaylor has quit IRC | 21:08 | |
| *** jkilpatr has joined #openstack-infra | 21:08 | |
| *** sdague has quit IRC | 21:10 | |
| *** aysyd has quit IRC | 21:11 | |
| *** nadya has joined #openstack-infra | 21:13 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: New Project - LOCI https://review.openstack.org/449224 | 21:14 |
| *** ldnunes has quit IRC | 21:15 | |
| *** mat128 has quit IRC | 21:16 | |
| *** mtanino has joined #openstack-infra | 21:16 | |
| *** tjones has joined #openstack-infra | 21:17 | |
| *** nadya has quit IRC | 21:18 | |
| tjones | hi i am looking for some advice. I want to configure our upstream CI (jenkins) not to run on patches that are abandoned or in merge conflict. does anyone have any pointers on how to do this? | 21:18 |
| clarkb | tjones: are you running a zuul? | 21:19 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul feature/zuulv3: Perform pre-launch merge checks https://review.openstack.org/446275 | 21:19 |
| tjones | no we are not | 21:20 |
| *** jcoufal has quit IRC | 21:20 | |
| *** jascott1_ has quit IRC | 21:20 | |
| clarkb | tjones: I'm not sure what facilities the gerrit jenkins plugin has for that. But zuul already solves both cases for us | 21:20 |
| *** jascott1_ has joined #openstack-infra | 21:20 | |
| openstackgerrit | Anthony Lin proposed openstack-infra/project-config master: Add Test Types to Patrole Jobs https://review.openstack.org/449289 | 21:21 |
| clarkb | you could have the jobs cehck when they start and fail fast if they don't meet the criteria | 21:21 |
| tjones | clarkb: is it this one ? https://docs.openstack.org/infra/zuul/ | 21:21 |
| clarkb | tjones: yes | 21:22 |
| tjones | thanks! | 21:22 |
| clarkb | tjones: the check early steps is basically what zuul does, it just happens to do them behind the scenes | 21:23 |
| tjones | that looks like exatcly what we need. we are wasting time/resources running on patches that don't need it | 21:24 |
| clarkb | mordred: if you are still around can we reenable puppet on review.o.o and keep an eye on the cron to make sure it doe what we want now? | 21:24 |
| *** jascott1_ has quit IRC | 21:25 | |
| mordred | yes! | 21:25 |
| *** jascott1_ has joined #openstack-infra | 21:25 | |
| clarkb | assuming the fix merged? did it merge? | 21:26 |
| clarkb | it did | 21:26 |
| *** matrohon has quit IRC | 21:27 | |
| *** esberglu has quit IRC | 21:29 | |
| clarkb | mordred: you removing it from emergency file or should I? | 21:30 |
| *** florianf has quit IRC | 21:30 | |
| clarkb | (I've got to go watch 1/2 of my children while the other half visit the doctor in a few minutes) | 21:30 |
| *** jascott1_ has quit IRC | 21:30 | |
| mordred | clarkb: i'll remove it | 21:30 |
| clarkb | thanks | 21:31 |
| *** annegentle has quit IRC | 21:31 | |
| *** jascott1_ has joined #openstack-infra | 21:32 | |
| *** jascott1_ has quit IRC | 21:35 | |
| *** zz_dimtruck is now known as dimtruck | 21:36 | |
| *** mtanino has quit IRC | 21:36 | |
| *** esikachev has joined #openstack-infra | 21:37 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: New Project - LOCI https://review.openstack.org/449224 | 21:40 |
| *** e0ne has quit IRC | 21:40 | |
| *** burgerk has quit IRC | 21:40 | |
| *** esikachev has quit IRC | 21:41 | |
| *** jascott1_ has joined #openstack-infra | 21:42 | |
| *** esikachev has joined #openstack-infra | 21:44 | |
| jhesketh | Morning | 21:44 |
| *** spzala has joined #openstack-infra | 21:45 | |
| *** adrian_otto has quit IRC | 21:47 | |
| *** eroux has quit IRC | 21:50 | |
| *** eroux has joined #openstack-infra | 21:52 | |
| *** esikachev has quit IRC | 21:52 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Update nodepool config syntax https://review.openstack.org/448814 | 21:52 |
| *** jascott1_ has quit IRC | 21:52 | |
| *** jascott1_ has joined #openstack-infra | 21:53 | |
| andreaf | AJaeger, yolanda: quick project-config patch if you have a moment: https://review.openstack.org/#/c/449300/ | 21:56 |
| *** jascott1_ has quit IRC | 21:57 | |
| *** Swami has quit IRC | 22:01 | |
| *** scottda has quit IRC | 22:04 | |
| *** jascott1_ has joined #openstack-infra | 22:06 | |
| *** mdrabe has quit IRC | 22:07 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Close testenv-client before zuul finishes https://review.openstack.org/410470 | 22:12 |
| *** erikmwilson has quit IRC | 22:12 | |
| *** nadya has joined #openstack-infra | 22:14 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/project-config master: New Project - LOCI https://review.openstack.org/449224 | 22:15 |
| *** erikmwilson has joined #openstack-infra | 22:16 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Update docs for configuration syntax change https://review.openstack.org/449339 | 22:18 |
| *** nadya has quit IRC | 22:19 | |
| *** kzaitsev_mb has quit IRC | 22:20 | |
| *** sshnaidm is now known as sshnaidm|off | 22:21 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Set agent modes for DVR+HA multinode job https://review.openstack.org/383827 | 22:21 |
| *** rbuzatu has joined #openstack-infra | 22:22 | |
| *** rbuzatu has quit IRC | 22:26 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Remove api-timeout and provider.image-type https://review.openstack.org/449342 | 22:29 |
| *** baoli has quit IRC | 22:30 | |
| openstackgerrit | Dave McCowan proposed openstack-infra/project-config master: Adding Python3 Functional Test Gate for Barbican https://review.openstack.org/449343 | 22:30 |
| *** dave-mcc_ has joined #openstack-infra | 22:32 | |
| *** xyang1 has quit IRC | 22:33 | |
| *** dave-mccowan has quit IRC | 22:36 | |
| *** rbrndt has joined #openstack-infra | 22:36 | |
| *** dave-mccowan has joined #openstack-infra | 22:38 | |
| *** gildub has joined #openstack-infra | 22:38 | |
| *** dave-mcc_ has quit IRC | 22:40 | |
| *** aeng has quit IRC | 22:46 | |
| *** gordc has quit IRC | 22:46 | |
| *** aeng has joined #openstack-infra | 22:47 | |
| *** rbrndt has quit IRC | 22:48 | |
| *** rbrndt has joined #openstack-infra | 22:48 | |
| *** rbrndt has quit IRC | 22:48 | |
| *** vhosakot_ has quit IRC | 22:50 | |
| *** marst has quit IRC | 22:50 | |
| *** jamesden_ has joined #openstack-infra | 22:50 | |
| *** jamesden_ has quit IRC | 22:50 | |
| *** tjones has quit IRC | 22:50 | |
| *** wolverineav has quit IRC | 22:54 | |
| *** wolverineav has joined #openstack-infra | 22:55 | |
| zaro | clarkb: i think the default reindex will reindex both accounts and changes | 22:59 |
| zaro | clarkb: probably don’t need any extra work for hooks, the puppet will install all core plugins | 22:59 |
| clarkb | zaro: oh cool | 23:01 |
| clarkb | zaro: it does say you have to do the accounts one offline | 23:04 |
| clarkb | but thats easy enough to do once | 23:04 |
| *** skelso has quit IRC | 23:04 | |
| *** esikachev has joined #openstack-infra | 23:06 | |
| *** hashar has quit IRC | 23:07 | |
| *** wolverineav has quit IRC | 23:07 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Refactor out Changeish https://review.openstack.org/448913 | 23:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Remove deprecated networks syntax https://review.openstack.org/449354 | 23:10 |
| *** esikachev has quit IRC | 23:10 | |
| *** ihrachys has quit IRC | 23:12 | |
| *** sree has joined #openstack-infra | 23:13 | |
| *** esikachev has joined #openstack-infra | 23:15 | |
| *** andreww has quit IRC | 23:16 | |
| *** sree has quit IRC | 23:18 | |
| *** MasterOfBugs has quit IRC | 23:19 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Refactor out Changeish https://review.openstack.org/448913 | 23:20 |
| *** esikachev has quit IRC | 23:20 | |
| *** pahuang has joined #openstack-infra | 23:21 | |
| *** rlandy is now known as rlandy|afk | 23:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Fetch server console log if ssh connection fails https://review.openstack.org/358730 | 23:26 |
| *** spzala has quit IRC | 23:27 | |
| *** spzala has joined #openstack-infra | 23:30 | |
| openstackgerrit | Annie Lezil proposed openstack-infra/elastic-recheck master: Adding e-r query for ironic-inspector gate failure https://review.openstack.org/449361 | 23:32 |
| *** spzala_ has joined #openstack-infra | 23:34 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Remove api-timeout and provider.image-type https://review.openstack.org/449342 | 23:34 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Remove deprecated networks syntax https://review.openstack.org/449354 | 23:34 |
| *** spzala has quit IRC | 23:34 | |
| *** spzala has joined #openstack-infra | 23:35 | |
| *** tjones has joined #openstack-infra | 23:36 | |
| *** spzala_ has quit IRC | 23:38 | |
| *** hongbin has quit IRC | 23:38 | |
| *** spzala has quit IRC | 23:39 | |
| *** nicolasbock has quit IRC | 23:40 | |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config master: Changing octavia service type to load-balancer https://review.openstack.org/449363 | 23:41 |
| *** adrian_otto has joined #openstack-infra | 23:42 | |
| *** tjones has quit IRC | 23:44 | |
| *** beslemon_ has quit IRC | 23:47 | |
| openstackgerrit | Dave McCowan proposed openstack-infra/project-config master: Adding Python3 Functional Test Gate for Barbican https://review.openstack.org/449343 | 23:50 |
| *** armax has quit IRC | 23:55 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add trigger capability on github pr review https://review.openstack.org/449365 | 23:56 |
| *** spzala has joined #openstack-infra | 23:58 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/glean master: Always define link_type in debian interfaces https://review.openstack.org/449366 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!