Thursday, 2018-10-11

*** kjackal has quit IRC00:00
*** kjackal has joined #openstack-infra00:00
*** rpittau has quit IRC00:03
*** rpittau has joined #openstack-infra00:03
*** rkukura_ has joined #openstack-infra00:09
*** vabada has quit IRC00:10
*** strigazi has quit IRC00:10
*** vabada has joined #openstack-infra00:10
*** yamamoto_ has joined #openstack-infra00:11
*** strigazi has joined #openstack-infra00:11
*** rkukura has quit IRC00:11
*** rkukura_ is now known as rkukura00:11
*** devananda has quit IRC00:12
*** wznoinsk has quit IRC00:12
*** devananda has joined #openstack-infra00:13
*** panda has quit IRC00:13
*** yamamoto has quit IRC00:13
*** carl_cai has quit IRC00:14
*** gyee has quit IRC00:15
*** panda has joined #openstack-infra00:16
*** rcernin has quit IRC00:29
*** bobh has joined #openstack-infra00:38
*** zhurong has joined #openstack-infra00:50
*** zhurong has quit IRC00:50
*** longkb has joined #openstack-infra00:52
*** eernst has quit IRC01:08
*** slaweq has joined #openstack-infra01:11
*** bobh has quit IRC01:15
*** slaweq has quit IRC01:15
*** bobh has joined #openstack-infra01:16
openstackgerritTony Breeds proposed openstack-infra/yaml2ical master: add monthly recurrence options  https://review.openstack.org/60868001:17
*** gagehugo_ has quit IRC01:24
*** gagehugo has joined #openstack-infra01:25
*** kaisers has quit IRC01:27
*** kaisers has joined #openstack-infra01:27
*** hongbin has joined #openstack-infra01:30
*** yamamoto_ has quit IRC01:32
*** yamamoto has joined #openstack-infra01:32
*** bobh has quit IRC01:34
*** mrsoul has quit IRC01:34
*** aspiers[m] has quit IRC01:37
*** clarkb has quit IRC01:38
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add version to info endpoint  https://review.openstack.org/60957101:41
*** studarus has joined #openstack-infra01:49
*** annp has quit IRC01:50
*** longkb has quit IRC01:50
*** longkb has joined #openstack-infra01:51
*** rcernin has joined #openstack-infra02:01
*** chrisyang_0660 has joined #openstack-infra02:06
*** longkb has quit IRC02:08
*** longkb has joined #openstack-infra02:09
*** jamesmcarthur has joined #openstack-infra02:09
*** dave-mccowan has quit IRC02:21
*** bobh has joined #openstack-infra02:22
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: builds: set missing start/end time for node errors  https://review.openstack.org/53554902:23
*** dave-mccowan has joined #openstack-infra02:24
*** stakeda has joined #openstack-infra02:28
*** bobh has quit IRC02:30
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: fix stream url from status json  https://review.openstack.org/60936502:34
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: remove legacy stream.html url support  https://review.openstack.org/60936602:34
*** felipemonteiro has joined #openstack-infra02:38
*** lbragstad has quit IRC02:40
openstackgerritMerged openstack-infra/irc-meetings master: Remove the nova cellsv2 meeting  https://review.openstack.org/60949602:42
*** rh-jelabarre has quit IRC02:43
*** rosmaita has left #openstack-infra02:44
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: use window.location in TableFilters to keep sub-url path  https://review.openstack.org/60957802:47
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add job page  https://review.openstack.org/59704802:53
*** armax has joined #openstack-infra02:53
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add config-errors notifications drawer  https://review.openstack.org/59714702:53
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add change status page  https://review.openstack.org/59947202:55
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add build page  https://review.openstack.org/59702402:57
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add project page  https://review.openstack.org/60426602:58
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add job page  https://review.openstack.org/59704802:58
*** armax_ has joined #openstack-infra03:05
*** armax has quit IRC03:06
*** armax_ is now known as armax03:06
*** jamesmcarthur has quit IRC03:06
*** psachin has joined #openstack-infra03:07
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add config-errors notifications drawer  https://review.openstack.org/59714703:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add change status page  https://review.openstack.org/59947203:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add project page  https://review.openstack.org/60426603:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add labels page  https://review.openstack.org/60468203:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add nodes page  https://review.openstack.org/60468303:15
*** ramishra has joined #openstack-infra03:16
*** dave-mccowan has quit IRC03:16
*** lifeless has joined #openstack-infra03:18
lifelessis the forced-registration thing likely to be reversed at any point? Its interacting with netsplits/disconnects poorly and all my channels get lost because irssi doesn't have a clue whats going on.03:19
*** armax has quit IRC03:26
*** hongbin has quit IRC03:34
*** roman_g has joined #openstack-infra04:07
*** felipemonteiro has quit IRC04:11
*** felipemonteiro has joined #openstack-infra04:17
*** longkb has quit IRC04:23
*** longkb has joined #openstack-infra04:24
*** janki has joined #openstack-infra04:26
*** felipemonteiro has quit IRC04:41
*** carl_cai has joined #openstack-infra04:52
*** ykarel has joined #openstack-infra05:07
*** slaweq has joined #openstack-infra05:11
*** roman_g has quit IRC05:12
*** slaweq has quit IRC05:16
*** sshnaidm|afk has quit IRC05:19
*** olivierb has joined #openstack-infra05:21
*** chkumar|off is now known as chandankumar05:26
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: fix stream url from status json  https://review.openstack.org/60936505:37
*** udesale has joined #openstack-infra05:39
*** quiquell|off is now known as quiquell05:42
*** auristor has quit IRC05:45
*** auristor has joined #openstack-infra05:52
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix broken code block in the docs  https://review.openstack.org/60959005:55
*** pcaruana has joined #openstack-infra06:01
*** e0ne has joined #openstack-infra06:02
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove converted tempest jobs  https://review.openstack.org/60959306:04
*** kushinauwu has joined #openstack-infra06:06
fricklerlifeless: you can set up irssi to use sasl so that your nick gets registered on login automatically06:09
fricklerlifeless: sadly the spammers don't seem to go away, so I expect this measure to be needed for an extended time06:09
*** olivierb has quit IRC06:11
*** olivierb has joined #openstack-infra06:12
openstackgerritGhanshyam Mann proposed openstack-infra/openstack-zuul-jobs master: Remove the legacy jobs for Tempest pre-provisioned cred  https://review.openstack.org/60959406:12
*** apetrich has quit IRC06:20
*** dims has quit IRC06:29
*** e0ne has quit IRC06:29
*** slaweq has joined #openstack-infra06:29
*** xinliang has joined #openstack-infra06:31
*** dims has joined #openstack-infra06:33
*** jrist has joined #openstack-infra06:33
*** janki has quit IRC06:34
*** slaweq has quit IRC06:34
*** jchhatba_ has joined #openstack-infra06:35
*** dims has quit IRC06:38
*** dims has joined #openstack-infra06:39
*** slaweq has joined #openstack-infra06:41
*** apetrich has joined #openstack-infra06:43
*** jchhatba_ has quit IRC06:46
*** slaweq has quit IRC06:46
*** kushinauwu has quit IRC06:47
*** janki has joined #openstack-infra06:47
*** aojea has joined #openstack-infra06:48
*** longkb has quit IRC06:49
*** kushinauwu has joined #openstack-infra06:50
*** slaweq has joined #openstack-infra06:50
*** longkb has joined #openstack-infra06:50
*** roman_g has joined #openstack-infra06:50
*** radez has quit IRC06:52
*** longkb has quit IRC06:53
*** longkb has joined #openstack-infra06:54
*** stakeda has quit IRC06:59
*** rcernin has quit IRC07:01
*** carl_cai has quit IRC07:02
*** studarus has quit IRC07:02
*** slaweq has quit IRC07:03
*** slaweq has joined #openstack-infra07:05
*** shardy has joined #openstack-infra07:06
quiquellfrickler: Good morning, zuul question07:09
*** jrist has quit IRC07:09
quiquellfrickler: Do you know why we have only one queue for all tripleo projects ?07:10
*** udesale has quit IRC07:12
*** udesale has joined #openstack-infra07:13
*** udesale has quit IRC07:15
fricklerquiquell: IIUC this is a decision of the project team07:15
*** ssbarnea has joined #openstack-infra07:17
quiquellfrickler: Do you know if there is any benefict from having different projects in same gate queue ?07:20
AJaegerquiquell: please read https://zuul-ci.org/docs/zuul/user/gating.html for full theory. Basically: You have cross-dependencies between projects and want to ensure that a change in one does not affect the other.07:22
quiquellAJaeger: Thanks, will check it.07:23
quiquellAhh found it ">If changes with cross-project dependencies do not share a change queue then Zuul is unable to enqueue them together, and the first will be required to merge before the second is enqueued."07:28
quiquellAJaeger: Thanks a lot07:28
fricklerlogan-: just found a timeout from last night, maybe you can crosscheck whether that ran on the same hv? http://logs.openstack.org/69/608869/1/gate/tempest-full/cfb0a8b/07:30
*** ssbarn___ has joined #openstack-infra07:33
*** Adri2000 has joined #openstack-infra07:33
*** lpetrut has joined #openstack-infra07:40
*** roman_g has quit IRC07:47
*** udesale has joined #openstack-infra07:51
*** jpich has joined #openstack-infra07:59
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: remove legacy stream.html url support  https://review.openstack.org/60936607:59
*** kopecmartin has joined #openstack-infra08:00
*** kopecmartin is now known as kopecmartin|ruck08:01
AJaegerfrickler: want to review https://review.openstack.org/#/c/609594/ and https://review.openstack.org/609522 as two tempest changes, please?08:08
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Replace openSUSE tempest check with newer version  https://review.openstack.org/60952208:10
*** ssbarnea has quit IRC08:13
*** kopecmartin|ruck has quit IRC08:15
*** kopecmartin has joined #openstack-infra08:15
*** kopecmartin is now known as kopecmartin|ruck08:15
*** AJaeger has quit IRC08:15
*** roman_g has joined #openstack-infra08:23
*** AJaeger has joined #openstack-infra08:25
*** kushinauwu has quit IRC08:30
*** kushinauwu has joined #openstack-infra08:30
*** electrofelix has joined #openstack-infra08:42
*** derekh has joined #openstack-infra08:42
*** dtantsur|afk is now known as dtantsur|08:44
*** dtantsur| is now known as dtantsur08:44
*** ssbarnea has joined #openstack-infra08:49
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Optionally disable service worker in zuul-web  https://review.openstack.org/60962608:51
*** carl_cai has joined #openstack-infra08:53
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove the legacy jobs for Tempest pre-provisioned cred  https://review.openstack.org/60959408:54
*** ssbarnea has quit IRC08:56
openstackgerritMerged openstack-infra/system-config master: Increase sync frequency for OpenSUSE repo  https://review.openstack.org/60939509:04
*** e0ne has joined #openstack-infra09:05
*** panda is now known as panda|mtg09:05
*** ykarel_ has joined #openstack-infra09:06
*** tosky has joined #openstack-infra09:06
*** ykarel_ is now known as ykarel|lunch09:07
*** ssbarn___ has quit IRC09:08
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Minor documentation updates  https://review.openstack.org/60963609:08
*** ykarel has quit IRC09:08
*** yamamoto has quit IRC09:10
*** yamamoto has joined #openstack-infra09:11
*** kopecmartin has joined #openstack-infra09:12
*** nicolasbock has quit IRC09:12
*** jistr is now known as jistr|call09:12
lifelessfrickler: I do have it setup for sasl09:12
*** yamamoto has quit IRC09:15
openstackgerritMark Goddard proposed openstack-infra/project-config master: Add publish-to-pypi-python3 template for kayobe  https://review.openstack.org/60964109:19
*** ssbarnea has joined #openstack-infra09:19
*** e0ne has quit IRC09:27
*** AJaeger_ has joined #openstack-infra09:37
*** AJaeger has quit IRC09:40
*** AJaeger_ is now known as AJaeger09:41
*** e0ne has joined #openstack-infra09:43
*** jtomasek has joined #openstack-infra09:46
openstackgerritMerged openstack/diskimage-builder master: Fail build due to missing kauditd only when SELinux is enabled  https://review.openstack.org/60818709:46
*** kopecmartin|ruck has quit IRC09:47
*** kopecmartin is now known as kopecmartin|ruck09:47
*** yamamoto has joined #openstack-infra09:50
*** imacdonn has quit IRC09:52
*** imacdonn has joined #openstack-infra09:53
*** yamamoto has quit IRC10:02
*** yamamoto has joined #openstack-infra10:02
*** jamesmcarthur has joined #openstack-infra10:04
*** longkb has quit IRC10:06
*** miouge has quit IRC10:08
*** jamesmcarthur has quit IRC10:09
*** Miouge has joined #openstack-infra10:09
*** gouthamr has quit IRC10:14
*** stevebaker has quit IRC10:16
chandankumarAJaeger: how to bring back openstackgerrit bot to tripleo channel?10:17
*** dmellado has quit IRC10:17
*** ykarel|lunch is now known as ykarel10:19
*** kushinauwu has quit IRC10:22
*** kushinauwu has joined #openstack-infra10:22
*** kushinauwu has left #openstack-infra10:25
*** yamamoto has quit IRC10:30
*** yamamoto has joined #openstack-infra10:30
*** yamamoto has quit IRC10:34
*** e0ne has quit IRC10:35
*** e0ne has joined #openstack-infra10:38
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Make update intercal configurable at compile time  https://review.openstack.org/60966410:44
*** sshnaidm|afk has joined #openstack-infra10:45
*** mgoddard has joined #openstack-infra10:52
mgoddardHi, could someone take a look at https://review.openstack.org/609641 which will help us to produce a release of Kayobe10:52
*** yamamoto has joined #openstack-infra10:53
*** jistr|call is now known as jistr11:00
*** sshnaidm|afk is now known as sshnaidm11:02
*** e0ne has quit IRC11:04
*** aojea has quit IRC11:08
AJaegerconfig-core ^11:08
*** Emine has joined #openstack-infra11:13
*** quiquell is now known as quiquell|afk11:15
openstackgerritGary Perkins proposed openstack-infra/system-config master: Add Arm64 CI cloud  https://review.openstack.org/60243611:16
*** dtantsur is now known as dtantsur|brb11:21
*** rh-jelabarre has joined #openstack-infra11:21
*** owalsh_away has quit IRC11:22
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: use PUBLIC_URL instead of package.json patch  https://review.openstack.org/60967211:28
*** owalsh has joined #openstack-infra11:28
*** udesale has quit IRC11:39
*** janki has quit IRC11:41
*** ramishra has quit IRC11:46
*** e0ne has joined #openstack-infra11:51
*** janki has joined #openstack-infra11:51
*** priteau has joined #openstack-infra11:55
*** derekh has quit IRC12:00
*** derekh has joined #openstack-infra12:00
*** ramishra has joined #openstack-infra12:01
*** jpena|off has joined #openstack-infra12:02
*** gouthamr has joined #openstack-infra12:05
*** dave-mccowan has joined #openstack-infra12:07
*** dmellado has joined #openstack-infra12:10
*** trown|outtypewww is now known as trown12:10
*** quiquell|afk is now known as quiquell|lunch12:11
*** boden has joined #openstack-infra12:15
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: executor: enable zuul_return to update Ansible inventory  https://review.openstack.org/59009212:17
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: executor: add support for generic build resource  https://review.openstack.org/57066812:17
*** dave-mccowan has quit IRC12:19
*** mriedem has joined #openstack-infra12:21
*** stevebaker has joined #openstack-infra12:22
*** rh-jelabarre has quit IRC12:23
*** ramishra has quit IRC12:26
*** ramishra has joined #openstack-infra12:29
*** rlandy has joined #openstack-infra12:32
fungilifeless: we might be due to try reversing it again. we tried to disable it a couple weeks ago and within a week starting getting increasing amounts of in-channel spam again12:33
fungiit definitely wasn't intended as a permanent measure12:34
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: fix rewrite rule installation doc  https://review.openstack.org/60885012:39
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Make update interval configurable at compile time  https://review.openstack.org/60966412:41
*** jamesmcarthur has joined #openstack-infra12:41
*** ansmith has joined #openstack-infra12:46
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: [WIP] web: enable status update interval set in info endpoint.  https://review.openstack.org/60969012:46
*** quiquell|lunch is now known as quiquell12:47
*** armax has joined #openstack-infra12:54
*** mriedem is now known as mriedem_afk12:54
*** janki has quit IRC13:00
*** jamesmcarthur has quit IRC13:02
*** bobh has joined #openstack-infra13:05
*** ykarel has quit IRC13:05
*** ykarel has joined #openstack-infra13:05
*** kgiusti has joined #openstack-infra13:08
*** e0ne has quit IRC13:11
*** dims has quit IRC13:12
coreycbfungi: to install python3.7, would a pre-run playbook be the preferred way to go about that?13:13
*** dims has joined #openstack-infra13:14
*** armax has quit IRC13:15
*** dtantsur|brb is now known as dtantsur13:15
*** ansmith has quit IRC13:16
*** felipemonteiro has joined #openstack-infra13:17
*** dims has quit IRC13:19
openstackgerritArx Cruz proposed openstack-infra/devstack-gate master: Enable process_stackviz function to be optional  https://review.openstack.org/47578813:19
*** e0ne has joined #openstack-infra13:21
*** carl_cai has quit IRC13:22
*** ansmith has joined #openstack-infra13:24
*** eharney has joined #openstack-infra13:25
*** abregman has joined #openstack-infra13:25
*** abregman has left #openstack-infra13:25
*** abregman has joined #openstack-infra13:25
openstackgerritDan Prince proposed openstack-infra/project-config master: Import puppet-crane for TripleO  https://review.openstack.org/60953113:26
*** udesale has joined #openstack-infra13:28
*** lbragstad has joined #openstack-infra13:31
fricklerinfra-root: headsup, root zone ksk rollover is scheduled for 1600 UTC today, if anything weird should happen at that time, my first guess would be to assume some correlation13:32
*** jrist has joined #openstack-infra13:33
fricklerfungi: iirc you were looking at gerritbot missing from some channels earlier, like from #tripleo above, was that resolved?13:34
*** ginopc has joined #openstack-infra13:34
*** psachin has quit IRC13:38
fungicoreycb: i think we probably want an ensure-python role which takes a variable to indicate the version you want installed, and then use that role in the pre playbook for the tox jobs (and have the role short-circuit if that variable is unset/empty so that it's backward-compatible)13:43
*** felipemonteiro has quit IRC13:44
fungiit could also start out with debian/ubuntu-specific package names i think, and if people want to improve it to be cross-distro later they can do that13:45
*** ijw has joined #openstack-infra13:45
*** ijw has quit IRC13:46
*** ijw has joined #openstack-infra13:46
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi-python3 template for kayobe  https://review.openstack.org/60964113:51
coreycbfungi: ok thanks I'll work on getting something put together for that13:55
*** mriedem_afk is now known as mriedem13:56
*** quiquell is now known as quiquell|off13:59
*** smarcet has joined #openstack-infra14:00
smarcetfungi: morning, i am looking for a openstack sdk for php i came with this one https://github.com/php-opencloud/openstack but wonder its there is an official one ? or recommend one ?14:01
mordredsmarcet: to my knowledge, no. there was an attempt at one point to do an official one, but it died on th evine14:02
smarcetmordred: oh ok14:02
mordredsmarcet: that's the best one for php that I'm aware of - although I have not used it so cannot actually vouch for it14:03
smarcetwell try that one then, thx14:03
smarcet:)14:03
*** ssbarne__ has joined #openstack-infra14:03
AJaegerinfra-root, anybody to pick up http://lists.openstack.org/pipermail/openstack-dev/2018-October/135527.html - force merge a couple of changes for python3-first, please?14:04
mordredsmarcet: I would be curious to learn how it goes so I can answer that question better in the future14:04
smarcetmordred: sure i will let u know :)14:04
AJaegerinfra-root, can we enable OVH GRA1 again? The maintenance window is over - do you have a way to check whether we can use the region again? See https://review.openstack.org/60317814:05
*** rh-jelabarre has joined #openstack-infra14:06
fungimordred: smarcet: if it's usable, would be a good idea to update https://developer.openstack.org/ with a working link for a php sdk14:06
smarcetfungi:mordred: only issue its that has a hard requirement of php >= 7.014:07
fungithat's i suppose still better than no sdk at all14:09
smarcetfungi: indeed :P14:10
*** panda|mtg has quit IRC14:12
*** panda has joined #openstack-infra14:14
*** jrist has quit IRC14:20
*** dims has joined #openstack-infra14:22
*** knikolla has joined #openstack-infra14:22
*** dave-mccowan has joined #openstack-infra14:31
*** e0ne has quit IRC14:33
openstackgerritJames E. Blair proposed openstack-infra/zuul master: web: use PUBLIC_URL instead of package.json patch  https://review.openstack.org/60967214:33
*** jtomasek has quit IRC14:36
smcginnisCould someone tell me what x-vrif-minus-2 is for GerritBot channel notifications?14:36
AJaegersmcginnis: today: a noop ;)14:37
AJaegersmcginnis: that event is not send14:37
smcginnisAJaeger: Hah, OK. That explains why it's on openstack-cinder but I've never seen anything.14:37
fungismcginnis: it was supposed to be for notifying on a verified -2 (e.g. gate failure) but as AJaeger points out it's not wired up14:37
smcginnisDo we still want it? Looks like channels.yaml could lose a lot of lines if we stripped it.14:38
* smcginnis likes to clean up14:38
fungiit's been cargo-culted all over, yeah. i'd be fine with cleaning it out14:38
AJaegersmcginnis: https://review.openstack.org/#/c/545502/ fixes it14:38
AJaegerbut we look for the successor of gerritbot and don't want to add new "features" to gerritbot anymore14:39
smcginnisAJaeger: Do you think I should clean up the existing x-vrif-minus-2 and let folks add back x-verified-minus-2 after that lands if they want that notification?14:40
fungispeaking of things we _might_ still want to add to gerritbot (and statusbot), topic:sasl-auth will get them joining all channels successfully again. right now they're both missing from a handful of channels due to identify/join races14:41
AJaegersmcginnis: I'm fine with removing it...14:41
*** sthussey has joined #openstack-infra14:41
fungii was planning to manually apply 608314 later today and make sure it works14:42
*** e0ne has joined #openstack-infra14:42
*** dims has quit IRC14:43
abregmanhello14:44
abregmananyone knows how can I create a devstack job that runs tempest twice?14:45
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Remove x-vrif-minus-2 from gerritbot notifications  https://review.openstack.org/60972814:45
AJaegerabregman: better ask on #openstack-qa . This sounds like grenade...14:46
abregmanAJaeger: I don't want to perform upgrade. just run it twice with different regex each time14:46
abregmanAJaeger: is it possible to create such customized job? still better to ask on openstack-qa?14:47
AJaegersmcginnis: let's also update the infra-manual - see http://git.openstack.org/cgit/openstack-infra/infra-manual/tree/doc/source/creators.rst#n47014:47
AJaegerabregman: you can create any kind of customized jobs ;) Please discuss first with QA team - and if questions remain, we're happy to help...14:48
*** dims has joined #openstack-infra14:48
smcginnisAJaeger: Will do!14:48
fungicoreycb: for adding an ensure-python role, https://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles is where most of zuul's other ensure roles are managed. you can also seen how ensure-babel and ensure-sphinx roles there make sure the gettext distro package is installed if you need a more specific example14:48
abregmanAJaeger: ack. thanks14:50
coreycbfungi: great, thanks. i was just poking around various roles to get familiar. i'll have something up a little later.14:50
fungicoreycb: the python_version variable can be done similar to how constraints_file is passed into those others14:50
coreycbfungi: ok14:50
fungithough in this case you shouldn't need constraints stuff as this role won't be pip-oriented14:51
fungii expect it should be a very, very small role14:51
openstackgerritSean McGinnis proposed openstack-infra/infra-manual master: Remove reference to x-vrif-minus-2  https://review.openstack.org/60973114:51
*** Swami has joined #openstack-infra14:51
openstackgerritMerged openstack-infra/zuul master: Fix broken code block in the docs  https://review.openstack.org/60959014:54
*** ginopc has quit IRC14:54
*** ramishra has quit IRC14:58
*** openstackgerrit has quit IRC14:58
AJaegersmcginnis: what kind of hotpots are you sharing in 609731 ?14:59
*** openstackgerrit has joined #openstack-infra14:59
*** ginopc has joined #openstack-infra14:59
smcginnisAJaeger: Huh, good question. :)15:02
openstackgerritSean McGinnis proposed openstack-infra/infra-manual master: Remove reference to x-vrif-minus-2  https://review.openstack.org/60973115:03
*** mriedem has quit IRC15:03
AJaegersmcginnis: now I'm happy ;)15:03
smcginnisI need to pay more attention some times, obviously. ;)15:04
*** lbragstad has quit IRC15:04
*** lbragstad has joined #openstack-infra15:05
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Add a quick-start test job  https://review.openstack.org/60951415:05
*** ssbarne__ has quit IRC15:07
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Fixes for quick-start playbooks  https://review.openstack.org/60955115:08
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Remove disable-recommends in quick-start test  https://review.openstack.org/60955315:08
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Quick-start: remove ansible Dockerfile  https://review.openstack.org/60955515:08
*** gyee has joined #openstack-infra15:08
*** e0ne has quit IRC15:09
*** clarkb has joined #openstack-infra15:10
clarkbmaybe I should sort out sasl, though logs show I was also disconnected for quite some time15:14
fungion that note, i'm testing gerritbot with sasl (topic:sasl-auth) on review.o.o right now and the patch seems to work fine15:15
fungiit appears to have joined the expected channels15:16
*** yamamoto has quit IRC15:16
*** yamamoto has joined #openstack-infra15:17
*** yamamoto has quit IRC15:17
clarkb15:18
*** lbragstad has quit IRC15:18
*** yamamoto has joined #openstack-infra15:18
clarkbdhellmann: AJaeger: do we still plan to merge those job updates?15:18
*** lbragstad has joined #openstack-infra15:19
*** yamamoto has quit IRC15:19
*** e0ne has joined #openstack-infra15:20
*** e0ne has quit IRC15:20
dhellmannclarkb : yes! I'm in TC office hours right now but I can monitor in the background15:20
clarkbok Im still getting the day started but will pull up the list shortly15:22
*** openstackgerrit has quit IRC15:22
dhellmannclarkb : ack, thanks15:22
*** mugsie has joined #openstack-infra15:22
dhellmannclarkb : for your convenience: http://lists.openstack.org/pipermail/openstack-dev/2018-October/135527.html15:23
*** roman_g has quit IRC15:24
*** yamamoto has joined #openstack-infra15:24
*** roman_g has joined #openstack-infra15:26
*** yamamoto has quit IRC15:29
*** chandankumar is now known as chkumar|off15:32
*** mriedem has joined #openstack-infra15:32
*** jamesmcarthur has joined #openstack-infra15:34
*** openstackgerrit has joined #openstack-infra15:34
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox functional testing for drivers  https://review.openstack.org/60951515:34
*** ijw has quit IRC15:44
clarkbdhellmann: did the first merges on monasca-analytics https://review.openstack.org/#/c/595658/ I also had to force merge the parent which fixed pep8 things. I went ahead and did this one first since the ml thread indicated it was fine with this project. Once your meeting is over maybe take a look at the comments on that and make sure you are happy with it and I'll do the rest15:45
dhellmannclarkb : those comments/messages look good in terms of recording the decision15:46
dhellmannI think we're good to go ahead with the rest when you're able15:46
clarkbok15:46
*** ijw has joined #openstack-infra15:47
clarkbdhellmann: I'm noticing the barbican changes are +1 by zuul. Would you like to get barbican to approve those normally?15:48
*** jamesmcarthur has quit IRC15:49
dhellmannclarkb : at this point, let's just go ahead and do them all15:49
dhellmannif they pass, all the better15:49
dhellmannteams have had a couple of months to do this15:49
clarkbwfm15:49
*** jrist has joined #openstack-infra15:50
*** jamesmcarthur has joined #openstack-infra15:52
*** dklyle has quit IRC15:53
*** rh-jelabarre has quit IRC15:55
*** Emine has quit IRC15:55
*** dpawlik has joined #openstack-infra15:57
*** ginopc has quit IRC15:57
*** dklyle has joined #openstack-infra15:58
*** HenryG has joined #openstack-infra15:58
*** jrist has quit IRC15:59
fungiinfra-root: infra-puppet-core: it would be nice to get https://review.openstack.org/#/q/topic:sasl-auth+status:open merged so that statusbot and gerritbot can join all channels again. right now they're each missing from a handful of channels15:59
*** dtantsur is now known as dtantsur|afk16:00
*** dpawlik has quit IRC16:01
*** yamamoto has joined #openstack-infra16:03
AJaegerclarkb, thanks for merging these. We can now merge https://review.openstack.org/#/c/597865/9 https://review.openstack.org/#/c/598671/ https://review.openstack.org/#/c/599663/ https://review.openstack.org/#/c/607560/16:04
AJaegerconfig-core, please review ^16:04
clarkbdhellmann: AJaeger I think I have managed to merge all of the CI updates. Please double check when you have a moment16:04
AJaegerclarkb: will do in a bit...16:05
* dhellmann runs his query tool16:05
dhellmannif AJaeger removes his WIP from https://review.openstack.org/#/c/598671 that can be approved16:06
dhellmannoh, and https://review.openstack.org/#/c/59966316:06
dhellmannhttps://review.openstack.org/#/c/597865 is already ready for voting16:06
dhellmannso yeah, I think we're good, clarkb16:06
dhellmannthank you!16:06
*** armax has joined #openstack-infra16:08
*** jamesmcarthur has quit IRC16:08
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add tox functional testing for drivers  https://review.openstack.org/60951516:09
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Do only one status request at a time  https://review.openstack.org/60975716:10
clarkbfungi: should we consider single core approving those?16:10
*** rpioso has joined #openstack-infra16:11
clarkbI'm going to check on ovh gra1 right now (probably boot a test instance there) and if it looks good restore the nodepool max servers value for that cloud16:11
AJaegercould I get a second core reviewer for https://review.openstack.org/#/c/597865/ , please? I had to do some non-trivial rebasing on it and thus don't want to +2...16:13
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Do only one status request at a time  https://review.openstack.org/60975716:14
*** lpetrut has quit IRC16:18
openstackgerritMerged openstack-infra/project-config master: remove job settings for Telemetry repositories  https://review.openstack.org/59867116:18
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Use yaml anchor for common gerritbot events  https://review.openstack.org/60975816:20
*** panda has quit IRC16:20
openstackgerritMerged openstack-infra/project-config master: remove job settings for openstack/monasca-analytics repo  https://review.openstack.org/60756016:21
*** panda has joined #openstack-infra16:21
*** ijw has quit IRC16:22
*** agopi is now known as agopi|brb16:22
*** ijw has joined #openstack-infra16:23
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Quick-start: remove ansible Dockerfile  https://review.openstack.org/60955516:26
clarkbI am very patiently waiting for gra1 to finish building my xenial test instance16:27
openstackgerritMerged openstack-infra/project-config master: remove job settings for barbican repositories  https://review.openstack.org/59966316:28
*** jamesmcarthur has joined #openstack-infra16:29
*** SergeyLukjanov has quit IRC16:29
clarkbinfra-root config-core anyone want to be second reviewer on https://review.openstack.org/#/c/603178/ to reenable ovh gra1? I was able to successfully boot an instance there that I will now delete16:32
*** SergeyLukjanov has joined #openstack-infra16:33
*** trown is now known as trown|lunch16:34
*** gouthamr has quit IRC16:34
*** stevebaker has quit IRC16:34
*** dmellado has quit IRC16:34
AJaegerfungi: could you review https://review.openstack.org/#/c/597865 as well, please? That finishes the job move to projects for official repos...16:36
fungithat file is getting smaller and smaller16:36
AJaegerfungi: yeah!16:37
*** armax has quit IRC16:37
AJaegerit was 30k lines long, is now less than 10k - need to update and I guess less than 8k ;)16:37
AJaeger(after that change merges)16:37
AJaegerno - less than 9k. It's now 9645 lines - and neutron removes 750.16:38
*** jpich has quit IRC16:38
AJaegerconfig-core, a tiny (3 liner) change to rename/remoe some jobs is https://review.openstack.org/609522 - reviews also welcome.16:39
openstackgerritCorey Bryant proposed openstack-infra/zuul-jobs master: Add ensure-python role  https://review.openstack.org/60976116:41
openstackgerritMerged openstack-infra/project-config master: Revert "OVH GRA1 Maintenance" - 2018-10-11 0000UTC  https://review.openstack.org/60317816:41
*** gouthamr has joined #openstack-infra16:43
*** mriedem is now known as mriedem_afk16:43
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Build new container images for quick-start  https://review.openstack.org/60976316:45
*** Swami has quit IRC16:45
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Use zuul/web/static in setup_hook  https://review.openstack.org/60976416:48
openstackgerritChandan Kumar proposed openstack-infra/zuul master: Added steps to install docker for all distros  https://review.openstack.org/60976516:49
openstackgerritMerged openstack-infra/project-config master: remove job settings for neutron repositories  https://review.openstack.org/59786516:50
*** dpawlik has joined #openstack-infra16:52
openstackgerritChandan Kumar proposed openstack-infra/zuul master: Added steps to install docker for all distros  https://review.openstack.org/60976516:52
*** dpawlik has quit IRC16:53
*** dpawlik has joined #openstack-infra16:53
pabelangerchkumar|off: clarkb: left comment on ^ for zuul16:54
openstackgerritCorey Bryant proposed openstack-infra/zuul-jobs master: Add ensure-python role  https://review.openstack.org/60976116:56
openstackgerritMerged openstack-infra/project-config master: Replace openSUSE tempest check with newer version  https://review.openstack.org/60952216:56
pabelangerfungi: +3 on sasl-auth topic16:57
*** agopi|brb is now known as agopi16:57
fungithanks pabelanger16:57
chkumar|offpabelanger: thanks, I will uodate the same16:58
openstackgerritMerged openstack-infra/zuul master: web: use window.location in TableFilters to keep sub-url path  https://review.openstack.org/60957816:59
*** derekh has quit IRC16:59
openstackgerritCorey Bryant proposed openstack-infra/openstack-zuul-jobs master: Ensure python3.7 installed for openstack-tox-py37  https://review.openstack.org/60976617:02
coreycbfungi: i have 2 reviews up now for ensure-python17:02
*** dklyle has quit IRC17:04
*** ykarel_ has joined #openstack-infra17:06
*** ykarel has quit IRC17:07
*** udesale has quit IRC17:11
clarkbwe are using gra1 again. There are errors that appear to be | fault                       | {'message': 'No valid host was found. There are not enough hosts available.', 'code': 500, 'created': '2018-10-11T17:10:12Z'}17:11
clarkbdifferent than the previous errors but possibly interesting to amorin and dpawlik17:12
*** electrofelix has quit IRC17:13
clarkbcoreycb: using codesearch.openstack.org I see some examples of preexisting python_version for tox jobs like those in http://git.openstack.org/cgit/openstack/rally/tree/.zuul.d/python-jobs.yaml makes me wonder if something is already doing this for us?17:14
clarkbcoreycb: but I'm not seeing ainythign in the ansible so it might just be rally specific17:15
*** olivierb has quit IRC17:15
*** AJaeger has quit IRC17:15
*** Swami has joined #openstack-infra17:15
fungiyeah, the only instances i know of so far where people "solved" this particular problem was some telemetry repos where they added a py37 bnidep profile to install python3.7 on bionic and then used the current tox jobs17:16
fungiand obviously expecting every project to add bindep entries just to be able to run py37 jobs is not the right approach17:16
clarkbya, I +2'17:16
clarkber17:16
clarkbI +2'd coreycb's changes, but wanted to point out there may be existing tooling but it wasn't clear to me if that is the case17:17
openstackgerritMerged openstack-infra/puppet-statusbot master: Use port 6697 for IRC over SSL/TLS  https://review.openstack.org/59906117:17
*** ijw has quit IRC17:18
*** ijw_ has joined #openstack-infra17:18
*** rh-jelabarre has joined #openstack-infra17:20
openstackgerritMerged openstack-infra/puppet-gerritbot master: Use port 6697 for IRC over SSL/TLS  https://review.openstack.org/60831317:20
*** AJaeger has joined #openstack-infra17:21
openstackgerritChandan Kumar proposed openstack-infra/zuul master: Added steps to install dependencies using bindep  https://review.openstack.org/60976517:22
chkumar|offpabelanger: ^^17:23
coreycbclarkb: fungi: thanks. I was curious how this was solved for py36 on xenial as py36 isn't installed by default on xenial.17:29
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Build new container images for quick-start  https://review.openstack.org/60976317:30
clarkbcoreycb: likely via bindep17:31
clarkbas python3 meta package17:31
coreycbclarkb: actually i'm confusing myself, 3.6 isn't available on xenial17:32
fungiyeah, we run py36 jobs on bionic and py35 jobs on xenial17:33
fungiwe ran py34 jobs on trusty by specially baking the package into our images, as it wasn't installed by default, but adding more and more things to images also turns out not to scale especially well17:33
openstackgerritChandan Kumar proposed openstack-infra/zuul master: Added steps to install docker for all distros  https://review.openstack.org/60976517:33
*** jamesmcarthur has quit IRC17:34
*** trown|lunch is now known as trown17:34
openstackgerritTobias Henkel proposed openstack-infra/zuul-jobs master: WIP: Add roles to work with a buildset docker registry  https://review.openstack.org/60977317:35
*** dklyle has joined #openstack-infra17:35
*** dpawlik has quit IRC17:39
*** jamesmcarthur has joined #openstack-infra17:39
AJaegerfungi, clarkb, should we update https://review.openstack.org/#/c/609766 for python 3.6 in the same way?17:39
*** ianychoi has quit IRC17:41
clarkbmaybe? I think if it is the 3.6 job then ya but if it is just the python3 that is current on $platform maybe less important?17:41
clarkbI like being speciofic though and failing if it won't work17:41
fungii'm fine with updating the jobs for other versions once the solution for doing this with 3.7 is confirmed and approved17:42
AJaegeryeah, let's test and iterate - I'll push a followup...17:42
fungibut from a standpoint of necessity, this is mostly to solve the case of "the declared image doesn't provide the interpreter version you need"17:43
AJaegerwe don't have python3.6 in our images - or do we?17:43
fungiit's installed by default on ubuntu-bionic17:44
fungipython3.7 is not installed by default and is optional but can be coinstalled alongside the preinstalled 3.617:44
AJaegerah, so no followup needed?17:45
funginot strictly needed, no17:45
AJaegerbtw. I don't think https://review.openstack.org/#/c/609761/2/roles/ensure-python/tasks/main.yaml will work - we do not set python_version as default17:45
AJaegercoreycb: -1 ^17:47
fungiyeah, i think instead we want that to be conditional and only fire if python_version is set17:47
fungiso basically short-circuit/no-op if that var isn't set17:47
AJaegeryeah17:47
clarkbAJaeger: there is a when just a couple lines above17:47
*** shardy has quit IRC17:47
clarkbit should do the right thing if I understand ansible correclty17:47
fungiyep, agreed17:47
AJaegerclarkb: Ah, missed that one .17:48
AJaegerdouble checking...17:48
fungiwhen: python_version is defined17:48
fungiso it looks like it already does what i suggested17:48
AJaegeryes, exactly - I'm fine. Waiting now for tests to finish...17:49
coreycbthanks for the reviews. first time ansibler here so appreciate a close eye on those. :)17:50
*** andreykurilin has quit IRC17:50
*** andreykurilin has joined #openstack-infra17:51
clarkbas a heads up I may try to reorganize my home office situation at some point today. Will likely be afk for an hour ish while I move furniture with computers turned off17:51
fungialso possible to have a change adding a py37 job declare a depends-on https://review.openstack.org/609766 and prove it's working as designed17:51
AJaegercoreycb: that would be nice ^17:52
chkumar|offclarkb: fungi do we have low hanging fruits to involve in openstack-infra?17:52
coreycbfungi: AJaeger: yep i'll do that17:52
AJaegerchkumar|off: reviews ;)17:52
fungicoreycb: AJaeger: maybe just update the broken one mentioned in the ml thread?17:52
* chkumar|off wants to learn first then jump on reviews17:52
AJaegerreviews are a great way to learn ;)17:53
coreycbfungi: sure but will there need to be 2 depends-on in that case since nothings landed yet17:53
coreycb?17:53
AJaegercoreycb: yes17:53
coreycbAJaeger: ok17:53
fungicoreycb: yeah, actually 609766 needs to have a depends-on https://review.openstack.org/60976117:56
*** jamesmcarthur has quit IRC17:56
AJaegerfungi: it does not strictly need it - it's just a variable declaration, it can merge without - but will be a noop without the other one merged17:57
fungiright17:57
fungii suppose both depends-on footers could go in the change adding the job17:57
fungieither way will work17:57
*** jamesmcarthur has joined #openstack-infra17:57
openstackgerritTobias Henkel proposed openstack-infra/zuul-jobs master: WIP: Add roles to work with a buildset docker registry  https://review.openstack.org/60977317:58
fungichkumar|off: what do you enjoy working on? we have a lot of unfinished or not-started things that could use some extra hands17:58
openstackgerritCorey Bryant proposed openstack-infra/openstack-zuul-jobs master: Ensure python3.7 installed for openstack-tox-py37  https://review.openstack.org/60976617:59
fungichkumar|off: configuration management? ci jobs? javascript web interfaces? python api servers?17:59
coreycbfungi: AJaeger: in addition to that ^ I updated:18:00
coreycbhttps://review.openstack.org/#/c/609558/  and https://review.openstack.org/#/c/609557/18:00
fungioof, we're up to nearly 3 hours to get a node assignment in check right now18:01
fungicoreycb: thanks!18:01
coreycbfungi: np thanks for the guidance!18:01
*** jamesmcarthur has quit IRC18:02
fungialso be aware the old change id based depends-on footers are deprecated in favor of url-based18:02
fungisince zuul can now support multiple connections to additional gerrits, github, et cetera18:03
fungi(soon probably gitlabs, maybe also bitbucket)18:03
mordredfungi: so many gitlabs18:04
*** ansmith has quit IRC18:04
coreycbfungi: good to know. i've seen both.18:05
*** dpawlik has joined #openstack-infra18:07
clarkbre low hanging fruit that is a good question. I've been so focused on make clodus work and the config mgmt updates and various conferences the last few weeks I'm not entirely sure. One that comes to mind is leaked port cleanup support in nodepool18:08
clarkbchkumar|off: ^ that is a python programming task in zuul/nodepool that will have a big impact on infra's ability to consume clouds18:09
*** mriedem_afk is now known as mriedem18:09
*** dpawlik has quit IRC18:09
*** dpawlik has joined #openstack-infra18:09
*** dpawlik has quit IRC18:10
*** dpawlik has joined #openstack-infra18:11
clarkbon the not so low hanging fruit list we've got a gerrit upgrade we should start figuring out as well as move to containers and config management updates. We also need to update servers that are running trusty to xenial (or bionic) for which we'll have an informal sprint next week to focus on that18:11
*** dpawlik has quit IRC18:13
*** dpawlik has joined #openstack-infra18:14
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Small cleanups for zuul.d/projects.yaml  https://review.openstack.org/60977818:17
*** dpawlik has quit IRC18:19
*** ansmith has joined #openstack-infra18:20
*** openstackgerrit has quit IRC18:20
clarkbfungi: ^ in theory openstackgerrit comes back shortly?18:21
clarkboh wait openstackgerrit does least recently used channel connectiosn right?18:21
clarkbquick merge code :P18:21
*** gouthamr_ has joined #openstack-infra18:25
*** openstackgerrit has joined #openstack-infra18:27
openstackgerritMerged openstack-infra/zuul master: web: use PUBLIC_URL instead of package.json patch  https://review.openstack.org/60967218:27
clarkbit works! :)18:27
clarkbfungi: ^ I expect that is happening over sasl identified connection now18:27
*** dmellado has joined #openstack-infra18:27
*** openstackstatus has quit IRC18:28
AJaegeryeah!18:28
*** jamesmcarthur has joined #openstack-infra18:29
clarkbAJaeger: should we #success the python job config moves?18:29
*** openstackstatus has joined #openstack-infra18:29
*** ChanServ sets mode: +v openstackstatus18:29
AJaegerclarkb: yeah, let's do it. Will you?18:33
openstackgerritCorey Bryant proposed openstack-infra/zuul-jobs master: Add ensure-python role  https://review.openstack.org/60976118:33
clarkbAJaeger: how about #success Job configuration for OpenStack projects moved into individual repositories enabling python3 first goal work.18:35
*** panda has quit IRC18:35
AJaegerLGTM18:36
clarkb#success Job configuration for OpenStack projects moved into individual repositories enabling python3 first goal work.18:36
openstackstatusclarkb: Added success to Success page (https://wiki.openstack.org/wiki/Successes)18:36
AJaegerare you aware of packethost trouble? We only use 30 out of 100 nodes according to http://grafana.openstack.org/d/U462abNik/nodepool-packethost?orgId=118:37
AJaegerTime to sign off for today - I wish all a great day...18:37
clarkbI want to say fungi and shrews looked on tuesdy and it was floating ip leaks?18:37
clarkbI'll take a look to see if we can clean any of that up18:37
lifelessfungi: cool. fingers cross18:38
*** panda has joined #openstack-infra18:38
fungiwell, i didn't look, but Shrews reported that it was complaining about being unable to allocate a fip due to being at quota18:38
clarkb| fault                       | {'message': "Build of instance 6c6d45bd-5ed6-4b63-b300-bcd73b0996b7 aborted: Failed to read or write disk info file: [Errno 24] Too many open files: '/opt/pf9/data/instances/6c6d45bd-5ed6-4b63-b300-bcd73b0996b7/disk.info'", 'code': 500, 'created': '2018-10-11T18:34:35Z'} is what server show on an ERROR'd server says18:39
clarkbfloating IPs don't seem to be leaked18:40
clarkbI'm guessing its more of the file dscriptor problem than fips18:40
fungiagreed18:40
fungistudarus seems to not be around at the moment18:40
*** dpawlik has joined #openstack-infra18:40
*** dpawlik has quit IRC18:41
*** dpawlik has joined #openstack-infra18:41
*** stevebaker has joined #openstack-infra18:42
*** gouthamr has quit IRC18:45
*** AJaeger_ has joined #openstack-infra18:46
*** AJaeger has quit IRC18:48
*** AJaeger_ is now known as AJaeger18:50
chkumar|offfungi: clarkb I can help on python programming / Ansible config management side18:50
clarkbchkumar|off: having nodepool clean up leaked ports might be a good task (python programming) then18:51
clarkbchkumar|off: if you look in nodepool there is preexisting code to do similar with floating IPs, we would need to add code that handles ports18:51
clarkbwe should double check with ianw when he wakes that work hasn't started on that yet18:51
chkumar|offclarkb: sure, I will take a look18:52
clarkbbut otherwise that is probably a good task since there is some preexisting code that does similar and it will help us make use of some of our clouds18:52
fungianother place python programmers would be welcome to help is on storyboard18:53
clarkb++18:54
fungithere's even a new wip spec posted for implementing a story attachments feature18:54
*** dave-mccowan has quit IRC18:54
fungithough some of that dips into javascript webapp land too18:54
*** ykarel_ has quit IRC18:54
*** ykarel_ has joined #openstack-infra18:55
*** dave-mccowan has joined #openstack-infra18:58
*** smarcet has quit IRC19:01
*** rh-jelabarre has quit IRC19:02
*** jamesmcarthur has quit IRC19:03
*** dave-mccowan has quit IRC19:03
*** ansmith has quit IRC19:05
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove build-openstack-sphinx-docs-python3  https://review.openstack.org/60979319:20
openstackgerritMerged openstack-infra/zuul-sphinx master: Use OrderedDict for object tracking  https://review.openstack.org/60524019:22
openstackgerritMerged openstack-infra/zuul-sphinx master: Add example and type options to attributes  https://review.openstack.org/60426719:22
*** dpawlik has quit IRC19:26
openstackgerritMerged openstack-infra/zuul-jobs master: Add zuul to the linters tox env  https://review.openstack.org/60702619:27
openstackgerritMerged openstack-infra/zuul master: web: fix multiple builds query parameters for a single column  https://review.openstack.org/59360519:28
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove obsolete legacy-tempest-dsvm-py35-full-devstack-plugin-ceph  https://review.openstack.org/60979519:36
*** ram5391 has joined #openstack-infra19:39
*** eernst has joined #openstack-infra19:43
*** eharney has quit IRC19:43
dhellmannconfig-core, could we get another reviewer to look at https://review.openstack.org/#/c/609472/ please? we would like to release that library for folks working on the upgrade check goal and we're blocked by not having a release job configured19:45
clarkbdhellmann: I'll take a look19:46
dhellmannclarkb : thanks!19:46
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Add a quick-start test job  https://review.openstack.org/60951419:53
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Fixes for quick-start playbooks  https://review.openstack.org/60955119:53
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Remove disable-recommends in quick-start test  https://review.openstack.org/60955319:53
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Quick-start: remove ansible Dockerfile  https://review.openstack.org/60955519:53
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Build new container images for quick-start  https://review.openstack.org/60976319:53
openstackgerritJeremy Stanley proposed openstack-infra/zuul-base-jobs master: Correct zuul-jobs path  https://review.openstack.org/59960719:54
openstackgerritMerged openstack-infra/project-config master: Add publish-to-pypi job for oslo.upgradecheck  https://review.openstack.org/60947219:54
*** manjeets has quit IRC20:04
clarkbpabelanger: sure, I think the restarts will largely be a noop since the changes are in testing and zuul config for nodepool20:05
clarkbshould be fine to do at any time20:05
pabelangerclarkb: okay, let me start with nodepool-builders20:05
clarkbthe buildres likely haven't been restarted in a while so starting with those is ++20:06
pabelangerokay, no DIBs building, starting with nb0120:06
clarkb(don't forget 03 too)20:06
pabelanger++20:07
clarkbI've approved the 3pci direction spec20:07
*** ansmith has joined #openstack-infra20:09
logan-http://grafana.openstack.org/d/BhcSH5Iiz/nodepool-ovh?panelId=20&fullscreen&orgId=1 - port leaks for the newly upgraded region maybe?20:10
clarkblogan-: I checked and we are getting placement errors of no valid host. I think maybe the host aggregates moved or got updated?20:10
pabelangerclarkb: nodepool-builders done, moving to nodepool-launcher20:11
logan-ah20:11
*** smarcet has joined #openstack-infra20:14
pabelangerclarkb: and nodepool-lauchers restarted20:15
openstackgerritMerged openstack-infra/zuul master: Do only one status request at a time  https://review.openstack.org/60975720:15
clarkbpabelanger: ok thanks I'll try to pay attention to it too20:16
clarkblogan-: unfortunately timezones are such that when I wake up they are off to dinner I think20:16
clarkbmaybe ajaeger frickler or fungi can catch amorin/dpawlik tomorrow20:16
*** tosky__ has joined #openstack-infra20:16
clarkbok I'm setting up sasl with freenode which should all work fine but I may drop for a short time if I get it wrong20:17
*** tosky has quit IRC20:17
pabelanger#status log all nodepool-builders / nodepool-launchers restarted to pick up latest code base (32b8f58)20:17
openstackstatuspabelanger: finished logging20:17
*** clarkb has quit IRC20:18
*** clarkb has joined #openstack-infra20:18
clarkbseems to have worked20:19
clarkbalso notice that freenode set a global +r?20:19
fungiyeah, they did20:21
clarkbI guess we can feel less bad about setting it on our channels then20:21
fungiyou now have to opt into privmsg from unauthenticated nicks20:21
clarkbwow20:21
openstackgerritMerged openstack-infra/infra-specs master: Direction setting for 3rd Party CI  https://review.openstack.org/56384920:22
clarkbianw: ^ fyi20:22
*** tosky__ is now known as tosky20:23
ianwcool20:23
clarkbI'm in the odd position of having worked through my todo list before the end of the day. I guess I go do code reviews now. Any changes people want me to look at ?20:23
ianwhttp://grafana.openstack.org/d/BhcSH5Iiz/nodepool-ovh doesn't look happy?20:23
clarkbianw: ya logan- poitned it out above too. When I looked after reenabling gra1 there are errors indicating there are no valid hosts to deploy to20:24
clarkb| fault                       | {'message': 'No valid host was found. There are not enough hosts available.', 'code': 500, 'created': '2018-10-11T17:10:12Z'}20:24
clarkbI'm going to guess that the upgrade is either still in progress on some hypervisors or the host aggregates got missed/messed up20:25
clarkbianw: maybe you can see if amorin or dpawlik are around later today/this evening for you? otherwise I'll likely send email in the morning20:25
*** kgiusti has left #openstack-infra20:27
ianwok, is it currently disabled?20:27
*** ijw_ has quit IRC20:28
clarkbianw: no it was successfully booting some nodes so I left it on20:28
clarkbbut looks like that has changed since20:28
clarkbactually I wonder if it is hitting the same port issue as bhs1 now too20:28
clarkbcould be multiple things :)20:28
ianwhrm, that could be right.  i still have my script running in the background deleting hte bhs1 ports20:28
clarkbI'm checking gra1 for port leaks20:29
ianwlet me run it against gra1 ...20:29
clarkb616 DOWN ports in gra120:29
clarkbyup I think its the same problem there now too20:29
clarkbexciting20:29
ianwok; let me update my script and that should clear that out ...20:30
clarkbyup then we'll only have to deal with the no valid host issue which hopefully amorin and or dpawlik can help with. And maybe explain we are periodically deleting leaked ports too :/20:30
clarkbianw: I pointed chkumar|off to the task of having nodepool do those port cleanups internally like it does for floating IPs20:32
clarkbso we might have a better answer for it in ndoepool in the near future too20:32
fungiclarkb: if you're looking for quick reviews, topic:its-storyboard will get task footers hyperlinked in gerrit20:32
ianwRemove port 0042adea-af67-40bd-89df-f568e88b2394 [1/616] ... going to take a while20:33
fungiclarkb: also i tried to refresh our zuul operational docs in system-config with 60555620:33
clarkbianw: we might need to temporarily stop nodepool booting stuff there to avoid having it work against us there?20:33
ianwthe script take's fungi's approach of comparing after a few minutes to see what's the same, so these should all be old ports20:35
clarkbianw: ya, but nodepool will leak new old ports while it fails I think20:35
clarkbwe can worry about that if the script doesn't catch up20:35
clarkbfungi: I approved the review-dev change and +2'd the review.o.o change. Figure you can approve the review.o.o change when we are able to restart gerrit quickly? Maybe tomorrow sometime?20:36
ianwi was going to leave it running in the same screen session as bhs1 on bridge.o.o20:36
clarkbianw: sounds good to me20:36
openstackgerritMerged openstack-infra/gerritbot master: Identify with SASL  https://review.openstack.org/60831420:39
ianwinteresting, it's now taking about 20/30 seconds to remove a port20:40
*** pcaruana has quit IRC20:40
clarkbwe saw similar with bhs1 initially20:40
fungiclarkb: sure, sounds great. thanks!20:40
clarkbI wonder if there is a performance degredation as part of the upgrade process20:40
openstackgerritMerged openstack-infra/statusbot master: Identify with SASL  https://review.openstack.org/59904920:40
clarkblike maybe they are udpating database tables in the background or something20:40
*** ansmith has quit IRC20:41
*** armax has joined #openstack-infra20:44
*** ijw has joined #openstack-infra20:54
ianwyeah, it's sped up, so something like that seems likely20:55
ianwanyway, the screen session now has two windows, one for BHS1 and one for GRA120:55
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add a helper for manual port cleanup  https://review.openstack.org/60913320:56
clarkbianw: re ^ I almost think we should merge it because this isn't the first time we've done similar and it gives us a concrete thing to point at for the neutron and ovh etc folks to understand what the issue is20:59
openstackgerritMerged openstack-infra/system-config master: Hyperlink task footers on review-dev  https://review.openstack.org/60769821:01
clarkbin fact I may as well go bug #openstack-neutron about this now21:01
Shrewsclarkb: ianw: if deleting ports takes that long, we'll have to start a background thread in nodepool to do that work21:03
clarkbShrews: it was only that long after the initial upgrade iirc21:03
clarkbShrews: its normally quite quick21:03
Shrewsclarkb: last couple of times i caught this conversation, folks were lamenting how long it takes21:04
clarkbShrews: yes immediately after hte upgrade (ovh just finished the upgrade in gra1 region but bhs1 hasn't been slow since that initial set of deletes that fungi did)21:05
*** bobh has quit IRC21:05
Shrewsok good21:06
*** ykarel__ has joined #openstack-infra21:06
*** bobh has joined #openstack-infra21:07
openstackgerritIan Wienand proposed openstack-infra/system-config master: Add a helper for manual port cleanup  https://review.openstack.org/60913321:07
ianwShrews: yeah, it's probably *everything* taking so long on the post upgrade clouds :)21:07
ianwseems to go about as well as most of the "upgrades" i try around my house, which usually ends with whining family saying "why did you touch it!" :)21:08
*** ykarel_ has quit IRC21:09
ianwno omelettes without breaking some eggs and all that21:09
clarkbianw: Shrews slaweq over in #openstack-neutron says they will take a look tomorrow with ovh21:09
*** bobh has quit IRC21:11
Shrewsclarkb: cool. i think i can have a nodepool implementation of ianw's script tomorrow, unless someone else is already hacking that out21:12
clarkbShrews: chkumar|off mentioned an interest in it21:12
Shrewsok21:12
ianwi haven't looked, happy to review :)21:13
*** eernst has quit IRC21:20
*** priteau has quit IRC21:22
Shrewsianw: does admin_state_down field identify the port being down?21:22
Shrewserr, admin_state_up rather21:22
clarkbShrews: osc shows it at Status field and value is DOWN21:24
clarkbShrews: admin_state_up appears to say UP even when ports are status DOWN21:24
Shrewsweird21:25
clarkbI think that may be neutron says this port should be up and working but its down beacuse not attached21:25
clarkbstatus i the running status, admin_state_up is the power button21:25
Shrewsah! yes, missed that in the api doc21:25
*** eernst has joined #openstack-infra21:26
openstackgerritMerged openstack-infra/zuul master: Added steps to install docker for all distros  https://review.openstack.org/60976521:26
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Remove x-vrif-minus-2 from gerritbot notifications  https://review.openstack.org/60972821:27
*** ykarel__ has quit IRC21:29
*** dave-mccowan has joined #openstack-infra21:30
*** priteau has joined #openstack-infra21:32
*** eernst has quit IRC21:34
*** panda has quit IRC21:34
*** priteau has quit IRC21:37
*** panda has joined #openstack-infra21:37
*** bobh has joined #openstack-infra21:40
openstackgerritDoug Hellmann proposed openstack-infra/project-config master: update the branching script to include a semver bump  https://review.openstack.org/60982721:40
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Quick-start: switch to polygerrit  https://review.openstack.org/60982821:40
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Skeleton code to cleanup down ports  https://review.openstack.org/60982921:40
Shrewsclarkb: if chkumar|off wants to pick up 609829 and run with it, i'm fine with that. just had the code in my head (and it is rather simple) and wanted to get it out somewhere21:41
Shrewsotherwise i'll finish it up tomorrow21:41
*** smarcet has quit IRC21:41
Shrews(and seems like we need it rather soon)21:41
clarkbShrews: ya depending on how quickly ovh is able to address it (if at all) having it soon is likely good21:43
*** armax has quit IRC21:43
*** armax has joined #openstack-infra21:44
*** bobh has quit IRC21:44
Shrewsand i lost power...  beer time21:45
*** bobh has joined #openstack-infra21:49
*** gouthamr_ is now known as gouthamr21:53
*** ijw has quit IRC21:57
*** ram5391 has quit IRC22:03
*** armax has quit IRC22:05
*** boden has quit IRC22:09
*** bobh has quit IRC22:12
*** pbourke has quit IRC22:19
*** openstackgerrit has quit IRC22:19
*** openstackgerrit has joined #openstack-infra22:20
*** mriedem is now known as mriedem_gone22:23
*** rcernin has joined #openstack-infra22:23
*** mriedem_gone has quit IRC22:24
*** ijw has joined #openstack-infra22:24
*** rpioso is now known as rpioso|afk22:26
*** openstackstatus has quit IRC22:28
*** kjackal has quit IRC22:28
*** openstackstatus has joined #openstack-infra22:29
*** ChanServ sets mode: +v openstackstatus22:29
*** pbourke has joined #openstack-infra22:31
openstackgerritClark Boylan proposed openstack-infra/zuul-jobs master: Install docker from distro properly  https://review.openstack.org/60984422:39
fungiShrews: better drink it before it all goes warm!22:42
*** smarcet has joined #openstack-infra22:44
openstackgerritClark Boylan proposed openstack-infra/zuul master: Build new container images for quick-start  https://review.openstack.org/60976322:45
openstackgerritClark Boylan proposed openstack-infra/zuul master: Quick-start: switch to polygerrit  https://review.openstack.org/60982822:45
*** smarcet has quit IRC22:46
*** xarses has quit IRC22:51
*** xarses has joined #openstack-infra22:51
*** eernst has joined #openstack-infra22:57
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Minor documentation updates  https://review.openstack.org/60963623:13
*** owalsh has quit IRC23:15
*** tosky has quit IRC23:15
*** mudpuppy has quit IRC23:16
*** spotz has quit IRC23:17
ianwgra1 seems to be taking nodes; the script has cleared out about 565 ports23:18
*** mattoliverau has quit IRC23:18
*** sthussey has quit IRC23:21
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fix syntax erro with zuul-tarball job  https://review.openstack.org/60985523:26
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fix syntax error with zuul-tarball job  https://review.openstack.org/60985523:27
*** owalsh has joined #openstack-infra23:30
*** rlandy is now known as rlandy|bbl23:46
*** dpawlik has joined #openstack-infra23:51
*** spotz has joined #openstack-infra23:55
*** mudpuppy has joined #openstack-infra23:55
*** dpawlik has quit IRC23:55
*** lbragstad has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!