Friday, 2017-10-27

*** andreas_s has joined #openstack-infra00:00
*** lindycoder has quit IRC00:00
ianwproject-core: i wouldn't mind moving on https://review.openstack.org/514117 and https://review.openstack.org/513233 .  it's moving zuul.projects to a dict ... we're halfway there with zuul exporting zuul._projects, but i'd like to get rid of that asap00:00
*** hongbin has quit IRC00:01
ianwfungi: did you notice the puppet-agent errors when you were launching?  https://review.openstack.org/#/c/515279/ worked for me, but also we determined i was on an older checkout00:03
ianwi can abandon that if it's fixed elsewhere00:03
*** hughhalf has joined #openstack-infra00:07
dhineshstill hitting https://pastebin.com/dVXAH3ia on nodepool 0.4.1 , 0.4.0 works fine without issues any suggestions?00:09
clarkbdhinesh: yes as explained earlier 0.4.1 isn't a "real" version because we have not tagged that version. This means you can't specify it as the version for pip to install00:10
clarkbdhinesh: it is a logical version for all commits after 0.4.0 because the next version will have to be at least 0.4.100:10
*** catintheroof has joined #openstack-infra00:11
*** burgerk has joined #openstack-infra00:13
*** rcernin has joined #openstack-infra00:21
jeblairianw: +2 from me on both of those.  i'd also like to wrap that up.  :)00:21
*** bobh has joined #openstack-infra00:22
dhineshclarkb: ok, looks like 0.4.1 start the service as nodepool user, i was able to fix it after i installed it through nodepool user00:22
*** eroux has quit IRC00:25
*** andreas_s has quit IRC00:26
*** andreas_s has joined #openstack-infra00:28
*** catintheroof has quit IRC00:32
*** burgerk has quit IRC00:36
*** catintheroof has joined #openstack-infra00:38
*** gyee has quit IRC00:40
*** sree has joined #openstack-infra00:40
*** catintheroof has quit IRC00:42
*** andreas_s has quit IRC00:43
*** xarses has joined #openstack-infra00:43
*** Apoorva_ has joined #openstack-infra00:43
*** catintheroof has joined #openstack-infra00:44
*** andreas_s has joined #openstack-infra00:44
*** sree has quit IRC00:45
*** Apoorva has quit IRC00:47
*** Apoorva_ has quit IRC00:47
*** andreas_s has quit IRC00:48
*** gouthamr has joined #openstack-infra00:48
dhineshafter upgrading nodepool from 3.1 to 4.1 none of the commands seem to be working, https://pastebin.com/cDEQFrGU00:50
dhineshis the hostname parameter need to be set anywhere?00:50
*** kiennt26 has joined #openstack-infra00:55
*** andreas_s has joined #openstack-infra00:55
*** catintheroof has quit IRC00:55
*** catintheroof has joined #openstack-infra00:56
*** LindaWang has joined #openstack-infra00:56
*** zzzeek has quit IRC00:58
*** xarses has quit IRC01:00
*** andreas_s has quit IRC01:00
*** zzzeek has joined #openstack-infra01:00
*** andreas_s has joined #openstack-infra01:01
*** smatzek has joined #openstack-infra01:02
fungiianw: yeah, for some reason the puppet packages on xenial now enable the puppet-agent daemon by default and it gets unhappy because we don't configure it to use a puppetmaster01:12
*** andreas_s has quit IRC01:12
ianwyep that was what i found, it was disabled, but not actually stopped01:13
fungidmsimard: i didn't get a quote for the shipping, we didn't get that far. i got a preliminary quote from the osl for what the setup and ongoing hosting costs would be, but that was contingent on us figuring out switches and mounting rails01:13
*** huanxie has joined #openstack-infra01:13
dmsimardfungi: Ok, thanks !01:14
*** liujiong has joined #openstack-infra01:15
*** mriedem has joined #openstack-infra01:15
*** mriedem_away has quit IRC01:15
*** thorst has joined #openstack-infra01:16
*** thorst has quit IRC01:17
*** andreas_s has joined #openstack-infra01:18
*** catintheroof has quit IRC01:20
*** namnh has joined #openstack-infra01:21
*** catintheroof has joined #openstack-infra01:21
*** Kevin_Zheng has joined #openstack-infra01:21
*** fanzhang has quit IRC01:24
*** fanzhang_ has joined #openstack-infra01:24
*** catintheroof has quit IRC01:24
*** gmann_afk is now known as gmann01:25
*** dave-mccowan has quit IRC01:26
*** andreas_s has quit IRC01:28
*** dave-mccowan has joined #openstack-infra01:28
*** psachin has joined #openstack-infra01:35
*** andreas_s has joined #openstack-infra01:37
*** csomerville has joined #openstack-infra01:42
*** rcernin has quit IRC01:43
*** yamamoto has joined #openstack-infra01:43
*** rcernin has joined #openstack-infra01:44
*** dhinesh has quit IRC01:44
*** fanzhang_ is now known as fanzhang01:44
*** cody-somerville has quit IRC01:46
*** andreas_s has quit IRC01:47
*** yamamoto has quit IRC01:49
*** smatzek has quit IRC01:50
*** catintheroof has joined #openstack-infra01:55
*** salv-orlando has joined #openstack-infra01:55
*** andreas_s has joined #openstack-infra01:56
*** askb has quit IRC01:56
*** salv-orlando has quit IRC02:00
*** catintheroof has quit IRC02:00
*** andreas_s has quit IRC02:02
*** hongbin has joined #openstack-infra02:05
*** stakeda has joined #openstack-infra02:09
*** askb has joined #openstack-infra02:10
*** bobh has quit IRC02:11
*** xarses has joined #openstack-infra02:13
*** rkukura has quit IRC02:13
*** gcb has joined #openstack-infra02:16
*** thorst has joined #openstack-infra02:18
*** liujiong has quit IRC02:18
*** liujiong has joined #openstack-infra02:18
*** bh526r has quit IRC02:19
*** huanxie has quit IRC02:22
*** andreas_s has joined #openstack-infra02:22
*** thorst has quit IRC02:23
*** annp has joined #openstack-infra02:23
*** andreas_s has quit IRC02:26
*** catintheroof has joined #openstack-infra02:28
*** lindycoder has joined #openstack-infra02:29
*** yamahata has quit IRC02:29
*** iyamahat has quit IRC02:30
*** andreas_s has joined #openstack-infra02:33
*** catintheroof has quit IRC02:33
*** fanzhang has left #openstack-infra02:34
*** cuongnv has joined #openstack-infra02:34
*** andreas_s has quit IRC02:44
*** yamamoto has joined #openstack-infra02:45
*** lindycoder has quit IRC02:49
*** spzala has quit IRC02:50
*** yamamoto has quit IRC02:51
Jeffrey4lwhen one project depends on another project's jobs, the second project is not added into the first project's dependency. is this correct? and how can i fix this?02:53
Jeffrey4llike this patch https://review.openstack.org/#/c/513971/02:53
*** andreas_s has joined #openstack-infra02:57
*** armaan has joined #openstack-infra02:59
kazshHi infra core team, could you someone review my change on project-config ?   https://review.openstack.org/#/c/514895/  this is show-stopper for our project...03:01
*** spzala has joined #openstack-infra03:02
openstackgerritMerged openstack-infra/project-config master: Remove Tacker legacy job  https://review.openstack.org/51534103:03
openstackgerritMerged openstack-infra/project-config master: Update neutron-failure-rate grafana dashboard  https://review.openstack.org/51543903:03
openstackgerritMerged openstack-infra/project-config master: Add nodejs4-publish jobs  https://review.openstack.org/51533203:03
openstackgerritMerged openstack-infra/project-config master: Switch kuryr back to publish-to-pypi  https://review.openstack.org/51551103:05
openstackgerritMerged openstack-infra/project-config master: Switch octavia back to publish-to-pypi  https://review.openstack.org/51551803:05
clarkbJeffrey4l: I'm not sure I understand the question. But my understanding is that both repos are pushed into the job when there is a depends on03:06
*** spzala has quit IRC03:06
*** andreas_s has quit IRC03:07
clarkbas for using other projects jobs and what repos are available the devstack job may be a good example for that? I think shade is using jobs defined in devstack now03:07
*** andreas_s has joined #openstack-infra03:12
openstackgerritMerged openstack-infra/project-config master: Add docs job for VPNaaS and VPNaaS dashboard  https://review.openstack.org/51490603:12
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364503:13
*** dave-mccowan has quit IRC03:14
Jeffrey4lcheck the newer patch in https://review.openstack.org/513971 , i think i found the solution.03:14
Jeffrey4lclarkb,03:14
openstackgerritMerged openstack-infra/project-config master: Updated templates for heat-dashboard  https://review.openstack.org/51489503:15
*** bhavik1 has joined #openstack-infra03:15
*** yamamoto has joined #openstack-infra03:16
*** ihrachys has quit IRC03:17
johnsomLow priority, but stackalytics is throwing 502s (nginx)03:17
*** andreas_s has quit IRC03:18
*** andreas_s has joined #openstack-infra03:19
*** huanxie has joined #openstack-infra03:19
clarkbjohnsom: which stackalytics?03:23
clarkbjohnsom: we don't run the production one03:23
johnsomhttp://stackalytics.com03:23
clarkb(there is a stackalytics.openstack.org though that we've fiddled with running)03:23
johnsomAh, didn't know03:23
johnsomJust happened to notice so thought I would mention it.03:23
*** andreas_s has quit IRC03:24
*** kiennt26 has quit IRC03:25
*** kiennt26 has joined #openstack-infra03:26
*** bhavik1 has quit IRC03:27
clarkbya unfortunately I don't think there is much we can do03:29
*** nicolasbock has quit IRC03:30
openstackgerritTony Breeds proposed openstack-infra/system-config master: Remove unneeded encoding change.  https://review.openstack.org/51558003:33
*** andreas_s has joined #openstack-infra03:34
*** hongbin has quit IRC03:35
*** yamamoto has quit IRC03:36
*** andreas_s has quit IRC03:38
*** kiennt26 has quit IRC03:39
*** yamahata has joined #openstack-infra03:40
*** kiennt26 has joined #openstack-infra03:40
*** markvoelker has quit IRC03:43
*** andreas_s has joined #openstack-infra03:47
*** andreas_s has quit IRC03:52
*** gongysh has joined #openstack-infra03:53
*** yamamoto has joined #openstack-infra03:54
*** yamamoto has quit IRC03:54
*** yamamoto has joined #openstack-infra03:55
*** kiennt26 has quit IRC03:56
*** salv-orlando has joined #openstack-infra03:56
*** udesale has joined #openstack-infra03:58
*** salv-orlando has quit IRC04:00
*** huanxie has quit IRC04:01
*** catintheroof has joined #openstack-infra04:01
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove non-voting from legacy-puppet-beaker-rspec  https://review.openstack.org/51558204:03
*** psachin has quit IRC04:03
*** spzala has joined #openstack-infra04:03
AJaegerianw: that's my proposed fix for https://review.openstack.org/514872 ^04:04
*** catintheroof has quit IRC04:05
*** spzala has quit IRC04:07
ianwAJaeger: ok, so the job def was wrong, that makes sense04:09
*** bstinson has quit IRC04:10
*** ykarel has joined #openstack-infra04:10
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Silence ansible-lint  https://review.openstack.org/51552004:11
AJaegerand here's one fix I like in as well ^04:11
*** bstinson has joined #openstack-infra04:14
*** mriedem has quit IRC04:15
*** kiennt26 has joined #openstack-infra04:15
*** Apoorva has joined #openstack-infra04:16
*** rosmaita has quit IRC04:18
*** thorst has joined #openstack-infra04:19
*** zoli has quit IRC04:19
*** dbecker has quit IRC04:20
*** annp has quit IRC04:20
*** claudiub|2 has joined #openstack-infra04:20
*** psachin has joined #openstack-infra04:20
*** pleia2 has quit IRC04:21
*** zoli has joined #openstack-infra04:21
*** pleia2 has joined #openstack-infra04:22
*** thorst has quit IRC04:24
openstackgerritMerged openstack-infra/project-config master: Remove legacy-osc-dsvm-functional* jobs  https://review.openstack.org/51507504:24
*** gildub has quit IRC04:24
*** annp has joined #openstack-infra04:28
*** janki has joined #openstack-infra04:36
*** dbecker has joined #openstack-infra04:36
*** sree has joined #openstack-infra04:37
*** logan- has quit IRC04:38
*** gildub has joined #openstack-infra04:40
*** ramishra has quit IRC04:42
*** logan- has joined #openstack-infra04:42
*** gongysh has quit IRC04:42
*** ramishra has joined #openstack-infra04:44
*** csomerville has quit IRC04:44
*** csomerville has joined #openstack-infra04:45
*** Apoorva has quit IRC04:46
*** xarses has quit IRC04:48
*** lamt has quit IRC04:51
*** _Cyclone_ has quit IRC04:51
*** pleia2 has quit IRC04:51
*** bstinson has quit IRC04:51
*** kuromagi has quit IRC04:51
*** cburgess has quit IRC04:51
*** cody-somerville has joined #openstack-infra04:53
*** cody-somerville has joined #openstack-infra04:53
*** csomerville has quit IRC04:55
*** shu-mutou has joined #openstack-infra05:02
*** spzala has joined #openstack-infra05:02
*** bhavik1 has joined #openstack-infra05:02
*** _Cyclone_ has joined #openstack-infra05:03
*** cburgess has joined #openstack-infra05:03
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: DNM: test ansible crash logging  https://review.openstack.org/51547005:04
*** lamt has joined #openstack-infra05:04
*** bstinson has joined #openstack-infra05:04
*** lamt is now known as Guest5385005:04
*** jaosorior has joined #openstack-infra05:05
*** pleia2 has joined #openstack-infra05:05
*** kuromagi has joined #openstack-infra05:05
*** spzala has quit IRC05:06
*** huanxie has joined #openstack-infra05:08
openstackgerritMerged openstack-infra/zuul-jobs master: Silence ansible-lint  https://review.openstack.org/51552005:10
chandankumarianw: please have a look on this https://review.openstack.org/#/c/513698/ thanks :-)05:10
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs  https://review.openstack.org/51245005:10
*** slaweq has joined #openstack-infra05:11
*** psachin has quit IRC05:14
*** psachin has joined #openstack-infra05:15
*** slaweq has quit IRC05:16
*** bstinson has quit IRC05:19
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: WIP: Run ansible-lint on repo  https://review.openstack.org/51559305:19
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: WIP: Run ansible-lint on repo  https://review.openstack.org/51559305:19
*** yamamoto has quit IRC05:21
*** bstinson has joined #openstack-infra05:22
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove non-voting from legacy-puppet-beaker-rspec  https://review.openstack.org/51558205:24
AJaegerianw: missed one place ^05:25
*** salv-orlando has joined #openstack-infra05:25
*** gongysh has joined #openstack-infra05:25
*** Guest87185 has quit IRC05:31
*** dhajare|mtg has joined #openstack-infra05:36
*** salv-orlando has quit IRC05:38
*** salv-orlando has joined #openstack-infra05:38
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: zuul-jobs: Run ansible-lint  https://review.openstack.org/51559605:39
*** iyamahat has joined #openstack-infra05:39
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: WIP: Run ansible-lint on repo  https://review.openstack.org/51559305:39
AJaegerpabelanger, ianw, frickler, tobiash, with these two changes above, we can run ansible-lint on zuul-jobs ^05:39
AJaegerMade it non-voting for now - like it's on openstack-zuul-jobs05:39
tobiashyay05:40
tobiashAJaeger: the first has a syntax error reported from zuul05:40
AJaegertobiash: coming...05:40
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: zuul-jobs: Run ansible-lint  https://review.openstack.org/51559605:40
AJaegeronce that is in, we can tackle project-config... My previous test run succeeded on zuul-jobs, so now doing it properly05:41
tobiashAJaeger: shall this replace tox-linters?05:41
AJaegertobiash: yes05:41
AJaegertobiash: uses same tox environment05:42
AJaegertobiash: pabelanger plans to make an ansible job, so we might reintroduce tox-linters later ;)05:42
AJaegeransible job instead of calling tox05:42
tobiashah, ok05:42
*** yolanda has joined #openstack-infra05:42
tobiashthen you have my +105:43
AJaegerthanks05:43
AJaegerdo you want to review zuul-jobs as well? I'll remove WIP now ;(05:43
*** markvoelker has joined #openstack-infra05:43
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Run ansible-lint on repo  https://review.openstack.org/51559305:44
tobiashah, thought it is a not ready for review yet05:44
AJaegertobiash: recheck will not help, it depends on the project-config. I left the WIP in by accident ;(05:44
AJaegerSee the first run for success05:44
AJaeger(first changeset)05:44
tobiashAJaeger: is that 'voting: false' correct there: https://review.openstack.org/#/c/515593/4/tox.ini05:48
tobiash?05:48
tobiashthat's new to me05:48
AJaegertobiash: good catch, I screwed up ;(05:48
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Run ansible-lint on repo  https://review.openstack.org/51559305:49
AJaegertrying again ...05:49
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: DNM: test ansible crash logging  https://review.openstack.org/51547005:49
*** cshastri has joined #openstack-infra05:51
*** iyamahat has quit IRC05:52
tobiash+205:53
*** dsariel has joined #openstack-infra05:56
AJaegerthanks05:56
*** aeng has quit IRC06:01
*** yolanda__ has joined #openstack-infra06:02
*** spzala has joined #openstack-infra06:03
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Ansible-lint project-config  https://review.openstack.org/51560306:04
AJaegerpabelanger: and one job for project-config ^06:04
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Do late decoding of log stream buffer  https://review.openstack.org/51504306:04
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Correctly stream the remaining buffer  https://review.openstack.org/51547006:04
*** spzala has quit IRC06:07
*** jbadiapa has joined #openstack-infra06:10
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Add ansible-linter job for project-config  https://review.openstack.org/51560706:12
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Ansible-lint project-config  https://review.openstack.org/51560306:12
*** vsaienk0 has joined #openstack-infra06:17
*** markvoelker has quit IRC06:18
*** bhavik1 has quit IRC06:18
*** gildub has quit IRC06:19
*** thorst has joined #openstack-infra06:20
*** gouthamr has quit IRC06:21
*** spectr has joined #openstack-infra06:22
*** andreas_s has joined #openstack-infra06:24
*** thorst has quit IRC06:25
*** gongysh has quit IRC06:36
*** cody-somerville has quit IRC06:36
*** kzaitsev1pi has joined #openstack-infra06:38
*** kzaitsev_pi has quit IRC06:40
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Correctly stream the remaining buffer  https://review.openstack.org/51547006:45
*** ccamacho has quit IRC06:47
*** pgadiya has joined #openstack-infra06:48
tobiashinfra-root: I noticed a strange failure in a pre playbook of a pep8 job on the zuul project: http://paste.openstack.org/show/624784/06:51
tobiashlooks like something infrastructure (mirror?) related06:51
tobiashthe change is 515043,506:54
tobiashzuul did correctly retry and the overall build is green06:54
*** cody-somerville has joined #openstack-infra06:56
*** csomerville has joined #openstack-infra06:58
*** pblaho has joined #openstack-infra07:00
*** cody-somerville has quit IRC07:01
*** armaan has quit IRC07:04
*** armaan has joined #openstack-infra07:05
*** rossella_s has joined #openstack-infra07:06
ianwtobiash: 37.153.172.245 ... hmm07:09
tobiashianw: so bindep timed out and collect tox logs threw this ssh error07:10
tobiashjust wanted to inform that there might be something wrong07:10
tobiashbut could also be some transient network issue07:11
*** florianf has joined #openstack-infra07:11
*** slaweq has joined #openstack-infra07:12
ianwyeah, all the cityhost mirrors seem up07:14
*** markvoelker has joined #openstack-infra07:14
*** armaan has quit IRC07:15
*** armaan has joined #openstack-infra07:15
*** slaweq has quit IRC07:16
openstackgerritIan Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs  https://review.openstack.org/51245007:17
*** vsaienk0 has quit IRC07:20
*** vsaienk0 has joined #openstack-infra07:20
*** armax has quit IRC07:21
*** tesseract has joined #openstack-infra07:22
*** armax has joined #openstack-infra07:22
*** armax has quit IRC07:22
*** armax has joined #openstack-infra07:23
*** armax has quit IRC07:23
*** armax has joined #openstack-infra07:24
*** armax has quit IRC07:24
*** armax has joined #openstack-infra07:24
*** armax has quit IRC07:25
*** armax has joined #openstack-infra07:25
*** armax has quit IRC07:25
*** armax has joined #openstack-infra07:26
*** armax has quit IRC07:26
*** slaweq has joined #openstack-infra07:26
*** armax has joined #openstack-infra07:27
*** yolanda__ has quit IRC07:27
*** armax has quit IRC07:27
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-lib jobs  https://review.openstack.org/51562007:28
*** salv-orlando has quit IRC07:30
openstackgerritVasyl Saienko proposed openstack-infra/openstack-zuul-jobs master: Cleanup ironic-lib legacy jobs  https://review.openstack.org/51562307:31
*** salv-orlando has joined #openstack-infra07:31
*** thorst has joined #openstack-infra07:33
*** namnh has quit IRC07:34
*** salv-orlando has quit IRC07:35
*** pblaho has quit IRC07:37
*** thorst has quit IRC07:38
*** pblaho has joined #openstack-infra07:42
*** CrayZee has quit IRC07:43
vsaienk0AJaeger: yolanda please add to your review queue - cleanup ironic-lib legacy jobs https://review.openstack.org/515620 and https://review.openstack.org/51562307:43
*** e0ne has joined #openstack-infra07:45
*** kazsh has left #openstack-infra07:47
*** markvoelker has quit IRC07:48
*** jpena|off is now known as jpena07:49
*** ralonsoh has joined #openstack-infra07:50
*** amoralej|off is now known as amoralej07:51
*** salv-orlando has joined #openstack-infra07:52
*** makowals has joined #openstack-infra07:58
*** rcernin has quit IRC07:58
*** gcb has quit IRC08:00
*** liujiong has quit IRC08:02
*** liujiong has joined #openstack-infra08:02
*** gcb has joined #openstack-infra08:03
*** slaweq has quit IRC08:04
*** Hal has joined #openstack-infra08:05
*** slaweq has joined #openstack-infra08:05
*** Hal is now known as Guest7285208:05
*** hashar has joined #openstack-infra08:09
*** ccamacho has joined #openstack-infra08:10
AJaegervsaienk0: commented. What about moving tox-cover into project-config, see my comments08:14
*** yamahata has quit IRC08:14
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-lib jobs  https://review.openstack.org/51562008:17
vsaienk0AJaeger: updated08:18
AJaegervsaienk0: +208:18
*** armaan has quit IRC08:18
*** armaan has joined #openstack-infra08:19
vsaienk0AJaeger: thanks!08:22
*** jascott1 has quit IRC08:22
*** jascott1 has joined #openstack-infra08:23
*** martinkopec has joined #openstack-infra08:25
*** lucas-afk is now known as lucasagomes08:26
*** jascott1 has quit IRC08:27
*** panda|ruck|off is now known as panda|ruck08:29
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564308:29
*** mhickey has joined #openstack-infra08:30
openstackgerritMerged openstack-infra/project-config master: Remove ansible-role-jobs template from openstack-ansible-ops  https://review.openstack.org/51537708:33
*** udesale has quit IRC08:35
hwoarangjeblair: mordred: we are facing an issue in openstack-ansible where our post-run playbook becomes available (cloned) during the run phase. However, zuul complains that 'ERROR Unable to find playbook ...' blah blah. Do you know if it's possible for zuulv3 to do the post-run evaluation after the 'run' phase?08:38
SamYaplehwoarang: i wonder if you could drop a placeholder in there to trick zuul08:38
SamYapleor if its rendering all of that ahead of time08:39
hwoarangyeah we can't drop a dummy file easily there. if we go down that road we may as well copy the playbook08:40
hwoarangjust wanted to check if there is a magic flag to turn on like static/dynamic playbook inclusion in ansible08:40
SamYaplehwoarang: not sure your exact use case, but why not clone the project initially?08:40
SamYapledo you have a patchset you are workign with08:40
vsaienk0AJaeger: we found that after migration to zuulv3 synchronize task stop working for ironic-python-agent project. When we fixed  https://review.openstack.org/#/c/515356/ we found that now it uploads UPLOAD_RAW/ and UPLOAD_TAR folders instead of copying content should be address by https://review.openstack.org/#/c/515484/ . Just curious if it is known issue?08:41
hwoarangSamYaple: the 'run' playbook is responsible to clone the post-run one08:41
hwoarangbut it will not run08:41
hwoarangwait08:41
SamYaplehwoarang: right, but why do it that way? why not clone it ahead of time ala required-projects08:42
SamYapleactually, now that im saying that that wont work08:42
hwoaranghmm08:42
hwoarangthat sounds reasonable. Why would it not work08:43
SamYaplehwoarang: you can try it, but i suspect you wont be able to since its only going to be looking in that repos relative directories08:44
SamYaplethe job would need to be declared in teh repo where the post-playbook lives and parented in the other repo you want to consume that playbook i believe08:44
AJaegervsaienk0: better ask rest of team, I'm not yet familiar with these ;(08:44
hwoarangyeah that's true08:44
*** huanxie has quit IRC08:44
*** markvoelker has joined #openstack-infra08:45
*** derekh has joined #openstack-infra08:45
*** sflanigan has joined #openstack-infra08:46
*** sflanigan has joined #openstack-infra08:46
SamYaplehwoarang: i bet you could declare the base job in repo A, then parent it in repo B with your post-playbook, then parent that job in repo A again to consume it08:46
*** shu-mutou is now known as shu-mutou-AWAY08:46
SamYaple(depending on what youre trying to do)08:46
AJaegerfrickler: could you review https://review.openstack.org/#/q/status:open+topic:newton-eol and https://review.openstack.org/#/q/status:open+topic:ansible-lint , please? Neither contains urgent fixes, just allow me to move forward with testing and cleanup ;)08:46
vsaienk0infra team could you please check issue with synchronize task described above ^08:47
tobiashhwoarang: zuul prepares all playbooks it runs by itself before running each playbook, the job cannot clone further playbooks and expect zuul to run them08:48
*** electrofelix has joined #openstack-infra08:48
tobiashhwoarang: what are you trying to achieve?08:49
*** spiffxp_ has quit IRC08:50
hwoarangtobiash: we have a post-run playbook that we want to re-use across different repos. So we were hopping to clone the common playbook from that repo in our other repos and use it08:50
hwoarangsomething like that08:50
tobiashhwoarang: for this you can define a job in the repo with the common playbook only specifying the post-run08:51
*** ykarel is now known as ykarel|lunch08:51
*** spiffxp has joined #openstack-infra08:51
*** namnh has joined #openstack-infra08:51
tobiashhwoarang: your other jobs can inherit from this one and all will use this post playbook08:51
tobiashhwoarang: regardless of the repo they are defined in08:51
*** pblaho has quit IRC08:51
*** udesale has joined #openstack-infra08:53
openstackgerritMerged openstack-infra/project-config master: Remove legacy ironic-lib jobs  https://review.openstack.org/51562008:53
hwoarangtobiash: interesting. let me try08:54
*** dizquierdo has joined #openstack-infra08:54
*** sdague has joined #openstack-infra08:54
tobiashhwoarang: the key for this is job inheritence: https://docs.openstack.org/infra/zuul/user/config.html#job08:56
AJaegerhwoarang: that should work without any change, zuul should do the magic08:56
hwoarangok i will check thank you for all the pointers08:57
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Newton-EOL: Remove obsolete jobs  https://review.openstack.org/51549308:59
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564309:02
*** sflanigan has quit IRC09:02
*** spzala has joined #openstack-infra09:04
*** spzala has quit IRC09:09
*** bmjen has quit IRC09:11
SamYapletobiash: they are just reusing the job from repo A in repo B, without parenting it. *however* they were not explictly setting repo A in the required-projects list of the job09:13
tobiashSamYaple: I think that should work automatically without having A in required-projects09:14
SamYapletobiash: it does not09:15
SamYapleit will implicitly clone the current repo, but when you run in repo B, that means its only cloning repo B09:15
SamYapleyou have to explicity declare repo A to use it when running the job in repo B09:15
* SamYaple just went through this for cross-project loci jobs09:16
*** slaweq has quit IRC09:16
*** slaweq has joined #openstack-infra09:17
tobiashSamYaple: but that sounds like a bug (and might also be related to the branching issues jeblair is fixing currently)09:17
*** tosky has joined #openstack-infra09:17
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Fuel branch cleanup  https://review.openstack.org/51550109:17
AJaegerfrickler: thanks for reviewing, fixed ^09:17
*** bmjen has joined #openstack-infra09:17
*** markvoelker has quit IRC09:18
tobiashSamYaple: you won't have repo A in your workspace but the playbook must be available to zuul09:18
SamYapletobiash: there was some conversations about it. in the past. ill have to check my logs. im not sure if it was a bug, or making the clone automatically work was a new feature, but it is documented that you have to explictly declare it, so someone, at sometime knew about this09:18
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Fuel branch cleanup  https://review.openstack.org/51550109:18
tobiashSamYaple: can you point me to where this is documented?09:19
*** ykarel|lunch is now known as ykarel09:20
SamYapletobiash: yea im looking through my logs. someone in infra walked me through this when we first rolled to v3 (before the rollback)09:20
*** slaweq has quit IRC09:22
*** slaweq has joined #openstack-infra09:22
SamYapleoops tell a lie. this was only a couple weeks ago. time flies right09:23
SamYapletobiash: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2017-10-16.log.html#t2017-10-16T19:55:4709:23
SamYapletobiash: i did absoluely require that to make this work, just fyi09:25
SamYapleperhaps mordred can explain why its needed (and if its a bug)? he doesnt seem to be talking like its a bug09:25
tobiashSamYaple: because you use the other repo in your job or because the jobs playbooks weren't there?09:26
tobiashSamYaple: there is a difference in having the playbook available on the executor (as part of the job)09:26
SamYapletobiash: because i was putting a loci job (repo A) in the cinder pipelines (repo B) i needed to explictly declare repo A in my required projects list09:26
tobiashSamYaple: and having the repo available in the node where the job is executed09:27
SamYaplethis is the same thing openstack-ansible is doing09:27
SamYapletobiash: https://review.openstack.org/#/c/512400/ was required for https://review.openstack.org/#/c/512398/09:28
*** LindaWang has quit IRC09:29
*** huanxie has joined #openstack-infra09:29
*** LindaWang has joined #openstack-infra09:29
SamYapletobiash: yea i get what youre saying. im not sure then. its like 530am right now. if you find out my understanding is incorrect, please correct me :D i will check scrollback when i wake up09:30
tobiashSamYaple: does the loci-cinder job something with the loci repo?09:31
tobiashthat would be something what would require required-projects09:32
SamYaplethe run job uses files from the loci repo, yes. which i saw was your point earlier09:32
*** armaan has quit IRC09:32
SamYaplebut thats what OSA is doign too09:32
*** armaan has joined #openstack-infra09:32
tobiashah I see09:32
tobiashthen required-projects is needed09:32
SamYapleyea i understand the distinction you are making too09:32
SamYapleim still happy to be wrong here, but i cant process any more zuul stuff tonight, so im heading out09:33
SamYaplethanks for explaining more to me09:33
tobiashI have to check the doc, maybe it's not clear enough about that09:33
tobiashnp09:33
*** thorst has joined #openstack-infra09:34
AJaegerwe might have some subtile differences here - http://git.openstack.org/cgit/openstack/openstack-manuals/tree/.zuul.yaml#n7 is used in http://git.openstack.org/cgit/openstack/api-site/tree/.zuul.yaml#n7 - without required-projects09:35
AJaegerbut since it's all self-testing, hwoarang should be be able to iterate on it ;)09:35
tobiashyes, the difference is if the job needs the repo in the work dir on the node it needs required-projects09:36
*** gcb has quit IRC09:37
*** LindaWang has quit IRC09:38
*** stakeda has quit IRC09:38
AJaegertobiash: yeah. for api-site, we need *only* the playbook09:38
*** liujiong has quit IRC09:39
*** LindaWang has joined #openstack-infra09:39
*** thorst has quit IRC09:39
*** liujiong has joined #openstack-infra09:39
*** links has quit IRC09:42
*** liujiong has quit IRC09:46
*** dsariel has quit IRC09:47
*** LindaWang has quit IRC09:52
*** maciejjozefczyk has joined #openstack-infra09:52
*** slaweq_ has joined #openstack-infra09:52
*** dpawlik has joined #openstack-infra09:52
*** dizquierdo is now known as dizquierdo_afk09:54
*** slaweq has quit IRC09:54
*** danpawlik has quit IRC09:55
*** maciejjo1 has quit IRC09:55
*** dizquierdo_afk is now known as dizquierdo09:55
*** armaan has quit IRC09:58
*** gmann is now known as gmann_afk10:02
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564310:04
*** kaisers1 has quit IRC10:05
*** psachin has quit IRC10:05
*** lindycoder has joined #openstack-infra10:10
*** LindaWang has joined #openstack-infra10:10
fricklerinfra-root: sdague: mtreinish: https://review.openstack.org/509313 seems ready now, getting elastic-recheck working with Zuul v3 again10:10
*** dsariel has joined #openstack-infra10:11
sdaguefrickler: cool, though maybe just convert the console.html to tags:console?10:11
sdaguethat simplifies things, right?10:12
*** daidv has quit IRC10:12
sdaguefrickler: I'm +A as is10:12
sdaguebut that might be a nice cleanup afterwards10:12
*** hashar is now known as hasharLunch10:13
*** kaisers has joined #openstack-infra10:14
fricklersdague: yeah, this is still trying to keep compatibility with v2 for now, just in case10:14
sdaguefrickler: sure10:14
sdaguethough tags:console should also match things that were tags:console.html10:14
sdagueit was mostly us not being consistent with bugs10:15
*** markvoelker has joined #openstack-infra10:15
*** kiennt26 has quit IRC10:17
*** mhickey has quit IRC10:17
*** Goneri has joined #openstack-infra10:20
openstackgerritMerged openstack-infra/elastic-recheck master: Add support for Zuulv3-specific parameters in elastic-recheck  https://review.openstack.org/50931310:22
*** yamamoto has joined #openstack-infra10:24
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Cleanup ironic-lib legacy jobs  https://review.openstack.org/51562310:26
*** andreas_s has quit IRC10:26
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove osc-dsvm-functional* jobs  https://review.openstack.org/51508810:26
*** andreas_s has joined #openstack-infra10:27
*** ldnunes has joined #openstack-infra10:27
*** armaan has joined #openstack-infra10:31
*** cuongnv has quit IRC10:33
openstackgerritMerged openstack-infra/project-config master: Enable documentation building for monascaclient  https://review.openstack.org/51495410:34
*** boden has joined #openstack-infra10:34
*** namnh has quit IRC10:36
*** vsaienk0 has quit IRC10:37
*** andreas_s has quit IRC10:37
*** andreas_s has joined #openstack-infra10:39
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove tacker legacy job  https://review.openstack.org/51534610:39
*** dizquierdo is now known as dizquierdo_afk10:41
*** armaan has quit IRC10:43
*** pblaho has joined #openstack-infra10:44
*** pbourke has quit IRC10:44
*** yamamoto has quit IRC10:45
*** pbourke has joined #openstack-infra10:46
*** markvoelker has quit IRC10:49
*** andreas_s has quit IRC10:50
*** rossella_s has quit IRC10:51
openstackgerritMerged openstack-infra/project-config master: Upload to pypi if pre / run was successful  https://review.openstack.org/51439810:52
*** Goneri has quit IRC10:53
*** andreas_s has joined #openstack-infra10:55
*** Goneri has joined #openstack-infra10:55
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove legacy oslo.db job  https://review.openstack.org/51141210:55
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Remove migrated legacy puppet openstack check jobs  https://review.openstack.org/51526310:55
*** vsaienk0 has joined #openstack-infra11:00
*** andreas_s has quit IRC11:01
*** andreas_s has joined #openstack-infra11:02
*** robcresswell has quit IRC11:03
*** annp has quit IRC11:03
*** yamamoto has joined #openstack-infra11:05
*** spzala has joined #openstack-infra11:06
*** lucasagomes is now known as lucas-hungry11:07
*** dsariel has quit IRC11:09
*** andreas_s has quit IRC11:12
*** spzala has quit IRC11:12
openstackgerritWitold Bedyk proposed openstack-infra/project-config master: Add openstack/monasca-specs repo  https://review.openstack.org/51568511:13
*** andreas_s has joined #openstack-infra11:13
*** smatzek has joined #openstack-infra11:15
*** andreas_s has quit IRC11:15
*** andreas_s has joined #openstack-infra11:16
*** mat128 has joined #openstack-infra11:17
*** yamamoto has quit IRC11:21
ykareldhellmann, hi11:22
ykarelI was trying to understand why magnum was not included in: https://review.openstack.org/#/c/493239/ add stub deliverable files for queens11:22
*** dizquierdo_afk is now known as dizquierdo11:23
ykareldo you know why?11:23
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Add missing role for deploy-guide.yaml  https://review.openstack.org/51568711:26
AJaegerykarel: there's an #openstack-release channel where you'll find the  release management team11:27
ykarelAJaeger, Ok11:27
AJaegerconfig-core, we missed one case with role, see 51568711:27
*** Goneri has quit IRC11:30
*** hamzy has joined #openstack-infra11:30
*** sdague has quit IRC11:31
*** sdague has joined #openstack-infra11:32
*** baoli has joined #openstack-infra11:34
dmsimardAJaeger: +311:35
*** thorst has joined #openstack-infra11:35
*** Goneri has joined #openstack-infra11:36
*** sdague has quit IRC11:37
AJaegerthanks, dmsimard11:37
*** baoli has quit IRC11:37
*** sdague has joined #openstack-infra11:38
*** sdague has quit IRC11:38
*** baoli has joined #openstack-infra11:38
*** dizquierdo is now known as dizquierdo_afk11:38
*** martinkopec has quit IRC11:38
openstackgerritMerged openstack-infra/project-config master: Add missing role for deploy-guide.yaml  https://review.openstack.org/51568711:39
AJaegerpabelanger: Some more work on ansible-lint, see https://review.openstack.org/#/q/status:open+topic:ansible-lint11:39
*** thorst has quit IRC11:40
*** sambetts|afk is now known as sambetts11:41
sambettsAJaeger: hey I'm seeing a failure in one of my legacy jobs: http://logs.openstack.org/43/510243/7/check/legacy-networking-cisco-tox-ocata/162268f/job-output.txt.gz#_2017-10-27_11_25_28_10948311:42
*** electrofelix has quit IRC11:43
sambettsAJaeger: it seems to be related to infra code but I'm not sure11:43
*** electrofelix has joined #openstack-infra11:43
*** nicolasbock has joined #openstack-infra11:43
dmsimardsambetts: looking11:43
sambettsdmsimard: thanks11:45
*** thorst has joined #openstack-infra11:46
*** sdague has joined #openstack-infra11:46
dmsimardsambetts: has that worked before or did it just break ?11:46
*** markvoelker has joined #openstack-infra11:46
sambettsdmsimard: it passed on a different patch 3 minutes ago https://review.openstack.org/#/c/512309/11:48
*** hasharLunch is now known as hashar11:48
dmsimardsambetts: okay, it's a bit weird -- I'm not familiar with those chunks of code but the error came from here: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/run-tox.sh#n7611:49
dmsimardSo this script returned 1 .. http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/jenkins-sudo-grep.sh11:50
*** yamamoto has joined #openstack-infra11:50
*** rosmaita has joined #openstack-infra11:50
dmsimardbecause it found this in the logs "> Oct 27 11:03:16 ubuntu sudo:     zuul : command not allowed ; TTY=unknown ; PWD=/home/zuul/workspace ; USER=root ; COMMAND=/bin/true"11:50
dmsimardSomething tried to run sudo /bin/true ? :/11:51
dmsimardMaybe someone else would know ^11:51
tobiashdmsimard: the revoke-sudo role runs sudo to test if it's not allowed: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/revoke-sudo/tasks/main.yaml11:56
dmsimardtobiash: that's not this role, though11:56
dmsimardtobiash: oh, wait, yes it is11:56
dmsimardtobiash: so we must fix the script to avoid catching /bin/true then, good catch !11:57
* dmsimard sends patch11:57
openstackgerritThomas Herve proposed openstack-infra/tripleo-ci master: Fail on curl error when setting up repo  https://review.openstack.org/51569511:57
tobiash:)11:57
dmsimardtobiash++11:57
*** dizquierdo_afk is now known as dizquierdo11:57
*** jpena is now known as jpena|lunch11:58
*** dtantsur|afk is now known as dtantsur11:58
*** yamamoto has quit IRC11:59
*** huanxie has quit IRC12:01
evrardjpI am clueless on why some job don't trigger, may I borrow a pair of eyes?12:01
evrardjpdmsimard maybe?12:01
sambettstobiash, dmsimard: thanks guys! is quite frustrating that http://logs.openstack.org/43/510243/7/check/legacy-networking-cisco-tox-ocata/162268f/job-output.txt.gz#_2017-10-27_11_03_15_159733 doesn't log the role name or task name to make it easy to grep for12:02
dmsimardsambetts: we'll eventually make things easier to find12:04
dmsimardsambetts: once the big fires have been put off, there's a lot of great opportunities for improving people's lives :D12:05
dmsimardevrardjp: what's up ?12:05
evrardjpapparently not my eyelids!12:05
evrardjp:p12:05
evrardjplet me show you something12:05
sambettsdmsimard: of course :)12:05
evrardjphttps://github.com/openstack/openstack-ansible/blob/master/zuul.d/jobs.yaml#L40-L4512:05
*** dprince has joined #openstack-infra12:05
evrardjpas you can see, pike is defined there12:06
evrardjpand the job is defined in the project12:06
evrardjphttps://github.com/openstack/openstack-ansible/blob/master/zuul.d/project.yaml#L2312:06
evrardjpso question12:06
evrardjpwhy this: https://review.openstack.org/#/c/512046/12:06
evrardjphas not tested this job?12:07
*** askb has quit IRC12:08
*** nsingh has joined #openstack-infra12:08
*** abelur_ has quit IRC12:08
*** spzala has joined #openstack-infra12:08
tobiashevrardjp: there are currently some branch related fixes in progress jeblair is working on12:09
*** niraj_singh has quit IRC12:09
tobiashevrardjp: it could be that you are affected by the current branch related bugs12:10
evrardjpthat must be a nightmare12:10
AJaegerevrardjp: read also https://docs.openstack.org/infra/manual/zuulv3.html#stable-branches12:10
AJaegerevrardjp: I don#t see a zuul.d directoy in stable/pike branch12:11
evrardjpAJaeger: yeah, because this time I defined branches12:11
evrardjpin master12:11
evrardjpbut ok I will backport those changes one per one12:11
AJaegerevrardjp: that will not help as explained in the link I gave.12:11
evrardjpyeah12:11
evrardjpI've seen12:11
*** jaypipes has joined #openstack-infra12:12
evrardjpAJaeger: on your link there is something saying12:12
evrardjp"unless branches are explicitly specified for the jobs in the project stanza,"12:12
AJaegerevrardjp: but as tobiash pointed out, there's bugs in branch handling as well12:12
evrardjpAJaeger: I am merely following the docs there :p12:12
evrardjpI am fine backporting those repo things, I just thought I was covered, due to the branches thing.12:13
AJaegerevrardjp: indeed - consider me confused. Let's wait for the jeblair magic to happen ;)12:13
*** spzala has quit IRC12:13
evrardjp:D12:13
*** yamamoto has joined #openstack-infra12:14
evrardjpin the meantime I will propose the patches. It's maybe safer in the long run.12:14
AJaegerdmsimard: care to review https://review.openstack.org/515501, please?12:14
*** lucas-hungry is now known as lucasagomes12:14
*** trown|outtypewww is now known as trown12:18
*** markvoelker has quit IRC12:20
*** Guest72852 has quit IRC12:24
*** camunoz has joined #openstack-infra12:24
*** askb has joined #openstack-infra12:25
*** rhallisey has quit IRC12:25
*** abelur_ has joined #openstack-infra12:26
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Make prepare-docs-for-afs useable for different paths  https://review.openstack.org/51569712:26
dmsimardevrardjp: sorry got distracted, did you get your answer ?12:27
dmsimardAJaeger: ouch, is someone from fuel weighing in on that patch ?12:27
*** markvoelker has joined #openstack-infra12:27
dmsimardAJaeger: oh wait, nevermind12:27
*** pgadiya has quit IRC12:28
AJaegerdmsimard: no - this is result of branch retiring and careful review of our automatic conversion12:28
*** StevenK has quit IRC12:29
*** rm_work has quit IRC12:29
evrardjpdmsimard: yes and now12:29
evrardjpno*12:29
*** rm_work has joined #openstack-infra12:30
*** rm_work has quit IRC12:30
*** rm_work has joined #openstack-infra12:30
*** jcoufal has joined #openstack-infra12:30
*** sflanigan has joined #openstack-infra12:31
*** sflanigan has joined #openstack-infra12:31
*** rlandy has joined #openstack-infra12:32
*** baoli has quit IRC12:32
dmsimardAJaeger: added a comment before +312:32
AJaegerdmsimard: commented - that line was broken conversoin, carefull analysis of zuul/layout shows that we never run that job at all12:34
dmsimardAJaeger: ack12:35
AJaegerthose fuel regexes were a mess ;(12:35
dmsimardAJaeger: btw does the zuul layout compile in yaml, json or xml in your brain ? :p12:35
AJaegerdmsimard: it's yaml ;) - and there once was a job that outputs the jobs with conditions12:36
*** baoli has joined #openstack-infra12:37
*** armaan has joined #openstack-infra12:37
openstackgerritDavid Moreau Simard proposed openstack-infra/project-config master: Exclude COMMAND=/bin/true from sudoers verification  https://review.openstack.org/51570012:38
openstackgerritMerged openstack-infra/project-config master: Fuel branch cleanup  https://review.openstack.org/51550112:38
dmsimardsambetts, tobiash, AJaeger ^ I *think* that should be fine, tested it locally12:38
AJaegerdmsimard: thanks - I wonder why this shows up now and not earlier...12:39
dmsimardAJaeger: sambetts seemed to suggest that it flapped, it is related to revoke sudo role.. maybe there are tox jobs running revoke-sudo and some aren't.12:39
AJaegerchange looks fine but I let others look at it first...12:40
*** StevenK has joined #openstack-infra12:41
dmsimardAJaeger: only thing IMO is that I might wrongly assume egrep (grep -P) is available everywhere12:41
dmsimard¯\_(ツ)_/¯12:41
dmsimardpretty sure it is, but who knows12:41
*** dsariel has joined #openstack-infra12:45
*** vsaienk0 has quit IRC12:46
*** jpena|lunch is now known as jpena12:50
*** spzala has joined #openstack-infra12:50
*** dsariel has quit IRC12:51
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Move test_model.test_job_inheritance_configloader  https://review.openstack.org/50949512:51
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove test_job_inheritance  https://review.openstack.org/50950912:51
*** Hal has joined #openstack-infra12:52
*** dhajare|mtg has quit IRC12:52
*** wolverineav has joined #openstack-infra12:52
*** Hal is now known as Guest4351812:52
*** dsariel has joined #openstack-infra12:55
*** amoralej is now known as amoralej|lunch12:55
*** mdbooth has quit IRC12:58
*** ari[m] has quit IRC12:58
evrardjpegrep is not available everywhere dmsimard, but there is a change I might be trolling you for large distros.12:59
evrardjpchance*12:59
evrardjpbut then you need gnu grep and all.13:00
evrardjpjeblair: could you enlighten us with the branch issue you're working on right now?13:00
dmsimardevrardjp: I mean I'm PRETTY confident it's there for fedora, centos, rhel, debian, ubuntu13:00
evrardjp:D13:00
dmsimardbut I don't know for like... gentoo or opensuse13:00
*** marst has joined #openstack-infra13:01
kklimondahmm, how are various mirrors updated for nodepool-provisioned machines? `ready-script: create_mirror.sh` is no longer there in nodepool v313:01
*** pblaho has quit IRC13:01
*** pblaho has joined #openstack-infra13:02
*** jaypipes is now known as leakypipes13:02
tobiashevrardjp: he wrote a lengthly mail about that on the *-infra mailing list: http://lists.openstack.org/pipermail/openstack-infra/2017-October/005634.html13:02
*** Mutter has joined #openstack-infra13:02
tobiashkklimonda: in zuulv3 the base job does this13:02
evrardjpthanks will read.13:02
AJaegerdmsimard: it's there fore openSUSE13:02
openstackgerritMerged openstack-infra/project-config master: zuul-jobs: Run ansible-lint  https://review.openstack.org/51559613:02
*** armaan has quit IRC13:02
kklimondatobiash: oh right, I just had that idea - configure-mirrors, right?13:02
tobiashyepp13:03
*** spzala has quit IRC13:04
*** bauzas is now known as bauwser13:04
*** Mutter has quit IRC13:04
*** catintheroof has joined #openstack-infra13:04
dmsimardAJaeger: comment on https://review.openstack.org/#/c/515603/13:05
*** shardy has quit IRC13:07
AJaegerdmsimard: thanks, fixing...13:07
AJaegergood spot!13:07
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Ansible-lint project-config  https://review.openstack.org/51560313:07
*** udesale has quit IRC13:08
*** baoli has quit IRC13:09
dmsimardAJaeger: +2 :)13:09
*** kgiusti has joined #openstack-infra13:13
*** andreas_s has quit IRC13:15
*** gibi is now known as pow_2_3013:17
*** mriedem has joined #openstack-infra13:18
*** vsaienk0 has joined #openstack-infra13:18
*** fdegir has quit IRC13:19
*** pahuang has quit IRC13:19
*** andreas_s has joined #openstack-infra13:19
*** fdegir has joined #openstack-infra13:19
evrardjptobiash: thanks for the link. It's a intense read.13:20
*** sflanigan has quit IRC13:20
*** mdbooth has joined #openstack-infra13:21
*** ari[m] has joined #openstack-infra13:21
fungievrardjp: however, the changes to solve it are up for review as of ~12 hours ago13:22
*** dsariel has quit IRC13:23
evrardjpI am not sure I can technically have a look at those, because I am not inside the system enough to take a good decision or not.13:23
evrardjpbut I will have a look at them anyway :D13:23
*** pahuang has joined #openstack-infra13:24
*** andreas_s has quit IRC13:25
*** andreas_s has joined #openstack-infra13:25
*** hemna_ has joined #openstack-infra13:25
vsaienk0infra-root: ironic-python-agent post upload job was corrupted after switching to zuulv3, and on tarballs.openstack.org now we have corrupted directory structure, we need to remove the following folders and its content completely http://tarballs.openstack.org/ironic-python-agent/coreos/UPLOAD_TAR/  http://tarballs.openstack.org/ironic-python-agent/coreos/files/UPLOAD_RAW/  http://tarballs.openstack.org/ironic-python-agent/tinyipa/UPLOAD_TAR/ and13:26
vsaienk0dtantsur: ^ fyi13:26
AJaegerconfig-core, please review https://review.openstack.org/515697 - fixes are needed for deploy-guide publishing13:27
dtantsurvsaienk0: ugh :( thanks for tracking this13:27
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Add ansible-linter job for project-config  https://review.openstack.org/51560713:28
*** claudiub has joined #openstack-infra13:28
*** dchen has joined #openstack-infra13:28
*** dingyichen has quit IRC13:28
fungivsaienk0: i can take care of that in just a moment13:29
*** dansmith is now known as superdan13:30
*** claudiub|2 has quit IRC13:31
*** dizquierdo has quit IRC13:31
*** dizquierdo has joined #openstack-infra13:32
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Make linter job voting on zuul-jobs  https://review.openstack.org/51570813:32
fungivsaienk0: http://tarballs.openstack.org/ironic-python-agent/tinyipa/files/UPLOAD_RAW/ too?13:33
vsaienk0fungi: yes all UPLOAD_RAW/ and UPLOAD_TAR/ folders13:33
*** mhickey has joined #openstack-infra13:33
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Run linters as voting  https://review.openstack.org/51571013:34
*** catintheroof has quit IRC13:34
fungivsaienk0: cool, done13:34
vsaienk0fungi: thanks for help!13:34
*** Goneri has quit IRC13:35
fungivsaienk0: you bet. i've stashed them in my homedir temporarily for safekeeping in case you discover i removed something i shouldn't have (i go back through periodically and clean up my homedir)13:36
AJaegerdmsimard, fungi, please +2a https://review.openstack.org/#/c/515593/ - we're ready for the linters on zuul-jobs13:36
*** janki has quit IRC13:37
*** esberglu has joined #openstack-infra13:40
*** Goneri has joined #openstack-infra13:41
*** catintheroof has joined #openstack-infra13:42
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: Make linter job voting on zuul-jobs  https://review.openstack.org/51570813:43
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Run linters as voting  https://review.openstack.org/51571013:43
AJaegerconfig-core, added proper dependencies for the ansible-lint jobs, for full list of changes see https://review.openstack.org/#/q/status:open+topic:ansible-lint13:44
AJaegerbbl13:44
fungilooks like frickler beat me to approving13:47
fungii need to pop out to run an errand but hope to be back in an hour or so13:47
*** jascott1 has joined #openstack-infra13:47
*** spectr has quit IRC13:49
*** jascott1 has quit IRC13:52
*** dave-mccowan has joined #openstack-infra13:53
*** sree has quit IRC13:54
*** dhajare|mtg has joined #openstack-infra13:55
*** niraj_singh has joined #openstack-infra13:55
*** sree has joined #openstack-infra13:55
*** nsingh has quit IRC13:55
*** owalsh has quit IRC13:56
*** amoralej|lunch is now known as amoralej13:56
*** arxcruz is now known as arxcruz|pto13:56
*** owalsh has joined #openstack-infra13:59
openstackgerritMerged openstack-infra/zuul-jobs master: Run ansible-lint on repo  https://review.openstack.org/51559313:59
*** sree has quit IRC14:00
*** gouthamr has joined #openstack-infra14:01
AJaegerthree more to go: https://review.openstack.org/#/q/status:open+topic:ansible-lint ;)14:01
*** baoli has joined #openstack-infra14:01
*** Dinesh_Bhor has quit IRC14:04
*** d0ugal_ has joined #openstack-infra14:04
*** d0ugal has quit IRC14:05
*** eumel8 has quit IRC14:06
*** andreas_s has quit IRC14:06
*** tosky has quit IRC14:06
*** tosky has joined #openstack-infra14:08
*** spectr has joined #openstack-infra14:09
*** andreas_s has joined #openstack-infra14:10
*** claudiub has quit IRC14:13
*** andreas_s has quit IRC14:14
*** andreas_s has joined #openstack-infra14:14
*** andreas_s has quit IRC14:14
openstackgerritSamuel Matzek proposed openstack-infra/project-config master: Remove tempest job  https://review.openstack.org/51476914:14
*** hongbin has joined #openstack-infra14:15
*** andreas_s has joined #openstack-infra14:15
*** armax has joined #openstack-infra14:17
*** dsariel has joined #openstack-infra14:17
*** andreas_s has quit IRC14:19
*** dave-mccowan has quit IRC14:20
pabelangerAJaeger: dmsimard: frickler: 515607 isn't going to work, as we cannot parent to jobs in ozj from project-config.  We can just use openstack-zuul-jobs-linters job directly however14:23
*** felipemonteiro_ has joined #openstack-infra14:26
*** baoli has quit IRC14:29
jeblairevrardjp: you can use branch matchers in-repo, but my recommendation would instead be to copy the jobs + playbooks to the various branches.  and hopefully that should work correctly some time today.14:31
*** vsaienk0 has quit IRC14:31
*** dsariel has quit IRC14:31
evrardjpjeblair: yes, I've read the whole conversation and thought that would be the recommendation. I've started to port all these definitions back. I think a bug will emerge though....14:31
*** felipemonteiro__ has joined #openstack-infra14:32
evrardjpgive you more feedback soon, when your series of patches has landed, this way I don't disturb you!14:33
*** dsariel has joined #openstack-infra14:33
jeblairevrardjp: i'd love to hear about the bug you suspect!  hopefully it's already addressed and i can just make sure it's clear in the docs, but if it will be a problem, before we roll it out would be a good time to know.  :)14:34
*** rbrndt has joined #openstack-infra14:34
evrardjpok I'll shoot then:p14:34
*** baoli has joined #openstack-infra14:35
*** dbecker has quit IRC14:35
*** dbecker has joined #openstack-infra14:35
*** felipemonteiro_ has quit IRC14:36
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Switch vitrage service to service release job  https://review.openstack.org/51572614:36
evrardjpon this patchset, where I (now) define in-tree the jobs for this stable branch (newton), I see none of the new in-tree project jobs defined.14:36
evrardjphttps://review.openstack.org/#/c/515713/1/zuul.d/project.yaml14:37
evrardjpanything that was defined in master is fine.14:37
evrardjpwhich makes me think that the inheritence here: https://review.openstack.org/#/c/515713/1/zuul.d/jobs.yaml would be broken if not completely redefined.14:37
evrardjpwould that be a correct assumption?14:38
*** vsaienk0 has joined #openstack-infra14:38
AJaegerpabelanger: so, what do you mean - move the job to project-config?14:38
jeblairevrardjp: where is openstack-ansible-deploy-aio-ubuntu-xenial defined?14:39
evrardjpmaster14:39
*** spectr has quit IRC14:39
*** dbecker has quit IRC14:40
evrardjphttps://github.com/openstack/openstack-ansible/blob/master/zuul.d/jobs.yaml#L4014:40
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Revert "Add ansible-linter job for project-config"  https://review.openstack.org/51572714:40
*** dbecker has joined #openstack-infra14:40
evrardjpOh I may need to remove the branch matching on master. That may be the cause.14:40
evrardjpjeblair: my bad!14:40
*** baoli has quit IRC14:42
pabelangerAJaeger: lets just use openstack-zuul-jobs-linters job for now, and I believe jeblair has patches up fix the late binding issue14:42
jeblairevrardjp: yep that's it14:42
evrardjpsorry :(14:43
evrardjpoverlook apparently.14:43
pabelangerAJaeger: or rename openstack-zuul-jobs-linters to something else, that you are comfortable with14:43
openstackgerritJulia Kreger proposed openstack-infra/project-config master: Remove legacy bifrost jobs  https://review.openstack.org/51572814:45
jeblairevrardjp: np.  aside from that, the way you've constructed things should work.  i'd recommend the other approach because it will be more natural as time passes and we make new branches off of master (all of which will contain all of this content).  however, if you want to maintain most or all the logic in master, that should be possible, just with a bit more upkeep.  and maybe you want to get the jobs working and stable in master with ...14:45
jeblair... branch matchers and then backport them when ready.  that's fine too.  :)14:45
openstackgerritJulia Kreger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy bifrost jobs  https://review.openstack.org/51572914:45
*** e0ne has quit IRC14:45
*** andreas_s has joined #openstack-infra14:45
AJaegerpabelanger: I copy it over to project-config and rename on that way...14:45
*** baoli has joined #openstack-infra14:45
*** HeOS has quit IRC14:45
evrardjpjeblair: I think it's more self contained that way, like you suggested. Thanks for the advice!14:46
jeblairevrardjp: np14:46
*** baoli has quit IRC14:48
AJaegerpabelanger: https://review.openstack.org/515603 passed the tests - I'm not convinced why it would not work14:48
*** armaan has joined #openstack-infra14:48
*** HeOS has joined #openstack-infra14:48
AJaegerpabelanger: we have other jobs in project-config running for project-config in similar way.14:49
AJaegerLike project-config-gerrit in openstack-zuul-jobs14:49
pabelangerAJaeger: do they parent to something outside project-config?14:49
AJaegerhttp://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/jobs.yaml#n66714:50
AJaegerparents to tox which is in zuul-jobs14:50
pabelangerif you use openstack-zuul-jobs-linters directly, it should work. But IIRC, because project-config-zuul-jobs-linter parents to openstack-zuul-jobs-linters, that isn't going to work because ozj is loaded after project-config14:50
pabelangerAJaeger: no, that is fine. ozj, is after zuul-jobs in main14:50
pabelangermain.yaml14:51
*** andreas_s has quit IRC14:51
*** LindaWang1 has joined #openstack-infra14:51
openstackgerritJames E. Blair proposed openstack-infra/infra-manual master: Zuul v3: clarify stable branches  https://review.openstack.org/51573214:52
AJaegerpabelanger: I'm not getting it yet. we *use* project-config-gerrit in project-config the same way we use  project-config-zuul-jobs-linters in it IMHO.14:52
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Remove publish to pypi for aodh and panko  https://review.openstack.org/51573314:52
AJaegerWhy would openstack-zuul-jobs-linters work but project-config-zuul-jobs-linters not?14:52
AJaegerAnd why is zuul v3 check ok?14:52
*** andreas_s has joined #openstack-infra14:52
*** LindaWang has quit IRC14:53
*** LindaWang1 is now known as LindaWang14:53
*** Guest43518 has quit IRC14:53
*** e0ne has joined #openstack-infra14:53
*** e0ne has quit IRC14:53
pabelangerAJaeger: okay, nevermind14:53
*** e0ne has joined #openstack-infra14:54
AJaegerpabelanger: you might be right, I might just not getting it ;)14:54
pabelangerAJaeger: For some reason I tought project-config-zuul-jobs-linters was in project-config .zuul.yaml14:54
openstackgerritRabi Mishra proposed openstack-infra/devstack-gate master: Set DEVSTACK_GATE_CEILOMETER_BACKEND to gnocchi  https://review.openstack.org/51573414:54
pabelangerAJaeger: so, that will work correctly14:54
pabelangerAJaeger: if it was in project-config .zuul.yaml, it would not work14:54
*** e0ne has quit IRC14:54
pabelangerAJaeger: +314:54
AJaegercorrect, that was my first try- cpoying it over. it didn't know parent tox ;(14:54
AJaegerthanks, pabelanger14:54
*** bobh has joined #openstack-infra14:55
AJaegerpabelanger: thanks! I still learned something ;)14:55
*** e0ne has joined #openstack-infra14:55
AJaegerpabelanger: what do you think of my two other changes - see https://review.openstack.org/#/q/status:open+topic:ansible-lint14:55
*** e0ne has quit IRC14:56
*** e0ne has joined #openstack-infra14:56
*** e0ne has quit IRC14:57
*** andreas_s has quit IRC14:57
*** ethfci has quit IRC14:57
pabelangerlooking14:57
pabelangerand drinking coffee :)14:57
AJaegerpabelanger: thanks - and enjoy!14:58
*** pblaho has quit IRC14:58
*** ldnunes has quit IRC14:58
*** bobh has quit IRC14:58
AJaegerand could I persuade some config-core to review https://review.openstack.org/#/c/515697/ and https://review.openstack.org/514053 as well, please? those fix document publishing14:59
*** ihrachys has joined #openstack-infra15:00
pabelangerAJaeger: -1 on 515708, want to see results from job run first. Once 515603 lands we can recheck15:01
*** ijw has joined #openstack-infra15:01
*** ijw_ has joined #openstack-infra15:02
AJaegerpabelanger: thta's for zuul-jobs , see https://review.openstack.org/#/c/515593/ for positive result15:03
AJaegeror would project-config usage find new problems in zuul-jobs?15:04
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-inspector jobs  https://review.openstack.org/51573715:05
pabelangerAJaeger: project-config should find new issues in zuul-jobs, but we can turn the job voting15:05
pabelangerthanks15:05
AJaegerpabelanger: ok.. So, let's see what 515603 brings ;)15:06
*** marst has quit IRC15:06
*** ijw has quit IRC15:06
openstackgerritMerged openstack-infra/project-config master: Ansible-lint project-config  https://review.openstack.org/51560315:06
*** marst has joined #openstack-infra15:06
*** baoli has joined #openstack-infra15:07
*** dsariel has quit IRC15:07
openstackgerritMichael Johnson proposed openstack-infra/project-config master: Remove migrated octavia zuul v3 jobs  https://review.openstack.org/51548615:07
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Run linters as voting  https://review.openstack.org/51571015:08
openstackgerritVasyl Saienko proposed openstack-infra/openstack-zuul-jobs master: Cleanup ironic-inspector legacy jobs  https://review.openstack.org/51573815:08
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role  https://review.openstack.org/51564315:09
AJaegerfungi, pabelanger, clarkb , could you review https://review.openstack.org/515700 , please? That should fix sambetts' problem but I'm surprised to see this now15:09
openstackgerritRabi Mishra proposed openstack-infra/devstack-gate master: Don't set CEILOMETER_BACKEND in localrc  https://review.openstack.org/51573415:09
*** ldnunes has joined #openstack-infra15:10
openstackgerritMerged openstack-infra/project-config master: Make linter job voting on zuul-jobs  https://review.openstack.org/51570815:10
*** dave-mccowan has joined #openstack-infra15:11
*** vhosakot has joined #openstack-infra15:11
*** tosky has quit IRC15:12
AJaegerthanks everybody for reviewing my newton eol and ansible-lint changes!15:12
*** yamahata has joined #openstack-infra15:14
fricklerI'm seeing a post failure for openstack-tox-cover running on designate-dashboard because the "tox -ecover" there is different and doesn't seem to produce a results dir, see http://logs.openstack.org/89/515689/1/check/openstack-tox-cover/2e35d62/job-output.txt.gz#_2017-10-27_11_56_15_61685615:14
fricklershould I change openstack-tox-cover to not fail in that case or rather create a variant of that job?15:15
*** hemna_ has quit IRC15:15
AJaegeransible-lint experts, we have failures for project-config at http://logs.openstack.org/86/515486/3/infra-check/project-config-zuul-jobs-linters/5f1d85a/job-output.txt.gz#_2017-10-27_15_13_36_71577115:15
*** sree has joined #openstack-infra15:16
fungifrickler: what's the point of running a coverage job that doesn't generate results? seems like something that should be fixed in their repo probably15:16
*** iyamahat has joined #openstack-infra15:16
fricklerfungi: I think that is because it is more javascript than python, but I'm not deep enough into that to decide. the other option would be to just remove the coverage job, like e.g. horizon seems to do15:17
pabelangerAJaeger: you'll need to skip that, zuul_return isn't shipped by ansible.  Another option is to install zuul, and add our libraries to ansible path15:18
*** links has joined #openstack-infra15:19
fungifrickler: right, i'm leaning toward fix things so that designate-dashboard's coverage job produces a report or stop running it. may be best to ask mugsie for input too15:19
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Make prepare-docs-for-afs useable for different paths  https://review.openstack.org/51569715:19
*** dave-mccowan has quit IRC15:21
fricklerfungi: o.k., will do, thx15:21
*** dhinesh has joined #openstack-infra15:22
*** armax_ has joined #openstack-infra15:22
*** salv-orlando has quit IRC15:22
*** armax has quit IRC15:22
*** armax_ is now known as armax15:22
*** salv-orlando has joined #openstack-infra15:23
*** yamamoto has quit IRC15:24
dhineshafter installing nodepool 4.1,  unable to do operations on it, i get this error https://pastebin.com/cDEQFrGU, is zookeeper required for it to work?15:25
*** cshastri has quit IRC15:25
*** slaweq_ has quit IRC15:25
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: ansible-lint: install zuul  https://review.openstack.org/51574215:26
AJaegerpabelanger: like this? ^15:26
*** salv-orlando has quit IRC15:27
*** ramishra has quit IRC15:27
*** andreas_s has joined #openstack-infra15:27
pabelangerAJaeger: we should use required-projects, then pip install into save virtualenv. You'll then need to setup ANSIBLE_LIBRARY_PATH (I think) to point to the library folder for zuul15:29
AJaegerpabelanger: I see...15:31
*** andreas_s has quit IRC15:31
pabelangerAJaeger: might not need to pip install, we could get away at just pointing to zuul library directory15:31
*** LindaWang has quit IRC15:31
pabelangerAJaeger: like we did for ANSIBLE_ROLES_PATH15:31
fungidirk: yes, <=0.3.x does not require zookeeper, >=0.4.x does: http://lists.openstack.org/pipermail/openstack-infra/2016-December/004972.html15:32
AJaegerpabelanger: copying over what zuul-jobs does15:32
fungier, sorry dirk. dinesh apparently likes to ask questions in channel and then disappear15:32
AJaegerDo we need ANSIBLE_CONFIG and set it to tests/ansible.cfg as well?15:32
fungidhinesh: yes, <=0.3.x does not require zookeeper, >=0.4.x does: http://lists.openstack.org/pipermail/openstack-infra/2016-December/004972.html15:33
fungii guess dhinesh did not leave the channel, i simply cannot type15:33
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: ansible-lint: install zuul  https://review.openstack.org/51574215:34
*** ykarel has quit IRC15:34
fungiAJaeger: any clue if there's supposed to be content at https://docs.openstack.org/pike/deploy/ ?15:35
dhineshfungi: yes is am well and alive :) , i will checkout the link15:35
*** kjackal_ has quit IRC15:35
AJaegerfungi: there was quite some discussion the last hours in the docu channel.15:36
fungiAJaeger: thanks, i'll point the person asking at that discussion15:36
AJaegerwe found some bugs in our scripts that let to endless redirects, those are fixed and now we need to add them back in. So, known issue, needs some more work15:37
*** andreas_s has joined #openstack-infra15:37
*** Goneri has quit IRC15:39
jeblairAJaeger, fungi: can you look at https://review.openstack.org/515732 when you have a moment?15:42
*** andreas_s has quit IRC15:46
fungijeblair: is it at all worth pointing out the reason, being that once you create a new branch from master the branched configuration will apply to that branch, but the master configuration will keep getting applied to your earlier branches instead if you haven't backported? or is that just going to make things more confusing?15:47
*** andreas_s has joined #openstack-infra15:47
fungii'm good with the proposed content as-is, fwiw15:47
jeblairfungi: i'll propose a followup15:48
*** jaosorior has quit IRC15:48
AJaegerjeblair: that is fine! One suggestion inline on what I missed - once zuul has defined that a job is run, later variants cannot change that with files/irrelevant files15:48
jeblairfungi: as i write it, i realize that i was about to repeat the sentence in the middle of teh paragraph.  this could be tricky.  :)15:49
fungijeblair: yeah, i guess that's pretty clear about it already15:50
jeblairfungi: oh, the master configuration *won't* apply to branches if you haven't backported15:50
*** wolverineav has quit IRC15:50
jeblairfungi: (it does now, but won't once the new changes land; unless you set the pragma directive)15:50
fungijeblair: it won't? i thought branches inherited configuration from master when not otherwise supplied. i clearly need to brush back up on ths15:50
fungiaha, i haven't looked at the pragma change yet, sorry15:51
*** wolverineav has joined #openstack-infra15:51
jeblairfungi: yeah that's basically untenable because of the thing AJaeger brings up -- you wouldn't be able to *undo* anything added to the master branch on stable.15:52
*** rhallisey has joined #openstack-infra15:52
jeblairfungi: (eg -- adding a new default service to devstack on master would affect stable if we left things this way)15:52
*** derekh has quit IRC15:52
fungiright. okay15:52
*** andreas_s has quit IRC15:52
fungimakes sense15:52
openstackgerritDoug Hellmann proposed openstack-infra/project-config master: fix release jobs for designate-dashboard and zaqar-ui  https://review.openstack.org/51575215:52
*** andreas_s has joined #openstack-infra15:53
AJaegerjeblair: so, I'm fine with change as is - just wonder whether my comment is worth adding. Could be a followup - then I'll +2A...15:53
jeblairAJaeger: yeah, i'm looking at that now15:54
jlvillalReview request: https://review.openstack.org/#/c/513444/ Add result_TIMED_OUT & result_RETRY_LIMIT as failure colors to Gerrit patch pages15:55
*** Goneri has joined #openstack-infra15:55
*** sree has quit IRC15:55
*** sree has joined #openstack-infra15:55
jeblairjlvillal: question on that15:55
jeblair(inline)15:55
jlvillaljeblair,15:55
AJaegerbbl15:56
jlvillaljeblair, Ah, good question. I'll add them!15:56
*** iyamahat has quit IRC15:56
openstackgerritJohn L. Villalovos proposed openstack-infra/system-config master: Add result_TIMED_OUT & result_RETRY_LIMIT as failure colors  https://review.openstack.org/51344415:57
*** vsaienk0 has quit IRC15:57
jlvillaljeblair, As you can tell, I'm not a CSS expert :)15:57
*** iyamahat has joined #openstack-infra15:59
openstackgerritJames E. Blair proposed openstack-infra/infra-manual master: Zuul v3: clarify stable branches  https://review.openstack.org/51573215:59
jeblairfungi, AJaeger: i tried to address both comments and also found something else i needed to change, so pushed up new rev ^15:59
*** sree has quit IRC16:00
dhineshfungi: the document suggests to have a separate one node setup for zookeeper and then point it to nodepool, am trying to make all my CI components into a single server, any suggestions around it?16:00
smcginnisWe got a new odd failure: http://lists.openstack.org/pipermail/release-job-failures/2017-October/000646.html16:01
smcginnisIs this a zuul issue?16:01
*** yamamoto has joined #openstack-infra16:01
*** andreas_s has quit IRC16:02
*** Apoorva has joined #openstack-infra16:03
fungidhinesh: i don't personally know of any reason why you couldn't run zookeeper on the same server unless resource constraints and scalability become a concern16:03
*** dave-mccowan has joined #openstack-infra16:04
*** andreas_s has joined #openstack-infra16:04
dhineshfungi: ok thanks16:05
jeblairsmcginnis: that's zuul saying that the publish-openstack-releasenotes job can't be modified to add openstack/horizon to it in a project stanza16:06
smcginnisjeblair: Is there something we need to fix in that project's configuration?16:08
jeblairsmcginnis: well, the job has "final: True" set, which means that it can't be modified.16:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: ansible-lint: install zuul  https://review.openstack.org/51574216:10
jeblairsmcginnis: probably because if it were, someone could inherit from it and cause something other than the releasenotes to be published16:10
*** jascott1 has joined #openstack-infra16:10
jeblairsmcginnis: but it also prevents adding new projects to 'required-projects'16:10
dhellmannI wonder why they need horizon in order to build release notes16:11
dhellmannmaybe we need variants for this like for the pypi job template16:11
jeblairdhellmann: yeah, not requiring horizon would be the ideal solution16:12
dhellmannsure, we should start there16:13
smcginnisAnother failure: http://logs.openstack.org/80/802206a2fcbacd79ec3f7ce1fd2856c2d3927bcf/release/publish-openstack-python-tarball/f691362/job-output.txt.gz#_2017-10-27_16_10_08_04549716:13
fungidhellmann: i expect the reason they "require" horizon to build release notes (or sdists/wheels, or documentation) is because their tox configuration attempts to install their project before doing those things and they declare that horizon is a runtime dependency required by all installations16:13
*** lucasagomes is now known as lucas-afk16:13
jeblairbut otherwise, assuming we want to maintain the 'final' setting, we'll actually just need to duplicate the job rather than making a variant. (the new job can use the same playbooks)16:13
jeblairso we'd duplicate https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/jobs.yaml#n21816:13
fungiif we didn't perform those actions under tox or told tox to install a different set of software for those particular task environments, we could likely drop the dependency16:13
*** andreas_s has quit IRC16:14
jeblairit is also possible that 'required-projects' should be considered something that can be overridden in a project-pipeline16:15
jeblairthat would be a change to zuul, but it's an option.  i just tried to be very conservative when i made the split.16:15
dhellmannsmcginnis : that 2nd one is a config issue in the repo: http://logs.openstack.org/80/802206a2fcbacd79ec3f7ce1fd2856c2d3927bcf/release/publish-openstack-python-tarball/f691362/job-output.txt.gz#_2017-10-27_16_09_52_54059616:15
jeblairbasically anything that could potentially change what happens when the job runs is not allowed.16:15
*** andreas_s has joined #openstack-infra16:16
smcginnisThird one - release-openstack-javascript release-openstack-javascript : ERROR No valid playbook found16:16
dhellmannin general for doc building we need to support dependencies for auto-generation of python api docs. That's not really the case for release notes, but that job uses the same requirements file and tox setup.16:16
*** hashar is now known as hasharAway16:18
*** hemna_ has joined #openstack-infra16:18
pabelangerfungi: is that in the venv section of tox.ini?16:18
smcginnisdhellmann: For that puppet-tripleo one, doesn't look like they've ever had a venv tox target?16:18
*** ijw_ has quit IRC16:18
dhellmannmaybe it's running the wrong job16:19
fungipabelanger: well, in options inherited by testenv:venv at any rate16:19
*** robcresswell has joined #openstack-infra16:19
pabelangerdhellmann: smcginnis: yah, puppet jobs for release don't use tox, they have another puppet module build process16:19
*** tosky has joined #openstack-infra16:19
*** iyamahat has quit IRC16:19
fungipabelanger: but as dhellmann points out, a lot of projects do need to be minimally importable or perhaps even runnable to support some specific sphinx extensions which inspect their objects or output16:20
fungijeblair: mentioned that too16:20
jeblairsmcginnis: release-openstack-javascript specifies a playbook (playbooks/javascript/run) that doesn't exist16:20
*** andreas_s has quit IRC16:21
*** e0ne has joined #openstack-infra16:21
smcginnisHmm. Is this something new?16:22
*** catintheroof has quit IRC16:23
*** salv-orlando has joined #openstack-infra16:23
smcginnisjeblair: Looks like that is probably supposed to be playbooks/javascript/tarball.yaml maybe?16:23
*** andreas_s has joined #openstack-infra16:23
smcginnisHmm, nope. That appears to be for16:25
smcginnisrelease-openstack-javascript16:25
smcginnisErr, publish-openstack-javascript-tarball I mean.16:25
*** dave-mccowan has quit IRC16:26
pabelangerhttps://review.openstack.org/502185/16:27
*** catintheroof has joined #openstack-infra16:27
pabelangerthat's the original from mordred16:27
*** rhallisey has quit IRC16:27
pabelangersmcginnis: I think you might be right, playbooks/javascript/tarball.yaml16:27
*** salv-orlando has quit IRC16:27
*** andreas_s has quit IRC16:28
pabelangerthat file should likely be run.yaml16:28
pabelangerdon't see anything else using it16:28
dhellmannso puppet-tripleo is correctly configured to use puppet-release-jobs but that template runs publish-openstack-python-tarball which is the job that failed16:28
smcginnispabelanger: We have two jobs: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/project-templates.yaml#n54716:29
smcginnisJust as an example16:29
dhellmannwhy would the puppet-release-jobs template use a python tarball job?16:29
*** ccamacho has quit IRC16:29
smcginnispabelanger: The tarball one looks like it's correct for publish-openstack-javascript-tarball.16:29
*** dizquierdo has quit IRC16:29
*** andreas_s has joined #openstack-infra16:29
smcginnispabelanger: So I think we may just be missing a run playbook for release-openstack-javascript, though I don't know what it would be doing.16:30
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Fix release-openstack-javascript job  https://review.openstack.org/51577216:30
pabelangersmcginnis: ^should fix the release job16:31
smcginnispabelanger: That's going to break this: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/jobs.yaml#n40116:31
smcginnisRight?16:31
pabelangersmcginnis: no, that is another job in a different repo16:31
pabelangerit has isn't own local playbook16:31
*** jamesden_ has joined #openstack-infra16:31
pabelangerjobs only look in the repo they are defined for playbooks16:32
*** jamesdenton has quit IRC16:32
smcginnispabelanger: Ah, OK. Too many moving parts.16:32
pabelangerso that is http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/javascript/tarball.yaml16:32
*** dizquierdo has joined #openstack-infra16:33
smcginnispabelanger: Then while we wait for that, any ideas for the issue dhellmann pointed out above? puppet job trying to do a python tarball job.16:33
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: ansible-lint: install zuul  https://review.openstack.org/51574216:33
dhellmannthe puppet job template uses a python job to build tarballs for release16:34
*** Goneri has quit IRC16:34
dhellmannsorry, project template16:34
*** kjackal_ has joined #openstack-infra16:35
*** catintheroof has quit IRC16:35
*** ccamacho has joined #openstack-infra16:35
* dhellmann wanders off in search of food16:36
AJaegerjeblair: great. One sentence confused me, -1 for that.16:36
pabelangersmcginnis: dhellmann: yah, looking here in a moment16:36
*** dtantsur is now known as dtantsur|afk16:36
*** catintheroof has joined #openstack-infra16:36
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: Use rdoproject-based settings for containers and dlrn_hash for images  https://review.openstack.org/51466816:38
clarkbcatching up, looks like most things are happy and/or being debugged? I'm going to take a look at e-r and why it still doesn't seem happy this morning after dmsimard's fixes for it merged16:38
*** links has quit IRC16:38
*** catintheroof has quit IRC16:39
smcginnisclarkb: I have more release failures if you want those. :)16:39
smcginnishttp://logs.openstack.org/ef/ef1677a3f4ad19833ffeb9ed652a23a8fcc760c1/release/propose-update-constraints/fd633f3/job-output.txt.gz#_2017-10-27_16_20_10_54148716:39
*** catintheroof has joined #openstack-infra16:39
*** andreas_s has quit IRC16:40
*** ccamacho has left #openstack-infra16:40
*** sambetts is now known as sambetts|afk16:41
pabelangersmcginnis: dhellmann: looks like we need to build release-openstack-puppet jobs, seems we never did that16:41
*** hasharAway has quit IRC16:41
smcginnisOh, that sounds like it could be an issue. :)16:42
pabelangeryah, let me see what that would look like16:42
openstackgerritMerged openstack-infra/project-config master: Remove migrated octavia zuul v3 jobs  https://review.openstack.org/51548616:43
pabelangersmcginnis: k, this is going to take some time. I'll start converting it to zuulv3 format after some food16:45
smcginnispabelanger: ++ Thanks16:45
openstackgerritKaitlin Farr proposed openstack-infra/project-config master: Remove legacy Castellan jobs  https://review.openstack.org/51577416:46
*** rkukura has joined #openstack-infra16:48
*** dave-mccowan has joined #openstack-infra16:50
openstackgerritKaitlin Farr proposed openstack-infra/openstack-zuul-jobs master: Remove legacy Castellan jobs  https://review.openstack.org/51577616:50
*** andreas_s has joined #openstack-infra16:51
openstackgerritKaitlin Farr proposed openstack-infra/project-config master: Remove legacy Castellan jobs  https://review.openstack.org/51577416:53
*** sshnaidm|off has quit IRC16:54
openstackgerritKaitlin Farr proposed openstack-infra/openstack-zuul-jobs master: Remove legacy Castellan jobs  https://review.openstack.org/51577616:54
*** trown is now known as trown|lunch16:54
*** florianf has quit IRC16:54
*** andreas_s has quit IRC16:56
*** electrofelix has quit IRC16:56
smcginnisStill not sure where this "Waiting on logger" thing is coming from.16:57
smcginnishttp://logs.openstack.org/ef/ef1677a3f4ad19833ffeb9ed652a23a8fcc760c1/release/propose-update-constraints/fd633f3/job-output.txt.gz#_2017-10-27_16_20_10_54148716:58
*** andreas_s has joined #openstack-infra16:58
*** mhickey has quit IRC16:59
clarkb#status log Killed elastic recheck static status page update processes from october first to unlock processing lock. Status page updates seem to be processing now.17:00
openstackstatusclarkb: finished logging17:00
clarkb#status log Restarted elasticsearch on elasticsearch07 as it's process had crashed. Log doesn't give many clues as to why. Restarted log workers afterwards.17:00
openstackstatusclarkb: finished logging17:00
*** ralonsoh has quit IRC17:00
clarkbI think with ^ we should start seeing e-r be happier now17:00
*** dave-mccowan has quit IRC17:03
clarkbsmcginnis: thats the process on the remote nodes that streams the finger console data back to you17:03
*** caphrim007 has joined #openstack-infra17:03
*** dave-mccowan has joined #openstack-infra17:03
clarkbmordred: jeblair ^ it looks like the logger process never started (maybe port was already taken?) and as a result it couldn't stop the logger and failed the task?17:03
openstackgerritJames E. Blair proposed openstack-infra/infra-manual master: Zuul v3: clarify stable branches  https://review.openstack.org/51573217:04
jeblairAJaeger, fungi: i chose to resolve that confusion by *removing* a sentence.  we don't have to cover everything right there.  :)17:04
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Ansible linting fixes  https://review.openstack.org/51577817:05
AJaegerjeblair: ;)17:06
*** ykarel has joined #openstack-infra17:06
*** ykarel is now known as ykarel|away17:06
pabelangersmcginnis: I think that is zuul_stream generating that17:06
*** andreas_s has quit IRC17:07
*** dhinesh has quit IRC17:07
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: DNM: testing ansible-lint fixes  https://review.openstack.org/51578117:08
AJaegerpabelanger, jeblair I think 515742 is ready now - and produces some lint issues. I fix some in 51577817:08
*** kjackal_ has quit IRC17:09
fungijeblair: wfm, thanks!17:10
openstackgerritClark Boylan proposed openstack-infra/elastic-recheck master: Balance parens in all fails query  https://review.openstack.org/51578217:11
clarkbdmsimard: ^ fyi17:11
clarkbmtreinish: sdague if you are around review on ^ would be great, I think that may be the last thing to get e-r going properly again17:11
*** andreas_s has joined #openstack-infra17:13
*** yamamoto has quit IRC17:13
openstackgerritPetr Kovar proposed openstack-infra/irc-meetings master: Update docs team mtg time and channel  https://review.openstack.org/51578317:15
openstackgerritMerged openstack-infra/infra-manual master: Zuul v3: clarify stable branches  https://review.openstack.org/51573217:15
jeblairclarkb: do you want to push in the zuul stable branch stack?  starts at https://review.openstack.org/51445917:16
clarkbjeblair: let me look17:17
*** Goneri has joined #openstack-infra17:17
*** inc0 has quit IRC17:17
clarkbalso smcginnis' logger thing is from zuul/ansible/callback/zuul_stream.py where it appears the executor side of anisble is attempting to connect to the test node side logger daemon port17:17
*** rbrndt has quit IRC17:17
*** inc0 has joined #openstack-infra17:17
*** ianychoi_ has quit IRC17:17
openstackgerritPetr Kovar proposed openstack-infra/irc-meetings master: Update docs team mtg time and channel  https://review.openstack.org/51578317:18
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Ansible linting fixes  https://review.openstack.org/51577817:18
*** jpena is now known as jpena|off17:19
jeblairclarkb, smcginnis: inspection of ara or job-output.json may help with that17:19
*** andreas_s has quit IRC17:19
jeblairwell, ara didn't report any problems starting zuul_console.17:20
*** yamamoto has joined #openstack-infra17:21
AJaegerargh, I broke docs publishing ;( On it...17:21
*** aarefiev is now known as aarefiev_afk17:21
*** sshnaidm has joined #openstack-infra17:24
*** kjackal_ has joined #openstack-infra17:24
*** ldnunes has quit IRC17:24
*** Swami has joined #openstack-infra17:29
*** dave-mccowan has quit IRC17:29
*** sshnaidm has quit IRC17:30
*** kjackal_ has quit IRC17:30
clarkbjeblair: one small thing on https://review.openstack.org/#/c/514459/4 inline that may indicate a bug? I +2'd because I think it likely was missed in git add -p or something like that17:30
*** dhinesh has joined #openstack-infra17:31
sdagueclarkb: +A17:31
clarkbsdague: thanks17:31
*** ykarel|away has quit IRC17:32
*** kjackal_ has joined #openstack-infra17:33
*** rkukura has quit IRC17:34
jeblairclarkb: oh that's likely erroneous.  how about i clean it up in a followup?17:35
*** ldnunes has joined #openstack-infra17:36
clarkbjeblair: sure17:38
openstackgerritMerged openstack-infra/elastic-recheck master: Balance parens in all fails query  https://review.openstack.org/51578217:38
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Fix prepare-docs-for-afs  https://review.openstack.org/51579117:39
openstackgerritAndreas Jaeger proposed openstack-infra/project-config master: fix publishing of deploy and install guide  https://review.openstack.org/51579217:40
AJaegerconfig-core, could you review these two changes above, please? I broke *all* docs publishing ;( ^17:40
*** xarses has joined #openstack-infra17:41
*** ldnunes has quit IRC17:41
fungiAJaeger: both changes lgtm17:43
AJaegerthanks, fungi17:43
*** rhallisey has joined #openstack-infra17:45
*** iyamahat has joined #openstack-infra17:45
*** yamamoto has quit IRC17:49
*** kjackal_ has quit IRC17:49
AJaegertests pass on  https://review.openstack.org/515791  and https://review.openstack.org/515792 - anybody for a second review, please?17:52
*** yamahata has quit IRC17:52
*** iyamahat has quit IRC17:52
*** yamamoto has joined #openstack-infra17:53
*** ldnunes has joined #openstack-infra17:54
clarkbjeblair: these are not fast reviews :)17:56
*** yamamoto has quit IRC17:59
*** jbadiapa has quit IRC18:02
AJaegerjeblair: just reviewed the documentation changes on the stack, those are fine ;)18:02
clarkbjeblair: question on https://review.openstack.org/#/c/511352/2118:03
*** rkukura has joined #openstack-infra18:04
*** harlowja has quit IRC18:05
*** harlowja has joined #openstack-infra18:05
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Create new repo cinder tempest plugin  https://review.openstack.org/48630318:13
openstackgerritSean McGinnis proposed openstack-infra/project-config master: Add jobs for cinder-tempest-repo  https://review.openstack.org/51579718:13
jeblairclarkb: replied; let me know if that clarifies or makes it worse :)18:14
*** Apoorva has quit IRC18:15
*** e0ne has quit IRC18:15
openstackgerritPaul Belanger proposed openstack-infra/zuul-jobs master: Create fetch-puppet-module-output role  https://review.openstack.org/51579818:16
clarkbjeblair: ok so parent runs aren't run if you supply your own run?18:16
smcginnisdhellmann: We're still waiting on this one for javascript jobs: https://review.openstack.org/#/c/515772/18:16
*** jascott1 has quit IRC18:16
jeblairclarkb: correct.  strict override.  (and that behavior is unchanged here)18:16
clarkbgot it18:16
*** Apoorva has joined #openstack-infra18:17
AJaegerconfig-core, could you review https://review.openstack.org/#/c/515772/ to help release team, please?18:17
smcginnisjeblair, clarkb: Sorry, I had stepped out. I see your responses above, but did figure out the Waiting on logger stuff?18:17
smcginnisIs it an issue we need to fix yet, or just a transient thing?18:18
smcginnisAJaeger: Thanks!18:18
clarkbsmcginnis: I haven't figured it out but my guess is it is a transient thing18:19
AJaegersmcginnis: note that docs publishing is broken currently - we need https://review.openstack.org/515791 at least (releasenotes work)18:19
jeblairsmcginnis: let's assume transient for now18:19
smcginnispabelanger: Were you still working on release-openstack-puppet jobs? Just want to make sure I didn't miss a patch.18:19
clarkbsmcginnis: baiscally the test node didn't end up listing on the port we expected (so maybe firewall issue or port conflict I dunno)18:19
smcginnisclarkb: OK, good. I was hoping it was.18:19
rm_workhey -- who do you guys work with at OVH for their donated nodes?18:19
pabelangersmcginnis: i am, starting with puppet-branch-tarball job first, almost have that finished. Then we can build on that into a puppet-tarball jobs, then release job18:20
smcginnispabelanger: OK, cool. Didn't want to pressure you, just wasn't sure if I missed anything.18:20
*** fultonj has quit IRC18:21
rm_workwe're trying to track down an issue and we need a really quick data dump from the *host* of a vm... like, `uname -a` and `cat /proc/cpuinfo`; I assume you guys don't have visibility into their hosts, as we just treat them as a generic cloud provider, right? But, maybe there is someone I can talk to?18:21
openstackgerritJeremy Stanley proposed openstack-infra/puppet-subunit2sql master: Install python-netifaces to avoid building  https://review.openstack.org/51580018:21
pabelangersmcginnis: no, happy to help18:21
fungimtreinish: ^ i don't know when that regressed, and i dug around in oslo git histories for a while18:21
fungirm_work: _if_ a job runs in infra-cloud vanilla or chocolate, we've got you covered there18:22
clarkbjeblair: and on https://review.openstack.org/#/c/515249/8/tests/unit/test_v3.py to make sure I understand the intended behavior. Untrusted playbooks still have secrets and we assert that they have access to them, but then we only run the jobs with those secrets in a "safe" post review manner18:22
clarkbjeblair: that is why _checkUntrustedSecrets() shows secrets present, _gate test runs job(s) and _check test does not18:23
rm_workfungi: err, i don't quite follow. as in, you have access to those hosts?18:23
clarkbrm_work: it might help to describe the problem you are trying to address18:23
jeblairclarkb: that one is intended to be a move of an existing test.  i would have done it as its own change, but the old test used an internal interface, and you could not have written the old test in real zuul config yaml (the test used forward-inheritance)18:24
fungirm_work: the infra team operates infra-cloud, so we have access to the hosts running in it. but not for our other providers18:24
rm_workso this is for OVH nodes18:24
rm_workfor example18:24
rm_workhttp://logs.openstack.org/44/515544/2/check/octavia-v1-dsvm-scenario-multinode/235a97c/logs/libvirt/qemu/instance-00000002.txt.gz18:24
*** salv-orlando has joined #openstack-infra18:24
fungirm_work: right, we don't have any access to the host layer at ovh, and i don't think we have any ovh operators present in irc either18:24
rm_workubuntu-xenial-ovh-gra1-000052687718:24
clarkbrm_work: we don't support nested virt18:24
clarkbrm_work: for this reason18:25
fungialso, yes, that18:25
rm_workok, yes, thank you :P18:25
jeblairclarkb: so there shouldn't be any changes there from the old one.18:25
rm_workwe know18:25
rm_workbut we believe it's a kernel bug18:25
rm_workand we'd like to finish filing it as such18:25
jeblairclarkb: but that test runs different jobs in the different pipelines because, yes, the jobs with untrusted secrets can't be run in check.18:25
rm_workbut we need just a couple of things from the underlying host18:25
clarkboh it surely is a kernel bug :)18:25
funginested virtualization acceleration exposes a lot of hardware _and_ kernel bugs18:26
rm_workspecifically `uname -a` and `cat /proc/cpuinfo`18:26
rm_workyeah, so i feel like the asks here aren't huge18:26
clarkbrm_work: so in the past we've basically said we can't file those bugs because we don't have that visibility18:26
rm_workwe just need to find someone we can talk to there18:26
rm_workright, we're not asking you to file it :)18:26
rm_workI'm asking if you have a contact at OVH18:26
rm_workso I can follow-up18:26
clarkbI suppose you have the dpkg listing from the VM so you just need uuids from us?18:27
rm_worki already knew what your response would be (we've done this a number of times)18:27
clarkbrm_work: but ya we've talked to $clouds in the past and sorting this out tends to be far too much work for both sides18:27
rm_workall we need is about 2 minutes of work18:28
rm_workand a pastebin18:28
clarkbso I don't really want to just set you loose on $cloud when we already have decided its not somethign we deal with18:28
*** vsaienk0 has joined #openstack-infra18:28
rm_workthat's why i specifically didn't want to ask you to deal with it18:29
*** salv-orlando has quit IRC18:29
clarkbexcept you'll need instance uuids at the very least I imagine right?18:29
clarkbso that the cloud can determine what hypervisor ran the VM in order to determine kernel and cpu18:29
rm_workhmm, perhaps18:29
rm_workis there a way to get that from logs?18:29
rm_workwe have a host18:29
clarkbyes we can get the data but this is explicilty not supported :)18:30
rm_workbut i assume those are kinda dynamic and unrelated to the scheduling18:30
clarkbfwiw speaking to Intel nova people at the ptg there seemed to be some interest in nova publishing a "this is how you make things actually work with nested virt and intel" document of some sort18:31
clarkbebcause apparently there are lots of tricks intel has sorted out themselves but that isn't necessarily common knowledge where people are running hypervisors18:31
openstackgerritVasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-inspector jobs  https://review.openstack.org/51573718:31
*** rbrndt has joined #openstack-infra18:32
rm_workout of curiosity, would it take longer or shorter than this conversation for you to just help me get the info I need? :P18:32
jeblairovh is a public cloud provider -- if this is something you're tracking down on your own, you can do that without our involvement.18:32
rm_workjeblair: i don't need your direct involvment... i am asking for *contact info*18:32
rm_workfor whoever we deal with on their side18:32
clarkbrm_work: ovh should publish contact info18:32
fungirm_work: "normal" support requests to tend to take weeks for us to hear back from ovh, so asking them for details from their hypervisors may never get any response. aside from trouble-tickets, the only real contact we have is with one of their vice presidents18:33
fungithe one who authorized our pro-bono account18:33
rm_workok, so we don't have any specific contact, like an AM or something18:33
clarkband one of their asks when getting involved with infra donations is that we don't send all these little things in as support requests18:33
* rm_work sighs18:34
jeblairthey give out free trial accounts all the time18:34
*** armaan has quit IRC18:34
*** rtjure has quit IRC18:34
fungirm_work: the contact we have with them is the same one listed at https://www.openstack.org/foundation/companies/profile/ovh-group18:34
jeblairjust sign up for an account, spin up a server, and open a ticket18:34
rm_workalright, thanks18:35
*** rtjure has joined #openstack-infra18:35
mtreinishfungi: I expect it was when we landed this: https://review.openstack.org/#/c/492174/ we didn't touch the min version for a very long time until that landed18:36
AJaegerjeblair, clarkb, could you review https://review.openstack.org/515791 to fix docs publishing again (Sorry, I broke it), please? And then also https://review.openstack.org/515792 (followup for two guides) and https://review.openstack.org/#/c/515772/ (release team help), please?18:36
AJaegerall have one +2 already...18:36
fungimtreinish: thanks, i suspected that. there was also a long-lived oslo.db cap as well18:37
fungiwhich i saw separately in the git log18:37
clarkbAJaeger: commented18:37
fungimtreinish: anyway, that's the latest blocker for deploying the new worker18:38
mtreinishfungi: anyway I +A'd the extra package in the puppet, seemed straightforward enough18:38
fungimtreinish: thanks! i'll try to fire up another once that merges18:38
clarkbjeblair: should I go ahead and approve these changes as I go or do you want them to get in all at relatively the same time?18:38
*** dave-mccowan has joined #openstack-infra18:38
*** rkukura has quit IRC18:39
openstackgerritAndreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Fix prepare-docs-for-afs  https://review.openstack.org/51579118:39
*** rtjure has quit IRC18:39
AJaegerclarkb: indeed! THANKS! updated ^18:39
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Create publish-openstack-puppet-branch-tarball job  https://review.openstack.org/51580218:41
pabelangermnaser: EmilienM: dhellmann: smcginnis: ^ first attempt at puppet-branch-tarball job, we'll be able to build atop of that for release jobs.18:41
smcginnispabelanger: Thanks!18:41
AJaegerfungi, could you review 515791 again, please? I fixed a critical typo that clarkb found18:41
fungiAJaeger: clarkb: ouch, i can't believe i didn't spot that18:42
jeblairclarkb: all at once would be best i think.18:42
clarkbjeblair: ok18:42
*** vsaienk0 has quit IRC18:43
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Create publish-openstack-puppet-branch-tarball job  https://review.openstack.org/51580218:43
AJaegerfungi: I shouldn't have written that ;(18:43
clarkbjeblair: in https://review.openstack.org/#/c/515536/2/tests/fixtures/config/ansible/git/common-config/zuul.yaml some playbooks have .yaml suffixes and some don't18:43
*** rkukura has joined #openstack-infra18:43
clarkbjeblair: is there a right or a wrong way to do that?18:43
*** trown|lunch is now known as trown18:44
clarkbjeblair: for example the base-urls job18:44
*** dave-mccowan has quit IRC18:44
AJaegerclarkb: my understanding reading the docs: All shouldl have .yaml suffixes going forward18:44
*** rtjure has joined #openstack-infra18:45
*** dprince has quit IRC18:48
clarkbjeblair: I guess if everything needs the .yaml that would be a separate change to update the non run playbooks?18:48
*** amoralej is now known as amoralej|off18:48
*** rtjure has quit IRC18:50
pabelangerfungi: clarkb: AJaeger: mind looking at 515802 too, first zuulv3 jobs for puppet-branch-tarballs, we'll build on that for release jobs18:50
fungisure18:50
jeblairclarkb: yep.  i want to deprecate the non-extension form.  with these changes, we support both for a transition period.  when i updated the test fixtures, i went ahead and added .yaml to those to save time later.18:51
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Fix prepare-docs-for-afs  https://review.openstack.org/51579118:51
jeblairclarkb: so yeah, once we remove the non-extension form, we'll need a similar change to update the stuff that was already there.18:51
*** smatzek has quit IRC18:52
fungipabelanger: are the ansible-lint failures on 515802 expected?18:54
funginon-voting, but didn't know what the status is on whether we should be paying attention to those yet18:55
AJaegerclarkb: could you put your eagle eyes on  https://review.openstack.org/515792 as well,  please?18:55
clarkbdmsimard: we seem to be hitting the continue at https://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/elastic_recheck/cmd/uncategorized_fails.py#n365 which is why uncategorized e-r pages are not currently updating18:55
AJaegerfungi: yes, expected - let me give you a few changes...18:55
fungithanks AJaeger!18:55
dmsimardclarkb: I have to do to an appointment now, I won't have the time to check soon :/18:56
AJaegerfungi: https://review.openstack.org/515742 and https://review.openstack.org/515778 are steps to fix them. But more work is needed - that's why we made those non-voting so that we can iterate on them.18:56
AJaegerfungi: we need 515742 in so that the job gives good results - and then we should check that new code does not introduce regressions.18:56
clarkbdmsimard: thats fine, just passing info along18:56
AJaegerhelp on remaining ansible issues is welcome, I've done what I could do easily...18:57
*** csomerville has quit IRC18:57
*** csomerville has joined #openstack-infra18:57
*** dhajare|mtg has quit IRC18:57
AJaegernow time to sign off - have a great weekend!18:57
fungithanks for all your help agani, AJaeger!18:58
fungis/agani/again/18:58
smcginnispabelanger: Was there anything else after that puppet patch needed? Or should we be OK to try once that makes it through?18:58
*** yamamoto has joined #openstack-infra18:59
openstackgerritMerged openstack-infra/project-config master: fix publishing of deploy and install guide  https://review.openstack.org/51579219:00
*** rbrndt has quit IRC19:02
*** rbrndt has joined #openstack-infra19:02
openstackgerritIhar Hrachyshka proposed openstack-infra/irc-meetings master: Moved Neutron upgrades meeting 1h earlier  https://review.openstack.org/51580319:03
clarkbjeblair: ok I have +2'd the entire stack that wasn't already approved19:05
openstackgerritMerged openstack-infra/zuul-jobs master: Create fetch-puppet-module-output role  https://review.openstack.org/51579819:05
clarkbjeblair: I think you can approve them in the sets that you want (maybe all together at once) whenever you are ready19:05
*** yamamoto has quit IRC19:06
pabelangerinfra-root: I have to run a few errands, but will return to continue work on puppet release jobs.19:06
fungithanks pabelanger!19:06
clarkbI'm going to grab lunch now19:08
*** sree has joined #openstack-infra19:11
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add multi-branch support for project-templates  https://review.openstack.org/51580519:12
openstackgerritMerged openstack-infra/project-config master: Create publish-openstack-puppet-branch-tarball job  https://review.openstack.org/51580219:13
jeblairclarkb: cool... i er, have one more change i'd like to get in when you're back from lunch.  :)  ^19:13
*** dprince has joined #openstack-infra19:13
jeblairthat one wasn't quite as urgent as the others, but with it, i think we've got the stable branch story pretty solid.19:14
fungiooh, this is fun... for some reason openstack-infra/puppet-subunit2sql got lumped into the tripleo gate queue19:15
* fungi fixes19:15
*** sree has quit IRC19:16
fungiconfig-core: er, where do we (un)set that?19:16
jeblairfungi: on the "project:" stanza for puppet-subunit2sql in project-config19:17
fungiis it maybe getting set by a project-template then?19:18
jeblairpossibly19:19
fungiand those are in osj, right?19:19
jeblairyeah19:19
jeblairi think that should be an anti-pattern19:20
jeblairputting queues on project-templates19:20
*** andreas_s has joined #openstack-infra19:21
fungihrm, i don't see it in project-templates.yaml19:21
jeblairperhaps it should even be a config error :)19:21
jeblairfungi: zuul.d/zuul-legacy-project-templates.yaml19:21
jeblairfungi: was added in this change: https://review.openstack.org/51214219:21
fungiaha, thanks19:21
fungimnaser: ^ any concerns with me disconnecting the puppet-beaker-jobs and puppet-beaker-jobs-xenial job-templates from the tripleo change queue? they're also used on hundreds of infra puppet modules19:24
*** Goneri has quit IRC19:25
openstackgerritJeremy Stanley proposed openstack-infra/openstack-zuul-jobs master: Disjoin puppet-beaker-jobs-.* from tripleo queue  https://review.openstack.org/51580819:27
fungimnaser: ^19:27
*** smatzek has joined #openstack-infra19:27
*** Apoorva has quit IRC19:28
*** Apoorva has joined #openstack-infra19:29
*** catintheroof has quit IRC19:29
*** e0ne has joined #openstack-infra19:29
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add implied branch matchers on 'master'  https://review.openstack.org/51445919:30
*** e0ne has quit IRC19:30
*** andreas_s has quit IRC19:31
*** andreas_s has joined #openstack-infra19:32
*** Apoorva has quit IRC19:33
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Switch to late-binding inheritance  https://review.openstack.org/51135219:33
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Move test_job_auth_inheritance to test_v3  https://review.openstack.org/51524919:33
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add pragma directive  https://review.openstack.org/51548319:33
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Support file extension in playbook path  https://review.openstack.org/51553519:33
*** tesseract has quit IRC19:33
*** andreas_s has quit IRC19:37
*** dave-mccowan has joined #openstack-infra19:37
*** andreas_s has joined #openstack-infra19:38
*** iyamahat has joined #openstack-infra19:40
*** catintheroof has joined #openstack-infra19:41
*** iyamahat_ has joined #openstack-infra19:42
dhellmannI'm seeing a parse error in a linter job on a project-config change that doesn't seem to be related to the error message. Is that a known issue? http://logs.openstack.org/52/515752/1/infra-check/project-config-zuul-jobs-linters/2116b8f/job-output.txt.gz19:43
*** Apoorva has joined #openstack-infra19:43
dhellmannthe patch is https://review.openstack.org/#/c/515752/119:43
jeblairdhellmann: yep; fix for that is either in progress or landed.  safe to ignore.19:44
dhellmannjeblair : ok, thanks19:45
jeblairdhellmann: in progress, it looks like: https://review.openstack.org/51574219:45
*** catintheroof has quit IRC19:45
*** iyamahat has quit IRC19:45
*** sree has joined #openstack-infra19:46
*** smatzek has quit IRC19:47
*** dave-mccowan has quit IRC19:49
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Increase github delay to 10 seconds  https://review.openstack.org/51581219:49
AJaegerdhellmann: the non-voting job is new - we're stabilizing it right now...19:50
mnaserfungi i dont oppose to it, but i'd like mwhahaha or EmilienM to chime in about removing beaker jobs from tripleo queue.19:51
*** sree has quit IRC19:51
mnaserfrom puppet side of things, its not necessary, but it might affect tripleo more than anything19:52
dhellmannAJaeger : the patch got a -1 because it failed19:53
dhellmannoh, wait, no, it didn't19:53
dhellmannsorry19:53
jeblairmnaser: projects are added to queues rather than jobs, so we should add the projects you intended to that queue instead19:53
jeblairmnaser: eg, all of the tripleo projects themselves, so we don't pull in the infra puppet projects.19:53
*** andreas_s has quit IRC19:54
EmilienMmnaser: beaker jobs? it has nothing to do with tripleo queue19:55
mnaserjeblair https://review.openstack.org/#/c/512142/ - so does that mean patch is technically not correct?  is there somewhere we can add projects to a queue?19:55
*** smatzek has joined #openstack-infra19:55
fungimnaser: in the projects list19:55
mnaseroooh i see what you mean19:56
jeblairmnaser: it's *technically* correct, just probably not the best approach since lots of unrelated projects use those jobs19:56
mnasernot in the job definiton19:56
mnaserok, makes sense19:56
jeblair("the best kind of correct!)19:56
*** andreas_s has joined #openstack-infra19:56
mnaseraha19:56
*** rcernin has joined #openstack-infra19:57
mnaserfungi: +2'd 515808 in that case19:57
smcginnisconfig-core: Could we get a second on https://review.openstack.org/#/c/515752/19:57
* fungi promotes jeblair to grade 37 bureaucrat19:57
mnaserfungi actually19:57
mnaserfungi https://review.openstack.org/#/c/512142/ what do you think of reverting that entire patch19:57
mnaserthose are all generic puppet jobs19:57
jeblairmnaser, EmilienM: we should get a replacement patch that adds projects to the tripleo queue prepared so we don't lose the queue19:58
fungimnaser: i'd be fine with that too. mostly just trying to make sure my puppet-subunit2sql change doesn't take 48 hours to merge19:58
*** sree has joined #openstack-infra19:58
mnaserfungi let me push up a patch to move the projects to the right tripleo queue19:59
mnaserand then we can put the revert on top of it19:59
mnaserdoes that sound ok?19:59
fungihappy to help iterate on a more thorough fix, just trying to get a new subunit2sql worker deployed19:59
fungiand was hoping not to have to set up a shadow module tree to accomplish that20:00
jeblairfungi: there's a quick fix we can do20:00
fungisure, i could promote that change for now20:00
jeblairi'll propose a patch that will let fungi move without disturbing the rest of the tripleo queue until mnaser's patches are ready20:00
*** andreas_s has quit IRC20:01
mnaserjeblair ok cool, mine should be up in a few minutes20:01
*** yamamoto has joined #openstack-infra20:02
*** andreas_s has joined #openstack-infra20:02
*** bobh has joined #openstack-infra20:02
fungijeblair: separate/dropping project-templates? or can the project list override the project-templates for queue?20:02
openstackgerritJames E. Blair proposed openstack-infra/project-config master: Put puppet-subunit2sql into its own queue  https://review.openstack.org/51581620:03
*** sree has quit IRC20:03
jeblairfungi: ^ first one wins20:03
fungithe latter apparently20:03
fungithanks!20:03
*** bobh has quit IRC20:04
jeblairi propose we add the tripleo queue to the projects that need it, then revert 512142, then revert 515816.  should be minimally disruptive for everyone.20:04
fungineat, so you add two project entries to accomplish that. yaml will allow duplicate list items?20:04
*** bobh has joined #openstack-infra20:05
fungiright, i guess duplicate list items is nothing outlandish now that i think about it20:05
jeblairfungi: yeah, that's why zuul's config is a list rather than a dict.20:05
fungiassociative array keys would have been a problem, yeah20:05
clarkbjeblair: looks like you WIP'd that last zuul change?20:05
jeblairclarkb: yeah, i realized i missed something (details in #zuul)20:05
*** smatzek has quit IRC20:05
jeblairclarkb: i'm about halfway through addressing that20:06
*** mat128 has quit IRC20:06
openstackgerritMerged openstack-infra/project-config master: fix release jobs for designate-dashboard and zaqar-ui  https://review.openstack.org/51575220:07
*** yamamoto has quit IRC20:08
clarkbok20:08
mnaserjeblair: what if i changed/edited the tripleo job project templates to run in the tripleo queue, would that be a good approach?20:08
clarkbpabelanger: fungi ianw any reason to not use presentty for the project update presentation in sydney?20:09
mnaserconsidering we won't weirdly list tripleo queue in the definition of a puppet job and the tripleo jobs control their queue?20:09
mnasers/puppet job/puppet project/20:09
*** bobh has quit IRC20:09
fungiclarkb: since you're starting from the deck i used in boston and it's already presentty, seems reasonable to me20:10
*** rbrndt has quit IRC20:10
fungialso it helps show off that we're console nuts20:10
*** rbrndt has joined #openstack-infra20:10
fungimight tempt a few new recruits just from that ;)20:11
clarkbfungi: well there are two presentations, the onboarding one I'm definitely just gonna reuse your boston deck from and then the project update deck where we get to talk about zuul and stuff20:11
*** ijw has joined #openstack-infra20:11
clarkbbut probably easiest to just use a single tool for both cases20:11
fungiclarkb: oh, right. color me confused. but yes, sounds swell to me20:12
jeblairmnaser: i'm sort of inclined to want to have queue membership be explicit per-project in project-config, rather than something that you get automatically when you add a specific project-template.20:13
mnaserjeblair: okay, i have no problems in either or, just thought i'd hear feedback20:14
jeblairmnaser: mostly just from a perspective of not wanting surprises :).  though it is more work.20:14
*** ldnunes has quit IRC20:14
jeblairmnaser: i'm not firmly opposed or anything.  this is definitely a situation where we are figuring out the best way to use the new tool.  :)20:14
*** andreas_s has quit IRC20:15
*** ijw has quit IRC20:16
*** andreas_s has joined #openstack-infra20:17
*** wolverineav has quit IRC20:18
*** wolverineav has joined #openstack-infra20:18
*** hongbin has quit IRC20:20
*** hashar has joined #openstack-infra20:22
clarkbfungi: https://review.openstack.org/#/c/515853/ fyi, also seems to work with whatever tox installs20:23
*** salv-orlando has joined #openstack-infra20:25
*** yamahata has joined #openstack-infra20:25
fungimnaser: for what it's worth, it definitely took a little time to track down what was causing the infra puppet modules to end up in the tripleo change queue, since it was applied indirectly through project-templates... but also avoiding doing it anywhere besides the projects list helps avoid confusion if you end up applying two project-templates which each declare a different change queue too20:25
*** hongbin has joined #openstack-infra20:27
*** dprince has quit IRC20:27
*** salv-orlando has quit IRC20:29
*** andreas_s has quit IRC20:31
*** cody-somerville has joined #openstack-infra20:31
*** jascott1 has joined #openstack-infra20:32
*** andreas_s has joined #openstack-infra20:32
*** dave-mccowan has joined #openstack-infra20:33
openstackgerritMohammed Naser proposed openstack-infra/project-config master: Refactor TripleO jobs to use project-template for scenario001  https://review.openstack.org/51588220:33
mnaserstep #1, cleanup ^20:33
*** csomerville has quit IRC20:34
*** kjackal_ has joined #openstack-infra20:34
*** rhallisey has quit IRC20:35
*** ijw has joined #openstack-infra20:35
*** kgiusti has left #openstack-infra20:35
*** salv-orlando has joined #openstack-infra20:35
*** ijw has joined #openstack-infra20:36
*** mdbooth has quit IRC20:37
*** ari[m] has quit IRC20:37
*** andreas_s has quit IRC20:37
*** dave-mccowan has quit IRC20:38
*** thorst has quit IRC20:38
*** andreas_s has joined #openstack-infra20:38
zxiiroDoes anyone know how to check documentation for a released version of something? For example I want to see JJB 1.6.2 documentation specificilly.20:39
zxiirothe default url always gives you the latest master branch20:39
openstackgerritMohammed Naser proposed openstack-infra/project-config master: Add modules tested by TripleO to their queue  https://review.openstack.org/51590320:39
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Add multi-branch support for project-templates  https://review.openstack.org/51580520:41
openstackgerritMohammed Naser proposed openstack-infra/openstack-zuul-jobs master: Revert "move puppet jobs to tripleo queue"  https://review.openstack.org/51591820:42
*** andreas_s has quit IRC20:44
*** ari[m] has joined #openstack-infra20:44
jeblairzxiiro: there's https://docs.openstack.org/infra/jenkins-job-builder/1.6/20:44
*** mdbooth has joined #openstack-infra20:44
*** yamahata has quit IRC20:44
*** andreas_s has joined #openstack-infra20:44
jeblairzxiiro: i think that's based on branches, not releases though20:45
jeblairwhich is why it's 1.6 and not 1.6.220:45
*** iyamahat_ has quit IRC20:45
jeblairzxiiro: we might be missing the release doc jobs on jjb.20:46
fungii was still not entirely convinced by the docs team's arguments that nobody wants to look at tag-specific versions of documentation (it only really makes sense in the openstack context where clients/libraries all get stable branches)20:46
mnaserjeblair fungi https://review.openstack.org/#/q/topic:tripleo-queue20:47
zxiirojeblair: ah great, that's actually all i need20:47
fungijeblair: unless we undid it, all projects stopped running tag-specific docs publication because the jobs were simplified to no longer support it20:48
*** felipemonteiro__ has quit IRC20:48
jeblairfungi: even library projects?20:49
fungijeblair: yeah, because "proper" (read "openstack") libraries use stable branches20:49
*** andreas_s has quit IRC20:50
*** bocon has joined #openstack-infra20:50
*** andreas_s has joined #openstack-infra20:51
mnaserwould infra be able to give us the ability to force merge a set of patches?  for puppet build jobs to work, we needed to add "puppet" to our bindep.txt across all branches  .. https://review.openstack.org/#/q/topic:bindep/puppet .. this is going to waste a lot of CI resources, could we have access to force merge it for now (or have infra do it for us?) to avoid the infra churn20:51
fungii've been meaning to come up with new jobs we can use in tag-based pipelines for branchless projects people consume releases of, just haven't gotten around to it20:51
jeblairfungi: yeah, i think that will be appropriate for some projects20:52
*** jgr has quit IRC20:53
jeblairmnaser: that's a lot of changes.  :)20:53
mnaserjeblair yeah... well its a change for every puppet openstack repo across all branches :(20:53
*** lindycoder has quit IRC20:53
*** dchen has quit IRC20:53
jeblairwhat is that, like 125 changes?20:53
mnaseri dont mind having to do all the clicking if i can be given the ability to do that for puppet projects and do it for those only20:53
*** jgr has joined #openstack-infra20:54
*** rlandy has quit IRC20:54
fungimnaser: i guess what you're asking for is the ability to push changes directly into the git repositories so that they don't run any check/gate jobs at all?20:54
fungirather than proposing them for review first?20:54
mnaser33 changes per branch, master + 3 stable => 13220:54
mnaserwell, either that, or if we can force merge those changes20:54
jeblairfungi: they're already up for review, so i think ^ that20:55
fungiahh20:55
fungii didn't realize they'd already been proposed to gerrit20:55
mnaserits ok, EmilienM is already gone to "somewhere with no ip addresses" so if the solution be easier not using the gerrit changes20:55
mnaserwe wont tell him20:55
mnaser:P20:55
EmilienMwat20:56
EmilienMno i'm still here20:56
EmilienM++ for force push on these patches20:56
mnaser:P20:56
EmilienMif we can save resources here20:57
mnaserand a lot of time, given tripleo queue is already so backed up and these jobs can take up to 1h-ish to run each20:57
jeblairwhat change prompted this?20:57
mnaserjeblair https://review.openstack.org/#/c/515802/20:58
mnaserwe dont have puppet in our bindep so that job would fail for puppet modules20:58
*** Apoorva_ has joined #openstack-infra20:59
jeblairan alternative would be to install puppet in that playbook, then let all those bindep changes work their way in slowly, then remove it from the playbook.20:59
mnaseri was thinking of creating something like an ensure-puppet type of role21:00
jeblairyeah, though having it in bindep sounds like a good idea regardless, right?21:00
mnaserbut i feel like if we start using too much of those ensure-foo roles then whats the point of bindep.. so yeah its better to rely on bindep for that21:00
*** andreas_s has quit IRC21:01
*** hemna_ has quit IRC21:01
mnaserok, should i add a patch to install puppet in pre_tasks and we can let those changes make their way slowly but surely..21:01
mnaserand maybe with a note to drop it once everything in that topic merges21:02
*** Apoorva has quit IRC21:02
mnaser(this solves the issue, but still leaves a ton of wasted resources :<)21:02
*** andreas_s has joined #openstack-infra21:02
*** camunoz has quit IRC21:04
*** yamamoto has joined #openstack-infra21:05
fungithe human effort wasted in cleanup if something goes wrong is a lot more valuable than a few hundred server-hours spread out over some period where the tripleo queue has reached greater stability21:05
fungi(obviously dumping those into the gate right now would not be an excellent choice)21:05
jeblairmnaser: that's the slowest and safest route.  while i can't see how force-merging these changes would cause a problem, i'm surprised every time we do it.  if other infra-root and puppet-core folks would prefer to force-merge, i won't object, but it's not my first choice.21:06
*** trown is now known as trown|outtypewww21:06
*** iyamahat has joined #openstack-infra21:06
mnaserhere's where i'm at: its friday evening, weekend is ahead of us, we'll churn in as much as we can get21:06
mnaserand see where we stand on monday21:07
mnaserif we got 5 patches merged, let's re-evaluate... if there's only 5 left, then not an issue21:07
*** bocon has quit IRC21:07
jeblairsounds good21:07
*** smatzek has joined #openstack-infra21:07
mnaserand if i can have eyes on this before the weekend so it helps some of the stuff that is not in the tripleo queue: https://review.openstack.org/#/q/topic:tripleo-queue !21:07
mnaserill focus on getting those in first21:08
*** andreas_s has quit IRC21:08
*** andreas_s has joined #openstack-infra21:09
*** yamamoto has quit IRC21:10
jeblairmnaser: regarding 515903 -- what about puppet-nova?21:10
EmilienMI'll be 100% offline this weekend, just fyi21:12
*** marst has quit IRC21:15
*** andreas_s has quit IRC21:18
mnaserjeblair i was looking for jobs which ran scenario in gate, puppet-nova only runs tripleo-ci in `check-tripleo` pipeline.. which doesnt seem to have any queues defined for it21:18
*** andreas_s has joined #openstack-infra21:19
*** dizquierdo has quit IRC21:23
*** leakypipes has quit IRC21:23
*** andreas_s has quit IRC21:24
*** yamahata has joined #openstack-infra21:25
jeblairif there are any infra-core folks who could review 515805 i'd appreciate it.21:27
*** rcernin has quit IRC21:28
*** andreas_s has joined #openstack-infra21:29
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Update test fixtures to use explicit run  https://review.openstack.org/51553621:30
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Remove implied run  https://review.openstack.org/51548721:30
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Validate that a job has a run playbook on freeze  https://review.openstack.org/51553721:30
jeblair515805 is the last thing holding up the stable branch fix restart21:30
* clarkb looks21:31
*** sree has joined #openstack-infra21:32
*** dingyichen has joined #openstack-infra21:33
*** esberglu has quit IRC21:34
*** sree has quit IRC21:37
*** andreas_s has quit IRC21:39
*** hashar has quit IRC21:39
*** jamesden_ is now known as jamesdenton21:41
*** andreas_s has joined #openstack-infra21:41
clarkbjeblair: lgtm21:42
*** baoli has quit IRC21:42
*** salv-orlando has quit IRC21:50
*** wolverineav has quit IRC21:50
*** salv-orlando has joined #openstack-infra21:50
*** wolverineav has joined #openstack-infra21:50
*** wolverineav has quit IRC21:54
*** salv-orlando has quit IRC21:55
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364522:00
*** andreas_s has quit IRC22:01
openstackgerritOpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements  https://review.openstack.org/50364522:01
*** salv-orlando has joined #openstack-infra22:03
*** ijw has quit IRC22:05
*** ijw has joined #openstack-infra22:05
*** yamamoto has joined #openstack-infra22:06
*** andreas_s has joined #openstack-infra22:06
*** smatzek has quit IRC22:09
*** smatzek has joined #openstack-infra22:10
*** smatzek has quit IRC22:11
*** yamamoto has quit IRC22:11
*** smatzek has joined #openstack-infra22:11
*** iyamahat_ has joined #openstack-infra22:13
*** jtomasek has quit IRC22:14
*** dave-mccowan has joined #openstack-infra22:14
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Switch puppet-release-jobs to use puppet-branch-tarball job  https://review.openstack.org/51595722:15
*** iyamahat has quit IRC22:15
clarkbjeblair: I'll be around for the afternoon and early evening if you wnt to give it a go22:15
pabelangerclarkb: fungi: mnaser: jeblair: smcginnis: ^ should start to let puppet jobs publish branch tarballs correctly.22:16
*** smatzek has quit IRC22:16
clarkbpabelanger: I'll add it to the review list22:16
clarkbwant to get a change up against publications for the project update first though22:16
*** andreas_s has quit IRC22:16
pabelangersure22:17
pabelangerclarkb: also, presentty wfm22:17
mnaserhttps://review.openstack.org/#/q/topic:tripleo-queue <-- this would help us move some jobs out of puppet queue22:17
*** andreas_s has joined #openstack-infra22:17
mnaserto get the bindep changes to land easier22:18
mnaserpabelanger is "publish-openstack-python-branch-tarball" in post for puppet jobs wrong in that case?22:19
pabelangerlooking22:19
pabelangermnaser: it is, but we need to write the release job still22:19
*** sree has joined #openstack-infra22:19
pabelangerthat is what I am working22:19
pabelangerbut first want to make sure branch-tarballs are working, since we'll depend on that22:19
mnaserpabelanger ok, because both "puppet-release-jobs" template is defined and "publish-openstack-python-branch-tarball" is manually listed in post for puppet projects, so ill drop "publish-openstack-python-branch-tarball" from the puppet projects.yaml22:20
mnaserbecause i dont think they should be there22:20
mnaserpuppet-release-jobs template being there will mean that there is no functional change22:20
*** iyamahat__ has joined #openstack-infra22:21
pabelangermnaser: right, you'll have to remove the manualy defined post pipelines too22:21
pabelangerwhere is that?22:21
mnaserpabelanger im seeing it for a few puppet modules in projects.yaml in project-config (removing them now)22:21
*** iyamahat__ has quit IRC22:21
*** iyamahat has joined #openstack-infra22:22
*** andreas_s has quit IRC22:22
*** sree has quit IRC22:23
*** thorst has joined #openstack-infra22:24
*** dave-mccowan has quit IRC22:24
*** panda|ruck is now known as panda|ruck|off22:24
pabelangermnaser: yah, we should have coverage in puppet-release-jobs tempalte22:24
pabelangertemplate*22:24
*** iyamahat_ has quit IRC22:24
openstackgerritMerged openstack-infra/project-config master: Refactor TripleO jobs to use project-template for scenario001  https://review.openstack.org/51588222:25
openstackgerritMohammed Naser proposed openstack-infra/project-config master: Drop publish-openstack-python-branch-tarball from Puppet post  https://review.openstack.org/51595922:25
openstackgerritMerged openstack-infra/project-config master: Add modules tested by TripleO to their queue  https://review.openstack.org/51590322:26
mnaserpabelanger btw, does this mean that we should technically drop "puppet-openstack-module-build" eventually?22:26
*** salv-orlando has quit IRC22:27
mnaserand use publish-openstack-puppet-branch-tarball ?22:27
pabelangerno, we'll create a new job and add it to ozj, we might be able to result job name22:27
pabelangerworking on that after release jobs22:27
mnaserok cool, makes sense22:28
*** andreas_s has joined #openstack-infra22:28
*** salv-orlando has joined #openstack-infra22:28
*** thorst has quit IRC22:29
jeblairclarkb, fungi, mnaser: how does this look? https://etherpad.openstack.org/p/EODwIgRLgk22:30
*** iyamahat_ has joined #openstack-infra22:30
mnaserjeblair lgtm, makes perfect sense, maybe add [stable] tag?22:31
clarkbya lgtm22:31
jeblaircool, will do.  i'll send that as soon as we restart22:32
jeblairwhich is not right now because the change bounced off the gate after getting an infra-cloud node.22:32
*** iyamahat__ has joined #openstack-infra22:33
mnaser:p22:33
*** iyamahat has quit IRC22:33
*** salv-orlando has quit IRC22:35
*** salv-orlando has joined #openstack-infra22:36
*** iyamahat_ has quit IRC22:37
*** sshnaidm has joined #openstack-infra22:38
*** iyamahat__ has quit IRC22:38
*** andreas_s has quit IRC22:38
*** salv-orlando has quit IRC22:40
*** jcoufal has quit IRC22:44
*** wolverineav has joined #openstack-infra22:46
fungisorry, entered an hour-long food prep and consumption tunnel without warning. back and catching up now22:48
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Revert "move puppet jobs to tripleo queue"  https://review.openstack.org/51591822:48
*** ijw has quit IRC22:48
*** ijw has joined #openstack-infra22:49
*** sdague has quit IRC22:50
openstackgerritMerged openstack-infra/project-config master: ansible-lint: install zuul  https://review.openstack.org/51574222:51
clarkbfungi: pabelanger ianw https://review.openstack.org/51595022:53
clarkbthat is a first draft for the update presentation22:53
*** andreas_s has joined #openstack-infra22:54
clarkbjeblair: what would be some good zuulv4 features to call out on the zuul v4 future?22:54
*** rbrndt has quit IRC22:55
clarkbpabelanger: your change doesn't update pre-release and release, just post22:55
clarkbpabelanger: is that intentional?22:55
clarkb*your puppet change22:55
pabelangerclarkb: it is, I am writing pre-release / release job now22:57
pabelangerbut wanted to first make sure post worked since it builds atop of it22:57
clarkbgot it22:57
clarkblooks like fungi just approved22:57
*** andreas_s has quit IRC22:58
pabelangergreat22:58
fungijeblair: draft announcement lgtm, thanks!23:00
*** tosky has quit IRC23:00
pabelangerclarkb: fungi: we also can land https://review.openstack.org/515959/ to, will remove duplicate jobs from puppet23:01
fungiclarkb: yep, sorry didn't realize you were reviewing it as i was going through things i found in scrollback chronologically23:01
fungiand i had a fair bit of context on that one owing to discussions in the -release channel23:01
fungiplus, was basically a one-liner23:02
openstackgerritMerged openstack-infra/openstack-zuul-jobs master: Switch puppet-release-jobs to use puppet-branch-tarball job  https://review.openstack.org/51595723:02
clarkbpabelanger: done23:03
clarkbfungi: np23:03
fungiclarkb: maybe we don't want to make references to "zuul v4" per se, and freak everyone out. better to perhaps talk about "post-v3 improvements"?23:03
clarkbhahaha23:03
clarkbv3.1 ?23:03
pabelangerthanks23:03
jeblairclarkb: well, actually i have a whole list of things that still need to go into v3 before it's done... lemme find it23:07
*** yamamoto has joined #openstack-infra23:07
*** xarses has quit IRC23:07
clarkbjeblair: that probably works better thanks23:08
jeblairclarkb: http://paste.openstack.org/show/624856/23:08
fungithat might be better to talk about... the road to the 3.0.0 release23:08
jeblairclarkb: my plan for early next week is to polish that up and send it out as a proposed update to our roadmap23:08
fungiwe're still very much beta-testing the future v323:08
jeblairthat's from the brainstorming etherpad at the ptg23:08
*** andreas_s has joined #openstack-infra23:08
fungior at least gamma-testing release candidates23:09
clarkbjeblair: perfect thanks23:09
openstackgerritMerged openstack-infra/project-config master: Drop publish-openstack-python-branch-tarball from Puppet post  https://review.openstack.org/51595923:09
openstackgerritMerged openstack-infra/zuul feature/zuulv3: Add multi-branch support for project-templates  https://review.openstack.org/51580523:09
jeblair\o/ back just in time :)23:09
*** ansiwen[q] has quit IRC23:10
*** yamahata has quit IRC23:10
*** yamamoto has quit IRC23:12
pabelangerokay, publish-openstack-puppet-branch-tarball worked: http://tarballs.openstack.org/puppet-qdr/23:14
pabelangernow for release job23:14
*** ansiwen[q] has joined #openstack-infra23:15
*** andreas_s has quit IRC23:17
jeblairzuul has been upgraded on all the hosts23:18
jeblairi'm going to install the repl on zuulv3 though real quick23:18
*** andreas_s has joined #openstack-infra23:19
jeblairi've saved queues and am going to restart zuulv3 scheduler and all executors now.  it will be down for a few minutes for this.23:19
clarkbfungi: jeblair pabelanger ianw ok new patchset to infra update deck with path to zuul v3 release23:20
clarkbI grabbed an item out of each of jeblair's * term sections in the paste23:20
jeblairclarkb: link to change?23:21
*** hongbin has quit IRC23:21
clarkbjeblair: https://review.openstack.org/#/c/51595023:21
*** bauwser is now known as bauzas23:21
*** ansiwen[q] has quit IRC23:22
*** andreas_s has quit IRC23:24
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Creaate release-openstack-puppet job  https://review.openstack.org/51596823:26
pabelangerjeblair: fungi: clarkb: mnaser: smcginnis: dhellmann: ^ should be our brand new zuulv3 release jobs for puppet modules23:27
*** thorst has joined #openstack-infra23:27
pabelangeractually, 1 sec. I think I see an issue23:27
openstackgerritPaul Belanger proposed openstack-infra/project-config master: Creaate release-openstack-puppet job  https://review.openstack.org/51596823:28
pabelangerthere, with signing goodness23:28
*** owalsh is now known as owalsh_pto23:29
jeblairstill waiting on a few executors to finish cleaning up23:29
*** andreas_s has joined #openstack-infra23:30
clarkbjeblair: are you restarting the scheduler too?23:30
jeblairclarkb: yes23:30
jeblairthere were changes to both23:30
clarkbcool I'll check on my cgit thing afterweards23:30
pabelangeri admit, now that we have our base jobs setup, it is getting faster to write new publishing jobs23:30
mnaserpabelanger shouldnt that job have something which does the actual publish23:31
mnaseror the post of the parent handles that?23:31
pabelangerright, parent does that for us23:32
mnaserok cool23:32
jeblairinteresting... there are a lot of ssh processes sticking around now23:35
jeblairthey are controlpersist servers, even though the timeout for that is 60s23:35
fungipabelanger: was that renaming find copied from the v2 job, by any chance?23:36
pabelangerfungi: yes23:36
clarkbjeblair: last time I did restarts it was the ssh-agents not the sshclients23:36
pabelangerfungi: crazy right?23:36
clarkband typically no more than a handful23:36
fungipabelanger: okay, i'll approve then. seems to have been written by someone who was either unaware of the rename command or afraid we'd try to port it to a platform where it was lacking23:37
jeblairselect(10, [3 5], [], NULL, NULL23:37
jeblaira select with no timeout? :/23:37
jeblairwe may need to try to keep track of those and clean them up ourselves :/23:38
jeblairSpamapS: ^ this is the ssh processes that implements the controlpersist server23:38
*** thorst has quit IRC23:38
jeblairclarkb: we actually didn't leake a single ssh-agent this time23:38
*** andreas_s has quit IRC23:38
SpamapSjeblair: gross23:39
fungiyeah, when i was doing executor restarts these last couple of times, the stop left behind 50-100 ssh processes23:39
jeblairansible 'ze*' -m shell -a 'pkill -u zuul ssh'23:40
SpamapSI guess they figured the select would die when the fifo was closed on the other side?23:40
fungithat's basically what i was doing (though server by server as i was performing rolling restarts)23:40
SpamapSit should result in an error23:40
jeblairSpamapS: yeah.  we have a way of finding network edge cases don't we? :)23:41
SpamapSoh wait no these are just on disk sockets23:41
jeblairssh     8677 zuul    3u  IPv4         2016013959      0t0        TCP ze08.openstack.org:42430->104.239.146.107:ssh (ESTABLISHED)23:41
jeblairssh     8677 zuul    5u  unix 0xffff88000ff61000      0t0 2016014183 /var/lib/zuul/builds/383ff3e5a94b464991577d84635ac569/.ansible/cp/df1b1e19ba.FOTfgbjxywzxIW37 type=STREAM23:41
jeblairSpamapS: ^ one of each it was waiting on23:41
jeblairokay, i have cleaned up everything, starting executors now23:42
jeblairstarting scheduler23:42
*** andreas_s has joined #openstack-infra23:44
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Switch to release-openstack-puppet jobs for release-puppet-jobs  https://review.openstack.org/51597023:45
pabelangermnaser: fungi: clarkb: dhellmann: smcginnis: ^and that brings the new release jobs for puppet modules online23:45
jeblairoh, hrm. the initial config loading now has to get all the branches before it does the cat jobs... that's annoying, but workable.  but if it does that on *reconfiguration* that's unworkable.23:46
openstackgerritPaul Belanger proposed openstack-infra/openstack-zuul-jobs master: Switch to release-openstack-puppet jobs for release-puppet-jobs  https://review.openstack.org/51597023:46
*** Swami has quit IRC23:46
jeblairi'll look into that.23:46
*** thorst has joined #openstack-infra23:49
*** caphrim007 has quit IRC23:49
*** thorst has quit IRC23:49
*** andreas_s has quit IRC23:49
clarkbyou might need a different type of merge job to really quuckly list the branches?23:49
*** oanson has quit IRC23:54
*** andreas_s has joined #openstack-infra23:54
*** oanson has joined #openstack-infra23:55
jeblairyeah, that's an idea23:56
*** ianychoi has joined #openstack-infra23:56
jeblairthis doesn't happen on dynamic reconfiguration.  that's good, we should be able to press forward.23:57
jeblairit looks like it added about 1 minute to the full reconfiguration time (went from 3m to 4m)23:57
jeblairhowever, i think it's going to happen on tenant reconfiguration too (whenever a change lands)23:58
jeblairthat was running at about 1m23:58
jeblairso it could end up doubling it23:58
jeblairthat's a moderate problem23:58
jeblairi think maybe we should find a way to use cached data for tenant reconfig, but not on full reconfig23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!