| ianw | yeah, the test maybe has more problems than just that | 00:00 |
|---|---|---|
| ianw | http://logs.openstack.org/58/545158/2/experimental/nodepool-zuul-functional/472195d/job-output.txt.gz | 00:00 |
| ianw | Submited node request <NodeRequest 200-0000000000 <NodeSet OrderedDict([(('c', 'o', 'n', 't', 'r', 'o', 'l', 'l', 'e', 'r'), | 00:00 |
| mordred | ianw, pabelanger: feel like +A'ing https://review.openstack.org/#/c/545139/ (follow up to the puppet-zuul patch from earlier) | 00:00 |
| mordred | ianw: :) yes- that does seem like more problems than just bindep | 00:01 |
| ianw | i've stuffed that up in the config before where you forget the "-" in the yaml | 00:02 |
| *** VW has joined #openstack-infra | 00:04 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:05 |
| *** salv-orlando has quit IRC | 00:06 | |
| *** salv-orlando has joined #openstack-infra | 00:07 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added logic to manage calendar sync error due credentials revocation. https://review.openstack.org/545166 | 00:08 |
| *** claudiub has quit IRC | 00:09 | |
| *** abelur has joined #openstack-infra | 00:10 | |
| *** salv-orlando has quit IRC | 00:12 | |
| openstackgerrit | Merged openstack-infra/system-config master: Set zuul openstack tenant name explicitly https://review.openstack.org/545139 | 00:17 |
| *** abelur has quit IRC | 00:19 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: handle missing lanes arg when creating a board https://review.openstack.org/545168 | 00:19 |
| *** abelur has joined #openstack-infra | 00:19 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: handle missing filters arg when creating a worklist https://review.openstack.org/545169 | 00:19 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: use required enums to validate 'type' args https://review.openstack.org/545170 | 00:19 |
| *** Thaelim has joined #openstack-infra | 00:22 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:22 |
| *** abelur has quit IRC | 00:25 | |
| *** abelur has joined #openstack-infra | 00:25 | |
| *** Thaelimm has joined #openstack-infra | 00:31 | |
| *** Thaelimm has quit IRC | 00:32 | |
| *** Thaelim has quit IRC | 00:32 | |
| *** Thaelimm has joined #openstack-infra | 00:32 | |
| *** ijw has quit IRC | 00:32 | |
| pabelanger | clarkb: ah, yes. I should have done that | 00:36 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:38 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 00:39 |
| *** gyee has quit IRC | 00:41 | |
| *** andreas_s has joined #openstack-infra | 00:45 | |
| *** bobh has joined #openstack-infra | 00:48 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 00:49 |
| *** andreas_s has quit IRC | 00:49 | |
| *** bobh has quit IRC | 00:52 | |
| *** SumitNaiksatam has quit IRC | 00:53 | |
| *** persia has joined #openstack-infra | 00:54 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 01:00 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 01:00 |
| *** hamzy has joined #openstack-infra | 01:02 | |
| *** felipemonteiro_ has quit IRC | 01:03 | |
| *** rlandy|rover|bbl is now known as rlandy|rover | 01:07 | |
| *** salv-orlando has joined #openstack-infra | 01:08 | |
| *** salv-orlando has quit IRC | 01:12 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:12 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:14 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 01:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add opensuse-tumbleweed to nodepool dsvm testing https://review.openstack.org/545183 | 01:18 |
| pabelanger | lets see if tumbleweed DIBs work | 01:18 |
| clarkb | greghaynes: ^ | 01:19 |
| clarkb | pabelanger: I think greghaynes was finding that glean may not work properly but we'll find out soon enough | 01:19 |
| greghaynes | Haha, I just made one today and yes glean thinks it's gentoo, but otherwise it seemed to work | 01:19 |
| *** jlabarre has quit IRC | 01:19 | |
| pabelanger | clarkb: ah, cool. Good to know, we have some patchs up to enable tumbleweed in nodepool | 01:19 |
| EmilienM | gerrit is super slow for me but it's probably just me again | 01:21 |
| persia | EmilienM: It's slow enough to not render new changes for me, so I think it's not just you. I was just checking mulitple devices and browsers. | 01:22 |
| persia | Now it is just super slow, so things are better. No idea why it was nearly blocked. | 01:24 |
| EmilienM | I know there is the re-index thing | 01:26 |
| EmilienM | but tbh I never remember when it is | 01:26 |
| *** xarses_ has joined #openstack-infra | 01:26 | |
| clarkb | EmilienM: we do a database backup at 0000UTC daily | 01:26 |
| clarkb | but this appears to be garbage collector related | 01:26 |
| EmilienM | we should write an IRC bot that notify here when the cron starts :D | 01:26 |
| EmilienM | ok | 01:26 |
| clarkb | EmilienM: the db backup no longer cuases a downtime either | 01:27 |
| EmilienM | good!! | 01:27 |
| pabelanger | exit | 01:27 |
| pabelanger | doh | 01:27 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:28 |
| clarkb | though looking at melody the last appreciable gc'ing happened right before the db backups | 01:28 |
| clarkb | about an hour and 35 minutes ago | 01:28 |
| pabelanger | clarkb: was that an SQL change that helped backups improve? | 01:28 |
| clarkb | it was a mysqldump change | 01:29 |
| clarkb | monty had a lightbulb moment and we were able to update the mysql dump to take advantage of innodb things or something | 01:29 |
| *** ijw has joined #openstack-infra | 01:29 | |
| *** rlandy|rover is now known as rlandy|rover|bbl | 01:31 | |
| *** Thaelimm has quit IRC | 01:31 | |
| clarkb | there is/was a spike in active threads | 01:34 |
| clarkb | thats likely related to the slowness | 01:34 |
| clarkb | however http requests per minute is down | 01:34 |
| clarkb | and cpu usage isn't crazy | 01:35 |
| clarkb | could be ssh related maybe? | 01:35 |
| clarkb | the spike seems to be mostly gone though so whatever it was I think it is tempoarary | 01:35 |
| pabelanger | hmm | 01:38 |
| pabelanger | 2018-02-16 01:15:59,082 p=29027 u=root | ze02.openstack.org : ok=0 changed=0 unreachable=1 failed=0 | 01:38 |
| pabelanger | seems ansible wheel on puppetmaster can't connect to ze02 | 01:38 |
| pabelanger | ianw: clarkb: ^any ideas? | 01:39 |
| pabelanger | 2018-02-16 01:37:07,363 p=27096 u=root | fatal: [ze02.openstack.org]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 2001:4800:7819:104:be76:4eff:fe05:302 port 22: Connection timed out\r\n", "unreachable": true} | 01:39 |
| clarkb | pabelanger: the ip address failed | 01:39 |
| clarkb | er | 01:39 |
| pabelanger | didn't IPv6 change recently? | 01:39 |
| clarkb | s/failed/changed | 01:39 |
| pabelanger | ah | 01:39 |
| clarkb | so likely need to just clear out the inventory cache file and let it rebuild it | 01:39 |
| pabelanger | kk | 01:40 |
| ianw | yeah ... should have just rebuilt it | 01:40 |
| pabelanger | rebuilding cache now | 01:41 |
| pabelanger | Hmm, no change | 01:44 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 01:45 |
| pabelanger | Connecting to ze02.openstack.org [2001:4800:7819:104:be76:4eff:fe02:8373] port 22 | 01:45 |
| pabelanger | that is new IPv6 | 01:45 |
| pabelanger | "ansible_ssh_host": "2001:4800:7819:104:be76:4eff:fe05:302", | 01:45 |
| pabelanger | is what inventory sees | 01:45 |
| pabelanger | possible openstack API is confused? | 01:45 |
| ianw | i was using the api and *pretty* sure it all looked right | 01:46 |
| pabelanger | | accessIPv6 | 2001:4800:7819:104:be76:4eff:fe05:302 | | 01:46 |
| pabelanger | thats what I get back from openstack server show ze02.openstack.org | 01:46 |
| *** olaph1 has joined #openstack-infra | 01:47 | |
| pabelanger | which, fails | 01:47 |
| pabelanger | ianw: did you reboot the server via API or just reattach network? | 01:47 |
| *** olaph has quit IRC | 01:48 | |
| ianw | that's the old address! | 01:48 |
| pabelanger | yah | 01:48 |
| ianw | i shutdown the server, detacted and reattached then rebooted | 01:48 |
| pabelanger | but new one via DNS works as expected | 01:48 |
| pabelanger | I want to say I seen this before in testing with citycloud, trying to remember how I get the info to update | 01:49 |
| ianw | cached? | 01:49 |
| pabelanger | ianw: yah, I was trying to enable ipv6 on citycloud region, and show how the address was stale like this. | 01:50 |
| pabelanger | but, can't remember what I did to refresh it | 01:50 |
| pabelanger | Was for a mirror | 01:50 |
| pabelanger | but, in the end, we just deleted the server and recreated I think | 01:51 |
| pabelanger | woot | 01:56 |
| pabelanger | 2018-02-16 01:52:38.420137 | controller | | 0000000001 | 0000000001 | devstack | opensuse-tumbleweed | opensuse-tumbleweed-1518745952 | d70c20fc-89e6-4641-9d38-5d8e3cd51a3e | ready | 00:00:00:03 | | 01:56 |
| pabelanger | just waiting for node to come online now | 01:56 |
| pabelanger | likely going to see glean issue :) | 01:56 |
| *** agopi_out has joined #openstack-infra | 02:04 | |
| *** yamahata has quit IRC | 02:04 | |
| gmann_ | AJaeger: can you re-confirm your comment on this - https://review.openstack.org/#/c/521888/10 | 02:05 |
| *** salv-orlando has joined #openstack-infra | 02:08 | |
| *** salv-orlando has quit IRC | 02:13 | |
| *** xarses_ has quit IRC | 02:14 | |
| *** hongbin has joined #openstack-infra | 02:18 | |
| *** ijw has quit IRC | 02:19 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:23 |
| *** mriedem1 has joined #openstack-infra | 02:29 | |
| *** mriedem has quit IRC | 02:29 | |
| *** mriedem1 is now known as mriedem | 02:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:38 |
| *** slaweq has joined #openstack-infra | 02:46 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:46 |
| *** slaweq has quit IRC | 02:50 | |
| *** jogo has quit IRC | 02:51 | |
| *** kiennt26 has joined #openstack-infra | 02:54 | |
| *** jogo has joined #openstack-infra | 02:58 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 02:59 |
| *** dave-mccowan has joined #openstack-infra | 03:00 | |
| *** claudiub has joined #openstack-infra | 03:02 | |
| *** xarses_ has joined #openstack-infra | 03:02 | |
| *** andreww has joined #openstack-infra | 03:03 | |
| *** kiennt26 has quit IRC | 03:04 | |
| *** xarses_ has quit IRC | 03:06 | |
| *** mriedem has quit IRC | 03:07 | |
| *** olaph has joined #openstack-infra | 03:09 | |
| *** salv-orlando has joined #openstack-infra | 03:09 | |
| *** olaph1 has quit IRC | 03:12 | |
| *** salv-orlando has quit IRC | 03:14 | |
| *** dave-mcc_ has joined #openstack-infra | 03:15 | |
| *** dave-mccowan has quit IRC | 03:15 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Add legacy-copy-jenkins-scripts https://review.openstack.org/545158 | 03:18 |
| *** sree has joined #openstack-infra | 03:19 | |
| *** numans_ has joined #openstack-infra | 03:21 | |
| *** numans has quit IRC | 03:22 | |
| *** claudiub has quit IRC | 03:22 | |
| *** harlowja_ has quit IRC | 03:23 | |
| *** vivsoni has joined #openstack-infra | 03:24 | |
| *** kiennt26 has joined #openstack-infra | 03:25 | |
| *** rlandy|rover|bbl is now known as rlandy|rover | 03:27 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: nodepool-integration fixes https://review.openstack.org/545163 | 03:28 |
| *** SumitNaiksatam has joined #openstack-infra | 03:28 | |
| *** numans_ has quit IRC | 03:28 | |
| *** rlandy|rover has quit IRC | 03:29 | |
| *** dave-mcc_ has quit IRC | 03:41 | |
| *** numans has joined #openstack-infra | 03:44 | |
| *** bobh has joined #openstack-infra | 03:52 | |
| *** bobh has quit IRC | 03:57 | |
| *** cshastri has joined #openstack-infra | 04:01 | |
| *** yamamoto has joined #openstack-infra | 04:05 | |
| *** andreas_s has joined #openstack-infra | 04:13 | |
| *** andreas_s has quit IRC | 04:17 | |
| *** salv-orlando has joined #openstack-infra | 04:23 | |
| *** psachin has joined #openstack-infra | 04:25 | |
| *** salv-orlando has quit IRC | 04:29 | |
| *** bobh has joined #openstack-infra | 04:30 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 04:33 |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 04:33 |
| *** sree has quit IRC | 04:34 | |
| *** sree has joined #openstack-infra | 04:34 | |
| *** janki|afk has joined #openstack-infra | 04:35 | |
| *** lpetrut has joined #openstack-infra | 04:36 | |
| *** ramishra has joined #openstack-infra | 04:39 | |
| *** ykarel has joined #openstack-infra | 04:44 | |
| *** hongbin has quit IRC | 04:45 | |
| *** harlowja has joined #openstack-infra | 04:46 | |
| *** slaweq has joined #openstack-infra | 04:46 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 04:50 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Add nodepool-zuul-functional as non-voting check job https://review.openstack.org/545203 | 04:50 |
| *** slaweq has quit IRC | 04:51 | |
| *** yamamoto_ has joined #openstack-infra | 04:51 | |
| *** janki|afk has quit IRC | 04:54 | |
| *** janki has joined #openstack-infra | 04:54 | |
| *** yamamoto has quit IRC | 04:55 | |
| *** harlowja has quit IRC | 04:59 | |
| *** bobh has quit IRC | 05:03 | |
| *** mdbooth has quit IRC | 05:13 | |
| *** ari[m] has quit IRC | 05:13 | |
| *** lpetrut has quit IRC | 05:13 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/gerritbot master: Add more unit tests including testing of Gerrit() class https://review.openstack.org/545128 | 05:15 |
| *** priteau has joined #openstack-infra | 05:19 | |
| *** lpetrut has joined #openstack-infra | 05:19 | |
| *** harlowja has joined #openstack-infra | 05:19 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/gerritbot master: Add more unit tests including testing of Gerrit() class https://review.openstack.org/545128 | 05:22 |
| *** priteau has quit IRC | 05:23 | |
| *** niska has quit IRC | 05:24 | |
| *** arxcruz|ruck has quit IRC | 05:24 | |
| *** salv-orlando has joined #openstack-infra | 05:25 | |
| *** harlowja has quit IRC | 05:27 | |
| *** arxcruz has joined #openstack-infra | 05:27 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/gerritbot master: Add more unit tests including testing of Gerrit() class https://review.openstack.org/545128 | 05:28 |
| *** salv-orlando has quit IRC | 05:29 | |
| *** niska has joined #openstack-infra | 05:32 | |
| *** lpetrut has quit IRC | 05:35 | |
| *** rosmaita has quit IRC | 05:39 | |
| *** claudiub has joined #openstack-infra | 05:40 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Use $NODEPOOL_KEY_NAME for dsvm testing https://review.openstack.org/545176 | 05:44 |
| *** dougwig has quit IRC | 05:49 | |
| *** aeng has quit IRC | 05:51 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Pass at cleaning up the bootloader https://review.openstack.org/541129 | 05:51 |
| openstackgerrit | Merged openstack-infra/system-config master: Add #openstack-kayobe to statusbot and meetbot https://review.openstack.org/531595 | 05:57 |
| *** kiennt26 has quit IRC | 05:57 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add #networking-sfc to statusbot and meetbot https://review.openstack.org/535390 | 05:58 |
| openstackgerrit | Merged openstack-infra/system-config master: Add statusbot/meetbot/eavesdrop to #openstack-self-healing https://review.openstack.org/531136 | 05:58 |
| *** dhajare|mtg has joined #openstack-infra | 06:01 | |
| *** ankkumar has joined #openstack-infra | 06:01 | |
| *** coolsvap has joined #openstack-infra | 06:03 | |
| *** pgadiya has joined #openstack-infra | 06:08 | |
| ankkumar | Hi All | 06:09 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/gerritbot master: Add more unit tests including testing of Gerrit() class https://review.openstack.org/545128 | 06:09 |
| ankkumar | I am getting this error while running my ci Ansible complete, result RESULT_TIMED_OUT code None | 06:10 |
| ankkumar | in zuul v3 | 06:11 |
| ankkumar | My test is passing but the ansible is timing out.. what timeout i need to set? | 06:11 |
| *** pgadiya has quit IRC | 06:17 | |
| *** threestrands has quit IRC | 06:20 | |
| *** salv-orlando has joined #openstack-infra | 06:25 | |
| *** openstackstatus has quit IRC | 06:27 | |
| *** openstack has joined #openstack-infra | 06:29 | |
| *** ChanServ sets mode: +o openstack | 06:29 | |
| *** pgadiya has joined #openstack-infra | 06:29 | |
| *** salv-orlando has quit IRC | 06:30 | |
| *** salv-orlando has joined #openstack-infra | 06:32 | |
| *** eroux has joined #openstack-infra | 06:33 | |
| *** yamahata has joined #openstack-infra | 06:37 | |
| *** olaph1 has joined #openstack-infra | 06:40 | |
| *** olaph has quit IRC | 06:41 | |
| *** andreww has quit IRC | 06:45 | |
| AJaeger | clarkb: regarding nodepool-zuul-functional: That's a native job, it should use bindep instead of using install-distro-packages | 06:49 |
| AJaeger | sorry, now reading all of backscroll | 06:51 |
| *** andreas_s has joined #openstack-infra | 06:57 | |
| *** andreas_s has quit IRC | 07:01 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs in tap-as-a-service https://review.openstack.org/513228 | 07:02 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Use bindep for nodepool-zuul-functional https://review.openstack.org/545219 | 07:06 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:08 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Remove copying of legacy install-distro script https://review.openstack.org/545222 | 07:09 |
| AJaeger | ianw: I think that's the way to do it - but we can go with your change and I'll do mine on top ^ Or merge my base change into yours ^ | 07:09 |
| *** gmmaha has joined #openstack-infra | 07:10 | |
| *** rwsu has quit IRC | 07:12 | |
| gmmaha | Hi, two of openstack-helm CI have timed out https://review.openstack.org/523513 and https://review.openstack.org/539019 but there are no links for the tests. | 07:13 |
| gmmaha | any help will be much appreciated. Not sure why it is timing out. | 07:14 |
| *** eumel8 has joined #openstack-infra | 07:17 | |
| *** pgadiya has quit IRC | 07:18 | |
| *** threestrands has joined #openstack-infra | 07:22 | |
| *** threestrands has quit IRC | 07:22 | |
| *** threestrands has joined #openstack-infra | 07:23 | |
| *** rwsu has joined #openstack-infra | 07:25 | |
| ianw | AJaeger: feel free to merge away :) I think i sent it through like 15 reviews so ended up a bit cross-eyed :) | 07:32 |
| *** pgadiya has joined #openstack-infra | 07:32 | |
| AJaeger | ianw: and sorry for that - I missed this usage when grepping for install-distro | 07:33 |
| AJaeger | ianw: ok, will merge | 07:33 |
| ianw | AJaeger: no problems, the job had bit-rotted anyway | 07:34 |
| *** threestrands has quit IRC | 07:34 | |
| ianw | we should be able to drop the zuul-cloner in https://review.openstack.org/#/c/545221/1/tools/nodepool-integration-setup.sh too; i'm not sure if we have a predone role for installing though | 07:35 |
| AJaeger | ianw: let's do the zuul-cloner separately | 07:37 |
| AJaeger | required-projects should be enough for isntallation | 07:37 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 07:38 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Fix nodepool-zuul-functional https://review.openstack.org/545158 | 07:39 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:40 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Remove copying of legacy install-distro script https://review.openstack.org/545222 | 07:40 |
| AJaeger | ianw: merged ^ | 07:40 |
| *** yamamoto_ has quit IRC | 07:40 | |
| *** rcernin has quit IRC | 07:41 | |
| *** yamamoto has joined #openstack-infra | 07:42 | |
| *** yamamoto has quit IRC | 07:42 | |
| *** yamamoto has joined #openstack-infra | 07:43 | |
| *** lpetrut has joined #openstack-infra | 07:44 | |
| *** slaweq has joined #openstack-infra | 07:44 | |
| *** dtantsur|afk is now known as dtantsur | 07:46 | |
| *** yamamoto has quit IRC | 07:48 | |
| *** sree has quit IRC | 07:50 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Avoid tools/nodepool-integration-setup.sh from zuul https://review.openstack.org/545228 | 07:52 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 07:52 |
| AJaeger | ianw: these two on top should do the trick - let's see whether that's all, I fear log copying will be wrong | 07:53 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/nodepool master: Avoid tools/nodepool-integration-setup.sh from zuul https://review.openstack.org/545228 | 07:57 |
| AJaeger | ianw: I understand why you're cross-eyed - lots of dependencies between these two ;( | 07:57 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 07:58 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 07:58 |
| *** hashar has joined #openstack-infra | 07:58 | |
| AJaeger | ianw: now wait and see ;) | 08:03 |
| * AJaeger will be back later... | 08:03 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/zuul-jobs master: Improve file renaming in stage-output https://review.openstack.org/544606 | 08:04 |
| *** yee37934 has quit IRC | 08:07 | |
| *** yee379 has joined #openstack-infra | 08:07 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Rework base-publish-static jobs using protected https://review.openstack.org/545234 | 08:08 |
| AJaeger | corvus: ^ | 08:09 |
| AJaeger | corvus, this uses protected now as you suggested ^ | 08:09 |
| *** andreas_s has joined #openstack-infra | 08:10 | |
| *** alexchadin has joined #openstack-infra | 08:13 | |
| *** aviau has quit IRC | 08:19 | |
| *** aviau has joined #openstack-infra | 08:19 | |
| *** HeOS has joined #openstack-infra | 08:24 | |
| *** alexchadin has quit IRC | 08:24 | |
| *** alexchadin has joined #openstack-infra | 08:25 | |
| *** tesseract has joined #openstack-infra | 08:27 | |
| *** sree has joined #openstack-infra | 08:31 | |
| *** salv-orlando has quit IRC | 08:32 | |
| *** amoralej|off is now known as amoralej | 08:32 | |
| *** salv-orlando has joined #openstack-infra | 08:32 | |
| *** gfidente has joined #openstack-infra | 08:35 | |
| *** e0ne has joined #openstack-infra | 08:35 | |
| *** salv-orlando has quit IRC | 08:37 | |
| *** gibi has joined #openstack-infra | 08:40 | |
| *** jtomasek has joined #openstack-infra | 08:40 | |
| *** gibi has quit IRC | 08:41 | |
| *** gibi has joined #openstack-infra | 08:41 | |
| *** jpena|off is now known as jpena | 08:41 | |
| *** gibi is now known as giblet | 08:42 | |
| *** florianf has joined #openstack-infra | 08:44 | |
| *** yamamoto has joined #openstack-infra | 08:44 | |
| *** salv-orlando has joined #openstack-infra | 08:45 | |
| *** david-lyle has quit IRC | 08:46 | |
| *** slaweq_ has joined #openstack-infra | 08:48 | |
| *** yamamoto has quit IRC | 08:50 | |
| *** ralonsoh has joined #openstack-infra | 08:51 | |
| *** slaweq_ has quit IRC | 08:53 | |
| *** e0ne has quit IRC | 08:54 | |
| *** jpich has joined #openstack-infra | 08:54 | |
| *** yamamoto has joined #openstack-infra | 08:59 | |
| *** salv-orlando has quit IRC | 09:04 | |
| *** salv-orlando has joined #openstack-infra | 09:05 | |
| *** rossella_s has quit IRC | 09:05 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/grafyaml master: Cap voluptuous version https://review.openstack.org/545016 | 09:08 |
| *** rossella_s has joined #openstack-infra | 09:09 | |
| *** salv-orlando has quit IRC | 09:09 | |
| *** pcaruana has joined #openstack-infra | 09:11 | |
| *** tosky has joined #openstack-infra | 09:12 | |
| *** rossella_s has quit IRC | 09:14 | |
| *** yamamoto has quit IRC | 09:15 | |
| *** rossella_s has joined #openstack-infra | 09:19 | |
| *** yamahata has quit IRC | 09:21 | |
| *** vabada has quit IRC | 09:25 | |
| *** priteau has joined #openstack-infra | 09:26 | |
| *** derekh has joined #openstack-infra | 09:29 | |
| *** bauzas is now known as bauwser | 09:30 | |
| *** ethfci has joined #openstack-infra | 09:31 | |
| *** stephenfin is now known as finucannot | 09:35 | |
| *** links has joined #openstack-infra | 09:47 | |
| *** links has quit IRC | 09:49 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Queens gerritbot tracking for #openstack-dns https://review.openstack.org/545039 | 09:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add Queens gerritbot tracking for OSA chan https://review.openstack.org/545032 | 09:50 |
| *** priteau has quit IRC | 09:56 | |
| *** priteau has joined #openstack-infra | 09:57 | |
| *** links has joined #openstack-infra | 09:58 | |
| *** alexchadin has quit IRC | 10:01 | |
| *** priteau has quit IRC | 10:03 | |
| *** giblet has quit IRC | 10:03 | |
| openstackgerrit | Merged openstack-infra/storyboard master: remove unused variable in test-cleanup.sh https://review.openstack.org/539695 | 10:07 |
| *** agopi_out has quit IRC | 10:08 | |
| *** links has quit IRC | 10:12 | |
| *** yamamoto has joined #openstack-infra | 10:15 | |
| *** gfidente has quit IRC | 10:18 | |
| *** openstackgerrit has quit IRC | 10:18 | |
| *** yamamoto has quit IRC | 10:20 | |
| aspiers | hi all, should eavesdrop kick in immediately after https://review.openstack.org/#/c/531136/ (which adds an IRC channel to system-config/hiera/common.yaml) is merged, or does it take a while? | 10:20 |
| *** priteau has joined #openstack-infra | 10:23 | |
| *** gfidente has joined #openstack-infra | 10:30 | |
| *** gfidente has joined #openstack-infra | 10:30 | |
| *** zoli is now known as zoli|lunch | 10:31 | |
| *** zoli|lunch is now known as zoli | 10:31 | |
| frickler | aspiers: looks like that patch only added statusbot and not meetbot, there is another list called "meetbot_channels" for the latter in the same file | 10:33 |
| *** dosaboy has quit IRC | 10:33 | |
| *** stakeda has quit IRC | 10:33 | |
| *** e0ne has joined #openstack-infra | 10:34 | |
| *** ldnunes has joined #openstack-infra | 10:34 | |
| *** dosaboy has joined #openstack-infra | 10:38 | |
| *** sambetts|afk is now known as sambetts | 10:41 | |
| *** sree has quit IRC | 10:42 | |
| *** sree has joined #openstack-infra | 10:43 | |
| *** kiennt26 has joined #openstack-infra | 10:43 | |
| *** sree has quit IRC | 10:47 | |
| *** yamamoto has joined #openstack-infra | 10:51 | |
| *** alexchadin has joined #openstack-infra | 10:54 | |
| *** olaph1 is now known as olaph | 11:03 | |
| *** lucas-afk is now known as lucasagomes | 11:03 | |
| *** giblet has joined #openstack-infra | 11:05 | |
| *** openstackgerrit has joined #openstack-infra | 11:09 | |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/openstack-zuul-jobs master: Add stable/queens to periodic-stable templates https://review.openstack.org/545268 | 11:09 |
| *** alexchadin has quit IRC | 11:10 | |
| *** e0ne has quit IRC | 11:12 | |
| *** e0ne has joined #openstack-infra | 11:13 | |
| *** alexchadin has joined #openstack-infra | 11:22 | |
| *** tpsilva has joined #openstack-infra | 11:23 | |
| *** armaan has joined #openstack-infra | 11:27 | |
| *** pbourke has quit IRC | 11:32 | |
| *** pbourke has joined #openstack-infra | 11:33 | |
| *** armaan has quit IRC | 11:35 | |
| AJaeger | andreaf: should I help you moving the grenade jobs over to grenade repo? Still as legacy ones... | 11:41 |
| *** alexchadin has quit IRC | 11:41 | |
| *** alexchadin has joined #openstack-infra | 11:42 | |
| andreaf | AJaeger: we can either move them and then replace them or replace them with new ones in grenade | 11:42 |
| andreaf | AJaeger: but given that the grenade jobs may take some more time we could go for move and replace | 11:42 |
| andreaf | AJaeger: I'm a bit concerned about the churn in job names though | 11:42 |
| AJaeger | andreaf: we can wait as well - and you create new native jobs and then delete in neutron. | 11:43 |
| *** wolverineav has joined #openstack-infra | 11:43 | |
| andreaf | AJaeger: but if we go for move and then repalce your help would be much appreciated, thank you :) | 11:44 |
| AJaeger | andreaf: yeah, it needs a few changes with renaming | 11:44 |
| AJaeger | andreaf: so, do you want help *now* to move them over? Then I'll do it... | 11:45 |
| andreaf | AJaeger well you've done this kind of changes more than me - what do you think would be best? | 11:46 |
| andreaf | AJaeger: I plan to have the grenade native job done by the end of PTG - at least the single node one | 11:46 |
| AJaeger | andreaf: then let's wait ;) | 11:47 |
| andreaf | ok | 11:47 |
| *** pgadiya has quit IRC | 11:49 | |
| *** wolverineav has quit IRC | 11:51 | |
| *** yamamoto has quit IRC | 11:56 | |
| *** yamamoto has joined #openstack-infra | 11:59 | |
| *** links has joined #openstack-infra | 12:03 | |
| *** yamamoto has quit IRC | 12:08 | |
| *** salv-orlando has joined #openstack-infra | 12:10 | |
| *** yamamoto has joined #openstack-infra | 12:11 | |
| *** olaph1 has joined #openstack-infra | 12:12 | |
| *** armaan has joined #openstack-infra | 12:13 | |
| *** psachin has quit IRC | 12:14 | |
| *** olaph has quit IRC | 12:14 | |
| *** wolverineav has joined #openstack-infra | 12:17 | |
| *** jcoufal has joined #openstack-infra | 12:21 | |
| *** wolverineav has quit IRC | 12:22 | |
| *** psachin has joined #openstack-infra | 12:25 | |
| *** rossella_s has quit IRC | 12:27 | |
| *** rossella_s has joined #openstack-infra | 12:29 | |
| *** janki has quit IRC | 12:35 | |
| *** yamamoto has quit IRC | 12:41 | |
| *** pcichy has joined #openstack-infra | 12:42 | |
| *** rosmaita has joined #openstack-infra | 12:42 | |
| *** sree has joined #openstack-infra | 12:43 | |
| *** kiennt26 has quit IRC | 12:43 | |
| *** jlabarre has joined #openstack-infra | 12:47 | |
| *** jpena is now known as jpena|lunch | 12:48 | |
| *** slaweq_ has joined #openstack-infra | 12:50 | |
| *** dprince has joined #openstack-infra | 12:51 | |
| *** slaweq_ has quit IRC | 12:54 | |
| *** sree has quit IRC | 12:55 | |
| *** ankkumar has quit IRC | 12:56 | |
| *** cshastri has quit IRC | 12:57 | |
| *** ansmith has joined #openstack-infra | 12:57 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/nodepool master: WIP Add Nodepool Ansible generic driver https://review.openstack.org/545286 | 12:59 |
| *** janki has joined #openstack-infra | 13:06 | |
| *** sree has joined #openstack-infra | 13:06 | |
| *** dave-mccowan has joined #openstack-infra | 13:08 | |
| *** nicolasbock has joined #openstack-infra | 13:10 | |
| *** nicolasbock has quit IRC | 13:10 | |
| *** dsariel has joined #openstack-infra | 13:12 | |
| *** dsariel_ has joined #openstack-infra | 13:12 | |
| *** dsariel has quit IRC | 13:13 | |
| *** dsariel has joined #openstack-infra | 13:14 | |
| *** sree has quit IRC | 13:14 | |
| *** r-daneel has joined #openstack-infra | 13:14 | |
| *** kopecmartin has joined #openstack-infra | 13:15 | |
| *** ramishra has quit IRC | 13:15 | |
| *** yamamoto has joined #openstack-infra | 13:15 | |
| *** dsariel has quit IRC | 13:18 | |
| *** dsariel_ has quit IRC | 13:19 | |
| *** armaan has quit IRC | 13:19 | |
| *** zhongjun has quit IRC | 13:20 | |
| *** wolverineav has joined #openstack-infra | 13:22 | |
| *** coolsvap has quit IRC | 13:22 | |
| *** janki has quit IRC | 13:25 | |
| *** armaan has joined #openstack-infra | 13:25 | |
| *** trown|outtypewww is now known as trown | 13:26 | |
| *** dsariel_ has joined #openstack-infra | 13:26 | |
| *** wolverineav has quit IRC | 13:30 | |
| *** dave-mccowan has quit IRC | 13:31 | |
| *** armaan has quit IRC | 13:32 | |
| *** rlandy has joined #openstack-infra | 13:37 | |
| *** VW has quit IRC | 13:37 | |
| *** rlandy is now known as rlandy|rover | 13:37 | |
| *** VW has joined #openstack-infra | 13:39 | |
| *** psachin has quit IRC | 13:42 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config master: Create 'Review-Priority' for designate repos https://review.openstack.org/295253 | 13:43 |
| *** eharney has quit IRC | 13:43 | |
| *** VW has quit IRC | 13:43 | |
| *** rossella_s has quit IRC | 13:49 | |
| *** rossella_s has joined #openstack-infra | 13:50 | |
| *** jpena|lunch is now known as jpena | 13:51 | |
| *** jlabarre has quit IRC | 13:54 | |
| openstackgerrit | Akihiro Motoki proposed openstack-infra/project-config master: Setup queens translation sync for heat-dashboard https://review.openstack.org/545299 | 13:55 |
| *** jcoufal has quit IRC | 13:59 | |
| *** ralonsoh_ has joined #openstack-infra | 14:00 | |
| *** jcoufal has joined #openstack-infra | 14:01 | |
| *** Adri2000 has quit IRC | 14:02 | |
| *** mriedem has joined #openstack-infra | 14:03 | |
| *** panda|off is now known as panda|odf | 14:03 | |
| *** ralonsoh has quit IRC | 14:03 | |
| *** dave-mccowan has joined #openstack-infra | 14:04 | |
| *** bobh has joined #openstack-infra | 14:06 | |
| *** rossella_s has quit IRC | 14:06 | |
| *** kgiusti has joined #openstack-infra | 14:07 | |
| *** rossella_s has joined #openstack-infra | 14:09 | |
| *** Adri2000 has joined #openstack-infra | 14:09 | |
| *** Adri2000 has quit IRC | 14:09 | |
| *** Adri2000 has joined #openstack-infra | 14:09 | |
| *** abelur_ has quit IRC | 14:10 | |
| *** sree has joined #openstack-infra | 14:10 | |
| *** ykarel has quit IRC | 14:13 | |
| *** sree has quit IRC | 14:15 | |
| *** pcichy has quit IRC | 14:16 | |
| *** jcoufal has quit IRC | 14:16 | |
| *** jcoufal has joined #openstack-infra | 14:16 | |
| *** Goneri has joined #openstack-infra | 14:17 | |
| *** arxcruz is now known as arxcruz|ruck | 14:19 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config master: gerrit tests should return to checked out branch https://review.openstack.org/545313 | 14:21 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Cap voluptuous version https://review.openstack.org/545016 | 14:22 |
| *** jcoufal has quit IRC | 14:23 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: Switch to platform.linux_distribution() https://review.openstack.org/545314 | 14:23 |
| pabelanger | clarkb: AJaeger: dirk: greghaynes: can either one of you confirm the output of platform.linux_distribution() in ^ | 14:24 |
| pabelanger | on tumbleweed | 14:24 |
| *** andreas_s has quit IRC | 14:24 | |
| dirk | pabelanger: looking | 14:24 |
| *** andreas_s has joined #openstack-infra | 14:25 | |
| dirk | >>> platform.linux_distribution(full_distribution_name=0) | 14:25 |
| dirk | ('', '', '') | 14:25 |
| pabelanger | okay, that explains why glean doesn't like tumbleweed | 14:26 |
| *** eharney has joined #openstack-infra | 14:26 | |
| dirk | pabelanger: iirc platform. checks for legacy stuff like /etc/SuSE-release, and that doesn't exist anymore in newer opensuse releases | 14:26 |
| dirk | pabelanger: we only have /etc/os-release, but the python platform module does not read that (because they consider platform deprecated and refuse to fix it) | 14:26 |
| pabelanger | dirk: kk, do you have a suggestion how to detect the distro? | 14:26 |
| dirk | there is a python bug report about it | 14:26 |
| dirk | yep, second, we have code for that | 14:26 |
| *** VW has joined #openstack-infra | 14:27 | |
| pabelanger | great! | 14:27 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/grafyaml master: Unlock yaxes min limit and allow free value https://review.openstack.org/545006 | 14:27 |
| dirk | pabelanger: https://github.com/openstack/renderspec/blob/master/renderspec/__init__.py#L82 | 14:27 |
| *** jlabarre has joined #openstack-infra | 14:28 | |
| *** sree has joined #openstack-infra | 14:28 | |
| dirk | pabelanger: its not the prettiest thing on the planet.. there is I think a python module for that though | 14:28 |
| *** felipemonteiro_ has joined #openstack-infra | 14:30 | |
| *** lucasagomes is now known as lucas-hungry | 14:31 | |
| dirk | pabelanger: https://pypi.python.org/pypi/distro/1.2.0 | 14:31 |
| pabelanger | dirk: okay, we'll have to import something like that into glean, as we try not to have any external dependencies | 14:32 |
| *** felipemonteiro__ has joined #openstack-infra | 14:34 | |
| *** efried is now known as fried_rice | 14:34 | |
| *** sree has quit IRC | 14:35 | |
| *** andreas_s has quit IRC | 14:35 | |
| *** alexchadin has quit IRC | 14:37 | |
| *** felipemonteiro_ has quit IRC | 14:37 | |
| AJaeger | pabelanger: regarding https://review.openstack.org/#/c/543139/ - I don't see it as viable to install that only on demand. We have legacy jobs in openstack-zuul-jobs and we have bindep role in zuul-jobs. We need the file on the node for both. | 14:38 |
| AJaeger | pabelanger: commented on the review as well. With the configuration and roles in three repos, I see no nice way of copying over on demand. | 14:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: Fix SUSE detection on Tumbleweed and SLE15 https://review.openstack.org/545314 | 14:42 |
| *** sree has joined #openstack-infra | 14:42 | |
| *** amoralej is now known as amoralej|lunch | 14:43 | |
| *** dansmith is now known as superdan | 14:43 | |
| fungi | i see in scrollback ankkumar (06:11z) and gmmaha (07:13z) complained about unanticipated job timeouts. i wonder if we're seeing fallout from the timeout change suggesting some jobs need retuning as a result? | 14:43 |
| AJaeger | fungi, gmmaha's case confused me - there was no log file | 14:44 |
| AJaeger | fungi: "openstack-helm-infra-ubuntu finger://ze08.openstack.org/56b38ac6f57749a39ed7e07d7f49985f : TIMED_OUT in 30m 08s" in https://review.openstack.org/#/c/523513/ | 14:45 |
| fungi | possible log collection in that job is happening in the run phase instead of post? | 14:45 |
| AJaeger | maybe - didn't dig into that | 14:45 |
| fungi | or are we applying job timeouts to post phase as well? | 14:45 |
| AJaeger | fungi: yes - and that could be the reason, that the changed timeout handling let us to not collect anything. | 14:46 |
| AJaeger | didn't we have in the past we had a 5 min buffer for the log collectoin? So that we always could get that? Did clarkb implement something like that? | 14:46 |
| * AJaeger needs to run some errands, will be back later | 14:46 | |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Add links to GrafYaml https://review.openstack.org/542395 | 14:47 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Add more units for graph panel https://review.openstack.org/543937 | 14:48 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Add Custom Template Type https://review.openstack.org/544305 | 14:49 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Allow hidden templated vars https://review.openstack.org/542979 | 14:49 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Allow decimals to be specificed to limit decimals graphs https://review.openstack.org/545112 | 14:50 |
| openstackgerrit | Alex Krzos proposed openstack-infra/grafyaml master: Allow repeating panels and to set minspan https://review.openstack.org/545062 | 14:50 |
| pabelanger | okay, I have no idea how glean is working on fedora 27, we have /etc/os-release too | 14:52 |
| pabelanger | also, https://bugs.python.org/issue1322 is pretty messed up | 14:52 |
| *** jistr is now known as jistr|mtg | 14:52 | |
| *** sree has quit IRC | 14:52 | |
| pabelanger | platform.dist and platform.linux_distribution are both deprecated now. And I don't see an intree replacement | 14:53 |
| *** andreas_s has joined #openstack-infra | 14:54 | |
| *** salv-orl_ has joined #openstack-infra | 14:55 | |
| *** jcoufal has joined #openstack-infra | 14:58 | |
| *** ari[m] has joined #openstack-infra | 14:58 | |
| *** r-daneel has quit IRC | 14:58 | |
| *** salv-orlando has quit IRC | 14:58 | |
| *** andreas_s has quit IRC | 14:58 | |
| *** mdbooth has joined #openstack-infra | 14:59 | |
| *** hrybacki is now known as hybacki|mtg | 15:00 | |
| *** dsariel_ has quit IRC | 15:01 | |
| *** ykarel has joined #openstack-infra | 15:01 | |
| *** andreas_s has joined #openstack-infra | 15:04 | |
| *** sree has joined #openstack-infra | 15:04 | |
| *** felipemonteiro__ has quit IRC | 15:06 | |
| *** felipemonteiro_ has joined #openstack-infra | 15:06 | |
| mordred | pabelanger: that's my reading too | 15:08 |
| *** andreas_s has quit IRC | 15:08 | |
| *** trown is now known as trown|brb | 15:11 | |
| *** dsariel_ has joined #openstack-infra | 15:14 | |
| *** lbragstad has quit IRC | 15:14 | |
| fungi | i think this is why bindep resorts to running an external tool | 15:15 |
| *** jistr|mtg is now known as jistr | 15:15 | |
| *** lbragstad has joined #openstack-infra | 15:16 | |
| pabelanger | yah, I'm trying to intree the distro package first, to see if I can get it working | 15:17 |
| pabelanger | maybe we just vendor it or see about moving bindep logic into glean | 15:18 |
| *** olaph1 is now known as olaph | 15:19 | |
| fungi | for the record, i wouldn't want to see bindep's logic abstracted into a separate lib because the goal is for it to have no dependencies... but i wonder if we could massage it however much is needed to use bindep as an optional dependency of glean | 15:20 |
| fungi | (that is to say, no dependencies aside from the lsb utility it runs to discover the operating system details) | 15:21 |
| fungi | or, yeah, consider maybe that's so straightforward that it can just live in both tools | 15:21 |
| fungi | it's not like we probably gain a lot by deduplicating that when it's mostly a subprocess call and pattern matching the result | 15:22 |
| mordred | fungi: ++ | 15:25 |
| *** amoralej|lunch is now known as amoralej | 15:25 | |
| mordred | fungi, pabelanger: alternately, we COULD rewrite glean in rust - although that's obviously a much longer task | 15:25 |
| *** felipemonteiro__ has joined #openstack-infra | 15:26 | |
| fungi | does rust actually solve this particular problem? | 15:27 |
| *** trown|brb is now known as trown | 15:27 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: WIP: Fix SUSE detection on Tumbleweed and SLE15 https://review.openstack.org/545314 | 15:28 |
| *** lucas-hungry is now known as lucasagomes | 15:28 | |
| pabelanger | mordred: fungi: ^ fixes unit tests locally, lets see if we can build / upload image with nodepool dsvm | 15:29 |
| pabelanger | as for vendoring distro or adding dependency. We could start installing glean into virtualenv to protect operating system pip packages | 15:29 |
| *** felipemonteiro_ has quit IRC | 15:29 | |
| AJaeger | pabelanger: time now to discuss https://review.openstack.org/543139 ? | 15:30 |
| pabelanger | AJaeger: can we push back another few hours? | 15:31 |
| AJaeger | let's try ;) | 15:31 |
| pabelanger | AJaeger: thanks | 15:31 |
| corvus | AJaeger, fungi, clarkb: there's no special handling of post vs run timeout, so yeah, once we hit the timeout we're not going to complete any post playbooks | 15:31 |
| *** lpetrut has quit IRC | 15:31 | |
| fungi | got it, so if jobs want to save time for post playbooks to collect logs they should probably set a separate, lower timeout on their run phase? | 15:34 |
| Roamer` | hm, speaking of bindep, how is it supposed to handle the fact that there is no libmysqlclient-dev in Debian any more? There's just libmariadbclient-dev now. | 15:34 |
| fungi | Roamer`: that can be done by editing bindep.txt files assuming libmariadbclient-dev does't do a provides for the old libmysqlclient-dev package name | 15:35 |
| fungi | but worset case you can specify a different package for particular debian releases | 15:35 |
| fungi | er, worst case | 15:35 |
| *** andreas_s has joined #openstack-infra | 15:36 | |
| *** rossella_s has quit IRC | 15:37 | |
| * fungi is running debian/unstable, so checks | 15:38 | |
| fungi | looks like it has a conflicts on libmysqlclient-dev but no provides | 15:38 |
| *** hybacki|mtg is now known as hrybacki | 15:39 | |
| *** kopecmartin has quit IRC | 15:39 | |
| fungi | Roamer`: though i have libmysqlclient-dev installed in a recently (last 24-hours or so) updated unstable system, so i guess that transition hasn't hit yet? | 15:39 |
| *** rossella_s has joined #openstack-infra | 15:40 | |
| *** r-daneel has joined #openstack-infra | 15:40 | |
| fungi | SpamapS: ^ you probably have more detail on this transition than i do | 15:40 |
| fungi | (if you're around) | 15:40 |
| *** slaweq has quit IRC | 15:42 | |
| fungi | https://packages.debian.org/libmysqlclient-dev indicates 5.7.21-1 is still very much present in sid | 15:42 |
| fungi | oh! | 15:42 |
| fungi | it was removed from stretch but still lingers in unstable | 15:42 |
| *** slaweq has joined #openstack-infra | 15:42 | |
| tosky | the transition of mysql->mariadb in sid was not exactly smooth | 15:43 |
| *** e0ne_ has joined #openstack-infra | 15:43 | |
| fungi | this explains my confusion | 15:43 |
| tosky | but it should have been completed already at this point | 15:43 |
| Roamer` | fungi, ah, that would explain why I don't have it, I live on testing | 15:43 |
| fungi | yeah, it's not in stretch or buster | 15:43 |
| *** e0ne has quit IRC | 15:43 | |
| fungi | looks like jessie was the last stable to include it | 15:43 |
| fungi | seeing what the pts has to say about that | 15:44 |
| *** david-lyle has joined #openstack-infra | 15:44 | |
| fungi | "Not touching package due to block request by pochu" | 15:45 |
| *** e0ne has joined #openstack-infra | 15:45 | |
| *** slaweq has quit IRC | 15:45 | |
| *** slaweq has joined #openstack-infra | 15:45 | |
| fungi | yeah, looks like there's an rc bug open on it for the past year | 15:46 |
| fungi | https://bugs.debian.org/853008 | 15:46 |
| openstack | Debian bug 853008 in mysql-server-5.7 "mysql-server-5.7: purge could delete mariadb-server files with inadequate warning" [Serious,Open] | 15:46 |
| *** olaph1 has joined #openstack-infra | 15:47 | |
| *** vaidy has quit IRC | 15:47 | |
| *** isviridov_away has quit IRC | 15:47 | |
| fungi | i have vague memories of this getting discussed on the debian-devel ml but have since forgotten the details | 15:47 |
| *** e0ne_ has quit IRC | 15:47 | |
| *** olaph has quit IRC | 15:48 | |
| fungi | and also the version in sid is vulnerable to https://security-tracker.debian.org/tracker/CVE-2017-15365 | 15:48 |
| *** camunoz has joined #openstack-infra | 15:49 | |
| Roamer` | fungi, FTR, https://tracker.debian.org/pkg/mysql-5.7 gives pretty much the same info at a glance, including a link to the bug; but yeah, I guess it takes time to retrain one's fingers, it took me months to stop typing packages.d.o :) | 15:49 |
| Roamer` | (and you can get there from https://tracker.debian.org/libmysqlclient-dev) | 15:49 |
| *** olaph1 is now known as olaph | 15:49 | |
| fungi | that's where i was, i tend to just follow the link from the package page to the tdo page for the source package | 15:49 |
| *** slaweq has quit IRC | 15:49 | |
| Roamer` | ah, right, sorry | 15:50 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 15:52 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Add nodepool-zuul-functional as non-voting check job https://review.openstack.org/545203 | 15:52 |
| fungi | Roamer`: so anyway, could probably tweak a bindep.txt to do libmysqlclient-dev [platform:dpkg !platform:debian-stretch !platform:debian-buster] and then libmariadbclient-dev [platform:debian-stretch platform:debian-buster] | 15:54 |
| fungi | or something along those lines | 15:54 |
| fungi | depending on which platforms your project is intended to support running on | 15:54 |
| fungi | if jessie is considered too old, then you could even do libmysqlclient-dev [platform:dpkg !platform:debian] | 15:55 |
| *** r-daneel has quit IRC | 15:55 | |
| fungi | or perhaps just libmysqlclient-dev [platform:ubuntu] | 15:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool integration tests https://review.openstack.org/540983 | 15:55 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Fix nodepool-zuul-functional https://review.openstack.org/545163 | 15:56 |
| fungi | Roamer`: and then libmariadbclient-dev [platform:debian] or something | 15:56 |
| fungi | bindep --profiles tells you what platform names the current system has | 15:56 |
| *** pcaruana has quit IRC | 15:57 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove install-distro step for nodepool integration https://review.openstack.org/545221 | 15:59 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul master: Remove tools/nodepool-integration-setup.sh https://review.openstack.org/545229 | 15:59 |
| *** salv-orl_ has quit IRC | 16:00 | |
| *** salv-orlando has joined #openstack-infra | 16:00 | |
| *** dhajare|mtg has quit IRC | 16:02 | |
| *** felipemonteiro__ has quit IRC | 16:02 | |
| *** rossella_s has quit IRC | 16:02 | |
| *** bnemec is now known as beekneemech | 16:03 | |
| *** r-daneel has joined #openstack-infra | 16:03 | |
| *** rossella_s has joined #openstack-infra | 16:05 | |
| *** salv-orlando has quit IRC | 16:05 | |
| *** rfolco is now known as rfolco|away | 16:05 | |
| pabelanger | woot, tumbleweed properly started | 16:06 |
| *** slaweq has joined #openstack-infra | 16:06 | |
| pabelanger | and nodepool was able to ssh into it | 16:06 |
| pabelanger | http://logs.openstack.org/14/545314/3/check/nodepool-functional-py35-suse-src/5cd0434/ | 16:06 |
| AJaeger | yeah | 16:07 |
| *** yamahata has joined #openstack-infra | 16:07 | |
| *** andreas_s has quit IRC | 16:09 | |
| pabelanger | fungi: mordred: So, what are you thoughts on https://review.openstack.org/545314/ do we want to consider vendoring the distro package? Or maybe consider updating diskimage-builder to support installing glean into a virtualenv and override systemd unit files to location | 16:09 |
| pabelanger | clarkb: ^might have thoughts too | 16:10 |
| *** finucannot is now known as stephenfin | 16:10 | |
| *** ykarel_ has joined #openstack-infra | 16:10 | |
| *** slaweq has quit IRC | 16:11 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul web: add admin endpoint, enqueue & autohold commands https://review.openstack.org/539004 | 16:11 |
| Roamer` | fungi, thanks, I was mainly missing the idea that bindep defines a debian-stretch profile, which should've been obvious in hindsight from the fact that it defines an ubuntu-trusty one | 16:11 |
| Roamer` | but also thanks for the bindep --profiles hint | 16:12 |
| fried_rice | jlvillal: Are you mucking with (or on top of) https://review.openstack.org/#/c/545128/ locally? | 16:12 |
| Roamer` | s/profile/platform/ of course | 16:12 |
| fried_rice | jlvillal: I'd like to enhance your GerritTestCase to go through the filters | 16:12 |
| fungi | Roamer`: we also have docs!!! https://docs.openstack.org/infra/bindep/ | 16:13 |
| fungi | (though it's basically just the same content as the readme) | 16:13 |
| *** lathiat has quit IRC | 16:13 | |
| *** ykarel has quit IRC | 16:13 | |
| jlvillal | fried_rice, I am not. I was hoping it was in a good enough stage to be merged as is. And then additional patches to follow to slowly improve it. | 16:14 |
| fried_rice | jlvillal: That's my plan, was going to propose one on top. | 16:14 |
| *** rossella_s has quit IRC | 16:14 | |
| jlvillal | fried_rice, Cool :) | 16:14 |
| *** david-lyle has quit IRC | 16:15 | |
| Roamer` | fungi, yeah, sorry about that :) I was mainly having a knee-jerk reaction to the fact that yesterday I couldn't easily test a quick openstack/requirements "tinyrpc===0.8" fix on my laptop, and I was thinking "does requirements/bindep.txt need to reflect that?" I still wonder, but maybe I'll raise it in #openstack-requirements instead, maybe not right now | 16:15 |
| jlvillal | fried_rice, We just need to get infra to review them :) | 16:15 |
| clarkb | pabelanger: it doesn't look like you've vendored the package but instead just listed it as a requirement? | 16:15 |
| *** hongbin has joined #openstack-infra | 16:16 | |
| fungi | Roamer`: yeah, we intend for this to be entirely under the control of those individual teams to make system dependency and support decisions for their deliverables. we just provide a tool and some guidance/examples | 16:16 |
| fungi | (and testing if they want it) | 16:16 |
| *** kjackal has quit IRC | 16:16 | |
| *** rossella_s has joined #openstack-infra | 16:16 | |
| clarkb | pabelanger: re platform.linux_distribution() its ('', '', '') my hunch is that that is a bug in the python packaging for tumblweed | 16:16 |
| jlvillal | fried_rice, If you are Eric Fried. I was thinking maybe the logic for the regex could be put in the ChannelConfig class. What do you think? | 16:16 |
| clarkb | pabelanger: so something that would probably get fixed quickly if we reported it | 16:16 |
| pabelanger | clarkb: yah, I haven't vendored it yet. Just want to make sure it first worked. I know we don't want glean to start growing external requirements too, so was going to upload a new patch if we decided vendoring was okay | 16:17 |
| clarkb | probably the best thing is to get the distro to fix their python packaging | 16:17 |
| pabelanger | clarkb: it isn't a simple fix. See https://bugs.python.org/issue1322 | 16:17 |
| jlvillal | fried_rice, I was thinking could add some functions and a cache. So don't have to do the re.match() every time through. aka Dynamic Programming (I think) | 16:17 |
| pabelanger | clarkb: and I can confirm it is broken on fedora-27 too | 16:18 |
| pabelanger | because of /etc/os-release | 16:18 |
| pabelanger | rather then fixing, they've seem to just deprecate it, with no replacement | 16:19 |
| jlvillal | If anyone would like to review: https://review.openstack.org/#/c/545128/ More unit tests for openstack/gerritbot | 16:19 |
| clarkb | pabelanger: maybe the simple-init shell wrapper around glean can source /etc/os-release and feed the data into glean? I dunno | 16:19 |
| fried_rice | jlvillal: That all sounds lovely, but I'd like to keep the initial regex patch simple | 16:21 |
| fried_rice | jlvillal: And we can iterate, as you say. | 16:21 |
| jlvillal | fried_rice, Okay. | 16:21 |
| *** lathiat has joined #openstack-infra | 16:22 | |
| *** david-lyle has joined #openstack-infra | 16:22 | |
| *** sree has quit IRC | 16:23 | |
| *** markvoelker has quit IRC | 16:23 | |
| *** VW_ has joined #openstack-infra | 16:24 | |
| *** sree has joined #openstack-infra | 16:26 | |
| *** VW_ has quit IRC | 16:27 | |
| *** VW_ has joined #openstack-infra | 16:27 | |
| *** VW has quit IRC | 16:27 | |
| *** andreas_s has joined #openstack-infra | 16:27 | |
| *** markvoelker_ has joined #openstack-infra | 16:27 | |
| openstackgerrit | Ian Y. Choi proposed openstack-infra/project-config master: [translation] Supports pot for project docs https://review.openstack.org/545377 | 16:29 |
| *** sree has quit IRC | 16:31 | |
| *** aspiers has quit IRC | 16:33 | |
| *** agopi_out has joined #openstack-infra | 16:34 | |
| *** jbadiapa has quit IRC | 16:34 | |
| *** pcichy has joined #openstack-infra | 16:34 | |
| *** slaweq has joined #openstack-infra | 16:34 | |
| *** imacdonn has quit IRC | 16:37 | |
| *** imacdonn has joined #openstack-infra | 16:38 | |
| clarkb | pabelanger: another appraoch could be to identify systemd vs not systemd but even then I Think we run into distro specific network configuration stuff (since its not all done by systemd) | 16:38 |
| *** aspiers has joined #openstack-infra | 16:38 | |
| *** slaweq has quit IRC | 16:39 | |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Test Gerrit message filtering https://review.openstack.org/545381 | 16:39 |
| fried_rice | jlvillal: There ya go ^ | 16:39 |
| jlvillal | fried_rice, Thanks | 16:40 |
| fried_rice | jlvillal: Now I'll rebase the regex patch on top of that one, and change it to key regex-ness off of '^' instead of the new yaml section. | 16:40 |
| jlvillal | Cool | 16:40 |
| *** dsariel_ has quit IRC | 16:43 | |
| *** felipemonteiro has joined #openstack-infra | 16:45 | |
| *** andreas_s has quit IRC | 16:46 | |
| *** ykarel_ is now known as ykarel|away | 16:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/glean master: Fix SUSE detection on Tumbleweed and SLE15 https://review.openstack.org/545314 | 16:47 |
| pabelanger | clarkb: updated to vendor^ | 16:48 |
| *** SumitNaiksatam has quit IRC | 16:49 | |
| pabelanger | clarkb: yah, it is possible we could remove the autodetect logic out of glean, and depend more on --distro flag. Updating wrapper to always pass in distro info | 16:49 |
| pabelanger | clarkb: it also seem pip vendors this file too | 16:50 |
| clarkb | nice | 16:50 |
| pabelanger | https://github.com/pypa/pip/pull/3906/ | 16:50 |
| rbergeron | ohhh. speaking of glean (I was just trying to figure a thing out there) -- I haz possibly dumb question | 16:50 |
| rbergeron | (as in, i'm pretty sure it's a bug and uh, i should file it but figured i would sanity check myself here, imposter syndrome and all) -- | 16:50 |
| rbergeron | on https://docs.openstack.org/infra/glean/ -- the first link on the page (for configuration drive) points at http://docs.openstack.org/user-guide/cli_config_drive.html -- | 16:51 |
| clarkb | pabelanger: in that case vendoring it is probably a reasonable thing to do if even pip has to do it | 16:51 |
| rbergeron | which just brings up ... all the user guides? | 16:51 |
| rbergeron | and it seems to be ... in other places too. anything pointing at any part of the user guide in docs is like, just going to the super meta giant main index of all pike user guides | 16:52 |
| clarkb | rbergeron: that is probably fallout of moving more of the docs into projects rather than having central guides | 16:52 |
| clarkb | rbergeron: we probably just need to find wherever nova documents config-drive now and update the link. | 16:52 |
| *** VW_ has quit IRC | 16:53 | |
| clarkb | https://docs.openstack.org/nova/latest/user/config-drive.html maybe? | 16:53 |
| *** VW has joined #openstack-infra | 16:53 | |
| clarkb | rbergeron: as someone reading the docs currently does ^ appropriately indicate what config drive is? | 16:54 |
| rbergeron | clarkb: it's also, like, not just that doc -- seems like other stuff that was pointing to anything in user-guide is doing the same behavior | 16:55 |
| rbergeron | clarkb: i eventually found -- https://docs.openstack.org/ocata/user-guide/cli-config-drive.html | 16:55 |
| clarkb | rbergeron: ya the user-guide is gone I think | 16:55 |
| clarkb | rbergeron: there was a great doc reorganization that happened a cycle or two back where the docs were pushed down into the projects themselves rather than maintaining a bunch of separate documents | 16:56 |
| clarkb | AJaeger could probably explain it better than me | 16:56 |
| rbergeron | which seems to be what *used* to be around.. and matches (more or less, though the "last updated" dates are like, different) | 16:56 |
| rbergeron | clarkb: yeah, I get it, i think. It just made me wonder if it was "i should just make a pr" or something more complicated like "someone needs to do global search and replace to fix things to complete the string and point to the right spot" | 16:57 |
| rbergeron | (https://docs.openstack.org/user-guide/common/cli-manage-images.html#create-or-update-an-image-glance is in the RDO install guide, and goes to the same "look, all the guides" front page) | 16:58 |
| *** VW has quit IRC | 16:58 | |
| *** caphrim007 has joined #openstack-infra | 16:58 | |
| *** VW has joined #openstack-infra | 16:58 | |
| clarkb | rbergeron: what we have done is generate daily 404 counts at http://files.openstack.org/docs-404s/2018-02-16.txt | 16:59 |
| clarkb | the idea being people could work on appropriate redirects for old content which happened a bunch at the start but may have fallen off since then | 16:59 |
| rbergeron | Ahh -- very nice :) | 16:59 |
| * rbergeron nods, a useful new contributor type thing to do | 17:00 | |
| clarkb | in this particular case I think it would be great to have a change to glean to update the url to the new location | 17:00 |
| *** dhill__ has joined #openstack-infra | 17:01 | |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Support regexes in channel config yaml https://review.openstack.org/499367 | 17:02 |
| rbergeron | ack -- i will... attempt that today. :) mostly because was going to point someone at it and i went to read through it to figure out what to highlight for them, and that's when i found all the things | 17:02 |
| *** r-daneel has quit IRC | 17:02 | |
| * rbergeron is attempting mostly (at the heart of all this) to decipher the future of how fedora (and thusly, in the more distant future, centos / rhel) is composing images as they do this whole modularity thing | 17:03 | |
| rbergeron | since it seems to be slightly different and i got to reading a thing and was like "zomg i do hope someone maybe considers how other projects also have to build images around your stuff" | 17:03 |
| * rbergeron has out her magnifying glass | 17:03 | |
| fungi | glad someone is thinking about that! | 17:03 |
| *** slaweq has joined #openstack-infra | 17:04 | |
| *** rossella_s has quit IRC | 17:05 | |
| *** e0ne has quit IRC | 17:06 | |
| * fungi must venture out on an errand-like quest, but shall return forthwith | 17:06 | |
| rbergeron | spying those things is one of the great joys of having worked in or near a lot of projects. ;) | 17:06 |
| rbergeron | (but also means I'm not particularly great at solving it myself, but at least can put together humans and light a fire under behinds) | 17:07 |
| clarkb | rbergeron: fwiw at a super high level our image build process is make chroot then have yum/dnf install distro into it. Modify as necessary, copy and convert onto images | 17:07 |
| *** rossella_s has joined #openstack-infra | 17:07 | |
| clarkb | one of our modify as necessary steps is adding glean | 17:07 |
| *** sambetts is now known as sambetts|afk | 17:08 | |
| *** andreww has joined #openstack-infra | 17:10 | |
| *** felipemonteiro has quit IRC | 17:10 | |
| *** xarses_ has joined #openstack-infra | 17:11 | |
| gmmaha | fungi: AJaeger: thanks for looking into the issue for me | 17:11 |
| gmmaha | is there anything i can provide to help you narrow down.. f | 17:12 |
| clarkb | gema: if you are around I wanted to make usre you saw my proposed ptg schedule https://ethercalc.openstack.org/cvro305izog2 and that thursday afternoon worked for you to go over the aarch64 stuff | 17:12 |
| *** dougwig has joined #openstack-infra | 17:12 | |
| gema | clarkb: that works fine | 17:13 |
| gema | clarkb: thanks :D | 17:13 |
| clarkb | gmmaha: fungi AJaeger we can add support into zuul for specifying specific run phase timeouts and have those win over the main job timeout value | 17:13 |
| clarkb | we could treat post as special beacuse its there to clean up and recover | 17:13 |
| gema | clarkb: it's already on our schedule pinned, so we are good | 17:13 |
| clarkb | gema: good to kow thanks | 17:14 |
| clarkb | *know | 17:14 |
| gema | clarkb: next week we'll be doing the bring up of the new cloud in london, so we'll mostly be awol | 17:14 |
| gema | see you in dublin | 17:14 |
| clarkb | gema: see you there gl with the cloud deployment | 17:14 |
| *** derekh has quit IRC | 17:14 | |
| gema | thx | 17:14 |
| *** andreww has quit IRC | 17:14 | |
| *** jpich has quit IRC | 17:15 | |
| *** zoli is now known as zoli|gone | 17:15 | |
| *** zoli|gone is now known as zoli | 17:15 | |
| gmmaha | clarkb: I am not proficient with zuul to be able to comment on that. but it seems the jobs timing out with nothing in the logs to show was the thing that confused me the most. haven't seen that in the past | 17:15 |
| clarkb | gmmaha: ya we updated the timeout behavior to avoid having jobs take 6 hours to timeout on a 2 hour timeout but the side effect is if you timeout before post then post doesn't run :/ | 17:16 |
| gmmaha | clarkb: aaah.. :| | 17:18 |
| *** links has quit IRC | 17:18 | |
| openstackgerrit | Pino de Candia proposed openstack-infra/project-config master: Add new project for Tatu (SSH as a Service) Horizon Plugin. https://review.openstack.org/537653 | 17:21 |
| openstackgerrit | Pino de Candia proposed openstack-infra/project-config master: Zuul templates for new project tatu-dashboard. https://review.openstack.org/538307 | 17:21 |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Support regexes in channel config yaml https://review.openstack.org/499367 | 17:24 |
| *** vaidy has joined #openstack-infra | 17:25 | |
| fried_rice | jlvillal, smcginnis, fungi, clarkb, evrardjp: ^ rebased onto jlvillal's tests, and updated to use '^' prefix instead of use-regex-for-sections section. | 17:25 |
| fried_rice | mriedem: I seem to recall you cared about this a little bit too, once. ^ | 17:25 |
| *** isviridov_away has joined #openstack-infra | 17:25 | |
| evrardjp | oh great. | 17:26 |
| mriedem | i do care about it, but not enough to review it | 17:26 |
| evrardjp | s/./!/ | 17:26 |
| evrardjp | mriedem: how friendly! | 17:26 |
| evrardjp | :) | 17:26 |
| *** agopi_out has quit IRC | 17:26 | |
| mriedem | i am a friendly guy | 17:27 |
| *** gfidente is now known as gfidente|afk | 17:27 | |
| SpamapS | fungi: MySQL was removed from testing/stable by edict of release and security team. | 17:28 |
| SpamapS | Roamer`: ^ | 17:28 |
| SpamapS | You will in fact need to switch to libmariadbclient-dev, but it should include a compatability fix for -lmysqlclient to still work | 17:29 |
| SpamapS | (Be careful, libmariadbclient has extra symbols that aren't in libmysqlclient. ;) | 17:29 |
| SpamapS | It's the chief source of evil in that transition that I was sad never got addressed. | 17:29 |
| SpamapS | Making a new library, and stealing an established SONAME for it, is not ok. | 17:30 |
| SpamapS | Even if it is 99% compatible. | 17:30 |
| clarkb | where are we using libmysqlclient anymore? | 17:30 |
| clarkb | I thought we had pretty much completely transitioned to pymysql? | 17:30 |
| openstackgerrit | Pino de Candia proposed openstack-infra/project-config master: Add new project for Tatu (SSH as a Service) Horizon Plugin. https://review.openstack.org/537653 | 17:30 |
| openstackgerrit | Pino de Candia proposed openstack-infra/project-config master: Zuul templates for new project tatu-dashboard. https://review.openstack.org/538307 | 17:30 |
| clarkb | Roamer`: ^ you may want to double check that you need the dep in the first place. pymysql is a native python implementation that under real world ish testing performs at par or reasonably well to mysql-python | 17:34 |
| clarkb | its particularly happy with our eventlet based systems because it can be properly greenthreaded | 17:34 |
| *** giblet is now known as gibi_off | 17:38 | |
| *** bobh has quit IRC | 17:40 | |
| AJaeger | rbergeron: http://specs.openstack.org/openstack/docs-specs/specs/pike/os-manuals-migration.html is the spec that explain what happens with the guides | 17:45 |
| *** panda|odf is now known as panda|off | 17:48 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 17:49 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: use required enums to validate 'type' args https://review.openstack.org/545170 | 17:52 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: mark worklist filter_criteria as a required field https://review.openstack.org/545405 | 17:52 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: mark FilterCriterion title as a mandatory field https://review.openstack.org/545406 | 17:52 |
| *** harlowja has joined #openstack-infra | 17:55 | |
| *** lbragstad has quit IRC | 17:55 | |
| *** lbragstad has joined #openstack-infra | 17:56 | |
| pabelanger | AJaeger: dirk: clarkb: mordred: greghaynes: ianw https://review.openstack.org/545314/ and https://review.openstack.org/545183/ our the glean fixes needed to bring opensuse-tumbleweed DIBs and nodepool testing online. If you'd like to review. | 17:57 |
| *** hemna_ has quit IRC | 17:57 | |
| *** mriedem is now known as mriedem_lunch | 17:58 | |
| rbergeron | AJaeger: thx | 18:00 |
| AJaeger | thanks, pabelanger ! | 18:00 |
| *** hemna_ has joined #openstack-infra | 18:00 | |
| *** gyee has joined #openstack-infra | 18:00 | |
| *** caphrim007_ has joined #openstack-infra | 18:01 | |
| fungi | clarkb: it's entirely possible that the libmysqlclient-dev and mysql-devel entries in, e.g., nova's bindep.txt are cruft | 18:02 |
| AJaeger | anybody for a quick queens translations review, please? https://review.openstack.org/#/c/545299/ is a one-liner for heat-dashboard | 18:03 |
| *** r-daneel has joined #openstack-infra | 18:05 | |
| *** caphrim007 has quit IRC | 18:05 | |
| *** Swami has joined #openstack-infra | 18:06 | |
| *** olaph has quit IRC | 18:06 | |
| pabelanger | clarkb: other issue is ze02 ipv6 address is reporting wrong in openstack, do you remember how to refresh that? Otherwise, it think we'll need to delete / create ze02.o.o again | 18:07 |
| pabelanger | clarkb: dynamic inventory in ansible is trying old IP address, not address from DNS | 18:07 |
| *** olaph has joined #openstack-infra | 18:07 | |
| *** jpena is now known as jpena|off | 18:09 | |
| *** hemna_ has quit IRC | 18:09 | |
| *** salv-orlando has joined #openstack-infra | 18:10 | |
| *** bobh has joined #openstack-infra | 18:10 | |
| *** SumitNaiksatam has joined #openstack-infra | 18:11 | |
| *** hemna_ has joined #openstack-infra | 18:11 | |
| *** yamahata has quit IRC | 18:11 | |
| *** suhdood has joined #openstack-infra | 18:11 | |
| openstackgerrit | Merged openstack-infra/zuul master: Fix nodepool integration tests https://review.openstack.org/540983 | 18:12 |
| *** david-lyle has quit IRC | 18:12 | |
| mordred | evrardjp, cloudnull, odyssey4me: we landed a rather large feature in ansible 2.5 for the os_ modules - wanted to make sure y'all are aware and that it doesn't mess you up | 18:13 |
| mordred | pabelanger: you mean nova/neutron is returning the wrong address for the server? | 18:13 |
| cloudnull | mordred: nice! | 18:14 |
| cloudnull | do you have the pr ? | 18:14 |
| cloudnull | happy to give it a run through | 18:14 |
| AJaeger | pabelanger: I'm here again - please tell me once you find some minutes to discuss the bindep-fallback move | 18:14 |
| *** ihrachys has joined #openstack-infra | 18:14 | |
| *** bobh has quit IRC | 18:15 | |
| mordred | cloudnull: https://github.com/ansible/ansible/pull/20974 | 18:16 |
| mordred | cloudnull: tl;dr - made a centralized constructor and added the ability to pass an entire cloud config to each module invocation | 18:17 |
| mordred | cloudnull: so if you have your config as ansible variables and not clouds.yaml - you can just pass a dict to the cloud: parameter of all of the os_ modules now | 18:17 |
| cloudnull | very cool | 18:18 |
| mordred | cloudnull: also, fwiw, we're live with zuul reporting on os_ module PRs and running shade integration tests on them | 18:18 |
| pabelanger | mordred: yah, it is the old address. I seem to remember citycloud caching that too, but cannot remember how I fixed it. | 18:18 |
| pabelanger | mordred: maybe you have an idea | 18:18 |
| cloudnull | we have a gate job to test the head of ansible so we should be picking up this change when we check experimental but I'll give that a run through and adjust where needed | 18:18 |
| cloudnull | thanks for the heads up mordred | 18:18 |
| mordred | pabelanger: this is for ze02 in rax yeah? | 18:18 |
| cloudnull | change looks great too :) | 18:19 |
| pabelanger | mordred: yup | 18:19 |
| mordred | cloudnull: awesome. we could also wire up your head-of-ansible job | 18:19 |
| cloudnull | that'd be great -cc evrardjp | 18:19 |
| mordred | pabelanger: then no, I don't have any ideas on that one ... in other contexts it could be a nova/neutron cache mismatch and we solve it by having shade pull addresses from neutron instead ... but there is no available neutron api at rax | 18:20 |
| *** hemna_ has quit IRC | 18:20 | |
| pabelanger | k | 18:20 |
| *** VW has quit IRC | 18:22 | |
| *** VW has joined #openstack-infra | 18:23 | |
| fungi | i'm a fan of just rebuilding it | 18:23 |
| *** VW has quit IRC | 18:23 | |
| fungi | we have 9 more, we won't miss it for however long it's being swapped out | 18:24 |
| *** bobh has joined #openstack-infra | 18:24 | |
| *** VW has joined #openstack-infra | 18:24 | |
| *** hemna_ has joined #openstack-infra | 18:25 | |
| clarkb | ya I don't know how to get openstack to realize the ip changed | 18:25 |
| *** ralonsoh_ has quit IRC | 18:26 | |
| *** tesseract has quit IRC | 18:26 | |
| *** rossella_s has quit IRC | 18:28 | |
| mordred | cloudnull, evrardjp: https://review.openstack.org/545430 Have zuul check out ansible for devel AIO job | 18:29 |
| *** dtantsur is now known as dtantsur|afk | 18:29 | |
| *** rossella_s has joined #openstack-infra | 18:30 | |
| odyssey4me | mordred ooh, shiny things | 18:33 |
| *** trown is now known as trown|lunch | 18:33 | |
| pabelanger | fungi: clarkb: okay, I'll get started here shortly | 18:34 |
| mordred | odyssey4me: I *think* I remember you asking for something similar in the past | 18:34 |
| odyssey4me | currently my brain is the equivalent to a bag of mushy peas, you could tell me anything and I'd believe you | 18:35 |
| pabelanger | okay, zuul-executor stopped on ze02 | 18:35 |
| odyssey4me | mordred thanks for the demo - I think that's going to spark some ideas for evrardjp :) | 18:36 |
| *** dhajare|mtg has joined #openstack-infra | 18:36 | |
| *** eroux has quit IRC | 18:36 | |
| *** r-daneel has quit IRC | 18:37 | |
| openstackgerrit | Merged openstack-infra/project-config master: Setup queens translation sync for heat-dashboard https://review.openstack.org/545299 | 18:37 |
| *** r-daneel has joined #openstack-infra | 18:38 | |
| mordred | odyssey4me: woot! also wanted y'all to be aware that we've got integration testing of os_ modules going now - which means if we find breaks or holes we should be sure to not only fix them but to add tests too | 18:38 |
| fungi | and you can depends-on their pull requests, right? | 18:38 |
| *** agopi_out has joined #openstack-infra | 18:39 | |
| fungi | (if you're upstreaming a fix to ansible that you subsequently want to use in a gerrit change) | 18:39 |
| *** jamesdenton has quit IRC | 18:39 | |
| *** psachin has joined #openstack-infra | 18:41 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add stable/queens to periodic-stable templates https://review.openstack.org/545268 | 18:43 |
| *** olaph1 has joined #openstack-infra | 18:43 | |
| *** harlowja has quit IRC | 18:44 | |
| *** olaph has quit IRC | 18:44 | |
| beekneemech | Hey, how would I get access to change the topic in #openstack-oslo? It's stuck on a meeting topic even though the meeting is over. | 18:46 |
| *** harlowja has joined #openstack-infra | 18:48 | |
| pabelanger | beekneemech: #endmeeting should work after 1 hour after start I believe | 18:48 |
| fungi | odds are the meetbot got disconnected/restarted mid-meeting and left a stale topic behind | 18:48 |
| pabelanger | that is possible too | 18:48 |
| fungi | as ptl of oslo, i have no qualms adding you to the access list for that channel | 18:48 |
| beekneemech | fungi: That would be good, thanks. | 18:49 |
| *** weshay is now known as weshay|bbiab | 18:50 | |
| fungi | beekneemech: you should now be able to /msg chanserv #openstack-oslo topic blah blah blah shared libraries blah | 18:50 |
| beekneemech | I think we may have had a global status update during the meeting, so when it went back to normal it reset to the meeting topic. | 18:50 |
| *** agopi_out has quit IRC | 18:50 | |
| *** ldnunes has quit IRC | 18:50 | |
| fungi | beekneemech: ahh, yes that also sounds likely. we stopped having statusbot hang out in our official meeting channels for precisely that reason | 18:50 |
| *** ldnunes has joined #openstack-infra | 18:51 | |
| fungi | having two bots trying to change channel topics and remember the previous topic is a recipe for topic races | 18:51 |
| *** jbadiapa has joined #openstack-infra | 18:52 | |
| openstackgerrit | Chris Hoge proposed openstack-infra/puppet-refstack master: Added option to set enable_anonymous_upload https://review.openstack.org/544145 | 18:53 |
| fungi | hogepodge: ^ is that urgent to get merged in advance of the maintenance? | 18:54 |
| *** r-daneel has quit IRC | 18:54 | |
| * fungi reads to see if it says | 18:55 | |
| *** lpetrut has joined #openstack-infra | 18:55 | |
| fungi | doesn't say | 18:55 |
| fungi | mguiney: ^ perhaps you know? | 18:55 |
| fungi | i guess that's the "push new code with anonymous upload turned off" from the maintenance plan you sent me? | 18:56 |
| *** olaph1 is now known as olaph | 18:57 | |
| *** ykarel|away has quit IRC | 18:58 | |
| *** r-daneel has joined #openstack-infra | 18:58 | |
| mguiney | I believe so | 18:59 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 19:02 |
| *** yamahata has joined #openstack-infra | 19:05 | |
| *** camunoz has quit IRC | 19:07 | |
| *** florianf has quit IRC | 19:08 | |
| *** harlowja has quit IRC | 19:10 | |
| *** hemna_ has quit IRC | 19:11 | |
| *** lucasagomes is now known as lucas-afk | 19:12 | |
| *** lpetrut has quit IRC | 19:12 | |
| *** david-lyle has joined #openstack-infra | 19:13 | |
| *** hemna_ has joined #openstack-infra | 19:15 | |
| *** gyee has quit IRC | 19:16 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 19:20 |
| *** lpetrut has joined #openstack-infra | 19:20 | |
| *** sambetts|afk has quit IRC | 19:22 | |
| *** camunoz has joined #openstack-infra | 19:22 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 19:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 19:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 19:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 19:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 19:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 19:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 19:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 19:23 |
| AJaeger | corvus: I followed your idea from yesterday and used protected for the static publish jobs - could you review https://review.openstack.org/545234 , please? | 19:23 |
| *** sambetts_ has joined #openstack-infra | 19:24 | |
| pabelanger | okay, replacement ze02 is online, DNS is updated. Just waiting for firewalls to reload now | 19:29 |
| dmsimard | pabelanger: yes, I can take care of deleting the old logserver and volumes | 19:30 |
| *** olaph1 has joined #openstack-infra | 19:30 | |
| pabelanger | dmsimard: great, thanks | 19:30 |
| corvus | AJaeger: lgtm; mordred you may want to look at https://review.openstack.org/545234 and its parent | 19:31 |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 19:32 |
| *** trown|lunch is now known as trown | 19:32 | |
| *** olaph has quit IRC | 19:32 | |
| AJaeger | corvus, one question about publish-openstack-artifacts - it has access to secrets and I see it inherited in mistral. Is that ok? Or should we add protected to publish-openstack-artifacts as well? | 19:32 |
| *** suhdood_ has joined #openstack-infra | 19:32 | |
| AJaeger | won't work with major reorg - the job is inherited in openstack-zuul-jobs as well | 19:33 |
| *** suhdood_ has quit IRC | 19:34 | |
| AJaeger | I'm talking about http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/jobs.yaml#n124 ... | 19:35 |
| *** suhdood has quit IRC | 19:35 | |
| *** kjackal has joined #openstack-infra | 19:37 | |
| *** bobh has quit IRC | 19:38 | |
| *** suhdood has joined #openstack-infra | 19:39 | |
| *** dhajare|mtg has quit IRC | 19:39 | |
| smcginnis | So kata containers will not be using Infra? | 19:40 |
| fungi | depends on what you mean by that | 19:41 |
| fungi | they're already using us to host their mailing lists | 19:41 |
| smcginnis | I assumed part of joining the foundation was also to use infra for running CI. | 19:41 |
| fungi | we present it as an option, but it's in no way mandated | 19:42 |
| gmmaha | clarkb: AJaeger: fungi: Just updated a patch https://review.openstack.org/#/c/539019/ and zuul has hit a time out again. Just want to see if it something that i am doing with the patches or if there is anyway i can help to fix it. | 19:42 |
| fungi | smcginnis: they can choose to when they're ready to do so | 19:42 |
| *** pcichy has quit IRC | 19:42 | |
| smcginnis | fungi: Looks like they aren't starting off with it: http://lists.katacontainers.io/pipermail/kata-dev/2018-February/000026.html | 19:42 |
| fungi | smcginnis: that was my understanding as well | 19:42 |
| *** dprince has quit IRC | 19:43 | |
| mordred | I've got to say, the link "http://kata-jenkins-ci.westus2.cloudapp.azure.com/" is disappointing on so many levels | 19:43 |
| smcginnis | :) | 19:43 |
| fungi | i'm happy to see that they at least think testing their software is a good idea | 19:44 |
| *** mriedem_lunch is now known as mriedem | 19:44 | |
| fungi | but yeah, not exactly my preferred toolchain | 19:44 |
| mordred | "we joined the OpenStack Foundation, which produces Open Source Cloud software as well as an advanced and scalable CI platform, but we've decided to deploy a jenkins server on a non-open source cloud" | 19:45 |
| clarkb | gmmaha: I am working in a change to zuul to change how timeouts work in post | 19:45 |
| clarkb | gmmaha: once we get that in and restart zuul it should allow your joba to timeout then still collect logs | 19:45 |
| gmmaha | clarkb: woot woot.. thanks a lot for looking into this. | 19:45 |
| gmmaha | shall wait and watch for that to happen | 19:45 |
| pabelanger | mordred: yah, noticed that too | 19:46 |
| pabelanger | I'm going to try an sit in some of the kata PTG sessions, but haven't see much about it yet | 19:46 |
| mordred | like, shrug and everything, I've got enough problems in my world and nothing said they *had* to do anything - but from a feels perspective it's not great | 19:46 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Protect some jobs https://review.openstack.org/545442 | 19:47 |
| AJaeger | corvus, mordred, one more proteced/abstract change ^ | 19:47 |
| corvus | demoralizing is the word that comes to mind | 19:47 |
| mordred | especially since the CI they're spinning up is clearly NEW - so it's not like they've got the whole "we have an existing something else and migrating would be hard" | 19:47 |
| pabelanger | +1 | 19:47 |
| mordred | corvus: ++ | 19:47 |
| fungi | gmmaha: as for what changed, my guess is that openstack-helm-infra-{ubuntu,centos,fedora} were probably taking close to 30 minutes already with their run phase playbooks, and so switching to putting all the phases together under that timeout has caused you to need a slightly longer timeout on those | 19:47 |
| fungi | gmmaha: and indeed, before this your average runtimes on those jobs look to have been around 40 minutes total | 19:48 |
| gmmaha | fungi: aaah.. thanks for that explanation.. now i understand better. is the timeout something i can configure in the zuul.yaml of openstack-helm-infra repo? | 19:49 |
| AJaeger | I don't know where the kata folks discuss those things (note I'm aware of their channels, didn't follow them). I'm on the dev mailing list and there is very low traffic and no discussion about any of this. | 19:49 |
| * gmmaha goes to see if there is an option | 19:49 | |
| pabelanger | FWIW: https://github.com/kata-containers/ci/issues/4 appears to be their discussion about CI | 19:49 |
| *** weshay|bbiab is now known as weshay | 19:50 | |
| fungi | seems like a jolly place to have an infrastructure discussion ;) | 19:50 |
| *** harlowja has joined #openstack-infra | 19:50 | |
| persia | https://github.com/WorksOnArm/cluster/issues/31 is something else I encountered recently related to their CI | 19:51 |
| fungi | gmmaha: yes, you can set your job timeout explicitly either in the job definition or in a variant | 19:51 |
| fungi | getting the appropriate documentation link now | 19:52 |
| fungi | gmmaha: https://docs.openstack.org/infra/zuul/user/config.html?highlight=timeout#attr-job.timeout | 19:53 |
| corvus | fungi, gmmaha: if you bump the timeout, you may want to track clarkb's change and lower it once it's in production (otherwise, you're likely to end up with a higher-than-anticipated timeout on the main job). | 19:53 |
| *** VW_ has joined #openstack-infra | 19:53 | |
| fungi | agreed | 19:53 |
| *** amoralej is now known as amoralej|off | 19:53 | |
| gmmaha | fungi: thanks.. | 19:53 |
| *** VW_ has quit IRC | 19:53 | |
| *** VW_ has joined #openstack-infra | 19:54 | |
| gmmaha | fungi: corvus: given that clarkb is already working on his patch, will wait and see what it lends and then figure out if need to increase the timeout. | 19:54 |
| fungi | sounds sane | 19:54 |
| *** ansmith has quit IRC | 19:54 | |
| *** olaph1 is now known as olaph | 19:56 | |
| *** VW has quit IRC | 19:57 | |
| *** anticw has joined #openstack-infra | 19:58 | |
| anticw | re: https://review.openstack.org/Documentation/user-search.html#_negation ... negation in searches still doesn't work | 19:58 |
| anticw | searching for -foo ... seems to search for "-foo" i guess, certainly it's not excluding things in expected ways | 19:58 |
| *** caphrim007_ has quit IRC | 19:58 | |
| AJaeger | pabelanger: I'll sign off in a bit - would appreciate if you could again look at https://review.openstack.org/543139 and see mordred's and my comments. | 19:59 |
| clarkb | anticw: -message:foo iirc | 19:59 |
| *** njohnston_ has joined #openstack-infra | 19:59 | |
| pabelanger | AJaeger: will do | 19:59 |
| AJaeger | thanks! | 20:00 |
| *** bobh has joined #openstack-infra | 20:01 | |
| *** dprince has joined #openstack-infra | 20:02 | |
| *** r-daneel_ has joined #openstack-infra | 20:02 | |
| *** salv-orlando has quit IRC | 20:03 | |
| *** salv-orlando has joined #openstack-infra | 20:04 | |
| *** caphrim007 has joined #openstack-infra | 20:04 | |
| *** r-daneel has quit IRC | 20:05 | |
| *** r-daneel_ is now known as r-daneel | 20:05 | |
| openstackgerrit | Merged openstack-infra/grafyaml master: Unlock yaxes min limit and allow free value https://review.openstack.org/545006 | 20:05 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Allow repeating panels and to set minspan https://review.openstack.org/545062 | 20:05 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Allow decimals to be specificed to limit decimals graphs https://review.openstack.org/545112 | 20:05 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Allow hidden templated vars https://review.openstack.org/542979 | 20:05 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Add more units for graph panel https://review.openstack.org/543937 | 20:05 |
| *** salv-orlando has quit IRC | 20:08 | |
| *** caphrim007 has quit IRC | 20:09 | |
| openstackgerrit | Merged openstack-infra/grafyaml master: Add links to GrafYaml https://review.openstack.org/542395 | 20:09 |
| *** pcichy has joined #openstack-infra | 20:09 | |
| anticw | clarkb: thanks, let me try that ... in the past -foo worked, i didn't think message: was needed | 20:11 |
| *** caphrim007 has joined #openstack-infra | 20:11 | |
| *** caphrim007 has quit IRC | 20:11 | |
| anticw | clarkb: great, that works well! thanks | 20:12 |
| *** caphrim007 has joined #openstack-infra | 20:12 | |
| *** ijw has joined #openstack-infra | 20:12 | |
| *** eharney has quit IRC | 20:13 | |
| dhellmann | can anyone help diagnose this doc build error with the message "remote copy does not support recursive copy of directory"? http://logs.openstack.org/ce/ceab113c77bec6b218cac28665684d20d7f8ef76/release/publish-openstack-sphinx-docs-infra/bede315/job-output.txt.gz#_2018-02-16_19_57_04_366966 | 20:15 |
| *** lpetrut has quit IRC | 20:16 | |
| dhellmann | as far as I know there is nothing special about the python-storyboardclient documentation build | 20:16 |
| dhellmann | and in fact older versions of the docs are available at https://docs.openstack.org/infra/python-storyboardclient/ | 20:16 |
| fungi | i wonder are we doing copy instead of sync in the publish-openstack-sphinx-docs-infra job? | 20:16 |
| dhellmann | oh, I wonder if it's because -- yeah, different job | 20:17 |
| *** ldnunes has quit IRC | 20:17 | |
| *** lpetrut has joined #openstack-infra | 20:17 | |
| fungi | it mostly exists to root the infra documentation under infra/ but it's possible improvements were made to the publish-openstack-sphinx-docs job and not publish-openstack-sphinx-docs-infra | 20:17 |
| fungi | i'm digging the source up now | 20:18 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/python-storyboardclient master: build universal wheels https://review.openstack.org/545446 | 20:19 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added logic to manage calendar sync error due credentials revocation. https://review.openstack.org/545166 | 20:23 |
| hogepodge | fungi yes, we can't update the server without that | 20:24 |
| fungi | hogepodge: just wondering whether it should merge before the maintenance or during | 20:25 |
| pabelanger | okay, ze02.o.o is back online | 20:25 |
| *** VW_ has quit IRC | 20:25 | |
| hogepodge | fungi: what's your preference? We can hold off on the upgrade. We're not on a hard schedule if maintenance is a better time for it. | 20:25 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/python-storyboardclient master: remove sphinx-isms from README.rst and clean up docs https://review.openstack.org/545447 | 20:25 |
| *** VW has joined #openstack-infra | 20:25 | |
| fungi | hogepodge: i have no preference... i was asking when you want that change approved | 20:26 |
| fungi | so that we do it at the time you need it done | 20:26 |
| fungi | i didn't want to have it merge earlier than you intended and cause some sort of disruption for the service | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 20:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 20:26 |
| fungi | hogepodge: perhaps i can rephrase... is it safe to approve that change now or do we need to wait until you're ready for the other steps in your maintenance plan? | 20:27 |
| fungi | dhellmann: strange, i don't see any obvious difference between that task for the prepare-infra-docs-for-afs and prepare-docs-for-afs roles | 20:28 |
| fungi | http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/prepare-docs-for-afs/tasks/tagged.yaml#n6 vs http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/prepare-infra-docs-for-afs/tasks/tagged.yaml#n6 | 20:29 |
| pabelanger | #status log replacement ze02.o.o server is online and processing jobs | 20:30 |
| openstackstatus | pabelanger: finished logging | 20:30 |
| *** VW has quit IRC | 20:30 | |
| dhellmann | fungi : is that the task that failed? | 20:30 |
| fungi | dhellmann: that's the one mentioned in the log | 20:30 |
| dhellmann | yeah, seems to be | 20:30 |
| fungi | i wonder if this job is supposed to only be running on tagged releases and not on change merges, per the comment in that role | 20:31 |
| dhellmann | fungi : it does mention "remote copy" so I wonder if for one job it runs locally and for another it runs remotely? | 20:31 |
| dhellmann | fungi : this build was triggered by a tagged release | 20:31 |
| fungi | ahh, okay | 20:31 |
| dhellmann | so maybe it's *not* supposed to run for tags? | 20:31 |
| *** olaph1 has joined #openstack-infra | 20:31 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 20:32 |
| fungi | clarkb: ^ you had been looking into updating docs on release for infra projects, right? | 20:32 |
| openstackgerrit | Merged openstack-infra/zuul master: Make NodeSetParser a regular class https://review.openstack.org/545148 | 20:32 |
| openstackgerrit | Merged openstack-infra/zuul master: Make SecretParser a regular class https://review.openstack.org/545149 | 20:32 |
| *** olaph has quit IRC | 20:32 | |
| *** njohnston_ has quit IRC | 20:33 | |
| *** salv-orlando has joined #openstack-infra | 20:34 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added logic to manage calendar sync error due credentials revocation. https://review.openstack.org/545166 | 20:35 |
| *** VW has joined #openstack-infra | 20:36 | |
| *** VW has joined #openstack-infra | 20:37 | |
| Roamer` | clarkb, about libmysqlclient: I was just trying to run the tox tests of openstack/requirements. I guess something in OpenStack still needs libmysqlclient, or, alternatively, all the dependencies have been ripped out but no one has been keeping a reference count. | 20:37 |
| *** agopi_out has joined #openstack-infra | 20:39 | |
| clarkb | Roamer`: ya I think both are possibilities. Its a housekeepign thing we should try to remember to look back into at some point | 20:41 |
| *** andreas_s has joined #openstack-infra | 20:43 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 20:43 |
| *** camunoz has quit IRC | 20:43 | |
| hogepodge | fungi: It's necessary to approve the change now | 20:44 |
| clarkb | gmmaha: ^ fyi I think that should work and has tests now | 20:44 |
| hogepodge | Our automatic reload on tag won't load properly without it | 20:44 |
| hogepodge | (not that it will fail, it just won't be able to set the options we need on the server) | 20:45 |
| fungi | hogepodge: cool, the lack of timeline requirements in the commit message or review comments led me to imagine all sorts of possibilities for that since it wasn't clear it was safe to approve in advance | 20:46 |
| fungi | any infra-puppet-core reviewers willing to review https://review.openstack.org/544145 as prep for today's refstack maintenance? | 20:47 |
| clarkb | yes I'll review here shortly | 20:47 |
| clarkb | unless you need a second non foundation staff member? | 20:47 |
| *** andreas_s has quit IRC | 20:47 | |
| fungi | might be good to get a non-osf core reviewer on it too since hogepodge is... yeah | 20:47 |
| hogepodge | :-) | 20:48 |
| hogepodge | fungi: can you leave a comment about more verbose messages on the review? As a learning experience. | 20:48 |
| fungi | you bet! | 20:48 |
| *** HeOS has quit IRC | 20:50 | |
| *** agopi_out has quit IRC | 20:51 | |
| *** gfidente|afk has quit IRC | 20:51 | |
| *** olaph1 is now known as olaph | 20:51 | |
| corvus | done | 20:53 |
| fungi | thanks corvus! | 20:53 |
| *** markvoelker_ has quit IRC | 20:55 | |
| *** markvoelker has joined #openstack-infra | 20:56 | |
| *** VW_ has joined #openstack-infra | 20:57 | |
| *** VW_ has quit IRC | 20:58 | |
| *** VW_ has joined #openstack-infra | 20:58 | |
| *** ijw has quit IRC | 20:58 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 20:58 |
| *** agopi_out has joined #openstack-infra | 20:58 | |
| *** kjackal has quit IRC | 20:58 | |
| *** VW has quit IRC | 21:00 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: mark worklist filter_criteria as a required field https://review.openstack.org/545405 | 21:00 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: mark FilterCriterion title as a mandatory field https://review.openstack.org/545406 | 21:01 |
| pabelanger | fungi: clarkb: mordred: ianw: do you mind looking at glean patch to fix opensuse / fedora: https://review.openstack.org/545314/ biggest change is vendoring of library. | 21:02 |
| *** ijw has joined #openstack-infra | 21:02 | |
| clarkb | looks like the license is compatibile and we've clearly stuck it in a _vendor path so thats good | 21:03 |
| fungi | pabelanger: i might be able to in a little while but about to step out to grab some food | 21:04 |
| *** eharney has joined #openstack-infra | 21:05 | |
| *** sambetts_ has quit IRC | 21:06 | |
| *** pcrews has quit IRC | 21:06 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 21:07 |
| *** pcrews has joined #openstack-infra | 21:08 | |
| *** r-daneel has quit IRC | 21:10 | |
| clarkb | pabelanger: are the jobs that ran on that change running with the ssh check? they must be right? | 21:10 |
| clarkb | pabelanger: just thinking the ssh check will give us a reasonably testing of this end to end | 21:11 |
| openstackgerrit | Merged openstack-infra/puppet-refstack master: Added option to set enable_anonymous_upload https://review.openstack.org/544145 | 21:11 |
| pabelanger | clarkb: yah, the tests in 545314 show SSh working, after we bring all OSes online | 21:11 |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 21:13 |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Test Gerrit message filtering https://review.openstack.org/545381 | 21:15 |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Support regexes in channel config yaml https://review.openstack.org/499367 | 21:15 |
| fried_rice | jlvillal: ^ | 21:15 |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: [WIP] docs sitemap generation automation https://review.openstack.org/524862 | 21:20 |
| openstackgerrit | Merged openstack-infra/zuul master: Add post-timeout setting https://review.openstack.org/545439 | 21:23 |
| *** kjackal has joined #openstack-infra | 21:24 | |
| openstackgerrit | Merged openstack-infra/zuul master: Add host/group vars https://review.openstack.org/544562 | 21:27 |
| *** kjackal has quit IRC | 21:32 | |
| *** trown is now known as trown|brb | 21:33 | |
| *** trown|brb is now known as trown|outtypewww | 21:33 | |
| *** jcoufal has quit IRC | 21:35 | |
| openstackgerrit | Frank Kloeker proposed openstack-infra/openstack-zuul-jobs master: docs sitemap generation automation https://review.openstack.org/524862 | 21:35 |
| corvus | infra-root: in order to pick up the 2 most recent zuul changes, we should restart the scheduler and executors together. | 21:38 |
| *** suhdood has quit IRC | 21:39 | |
| clarkb | I'm around to help with that. Also have the refstack thing in a 1.3 hours | 21:41 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/python-storyboardclient master: remove sphinx-isms from README.rst and clean up docs https://review.openstack.org/545447 | 21:42 |
| gmmaha | clarkb: thanks a lot for the patch.. let me go aehad and recheck the job and see how that foes. | 21:42 |
| gmmaha | goes* | 21:42 |
| corvus | clarkb: now looks like a fine time to do so, let's do it | 21:43 |
| corvus | clarkb: you want to do the scheduler while i do executors? | 21:43 |
| clarkb | gmmaha: you'll need to wait for things to be restarted | 21:43 |
| clarkb | corvus: sure. I will need to save queues, stop scheduler and web? then start scheduler and web and restore queues? | 21:44 |
| corvus | clarkb: yep | 21:44 |
| clarkb | corvus: I'll need to figure out process for saving queues | 21:44 |
| clarkb | python /opt/zuul/tools/zuul-changes.py https://zuul.openstack.org openstack check >check.sh | 21:44 |
| clarkb | corvus: ^ that look right for saving them? then I guess executre check.sh to restore them | 21:45 |
| corvus | clarkb: yep | 21:45 |
| clarkb | corvus: also do you want me to wait for the executors before starting the scheduler or just restart it? | 21:45 |
| gmmaha | clarkb: duh.. somehow i thought it was already done since i saw the patch merged.. blame it on friday | 21:45 |
| clarkb | I should also check that pip has installed the newer version | 21:45 |
| corvus | clarkb: let's at least wait until they're stopped | 21:45 |
| corvus | all executors are at zuul==2.6.1.dev614 | 21:46 |
| clarkb | zuul is up to date on zuul.o.o according to pbr freeze | 21:46 |
| clarkb | zuul==2.6.1.dev614 # git sha 9a21ad8 | 21:46 |
| *** hashar has quit IRC | 21:47 | |
| corvus | clarkb: i think you're clear to save and stop the scheduler; let me know when that's done | 21:47 |
| clarkb | ok starting now | 21:47 |
| clarkb | corvus: do I need to stop -web and -fingergw too? | 21:48 |
| clarkb | I guess it can't hurt | 21:48 |
| corvus | clarkb: can't hurt | 21:48 |
| *** r-daneel has joined #openstack-infra | 21:48 | |
| clarkb | I've told init to stop all three, the scheduler is the only one still running | 21:49 |
| clarkb | and now its completely stopped | 21:49 |
| corvus | stopping execs | 21:49 |
| corvus | ansible-playboo is the best process name | 21:52 |
| *** dprince has quit IRC | 21:52 | |
| andreaf | corvus I started reviewing https://review.openstack.org/#/c/522054/ so far so good - I'll finish the review over the weekend hopefully | 21:52 |
| corvus | they're all stopped except ze02 | 21:53 |
| corvus | which apparently was running 2 or something weird like that | 21:53 |
| clarkb | just let me know when you are ready to have the scheduler restarted and whenI should reenqueue things | 21:53 |
| *** sree has joined #openstack-infra | 21:54 | |
| corvus | clarkb: i killed ze02 manually. you can start now | 21:54 |
| clarkb | ok | 21:54 |
| corvus | clarkb: i'm going to clean up ze02 a bit before i start the executors, but no need to wait on that | 21:54 |
| pabelanger | odd, I just launched ze02. perhaps I did something wrong | 21:54 |
| pabelanger | s/launched/created | 21:55 |
| clarkb | all three services are starting now | 21:55 |
| clarkb | should I reenqueue or wait? | 21:55 |
| corvus | clarkb: i usually tail the log file to wait until it's loaded the config. you can re-enqueue at any point, but it won't proceed past the first item until it's running anyway. | 21:55 |
| clarkb | right now its doing a bunch of git uplaod packs I think to pull the config down | 21:56 |
| corvus | yeah, that's learning the branches so it can ask the mergers to get the config | 21:57 |
| clarkb | I'll have to push up a change to set a post-timeout to our base job too | 21:57 |
| clarkb | ah ya now there are cat jobs | 21:57 |
| *** sree has quit IRC | 21:59 | |
| *** panda|off is now known as panda | 21:59 | |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Refactor ChannelConfig with channels_for https://review.openstack.org/545469 | 21:59 |
| clarkb | corvus: if parent isn't specified is it implied to be base? | 21:59 |
| corvus | clarkb: yep | 21:59 |
| fried_rice | jlvillal: There's that refactor to move _channels_for into ChannelConfig | 21:59 |
| *** eernst has joined #openstack-infra | 22:00 | |
| *** lpetrut has quit IRC | 22:00 | |
| *** edmondsw has quit IRC | 22:01 | |
| corvus | starting executors | 22:01 |
| clarkb | ok I'm enqueing things now | 22:01 |
| *** edmondsw has joined #openstack-infra | 22:01 | |
| jlvillal | fried_rice, Thanks. I'll look at it in a bit. | 22:01 |
| *** VW_ has quit IRC | 22:02 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Set post-timeout value to half an hour on base jobs https://review.openstack.org/545470 | 22:02 |
| corvus | clarkb: looks like things are running again | 22:03 |
| clarkb | corvus: pabelanger ^ that should set our post timeout to 30 minutes by default | 22:03 |
| clarkb | check is still reenquing | 22:03 |
| *** VW has joined #openstack-infra | 22:03 | |
| corvus | andreaf: great, thanks! also, host/group vars just went live | 22:03 |
| pabelanger | wfm | 22:03 |
| clarkb | gmmaha: you can recheck things now fi they aren't already getting enqueued and we should see logs are collected despite timing out (if it does even manage to timeout) | 22:04 |
| *** psachin has quit IRC | 22:04 | |
| gmmaha | clarkb: thanks.. i already did the recheck and waiting to see what happens with zuul.. | 22:04 |
| clarkb | and reenqueue is done | 22:05 |
| *** sboeuf has joined #openstack-infra | 22:06 | |
| *** edmondsw has quit IRC | 22:06 | |
| *** VW_ has joined #openstack-infra | 22:06 | |
| *** VW has quit IRC | 22:06 | |
| pabelanger | clarkb: corvus: I'm no longer seeing available ram update on http://grafana.openstack.org/dashboard/db/zuul-status | 22:06 |
| corvus | pabelanger: then we're ready to merge this: https://review.openstack.org/544746 | 22:07 |
| andreaf | corvus: oh yeah? that's great I'll try to make some multinode patches that use that as well | 22:07 |
| clarkb | corvus: approved | 22:07 |
| pabelanger | corvus: Ha, +3 | 22:08 |
| *** fultonj has quit IRC | 22:08 | |
| sboeuf | mordred: hey ! I have seen your comments about Kata CI on the log. First, we have an issue opened here, and we expect the maximum of people to give their inputs about that but didn't see much activity about it. | 22:09 |
| mordred | sboeuf: heya! I was just working on a response to the issue | 22:09 |
| mordred | sboeuf: so - I'm now aware of it and will be happily engaging with the discussion :) | 22:10 |
| sboeuf | mordred: And then the explanation for Azure is that with Clear Containers (former project) we were using it for our CI. As we want to make sure we can test as quick as possible, we reused this solution | 22:10 |
| sboeuf | mordred: cool | 22:10 |
| sboeuf | mordred: but we're not against Infra, we just needed something up and running pretty quickly | 22:10 |
| sboeuf | that;s it | 22:11 |
| mordred | sboeuf: totally, and makes sense. | 22:11 |
| eernst | (fwiw, had been using azure for a few months) | 22:11 |
| *** suhdood has joined #openstack-infra | 22:11 | |
| jlvillal | clarkb, Not sure if you want to review: https://review.openstack.org/#/c/545128/5 as it is the parent patch of fried_rice's two patches that you gave +2s to. | 22:11 |
| clarkb | jlvillal: ya was slowly working my way backwards up that stack then got distracted by zuul updates | 22:12 |
| fried_rice | clarkb: FYI I'm still +1 on that guy - the talk about py3 and six is non blocking. | 22:12 |
| mordred | sboeuf, eernst: yah - I think we're in a good position (and the timing is actually pretty good now) to be able to connect some things up to y'all and start to make some incremental progress together | 22:12 |
| sboeuf | mordred: interesting, I look forward to your proposal on the github issue ! | 22:13 |
| clarkb | fwiw I find github issues in many different repositories hard to track | 22:13 |
| eernst | Yeah, any information/proposals/details on what's available is more than welcome | 22:13 |
| mordred | sboeuf, eernst: one of the outcomes of which I hope will be an azure plugin for nodepool - but I don't think that'll need to be a first thing done | 22:13 |
| clarkb | this is IMO exactly what the mailing list should be used for | 22:13 |
| *** rlandy|rover has quit IRC | 22:13 | |
| mordred | eernst, sboeuf: you say that now - but you havne't yet had to read one of my long-rambling text dumps :) | 22:14 |
| eernst | Sure, that's good feedback. | 22:14 |
| eernst | clarkb: ^ most of the conversation is on slack and github at this point, especially since its pretty early on | 22:14 |
| *** slaweq has quit IRC | 22:15 | |
| clarkb | the problem is github isn't really built for having conversattions | 22:15 |
| clarkb | the issue tracker is for tracking software defects | 22:15 |
| clarkb | mailing lists on the other hand are much better at this sort of conversation (and easier for people not involved 100% to follow along) | 22:15 |
| eernst | Thanks for the feedback | 22:15 |
| sboeuf | clarkb: yeah i get it, but at the same time you can give more details and it's easier to read thanks to the md format | 22:15 |
| clarkb | sboeuf: we'll have to disagree on that one :) I find plain text email the simplest thing to ready because its so simple :) | 22:16 |
| mordred | clarkb, sboeuf, eernst: https://github.com/kata-containers/ci/issues/4 <-- (very long) comment added | 22:17 |
| eernst | Anyway, clarkb, mordred -- if there's a way to get educated on what is available, I think all the kata members'll like the feedback. | 22:17 |
| eernst | so we better understand what's available. | 22:17 |
| mordred | eernst: ++ | 22:17 |
| clarkb | eernst: looks like mordred just did a large braindump on that info | 22:17 |
| * mordred apologizes to everyone who has to read his braindumps | 22:18 | |
| eernst | yea, thanks clarkb | 22:18 |
| *** rossella_s has quit IRC | 22:19 | |
| pcichy | are there any restrictions when it comes to new features in gerritbot? | 22:19 |
| fried_rice | pcichy: What did you have in mind? | 22:20 |
| fried_rice | pcichy: Restricted to what you can get the cores to agree to :) | 22:20 |
| persia | mordred: To your credit, your braindumps are easier to read when they are a single blob of text vs. slowly streamed over IRC. | 22:21 |
| eernst | mordred: regarding the actual test nodes (not zuul, necessarily): can you give some feedback on these? I assume these are VMs/supported nested virtualization? | 22:21 |
| eernst | Or are folks also using baremetal? | 22:21 |
| clarkb | eernst: we don't have access to baremetal nodes today, but other people using our tooling have used it with baremetal so it should be possible as long as we are given cloud access to baremetal nodes | 22:21 |
| mordred | eernst: right now they're all VMs from openstack public clouds ... but that's largely related to what resources have been provided to the openstack project | 22:22 |
| clarkb | ericyoung: for nested virt some of our clouds do support it, we've found it to not be the most relaible thing | 22:22 |
| pcichy | fried_rice: heh :) I really liked the idea and code so I wanted to add support for Slack / MS Teams | 22:22 |
| clarkb | er eernst ^ sorry ericyoung | 22:22 |
| eernst | k, thx for the info. | 22:22 |
| fried_rice | pcichy: We couldn't really call it gerritbot, then, could we? :P | 22:23 |
| *** rossella_s has joined #openstack-infra | 22:23 | |
| fried_rice | pcichy: But using this framework it should be fairly easy to extend thusly, I would think. | 22:23 |
| pabelanger | mordred: +1 for mentioning day 2 services (ELK, elastic-recheck, mirrors) :) | 22:23 |
| pcichy | fried_rice: we could. The idea stays the same. Notify about changes in gerrit. Any gerrit in fact :) | 22:23 |
| mordred | pcichy: we have an open todo-list item about consolidating our irc bot infrastructure into a single bot - and one of the options that we've looked at was to migrate to using errbot which supports non-irc backends. however, desire for re-using the bots for non-irc seemed to have dried up so we started looking at otheroptions for the consolidated bot | 22:23 |
| clarkb | eernst: I have/had an email thread with the clodu provider giving us the most trouble with nested virt but haven't heard back on that in a while. I should probably go rebug them about it | 22:24 |
| corvus | pcichy, fried_rice: http://specs.openstack.org/openstack-infra/infra-specs/specs/irc.html is the direction we've chosen to go | 22:24 |
| clarkb | mordred: pcichy keep in mind at least for slack the irc gateway works great and I'd just connect an irc bot to it | 22:24 |
| fried_rice | pcichy: Sorry, yeah, I was getting confused. The gerrit part stays the same; the IRC part is what you're expanding. | 22:24 |
| corvus | pcichy, fried_rice: if you're interested in pitching in on that, we'd love the help :) | 22:25 |
| eernst | ok, thx. At this point the kata project has nested virtualized compute available, so this isn't as big of an issue at all. I was looking to see if there was a setup that'd be more ideal for PnP (ie, baremetal) | 22:25 |
| pcichy | corvus I will get a read on that. If I will be able to help I sure will do that :) | 22:27 |
| mordred | eernst: yah- nodepool itself can support both nested-virt and baremetal -there are issues to discuss with both as far as using them for ci resources at scale | 22:27 |
| pcichy | clarkb: thanks for the tip with the gateway. I will give that one a try also | 22:27 |
| mordred | eernst: but the tradeoffs/risks for those may be different for kata than for openstack, so it's definitely an area where we'll want to talk about and see what is applicable or not | 22:28 |
| EmilienM | is zuul ok now? | 22:28 |
| EmilienM | 542906 is not enqueued for ex | 22:29 |
| EmilienM | and I don't see events in dashboard | 22:29 |
| fried_rice | corvus: What's the timeline for that bot migration? | 22:29 |
| fried_rice | corvus: Cause gerritbot just got a flurry of enhancements :) | 22:29 |
| pabelanger | EmilienM: possible you need to recheck, zuul was restarted a short while ago | 22:29 |
| EmilienM | I did recheck | 22:29 |
| EmilienM | nothing happens | 22:29 |
| eernst | Alright, thanks for the information. At this point we have our working CI to keep development sane as we work towards having an initial Kata release. After that I think we'll be able to settle into what makes sense long term. Make sense? | 22:30 |
| pabelanger | EmilienM: I see it in zuul.o.o | 22:30 |
| mordred | eernst: yup! | 22:30 |
| openstackgerrit | Merged openstack-infra/project-config master: Grafana: switch zuul executor ram metric https://review.openstack.org/544746 | 22:30 |
| openstackgerrit | Merged openstack-infra/project-config master: Set post-timeout value to half an hour on base jobs https://review.openstack.org/545470 | 22:30 |
| EmilienM | pabelanger: yeah now :D | 22:30 |
| EmilienM | mhh ok | 22:30 |
| *** kgiusti has left #openstack-infra | 22:30 | |
| pabelanger | EmilienM: it is possible zuul missed the gerrit events, when restart happened | 22:30 |
| mordred | eernst: and I think we can take a few small steps to get you started with zuul-related things so we can iterate on stuff | 22:30 |
| *** Goneri has quit IRC | 22:30 | |
| corvus | fried_rice: hasn't started yet (you can help!) so i reckon your changes to gerritbot should go along with it. | 22:31 |
| clarkb | pcichy: the best part is it tends to stay up when the web based interfaces are done too :) | 22:31 |
| EmilienM | pabelanger: np | 22:31 |
| clarkb | pcichy: everything gets super quiet then you hear people asking if slack is down on other mediums | 22:31 |
| eernst | k. thanks mordred, clarkb. | 22:31 |
| fried_rice | corvus: My understanding is limited to the python, really. I don't have a good handle on the networking, channel protocols, etc. But let me know if there's something specific. | 22:32 |
| *** tpsilva has quit IRC | 22:33 | |
| pabelanger | eernst: I'd recommends using jenkins-job-builder (JJB) to version jobs. Also help with longer term migration if kata decides down that path. | 22:33 |
| corvus | fried_rice: first steps are setting up limnoria anyway, so if you're not up for the sysadmin work, i'll just bug you when we start porting bots :) | 22:33 |
| fried_rice | corvus: Sounds good | 22:33 |
| clarkb | eernst: out of curiousity can your container vm things run with just qemu or do they require kvm too? | 22:36 |
| clarkb | eernst: put another way could you test basic functionality without nested virt? | 22:36 |
| eernst | Nested virtualization is a requirement (one of the early reasons to use compute nodes from azure) | 22:36 |
| *** zzzeek has quit IRC | 22:38 | |
| *** olaph has quit IRC | 22:39 | |
| *** olaph has joined #openstack-infra | 22:40 | |
| *** priteau has quit IRC | 22:41 | |
| *** zzzeek has joined #openstack-infra | 22:42 | |
| *** jlabarre has quit IRC | 22:43 | |
| *** pcichy has quit IRC | 22:44 | |
| pabelanger | clarkb: corvus: much better! http://grafana.openstack.org/dashboard/db/zuul-status | 22:45 |
| openstackgerrit | Eric Fried proposed openstack-infra/gerritbot master: Refactor ChannelConfig with channels_for https://review.openstack.org/545469 | 22:45 |
| fried_rice | jlvillal: I'm not sure what you're looking for in the way of explicit regex tests. Having regexes in the config yaml and making sure they hit... isn't enough? | 22:47 |
| *** suhdood_ has joined #openstack-infra | 22:49 | |
| *** suhdood has quit IRC | 22:49 | |
| *** suhdood_ has quit IRC | 22:54 | |
| clarkb | fungi: hogepodge mguiney ok I think it is just about time to do the things. How can I help? | 22:59 |
| clarkb | eernst: fwiw I think I'm in the same general physical location of some of you if you ever want to meet up and talk about things in person too (might help to get more of the initial brain dump stuff with high bw in person) | 23:06 |
| eernst | Where you at? | 23:06 |
| clarkb | I'm over in the bethany/oak hills area just north of 26 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 23:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove project.unparsed_config https://review.openstack.org/545485 | 23:07 |
| clarkb | about 20 minutes from jones farm if traffic isn't terrible | 23:07 |
| eernst | K, half the time I'm over at Intel. The other half I'm in pajamas in SE portland ;-) | 23:07 |
| *** dougwig has quit IRC | 23:07 | |
| mordred | infra-root: the zuul github app is still set to not validated SSL - but we've got SSL enabled on the endpoint now - so I'm going to toggle the checkbox | 23:08 |
| clarkb | mordred: sounds good | 23:08 |
| corvus | ++ | 23:09 |
| hogepodge | clarkb: we're figuring that out. I need to check the reviews to make sure everything's in place | 23:10 |
| clarkb | hogepodge: ok | 23:10 |
| *** VW_ has quit IRC | 23:10 | |
| *** VW has joined #openstack-infra | 23:11 | |
| hogepodge | clarkb: does the deployment job automatically use the upstream puppet module, or do we need to specify a version? | 23:12 |
| *** VW has quit IRC | 23:12 | |
| *** VW has joined #openstack-infra | 23:12 | |
| hogepodge | (I also lost the repo that the server is deployed from in my back logs) | 23:13 |
| clarkb | hogepodge: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules.env#n149 shows what will be used. in this case its master so should auto update | 23:13 |
| hogepodge | how often does auto-update happen? | 23:14 |
| *** weshay is now known as weshay_PTO | 23:14 | |
| clarkb | hogepodge: it auto-updates as the first step in the ansible run loop. The ansible loop is on a 15 minute cron but typically runs more like every half hour due to how long it takes to execute ansible | 23:14 |
| clarkb | let me check when the next run is | 23:14 |
| hogepodge | ok, the change that merged does a big chunk of our work since it defaults to the behavior we want. | 23:15 |
| clarkb | its running currently next run should be at the half hour | 23:15 |
| * fungi is home and catching up | 23:15 | |
| *** rossella_s has quit IRC | 23:15 | |
| * mguiney nods | 23:16 | |
| mguiney | cool | 23:16 |
| fungi | dinner errands took longer than anticipated but available to help with refstack now | 23:16 |
| *** sboeuf has left #openstack-infra | 23:17 | |
| *** rossella_s has joined #openstack-infra | 23:17 | |
| *** bobh has quit IRC | 23:20 | |
| clarkb | hogepodge: mguiney is there anything fungi and I can eb doing at this point to see get things rolling? | 23:23 |
| fungi | i see mguiney, hogepodge and clarkb are on the case but happy to pitch in | 23:23 |
| hogepodge | clarkb: I'm not sure. mguiney is the main point of contact for running the script against the database | 23:24 |
| hogepodge | I'm trying to tag the release right now, once that's in the server will be updated and disallow anonymous results from being uploaded. | 23:25 |
| hogepodge | Once that behavior is disallowed, we can update the database without fear of a race condition. | 23:25 |
| hogepodge | I can't get gpg to sign my tag though... | 23:25 |
| fungi | error message? | 23:27 |
| fungi | i spend a lot of time wrestling gnupg so happy to help | 23:28 |
| *** VW_ has joined #openstack-infra | 23:28 | |
| clarkb | yes fungi is my goto for gnupg info | 23:28 |
| * fungi is not sure whether that is a good thing to be or a bad thing to be | 23:29 | |
| mguiney | clarkb: i'm currently just doing some last minute testing | 23:30 |
| mguiney | the actual database is a different version (obviously) than the one i tested on, so it was cranky about the url changes, just getting those to check out now | 23:31 |
| fungi | take your time... here when you need! | 23:31 |
| mguiney | but that is fairly trivial, as far as checking goes. | 23:32 |
| *** VW has quit IRC | 23:32 | |
| *** VW_ has quit IRC | 23:33 | |
| fungi | sc`: your "state of the kitchen" was an awesome read--keep it up if you can! | 23:35 |
| *** caphrim007_ has joined #openstack-infra | 23:35 | |
| * fungi does _not_ want to know what the baby wipes are for, btw | 23:35 | |
| clarkb | fungi: I'm basically a baby wipe expert now, you sure you don't want to know? | 23:36 |
| fungi | really, no | 23:36 |
| mguiney | script appears to be running smoothly, with a few tweaks to represent differences in the data dump | 23:36 |
| *** caphrim007_ has quit IRC | 23:36 | |
| mguiney | should be ready to go. can we snapshot the current db state before doing so, though? | 23:37 |
| fungi | i can unless clarkb is already in place | 23:37 |
| clarkb | fungi: go for it | 23:37 |
| fungi | where "snapshot" just means i'm saving a mysqldump | 23:37 |
| fungi | hopefully that does not contradict your expectations | 23:37 |
| mguiney | nah | 23:38 |
| *** caphrim007_ has joined #openstack-infra | 23:38 | |
| mguiney | i know, just using incorrect language because bad old habits die hard | 23:38 |
| fried_rice | Presumably one uses fewer baby wipes when running smoothly than when saving a mysqldump | 23:38 |
| *** caphrim007 has quit IRC | 23:39 | |
| fungi | dumping to refstack.o.o:~fungi/refstack_pre-upgrade_2018-02-16.sql.gz for reference | 23:39 |
| clarkb | should be about 18MB iirc | 23:39 |
| fungi | fried_rice: this jives with my experience, yes | 23:39 |
| fungi | clarkb: yes, 19M compressed per du | 23:40 |
| fungi | mguiney: it's done | 23:40 |
| mguiney | excellent. i guess that would make it time to run the script then | 23:41 |
| mguiney | hogepodge: anything you can think of that needs to get done first? I can grab a new data dump from the powered db if that has been updated in the last hour or so | 23:41 |
| hogepodge | I just needed a helper program installed to ask for my passphrase | 23:41 |
| sc` | fungi: thanks! with the bulk of the deprecations out of the way, i hope i'll be able to turn around queens faster | 23:41 |
| hogepodge | Release tagged and pushed | 23:41 |
| fungi | mguiney: that's an api-using script you're running with your creds, right? | 23:42 |
| sc` | it's easy to get lost in gerrit/lp and not tell people what's going on | 23:42 |
| fungi | (just making sure it's not a script i need to find and run) | 23:42 |
| mguiney | yupyup | 23:42 |
| *** caphrim007_ has quit IRC | 23:42 | |
| fungi | sc`: i agree, happens to me faaar too often | 23:43 |
| fungi | mguiney: will you be in dublin, or classes keeping you stateside? i miss hanging out | 23:43 |
| mguiney | will be in dublin! | 23:44 |
| sc` | fungi: btw, the wipes are for the day after eating the output of the recipe | 23:44 |
| fungi | mguiney: great, will definitely be sure to get up with you! | 23:44 |
| mguiney | sounds good! | 23:45 |
| fungi | sc`: yeah, i kinda figured, was just hoping not to have my assumptions confirmeed | 23:45 |
| sc` | that's a subset of what i usually make when i'm craving chili | 23:46 |
| hogepodge | mguiney: we're good to move forward with the data cleaning | 23:46 |
| hogepodge | once the database has been backed up | 23:46 |
| fungi | sc`: over the years i've come to consider "chili powder" a black box, and provide my own roasted peppers and spices. getting pretty close but my recipe is all scribbles i should translate to electrons at some point | 23:48 |
| sc` | yup. chili powder is an umbrella term. i have my own blend | 23:48 |
| mguiney | ok sounds good | 23:48 |
| * mguiney wipes brow with baby wipe | 23:48 | |
| mguiney | ok im gonna run the update | 23:49 |
| clarkb | sc`: I like to use chuck cut up into cubes and pork sausage for mine | 23:49 |
| fungi | sc`: i have the added challenge that my wife is not a fan of cumin (what?!? ikr!) so i've had to adjust heavily | 23:49 |
| sc` | mine is wife-approved :) | 23:49 |
| *** hemna_ has quit IRC | 23:50 | |
| *** larainema has joined #openstack-infra | 23:50 | |
| fungi | clarkb: chuck and chorizo is a great combo, or chuck and cubed brisket | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make JobParser a regular class https://review.openstack.org/545150 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make PipelineParser a regular class https://review.openstack.org/545151 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make SemaphoreParser a regular class https://review.openstack.org/545152 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Make TenantParser a regular class https://review.openstack.org/545153 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ConfigLoader initializer https://review.openstack.org/545180 | 23:50 |
| sc` | i'm going to try chorizo in a future batch | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove tenant argument from UnparsedTenantConfig https://review.openstack.org/545181 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add ParseContext class https://review.openstack.org/545448 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix typo in PragmaParser https://review.openstack.org/545449 | 23:50 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove project.unparsed_config https://review.openstack.org/545485 | 23:50 |
| sc` | it usually never stays around long enough to be chili | 23:50 |
| fungi | sc`: i've only done it a couple times, but if you have a sausage grinder then making your own obviously allows you to control the seasoning going in | 23:51 |
| *** hongbin has quit IRC | 23:51 | |
| sc` | in the process of buying a house. the to-buy list is a bigger kitchenaid for the grinder attachment | 23:51 |
| fungi | which i find helps a lot to be able to balance flavors | 23:51 |
| sc` | or a 1.0 version | 23:52 |
| clarkb | gmmaha: looks like you have logs now (though still timing out) | 23:52 |
| sc` | whichever seems more appealing at the time | 23:52 |
| clarkb | gmmaha: possible you will have to increase the timeout afterall but now I think you'll be able to confirm that via the logs? | 23:52 |
| fungi | kitchenaid attachments have been garbage ever since hobart sold the product line off in the late '70s | 23:52 |
| gmmaha | clarkb: yeah.. the infra jobs need a bit more time to finish.. i have a patch up to increase the timeout https://review.openstack.org/#/c/545481/ | 23:52 |
| gmmaha | and that job itself has been running close to an hour now and hasn't died yet.. | 23:53 |
| gmmaha | so yaayy!! | 23:53 |
| sc` | the list is ever-changing, so it's not set in stone | 23:53 |
| gmmaha | clarkb: and yes the timeout trigger was visible in the logs and i saw it go ahead and get logs lwhich i was able to see.. | 23:54 |
| gmmaha | thanks for working on that fix. | 23:54 |
| mguiney | hogepodge: database updated | 23:55 |
| *** sree has joined #openstack-infra | 23:55 | |
| mguiney | at this point, anything unverified isn't in our set of records, and should be removable | 23:55 |
| hogepodge | mguiney: what do you mean database updated? | 23:56 |
| *** aviau has quit IRC | 23:56 | |
| hogepodge | Ah, all of the verified results have been tagged? | 23:56 |
| *** aviau has joined #openstack-infra | 23:56 | |
| mguiney | ^ that | 23:56 |
| hogepodge | Can you send me a link to one of the updated results so I can check on it? Also, do yo have a sample test run handy? I want to verify that anonymous uploads are disabled | 23:57 |
| mguiney | i can have it run through and remove unverified results, but figured i'd check in before taking any further action | 23:58 |
| mguiney | what format would you like the canned result sent via, and would you prefer full or partial run | 23:58 |
| hogepodge | mguiney: just the json file is enough | 23:58 |
| *** rossella_s has quit IRC | 23:58 | |
| mguiney | yes but how. via what method of send | 23:59 |
| *** SumitNaiksatam has quit IRC | 23:59 | |
| *** sree has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!