| *** threestrands_ has quit IRC | 00:48 | |
| *** threestrands has joined #zuul | 00:48 | |
| *** persia has joined #zuul | 00:49 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: check for client in properties https://review.openstack.org/534988 | 01:24 |
|---|---|---|
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers https://review.openstack.org/524620 | 01:25 |
| *** harlowja has quit IRC | 01:55 | |
| *** mattclay has quit IRC | 01:55 | |
| *** robcresswell has quit IRC | 01:55 | |
| *** gothicmindfood has quit IRC | 01:55 | |
| *** mnaser has quit IRC | 01:55 | |
| *** fdegir has quit IRC | 01:55 | |
| *** gothicmindfood has joined #zuul | 01:55 | |
| *** mattclay has joined #zuul | 01:56 | |
| *** mnaser has joined #zuul | 01:56 | |
| *** robcresswell has joined #zuul | 01:56 | |
| *** fdegir has joined #zuul | 01:56 | |
| mrhillsman | want to test if environment behind vpn will work, nodepool and zuul-executor seem to be talking to zookeeper and gearman respectively, is there a way to trigger a job run for this particular executor to confirm things are working? | 02:21 |
| mrhillsman | additionally, nodepool is not creating any images/servers, is this because the scheduler needs to be able to hit nodepool rather than communicate this via zookeeper? | 02:22 |
| mrhillsman | zookeeper and gearman are hosted outside of the environment | 02:22 |
| pabelanger | 2018-01-18 02:34:20,571 WARNING kazoo.client: Failed connecting to Zookeeper within the connection retry policy. | 02:34 |
| pabelanger | tristanC: Shrews: ^did we have a patch up to address that? keep trying zookeeper forever | 02:35 |
| pabelanger | right now, port to zookeeper is closed by firewall | 02:35 |
| pabelanger | because we haven't opened it yet, but we'd want nodepool-builder to keep trying until it does get open | 02:35 |
| tristanC | pabelanger: the patch is https://review.openstack.org/523640, but it's for retrying command after the connection is established, retrying connection attempt would need to set the "connection_retry" client argument | 02:42 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: automatically retry command when connection is lost https://review.openstack.org/523640 | 02:43 |
| pabelanger | tristanC: okay, I'll check back in morning with Shrews on how best to proceed | 02:44 |
| *** TheJulia has quit IRC | 03:30 | |
| *** timrc has quit IRC | 03:30 | |
| *** jamielennox has quit IRC | 03:30 | |
| *** timrc has joined #zuul | 03:30 | |
| *** TheJulia has joined #zuul | 03:31 | |
| *** jamielennox has joined #zuul | 03:37 | |
| *** jkt has quit IRC | 03:51 | |
| SpamapS | mrhillsman: if you have min-ready: set higher than 0, you should have servers. And your labels should all be built and uploaded, regardless of whether zuul has asked for any capacity. | 03:55 |
| SpamapS | or rather, your images | 03:55 |
| SpamapS | mrhillsman: I'd check the logs, there are likely things failing. | 03:55 |
| mrhillsman | min-ready set to 1 | 03:59 |
| mrhillsman | max-servers for the pool set to 5 and concurrency set to 5 | 03:59 |
| mrhillsman | got debug on and no errors at all | 04:00 |
| mrhillsman | :( | 04:00 |
| mrhillsman | maybe i'll change some stuff to see if anything acts up | 04:01 |
| clarkb | images happen first so start there | 04:02 |
| mrhillsman | let me try to build an image with dib logs set to debug and are not saying much of anything | 04:06 |
| mrhillsman | http://paste.openstack.org/show/646808/ < launcher | 04:06 |
| mrhillsman | http://paste.openstack.org/show/646809/ < builder | 04:07 |
| *** bhavik1 has joined #zuul | 04:09 | |
| *** bhavik1 has quit IRC | 04:13 | |
| clarkb | what does your builder config lool like? | 04:16 |
| mrhillsman | sec | 04:20 |
| SpamapS | Is there a known bug in the streaming plugins that makes command: ignore no_log: true? | 04:20 |
| clarkb | SpamapS: I want to say last I checked thw no logging atuff resulted in no logs but that may have regressed | 04:21 |
| SpamapS | I just had our artifactory passwords leaked into log files | 04:22 |
| mrhillsman | http://paste.openstack.org/show/646812/ | 04:22 |
| SpamapS | after decrypting them to stdout with no_log: true .. the stdout was logged. | 04:22 |
| SpamapS | So, guessing it doesn't respect the flag. | 04:22 |
| * SpamapS has reworked the code to output to a temp file and slurp it. | 04:23 | |
| clarkb | mrhillsman: that looks correct to me. And nodepool image-list shows no images? other than it thnking its work is done I'm stumped for now | 04:25 |
| mrhillsman | ok thx | 04:26 |
| mrhillsman | image-list shows images | 04:26 |
| mrhillsman | just nothing local | 04:26 |
| mrhillsman | which is strange to me since i removed information about the other environments | 04:27 |
| clarkb | well if it thinks its work is done for that cloud it will idle | 04:27 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: use kazoo retry facilities https://review.openstack.org/523640 | 04:27 |
| mrhillsman | but i figured it was reading that from zookeeper | 04:27 |
| SpamapS | if it shows images, that means they're at least built | 04:27 |
| mrhillsman | it shows images, just from other environments | 04:27 |
| tristanC | pabelanger: Shrews: i've rewrote the retry logic patch, apparently the KazooClient expect a dictionary of retry option instead of a retry object | 04:27 |
| mrhillsman | maybe cached? | 04:27 |
| mrhillsman | i copied the config from working environment and modified it | 04:28 |
| tristanC | pabelanger: Shrews: then i removed the nodepool local suspended logic. it does work as expected (e.g. nodepool survive when zookeeper is turned off randomly) | 04:29 |
| clarkb | mrhillsman: do they use the same zookeeper? and talk to the same cloud(s) | 04:30 |
| tristanC | SpamapS: i think zuul_stream is ignoring no_log yes, fixed it with https://review.openstack.org/#/c/533509/ | 04:31 |
| SpamapS | tristanC: thanks! | 04:35 |
| mrhillsman | clarkb they use the same zookeeper yes | 04:37 |
| mrhillsman | i do not have them all talking to the same clouds though | 04:37 |
| mrhillsman | this particular one does not allow ingress without vpn | 04:37 |
| mrhillsman | so my understanding, which could be wrong, was that i needed at least nodepool(builder+launcher) and zuul-executor inside, with zookeeper and gearman outside | 04:38 |
| mrhillsman | so dib works fine it appears | 04:39 |
| mrhillsman | disk-image-create ubuntu vm gave me image.qcow2 | 04:40 |
| mrhillsman | so will keep digging | 04:41 |
| clarkb | mrhillsman: if the providers and pools are names the same in the configs they will be treated as the same regardless of network connecrivity | 04:48 |
| clarkb | as long as those are unique it should be fine | 04:48 |
| clarkb | also the builder doesnt need access to where the VMs get IPs only to the cloud apis and artifacts/reaources to build the images themselves | 04:49 |
| mrhillsman | ok so i need to set the new environment in the nodepool builder config on the public server | 04:50 |
| mrhillsman | so it can put in zookeeper information about what should be available there | 04:51 |
| mrhillsman | and the local builder will read that and do its thing | 04:51 |
| mrhillsman | it seems that is the case because the local nodepool config i do not think is affecting what is in zookeeper | 04:52 |
| mrhillsman | i'm sure it is late for you, i'll draw a picture and drop in here tomorrow for some feedback | 04:53 |
| mrhillsman | the issue it would seem to me is that zookeeper does not have any info about the new environment so when builder checks it, nothing happens, at least on starting that is what the logs suggest | 04:54 |
| tobiash | mrhillsman: if you share zookeeper by several unrelated nodepools you need to take care of having provider named unique | 04:56 |
| tobiash | The truth of the nodepool state is saved in zk and the provider name is a key for accessing that | 04:57 |
| mrhillsman | the provider names are all unique | 04:57 |
| mrhillsman | i see it creating ProviderManager for local < new provider/pool | 04:58 |
| mrhillsman | and i see it reading info on the other providers | 04:58 |
| mrhillsman | and their images | 04:58 |
| mrhillsman | and nodepool image-list confirms | 04:58 |
| tobiash | You probably also need to name the images unique | 04:59 |
| mrhillsman | ok | 04:59 |
| mrhillsman | will try that | 04:59 |
| tobiash | Nodepool has entities for local and uploaded imaged | 05:00 |
| tobiash | You can query the local images with nodepool dib-image-list | 05:01 |
| mrhillsman | that was it | 05:01 |
| mrhillsman | the image names | 05:01 |
| tobiash | So if you have the same image names the following will happen | 05:01 |
| mrhillsman | i change image and label | 05:02 |
| mrhillsman | and i see builder trying to build images now | 05:02 |
| tobiash | Nodepool1 builds and uploads image, nodepool2 sees the local image and tells to upload it, but no builder feels responsle for the local image target cloud combination | 05:04 |
| tobiash | Because only the original builder can upload it but doesn't know the target | 05:05 |
| mrhillsman | i understand what you are getting at in a sense though not entirely clear | 05:07 |
| mrhillsman | will have to work it out in my head but i get what you mean | 05:07 |
| mrhillsman | ok, had to talk it through to myself hehe, got it, thanks for clarity | 05:08 |
| tobiash | You also could use the chroot setting: | 05:09 |
| tobiash | https://docs.openstack.org/infra/nodepool/feature/zuulv3/configuration.html#zookeeper-servers | 05:09 |
| tobiash | That separates their namespaces and inhibits such name collisions | 05:09 |
| tobiash | But zuul needs to use the same chroot, have to check if zuul supports that | 05:10 |
| *** jaianshu has joined #zuul | 05:12 | |
| tobiash | Hrm, looks like zuul doesn't support that yet | 05:12 |
| tobiash | Chroot is currently only used in the test suite of nodepool where we would have the same problems | 05:13 |
| tobiash | So once zuul is enabled for this you should chroot this | 05:13 |
| tobiash | Until then you have the choice to keep an eye on the naming or use separate zk clusters | 05:14 |
| mrhillsman | that makes sense | 05:20 |
| mrhillsman | thanks! | 05:20 |
| SpamapS | I am not sure it's wise to share zookeeper's without chrooting inside the zookeeper. | 05:20 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Link to zuul-base-jobs docs from User's Guide https://review.openstack.org/531912 | 05:21 |
| mrhillsman | i can use chroot | 05:21 |
| mrhillsman | just was not aware | 05:21 |
| mrhillsman | still new to using zuul as a whole and everything was all in one environment | 05:22 |
| mrhillsman | will implement it now | 05:22 |
| tobiash | SpamapS: it's not (at least for production) | 05:31 |
| *** flepied_ has quit IRC | 06:38 | |
| *** flepied has joined #zuul | 06:39 | |
| *** flepied_ has joined #zuul | 06:50 | |
| *** flepied has quit IRC | 06:53 | |
| *** flepied_ has quit IRC | 07:01 | |
| *** flepied_ has joined #zuul | 07:01 | |
| *** ianw has quit IRC | 07:02 | |
| *** flepied_ has quit IRC | 07:03 | |
| *** flepied has joined #zuul | 07:04 | |
| *** flepied has quit IRC | 07:16 | |
| *** Wei_Liu has joined #zuul | 07:18 | |
| *** flepied has joined #zuul | 07:22 | |
| *** flepied has quit IRC | 07:23 | |
| *** ankkumar has joined #zuul | 07:27 | |
| *** threestrands has quit IRC | 08:03 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add buildset-artifacts-location and fetch roles https://review.openstack.org/530679 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add linters job and role https://review.openstack.org/530682 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-lint job https://review.openstack.org/532083 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-import-to-galaxy job https://review.openstack.org/532084 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-spec job https://review.openstack.org/532085 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-review job https://review.openstack.org/535223 | 08:37 |
| *** jpena|off is now known as jpena | 08:46 | |
| *** sshnaidm is now known as sshnaidm|afk | 08:55 | |
| *** ianw has joined #zuul | 08:58 | |
| *** saop has joined #zuul | 11:05 | |
| saop | tristanC, My ansible was failing to adding logserver in inventory, we are trying to run ssh-agent there, with this command: "eval `ssh-agent -s`" but it was failing? | 11:06 |
| saop | tristanC, we tried many methods running ssh-agent but not working anything? | 11:07 |
| saop | tristanC, any suggestion? | 11:07 |
| *** tobiash has quit IRC | 11:19 | |
| *** tobiash has joined #zuul | 11:22 | |
| *** jkilpatr has joined #zuul | 11:52 | |
| *** jkilpatr has quit IRC | 11:53 | |
| *** jkilpatr_ has joined #zuul | 11:54 | |
| *** jpena is now known as jpena|lunch | 11:59 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove testr and stestr specific roles https://review.openstack.org/529340 | 12:43 |
| *** jpena|lunch is now known as jpena | 13:06 | |
| *** jaianshu has quit IRC | 13:27 | |
| *** rlandy has joined #zuul | 13:27 | |
| *** weshay is now known as weshay|rover | 13:52 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:57 | |
| *** sshnaidm is now known as sshnaidm|afk | 14:06 | |
| *** sshnaidm|afk is now known as sshnaidm|mtg | 14:07 | |
| *** saop has quit IRC | 14:15 | |
| *** ankkumar has quit IRC | 14:17 | |
| tobiash | Shrews, corvus: do you know about a state 'init' in nodepool? | 14:48 |
| tobiash | I have a node in this state without server id | 14:48 |
| tobiash | ah, maybe it was killed during creation of the node | 14:50 |
| *** dkranz has joined #zuul | 14:52 | |
| pabelanger | tristanC: Shrews: thoughts on adding zuul.d support to nodepool (nodepool.d)? I think it would help to manage the since of our yaml files | 15:48 |
| Shrews | i have no preference either way | 15:50 |
| clarkb | corvus: do you want to rereview https://review.openstack.org/#/c/533771/9 so we can get that bugfix in before merging branches today? | 16:10 |
| corvus | pabelanger: i could see nodepool.d being useful, yeah. | 16:14 |
| pabelanger | cool, I want to say tristanC might have some existing patches in SF nodepool. Hopefully he has more info | 16:16 |
| corvus | clarkb: i see 2 changes: useBuilder and the labels, yeah? lgtm. | 16:17 |
| clarkb | corvus: yup, the labels were to fix an actual issue with the test (avoids a race) and useBuilder was a base test class api change | 16:18 |
| *** dkranz has quit IRC | 16:26 | |
| mrhillsman | ok, so i got builder to build images now and instances have spawned, next of course is to see if the scheduler hosted outside of this environment can actually use it for jobs, i have zuul-executor running on the same server as nodepool (builder+launcher), is there a way to trigger a job run to this executor? | 16:27 |
| mrhillsman | or should it simply pick up jobs as it is able to from having access to gearman | 16:28 |
| clarkb | it should pick up jobs as it is able via gearman | 16:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient https://review.openstack.org/534431 | 16:35 |
| mrhillsman | ok cool | 16:38 |
| *** dkranz has joined #zuul | 16:38 | |
| corvus | clarkb: it looks like no one approved 533372 so i just did | 16:54 |
| *** dkranz has quit IRC | 16:54 | |
| *** sshnaidm|mtg has quit IRC | 16:56 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Handle nodesets in branches https://review.openstack.org/534974 | 17:00 |
| *** dkranz has joined #zuul | 17:07 | |
| * SpamapS kind of wishes nodepool built images by submitting a job to the zuul-executor so it could be shown in builds. | 17:17 | |
| corvus | that's probably not infeasible with things like static nodes or cloud-provider images. | 17:18 |
| clarkb | The upside to nodepool builders running externally is you can use it as an image build service on its own. I know there is a non zero number of people that are or were doing this. | 17:19 |
| corvus | a bit inceptiony | 17:19 |
| corvus | clarkb: oh of course, i would never suggest that as the exclusive way. just an additional way. | 17:20 |
| corvus | s/never/probably never unless drunk/ | 17:20 |
| corvus | i like a toolkit with options though :) | 17:21 |
| clarkb | ya, to make that more simple we might want to add a nodepool image build command that doesn't use a daemon at all | 17:21 |
| clarkb | justbuilds and image and optionally uploads it | 17:22 |
| SpamapS | The way it is now the debugging process for "why did my job break" is quite different than "why did my image build break" .. so while I respect nodepool-doesnt-need-zuul as an argument, I also think maybe nodepool does need {something} and I like zuul for that something. :) | 17:22 |
| clarkb | SpamapS: the other downside (and you can kinda hack around this with gearman a bit) is being at the mercy of job demand for image freshness | 17:23 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Only fail requests if no cloud can service them https://review.openstack.org/533372 | 17:24 |
| clarkb | waiting on new images to fix a problem causing everyone to recheck taking forever because everyone is rechecking sort of issue | 17:24 |
| corvus | also, exposing nodepool build logs through a web api could help with "why did my build break?" | 17:24 |
| SpamapS | Yeah, that's what SUBMIT_JOB_HIGH is for :) | 17:25 |
| corvus | oh *that's* what it means by "high" | 17:25 |
| corvus | no wonder my jobs are taking so long. i was so confused. | 17:25 |
| SpamapS | corvus: yeah I do actually have a pretty extensive Fluentd/ElasticSearch cluster available to shove them into. But it's the iterating-on-fixing-it part that is a bit weird. | 17:25 |
| corvus | i thought... nevermind. | 17:26 |
| SpamapS | :) | 17:26 |
| SpamapS | I was gonna build you a new image... but then somebody SUBMIT_JOB_HIGH'd | 17:26 |
| clarkb | SpamapS: I do that locally fwiw. Its just way easier to stick a bash call into dib elements and debug from there or mount the image and poke around or boot it | 17:26 |
| SpamapS | clarkb: because you run Linux locally. :) | 17:27 |
| clarkb | yes, I highly recommend it to everyone else too :) | 17:28 |
| clarkb | (that said I have a vm I do all my dib in because dib is rooty so anyone should eb able to do it regardless of platform as long as they are willing to run a linux VM) | 17:30 |
| *** hashar has joined #zuul | 17:31 | |
| *** hashar is now known as hasharDinner | 17:31 | |
| *** sshnaidm|mtg has joined #zuul | 17:31 | |
| *** sshnaidm|mtg is now known as sshnaidm | 17:32 | |
| SpamapS | Yeah I dib in a clean VM too ;) | 17:33 |
| SpamapS | But that sort of feels like what zuul is good at :) | 17:33 |
| SpamapS | hence we've come full circle :) | 17:34 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add test_launcher test https://review.openstack.org/533771 | 17:34 |
| corvus | clarkb: ^ | 17:35 |
| corvus | okay, how about we start this merge thing? | 17:35 |
| clarkb | ++ | 17:35 |
| corvus | let's call the repos frozen now | 17:35 |
| clarkb | electrofelix had some comments on the commands to run | 17:35 |
| corvus | i'll record a list of all the changes on feature/zuulv3, then abandon them all with a nice message | 17:35 |
| clarkb | I'm not entirely sure how they are different than what I suggested since aiui ours means always use the new stuff | 17:35 |
| SpamapS | \o/ | 17:35 |
| clarkb | or s/new/our/ | 17:35 |
| corvus | clarkb: i think electrofelix solved the 'manually delete files' step... | 17:36 |
| clarkb | ah | 17:36 |
| clarkb | its like 4 files so not a huge deal | 17:36 |
| corvus | actually, let's do an etherpad for this: https://etherpad.openstack.org/p/3mSln6ATMP | 17:36 |
| corvus | i think that's got it, that look okay to everyone? | 17:41 |
| clarkb | lgtm | 17:42 |
| mordred | yay merging! | 17:43 |
| -corvus- Zuul and Nodepool feature/zuulv3 branches are now frozen while we merge them into master. Please don't upload or approve any changes to those branches. | 17:43 | |
| corvus | do folks see the notice i just made ^ ? | 17:43 |
| dmsimard | corvus: yes | 17:43 |
| corvus | cool. it's nice and red-highlighted on my client; just wanted to make sure it's visible | 17:43 |
| clarkb | I see it | 17:43 |
| mordred | yup. I see it | 17:44 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Add provider info command https://review.openstack.org/535423 | 17:44 |
| dmsimard | tobiash, tristanC: btw I had an idea for the zuul web dashboard.. a schedule of periodic jobs (like a queue that shows which jobs will be triggered when) | 17:44 |
| tobiash | dmsimard: that sounds cool | 17:45 |
| dmsimard | periodic jobs are easy when you have just one pipeline that triggers every 4hrs but if you have multiple periodic pipelines that trigger on different timers it gets confusing | 17:45 |
| Shrews | pabelanger: 535423 ^^ will give us a tool to remove vanilla's zookeeper data w/o having to manually do that | 17:45 |
| dmsimard | tobiash: I'll create a story for it | 17:45 |
| corvus | Shrews: ^ we're freezing the feature/zuulv3 branches | 17:45 |
| tobiash | also the current number of changes in pipelines in the tenant view would be useful | 17:45 |
| Shrews | corvus: that's fine. i can re-home it later, it's not ready yet | 17:46 |
| corvus | i think this etherpad is mostly a one-person thing, so i'll plan on doing most of it. | 17:46 |
| Shrews | i did not see your notice until just now :) | 17:46 |
| corvus | maybe someone wants to start looking at item #7 -- update configs where we point to feature/zuulv3 ? | 17:47 |
| mordred | corvus: that should mostly be in system-config I think, right? | 17:47 |
| mordred | or, rather - I'll take system-config | 17:48 |
| pabelanger | Shrews: great, thanks | 17:49 |
| clarkb | pabelanger: ^ your recent change to test-requirements in project-config made me think of number 7 above | 17:50 |
| clarkb | but system-cofnig has it too for the puppet based git repo checkouts and pip installs | 17:51 |
| * clarkb updates system-config | 17:51 | |
| pabelanger | ah, good point | 17:51 |
| mordred | how about topic 'zuulv3-merge' ? | 17:51 |
| clarkb | wfm | 17:52 |
| clarkb | https://review.openstack.org/535429 is up for system-config | 17:56 |
| mordred | ok. I added a list of all of the places feature/zuulv3 happens across all our repos to the etherpad | 17:57 |
| dmsimard | tobiash, tristanC: https://storyboard.openstack.org/#!/story/2001486 | 17:57 |
| mordred | clarkb: jinx. we made the same patch - but slightly differently | 17:57 |
| clarkb | mordred: oh I totally missed you saying you would do system config :P | 17:58 |
| clarkb | I can abandon mine as you were first | 17:58 |
| corvus | http://paste.openstack.org/show/646946/ | 18:00 |
| tobiash | dmsimard: nice idea | 18:00 |
| corvus | how's that look for the list of open changes? | 18:00 |
| tobiash | wow, even with dependencies | 18:01 |
| clarkb | corvus: if you abandon all of them and gerrit lets us delete the branch then you know you got them all :) | 18:01 |
| mordred | looks great | 18:01 |
| tobiash | like it | 18:01 |
| mordred | clarkb: perhaps if we land https://review.openstack.org/#/c/534458/ real quick, when people re-propose all of their patches we won't burn the VMs on the cover jobs | 18:02 |
| corvus | tobiash: it's a screenshot from gertty | 18:02 |
| mordred | gah | 18:02 |
| mordred | corvus: ^^ | 18:02 |
| corvus | mordred: or if we land that asap after the merge :) | 18:02 |
| mordred | corvus: either way :) | 18:02 |
| corvus | let's do it post-merge, so we don't have to pause here for 15 minutes :) | 18:03 |
| corvus | okay, if it lands it lands :) | 18:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-base-jobs master: Remove feature/zuulv3 from pip depend https://review.openstack.org/535432 | 18:05 |
| corvus | actually -- we should probably perform the merge first, *then* abandon all the changes. | 18:06 |
| clarkb | mordred: re system-config the reason I explicitly switched feature/zuulv3 -> master was I didn't want to have to cross check against where we put in the default to old master tags for things like openstackci | 18:07 |
| clarkb | mordred: I think chances are what you did are fine, but it may install zuulv2 in some cases | 18:07 |
| corvus | clarkb, mordred: ^ should we merge the branches first, then abandon the changes? | 18:08 |
| mordred | corvus: I'm good with that | 18:09 |
| corvus | also, i put in a proposed 'abandon' message for that step, does that look good? | 18:09 |
| mordred | clarkb: let's do your version | 18:09 |
| clarkb | I think we can abandon changes whenever. The branch delete is the tricky one. | 18:09 |
| mordred | yah. we have a few tox.ini files and required-projects lines referencing it - I thinkn we should merge, abandon, then land the changes updating things to pointto master, then delete branch | 18:10 |
| corvus | kk | 18:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool master: Replace master with feature/zuulv3 https://review.openstack.org/535435 | 18:14 |
| Shrews | hrm, the sitemap.xml in openstack-manuals seems to be regenerated rather than manually edited. anyone know offhand how that's done? | 18:14 |
| corvus | Shrews: ask in #openstack-infra ? | 18:15 |
| corvus | or #openstack-doc | 18:15 |
| corvus | i've updated the steps in 3 to reflect what i'm doing on a fresh clone of the repos | 18:15 |
| mordred | corvus: we have a chicken-and-egg issue | 18:16 |
| mordred | corvus: oh, no we don't - nevermind | 18:16 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:17 |
| mordred | corvus: you may want to make a follow-up that removes the feature/zuulv3 entries from README | 18:17 |
| corvus | mordred: yep | 18:17 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Stop running tox-cover job https://review.openstack.org/534458 | 18:18 |
| mordred | oh - although we do have an issue with the .zuul.yaml file | 18:18 |
| corvus | 535436 is failing due to the issue i'm fixing in 534974 | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:19 |
| mordred | corvus: should we land 534974 first? or land a feature/zuulv3 patch removing the .zuul.yaml ? | 18:19 |
| corvus | mordred: 534974 needs careful review and a restart | 18:19 |
| corvus | mordred: i've modified 535436 to use a different nodeset name | 18:20 |
| mordred | nod. wfm | 18:20 |
| corvus | so if you download that patch and diff the two branches, you should see .gitreview and .zuul.yaml are different. | 18:20 |
| AJaeger | we also need to change some jobs, like http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/jobs.yaml for landing the change | 18:20 |
| AJaeger | and http://git.openstack.org/cgit/openstack-infra/zuul-base-jobs/tree/tox.ini#n38 | 18:21 |
| mordred | AJaeger: yah - we have a topic up - zuul-merge | 18:21 |
| mordred | or zuulv3-merge rather, sorry | 18:21 |
| clarkb | mordred: should I unabandon mine? | 18:21 |
| clarkb | and ya I don't think there is a chicken and egg, we just have to get merge in onto master first then do everything else before deleting the brnach | 18:21 |
| mordred | clarkb: yah - I think youmake a good point about being explicit for now | 18:21 |
| corvus | i've updated my local copy of the change list to remove the cover job. | 18:21 |
| corvus | er, the cover job change | 18:22 |
| AJaeger | mordred: good, I see it's covered already | 18:22 |
| clarkb | mordred: https://review.openstack.org/#/c/535429/ is restored | 18:22 |
| mordred | clarkb: +2 - abandoned mine | 18:24 |
| corvus | clarkb, mordred, tobiash: 535435 and 535436 are ready for reviews... in whatever way we think is best to review those. :) | 18:27 |
| mordred | corvus: +2 on both from me | 18:28 |
| tobiash | the nodepool change looks suprisingly small | 18:31 |
| corvus | tobiash: the builder part of nodepool v3 is already on master | 18:31 |
| clarkb | ya I'm not sure the best way to review that. When I did it locally I trusted git and tox for the most part | 18:32 |
| clarkb | I suppose I can reproduce the merge and see if it looks different | 18:32 |
| corvus | mordred: my merge did not include your tox-cover change | 18:33 |
| tobiash | sure that's correct? I don't see driver related stuff in there. | 18:33 |
| corvus | i guess i will redo it. | 18:33 |
| mordred | corvus: piddle. oh well | 18:34 |
| Shrews | oh, hrm, i'm not seeing the nodepool/launcher.py file in that change | 18:34 |
| corvus | tobiash: be aware that gerrit is not very good at rendering merge commit diffs. | 18:34 |
| corvus | Shrews: ^ | 18:34 |
| mordred | tobiash: for merge commits gerrit only shows you conflicts | 18:34 |
| mordred | and their resolutions | 18:34 |
| tobiash | ah, that might explain my confusion | 18:34 |
| corvus | gertty will show you the whole diff | 18:35 |
| Shrews | ah | 18:35 |
| corvus | but the best way is probably to check it out locally and verify the result looks correct | 18:35 |
| tobiash | that's what I just was going to do | 18:35 |
| corvus | anyway, hold off on zuul for now. proceed with nodepool. | 18:35 |
| Shrews | yeah, checking out the change and visually inspecting looks good | 18:36 |
| tobiash | ok, the checkout looks familiar | 18:37 |
| tobiash | :) | 18:37 |
| tobiash | so +2 from me | 18:37 |
| clarkb | corvus: electrofelix's steps and mine do result in slightly different diffs. Things like tools/check_devstack_plugin.sh in nodepool appear to have been updated on master but not on feature/zuulv3 | 18:37 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:38 |
| clarkb | and we've got stuff like precise instead of xenial in the docs on the feature/zuulv3 side that appear to have been updated on the master side | 18:39 |
| corvus | apparently master wasn't as frozen as we thought it was? | 18:39 |
| clarkb | ya I guess not | 18:39 |
| corvus | clarkb: is the diff large? | 18:40 |
| clarkb | http://paste.openstack.org/show/646948/ | 18:40 |
| clarkb | - is yours + is mine | 18:40 |
| clarkb | er did I get that backwards /me double checks | 18:40 |
| tobiash | ah, diff against parent 2 is what I want to look at | 18:40 |
| clarkb | oh I may have ogtten it backwards - is mine + is yours | 18:41 |
| clarkb | my read on that is your diff is safe and only tests should be affected but we will be running tests for master now so that should be fine | 18:42 |
| corvus | ya, i mean, we've also been running tests for feature/zuulv3 :) | 18:42 |
| corvus | i'm inclined to just overwrite master with v3; i think the little we lose is okay and easily fixable. if anyone disagrees, you're welcome to merge master into v3 and then v3 back into master. | 18:44 |
| clarkb | I've +2'd the change to indicate I think its fine | 18:44 |
| corvus | clarkb: are you going to check and review the zuul change too? | 18:45 |
| mordred | corvus: I agree with your inclination | 18:45 |
| *** jpena is now known as jpena|off | 18:46 | |
| corvus | and does anyone else want to review either of the changes, or should we go aheand and approve them? | 18:46 |
| mordred | corvus: I vote approve. | 18:48 |
| pabelanger | corvus: no objection here | 18:48 |
| tobiash | zuul failed with test failures I've seen already several times on unrelated changes | 18:52 |
| clarkb | corvus: I can if you like I'm less concerned about it after seeing nodepools since I think zuul was frozen harder | 18:53 |
| mordred | tobiash: I thnk that means the merge worked :) | 18:53 |
| tobiash | yes ;) | 18:54 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul feature/zuulv3: Do not call merger:cat when all config items are excluded https://review.openstack.org/535450 | 19:02 |
| corvus | fbo: ^ feature/zuulv3 is frozen for the merge into master; you'll need to repropose that | 19:03 |
| AJaeger | ok to merge the infra-manual change already? | 19:04 |
| corvus | i've gone ahead and approved the zuul change; it timed out on a test, but tobiash rechecked | 19:05 |
| corvus | i'm going to afk until after lunch; hopefully those will have merged by then. if folks could help push things along if you see any issues, i'd appreciate it. | 19:05 |
| mordred | corvus: ++ | 19:06 |
| AJaeger | so, zuul change is in gate - passed check queue.. | 19:17 |
| clarkb | nodepool has a couple longer check tests but once it gets to gate should go quickly | 19:18 |
| SpamapS | very exciting | 19:34 |
| * AJaeger rechecked the zuul merge change again | 19:43 | |
| tobiash | so again check and gate :-/ | 19:44 |
| AJaeger | tobiash: yeah ;( | 19:44 |
| clarkb | hrm should we be concerned the merge is causing that instability or is this normal? | 19:44 |
| AJaeger | tobiash: hope we see it before we call it a day ;) | 19:44 |
| tobiash | does nodepool really take soo long or is it hanging? | 19:45 |
| AJaeger | clarkb: the py35 job is not stable AFAIK ;( | 19:45 |
| SpamapS | haven't seen too many non-deterministic fails, but it does happen | 19:45 |
| SpamapS | there are a few "wait a second" tests that are susceptible to cloud blips I think | 19:46 |
| corvus | the two test failures are slow and busy tests; probably recently made slightly less stable by the recent addition of a bunch of new tests throwing off the scheduling. | 19:46 |
| clarkb | tobiash: it takes a long time, its building an image and booting it all without nested virt | 19:47 |
| clarkb | tobiash: also without a warm cache for image builds | 19:47 |
| clarkb | tobiash: I think ~40 minutes is normal | 19:47 |
| tobiash | AJaeger: actually I called it a day some hours ago but lurking around here in my free time a bit ;) | 19:47 |
| tobiash | clarkb: the current nodepool job is running since 90min | 19:48 |
| AJaeger | tobiash: ;) | 19:48 |
| clarkb | tobiash: hrm it actually looks like it never finished making the opnstack cloud with devstack | 19:49 |
| tobiash | the log lying... 'running devstack now this takes 10 - 15 minutes' | 19:52 |
| *** harlowja has joined #zuul | 19:52 | |
| clarkb | that almost looks like the node "went away" | 19:58 |
| clarkb | we'll have to check logs | 19:58 |
| tobiash | no luck today | 20:13 |
| tobiash | nodepool also failed the gate | 20:13 |
| tobiash | rechecked | 20:13 |
| AJaeger | and zuul merge failing again ;( | 20:15 |
| tobiash | looks like a recheck party | 20:18 |
| pabelanger | what failed? test or something on network? | 20:18 |
| * AJaeger passes on the recheck torch to the next person - if needed | 20:19 | |
| AJaeger | pabelanger: test ;( | 20:19 |
| pabelanger | possible something in merge is creating race? | 20:20 |
| pabelanger | or existing failure | 20:20 |
| pabelanger | haven't looked yet myself | 20:20 |
| clarkb | people seemed to think its due to extra tests that were recently added (and not due to the merge) | 20:20 |
| AJaeger | I saw these on zuul-jobs already in the last days... | 20:20 |
| * clarkb wanders off to find food too while we wait | 20:22 | |
| pabelanger | ah, see that in backscroll now | 20:22 |
| * tobiash gives up and raises eod | 20:25 | |
| Shrews | clarkb: seems nodepool failed on your new test | 20:25 |
| Shrews | must still be a race there somewhere? | 20:25 |
| clarkb | shrews have a link? I'll look after lunch | 20:27 |
| Shrews | clarkb: http://logs.openstack.org/35/535435/1/gate/tox-py35/d58d61e/testr_results.html.gz | 20:27 |
| Shrews | not entirely sure what has happened there | 20:28 |
| Shrews | clarkb: oh, i think maybe it's the map() call. getNode() can return None if the id you give it has gone away by the time you call it | 20:31 |
| clarkb | ah | 20:31 |
| clarkb | ya that will do it | 20:31 |
| clarkb | so we need if node and node.provider == | 20:31 |
| Shrews | yup | 20:31 |
| mordred | corvus, Shrews, clarkb, pabelanger: I got sucked in to fixing a bug caused by something getting released when I wasn't expecting it - looks like we're still waiting on the recheck dance, but ping me if I seem absent - I'm trying to bounce back and forth though | 20:40 |
| openstackgerrit | Merged openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 20:47 |
| AJaeger | party time ;) | 20:47 |
| mrhillsman | ++ | 20:59 |
| corvus | nodepool just moved to gate | 21:00 |
| AJaeger | now time to merge the job changes from https://review.openstack.org/#/q/topic:zuulv3-merge ? | 21:00 |
| corvus | AJaeger: the ones for zuul should be safe to merge, yes | 21:01 |
| corvus | 535429 has some nodepool stuff in it and so should wait | 21:02 |
| * AJaeger already gave +2 - so need second reviewer... | 21:02 | |
| corvus | approved 535432 | 21:02 |
| corvus | and 535433 | 21:03 |
| corvus | and 535434 | 21:03 |
| AJaeger | yeah, those were the job changes - thanks | 21:04 |
| AJaeger | quite a large branch merge: | 21:05 |
| AJaeger | 995 files changed, 58394 insertions(+), 19548 deletions(-) | 21:05 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove feature/zuulv3 references from README https://review.openstack.org/535481 | 21:06 |
| *** maxamillion has quit IRC | 21:08 | |
| fungi | corvus: are we considering master unfrozen just for ^ i guess? | 21:09 |
| fungi | (and maybe any other potentially misleading v3-related references that need cleaning up before merge) | 21:10 |
| AJaeger | fungi: we're after merge... | 21:10 |
| openstackgerrit | Merged openstack-infra/nodepool master: Replace master with feature/zuulv3 https://review.openstack.org/535435 | 21:10 |
| fungi | oh! | 21:10 |
| fungi | i missed the event | 21:10 |
| AJaeger | fungi: 20:47 | 21:10 |
| fungi | thought it had gotten suspiciously quiet in #openstack-infra | 21:10 |
| corvus | and there's the other event! | 21:10 |
| AJaeger | fungi: you're here just in time for the nodepool party ;) | 21:10 |
| fungi | heh | 21:11 |
| corvus | approving 535429 now | 21:11 |
| corvus | all zuulv3-merge changes are approved | 21:11 |
| clarkb | I've eaten /me is back now | 21:11 |
| corvus | how does my abandon message look in https://etherpad.openstack.org/p/3mSln6ATMP section 5? | 21:11 |
| fungi | i should have checked in here earlier in the day, forgot to consider the great merging would probably get coordinated here and not in the other channel | 21:12 |
| AJaeger | corvus: friendly ;) | 21:12 |
| corvus | sorry, i guess it's about a 50/50 split we have going on | 21:12 |
| *** maxamillion has joined #zuul | 21:12 | |
| clarkb | corvus: lgtm | 21:12 |
| AJaeger | corvus: change "is being merged into" to "has merged" | 21:13 |
| fungi | apologies if i seemed strangely absent | 21:13 |
| corvus | AJaeger: ++ | 21:13 |
| openstackgerrit | Merged openstack-infra/zuul-base-jobs master: Remove feature/zuulv3 from pip depend https://review.openstack.org/535432 | 21:13 |
| *** maxamillion has quit IRC | 21:13 | |
| *** maxamillion has joined #zuul | 21:13 | |
| fungi | abandon message in step #5 lgtm | 21:13 |
| clarkb | I'm going to get a patch up for the bug that shrews pointed out in my test | 21:14 |
| AJaeger | #success Zuul and nodepool feature/zuulv3 branches have merged into master | 21:14 |
| corvus | okay, gertty is chewing through that list now | 21:15 |
| corvus | "Sync: 114" | 21:15 |
| AJaeger | success not working here? | 21:15 |
| fungi | AJaeger: looks like openstackstatus may be out to lunch | 21:15 |
| fungi | i'll check on it | 21:15 |
| AJaeger | it's in #openstack-infra, I'll do it there... | 21:16 |
| AJaeger | fungi, openstackstatus is not run here according to system-config | 21:17 |
| fungi | yep, am nearly done writing the commit message | 21:18 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool master: Fix node data retrieval race in test_failed_provider https://review.openstack.org/535484 | 21:20 |
| clarkb | Shrews: ^ I think that will do it | 21:20 |
| AJaeger | enough party - /me calls it a day | 21:23 |
| clarkb | AJaeger: good night | 21:23 |
| fungi | thanks AJaeger! | 21:23 |
| fungi | <proxying openstackgerrit> https://review.openstack.org/535485 Have statusbot join #zuul | 21:24 |
| *** weshay|rover is now known as weshay|rover|den | 21:24 | |
| *** weshay|rover|den is now known as weshay|dentist | 21:24 | |
| corvus | email to zuul-discuss about the merge and abandoned changes sent | 21:25 |
| corvus | still waiting for gertty to confirm that all were abandoned (i think it's done abandoning and it's just resyncing them now for confirmation) | 21:25 |
| corvus | and it just finished | 21:25 |
| corvus | now we should wait until the zuulv3-merge changes have merged before deleting the branches | 21:26 |
| *** openstackgerrit has quit IRC | 21:33 | |
| mordred | corvus: ++ | 21:34 |
| *** sshnaidm is now known as sshnaidm|off | 21:36 | |
| fungi | looks like the last of them just merged | 21:38 |
| clarkb | we should probably wait for ansible puppet to idle (if not already) before deleting the branch? | 21:39 |
| corvus | clarkb, mordred: some of the items in #7 don't have changes -- is it the case that they don't need any? are we ready to delete the branch once the puppetmaster idles as clarkb suggests? | 21:40 |
| clarkb | looks like it is running right now | 21:40 |
| fungi | or are there more needed for the remaining subparts under step #7 in the etherpad? | 21:40 |
| mordred | corvus: they were just less-high priority | 21:41 |
| mordred | the openstack-ansible-* ones are plentiful - but it's all a mention in a comment | 21:41 |
| clarkb | TIL infra-core has core on ansible-role-zuul | 21:41 |
| fungi | the one listed for system-config (535426) seems to have been unceremoniously abandoned too | 21:41 |
| mordred | fungi: oh - sorry - clarkb made one too that has been approved | 21:41 |
| fungi | okay, cool, just making sure | 21:42 |
| clarkb | we wrote two patches both of them at the same time because i am a derp | 21:42 |
| fungi | no worries | 21:42 |
| mordred | but then clarkb's wound up being nicer | 21:42 |
| clarkb | I don't see changes to ansible-role-zuul or nodepool | 21:42 |
| clarkb | pabelanger: ^ is that urgent for you? I've never used or read the roles before so hesitate to start doing surgery on them | 21:42 |
| fungi | clarkb: which one did you say was still running? i don't see any open under https://review.openstack.org/#/q/is:open+topic:zuulv3-merge | 21:43 |
| clarkb | fungi: ansible puppet on puppetmaster | 21:44 |
| fungi | oh, _that_ | 21:44 |
| fungi | yup | 21:44 |
| clarkb | I think it may have just stopped and will start a new run in a minute (which will pull updated code which should be fine) | 21:44 |
| *** hasharDinner has quit IRC | 21:45 | |
| clarkb | ya it started again so I think we are clear on that front | 21:47 |
| clarkb | so just ansible-role-zuul and ansible-role-nodepool | 21:47 |
| *** rlandy is now known as rlandy|brb | 21:48 | |
| corvus | i guess let's give pabelanger a few minutes to weigh in | 21:49 |
| *** threestrands has joined #zuul | 21:56 | |
| *** threestrands has quit IRC | 21:56 | |
| *** threestrands has joined #zuul | 21:56 | |
| Shrews | the openstack-manuals change is slightly more complicated since the sitemap.xml is generated by walking docs.o.o (according to AJaeger), so I guess that one needs to wait | 21:57 |
| clarkb | oh that brings up an interesting problem. For feature branches do we manually delete their docs? | 21:58 |
| clarkb | or juts let them live on because lazy | 21:59 |
| clarkb | actually we get ref updated events for branch deletes we could have jobs unpublish stuff | 21:59 |
| fungi | i suppose we could add redirects in case anyone had bookmarked some deep link | 21:59 |
| corvus | i'd be fine just deleting it. it's a feature branch :) | 22:01 |
| clarkb | ++ | 22:01 |
| fungi | wfm | 22:01 |
| corvus | shall we go ahead and delete the branches now? | 22:02 |
| fungi | doesn't seem to be any other blockers in the pad | 22:02 |
| corvus | i don't think we need to eol-tag these branches since they were merged. | 22:04 |
| mordred | corvus: agree. and also think deleting is a great idea | 22:04 |
| fungi | agreed | 22:04 |
| corvus | and the merge commit second parent should be sufficient recording of the sha for posterity | 22:04 |
| corvus | done | 22:05 |
| clarkb | ya you can follow the code back from the merge commit so I agree | 22:05 |
| corvus | i believe that concludes this operation :) | 22:06 |
| mordred | \o/ | 22:06 |
| Shrews | end of an era | 22:06 |
| mordred | srrsly | 22:06 |
| corvus | beginning of a new one? | 22:06 |
| *** rlandy|brb is now known as rlandy | 22:09 | |
| fungi | time to create the feature/zuulv4 branch! ;) | 22:12 |
| * fungi ducks | 22:13 | |
| *** openstackgerrit has joined #zuul | 22:18 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix node data retrieval race in test_failed_provider https://review.openstack.org/535484 | 22:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Handle secrets in branches https://review.openstack.org/535501 | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Handle nodesets in branches https://review.openstack.org/535502 | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Normalize semaphore branch handling https://review.openstack.org/535503 | 22:25 |
| corvus | i'm going to abandon the original github integration changes with a nice message | 22:32 |
| SpamapS | this is not the end. This is not the beginning of the end. But perhaps, this is the end of the beginning of github support. ;) | 22:33 |
| corvus | that's, erm, shorter than what i wrote. | 22:33 |
| corvus | that's 97->75 open changes | 22:34 |
| *** dkranz has quit IRC | 22:36 | |
| dmsimard | How do you drain all the nodes from nodepool again ? need to set max-servers to -1 right ? does that kill ongoing jobs ? | 22:37 |
| dmsimard | (This is for RDO) | 22:37 |
| corvus | dmsimard: does not kill jobs. max servers can be 0, or you can drop the label from the providers. | 22:37 |
| corvus | -1 also works | 22:38 |
| corvus | i'm planning on using this message for changes that look like they are no longer relevant: http://paste.openstack.org/show/646960/ | 22:39 |
| corvus | how does that sound? | 22:39 |
| dmsimard | iirc there was a difference between max-servers 0 and -1 | 22:40 |
| mordred | corvus: sounds good | 22:40 |
| dmsimard | I think 0 lets the servers drain over time (i.e, after they're consumed nodepool doesn't respawn more nodes) while -1 would kill "ready" nodes for example | 22:41 |
| corvus | dmsimard: the difference was whether images are deleted | 22:41 |
| corvus | my preferenc is for the new method which is just to remove things from the config which you don't want. that seems clear to me. :) | 22:41 |
| corvus | (though, if you remove the provider entirely, then obviously it can't delete things, so sometimes you want a provider with an empty label list) | 22:42 |
| corvus | dmsimard: just 'nodepool delete' the ready nodes after you've quiesced the provider | 22:42 |
| dmsimard | last time I put max-servers 0 it would just say in the logs "woops I'm over quota" but wouldn,t actively delete things | 22:42 |
| dmsimard | corvus: yeah that sounds like a plan | 22:43 |
| corvus | i don't believe anything will delete ready nodes other than a timeout | 22:43 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Do not call merger:cat when all config items are excluded https://review.openstack.org/535509 | 22:43 |
| corvus | (*automatically) | 22:43 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config https://review.openstack.org/535511 | 22:45 |
| dmsimard | corvus: yeah that's fine, I just thought -1 did it automatically | 22:46 |
| corvus | okay, i did a pass through open changes and removed a bunch that looked like they no longer applied. i'm sure there's more, but the remainders may take a little more thought. | 22:49 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Normalize semaphore branch handling https://review.openstack.org/535503 | 22:53 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Remove zuul._projects https://review.openstack.org/535518 | 23:10 |
| openstackgerrit | Merged openstack-infra/zuul master: Remove feature/zuulv3 references from README https://review.openstack.org/535481 | 23:15 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Remove zuul._projects https://review.openstack.org/535518 | 23:46 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!