| *** lathiat has quit IRC | 00:00 | |
| *** akhilaki_ has quit IRC | 00:01 | |
| *** sdake has quit IRC | 00:01 | |
| *** vkmc has quit IRC | 00:02 | |
| *** rcernin_ has quit IRC | 00:02 | |
| *** zhipeng has quit IRC | 00:03 | |
| *** sdake has joined #openstack-infra | 00:06 | |
| *** sdake has quit IRC | 00:06 | |
| *** sdake has joined #openstack-infra | 00:06 | |
| *** lathiat_ has joined #openstack-infra | 00:06 | |
| *** s-shiono has joined #openstack-infra | 00:06 | |
| *** jhesketh has quit IRC | 00:06 | |
| *** lathiat_ has quit IRC | 00:06 | |
| *** lathiat_ has joined #openstack-infra | 00:06 | |
| *** vkmc has joined #openstack-infra | 00:06 | |
| *** s-shiono has quit IRC | 00:06 | |
| *** lathiat_ has quit IRC | 00:06 | |
| *** lbragstad has quit IRC | 00:06 | |
| *** annp has quit IRC | 00:06 | |
| *** mschuppert has quit IRC | 00:06 | |
| *** lucasagomes has quit IRC | 00:06 | |
| *** fabo_ has quit IRC | 00:06 | |
| *** calbers_ has quit IRC | 00:06 | |
| *** sshnaidm|off has quit IRC | 00:06 | |
| *** jamesdenton has quit IRC | 00:06 | |
| *** russellb has quit IRC | 00:06 | |
| *** rnoriega has quit IRC | 00:06 | |
| *** adam_g has quit IRC | 00:06 | |
| *** jogo has quit IRC | 00:06 | |
| *** flaper87 has quit IRC | 00:06 | |
| *** gagehugo has quit IRC | 00:06 | |
| *** amotoki has quit IRC | 00:06 | |
| *** mudpuppy has quit IRC | 00:06 | |
| *** frickler has quit IRC | 00:06 | |
| *** dosaboy has quit IRC | 00:06 | |
| *** gmann has quit IRC | 00:06 | |
| *** tbarron has quit IRC | 00:06 | |
| *** eandersson has quit IRC | 00:06 | |
| *** shoogz has quit IRC | 00:09 | |
| *** bandini has quit IRC | 00:09 | |
| *** linshuicheng[m] has quit IRC | 00:10 | |
| *** warlord77[m] has quit IRC | 00:10 | |
| *** aspiers[m] has quit IRC | 00:10 | |
| *** lbragstad[m] has quit IRC | 00:10 | |
| *** dhellmann[m] has quit IRC | 00:10 | |
| *** austinsun[m] has quit IRC | 00:10 | |
| *** markvoelker has quit IRC | 00:10 | |
| *** smcginnis[m] has quit IRC | 00:10 | |
| *** felipemonteiro has joined #openstack-infra | 00:11 | |
| *** slaweq has joined #openstack-infra | 00:11 | |
| *** bandini has joined #openstack-infra | 00:11 | |
| *** shoogz has joined #openstack-infra | 00:14 | |
| *** jhesketh has joined #openstack-infra | 00:15 | |
| *** rcernin_ has joined #openstack-infra | 00:15 | |
| *** s-shiono has joined #openstack-infra | 00:15 | |
| *** lathiat_ has joined #openstack-infra | 00:15 | |
| *** lbragstad has joined #openstack-infra | 00:15 | |
| *** annp has joined #openstack-infra | 00:15 | |
| *** mschuppert has joined #openstack-infra | 00:15 | |
| *** lucasagomes has joined #openstack-infra | 00:15 | |
| *** fabo_ has joined #openstack-infra | 00:15 | |
| *** calbers_ has joined #openstack-infra | 00:15 | |
| *** sshnaidm|off has joined #openstack-infra | 00:15 | |
| *** jamesdenton has joined #openstack-infra | 00:15 | |
| *** russellb has joined #openstack-infra | 00:15 | |
| *** rnoriega has joined #openstack-infra | 00:15 | |
| *** adam_g has joined #openstack-infra | 00:15 | |
| *** jogo has joined #openstack-infra | 00:15 | |
| *** flaper87 has joined #openstack-infra | 00:15 | |
| *** gagehugo has joined #openstack-infra | 00:15 | |
| *** mudpuppy has joined #openstack-infra | 00:15 | |
| *** amotoki has joined #openstack-infra | 00:15 | |
| *** frickler has joined #openstack-infra | 00:15 | |
| *** dosaboy has joined #openstack-infra | 00:15 | |
| *** gmann has joined #openstack-infra | 00:15 | |
| *** tbarron has joined #openstack-infra | 00:15 | |
| *** eandersson has joined #openstack-infra | 00:15 | |
| *** aeng has joined #openstack-infra | 00:15 | |
| *** slaweq has quit IRC | 00:16 | |
| corvus | okay that took a little longer than i thought it should, due to tracing a bunch of code through requests/httplib, but here's what i came up with: http://paste.openstack.org/show/726314/ | 00:16 |
|---|---|---|
| corvus | seems to work | 00:16 |
| clarkb | that look about right | 00:18 |
| corvus | what that ends up doing in practice is putting about 4 16k chunks into the compressor with nothing coming out, then a 5th chunk producing 30k of output. generally speaking. | 00:18 |
| clarkb | I'm fighting weird routes to my ovh server. I was getting to montreal area by way of singapore and france | 00:19 |
| clarkb | seems fixed now but irc was basically unusable for 5-10 minutes | 00:19 |
| corvus | that sounds like a lovely trip | 00:20 |
| *** felipemonteiro has quit IRC | 00:22 | |
| *** threestrands has joined #openstack-infra | 00:23 | |
| *** threestrands has quit IRC | 00:23 | |
| *** threestrands has joined #openstack-infra | 00:23 | |
| *** markvoelker has joined #openstack-infra | 00:24 | |
| *** dpawlik has joined #openstack-infra | 00:24 | |
| *** dklyle has quit IRC | 00:24 | |
| *** dpawlik has quit IRC | 00:28 | |
| *** slaweq has joined #openstack-infra | 00:35 | |
| *** jamesmcarthur has joined #openstack-infra | 00:36 | |
| *** slaweq has quit IRC | 00:40 | |
| *** rcernin_ has quit IRC | 00:43 | |
| *** rcernin has joined #openstack-infra | 00:44 | |
| *** zaneb has left #openstack-infra | 00:48 | |
| *** warlord77[m] has joined #openstack-infra | 00:49 | |
| *** robled has quit IRC | 00:50 | |
| *** robled has joined #openstack-infra | 00:53 | |
| *** robled has joined #openstack-infra | 00:53 | |
| corvus | clarkb, mordred: something to ponder: how should we get an environment with python-openstacksdk on the executor? i don't think we'll want to create a venv for every job. should we install it globally? as the zuul user and map it in via bubblewrap? in a dedicated venv on the system and map it in via bubblewrap? | 00:54 |
| mordred | corvus: I'd say either globally or in a dedicated. we're instlaling zuul globally though so I don't think installing sdk globally would be much worse | 00:56 |
| *** akhilaki__ has quit IRC | 00:56 | |
| *** akhilaki has quit IRC | 00:56 | |
| mordred | but then yeah- map in via bwrap | 00:56 |
| mordred | oh, wait | 00:56 |
| mordred | I mean - it'll get in to bwrap just cause it's there | 00:56 |
| * mordred is not saying good word noises | 00:56 | |
| corvus | if it's installed system-wide, yeah | 00:57 |
| corvus | if we do a user or venv install, we'd need to map something in | 00:57 |
| *** r-daneel has quit IRC | 00:57 | |
| mordred | yah | 00:57 |
| corvus | i guess if we were to implement it as an ansible module, it'd need to either be system or user installed (not a dedicated venv) | 00:58 |
| corvus | this may be an interesting thing to keep in mind when we think about supporting multiple ansible versions. like, maybe however we create *those* venvs, we should think about having the ability to specify other packages to install in them. | 00:59 |
| *** dave-mccowan has quit IRC | 00:59 | |
| corvus | dinner is ready... will think more and regroup tomorrow | 00:59 |
| *** felipemonteiro has joined #openstack-infra | 00:59 | |
| *** r-daneel has joined #openstack-infra | 01:00 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: configure-mirror: install unbound if missing https://review.openstack.org/584171 | 01:07 |
| *** yamahata has quit IRC | 01:08 | |
| *** tpatil has joined #openstack-infra | 01:09 | |
| *** harlowja has quit IRC | 01:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: configure-mirror: install unbound if missing https://review.openstack.org/584171 | 01:09 |
| *** slaweq has joined #openstack-infra | 01:11 | |
| tpatil | Hi Infra team, I need your help - We are trying to build a multi-node job for tacker project in this patch https://review.openstack.org/#/c/578170/33 | 01:11 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: configure-mirror: install unbound if missing https://review.openstack.org/584171 | 01:13 |
| tpatil | Compute service fails to start because n-cond service on the controller node is not started by the time compute service expects response from RPC call. It timed out and hence compute service fails to start. | 01:13 |
| tpatil | How do we ensure all services on controller node are started before those on compute node? | 01:13 |
| *** slaweq has quit IRC | 01:16 | |
| *** ijw has quit IRC | 01:18 | |
| *** r-daneel has quit IRC | 01:18 | |
| *** mrsoul has joined #openstack-infra | 01:19 | |
| *** jamesmcarthur has quit IRC | 01:19 | |
| *** eernst has joined #openstack-infra | 01:20 | |
| *** ijw has joined #openstack-infra | 01:20 | |
| *** jamesmcarthur has joined #openstack-infra | 01:21 | |
| *** mschuppert has quit IRC | 01:23 | |
| pabelanger | config-core: https://review.openstack.org/584171/ is a small update to configure-unbound that will now install unbound if missing, this fixes a use case where DIB didn't actually install unbound to start with | 01:23 |
| pabelanger | dmsimard: ^ | 01:23 |
| *** linshuicheng[m] has joined #openstack-infra | 01:24 | |
| *** smcginnis[m] has joined #openstack-infra | 01:24 | |
| *** austinsun[m] has joined #openstack-infra | 01:24 | |
| *** dhellmann[m] has joined #openstack-infra | 01:24 | |
| *** lbragstad[m] has joined #openstack-infra | 01:24 | |
| *** aspiers[m] has joined #openstack-infra | 01:24 | |
| ianw | istr we did it like that for a reason | 01:25 |
| pabelanger | it should be a noop if package is found | 01:25 |
| dmsimard | I don't think there was a good reason we did it that way | 01:26 |
| dmsimard | other than failing because we're not expecting unbound to not be there | 01:26 |
| pabelanger | if package does refresh indexes, we could query another way to see if package is missing | 01:26 |
| dmsimard | it's just a cleaner verbose way to fail there than, say, on a random file task because unbound isn't installed | 01:26 |
| pabelanger | dmsimard: right, seems implied in role name you want to actually configure-unbound :) | 01:26 |
| ianw | no, iirc it was more that unbound *not* being there at that point was an error, because it completely relies on the configure-unbound element to set it up before that | 01:26 |
| dmsimard | installing unbound if it's not installed probably makes sense | 01:26 |
| ianw | that role is not really configuring unbound, i think | 01:27 |
| dmsimard | it has some stuff for TTLs | 01:27 |
| ianw | right, but that *needs* the setup done by nodepool-base element | 01:27 |
| pabelanger | it uses default configuration and sets up forwarding.conf | 01:27 |
| pabelanger | I don't think it needs to be setup in DIB element, just we still do that in openstack-infra | 01:28 |
| pabelanger | other thing element does is setup resolv.conf | 01:29 |
| ianw | see the note : https://docs.openstack.org/infra/openstack-zuul-jobs/roles.html#role-configure-unbound | 01:29 |
| pabelanger | which, I'm writing another role to do that | 01:29 |
| *** yamahata has joined #openstack-infra | 01:30 | |
| pabelanger | I'll double check the default for centos-7, but think most of that code in DIB could be a role now. biggest thing to check here, is that forwarding.conf loads on default settings | 01:31 |
| *** mriedem_away has quit IRC | 01:31 | |
| pabelanger | which I am pretty sure it does | 01:32 |
| pabelanger | include: /etc/unbound/conf.d/*.conf | 01:32 |
| *** hongbin_ has joined #openstack-infra | 01:32 | |
| ianw | i'm thinking the issue that keeps it in nodepool is that we want it on boot | 01:32 |
| *** eernst has quit IRC | 01:32 | |
| ianw | sorry, dib, not nodepool | 01:32 |
| *** felipemonteiro has quit IRC | 01:32 | |
| pabelanger | I think that is for legacy reasons now, we used to run a ready-script in nodepool | 01:33 |
| pabelanger | and did package installs | 01:34 |
| *** gcb has quit IRC | 01:34 | |
| pabelanger | with zuulv3, I can't think of a reason for keeping that in a DIB element | 01:34 |
| pabelanger | in the case here, rdoproject doesn't setup unbound on elements for some reason | 01:34 |
| pabelanger | and this seems to just work by using the configure-unbound role and some updates to resolvconf to use localhost | 01:35 |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config master: Remove legacy scenario multinode job for nova https://review.openstack.org/583849 | 01:35 |
| ianw | finalise.d/89-unbound does a lot of stuff | 01:36 |
| *** jamesmcarthur has quit IRC | 01:37 | |
| pabelanger | yah for openstack-infra, there is a lot of images to deal with. downstream in rdoproject, just centos-7 and fedora right now | 01:37 |
| pabelanger | I'll do more testing, but so far don't need any of finalise.d stuff for the image | 01:38 |
| *** dave-mccowan has joined #openstack-infra | 01:38 | |
| pabelanger | there is also a use case where we might be using cloud-images in nodepool, so won't actually be able to configure unbound via DIB. We'll likely need to shore the configure-unbound role for that use case too | 01:39 |
| *** jamesmcarthur has joined #openstack-infra | 01:39 | |
| ianw | do you really even need configure unbound anyway? | 01:39 |
| pabelanger | would like dns caching | 01:39 |
| pabelanger | been an issue in the past for downstream jobs | 01:39 |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/openstack-zuul-jobs master: Remove legacy scenario job https://review.openstack.org/584190 | 01:44 |
| *** dave-mccowan has quit IRC | 01:52 | |
| *** Bhujay has joined #openstack-infra | 01:52 | |
| *** AhmadM has quit IRC | 01:54 | |
| *** AhmadM_ is now known as AhmadM | 01:54 | |
| *** yamahata has quit IRC | 02:06 | |
| *** slaweq has joined #openstack-infra | 02:11 | |
| *** slaweq has quit IRC | 02:16 | |
| *** rlandy|bbl is now known as rlandy | 02:17 | |
| *** agopi has joined #openstack-infra | 02:17 | |
| *** roman_g has quit IRC | 02:26 | |
| *** Bhujay has quit IRC | 02:30 | |
| tonyb | Silly question does the old Change-Id based Depends-On still work? | 02:33 |
| pabelanger | yes, but going to be removed in the future, IIRC | 02:33 |
| * tonyb would like to get a cross-project series tother and publish it in a hit rather then dribs and drabs | 02:33 | |
| *** felipemonteiro has joined #openstack-infra | 02:34 | |
| tonyb | pabelanger: Okay as long "the future" isn't in the next 7 days I'm cool with that ;P | 02:34 |
| pabelanger | tonyb: you're don't doing cross project testing with rdoproject by chance? | 02:34 |
| *** tpatil has quit IRC | 02:35 | |
| tonyb | pabelanger: Nope, it's openstack/{governance,governance-sigs,election} | 02:35 |
| tonyb | pabelanger: but that's a good point, could I have added a Depends-On: https://review.rdoproject.org/14519 (or similar) to a python-tripleoclient change? | 02:37 |
| *** eernst has joined #openstack-infra | 02:37 | |
| * tonyb doubts the dlrn stuff would have worked correctly but it's be super aweom if it did | 02:37 | |
| tonyb | and it'd also be nice if the python-tripleoclient wouldn't merge until the rdo one did | 02:37 |
| pabelanger | tonyb: no in zuul.o.o, but downstream in rdoproject, you can do depends-on: https://review.o.o and will properly work | 02:37 |
| tonyb | pabelanger: Okay. good to know | 02:38 |
| pabelanger | to go other way, we'd need to configure zuul.o.o to talk with rdoproject gerrit, would need a discussion I think | 02:38 |
| *** yamahata has joined #openstack-infra | 02:39 | |
| tonyb | pabelanger: that makes sense | 02:39 |
| clarkb | tpatil is gone but we run controller setup before compute hosts on existing multinode jobs for this reason | 02:39 |
| *** links has joined #openstack-infra | 02:40 | |
| clarkb | pabelanger: we shouldnt remove setup like unbound or firewall from dib | 02:40 |
| clarkb | you want that stuff on boot | 02:40 |
| pabelanger | clarkb: right, not suggesting removing it here in openstack-infra, but there is use case down stream where that isn't done in DIB, and think there is an argument to have a base job configure unbound | 02:41 |
| clarkb | pabelanger: maybe, but why not fix the image instead? | 02:42 |
| pabelanger | clarkb: nodepool elements today live in another code review system, so it will take time and energy to land them, I don't have +A for those. Best I can do now, is fix jobs which I have control over | 02:43 |
| *** adrianreza has joined #openstack-infra | 02:44 | |
| pabelanger | there is also the use case of cloud-images, I may not want to deal with building them again, but haven't figured that out yet | 02:44 |
| clarkb | if you control the nodepool config you can load whatever elements... | 02:44 |
| pabelanger | I don't today, that is in different zuul tenant | 02:45 |
| clarkb | as for using cloud provided images I think you are atthe mercy of hwoever cloud configuresstuff in that case | 02:45 |
| pabelanger | rdoproject is another, which doesn't manage nodepool | 02:45 |
| clarkb | its low effort but may not be the most reliable | 02:45 |
| clarkb | (generally though you'd expect working dns from a cloud and its images but experience saysnot always :( ) | 02:46 |
| pabelanger | Yah, I hear that. But honestly, I think it is a fair use case for a zuul user to want to configure DNS unbound role, then don't have to touch openstack tenant or nodepool | 02:47 |
| pabelanger | but brain is melted, will think more on it with fresh eyes in morning | 02:47 |
| clarkb | I dont think its the wrong idea I just think its the wrong place to do it | 02:47 |
| clarkb | you need a set of base functionality at boot, firewall, dns, ssh, ntp and probably a few others I dont remember | 02:48 |
| *** psachin` has joined #openstack-infra | 02:48 | |
| clarkb | otherwise you may not get far enough to bootstrap reliable ansible necessary to run those roles | 02:49 |
| pabelanger | right, there is a small window of DNS failure, but might be willing to take that over dealing with DIB element or cloud config | 02:50 |
| pabelanger | configure-unbound is one of the first roles to run after SSH into the node | 02:50 |
| pabelanger | so, really, if DNS lookup on SSH fails, zuul would retry launch I think | 02:51 |
| pabelanger | or just timeout SSH | 02:51 |
| *** lifeless has quit IRC | 02:51 | |
| clarkb | it would delete the node and try again up to retry failure I think | 02:51 |
| mordred | I could see unbound a base job instead of element - one could imagine that the images from the cloud work for the most part, and unbound being a way to make those nodes "better" for testing purposes | 02:52 |
| mordred | in that way, unbound is really more like the per-region mirrors | 02:52 |
| clarkb | mordred: except for clouds that blacklist your dns traffic :) | 02:52 |
| clarkb | that was what pushed us to unbound in the first place | 02:52 |
| mordred | yay broken dns | 02:53 |
| mordred | there's definitely a "tenant admin" vs. "zuul admin" roles and responsibilities thing going on here | 02:53 |
| mordred | which is an interesting thing we haven't really had to deal with in infra land yet | 02:53 |
| pabelanger | ++ | 02:54 |
| pabelanger | 2018 and still 20 ways to configure /etc/resolv.conf | 02:54 |
| *** threestrands has quit IRC | 02:54 | |
| mordred | pabelanger: I feel that we've only grown new ways | 02:54 |
| mordred | without removing any of the old ways | 02:54 |
| clarkb | what I'd be concerned about is making it appear the best way isto fix this stuff in jobs | 02:54 |
| clarkb | its too late | 02:55 |
| pabelanger | mordred: IKR | 02:55 |
| pabelanger | clarkb: yah, in the use case of blacklisted DNS I can see that | 02:55 |
| mordred | yah - I think there's two different use cases for the same sort of thing - fix broken, or optimize | 02:56 |
| clarkb | ultimately the idea is you get root in jobs to do what you need thats fine but the images should have base level of functionality too | 02:56 |
| pabelanger | yes, I'm trying to optimize right now | 02:56 |
| mordred | yup | 02:56 |
| clarkb | and working dns on boot is useful :) | 02:57 |
| pabelanger | ++ | 02:57 |
| mordred | clarkb: that's not very cloud native of you | 02:57 |
| *** lifeless has joined #openstack-infra | 02:58 | |
| *** jamesmcarthur has quit IRC | 02:58 | |
| mordred | dns is legacy. you should be doing service discovery using a network-based microservice | 02:58 |
| clarkb | if you do dna right it is service discover :P see afs | 02:59 |
| mordred | clarkb: my point exactly. it's possible my delivery was too dry | 02:59 |
| mordred | clarkb: did you ever see or use hesiod? | 03:00 |
| clarkb | I know | 03:00 |
| clarkb | I dont thibk so | 03:00 |
| clarkb | wow its yp on dns | 03:00 |
| mordred | yup | 03:01 |
| mordred | any time anyone tells me anything about service discovery, I cannot help but think of hesiod | 03:02 |
| *** rh-jelabarre has quit IRC | 03:03 | |
| *** armax has joined #openstack-infra | 03:05 | |
| *** rh-jelabarre has joined #openstack-infra | 03:06 | |
| *** ijw has quit IRC | 03:09 | |
| *** threestrands has joined #openstack-infra | 03:10 | |
| *** threestrands has quit IRC | 03:10 | |
| *** threestrands has joined #openstack-infra | 03:10 | |
| *** slaweq has joined #openstack-infra | 03:11 | |
| *** jamesmcarthur has joined #openstack-infra | 03:14 | |
| *** slaweq has quit IRC | 03:15 | |
| *** jamesmcarthur has quit IRC | 03:18 | |
| *** eernst has quit IRC | 03:18 | |
| *** felipemonteiro_ has joined #openstack-infra | 03:18 | |
| *** felipemonteiro has quit IRC | 03:21 | |
| *** ramishra has joined #openstack-infra | 03:33 | |
| *** dklyle has joined #openstack-infra | 03:37 | |
| mnaser | is there some sort of character control over job names | 03:38 |
| *** bhavik1 has joined #openstack-infra | 03:42 | |
| clarkb | mnaser I dont think so other thanno / | 03:47 |
| mnaser | i just saw an interesting use of ':' and im wondering if that might impact things | 03:47 |
| *** aeng has quit IRC | 03:48 | |
| *** ykarel has joined #openstack-infra | 03:52 | |
| *** lpetrut has joined #openstack-infra | 03:53 | |
| *** udesale has joined #openstack-infra | 04:07 | |
| *** slaweq has joined #openstack-infra | 04:11 | |
| *** Bhujay has joined #openstack-infra | 04:14 | |
| *** slaweq has quit IRC | 04:16 | |
| *** hongbin_ has quit IRC | 04:22 | |
| *** ijw has joined #openstack-infra | 04:24 | |
| *** dpawlik has joined #openstack-infra | 04:24 | |
| *** dklyle has quit IRC | 04:26 | |
| *** dpawlik has quit IRC | 04:28 | |
| *** ijw has quit IRC | 04:28 | |
| *** harlowja has joined #openstack-infra | 04:30 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/job/{job_name} route https://review.openstack.org/550978 | 04:34 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/projects and /{tenant}/project/{project} routes https://review.openstack.org/550979 | 04:34 |
| *** lpetrut has quit IRC | 04:35 | |
| *** bhavik1 has quit IRC | 04:38 | |
| *** threestrands has quit IRC | 04:40 | |
| *** jiapei has joined #openstack-infra | 04:44 | |
| *** lpetrut has joined #openstack-infra | 04:46 | |
| *** harlowja has quit IRC | 04:52 | |
| *** lpetrut has quit IRC | 04:57 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/projects and /{tenant}/project/{project} routes https://review.openstack.org/550979 | 05:00 |
| *** pcaruana has joined #openstack-infra | 05:09 | |
| *** felipemonteiro has joined #openstack-infra | 05:11 | |
| *** slaweq has joined #openstack-infra | 05:11 | |
| *** psachin` has quit IRC | 05:12 | |
| *** felipemonteiro has quit IRC | 05:15 | |
| *** slaweq has quit IRC | 05:16 | |
| *** felipemonteiro has joined #openstack-infra | 05:19 | |
| tonyb | How/can do I map an nodepool provider name back to the company that physically hosts the systems | 05:22 |
| *** pcichy has quit IRC | 05:25 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Add variables to project https://review.openstack.org/584230 | 05:25 |
| ianw | tonyb: it's pretty much in the name? | 05:25 |
| tonyb | ianw: Okay. | 05:26 |
| tonyb | ianw: I thought that the arm64 systems were provided by linaro but hosted elsewhere | 05:26 |
| ianw | i'm not sure of the *exact* situation there (gema?) but AIUI linaro provides the hardware+networking+people for it all | 05:28 |
| ianw | where i guess (hardware+networking+people) = a cloud :) | 05:28 |
| tonyb | ianw: Okay. I could be very mistaken .. beer may have been involved ;p | 05:29 |
| ianw | tonyb: you can see the end-points hte nodepool clouds.yaml talks to via https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/nodepool/clouds.yaml.erb | 05:31 |
| ianw | to be specific :) | 05:31 |
| *** quiquell|off is now known as quiquell | 05:31 | |
| tonyb | ianw: Cool thanks. | 05:32 |
| *** pawelzny has joined #openstack-infra | 05:33 | |
| *** aeng has joined #openstack-infra | 05:39 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Add variables to project https://review.openstack.org/584230 | 05:48 |
| *** zoli has quit IRC | 05:48 | |
| *** dmellado has quit IRC | 05:48 | |
| *** dulek has quit IRC | 05:49 | |
| *** gouthamr has quit IRC | 05:49 | |
| *** graphene has quit IRC | 05:49 | |
| *** stevebaker has quit IRC | 05:49 | |
| *** graphene has joined #openstack-infra | 05:51 | |
| *** zoli has joined #openstack-infra | 05:53 | |
| *** dulek has joined #openstack-infra | 05:55 | |
| *** aeng has quit IRC | 05:57 | |
| *** ramishra has quit IRC | 05:58 | |
| *** ramishra has joined #openstack-infra | 05:59 | |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** Seb-Solo` has joined #openstack-infra | 06:14 | |
| *** Seb-Solon has quit IRC | 06:15 | |
| *** slaweq has quit IRC | 06:16 | |
| *** kjackal has joined #openstack-infra | 06:17 | |
| *** lpetrut has joined #openstack-infra | 06:19 | |
| *** ijw has joined #openstack-infra | 06:25 | |
| *** mrsoul is now known as mschuppert | 06:25 | |
| *** udesale_ has joined #openstack-infra | 06:28 | |
| *** stevebaker has joined #openstack-infra | 06:29 | |
| *** dmellado has joined #openstack-infra | 06:29 | |
| *** ijw has quit IRC | 06:30 | |
| *** udesale has quit IRC | 06:30 | |
| *** gouthamr has joined #openstack-infra | 06:31 | |
| *** janki has joined #openstack-infra | 06:35 | |
| *** rlandy has quit IRC | 06:45 | |
| *** zhangfei has joined #openstack-infra | 06:48 | |
| *** kjackal has quit IRC | 06:49 | |
| *** aojea has joined #openstack-infra | 06:51 | |
| *** pguimaraes has joined #openstack-infra | 06:51 | |
| *** felipemonteiro has quit IRC | 06:56 | |
| *** armax has quit IRC | 06:57 | |
| *** tesseract has joined #openstack-infra | 07:02 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: add rocky branch to features.yaml https://review.openstack.org/584100 | 07:02 |
| *** rcernin has quit IRC | 07:02 | |
| *** jtomasek has joined #openstack-infra | 07:05 | |
| *** slaweq has joined #openstack-infra | 07:10 | |
| *** alexchadin has joined #openstack-infra | 07:11 | |
| *** hashar has joined #openstack-infra | 07:11 | |
| *** e0ne has joined #openstack-infra | 07:13 | |
| *** jamesmcarthur has joined #openstack-infra | 07:14 | |
| *** jamesmcarthur has quit IRC | 07:19 | |
| *** ccamacho has joined #openstack-infra | 07:22 | |
| *** jiapei has quit IRC | 07:24 | |
| *** peereb has joined #openstack-infra | 07:26 | |
| *** tosky has joined #openstack-infra | 07:27 | |
| *** kjackal has joined #openstack-infra | 07:27 | |
| *** annp has quit IRC | 07:28 | |
| *** dpawlik has joined #openstack-infra | 07:29 | |
| *** AhmadM_ has joined #openstack-infra | 07:30 | |
| *** vaidy has quit IRC | 07:30 | |
| *** zhongjun___ has joined #openstack-infra | 07:30 | |
| *** zhongjun__ has quit IRC | 07:31 | |
| *** isviridov_away has quit IRC | 07:31 | |
| *** zhongjun___ is now known as zhongjun__ | 07:31 | |
| *** hamzy_ has joined #openstack-infra | 07:31 | |
| *** agopi_ has joined #openstack-infra | 07:31 | |
| *** hamzy has quit IRC | 07:31 | |
| *** jamielennox has quit IRC | 07:31 | |
| *** Bhujay has quit IRC | 07:32 | |
| *** agopi has quit IRC | 07:32 | |
| *** vkmc has quit IRC | 07:32 | |
| *** AhmadM has quit IRC | 07:32 | |
| *** AhmadM_ is now known as AhmadM | 07:32 | |
| *** Bhujay has joined #openstack-infra | 07:33 | |
| *** udesale__ has joined #openstack-infra | 07:33 | |
| *** udesale_ has quit IRC | 07:36 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Stop installing puppet with puppet https://review.openstack.org/581596 | 07:39 |
| *** jchhatbar has joined #openstack-infra | 07:46 | |
| *** janki has quit IRC | 07:48 | |
| *** hamzy_ has quit IRC | 07:50 | |
| *** pcichy has joined #openstack-infra | 07:55 | |
| *** electrofelix has joined #openstack-infra | 07:56 | |
| *** jchhatbar has quit IRC | 07:56 | |
| *** jchhatbar has joined #openstack-infra | 07:57 | |
| *** gfidente has joined #openstack-infra | 08:00 | |
| *** gfidente has joined #openstack-infra | 08:00 | |
| *** kjackal has quit IRC | 08:02 | |
| *** jamielennox has joined #openstack-infra | 08:05 | |
| *** ykarel is now known as ykarel|lunch | 08:05 | |
| *** jchhatbar has quit IRC | 08:08 | |
| *** jchhatbar has joined #openstack-infra | 08:08 | |
| *** shardy has joined #openstack-infra | 08:10 | |
| *** jchhatbar has quit IRC | 08:13 | |
| *** jchhatbar has joined #openstack-infra | 08:14 | |
| *** zhangfei has quit IRC | 08:14 | |
| *** s-shiono has quit IRC | 08:16 | |
| *** owalsh is now known as owalsh_afk | 08:20 | |
| *** Gorian has quit IRC | 08:21 | |
| *** zhangfei has joined #openstack-infra | 08:26 | |
| *** jchhatbar has quit IRC | 08:30 | |
| *** Gorian has joined #openstack-infra | 08:30 | |
| *** jchhatbar has joined #openstack-infra | 08:30 | |
| *** dbecker has joined #openstack-infra | 08:44 | |
| *** panda|off is now known as panda | 08:51 | |
| *** pbourke has quit IRC | 08:53 | |
| *** pbourke has joined #openstack-infra | 08:53 | |
| *** hashar is now known as hasharAway | 08:59 | |
| *** ykarel|lunch is now known as ykarel | 09:04 | |
| *** alexchadin has quit IRC | 09:10 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-drupal master: Fix template variable https://review.openstack.org/584293 | 09:20 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-drupal master: Add beaker tests https://review.openstack.org/584294 | 09:20 |
| *** roman_g has joined #openstack-infra | 09:23 | |
| *** jbadiapa has quit IRC | 09:28 | |
| *** alexchadin has joined #openstack-infra | 09:28 | |
| *** links has quit IRC | 09:29 | |
| *** hamzy_ has joined #openstack-infra | 09:31 | |
| *** warlord77[m] has quit IRC | 09:42 | |
| *** austinsun[m] has quit IRC | 09:42 | |
| *** lbragstad[m] has quit IRC | 09:42 | |
| *** linshuicheng[m] has quit IRC | 09:42 | |
| *** aspiers[m] has quit IRC | 09:42 | |
| *** dhellmann[m] has quit IRC | 09:43 | |
| *** smcginnis[m] has quit IRC | 09:43 | |
| *** jbadiapa has joined #openstack-infra | 09:43 | |
| *** notmyname has quit IRC | 09:45 | |
| *** shoogz has quit IRC | 09:46 | |
| *** shoogz has joined #openstack-infra | 09:47 | |
| *** notmyname has joined #openstack-infra | 09:48 | |
| *** warlord77[m] has joined #openstack-infra | 09:48 | |
| *** vkmc has joined #openstack-infra | 09:49 | |
| *** sshnaidm|off has quit IRC | 09:50 | |
| *** isviridov_away has joined #openstack-infra | 09:52 | |
| *** vaidy has joined #openstack-infra | 09:52 | |
| *** kjackal has joined #openstack-infra | 09:53 | |
| *** alexchad_ has joined #openstack-infra | 09:54 | |
| *** alexchadin has quit IRC | 09:56 | |
| *** jbadiapa is now known as jbadiapa|away | 10:06 | |
| *** alexchadin has joined #openstack-infra | 10:19 | |
| *** alexchad_ has quit IRC | 10:22 | |
| *** quiquell is now known as quiquell|brb | 10:26 | |
| *** quiquell|brb is now known as quiquell | 10:29 | |
| *** dtantsur|afk is now known as dtantsur | 10:31 | |
| *** lbragstad[m] has joined #openstack-infra | 10:32 | |
| *** smcginnis[m] has joined #openstack-infra | 10:32 | |
| *** aspiers[m] has joined #openstack-infra | 10:32 | |
| *** dhellmann[m] has joined #openstack-infra | 10:32 | |
| *** austinsun[m] has joined #openstack-infra | 10:32 | |
| *** linshuicheng[m] has joined #openstack-infra | 10:32 | |
| *** aojea has quit IRC | 10:37 | |
| *** pguimaraes has quit IRC | 10:48 | |
| *** pguimaraes has joined #openstack-infra | 10:52 | |
| *** owalsh_afk is now known as owalsh | 10:54 | |
| *** shardy has quit IRC | 10:56 | |
| *** kjackal has quit IRC | 11:04 | |
| *** wolverineav has joined #openstack-infra | 11:16 | |
| *** udesale__ has quit IRC | 11:19 | |
| *** wolverineav has quit IRC | 11:24 | |
| *** lewo`` has quit IRC | 11:24 | |
| *** ijw has joined #openstack-infra | 11:24 | |
| *** wolverineav has joined #openstack-infra | 11:25 | |
| *** ijw has quit IRC | 11:29 | |
| *** links has joined #openstack-infra | 11:32 | |
| *** stephenfin is now known as finucannot | 11:34 | |
| *** tpsilva has joined #openstack-infra | 11:39 | |
| *** kjackal has joined #openstack-infra | 11:44 | |
| *** xinliang has quit IRC | 11:55 | |
| *** xinliang has joined #openstack-infra | 11:56 | |
| *** bhavik1 has joined #openstack-infra | 11:57 | |
| *** jbadiapa|away is now known as jbadiapa | 11:58 | |
| *** pawelzny has quit IRC | 11:58 | |
| *** lewo has joined #openstack-infra | 11:59 | |
| *** zhangfei has quit IRC | 12:00 | |
| *** trown|outtypewww is now known as trown | 12:04 | |
| *** zul has joined #openstack-infra | 12:04 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-drupal master: Add beaker tests https://review.openstack.org/584294 | 12:06 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-drupal master: Monkey-patch drush module for puppet 4 https://review.openstack.org/584341 | 12:06 |
| *** alexchadin has quit IRC | 12:07 | |
| *** bhavik1 has quit IRC | 12:08 | |
| *** eharney has joined #openstack-infra | 12:10 | |
| *** alexchadin has joined #openstack-infra | 12:12 | |
| *** niska has quit IRC | 12:19 | |
| *** agopi_ has quit IRC | 12:22 | |
| *** Seb-Solo` is now known as Seb-Solon | 12:23 | |
| *** dtantsur is now known as dtantsur|brb | 12:23 | |
| *** boden has joined #openstack-infra | 12:23 | |
| *** niska has joined #openstack-infra | 12:25 | |
| *** alexchadin has quit IRC | 12:26 | |
| *** alexchadin has joined #openstack-infra | 12:26 | |
| *** alexchadin has quit IRC | 12:27 | |
| *** alexchadin has joined #openstack-infra | 12:27 | |
| *** alexchadin has quit IRC | 12:28 | |
| *** alexchadin has joined #openstack-infra | 12:28 | |
| *** alexchadin has quit IRC | 12:28 | |
| *** camunoz has joined #openstack-infra | 12:29 | |
| *** rlandy has joined #openstack-infra | 12:31 | |
| *** sthussey has joined #openstack-infra | 12:32 | |
| *** graphene has quit IRC | 12:32 | |
| *** dpawlik has quit IRC | 12:32 | |
| *** graphene has joined #openstack-infra | 12:34 | |
| *** dpawlik has joined #openstack-infra | 12:35 | |
| *** dpawlik has quit IRC | 12:35 | |
| *** mriedem has joined #openstack-infra | 12:36 | |
| *** ccamacho has quit IRC | 12:39 | |
| *** dpawlik has joined #openstack-infra | 12:42 | |
| *** edmondsw has joined #openstack-infra | 12:42 | |
| *** quiquell is now known as quiquell|lunch | 12:44 | |
| *** d0ugal has quit IRC | 12:45 | |
| *** d0ugal has joined #openstack-infra | 12:45 | |
| *** d0ugal has quit IRC | 12:45 | |
| *** d0ugal has joined #openstack-infra | 12:45 | |
| *** sshnaidm|off has joined #openstack-infra | 12:48 | |
| *** graphene has quit IRC | 12:52 | |
| *** graphene has joined #openstack-infra | 12:53 | |
| *** graphene has quit IRC | 12:54 | |
| *** rpioso|afk is now known as rpioso | 12:55 | |
| mordred | ianw: "(hardware+networking+people) = a cloud" ++ | 12:55 |
| *** graphene has joined #openstack-infra | 12:56 | |
| *** d0ugal has quit IRC | 12:57 | |
| *** armaan has joined #openstack-infra | 13:00 | |
| *** ijw has joined #openstack-infra | 13:01 | |
| *** jamesmcarthur has joined #openstack-infra | 13:03 | |
| *** quiquell|lunch is now known as quiquell | 13:05 | |
| *** jcoufal has joined #openstack-infra | 13:05 | |
| *** ijw has quit IRC | 13:06 | |
| *** d0ugal has joined #openstack-infra | 13:09 | |
| *** ccamacho has joined #openstack-infra | 13:10 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add pbrx-release team for pbrx https://review.openstack.org/584366 | 13:13 |
| *** jamesmcarthur has quit IRC | 13:22 | |
| *** agopi has joined #openstack-infra | 13:25 | |
| *** graphene has quit IRC | 13:27 | |
| *** armaan has quit IRC | 13:28 | |
| *** graphene has joined #openstack-infra | 13:29 | |
| *** yamamoto has joined #openstack-infra | 13:34 | |
| *** pblaho has quit IRC | 13:37 | |
| *** zhangfei has joined #openstack-infra | 13:39 | |
| *** TheJulia is now known as needssleep | 13:40 | |
| *** EmilienM is now known as EvilienM | 13:41 | |
| *** jcoufal has quit IRC | 13:44 | |
| *** alexchadin has joined #openstack-infra | 13:44 | |
| *** jcoufal has joined #openstack-infra | 13:45 | |
| *** graphene has quit IRC | 13:47 | |
| *** graphene has joined #openstack-infra | 13:48 | |
| *** ykarel is now known as ykarel|away | 13:49 | |
| *** ihar has joined #openstack-infra | 13:49 | |
| *** jchhatbar has quit IRC | 13:49 | |
| *** hongbin has joined #openstack-infra | 13:55 | |
| *** alexchadin has quit IRC | 13:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Build container images using pbrx https://review.openstack.org/582732 | 13:58 |
| *** ijw has joined #openstack-infra | 13:59 | |
| *** eernst has joined #openstack-infra | 14:01 | |
| *** quiquell is now known as quiquell|off | 14:03 | |
| *** dansmith is now known as superdan | 14:04 | |
| *** ijw has quit IRC | 14:04 | |
| *** peereb has quit IRC | 14:05 | |
| *** jcoufal has quit IRC | 14:14 | |
| *** pcichy has quit IRC | 14:14 | |
| openstackgerrit | Tobias Rydberg proposed openstack-infra/irc-meetings master: Added new bi-weeky meeting on even weeks. https://review.openstack.org/584389 | 14:14 |
| *** jcoufal has joined #openstack-infra | 14:14 | |
| *** pcichy has joined #openstack-infra | 14:15 | |
| *** dtantsur|brb is now known as dtantsur | 14:15 | |
| *** ykarel|away has quit IRC | 14:17 | |
| *** armax has joined #openstack-infra | 14:17 | |
| *** florianf has joined #openstack-infra | 14:22 | |
| *** ijw has joined #openstack-infra | 14:24 | |
| *** jtomasek has quit IRC | 14:28 | |
| *** ijw has quit IRC | 14:29 | |
| *** efried is now known as fried_rice | 14:32 | |
| *** beagles is now known as beagles_afk | 14:33 | |
| *** felipemonteiro_ has quit IRC | 14:33 | |
| *** neiloy has joined #openstack-infra | 14:37 | |
| *** Bhujay has quit IRC | 14:42 | |
| *** eernst has quit IRC | 14:44 | |
| *** pguimaraes has quit IRC | 14:45 | |
| *** lpetrut has quit IRC | 14:46 | |
| *** armaan has joined #openstack-infra | 14:47 | |
| *** felipemonteiro has joined #openstack-infra | 14:51 | |
| *** links has quit IRC | 14:52 | |
| *** eernst has joined #openstack-infra | 14:54 | |
| boden | hi, left field question… we have an openstack based PYPI project for which the only human maintainer for the PYPI project is no longer on our team or reachable… however the ‘openstackci’ user on PYPI does have access.. is it possible someone can acutally use that openstackci PYPI user to add some active maintainers to the project? | 14:56 |
| *** armaan has quit IRC | 14:57 | |
| *** dpawlik has quit IRC | 14:57 | |
| *** rpittau_ has quit IRC | 14:59 | |
| *** e0ne has quit IRC | 14:59 | |
| *** r-daneel has joined #openstack-infra | 14:59 | |
| *** ijw has joined #openstack-infra | 15:00 | |
| *** zhangfei has quit IRC | 15:01 | |
| *** lbragstad has quit IRC | 15:03 | |
| *** pcichy has quit IRC | 15:03 | |
| *** r-daneel_ has joined #openstack-infra | 15:04 | |
| *** r-daneel has quit IRC | 15:04 | |
| *** r-daneel_ is now known as r-daneel | 15:04 | |
| pabelanger | which project? I could see a case we're we as infra might not want to be responsible for handing out access to a project no longer in our care | 15:04 |
| *** ijw has quit IRC | 15:05 | |
| *** e0ne has joined #openstack-infra | 15:05 | |
| *** e0ne has quit IRC | 15:06 | |
| *** e0ne has joined #openstack-infra | 15:06 | |
| mordred | boden: yeah - it is potentially feasible - although if it's one that openstackci is pushing to / managing, there really shouldn't be a need for a maintainer otherwise ... unless the intent is to move it out of being managed, in which case yeah, what pabelanger said | 15:08 |
| *** bnemec is now known as beekneemech | 15:09 | |
| boden | pabelanger mordred the project is https://pypi.org/project/vmware-nsx-tempest/ we are still managing the actual git repo and in fact want to start pushing releases to PYPI… Today we don’t have a real need to access the PYPI account, my concern is that in the future we may need to (to delete a release or something) | 15:10 |
| boden | perhaps I’m being overly proactive here | 15:11 |
| mordred | yeah - I say let's just cross that bridge when we come to it | 15:11 |
| *** rcarrill1 has joined #openstack-infra | 15:12 | |
| mordred | also - never delete a release - just make a new one ... deleting things has a greater ability to break humans in the wild | 15:12 |
| *** rcarrillocruz has quit IRC | 15:12 | |
| *** lbragstad has joined #openstack-infra | 15:12 | |
| boden | mordred it was suggested otherwise to me.. for example https://pypi.org/project/vmware-nsx/#history you can see 2 old versions show up as recent; because the versioning is not consistent | 15:13 |
| boden | I was told to delete those 2 old ones if we are not supporting/using | 15:13 |
| mordred | oh - the old 2015 ones ... | 15:14 |
| mordred | dhellmann: ^^ how did we handle that for the other openstack projects? did we delete? | 15:14 |
| dhellmann | mordred : yeah, we removed a few things that were extremely old and not pip installable | 15:15 |
| dhellmann | setuptools doesn't know how to do epochs | 15:15 |
| mordred | kk | 15:17 |
| mordred | boden: I can take care of those two - both of the 2015 versions? | 15:19 |
| *** dave-mccowan has joined #openstack-infra | 15:20 | |
| *** florianf has quit IRC | 15:22 | |
| *** dave-mcc_ has joined #openstack-infra | 15:23 | |
| *** wolverineav has quit IRC | 15:23 | |
| EvilienM | pabelanger: can you add me core in openstack/ansible-role-openstack-operations please, we created this role in tripleo | 15:24 |
| EvilienM | I need to setup gerrit perms, for tripleo core group | 15:24 |
| *** dave-mccowan has quit IRC | 15:25 | |
| EvilienM | or any infra-root ^ | 15:27 |
| *** wolverineav has joined #openstack-infra | 15:28 | |
| corvus | EvilienM: on it | 15:29 |
| EvilienM | thx | 15:30 |
| corvus | EvilienM: done | 15:31 |
| corvus | also, i like the commitment to the new identity. | 15:32 |
| EvilienM | corvus: thanks a lot | 15:37 |
| EvilienM | corvus: I'm trying | 15:37 |
| *** mschuppert has quit IRC | 15:38 | |
| boden | mordred re the 2015 old ones… yes that’s them, but I’m still working to confirm we are not using/supporting them; so I think we need to wait on the delete | 15:39 |
| mordred | boden: kk | 15:40 |
| *** ykarel has joined #openstack-infra | 15:41 | |
| *** mriedem is now known as mriedem_lunch | 15:43 | |
| *** ykarel is now known as ykarel|away | 15:46 | |
| *** electrofelix has quit IRC | 15:47 | |
| *** trown is now known as trown|lunch | 15:51 | |
| *** agopi is now known as agopi|food | 15:52 | |
| *** r-daneel has quit IRC | 15:53 | |
| *** sshnaidm|off has quit IRC | 15:53 | |
| *** r-daneel has joined #openstack-infra | 15:53 | |
| *** udesale__ has joined #openstack-infra | 16:00 | |
| *** dbecker has quit IRC | 16:04 | |
| *** udesale__ has quit IRC | 16:05 | |
| *** ykarel|away has quit IRC | 16:05 | |
| clarkb | mordred: follow up on packethost resource leaks. We appear to have leaked two floating IPs. Since I cleaned everything up (12 hours maybe longer) | 16:05 |
| mordred | clarkb: oh joy | 16:05 |
| mordred | clarkb: ok - so - next step tracking that down, I think, is restarting the launchers which should get us slightly better logging in those errors | 16:05 |
| mordred | clarkb: like, we should at least get the error message from the cloud | 16:06 |
| clarkb | mordred: this is the changes to consuming sdk? | 16:06 |
| mordred | clarkb: you're probably right about that section of code you were mentioning the other day - but if it's still happening it might be useful to get the error messages | 16:06 |
| clarkb | ++ | 16:06 |
| clarkb | I'm trying to dig up logs on one of the floating IPs | 16:07 |
| *** akhilaki has joined #openstack-infra | 16:07 | |
| clarkb | we are reusing IPs so tracking to the point where it actually failed is fun | 16:08 |
| openstackgerrit | Merged openstack-infra/infra-specs master: Update config management for the Infra Control Plane https://review.openstack.org/565550 | 16:08 |
| mordred | oh dear god that actually landed | 16:08 |
| mordred | I actually have to do stuff now don't I? | 16:08 |
| clarkb | mordred: if you want to get a head start you could review cmurphy's topic:puppet-4 changes :P | 16:09 |
| clarkb | mordred: ya reading the logs for this FIP we failed in the way I described it hours before the FIP with that ip was used and the point where it was last used and leaked seem pretty normal | 16:09 |
| mordred | clarkb: https://review.openstack.org/#/c/582732/ <-- fix patch tagged with 'topic:update-cfg-mgmt' :) | 16:10 |
| mordred | clarkb: oh - wait - we're re-using FIPs not deleting and recreating? | 16:10 |
| clarkb | mordred: no, reuse in that the same ip is reused but should be a new fip resource | 16:10 |
| mordred | ok. PHEW | 16:10 |
| clarkb | so if I grep the ip address it shows up all over | 16:10 |
| clarkb | but tracking it down against update times in neutron and nodepool logs the last useage of it actually looks clean | 16:11 |
| clarkb | no errors deleting the server because fip failed to delete in this case, happened once or twice on earlier cases of using that ip address but htat was hours before | 16:11 |
| clarkb | also, we don't appear to have leaked any ports, I think those two leaks may be independent | 16:11 |
| clarkb | we may actually need Johns help to look at why the FIP wasn't deleted. My earlier assumption that it was something we were doing seems to no longer be valid | 16:14 |
| *** e0ne has quit IRC | 16:14 | |
| clarkb | that said, I think we should have floating ip cleanup in nodepool so going to look at why that isn't running | 16:14 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Clean packethost floating IPs https://review.openstack.org/584440 | 16:18 |
| clarkb | mordred: ^ that should get nodepool to clean this up itself | 16:18 |
| clarkb | I'll leave the two that leaked in place as watching that they go away will be good confirmation that ^ is working | 16:18 |
| mordred | ++ | 16:21 |
| *** tesseract has quit IRC | 16:24 | |
| *** fried_rice is now known as fried_rolls | 16:24 | |
| clarkb | I wish that was safe enough to be a default | 16:25 |
| clarkb | it is for infra the way we ask for accounts and tenants to be set up but probably not safe for pabelanger | 16:25 |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Added new bi-weeky meeting on even weeks. https://review.openstack.org/584389 | 16:25 |
| *** yamahata has quit IRC | 16:27 | |
| *** pguimaraes has joined #openstack-infra | 16:28 | |
| *** hasharAway has quit IRC | 16:32 | |
| pabelanger | clarkb: I'd be okay with it being default, don't block on unsafe for bad practices :) | 16:35 |
| clarkb | pabelanger: its not your fault, its mostly a problem with how the api works | 16:36 |
| clarkb | if you create FIPs then attach them then valid FIPs can be unattached state for some time | 16:36 |
| clarkb | nodepool does create on attach so its mostly atomic | 16:36 |
| clarkb | (but if you have users other than nodepool you cannot rely on that) | 16:37 |
| clarkb | pabelanger: if you can review the change that would be helpful for getting packethost to a happy place | 16:37 |
| pabelanger | sure, +3 | 16:38 |
| *** sshnaidm|off has joined #openstack-infra | 16:38 | |
| *** kjackal has quit IRC | 16:41 | |
| *** felipemonteiro has quit IRC | 16:45 | |
| *** felipemonteiro has joined #openstack-infra | 16:46 | |
| *** felipemonteiro has quit IRC | 16:46 | |
| *** yamamoto has quit IRC | 16:51 | |
| *** myoung is now known as myoung|lunch | 16:52 | |
| *** myoung|lunch is now known as myoung | 16:53 | |
| *** myoung is now known as myoung|lunch | 16:58 | |
| *** yamahata has joined #openstack-infra | 17:00 | |
| *** esarault has joined #openstack-infra | 17:02 | |
| openstackgerrit | Merged openstack-dev/pbr master: Switch to stestr https://review.openstack.org/580413 | 17:02 |
| openstackgerrit | Merged openstack-dev/pbr master: doc: Fix formatting of packagers guide https://review.openstack.org/580418 | 17:02 |
| openstackgerrit | Merged openstack-dev/pbr master: doc: Add documentation for missing '[pbr]' options https://review.openstack.org/580419 | 17:02 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Build container images using pbrx https://review.openstack.org/582732 | 17:03 |
| *** ramishra has quit IRC | 17:07 | |
| *** graphene has quit IRC | 17:10 | |
| *** dtantsur is now known as dtantsur|afk | 17:11 | |
| *** graphene has joined #openstack-infra | 17:12 | |
| *** trown|lunch is now known as trown | 17:12 | |
| *** wendar has quit IRC | 17:13 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix indentation of executor monitoring docs https://review.openstack.org/584448 | 17:14 |
| *** dklyle has joined #openstack-infra | 17:15 | |
| clarkb | we ran 24k jobs in the last 24 hours | 17:15 |
| *** jamesmcarthur has joined #openstack-infra | 17:15 | |
| *** wendar has joined #openstack-infra | 17:15 | |
| *** david-lyle has joined #openstack-infra | 17:16 | |
| mordred | clarkb: nice | 17:19 |
| *** dklyle has quit IRC | 17:20 | |
| *** david-lyle has quit IRC | 17:21 | |
| corvus | i'm polishing up my work from the past 2 days; hope to upload something today | 17:22 |
| *** david-lyle has joined #openstack-infra | 17:22 | |
| *** myoung|lunch is now known as myoung | 17:25 | |
| *** anteaya has joined #openstack-infra | 17:26 | |
| *** neiloy has quit IRC | 17:26 | |
| *** slaweq has quit IRC | 17:27 | |
| *** panda is now known as panda|off | 17:27 | |
| *** jtomasek has joined #openstack-infra | 17:28 | |
| mordred | cmurphy: thanks for all the fun reviewing opportunities you have provided me | 17:29 |
| *** david-lyle has quit IRC | 17:33 | |
| *** dklyle has joined #openstack-infra | 17:33 | |
| *** jamesmcarthur has quit IRC | 17:33 | |
| cmurphy | mordred: you are very welcome | 17:33 |
| *** graphene has quit IRC | 17:35 | |
| *** Randall_ has joined #openstack-infra | 17:35 | |
| *** graphene has joined #openstack-infra | 17:36 | |
| clarkb | mordred: ya I was double checking that we werent just massively behind because something was broken (especially after fixing packethost) and seems like there is simply a lot if work to do | 17:37 |
| *** r-daneel has quit IRC | 17:39 | |
| *** myoung is now known as myoung|bbl | 17:39 | |
| *** Randall_ has quit IRC | 17:40 | |
| *** lpetrut has joined #openstack-infra | 17:42 | |
| *** pradk has joined #openstack-infra | 17:43 | |
| *** akhilaki has quit IRC | 17:43 | |
| *** agopi|food is now known as agopi | 17:45 | |
| *** lpetrut has quit IRC | 17:47 | |
| *** jtomasek has quit IRC | 17:47 | |
| *** jtomasek has joined #openstack-infra | 17:48 | |
| *** mriedem_lunch is now known as mriedem | 17:50 | |
| *** jtomasek has quit IRC | 17:54 | |
| openstackgerrit | Merged openstack-infra/project-config master: Clean packethost floating IPs https://review.openstack.org/584440 | 17:55 |
| *** pguimaraes has quit IRC | 17:56 | |
| *** dhill_ has joined #openstack-infra | 18:02 | |
| corvus | i've written 2 unit tests so far for the directory walker :) | 18:03 |
| openstackgerrit | Merged openstack-infra/puppet-meetbot master: Use httpd::vhost $content parameter https://review.openstack.org/564904 | 18:06 |
| openstackgerrit | Merged openstack-infra/puppet-mailman master: Add beaker tests https://review.openstack.org/567465 | 18:06 |
| *** kzaitsev1pi has joined #openstack-infra | 18:14 | |
| openstackgerrit | Merged openstack-infra/puppet-mailman master: Use httpd::vhost $content parameter https://review.openstack.org/567440 | 18:15 |
| openstackgerrit | Merged openstack-infra/puppet-cgit master: Add beaker check of vhost template https://review.openstack.org/570991 | 18:15 |
| openstackgerrit | Merged openstack-infra/puppet-cgit master: Use httpd::vhost $content parameter https://review.openstack.org/570825 | 18:15 |
| openstackgerrit | Merged openstack-infra/puppet-nodepool master: Fix name of libxslt dev package https://review.openstack.org/577519 | 18:15 |
| openstackgerrit | Merged openstack-infra/puppet-openstackci master: Add beaker-rspec bindep dependencies https://review.openstack.org/578559 | 18:15 |
| *** kzaitsev_pi has quit IRC | 18:17 | |
| clarkb | packethost FIPs seem to be cleaned up now | 18:17 |
| *** e0ne has joined #openstack-infra | 18:18 | |
| corvus | clarkb: does that mean the cleaup thread is working? | 18:23 |
| corvus | clarkb: or that you manually cleaned them up? | 18:24 |
| clarkb | corvus: I didn't manually clean them up. The config change to have nodepool do it merged and appears to be working | 18:24 |
| corvus | ack | 18:26 |
| clarkb | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=64792&rra_id=all zuul continues to look happy. There is slow growth but slower than we saw with the leaks (but time frame is still much shorter) | 18:27 |
| *** dklyle has quit IRC | 18:29 | |
| corvus | yeah, that doesn't look unlike it did before the last restart. we'd need more time to confirm the new leak is fixed, but so far we've at least failed to confirm it's still there. :) | 18:29 |
| *** shaner has quit IRC | 18:34 | |
| clarkb | mordred: I just learned a new thing managing my local server. Upgrading docker reuires a daemon restart and it wants to restart all containers too | 18:34 |
| clarkb | we may need to accomodate that with our automatic updates | 18:34 |
| clarkb | (the update text said that if you don't do it docker can fail to properly manage the contianers going forward) | 18:35 |
| *** dklyle has joined #openstack-infra | 18:36 | |
| *** r-daneel has joined #openstack-infra | 18:37 | |
| *** fried_rolls is now known as fried_rice | 18:38 | |
| *** dklyle has quit IRC | 18:38 | |
| *** dklyle has joined #openstack-infra | 18:38 | |
| *** shaner has joined #openstack-infra | 18:39 | |
| *** eernst has quit IRC | 18:40 | |
| mordred | clarkb: ++ | 18:41 |
| *** pradk has quit IRC | 18:43 | |
| *** dklyle has quit IRC | 18:43 | |
| *** ijw has joined #openstack-infra | 18:44 | |
| *** esarault_ has joined #openstack-infra | 18:44 | |
| *** ijw has quit IRC | 18:44 | |
| corvus | what are our options? never update docker? | 18:45 |
| clarkb | corvus: or do so with coordinated short downtime? | 18:46 |
| clarkb | in theory its quick but likely noticeable | 18:46 |
| corvus | clarkb: restarting gerrit and zuul are never quick | 18:47 |
| *** esarault has quit IRC | 18:47 | |
| fungi | boden: mordred: dhellmann: yeah, for cleaning up year-based releases of other projects we had a brief thread on the -dev ml to give people a heads up, and then i deleted their files followed by the releases themselves after a few days/week with no objections | 18:47 |
| boden | fungi ack thx | 18:47 |
| corvus | clarkb: but anyway, that still means excluding docker in unattended-upgrades, right? | 18:48 |
| *** gfidente has quit IRC | 18:52 | |
| pabelanger | how are openstack projects doing things with docker and lifecycle management? | 18:54 |
| pabelanger | never realized restarting docker also took down containers | 18:54 |
| *** dhill_ has quit IRC | 18:57 | |
| *** esarault_ has quit IRC | 18:57 | |
| mordred | corvus: yah - I think we should excludeit from unattended-upgrades - and then we'll need to figure out a strategy | 18:57 |
| corvus | eventually it'll get better, for zuul at least, but not for a while | 18:58 |
| corvus | we could probably let it restart gerrit, it will just be random and slow | 18:59 |
| mordred | yah | 18:59 |
| *** neiloy has joined #openstack-infra | 18:59 | |
| mordred | also - for some of these things we should likely go straight to runc | 18:59 |
| mordred | like gerrit | 18:59 |
| mordred | with runc, we'd be running the container like a normal process - so there is no daemon to restart | 19:00 |
| mordred | same with zuul | 19:00 |
| *** jcoufal has quit IRC | 19:00 | |
| corvus | oh ok | 19:00 |
| mordred | so - maybe it's a good reason to consider that approach sooner - since using docker from the start was supposedely about making things easier | 19:00 |
| corvus | yeah, worth considering | 19:01 |
| mordred | the downside to that is that we'd still be using systemd ;) | 19:01 |
| corvus | tbh, this is the one leg that systemd has to stand on. | 19:01 |
| mordred | yah | 19:01 |
| corvus | (i've liked this idea since i heard lennart talk about it mumblemumble years ago) | 19:02 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Allow launch-retries to indefinitely retry for openstack https://review.openstack.org/584488 | 19:02 |
| mordred | clarkb: how's https://review.openstack.org/#/c/582732/ look to you now? | 19:03 |
| *** yamamoto has joined #openstack-infra | 19:03 | |
| *** rcarrillocruz has joined #openstack-infra | 19:04 | |
| clarkb | pabelanger I think it might be more the upgrade needs a restart and upgraded daemon cant necessarily talk to older managed containers | 19:06 |
| clarkb | but dist-upgrade made me manually confirm I was ok with this restart and had a big warning | 19:06 |
| pabelanger | yah, I _think_ if it was just update, docker should not have changed | 19:07 |
| *** rcarrill1 has quit IRC | 19:07 | |
| *** yamamoto has quit IRC | 19:07 | |
| *** felipemonteiro has joined #openstack-infra | 19:07 | |
| *** rcarrill1 has joined #openstack-infra | 19:08 | |
| *** e0ne has quit IRC | 19:08 | |
| *** rcarrillocruz has quit IRC | 19:10 | |
| *** rcarrillocruz has joined #openstack-infra | 19:11 | |
| *** ijw has joined #openstack-infra | 19:12 | |
| *** graphene has quit IRC | 19:13 | |
| *** rcarrill1 has quit IRC | 19:13 | |
| clarkb | I dont think update vs upgrade makes a difference to docker | 19:14 |
| clarkb | I mean you can restart docker if not upgrading and I think containers are fine | 19:14 |
| clarkb | its just when docker updates that it wants that? | 19:14 |
| *** graphene has joined #openstack-infra | 19:14 | |
| mordred | clarkb: was yours coming from suse or from upstream docker? (just curious - mechanics are the same) | 19:14 |
| mordred | I've been thinking (and I think it's in the spec?) that we should track the docker upstream stable repo instead of whichever thing happens to come in the base os | 19:15 |
| clarkb | mordred: xenial | 19:17 |
| *** pcichy has joined #openstack-infra | 19:18 | |
| mordred | clarkb: nod. so docker.io instead of docker-ce | 19:18 |
| *** edmondsw has quit IRC | 19:19 | |
| openstackgerrit | Jill Rouleau proposed openstack-infra/project-config master: Add Tripleo Ansible repo https://review.openstack.org/583415 | 19:19 |
| *** edmondsw has joined #openstack-infra | 19:20 | |
| *** jamesmcarthur has joined #openstack-infra | 19:24 | |
| *** jamesmcarthur has quit IRC | 19:25 | |
| *** jamesmcarthur has joined #openstack-infra | 19:25 | |
| *** felipemonteiro has quit IRC | 19:25 | |
| *** e0ne has joined #openstack-infra | 19:26 | |
| *** edmondsw has quit IRC | 19:26 | |
| openstackgerrit | Merged openstack-infra/puppet-openafs master: Add beaker tests https://review.openstack.org/579394 | 19:30 |
| mriedem | clarkb: do you know if we already have something tracking this type of issue? http://logs.openstack.org/20/582920/1/gate/tempest-full/101c850/ara-report/reports/85a10e09-1e8b-49cd-ac8f-b3aa23193168.html | 19:31 |
| mriedem | i'm going to write an e-r fingerprint | 19:32 |
| mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22SSH%20Error%3A%20data%20could%20not%20be%20sent%20to%20remote%20host%5C%22%20AND%20message%3A%5C%22Make%20sure%20this%20host%20can%20be%20reached%20over%20ssh%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | 19:32 |
| mriedem | https://bugs.launchpad.net/tripleo/+bug/1781871 ? | 19:33 |
| openstack | Launchpad bug 1781871 in tripleo "RDO cloud jobs failing with SSH Error: data could not be sent to remote host \"38.145.32.100\". Make sure this host can be reached over ssh" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 19:33 |
| clarkb | mriedem: I'll look after lunvh | 19:35 |
| *** eernst has joined #openstack-infra | 19:36 | |
| mriedem | looks like it was a major issue with packethost-us-west-1 | 19:36 |
| openstackgerrit | boden proposed openstack-infra/project-config master: add publish-to-pypi to vmware-nsx-tempest-plugin https://review.openstack.org/584498 | 19:36 |
| pabelanger | mriedem: clarkb: sounds like executor from rax might have lost SSH connect to those nodes | 19:39 |
| *** graphene has quit IRC | 19:39 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for packethost-us-west-1 node setup bug 1781871 https://review.openstack.org/584499 | 19:39 |
| *** r-daneel has quit IRC | 19:39 | |
| openstack | bug 1781871 in tripleo "RDO cloud jobs failing with SSH Error: data could not be sent to remote host \"38.145.32.100\". Make sure this host can be reached over ssh" [Critical,Triaged] https://launchpad.net/bugs/1781871 - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 19:40 |
| mriedem | pabelanger: is that node provider rax? | 19:40 |
| *** r-daneel has joined #openstack-infra | 19:40 | |
| mriedem | i don't know what packethost is | 19:40 |
| mriedem | i guess it's not rax https://www.packet.net/ | 19:40 |
| pabelanger | mriedem: no, all our zuul-executors are in rackspace, so thinking the SSH connection from them into packet.net might be having issue. We are also using FIPs for packet.net too | 19:41 |
| *** graphene has joined #openstack-infra | 19:41 | |
| *** felipemonteiro has joined #openstack-infra | 19:43 | |
| *** felipemonteiro has quit IRC | 19:44 | |
| *** dklyle has joined #openstack-infra | 19:44 | |
| *** felipemonteiro has joined #openstack-infra | 19:45 | |
| clarkb | it will be interesting to see if it is a single cloud affevted or not | 19:46 |
| clarkb | in the past its been all clouds I think | 19:46 |
| clarkb | looks like mostly packethost (but a few in rax and one in ovh) | 19:52 |
| clarkb | over the last day | 19:52 |
| scas | might it be due to utilization? they tweeted out 100% utilization yesterday | 19:52 |
| clarkb | scas: aiui we are running on baremetal nodes allocated out of their baremetal cloud and those run openstack | 19:53 |
| scas | just an armchair observation | 19:53 |
| clarkb | I expect we are mostly segregated as a result | 19:53 |
| pabelanger | clarkb: mordred: fungi: corvus: following up from #zuul, tobiash has pointed out we might want to launch another zuul-executor or 2 to account for extra 100 nodes from packet now. This is based on the executor queue at http://grafana.openstack.org/d/T6vSHcSik/zuul-status | 19:53 |
| clarkb | what is odd is that nodepool does the ssh keyscan | 19:53 |
| clarkb | so there was a route up to that point | 19:53 |
| pabelanger | I have some spare time this afternoon to work on it, if wanted | 19:53 |
| scas | clarkb: understood https://twitter.com/john_studarus/status/1020026116038258688 | 19:54 |
| clarkb | ya thats our graph I think | 19:54 |
| corvus | pabelanger: i'm in favor of another executor | 19:54 |
| clarkb | one of the IPs (chosen arbitrarily) works for ssh now so I don't think it is a systemic route problem | 19:56 |
| clarkb | possibly too small mtu somewhere? | 19:56 |
| clarkb | if VM thinks it can use mtu of 1500 but that is running vxlan under it and host mtu is also 1500 I could see that being sad, but work well enoughto key scan | 19:57 |
| *** graphene has quit IRC | 19:57 | |
| *** graphene has joined #openstack-infra | 19:58 | |
| corvus | clarkb: that would only affect multinode jobs? | 19:58 |
| fungi | ssh keyscan may not need a full-size tcp packet? | 19:59 |
| fungi | if the request/response are short enough to fit in a single packet | 20:00 |
| clarkb | corvus: no, in this case if the cloud itslef had a bad link somewhere we could see that too | 20:00 |
| pabelanger | okay, I'll bring up ze11 now | 20:00 |
| *** myoung|bbl is now known as myoung | 20:01 | |
| fungi | so the indication is that something is impacting node-to-node communication but maybe not control-plane-to-node? | 20:02 |
| clarkb | fungi: well all we know is ssh from rax to packethost sometimes fails after nodepool did so successfully | 20:02 |
| clarkb | fungi: that implies to me that the instance booted and its IP was routable long enough to get the ssh host key scanned | 20:02 |
| clarkb | but then when zuul runs the job it times out | 20:03 |
| clarkb | later that same IP routes and ssh works fine | 20:03 |
| clarkb | the upside is due to happening in pre run it should cause the job to be rerun | 20:04 |
| fungi | and these go through fips right? | 20:05 |
| fungi | i wonder if their fips could be getting reassigned or unmapped or something | 20:06 |
| clarkb | yes all are FIPs | 20:06 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for packethost-us-west-1 node setup bug 1781871 https://review.openstack.org/584499 | 20:09 |
| openstack | bug 1781871 in tripleo "RDO cloud jobs failing with SSH Error: data could not be sent to remote host \"38.145.32.100\". Make sure this host can be reached over ssh" [Critical,Triaged] https://launchpad.net/bugs/1781871 - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 20:09 |
| clarkb | fungi: also happening about 3 times an hour according to logstash | 20:09 |
| fungi | no correlation across ip addresses i suppose (different addresses each time)? | 20:09 |
| clarkb | not different each time but across a few IPs | 20:10 |
| clarkb | we have to recycle IPs there in general so not super surprising if there are dups | 20:10 |
| fungi | if it were the same addresses over and over i'd suspect arp overwrite fights with something statically addressed on the segment, but i guess not | 20:10 |
| *** felipemonteiro has quit IRC | 20:10 | |
| *** slaweq has joined #openstack-infra | 20:11 | |
| fungi | but yeah, maybe some gear is getting confused as to what addresses are in which parts of that lan or holding onto stale information when an address moves from one vm to the next | 20:11 |
| fungi | though that's usually more of a problem with recycling hardware addresses than ip addresses | 20:12 |
| clarkb | and in this case we should've ssh keyscanned which should fail if the old user of the IP has been deleted and new user is booted | 20:12 |
| clarkb | (but that could be racy and potentially still explaing it, we scan the old host then tell zuul to run job on new host and it fails) | 20:12 |
| *** kjackal has joined #openstack-infra | 20:13 | |
| *** slaweq has quit IRC | 20:15 | |
| *** camunoz has quit IRC | 20:19 | |
| *** edmondsw has joined #openstack-infra | 20:20 | |
| *** eernst has quit IRC | 20:25 | |
| *** rh-jelabarre has quit IRC | 20:29 | |
| *** akhilaki has joined #openstack-infra | 20:29 | |
| *** rh-jelabarre has joined #openstack-infra | 20:34 | |
| *** slaweq has joined #openstack-infra | 20:35 | |
| *** dklyle has quit IRC | 20:35 | |
| *** owalsh has quit IRC | 20:38 | |
| *** r-daneel has quit IRC | 20:40 | |
| *** akhilaki has quit IRC | 20:42 | |
| *** owalsh has joined #openstack-infra | 20:44 | |
| *** jamesmcarthur has quit IRC | 20:51 | |
| pabelanger | seems HWE kernel on zuul-executor is pushing puppet apply timeout over the edge and failing to bring ze11 online. I'll dig more into it this evening, but now I go out to eat | 20:52 |
| clarkb | pabelanger: it is compiling afs I think | 20:54 |
| pabelanger | yah, something is compiling | 20:55 |
| pabelanger | oh | 20:56 |
| pabelanger | I see | 20:56 |
| pabelanger | | Jul 20 20:19:47 ze11 puppet-user[14395]: (/Stage[main]/Zuul/Exec[install_zuul]/returns) Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall. | 20:56 |
| pabelanger | | Jul 20 20:19:47 ze11 puppet-user[14395]: (/Stage[main]/Zuul/Exec[install_zuul]) Failed to call refresh: pip3 install -U -r/opt/zuul/requirements.txt && pip3 install --no-deps -U /opt/zuul returned 1 instead of one of [0] | 20:57 |
| pabelanger | okay, have to run, will debug more | 20:57 |
| clarkb | pip10 problems | 21:03 |
| clarkb | I think cmurphy has a fix for that one (stop installing it from distro as its a zuul dep) | 21:03 |
| *** trown is now known as trown|outtypewww | 21:04 | |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Skip releases jobs on doc updates https://review.openstack.org/584515 | 21:05 |
| *** agopi is now known as agopi|brb | 21:08 | |
| *** graphene has quit IRC | 21:11 | |
| *** agopi|brb has quit IRC | 21:12 | |
| *** graphene has joined #openstack-infra | 21:12 | |
| *** tdasilva has quit IRC | 21:16 | |
| *** linshuicheng[m] has quit IRC | 21:16 | |
| *** austinsun[m] has quit IRC | 21:16 | |
| *** warlord77[m] has quit IRC | 21:16 | |
| *** dhellmann[m] has quit IRC | 21:16 | |
| *** lbragstad[m] has quit IRC | 21:16 | |
| *** smcginnis[m] has quit IRC | 21:16 | |
| *** aspiers[m] has quit IRC | 21:17 | |
| *** bandini has quit IRC | 21:19 | |
| *** r-daneel has joined #openstack-infra | 21:20 | |
| *** bandini has joined #openstack-infra | 21:21 | |
| *** jamesmcarthur has joined #openstack-infra | 21:23 | |
| *** e0ne has quit IRC | 21:27 | |
| *** e0ne has joined #openstack-infra | 21:27 | |
| *** e0ne has quit IRC | 21:30 | |
| *** tdasilva has joined #openstack-infra | 21:31 | |
| *** hashar has joined #openstack-infra | 21:31 | |
| mordred | clarkb: ++ | 21:33 |
| *** slaweq has quit IRC | 21:33 | |
| *** felipemonteiro has joined #openstack-infra | 21:38 | |
| *** felipemonteiro has quit IRC | 21:41 | |
| *** felipemonteiro has joined #openstack-infra | 21:41 | |
| *** boden has quit IRC | 21:48 | |
| *** felipemonteiro has quit IRC | 21:59 | |
| *** yamamoto has joined #openstack-infra | 22:03 | |
| *** warlord77[m] has joined #openstack-infra | 22:04 | |
| *** tpsilva has quit IRC | 22:06 | |
| *** yamamoto has quit IRC | 22:07 | |
| *** fried_rice is now known as efried | 22:09 | |
| *** roman_g has quit IRC | 22:10 | |
| clarkb | mordred: pabelanger https://review.openstack.org/#/c/582518/ we might need a change for zuul too | 22:15 |
| clarkb | I only seechanges for dib and subunit2sql | 22:15 |
| *** weshay is now known as weshay_PTO | 22:15 | |
| corvus | clarkb: agreed | 22:19 |
| openstackgerrit | Fatema Khalid Sherif proposed openstack-infra/storyboard master: Validating task status in the API https://review.openstack.org/556648 | 22:25 |
| *** hashar has quit IRC | 22:29 | |
| *** myoung has quit IRC | 22:29 | |
| *** eernst has joined #openstack-infra | 22:40 | |
| *** imacdonn has quit IRC | 22:45 | |
| *** imacdonn has joined #openstack-infra | 22:45 | |
| *** aspiers[m] has joined #openstack-infra | 22:46 | |
| *** lbragstad[m] has joined #openstack-infra | 22:46 | |
| *** dhellmann[m] has joined #openstack-infra | 22:47 | |
| *** smcginnis[m] has joined #openstack-infra | 22:47 | |
| *** austinsun[m] has joined #openstack-infra | 22:47 | |
| *** linshuicheng[m] has joined #openstack-infra | 22:47 | |
| clarkb | not to open a giant can of worms and the spec we just merged is probably going to negate the usefulness of this but maybe we should consider venvs | 22:48 |
| clarkb | mordred: ^ does pbrx install into virtualenvs in the image to avoid pip problems? or maybe alpine is minimal enough that it doesn't have python package pollution from the upstream distro | 22:48 |
| *** hongbin has quit IRC | 22:50 | |
| *** alex_xu has quit IRC | 22:53 | |
| clarkb | cmurphy: you have probably called it a week by now, but thinking about puppet4, is the next step on review-dev to upgrade to puppet4? | 22:54 |
| openstackgerrit | Merged openstack-dev/pbr master: Add Sphinx extension https://review.openstack.org/580420 | 22:55 |
| openstackgerrit | Merged openstack-dev/pbr master: Deprecate 'build_sphinx' integration https://review.openstack.org/580421 | 22:55 |
| openstackgerrit | Merged openstack-dev/pbr master: Deprecate 'test' integration https://review.openstack.org/580422 | 22:55 |
| clarkb | corvus: https://review.openstack.org/#/c/581565/3 is something to have on your radar with possible log hosting changes | 22:59 |
| *** neiloy has quit IRC | 22:59 | |
| *** alex_xu has joined #openstack-infra | 23:00 | |
| clarkb | corvus: also can you respond to https://review.openstack.org/#/c/546858/ my understanding of it is that if you say run job foo in check pipeline and then delete job foo's definition from your in project config then zuul will just noop | 23:01 |
| clarkb | mriedem: ^ fyi | 23:01 |
| clarkb | I feel like we talked about that at the PTG | 23:01 |
| *** agopi has joined #openstack-infra | 23:03 | |
| *** greghaynes has quit IRC | 23:04 | |
| *** greghaynes has joined #openstack-infra | 23:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Zuul job for stx-docs https://review.openstack.org/579303 | 23:13 |
| *** bobh has joined #openstack-infra | 23:18 | |
| *** sthussey has quit IRC | 23:21 | |
| *** r-daneel has quit IRC | 23:26 | |
| *** r-daneel has joined #openstack-infra | 23:29 | |
| corvus | clarkb, ianw: would it be okay to freeze work on download-logs until the swift stuff is done? | 23:33 |
| clarkb | I'm ok with it. I just use wget to crawl and download everyhting when I need that | 23:34 |
| corvus | clarkb: that is an old change :) | 23:34 |
| clarkb | corvus: yup, I think we can abandon it if I remember the PTG properly | 23:35 |
| corvus | clarkb: replied | 23:37 |
| corvus | (i think you are right) | 23:38 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: WIP: Add upload-logs-swift role https://review.openstack.org/584541 | 23:41 |
| *** rlandy has quit IRC | 23:43 | |
| *** eernst has quit IRC | 23:45 | |
| *** tosky has quit IRC | 23:48 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool master: Use detail listing in integration testing https://review.openstack.org/584542 | 23:50 |
| clarkb | tristanC: Shrews ^ FYI wrote that up really quickly as a way to check the pool listing as our tests of list command is pretty simple | 23:50 |
| clarkb | er wrong window but eh | 23:51 |
| *** wolverineav has quit IRC | 23:58 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Build container images using pbrx https://review.openstack.org/582732 | 23:58 |
| *** wolverineav has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!