| *** thorst has joined #openstack-powervm | 00:41 | |
| *** thorst has quit IRC | 00:45 | |
| *** thorst has joined #openstack-powervm | 00:46 | |
| *** thorst has quit IRC | 00:46 | |
| *** seroyer has joined #openstack-powervm | 00:56 | |
| *** seroyer has quit IRC | 00:57 | |
| *** seroyer has joined #openstack-powervm | 01:58 | |
| *** thorst has joined #openstack-powervm | 02:47 | |
| *** thorst has quit IRC | 02:52 | |
| *** tlian has quit IRC | 03:13 | |
| *** thorst has joined #openstack-powervm | 03:14 | |
| *** apearson has joined #openstack-powervm | 03:14 | |
| *** thorst has quit IRC | 03:14 | |
| *** seroyer has quit IRC | 03:26 | |
| openstackgerrit | Tuan Luong-Anh proposed openstack/nova-powervm: Use ConfigParser instead of SafeConfigParser in Python 3 https://review.openstack.org/403474 | 03:57 |
|---|---|---|
| *** nbante has joined #openstack-powervm | 05:04 | |
| *** apearson has quit IRC | 05:06 | |
| *** thorst has joined #openstack-powervm | 05:15 | |
| *** thorst has quit IRC | 05:20 | |
| *** YuYangWang has joined #openstack-powervm | 07:14 | |
| *** newbie|4 has joined #openstack-powervm | 07:14 | |
| *** nbante has quit IRC | 07:16 | |
| *** thorst has joined #openstack-powervm | 07:16 | |
| *** thorst has quit IRC | 07:21 | |
| *** newbie|4 has quit IRC | 07:45 | |
| *** nbante has joined #openstack-powervm | 07:47 | |
| *** thorst has joined #openstack-powervm | 09:17 | |
| *** thorst has quit IRC | 09:22 | |
| *** thorst has joined #openstack-powervm | 11:18 | |
| *** thorst has quit IRC | 11:23 | |
| *** nbante has quit IRC | 12:04 | |
| *** smatzek has joined #openstack-powervm | 12:17 | |
| *** k0da has joined #openstack-powervm | 12:28 | |
| *** thorst has joined #openstack-powervm | 12:45 | |
| *** tblakes has joined #openstack-powervm | 13:05 | |
| *** svenkat has joined #openstack-powervm | 13:23 | |
| *** mdrabe has joined #openstack-powervm | 13:31 | |
| *** mdrabe has quit IRC | 13:31 | |
| *** mdrabe has joined #openstack-powervm | 13:47 | |
| *** tlian has joined #openstack-powervm | 13:55 | |
| *** jwcroppe_ has quit IRC | 14:00 | |
| *** nbante has joined #openstack-powervm | 14:06 | |
| *** jwcroppe has joined #openstack-powervm | 14:13 | |
| *** esberglu has joined #openstack-powervm | 14:14 | |
| *** seroyer has joined #openstack-powervm | 14:22 | |
| *** k0da has quit IRC | 14:24 | |
| thorst | esberglu: you back? | 15:14 |
| *** edmondsw has joined #openstack-powervm | 15:20 | |
| *** edmondsw has quit IRC | 15:30 | |
| *** tjakobs has joined #openstack-powervm | 15:32 | |
| *** seroyer has quit IRC | 16:12 | |
| *** burgerk has joined #openstack-powervm | 16:12 | |
| esberglu | thorst: Yep | 16:14 |
| *** apearson has joined #openstack-powervm | 16:18 | |
| esberglu | thorst: adreznec: I'm just trying to figure out where we are at. Did the CI get redeployed while I was gone? | 16:22 |
| esberglu | We switched off voting correct? | 16:22 |
| adreznec | No, we hand fixed things because there were so many outstanding patches... | 16:23 |
| adreznec | and we fixed that and turned it back on | 16:23 |
| adreznec | It had been working until Friday | 16:23 |
| esberglu | The zuul logs are getting swamped with errors on the management node | 16:24 |
| adreznec | What kind of errors? | 16:24 |
| esberglu | http://paste.openstack.org/show/595069/ | 16:25 |
| esberglu | Just that over and over | 16:25 |
| esberglu | Crap didn't get the whole trace | 16:26 |
| esberglu | http://paste.openstack.org/show/595070/ | 16:26 |
| adreznec | That's... odd | 16:26 |
| *** seroyer has joined #openstack-powervm | 16:28 | |
| adreznec | What version of zuul are we on | 16:28 |
| adreznec | Does it contain https://github.com/openstack-infra/zuul/commit/07cc33c85c2979c9584d8473cb0a3d7934f146dd | 16:29 |
| adreznec | ? | 16:29 |
| esberglu | We are on 2.5.0, looks like that is 2.5.1 | 16:31 |
| esberglu | Brb, need more coffee. This 11 hour jet lag is getting the best of me | 16:31 |
| adreznec | Hmm ok | 16:31 |
| adreznec | Might need to bump and redeploy | 16:31 |
| thorst | agree. | 16:34 |
| thorst | we know we need a redeploy anyway | 16:34 |
| thorst | for that staging server (well, the file repo server) | 16:35 |
| *** k0da has joined #openstack-powervm | 16:40 | |
| esberglu | thorst: adreznec: Okay. Couple of other things that I'm not sure where we landed on. What happened with the multiple ceilometer-powervm processes? | 17:07 |
| thorst | nothing | 17:12 |
| thorst | we didn't update them to turn off | 17:12 |
| thorst | but thought that they could be shut down | 17:12 |
| esberglu | Alright. And it looks like the "abort the fifo pipe" patch that we were using for CI has been abandoned since the branch was removed. So probably need to redo that on the dev branch | 17:16 |
| thorst | yeah, mind cherry picking to that? | 17:17 |
| esberglu | Oh just saw that you already emailed about that. Yeah I can make that change | 17:18 |
| esberglu | And my devstack patch finally made it in while I was gone. But since we have been running on that patch we are now lagging behind master devstack. | 17:20 |
| esberglu | So I need to get master devstack running in staging CI and see if any changes to devstack since then have hit us | 17:20 |
| thorst | k | 17:23 |
| thorst | esberglu: did you see the update from adreznec about a new DNS host? | 17:23 |
| *** burgerk has quit IRC | 17:23 | |
| esberglu | Saw the email, haven't actually had a chance to look at the change yet | 17:24 |
| adreznec | Ok | 17:24 |
| adreznec | We can chat about it | 17:24 |
| adreznec | It's partially | 17:24 |
| thorst | It's partially? | 17:25 |
| adreznec | Oops sorry, got pulled back to writing notes | 17:25 |
| adreznec | Partially tested | 17:26 |
| thorst | esberglu: when we use that server, there is also a conf option that we need to set in tempest | 17:26 |
| thorst | to tell it to pull the cirros image from there | 17:26 |
| adreznec | Yeah | 17:26 |
| thorst | which was where we had a few tests (maybe 10 a day) failing last week | 17:26 |
| adreznec | I have a patch that's half finished that adds that support | 17:26 |
| adreznec | Need to finish that today | 17:26 |
| adreznec | Obviously untested | 17:26 |
| thorst | test test test! | 17:27 |
| esberglu | So are we sharing these between production and staging? | 17:29 |
| thorst | that server would support both | 17:31 |
| thorst | the idea is all the VMs use that at least as the DNS host | 17:31 |
| thorst | and it is effectively a DNS repeater, so we can change the DNS server in one place rather than having to rebuild all the VMs when one of the many DNS servers dies. | 17:31 |
| *** k0da has quit IRC | 17:32 | |
| *** nbante has quit IRC | 17:59 | |
| *** esberglu has quit IRC | 18:02 | |
| *** esberglu has joined #openstack-powervm | 18:03 | |
| *** esberglu has quit IRC | 18:07 | |
| *** esberglu has joined #openstack-powervm | 18:26 | |
| *** esberglu has quit IRC | 18:27 | |
| *** esberglu has joined #openstack-powervm | 18:27 | |
| BorD__ | if there a way to blow up the rpm database on aix and start over? dependency hell strikes again. | 19:14 |
| thorst | apearson adreznec: know who could possibly answer that? | 19:16 |
| adreznec | Sorry, outside my area of expertise thorst. Maybe seroyer might know someone? ^ | 19:18 |
| seroyer | BorD__ I haven't tried it but you can try this: http://www-01.ibm.com/support/docview.wss?uid=isg3T1000657 | 19:21 |
| BorD__ | that will rebuild.. | 19:21 |
| seroyer | Ah, you want to delete it along with any packages that are already installed? | 19:22 |
| BorD__ | correct | 19:22 |
| seroyer | That's beyond my experience, sorry. Might need to contact AIX support. | 19:24 |
| BorD__ | ha. have you called support lately? | 19:25 |
| seroyer | Never once in my life. | 19:25 |
| seroyer | I can try to poke around and see if I can get an answer. No promises, though... | 19:26 |
| esberglu | thorst: adreznec: Okay the abort fifo is in develop. I'm about to redeploy with that and with zuul 2.5.1 | 19:34 |
| thorst | wooooo | 19:34 |
| adreznec | Cool | 19:34 |
| adreznec | Is this a redeploy of prod or staging? | 19:35 |
| esberglu | Production | 19:38 |
| esberglu | Unless we want to test 2.5.1 on staging first | 19:38 |
| thorst | well, prod is down? | 19:39 |
| adreznec | Nah, that's fine | 19:39 |
| thorst | so my vote would be do whatever we need to to get prod running reliably again | 19:39 |
| adreznec | Just checking. Which patches are you all pilling in | 19:39 |
| thorst | (and then move to staging to test the new things) | 19:39 |
| adreznec | *pulling | 19:39 |
| esberglu | Moving zuul to 2.5.1 | 19:39 |
| esberglu | Applying the new abort fifo patch | 19:40 |
| esberglu | Checking out the known working devstack | 19:40 |
| *** svenkat has quit IRC | 19:40 | |
| esberglu | That's it | 19:40 |
| adreznec | Ok | 19:40 |
| adreznec | Just curious since there are so many at this point | 19:41 |
| thorst | what about that dns thing? | 19:41 |
| thorst | too much change? | 19:41 |
| adreznec | Yeah, lets wait on that | 19:41 |
| thorst | fair enough | 19:41 |
| adreznec | I'd rather test that in staging first since I haven't tested the client side at all | 19:42 |
| thorst | how much data are you hosting out of there adreznec | 19:42 |
| adreznec | Right now? Pretty much nothing since nothing is requesting against it | 19:42 |
| adreznec | It will increase based on the requests clients make of it | 19:42 |
| adreznec | Both for recursive DNS requests and apt packages | 19:43 |
| thorst | guesstimate of how much it'd host? | 19:43 |
| thorst | I'm wondering if I should move that to our SAN | 19:43 |
| thorst | or keep it on the rusty spinner | 19:43 |
| adreznec | Hmm | 19:44 |
| thorst | well, time will tell | 19:44 |
| thorst | I won't get the SAN for another week or two | 19:44 |
| thorst | they have to move a SMI-S provider off the box | 19:44 |
| adreznec | Maybe a couple of gigs | 19:44 |
| thorst | and if you don't know what a SMI-S provider is, consider yourself lucky | 19:45 |
| adreznec | Isn't that some emc thing? | 19:45 |
| adreznec | sounds gross | 19:45 |
| thorst | lots of storage's have it...and not just storage's | 19:46 |
| thorst | but meh...ignore it. | 19:46 |
| esberglu | Here's the plan for the staging CI deploy. | 19:49 |
| esberglu | - Reinstall staging neo (neo14) | 19:49 |
| esberglu | - Do a full staging redeploy with the following | 19:49 |
| esberglu | 1) Use the latest devstack (prod. is behind from pinning on my devstack change) | 19:49 |
| esberglu | 2) Test out the DNS and apt-cache stuff | 19:49 |
| esberglu | 3) Make sure that the skip list with ids instead of strings works | 19:49 |
| esberglu | I think that covers all of the major outstanding CI changes, the rest are either trivial or already tested | 19:49 |
| thorst | excellent | 19:50 |
| thorst | nice to have you back :-) | 19:50 |
| esberglu | : ) | 19:51 |
| *** k0da has joined #openstack-powervm | 19:52 | |
| *** apearson has quit IRC | 20:01 | |
| *** apearson has joined #openstack-powervm | 20:02 | |
| *** svenkat has joined #openstack-powervm | 20:14 | |
| *** edmondsw has joined #openstack-powervm | 20:18 | |
| *** edmondsw has quit IRC | 20:23 | |
| *** apearson has quit IRC | 21:00 | |
| *** apearson has joined #openstack-powervm | 21:02 | |
| efried | thorst: FYI, I'm working hard to backport "don't use pypowervm's test data files" to nova-powervm... but I don't fully agree we should do that. | 21:03 |
| thorst | at least for in tree, I don't want to | 21:03 |
| thorst | I don't like testing pypowervm features up in nova-powervm | 21:03 |
| thorst | and won't in nova. | 21:03 |
| efried | We're not testing pypowervm features. We're piggybacking on pypowervm's test data as a convenient way to build wrapper objects for purposes of testing nova-powervm features. | 21:03 |
| thorst | right...but that's still gross. | 21:04 |
| thorst | I'm OK with it up in nova-powervm, grumpy about it, but OK with it | 21:04 |
| efried | Is it really grosser than having a mock with a dozen or more properties? | 21:04 |
| efried | some nested | 21:04 |
| thorst | to a core that didn't spend 2 years on it...yeah, I think so | 21:04 |
| efried | okaaaay. | 21:04 |
| efried | fwiw, tracking some of these down is a super PITA. | 21:05 |
| thorst | heh, I bet... | 21:05 |
| thorst | I was mr. grumpy back then | 21:05 |
| thorst | but I was convinced that 'its testing more of the stack' | 21:05 |
| thorst | and then I think we eased back on a LOT of those | 21:05 |
| thorst | but not all | 21:05 |
| thorst | erlarese did some of that, and a few others I think | 21:05 |
| *** tjakobs has quit IRC | 21:05 | |
| *** smatzek has quit IRC | 21:33 | |
| *** tjakobs has joined #openstack-powervm | 21:37 | |
| *** edmondsw has joined #openstack-powervm | 21:40 | |
| *** edmondsw has quit IRC | 21:44 | |
| *** thorst has quit IRC | 21:55 | |
| *** smatzek has joined #openstack-powervm | 22:05 | |
| *** seroyer has quit IRC | 22:06 | |
| *** smatzek has quit IRC | 22:19 | |
| *** thorst has joined #openstack-powervm | 22:24 | |
| *** svenkat has quit IRC | 22:26 | |
| *** thorst has quit IRC | 22:28 | |
| *** tblakes has quit IRC | 22:39 | |
| *** mdrabe has quit IRC | 22:58 | |
| *** jwcroppe has quit IRC | 23:26 | |
| *** seroyer has joined #openstack-powervm | 23:28 | |
| *** apearson has quit IRC | 23:33 | |
| *** edmondsw has joined #openstack-powervm | 23:55 | |
| *** edmondsw has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!