| *** esberglu has quit IRC | 00:10 | |
| *** thorst_ has joined #openstack-powervm | 00:12 | |
| *** jwcroppe has joined #openstack-powervm | 00:29 | |
| *** jwcroppe has quit IRC | 00:30 | |
| *** jwcroppe has joined #openstack-powervm | 00:30 | |
| *** chas_ has joined #openstack-powervm | 00:44 | |
| *** chas_ has quit IRC | 00:49 | |
| *** seroyer has joined #openstack-powervm | 00:56 | |
| *** apearson has joined #openstack-powervm | 01:04 | |
| *** wangqwsh has joined #openstack-powervm | 01:22 | |
| *** chas_ has joined #openstack-powervm | 01:45 | |
| *** chas_ has quit IRC | 01:49 | |
| *** smatzek has joined #openstack-powervm | 01:56 | |
| thorst_ | hola | 02:00 |
|---|---|---|
| adreznec | Hey thorst_. wangqwsh are you there? | 02:01 |
| wangqwsh | yes | 02:01 |
| thorst_ | o/ | 02:01 |
| adreznec | All right, lets kick things off then | 02:02 |
| adreznec | #startmeeting PowerVM CI Meeting | 02:02 |
| openstack | Meeting started Wed Nov 9 02:02:35 2016 UTC and is due to finish in 60 minutes. The chair is adreznec. Information about MeetBot at http://wiki.debian.org/MeetBot. | 02:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 02:02 |
| openstack | The meeting name has been set to 'powervm_ci_meeting' | 02:02 |
| adreznec | #topic Current status | 02:03 |
| adreznec | So, I haven't had a chance to look at the latest revision of your patchset that was put up wangqwsh. Do you want to run us through status? | 02:03 |
| wangqwsh | 1. pypowervm adapter patch's done. 2. we can install openstack with osa | 02:04 |
| thorst_ | whoa, that's pretty good. | 02:04 |
| wangqwsh | I update the script related to tempest variables | 02:05 |
| thorst_ | so what's next there? Tempest tests? | 02:05 |
| adreznec | Awesome, you've gotten things running in the staging environment then? | 02:05 |
| thorst_ | ahh | 02:05 |
| wangqwsh | there is an issue in galera while running tempest | 02:05 |
| wangqwsh | i am trying to find the root casue. | 02:06 |
| wangqwsh | casue -> cause | 02:06 |
| adreznec | In galera while running tempest? Are we seeing database failures? | 02:06 |
| wangqwsh | yes | 02:06 |
| wangqwsh | the mysql is not running | 02:06 |
| adreznec | Hmm ok | 02:06 |
| adreznec | How much memory are we assigning to these VMs? | 02:06 |
| wangqwsh | 8g | 02:07 |
| adreznec | I know we've been seeing some issues in OSA AIO testing with 8g lately | 02:07 |
| thorst_ | that's too little? | 02:07 |
| wangqwsh | yes | 02:07 |
| adreznec | lately being yesterday/today | 02:07 |
| adreznec | Yeah, issues with mysql running out of memory. Still trying to debug exactly why | 02:07 |
| thorst_ | yikes...that's a fair amount. | 02:07 |
| thorst_ | Lets definitely not double that...but instead maybe step to 10... | 02:07 |
| adreznec | It's the absolute minimum req for OSA AIO FWIW | 02:08 |
| thorst_ | and it probably doesn't help that we're SMT-8 | 02:08 |
| adreznec | Recommended is 16 | 02:08 |
| thorst_ | though I doubt that has the same impact to OSA as it does devstck | 02:08 |
| thorst_ | yikes...and this would be all our nodes... | 02:08 |
| thorst_ | not just OSA nodes? | 02:08 |
| adreznec | Yeah, I'm not sure how much we can tie back to that | 02:08 |
| adreznec | Unless we decide to split nodes | 02:08 |
| adreznec | I know, we already have memory issues | 02:09 |
| adreznec | The OSA gate upstream uses 8gb | 02:09 |
| thorst_ | hmm...well, it generally isn't too stressed (each node runs 4 at once, each with 8 GB, but has 128 GB memory) | 02:09 |
| adreznec | So I think we should be able to make it work with that amount with tweaking... just not sure what's the issue | 02:09 |
| thorst_ | so we have SOME space...but we need to leave more for the VMs during the tempest runs themselves. | 02:09 |
| adreznec | Right | 02:09 |
| thorst_ | lets maybe try 12 and back track from there... | 02:10 |
| adreznec | wangqwsh: how far into tempest are you making it? | 02:10 |
| wangqwsh | 3h | 02:11 |
| adreznec | Whoa | 02:11 |
| adreznec | Tempest definitely shouldn't take that long to run | 02:11 |
| wangqwsh | because I restart the db by manual :) | 02:11 |
| adreznec | Ah, I see | 02:12 |
| thorst_ | well... | 02:12 |
| wangqwsh | if not, a few seconds | 02:12 |
| thorst_ | tempest used to take 4 hours until we bumped up the threads | 02:12 |
| adreznec | Yeah, I was just hoping we wouldn't have that same issue here | 02:13 |
| thorst_ | yeah....just need to make sure we have the right thread count | 02:14 |
| thorst_ | but it sounds like overall we're making good progress there :-) | 02:15 |
| adreznec | All right, so wangqwsh do you want to try bumping things to 10gb (or 12gb) and see if we still have galera issues? | 02:15 |
| wangqwsh | sure. | 02:15 |
| adreznec | All right | 02:15 |
| wangqwsh | i used this image for CI, Ubuntu_16-04-01_2g.img | 02:16 |
| adreznec | #action wangqwsh to try increasing OSA VM memory to 10gb/12gb and retesting | 02:16 |
| wangqwsh | is that fine? | 02:16 |
| adreznec | That should be right | 02:16 |
| wangqwsh | ok | 02:16 |
| adreznec | All right. On to other status | 02:17 |
| adreznec | thorst_ has his well named fifo_pipo patch on deck | 02:17 |
| thorst_ | actually efried has taken over driving that | 02:17 |
| thorst_ | and PipesOutLightsOut has become RESTAPIPipe | 02:17 |
| thorst_ | I like my name better. | 02:17 |
| adreznec | It'll always be fifo_pipo to me | 02:18 |
| adreznec | and git | 02:18 |
| thorst_ | :-) | 02:18 |
| thorst_ | so that's under testing | 02:18 |
| adreznec | All right | 02:18 |
| thorst_ | making progress...hopefully more known tomorrow | 02:18 |
| adreznec | I believe esberglu was testing that, right? | 02:19 |
| thorst_ | yep | 02:20 |
| adreznec | #action esberglu to continue testing RESTAPIPipe patch with assistance from efried | 02:20 |
| adreznec | Who are here only in spirit | 02:21 |
| adreznec | All righty then | 02:21 |
| adreznec | #topic Next steps | 02:21 |
| adreznec | Looking forward then, it seems like things are starting to shape up | 02:21 |
| thorst_ | definitely. I think the next thing - perhaps out of CI - is the pypowervm integrated into global-requirements | 02:22 |
| thorst_ | asking for that to be done...but - not really tied to CI | 02:22 |
| adreznec | wangqwsh: it sounds like next up after the galera you're going to be digging into tempest config? | 02:22 |
| wangqwsh | yes | 02:22 |
| adreznec | thorst_: yeah, I spent some time with that today. I'm having some issues with getting setuptools to build with appropriate versioning | 02:23 |
| thorst_ | :-/ alright. We need to get that into the priority view next week...once CI stabilizes | 02:23 |
| adreznec | I talked with esberglu about how he supposedly got setuptools/pbr to disregard PEP440 before, but he couldn't remember and also had no notes | 02:23 |
| adreznec | I'll continue digging and see what I can get worked out here | 02:24 |
| adreznec | Hopefully the necessary pypowervm version will be up tomorrow | 02:24 |
| adreznec | #action adreznec to get pypowervm update published to pypi to drive towards getting pypowervm in g-r | 02:24 |
| thorst_ | yikes...I thought he had a wiki page of notes somewhere on it... | 02:25 |
| adreznec | Nope... | 02:25 |
| thorst_ | gr8 | 02:25 |
| adreznec | unfortunate because this is the same issue I hit last time I tried | 02:25 |
| thorst_ | maybe we ask him to do it from scratch and he stumbles upon the same result? | 02:26 |
| adreznec | This is where I'd normally put a random, semi-related giphy | 02:26 |
| thorst_ | heh | 02:26 |
| adreznec | I'll give it a bit more trying tomorrow | 02:26 |
| adreznec | and if I can't get it I'll walk over to his desk | 02:26 |
| thorst_ | sounds good | 02:26 |
| adreznec | and we'll figure it out | 02:26 |
| thorst_ | agenda for next week...getting the logs published on nova patches... | 02:26 |
| adreznec | So I know esberglu isn't here, but it sounds like mriedem wants us to change the way we're handling our skip list? | 02:27 |
| thorst_ | yeah, I sent that to esberglu already and he took that as a TODO | 02:28 |
| thorst_ | but...for good measure | 02:28 |
| thorst_ | #action esberglu to switch temptest skip list from name to test id | 02:28 |
| adreznec | Yeah, I saw that mentioned earlier | 02:28 |
| adreznec | Are we going to have to start maintaining 2 skip lists now? | 02:28 |
| adreznec | One for dsvm and one for osa? | 02:28 |
| thorst_ | I don't see a need ... yet | 02:29 |
| adreznec | Right | 02:29 |
| thorst_ | might need for integrated nova driver versus out of tree though | 02:29 |
| adreznec | in osa since we're using ovs we'll be supporting a theoretical superset of function | 02:29 |
| adreznec | sec groups, l3, etc | 02:29 |
| thorst_ | right...but lets start small and keep consistent | 02:30 |
| thorst_ | then add complexity as we go | 02:30 |
| adreznec | k | 02:30 |
| adreznec | integrated vs OOT we'll definitely need two lists, at least at first | 02:30 |
| adreznec | All right, so wangqwsh should be fine using the existing tempest configs then | 02:31 |
| adreznec | in terms of tests to run/skip | 02:31 |
| adreznec | Ok, so I know we're over time here | 02:32 |
| thorst_ | its because we're all watching florida | 02:32 |
| adreznec | It's always Florida | 02:32 |
| thorst_ | any other aspects we need to cover? | 02:32 |
| adreznec | wangqwsh anything else from your end? | 02:32 |
| wangqwsh | nope | 02:32 |
| adreznec | All right | 02:33 |
| adreznec | That's a wrap then | 02:33 |
| adreznec | Thanks all! | 02:33 |
| adreznec | #endmeeting | 02:33 |
| thorst_ | thanks! | 02:33 |
| openstack | Meeting ended Wed Nov 9 02:33:27 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 02:33 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_ci_meeting/2016/powervm_ci_meeting.2016-11-09-02.02.html | 02:33 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_ci_meeting/2016/powervm_ci_meeting.2016-11-09-02.02.txt | 02:33 |
| wangqwsh | thanks | 02:33 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_ci_meeting/2016/powervm_ci_meeting.2016-11-09-02.02.log.html | 02:33 |
| *** thorst_ has quit IRC | 02:39 | |
| *** thorst_ has joined #openstack-powervm | 02:40 | |
| *** chas_ has joined #openstack-powervm | 02:45 | |
| *** thorst_ has quit IRC | 02:48 | |
| *** chas_ has quit IRC | 02:50 | |
| *** seroyer has quit IRC | 03:06 | |
| *** smatzek has quit IRC | 03:36 | |
| *** thorst_ has joined #openstack-powervm | 03:46 | |
| *** chas_ has joined #openstack-powervm | 03:46 | |
| *** tjakobs has joined #openstack-powervm | 03:48 | |
| *** chas_ has quit IRC | 03:51 | |
| *** tjakobs has quit IRC | 03:54 | |
| *** thorst_ has quit IRC | 03:54 | |
| *** kylek3h has quit IRC | 04:08 | |
| *** tjakobs has joined #openstack-powervm | 04:16 | |
| *** tjakobs has quit IRC | 04:45 | |
| *** thorst_ has joined #openstack-powervm | 04:52 | |
| *** thorst_ has quit IRC | 04:59 | |
| *** kylek3h has joined #openstack-powervm | 05:09 | |
| *** kylek3h has quit IRC | 05:15 | |
| *** apearson has quit IRC | 05:31 | |
| *** chas_ has joined #openstack-powervm | 05:48 | |
| *** tjakobs has joined #openstack-powervm | 05:49 | |
| *** chas_ has quit IRC | 05:52 | |
| *** tjakobs has quit IRC | 05:56 | |
| *** thorst_ has joined #openstack-powervm | 05:56 | |
| *** thorst_ has quit IRC | 06:04 | |
| *** wangqwsh has quit IRC | 06:26 | |
| *** chas_ has joined #openstack-powervm | 06:48 | |
| *** esberglu has joined #openstack-powervm | 06:49 | |
| *** chas_ has quit IRC | 06:53 | |
| *** esberglu has quit IRC | 06:53 | |
| *** thorst_ has joined #openstack-powervm | 07:01 | |
| *** thorst_ has quit IRC | 07:09 | |
| *** kylek3h has joined #openstack-powervm | 07:11 | |
| *** kylek3h has quit IRC | 07:16 | |
| *** chas_ has joined #openstack-powervm | 07:19 | |
| *** esberglu has joined #openstack-powervm | 07:48 | |
| *** esberglu has quit IRC | 07:53 | |
| *** thorst_ has joined #openstack-powervm | 08:08 | |
| *** thorst_ has quit IRC | 08:13 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #openstack-powervm | 08:48 | |
| *** thorst_ has joined #openstack-powervm | 09:11 | |
| *** kylek3h has joined #openstack-powervm | 09:12 | |
| *** kylek3h has quit IRC | 09:17 | |
| *** thorst_ has quit IRC | 09:19 | |
| *** thorst_ has joined #openstack-powervm | 10:16 | |
| *** thorst_ has quit IRC | 10:23 | |
| *** esberglu has joined #openstack-powervm | 10:32 | |
| *** esberglu has quit IRC | 10:36 | |
| *** kylek3h has joined #openstack-powervm | 11:13 | |
| *** kylek3h has quit IRC | 11:17 | |
| *** thorst_ has joined #openstack-powervm | 11:20 | |
| *** thorst_ has quit IRC | 11:27 | |
| *** smatzek has joined #openstack-powervm | 12:08 | |
| *** esberglu has joined #openstack-powervm | 12:11 | |
| *** esberglu has quit IRC | 12:16 | |
| *** seroyer has joined #openstack-powervm | 12:20 | |
| *** thorst_ has joined #openstack-powervm | 12:48 | |
| *** thorst_ has quit IRC | 12:48 | |
| *** thorst_ has joined #openstack-powervm | 12:48 | |
| *** seroyer has quit IRC | 12:59 | |
| *** svenkat has joined #openstack-powervm | 12:59 | |
| *** kylek3h has joined #openstack-powervm | 13:09 | |
| *** edmondsw has joined #openstack-powervm | 13:15 | |
| *** jwcroppe has quit IRC | 13:16 | |
| *** tblakes has joined #openstack-powervm | 13:20 | |
| *** seroyer has joined #openstack-powervm | 13:33 | |
| *** jwcroppe has joined #openstack-powervm | 13:51 | |
| *** seroyer has quit IRC | 13:57 | |
| *** apearson has joined #openstack-powervm | 13:58 | |
| *** seroyer has joined #openstack-powervm | 14:05 | |
| *** mdrabe has joined #openstack-powervm | 14:11 | |
| *** smatzek has quit IRC | 14:32 | |
| *** esberglu has joined #openstack-powervm | 14:33 | |
| esberglu | thorst_: Seeing that read only filesystem issue on staging now as well as some of the production nodes. Any idea what could be causing this? | 14:44 |
| openstackgerrit | Drew Thorstensen (thorst) proposed openstack/nova-powervm: Add delay queue for events https://review.openstack.org/391164 | 14:44 |
| thorst_ | esberglu: check the backing SAN...lets see if we have a true SAN issue | 14:45 |
| thorst_ | or if its network. | 14:45 |
| thorst_ | do you know how to do that? | 14:46 |
| esberglu | nope | 14:50 |
| thorst_ | esberglu: PM'd you the SAN connection info. Go into it, and look for critical errors. I think bottom right. See if any are new | 14:54 |
| *** efried has joined #openstack-powervm | 14:56 | |
| esberglu | Nothing there, last status alert is from a month ago | 14:56 |
| thorst_ | then its got to be network blips | 15:08 |
| thorst_ | I'd restart the VIOSes. Clear the systems and reboot the system basically | 15:08 |
| thorst_ | let me check for a broadcast storm first. | 15:08 |
| thorst_ | doesn't seem like a broadcast storm | 15:11 |
| esberglu | thorst_: Okay I will start rebooting. | 15:22 |
| efried | adreznec, did you really mean to make the CI meeting 7:30am Central? | 15:22 |
| adreznec | efried: that's when we'd been doing it to make it easier for wangqwsh to attend | 15:24 |
| adreznec | If that doesn't work we could shift it around | 15:24 |
| efried | Okay, I'm fine with that answer. Just wanted to make sure it was intentional. | 15:24 |
| adreznec | Yep. Like I said, if either of these meeting times end up not working out we can definitely reschedule. | 15:24 |
| adreznec | Just trying to deal with timezones | 15:25 |
| *** seroyer has quit IRC | 15:31 | |
| *** tjakobs has joined #openstack-powervm | 15:33 | |
| *** seroyer has joined #openstack-powervm | 15:47 | |
| *** mdrabe has quit IRC | 16:01 | |
| *** mdrabe has joined #openstack-powervm | 16:15 | |
| *** smatzek has joined #openstack-powervm | 16:26 | |
| thorst_ | esberglu: neo40 still working? | 16:32 |
| thorst_ | I'm going to try that remote debug debacle | 16:32 |
| esberglu | efried: ^^ | 16:32 |
| efried | thorst_ Yes. This is my LPM stack, which finally seems to be relatively stable, so don't eff it up too badly. | 16:33 |
| * thorst_ apologizes in advance | 16:33 | |
| thorst_ | I actually shouldn't be able to screw it up | 16:33 |
| thorst_ | unless I don't clean up my files... | 16:33 |
| efried | Note that my SSP currently has the 2G Ubuntu 1604 image already in it. | 16:34 |
| efried | So you may want to use a different one. | 16:34 |
| thorst_ | efried: I plan to use a 0's file | 16:38 |
| efried | dig | 16:38 |
| thorst_ | adreznec: I didn't realize I had actually named my branch fifo_pipo | 16:40 |
| thorst_ | that's 1 week ago me making a joke to today me | 16:40 |
| thorst_ | efried: can we install the package that allows remote connections? | 16:44 |
| efried | thorst_, sure. | 16:49 |
| adreznec | thorst_: lol, that makes more sense | 16:53 |
| thorst_ | efried: This is the hidden error: SpecialFileError: SpecialF...d pipe',) | 17:12 |
| efried | thorst_ - did you truncate that or does it show up that way in the log? | 17:12 |
| thorst_ | that's what shows up in pdb | 17:13 |
| thorst_ | "`/tmp/tmpNY1Ukv/REST_API_Pipe` is a named pipe" | 17:15 |
| thorst_ | probably just how I did this... | 17:17 |
| efried | thorst_ - what's the stack when that happens? The open on one side or the other? The read or write? | 17:25 |
| thorst_ | its how I did it. shutil can't copy (dunno why) to a named pipe | 17:25 |
| thorst_ | its not the real error | 17:25 |
| thorst_ | I see the issue. | 17:26 |
| thorst_ | we've got a threading issue. | 17:26 |
| thorst_ | glance issues the write command to the file. It waits for the write command to come back. The read of the named pipe never gets invoked because its blocked on the thread that is issuing the write. | 17:27 |
| thorst_ | the RESTAPIPipe needs to run in a different thread... | 17:27 |
| thorst_ | does that make sense? | 17:27 |
| efried | thorst_, I thought it was. | 17:31 |
| efried | thorst_, how determined are we to keep this in memory? | 17:33 |
| efried | thorst_, and what's using shutil? | 17:40 |
| efried | thorst_ must be deep in a burrito. | 17:45 |
| thorst_ | efried: just walked to cafe | 17:53 |
| thorst_ | all from the same thread we invoke the 'write to file' before we invoke the 'read from file' so we are stuck in io block | 17:53 |
| efried | thorst_, where's the code that writes to the pipe? | 17:57 |
| thorst_ | customer provided function | 17:57 |
| thorst_ | that we invoke | 17:57 |
| efried | where's that invocation? | 17:57 |
| *** k0da has joined #openstack-powervm | 17:59 | |
| thorst_ | line 418 | 18:00 |
| thorst_ | 'Have the client start writing to the pipe' | 18:00 |
| efried | I see it | 18:00 |
| thorst_ | that is a blocking call | 18:00 |
| efried | Okay, so let's shove that guy into its own thread. | 18:00 |
| thorst_ | efried: +1 | 18:00 |
| efried | thorst_, you or me? | 18:01 |
| thorst_ | but do we make it part of the bigger thread pool (in upload_stream_coordinated) or down IN that method | 18:01 |
| thorst_ | I can...just want to make sure my approach isn't something you'll hate | 18:02 |
| efried | How would you get that thread pool down in there? You would have to pass it through two method calls and the RESTAPIPipe constructor. | 18:02 |
| thorst_ | see ps1 | 18:03 |
| thorst_ | easier there | 18:03 |
| thorst_ | efried: I'd just stick that thing in the thread wait pool. | 18:05 |
| efried | what 'thing'? | 18:05 |
| thorst_ | efried: let me explain with code...I'll get a rev up in a few. | 18:06 |
| efried | ight | 18:06 |
| *** efried has quit IRC | 18:35 | |
| svenkat | efried: working on a problem with vNIC create events flowing to SEA agent which provisions vlan … it is due to VIFEventHandler in agent base treating all events same and send it to both agents. I want to know how do I distinguish between CNA vs VNIC events in ProvisionRequest.for_event method. which bit in the invoming event can I use? | 18:44 |
| thorst_ | svenkat: efried hopped offline. Lets get the bug open in launchpad for now | 18:48 |
| svenkat | sure. i will. | 18:48 |
| thorst_ | svenkat: thx. I think its a solid bug | 18:48 |
| svenkat | thorst_ : opened https://bugs.launchpad.net/networking-powervm/+bug/1640564 | 18:53 |
| openstack | Launchpad bug 1640564 in networking-powervm "agent_base.VIFEventHandler not distinguishing between CNA and VNIC events" [Undecided,New] | 18:53 |
| adreznec | thorst_: have you ever seen this with the new nova-powervm/pypowervm image code? http://paste.openstack.org/show/588607/ | 19:24 |
| adreznec | I just did a redeploy of the latest OSA and I'm hitting that | 19:24 |
| adreznec | pypowervm is 1.0.0.4-161108 | 19:24 |
| thorst_ | adreznec: that is eerily similar to the chunk errors we hit with our own chunky reader | 19:25 |
| *** chas_ has quit IRC | 19:33 | |
| *** efried has joined #openstack-powervm | 19:42 | |
| *** kriskend has joined #openstack-powervm | 19:48 | |
| seroyer | I seem to remember seeing discussions around None getting returned when we were trying to figure out what was wrong with the chunky method. We weren't hitting the None error, but I think others did... Will try and search for it again. | 20:04 |
| seroyer | Ah. This is what I was remembering: | 20:06 |
| seroyer | https://bugs.launchpad.net/python-glanceclient/+bug/1434040 | 20:06 |
| openstack | Launchpad bug 1434040 in Glance Client "Integrity check for images with checksum = "None"" [Undecided,Incomplete] | 20:06 |
| seroyer | Not the same problem. | 20:06 |
| *** tblakes has quit IRC | 20:33 | |
| *** chas_ has joined #openstack-powervm | 20:34 | |
| thorst_ | well, I may have remote API working | 20:35 |
| thorst_ | definition of working varies, but it limps along | 20:36 |
| *** chas_ has quit IRC | 20:39 | |
| adreznec | thorst_: slow? | 20:40 |
| thorst_ | adreznec: actually quite speedy | 20:43 |
| thorst_ | I'm just 100% sure that mr. efried will hack it apart into something better | 20:43 |
| thorst_ | efried esberglu: I put a new patch up on 4458. Can we try that out? It worked locally (to a remote server) | 20:44 |
| efried | You do the hard part of making it function; I'll do the monkey work of reorganizing it so it's readable. | 20:44 |
| thorst_ | efried: see, I think its readable | 20:44 |
| thorst_ | :-) | 20:44 |
| efried | I haven't looked yet. | 20:44 |
| efried | But what we had before 4458 even started was approaching unreadable. | 20:44 |
| thorst_ | approaching? | 20:44 |
| thorst_ | :-) | 20:44 |
| thorst_ | it is certainly a jack of all trades. | 20:45 |
| esberglu | thorst_: Yeah I will try it out. Hopefully my systems don't die before then. Rebooting seemed to work temporarily | 20:45 |
| thorst_ | esberglu: we can't recreate the network issue at the moment....so | 20:50 |
| thorst_ | here's hoping | 20:50 |
| adreznec | Yay | 20:50 |
| efried | So thorst_, why did you find you had to use DISK_IMAGE? | 20:50 |
| adreznec | FYI thorst_ that glance issue appears to have somehow been resolved, but I'm not entirely sure how I resolved it | 20:50 |
| adreznec | So... that's uncomforting | 20:50 |
| thorst_ | efried: do you mean stream instead of coordinated? | 20:51 |
| efried | yes | 20:51 |
| thorst_ | because stream goes to the REST API | 20:51 |
| thorst_ | coordinated requires the REST API to be on the local server | 20:51 |
| thorst_ | which is kinda opposite of what we're doing for this CI env | 20:51 |
| efried | figgered it was something like that. | 20:51 |
| efried | I'm asking what about coordinated requires local? | 20:51 |
| efried | fact that you're passing a file path? | 20:52 |
| thorst_ | the REST API makes a local file | 20:52 |
| thorst_ | they're doing a FIFO pipe themselves | 20:53 |
| thorst_ | so if the REST API is local, we can let them do the FIFO pipe | 20:53 |
| thorst_ | but if we're remote, then we need to do the FIFO pipe and send it to the HTTP server stream | 20:54 |
| thorst_ | esberglu: I think the edge switch is in a bad state...I think I can just reboot it | 20:56 |
| thorst_ | but your SSPs may be a little cranky after that | 20:57 |
| thorst_ | (but ultimately happier?) | 20:57 |
| esberglu | Okay. Want to hold off and see if that patch works first? | 20:58 |
| thorst_ | no | 20:58 |
| thorst_ | this takes a minute or two? | 20:58 |
| esberglu | Okay go for it | 20:58 |
| esberglu | thorst_: what do you mean by cranky? | 20:59 |
| thorst_ | esberglu: ports were flapping, it was claiming there were broadcast storms and none of its peers agreed, things like that | 21:00 |
| thorst_ | the silence....of a network switch reboot... | 21:01 |
| thorst_ | will it come back... | 21:01 |
| thorst_ | and we're back | 21:01 |
| adreznec | [15:01:13] <thorst_>will it come back... | 21:04 |
| adreznec | [15:01:24] <thorst_>and we're back | 21:04 |
| adreznec | Such suspense, 11 seconds | 21:04 |
| apearson | Back, but... | 21:04 |
| thorst_ | apearson: still looking - cool yer jets | 21:05 |
| thorst_ | the two systems that work versus fail are one hop away | 21:05 |
| thorst_ | so comparing paths | 21:06 |
| efried | thorst_, totally on board with the theme of 4458, but have a number of gripes around the flow and use of vars and such. | 21:26 |
| efried | Do you want me to futz with it myself, or comment it up and let you do it? | 21:26 |
| *** smatzek has quit IRC | 21:30 | |
| *** chas_ has joined #openstack-powervm | 21:35 | |
| *** edmondsw has quit IRC | 21:37 | |
| thorst_ | efried: I'm cool if you want to do it and I'll poke around the network to see what's what | 21:38 |
| efried | thorst_, roger wilco. | 21:39 |
| *** chas_ has quit IRC | 21:40 | |
| *** thorst_ has quit IRC | 21:49 | |
| *** apearson has quit IRC | 21:50 | |
| *** jwcroppe has quit IRC | 21:57 | |
| *** jwcroppe has joined #openstack-powervm | 21:58 | |
| *** jwcroppe has quit IRC | 22:03 | |
| *** svenkat has quit IRC | 22:03 | |
| *** seroyer has quit IRC | 22:31 | |
| *** kriskend has quit IRC | 22:31 | |
| *** tblakes has joined #openstack-powervm | 22:36 | |
| *** chas_ has joined #openstack-powervm | 22:36 | |
| *** thorst_ has joined #openstack-powervm | 22:39 | |
| *** chas_ has quit IRC | 22:41 | |
| *** mdrabe has quit IRC | 22:42 | |
| *** mdrabe has joined #openstack-powervm | 22:45 | |
| *** k0da has quit IRC | 22:50 | |
| *** dwayne_ has quit IRC | 23:00 | |
| *** esberglu has quit IRC | 23:07 | |
| *** tjakobs has quit IRC | 23:10 | |
| *** mdrabe has quit IRC | 23:18 | |
| *** esberglu has joined #openstack-powervm | 23:21 | |
| *** seroyer has joined #openstack-powervm | 23:23 | |
| *** jwcroppe has joined #openstack-powervm | 23:24 | |
| *** jwcroppe has quit IRC | 23:25 | |
| *** esberglu has quit IRC | 23:25 | |
| *** jwcroppe has joined #openstack-powervm | 23:25 | |
| *** jwcroppe has quit IRC | 23:30 | |
| *** thorst_ has quit IRC | 23:32 | |
| *** thorst_ has joined #openstack-powervm | 23:32 | |
| *** seroyer has quit IRC | 23:37 | |
| *** thorst_ has quit IRC | 23:40 | |
| *** dwayne_ has joined #openstack-powervm | 23:42 | |
| *** chas_ has joined #openstack-powervm | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!