| *** sdake_ has joined #openstack-ansible | 00:03 | |
| *** sdake has quit IRC | 00:07 | |
| *** darrenc is now known as darrenc_Afk | 00:40 | |
| *** markvoelker has joined #openstack-ansible | 00:50 | |
| *** stevemar has quit IRC | 00:53 | |
| *** markvoelker has quit IRC | 00:55 | |
| *** darrenc_Afk is now known as darrenc | 01:25 | |
| *** britthouser has quit IRC | 01:25 | |
| *** britthouser has joined #openstack-ansible | 01:27 | |
| *** sdake_ has quit IRC | 01:40 | |
| *** markvoelker has joined #openstack-ansible | 01:51 | |
| *** markvoelker has quit IRC | 01:56 | |
| *** stevemar has joined #openstack-ansible | 02:29 | |
| *** javeriak_ has quit IRC | 02:46 | |
| *** markvoelker has joined #openstack-ansible | 02:52 | |
| *** markvoelker has quit IRC | 02:56 | |
| *** sdake has joined #openstack-ansible | 03:02 | |
| *** sdake_ has joined #openstack-ansible | 03:06 | |
| *** sdake has quit IRC | 03:09 | |
| *** sdake has joined #openstack-ansible | 03:21 | |
| *** sdake_ has quit IRC | 03:24 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: updated dev requirements to allow for linting python https://review.openstack.org/171035 | 03:37 |
|---|---|---|
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Flake8 update - dynamic_inventory.py https://review.openstack.org/171036 | 03:41 |
| palendae | cloudnull: You don't have to do everything yourself :p | 03:49 |
| Sam-I-Am | heh | 03:50 |
| *** markvoelker has joined #openstack-ansible | 03:53 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Flake8 update - swift_rings.py https://review.openstack.org/171038 | 03:54 |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Flake8 update - dynamic_inventory.py https://review.openstack.org/171036 | 03:54 |
| *** markvoelker has quit IRC | 03:57 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Flake8 update - openstack-wheel-builder.py https://review.openstack.org/171041 | 04:00 |
| *** sdake_ has joined #openstack-ansible | 04:01 | |
| *** sdake has quit IRC | 04:05 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Flake8 update - openstack-branch-grabber.py https://review.openstack.org/171042 | 04:06 |
| cloudnull | hey palendae :) | 04:15 |
| cloudnull | no worries i figured i'd do a few and let others do the rest. | 04:16 |
| cloudnull | hows it going ? | 04:16 |
| cloudnull | palendae if you get a chance, you mind reviewing https://review.openstack.org/#/c/167460/ <- gating is stuck without it. | 04:18 |
| cloudnull | Sam-I-Am ^ that resolves the issue you found with horizon | 04:18 |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: updated dev requirements to allow for linting python https://review.openstack.org/171035 | 04:22 |
| palendae | cloudnull: Did the thing | 04:29 |
| cloudnull | palendae for prez! | 04:30 |
| cloudnull | :) | 04:30 |
| cloudnull | how you doing tonight ? | 04:30 |
| cloudnull | you going to pycon? | 04:30 |
| palendae | cloudnull: Doing ok. Gaming. And yeah, Pycon | 04:32 |
| cloudnull | nice! | 04:32 |
| cloudnull | i've never been to pycon, id imagine its an all around good time. | 04:33 |
| palendae | Oh, yeah | 04:33 |
| palendae | I love it | 04:33 |
| palendae | This'll be my....5th | 04:34 |
| palendae | I think I'm actually gonna go without my laptop | 04:34 |
| cloudnull | go analog! | 04:34 |
| palendae | cloudnull: A request - when you include the command for doing the actual linting, could you annotate what F403 and H303 are for reference? | 04:40 |
| cloudnull | sure can. | 04:41 |
| cloudnull | you mean in the lp issue ? or the os paste ? | 04:41 |
| cloudnull | i've not created a lint task yet. | 04:42 |
| palendae | In the code base | 04:43 |
| *** persia_ has quit IRC | 04:44 | |
| cloudnull | ah , i've not done that yet, ill add the command to the lp issue as we'll likely need to wait for all the flake8 pr's to merge. | 04:45 |
| *** persia_ has joined #openstack-ansible | 04:45 | |
| cloudnull | palendae you thinking something like the last comment i made : https://bugs.launchpad.net/openstack-ansible/+bug/1440462/comments/5 | 04:46 |
| openstack | Launchpad bug 1440462 in openstack-ansible trunk "Enable Python syntax linting on Ansible modules" [Medium,Confirmed] | 04:46 |
| palendae | Yeah. F403 - * imports H303 - ... | 04:46 |
| cloudnull | https://bugs.launchpad.net/openstack-ansible/+bug/1440462/comments/6 | 04:49 |
| openstack | Launchpad bug 1440462 in openstack-ansible trunk "Enable Python syntax linting on Ansible modules" [Medium,Confirmed] | 04:49 |
| palendae | cloudnull: great | 04:50 |
| *** markvoelker has joined #openstack-ansible | 04:53 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Adds Flake8 job to the lint check https://review.openstack.org/171045 | 04:54 |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Adds Flake8 job to the lint check https://review.openstack.org/171045 | 04:55 |
| cloudnull | ^ there it is. | 04:55 |
| cloudnull | and on that note , im going to sleep. | 04:55 |
| *** markvoelker has quit IRC | 04:58 | |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Horizon config changes for kilo https://review.openstack.org/167460 | 05:20 |
| *** sdake has joined #openstack-ansible | 05:43 | |
| *** sdake_ has quit IRC | 05:46 | |
| *** markvoelker has joined #openstack-ansible | 05:54 | |
| *** markvoelker has quit IRC | 05:58 | |
| *** ishant has joined #openstack-ansible | 06:23 | |
| *** markvoelker has joined #openstack-ansible | 06:55 | |
| *** markvoelker has quit IRC | 06:59 | |
| *** jwagner is now known as jwagner_away | 07:18 | |
| *** subscope has joined #openstack-ansible | 07:41 | |
| *** stevemar has quit IRC | 07:54 | |
| *** markvoelker has joined #openstack-ansible | 07:56 | |
| *** markvoelker has quit IRC | 08:00 | |
| odyssey4me | o/ howdy everyone | 08:32 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - swift_rings.py https://review.openstack.org/171038 | 08:48 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - dynamic_inventory.py https://review.openstack.org/171036 | 08:49 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - openstack-wheel-builder.py https://review.openstack.org/171041 | 08:49 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - openstack-branch-grabber.py https://review.openstack.org/171042 | 08:49 |
| *** markvoelker has joined #openstack-ansible | 08:56 | |
| *** markvoelker has quit IRC | 09:01 | |
| *** persia_ is now known as persia | 09:14 | |
| openstackgerrit | git-harry proposed stackforge/os-ansible-deployment: Add network.yml monitoring playbook https://review.openstack.org/170062 | 09:24 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: updated dev requirements to allow for linting python https://review.openstack.org/171035 | 09:25 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Adds Flake8 job to the lint check https://review.openstack.org/171045 | 09:28 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Updating CONTRIBUTING for commit messages https://review.openstack.org/170899 | 09:28 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - openstack-branch-grabber.py https://review.openstack.org/171042 | 09:29 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - openstack-wheel-builder.py https://review.openstack.org/171041 | 09:29 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - dynamic_inventory.py https://review.openstack.org/171036 | 09:29 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - swift_rings.py https://review.openstack.org/171038 | 09:29 |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Flake8 update - inventory-manage.py https://review.openstack.org/171102 | 09:38 |
| odyssey4me | hughsaunders I see http://logs.openstack.org/87/170187/1/check/os-ansible-deployment-dsvm-check-commit/bb67bee/console.html#_2015-04-07_09_38_08_028 is showing a timeout for https://review.openstack.org/170187 | 09:44 |
| odyssey4me | does that timeout help identify the root cause? | 09:44 |
| hughsaunders | odyssey4me: possibly, though it may indicate general network wonkyness | 09:45 |
| odyssey4me | wonkyness, haha - that's a technical term for sure :p | 09:51 |
| *** markvoelker has joined #openstack-ansible | 09:57 | |
| *** markvoelker has quit IRC | 10:02 | |
| odyssey4me | so the network test we're failing on now is the same one that we've always been failing on, even prior to Kilo | 10:03 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Updated log file entries for neutron https://review.openstack.org/170284 | 10:17 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Move SYMLINK_DIR declaration to bootstrap-aio.sh https://review.openstack.org/170303 | 10:20 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Enable offline compression for resource files https://review.openstack.org/170298 | 10:21 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Update to readme to reference doc source more generally https://review.openstack.org/169358 | 10:29 |
| *** markvoelker has joined #openstack-ansible | 10:58 | |
| *** markvoelker has quit IRC | 11:03 | |
| *** sdake_ has joined #openstack-ansible | 11:03 | |
| *** sdake has quit IRC | 11:07 | |
| openstackgerrit | Miguel Grinberg proposed stackforge/os-ansible-deployment: Restart apache2 after local_settings.py is changed https://review.openstack.org/171132 | 11:20 |
| openstackgerrit | Hugh Saunders proposed stackforge/os-ansible-deployment: Update tempest config for current master https://review.openstack.org/171142 | 11:42 |
| *** markvoelker has joined #openstack-ansible | 11:59 | |
| *** markvoelker has quit IRC | 12:03 | |
| *** stevemar has joined #openstack-ansible | 12:39 | |
| *** KLevenstein has joined #openstack-ansible | 12:42 | |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Set containers to use the same apt repo as the host https://review.openstack.org/171175 | 12:54 |
| *** markvoelker has joined #openstack-ansible | 12:59 | |
| *** markvoelker has quit IRC | 13:04 | |
| *** openstackgerrit has quit IRC | 13:07 | |
| *** openstackgerrit has joined #openstack-ansible | 13:07 | |
| *** markvoelker has joined #openstack-ansible | 13:37 | |
| *** cbaesema has quit IRC | 13:51 | |
| *** fawadkhaliq has joined #openstack-ansible | 13:54 | |
| *** fawadk has joined #openstack-ansible | 14:01 | |
| *** fawadkhaliq has quit IRC | 14:04 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:16 | |
| openstackgerrit | git-harry proposed stackforge/os-ansible-deployment: Add HP monitoring playbook https://review.openstack.org/171223 | 14:32 |
| palendae | So many patches | 14:35 |
| *** devlaps has quit IRC | 14:38 | |
| *** Mudpuppy has joined #openstack-ansible | 14:40 | |
| odyssey4me | palendae yeah, all stuck in the queue thanks to the tempest/neutron networking shenanigans | 14:42 |
| palendae | =\ | 14:42 |
| palendae | And yes, same network test failure | 14:42 |
| odyssey4me | have we logged a bug upstream for this yet? it seems that we're fairly convinced that it's an upstream issue that's just never come up because there's no upstream testing for linuxbridge | 14:43 |
| palendae | Talk to Apsu | 14:43 |
| Apsu | odyssey4me: Which upstream? | 14:43 |
| odyssey4me | good question, haha | 14:44 |
| odyssey4me | I would guess that this is a neutron bug? | 14:44 |
| Apsu | That's kind of the thing here. | 14:44 |
| Apsu | I'm not so sure. I added time.sleep(1) to every single interface operation in neutron. Added 100 seconds to gate tests. It still failed. | 14:44 |
| Apsu | So I'm thinking there's really something else going on here. | 14:45 |
| odyssey4me | :( | 14:45 |
| Apsu | I'm going to have to try a much newer version of Ubuntu with a much different kernel. | 14:45 |
| sigmavirus24 | so, I'm trying to test patches on AIOs and they keep stalling trying to install rabbitmq | 14:45 |
| sigmavirus24 | I kicked this off before going for a ride last night and it's still at: "+ openstack-ansible -v --forks 10 rabbitmq-install.yml" | 14:45 |
| odyssey4me | sigmavirus24 that's different to all the failures we've seen today | 14:46 |
| sigmavirus24 | Yeah, I get that. It's just that was the second or third time it happened yesterday | 14:46 |
| sigmavirus24 | Kind of frustrating to be quite honest | 14:46 |
| sigmavirus24 | Anyone else seeing that or is it just me? | 14:46 |
| mattt | sigmavirus24: i have | 14:46 |
| mattt | sigmavirus24: not frequently but i have def. seen it | 14:46 |
| Apsu | sigmavirus24: I'll see your kind of frustrating and raise you NetworkBasicOMFG | 14:47 |
| odyssey4me | every day brings fresh external factor nonsense :/ | 14:47 |
| sigmavirus24 | Apsu: you chose the packet life, not me | 14:47 |
| Apsu | haha | 14:47 |
| sigmavirus24 | (Or did the packet life choose you?) | 14:47 |
| Apsu | No, see.... let this be a lesson to you kids. | 14:47 |
| Apsu | When you fail How Not To Be Seen by admitting you understand networking... this is what happens. | 14:47 |
| sigmavirus24 | "admitting you understand networking" <- That will never be me =P | 14:48 |
| sigmavirus24 | "A packet is made. A packet is sent. A packet dies. And it's all vanity" | 14:48 |
| Apsu | Eventually you end up in a ditch covered in discarded packets with a protocol shovel that can only be used to dig yourself deeper. | 14:48 |
| Apsu | sigmavirus24: @NihilistNetworking needs to be a thing | 14:48 |
| sigmavirus24 | Apsu: that was just an egregious misquote of a Cake song | 14:48 |
| Apsu | sigmavirus24: That's fine. Reminded me of Nihilist Arby's :P | 14:49 |
| sigmavirus24 | https://www.youtube.com/watch?v=MizOftmSc7k | 14:49 |
| odyssey4me | Apsu is there perhaps someone in #openstack-dev or #openstack-neutron who could help to investigate? | 14:49 |
| sigmavirus24 | odyssey4me: they all know to hide when we come calling ;) | 14:50 |
| Apsu | odyssey4me: Maybe. I'm going to troll around there and #netfilter and #kernel. I know a few folks in each | 14:51 |
| Apsu | But the first thing #kernel will tell me is "try the latest stable kernel" | 14:52 |
| Apsu | Also... something very important to consider... | 14:52 |
| odyssey4me | true | 14:52 |
| Apsu | We're doing network namespaces inside of network namespaces (netns in an lxc container with its own netns), and that's the only place this is failing. | 14:52 |
| Apsu | The host's root netns isn't having issues adding VM taps to bridges | 14:53 |
| odyssey4me | ah, so we have have just found an edge case | 14:53 |
| Apsu | So I'm going to disable apparmor, try a newer kernel, try to investigate all the possible namespace/lxc options, etc | 14:53 |
| cloudnull | odyssey4me the issue that i've seen is that neutron is not gating on linuxbridge. at least not running a comprehensive set of tests using linuxbridge. | 14:54 |
| Apsu | I was really hoping it was a timing issue given the kernel's asynchronous interface behavior... | 14:54 |
| cloudnull | from what i've gathered the neutron people are looking to introduce gate tests for linuxbridge soon-ish but they've not done that quite yet. | 14:54 |
| cloudnull | Sam-I-Am ^ ] | 14:54 |
| *** fawadk has quit IRC | 14:54 | |
| andymccr | there are comments on dev list about that! | 14:55 |
| andymccr | that'd be great | 14:55 |
| odyssey4me | cloudnull yeah, I've seen some talk in the dev lists recently around them wanting to pull the tests for neutron into the code tree | 14:55 |
| cloudnull | prometheanfire was working on some of that too. | 14:55 |
| Sam-I-Am | cloudnull: yeah, i'm talking to some other nerds about this | 14:56 |
| Sam-I-Am | the only reason being easier migration from nova-net | 14:57 |
| Sam-I-Am | maybe lb will get more lub | 14:57 |
| odyssey4me | Sam-I-Am yeah, watching the discussions around nova-net to neutron I was thinking the same thing - using linuxbridge is much easier and would appeal more broadly to current nova-net users | 14:58 |
| Apsu | afk a few | 14:58 |
| cloudnull | personally, i'd like to remove the network basis ops test. while it does provide some testing that we might be able to derive some value from it fails roughly 50% of the time which requires a retest. at present there is no faith in that test passing so people dont really investigate the failure, we're just retesting and calling it normal. | 15:00 |
| sigmavirus24 | cloudnull: so we're just like a real openstack project? | 15:00 |
| cloudnull | yup | 15:00 |
| openstackgerrit | David Wilde proposed stackforge/os-ansible-deployment: Nova Kilofication Work https://review.openstack.org/170952 | 15:00 |
| mattt | cloudnull: yeah i think core networking is tested via other scenarios, so i think that may be the way forward | 15:01 |
| cloudnull | so while we're off spinning our wheels trying to figure out kernel problems and neutron race conditions i think that test should be removed. | 15:01 |
| cloudnull | and if we fix it, lets re-introduce it. | 15:01 |
| cloudnull | looking at you Apsu | 15:01 |
| cloudnull | :) | 15:01 |
| mattt | haha | 15:01 |
| cloudnull | but shits broke . | 15:01 |
| odyssey4me | cloudnull I agree, although when we started on this adventure there were many issues which have been resolved because not doing so cripples us | 15:02 |
| odyssey4me | when we remove the blockage using a skip, we also remove the imperitive to fix it | 15:02 |
| mattt | that is true also :-/ | 15:02 |
| andymccr | agree with cloudnull, we did due diligence on that | 15:03 |
| odyssey4me | that said, in this particular scenario we don't have many people who have enough skill depth to work on it - so we have 90% of us wasting time doing rechecks | 15:03 |
| andymccr | as long as we commit/recognize we should fix this longer term | 15:03 |
| andymccr | right now its just blocking us with no fix in sight. | 15:03 |
| andymccr | i don't feel like we just went "well, networking is hard lolz. #disabled" | 15:03 |
| odyssey4me | so yes, I agree that our best option right now is to remove the network ops test | 15:03 |
| andymccr | but - we should still put somebody on working longer term to fix it | 15:03 |
| andymccr | (puts hand down though) | 15:04 |
| andymccr | hopefully, if linuxbridge gets added to nova gate, it'll raise this issue to the community more clearly and that'd force a fix | 15:04 |
| mattt | perhaps that should be part of the blueprint in going from release x -> y | 15:04 |
| odyssey4me | yeah, perhaps Apsu and rackertom can tag team it a bit for a week or two? | 15:04 |
| mattt | we should always be looking for new scenarios that we can introduce to our gating | 15:04 |
| rackertom | How dare you use our names in vain. :) | 15:05 |
| andymccr | rackertom: there is no other way to use your names :( | 15:05 |
| odyssey4me | mattt I do think that the gate testing should be reviewed every major/minor release - for sure | 15:05 |
| rackertom | More true words have never been spoken andymccr | 15:05 |
| cloudnull | we are exercising neutron by simply having the compute scenario tests and maybe adding a few more just for the sake of argument. | 15:08 |
| cloudnull | but that NeutronBasicOpsOFMG test is broken. | 15:08 |
| *** erikmwilson is now known as Guest190 | 15:08 | |
| *** erikmwilson has joined #openstack-ansible | 15:08 | |
| *** erikmwilson_ has joined #openstack-ansible | 15:08 | |
| palendae | Yeah, I'd agree; I want TestNetworkBasicOps fixed for LinuxBridge, but I think a fix may be out of scope for os-a-d | 15:09 |
| andymccr | agree | 15:10 |
| palendae | And by 'I want' i mean 'I'd like it if someone not me/us did it' ;p | 15:12 |
| andymccr | very charitable, i would've just gone with "not me" - but i'm selfish like that | 15:13 |
| *** ishant has quit IRC | 15:13 | |
| sigmavirus24 | palendae: I'd like it if you'd just go ahead and fix that by Friday. Thanks | 15:14 |
| sigmavirus24 | That'd be greeeeeaaaaat | 15:14 |
| palendae | sigmavirus24: Nah | 15:14 |
| odyssey4me | have we got a bug for it in openstack-ansible? | 15:14 |
| palendae | No computer after wednesday | 15:14 |
| palendae | odyssey4me: After a fashion, 1s | 15:14 |
| odyssey4me | I'll happily put in a review to remove it | 15:14 |
| sigmavirus24 | palendae: is that "How not to PyCon"? | 15:15 |
| palendae | https://bugs.launchpad.net/openstack-ansible/+bug/1425255 | 15:15 |
| openstack | Launchpad bug 1425255 in openstack-ansible juno "Tempest NetworkBasicOps tests cannot ping from within instances" [Critical,New] - Assigned to Evan Callicoat (diopter) | 15:15 |
| palendae | sigmavirus24: How to disconnect for a bit | 15:15 |
| odyssey4me | that'll do | 15:15 |
| sigmavirus24 | palendae: at a tech conf | 15:15 |
| palendae | sigmavirus24: Yep | 15:15 |
| sigmavirus24 | ಠ_ಠ| 15:16 |
| palendae | Oh, right. Tuesdays and Thursdays are meetings over the noon hour | 15:16 |
| palendae | sigmavirus24: I can like, talk to people instead | 15:16 |
| sigmavirus24 | I can multitask =P | 15:16 |
| sigmavirus24 | requests needs to set up a bof | 15:17 |
| palendae | I updated 1425255 to be in progress instead of fix committed | 15:17 |
| *** Bjoern__ has joined #openstack-ansible | 15:19 | |
| openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Remove Tempest NetworkBasicOps from commit tests https://review.openstack.org/171247 | 15:20 |
| openstackgerrit | David Wilde proposed stackforge/os-ansible-deployment: Add env.d extra configuration directory to etc/openstack_deploy/ https://review.openstack.org/170558 | 15:21 |
| openstackgerrit | David Wilde proposed stackforge/os-ansible-deployment: Add env.d extra configuration directory to etc/openstack_deploy/ https://review.openstack.org/170558 | 15:23 |
| *** sacharya has joined #openstack-ansible | 15:24 | |
| openstackgerrit | David Wilde proposed stackforge/os-ansible-deployment: Add env.d extra configuration directory to etc/openstack_deploy/ https://review.openstack.org/170558 | 15:30 |
| openstackgerrit | David Wilde proposed stackforge/os-ansible-deployment: Add env.d extra configuration directory to etc/openstack_deploy/ https://review.openstack.org/170558 | 15:31 |
| *** ccrouch has quit IRC | 15:36 | |
| odyssey4me | Apsu if you could add more information into the bug about the network tests that are failing that'd be great - the bug is looking a bit sparse | 15:36 |
| odyssey4me | palendae perhaps you could add some meat based on the work you guys did over the last two days? | 15:37 |
| Sam-I-Am | could probably just copy the e-mail thread bits to the bug | 15:38 |
| *** devlaps has joined #openstack-ansible | 15:40 | |
| *** jwagner_away is now known as jwagner | 15:47 | |
| openstackgerrit | Miguel Alejandro Cantu proposed stackforge/os-ansible-deployment-specs: Add spec for BP implement-ceilometer https://review.openstack.org/169417 | 15:51 |
| d34dh0r53 | triage times | 16:00 |
| rackertom | o/ hello | 16:00 |
| b3rnard0 | yeah, more meetings! | 16:00 |
| rackertom | Today is wall to wall with them. | 16:00 |
| d34dh0r53 | First up: https://bugs.launchpad.net/openstack-ansible/+bug/1441189 | 16:00 |
| openstack | Launchpad bug 1441189 in openstack-ansible "HP hardware not monitored" [Undecided,New] | 16:00 |
| b3rnard0 | Notes here: https://etherpad.openstack.org/p/openstack_ansible_bug_triage.2015-04-07-16.00 | 16:01 |
| d34dh0r53 | self explanatory and there is a trello card for it assigned to myself and git-harry | 16:01 |
| palendae | What's a trello? | 16:01 |
| d34dh0r53 | I don't know, ask the scrum master | 16:02 |
| palendae | I only know of launchpad and gerrit here | 16:02 |
| b3rnard0 | this OS. there are no scum bags here | 16:02 |
| d34dh0r53 | ohh, sorry | 16:02 |
| stevelle | A trello is a large bird, drops things on your car to set off the alarm | 16:02 |
| d34dh0r53 | assigned it to git-harry | 16:03 |
| b3rnard0 | i've heard of a trollo but i don't want to talk about sigmavirus24 if he's not available | 16:03 |
| d34dh0r53 | next up: https://bugs.launchpad.net/openstack-ansible/+bug/1441110 | 16:03 |
| openstack | Launchpad bug 1441110 in openstack-ansible "the horizon playbook does not restart apache2 when local_settings.py is changed" [Undecided,New] - Assigned to Miguel Grinberg (miguelgrinberg) | 16:03 |
| sigmavirus24 | I'm here | 16:03 |
| d34dh0r53 | miguelgrinberg: in progress? | 16:03 |
| b3rnard0 | miguelgrinberg may be at pyconn and unavailable | 16:04 |
| palendae | Yeah, that should be marked as in prgress, there's a review for it | 16:04 |
| d34dh0r53 | ahh, anyone have thoughts on priority? | 16:04 |
| palendae | https://review.openstack.org/#/c/171132/ | 16:04 |
| palendae | Not sure why that didn't link | 16:05 |
| d34dh0r53 | s/priority/importance | 16:05 |
| d34dh0r53 | palendae: not sure either | 16:05 |
| d34dh0r53 | topic needs to have bug/1441110 in it | 16:06 |
| palendae | Ah | 16:06 |
| palendae | Yeah | 16:06 |
| Apsu | odyssey4me: Yes I can. In a meeting right now | 16:06 |
| palendae | So it looks like it's in progress and just waiting on the gate to stop failing | 16:06 |
| stevelle | med for that? | 16:06 |
| d34dh0r53 | sounds good to me | 16:06 |
| d34dh0r53 | next up: https://bugs.launchpad.net/openstack-ansible/+bug/1440784 | 16:07 |
| openstack | Launchpad bug 1440784 in openstack-ansible "Spice console is freezing" [Undecided,New] | 16:07 |
| d34dh0r53 | Bjoern__: thoughts? | 16:08 |
| *** Bjoern__ is now known as BjoernT | 16:08 | |
| BjoernT | I dumped everything I know so far in the ticket | 16:08 |
| d34dh0r53 | BjoernT: how frequently are you seeing this? | 16:08 |
| BjoernT | We see this at one customer every week | 16:08 |
| BjoernT | I have not found the root cause | 16:08 |
| d34dh0r53 | reproducible or transient? | 16:09 |
| BjoernT | it comes back, so transient | 16:10 |
| BjoernT | but we can debug the issue then | 16:10 |
| BjoernT | since it only happens to one container at a time | 16:10 |
| d34dh0r53 | can you make it happen by say continually launching spice consoles or by using a certain console for more than X time? | 16:10 |
| BjoernT | I was not able to reproduce it yet. All I know it comes back in this environment every week or every other week on different containers | 16:11 |
| d34dh0r53 | hmm, ok confirmed medium for now then | 16:12 |
| d34dh0r53 | next up: https://bugs.launchpad.net/openstack-ansible/+bug/1440418 | 16:13 |
| openstack | Launchpad bug 1440418 in openstack-ansible "ec2 Keystone Auth Endpoint Not Configured in nova.conf" [Undecided,New] | 16:13 |
| BjoernT | there is a 2nd bug to ec2 in regards to the endpoint port 3333 | 16:14 |
| BjoernT | so we have customer to getting this fixed and working for him | 16:14 |
| BjoernT | + waiting | 16:14 |
| *** fawadkhaliq has joined #openstack-ansible | 16:14 | |
| d34dh0r53 | I agree with cloudnull's comments on both of those bugs, I think this should be handled as a feature request with a blueprint. | 16:18 |
| d34dh0r53 | open to disagreement though | 16:18 |
| BjoernT | either way we need to move here | 16:18 |
| d34dh0r53 | ok, can someone from your end please register a blueprint for this? and we'll get it prioritized appropriately | 16:20 |
| stevelle | BjoernT: is your fire for this going to imply the return of S3 compat as well? | 16:20 |
| *** Mudpuppy has quit IRC | 16:20 | |
| BjoernT | I don't know yet what's working or what not. the euca tools aren | 16:21 |
| *** Mudpuppy has joined #openstack-ansible | 16:21 | |
| BjoernT | 't and this bug 1440418 is one issue to solve | 16:21 |
| openstack | bug 1440418 in openstack-ansible "ec2 Keystone Auth Endpoint Not Configured in nova.conf" [Undecided,New] https://launchpad.net/bugs/1440418 | 16:21 |
| BjoernT | but I don't now what's next especially with the ec3 api endpoint port 3333 | 16:22 |
| BjoernT | i meant ec2 | 16:23 |
| BjoernT | ec3 is not out yet, lol | 16:23 |
| d34dh0r53 | elastic compute cloud on a cloud? | 16:24 |
| stevelle | elastic compute cloud compatibility, obv. | 16:24 |
| d34dh0r53 | lol much better | 16:24 |
| BjoernT | So Sean tested 1440418 and the keystone_ec2_url need to be test to go one step ahead but the api endpoint 3333 is still not existing, at least it has no running backends | 16:26 |
| BjoernT | ... ? any comment ? I can help you guys working on a blue print but in the end I ned it for juno | 16:29 |
| d34dh0r53 | Ok, reading more into this we are really going to need to handle this via a blueprint, as the implementations are going to very quite a bit between Juno and Kilo (there will be no more nova_api_ec2_container in kilo). It looks like the S3 package is in an unstable state, etc... We should probably set up a meeting to talk about this one as I feel that it's going to be quite a bit of work. | 16:31 |
| BjoernT | That's ok | 16:31 |
| d34dh0r53 | cool | 16:32 |
| d34dh0r53 | Next up: https://bugs.launchpad.net/openstack-ansible/+bug/1439892 | 16:32 |
| openstack | Launchpad bug 1439892 in openstack-ansible "Bindmount for Glance" [Undecided,New] | 16:32 |
| andymccr | im not a massive fan of bind mounting - it forces a 1:1 tie between host and container | 16:33 |
| d34dh0r53 | the bug to me seems to be that glance is ignoring image_cache_max_size | 16:34 |
| *** fawadkhaliq has quit IRC | 16:34 | |
| andymccr | sigmavirus24 can you weigh in ? :D | 16:34 |
| sigmavirus24 | It shouldn't ignore it | 16:34 |
| * sigmavirus24 weighed in | 16:34 | |
| andymccr | haha | 16:34 |
| d34dh0r53 | lol | 16:34 |
| andymccr | care to fix it then? | 16:35 |
| sigmavirus24 | oh | 16:35 |
| sigmavirus24 | I think I understand the problem | 16:35 |
| andymccr | hahahaha | 16:35 |
| andymccr | oh now he understands | 16:35 |
| andymccr | typical! | 16:35 |
| sigmavirus24 | I think that config setting is per image size | 16:35 |
| sigmavirus24 | Not total cache size | 16:35 |
| d34dh0r53 | ahh | 16:35 |
| d34dh0r53 | is there a total_cache_size we can configure? | 16:35 |
| * sigmavirus24 thinks he remembers someone else having this discussion in #openstack-glance before | 16:35 | |
| sigmavirus24 | d34dh0r53: I don't think so | 16:35 |
| stevelle | d34dh0r53: checking to be sure | 16:35 |
| sigmavirus24 | We can do a cron-job to manage the cache though | 16:36 |
| stevelle | image_cache_max_size is doc'd as total cache size | 16:36 |
| sigmavirus24 | Hm | 16:36 |
| BjoernT | I actually have an issue with a related bug | 16:37 |
| BjoernT | I heard we up sized the glance container to 10g ? | 16:37 |
| sigmavirus24 | d34dh0r53: andymccr assign me to that bug and I'll look into it | 16:38 |
| sigmavirus24 | Is this on Icehouse/Juno/Kilo? | 16:39 |
| stevelle | https://github.com/openstack/glance/blob/803c54051bdbfc1e4d695a2838906377e94611b6/doc/source/cache.rst | 16:39 |
| d34dh0r53 | sigmavirus24: will do | 16:39 |
| BjoernT | the problem is for juno | 16:39 |
| d34dh0r53 | thoughts on importance? | 16:39 |
| *** devlaps has quit IRC | 16:39 | |
| stevelle | it's the total cache size, but the cache is arguably bugged by design | 16:39 |
| andymccr | haha "bugged by design" i need to use that more often | 16:39 |
| BjoernT | we got now containers getting full silently | 16:39 |
| BjoernT | so important | 16:39 |
| BjoernT | can we turn off the cache ? | 16:40 |
| d34dh0r53 | tagged it as high and assigned to sigmavirus24 you're welcome | 16:40 |
| sigmavirus24 | stevelle: your problem is thinking the documentation is correct when it's maintained inside glance | 16:40 |
| stevelle | BjoernT: yes, cache can be disabled in principle | 16:40 |
| sigmavirus24 | And a different core just confirmed my expectation that the max-size is a per-image max | 16:40 |
| BjoernT | So i would vote for that ant not even bother with the cron job | 16:41 |
| stevelle | sigmavirus24: staring at the source too | 16:41 |
| sigmavirus24 | I would vote that the community should be able to configure it (or have a vote before we do anything so drastically harmful to performance) | 16:42 |
| d34dh0r53 | ok, going to move on to finish the triage but feel free to keep the glance convo going | 16:43 |
| d34dh0r53 | next: https://bugs.launchpad.net/openstack-ansible/+bug/1439755 | 16:43 |
| openstack | Launchpad bug 1439755 in openstack-ansible "defcore: test_list_servers_detailed_filter_by_image is being skipped" [Undecided,New] | 16:43 |
| stevelle | https://github.com/openstack/glance/blob/803c54051bdbfc1e4d695a2838906377e94611b6/doc/source/cache.rst sigmavirus24 | 16:44 |
| stevelle | then line 193 does per image | 16:44 |
| sigmavirus24 | stevelle: you linked to the docs again | 16:45 |
| stevelle | grr | 16:45 |
| stevelle | https://github.com/openstack/glance/blob/bb59c33ffcc6e1cde23c93bb25d38846e84c2cb9/glance/image_cache/__init__.py#L192-L193 | 16:46 |
| d34dh0r53 | any issues with confirmed/medium on 1439755 | 16:46 |
| palendae | d34dh0r53: Nope | 16:47 |
| stevelle | none here | 16:47 |
| d34dh0r53 | ty | 16:47 |
| d34dh0r53 | next: https://bugs.launchpad.net/openstack-ansible/+bug/1437045 | 16:47 |
| openstack | Launchpad bug 1437045 in openstack-ansible "Extend repo vars documentation" [Undecided,New] - Assigned to Nolan Brubaker (nolan-brubaker) | 16:47 |
| d34dh0r53 | palendae: status/importance preference? | 16:48 |
| palendae | d34dh0r53: Just changed it to Triaged/Low | 16:48 |
| d34dh0r53 | palendae: tys | 16:48 |
| d34dh0r53 | https://bugs.launchpad.net/openstack-ansible/+bug/1436999 | 16:49 |
| openstack | Launchpad bug 1436999 in openstack-ansible "LVM/DM/UDEV out of sync inside cinder-volumes container" [Undecided,New] | 16:49 |
| d34dh0r53 | BjoernT: thoughts? | 16:50 |
| BjoernT | As last time, I requested some dev support since I was not able to reproduce it yet | 16:51 |
| BjoernT | I believe apsu volunteered | 16:51 |
| d34dh0r53 | ok | 16:51 |
| Apsu | BjoernT: To my detriment I keep failing at not being seen. | 16:52 |
| d34dh0r53 | :) | 16:53 |
| b3rnard0 | BjoernT: much better than in the other bug triage :-) | 16:53 |
| BjoernT | lol, you can't hide. I can see dead bugs | 16:53 |
| * Apsu whispers "I see dead bugs" o.o | 16:54 | |
| d34dh0r53 | They're everywhere | 16:54 |
| BjoernT | apsu: you got the analogy. So next bug i guess | 16:55 |
| Apsu | </GotThatReference> | 16:56 |
| andymccr | would that mean that Apsu actually IS a bug? | 16:56 |
| Apsu | andymccr: I'm a glitch in the matrix. It means they changed something. | 16:57 |
| d34dh0r53 | and: https://bugs.launchpad.net/openstack-ansible/+bug/1436761 | 16:58 |
| openstack | Launchpad bug 1436761 in openstack-ansible "No rabbitmq_package_sha256 in icehouse/juno branches" [Undecided,New] | 16:58 |
| *** fawadkhaliq has joined #openstack-ansible | 16:58 | |
| *** fawadk has joined #openstack-ansible | 16:59 | |
| palendae | I'd say Medium on importance | 17:00 |
| d34dh0r53 | yep, mattt you want this one? | 17:00 |
| andymccr | mattt's afk | 17:01 |
| d34dh0r53 | ok | 17:01 |
| d34dh0r53 | well, that does it for the bug triage meeting. Thanks all for the participation. You may now resume your regularly scheduled shenanigans | 17:01 |
| *** jwagner is now known as jwagner_away | 17:01 | |
| b3rnard0 | thanks to all the bugs and bug hunters | 17:02 |
| stevelle | shenanigans | 17:03 |
| *** fawadkhaliq has quit IRC | 17:03 | |
| *** Mudpuppy_ has joined #openstack-ansible | 17:04 | |
| *** Mudpuppy has quit IRC | 17:07 | |
| *** sdake_ has quit IRC | 17:08 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 17:29 | |
| BjoernT | do we have a working kilo installation ? in what state is the project? | 17:39 |
| miguelgrinberg | d34dh0r53 cloudnull: sorry missed the bug review meeting. Just wanted to say that https://bugs.launchpad.net/openstack-ansible/+bug/1441110 is what fixes the issues with the new launch instance dialog in horizon, in case it wasn't clear, so it is important to include. | 17:52 |
| openstack | Launchpad bug 1441110 in openstack-ansible "the horizon playbook does not restart apache2 when local_settings.py is changed" [Medium,In progress] - Assigned to Miguel Grinberg (miguelgrinberg) | 17:52 |
| stevelle | BjoernT: not really. We are working through some gate instability right now. | 17:57 |
| *** fawadk has quit IRC | 17:58 | |
| *** fawadkhaliq has joined #openstack-ansible | 18:00 | |
| *** jwagner_away is now known as jwagner | 18:07 | |
| *** sdake has joined #openstack-ansible | 18:10 | |
| *** sdake_ has joined #openstack-ansible | 18:11 | |
| *** sdake has quit IRC | 18:15 | |
| openstackgerrit | Matthew Kassawara proposed stackforge/os-ansible-deployment: Update keystone middleware in glance for Kilo https://review.openstack.org/170695 | 18:17 |
| openstackgerrit | Matthew Kassawara proposed stackforge/os-ansible-deployment: Update Glance service to Kilo https://review.openstack.org/170211 | 18:17 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 18:17 | |
| prometheanfire | master is failing now, right? | 18:30 |
| * prometheanfire wants to test kilo with mtuness | 18:30 | |
| *** fawadkhaliq has quit IRC | 18:33 | |
| Sam-I-Am | prometheanfire: mtuness? is that like a new version of itunes? | 18:40 |
| prometheanfire | mtu-ness | 18:41 |
| palendae | We're pinning to master of all the upstream projects right now, so I think that brings with it some instability | 18:41 |
| prometheanfire | ya | 18:42 |
| palendae | Even though feature freeze | 18:42 |
| prometheanfire | another question, I'm guessing we test on 14.04? | 18:42 |
| palendae | Yeah, I don't think we're testing on non-LTS | 18:43 |
| prometheanfire | cool | 18:43 |
| Sam-I-Am | stevelle: looking at the gerrit bot in here, did i do something wrong with my last commit? | 18:44 |
| stevelle | I believe you could have just hit the rebase button. Distracted with other things this moment. | 18:45 |
| Sam-I-Am | stevelle: ok, i didnt see your patch approved | 18:47 |
| Sam-I-Am | so for now its just a dependency | 18:47 |
| Sam-I-Am | or it was, but gating :/ | 18:47 |
| stevelle | just thinking it might be less effort to press butan. also it makes it clear in gerrit that it's a rebase | 18:53 |
| stevelle | haven't looked at it since you did it to see what gerrit is doing w. it now | 18:54 |
| palendae | Hey Apsu: someone's trying to make LinuxBridge work on devstack and sees interfaces not showing up in the bridge...https://review.openstack.org/#/c/168423/ | 18:55 |
| prometheanfire | Sam-I-Am: bootstrap-aio right? | 18:56 |
| Sam-I-Am | prometheanfire: there's a bit more than that | 18:57 |
| Sam-I-Am | prometheanfire: what are you trying to do? | 18:57 |
| prometheanfire | Sam-I-Am: build an aio host to fiddle with neutron mtu stuff | 18:58 |
| Sam-I-Am | ah. so... something to think about... we use containers that have interfaces, and veth pairs, and things. | 18:58 |
| Sam-I-Am | not sure if it will take more work to initially test with osad or plain source | 18:59 |
| prometheanfire | dunno | 18:59 |
| Sam-I-Am | prometheanfire: mtus on the container interfaces are 1500 | 19:00 |
| prometheanfire | I can change that on the host side if I want | 19:00 |
| prometheanfire | ya | 19:00 |
| prometheanfire | saw that | 19:00 |
| Sam-I-Am | so... we might be able to set that somewhere | 19:00 |
| prometheanfire | 21: veth4-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 | 19:00 |
| Sam-I-Am | i wonder of any of those are configurable yet | 19:01 |
| Sam-I-Am | its probably in the lxc library | 19:01 |
| *** fawadkhaliq has joined #openstack-ansible | 19:01 | |
| prometheanfire | anyway, building it first just to get it running | 19:02 |
| prometheanfire | gate-check-commit.sh is what I needed I think | 19:02 |
| Sam-I-Am | prometheanfire: yeah, you can pick apart that script too | 19:02 |
| Sam-I-Am | especially if you dont want tempest, or if you want to rebuild bits that dont work (like yaprt?) | 19:03 |
| prometheanfire | well, I'll be able to verify the yaprt bug :P | 19:03 |
| Sam-I-Am | stevelle hit it | 19:03 |
| Sam-I-Am | so i'm not totally crazy | 19:04 |
| prometheanfire | twice then | 19:04 |
| Sam-I-Am | so... i ran the pip install command and it worked | 19:04 |
| prometheanfire | well, something fucked up :P | 19:04 |
| prometheanfire | blame kevin | 19:04 |
| Sam-I-Am | i will when i see him again :) | 19:04 |
| prometheanfire | ya, I have it in the keystone package this time | 19:15 |
| prometheanfire | "git_url": "https://github.com/openstack/keystone", | 19:15 |
| prometheanfire | I wonder if github is ratelimiting or something | 19:16 |
| *** cbaesema has joined #openstack-ansible | 19:16 | |
| Sam-I-Am | do you also see it in glance_store? | 19:16 |
| prometheanfire | no | 19:17 |
| Sam-I-Am | interesting | 19:18 |
| Sam-I-Am | i see it in >1 place | 19:18 |
| prometheanfire | I don't have a pip_install_url defined where it broke for me | 19:18 |
| prometheanfire | so it's not likely that | 19:18 |
| prometheanfire | I have it in more then one place too though | 19:19 |
| prometheanfire | quite a few places | 19:19 |
| Sam-I-Am | i cant even build the repo container now :/ | 19:20 |
| prometheanfire | keystone, python-glanceclient, python-ironicclient, python-tuskarclient, tempest, tempest-lib | 19:20 |
| prometheanfire | https://developer.github.com/v3/troubleshooting/#why-am-i-getting-a-404-error-on-a-repository-that-exists | 19:22 |
| *** openstackgerrit has quit IRC | 19:22 | |
| *** openstackgerrit has joined #openstack-ansible | 19:23 | |
| Sam-I-Am | prometheanfire: i just got it on another gate job | 19:23 |
| prometheanfire | I asked in #github if mass cloinging could be 404'd on their side as dos prevention | 19:24 |
| Apsu | prometheanfire: http://specs.openstack.org/openstack/neutron-specs/specs/kilo/mtu-selection-and-advertisement.html MTUs are a significantly complex issue, and pretty fundamentally broken right now. | 19:33 |
| Apsu | Read, enjoy, weep, and move on | 19:33 |
| Apsu | Nothing to see here. | 19:34 |
| prometheanfire | Apsu: I've read it | 19:34 |
| Sam-I-Am | prometheanfire: i'm talking to cloudnull about the github thing | 19:34 |
| prometheanfire | https://developer.github.com/v3/#rate-limiting | 19:36 |
| prometheanfire | Sam-I-Am: if you forward your ssh agent, then you may be authing for each repo clone | 19:37 |
| prometheanfire | verifying now | 19:38 |
| sigmavirus24 | prometheanfire: that ratelimiting doc only applies to their API | 19:49 |
| prometheanfire | right | 19:50 |
| prometheanfire | I'm trying to find out if they do so with clones (authed) | 19:51 |
| *** britthou_ has joined #openstack-ansible | 19:56 | |
| *** britthouser has quit IRC | 19:58 | |
| prometheanfire | is the repo server suposed to build twice RETRY=1 | 19:59 |
| prometheanfire | 2015-04-07 19:59:30,160 - utils:ERROR => Failed to get requirement file contents for [ https://raw.githubusercontent.com/openstack/neutron-fwaas/master/requirements.txt ] | 20:01 |
| Sam-I-Am | prometheanfire: yeah | 20:02 |
| Sam-I-Am | prometheanfire: and try re-running the repo-build play... cloudnull fixed the problem (we think) | 20:02 |
| *** jmeridth has quit IRC | 20:03 | |
| Sam-I-Am | bbiab, busing | 20:03 |
| cloudnull | prometheanfire i just pushed an update to yaprt which will now retry when the head request succeeds but the get fails due to 404 | 20:03 |
| cloudnull | rerun the repo-server.yml and repo-build.yml plays and try again | 20:03 |
| prometheanfire | ok, I'll do a pull and retry | 20:03 |
| stevelle | cloudnull: so that suggests there was a rate limiting or something going on | 20:04 |
| prometheanfire | I did git a 404 on a basic curl | 20:04 |
| cloudnull | github is intermittently throwing 404's | 20:04 |
| stevelle | Downloading yaprt-0.1.2.tar.gz | 20:09 |
| cloudnull | stevelle its up on pypi as of now. | 20:09 |
| cloudnull | https://pypi.python.org/pypi/yaprt/0.1.2 | 20:10 |
| stevelle | confimed. that msg was from pip | 20:10 |
| palendae | we're using pre-release!? | 20:10 |
| cloudnull | ah. | 20:10 |
| palendae | ;) | 20:10 |
| cloudnull | palendae: what? | 20:10 |
| palendae | yaprt isn't 1.0! | 20:10 |
| cloudnull | ah. yes. | 20:10 |
| cloudnull | because i dont always believe in the linus school of thinking. | 20:11 |
| prometheanfire | he doesn't quite use symantic versioning I think | 20:12 |
| cloudnull | im thinking that i skip 1 and go straight to 3. | 20:12 |
| palendae | Skip numerals | 20:12 |
| palendae | use words | 20:12 |
| prometheanfire | palendae: some distros handle that just fine | 20:12 |
| palendae | prometheanfire: Some :) | 20:12 |
| prometheanfire | :P | 20:12 |
| palendae | I don't think pip does | 20:12 |
| palendae | num_to_word(1) | 20:13 |
| palendae | num_to_word(32481) | 20:13 |
| prometheanfire | pip isn't a distro | 20:13 |
| palendae | But it is a package manager | 20:13 |
| prometheanfire | sadly | 20:13 |
| stevelle | https://review.openstack.org/#/c/171247/ is another remaining known thorn in gating | 20:13 |
| * prometheanfire hates out of tree PM | 20:13 | |
| stevelle | I can trigger rechecks but this will snare some of them | 20:14 |
| *** alextricity has quit IRC | 20:22 | |
| *** jmeridth has joined #openstack-ansible | 20:23 | |
| *** Mudpuppy_ is now known as Mudpuppy | 20:24 | |
| *** jwagner is now known as jwagner_away | 20:25 | |
| *** jmeridth has left #openstack-ansible | 20:27 | |
| Sam-I-Am | prometheanfire: making any progress? | 20:28 |
| prometheanfire | + openstack-ansible -v --forks 10 repo-build.yml | 20:28 |
| prometheanfire | waiting | 20:28 |
| prometheanfire | so I guess it's good? | 20:28 |
| Sam-I-Am | heh | 20:29 |
| prometheanfire | passed it to galera now, so good? | 20:29 |
| Sam-I-Am | probably | 20:29 |
| Sam-I-Am | i'm re-kicking my aio box | 20:29 |
| prometheanfire | I just run the cleanup between runs, worksforme so far | 20:30 |
| Sam-I-Am | it was all garblypoops after todays events | 20:30 |
| *** fawadkhaliq has quit IRC | 20:32 | |
| prometheanfire | it's doing the keystone piece, so, good | 20:36 |
| Sam-I-Am | this is progress | 20:36 |
| cloudnull | yay for all the things ! | 20:39 |
| Sam-I-Am | cloudnull: i'm a bit confused as to why rechecking my glance patch aborted the build on steve's glance patch | 20:40 |
| prometheanfire | the deployments deploy with ml2/LB/vxlan right? | 20:41 |
| Sam-I-Am | prometheanfire: it supports flat/vlan/vxlan | 20:42 |
| prometheanfire | ah | 20:42 |
| Sam-I-Am | tempest tests with flat/vxlan iirc | 20:42 |
| prometheanfire | so this won't even build a modifyable test for me | 20:42 |
| cloudnull | Sam-I-Am cosmic rays | 20:45 |
| Sam-I-Am | cloudnull: i thought we turned the ray gun off | 20:46 |
| Sam-I-Am | prometheanfire: i got to galera yay | 20:54 |
| prometheanfire | - [ Running OpenStack Smoke Tests ] ---------------------------------- | 20:55 |
| prometheanfire | get with the times man | 20:55 |
| Sam-I-Am | this is a no-smoking environment sir | 20:55 |
| *** sdake_ has quit IRC | 20:57 | |
| Sam-I-Am | crap, i forgot to tmux this | 20:57 |
| prometheanfire | ha | 20:57 |
| prometheanfire | disown | 20:57 |
| prometheanfire | - [ Status: Success ] ------------------------------------------------ | 20:59 |
| Sam-I-Am | prometheanfire: fancy | 20:59 |
| Sam-I-Am | so, if you're going to muck with the mtu thing, i guess i'd see first whether the neutron bits actually work with lb... and only then dive into the lxc module for mtu on the container bits | 21:00 |
| prometheanfire | ya | 21:00 |
| prometheanfire | core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin | 21:01 |
| prometheanfire | well, there's that | 21:01 |
| Sam-I-Am | what about it? vs. "ml2" ? | 21:01 |
| prometheanfire | tenant_network_types = vxlan,flat,vlan | 21:01 |
| prometheanfire | mechanism_drivers = linuxbridge,l2population | 21:01 |
| prometheanfire | so looks like it | 21:01 |
| Sam-I-Am | transferring self home, brb | 21:02 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Remove Tempest NetworkBasicOps from commit tests https://review.openstack.org/171247 | 21:08 |
| palendae | hah, didn't need to do rechecks on that one | 21:10 |
| sigmavirus24 | palendae: weird, no? | 21:11 |
| stevelle | I'll continue to poke things to try and get the red out. Doing a few at a time to ensure we don't set off a bunch of new transients | 21:13 |
| sigmavirus24 | stevelle: to be fair | 21:16 |
| sigmavirus24 | I think the keystone stuff is failing legitimately | 21:16 |
| stevelle | yup. I haven't rechecked that one for a reason :) | 21:16 |
| stevelle | focusing on the things that are +W or at least +2 | 21:17 |
| sigmavirus24 | +1 | 21:18 |
| stevelle | cloudnull: d34dh0r53 palendae etc, thinking that maybe the outstanding juno branch changes should be -W until after the cut on Friday so we don't slip broken stuff out. | 21:19 |
| stevelle | thinking that neither of those are targeted for 10.1.3 | 21:20 |
| prometheanfire | Sam-I-Am: a manual editing of files shows this | 21:20 |
| prometheanfire | Sam-I-Am: 2: qg-5baca3ac-1c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 8000 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 | 21:20 |
| prometheanfire | within the router namespace | 21:20 |
| palendae | stevelle: Cool with me | 21:23 |
| stevelle | https://review.openstack.org/#/c/171041/ may need a nudge from a core to get it to Verify | 21:24 |
| stevelle | looks like it got stuck | 21:25 |
| openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Update Keystone config and policy for Kilo https://review.openstack.org/167475 | 21:28 |
| d34dh0r53 | stevelle: which juno branch changes are you referring to? | 21:30 |
| stevelle | d34dh0r53: https://review.openstack.org/170062 and https://review.openstack.org/171223 | 21:30 |
| stevelle | as I understand they are not going to make 10.1.3 | 21:30 |
| *** sacharya has quit IRC | 21:31 | |
| prometheanfire | if someone can tell me how to boot a test instance within this aio setup I could test the mtu things | 21:31 |
| prometheanfire | everything seems right on the neutron side | 21:31 |
| d34dh0r53 | stevelle: good catches, I've -W'd them both | 21:32 |
| stevelle | prometheanfire: https://github.com/stackforge/os-ansible-deployment/blob/master/scripts/run-aio-build.sh | 21:33 |
| palendae | Heh | 21:33 |
| Sam-I-Am | prometheanfire: yo | 21:33 |
| palendae | run-aio-build calls a scrip that calls bootstrap-aio | 21:33 |
| stevelle | set ENV as needed: a REPO_URL to your fork if any, REPO_BRANCH if any, | 21:34 |
| palendae | Should wrap it in do-aio.sh | 21:34 |
| stevelle | possibly RUN_TEMPEST=false from https://github.com/stackforge/os-ansible-deployment/blob/master/scripts/gate-check-commit.sh | 21:34 |
| Sam-I-Am | prometheanfire: what do you need? | 21:34 |
| prometheanfire | Sam-I-Am: that'll likey overwrite my configs | 21:34 |
| prometheanfire | I don't see a container running compute | 21:34 |
| Sam-I-Am | its on the host | 21:34 |
| Sam-I-Am | we dont containerize compute due to Issues | 21:35 |
| prometheanfire | oh ok | 21:35 |
| Apsu | $issues | 21:35 |
| Sam-I-Am | prometheanfire: so you want to build some networks and launch an instance? | 21:35 |
| Apsu | =(cinder) | 21:35 |
| prometheanfire | ya | 21:35 |
| Sam-I-Am | prometheanfire: do you have an image in there yet? | 21:35 |
| prometheanfire | gimme your ssh key and I can let you at this box | 21:35 |
| prometheanfire | nope | 21:35 |
| Sam-I-Am | prometheanfire: actually, can you tmux me in? | 21:35 |
| Sam-I-Am | yeah... | 21:35 |
| Sam-I-Am | one sec | 21:35 |
| prometheanfire | I haven't used tmux, ever :P | 21:35 |
| Sam-I-Am | pfft, gentoo people | 21:36 |
| * palendae gasps | 21:36 | |
| Sam-I-Am | how about screen? :) | 21:36 |
| prometheanfire | yep, screen | 21:36 |
| Sam-I-Am | prometheanfire: emailed you | 21:37 |
| *** openstackgerrit has quit IRC | 21:37 | |
| *** openstackgerrit has joined #openstack-ansible | 21:37 | |
| prometheanfire | Sam-I-Am: I still need to edit config on the host | 21:38 |
| prometheanfire | containers are done I think | 21:38 |
| Sam-I-Am | prometheanfire: are you editing live configs? | 21:39 |
| prometheanfire | yes | 21:40 |
| prometheanfire | just to see if it works | 21:40 |
| prometheanfire | I restart the service manually | 21:40 |
| prometheanfire | nice and hacky | 21:40 |
| Sam-I-Am | heh | 21:40 |
| Sam-I-Am | can you tmux attach? | 21:41 |
| Sam-I-Am | i started a session | 21:41 |
| Sam-I-Am | 'tmux a' perhaps? | 21:41 |
| prometheanfire | http://i.imgur.com/ZlQ2ADm.png | 21:42 |
| prometheanfire | tiny tiny screen | 21:42 |
| Sam-I-Am | ok | 21:42 |
| prometheanfire | thanks :P | 21:42 |
| Sam-I-Am | prometheanfire: did you put that in the agents container too? | 21:43 |
| Sam-I-Am | and in the neutron-server container | 21:43 |
| Sam-I-Am | btw, it seemed like the neutron codez would write the dhcp option for you | 21:44 |
| prometheanfire | yep | 21:44 |
| prometheanfire | coppied | 21:44 |
| prometheanfire | it probably done | 21:44 |
| prometheanfire | it probably does | 21:44 |
| Sam-I-Am | mmkay | 21:46 |
| Sam-I-Am | gonna make some networkzes | 21:46 |
| prometheanfire | the dhcp interface got created with the correct mtu (8k in this case) | 21:49 |
| Sam-I-Am | prometheanfire: dhcp interface as in... the tap? | 21:50 |
| Sam-I-Am | oh, i see | 21:50 |
| Sam-I-Am | all the crap got 8000 :) | 21:50 |
| prometheanfire | which is fine, it's pulling it from the config | 21:50 |
| Sam-I-Am | i was just testing basic connectivity with those ping | 21:50 |
| prometheanfire | that's one of the options I set | 21:50 |
| prometheanfire | ok | 21:51 |
| Sam-I-Am | trying to make sure openstack works | 21:51 |
| prometheanfire | don't let me interupt :P | 21:51 |
| Sam-I-Am | hmm interesting, did you set the mtus on those interfaces? | 21:51 |
| prometheanfire | sec | 21:51 |
| prometheanfire | only the eth* ones | 21:52 |
| Sam-I-Am | ok, manually i take it? | 21:52 |
| prometheanfire | because that's host config that neutron doesn't touch | 21:52 |
| prometheanfire | ya, manually | 21:52 |
| Sam-I-Am | and the veth pairs? | 21:52 |
| prometheanfire | set those too | 21:52 |
| prometheanfire | because host config | 21:52 |
| Sam-I-Am | oo fancy | 21:52 |
| prometheanfire | for link in $(ls /sys/class/net | grep '^veth"; do ip link set mtu 9300 dev ${link}; done | 21:53 |
| Sam-I-Am | coo | 21:54 |
| Sam-I-Am | tempest left a few turds, cleaning up a bit | 21:55 |
| Sam-I-Am | lets boot a vm | 21:56 |
| Sam-I-Am | prometheanfire: say what? | 21:57 |
| *** stevemar has quit IRC | 21:58 | |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Heat config changes for kilo https://review.openstack.org/167947 | 22:03 |
| *** KLevenstein has quit IRC | 22:03 | |
| *** erikmwilson has quit IRC | 22:04 | |
| Sam-I-Am | prometheanfire: that would be my next thing | 22:04 |
| Sam-I-Am | prometheanfire: whats happening on the compute side | 22:05 |
| Sam-I-Am | prometheanfire: might be easier on irc | 22:07 |
| prometheanfire | maybe | 22:07 |
| Sam-I-Am | so you edited the ml2 config files everywhere, and changed all the host interfaces? | 22:08 |
| *** erikmwilson has joined #openstack-ansible | 22:08 | |
| Sam-I-Am | but didnt restart the agent except in the containers? | 22:08 |
| Sam-I-Am | and neutron-server ? | 22:08 |
| prometheanfire | tailing ml2 log | 22:08 |
| prometheanfire | right | 22:08 |
| prometheanfire | # 2015-04-07 22:08:37.063 9896 INFO neutron.plugins.linuxbridge.agent.linuxbridge_neutron_agent [req-c8dd9305-3bf7-4272-b48e-bf1cd06efe93 ] Port tapabda39fb-ce updated. Details: {u'profile': {}, u'admin_state_up': True, u'network_id': u'2fb6d721-5edd-4d70-88e5-fe0abef0dd83', u'segmentation_id': 1, u'device_owner': u'compute:None', u'physical_network': None, u'mac_address': u'fa:16:3e:1f:11:8a', u'device': u'tapabda39fb- | 22:09 |
| prometheanfire | no mtu info | 22:09 |
| Sam-I-Am | prometheanfire: do you see the actual command its running? | 22:09 |
| Sam-I-Am | might need to throw into debug mode | 22:10 |
| prometheanfire | ya, debug needed for that | 22:10 |
| Sam-I-Am | lets do that on the host | 22:10 |
| prometheanfire | that was host | 22:11 |
| prometheanfire | I deleted the instance | 22:12 |
| prometheanfire | Sam-I-Am: the only mtu in that output | 22:13 |
| prometheanfire | Stdout: 495: tapb1ec7028-3c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master brq2fb6d721-5e state UNKNOWN mode DEFAULT group default qlen 500\ link/ether fe:16:3e:9f:16:61 brd ff:ff:ff:ff:ff:ff | 22:13 |
| prometheanfire | Stdout: 494: brq2fb6d721-5e: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default \ link/ether 56:48:fa:d3:35:53 brd ff:ff:ff:ff:ff:ff | 22:13 |
| *** BjoernT has quit IRC | 22:14 | |
| Sam-I-Am | prometheanfire: yeah :/ | 22:14 |
| Sam-I-Am | there are some settings in nova, iirc | 22:15 |
| prometheanfire | ya | 22:15 |
| Sam-I-Am | hmm, were there any commands added to neutron to set network mtu? | 22:16 |
| Sam-I-Am | from this patch | 22:16 |
| prometheanfire | don't think so | 22:17 |
| Sam-I-Am | you set advertise_mtu and path_mtu right? | 22:17 |
| prometheanfire | https://blueprints.launchpad.net/neutron/+spec/mtu-selection-and-advertisement | 22:17 |
| prometheanfire | I did | 22:17 |
| Sam-I-Am | and physnet mtus, which doesnt apply to vxlan iirc | 22:18 |
| prometheanfire | the source of this problem is the tap of the guest atm | 22:18 |
| Sam-I-Am | yeah, and the bridge | 22:19 |
| prometheanfire | ya | 22:19 |
| prometheanfire | looks like that patch in the email thread does client side setting | 22:20 |
| Sam-I-Am | it should | 22:20 |
| Sam-I-Am | sort of splitting my brain between meeting and this | 22:21 |
| Sam-I-Am | prometheanfire: interestingly enough, even if this patch only supported ovs, ovs doesnt plug directly into instances anyway | 22:22 |
| Sam-I-Am | so... | 22:23 |
| prometheanfire | talking with ijw about it now | 22:23 |
| Sam-I-Am | ijw? | 22:23 |
| prometheanfire | author | 22:23 |
| Sam-I-Am | ahhh | 22:23 |
| Sam-I-Am | this is helpful :) | 22:23 |
| Sam-I-Am | use the source | 22:23 |
| prometheanfire | #os-neutron | 22:23 |
| prometheanfire | luke | 22:23 |
| Sam-I-Am | openstack-neutron, not os-neutron? | 22:24 |
| prometheanfire | ya | 22:25 |
| prometheanfire | shorthand | 22:25 |
| Sam-I-Am | ok, i didnt see anything in there | 22:26 |
| prometheanfire | no one's talking now | 22:26 |
| Sam-I-Am | oh, i thought you were talking to the guy | 22:26 |
| Sam-I-Am | so... not sure how that would apply to kilo | 22:28 |
| Sam-I-Am | bbiab 1:1 meeting | 22:28 |
| prometheanfire | Sam-I-Am: kk, cya, going to eat now | 22:28 |
| Sam-I-Am | ok, enjoy foodage | 22:29 |
| *** erikmwilson has quit IRC | 22:29 | |
| *** sdake has joined #openstack-ansible | 22:31 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:36 | |
| *** sdake_ has joined #openstack-ansible | 22:37 | |
| *** sdake has quit IRC | 22:40 | |
| openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Correct a syntax error in some hosts: patterns https://review.openstack.org/169506 | 22:48 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Test instance connectivity in compute scenarios https://review.openstack.org/170187 | 22:50 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Set containers to use the same apt repo as the host https://review.openstack.org/171175 | 23:30 |
| openstackgerrit | Merged stackforge/os-ansible-deployment: Update Glance service to Kilo https://review.openstack.org/170211 | 23:31 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
