Tuesday, 2015-09-01

*** sdake_ has joined #openstack-ansible00:08
*** britthou_ has joined #openstack-ansible00:09
*** sdake has quit IRC00:11
*** britthouser has quit IRC00:12
*** cloudtrainme has joined #openstack-ansible00:15
cloudnullevrardjp: it took all day but I finally got to the review of the patch https://review.openstack.org/218818 - looks like a good start btw. I've done my best to review the entire role / code and I like where its going.00:16
cloudnullI think that the effort in https://review.openstack.org/#/c/217517/ should merge with https://review.openstack.org/21881800:17
*** sdake has joined #openstack-ansible00:18
cloudnullthat said, I'll let you and Shu Shen (sshen) work that out if needed.00:18
cloudnullevrardjp: great work on that role and i'll try to sync up with you in the morrow00:19
*** sdake_ has quit IRC00:21
*** sdake_ has joined #openstack-ansible00:47
*** sdake__ has joined #openstack-ansible00:49
*** sdake has quit IRC00:50
*** sdake_ has quit IRC00:53
*** sdake__ is now known as sdake00:54
*** sigmavirus24_awa is now known as sigmavirus2401:00
openstackgerritBrian Moss proposed stackforge/os-ansible-deployment: [OSAD doc] Update link to Cloud Admin Guide  https://review.openstack.org/21908501:11
*** BjoernT has quit IRC01:15
*** cloudtrainme has quit IRC01:55
*** tlian2 has quit IRC02:19
mhaydenokay, so do i make a spec first or a blueprint?02:38
*** britthou_ has quit IRC02:53
*** britthouser has joined #openstack-ansible02:53
sigmavirus24mhayden: yes03:03
sigmavirus24mhayden: we used to primarily use blueprints03:03
mhaydensigmavirus24: i see what you did there03:03
sigmavirus24I think now they're sort of placeholders for tracking development via launchpad sort of03:03
sigmavirus24So my workflow is write a spec, before pushing create a blueprint with the summary. Then create spec referencing the blueprint name03:03
sigmavirus24Make sense/03:03
mhaydengotcha03:08
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment-specs: Adding spec for named veth interfaces  https://review.openstack.org/21910503:12
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment-specs: Adding spec for named veth interfaces  https://review.openstack.org/21910503:14
mhaydeni failed at rst :/03:17
*** Apsu is now known as JorgeSonata03:20
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment-specs: Adding spec for named veth interfaces  https://review.openstack.org/21910503:20
mhaydennow it should pass03:21
*** JorgeSonata is now known as Apsu03:22
*** shausy has joined #openstack-ansible03:26
*** darrenc is now known as darrenc_afk03:28
sigmavirus24mhayden: everyone fails at rst the first few times03:33
sigmavirus24I've been writing rst for years now and still fail03:34
mhaydeni missed the tox stuff in the instructions :)03:34
mhaydensigmavirus24: next spec will suggest a migration to cfengine03:34
* mhayden cackles03:35
sigmavirus24dude I cfengine'd in college03:38
*** britthou_ has joined #openstack-ansible03:46
*** sigmavirus24 has quit IRC03:48
*** britthouser has quit IRC03:48
*** sigmavirus24 has joined #openstack-ansible03:49
*** THiagoCMC has quit IRC03:52
*** britthou_ has quit IRC03:55
*** sigmavirus24 is now known as sigmavirus24_awa03:59
*** markvoelker has joined #openstack-ansible04:00
*** markvoelker_ has joined #openstack-ansible04:02
*** shausy has quit IRC04:04
*** markvoelker has quit IRC04:04
*** darrenc_afk is now known as darrenc04:54
*** cloudtrainme has joined #openstack-ansible04:56
*** shausy has joined #openstack-ansible05:21
*** cloudtrainme has quit IRC05:39
*** markvoelker_ has quit IRC05:53
*** shausy has quit IRC06:04
*** sdake_ has joined #openstack-ansible06:05
*** shausy has joined #openstack-ansible06:05
*** sdake has quit IRC06:08
*** sdake has joined #openstack-ansible06:11
*** sdake has quit IRC06:11
*** sdake has joined #openstack-ansible06:13
*** sdake_ has quit IRC06:15
*** shoutm has quit IRC06:17
*** Mudpuppy_ has quit IRC06:26
*** Mudpuppy has joined #openstack-ansible06:26
*** Mudpuppy has quit IRC06:30
*** alextricity has quit IRC07:22
*** shoutm has joined #openstack-ansible07:24
evrardjpgood morning everyone07:41
*** gparaskevas has joined #openstack-ansible08:13
matttmorning evrardjp08:14
evrardjpwhat does "NIT" mean in a review? it's like a software bug so ugly it looks like its animal counterpart with the same name?08:16
odyssey4memorning all08:17
odyssey4meevrardjp a nit is just a small minor comment which ideally should be fixed, but is generally not enought o hold back a review08:18
evrardjpgood morning odyssey4me08:18
evrardjpthanks for the info08:18
evrardjpit's some kind of abreviation? like "not important topic"?08:19
odyssey4mefyi mattt hughsaunders andymccr https://review.openstack.org/218572 unblocks master by resolving the ssh conneciton failures08:21
odyssey4meevrardjp your comments on your changes to ssh config in ansible.cfg interest me though08:21
odyssey4meevrardjp nit is short for nit-pick I think08:21
evrardjpthis is what I perceived08:22
odyssey4me'Look for and criticize trivial faults'08:22
evrardjpI think it should be more tested08:22
evrardjpif it's real, it's a damn obscure bug08:23
evrardjpI understand with the example given the dictionary, thanks for improving my english skills ;)08:24
odyssey4mesvg yes, 11.2.0 is being held back due to some breaking changes we found - if we can get https://review.openstack.org/218572 merged today, then I'll finalise https://review.openstack.org/217014 and we can get that merged which is a final patch for 11.2.008:39
svgthx odyssey4me08:40
odyssey4mewe had a holiday yesterday - I did hope that we'd have more votes on the patch from the US guys, but that didn't happen :/08:41
*** shausy has quit IRC09:04
*** shausy has joined #openstack-ansible09:05
openstackgerritJean-Philippe Evrard proposed stackforge/os-ansible-deployment: [WIP] Implementation of keepalived for haproxy  https://review.openstack.org/21881809:36
evrardjpodyssey4me: I've seen your review after this commit, but I'm glad I don't have to change anything... I thought of it without you... You see, progress!09:39
svgevrardjp: Would that patch allow for using haproxy in production?09:40
evrardjpthat's what we'll try09:40
evrardjphaproxy definitely needs fine tuning in terms of timers09:41
evrardjpwe've got our own values09:41
evrardjpit's really painful to setup09:41
evrardjpto correctly setup09:41
evrardjpthis just improves HA of haproxy09:41
evrardjpby setting it in one or multiple vrrp groups, depending on what you're looking for09:42
svgright now we use an extrenal BigIP F5 device; might be need to not depend on that anymore09:47
svg*neat09:47
*** Mudpuppy has joined #openstack-ansible10:00
*** Mudpuppy has quit IRC10:04
odyssey4meevrardjp agreed, I think some attention should be paid to some of the backend config in haproxy10:07
odyssey4mesvg using the F5 would still be something that we recommend, bu haproxy is becoming more usable for production in smaller deployments10:08
*** javeriak has joined #openstack-ansible10:13
*** javeriak has quit IRC10:18
*** britthouser has joined #openstack-ansible11:09
*** jaypipes has joined #openstack-ansible11:14
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Removed default lxc profile on container create  https://review.openstack.org/21701411:15
*** shoutm has quit IRC11:19
evrardjpodyssey4me: the interesting thing is that haproxy's price doesn't scale the same way as F5 depending on your bandwidth11:25
odyssey4meevrardjp yes :) what I am saying though is that RPC has decided that it will not be using haproxy for load balancing, so the maintenance of the haproxy role for production use is totally up to the community that wants to use it for production11:28
evrardjpyes, I understood, and it seems logical11:28
evrardjpwe are working on OSS after all11:28
*** javeriak has joined #openstack-ansible11:29
odyssey4me:)11:29
*** pradk has joined #openstack-ansible11:50
*** javeriak_ has joined #openstack-ansible12:00
*** javeriak has quit IRC12:02
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment-specs: Adding spec for named veth interfaces  https://review.openstack.org/21910512:16
mgariepygood morning everyone12:20
evrardjpgood morning mgariepy12:21
evrardjpis someone interested by having the nova_console secured?12:22
evrardjpor I am the only one? Because it's an upstream bug I think12:22
*** javeriak_ has quit IRC12:39
odyssey4meevrardjp I think we all are, but yes as I recall there's some sort of upstream bug - don't quote me on that12:43
evrardjpI've edited the js for nova console, and I got it working with ssl termination on my load balancers12:48
evrardjpno problem of protocol mismatch anymore12:48
evrardjphowever it's not real security, the connection is secure, but the spice page still shows: "keyboard channel is insecure"12:49
*** sigmavirus24_awa is now known as sigmavirus2412:51
*** javeriak_ has joined #openstack-ansible12:52
shausy@here during one of my play, im getting the following error --> undefined variables: 'dict object' has no attribute 'vips'", 'failed': True}12:55
shausyany pointer for this issue12:55
*** tlian has joined #openstack-ansible12:59
evrardjpshausy: check you identation first13:00
evrardjpand see if you your variable has "vips"13:00
evrardjpthe easiest way for us to help you is probably by copy pasting more content somewhere13:00
*** scarlisle has joined #openstack-ansible13:04
shausyevrardjp, i have checked, indn with yml verifier.. no issues there13:09
matttshausy: i see no reference to "vips" on master, which branch are you using?13:11
*** gparaskevas has quit IRC13:13
*** sdake has quit IRC13:17
shausymattt, 1.7.2 version stable we are using13:17
odyssey4meshausy is that your ansible version? I thikn that mattt meant what version of openstack-ansible/os-ansible-deployment are you using?13:19
*** cloudtrainme has joined #openstack-ansible13:19
matttshausy: yep which branch of os-ansible-deployment13:19
*** cloudtrainme has quit IRC13:19
openstackgerritHugh Saunders proposed stackforge/os-ansible-deployment: Add variable for cirros url  https://review.openstack.org/21731013:20
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Update how neutron migrations are handled  https://review.openstack.org/21558413:20
*** sdake has joined #openstack-ansible13:21
*** sigmavirus24 is now known as sigmavirus24_awa13:24
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Update how neutron migrations are handled  https://review.openstack.org/21558413:30
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Updated juno to include fix for CVE-2015-3241 - 26 Aug 2015  https://review.openstack.org/21709813:32
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Update how neutron migrations are handled  https://review.openstack.org/21558413:34
*** sigmavirus24_awa is now known as sigmavirus2413:39
palendaeodyssey4me: Does https://review.openstack.org/#/c/218572/ not have a bug associated?13:45
odyssey4mepalendae there is - check the bottom of the commit msg13:46
palendaeHuh, so the topic didn't update13:46
odyssey4mepalendae ah, that's a legacy of me merging two patches from my wip testing over the weekend13:47
odyssey4methe topic is usually based on your branch, not on the referenced bug13:47
*** Mudpuppy has joined #openstack-ansible13:49
palendaeUnless you have a bug in the first commit message13:51
palendaeBut cool, silly me for not to check the commit message :)13:53
odyssey4mepalendae the first commit message had one ;) https://review.openstack.org/#/c/218572/1//COMMIT_MSG,cm13:53
*** KLevenstein has joined #openstack-ansible13:53
cloudnullmorning13:54
odyssey4meo/ cl13:55
odyssey4mecloudnull13:55
cloudnullhow was your bank holiday ? :)13:58
odyssey4mecloudnull much sleeping and movie watching was done, oh and good food13:59
cloudnullsounds like a perfect day!13:59
palendaeHuh13:59
cloudnullevrardjp: did you ever get the dynamic inventory bits sorted ?14:00
* cloudnull still reading scroll back14:00
evrardjpcloudnull:  didn't got the chance to do so14:01
evrardjpit's not that it doesn't work14:01
evrardjpit's that it overrides some of my work14:01
cloudnullah. where is it overriding  ?14:01
evrardjpdo you mind if we take this one as example? https://review.openstack.org/#/c/218818/14:02
cloudnullsur e14:03
evrardjpIf I rename the conf.d/keepalived.yml.example into keepalived.yml14:03
evrardjpI do any operation14:03
evrardjpI will see that in my openstack_inventory.json:14:03
*** yaya has joined #openstack-ansible14:03
cloudnullyes it merges on ansible playbook execution.14:04
*** cloudtrainme has joined #openstack-ansible14:04
evrardjpphysical_host_group: "keepalived_hosts"14:04
evrardjpit must be ok, but if I use the same hosts for another purpose (let's say haproxy)?14:05
evrardjpif it overwrites the physical_host_group, won't it be problematic?14:05
*** KLevenstein has quit IRC14:05
evrardjpatm the inventory seems fine for me14:05
evrardjpbut I'd like to make sure it will always be the same for everyone14:06
*** spotz_zzz is now known as spotz14:06
evrardjpand good morning cloudnull ;)14:06
*** KLevenstein has joined #openstack-ansible14:07
*** phalmos has joined #openstack-ansible14:07
evrardjpodyssey4me: you said you had a nice holiday with good food? Is that even possible in the UK? (I meant having good food?) ><14:07
evrardjpI guess lots of restaurants with a different origin than UK :p14:08
*** javeriak has joined #openstack-ansible14:10
cloudnullevrardjp:  yes that could be problematic as the physical_host_group is used in a few places. that said its a legacy variable and only used within inventory.14:11
cloudnullhttp://cdn.pasteraw.com/9j4gfvej1i3nyat7ez3dvbb1tyd8mru14:11
palendaemhayden: Updated https://review.openstack.org/#/c/219105/ with comments14:11
*** jmckind has joined #openstack-ansible14:11
cloudnull^ thats where the physical_host_group variable is defined.14:11
*** javeriak_ has quit IRC14:11
evrardjpit's maybe because I used a wrong way to define a new type of service14:12
evrardjpprobably*14:12
mhaydenpalendae: thanks -- looking them over14:12
cloudnullso it should not cause deployment problems and in the liberty/mitaka timeframe, if palendae gets some time to work on upgrade implementations, we should be able to clean up some of the todos and make those kinds of things go away14:12
evrardjpthat's good news cloudnull14:13
evrardjpjust to be sure:14:13
evrardjpon this: https://review.openstack.org/#/c/218818/3/etc/openstack_deploy/env.d/keepalived.yml14:13
palendaeSo cloudnull and evrardjp, I'm working on specs for upgrade stuff (like liberty's upgrade path and an upgrade gate job) and docs for the env, conf, and inventory, but it's been an 'as we have time' thing14:13
evrardjpI think if ppl want to bring their own kind of components, it could be useful to understand how env.d really works14:14
palendaeYep14:15
palendaeI agree14:15
evrardjpwhile it seemed obvious at first, when reading the env.d files, creating some env.d has overwrite impact14:15
evrardjp(if hosts are reused for example)14:15
evrardjpalso here:14:16
evrardjphttps://review.openstack.org/#/c/218818/3/etc/openstack_deploy/env.d/keepalived.yml14:16
evrardjpin container_skel:14:16
evrardjpthe first line, here keepalived_container, should be "standardized"14:16
evrardjpI had the feeling it was weird to have some kind of group of containers called keepalived_containers and another group called keepalived_container14:17
openstackgerritMatthew Kassawara proposed stackforge/os-ansible-deployment: Disable scatter-gather offload on host bridges  https://review.openstack.org/21929214:21
openstackgerritMatthew Kassawara proposed stackforge/os-ansible-deployment: Disable scatter-gather offload on host bridges  https://review.openstack.org/21929214:25
*** alejandrito has joined #openstack-ansible14:26
*** javeriak has quit IRC14:28
cloudnullsorry was in side convos here. evrardjp host wise it should be ok to reuse hosts in various files as long as the groups are different in the confi files.14:37
evrardjpthat makes sense14:37
cloudnullmaybe we need to revist the precedence system in how those conf files are used to generate inv json.14:38
odyssey4mecloudnull so fyi I had some conversations with -infra and did a little research on a few topics14:38
evrardjpso there is nothing to change for the env.d/conf.d in my commit14:38
cloudnullso that we can have it be more deterministic instead of doing an inventory hash merge.14:38
evrardjpI was just surprised that merging was doing overwrite14:38
cloudnullevrardjp:  i dont think so, and the physical_host_group variable can be ignored.14:39
cloudnullodyssey4me:  on what ?14:39
evrardjpbut if it's on non-important stuff, then it's ok14:39
odyssey4mewith regards to https://review.openstack.org/218611 I was investigating whether there was a difference made to the success of builds if we don't use the rackspace mirror specifically14:39
odyssey4meit does seem to improve things when builds are running in hpcloud14:39
odyssey4methere's some speculation in -infra about whether hpcloud intercepts apt requests to archive.ubuntu and directs them to local caches14:40
cloudnullyea in ci there were 6 successes 5 of which were in hp on https://review.openstack.org/#/c/21861114:40
odyssey4mebut nothing concrete - I'd like to run some tests in your hpcloud account if that's ok?14:40
cloudnullsure.14:40
cloudnullill make you a tenant14:41
odyssey4meI saw a post that hpcloud has a specific mirror, but -infra doesn't appear to be using it - I'm thinking that perhaps we should use the generic approach of using the hosts repositories for AIO's - but then for gate jobs we can select a repo based on known upstream repository URL's that give the best success.14:41
palendaeodyssey4me: Did the image-based approach fail14:42
palendae?14:42
odyssey4meie an AIO build should use whatever's configured on the initial host build - then the gate checks should use known repositories which work best for rax/hpcloud14:42
odyssey4mepalendae image-based approach?14:42
palendaeodyssey4me: I thought you linked to something that showed we could deploy our own custom OS images on infra14:43
*** alextricity_h has joined #openstack-ansible14:43
odyssey4meI think that the above approach for the AIO vs Gate configs of the repositories is the best way to be dev friendly, but also configure what we think is best for the gate. It seperates the concerns better.14:43
palendaeTrue enough14:43
odyssey4mepalendae ah, that was the next thing I wanted to mention to cloudnull14:44
palendaeThe image based approach would save time, but also not exercise certain things14:44
palendaeBut if those things are broken in infra, not much point in exercising them either14:44
odyssey4meI didn't actually do an image build test of any sort (yet) - but it does seem that there are projects that are building images and publishing them.14:44
*** javeriak has joined #openstack-ansible14:44
palendaeOr teaming with infra to fix it14:44
odyssey4meThis would mean that we could publish LXC images however we choose - perhaps every tag or whatever.14:46
odyssey4meThis could be very helpful for upgrade testing - to make the check run a heck of a lot quicker.14:46
odyssey4meIt would also be useful for role specific gate checks where we may wish to use the images for the infrastructure, then only do the full deploys for the openstack bits... or something to that effect.14:47
odyssey4meAnyway, more specifically it opens up the options.14:47
*** shoutm has joined #openstack-ansible14:49
*** phalmos has quit IRC14:49
*** sdake_ has joined #openstack-ansible14:53
*** k_stev has joined #openstack-ansible14:54
*** phalmos has joined #openstack-ansible14:55
*** Mudpuppy has quit IRC14:57
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Remove hardcoded config drive enforcement  https://review.openstack.org/21848014:58
*** sdake has quit IRC14:58
*** Mudpuppy has joined #openstack-ansible14:58
*** KLevenstein has quit IRC14:58
*** yaya has quit IRC14:58
odyssey4mehere's another funny14:59
*** KLevenstein has joined #openstack-ansible14:59
odyssey4methe issue in hpcloud-b4 seems very specifically tied to the containers being unable to communicate with dns or the outside world in any way15:00
odyssey4meonly the containers though, the host is perfectly fine15:00
palendaeodyssey4me, cloudnull, mattt, andymccr, hughsaunders, sigmavirus24, other cores - I want to submit a spec for upgrade testing, however, I don't think it fits just in kilo or just in liberty. Anyone have opinions where to put it? Top of specs repo?15:05
sigmavirus24palendae: yes15:05
*** galstrom_zzz is now known as galstrom15:06
sigmavirus24put it in liberty because it's new feature development15:06
sigmavirus24just because it's in one doesn't mean it stops applying to future releases15:06
odyssey4mepalendae it needs a finite target15:06
*** sdake_ is now known as sdake15:06
sigmavirus24"This will be implemented for liberty"15:06
palendaeOk, my plan was to implement in kilo, too15:06
palendaeAnd I think kilo and liberty implementations may vary15:06
palendaeSo perhaps we have different specs?15:06
odyssey4meagreed with sigmavirus24 - the upgrade blueprint must specifically target kilo->liberty15:06
palendaeAlright, so are people saying to leave kilo as is?15:07
sigmavirus24palendae: that might work15:07
palendaeI'd really like to fix that script in kilo...15:07
palendaeBut Liberty is greenfield, too15:07
odyssey4mepalendae if at some point it's decided that kilo's method needs an overhaul, then that should be blueprinted then15:07
sigmavirus24palendae: that works too15:07
sigmavirus24kilo specs are still open15:07
sigmavirus24as far as I remember15:07
sigmavirus24odyssey4me: I think most of us agree it needs an overhall sadly15:08
palendaeodyssey4me: Fair. I was going to start there because I know at least part of what needs to be done15:08
odyssey4mefor now, liberty is soon - let's get it right in a greener field, then learn lessons and apply them to the kilo upgrade method if we can afford the time15:08
*** javeriak_ has joined #openstack-ansible15:08
odyssey4mesigmavirus24 sure we all agree, but do we have the time and resource allocation to do it15:08
sigmavirus24odyssey4me: sure15:09
sigmavirus24Deciding to do the work and deciding the work needs to be done are two different things15:09
odyssey4mepalendae I suppose I'm saying that we should focus on one and do it well - not on both15:09
palendaeOk, so I'll work on that. I'd really like to get cores looking at the liberty upgrade path spec too, and stop merging changes for kilo run-upgrades.sh into master15:09
sigmavirus24As in "we know this is something broken and should be fixed" versus "we can fix that now"15:09
palendaeodyssey4me: Yeah, and that makes sense15:09
sigmavirus24Specs are an indication of the former, not the later15:09
sigmavirus24*latter15:09
*** javeriak has quit IRC15:09
odyssey4mepalendae I haven't seen a run-upgrade patch into master for ages15:10
palendaeThere were enough to invalidate my WIP patch that guts it15:10
* palendae shrugs15:10
*** shoutm has quit IRC15:10
palendaeMostly just want to be clear, but I can't be too stern about it with out the spec accepted15:10
palendaeGonna go respond to comments in it right now15:11
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Update how neutron migrations are handled  https://review.openstack.org/21558415:11
palendaeodyssey4me, sigmavirus24 Do you remember the title for the epoch discussion on the openstack ML?15:15
sigmavirus24palendae: no but it was started by harlowja15:16
sigmavirus24That was back in June maybe?15:16
sigmavirus24Perhaps earlier?15:16
sigmavirus24If you search by month and look for harlowja and me in a conversation with epoch somewhere in the title you'll find it15:16
palendaehttp://lists.openstack.org/pipermail/openstack-dev/2015-July/069085.html ?15:17
odyssey4mepalendae [openstack-dev] [openstack-ansible] [os-ansible-deployment] Kilo -> Liberty Upgrade Problems15:18
sigmavirus24palendae: that's it15:19
odyssey4meoh, you're meaning the other discussion15:19
odyssey4methe one that spurred this one15:19
odyssey4methat was fragmented among many spats15:19
palendaeI'll link both15:19
palendaeWhy not15:19
sigmavirus24odyssey4me: I think you're confused. It was one thread but dotted over a couple of weeks15:20
sigmavirus24I didn't want to get too angry in one sitting15:20
sigmavirus24That isn't quite healthy15:20
odyssey4melol15:20
sigmavirus24There isn't enough alcohol in Wisconsin to calm me down when people say "downtream are the only people who should care about that"15:21
sigmavirus24Because that's not how that works15:21
sigmavirus24You're using versions semantically already, you should be as semantic as possible to be as clear to all users what those versions mean15:21
* sigmavirus24 steps away from keyboard for a short period of time15:21
*** galstrom is now known as galstrom_zzz15:23
bgmccollumvPOTAT.015:23
*** galstrom_zzz is now known as galstrom15:23
*** k_stev has quit IRC15:26
sigmavirus24Hey potatoes are actually useful bgmccollum15:26
sigmavirus24You can run a cloud on potato power15:26
sigmavirus24Not an openstack cloud because those just don't run15:26
sigmavirus24But any other cloud, certainly15:26
palendaeodyssey4me, cloudnull: Replied to comments on https://review.openstack.org/#/c/207713/, but working on a new draft too15:29
*** k_stev has joined #openstack-ansible15:31
*** jmckind has quit IRC15:31
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: [WIP] Diagnostic test  https://review.openstack.org/21931315:31
*** javeriak has joined #openstack-ansible15:32
*** galstrom is now known as galstrom_zzz15:33
*** cloudtra_ has joined #openstack-ansible15:33
*** javeriak_ has quit IRC15:33
*** cloudtrainme has quit IRC15:33
*** jmckind has joined #openstack-ansible15:34
*** alop has joined #openstack-ansible15:34
*** galstrom_zzz is now known as galstrom15:44
*** cloudtra_ has quit IRC15:44
openstackgerritMatt Thompson proposed stackforge/os-ansible-deployment: Update how neutron migrations are handled  https://review.openstack.org/21558415:51
*** shausy has quit IRC15:56
*** k_stev has quit IRC16:01
*** k_stev has joined #openstack-ansible16:01
*** k_stev has quit IRC16:02
*** k_stev has joined #openstack-ansible16:02
*** k_stev has quit IRC16:03
*** k_stev has joined #openstack-ansible16:03
*** jmckind has quit IRC16:03
*** sdake_ has joined #openstack-ansible16:05
palendaeBug tirage?16:06
palendaetriage, too16:06
*** k_stev1 has joined #openstack-ansible16:06
*** k_stev has quit IRC16:08
*** k_stev2 has joined #openstack-ansible16:08
odyssey4meuh cloudnull ^ ?16:08
*** k_stev2 has quit IRC16:08
*** sdake has quit IRC16:09
*** k_stev has joined #openstack-ansible16:09
cloudnullah . yea. sorry .16:10
* cloudnull in email hell16:10
cloudnullfirst up https://bugs.launchpad.net/openstack-ansible/+bug/148863916:10
openstackLaunchpad bug 1488639 in openstack-ansible "Profile Ansible tasks to find the slowest ones" [Undecided,New]16:10
*** phalmos has quit IRC16:11
*** k_stev1 has quit IRC16:11
cloudnullrelated PR https://review.openstack.org/#/c/216849/16:11
odyssey4memhayden if you had assigned it to yourself then the master review would have been auto-added :)16:11
odyssey4meto me it's a wishlist item, and should only be in gate tests - not run for production environments16:12
cloudnullI like the use case and change that said i can understand the desire to keep unneeded bits out of the repo16:12
cloudnullif some of the other cores can weigh in on it it would be great16:12
odyssey4meit can also make some of our scripts simpler, so I'm game for the change16:12
cloudnullnext https://bugs.launchpad.net/openstack-ansible/+bug/148865016:12
openstackLaunchpad bug 1488650 in openstack-ansible "Should be able to specify account names in user_variables" [Undecided,New]16:12
cloudnullthis is something that we have in kilo and beyond16:13
cloudnullthe question is do we need / want it for juno16:13
odyssey4meI am not a fan of backporting anything to Juno, personally.16:14
cloudnulli say no. being that juno goes sec patch only in about two months.16:14
odyssey4meany features I mean16:14
odyssey4mereal bugs, sure - but not features16:14
*** phalmos has joined #openstack-ansible16:14
cloudnullim of the oppinion that if thats a feature needed, kilo is where its at16:14
odyssey4meyup, kilo's been out for ages - it's time that those that use juno upgrade to it16:15
palendaeodyssey4me: Keep in mind Kilo upgrades are not simple16:15
palendaeSo I see why people would want it in Juno16:15
palendaeThat said, we can't keep taking features back16:15
odyssey4mepalendae if they're on juno, then they are already running in the way juno was designed16:15
palendaeSure16:15
odyssey4meit's pointless to change that way at this late stage16:15
palendaeAgreed16:16
cloudnullAgreed.16:16
*** jaypipes has quit IRC16:16
palendaeJust saying that waving it off as "upgrade to Kilo" is complicated16:16
odyssey4meif people are having trouble with upgrades, the issues with the upgrades or the changes made in kilo need to be adressed - rather than trying to make juno work like kilo.16:16
palendaeAgreed16:17
palendaeSo I think that bug needs to have a link to the change in Kilo with that explanation16:17
cloudnullok .16:18
*** jmckind has joined #openstack-ansible16:18
cloudnullupdated16:21
cloudnullnext https://bugs.launchpad.net/openstack-ansible/+bug/148867016:21
openstackLaunchpad bug 1488670 in openstack-ansible "configure_diskspace will attempt to use small partitions" [Undecided,New]16:21
*** sdake_ is now known as sdake16:21
odyssey4methere's a review in for that16:21
palendaeThanks16:22
cloudnullrelated https://review.openstack.org/#/c/216880/16:22
odyssey4meyup, that's it16:22
odyssey4mesome discussion there - I agree that we need to do better, but I'm thinking that the current method is a little too exclusionary16:23
odyssey4mewe've suggested a smaller limit or something16:23
odyssey4meanyway, in terms of bug this is assigned to neill and is low priority16:23
*** javeriak_ has joined #openstack-ansible16:24
cloudnullbug updated with context, assigned ot neil, and marked triaged .16:24
cloudnullnext https://bugs.launchpad.net/openstack-ansible/+bug/148916916:24
openstackLaunchpad bug 1489169 in openstack-ansible "Speed up deployment by upgrading cached container image" [Undecided,New]16:24
*** javeriak has quit IRC16:25
odyssey4meI like the idea16:26
odyssey4mewish list16:26
odyssey4meassign to mhayden16:26
cloudnullkk16:26
odyssey4mebut a cautionary note - putting it into the play would probably make it update the cache every time?16:26
odyssey4mehow does that affect already running containers16:27
odyssey4meand how does that affect consistency when adding new containers later on16:27
*** jaypipes has joined #openstack-ansible16:27
cloudnull+1 i like the approach and I think we can iterate on further improving that process for more speed gains.16:28
cloudnullwe can drop a when clause to only do it for new container creates.16:28
cloudnullor if a variable is specifically specified16:29
evrardjpthat would be even better16:29
palendaeYeah16:29
cloudnull+1 odyssey4me can you drop a not e in that bug to that effect?16:29
evrardjpby default, update (old process), and skip if variable is set16:29
odyssey4mecloudnull sure16:30
palendaeI know the apt module has a conditional you can add to only update things if it's been >n seconds since last update, but this looks like it's just using a command16:30
evrardjpthis is for the whole container thing, this is even better16:31
evrardjpwhy not think also of one update and then a distribution of the updated container template to all physical hosts?16:31
*** cloudtrainme has joined #openstack-ansible16:32
cloudnullyea theres an upstream bug in that module which makes it so the update doesn't "always" happen as one would assume. we have a fix upstream to address that but its currently in review https://github.com/ansible/ansible-modules-core/pull/151716:32
cloudnullpalendae:  ^ re apt module.16:32
palendaeAh, ok16:32
palendaeThat was really a tangent16:32
cloudnullevrardjp:  thats also a good idea .16:32
cloudnullodyssey4me:  and andymccr have been pondering how to do tha t16:32
cloudnullmaybe you three can come to gether to work out a solution ?16:33
evrardjpwhy not :)16:33
cloudnulli think that would be ideal in the end to be able to distribute an updated image internally using something like the repo container infrastrucutre however there are some logistical issues with that need to be address.16:34
odyssey4meAdded this note: It may be useful to download the cache to the deployment host (and if the file is there, don't download it again), then update it and distribute it to the rest of the hosts from there. That may cater for all needs.16:34
cloudnulland with a meeting of the minds antonym odyssey4me evrardjp we can make it all go :)16:34
palendaeI think you mean andymccr there :)16:35
cloudnulls/antonym/ andymccr/16:35
palendaeChannel's getting bigger, our tab completion strings are getting longer16:35
cloudnullindeed16:35
cloudnull:)16:35
cloudnullnext https://bugs.launchpad.net/openstack-ansible/+bug/148925116:35
openstackLaunchpad bug 1489251 in openstack-ansible "Wrong pip package versions after upgrade" [Undecided,New]16:36
palendaeLooks like this is Juno -> Juno minor versions16:36
palendaeBut may be relevant to any upgrades16:36
*** Mudpuppy has quit IRC16:37
cloudnullthis is an issue with us needing to downgrade a package due to upstream pin changes.16:37
prometheanfiredo we not support downgrades?16:37
cloudnullsadly pip does not have a good resolver to make that a reality without some manual intervention .16:37
cloudnullsigmavirus24: cc ^16:37
palendaeyeah, pip doesn't work very well there..16:38
odyssey4methis only affects the deployment host though16:38
cloudnull++16:38
palendaeodyssey4me: With these specific packages, but it may very well be a general problem, no?16:38
cloudnullon install of a given service pip will install all of the packages that are required. which will do a downgrade in general.16:39
odyssey4mewell, it will affect how the repo server builds out the final package list16:39
odyssey4meother than that, requirements.txt is only used by bootstrap-ansible as I recall16:40
palendaeYeah, true - though openstack services do tend to shift dependencies back when problems are found16:40
cloudnullif im not mistaken pyyaml would be replaced if nova was reinstalled in this case.16:40
palendaeAnd we don't clean it up16:40
*** KLevenstein_ has joined #openstack-ansible16:42
prometheanfireeasier to do a full regen every run?16:42
*** KLevenstein has quit IRC16:42
*** KLevenstein_ is now known as KLevenstein16:42
prometheanfirerecompile the deps?16:42
odyssey4meprometheanfire that is not possible for juno16:42
prometheanfireah16:42
odyssey4mejuno's repo is built upstream and is done per tag16:42
*** Mudpuppy has joined #openstack-ansible16:43
odyssey4methe bug in question doesn't really say that there's any serious problem other than inconsistency16:43
palendaeSo we should ask if there is any other serious problem16:44
odyssey4meyeah - if this doesn't affect any running services, then what's the problem?16:45
*** Bjoern_ has joined #openstack-ansible16:45
palendaeI asked16:45
palendaeIn a comment16:45
odyssey4meotherwise it sounds like a low priority issue16:45
palendaeCan be for now, until we get a response16:46
cloudnullok marked incomplete16:46
palendae^16:46
prometheanfirenext?16:46
cloudnullnext https://bugs.launchpad.net/openstack-ansible/+bug/148945116:47
openstackLaunchpad bug 1489451 in openstack-ansible "Not able to connect memcached_container while running setup-infrastructure.yml " [Undecided,New]16:47
cloudnulli think this is an issue with the key distribution bits that is being replaced by the process developed by andymccr.16:49
odyssey4meoh? I thought that python was already installed in the image?16:49
cloudnullit is.16:50
odyssey4meand why is this not happening in our gate checks?16:50
cloudnullit is on contianer create.16:51
cloudnullhttps://github.com/stackforge/os-ansible-deployment/blob/kilo/playbooks/roles/lxc_container_create/tasks/container_create.yml#L256-L26616:51
odyssey4mehe suggested using just 'python' as the package name instead of 'python2.7'16:52
odyssey4memaybe this relates to an old mirror or something?16:52
cloudnullmaybe... im pondering and dont really know how all that happened.16:53
javeriak_hey guys, somewhat off-topic, but I'm trying to upload a glance image with cloudfiles storage as the backend, the image is 1G, and forever stuck on saving, im guessing some glance size parameters need to be changed?16:53
evrardjpupload from url or directly? connection speed? what's forever?16:54
evrardjpdid you try with the progress bar using the cli?16:54
palendaeafk a few, sleeping laptop while I move16:55
*** cloudtrainme has quit IRC16:55
cloudnulljaveriak_:  what evrardjp said would be of help.16:55
javeriak_evrardjp upload through cli, and nope didnt try with progress bar so all i see is 'Saving'16:56
cloudnullin that way you'd have a bit more upload stats16:56
cloudnullwe're done with the general bug triage .16:56
cloudnullthanks everyone16:56
Bjoern_javariak_: glance_container-1fec626a:~# grep 5120 /etc/glance/glance-api.conf16:56
Bjoern_swift_store_large_object_size = 512016:56
evrardjpjaveriak_: using glance cli use --progress16:57
Bjoern_look inside the glance-registry.log for errors and turn on debugging so you would see how it uses the swift client16:57
evrardjpI'm off for today16:57
evrardjpsee you tomorrow, hope you'll find your solution javeriak_16:57
javeriak_Bjoern_ yes thats 5120 and no errors i checked all logs, evrardjp let me give that a try thanks16:58
*** Bjoern_ is now known as BjoernT16:58
*** sdake_ has joined #openstack-ansible17:06
*** sdake has quit IRC17:06
*** sdake has joined #openstack-ansible17:07
BjoernTjaveriak_: look inside the image_location table from the glance database and verify that the swift url is currect. Did you set swift_store_endpoint_type to internalURL or publicURL ?17:10
*** sdake_ has quit IRC17:10
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: [WIP] Temporarily pin netaddr to not use 0.7.16  https://review.openstack.org/21936417:11
*** k_stev1 has joined #openstack-ansible17:11
*** k_stev has quit IRC17:12
javeriak_BjoernT, the swift backend settings are fine, ive had success uploading cirros images, the custom image im trying with atm is just too big i suppose, and the connectivity also seems to be crap, the progress bar is moving snail pace, so atleast now i wont get impatient and terminate it17:14
javeriak_BjoernT: to answer your question, internalURL  wouldn't work so I set it to publicURL17:15
BjoernTlol, yeah I always run tcpump  from a glance container to see if stuff is still trickling into to cloudfiles which is rather slow. Yes I was hoping you had set it to publicUrl17:15
*** k_stev1 has quit IRC17:16
*** abitha has joined #openstack-ansible17:37
*** sigmavirus24 is now known as sigmavirus24_awa17:54
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: [WIP] Temporarily pin netaddr to not use 0.7.16  https://review.openstack.org/21938518:04
*** javeriak_ has quit IRC18:06
*** cloudtrainme has joined #openstack-ansible18:12
*** javeriak has joined #openstack-ansible18:19
*** javeriak_ has joined #openstack-ansible18:22
*** javeriak has quit IRC18:24
*** k_stev has joined #openstack-ansible18:27
*** jwagner_away is now known as jwagner18:30
*** jwagner is now known as jwagner_away18:41
*** cloudtrainme has quit IRC18:43
mhaydenodyssey4me: whee!18:43
*** jwagner_away is now known as jwagner18:49
odyssey4memhayden haha18:56
* odyssey4me shakes a fist at random library changes at inconvenient times18:56
mhaydenodyssey4me: i feel all open sourcey18:57
*** javeriak has joined #openstack-ansible18:57
*** javeriak_ has quit IRC18:57
*** k_stev has quit IRC18:59
*** woodard has joined #openstack-ansible19:00
*** BjoernT has quit IRC19:02
*** gparaskevas has joined #openstack-ansible19:03
*** sigmavirus24_awa is now known as sigmavirus2419:04
*** javeriak_ has joined #openstack-ansible19:05
*** javeriak has quit IRC19:06
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: [WIP] Temporarily pin netaddr to use <0.7.16  https://review.openstack.org/21936419:07
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: [WIP] Temporarily pin netaddr to use <0.7.16  https://review.openstack.org/21938519:08
*** sdake_ has joined #openstack-ansible19:12
*** cloudtrainme has joined #openstack-ansible19:12
*** sdake has quit IRC19:13
*** sdake has joined #openstack-ansible19:13
*** cloudtrainme has quit IRC19:14
*** sdake_ has quit IRC19:18
*** cloudtrainme has joined #openstack-ansible19:18
*** scarlisle has quit IRC19:34
*** mcarden has quit IRC19:34
*** tobasco_ has quit IRC19:34
*** mrodden has quit IRC19:34
*** raginbajin has quit IRC19:34
*** harlowja has quit IRC19:34
*** Sam-I-Am has quit IRC19:34
*** jmckind has quit IRC19:34
*** b3rnard0 has quit IRC19:34
*** bogeyon18 has quit IRC19:34
*** cloudnull has quit IRC19:34
*** jwagner has quit IRC19:34
*** b3rnard0 has joined #openstack-ansible19:34
*** bogeyon18 has joined #openstack-ansible19:34
*** cloudnull has joined #openstack-ansible19:34
*** jwagner has joined #openstack-ansible19:34
*** k_stev has joined #openstack-ansible19:34
*** scarlisle has joined #openstack-ansible19:34
*** mcarden has joined #openstack-ansible19:34
*** tobasco_ has joined #openstack-ansible19:34
*** mrodden has joined #openstack-ansible19:34
*** raginbajin has joined #openstack-ansible19:34
*** Sam-I-Am has joined #openstack-ansible19:34
*** harlowja has joined #openstack-ansible19:35
*** harlowja has quit IRC19:35
*** harlowja has joined #openstack-ansible19:35
*** abitha has quit IRC19:37
*** jmckind has joined #openstack-ansible19:39
*** sdake_ has joined #openstack-ansible19:45
*** harlowja has quit IRC19:47
*** k_stev has quit IRC19:47
*** scarlisle has quit IRC19:47
*** mcarden has quit IRC19:47
*** tobasco_ has quit IRC19:47
*** mrodden has quit IRC19:47
*** raginbajin has quit IRC19:47
*** Sam-I-Am has quit IRC19:47
*** sdake has quit IRC19:49
*** sdake has joined #openstack-ansible19:49
*** harlowja has joined #openstack-ansible19:52
*** k_stev has joined #openstack-ansible19:52
*** scarlisle has joined #openstack-ansible19:52
*** mcarden has joined #openstack-ansible19:52
*** tobasco_ has joined #openstack-ansible19:52
*** mrodden has joined #openstack-ansible19:52
*** raginbajin has joined #openstack-ansible19:52
*** Sam-I-Am has joined #openstack-ansible19:52
*** sdake_ has quit IRC19:53
*** javeriak has joined #openstack-ansible20:01
*** javeriak_ has quit IRC20:02
*** cloudtrainme has quit IRC20:12
*** javeriak_ has joined #openstack-ansible20:25
*** javeriak has quit IRC20:26
*** cloudtrainme has joined #openstack-ansible20:28
*** jwagner is now known as jwagner_away20:34
*** alextricity_h has quit IRC20:34
*** andymccr has quit IRC20:39
cooljcloudnull: https://bugs.launchpad.net/openstack-ansible/+bug/148922520:41
openstackLaunchpad bug 1489225 in openstack-ansible "Allow configuration of config_drive_format in nova.conf" [Undecided,New]20:41
*** afranc202 has joined #openstack-ansible20:46
matttcoolj: why do you need to use config drive (honest question)20:48
*** KLevenstein has quit IRC20:49
*** KLevenstein has joined #openstack-ansible20:49
*** k_stev has quit IRC20:50
*** sdake has quit IRC20:53
*** k_stev has joined #openstack-ansible20:59
*** k_stev has quit IRC21:09
cooljmattt: we don't *need* it per se, but we have had issues with api saturation from customers using puppet on a bunch of instances to poll metadata21:10
matttcoolj: ah, i see!21:10
matttcoolj: wonder if that was due to a misconfigured neutron server (not enough threads/workers?)21:11
*** pradk has quit IRC21:11
mattt(and not server per se, but neutron metadata server)21:11
*** pradk has joined #openstack-ansible21:12
cooljmattt: i can find the tickets to see exactly what we tuned but as i recall we tried upping api_workers any a bunch of other stuff but the bottleneck was that neutron-metadata-proxy was a single thread/process. i might be misremembering the exact details21:14
*** gparaskevas has quit IRC21:15
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment-specs: Adding spec for named veth interfaces  https://review.openstack.org/21910521:16
matttcoolj: we can look into it, there may be no other solution but to enable config drive, but it may be worth looking into in more detail21:18
cooljmattt: what is the downside of enabling config-drive if we can also use vfat for the drive? afaik live-migration was the only reason to disable and that works as long as vfat format is used instead of iso966021:20
*** javeriak_ has quit IRC21:21
matttcoolj: tbh i'm not really aware of any21:21
openstackgerritMajor Hayden proposed stackforge/os-ansible-deployment: Used named veth pairs that match container  https://review.openstack.org/21945721:21
cooljcool, just wanted to be aware if there are further issues with having it enabled21:23
mhaydencloudnull: bumped the named-veths spec and tossed in a WIP commit21:24
matttcoolj: oh, does vfat work on non-linux images ?21:28
*** tlian2 has joined #openstack-ansible21:31
*** tlian has quit IRC21:34
cooljmattt: good question! i can test! since it's fat32 i suspect 99% of our instances will have no problem with it (mostly linux with a sprinkling of windows), but i can testopenindiana and freebsd. any others you can think of to test?21:36
*** alop has quit IRC21:37
*** Mudpuppy has quit IRC21:39
*** jmckind has quit IRC21:52
*** phalmos has quit IRC21:52
matttcoolj: may be worht testing it on windows also :P21:55
*** Bjoern_ has joined #openstack-ansible22:08
*** galstrom is now known as galstrom_zzz22:11
*** sigmavirus24 is now known as sigmavirus24_awa22:18
*** scarlisle has quit IRC22:26
*** jaypipes has quit IRC22:40
*** alejandrito_ has joined #openstack-ansible22:43
*** alejandrito_ has quit IRC22:45
*** alejandrito has quit IRC22:45
*** spotz is now known as spotz_zzz22:49
*** arbrandes has quit IRC22:50
*** andymccr has joined #openstack-ansible22:57
*** KLevenstein has quit IRC23:00
*** cloudtrainme has quit IRC23:03
*** arbrandes has joined #openstack-ansible23:03
*** Bjoern_ has quit IRC23:47
*** shoutm has joined #openstack-ansible23:48
*** woodard has quit IRC23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!