| SpamapS | right, you would say "this_is_a_network_node: true" not "sysctl.net.ipv4.ip_forward: 1" | 00:00 |
|---|---|---|
| greghaynes | yerp | 00:00 |
| greghaynes | so, I think you would still need to support a runtime sysctl-set-value, right? | 00:03 |
| * greghaynes is not wondering if https://review.openstack.org/#/c/101784/1 needs to be smart-ish and detect if its build or runtime | 00:03 | |
| greghaynes | s/not/now | 00:03 |
| *** ddieterly has joined #tripleo | 00:11 | |
| *** ddieterly has quit IRC | 00:12 | |
| *** ddieterly has joined #tripleo | 00:12 | |
| *** yamahata has quit IRC | 00:13 | |
| *** Penick has quit IRC | 00:14 | |
| *** jerryz has joined #tripleo | 00:15 | |
| *** ddieterly has quit IRC | 00:17 | |
| jerryz | lifeless: is there an updated tutorial to play tripleo, deploying seed, under, overcloud? | 00:17 |
| kevinbenton | lifeless: is there a way to turn debug logs back on for neutron so i can see what’s going on in this patch? https://review.openstack.org/#/c/101447/ | 00:17 |
| greghaynes | kevinbenton: check out the ExtraConfig parameter in overcloud.yaml | 00:20 |
| greghaynes | neutron.conf supports adding options via that | 00:20 |
| *** rlandy has quit IRC | 00:21 | |
| *** matsuhashi has joined #tripleo | 00:21 | |
| *** nati_ueno has joined #tripleo | 00:23 | |
| kevinbenton | so what’s the process of doing that for a specific neutron patch? | 00:24 |
| kevinbenton | greghaynes: ^^ | 00:25 |
| *** eghobo has quit IRC | 00:26 | |
| tchaypo | jerryz: are you asking about a video? I'm not aware of such a thing | 00:31 |
| tchaypo | but http://docs.openstack.org/developer/tripleo-incubator/devtest.html (generated from devtest.sh) essentially does that in text | 00:31 |
| jerryz | tchaypo: not necessarily a video. doc.o.o or something | 00:31 |
| jerryz | tchaypo: thanks. so the devtest scripts is the literally the procedures. | 00:34 |
| tchaypo | it's *a* procedure | 00:34 |
| tchaypo | I'd expect that if you're doing your own deployment you'll probably find parts of the procedure that don't quite do what you need | 00:34 |
| tchaypo | but our scripts aim to be "small pieces, loosely joined" so it should be easy for you to tweak pieces to get a procedure that suits you | 00:35 |
| jerryz | tchaypo: got you | 00:35 |
| tchaypo | but running through the devtest scripts first should, in theory, give you an introduction to the bits and how they fit together, and give you a starting point for your own deployment | 00:35 |
| jerryz | tchaypo: what automation or CM tool do you use to deploy your CD overcloud? | 00:36 |
| jerryz | tchaypo: devtest looks devstack kind of thing to me. | 00:37 |
| tchaypo | good question, I should know the answer to that | 00:38 |
| tchaypo | for CI we use jenkins and zuul, and they drive a slightly modified process - our CI infra has a few things that don' | 00:39 |
| lifeless | jerryz: we do use devtest, because it deploys production ready - we keep moving stuff out of it and into stable repos as time goes by | 00:39 |
| tchaypo | t change much preconfigured | 00:39 |
| lifeless | SpamapS: greghaynes FWIW e20084f21aa80e47ce219b5cc050ec2a57ab3fea | 00:39 |
| tchaypo | The slight-modified devtest.sh we use for CI is at http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_devtest.sh - but that's CI, not CD | 00:40 |
| lifeless | greghaynes: kevinbenton is looking into the thing that broke overcloud gre networking | 00:40 |
| * jerryz reading | 00:41 | |
| lifeless | jerryz: tchaypo: meeet each other - jerryz is bringing up the huawei region | 00:41 |
| kevinbenton | lifeless: dprince mentioned in the patch that the heat auth errors were normal | 00:41 |
| tchaypo | jerryz: oh hey, you're my new favorite person! | 00:41 |
| lifeless | SpamapS: 101305 is ready for +A | 00:42 |
| kevinbenton | lifeless: so i need to figure out how it’s still breaking but for some reason all of the logging is gone now | 00:42 |
| lifeless | kevinbenton: righto so thats because we run defaults :) | 00:42 |
| lifeless | uhm | 00:42 |
| jerryz | tchaypo: nice to talk to you. going to have questions bugging you. | 00:43 |
| lifeless | kevinbenton: the review that we were using for this had a patch set to make everything default to debug, we don't have a generic knob for CI to say 'this run should be debug', but it sounds like a super useful thing to have | 00:43 |
| *** e0ne has joined #tripleo | 00:45 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Enable ip nonlocal binds https://review.openstack.org/101305 | 00:45 |
| jerryz | lifeless: sounds a potential zuul parameter to me. | 00:46 |
| clarkb | we run all tests as debug | 00:47 |
| clarkb | do you guys not do that? | 00:47 |
| greghaynes | clarkb: we dont | 00:47 |
| *** e0ne has quit IRC | 00:48 | |
| lifeless | clarkb: part of running defaults | 00:48 |
| tchaypo | Dear cores - after yesterday's adventure, https://review.openstack.org/#/c/92749/ is back to having one +2 and a couple of +1s; the only change from the last time it had 3 +2s is that I've merged it with some changes gfidente made. Your attention would be appreciated | 00:48 |
| jerryz | clarkb: a knob would be great or just debug open for rechecks. that would also reduce log volumes | 00:48 |
| lifeless | clarkb: it means we get to show devs what their logging /actually is like/ | 00:48 |
| clarkb | lifeless: we do that by only indexing > debug | 00:49 |
| clarkb | so if you want to take advantage of tooling you need sufficient logging | 00:49 |
| lifeless | tchaypo: one +2. | 00:49 |
| lifeless | clarkb: we'd need to run non-default to do that. | 00:50 |
| lifeless | clarkb: which violates a fairly important principle. | 00:50 |
| lifeless | clarkb: we'd welcome a change to the defaults :) | 00:50 |
| clarkb | well defaults shouldn't be debug | 00:51 |
| clarkb | but imo you should always test with debug | 00:51 |
| jerryz | clarkb, lifeless: missed e-r part that rely on debug mode, never mind | 00:51 |
| clarkb | because well testing | 00:51 |
| lifeless | clarkb: depends what you're testing | 00:51 |
| lifeless | clarkb: if you can't analyse the fault remotely with the logs we get, the logs are insufficient, because thats production too | 00:51 |
| clarkb | I agree | 00:51 |
| clarkb | but you have to have a realistic approach to the problem | 00:52 |
| lifeless | clarkb: we do :) | 00:52 |
| clarkb | ok | 00:52 |
| tchaypo | lifeless: yes, that's all it has now. It had three but then I had to rebase.. | 00:54 |
| greghaynes | kevinbenton: https://review.openstack.org/#/c/100374/ is the "debug all the things" patch | 00:54 |
| greghaynes | that is one way to do it | 00:55 |
| SpamapS | IMO there's just no good guidelines for what should be DEBUG and would should be INFO | 00:55 |
| SpamapS | so people throw everything in DEBUG | 00:55 |
| lifeless | tchaypo: ah yeup. anyhow -1 | 00:55 |
| lifeless | SpamapS: there is a doc now on the wiki. | 00:55 |
| SpamapS | oh good | 00:56 |
| lifeless | SpamapS: the big issue IMO is that most devs are not ops. | 00:56 |
| lifeless | anyhow, I have to write this up. excuse me. | 00:56 |
| greghaynes | Reasons its good for us to do !debug | 00:56 |
| greghaynes | kevinbenton: the passthrough config is another way, if youre wanting something more permanent I would look into doing passthrough config, but for debugging editing the config like in the debug all the things patch might be easiest | 00:57 |
| *** dkehn has joined #tripleo | 00:57 | |
| lifeless | greghaynes: wht kevinbenton needs is his Neutron patch run in debug mode. | 00:58 |
| *** yamahata has joined #tripleo | 00:58 | |
| greghaynes | ah | 00:59 |
| greghaynes | kevinbenton: diskimage-builder/elements/source-repositories/README.md has some info on how to do a build with a specific ref, basically DIB_REPOREF_neutron=refs/..., and then you can edit the neutron.conf in triple-image-elements similar to review 100374 to enable debugging | 01:01 |
| greghaynes | if I understand what youre trying to do this time :p | 01:01 |
| lifeless | greghaynes: in CI:) | 01:02 |
| greghaynes | ugh | 01:02 |
| * greghaynes finds example | 01:02 | |
| lifeless | greghaynes: since I very much doubt kevinbenton has the activation energy to get setup on TripleO dev right now. | 01:02 |
| greghaynes | haha, that fair | 01:02 |
| *** chuckC has quit IRC | 01:02 | |
| lifeless | greghaynes: I suspect a toci patch to fiddle things is probably the most viable route | 01:03 |
| greghaynes | yep | 01:03 |
| lifeless | greghaynes: e.g. to bring in the neutron patch | 01:03 |
| *** eghobo has joined #tripleo | 01:05 | |
| *** eghobo has quit IRC | 01:05 | |
| *** eghobo has joined #tripleo | 01:05 | |
| *** morganfainberg has quit IRC | 01:07 | |
| *** morganfainberg_Z is now known as morganfainberg | 01:07 | |
| *** ddieterly has joined #tripleo | 01:08 | |
| *** bthurber has quit IRC | 01:11 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 01:13 |
| tchaypo | Could we achieve that witht he usual reverts mechanism | 01:20 |
| tchaypo | ? | 01:20 |
| greghaynes | see that patch ^ | 01:20 |
| greghaynes | I *think* that might work ;) | 01:20 |
| tchaypo | I was thinking of the temprevert section above. But you're right, we're not trying to revert anything here, we're trying to pin a particular change. | 01:22 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Move user's group membership setup in separate script https://review.openstack.org/93954 | 01:23 |
| tchaypo | Hrm. | 01:26 |
| tchaypo | I was going to say that I'd like to see a comment on the line above, as we do in the temprevert section, explaining why this line is here | 01:27 |
| tchaypo | but then the demon on my other shoulder told me "nonsense, that's what git blame is for" | 01:27 |
| greghaynes | haha | 01:27 |
| tchaypo | so now I'm not sure which to listen to | 01:27 |
| greghaynes | that shouldnt ever be merged | 01:27 |
| tchaypo | that'd be why it's W:-1 | 01:28 |
| tchaypo | Urr | 01:29 |
| tchaypo | https://wiki.openstack.org/w/index.php?title=TripleO&diff=0&oldid=56304 has made me aware that in our page we link to github repos rather than git.openstack | 01:29 |
| tchaypo | That seems... contrary to my expectations.. | 01:29 |
| greghaynes | fixitfixitfixit | 01:29 |
| SpamapS | lifeless: https://etherpad.openstack.org/p/tripleo-heat-chunking-plan | 01:31 |
| * tchaypo kicks off a run to test https://review.openstack.org/#/c/101438/3 | 01:31 | |
| greghaynes | oh great, CI fail due to cloud sha being invalid | 01:32 |
| StevenK | greghaynes: Haha. We should totally find the youtube clip of Fry doing that | 01:32 |
| greghaynes | :) | 01:33 |
| *** cwolferh has quit IRC | 01:34 | |
| *** cwolferh has joined #tripleo | 01:34 | |
| StevenK | There we go. | 01:34 |
| StevenK | tchaypo: www.youtube.com/watch?v=xhy7dXWjpAA | 01:35 |
| StevenK | Argh. Mirror sync: universe/p/percona-xtradb-cluster-5.5 last seen | 01:35 |
| gfidente | StevenK, at 3.36 localtime, that was hard | 01:36 |
| gfidente | hit me so badly I wanted to go to sleep crying for not having fixed much | 01:37 |
| StevenK | Haha | 01:37 |
| gfidente | :) | 01:37 |
| * gfidente salute everyone | 01:38 | |
| *** gfidente has quit IRC | 01:38 | |
| *** chuckC has joined #tripleo | 01:43 | |
| *** nosnos has joined #tripleo | 01:44 | |
| *** e0ne has joined #tripleo | 01:45 | |
| lifeless | SpamapS: some quick thoughts there. | 01:45 |
| SpamapS | lifeless: yes they were some quick thoughts | 01:46 |
| SpamapS | :) | 01:46 |
| lifeless | SpamapS: mine, I meant :) | 01:46 |
| *** eguz has joined #tripleo | 01:46 | |
| *** e0ne has quit IRC | 01:46 | |
| SpamapS | lifeless: the idea of not changing merge.py has crossed my mind. | 01:47 |
| *** e0ne has joined #tripleo | 01:47 | |
| SpamapS | lifeless: and agree the the all hosts I/O may be generally useful. | 01:48 |
| *** eghobo has quit IRC | 01:50 | |
| *** eguz has quit IRC | 01:51 | |
| *** e0ne has quit IRC | 01:52 | |
| SpamapS | lifeless: ty.. I will work backwards anyway and so I might find that I can just manually write the binding document and call make overcloud.yaml as you suggest. | 01:52 |
| *** nati_ueno has quit IRC | 01:53 | |
| lifeless | boom | 01:56 |
| lifeless | greghaynes: next HA review plox | 01:57 |
| StevenK | lifeless: I'm waiting to see if you open the sprint with "Welcome, thanks for coming. HA ALL THE THINGS!" | 01:58 |
| lifeless | s/.*/\1 NOW/ | 01:58 |
| StevenK | lifeless: I was pondering ALL THE THINGS RIGHT NOW, but didn't want to make you seem needy :-P | 01:59 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-specs: TripleO and backwards compatibility https://review.openstack.org/102081 | 01:59 |
| *** jp_at_hp has quit IRC | 01:59 | |
| lifeless | now, I need someone to express strong support for ^ right now, to set the tone up right | 02:00 |
| lifeless | also I need food | 02:00 |
| lifeless | so, -> that. | 02:00 |
| lifeless | SpamapS: ^ | 02:00 |
| greghaynes | looks like the two I wanted failed CI :/ | 02:00 |
| lifeless | the nova thing? | 02:02 |
| lifeless | bbiaw | 02:02 |
| greghaynes | That too, but https://review.openstack.org/#/c/83883/ and the patch it depends on | 02:02 |
| greghaynes | on, no, not the nova thing | 02:04 |
| greghaynes | one is dns fail and the other might be legit | 02:05 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Add outputs and inputs for all hosts https://review.openstack.org/102082 | 02:09 |
| SpamapS | lifeless: hrm. ControlVirtualIP complicates things. :-P | 02:13 |
| SpamapS | actually hm.. maybe I don't need to do an overcloud.yaml CONTROLSCALE=0 .. I should be able to just use nova-compute-instance.yaml | 02:14 |
| *** ddieterly has quit IRC | 02:15 | |
| tchaypo | lifeless: so just to check that I'm on the right track - I'm currently trying to verify that the change in https://review.openstack.org/#/c/101438/3 does what I expect (create a new VIP on the overcloud control nodes), then I'm going to do another change to make haproxy listen on that vip as well | 02:16 |
| tchaypo | lifeless: is that roughly what you were thinking? | 02:17 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Add ControlVirtualIP as an overcloud output https://review.openstack.org/102085 | 02:18 |
| *** morazi has quit IRC | 02:23 | |
| *** mestery has quit IRC | 02:31 | |
| *** mestery has joined #tripleo | 02:31 | |
| tchaypo | SpamapS: this seems relevant to my interests | 02:33 |
| *** pcrews has quit IRC | 02:34 | |
| tchaypo | SpamapS: in 101438 I'm adding a new VIP to be used for things that want to be accessible from outside the cluster. I'm guessing I should add it as an output too? | 02:34 |
| tchaypo | If I understand heat correctly, an "output" can be used as "input" elsewhere in the template, right? | 02:34 |
| tchaypo | So I'd be adding it as an output so that eg the bit of the heat template that sets up haproxy could consume it, is what I'm thinking | 02:36 |
| *** killer_prince is now known as lazy_prince | 02:42 | |
| *** lazy_prince is now known as killer_prince | 02:42 | |
| lifeless | tchaypo: sounds like it | 02:42 |
| lifeless | tchaypo: I don't think clints work is relevant to your work | 02:43 |
| * tchaypo does some reading about heat | 02:43 | |
| lifeless | tchaypo: outputs are for inspecting output of the template-as-progra | 02:43 |
| lifeless | program | 02:43 |
| tchaypo | how do you do that inspection? | 02:43 |
| *** killer_prince is now known as lazy_prince | 02:43 | |
| tchaypo | looking at https://review.openstack.org/#/c/100848 | 02:45 |
| *** lazy_prince is now known as killer_prince | 02:45 | |
| *** e0ne has joined #tripleo | 02:45 | |
| tchaypo | if I'm understanding correctly - the fn:getatt says to look in "NovaCompute0" for the attribute "networks"; then the fn::select picks out just the attribute of networks called ctlplane; then the fn::select picks out just the 0th element of ctlplane | 02:45 |
| tchaypo | is that roughtly right? | 02:46 |
| *** e0ne has quit IRC | 02:49 | |
| *** ddieterly has joined #tripleo | 02:49 | |
| *** mestery_ has joined #tripleo | 02:52 | |
| *** nosnos has quit IRC | 02:54 | |
| *** ddieterly has quit IRC | 02:54 | |
| *** nosnos has joined #tripleo | 02:54 | |
| *** mestery has quit IRC | 02:55 | |
| *** nosnos has quit IRC | 02:59 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332458 | 03:00 |
| uvirtbot | Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332521 | 03:00 |
| uvirtbot | Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| lifeless | tchaypo: that review doesn't have outputs | 03:00 |
| tchaypo | 100848? I know, I'm haring off on a tangent looking at heat template changes to learn more about it | 03:01 |
| lifeless | gets | 03:01 |
| lifeless | yes | 03:01 |
| tchaypo | Fn::Join seems pretty simple. OS::Heat::RandomString looks handy. | 03:02 |
| *** vinsh has joined #tripleo | 03:02 | |
| *** e0ne has joined #tripleo | 03:02 | |
| tchaypo | When you use ref: to do a lookup... I'm going to have to figure out the scoping rules later. | 03:03 |
| *** nosnos has joined #tripleo | 03:05 | |
| tchaypo | okay steamophiles, the next "community choice" group has only one correct answer - it's the group with kerbal space program at -40% and simcity4 for $5 | 03:07 |
| *** e0ne has quit IRC | 03:07 | |
| *** akuznetsov has joined #tripleo | 03:11 | |
| *** ddieterly has joined #tripleo | 03:11 | |
| SpamapS | tchaypo: in the REST API to get the outputs you fetch the stack | 03:13 |
| fabbione | lifeless: ping | 03:15 |
| SpamapS | tchaypo: ugh.. that is a choice that leads to me never leaving the house for 3 weeks. | 03:15 |
| *** ddieterly has quit IRC | 03:16 | |
| tchaypo | SpamapS: sounds good, right? | 03:16 |
| tchaypo | When I'm using the command-line client and I "heat stack-show undercloud" and there's one part of the output that has a list of key:value pairs - those are the outputs? | 03:17 |
| SpamapS | tchaypo: I can't even begin to tell you how poisonous that would be to my life. ;) | 03:17 |
| tchaypo | I can't build anything today | 03:17 |
| tchaypo | + heat stack-create -e /home/polleyj/.cache/tripleo/undercloud-env.json -t 360 -f /home/polleyj/.cache/tripleo/tripleo-heat-templates/undercloud-vm-ironic.yaml undercloud | 03:17 |
| tchaypo | ERROR: The Parameter (PowerManager) was not defined in template. | 03:17 |
| * SpamapS is a recovering gamaholic | 03:17 | |
| tchaypo | please tell me I'm not the only one seeing that.. | 03:18 |
| SpamapS | you're not the only one seeing that | 03:18 |
| * SpamapS lies | 03:18 | |
| *** matsuhashi has quit IRC | 03:18 | |
| *** matsuhashi has joined #tripleo | 03:19 | |
| mordred | SpamapS: you were addicted to gama ? | 03:21 |
| *** matsuhashi has quit IRC | 03:24 | |
| *** ramishra has joined #tripleo | 03:27 | |
| lifeless | fabbione: pong | 03:29 |
| lifeless | tchaypo: you've got a non-ironic parameter in your heat env | 03:30 |
| lifeless | tchaypo: and heat no like that | 03:30 |
| tchaypo | and importantly | 03:31 |
| tchaypo | ~/.devtestrc has USE_IRONIC=1 | 03:31 |
| * greghaynes is sad https://review.openstack.org/#/c/93104/ got abandoned | 03:31 | |
| greghaynes | given that its needed for HA and the one it was abandoned for looks not close to mergable | 03:31 |
| * tchaypo turns that off and tries again | 03:32 | |
| *** nosnos has quit IRC | 03:34 | |
| *** nosnos has joined #tripleo | 03:35 | |
| *** ramishra_ has joined #tripleo | 03:37 | |
| *** eghobo has joined #tripleo | 03:38 | |
| *** ramishra has quit IRC | 03:38 | |
| *** nosnos has quit IRC | 03:39 | |
| *** e0ne has joined #tripleo | 03:45 | |
| greghaynes | tchaypo: likely caused by changing that parameter without rm'ing your -env.json | 03:47 |
| greghaynes | I have been meaning to write up a bug report for that | 03:47 |
| *** jml has quit IRC | 03:47 | |
| *** CaptTofu_ has quit IRC | 03:48 | |
| lifeless | ok, offlineish | 03:48 |
| *** e0ne has quit IRC | 03:48 | |
| *** michchap has quit IRC | 03:52 | |
| *** michchap has joined #tripleo | 03:52 | |
| *** jml has joined #tripleo | 03:55 | |
| *** GheRivero has quit IRC | 04:04 | |
| *** davidlenwell_ has joined #tripleo | 04:04 | |
| *** GheRivero has joined #tripleo | 04:04 | |
| *** davidlenwell has quit IRC | 04:05 | |
| *** SlickNik has quit IRC | 04:05 | |
| *** SlickNik has joined #tripleo | 04:06 | |
| *** matsuhashi has joined #tripleo | 04:10 | |
| *** ddieterly has joined #tripleo | 04:11 | |
| *** ddieterly has quit IRC | 04:16 | |
| *** nosnos has joined #tripleo | 04:19 | |
| *** killer_prince is now known as lazy_prince | 04:24 | |
| *** e0ne has joined #tripleo | 04:25 | |
| kevinbenton | lifeless, greghaynes: sorry, I had to leave in a rush earlier for a meeting | 04:29 |
| kevinbenton | lifeless, greghaynes: what was the consensus on the best way to have the CI test my neutron patch on tripleo with debug enabled? | 04:29 |
| *** e0ne has quit IRC | 04:30 | |
| greghaynes | kevinbenton: once our CI is unwedged from a new trusty image I think https://review.openstack.org/#/c/102073/ will work | 04:37 |
| kevinbenton | greghaynes: cool. what does unwedged mean in this context? | 04:38 |
| greghaynes | kevinbenton: a new trusty cloudimg got released, I think were caching one (but not both) of the image or sha256sums... so we keep failing tests with invalid sha | 04:40 |
| *** lazy_prince has quit IRC | 04:41 | |
| *** morganfainberg_L has joined #tripleo | 04:41 | |
| kevinbenton | greghaynes: oh okay. is this something that will clear up soon or should I check back tomorrow? | 04:41 |
| lifeless | I think I can clear it up, will take a minute | 04:42 |
| kevinbenton | lifeless: cool. thanks | 04:42 |
| lifeless | there is a patchset which does cache-busting requests | 04:42 |
| lifeless | if I can find it... doesn't look like its merged | 04:46 |
| lifeless | 88084 | 04:46 |
| lifeless | so thats rechecking now | 04:46 |
| lifeless | in a few minutes re-run greghaynes's one. It will be behind the other and should DTRT :) | 04:47 |
| kevinbenton | DTRT? | 04:49 |
| StevenK | kevinbenton: Do The Right Thing | 04:49 |
| *** pcrews has joined #tripleo | 04:49 | |
| kevinbenton | oooh, a new acronym :-) | 04:50 |
| lifeless | Ng: ^ might want to finish that patch off :> in your copious spare time | 04:51 |
| *** rakesh_hs4 has joined #tripleo | 04:57 | |
| *** nati_ueno has joined #tripleo | 04:58 | |
| *** eghobo has quit IRC | 05:00 | |
| *** eghobo has joined #tripleo | 05:01 | |
| *** CaptTofu_ has joined #tripleo | 05:01 | |
| *** bmahalakshmi has joined #tripleo | 05:02 | |
| *** cwolferh has quit IRC | 05:06 | |
| *** CaptTofu_ has quit IRC | 05:06 | |
| *** ajc_ has joined #tripleo | 05:07 | |
| kevinbenton | lifeless: do you have a link to some logs from a recent job that passed? I want to compare some outputs | 05:07 |
| *** lazy_prince has joined #tripleo | 05:09 | |
| *** ddieterly has joined #tripleo | 05:11 | |
| *** ddieterly has quit IRC | 05:16 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Separate Overcloud into plan and stack https://review.openstack.org/102106 | 05:16 |
| *** lazy_prince is now known as killer_prince | 05:22 | |
| *** amitpp has joined #tripleo | 05:29 | |
| *** rwsu has quit IRC | 05:41 | |
| *** gcha has quit IRC | 05:49 | |
| *** morganfainberg_L has quit IRC | 05:51 | |
| *** rdopieralski has joined #tripleo | 06:02 | |
| *** robsparker has quit IRC | 06:04 | |
| *** ddieterly has joined #tripleo | 06:11 | |
| tchaypo | That moment when people insist you remove the "as of $DATE we support" from your patch, so you do, then people say "this should say "as of $DATE"" | 06:12 |
| tchaypo | That other moment when you realise you have nested ""s | 06:12 |
| *** dshulyak_ has joined #tripleo | 06:15 | |
| *** killer_prince is now known as lazy_prince | 06:15 | |
| *** ddieterly has quit IRC | 06:16 | |
| *** ddieterly has joined #tripleo | 06:16 | |
| *** ddieterly has quit IRC | 06:21 | |
| tchaypo | oh wait | 06:22 |
| tchaypo | easycount is produced by the AEC, not by a third party? | 06:22 |
| tchaypo | fuck this "commercial in confidence" shit then. that's a fucking crock. | 06:22 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Add details of which OS releases are tested in CI https://review.openstack.org/92749 | 06:25 |
| StevenK | tchaypo: AEC is a commision, so has no commercial interests, right? | 06:25 |
| tchaypo | huh. | 06:25 |
| * tchaypo forgot to switch windows | 06:26 | |
| StevenK | Haha | 06:26 |
| tchaypo | StevenK: incorrect, they run elections for profit | 06:26 |
| tchaypo | according to latest post on linux-aus, total of like 1.5 million in profit each year, compared with 160 million to run a federal election | 06:26 |
| StevenK | All elections, or the state/federal ones are not for profit? | 06:26 |
| *** mrunge has joined #tripleo | 06:28 | |
| *** jang has joined #tripleo | 06:30 | |
| *** jml has quit IRC | 06:31 | |
| *** jml has joined #tripleo | 06:33 | |
| *** gcha has joined #tripleo | 06:38 | |
| *** gfidente has joined #tripleo | 06:44 | |
| *** ramishra_ has quit IRC | 06:44 | |
| *** ramishra has joined #tripleo | 06:46 | |
| StevenK | Woo, progress. | 06:46 |
| tchaypo | the state and federal ones are not for profit | 06:46 |
| StevenK | My setup-neutron in os-cloud-config can create nets. | 06:46 |
| tchaypo | they're required by law to run some of the union elections; not sure if they're for profit or not | 06:46 |
| *** CaptTofu_ has joined #tripleo | 06:49 | |
| gfidente | guys, still there? | 06:50 |
| gfidente | I left you a while back | 06:50 |
| tchaypo | gfidente: welcome back | 06:51 |
| tchaypo | gfidente: i hope you got some sleep | 06:51 |
| gfidente | I did! | 06:51 |
| tchaypo | gfidente: but your comment made me cry. | 06:51 |
| gfidente | have to buy some cofee though | 06:51 |
| tchaypo | you don't have coffee supplies at home? | 06:51 |
| tchaypo | ... are you at home? | 06:51 |
| gfidente | those are finished | 06:51 |
| tchaypo | speaking of supplies at home, i need to contact my dealer | 06:51 |
| gfidente | tchaypo, you mean the comment on which distro version we support? | 06:52 |
| tchaypo | yeah, the comment suggestiong we should have "as of $DATE ..." | 06:52 |
| tchaypo | not your fault, it's how i wanted it, but consensus seems to be that it's better without | 06:52 |
| gfidente | wait let me check the reviews :) | 06:53 |
| gfidente | I wanted to ask though if you know anything about SHA failing for the ubuntu images? | 06:53 |
| tchaypo | patchset 8 is where it changed :) | 06:53 |
| tchaypo | oh, i think greghaynes said something about that | 06:53 |
| *** ifarkas has joined #tripleo | 06:53 | |
| *** CaptTofu_ has quit IRC | 06:54 | |
| gfidente | tchaypo, oh I saw the comments on patch 8 | 06:56 |
| gfidente | I think I can be happy if we use Currently too :) | 06:57 |
| tchaypo | phew | 06:58 |
| *** eguz has joined #tripleo | 06:59 | |
| tchaypo | gfidente: if you have a momeny, I'd appreciate a review on https://review.openstack.org/#/c/97657/5 | 07:01 |
| tchaypo | bnemec and i both feel it's too early to be breaking backwards compatibility, | 07:02 |
| *** eghobo has quit IRC | 07:03 | |
| tchaypo | but we were involved in putting the code in its current state, maybe we're being too conservative | 07:04 |
| *** eguz has quit IRC | 07:05 | |
| gfidente | tchaypo, breaking backwards means you're not sourcing the file | 07:09 |
| gfidente | but script is not overwriting the passwords anyway | 07:10 |
| *** openstackgerrit has quit IRC | 07:10 | |
| tchaypo | not sourcing the file in $PWD; sure; but that only breaks people who were using devtest before my patch in march, and had password files created somewhere else, and are used to using them | 07:10 |
| tchaypo | his script doesn't overwrite the passwords; t:q | 07:11 |
| gfidente | so the thing is that I think we should make the passwords customizable | 07:11 |
| tchaypo | bnemec tweaked the scripts so that for any password already in the file it will be left untouched, but any new passwords added to the script will be appended to the file | 07:12 |
| gfidente | yep which I think is great | 07:12 |
| tchaypo | so calling it on an exisitng file will touch nothing, but when we have changes such as the recent one that added a ceilometer password, they get added to existing files | 07:12 |
| *** pblaho has joined #tripleo | 07:13 | |
| gfidente | so I am of the idea that this is an actual feature, not only something good for compatibility | 07:13 |
| gfidente | but I'd make it more explicit | 07:13 |
| * tchaypo waits for your comment on review | 07:14 | |
| gfidente | I'd write the file in a location which the user can customize, but keep the change | 07:14 |
| gfidente | ok sorry | 07:14 |
| gfidente | commenting there | 07:14 |
| tchaypo | and have $TRIPLEO_ROOT be the default location? | 07:14 |
| gfidente | yes | 07:15 |
| tchaypo | I can get behind that. It means that existing devs with existing files will have to, just once, move their files into $TRIPLEO_ROOT, or export something in their ~/.devtestrc | 07:15 |
| *** nati_ueno has quit IRC | 07:15 | |
| gfidente | yes I understand that, but in its current state it is sort of 'hidden' while I think it is a good feature to have | 07:16 |
| tchaypo | _1 | 07:18 |
| tchaypo | urr | 07:18 |
| tchaypo | +1 also | 07:18 |
| tchaypo | if I can type | 07:18 |
| tchaypo | in other news it's time for me to head pubwards | 07:18 |
| *** jcoufal has joined #tripleo | 07:21 | |
| *** beekneemech has joined #tripleo | 07:32 | |
| *** bnemec has quit IRC | 07:32 | |
| *** pblaho is now known as pblaho|afk | 07:34 | |
| *** dshulyak_ has quit IRC | 07:36 | |
| *** jprovazn has joined #tripleo | 07:37 | |
| *** andreaf has joined #tripleo | 07:39 | |
| *** jtomasek has joined #tripleo | 07:45 | |
| *** pblaho|afk is now known as pblaho | 07:52 | |
| *** jp_at_hp has joined #tripleo | 08:05 | |
| *** amitpp has quit IRC | 08:12 | |
| *** fandi has joined #tripleo | 08:14 | |
| *** akrivoka has joined #tripleo | 08:19 | |
| *** dtantsur|afk is now known as dtantsur | 08:24 | |
| *** bogdando has quit IRC | 08:29 | |
| *** ramishra has quit IRC | 08:30 | |
| *** ramishra has joined #tripleo | 08:32 | |
| *** jcoufal has quit IRC | 08:33 | |
| *** jcoufal has joined #tripleo | 08:34 | |
| *** athomas has joined #tripleo | 08:34 | |
| *** CaptTofu_ has joined #tripleo | 08:38 | |
| *** ramishra has quit IRC | 08:38 | |
| *** CaptTofu_ has quit IRC | 08:42 | |
| *** martyntaylor has joined #tripleo | 08:42 | |
| *** martyntaylor has quit IRC | 08:46 | |
| *** martyntaylor has joined #tripleo | 08:50 | |
| *** pblaho is now known as pblaho|afk | 08:59 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332458 | 09:00 |
| uvirtbot | Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332521 | 09:00 |
| uvirtbot | Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| *** pblaho|afk is now known as pblaho | 09:08 | |
| *** e0ne has joined #tripleo | 09:10 | |
| *** akrivoka has quit IRC | 09:11 | |
| *** pelix has joined #tripleo | 09:16 | |
| *** ramishra has joined #tripleo | 09:23 | |
| Ng | morning | 09:23 |
| * Ng hopes his neighbour's kids continue to practice their basic violin at this volume all day, it's extremely soothing to hear someone violate a shrill musical instrument when you've managed about 3 hours of sleep | 09:23 | |
| Ng | hang on, nope, that's not right at all ;) | 09:24 |
| *** rdopieralski has quit IRC | 09:24 | |
| *** ramishra has quit IRC | 09:28 | |
| StevenK | Ng: Sarah is learning the violin, and it sounds rather lovely when she practises :-) | 09:31 |
| Ng | StevenK: she's not 6 and physically incapable of playing a violin nicely :) | 09:31 |
| Ng | I now feel bad for all the people I annoyed when I was learning violin as a little kid :) | 09:32 |
| StevenK | Ng: When are you buying a quarter size violin for your eldest as payback? :-) | 09:33 |
| Ng | StevenK: never, I will encourage my kids to learn things that can be trivially electronicsised and thus played with headphones ;) | 09:33 |
| Ng | piano, drums, that sort of thing | 09:33 |
| StevenK | Electronic violins also exist | 09:33 |
| Ng | they do indeed, but I'm not sure there are cheap ones | 09:34 |
| Ng | but it's a lot of years since I was into violin | 09:34 |
| StevenK | Musical instrument and cheap and diametrically opposed concepts. | 09:35 |
| StevenK | s/cheap and/cheap are/ | 09:35 |
| *** akrivoka has joined #tripleo | 09:35 | |
| Ng | I'm not sure I agree, cheap keyboards aren't unplayable, and the cheap electronic drumkits are really surprisingly good these days | 09:35 |
| Ng | when I was taking drum lessons, they were thousands of pounds, but you can get an entirely acceptable kit for under £200 now | 09:36 |
| StevenK | Huh | 09:36 |
| StevenK | One of the friends of the family had a cheap electronic drumkit when I was growing up, and it was utterly terrible. | 09:37 |
| Ng | I'm actually hoping I can get the kids into drumming, because I really miss playing and this house is the first time in a couple of decades that I've legitimately had space to own a kit :) | 09:38 |
| StevenK | It's London, lies. :-P | 09:38 |
| Ng | hehe | 09:39 |
| Ng | StevenK: well once the kids are a bit older, I can start taking over their play room with more suitable toys :D | 09:39 |
| Ng | games consoles, drumkits | 09:39 |
| Ng | programmable lego! | 09:39 |
| StevenK | Hahaha | 09:40 |
| Ng | chemistry sets! | 09:40 |
| *** bmahalakshmi has quit IRC | 09:41 | |
| StevenK | Oooh, what could possibly go wrong | 09:41 |
| Ng | nothing* | 09:41 |
| *** amitpp has joined #tripleo | 09:41 | |
| Ng | Rike will hate it, but I will claim that all of the above are good for the kids ;) | 09:41 |
| StevenK | How do you get Copper Sulfate stains out of the carpet? :-P | 09:42 |
| *** bmahalakshmi has joined #tripleo | 09:43 | |
| Ng | StevenK: easy, we don't have carpet on that floor of the house ;) | 09:44 |
| StevenK | Ng: Tiles, floorboards, *shiver* lino? | 09:44 |
| Ng | StevenK: the funky interlocking boards | 09:44 |
| Ng | one continuous, smooth surface across four rooms and a hallway \o/ | 09:45 |
| StevenK | Wait | 09:45 |
| gfidente | so I have my son (who is 3) playing a lot with a kalimba and the kaossilator | 09:45 |
| StevenK | Give it 5 to 7 years, and it probably won't be so smooth | 09:45 |
| gfidente | I'm not sure the kaossilator was a good move | 09:46 |
| Ng | StevenK: probably true :) | 09:46 |
| Ng | gfidente: ♥ kalimbas | 09:46 |
| StevenK | dict can't spell anything with those two words | 09:46 |
| gfidente | Ng, indeed it is a very nice sounding thing and he enjoys the sound it makes | 09:47 |
| gfidente | the kaossilator instead, drives us all crazy | 09:47 |
| Ng | gfidente: I think I shall get one for my 3 year old :) | 09:47 |
| *** openstackgerrit has joined #tripleo | 09:47 | |
| gfidente | yeah they make a really nice sound | 09:47 |
| gfidente | and are kind of "easy" to use | 09:48 |
| *** dtantsur is now known as dtantsur|lunch | 09:50 | |
| openstackgerrit | Chris Jones proposed a change to openstack/diskimage-builder: Last ditch effort to correct a wrong shasum. https://review.openstack.org/88084 | 09:53 |
| *** fandi has quit IRC | 09:54 | |
| *** rcarrillocruz has joined #tripleo | 10:00 | |
| *** rcarrill` has quit IRC | 10:03 | |
| *** hewbrocca has joined #tripleo | 10:05 | |
| *** athomas has quit IRC | 10:14 | |
| *** nosnos has quit IRC | 10:19 | |
| gfidente | guys, regarding the overcloud networking, I noticed instances get by default a float on the same ctlplane we use for undercloud/overcloud interconnection | 10:19 |
| *** nosnos has joined #tripleo | 10:19 | |
| gfidente | I'm not sure if this is customizable though, is it possible to avoid that? anyone has a pointer to the relevant settings? | 10:20 |
| *** athomas has joined #tripleo | 10:23 | |
| *** matsuhashi has quit IRC | 10:23 | |
| *** matsuhashi has joined #tripleo | 10:24 | |
| *** nosnos has quit IRC | 10:24 | |
| *** amitpp has quit IRC | 10:24 | |
| *** CaptTofu_ has joined #tripleo | 10:26 | |
| *** matsuhashi has quit IRC | 10:28 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Check if stunnel.connect_ip is set https://review.openstack.org/102177 | 10:29 |
| *** CaptTofu_ has quit IRC | 10:31 | |
| *** e0ne has quit IRC | 10:33 | |
| *** markmc has joined #tripleo | 10:33 | |
| *** pblaho is now known as pblaho|afk | 10:41 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-image-elements: Allowing to optionally run sensors-detect https://review.openstack.org/102182 | 10:47 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-heat-templates: Add parameters for setting up keystone keys/certs in undercloud https://review.openstack.org/102183 | 10:56 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-incubator: Generate overcloud keystone keys/certs https://review.openstack.org/95101 | 10:57 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Require and set mode 0600 on heat env caches https://review.openstack.org/100994 | 11:00 |
| *** e0ne has joined #tripleo | 11:01 | |
| *** jprovazn has quit IRC | 11:07 | |
| *** rdopieralski has joined #tripleo | 11:11 | |
| TheJulia | good morning | 11:15 |
| *** xuhaiwei has quit IRC | 11:16 | |
| *** yamahata has quit IRC | 11:18 | |
| lazy_prince | please review https://review.openstack.org/#/c/79289/ (Adds local storage Boot support on PXE failures). It has under gone multiple review and is now ready for merge.. | 11:21 |
| *** bmahalakshmi has quit IRC | 11:26 | |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-specs: Remove merge.py from TripleO Heat Templates https://review.openstack.org/97939 | 11:28 |
| *** bmahalakshmi has joined #tripleo | 11:29 | |
| *** jprovazn has joined #tripleo | 11:31 | |
| *** pblaho|afk is now known as pblaho | 11:36 | |
| *** bmahalakshmi has quit IRC | 11:43 | |
| *** bmahalakshmi has joined #tripleo | 11:44 | |
| *** ddieterly has joined #tripleo | 11:50 | |
| *** e0ne_ has joined #tripleo | 11:53 | |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 | 11:54 |
| ekarlso | lifeless: greghaynes and who else it concerned ^ | 11:55 |
| *** e0ne has quit IRC | 11:56 | |
| *** weshay has joined #tripleo | 11:57 | |
| *** ramishra has joined #tripleo | 11:59 | |
| *** CaptTofu_ has joined #tripleo | 11:59 | |
| *** morazi has joined #tripleo | 12:00 | |
| *** rakesh_hs4 has quit IRC | 12:01 | |
| *** jdob has joined #tripleo | 12:01 | |
| *** rlandy has joined #tripleo | 12:07 | |
| *** ddieterly has quit IRC | 12:16 | |
| *** dprince has joined #tripleo | 12:19 | |
| *** dtantsur|lunch is now known as dtantsur | 12:20 | |
| *** bmahalakshmi has quit IRC | 12:21 | |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 | 12:31 |
| *** edmund1 has joined #tripleo | 12:36 | |
| *** dshulyak_ has joined #tripleo | 12:39 | |
| pblaho | d0ugal: ping | 12:46 |
| d0ugal | pblaho: hey | 12:46 |
| pblaho | d0ugal: hi | 12:47 |
| pblaho | will you be ok if I do +A on https://review.openstack.org/#/c/94523/ if gates pass? | 12:47 |
| d0ugal | pblaho: sure | 12:48 |
| d0ugal | pblaho: my suggestion was only minor, which is why I went with a +1 still :) | 12:48 |
| pblaho | d0ugal: ok... I was just asking ... | 12:48 |
| d0ugal | thanks! | 12:48 |
| pblaho | d0ugal: yeah... | 12:48 |
| pblaho | np | 12:49 |
| * pblaho hopes gates will be ok | 12:49 | |
| *** ajc_ has quit IRC | 12:50 | |
| gfidente | did we fix the ubuntu SHA mismatch? am I safe to recheck? | 12:53 |
| *** dshulyak_ has quit IRC | 12:56 | |
| *** julim has joined #tripleo | 12:59 | |
| *** ddieterly has joined #tripleo | 12:59 | |
| *** sseago has quit IRC | 12:59 | |
| *** sseago_ has joined #tripleo | 12:59 | |
| *** Matt1 has joined #tripleo | 13:00 | |
| *** lazy_prince has quit IRC | 13:13 | |
| *** andreaf has quit IRC | 13:13 | |
| *** sseago__ has joined #tripleo | 13:20 | |
| *** sseago_ has quit IRC | 13:20 | |
| *** pblaho_ has joined #tripleo | 13:30 | |
| *** pblaho has quit IRC | 13:30 | |
| *** casanch1 has joined #tripleo | 13:31 | |
| *** ramishra has quit IRC | 13:38 | |
| *** ramishra has joined #tripleo | 13:39 | |
| *** ramishra has quit IRC | 13:39 | |
| *** ramishra has joined #tripleo | 13:39 | |
| *** pblaho__ has joined #tripleo | 13:41 | |
| *** pblaho__ is now known as pblaho | 13:41 | |
| *** mestery_ is now known as mestery | 13:42 | |
| *** pblaho_ has quit IRC | 13:45 | |
| *** pblaho is now known as pblaho|afk | 13:52 | |
| *** martyntaylor has left #tripleo | 13:56 | |
| *** pblaho|afk is now known as pblaho | 13:57 | |
| *** pblaho has quit IRC | 13:58 | |
| lifeless | morning | 14:04 |
| *** beekneemech is now known as bnemec | 14:04 | |
| jdob | lifeless: are you traveling right now? isn't it like the middle of the night where you are? | 14:08 |
| jdob | also, morning :) | 14:08 |
| *** edmund1 has quit IRC | 14:12 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 | 14:16 |
| lifeless | jdob: I'm in sunnyvale right now | 14:16 |
| jdob | ahh, that makes more sense :) | 14:16 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 | 14:17 |
| *** cwolferh has joined #tripleo | 14:30 | |
| *** killer_prince has joined #tripleo | 14:33 | |
| *** andreaf has joined #tripleo | 14:37 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified https://review.openstack.org/102018 | 14:40 |
| *** killer_prince has quit IRC | 14:40 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 | 14:44 |
| *** killer_prince has joined #tripleo | 14:44 | |
| *** jprovazn is now known as jprovazn_afk | 14:46 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration when UI not specified https://review.openstack.org/102018 | 14:52 |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Fix Ceilometer undercloud endpoint registration https://review.openstack.org/102018 | 14:53 |
| *** bthurber has joined #tripleo | 14:54 | |
| *** morganfainberg_L has joined #tripleo | 14:58 | |
| *** andrearosa has quit IRC | 14:59 | |
| *** nati_ueno has joined #tripleo | 15:00 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332458 | 15:00 |
| uvirtbot | Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1333649 | 15:00 |
| uvirtbot | Launchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332521 | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] | 15:00 |
| *** chuckC has quit IRC | 15:01 | |
| *** marun has joined #tripleo | 15:01 | |
| *** casanch1 has quit IRC | 15:11 | |
| *** jcoufal has quit IRC | 15:11 | |
| *** ramishra has quit IRC | 15:19 | |
| *** ramishra has joined #tripleo | 15:30 | |
| gfidente | all, are there plans to switch from mysql-binary to the distro packaged versions of mysql and/or mariadb? | 15:32 |
| *** marun has quit IRC | 15:33 | |
| *** marun has joined #tripleo | 15:37 | |
| gfidente | lifeless ^^ ? | 15:40 |
| lifeless | gfidente: I think you can do a build like that if you want to, there's certainly been patch sets up to permit that | 15:41 |
| gfidente | lifeless, yeah I've seen elements for packaged mariadb as well | 15:42 |
| gfidente | but is there a plan to switch per default? | 15:43 |
| lifeless | I don't think so | 15:45 |
| lifeless | percona pacakges aren't available everywhere yet | 15:45 |
| gfidente | got it, thanks | 15:46 |
| *** ramishra has quit IRC | 15:49 | |
| *** ramishra has joined #tripleo | 15:50 | |
| *** chuckC has joined #tripleo | 15:52 | |
| openstackgerrit | Kevin Benton proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 15:53 |
| *** eghobo has joined #tripleo | 15:53 | |
| *** eghobo has quit IRC | 15:53 | |
| *** ramishra has quit IRC | 15:54 | |
| *** eghobo has joined #tripleo | 15:55 | |
| *** jtomasek has quit IRC | 15:55 | |
| kevinbenton | lifeless: shouldn’t this line be gone now? https://github.com/openstack-infra/tripleo-ci/blob/master/toci_devtest.sh#L35 | 15:56 |
| *** yamahata has joined #tripleo | 15:59 | |
| lifeless | kevinbenton: ah - the code for that checks the commit # and merge logic, but yes, lets remove it | 15:59 |
| kevinbenton | lifeless: does that code affect jobs run by ‘check experimental'? | 16:00 |
| lifeless | kevinbenton: everything but is has some smarts. | 16:00 |
| openstackgerrit | lifeless proposed a change to openstack-infra/tripleo-ci: Unrevert Neutron's ovs-agent patch https://review.openstack.org/102281 | 16:01 |
| vinsh | re-un-revert. ;) | 16:01 |
| lifeless | kevinbenton: you can however rebase your test patch on top of ^ to ensure you're not affected by it | 16:01 |
| *** eghobo has quit IRC | 16:01 | |
| lifeless | Ihave a call now, sorry | 16:01 |
| *** ramishra has joined #tripleo | 16:01 | |
| *** eghobo has joined #tripleo | 16:01 | |
| kevinbenton | lifeless: no prob. i just commented it out on mine | 16:01 |
| *** martyntaylor has joined #tripleo | 16:07 | |
| *** martyntaylor has quit IRC | 16:11 | |
| *** dtantsur is now known as dtantsur|afk | 16:16 | |
| *** nati_ueno has quit IRC | 16:17 | |
| *** e0ne_ has quit IRC | 16:18 | |
| *** andreaf has quit IRC | 16:18 | |
| *** andreaf has joined #tripleo | 16:19 | |
| *** rdopieralski has quit IRC | 16:31 | |
| *** martyntaylor has joined #tripleo | 16:31 | |
| *** martyntaylor has left #tripleo | 16:32 | |
| *** yjiang5 is now known as yjiang5_away | 16:33 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Allow use of dib-utils https://review.openstack.org/100003 | 16:39 |
| *** jcoufal has joined #tripleo | 16:41 | |
| *** noslzzp has quit IRC | 16:42 | |
| *** noslzzp has joined #tripleo | 16:43 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: avoid failure if /lib/firmware doesn't exist https://review.openstack.org/100026 | 16:46 |
| *** ramishra has quit IRC | 16:49 | |
| *** rcarrill` has joined #tripleo | 16:51 | |
| *** rcarrillocruz has quit IRC | 16:51 | |
| *** yjiang5_away is now known as yjiang5 | 16:51 | |
| *** pelix has quit IRC | 16:54 | |
| *** athomas has quit IRC | 16:56 | |
| *** ramishra has joined #tripleo | 16:56 | |
| *** dshulyak_ has joined #tripleo | 16:58 | |
| *** mrunge has quit IRC | 17:02 | |
| *** dtantsur|afk is now known as dtantsur | 17:06 | |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 | 17:07 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's https://review.openstack.org/93039 | 17:12 |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 | 17:12 |
| *** akrivoka has quit IRC | 17:15 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Do not use cat in sysctl-set-value https://review.openstack.org/101116 | 17:18 |
| *** martyntaylor1 has joined #tripleo | 17:21 | |
| greghaynes | jprovazn_afk: you seee my note on https://review.openstack.org/#/c/93104/ ? | 17:25 |
| *** rwsu has joined #tripleo | 17:25 | |
| *** nealph has quit IRC | 17:26 | |
| ekarlso | greghaynes: u got time to signoff to changes ? ;9 | 17:27 |
| greghaynes | referring to 102198? | 17:27 |
| ekarlso | yah, for one | 17:28 |
| greghaynes | I have it open :) | 17:28 |
| ekarlso | :) | 17:29 |
| ekarlso | https://review.openstack.org/#/c/102292/ this also | 17:29 |
| ekarlso | should fix issues with healing upon partitioning | 17:29 |
| greghaynes | ooo nice | 17:30 |
| greghaynes | aye, testing my awk fu | 17:32 |
| ccrouch | jp_at_hp: have you had any joy with https://bugs.launchpad.net/tripleo/+bug/1318750 ? | 17:35 |
| uvirtbot | Launchpad bug 1318750 in tripleo "overcloud create failing; "Command output matched '(CREATE|UPDATE)_FAILED'" (dup-of: 1317923)" [Critical,In progress] | 17:35 |
| uvirtbot | Launchpad bug 1317923 in tripleo "tripleo failing to come up - baremetal configuration broked" [Critical,Fix released] | 17:35 |
| *** openstackgerrit has quit IRC | 17:35 | |
| ccrouch | i'm having an issue with an authentication failure from nova baremetal to one of the nodes and I came across that bug | 17:36 |
| ccrouch | which has the same error in the attached log snippet, not sure its related | 17:36 |
| *** openstackgerrit has joined #tripleo | 17:36 | |
| *** akuznetsov has quit IRC | 17:37 | |
| *** e0ne has joined #tripleo | 17:38 | |
| *** jprovazn_afk has quit IRC | 17:40 | |
| *** e0ne has quit IRC | 17:40 | |
| *** andreaf has quit IRC | 17:40 | |
| *** ramishra has quit IRC | 17:40 | |
| *** e0ne has joined #tripleo | 17:42 | |
| lifeless | greghaynes: morning :) | 17:43 |
| lifeless | greghaynes: next HA patch ? | 17:43 |
| greghaynes | hihi | 17:43 |
| jang | ooh. potentially good news. Think we've got a fix for eventlet. | 17:44 |
| jang | mind you, I've said that about six times now. | 17:44 |
| lifeless | jang: yay | 17:44 |
| lifeless | jang: noo | 17:44 |
| jang | but I think we caught the bug that's actually been killing us | 17:44 |
| greghaynes | lifeless: hrm, youre running me dry as for ones that have passed CI | 17:45 |
| lifeless | dumdumdummmmmmm duuuum! | 17:47 |
| *** jp_at_hp has quit IRC | 17:47 | |
| greghaynes | http://logs.openstack.org/77/102177/1/check-tripleo/check-tripleo-undercloud-precise/7494d73/logs/seed_logs/nova-compute.txt.gz | 17:49 |
| greghaynes | PXE deploy failed? | 17:49 |
| *** mrunge has joined #tripleo | 17:49 | |
| greghaynes | Havent seen that one before | 17:49 |
| greghaynes | huh, no such file or directory <iscsi target> | 17:50 |
| greghaynes | lifeless: Theres two that *should* pass CI soon and are needed, ill ping when done | 17:53 |
| lifeless | greghaynes: same here - logs.openstack.org/35/96235/6/check-tripleo/check-tripleo-undercloud-precise/323ebb2/logs/seed_logs/nova-compute.txt.gz | 17:53 |
| greghaynes | aye | 17:54 |
| lifeless | apparently the seeds are 2G VMs, which will be hitting the adm64 footprint limit | 17:54 |
| *** Penick has joined #tripleo | 17:54 | |
| lifeless | Ng: ping | 17:55 |
| lifeless | Ng: is there an hp2 bastion now? network folk want to put the network into final configuration | 17:56 |
| *** jcoufal has quit IRC | 17:56 | |
| *** yjiang5 is now known as yjiang5_away | 17:57 | |
| lifeless | greghaynes: check the output of free in the host state | 17:57 |
| lifeless | total used free shared buffers cached | 17:57 |
| lifeless | Mem: 2.0G 1.6G 387M 448K 13M 78M | 17:57 |
| lifeless | -/+ buffers/cache: 1.5G 478M | 17:57 |
| lifeless | Swap: 0B 0B 0B | 17:57 |
| lifeless | tight but not impossible | 17:58 |
| lifeless | I'd be a lot happier if that was a 4G VM | 17:58 |
| lifeless | dprince: hey - so - suggestions on how to deploy/fix that? | 17:58 |
| lifeless | dprince: I'm still not entirely familiar with the layout within rh1 | 17:59 |
| ekarlso | any chance to get some views on https://review.openstack.org/#/c/102292/ ? | 17:59 |
| lifeless | greghaynes: filing a bug ? | 17:59 |
| greghaynes | yep | 17:59 |
| jang | Right. That github repo may have the latest and greatest in. It's chock full of debug crap but there's the gem of a working eventlet in there, we think. Proving it out with ironic and tidying it back up for shoving upstream, for tomorrow. | 18:00 |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 | 18:00 |
| lifeless | jang: did you get a suggestion in reply to your mail? or you just kept plugging at the issue ? | 18:01 |
| greghaynes | oh, bnemec might have already | 18:03 |
| greghaynes | https://bugs.launchpad.net/tripleo/+bug/1316350 | 18:03 |
| uvirtbot | Launchpad bug 1316350 in tripleo "Deployment failure due to missing iscsi target" [Medium,Triaged] | 18:03 |
| openstackgerrit | Haneef Ali proposed a change to openstack/tripleo-image-elements: Configure keystone with apache https://review.openstack.org/102304 | 18:06 |
| jang | lifeless: jsut kept plugging at it. My first patch had "moved" the problem, we kept on with it until we'd plugged every leak. | 18:10 |
| jang | now we jsut need to figure out what the minimal invasive patch is. | 18:10 |
| openstackgerrit | A change was merged to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/96234 | 18:11 |
| jang | I swear, though, coroutines are actually _harder_ to grok than straight up continuations. It'd be a hell of a lot simpler to recast eventlet atop that as an abstraction. | 18:11 |
| ekarlso | is it my patch that makes it bork or someone elses http://logs.openstack.org/92/102292/1/check-tripleo/check-tripleo-ironic-seed-precise/1368845/console.html ? | 18:12 |
| lifeless | jang: funnily enough, thats been said before :) | 18:13 |
| lifeless | ekarlso: seed didn't come up proplerly; what was your patch changing ? | 18:14 |
| ekarlso | rabbitmq.config | 18:14 |
| lifeless | could be it :> | 18:15 |
| lifeless | ekarlso: you can check if the config came up properly | 18:15 |
| ekarlso | lifeless: where ? | 18:15 |
| lifeless | ekarlso: in the logs tgz | 18:17 |
| greghaynes | lifeless: how bad of form would it be to un-abandon https://review.openstack.org/#/c/93104/ | 18:17 |
| greghaynes | seems like it was already 2x+2 whish is why I want to | 18:17 |
| ekarlso | oh, wrong config format : p | 18:17 |
| dprince | lifeless: are you requesting that we increase the size of the seed VMs? | 18:18 |
| *** echohead has joined #tripleo | 18:18 | |
| dprince | lifeless: I'd have to do the match... but like we spoke yesterday the intent was to be able to save space and use a 2 Gig seed for CI (I think) | 18:19 |
| lifeless | dprince: or if we needed to go to i386, that we do that. I would rather 4G seeds though - because we IO thrash if the BM deploy nodes are too small | 18:19 |
| dprince | lifeless: how urgently is this needed? | 18:19 |
| lifeless | dprince: but right now we have 2G seed and amd64 nodes, that fails intermittently in my tests - we're too close to the RAM ceiling | 18:20 |
| dprince | lifeless: The cost is we'll have less capicity on the RH rack I think | 18:20 |
| dprince | lifeless: of course we can do it, but we'll be running even less jobs. | 18:21 |
| dprince | lifeless: I'd like to see us bring on more nodes first, I suppose and hold off on sending anything which breaks the seeds in this regard until we do | 18:22 |
| dprince | lifeless: which patch in particular are you concerned about? | 18:22 |
| lifeless | dprince: what about 3G ndoes w/i386 ? | 18:22 |
| lifeless | dprince: | 18:23 |
| lifeless | bah | 18:23 |
| lifeless | https://bugs.launchpad.net/tripleo/+bug/1316350 <- thinking that might be a low memory symptom | 18:23 |
| uvirtbot | Launchpad bug 1316350 in tripleo "Deployment failure due to missing iscsi target" [Medium,Triaged] | 18:23 |
| dprince | lifeless: seems like a reasonable compromise but again I'll have to check the numbers | 18:23 |
| *** martyntaylor1 has left #tripleo | 18:23 | |
| lifeless | StevenK: where is your patch to fixup testenv creation metadata? | 18:24 |
| lifeless | StevenK: we should get that in to any rebuild dprince does | 18:24 |
| lifeless | dprince: if we do rebuild, I'd like to follow along in a screen session or something, to see how its down in that rack | 18:24 |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 | 18:25 |
| dprince | lifeless: sure. I'm fine with that. Honestly this would be something nice to do in mid-July perhaps. | 18:25 |
| lifeless | dprince: do you mean at the sprint? | 18:26 |
| dprince | lifeless: we have good stability right now... I hate to rock the boat just because we think a small seed causes this iscsi failure though. | 18:26 |
| lifeless | dprince: some training for tripleo-cd-admins there would rock | 18:26 |
| dprince | lifeless: yes, the sprint | 18:26 |
| greghaynes | +1 | 18:26 |
| dprince | greghaynes: not sure what you are +1'ing. We'll take it as a general amen, tripleO rocks sort of thing | 18:27 |
| *** e0ne has quit IRC | 18:28 | |
| greghaynes | yep, just trying to cheer the project along | 18:28 |
| *** e0ne has joined #tripleo | 18:28 | |
| greghaynes | +1 to cd-admins training though :) | 18:28 |
| *** vinsh has quit IRC | 18:28 | |
| ekarlso | greghaynes: when u mean to compare the file, do a md5sum or so of the 2 hosts files ? | 18:29 |
| ekarlso | https://review.openstack.org/#/c/102198/3/elements/hosts/os-refresh-config/configure.d/51-hosts | 18:29 |
| greghaynes | ekarlso: I mean figure out if were actually going to modify it, and if so log what were going to change | 18:29 |
| dprince | greghaynes/lifeless: I need this done before ssh access to the RH cloud will get fixed https://review.openstack.org/#/c/101576/ | 18:29 |
| greghaynes | I could see that being useful for knowing what our hosts content was during a specific o-c-c run | 18:30 |
| ekarlso | greghaynes: that sounds like doing more advanced stuff then the awk that's currently there :| | 18:30 |
| dprince | The gist is our guys don't like using a single account for access (for obvious security reasons) | 18:33 |
| greghaynes | hehe, WCPGW with a single account and a ton of ssh keys | 18:33 |
| dprince | And there isn't a reasonable way to map the keys to usernames/IRC names otherwise | 18:33 |
| greghaynes | cp file, do awk, compare | 18:33 |
| greghaynes | ? | 18:34 |
| *** jp_at_hp has joined #tripleo | 18:34 | |
| lifeless | greghaynes: can you run the meeting today?\ | 18:34 |
| greghaynes | If its a bunch of work then im fine not doing it, seemed simple though | 18:34 |
| greghaynes | lifeless: sure | 18:34 |
| greghaynes | do youall have macros for all those #links things you do? | 18:34 |
| greghaynes | I notice youall spam lots quicly | 18:34 |
| lifeless | greghaynes: see the wiki page | 18:35 |
| lifeless | dprince: so its basically fine | 18:35 |
| lifeless | dprince: I'm just struggling to remember how we get *those* keys into the admin 'default' keypair in the CI region underclouds, so that we have access to the overcloud | 18:36 |
| dprince | lifeless: So, in the case where the undercloud is the gateway/access node (i.e. what the HP rack did) I don't think there is an issue. | 18:39 |
| dprince | lifeless: in the case where there is a bastion there would need to be a second sync script to allow you to access the undercloud as well. | 18:39 |
| dprince | lifeless: I use port forwarding | 18:39 |
| dprince | lifeless: not port forwarding, ssh forwarding :) | 18:40 |
| dprince | lifeless: we can have more scripts for whatever. All I really care about initially is how we store these keys... and keeping them in files per user is a step in the right direction | 18:40 |
| lifeless | dprince: yep, like I say I'm fine with that. Its their use in the admin keypair that I'm trying to remember. | 18:41 |
| lifeless | dprince: how is that setup in the rh rack? or do you use the id_rsa of the user account the deploy is intiated from ? | 18:41 |
| dprince | lifeless: I don't think they are used there. | 18:42 |
| lifeless | ah | 18:42 |
| dprince | lifeless: at least they aren't on our rack | 18:42 |
| ekarlso | greghaynes: would doing a diff be sufficient ? ; ) | 18:42 |
| lifeless | so in the hp1 region I had it up so we can all ssh directly in as heat-admin@ci-overcloud.... | 18:42 |
| lifeless | using our own keys | 18:42 |
| greghaynes | ekarlso: totally | 18:42 |
| dprince | lifeless: that is fine. But in our region everything is configured off of a seed host. As TripleO doesn't bootstrap that seed host I'd say lets just also have a script to sync things there too | 18:43 |
| dprince | lifeless: if someone wants to go directly from bastion to undercloud... that is fine too but I would rather consolidate things on the seed host as much as possible. As sort of an administration node | 18:44 |
| lifeless | dprince: FWIW I marked it +A about 3m back | 18:44 |
| dprince | lifeless: we're just talking here man, not trying to pressure you :) | 18:45 |
| lifeless | dprince: I know | 18:45 |
| lifeless | dprince: just letting you know since you need it | 18:45 |
| lifeless | dprince: I have to go do this PTL webinar now | 18:45 |
| dprince | lifeless: thanks, TTL | 18:45 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Revert "Try reverting Neutron ovs-agent change" https://review.openstack.org/101513 | 18:45 |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Hosts element: Use 'markers' for updating hosts https://review.openstack.org/102198 | 18:47 |
| ekarlso | there you go greghaynes | 18:47 |
| SpamapS | we on alternate meeting time today? | 18:50 |
| greghaynes | oh, right | 18:50 |
| greghaynes | lifeless: ^ | 18:50 |
| lifeless | oh | 18:51 |
| lifeless | I thought it was original. Easy :) | 18:51 |
| lifeless | tchaypo: tag, you get to run the meeting :> | 18:51 |
| SpamapS | Oh wow I didn't know there was an ical for the meetings | 18:55 |
| SpamapS | lovely | 18:55 |
| greghaynes | ekarlso: reviewed your other RMQ patch | 18:56 |
| greghaynes | I didnt realise rmq had that config value, that makes me a lot happier | 18:57 |
| *** jdob has quit IRC | 18:57 | |
| *** jdob has joined #tripleo | 18:58 | |
| *** jtomasek has joined #tripleo | 18:59 | |
| *** jcoufal has joined #tripleo | 18:59 | |
| ekarlso | greghaynes: i discovered it when testing it here by partitioning the cluster it doesn't autoheal | 19:01 |
| ekarlso | ;) | 19:01 |
| greghaynes | awesome | 19:01 |
| *** morganfainberg_L has quit IRC | 19:01 | |
| *** morganfainberg_L has joined #tripleo | 19:01 | |
| *** morganfainberg_L has quit IRC | 19:02 | |
| *** morganfainberg_L has joined #tripleo | 19:02 | |
| *** yamahata__ has joined #tripleo | 19:02 | |
| jp_at_hp | ccrouch - that fix should be in the tripleo codebase already, as per https://bugs.launchpad.net/tripleo/+bug/1317923 | 19:02 |
| uvirtbot | Launchpad bug 1317923 in tripleo "tripleo failing to come up - baremetal configuration broked" [Critical,Fix released] | 19:02 |
| *** yamahata_ has quit IRC | 19:03 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Split ssh keys into files per user. https://review.openstack.org/101576 | 19:03 |
| *** jp_at_hp has quit IRC | 19:03 | |
| *** vinsh has joined #tripleo | 19:08 | |
| *** jdob has quit IRC | 19:09 | |
| *** jdob has joined #tripleo | 19:09 | |
| *** jang1 has joined #tripleo | 19:12 | |
| *** mrunge has quit IRC | 19:12 | |
| jdob | bnemec: did you submit a patch for the tox.ini yet, or did I misread your email and you're still working on it? | 19:21 |
| ekarlso | greghaynes: what u want to call the ha_mode then ? | 19:21 |
| bnemec | jdob: https://review.openstack.org/#/c/102291/ | 19:21 |
| jdob | ah, crud, dib-utils isnt in my dashboard yet, that's why | 19:22 |
| lifeless | Ng: are you watching the webcast, or is it your evil clone? | 19:22 |
| *** killer_prince has quit IRC | 19:24 | |
| jdob | thanks bnemec | 19:24 |
| bnemec | jdob: np | 19:25 |
| *** dshulyak_ has quit IRC | 19:40 | |
| *** noslzzp has quit IRC | 19:41 | |
| kevinbenton | dprince: ping | 19:42 |
| dprince | kevinbenton: hi | 19:42 |
| kevinbenton | dprince: would this have affected the tests for my neutron patch? https://review.openstack.org/102281 | 19:43 |
| kevinbenton | dprince: whoops https://review.openstack.org/#/c/101513/ | 19:43 |
| kevinbenton | it would have removed one of the commits mine depended on if i understand it correctly | 19:44 |
| dprince | kevinbenton: oh, did you reference the same (already closed I thought) ticket? | 19:44 |
| lifeless | if you reference the ticket it doesn't apply the revery | 19:45 |
| dprince | kevinbenton: ah, yes. I think it might have. | 19:45 |
| dprince | kevinbenton: no, though. THe commit was different | 19:45 |
| dprince | kevinbenton: plus we'd see it in the logs | 19:45 |
| kevinbenton | dprince: yeah, i see the id changed | 19:45 |
| dprince | kevinbenton: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_devtest.sh#n20 | 19:46 |
| openstackgerrit | Jay Dobies proposed a change to openstack/tripleo-specs: Spec for Tuskar Juno Plan REST API changes https://review.openstack.org/94720 | 19:47 |
| kevinbenton | dprince: well that looks for a bug number, right? | 19:47 |
| kevinbenton | dprince: XXX isn’t exactly a valid bug :-) | 19:48 |
| dprince | kevinbenton: Correct, So this wouldn't have caused the problems. | 19:48 |
| *** jcoufal has quit IRC | 19:49 | |
| ekarlso | greghaynes: when u mean a diff name it would rock my world with a suggestion :) | 19:49 |
| dprince | kevinbenton: FWIW, I tried your patch 4-5 times locally today (on real bare metal) and it works for me. So I think it may be a race that occurs only on smaller/virtual test environments. | 19:49 |
| dprince | kevinbenton: did you see my note about the GRE ports missing on the br-tun bridge? | 19:49 |
| ekarlso | regards to https://review.openstack.org/#/c/102292/2 | 19:50 |
| kevinbenton | dprince: yes, but that can be related to lots of things | 19:50 |
| kevinbenton | dprince: without the debug logs for the vswitch agent there is nothing to work on | 19:51 |
| kevinbenton | dprince: I was hoping it would be as easy as this, but it doesn’t seem so https://review.openstack.org/#/c/102073/2/heat-templates/testenv-workers.yaml | 19:54 |
| *** jtomasek has quit IRC | 19:55 | |
| *** Penick has quit IRC | 19:58 | |
| devananda | dprince: if you have a moment to look at https://review.openstack.org/#/c/99460/ -- looks like it is passing everything now | 20:00 |
| dprince | devananda: ack, thanks. Glad to see that. let me re-review it in a minute | 20:01 |
| dprince | kevinbenton: left you some comments on 102073. You are on the right track... I think we just need two sets of DIB_REPOREF variables though | 20:02 |
| dprince | kevinbenton: one for your neutron patch (you've got that already). And another for a new tripleo-heat-templates branch which enables neutron debug logging. Alternately you could just hack the tripleo-image-elements neutron.conf file directly and reference that project/patch instead. | 20:03 |
| *** Penick has joined #tripleo | 20:03 | |
| dprince | kevinbenton: make sense? Once we have that you should essentially get the results you are looking for, ie. your neutron patch with debug logging enabled | 20:03 |
| *** spzala has joined #tripleo | 20:04 | |
| *** e0ne has quit IRC | 20:05 | |
| *** e0ne has joined #tripleo | 20:06 | |
| *** e0ne has quit IRC | 20:08 | |
| *** jml has quit IRC | 20:09 | |
| dprince | devananda: 99460 is +A | 20:10 |
| openstackgerrit | Kevin Benton proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 20:10 |
| kevinbenton | dprince: like that? ^^ | 20:10 |
| kevinbenton | dprince: forgot to remove the debug: true | 20:10 |
| devananda | dprince: awesome, thanks | 20:10 |
| *** dtantsur is now known as dtantsur|afk | 20:10 | |
| dprince | kevinbenton: Hmmm, actually I led you on the wrong path. TripleO image elements isn't installed by DIB!! | 20:12 |
| devananda | lifeless: ^ scheduling fixes should be landing soon | 20:12 |
| greghaynes | ekarlso: *shrug* why not just use the name of the config option in rabbit | 20:12 |
| greghaynes | rabbit.cluster_partition_handling | 20:12 |
| *** e0ne has joined #tripleo | 20:12 | |
| dprince | kevinbenton: I'm not sure we have a good way of enabling neutron debugging at this time. What we really need is Zuul dependencies :( | 20:12 |
| *** jml has joined #tripleo | 20:13 | |
| dprince | kevinbenton: I think we might should an an option to the tripleo-incubator (the devtest scripts themselves) to enable DEBUG via an environment variable. | 20:14 |
| kevinbenton | dprince: why not just debug by default in these projects? to keep the logs small? | 20:15 |
| openstackgerrit | Endre Karlson proposed a change to openstack/tripleo-image-elements: Allow ha_mode to be set. https://review.openstack.org/102292 | 20:17 |
| dprince | kevinbenton: yes, keeps the logs small. Also, we want production defaults | 20:17 |
| dprince | kevinbenton: Would you like me to take this? | 20:18 |
| greghaynes | dprince: cant we use ExtraParams | 20:18 |
| greghaynes | to enable debugging | 20:18 |
| kevinbenton | dprince: yes, i know nothing about tripleo so i’m in over my head | 20:18 |
| greghaynes | so export OVERCLOUD_EXTRA_CONFIG | 20:18 |
| dprince | greghaynes: thanks, good idea | 20:19 |
| * dprince double checks... | 20:19 | |
| greghaynes | neutron.conf does support passthrough | 20:19 |
| *** e0ne has quit IRC | 20:20 | |
| dprince | greghaynes: yes, it does. Are you up for wiring https://review.openstack.org/#/c/102073/3 up with that approach? | 20:20 |
| *** e0ne has joined #tripleo | 20:20 | |
| greghaynes | yep | 20:21 |
| dprince | greghaynes: if not I'll take your suggestions and see what works best | 20:21 |
| dprince | greghaynes: thanks man :) | 20:21 |
| greghaynes | np | 20:21 |
| dprince | kevinbenton: cool, hopefully we'll get you debug info soon then | 20:22 |
| *** dprince has quit IRC | 20:23 | |
| *** e0ne has quit IRC | 20:24 | |
| *** eguz has joined #tripleo | 20:26 | |
| *** davidlenwell_ is now known as davidlenwell | 20:28 | |
| *** eguz has quit IRC | 20:28 | |
| *** ChanServ sets mode: +v davidlenwell | 20:28 | |
| *** eguz has joined #tripleo | 20:28 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 20:29 |
| *** e0ne has joined #tripleo | 20:29 | |
| *** howleyt_ has joined #tripleo | 20:30 | |
| *** eghobo has quit IRC | 20:30 | |
| howleyt_ | Howdy, anyone around that could approve https://review.openstack.org/#/c/81920/ It has two +2's already. | 20:32 |
| lifeless | devananda: huh? | 20:32 |
| lifeless | howleyt_: not until we have CI ok | 20:32 |
| greghaynes | ci looks like it passed | 20:32 |
| lifeless | oh gertty hadn't picked that up. cool | 20:33 |
| greghaynes | +a'd | 20:33 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add keepalive option to rabbitmq config https://review.openstack.org/81920 | 20:33 |
| lifeless | hmm, no reviews on 102081 yet. | 20:35 |
| devananda | lifeless: scheduler bug in nova that wasn't picking up host stats from ironic -- the fix was just approved. | 20:36 |
| devananda | lifeless: once it lands, you may want to poke hp internal folks who were waiting for that | 20:36 |
| SpamapS | lifeless: how important do you think it will be that the chunking shim be able to take any and all heat CLI options and pass them along to the respective underlings? | 20:43 |
| *** nati_ueno has joined #tripleo | 20:48 | |
| tchaypo | The meeting was the early one today? | 20:49 |
| greghaynes | unsure what early is for us | 20:50 |
| greghaynes | its 00:00 PST | 20:50 |
| greghaynes | which is technically *very* early | 20:50 |
| tchaypo | Ie 12 hours from now | 20:50 |
| tchaypo | Oh | 20:51 |
| tchaypo | That'd be why lifeless is declining to run sit today | 20:51 |
| howleyt_ | greghaynes, lifeless: Thought I'd pounce when I saw the CI passing. Thanks. | 20:52 |
| tchaypo | I woke up, saw that he'd tagged me at 04:51, and thought I'd misjudged and slept through | 20:53 |
| openstackgerrit | Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 20:56 |
| greghaynes | bnemec: thanks for the repolocation protip | 20:56 |
| *** jdob has quit IRC | 20:58 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332458 | 21:00 |
| uvirtbot | Launchpad bug 1332458 in diskimage-builder "Not able to build armhf or arm64 images on amd64 arch systems." [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1333807 | 21:00 |
| uvirtbot | Launchpad bug 1333807 in tripleo "RabbitMQ cluster doesn't autoheal after a partition" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1333649 | 21:00 |
| uvirtbot | Launchpad bug 1333649 in tripleo "/etc/hosts entries written by hosts element doesn't survive reboot" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1332521 | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| uvirtbot | Launchpad bug 1332521 in diskimage-builder "failed to remove `/tmp/image.A2aupYJb/mnt/tmp/in_target.d': Device or resource busy" [Undecided,New] | 21:00 |
| *** nati_ueno has quit IRC | 21:02 | |
| *** nati_ueno has joined #tripleo | 21:02 | |
| greghaynes | Looking for core on https://review.openstack.org/#/c/102177/ | 21:03 |
| lifeless | devananda: I don't know who's waiting for it - I just heard of it now | 21:04 |
| greghaynes | lifeless: ^ HA patch that passed CI | 21:04 |
| lifeless | SpamapS: I think its very unimportant. Its not in a heat repo. | 21:04 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Check if stunnel.connect_ip is set https://review.openstack.org/102177 | 21:06 |
| *** julim has quit IRC | 21:08 | |
| *** cwolferh has quit IRC | 21:12 | |
| SpamapS | lifeless: I got it working anyway. :-P | 21:15 |
| SpamapS | lifeless: should help this have minimal impact on the way everything works. | 21:15 |
| SpamapS | Just working on pointing it at a real Heat now | 21:15 |
| greghaynes | lifeless: curious if youve read what happened on https://review.openstack.org/#/c/93104/ | 21:21 |
| greghaynes | its needed for HA | 21:21 |
| lifeless | greghaynes: i have not | 21:21 |
| greghaynes | got abandoned when it was 2x +2 for a patch that seems pretty far from being merged | 21:21 |
| lifeless | greghaynes: meetig, back soon | 21:21 |
| greghaynes | ok | 21:21 |
| lifeless | greghaynes: if you want it alive, I think you can do that or ping clarkb | 21:21 |
| clarkb | you all can do it now | 21:22 |
| clarkb | Cores have (un)abandon rights | 21:22 |
| *** eguz has quit IRC | 21:22 | |
| greghaynes | Yep, hoping for at least a second opinion that im not missing something | 21:22 |
| greghaynes | I do see a restore change button :) | 21:22 |
| clarkb | this is one of the new shiny gerrit 2.8 features | 21:22 |
| *** nealph has joined #tripleo | 21:28 | |
| nealph | lifeless: added you as a core reviewer on https://review.openstack.org/#/c/96527/. I think you have background on what's happening there...thanks again for working through the context with me | 21:29 |
| *** jang has quit IRC | 21:31 | |
| *** eghobo has joined #tripleo | 21:32 | |
| *** eghobo has quit IRC | 21:36 | |
| *** e0ne has quit IRC | 21:36 | |
| *** eghobo has joined #tripleo | 21:36 | |
| lifeless | nealph: whats wrong with it ? | 21:43 |
| lifeless | nealph: it looks like its passed CI and review | 21:43 |
| lifeless | greghaynes: I don't think they are mutually exclusive, no? fixing up reset-db is fine, on the path to deleting it :) | 21:44 |
| greghaynes | Yes, 93104 is just a subset of the other | 21:45 |
| lifeless | greghaynes: so... you have the force, greg | 21:49 |
| greghaynes | restored | 21:50 |
| *** yjiang5_away is now known as yjiang5 | 21:51 | |
| lifeless | greghaynes: r11 is really quite different to r10 | 21:51 |
| greghaynes | oh my, it is | 21:52 |
| lifeless | I'm not sure r11 will pass | 21:53 |
| *** nati_ueno has quit IRC | 21:53 | |
| lifeless | have you looked at why it failed ? | 21:53 |
| greghaynes | Its been working for me | 21:53 |
| ekarlso | Yap greghaynes that pause_minority thing for fixes teh partitioning issue when a node comes up again, no manual intervention needed : p | 21:53 |
| *** nati_ueno has joined #tripleo | 21:54 | |
| greghaynes | lifeless: pypi fail | 21:54 |
| lifeless | ah kk | 21:54 |
| greghaynes | ekarlso: w00t | 21:54 |
| lifeless | well its rechecking now | 21:54 |
| greghaynes | lifeless: when we switched to non-package install for mysql I think the data massaging part of that script became irrelevant | 21:55 |
| ekarlso | greghaynes: also I tested the /etc/hosts fix, works like a charm : I | 21:55 |
| ekarlso | Now I can reboot a node without the fear of it never coming back to the living : P | 21:56 |
| *** jtomasek has joined #tripleo | 22:00 | |
| nealph | lifeless: just lookin' for the +2/approval. | 22:01 |
| *** jpeeler has quit IRC | 22:04 | |
| *** e0ne has joined #tripleo | 22:07 | |
| *** jtomasek has quit IRC | 22:07 | |
| *** e0ne_ has joined #tripleo | 22:09 | |
| *** e0ne has quit IRC | 22:12 | |
| *** e0ne_ has quit IRC | 22:12 | |
| *** nijaba has quit IRC | 22:13 | |
| *** nijaba has joined #tripleo | 22:15 | |
| *** nijaba has quit IRC | 22:15 | |
| *** nijaba has joined #tripleo | 22:15 | |
| openstackgerrit | Longgeek proposed a change to openstack/tripleo-image-elements: loading of package `pbr` will break setuptools in tripleo https://review.openstack.org/102367 | 22:16 |
| jerryz | lifeless: how does heat decide the number of nodes for compute and controller, and the role mapping to the nodes? | 22:26 |
| *** markmc has quit IRC | 22:28 | |
| *** noslzzp has joined #tripleo | 22:31 | |
| *** yamahata has quit IRC | 22:35 | |
| *** nijaba_ has joined #tripleo | 22:36 | |
| *** nijaba has quit IRC | 22:36 | |
| *** nijaba_ has quit IRC | 22:45 | |
| *** boris-42 has quit IRC | 22:46 | |
| *** yamahata has joined #tripleo | 22:47 | |
| *** eghobo has quit IRC | 22:50 | |
| *** Penick has quit IRC | 22:51 | |
| *** nijaba has joined #tripleo | 22:51 | |
| *** howleyt_ has quit IRC | 22:54 | |
| *** Penick has joined #tripleo | 22:58 | |
| *** nijaba has quit IRC | 22:58 | |
| SpamapS | jerryz: Heat does not decide | 23:00 |
| SpamapS | jerryz: we generate a template that has the number of nodes | 23:00 |
| SpamapS | jerryz: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/Makefile | 23:00 |
| SpamapS | jerryz: COMPUTESCALE and CONTROLSCALE | 23:00 |
| jerryz | SpamapS: yeah, i see that. but how can i specify which is which | 23:01 |
| *** boris-42 has joined #tripleo | 23:02 | |
| jerryz | SpamapS: say, one BM node has larger storage, which is for cinder | 23:02 |
| lifeless | jerryz: you'd crete a flavor that matches that node size, and specify the right flavor for cinder nodes in the template | 23:03 |
| lifeless | jerryz: if this is for the tripleo region, I wouldn't bother - just run all the nodes the same size, and run all services on all of them | 23:03 |
| lifeless | jerryz: no need to super-optimise things | 23:03 |
| *** chuckC has quit IRC | 23:04 | |
| jerryz | lifeless: ok. i see. that is at the time registering the BM node. | 23:06 |
| kevinbenton | greghaynes: hey, was the latest result set for this patch supposed to have debug info? https://review.openstack.org/#/c/102073/ | 23:07 |
| jerryz | lifeless: but tripleO script itself just hardcodes to flavor baremetal | 23:07 |
| greghaynes | kevinbenton: I just saw that, am testing now. Those logs look like it didnt have debug info set, right? | 23:08 |
| lifeless | jerryz: yes, we're very simple today | 23:08 |
| kevinbenton | greghaynes: yeah, doesn’t seem to | 23:08 |
| *** yamahata has quit IRC | 23:08 | |
| *** e0ne has joined #tripleo | 23:09 | |
| jerryz | lifeless: another question, how do you get the mac address of the real BM machines not VM simulators | 23:09 |
| *** ddieterly has quit IRC | 23:09 | |
| kevinbenton | greghaynes: should be able to see rpc messages in http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/logs/overcloud-NovaCompute0-brah7pocqqon_logs/neutron-openvswitch-agent.txt.gz | 23:09 |
| greghaynes | lol, I think it might be because I didnt capitalize default | 23:10 |
| greghaynes | in the config section | 23:10 |
| *** Penick has quit IRC | 23:11 | |
| *** e0ne has quit IRC | 23:12 | |
| *** ifarkas has quit IRC | 23:13 | |
| lifeless | jerryz: we pass it in with the IPMI details - in nodes.json | 23:16 |
| greghaynes | oh wait, it didnt set that at all | 23:18 |
| *** Penick has joined #tripleo | 23:18 | |
| greghaynes | er, it didnt use the exta params to fill out the config file | 23:18 |
| jerryz | lifeless: i assume this nodes.json file needs lab admin's input, or some hardware discovery tool. | 23:20 |
| openstackgerrit | Chris Krelle proposed a change to openstack/diskimage-builder: update zypper element https://review.openstack.org/102382 | 23:20 |
| openstackgerrit | Chris Krelle proposed a change to openstack/diskimage-builder: update zypper element https://review.openstack.org/102382 | 23:21 |
| greghaynes | hrm, looks like the extra config thing should have worked: http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/console.html#_2014-06-24_21_53_43_551 | 23:23 |
| lifeless | jerryz: lab admin yeah | 23:23 |
| lifeless | jerryz: you need IPMI credentials, can't discover that :) | 23:23 |
| greghaynes | ok, yea, I think its a capitalization issue, retrying... | 23:24 |
| *** eghobo has joined #tripleo | 23:24 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack-infra/tripleo-ci: Testing using a specific patch from neutron in CI https://review.openstack.org/102073 | 23:24 |
| greghaynes | kevinbenton: http://logs.openstack.org/73/102073/5/check-tripleo/check-tripleo-overcloud-f20/d704ad5/logs/overcloud-controller0-ogh64dgoubfk_logs.tar.xz has /etc in it | 23:25 |
| greghaynes | kevinbenton: so you can see at the bottom the value is being set | 23:25 |
| *** eghobo has quit IRC | 23:25 | |
| *** eghobo has joined #tripleo | 23:25 | |
| *** morazi has quit IRC | 23:26 | |
| *** gfidente has quit IRC | 23:28 | |
| *** chuckC has joined #tripleo | 23:35 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/diskimage-builder: Ensure cache directory exists before use https://review.openstack.org/102386 | 23:36 |
| greghaynes | huh http://logs.openstack.org/54/93554/10/check-tripleo/check-tripleo-overcloud-f20/a44beb4/console.html#_2014-06-24_22_21_43_145 | 23:37 |
| greghaynes | so close! | 23:37 |
| SpamapS | lifeless: ok, I've got something that _tries_ to work.. let me squash and push for review | 23:37 |
| lifeless | SpamapS: woo | 23:38 |
| SpamapS | have to run get kids now.. but I think I could have a chunked stack deploying tonight | 23:38 |
| lifeless | SpamapS: fantastico | 23:38 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-incubator: Tool to tie disconnected stacks together https://review.openstack.org/102388 | 23:41 |
| SpamapS | about half of those lines are, sadly, copy/paste from python-heatclient | 23:41 |
| SpamapS | no time to refactor it to expose its parsers and keystone-client-endpoint-determining goodness. | 23:42 |
| ekarlso | greghaynes: I wonder if theres another issue now... | 23:43 |
| greghaynes | nah, im sure we fixed *all* the bugs :p | 23:44 |
| ekarlso | so when nodes have been powered off and then powered back on w rmq I get: mkdir: cannot create directory ‘/var/run/rabbitmq’: Permission denied in logs :) | 23:44 |
| ekarlso | is /var/run wiped on boot or ? | 23:44 |
| * SpamapS is out | 23:45 | |
| *** Penick has quit IRC | 23:45 | |
| greghaynes | something to note is that even if rabbitmq comes back, there is not much hope of galera coming back after a full cluster reboot | 23:45 |
| greghaynes | if you shut it all down at once | 23:45 |
| greghaynes | so not sure if that puts a damper on your plans | 23:45 |
| jerryz | lifeless: i see in devtest grizzly is used for overcloud. is there plan for icehouse? | 23:45 |
| ekarlso | greghaynes: we're more checking up on how stuff behaves and what intervention is needed | 23:46 |
| greghaynes | ok, just making sure you were aware | 23:46 |
| greghaynes | ekarlso: unsure about the /var/run thing, I dont think we wipe it though | 23:47 |
| greghaynes | jerryz: huh? We use HEAD | 23:48 |
| ekarlso | greghaynes: I'll check tmrw what happens then | 23:49 |
| *** Penick has joined #tripleo | 23:52 | |
| *** yamahata has joined #tripleo | 23:57 | |
| *** xuhaiwei has joined #tripleo | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!