| *** pcrews has joined #tripleo | 00:16 | |
| *** sseago has joined #tripleo | 00:35 | |
| *** pcrews has quit IRC | 00:41 | |
| *** yamahata has joined #tripleo | 00:53 | |
| tchaypo | hrm. | 00:58 |
|---|---|---|
| tchaypo | /home/polleyj/.cache/tripleo/tripleo-incubator/scripts/devtest_seed.sh: line 185: BM_VLAN_SEED_IP_ADDR: unbound variable | 00:58 |
| tchaypo | do I need to blow away my env files again? | 01:02 |
| lifeless | no | 01:15 |
| lifeless | it may be a bug in my patchset | 01:15 |
| lifeless | greghaynes: so I just had a terrible though | 01:16 |
| lifeless | greghaynes: do we setup mysql / rabbit differently in different compute scales? | 01:16 |
| lifeless | greghaynes: (please say no :)) | 01:16 |
| * tchaypo digs | 01:19 | |
| *** ci-overcloud has joined #tripleo | 01:27 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 01:27 |
| *** ci-overcloud has quit IRC | 01:27 | |
| *** mestery has joined #tripleo | 01:28 | |
| *** nati_ueno has quit IRC | 01:29 | |
| *** nati_ueno has joined #tripleo | 01:29 | |
| *** mestery has quit IRC | 01:39 | |
| *** mestery has joined #tripleo | 01:40 | |
| *** nosnos has joined #tripleo | 01:49 | |
| lifeless | hmm, uc disappearing off-net not good | 01:56 |
| *** ci-overcloud has joined #tripleo | 01:58 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 01:58 |
| *** ci-overcloud has quit IRC | 01:58 | |
| *** nosnos has quit IRC | 02:09 | |
| *** ci-overcloud has joined #tripleo | 02:14 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 02:14 |
| *** ci-overcloud has quit IRC | 02:14 | |
| *** Yanyanhu has joined #tripleo | 02:17 | |
| *** mestery has quit IRC | 02:20 | |
| *** mestery has joined #tripleo | 02:21 | |
| lifeless | so this is interesting | 02:21 |
| lifeless | heat triggers way worse failure rates than nova boot --num-instances xx | 02:22 |
| StevenK | lifeless: Can you re-review https://review.openstack.org/#/c/101784/ ? | 02:48 |
| *** morganfainberg_Z is now known as morganfainberg | 02:54 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 03:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340467 | 03:00 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341389 | 03:00 |
| uvirtbot | Launchpad bug 1340467 in tripleo "services cannot be decoupled from the boot-stack element" [Undecided,In progress] | 03:00 |
| uvirtbot | Launchpad bug 1341389 in tripleo "multiple dh-ctlplane dhclient processes and undercloud drops off network" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 03:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341192 | 03:00 |
| uvirtbot | Launchpad bug 1341192 in tripleo "If MySQL initialization fails, restart is not attempted" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Shift sysctl.d writing code to a new script https://review.openstack.org/101784 | 03:02 |
| lifeless | seriously | 03:03 |
| lifeless | why is | 03:03 |
| lifeless | /var/log/upstart/nova-compute.log:2014-07-14 02:13:50.626 20886 DEBUG nova.compute.manager [req-00895408-9a20-42b4-b755-a133af260638 None] [instance: d4d95fb0-7435-4436-ad81-f1ebb1706608] Insufficient compute resources: Free m | 03:03 |
| *** Yanyanhu has quit IRC | 03:03 | |
| lifeless | emory 0.00 MB < requested 98304 MB. _build_and_run_instance /opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/compute/manager.py:1981 | 03:03 |
| lifeless | DEBUG | 03:03 |
| *** Yanyanhu has joined #tripleo | 03:03 | |
| *** ramishra has joined #tripleo | 03:07 | |
| *** tzumainn has quit IRC | 03:14 | |
| *** bmahalakshmi has joined #tripleo | 03:17 | |
| *** Yanyanhu has quit IRC | 03:21 | |
| *** Yanyanhu has joined #tripleo | 03:23 | |
| *** pcrews has joined #tripleo | 03:29 | |
| lifeless | I'm starting to think nova-scheduler doesn't actually log the selected host. | 03:29 |
| lifeless | at debug level. | 03:29 |
| *** pcrews has quit IRC | 03:36 | |
| *** yamahata has quit IRC | 03:36 | |
| *** yamahata has joined #tripleo | 03:37 | |
| *** morganfainberg is now known as morganfainberg_Z | 03:41 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Correct SELinux file contexts https://review.openstack.org/99242 | 03:41 |
| lifeless | oh wow | 03:43 |
| lifeless | I think I've founda mammoth race :( | 03:43 |
| lifeless | Ng: I think I have a task for the virtual team | 03:45 |
| tchaypo | lifeless: with sabre-tooth tiger jockeys? | 03:45 |
| * StevenK wonders if apt-mirror is horrible enough to chown base_path | 03:45 | |
| *** akuznetsov has joined #tripleo | 03:47 | |
| StevenK | Huh, maybe making a symlink does | 03:47 |
| StevenK | Which it doesn't. How odd. | 03:48 |
| *** ci-overcloud has joined #tripleo | 04:03 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 04:03 |
| *** ci-overcloud has quit IRC | 04:03 | |
| *** eghobo has joined #tripleo | 04:04 | |
| lifeless | so I think I now know why my ironic vm tests were failing so damn hard | 04:17 |
| lifeless | https://bugs.launchpad.net/nova/+bug/1341420 | 04:17 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [Undecided,New] | 04:17 |
| lifeless | ^ new favourite bug | 04:17 |
| *** eghobo has quit IRC | 04:18 | |
| *** eghobo has joined #tripleo | 04:19 | |
| *** nosnos has joined #tripleo | 04:26 | |
| tchaypo | I would have expected that when the schedule says "You can use X" that I now have a reservation on X | 04:41 |
| tchaypo | it sounds like the reservation isn't made until I respond to confirm? | 04:42 |
| tchaypo | There must be some other thing I'm missing otherwise that would be a race every time | 04:42 |
| *** eghobo has quit IRC | 04:51 | |
| * StevenK ponders checking his pypi mirror by null routing pypi.python.org on his desktop | 04:57 | |
| *** nati_ueno has quit IRC | 05:01 | |
| *** nati_ueno has joined #tripleo | 05:02 | |
| *** nati_uen_ has joined #tripleo | 05:04 | |
| *** nati_ueno has quit IRC | 05:04 | |
| *** lazy_prince2 has joined #tripleo | 05:09 | |
| StevenK | steven@undermined:~% ip r g 103.245.222.175 | 05:12 |
| StevenK | RTNETLINK answers: Network is unreachable | 05:12 |
| StevenK | That should help | 05:12 |
| *** k4n0 has joined #tripleo | 05:13 | |
| *** lazy_prince2 has quit IRC | 05:19 | |
| StevenK | Downloading/unpacking pip | 05:21 |
| StevenK | Cannot fetch index base URL https://pypi.python.org/simple/ | 05:21 |
| StevenK | Well, yeah ... | 05:21 |
| *** eghobo has joined #tripleo | 05:28 | |
| *** rushiagr has joined #tripleo | 05:40 | |
| *** shausy has joined #tripleo | 05:42 | |
| *** rakesh_hs has joined #tripleo | 05:46 | |
| *** nati_uen_ has quit IRC | 05:47 | |
| Yanyanhu | hi, SpamapS, are you around? | 05:50 |
| tchaypo | StevenK: what are you doing to test it? | 05:55 |
| tchaypo | StevenK: as in, what flags are you giving pip (assuming you're running pip directly) | 05:55 |
| StevenK | tchaypo: So I've been hitting that wall, since pip *really* wants to talk to pypi.p.o | 06:00 |
| tchaypo | have you tried --no-index ? | 06:02 |
| *** ramishra has quit IRC | 06:03 | |
| StevenK | tchaypo: http://paste.openstack.org/show/86364 is what I've been hacking up tie with | 06:03 |
| tchaypo | I'm not seeing --no-index | 06:04 |
| tchaypo | i'm fairly certain --no-index turns off the pypi behaviour | 06:05 |
| StevenK | tchaypo: Right, but then where does pip know where to fetch from? I couldn't see any environment variables that influence it | 06:06 |
| *** nosnos has quit IRC | 06:20 | |
| StevenK | tchaypo: Ah ha. I don't need those tie hacks at all | 06:21 |
| tchaypo | you need to also pass --index-url to tell it what index to use | 06:22 |
| tchaypo | --no-index really means --no-pypi | 06:22 |
| StevenK | tchaypo: Actually, I just needed to add pypi to the list of elements, which configures pip.conf :-) | 06:22 |
| tchaypo | hah | 06:23 |
| StevenK | And it hasn't touched pypi.p.o at all | 06:23 |
| tchaypo | yes, but doesn't that use the (presently, maybe not in future) deprecated pypi-mirror to create a local index? | 06:23 |
| StevenK | tchaypo: No, the pypi element does not use pypi-mirror. It is mentioned in the README as "Perhaps you want to look at using it", but does not require it | 06:25 |
| *** ramishra has joined #tripleo | 06:26 | |
| tchaypo | right. those two elements always confuse me, i can never remember which is which | 06:28 |
| tchaypo | but i think the other one is called pypi-mirror which is proabably a hint | 06:28 |
| StevenK | tchaypo: Right. It is also made more confusing by that the pypi-mirror element is in tie, and the pypi element is in dib | 06:29 |
| tchaypo | heh | 06:29 |
| StevenK | As well as the pypi-openstack element, which is in tie | 06:31 |
| StevenK | But could also be done with PYPI_MIRROR_URL and the pypi element | 06:31 |
| tchaypo | time to fix gertty so it doesn't crash when i click on things | 06:33 |
| StevenK | tchaypo: Heh | 06:34 |
| *** rdopieralski has joined #tripleo | 06:35 | |
| *** mrunge has joined #tripleo | 06:35 | |
| *** rdopieralski has quit IRC | 06:35 | |
| *** rdopieralski has joined #tripleo | 06:35 | |
| StevenK | Right, and there is DIB_COMMON_ELEMENTS exported in my .devtestrc | 06:36 |
| tchaypo | which says? | 06:36 |
| *** slok184 has joined #tripleo | 06:38 | |
| StevenK | tchaypo: http://paste.openstack.org/show/86365/ | 06:40 |
| StevenK | Maybe -incubator should add pypi to DIB_COMMON_ELEMENTS if PYPI_MIRROR_URL is set, but it smells like a layering violation | 06:40 |
| tchaypo | plus you don't always want that behaviour | 06:41 |
| tchaypo | if your local mirror is out of date you might be fine with pip looking upstream | 06:42 |
| StevenK | Right | 06:42 |
| tchaypo | gah. I'm in a car. the gentle lulling motion keeps rocking me to sleep. every time I fall asleep for a few seconds i wake up with a bunch of dddddddddddddd on-screen | 06:43 |
| kviiri | I hope you're not the one driving | 06:43 |
| StevenK | Hahaha | 06:43 |
| kviiri | Both on accounts of that story and the fact you're telling it on IRC | 06:43 |
| *** eguz has joined #tripleo | 07:01 | |
| rdopieralski | the tradition is to use zzzzzzzz | 07:02 |
| *** eguz has quit IRC | 07:03 | |
| *** pblaho has joined #tripleo | 07:03 | |
| *** eghobo has quit IRC | 07:05 | |
| *** regebro has joined #tripleo | 07:06 | |
| *** jtomasek has joined #tripleo | 07:06 | |
| *** akuznetsov has quit IRC | 07:09 | |
| lifeless | righto | 07:10 |
| lifeless | tchaypo: you're not missing anythnig | 07:10 |
| pblaho | o/ | 07:10 |
| lifeless | tchaypo: its a race every time | 07:10 |
| lifeless | StevenK: pip won't touch pypi.p.o if you use the pypi element, no ? | 07:12 |
| StevenK | lifeless: But you can make it do so if you export PYPI_MIRROR_URL=http://pypi.o.o/simple | 07:14 |
| lifeless | StevenK: sure | 07:14 |
| lifeless | StevenK: I was just reading backscroll :) | 07:14 |
| StevenK | I can see that | 07:15 |
| lifeless | and saw your http://paste.openstack.org/show/86364/ :)[6~ | 07:15 |
| lifeless | btw what did you change in https://bugs.launchpad.net/nova/+bug/1341420/+activity? lp doesn't have a diff.... | 07:15 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [High,Triaged] | 07:15 |
| StevenK | lifeless: "The race is particularly noticable with Ironic, where every request" | 07:17 |
| StevenK | You had "where ever request" | 07:17 |
| lifeless | ah thanks | 07:18 |
| *** bmahalakshmi2 has joined #tripleo | 07:18 | |
| *** bmahalakshmi has quit IRC | 07:19 | |
| *** jcoufal has joined #tripleo | 07:20 | |
| *** jistr has joined #tripleo | 07:20 | |
| *** ifarkas has joined #tripleo | 07:32 | |
| *** xuhaiwei has joined #tripleo | 07:37 | |
| lxsli | lifeless: morning! Please would you check https://review.openstack.org/#/c/92356/ ? | 07:42 |
| lxsli | beekneemech: https://review.openstack.org/#/c/105460/ please? it's fixed | 07:48 |
| lxsli | slagle: ping re: https://review.openstack.org/#/c/94876 | 07:55 |
| GheRivero | morning all | 07:56 |
| lifeless | interesting - http://www.redhat.com/about/news/archive/2014/7/red-hat-and-google-collaborate-on-kubernetes-to-manage-docker-containers-at-scale and https://github.com/GoogleCloudPlatform/kubernetes/blob/master/DESIGN.md | 07:56 |
| lifeless | morning GheRivero lxsli | 07:57 |
| *** ci-overcloud has joined #tripleo | 08:00 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 08:00 |
| *** ci-overcloud has quit IRC | 08:00 | |
| lifeless | Ng: https://bugs.launchpad.net/nova/+bug/1341420 | 08:06 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [High,Triaged] | 08:06 |
| *** derekh__ has joined #tripleo | 08:13 | |
| Ng | any cores fancy looking at https://review.openstack.org/#/c/87758/ , see if we can land it today? :) | 08:18 |
| *** sseago has quit IRC | 08:19 | |
| lifeless | Ng: trade you a review for a review of 100000 | 08:20 |
| lxsli | do we have any bash standards? I like to quote all the things | 08:20 |
| Ng | sure | 08:20 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Virsh destroy more domains https://review.openstack.org/106358 | 08:20 |
| Ng | lifeless: also, what was the virtual team task you mentioned? | 08:20 |
| lifeless | Ng: we just spoke about it :) | 08:22 |
| Ng | lifeless: aha :) | 08:22 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Set default OVERCLOUD_CONTROLSCALE to 3 https://review.openstack.org/100000 | 08:27 |
| GheRivero | hey all, regarding https://review.openstack.org/#/c/87758/ the code upgrade path is not activated by default, but was in a previous patch and passed. Should be activated by default, add another check, just check for now with check-experimental...? | 08:27 |
| *** lucasagomes has joined #tripleo | 08:28 | |
| *** Yanyanhu has quit IRC | 08:28 | |
| lifeless | its only passing CI because someone made it optional | 08:29 |
| lifeless | it shouldn't be optional :) | 08:29 |
| lifeless | GheRivero: Ng: I'd really rather see the upgrade thing always on. Is that the plan? | 08:30 |
| GheRivero | I did it :) It adds ~25 min to the test | 08:30 |
| lifeless | GheRivero: and? | 08:30 |
| lifeless | GheRivero: when will we not test it but still deploy an overcloud in CI ? | 08:30 |
| lifeless | (also, it taking 25m is kindof terrifying, but thats an optimisation problem) | 08:31 |
| GheRivero | we shouldn't, but a couple of weeks ago, wasn't an option to have it on | 08:31 |
| GheRivero | anyway, will change the default right now | 08:32 |
| lifeless | cool | 08:33 |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 08:34 |
| lxsli | lifeless: posted on bug 1341420 | 08:34 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [High,Triaged] https://launchpad.net/bugs/1341420 | 08:34 |
| *** athomas has joined #tripleo | 08:36 | |
| lifeless | lxsli: thanks! | 08:42 |
| lifeless | lxsli: replied ;) | 08:42 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Add an Architecture field to the Register Nodes dialog https://review.openstack.org/106707 | 08:43 |
| *** IvanBerezovskiy has joined #tripleo | 08:45 | |
| *** martyntaylor has joined #tripleo | 08:45 | |
| *** Yanyanhu has joined #tripleo | 08:48 | |
| *** ramishra has quit IRC | 08:58 | |
| *** jp_at_hp has joined #tripleo | 08:59 | |
| *** xuhaiwei has quit IRC | 09:00 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 09:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 09:00 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340467 | 09:00 |
| uvirtbot | Launchpad bug 1340467 in tripleo "services cannot be decoupled from the boot-stack element" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341389 | 09:00 |
| uvirtbot | Launchpad bug 1341389 in tripleo "multiple dh-ctlplane dhclient processes and undercloud drops off network" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 09:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341192 | 09:00 |
| uvirtbot | Launchpad bug 1341192 in tripleo "If MySQL initialization fails, restart is not attempted" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| *** akrivoka has joined #tripleo | 09:05 | |
| lifeless | tchaypo: did you get past the issue with an unbound variable? | 09:05 |
| *** ramishra has joined #tripleo | 09:06 | |
| tchaypo | I haven't had a stable connection all day so I haven't been able to poke at it. Should be looking at it over the next hour or two on that I've stopped moving. | 09:16 |
| tchaypo | Been using Getty to look at code reviews instead | 09:16 |
| lifeless | tchaypo: are you becoming like NobodyCam, living in a mobile home? | 09:20 |
| lifeless | righto, workaround - horrid, but appears to work | 09:23 |
| *** athomas has quit IRC | 09:23 | |
| *** lazy_prince has quit IRC | 09:23 | |
| lifeless | Ng: https://review.openstack.org/106716 may help adam_g | 09:24 |
| lifeless | derekh__: so, I have a CI overcloud deploy finally robust enough. yay. | 09:24 |
| derekh__ | lifeless: sweet :-) | 09:25 |
| lifeless | derekh__: I found a truely horrid set of interactions with the scheduler today | 09:25 |
| lifeless | derekh__: https://bugs.launchpad.net/nova/+bug/1341420 | 09:25 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [High,Triaged] | 09:26 |
| Ng | lifeless: thanks | 09:29 |
| *** athomas has joined #tripleo | 09:29 | |
| lifeless | Ng: I was shocked that it helped. | 09:29 |
| Ng | that is a horrible, horrible patch :D | 09:29 |
| lifeless | Ng: isn't it just ? | 09:31 |
| *** tserong has quit IRC | 09:31 | |
| derekh__ | lifeless: so this is causing the failed instance when creating the overcloud stack | 09:31 |
| derekh__ | ? | 09:31 |
| *** dtantsur has quit IRC | 09:31 | |
| lifeless | derekh__: yes | 09:33 |
| lifeless | derekh__: well, there was other stuff | 09:33 |
| lifeless | derekh__: we had two wrong macs | 09:33 |
| lifeless | derekh__: and one of the machines we fixed has gone bad again, so its now in maintenance mode | 09:33 |
| *** tserong has joined #tripleo | 09:34 | |
| *** tserong has joined #tripleo | 09:34 | |
| derekh__ | lifeless: ok, so it the ci-overcloud running now, anything I can do? | 09:34 |
| *** dtantsur has joined #tripleo | 09:34 | |
| derekh__ | hmm the rh1 cluod seems to be getting a lot of instances with 2 IP on each netowrk and then going to an ERROR state | 09:35 |
| lifeless | derekh__: well, waiting on heat stack-ready | 09:35 |
| derekh__ | lifeless: ok | 09:35 |
| lifeless | derekh__: at which point I think we can usefully start on the changes to devtest_overcloud.sh to gracefully encompass the needed ci-overcloud things | 09:35 |
| lifeless | derekh__: I know you had a post-deploy script | 09:35 |
| lifeless | derekh__: I'd like to at least see about integrating stuff into _overcloud - e.g. network modelling etc | 09:36 |
| lifeless | derekh__: ahh | 09:37 |
| lifeless | + ip route prepend default via 138.35.77.1 | 09:37 |
| lifeless | RTNETLINK answers: Network is unreachable | 09:37 |
| lifeless | [2014-07-14 09:36:42,947] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/opt/stack/os-config-refresh/configure.d']' returned non-zero exit status 2] | 09:37 |
| lifeless | derekh__: so os-collect-config is failing in the oc control plane | 09:37 |
| lifeless | derekh__: we need to debug that and fix my patches, clearly. | 09:38 |
| lifeless | tchaypo: I was hoping to ask you to poke at https://bugs.launchpad.net/tripleo/+bug/1341389 | 09:38 |
| uvirtbot | Launchpad bug 1341389 in tripleo "multiple dh-ctlplane dhclient processes and undercloud drops off network" [Undecided,Incomplete] | 09:38 |
| lifeless | tchaypo: which may be a local problem in my patches, or it may exist in trunk already | 09:38 |
| derekh__ | lifeless: ok, I can take a look at fixing those patches today | 09:39 |
| lifeless | cool | 09:39 |
| lifeless | I'm going to make sure my local fixes are pushed up | 09:39 |
| lifeless | and retry this stack with a fresh set of images | 09:39 |
| lifeless | now that the deploy actually, well, deploys. | 09:39 |
| derekh__ | lifeless: ok, just shout when your done and I'll jump in | 09:40 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Make use of ip route replace in ensure-bridge https://review.openstack.org/104461 | 09:40 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Set the MAC address for ensure-bridge bridges https://review.openstack.org/99604 | 09:40 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Fix issues with provider networks and public ips https://review.openstack.org/103449 | 09:40 |
| lifeless | StevenK: sorry for the rebase, I needed your patch :) | 09:40 |
| *** lazy_prince has joined #tripleo | 09:40 | |
| lifeless | derekh__: this sleep(5) may help with the virtual environment tests too, come to think of it | 09:42 |
| derekh__ | lifeless: sleep(5) ? | 09:43 |
| *** ramishra has quit IRC | 09:43 | |
| lifeless | derekh__: see the review linked at teh bottom of https://bugs.launchpad.net/tripleo/+bug/1341389 | 09:43 |
| uvirtbot | Launchpad bug 1341389 in tripleo "multiple dh-ctlplane dhclient processes and undercloud drops off network" [Undecided,Incomplete] | 09:43 |
| lifeless | derekh__: I'm deploying a 28 hypervisor, allowing the rest for testenvs, for now. | 09:45 |
| *** ramishra has joined #tripleo | 09:45 | |
| derekh__ | lifeless: there isn't any review on the bug | 09:46 |
| derekh__ | lifeless: ok | 09:46 |
| lifeless | derekh__: https://bugs.launchpad.net/nova/+bug/1341420/comments/7 | 09:47 |
| uvirtbot | Launchpad bug 1341420 in nova "gap between scheduler selection and claim causes spurious failures when the instance is the last one to fit" [High,Triaged] | 09:47 |
| derekh__ | lifeless: ahh found it, looks like you linked the wrong bug | 09:47 |
| lifeless | derekh__: oh, I did. DOH | 09:48 |
| tchaypo | lifeless: I can poke, but im not sure how I'd reproduce | 09:48 |
| tchaypo | Maybe running ensure-bridge a lot? | 09:49 |
| lifeless | tchaypo: the bug presumably? set the neutron dhcp time down to like 2 minutes and bring up a UC with vlan on your seed | 09:49 |
| lifeless | tchaypo: then see if it loses the plot | 09:49 |
| lifeless | tchaypo: and also you can run os-refresh-config --force --one a lot | 09:49 |
| lifeless | and see if that adds more dhclient processes | 09:50 |
| tchaypo | Neutron dhcp time meaning the lease time? | 09:50 |
| lifeless | tchaypo: and log into the freshly deployed UC and check whether it has one or more than one dhclients | 09:50 |
| lifeless | tchaypo: yes, you can add arbitrary settings - see the overcloud where we clamp mtu; shoudl be able to use the same to set dhcp lease time | 09:50 |
| tchaypo | Are you seeing this on trunk or with your psyched applied? | 09:51 |
| lifeless | all my psyched | 09:51 |
| lifeless | thus one of the questions about whether its a pre-existing / latent bug or local to my patches | 09:51 |
| lifeless | I'd start with my patchs where I see it to be sure the reproduction methodology is sound then check trunk | 09:51 |
| tchaypo | All your psyched. | 09:52 |
| tchaypo | I like | 09:52 |
| lifeless | tchaypo: you should, you started it :> | 09:53 |
| tchaypo | Eep | 09:53 |
| tchaypo | I did too | 09:53 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: HP1 region deploy config fixups. https://review.openstack.org/99799 | 09:56 |
| lifeless | derekh__: right, stuff pushed up, no local deltas anymore - all committed and referenced by tripleo-end-to-end | 09:57 |
| *** k4n0 has quit IRC | 09:57 | |
| lifeless | derekh__: deploy-ci-overcloud invoked and churning | 09:58 |
| lifeless | derekh__: so, in principle that comes up, and then we figure out what needs to be done post bring-up to make it usable | 09:58 |
| derekh__ | lifeless: ok, sounds good, will jump in in a few minutes | 09:59 |
| lifeless | derekh__: I'm going to keep an eye out - not going to sleep just yet. Need to get some game on :) | 10:01 |
| derekh__ | lifeless: ok | 10:01 |
| *** pelix has joined #tripleo | 10:03 | |
| *** andreaf has quit IRC | 10:07 | |
| *** akuznetsov has joined #tripleo | 10:07 | |
| *** yamahata has quit IRC | 10:08 | |
| *** k4n0 has joined #tripleo | 10:10 | |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Add the ability to create databases from HEAT metadata https://review.openstack.org/96448 | 10:14 |
| *** ramishra has quit IRC | 10:14 | |
| proffalken | lxsli, slagle: https://review.openstack.org/#/c/96448/ has the latest approach to this. I've nto been able to get a good devtest run yet to prove it works fully however I'd welcome comments on whether this is a better approach thatn previously | 10:15 |
| proffalken | s/nto/not | 10:15 |
| proffalken | It has made me have one concern though - in order to create the database on a host other than the one that my element is installed to, I need to install my element on both nodes with this approach | 10:16 |
| proffalken | reverting to using HEAT variables removes this dependency | 10:16 |
| proffalken | I'm thinking about a "classic" web stack here where you might have a DB Node, an API server that talks to it and a front-end website | 10:16 |
| lifeless | proffalken: what problem are you trying to solve? | 10:17 |
| proffalken | with the old approach, the HEAT stuff was read only on the server with the mysql-common element, whereas with this new approach the "api" element would need to also be installed on the databse server | 10:18 |
| lifeless | proffalken: there's no bug or spec on the review, so I'm having to guess | 10:18 |
| proffalken | lifeless: the specific usecase is that icinga-web (shiney PHP-based web interface to icinga) requries a MySQL database to interact with | 10:18 |
| proffalken | lifeless: good point, I'll work out what this should be an address that | 10:18 |
| lifeless | proffalken: so, I'd use trove for that. | 10:18 |
| lifeless | proffalken: if I may sketch medium term for a second | 10:18 |
| proffalken | lifeless: does trove work on the undercloud? | 10:19 |
| lifeless | proffalken: assume we have the multi-hypervisor feature landed. | 10:19 |
| proffalken | ok | 10:19 |
| lifeless | proffalken: then, we can deploy infrastructure workloads on the deploy cloud in containers/VMs/baremetal as desired. | 10:19 |
| *** ci-overcloud has joined #tripleo | 10:19 | |
| ci-overcloud | ************** ci-overcloud complete status=127 ************ | 10:19 |
| *** ci-overcloud has quit IRC | 10:19 | |
| lifeless | proffalken: all we need on the actual UC baremetal images then is the transitive closure over nova | 10:20 |
| *** ramishra has joined #tripleo | 10:20 | |
| lifeless | proffalken: to get icinga deployed we could do it as part of the deploy cloud itself, or as a privileged tenant of the deploy cloud. | 10:20 |
| lifeless | proffalken: given the goal is to make the deploy cloud a tenant of itself, there's nothing particularly special about being part of the deploy cloud itself. | 10:21 |
| proffalken | lifeless: I like the medium term approach, however in the short-term if I want to run a service which will require a MySQL (or indeed any other type!) database how do I get that database created so that I can configure the app to talk to it? | 10:21 |
| lifeless | proffalken: so - long story short - if we get trove up, we'll be able to have a heat template + narrow focused images for icinga, and run it in VM/containers in the deploy cloud. | 10:22 |
| proffalken | sounds good | 10:22 |
| lifeless | proffalken: so, in the interim, make an element that you add to your control plane image that will create the appropriate DB | 10:24 |
| lifeless | proffalken: same as bootstack does, for instance. | 10:24 |
| lifeless | proffalken: its not super flexible or dynamic, but see under medium term - I don't think we want to invest a bunch of effort in a mini-trove | 10:24 |
| proffalken | lifeless: how does that differ from my patch to MySQL common? Surely it's better to add to an existing element with a caveat that this will go away when trove arrives than to create a new one? | 10:25 |
| lifeless | proffalken: you're adding a data-driven thing | 10:26 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-image-elements: adding local boot support for uboot arm arch https://review.openstack.org/106727 | 10:26 |
| proffalken | ah, ok, so the commit message needs rewriting too | 10:26 |
| lifeless | proffalken: I'm proposing #!/bin/bash \n os-db-create foo bar $Q@#$!R | 10:26 |
| proffalken | lifeless: that's effectively what this python script is | 10:26 |
| proffalken | it's just the title of the commit message wasnt changed | 10:27 |
| proffalken | you pass it the dbname, username, password, host you will be connecting from and the permissions you want to set and it adds it all for you | 10:28 |
| *** akrivoka has quit IRC | 10:28 | |
| proffalken | I'm just concerned that in a multi-tier app you will have to install the element that needs the database onto the DB server which is bad architectural design | 10:28 |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Add the ability to create databases from within Elements https://review.openstack.org/96448 | 10:29 |
| proffalken | lifeless: I just changed the commit message ;) | 10:29 |
| *** markmc has joined #tripleo | 10:31 | |
| lifeless | hhopefully gertty has uploaded my comments | 10:31 |
| lifeless | proffalken: thats still dynamic, no? heat asserts the set of DBs, your code ensures they exist? | 10:32 |
| tchaypo | lifeless: btw, that race condition.. | 10:32 |
| tchaypo | ouch | 10:32 |
| lifeless | proffalken: I'm not dead-set against it, just concerned about layers | 10:32 |
| lifeless | tchaypo: its generating some chatter :) | 10:32 |
| tchaypo | I'm guessing it's slipped under the radar because people don't normally fill up the whole allocation of VMs? | 10:32 |
| lifeless | proffalken: and how to make sure we don't structure ourselves to reinvent something someone else is working on | 10:32 |
| lifeless | tchaypo: there's a provider, bluebox i tink | 10:33 |
| openstackgerrit | lokesh s proposed a change to openstack/diskimage-builder: Solves chroot error while building cross arch images https://review.openstack.org/103483 | 10:33 |
| lifeless | tchaypo: who use nova to do whole-machine-vms | 10:33 |
| lifeless | derekh__: new symlink needed, redoing setup. | 10:33 |
| derekh__ | lifeless: ok | 10:34 |
| *** k4n0 has quit IRC | 10:35 | |
| *** ci-overcloud has joined #tripleo | 10:35 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 10:35 |
| *** ci-overcloud has quit IRC | 10:35 | |
| *** martyntaylor has left #tripleo | 10:35 | |
| lxsli | tchaypo lifeless: this is Nova working as designed | 10:35 |
| lxsli | it's not normally an issue because requests don't come in that fast and usually don't fill whole machines | 10:36 |
| lifeless | lxsli: so, the design used to be different ;) | 10:36 |
| lifeless | devananda: have lunch with hernan please :) | 10:36 |
| tchaypo | lifeless: so to answer your earlier question - no, still stuck on BM_VLA_SEED_IP_ADDR | 10:36 |
| tchaypo | s/VLA/VLAN/ | 10:36 |
| lifeless | tchaypo: whats the exact error ? | 10:37 |
| tchaypo | so I'm going to have to figure that one out | 10:38 |
| tchaypo | http://paste.openstack.org/show/86390/ | 10:38 |
| *** ci-overcloud has joined #tripleo | 10:38 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 10:38 |
| *** ci-overcloud has quit IRC | 10:38 | |
| lifeless | bwaaah | 10:39 |
| lifeless | ok that 5 second hack was insufficient :) | 10:39 |
| lifeless | or I just got lucky. | 10:39 |
| tchaypo | what hack did you do? | 10:39 |
| lifeless | tchaypo: see the bug there's a review linked | 10:40 |
| tchaypo | or | 10:40 |
| tchaypo | oh | 10:40 |
| tchaypo | 5 second hack for the vm scheduling | 10:40 |
| lxsli | sleep is always lucky | 10:40 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-image-elements: adding local boot support for uboot arm arch https://review.openstack.org/106727 | 10:40 |
| tchaypo | ah, here we go | 10:43 |
| openstackgerrit | lokesh s proposed a change to openstack/diskimage-builder: Solves chroot error while building cross arch images https://review.openstack.org/103483 | 10:44 |
| tchaypo | lines 89-98 are where BM_VLAN_SEED_IP_ADDR *may* get set | 10:44 |
| tchaypo | but then line 185 *always* references it | 10:44 |
| *** akuznetsov has quit IRC | 10:45 | |
| lifeless | tchaypo: so bug in my patches :) | 10:45 |
| tchaypo | the problem seems to be that I don't have baremetal-network.seed.public_vlan.ip set in my heat data | 10:46 |
| tchaypo | the fix is probably either always setting it, or skipping over that bit of code if it's not set | 10:46 |
| tchaypo | probably the latter, i think | 10:46 |
| lifeless | it's optional (in that vlans are optional) but required if any of the vlan stuff is being done) | 10:46 |
| proffalken | lifeless: the user stuf was already in there, I'm not removing any of that. The idea is that you hard-code into your element "mysql-create-database <variables>" and remove the heat stuff entirely as far as database creation is concerned | 10:47 |
| lifeless | proffalken: I am confused, there is already a command to do that | 10:48 |
| *** bmahalakshmi2 has quit IRC | 10:49 | |
| lifeless | proffalken: os-db-create | 10:49 |
| proffalken | really? I couldn't find it and all requests for help resulted in "just write your own" | 10:49 |
| * proffalken abandons the patch | 10:49 | |
| tchaypo | I think if I just tweak line 185 to use ${ :- } when it references that variable the comparison will fail and the next few lines should be skipped | 10:49 |
| * tchaypo chortles | 10:49 | |
| tchaypo | thank you proffalken | 10:49 |
| lifeless | proffalken: ./elements/os-svc-install/bin/os-db-create | 10:50 |
| lifeless | proffalken: in tripleo-image-element | 10:50 |
| lifeless | proffalken: now, it may not be suitable for your needs, but it shouldn't be far off | 10:50 |
| tchaypo | proffalken: it's so nice tosee this kind of thing happening to someone who isn't me :) | 10:51 |
| lifeless | proffalken: and I fully support tweaking its interface to be a little more featureful if needed, I don't think that 'create a db here for me' is in danger of trove-ness :) | 10:51 |
| proffalken | tchaypo: heh, stick around, this happens to me a lot! :P | 10:52 |
| proffalken | lifeless: thanks, I'll take a look | 10:52 |
| *** tzumainn has joined #tripleo | 10:54 | |
| *** akuznetsov has joined #tripleo | 10:54 | |
| lazy_prince | lifeless: I sent a mail to you regarding Windows Disk Image builder on July 10th. did you get a chance to look at it..? | 10:55 |
| lazy_prince | lifeless: We did not get much dissenting voices. So shall we start with infra to start the merge proposals... ? | 10:55 |
| *** killer_prince has joined #tripleo | 11:04 | |
| StevenK | killer_prince: I'm curious about how much code it is | 11:05 |
| *** pmyers has joined #tripleo | 11:06 | |
| lazy_prince | StevenK: its around 600LoC. | 11:07 |
| openstackgerrit | lokesh s proposed a change to openstack/tripleo-image-elements: adding local boot support for uboot arm arch https://review.openstack.org/106727 | 11:07 |
| *** lazy_prince has quit IRC | 11:07 | |
| *** killer_prince is now known as lazy_prince | 11:07 | |
| *** slok184 has quit IRC | 11:08 | |
| StevenK | There's ~2,200 lines of shell and 450 of python in dib already. I wonder about just including the ~600 lines of powershell in dib itself | 11:09 |
| *** lazy_prince2 has joined #tripleo | 11:11 | |
| *** lazy_prince2 is now known as killer_prince | 11:11 | |
| lifeless | killer_prince: I thought I replied on the list | 11:12 |
| killer_prince | StevenK: we had discussed with lifeless about it during blueprint approval and decided to keep it a seperate project as there is no code reusability between linux dib and Windows dib. | 11:13 |
| killer_prince | okay let me check the list then.. | 11:13 |
| killer_prince | The last reply that i received from you was on 7th July. is that what you are referring to.. | 11:15 |
| *** killer_prince has quit IRC | 11:20 | |
| lazy_prince | lifeless: so how does it work, do i need to drop an email to infra team for a project creation..? | 11:21 |
| StevenK | lazy_prince: There's a process, I think it's been detailed on the list. | 11:22 |
| lazy_prince | aha.. are you talking about Cookie-cutter..? i will check that.. | 11:23 |
| *** andreaf has joined #tripleo | 11:25 | |
| *** lucasagomes is now known as lucas-hungry | 11:31 | |
| lazy_prince | StevenK: i just have a doubt. cookiecutter is more suited for a python project. but this is more of a powershell. | 11:31 |
| StevenK | lazy_prince: How about http://ci.openstack.org/stackforge.html ? | 11:33 |
| * tchaypo learns more about what devtest_seed.sh does | 11:35 | |
| tchaypo | of course the way I do this is by figuring out what the hell is broken | 11:36 |
| *** yamahata has joined #tripleo | 11:40 | |
| tchaypo | ho hum. why did it route the traffic there? | 11:49 |
| *** rakesh_hs has quit IRC | 11:52 | |
| *** rushiagr has quit IRC | 11:57 | |
| kviiri | How do I use the RHEL element of DIB to create a CentOS image? | 12:00 |
| *** rushiagr has joined #tripleo | 12:00 | |
| *** jdob has joined #tripleo | 12:03 | |
| *** lazy_prince has quit IRC | 12:05 | |
| *** ddieterl_ has quit IRC | 12:09 | |
| *** ddieterly has joined #tripleo | 12:10 | |
| lifeless | derekh__: kicking off another attempt with a new scheduler patch, then really -> sleep() | 12:17 |
| *** mrunge has quit IRC | 12:17 | |
| Ng | lifeless: does scheduler_host_subset_size help us here? | 12:17 |
| Ng | not super familiar with the nova scheduler, just reading the code for the filter scheduler | 12:18 |
| lifeless | I tried it with a window of 10, didn't do much | 12:18 |
| lifeless | see the bug and internal mail thread where I mention this | 12:19 |
| Ng | oh, derp, so you did | 12:19 |
| Ng | I was thinking set it insanely high, so the code caps it at len(weighed_hosts) and then you're basically saying "pick a random host from the entire list of hosts" | 12:19 |
| Ng | assuming random.choice() is random, and not debian-random ;) | 12:19 |
| *** Yanyanhu has quit IRC | 12:19 | |
| * lxsli agrees | 12:19 | |
| *** amitpp has joined #tripleo | 12:20 | |
| *** ci-overcloud has joined #tripleo | 12:20 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 12:20 |
| *** ci-overcloud has quit IRC | 12:20 | |
| lifeless | hmm, new patch is bong. I shall dive on this grenade properly tomorrow. I have some ideas that should get us by, I've also sketched a more comprehensive BASE based approach in the bug | 12:20 |
| lifeless | derekh__: sorry, no go :/ - but have a look at the next bit of scripting if you have time (merge prepare -> devtest_overcloud) | 12:21 |
| *** killer_prince has joined #tripleo | 12:21 | |
| lifeless | lxsli: can you please mail my hp email about that review you wanted? | 12:21 |
| *** killer_prince is now known as lazy_prince | 12:21 | |
| lifeless | lxsli: I'll do it tomorrow, just need a visible reminder | 12:21 |
| *** Yanyanhu has joined #tripleo | 12:30 | |
| *** rushiagr_ has joined #tripleo | 12:31 | |
| *** rushiagr has quit IRC | 12:34 | |
| *** lucas-hungry is now known as lucasagomes | 12:36 | |
| lxsli | lifeless: sure thing - sleep well | 12:36 |
| *** akrivoka has joined #tripleo | 12:36 | |
| kviiri | So has anyone managed to create a centos image using the rhel element? | 12:39 |
| *** ramishra has quit IRC | 12:39 | |
| slagle | kviiri: not that i know of. i'm not aware of anyone using centos yet | 12:43 |
| slagle | kviiri: there is however a WIP patch up to add centos7 support: https://review.openstack.org/#/c/104449/ | 12:43 |
| kviiri | slagle: Thanks. I actually knew of the patch already but there was also someone saying I could use the rhel element | 12:44 |
| slagle | ok. well, maybe someone has done it and i just don't know of it :) | 12:45 |
| kviiri | slagle: It seems to be in a good shape to be tried out | 12:45 |
| kviiri | the patch, I mean | 12:45 |
| kviiri | "WIP: image is created correctly, but seem to be some grub issues with" | 12:45 |
| kviiri | boot | 12:45 |
| *** jang has quit IRC | 12:46 | |
| *** dprince has joined #tripleo | 12:47 | |
| *** morazi has joined #tripleo | 12:47 | |
| *** shausy has quit IRC | 12:53 | |
| *** julim has joined #tripleo | 12:55 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Improve readability of long JQ expression https://review.openstack.org/99681 | 12:58 |
| tchaypo | lifeless: i made a little progress but i still can't get even my seed up. Are we planning to make some of these bm_network parameters neccessary? If so I should probably just provide them instead of fighting | 13:00 |
| tchaypo | and now it's bedtime | 13:00 |
| *** sseago has joined #tripleo | 13:01 | |
| *** Yanyanhu has quit IRC | 13:06 | |
| *** Yanyanhu has joined #tripleo | 13:06 | |
| *** ramishra has joined #tripleo | 13:08 | |
| *** ddieterly has quit IRC | 13:09 | |
| rdopieralski | tzumainn: do you know where I should be getting the list of available architectures and/or drivers from? | 13:19 |
| rdopieralski | tzumainn: or should I just hardcode them? | 13:20 |
| tzumainn | rdopieralski, hm, I assumed they'd be hardcoded, to be honest | 13:21 |
| derekh__ | lifeless: I'm not sure about merging prepare-ci-overcloud into devtest_overcloud , it doesn't feel right complicating devtest_overcloud for the specific case of preparing an overcloud for our CI case | 13:24 |
| derekh__ | lifeless: I think keeping it tacked onto the end of deploy-ci-overcloud is better (to avoid complicating devtest_overcloud | 13:25 |
| rdopieralski | tzumainn: hardcoded is fine by me :) | 13:26 |
| rdopieralski | tzumainn: then again, should I hardocde them in the form, or better in the API? | 13:26 |
| tzumainn | rdopieralski, I'd suggest the api | 13:33 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Use a MultiMACField in the Register Nodes form https://review.openstack.org/106366 | 13:33 |
| *** andreaf has quit IRC | 13:34 | |
| *** andreaf has joined #tripleo | 13:36 | |
| *** amitpp has quit IRC | 13:40 | |
| *** hewbrocca has joined #tripleo | 13:42 | |
| *** amitpp has joined #tripleo | 13:42 | |
| *** weshay has joined #tripleo | 13:47 | |
| *** andreaf has quit IRC | 13:53 | |
| lazy_prince | StevenK: can you review this https://review.openstack.org/#/c/103483/ when you get time..? | 13:57 |
| *** ddieterly has joined #tripleo | 13:57 | |
| *** imcsk8 has quit IRC | 13:57 | |
| *** imcsk8 has joined #tripleo | 13:57 | |
| *** jistr has quit IRC | 13:58 | |
| *** jistr has joined #tripleo | 13:59 | |
| *** jrist has joined #tripleo | 14:04 | |
| *** rushiagr_ has quit IRC | 14:16 | |
| *** lazy_prince is now known as killer_prince | 14:17 | |
| *** beekneemech is now known as bnemec | 14:18 | |
| *** funzo has joined #tripleo | 14:22 | |
| *** al has quit IRC | 14:26 | |
| openstackgerrit | Ed Bak proposed a change to openstack/tripleo-incubator: Provide ability to move the gateway away from the seed https://review.openstack.org/104641 | 14:30 |
| *** jistr has quit IRC | 14:35 | |
| *** jistr has joined #tripleo | 14:36 | |
| *** al has joined #tripleo | 14:36 | |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/plans endpoint to API https://review.openstack.org/106388 | 14:38 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds GET /v2/plans/<plan-uuid> endpoint https://review.openstack.org/106779 | 14:38 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds DELETE /v2/plans/<plan-uuid> endpoint https://review.openstack.org/106780 | 14:38 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: [WIP] Adds POST /v2/plans endpoint to API https://review.openstack.org/106781 | 14:38 |
| derekh__ | lxsli: Would like to get https://review.openstack.org/#/c/102248 in but looks like its missing "=" , would you mind pushing an update | 14:39 |
| derekh__ | lxsli: CI failure rate is high ATM and I think it would help | 14:39 |
| lxsli | sure | 14:39 |
| lxsli | argh, how frustrating - just saw the error | 14:39 |
| *** martyntaylor has joined #tripleo | 14:40 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Wait longer for BM node registration https://review.openstack.org/102248 | 14:40 |
| derekh__ | lxsli: thanks | 14:41 |
| lxsli | derekh__: thank you for herding CI :o | 14:41 |
| derekh__ | :-) | 14:42 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add a sysctl-append-value script https://review.openstack.org/101117 | 14:46 |
| *** rdopieralski has quit IRC | 14:46 | |
| *** pblaho has quit IRC | 14:48 | |
| lxsli | Ng GheRivero: https://review.openstack.org/#/c/102248 please? derekh thinks it might help CI | 14:49 |
| * Ng looks | 14:49 | |
| *** rushiagr_ has joined #tripleo | 14:50 | |
| *** morganfainberg_Z is now known as morganfainberg | 14:51 | |
| *** jcoufal has quit IRC | 14:54 | |
| *** marun has joined #tripleo | 15:00 | |
| *** untriaged-bot has joined #tripleo | 15:01 | |
| untriaged-bot | Untriaged bugs so far: | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 15:01 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341192 | 15:01 |
| uvirtbot | Launchpad bug 1341192 in tripleo "If MySQL initialization fails, restart is not attempted" [Undecided,New] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 15:01 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340467 | 15:01 |
| uvirtbot | Launchpad bug 1340467 in tripleo "services cannot be decoupled from the boot-stack element" [Undecided,In progress] | 15:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 15:01 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 15:01 |
| *** untriaged-bot has quit IRC | 15:01 | |
| *** edmund has joined #tripleo | 15:02 | |
| bnemec | derekh__: The dib-utils thing I mentioned is https://review.openstack.org/#/c/100004/ | 15:04 |
| bnemec | It looks like dib-utils isn't included in /opt/stack/new, so it can't be found. | 15:05 |
| *** Yanyan has joined #tripleo | 15:05 | |
| bnemec | Would you happen to know how to get that added to the image? | 15:05 |
| derekh__ | bnemec: looking | 15:06 |
| *** gilliard has joined #tripleo | 15:07 | |
| *** Yanyanhu has quit IRC | 15:07 | |
| *** Yanyan has quit IRC | 15:10 | |
| derekh__ | bnemec: looks like the patch would require dib-run-parts to already be installed on the devtest server, is that right ? | 15:17 |
| derekh__ | bnemec: we could add it to the list of things to install in incubator, one sec | 15:17 |
| bnemec | derekh__: Yeah, I added it to pull-tools in devtest: https://github.com/openstack/tripleo-incubator/blob/master/scripts/pull-tools#L24 | 15:17 |
| bnemec | But that doesn't appear to be run in CI. | 15:18 |
| derekh__ | bnemec: http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate-wrap.sh | 15:23 |
| derekh__ | bnemec: to include it in /opt/stack/new, it needs to be added there | 15:23 |
| gilliard | Am I right in thinking that if you use VMs for devtest, they are all the same spec? NODE_MEM, NODE_DISK, NODE_CPU etc? | 15:24 |
| derekh__ | then | 15:24 |
| derekh__ | bnemec: once that merges you'll have to wait until a new template image is built by nodepool, I think this is done once a day (not sure) | 15:24 |
| derekh__ | bnemec: I got a funny feeling there is a second place to add it too | 15:25 |
| derekh__ | bnemec: checking | 15:25 |
| gilliard | I think w | 15:28 |
| *** pblaho has joined #tripleo | 15:28 | |
| gilliard | I think the answe to my Q is "yes", but I'm trying to think up a nice way to make it not have to be the case. | 15:28 |
| derekh__ | bnemec: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/prepare_tripleo.sh | 15:28 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Update node detail page for Juno https://review.openstack.org/106268 | 15:29 |
| gilliard | Once patch 105802 lands, we will have a way to target specific groups of nodes using flavors. The idea being to reduce the memory requirement for someone developing tripleo. | 15:30 |
| gilliard | If anyone's been thinking/working along those lines I'd be very grateful for a discussion. | 15:31 |
| lxsli | bnemec: re https://review.openstack.org/#/c/102248/ , it's not perfect but it is a noticeable improvement | 15:31 |
| lxsli | please could we merge then iterate? | 15:32 |
| *** amitpp has quit IRC | 15:32 | |
| bnemec | lxsli: I could live with that, but I do think we should fix it. The current version leaves the potential for a race condition, except an even smaller one that would be that much harder to find. :-) | 15:34 |
| bnemec | derekh__: Great, thanks! | 15:34 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Add new sosreport element https://review.openstack.org/104593 | 15:36 |
| lxsli | bnemec: sure and if you have some ideas on how to implement the improvement (EG in patch form...) I'll happily review them | 15:37 |
| *** cwolferh_ has joined #tripleo | 15:47 | |
| *** athomas has quit IRC | 15:50 | |
| *** derekh__ is now known as derekh_ | 15:53 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-incubator: WIP: Calculate expected number of vcpus https://review.openstack.org/106797 | 15:54 |
| bnemec | lxsli: ^ | 15:54 |
| bnemec | Bah, memory has the same problem. | 15:55 |
| bnemec | Oh well, that's why WIP. :-) | 15:56 |
| *** athomas has joined #tripleo | 15:56 | |
| *** jistr has quit IRC | 15:56 | |
| jp_at_hp | so - did anyone notice the 2 patches that drop devtest.sh build time from 1700 seconds to 460 seconds? | 15:58 |
| lxsli | :O | 15:58 |
| jp_at_hp | In case anyone cares ;-) they are https://review.openstack.org/106440 and https://review.openstack.org/106443 | 15:59 |
| lxsli | hmm, scary | 15:59 |
| *** eghobo has joined #tripleo | 16:00 | |
| lxsli | jp_at_hp: is ccache a typo? | 16:00 |
| jp_at_hp | lxsli: probably not, given that it is the ccache directory. | 16:01 |
| *** yamahata has quit IRC | 16:01 | |
| lxsli | what's that then? I only know about ~/.cache/{tripleo,image-create} | 16:01 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-incubator: WIP: Calculate expected number of vcpus/mem https://review.openstack.org/106797 | 16:02 |
| lxsli | bnemec: better to stash `nova flavor-show baremetal` and awk it twice | 16:03 |
| *** eghobo has quit IRC | 16:03 | |
| derekh_ | anybody know who own | 16:05 |
| derekh_ | | cb66b0d9-40c6-4806-820f-8e31d20bb8fb | test | ACTIVE | None | Running | default-net=10.2.9.109, 66.187.229.20; tripleo-bm-test=192.168.1.101 | | 16:05 |
| derekh_ | on the rh1 ci overcloud | 16:05 |
| derekh_ | can't ssh to it, | 16:05 |
| derekh_ | will delete if not claimed | 16:05 |
| *** eghobo has joined #tripleo | 16:10 | |
| *** pcrews has joined #tripleo | 16:11 | |
| *** martyntaylor1 has joined #tripleo | 16:17 | |
| *** martyntaylor has quit IRC | 16:18 | |
| *** pblaho has quit IRC | 16:19 | |
| *** derekh_ has quit IRC | 16:20 | |
| *** eghobo has quit IRC | 16:21 | |
| *** eghobo has joined #tripleo | 16:24 | |
| *** eghobo has quit IRC | 16:25 | |
| *** eghobo has joined #tripleo | 16:25 | |
| *** dsneddon has quit IRC | 16:34 | |
| *** dsneddon has joined #tripleo | 16:35 | |
| *** andreaf has joined #tripleo | 16:36 | |
| *** martyntaylor1 has quit IRC | 16:37 | |
| *** sballe has quit IRC | 16:39 | |
| *** sballe has joined #tripleo | 16:40 | |
| *** andreaf has quit IRC | 16:40 | |
| *** martyntaylor has joined #tripleo | 16:47 | |
| lxsli | bnemec: since CI isn't checking dib-utils yet, would you consider merging https://review.openstack.org/#/c/105461/ please? | 16:49 |
| greghaynes | aye, looks like fun CI fails | 16:51 |
| greghaynes | Im guessing its the same nova scheduler issue?} | 16:52 |
| greghaynes | lxsli: is there some reason we need to merge that before it passes CI? | 16:52 |
| bnemec | This whole two copies of dib-run-parts thing is giving me a headache. I should have just started -2'ing all the changes until the transition is complete. :-P | 16:53 |
| *** andreaf has joined #tripleo | 16:53 | |
| greghaynes | oh, its in the new fancy dib-util repo | 16:54 |
| lxsli | bnemec: not really, just seems a waste rechecking when no actual checking is occurring | 16:54 |
| bnemec | lxsli: I think I'd rather hold off until the dib change passes CI (looks like the latest patch set didn't yet). That's really the only way we have to ensure the dib-utils changes work right now. :-/ | 16:56 |
| *** Guest33108 is now known as bhunter71 | 16:56 | |
| lxsli | ok np :) | 16:57 |
| lxsli | I mean, that approach makes sense | 16:57 |
| *** cinerama has joined #tripleo | 16:58 | |
| bnemec | Anybody know if we have a bug open for the arch mismatch that was happening in CI last week? | 16:58 |
| bnemec | That's why the dib-run-parts check failed on Friday. | 16:58 |
| lxsli | waaah derekh is gone | 16:59 |
| *** pblaho has joined #tripleo | 17:07 | |
| *** rushiagr_ has quit IRC | 17:07 | |
| *** regebro has quit IRC | 17:20 | |
| *** nati_ueno has joined #tripleo | 17:28 | |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Add new sosreport element https://review.openstack.org/104593 | 17:33 |
| *** pblaho has quit IRC | 17:34 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Adds passthrough config for cinder rootwrap https://review.openstack.org/97312 | 17:35 |
| *** noslzzp has quit IRC | 17:41 | |
| *** noslzzp has joined #tripleo | 17:44 | |
| *** ramishra has quit IRC | 17:46 | |
| *** pelix has quit IRC | 17:47 | |
| *** Penick has joined #tripleo | 17:53 | |
| greghaynes | TheJulia: Hey, looking at https://review.openstack.org/#/c/105862/3/elements/keepalived/bin/keepalived-status im wondering how much of the checking is actually stuff we need to be doing | 17:56 |
| greghaynes | (thinking out loud) | 17:56 |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Change the Health status piechart to Power status https://review.openstack.org/106832 | 17:57 |
| greghaynes | we basically just want to make sure mysql is reachable before proceeding, right? | 17:57 |
| greghaynes | or was there some other use case? | 17:57 |
| * TheJulia shifts mental gears | 17:58 | |
| greghaynes | If so, I think we could make that process a *lot* more simple if we just used the notify property keepalived supports to write out a file with MASTER/BACKUP | 17:58 |
| greghaynes | we can chat later if youre busy? | 17:59 |
| *** jp_at_hp has quit IRC | 17:59 | |
| TheJulia | nah, trying to wrap up a few things mentally so i can go run a few errands | 17:59 |
| greghaynes | kk, so if we just wrote out a file with that property, thats a reasonable check that keepalived is up, running, and with our vrrp config added | 18:00 |
| greghaynes | then we could do a mysqlping or similar (I think theres a patch for this?) to check that mysql is reachable | 18:01 |
| TheJulia | greghaynes: When it is first run, mysql hasn't even been intialized yet. Agreed, if its writing out the file thats reasonable to assume that its working, although I think pinging the vips might still be a good idea... just in case | 18:01 |
| TheJulia | mysqlping before openstack-init? | 18:02 |
| greghaynes | yes | 18:02 |
| TheJulia | that could do it | 18:02 |
| greghaynes | Since it seems like the two states we want to enfoce is keepalived is happy, then mysql is reachable, then do_stuff | 18:03 |
| TheJulia | exactly | 18:03 |
| greghaynes | SpamapS: lifeless curious what youall think ^? This is re: mysql starting doesnt actually mean things can hit it on the VIP (keepalived binding that is an asynchronous process) | 18:05 |
| openstackgerrit | Julia Kreger proposed a change to openstack/tripleo-image-elements: Add local_bind flag to my.cf https://review.openstack.org/105870 | 18:06 |
| greghaynes | an additional note, it might be a nice trick to put the check for keepalived after mysql starts since those do not actually depend on each other and so we start mysql ASAP | 18:07 |
| *** akrivoka has quit IRC | 18:19 | |
| *** shakamunyi has joined #tripleo | 18:20 | |
| adam_g | greghaynes, is mysql restartable with OVERCLOUD_CONTROLSCALE=3, or are we back to the re-bootstrapping issue you fixed for the single node case? | 18:32 |
| greghaynes | adam_g: it is not | 18:33 |
| greghaynes | so, yes | 18:33 |
| greghaynes | Can youall get away with CONTROLSCALE=1? | 18:33 |
| *** rwsu has joined #tripleo | 18:51 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-specs: TripleO network configuration https://review.openstack.org/97859 | 18:56 |
| *** lucasagomes has left #tripleo | 18:57 | |
| gilliard | Re: different VM sizes for different roles in devtest. My idea is to have a file defining what nodes | 18:58 |
| gilliard | there are, and their different sizes. Then if that file exists, the usual method for creating the VMs (all the same size) is skipped. | 18:58 |
| *** nati_ueno has quit IRC | 18:59 | |
| *** nati_ueno has joined #tripleo | 19:00 | |
| gilliard | The creation of flavours would need to be based on the different node types specified in that file, and flavours would be matched to nodes using nova scheduler's ComputeCapbilitiesFilter. | 19:01 |
| gilliard | So, this touches quite a few parts of devtest (most of them, I guess). Would it make sense to clarify all this in tripleo-specs before starting? | 19:03 |
| greghaynes | absolutely, or if you want a bit more buy in before commiting to a spec an ML thread | 19:04 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-incubator: Document NeutronControlPlaneID https://review.openstack.org/106844 | 19:05 |
| gilliard | greghaynes: ya that makes sense. Thanks. | 19:07 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Move rabbitmq-server cluster port https://review.openstack.org/91704 | 19:07 |
| *** jang has joined #tripleo | 19:10 | |
| *** panda has quit IRC | 19:10 | |
| *** panda has joined #tripleo | 19:11 | |
| *** nati_ueno has quit IRC | 19:24 | |
| *** nati_ueno has joined #tripleo | 19:24 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Issue in openstack-client that stops nova being in a common-venv https://review.openstack.org/93075 | 19:33 |
| dprince | greghaynes: So... I'm not super keen on the new OVERCLOUD_CONTROLSCALE setting. Having devtest default to 3 controllers seems a bit greedy | 19:38 |
| dprince | common-venv https://review.openstack.org/93075 | 19:38 |
| dprince | oops... ignore that second post... | 19:38 |
| dprince | greghaynes: I specifically thought we had discussed *not* doing this at the Atlanta summit | 19:39 |
| greghaynes | There was a small amount of chat about it on https://review.openstack.org/#/c/100000/ | 19:39 |
| greghaynes | but I thought we were supposed to be production-able? | 19:39 |
| *** jp_at_hp has joined #tripleo | 19:39 | |
| dprince | greghaynes: I'm going to propose a revert. You've got 3 HP approvals there. | 19:40 |
| dprince | greghaynes: I'd like to see at least one non-HP approval on this. | 19:40 |
| greghaynes | Im all for more discussion :) | 19:40 |
| greghaynes | so, go for it and we can chat there | 19:40 |
| greghaynes | or chat here and there ;) | 19:40 |
| dprince | greghaynes: :), Sorry if this sounds a bit aggressive | 19:41 |
| * dprince isn't meanting to pick on greg | 19:41 | |
| greghaynes | haha, its fine | 19:41 |
| greghaynes | So, when you say aggressive, do you mean in CI or for users or both? | 19:41 |
| greghaynes | I think theres a pretty straightforward argument for why we need to do it in CI | 19:41 |
| *** nati_ueno has quit IRC | 19:42 | |
| dprince | greghaynes: right, well I'm all for testing this. I just don't agree that it should be our default | 19:42 |
| *** nati_ueno has joined #tripleo | 19:42 | |
| dprince | greghaynes: I would always like to ensure a single node works as well | 19:43 |
| dprince | greghaynes: may seem like a simple case but I find it hard to do the simple things w/ TripleO. Things like simply rebooting are often broken | 19:43 |
| greghaynes | Yes, I think there was a comment somewhere about how we should probably be testing that - especially when we start trying to support upgrades | 19:43 |
| greghaynes | My thinking is - one of the big things were aming to deliver is upgrades without downtime, and it is simply not possible without that value being set | 19:45 |
| *** nati_ueno has quit IRC | 19:46 | |
| *** nati_ueno has joined #tripleo | 19:46 | |
| *** bkero has joined #tripleo | 19:48 | |
| *** nightfly has joined #tripleo | 19:50 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-incubator: Revert "Set default OVERCLOUD_CONTROLSCALE to 3" https://review.openstack.org/106852 | 19:50 |
| *** andreaf has quit IRC | 19:54 | |
| openstackgerrit | Jay Dobies proposed a change to openstack/tuskar: Added template combination functionality https://review.openstack.org/106853 | 19:55 |
| *** blkperl has joined #tripleo | 19:58 | |
| *** Penick has quit IRC | 19:59 | |
| *** cody-somerville has joined #tripleo | 20:00 | |
| *** julim_ has joined #tripleo | 20:08 | |
| *** julim has quit IRC | 20:08 | |
| *** dtantsur is now known as dtantsur|afk | 20:08 | |
| *** dprince has quit IRC | 20:14 | |
| openstackgerrit | Haneef Ali proposed a change to openstack/tripleo-image-elements: Configure keystone with apache https://review.openstack.org/102304 | 20:16 |
| *** athomas has quit IRC | 20:18 | |
| *** Penick has joined #tripleo | 20:19 | |
| *** spzala has joined #tripleo | 20:20 | |
| *** ddieterly has quit IRC | 20:24 | |
| *** jang1 has joined #tripleo | 20:24 | |
| *** morazi has quit IRC | 20:29 | |
| *** morazi has joined #tripleo | 20:34 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack-infra/tripleo-ci: WIP: Parallel building and booting https://review.openstack.org/106019 | 20:34 |
| *** julim_ has quit IRC | 20:35 | |
| *** Hefeweizen has quit IRC | 20:35 | |
| *** cinerama has quit IRC | 20:37 | |
| *** ccrouch1 has quit IRC | 20:38 | |
| *** ccrouch has joined #tripleo | 20:38 | |
| *** ccrouch has quit IRC | 20:42 | |
| *** markmc has quit IRC | 20:43 | |
| *** edmund has quit IRC | 20:46 | |
| *** cinerama has joined #tripleo | 20:48 | |
| *** ccrouch has joined #tripleo | 20:49 | |
| *** martyntaylor has quit IRC | 20:55 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340087 | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1341192 | 21:00 |
| uvirtbot | Launchpad bug 1340087 in tripleo "PXE deploy failed with nova.virt.baremetal" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1339986 | 21:00 |
| uvirtbot | Launchpad bug 1341192 in tripleo "If MySQL initialization fails, restart is not attempted" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340467 | 21:00 |
| uvirtbot | Launchpad bug 1339986 in tripleo "when CONTROLSCALE>1 , services auth is still pointing at the local ip instead of virtual" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1340776 | 21:00 |
| uvirtbot | Launchpad bug 1340467 in tripleo "services cannot be decoupled from the boot-stack element" [Undecided,In progress] | 21:00 |
| uvirtbot | Launchpad bug 1340776 in tripleo "Swift fails to start in overcloud" [Undecided,In progress] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** yamahata has joined #tripleo | 21:01 | |
| *** jdob has quit IRC | 21:06 | |
| *** cody-somerville has quit IRC | 21:13 | |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-incubator: Keystone/LDAP integration https://review.openstack.org/106175 | 21:13 |
| *** mestery has quit IRC | 21:19 | |
| *** mestery has joined #tripleo | 21:20 | |
| *** cinerama has quit IRC | 21:26 | |
| *** openstackgerrit has quit IRC | 21:31 | |
| *** openstackgerrit has joined #tripleo | 21:32 | |
| *** killer_prince has quit IRC | 21:34 | |
| *** ci-overcloud has joined #tripleo | 21:37 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 21:37 |
| *** ci-overcloud has quit IRC | 21:37 | |
| *** cinerama has joined #tripleo | 21:39 | |
| lifeless | SpamapS: ping | 21:39 |
| *** shakamunyi has quit IRC | 21:40 | |
| *** killer_prince has joined #tripleo | 21:41 | |
| *** ci-overcloud has joined #tripleo | 21:41 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 21:41 |
| *** ci-overcloud has quit IRC | 21:41 | |
| SpamapS | lifeless: pong wassup? | 21:41 |
| *** killer_prince is now known as lazy_prince | 21:41 | |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-incubator: Parallelise image builds https://review.openstack.org/106443 | 21:41 |
| greghaynes | lifeless: if you havent seen https://review.openstack.org/#/c/106852/ might be worth a look | 21:41 |
| lifeless | SpamapS: hey hey hey, I want to talk about heat asserting global things like networks | 21:42 |
| lifeless | SpamapS: in the context of os-cloud-config | 21:42 |
| SpamapS | lifeless: cool | 21:42 |
| SpamapS | lifeless: have to step away just for < 10 min.. ask away, will respond the moment I sit back down. | 21:43 |
| lifeless | greghaynes: commented | 21:44 |
| jp_at_hp | lifeless: would appreciate eyes on that change - parallelisation of devtest image builds | 21:44 |
| lifeless | oh joy | 21:50 |
| lifeless | | fault | {"message": "Timed out waiting for a reply to message ID 2b86c058c4f240ec918b70da892984d1", "code": 500, "details": " File \"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/conductor/manager.py\", line 607, in build_instances | | 21:50 |
| lifeless | can't make the scheduler wait. That times out. sheese. | 21:50 |
| *** nati_ueno has quit IRC | 21:54 | |
| *** nati_ueno has joined #tripleo | 21:55 | |
| lifeless | jp_at_hp: reviewed | 21:57 |
| lifeless | jp_at_hp: at a high level | 21:57 |
| lifeless | jp_at_hp: hopin to start a discussion | 21:57 |
| lifeless | SpamapS: so, setup-neutron | 21:57 |
| *** nati_ueno has quit IRC | 21:57 | |
| lifeless | SpamapS: it occurs to me that that is very close to a heat template describing needed networks | 21:57 |
| *** nati_ueno has joined #tripleo | 21:57 | |
| *** Penick has quit IRC | 21:57 | |
| lifeless | SpamapS: and I'm wondering whether we're at the point of wanting to assert global networks (and perhaps other things) via heat | 21:58 |
| lifeless | SpamapS: I'm very concerned about its tendancy to take its toys home when things go wrong, since deleting out the global default network would be B A D | 21:58 |
| lifeless | SpamapS: and we can't do 'reliable in aggregate' on these things. | 21:58 |
| lifeless | SpamapS: also we'd want very strict update-only-never-replace policies on the things we'd be asserting | 21:59 |
| lifeless | SpamapS: IMNSHO | 21:59 |
| *** Penick has joined #tripleo | 22:01 | |
| tchaypo | lifeless: my latest (https://review.openstack.org/#/c/106882/) extends your stack | 22:01 |
| tchaypo | of gertty fixes | 22:01 |
| tchaypo | gertty no longer crashes when you click on unclickable things | 22:01 |
| lifeless | tchaypo: you probably want that low down so that its not floating on top of unmergables | 22:02 |
| lifeless | tchaypo: do you want me to sink it down to an appropriate place and push it up ? | 22:02 |
| lifeless | slagle: ping :) - multi hypervisor | 22:02 |
| *** jang1 has quit IRC | 22:02 | |
| tchaypo | lifeless: or possibly sync it up to an appropriate place and push it down? | 22:03 |
| tchaypo | sure, I could do just as easily though | 22:03 |
| tchaypo | ah poop. | 22:04 |
| *** nati_uen_ has joined #tripleo | 22:04 | |
| *** nati_ueno has quit IRC | 22:05 | |
| *** kevinbenton has left #tripleo | 22:05 | |
| tchaypo | less poop. | 22:05 |
| *** jp_at_hp has quit IRC | 22:05 | |
| tchaypo | i noticed that I seem to have updated your whole stack | 22:05 |
| SpamapS | lifeless: interesting | 22:05 |
| tchaypo | but that looks like it's just becaue I reabsed them all. | 22:05 |
| *** nati_uen_ has quit IRC | 22:05 | |
| *** nati_ueno has joined #tripleo | 22:06 | |
| SpamapS | lifeless: I'm pretty sure the best way to counteract "take its toys and go" is convergence. | 22:06 |
| lifeless | SpamapS: yes, thus I guess I'm less asking if this is a good idea and more whether we should stay with our current stuff for now | 22:06 |
| SpamapS | lifeless: note that zaneb is zeroing in on "retry any failure" | 22:06 |
| *** ccrouch has quit IRC | 22:11 | |
| * SpamapS needs to push up what should be final rev of convergence specs | 22:14 | |
| *** nati_ueno has quit IRC | 22:16 | |
| *** nati_ueno has joined #tripleo | 22:17 | |
| *** lazy_prince has quit IRC | 22:19 | |
| *** Shrews has quit IRC | 22:20 | |
| *** cinerama has quit IRC | 22:21 | |
| *** Shrews has joined #tripleo | 22:21 | |
| lifeless | tchaypo: where did you get to with the seed ? | 22:22 |
| tchaypo | I got it to not abort on the unbound variable, then i tweaked it some more so that it sets up the primary route and then considers adding the bm route seperately | 22:23 |
| tchaypo | so now i get a route to my seed | 22:23 |
| tchaypo | now I'm dingging into why I get "Could not find admin role" when running setup-endpoints | 22:23 |
| *** killer_prince has joined #tripleo | 22:23 | |
| lifeless | tchaypo: is there a patch somewhere for your incremental changes? they sounds a little odd to me | 22:24 |
| *** killer_prince is now known as lazy_prince | 22:24 | |
| tchaypo | but - the patch I had to make to set up the primary route - should I push that as a seperate review? | 22:24 |
| lifeless | tchaypo: pastebin perhaps? I want to eyeball it | 22:24 |
| lifeless | tchaypo: I'd just have to fold it into my patch if its needed | 22:25 |
| lifeless | tchaypo: so there's no point cluttering gerrit with it | 22:25 |
| tchaypo | the main reason I spllit them out is that the way I fixed the unbound variable problem ends up generating an empty string | 22:25 |
| *** ccrouch has joined #tripleo | 22:25 | |
| tchaypo | so the second half of the check becomes "-o $BM_NETWORK_GATEWAY = ]" and bash complains about "unary operator expected" | 22:25 |
| tchaypo | with the benefit of sleep i think I know how to fix that | 22:26 |
| lifeless | -o "$BM_NETWORK_GATEWAY" | 22:26 |
| lifeless | tchaypo: anyhow, let me see the diffs please :) | 22:26 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-specs: Decouple OS services from boot-stack element https://review.openstack.org/106885 | 22:27 |
| tchaypo | http://paste.openstack.org/show/86460/ | 22:27 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add public VLAN support. https://review.openstack.org/103757 | 22:29 |
| tchaypo | http://paste.openstack.org/show/86461/ | 22:30 |
| lifeless | tchaypo: ok, your change is wrong | 22:31 |
| lifeless | tchaypo: we don't want to set *either route* unless the default gateway on the deployed nodes will be the seed. | 22:31 |
| *** sseago has quit IRC | 22:32 | |
| tchaypo | with the benefit of sleep, I realised that all I needed to do was to add the " around the ${ :- } and my original change would have worked | 22:32 |
| tchaypo | http://paste.openstack.org/show/86462/ | 22:33 |
| lifeless | tchaypo: well, other than being incorrect :) | 22:33 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Add public VLAN support. https://review.openstack.org/103757 | 22:34 |
| tchaypo | ^^ that's my original change | 22:34 |
| lifeless | tchaypo: the line 185 test is looking to see if there is a real network or if we're emulating by having the seed be a router | 22:34 |
| lifeless | tchaypo: try my update please | 22:35 |
| tchaypo | I'm sure it will work. | 22:36 |
| lifeless | tchaypo: nobody expects the spanish inquisition | 22:36 |
| lifeless | so with controlscale=3 | 22:36 |
| tchaypo | okay, that's running. | 22:37 |
| lifeless | I suspect the VLAN stuff is now going to be blocked on the public VIP work again :/ | 22:37 |
| lifeless | hah | 22:37 |
| lifeless | ERROR: The specified reference "controller0_VLANPort" (in controller0_99_VLANPort.Properties.input_values.vlan_port.Fn::Join[1][0].Fn::Select[1].Fn::Select[1]) is incorrect. | 22:37 |
| *** ci-overcloud has joined #tripleo | 22:37 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 22:37 |
| *** ci-overcloud has quit IRC | 22:37 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Expose public_interface_tag support. https://review.openstack.org/103454 | 22:39 |
| lifeless | greghaynes: btw did you see my q about conditional stuff with different SCALE ? | 22:40 |
| greghaynes | no? | 22:40 |
| lifeless | greghaynes: oh ok so - | 22:40 |
| lifeless | greghaynes: I had a terrifying moment when I thought that maybe we're installing different software, or configuring RMQ/mysql differently depending on controlscale | 22:41 |
| lifeless | greghaynes: e.g. not installing keepalived and haproxy on scale=1 | 22:41 |
| greghaynes | We shouldnt be | 22:41 |
| lifeless | greghaynes: or not enabling galera on scale=1 | 22:41 |
| greghaynes | er, more clear, we should be doing mostly the same thing reguardless of scale | 22:41 |
| greghaynes | We dont run some commands in rmq to join with the rest of the cluster (obviously) and galera is a permanent master... | 22:42 |
| greghaynes | but other than that its basically the same AFAIK | 22:42 |
| lifeless | greghaynes: cool | 22:42 |
| *** ci-overcloud has joined #tripleo | 22:42 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 22:42 |
| *** ci-overcloud has quit IRC | 22:42 | |
| lifeless | heh | 22:42 |
| lifeless | ERROR: Stack ci-overcloud already has an action (DELETE) in progress. | 22:43 |
| greghaynes | lifeless: Did you see my discussion with TheJulia about ensuring keepalived and then mysql is alive and well before proceeding to db-create | 22:44 |
| lifeless | no | 22:44 |
| greghaynes | up at UTC 18:00 | 22:44 |
| greghaynes | and re: https://review.openstack.org/#/c/105862/ | 22:44 |
| greghaynes | I suggested we just ensure that keepalived thinks it is either in master or backup state for our vrrp instance, and then ensure we can mysqlping on VIP, | 22:45 |
| greghaynes | I know theres been a lot of talk about how much we wan to do checks like that in o-r-c though so curious what you and SpamapS thought | 22:46 |
| *** marun has quit IRC | 22:47 | |
| lifeless | so I think there is confusion here | 22:48 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Update node index view to match Juno wireframes https://review.openstack.org/106889 | 22:48 |
| lifeless | keepalived status doesn't matter to mysql. | 22:48 |
| greghaynes | correct | 22:48 |
| *** marun has joined #tripleo | 22:48 | |
| greghaynes | I was thinking start keepalived -> start mysql -> wait for keepalived alive -> wait for mysql pingable -> proceed | 22:49 |
| greghaynes | We could just not check keepalived at all though... if thats what youre thinking. It does make things a bit more annoying to debug as pretty much everything depends on it being alive | 22:51 |
| lifeless | reviewed | 22:55 |
| devananda | greghaynes: I'm cancelling one of your ironic meetup orders (you have two) | 22:55 |
| greghaynes | devananda: Ah, woops | 22:56 |
| greghaynes | ty | 22:56 |
| tchaypo | and on that note | 22:57 |
| tchaypo | anyone coming to the tripleo meetup needs to add their name to https://etherpad.openstack.org/p/juno-midcycle-meetup to confirm attendance (and dietary requirements) at dinner on wednesday night | 22:57 |
| tchaypo | lifeless: StevenK: I notice you aren't on the list ^^^ | 22:57 |
| greghaynes | tchaypo: I see lifeless | 22:58 |
| greghaynes | oh | 22:58 |
| greghaynes | dinner | 22:58 |
| greghaynes | not attendees | 22:58 |
| tchaypo | yes, dinner, the new list down the bottom | 22:58 |
| greghaynes | probably TheJulia ^ on there too | 22:58 |
| *** nati_ueno has quit IRC | 23:01 | |
| *** nati_ueno has joined #tripleo | 23:03 | |
| rwsu | lifeless: hello, please take a look at my reply to your comment in https://review.openstack.org/#/c/104692/ | 23:06 |
| *** nati_ueno has quit IRC | 23:07 | |
| *** ci-overcloud has joined #tripleo | 23:07 | |
| ci-overcloud | ************** ci-overcloud complete status=1 ************ | 23:07 |
| *** ci-overcloud has quit IRC | 23:07 | |
| *** nati_ueno has joined #tripleo | 23:07 | |
| *** sseago has joined #tripleo | 23:08 | |
| lifeless | rwsu: I'm still quite confused here - is this end user ports or in-cluster ports. What relevance does X have here? | 23:08 |
| rwsu | lifeless: this is end user ports | 23:10 |
| rwsu | lifeless: port 6000 is a X port , it really shouldn't impact us, because normally people wouldn't be running X with swift | 23:11 |
| lifeless | rwsu: so whats wrong with the default port numbers? I *know* they can be changed, but a founding principle is to run defaults. | 23:11 |
| lifeless | rwsu: we're a bit all over the map right now with the HA work, but i'm hoping we can bring it all back together quite soon | 23:12 |
| rwsu | lifeless: but the selinux policy developers don't want to open port 6000 to swift as it may pose a security risk if someone decides to run X with swift | 23:12 |
| lifeless | rwsu: so they should talk to swift upstream | 23:12 |
| lifeless | rwsu: because they're breaking the defaults if someone grabs swift from upstream | 23:13 |
| rwsu | lifeless: I think Pete (one of the swift dev) tried that once, but upstream swift says, just change the configuration files, they don't want to change the defaults | 23:14 |
| lifeless | so users lose out here | 23:14 |
| rwsu | lifeless: that's true for folks running from source, but for rdo the swift package will default to the 620x ports to match the selinux policy that allows 620x to swift | 23:15 |
| lifeless | sorry, I'm on a call now, I need to focus on that, will ping when I'm back | 23:15 |
| rwsu | ok | 23:15 |
| *** marun is now known as marun_afk | 23:17 | |
| SpamapS | rwsu: so they're violating RFC's and usurping IANA assigned port #'s. That's a _clear_ bug. | 23:18 |
| SpamapS | rwsu: "just change the config file" == head in the sand | 23:19 |
| rwsu | SpamapS: yeah, we haven't been able to get much traction asking them to change the default, we can try again :) | 23:20 |
| SpamapS | rwsu: but where is their rationale documented on violating IANA port assignments? | 23:21 |
| rwsu | SpamapS: I don't know | 23:23 |
| SpamapS | rwsu: Maybe I'm a little sadistic, but I like to make people publicly state why they want to disagree with "The internet". ;) | 23:24 |
| *** sseago has quit IRC | 23:29 | |
| lifeless | rwsu: so we've had a fairly consistent answer in tripleo which is that when *we* change the default, we do so with a bug reference to an upstream bug, *unless* its something where changing it really is needed. | 23:30 |
| lifeless | our goal is to build a working-by-default cloud, not the monster of a thousand knobs | 23:31 |
| rwsu | lifeless: I can file a bug upstream, to get more documentation around this issue, but if they come back again with no we can't change the default ports, is the patch I'm proposing reasonable? | 23:34 |
| lifeless | rwsu: no | 23:36 |
| lifeless | rwsu: you're making these ports a user visible feature | 23:37 |
| lifeless | rwsu: my objection is not to having the ports different to swift defaults - there is a process for that, and if we follow it I'm ok with that. | 23:37 |
| lifeless | rwsu: my objection is that this shouldn't be a user visible feature; there's simply no need to change the ports - externally, there is a VIP address and keystone knows the ports. | 23:38 |
| lifeless | rwsu: I'm concerned about surfacing lots of configuration details in the UI because it detracts from the UI - the plumbing-layer-UI can set these things already. | 23:39 |
| lifeless | rwsu: the issue for RDO AIUI is that you want to change it independently from *user* choice, you want it to be changed *on RDO only*, which you'd be doing by having a patch to tuskar/t-h-t, right ? | 23:40 |
| lifeless | rwsu: so an alternative I would be happy with is to set the values to the values that should work for everyone directly in tie or tht, with a reference to the swift bug. | 23:41 |
| lifeless | rwsu: there is of course another related discussion we perhaps should have | 23:41 |
| lifeless | rwsu: which is why these are dedicted ports at all- as APIs we should be able to put everything behind one port 80/443 url prefix | 23:42 |
| lifeless | https://thiscloud:443/swift/v1/... | 23:42 |
| lifeless | rwsu: which probably qualifies as a cross-project discussion but might be better in all regards | 23:42 |
| rwsu | lifeless: thanks I understand your concerns better now | 23:43 |
| rwsu | lifeless: I'll post a bug upstream and do what you suggest, and propose to change the port numbers in tie | 23:44 |
| *** yamahata has quit IRC | 23:45 | |
| rwsu | lifeless: for rdo, the configuration files in the packages will default to a different set of ports 620x range, the patch I was proposing would provide a way for us to configure tripleo to also use those ports. but having tie default to 620x ports would achieve the same goal | 23:46 |
| rwsu | lifeless: I had originally proposed the change the ports numbers in tie in https://review.openstack.org/#/c/102687/, but Gerry had some concerns about the swift port numbers being already encoded in existing swift deployments | 23:47 |
| rwsu | and changing the default would break existing users | 23:47 |
| rwsu | lifeless: are we at a point where we need to worry about breaking exisitng swift deployments using tripleo? | 23:50 |
| *** nosnos has joined #tripleo | 23:50 | |
| lifeless | thats a really good point | 23:53 |
| lifeless | I think we are | 23:53 |
| lifeless | so we'll need to listen to both ports for some deprecation period | 23:53 |
| *** jml has quit IRC | 23:54 | |
| lifeless | sda 0.00 45.75 0.00 58.75 0.00 3098.00 105.46 8.51 145.02 0.00 145.02 8.20 48.20 | 23:57 |
| lifeless | we are managing to push a single uc node a little hard | 23:58 |
| *** morazi has quit IRC | 23:58 | |
| *** jml has joined #tripleo | 23:58 | |
| lifeless | sda 0.00 425.50 0.00 351.25 0.00 98031.00 558.18 144.97 403.35 0.00 403.35 2.85 100.00 | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!