| *** rongze has quit IRC | 00:00 | |
| *** NobodyCam_afk is now known as NobodyCam | 00:04 | |
| *** jayg is now known as jayg|g0n3 | 00:04 | |
| *** cd-undercloud has joined #tripleo | 00:27 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 00:27 |
|---|---|---|
| *** cd-undercloud has quit IRC | 00:27 | |
| *** matsuhashi has joined #tripleo | 00:29 | |
| *** CaptTofu has quit IRC | 00:38 | |
| *** CaptTofu has joined #tripleo | 00:39 | |
| *** bcrochet is now known as bcrochet|whodat | 00:45 | |
| *** bcrochet|whodat is now known as bcrochet | 00:45 | |
| *** sdake_ has quit IRC | 00:47 | |
| *** sdake has quit IRC | 00:47 | |
| *** matsuhashi has quit IRC | 00:50 | |
| *** matsuhashi has joined #tripleo | 00:50 | |
| *** michchap_ has quit IRC | 00:53 | |
| *** matsuhas_ has joined #tripleo | 00:53 | |
| *** michchap has joined #tripleo | 00:53 | |
| *** matsuhashi has quit IRC | 00:54 | |
| *** krotscheck has quit IRC | 00:54 | |
| *** marun has joined #tripleo | 00:55 | |
| lifeless | if anyone needs me, I'm at the doctors - ring | 01:01 |
| *** marun has quit IRC | 01:02 | |
| *** sdake has joined #tripleo | 01:02 | |
| *** sdake has quit IRC | 01:15 | |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-refresh-config: `os-refresh-config` README enhancement https://review.openstack.org/59318 | 01:19 |
| *** nosnos has joined #tripleo | 01:26 | |
| *** rongze has joined #tripleo | 01:27 | |
| *** cd-undercloud has joined #tripleo | 01:27 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 01:27 |
| *** cd-undercloud has quit IRC | 01:27 | |
| *** rongze has quit IRC | 01:31 | |
| *** jcooley_ has joined #tripleo | 01:37 | |
| *** CaptTofu has quit IRC | 01:42 | |
| *** CaptTofu has joined #tripleo | 01:43 | |
| *** jcooley_ has quit IRC | 01:45 | |
| *** kui has joined #tripleo | 01:46 | |
| *** arata has joined #tripleo | 01:54 | |
| *** jcooley_ has joined #tripleo | 02:01 | |
| *** jcooley_ has quit IRC | 02:02 | |
| *** rongze has joined #tripleo | 02:06 | |
| *** vkozhukalov has joined #tripleo | 02:19 | |
| *** CaptTofu has quit IRC | 02:21 | |
| *** CaptTofu has joined #tripleo | 02:21 | |
| lifeless | back | 02:27 |
| *** cd-undercloud has joined #tripleo | 02:28 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 02:28 |
| *** cd-undercloud has quit IRC | 02:28 | |
| *** matsuhas_ has quit IRC | 02:31 | |
| *** matsuhashi has joined #tripleo | 02:31 | |
| *** sdake has joined #tripleo | 02:32 | |
| *** ccrouch has joined #tripleo | 02:50 | |
| *** sdake_ has joined #tripleo | 02:56 | |
| *** sdake_ has quit IRC | 02:56 | |
| *** sdake_ has joined #tripleo | 02:56 | |
| *** CaptTofu has quit IRC | 02:56 | |
| *** CaptTofu has joined #tripleo | 02:57 | |
| *** nosnos has quit IRC | 02:58 | |
| *** nosnos has joined #tripleo | 02:59 | |
| *** cwolferh has quit IRC | 03:27 | |
| *** jcooley_ has joined #tripleo | 03:29 | |
| *** cd-undercloud has joined #tripleo | 03:31 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 03:31 |
| *** cd-undercloud has quit IRC | 03:31 | |
| *** matsuhashi has quit IRC | 03:42 | |
| *** sdake has quit IRC | 03:46 | |
| *** sdake has joined #tripleo | 03:46 | |
| *** sdake has quit IRC | 03:46 | |
| *** sdake has joined #tripleo | 03:46 | |
| *** arata has left #tripleo | 03:55 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Use cleanup_flavor in setup-baremetal. https://review.openstack.org/59512 | 04:09 |
| *** michchap has quit IRC | 04:21 | |
| *** michchap has joined #tripleo | 04:21 | |
| *** matsuhashi has joined #tripleo | 04:25 | |
| *** rongze has quit IRC | 04:31 | |
| *** CaptTofu has quit IRC | 04:34 | |
| *** cd-undercloud has joined #tripleo | 04:35 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 04:35 |
| *** cd-undercloud has quit IRC | 04:35 | |
| *** CaptTofu has joined #tripleo | 04:35 | |
| *** matsuhashi has quit IRC | 05:00 | |
| *** jcoufal has joined #tripleo | 05:07 | |
| *** matsuhashi has joined #tripleo | 05:08 | |
| *** rushiagr has joined #tripleo | 05:14 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Drop default distribution root element support https://review.openstack.org/58601 | 05:20 |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Revert "Store DIB_* only env variables" https://review.openstack.org/59621 | 05:22 |
| openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Revert "Store DIB_* only env variables" https://review.openstack.org/59622 | 05:25 |
| *** matsuhashi has quit IRC | 05:27 | |
| *** rpodolyaka1 has joined #tripleo | 05:36 | |
| lifeless | rpodolyaka1: hi! | 05:37 |
| *** cd-undercloud has joined #tripleo | 05:38 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 05:38 |
| *** cd-undercloud has quit IRC | 05:38 | |
| rpodolyaka1 | lifeless: morning! | 05:39 |
| rpodolyaka1 | morning all | 05:39 |
| rpodolyaka1 | lifeless: it's hard to intersect with you online these days :) | 05:40 |
| rpodolyaka1 | lifeless: ok, so on nova-bm... I thought we were going to have some flag passed to rebuild API method and then down the stack to work_on_disk() | 05:41 |
| rpodolyaka1 | but it seems to be odd, as I have to pass it as a kernel parameter, and then return it back to baremetal deploy helper | 05:41 |
| rpodolyaka1 | and there seems to be no other way to do this except putting it to one of DB tables | 05:42 |
| rpodolyaka1 | (I mean, baremetal-deploy-helper is a separate process, that just listens on 10000 for HTTP POSTs from nodes) | 05:43 |
| rpodolyaka1 | on the other hand, having a flag is ok | 05:43 |
| rpodolyaka1 | but when do we want to set it? in the handler of API rebuild method? or when a baremetal node is created (so all its rebuilds will preserve ephemeral)? | 05:44 |
| lifeless | rpodolyaka1: so nearly | 05:44 |
| lifeless | rpodolyaka1: API -> driver; driver -> DB; then deploy helper with the changes I've put up | 05:45 |
| lifeless | rpodolyaka1: so you're like 99% right | 05:45 |
| rpodolyaka1 | ok, so are we postponing libvirt support a bit? and what the 1% left I am missing? :) | 05:46 |
| rpodolyaka1 | btw, your comment on possible race condition - good point. I was thinking about that too | 05:47 |
| lifeless | rpodolyaka1: so the 1% a) needing a rebuild method on the driver to avoid the race condition and b) using the DB rather than kernel parameter tunneling | 05:48 |
| rpodolyaka1 | ok. I think, we could for some kind of driver "extension". I.e. check if a driver supports 'native rebuild', call it, if not, do the existing procedure of destroy() + spawn() | 05:50 |
| rpodolyaka1 | basically, change as little code as possible | 05:50 |
| lifeless | rpodolyaka1: huh? I'd do it in the base class | 05:50 |
| lifeless | rpodolyaka1: there is 20 LOC right ? | 05:50 |
| lifeless | rpodolyaka1: so we make a new base driver method rebuild(...) and we put those 20 LOC in there. | 05:50 |
| rpodolyaka1 | which base do you mean? driver in virt? | 05:50 |
| SpamapS | hm | 05:50 |
| lifeless | virt.driver | 05:50 |
| SpamapS | no dnsmasq for neutron-dhcp-agent again | 05:51 |
| lifeless | rpodolyaka1: then, we override that baremetal; probably with one helper for the code between destroy and spawn | 05:51 |
| rpodolyaka1 | lifeless: (precoffee) | 05:52 |
| rpodolyaka1 | lifeless: yeah, seems reasonable | 05:52 |
| lifeless | the rebuild method on driver would accept preserve_ephemeral as a parameter | 05:53 |
| lifeless | so that avoids the race | 05:53 |
| lifeless | and lets us set the preserve value in the DB + initiate a deploy | 05:53 |
| SpamapS | dnsmasq is segfaulting | 05:54 |
| SpamapS | write(11, "<30>Dec 3 05:54:31 dnsmasq-dhcp[29600]: DHCPOFFER(tap56efc4c6-55) 10.10.16.171 "..., 98) = 98 | 05:54 |
| SpamapS | --- SIGSEGV (Segmentation fault) @ 0 (0) --- | 05:54 |
| lifeless | rotfl | 05:54 |
| lifeless | it's never easy | 05:55 |
| SpamapS | neutron-dhcp-agent never notices | 05:55 |
| rpodolyaka1 | lifeless: ack | 05:55 |
| rpodolyaka1 | I know, our deployers used some patches for dnsmasq | 05:55 |
| rpodolyaka1 | not sure if they were related to segfaults though | 05:55 |
| lifeless | rpodolyaka1: if we get pushback on that from nova-core, we can discuss it in more detail with them | 05:56 |
| lifeless | rpodolyaka1: who would have those patches ? | 05:56 |
| lifeless | SpamapS: filing a bug on Neutron ? | 05:56 |
| rpodolyaka1 | lifeless: sure, going to hack on this week, was side tracked mostly all the time last week :( | 05:57 |
| rpodolyaka1 | lifeless: I meant deployers here at mirantis, I'm going to ask one of my colleagues about problems they had with dnsmasq | 05:58 |
| lifeless | rpodolyaka1: yeah, I figure thats what you meant :) | 05:59 |
| *** matsuhashi has joined #tripleo | 05:59 | |
| rpodolyaka1 | oh, drop default distribution root element support is merged to diskimage-builder | 06:01 |
| rpodolyaka1 | was waiting for it before doing a release of dib | 06:01 |
| rpodolyaka1 | btw, in case I don | 06:02 |
| rpodolyaka1 | don't make tonights meeting, I'm taking releases for the next week too :) | 06:03 |
| lifeless | rpodolyaka1: awesome, thanks! | 06:03 |
| SpamapS | lifeless: this is the problem with having a frozen undercloud :-P | 06:05 |
| SpamapS | lifeless: reporting bugs on pre-havana code base... dicey | 06:05 |
| rpodolyaka1 | and on neutron bugs. the fix for policies bug was merged (though it is stated that it is partial in the commit message), and the one for multiple agents entries hasn't been reviewed yet. Review time in neutron is frustrating :( | 06:05 |
| lifeless | Median wait time: 5 days, 20 hours, 31 minutes | 06:06 |
| lifeless | 3rd quartile wait time: 20 days, 15 hours, 57 minutes | 06:06 |
| lifeless | dkehn: hey, if you're not already, I suggest you start doing two code-reviews a day in Neutron | 06:07 |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-refresh-config: `os-refresh-config` README enhancement https://review.openstack.org/59318 | 06:08 |
| lifeless | dkehn: thats would put you at the same rate or more, than most of neutron-core, and should [once you're up to speed on the local culture etc] significantly help Neutron. | 06:08 |
| *** arata has joined #tripleo | 06:15 | |
| *** noslzzp has quit IRC | 06:25 | |
| *** jpeeler has quit IRC | 06:34 | |
| *** jpeeler has joined #tripleo | 06:37 | |
| *** vkozhukalov has quit IRC | 06:47 | |
| *** tzumainn has quit IRC | 06:58 | |
| *** sdake_ has quit IRC | 07:09 | |
| *** toci-bot has joined #tripleo | 07:11 | |
| toci-bot | ERROR during toci run, see http://54.228.118.193/toci/toci_logs_P7VbroC/ | 07:11 |
| *** toci-bot has quit IRC | 07:11 | |
| lifeless | ruh roh | 07:14 |
| *** lsmola has joined #tripleo | 07:16 | |
| *** akuznetsov has quit IRC | 07:17 | |
| *** vipul has quit IRC | 07:32 | |
| *** vipul has joined #tripleo | 07:33 | |
| *** jcooley_ has quit IRC | 07:35 | |
| *** jcooley_ has joined #tripleo | 07:36 | |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/diskimage-builder: Modify /etc/selinux/config if it exists https://review.openstack.org/59642 | 07:41 |
| *** jprovazn has joined #tripleo | 07:43 | |
| *** rpodolyaka1 has quit IRC | 07:51 | |
| GheRivero | morning TripleO | 07:53 |
| *** jtomasek has joined #tripleo | 07:54 | |
| shadower | hey | 07:54 |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-apply-config: os-apply-config readme enhancement https://review.openstack.org/59320 | 07:59 |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-apply-config: os-apply-config readme enhancement https://review.openstack.org/59320 | 08:00 |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-apply-config: os-apply-config readme enhancement https://review.openstack.org/59320 | 08:04 |
| *** jtomasek has quit IRC | 08:07 | |
| *** jtomasek has joined #tripleo | 08:09 | |
| lifeless | shadower: GheRivero: o/ | 08:10 |
| shadower | hey lifeless | 08:12 |
| GheRivero | hey lifeless and shadower | 08:13 |
| *** pblaho has joined #tripleo | 08:20 | |
| *** rdopieralski has joined #tripleo | 08:24 | |
| *** vkozhukalov has joined #tripleo | 08:27 | |
| *** jcooley_ has quit IRC | 08:36 | |
| *** jcooley_ has joined #tripleo | 08:38 | |
| *** pblaho has quit IRC | 08:40 | |
| *** akrivoka has joined #tripleo | 08:45 | |
| *** jistr has joined #tripleo | 08:48 | |
| *** derekh has joined #tripleo | 08:57 | |
| *** rpodolyaka has quit IRC | 09:02 | |
| *** lynxman has quit IRC | 09:02 | |
| *** boris-42 has joined #tripleo | 09:03 | |
| *** rpodolyaka has joined #tripleo | 09:05 | |
| *** max_lobur_afk is now known as max_lobur | 09:14 | |
| Ng | morning | 09:18 |
| *** martyntaylor has joined #tripleo | 09:19 | |
| *** rushiagr has quit IRC | 09:28 | |
| *** rushiagr has joined #tripleo | 09:28 | |
| *** athomas has joined #tripleo | 09:29 | |
| *** lucasagomes has joined #tripleo | 09:32 | |
| *** jcooley_ has quit IRC | 09:34 | |
| *** lynxman has joined #tripleo | 09:43 | |
| matel | Hi TripleO, what is the best starting point to understand how bare metal deployment happens? (AFAIK it's exporting the disk as iSCSI) Is there any documentation around that? | 09:47 |
| *** markmc has joined #tripleo | 09:48 | |
| *** pblaho has joined #tripleo | 09:48 | |
| SpamapS | lifeless: you working on cd-undercloud at all? | 09:49 |
| *** kushi_ has joined #tripleo | 09:50 | |
| *** matsuhashi has quit IRC | 09:51 | |
| *** martyntaylor1 has joined #tripleo | 09:52 | |
| *** kui has quit IRC | 09:52 | |
| *** martyntaylor has quit IRC | 09:53 | |
| openstackgerrit | Jun Jie Nan proposed a change to openstack/os-apply-config: Converted README to .rst format https://review.openstack.org/59320 | 09:54 |
| *** matsuhashi has joined #tripleo | 09:54 | |
| *** kushi_ has quit IRC | 09:55 | |
| SpamapS | lifeless: anyway, I think dnsmasq was segfaulting becuase there were no nameserver's in the host resolv.conf | 09:58 |
| SpamapS | nope.. still segfaulting | 10:02 |
| SpamapS | wtf | 10:02 |
| lifeless | SpamapS: I am not | 10:02 |
| SpamapS | lifeless: I'm a bit out of ideas for the segfaulting dnsmasq. Thoughts before I build one with debugging symbols and attach gdb? | 10:04 |
| *** jcooley_ has joined #tripleo | 10:05 | |
| lifeless | SpamapS: retracer? | 10:11 |
| lifeless | SpamapS: enable core files and retrace with the detached symbols Ubuntu builds for every package | 10:12 |
| lifeless | SpamapS: you can do that offline | 10:12 |
| lifeless | SpamapS: or do an apport run - I bet there are crashdumps just pending | 10:12 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Modify /etc/selinux/config if it exists https://review.openstack.org/59642 | 10:12 |
| *** jcooley_ has quit IRC | 10:12 | |
| *** boris-42 has quit IRC | 10:13 | |
| SpamapS | lifeless: ubuntu does not actually build those for all packages. They try. | 10:13 |
| SpamapS | lifeless: I do have a core file already yes | 10:13 |
| SpamapS | lifeless: anyway, dnsmasq doesn't have a dbgsym package | 10:13 |
| *** marun has joined #tripleo | 10:20 | |
| *** pblaho has quit IRC | 10:21 | |
| *** pblaho has joined #tripleo | 10:22 | |
| * SpamapS runs debsums just in case it is as simple as corrupted binaries | 10:24 | |
| lifeless | SpamapS: uhm, you know about the dedicated archive with the symbols in it right | 10:27 |
| lifeless | ? | 10:27 |
| SpamapS | lifeless: ddebs.ubuntu.com yes | 10:27 |
| *** rongze has joined #tripleo | 10:28 | |
| lifeless | SpamapS: ack, ok | 10:28 |
| SpamapS | lifeless: no dnsmasq unfortunately. | 10:28 |
| SpamapS | lifeless: probably means it is stripped by its own build process | 10:28 |
| lifeless | nuts | 10:29 |
| lifeless | bug filin' time | 10:29 |
| lifeless | also what are you doing up at 4am? | 10:29 |
| SpamapS | lifeless: 2:31 | 10:31 |
| lifeless | so much better | 10:31 |
| SpamapS | lifeless: drank a diet coke at 20:00 .. it is starting to wear off | 10:31 |
| lifeless | ah pity the fool | 10:32 |
| lifeless | night | 10:32 |
| SpamapS | this is just eating away at my brain :-P | 10:32 |
| *** akrivoka1 has joined #tripleo | 10:32 | |
| *** jcoufal has quit IRC | 10:34 | |
| *** akrivoka has quit IRC | 10:35 | |
| *** rongze has quit IRC | 10:46 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add element to install a testenv worker https://review.openstack.org/58305 | 10:48 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add element to install a testenv client https://review.openstack.org/58306 | 10:48 |
| *** rdopieralski has quit IRC | 10:50 | |
| SpamapS | Dec 3 10:52:49 undercloud-notcompute-jws3awlsb2kh dnsmasq[7636]: bad IP address at line 8 of /var/run/neutron/dhcp/e85b44c7-1136-4217-954e-cdf0acdddfe1/opts | 10:53 |
| SpamapS | _hm_ | 10:53 |
| Ng | that sounds unfun | 10:54 |
| SpamapS | Ng: fyi current problem is dnsmasq segfaults after DHCPOFFER | 10:54 |
| SpamapS | I have a backtrace, and I'm now trying trusty's dnsmasq to see if it fixes the problem | 10:55 |
| Ng | *wat* | 10:56 |
| Ng | that's pretty crazy :) | 10:56 |
| SpamapS | yeah | 10:57 |
| SpamapS | will wait through one more DHCP cycle then sleeeep | 10:57 |
| SpamapS | ah, fixed | 10:59 |
| SpamapS | dnsmasq (2.67-1~ubuntu12.10.1) quantal-backports; urgency=low | 11:00 |
| SpamapS | * No-change backport to quantal | 11:00 |
| SpamapS | -- Clint "Entropy" Byrum <clint@undercloud-notcompute-jws3awlsb2kh.novalocal> Tue, 03 Dec 2013 10:51:17 +0000 | 11:00 |
| *** rongze has joined #tripleo | 11:03 | |
| *** jcooley_ has joined #tripleo | 11:05 | |
| *** jcooley_ has quit IRC | 11:10 | |
| *** kui has joined #tripleo | 11:17 | |
| vkozhukalov | Hey all, can anybody know if tripleo-ci still alive? We want to create functional (aka integration) testing scheme for ironic based on emulated bare metal. Is there any activities connected to such a task? | 11:21 |
| *** boris-42 has joined #tripleo | 11:27 | |
| vkozhukalov | And another question: is there any documentation or something about emulated baremetal? | 11:28 |
| *** sheep_ has joined #tripleo | 11:28 | |
| *** sheep_ is now known as rdopieralski | 11:29 | |
| rpodolyaka | vkozhukalov: hey, yep, tripleo-ci is alive | 11:30 |
| rpodolyaka | vkozhukalov: but I would suggest to start from tripleo devtest story | 11:30 |
| rpodolyaka | vkozhukalov: which is run on emulated baremetal | 11:30 |
| rpodolyaka | vkozhukalov: though, you can tweak it to run on real baremetal | 11:31 |
| *** jcoufal has joined #tripleo | 11:31 | |
| vkozhukalov | rpodolyaka, thanks. | 11:32 |
| vkozhukalov | rpodolyaka, will have a look | 11:32 |
| rpodolyaka | vkozhukalov: currently, tripleo-ci is duplicating devtest a lot, but I believe, the idea is to reuse devtest scripts in tripleo-ci | 11:32 |
| rpodolyaka | vkozhukalov: http://docs.openstack.org/developer/tripleo-incubator/devtest.html | 11:32 |
| rpodolyaka | vkozhukalov: https://github.com/openstack/tripleo-incubator (the doc is generated from devtest*.sh scripts) | 11:34 |
| vkozhukalov | rpodolyaka, do you know why we need shell scripts to make VMs instead of using openstack itself (compute)? | 11:36 |
| vkozhukalov | rpodolyaka, at the moment compute can not boot VMs via PXE. I think this is the only problem. | 11:37 |
| rpodolyaka | vkozhukalov: sort of. we are using SSH power driver of nova baremetal in order to do provisioning of nodes | 11:38 |
| *** rongze has quit IRC | 11:39 | |
| *** jtomasek has quit IRC | 11:39 | |
| *** jtomasek has joined #tripleo | 11:39 | |
| vkozhukalov | rpodolyaka, ok. FYI there is the blueprint https://blueprints.launchpad.net/nova/+spec/libvirt-empty-vm-boot-pxe connected with this task | 11:40 |
| rpodolyaka | vkozhukalov: cool! I'm looking forward for this to be approved and implemented then ;) | 11:41 |
| *** akrivoka1 has quit IRC | 11:43 | |
| *** arata has left #tripleo | 11:44 | |
| *** akrivoka has joined #tripleo | 11:44 | |
| *** agordeev has joined #tripleo | 11:45 | |
| *** jcooley_ has joined #tripleo | 11:51 | |
| GheRivero | vkozhukalov: any reason to implement that blueprint in nova and not ironic? nova-bm is going to be deprecated | 11:51 |
| GheRivero | vkozhukalov: got it | 11:52 |
| *** rongze_ has joined #tripleo | 11:56 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Add element to install a testenv client https://review.openstack.org/58306 | 11:58 |
| vkozhukalov | GheRivero, just to make it more clear. We already have advanced functionality in nova to make VMs. So instead of using shell scripts (devtest or tripleo-ci) and SshPower driver to deal with emulated baremetal VMs we could use nova for that. And the only obstacle here is that nova can not boot VMs via PXE. | 12:03 |
| *** cd-undercloud has joined #tripleo | 12:04 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 12:04 |
| *** cd-undercloud has quit IRC | 12:04 | |
| *** boris-42_ has joined #tripleo | 12:05 | |
| *** max_lobur has quit IRC | 12:05 | |
| *** rpodolyaka has quit IRC | 12:05 | |
| *** max_lobur has joined #tripleo | 12:05 | |
| *** rpodolyaka has joined #tripleo | 12:06 | |
| *** boris-42 has quit IRC | 12:08 | |
| *** CaptTofu has quit IRC | 12:08 | |
| *** CaptTofu has joined #tripleo | 12:08 | |
| openstackgerrit | A change was merged to openstack/os-apply-config: Converted README to .rst format https://review.openstack.org/59320 | 12:09 |
| *** jcooley_ has quit IRC | 12:25 | |
| *** boris-42_ has quit IRC | 12:27 | |
| *** lsmola has quit IRC | 12:30 | |
| *** morazi has joined #tripleo | 12:32 | |
| *** boris-42 has joined #tripleo | 12:33 | |
| *** akuznetsov has joined #tripleo | 12:38 | |
| *** lucasagomes is now known as lucas-hungry | 12:44 | |
| *** lsmola has joined #tripleo | 12:47 | |
| *** rbrady has joined #tripleo | 12:54 | |
| *** SpamapS has quit IRC | 12:59 | |
| *** noslzzp has joined #tripleo | 13:01 | |
| *** cd-undercloud has joined #tripleo | 13:06 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 13:06 |
| *** cd-undercloud has quit IRC | 13:06 | |
| *** jcooley_ has joined #tripleo | 13:08 | |
| *** jergerber has joined #tripleo | 13:11 | |
| *** jcooley_ has quit IRC | 13:13 | |
| *** CaptTofu has quit IRC | 13:15 | |
| *** CaptTofu has joined #tripleo | 13:16 | |
| *** jayg|g0n3 is now known as jayg | 13:19 | |
| *** jdob has joined #tripleo | 13:23 | |
| *** jcooley_ has joined #tripleo | 13:23 | |
| *** bcrochet has quit IRC | 13:25 | |
| *** bcrochet has joined #tripleo | 13:28 | |
| *** jcooley_ has quit IRC | 13:28 | |
| *** jdob has quit IRC | 13:35 | |
| *** jdob has joined #tripleo | 13:37 | |
| *** kui has quit IRC | 13:37 | |
| *** matsuhashi has quit IRC | 13:40 | |
| *** rushiagr has quit IRC | 13:42 | |
| openstackgerrit | Marios Andreou proposed a change to openstack/tripleo-heat-templates: WIP: make merge.py installable as a package using pbr https://review.openstack.org/58823 | 13:46 |
| *** jdob has quit IRC | 13:51 | |
| *** jtomasek has quit IRC | 13:52 | |
| *** jdob has joined #tripleo | 13:52 | |
| *** jtomasek has joined #tripleo | 13:56 | |
| *** dprince has joined #tripleo | 13:59 | |
| *** pblaho has quit IRC | 14:01 | |
| *** julim has joined #tripleo | 14:01 | |
| *** tzumainn has joined #tripleo | 14:02 | |
| *** lucas-hungry is now known as lucasagomes | 14:04 | |
| openstackgerrit | Ralf Haferkamp proposed a change to openstack/diskimage-builder: Add support for building openSUSE images https://review.openstack.org/56157 | 14:05 |
| *** cd-undercloud has joined #tripleo | 14:08 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:08 |
| *** cd-undercloud has quit IRC | 14:08 | |
| *** hewbrocca has joined #tripleo | 14:09 | |
| *** john-n-seattle1 has joined #tripleo | 14:14 | |
| *** jcooley_ has joined #tripleo | 14:17 | |
| *** nosnos has quit IRC | 14:20 | |
| *** hewbrocca has quit IRC | 14:20 | |
| *** matty_dubs|gone is now known as matty_dubs | 14:21 | |
| *** hewbrocca has joined #tripleo | 14:21 | |
| *** jcooley_ has quit IRC | 14:21 | |
| *** rongze_ has quit IRC | 14:27 | |
| *** CaptTofu has quit IRC | 14:31 | |
| *** CaptTofu has joined #tripleo | 14:31 | |
| openstackgerrit | A change was merged to openstack/os-refresh-config: `os-refresh-config` README enhancement https://review.openstack.org/59318 | 14:31 |
| *** rongze has joined #tripleo | 14:38 | |
| *** akuznetsov has quit IRC | 14:44 | |
| *** hewbrocca has quit IRC | 14:52 | |
| *** cd-undercloud has joined #tripleo | 15:10 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 15:10 |
| *** cd-undercloud has quit IRC | 15:10 | |
| *** agordeev has quit IRC | 15:16 | |
| *** jcooley_ has joined #tripleo | 15:17 | |
| *** jcoufal has quit IRC | 15:17 | |
| *** jcooley_ has quit IRC | 15:21 | |
| NobodyCam | Good Morning TripleO | 15:24 |
| jdob | morning :) | 15:28 |
| NobodyCam | good morning jdob :) | 15:29 |
| *** jprovazn has quit IRC | 15:33 | |
| *** akuznetsov has joined #tripleo | 15:33 | |
| *** vkozhukalov has quit IRC | 15:45 | |
| *** SpamapS has joined #tripleo | 15:51 | |
| openstackgerrit | Marios Andreou proposed a change to openstack/tripleo-heat-templates: WIP: make merge.py installable as a package using pbr https://review.openstack.org/58823 | 15:55 |
| openstackgerrit | Marios Andreou proposed a change to openstack/tripleo-heat-templates: Make merge.py installable as a package using pbr https://review.openstack.org/58823 | 16:03 |
| *** jcooley_ has joined #tripleo | 16:04 | |
| *** sdake_ has joined #tripleo | 16:06 | |
| *** sdake_ has quit IRC | 16:07 | |
| *** sdake_ has joined #tripleo | 16:07 | |
| *** sdake_ has quit IRC | 16:07 | |
| *** sdake_ has joined #tripleo | 16:07 | |
| *** cd-undercloud has joined #tripleo | 16:12 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 16:12 |
| *** cd-undercloud has quit IRC | 16:12 | |
| SpamapS | hm nope, still segfaulting dnsmasq | 16:13 |
| *** john-n-seattle1 has quit IRC | 16:16 | |
| openstackgerrit | Yuriy Zveryanskyy proposed a change to openstack/diskimage-builder: Add deploy ramdisk element for Ironic https://review.openstack.org/59770 | 16:17 |
| *** jcooley_ has quit IRC | 16:19 | |
| *** jcooley_ has joined #tripleo | 16:19 | |
| *** jcooley_ has quit IRC | 16:20 | |
| *** john-n-seattle1 has joined #tripleo | 16:20 | |
| *** john-n-seattle1 has quit IRC | 16:25 | |
| *** dprince has quit IRC | 16:25 | |
| *** jcooley_ has joined #tripleo | 16:35 | |
| *** marun has quit IRC | 16:37 | |
| *** john-n-seattle1 has joined #tripleo | 16:39 | |
| *** jcooley_ has quit IRC | 16:47 | |
| *** rdopieralski has quit IRC | 16:49 | |
| *** mestery has quit IRC | 16:49 | |
| *** mestery has joined #tripleo | 16:50 | |
| *** SpamapS has quit IRC | 16:52 | |
| *** jtomasek has quit IRC | 16:55 | |
| *** marun has joined #tripleo | 16:57 | |
| *** jtomasek has joined #tripleo | 16:59 | |
| *** cwolferh has joined #tripleo | 17:05 | |
| *** SpamapS has joined #tripleo | 17:05 | |
| *** SpamapS has quit IRC | 17:05 | |
| *** SpamapS has joined #tripleo | 17:05 | |
| *** jistr has quit IRC | 17:10 | |
| *** jcooley_ has joined #tripleo | 17:12 | |
| *** UtahDave has joined #tripleo | 17:13 | |
| *** marun has quit IRC | 17:13 | |
| *** jcooley_ has quit IRC | 17:15 | |
| *** jcooley_ has joined #tripleo | 17:16 | |
| *** jcooley_ has quit IRC | 17:20 | |
| *** jcooley_ has joined #tripleo | 17:23 | |
| *** markmc has quit IRC | 17:24 | |
| *** bauzas has quit IRC | 17:31 | |
| *** matty_dubs is now known as matty_dubs|lunch | 17:35 | |
| *** max_lobur is now known as max_lobur_afk | 17:37 | |
| *** sharp_ has joined #tripleo | 17:37 | |
| *** boris-42 has quit IRC | 17:38 | |
| *** sharp_ is now known as kmsWork | 17:41 | |
| *** derekh has quit IRC | 17:41 | |
| *** martyntaylor1 has quit IRC | 17:52 | |
| *** vkozhukalov has joined #tripleo | 17:55 | |
| *** bauzas has joined #tripleo | 17:55 | |
| *** rongze has quit IRC | 17:58 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic to setup-undercloud-passwords https://review.openstack.org/59800 | 18:06 |
| *** UtahDave has quit IRC | 18:08 | |
| * Ng dinners | 18:10 | |
| *** athomas has quit IRC | 18:15 | |
| *** vipul is now known as vipul-away | 18:15 | |
| *** sdake has quit IRC | 18:19 | |
| *** sdake has joined #tripleo | 18:19 | |
| *** sdake has quit IRC | 18:19 | |
| *** sdake has joined #tripleo | 18:19 | |
| *** sdake has quit IRC | 18:20 | |
| *** sdake has joined #tripleo | 18:20 | |
| *** sdake has quit IRC | 18:20 | |
| *** sdake has joined #tripleo | 18:20 | |
| *** SpamapS changes topic to "FIREDRILL: dnsmasq segfaults causing undercloud to fail | Using OpenStack to deploy OpenStack; meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 18:22 | |
| *** matty_dubs|lunch is now known as matty_dubs | 18:30 | |
| *** krotscheck has joined #tripleo | 18:31 | |
| *** vipul-away is now known as vipul | 18:32 | |
| *** openstackgerrit has quit IRC | 18:33 | |
| *** openstackgerrit has joined #tripleo | 18:33 | |
| *** dprince has joined #tripleo | 18:39 | |
| *** A-G has quit IRC | 18:39 | |
| SpamapS | ggaaaahhh | 18:42 |
| SpamapS | why must servers take so dang long to boot | 18:42 |
| SpamapS | and now tftp is failing for some weird reason | 18:45 |
| lifeless | morning | 18:47 |
| *** rongze has joined #tripleo | 18:49 | |
| SpamapS | lifeless: hello there | 18:50 |
| lifeless | hi | 18:52 |
| *** rongze has quit IRC | 18:54 | |
| dprince | SpamapS: those extra boot time memory checks are quite painful | 18:55 |
| SpamapS | what sucks is.. | 18:55 |
| SpamapS | I wait about 7 minutes for all the crap.. | 18:56 |
| dprince | SpamapS: if only you could have IPMI and turn off all those checks | 18:56 |
| SpamapS | then it displays the error for about 0.75s | 18:56 |
| *** pblaho has joined #tripleo | 18:58 | |
| SpamapS | dprince: eh? | 18:58 |
| SpamapS | dprince: is there a way to streamline POST | 18:59 |
| SpamapS | ? | 18:59 |
| *** pblaho has left #tripleo | 18:59 | |
| lifeless | meeting time | 18:59 |
| *** jistr has joined #tripleo | 19:00 | |
| dprince | SpamapS: on some machines you can tone down the wait time... but in general no it seems | 19:02 |
| dprince | SpamapS: real servers with all the extra memory checks always seem to take the longest | 19:03 |
| SpamapS | dprince: and I have 96G of memory to check.. so... :-P | 19:03 |
| lifeless | SpamapS: there is a way, beat your vendor up :P | 19:03 |
| pleia2 | dprince: apologies for not getting to your question yesterday re: ssh access, could use more details of exactly what you are asking | 19:04 |
| *** slagle_ has joined #tripleo | 19:05 | |
| dprince | pleia2: perhaps we can talk about it in our CI google hangout tomorrow too, but the issue is I'm getting pushback about obtaining public IPs for our CI/CD rack. I wanted to no exactly how many you had, and how many we might actually need. | 19:05 |
| *** slagle has quit IRC | 19:05 | |
| *** slagle_ is now known as slagle | 19:06 | |
| dprince | pleia2: which reminds me, I'll send out an invite for the meeting tomorrow. | 19:06 |
| *** lucasagomes is now known as lucas-dinner | 19:07 | |
| pleia2 | dprince: ok, tomorrow would be good, I'm not sure | 19:08 |
| SpamapS | hmmm | 19:10 |
| SpamapS | we may have a rogue DHCP server | 19:10 |
| SpamapS | ok no not rogue | 19:18 |
| * SpamapS foiled by netns again | 19:18 | |
| lifeless | dprince: can you get IPv6 for the test rack more easily? | 19:19 |
| lifeless | dprince: we can do IPv6 only | 19:19 |
| lifeless | dprince: I know IPv4 is super scarce | 19:19 |
| *** shardy is now known as shardy_afk | 19:20 | |
| dprince | lifeless: not sure about IPv6, I can push for IPv4 too but it is up in the air | 19:22 |
| pleia2 | dprince: any CI updates for #openstack-meeting-alt meeting? | 19:22 |
| lifeless | dprince: so we need a fair naumber - one per jenkins slave + one for the undercloud control plane/endpoint + one for the overcloud control plane/endpoint - at minimum | 19:22 |
| dprince | lifeless: yeah, I figured :( | 19:23 |
| lifeless | dprince: 15 servers, say 8 end up being test environments and 2 end up being hypervisors, that would be 30+IPs | 19:23 |
| dprince | lifeless: My initial estimate was 20, I'll bump it a bit | 19:23 |
| *** sdake has quit IRC | 19:24 | |
| *** sdake has joined #tripleo | 19:24 | |
| lifeless | dprince: I really suggest going the IPv6 route. | 19:24 |
| dprince | pleia2: last week was mostly a holiday, so not much CI stuff yet, Derek has made progress on the gearman eleements though. Soon I plan on tackling making TOCI use the devtest scripts... | 19:24 |
| SpamapS | ok so for some reason the freshly PXE dhcp'd machine doesn't see tftp on 10.10.16.172 | 19:25 |
| pleia2 | dprince: ok same here | 19:25 |
| dprince | lifeless: noted, will see what we can do | 19:25 |
| lifeless | dprince: because I'd rather not have to stress about hitting limits :) | 19:25 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-incubator: Test MAC address should be 6 octets. https://review.openstack.org/59812 | 19:27 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-incubator: Test MAC address should be 6 octets. https://review.openstack.org/59812 | 19:30 |
| *** vipul is now known as vipul-away | 19:33 | |
| *** vipul-away is now known as vipul | 19:33 | |
| *** akrivoka has quit IRC | 19:34 | |
| lifeless | sdake: got a pointer to the dib template for slagle | 19:34 |
| *** jcooley_ has quit IRC | 19:37 | |
| *** jcooley_ has joined #tripleo | 19:37 | |
| *** jcooley_ has quit IRC | 19:41 | |
| *** jcooley_ has joined #tripleo | 19:42 | |
| SpamapS | dkehn: ok, you have the baton | 19:44 |
| dkehn | SpamapS: thx | 19:44 |
| SpamapS | I need to do some reviews anyway | 19:44 |
| lifeless | dkehn: hold off fixing the live system, or filing the bug ? | 19:44 |
| dkehn | SpamapS: shouldn't take long | 19:44 |
| lifeless | SpamapS: 78:e7:d1:24:31:5d - I think thats one of ours, no? | 19:45 |
| lifeless | dprince: hah, I'm going to be difficult and suggest we hangout now; I have a dr appt tomorrow | 19:46 |
| *** jcooley_ has quit IRC | 19:47 | |
| *** jcooley_ has joined #tripleo | 19:47 | |
| dprince | lifeless: I can talk now (I think), But I'm not sure derekh can make it today | 19:47 |
| lifeless | ah | 19:47 |
| lifeless | perhaps you me lyz today, and you derekh lyz tomorrow? | 19:48 |
| pleia2 | I don't really have anything useful to say today | 19:48 |
| SpamapS | lifeless: yes that is bm-aw1az2-freecloud0010 | 19:48 |
| dprince | lifeless/pleia2: also, I'm working from home today so if we chat now I'll have my flaky headset only | 19:48 |
| pleia2 | I think just talking with dprince and derekh tomorrow would be valuable | 19:49 |
| lifeless | SpamapS: which is just a regular trunk POC node, so we can fix it by giving it the same settings for those opts the other deployed nodes have | 19:49 |
| lifeless | dprince: pleia2: your call | 19:49 |
| *** rongze has joined #tripleo | 19:51 | |
| *** jcooley_ has quit IRC | 19:51 | |
| *** jcooley_ has joined #tripleo | 19:52 | |
| sdake | lifeless reading backscroll sec | 19:52 |
| *** vipul is now known as vipul-away | 19:53 | |
| lifeless | sdake: oh if you want context - https://review.openstack.org/#/c/59361/ | 19:53 |
| lifeless | dkehn: did you see my suggestion about reviews - would have been your 1am or something | 19:54 |
| sdake | lifeless put in review | 19:55 |
| dkehn | lifeless: no I didn't but I saw that the terminal caught it, but didn;t seee where, will look at it | 19:55 |
| *** rongze has quit IRC | 19:56 | |
| dkehn | lifeless: lol, yepper trying to do 3 a day | 19:57 |
| dkehn | lifeless: reviews | 19:57 |
| dkehn | lifeless: unless my stuff gets in the way | 19:58 |
| *** lynxman has quit IRC | 19:58 | |
| lifeless | dkehn: ok, cool - is that new? - http://russellbryant.net/openstack-stats/neutron-reviewers-90.txt shows 10 reviews from you | 19:59 |
| *** lynxman has joined #tripleo | 19:59 | |
| dkehn | that is new | 19:59 |
| dkehn | 12 | 19:59 |
| dprince | lifeless/pleia2: lets chat now (if I can make my flaky headset work) | 19:59 |
| pleia2 | ok | 20:00 |
| lifeless | dkehn: Let me know if you need any input ;) | 20:00 |
| dkehn | lifeless: thx | 20:00 |
| lifeless | dkehn: sorry 12, yes :) | 20:00 |
| dkehn | np | 20:00 |
| *** epim has joined #tripleo | 20:03 | |
| *** vipul-away is now known as vipul | 20:04 | |
| *** jcooley_ has quit IRC | 20:06 | |
| lifeless | dprince: https://wiki.openstack.org/wiki/TripleO/TripleOCloud/Regions#Contributing_a_region_to_the_TripleOCloud | 20:08 |
| *** vipul is now known as vipul-away | 20:11 | |
| *** vipul-away is now known as vipul | 20:12 | |
| dprince | http://upstart.ubuntu.com/cookbook/#disabling-a-job-from-automatically-starting | 20:19 |
| *** vkozhukalov has quit IRC | 20:19 | |
| dkehn | SpamapS: write the bug, if you pass a '' it doesn't get loaded, but a public --fixed-ip subnet_id=dbf0a864-f836-4d39-ba81-787e8b6e9454,ip_address=172.24.4.230 --extra-dhcp-opt opt_name=bootfile-name,opt_value=0 --extra-dhcp-opt opt_name=tftp-server,opt_value=123.123.123.123 --extra-dhcp-opt opt_name=server-ip-address,opt_value=' ' gets it | 20:26 |
| dkehn | SpamapS: let me know the bug number and I'll put a patch on it | 20:27 |
| *** vipul is now known as vipul-away | 20:27 | |
| SpamapS | dkehn: seems like you have a grip, want to just write the bug yourself? | 20:28 |
| SpamapS | dkehn: I'm just aware of the symptoms.. :-P | 20:28 |
| dkehn | SpamapS: sure, will do | 20:28 |
| dkehn | SpamapS: thx for the catch | 20:28 |
| SpamapS | dkehn: thanks for looking into it. Thought I was going crazy. ;) | 20:32 |
| dkehn | SpamapS: hmm, can't be sure of the crazy part | 20:32 |
| SpamapS | my parents said the same thing.. ;) | 20:33 |
| SpamapS | hm | 20:36 |
| SpamapS | so tripleo-cd is now just stopping altogether if it hits the throttle | 20:36 |
| SpamapS | is that what we wanted? | 20:36 |
| lifeless | for 10m | 20:38 |
| lifeless | it was | 20:38 |
| lifeless | if you stop and start it it will try again | 20:38 |
| SpamapS | it stops on its own | 20:38 |
| SpamapS | forever | 20:39 |
| *** kui has joined #tripleo | 20:39 | |
| SpamapS | flock -x $THROTTLELOCK echo | 20:39 |
| SpamapS | if [ "0" != "$RESULT" ]; then | 20:39 |
| SpamapS | rm -f $THROTTLELOCK | 20:39 |
| SpamapS | exit $RESULT | 20:39 |
| slagle | the exit was there before wasn't it? | 20:39 |
| SpamapS | actually it just stops, period, if RESULT != 0 | 20:40 |
| slagle | it should get restarted by the service though | 20:40 |
| slagle | once it exits | 20:40 |
| SpamapS | in theory yes | 20:40 |
| SpamapS | but then the throttle is hit | 20:40 |
| SpamapS | and upstart says "you're restarting too fast" | 20:40 |
| SpamapS | I think | 20:40 |
| SpamapS | I actually don't understand the flock/throttle thing at all. Have not tried to think about it. | 20:41 |
| SpamapS | oh, block the exit if we failed in less than 10 minutes, ok. | 20:42 |
| SpamapS | now I'm actually even more confused as to why we exit and don't just let the loop repeat. :-P | 20:43 |
| lifeless | the lock file is random | 20:45 |
| *** vipul-away is now known as vipul | 20:45 | |
| lifeless | dprince: https://etherpad.openstack.org/p/tripleo-late-start-services | 20:48 |
| *** jistr has quit IRC | 20:48 | |
| *** jcooley_ has joined #tripleo | 20:51 | |
| SpamapS | ok so I'm not really sure where to "fix" that network to work right | 20:52 |
| *** jcooley_ has quit IRC | 20:53 | |
| SpamapS | dkehn: ? | 20:53 |
| *** rongze has joined #tripleo | 20:53 | |
| SpamapS | I mean, I see tftp-server and server-ip-address are blank | 20:53 |
| SpamapS | which I presume is "the problem" | 20:53 |
| dkehn | SpamapS: check the nova.conf baremetal section , I'm thinking | 20:53 |
| SpamapS | nova/lib/python2.7/site-packages/nova/tests/virt/baremetal/test_driver.py: {'opt_name': 'server-ip-address', 'opt_value': CONF.my_ip}, | 20:55 |
| SpamapS | doh thats tests | 20:55 |
| SpamapS | ah ok same diff | 20:55 |
| SpamapS | $ sudo grep my_ip /etc/nova/nova.conf | 20:56 |
| SpamapS | my_ip= | 20:56 |
| *** CaptTofu has quit IRC | 20:56 | |
| SpamapS | andmy_ip={{local-ipv4}} | 20:57 |
| SpamapS | ec2 metadata not working == the problem | 20:57 |
| *** CaptTofu has joined #tripleo | 20:57 | |
| lifeless | SpamapS: eyeball please - https://etherpad.openstack.org/p/tripleo-late-start-services | 20:58 |
| pleia2 | eek, my chrome crashed | 20:58 |
| *** rongze has quit IRC | 20:58 | |
| pleia2 | I think we were near the end anyway | 21:00 |
| *** krotscheck has quit IRC | 21:00 | |
| SpamapS | lifeless: hm. | 21:00 |
| dprince | pleia2: yep, we were done anyway. | 21:00 |
| SpamapS | lifeless: why are init files 'state' ? | 21:00 |
| *** vipul is now known as vipul-away | 21:00 | |
| dprince | pleia2: funny thing was your picture was still perfectly posed in the Google hangout. It really did look like you were still there, albiet mostly still/froze | 21:01 |
| *** krotscheck has joined #tripleo | 21:01 | |
| pleia2 | heh :) | 21:04 |
| SpamapS | lifeless: did ec2 metadata ever work on cd-undercloud? | 21:04 |
| lifeless | SpamapS: yes | 21:06 |
| lifeless | SpamapS: if the undercloud has it wrong, its a <->seed issue | 21:07 |
| *** boris-42 has joined #tripleo | 21:07 | |
| SpamapS | lifeless: Destination Gateway Genmask Flags Metric Ref Use Iface | 21:07 |
| SpamapS | 0.0.0.0 138.35.77.1 0.0.0.0 UG 0 0 0 vlan25 | 21:07 |
| SpamapS | lifeless: is the gateway right there? | 21:07 |
| *** shardy_afk is now known as shardy | 21:07 | |
| SpamapS | lifeless: seems like we'd have to route through the seed to make it work | 21:07 |
| lifeless | SpamapS: no, the seed pushes a route | 21:08 |
| SpamapS | lifeless: _ahh_ | 21:09 |
| lifeless | SpamapS: or should be; it may be too old | 21:09 |
| SpamapS | lifeless: we were still on file injection at the time | 21:09 |
| lifeless | SpamapS: yeah | 21:09 |
| lifeless | SpamapS: so I think we should add a 169.254.169.254 route to the dnsmasq config on the seed | 21:09 |
| SpamapS | lifeless: post-up ip route add 169.154.169.154/32 via 10.10.16.168 | 21:09 |
| SpamapS | its in /etc/network/interfaces | 21:09 |
| SpamapS | somehow lost though? | 21:09 |
| lifeless | SpamapS: remember we had a ip binding issue on boot | 21:10 |
| lifeless | SpamapS: bridge vs eth | 21:10 |
| lifeless | adding the route to the seed dnsmasq | 21:10 |
| SpamapS | right | 21:10 |
| lifeless | would let cloud-init get the data on boot | 21:10 |
| lifeless | which would let orc run properly | 21:10 |
| SpamapS | lifeless: ok, but we're not using the dhcp.. so? | 21:10 |
| lifeless | which would let it all come up normally I think | 21:11 |
| lifeless | SpamapS: we do, when we start the service when we reboot the undercloud node | 21:11 |
| lifeless | SpamapS: but right now, I'd be inclined to just a) file a note in trello, b) fix things right now, c) get onto automating our way down to the undercloud | 21:11 |
| SpamapS | right | 21:11 |
| lifeless | it's not even a bug | 21:12 |
| lifeless | since it's in old code | 21:12 |
| SpamapS | right | 21:12 |
| *** noslzzp has quit IRC | 21:13 | |
| SpamapS | hm, something happens where baremetal nodes get stuck as long dead instances | 21:15 |
| lifeless | SpamapS: how are init files state? They aren't *but* the assertion 'service X is properly configured' is. | 21:17 |
| lifeless | SpamapS: I'm not entirely happy with that etherpad, but I think it's strictly better than what we have (removes spurious errors in logs during deploy) | 21:18 |
| *** vipul-away is now known as vipul | 21:19 | |
| SpamapS | lifeless: so I may be wrong here, but I'm 90% sure upstart won't follow symlinks. | 21:21 |
| *** CaptTofu has quit IRC | 21:22 | |
| SpamapS | lifeless: we may have to have a set of 'start on starting' jobs that block until the right state has been achieved. | 21:24 |
| SpamapS | RESTRICTIONS The use of symbolic links in job configuration file directories is not supported since it can lead to unpredi‐ cable behaviour resulting from broken or inaccessible links (such as would be caused by a link crossing a filesystem boundary to a filesystem that has not yet been mounted). | 21:24 |
| *** jcooley_ has joined #tripleo | 21:24 | |
| SpamapS | oh and I just found a spelling mistake in the man page | 21:24 |
| * SpamapS submits patch | 21:24 | |
| lifeless | SpamapS: ok; so goal is to support two things | 21:24 |
| lifeless | A) / readonly | 21:25 |
| lifeless | B) don't start services that don't have valid config | 21:25 |
| lifeless | which is both initial, and new image done via cold deploy | 21:25 |
| lifeless | Suggestions? | 21:25 |
| SpamapS | lifeless: a simple "orc has run successfully" coarse grained state that we can record in /mnt with touch would do the trick. We can have one job which just does 'start on starting (list all ..)' which runs a script polling for that file. | 21:26 |
| SpamapS | newer upstarts can even start on inotify events | 21:28 |
| *** CaptTofu has joined #tripleo | 21:29 | |
| SpamapS | start on file FILE=/mnt/occ.done EVENT=create | 21:29 |
| lifeless | SpamapS: orc needs to start these things during post-configure though; migrations may depend on it | 21:36 |
| lifeless | so occ.done would be too coarse I think | 21:36 |
| SpamapS | ah right | 21:36 |
| SpamapS | lifeless: ok so configure.d 99-configuration-done touches /mnt/occ.configuration.done ? | 21:37 |
| SpamapS | /mnt/state rather | 21:37 |
| SpamapS | or something | 21:37 |
| *** vipul is now known as vipul-away | 21:39 | |
| *** vipul-away is now known as vipul | 21:39 | |
| *** jdob has quit IRC | 21:41 | |
| SpamapS | aaaaand now I've pissed off the ipmi | 21:41 |
| *** bauzas1 has joined #tripleo | 21:42 | |
| *** bauzas has quit IRC | 21:43 | |
| lifeless | SpamapS: or we could bind-mount (ugh) the file into /etc for upstart | 21:43 |
| lifeless | SpamapS: basically we need to unblock dprince though | 21:43 |
| SpamapS | lifeless: dirs in /etc/init will be used, so that could work. | 21:43 |
| * SpamapS tries it | 21:43 | |
| SpamapS | lifeless: so the jobs would have /'s in their names. | 21:46 |
| *** A-G has joined #tripleo | 21:50 | |
| lifeless | dirs? | 21:52 |
| lifeless | you can bind mount files... | 21:52 |
| SpamapS | lifeless: another option is to have htem 'start on openstack-configured' and emit that in configure.d/75-openstack-configured on upstart systems. | 21:52 |
| SpamapS | lifeless: yeah I dislike bind mounting all those files. :-P | 21:52 |
| *** rongze has joined #tripleo | 21:55 | |
| *** rpodolyaka1 has joined #tripleo | 21:56 | |
| *** rongze has quit IRC | 22:00 | |
| *** jcoufal has joined #tripleo | 22:02 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add diskimage-builder element. https://review.openstack.org/59361 | 22:09 |
| *** bauzas1 is now known as bauzas | 22:09 | |
| *** sdake_ has quit IRC | 22:10 | |
| *** sdake_ has joined #tripleo | 22:10 | |
| *** krotscheck has quit IRC | 22:14 | |
| *** noslzzp has joined #tripleo | 22:15 | |
| *** krotscheck has joined #tripleo | 22:17 | |
| *** dprince has quit IRC | 22:22 | |
| lifeless | SpamapS: OTP | 22:24 |
| SpamapS | \o/ .. got a box to boot finally | 22:28 |
| *** lsmola has quit IRC | 22:35 | |
| *** lsmola has joined #tripleo | 22:38 | |
| *** epim has quit IRC | 22:39 | |
| *** rpodolyaka1 has quit IRC | 22:41 | |
| lifeless | SpamapS: \o/ | 22:46 |
| dkehn | SpamapS: dekehn proposed a change to openstack/neutron: Add checks for empty strings https://review.openstack.org/59858, to deal with the issue earlier | 22:47 |
| SpamapS | dkehn: sweet thanks | 22:48 |
| dkehn | SpamapS: thanks for catching it | 22:48 |
| SpamapS | now.. this is where we tell echohead he was right and we should fail os-apply-config on missing keys. ;) | 22:48 |
| SpamapS | I think nova should also fail if my_ip is blank | 22:49 |
| SpamapS | which is where this came from | 22:49 |
| SpamapS | "fail fast" | 22:49 |
| SpamapS | lifeless: is there any chance we could get the test rack split into two physical/vlan l2 segments? | 22:50 |
| SpamapS | lifeless: it would be good to be able to have two concurrent running underclouds. | 22:51 |
| lifeless | SpamapS: We had that, I think its a bad idea | 22:51 |
| SpamapS | mmk | 22:51 |
| * SpamapS will keep trying to find things to do in between 40 minute fail cycles. :-/ | 22:51 | |
| lifeless | SpamapS: also we are already running two underclouds | 22:52 |
| lifeless | SpamapS: so its entirely not needed to do that | 22:52 |
| *** epim has joined #tripleo | 22:52 | |
| SpamapS | I suggested spinning up a new one and you said that it would cause problems. I may have misunderstood that to be a network segmentation issue. | 22:52 |
| lifeless | SpamapS: with native neutron dhcp there are no issues, as long as we don't have any machine owned by two at aonce | 22:53 |
| lifeless | SpamapS: but that sayid - why, what will we learn from a second undercloud ? | 22:53 |
| SpamapS | while the first is doing something, we can change something on the 2nd and run a second experiment. | 22:54 |
| SpamapS | basically debugging anything on the undercloud takes _days_ of waiting. | 22:55 |
| lifeless | SpamapS: mmm, so when we're doing bare metal ironic gating we get that massively and automatically | 22:55 |
| SpamapS | "when we're doing" .. if that is sometime that could be qualified as "soon" then agreed, thats the way to go. | 22:56 |
| *** michchap has quit IRC | 22:56 | |
| *** CaptTofu has quit IRC | 22:57 | |
| *** rongze has joined #tripleo | 22:57 | |
| *** michchap has joined #tripleo | 23:02 | |
| *** rongze has quit IRC | 23:02 | |
| *** jcoufal has quit IRC | 23:02 | |
| *** CaptTofu has joined #tripleo | 23:03 | |
| *** lsmola has quit IRC | 23:07 | |
| *** jtomasek has quit IRC | 23:07 | |
| *** noslzzp has quit IRC | 23:11 | |
| SpamapS | Hm ok so digging through my "bugs-to-file" notes.. neutron-dhcp-agent not noticing that its' dnsmasq has died... | 23:12 |
| SpamapS | Trying to decide what Neutron should do if dnsmasq does die. | 23:13 |
| lifeless | run around in circles | 23:18 |
| *** jcooley_ has quit IRC | 23:18 | |
| lifeless | SpamapS: it should log it logger.exception(...) | 23:18 |
| lifeless | or logger.error(...) | 23:18 |
| SpamapS | I'm starting to see the code required to do this | 23:23 |
| lifeless | SpamapS: just file the bug IMO | 23:24 |
| lifeless | SpamapS: will be eventlet fiddling to respond to the child died event | 23:24 |
| SpamapS | lifeless: well I was going to provide a patch if that is easy | 23:24 |
| SpamapS | lifeless: the eventlet fiddling is relatively straight forward.. ignore SIGCHLD, use .wait() in its own greenthread. | 23:25 |
| SpamapS | assuming wait doesn't use SIGCHLD actually | 23:25 |
| SpamapS | I used to know this stuff.. :-P | 23:25 |
| lifeless | so thats why I say just file the bug :) | 23:25 |
| lifeless | there's enough detail there to get right | 23:25 |
| *** cd-undercloud has joined #tripleo | 23:29 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 23:29 |
| *** cd-undercloud has quit IRC | 23:29 | |
| *** matty_dubs is now known as matty_dubs|gone | 23:30 | |
| SpamapS | https://bugs.launchpad.net/neutron/+bug/1257524 | 23:30 |
| lifeless | more on kexec - http://mjg59.dreamwidth.org/28746.html | 23:30 |
| uvirtbot | Launchpad bug 1257524 in neutron "If neutron spawned dnsmasq dies, neutron-dhcp-agent will be totally unaware" [Undecided,New] | 23:30 |
| lifeless | I love mjg59 | 23:30 |
| SpamapS | ok well back to having ACTIVE servers, but they've timed out | 23:32 |
| SpamapS | ec2 metadata now failed | 23:35 |
| *** bauzas has quit IRC | 23:40 | |
| *** CaptTofu has quit IRC | 23:44 | |
| *** jtomasek has joined #tripleo | 23:45 | |
| *** jayg is now known as jayg|g0n3 | 23:48 | |
| *** jtomasek has quit IRC | 23:52 | |
| *** CaptTofu has joined #tripleo | 23:55 | |
| *** openstackgerrit has quit IRC | 23:56 | |
| *** openstackgerrit has joined #tripleo | 23:56 | |
| *** kui has quit IRC | 23:56 | |
| *** rongze has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!