| *** jrist has joined #tripleo | 00:09 | |
| *** ramishra has quit IRC | 00:17 | |
| *** ramishra has joined #tripleo | 00:19 | |
| *** cd-undercloud has joined #tripleo | 00:19 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 00:19 |
|---|---|---|
| *** cd-undercloud has quit IRC | 00:19 | |
| *** epim has quit IRC | 00:26 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-heat-templates: Add username parameter for rabbit https://review.openstack.org/71430 | 00:28 |
| *** ramishra has quit IRC | 00:29 | |
| *** cadenzajon has quit IRC | 00:36 | |
| *** CaptTofu has joined #tripleo | 00:39 | |
| *** olaph has quit IRC | 00:49 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Remove rabbitmq-server from boot-stack's element-deps https://review.openstack.org/72516 | 00:56 |
| lifeless | SpamapS: ok so | 01:06 |
| *** ramishra has joined #tripleo | 01:08 | |
| *** epim has joined #tripleo | 01:09 | |
| openstackgerrit | lifeless proposed a change to openstack-infra/tripleo-ci: Allow virsh commands from the VirtualPowerDriver https://review.openstack.org/72415 | 01:11 |
| lifeless | SpamapS: slagle: ^ please. | 01:12 |
| lifeless | needed to unbreak tripleo-ci on undercloud | 01:12 |
| lifeless | we'll need to redeploy the TE after this too, I suspect. | 01:12 |
| *** ramishra has quit IRC | 01:13 | |
| *** e0ne has quit IRC | 01:16 | |
| *** CaptTofu has quit IRC | 01:16 | |
| *** CaptTofu has joined #tripleo | 01:19 | |
| *** cd-undercloud has joined #tripleo | 01:41 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 01:41 |
| *** cd-undercloud has quit IRC | 01:41 | |
| *** xuhanp has joined #tripleo | 01:41 | |
| lifeless | man, I had expenses. | 01:41 |
| lifeless | soul destroying | 01:41 |
| *** weshay has quit IRC | 01:51 | |
| *** epim has quit IRC | 01:52 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-incubator: Update .gitignore w/ overcloud/undercloud passwds https://review.openstack.org/72520 | 01:53 |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-incubator: Messaging backend selection https://review.openstack.org/72524 | 02:06 |
| *** ramishra has joined #tripleo | 02:09 | |
| *** ramishra has quit IRC | 02:14 | |
| *** cwolferh has quit IRC | 02:14 | |
| StevenK | lifeless: I did that yesterday. | 02:16 |
| *** e0ne has joined #tripleo | 02:27 | |
| mordred | lifeless: I'm coming to believe that other people shoudl do those for me | 02:29 |
| *** spzala has quit IRC | 02:30 | |
| *** e0ne has quit IRC | 02:31 | |
| lifeless | mordred: I would love that | 02:35 |
| lifeless | mordred: because, well - I'm expensive. | 02:35 |
| lifeless | mordred: and so are you | 02:35 |
| lifeless | this may be a bit egocentric of me | 02:36 |
| *** wusWWW has quit IRC | 02:36 | |
| lifeless | and ugh | 02:39 |
| lifeless | lost a 5 way split | 02:39 |
| lifeless | because FAIL | 02:39 |
| *** ramishra has joined #tripleo | 02:45 | |
| *** ramishra has quit IRC | 02:46 | |
| StevenK | mordred: I sat through the two hours of EEM training last night, you can delegate ... | 02:46 |
| StevenK | mordred: Not that I'm in any way, shape or form offering to do so :-) | 02:46 |
| mordred | :) | 02:47 |
| * StevenK stabs mox | 02:49 | |
| StevenK | And its docs for being useless | 02:49 |
| openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: Design fiction porting undercloud-bm to software-config https://review.openstack.org/72533 | 02:49 |
| stevebaker | lifeless, SpamapS, let me know if you'd prefer to discuss ^ here, in the review, or on the mailing list | 02:50 |
| mordred | StevenK: aren't we moving to mock? | 02:55 |
| StevenK | mordred: The nova test class I'm currently bashing my face against is using mox. If we are moving to mock, I can shave that yak first | 02:59 |
| *** AaronGr has quit IRC | 03:00 | |
| lifeless | StevenK: IIRC I bought you lunch at the stirling hotel on monday, right? | 03:02 |
| *** mikal has quit IRC | 03:02 | |
| lifeless | StevenK: I try to keep such yaks low | 03:02 |
| lifeless | StevenK: nova have explicitly said they don't want huge churn on that | 03:02 |
| *** cd-undercloud has joined #tripleo | 03:03 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 03:03 |
| *** cd-undercloud has quit IRC | 03:03 | |
| lifeless | StevenK: so I'd wait for a (core) reviewer to request it. | 03:03 |
| *** mikal has joined #tripleo | 03:03 | |
| StevenK | lifeless: With mjg, devananda? I thought that was Tuesday, but ICBW | 03:03 |
| devananda | hrm? | 03:03 |
| lifeless | StevenK: no, the stirling hotel | 03:03 |
| lifeless | it was a short drive | 03:03 |
| *** greghaynes has quit IRC | 03:04 | |
| * devananda realizes context, returns to being afk | 03:04 | |
| lifeless | it was someone :P | 03:04 |
| *** AaronGr has joined #tripleo | 03:05 | |
| StevenK | lifeless: I don't think it was me, but the lunches from LCA are pretty dim | 03:05 |
| lifeless | no worries | 03:05 |
| lifeless | it was 30$ - I can eat that ona hungry day | 03:05 |
| * lifeless claims it as one person | 03:05 | |
| *** CaptTofu has quit IRC | 03:06 | |
| lifeless | wow, I think EEM got restarted under me that time | 03:06 |
| *** greghaynes has joined #tripleo | 03:07 | |
| *** ramishra has joined #tripleo | 03:07 | |
| StevenK | lifeless: It's better than CA. Which isn't saying much. | 03:07 |
| StevenK | lifeless: Fairy nuff, I'll leave that yak alone. But I'm still trying to work out how to use mox for my purposes, and it's fighting me. | 03:08 |
| StevenK | lifeless: Hm, aren't you going to get yelled at by EEM for >30 days | 03:10 |
| StevenK | Since I did | 03:10 |
| lifeless | StevenK: certainly | 03:10 |
| lifeless | StevenK: and since this is a 5K expense... lots | 03:10 |
| StevenK | Holy crap | 03:10 |
| lifeless | nah, holy crap was when monty made me put a 5K dinner on the card in one hit | 03:11 |
| StevenK | Mine was a few hundred, which is roughly what I spent in cash, but you did buy lots of stuff | 03:11 |
| StevenK | I wait to see if mordred rejects my report with "Try harder" | 03:12 |
| StevenK | :-P | 03:12 |
| *** AaronGr_Zzz has joined #tripleo | 03:13 | |
| lifeless | now,I'm having mental blanks on some of the names here | 03:13 |
| lifeless | mikals minions mainly | 03:13 |
| StevenK | Hahaha | 03:13 |
| StevenK | I even questioned you at the time! | 03:14 |
| *** taps has quit IRC | 03:14 | |
| lifeless | I have photos | 03:14 |
| *** ramishra_ has joined #tripleo | 03:14 | |
| StevenK | Sure, but it doesn't help if you can't put the face to a name | 03:15 |
| StevenK | lifeless: So I'm mocking out the call to destroy_images so I can verify it's called. Except that I don't have one of the arguments when I call it after mocking but before calling spawn(). | 03:17 |
| StevenK | Sigh. That last sentence is probably unparseable | 03:17 |
| *** ramishra has quit IRC | 03:18 | |
| *** AaronGr_Zzz has quit IRC | 03:19 | |
| *** AaronGr_Zzz has joined #tripleo | 03:20 | |
| lifeless | asdst | 03:21 |
| lifeless | ba | 03:22 |
| lifeless | tcp I hate thee | 03:22 |
| lifeless | so there are three names at issue | 03:22 |
| lifeless | the russian due | 03:22 |
| lifeless | dude | 03:22 |
| lifeless | the guy across from StevenK beside mrda | 03:22 |
| lifeless | and the catalyst guys name escapes me for some reason | 03:24 |
| lifeless | though I know him! | 03:24 |
| lifeless | andrew | 03:24 |
| stevebaker | ruthven | 03:25 |
| *** AaronGr_Zzz has quit IRC | 03:25 | |
| lifeless | right | 03:25 |
| lifeless | so two names to fill | 03:26 |
| *** AaronGr_Zzz has joined #tripleo | 03:26 | |
| *** AaronGr_Zzz has quit IRC | 03:30 | |
| *** AaronGr_Zzz has joined #tripleo | 03:30 | |
| greghaynes | So ive got a hacked up heat template to spawn two notcompute overcloud instances, and stack-create fails with Resource create failed: WaitConditionTimeout: 0 of 1. I also notice that os-c-c fails on one due to not having the /m/s/v/l/b-s/init-openstack.ok written... does that seem like the correct path to be hunting down? | 03:30 |
| greghaynes | also \O/ for being able to semi-successfully hack up a heat template | 03:31 |
| StevenK | lifeless: Try two: I have a mocked out function, but I don't have one of the arguments when I'm calling it, since I believe you have to call mocks before the test will do so? | 03:33 |
| ccrouch | does anyone now of a nice way to make a server use its non-default NIC as the preferred boot device? | 03:50 |
| ccrouch | where nice==not through the BIOS, since that gets tiresome when you have lots of servers | 03:50 |
| ccrouch | I'm looking at you HP ProLiant BL465c G7 :-) | 03:51 |
| ccrouch | ilo and ipmitool aren't helping | 04:07 |
| greghaynes | Yea, ive only ever been able to say bootdev pxe or disk with ipmi, not which nic | 04:07 |
| greghaynes | vendor ilo? | 04:08 |
| mordred | wow | 04:09 |
| mordred | apparently I hired someone to a job that had the following title: | 04:09 |
| mordred | Coloud Auomation and Distribution Engineer | 04:09 |
| mordred | I mean | 04:10 |
| greghaynes | :p | 04:10 |
| mordred | those aren't even words | 04:10 |
| greghaynes | IKR | 04:10 |
| jesusaurus | proof that you need some sort of cloud automation... | 04:10 |
| StevenK | mordred: https://xkcd.com/1319/ | 04:10 |
| mordred | :) | 04:12 |
| *** ramishra_ has quit IRC | 04:12 | |
| *** cd-undercloud has joined #tripleo | 04:26 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 04:26 |
| *** cd-undercloud has quit IRC | 04:26 | |
| SpamapS | greghaynes: ^5 for hacking on heat templates :) | 04:37 |
| greghaynes | \O | 04:37 |
| SpamapS | greghaynes: regarding the .ok file being written.. I think the answer there is to make that process idempotent on a distributed level (as in, don't write it to the disk, but to a database) | 04:39 |
| SpamapS | greghaynes: that's actually exactly where I'd expect what you're doing to fail. | 04:39 |
| greghaynes | sweet | 04:40 |
| greghaynes | do you know what kicks off creating that file? | 04:40 |
| SpamapS | greghaynes: scripts | 04:41 |
| SpamapS | greghaynes: elements/boot-stack/os-refresh-config/configure.d/51-init-openstack:OK=/mnt/state/var/lib/boot-stack/init-openstack.ok | 04:41 |
| greghaynes | hahaa, I totally wasa looking at that file and didnt look down a couple lines | 04:42 |
| greghaynes | thanks :) | 04:42 |
| lifeless | greghaynes: hey cool | 04:43 |
| lifeless | greghaynes: so the control plane isn't read for N nodes | 04:44 |
| lifeless | greghaynes: specifically, you need to cope with >1 node running rabbit and DB, because they have shared state. | 04:44 |
| greghaynes | My thinking was do this where theres data everywhere, then do the rabbitmq, then db, and so on | 04:44 |
| lifeless | greghaynes: there may be a cinder issue too; swift should work (but we may find glitches) | 04:44 |
| lifeless | greghaynes: so - which node should neutron-openvswitch-agent connect to to do DB updates? | 04:45 |
| greghaynes | Although im happy to try something else if thats a pretty crazy idea | 04:45 |
| lifeless | greghaynes: in your approach ? | 04:45 |
| greghaynes | Yep, figured itd break all the things :) | 04:46 |
| greghaynes | so the db thing is probably first deal to poke at? | 04:46 |
| greghaynes | Ive noticed the clustered mq has a fair amount of work done already | 04:47 |
| lifeless | yeah | 04:48 |
| lifeless | so the plan is to switch from distro mysql to percona xtradb | 04:49 |
| lifeless | in prrinciple its simple: we need a config for running that on one node; and a config for running in on 3 | 04:49 |
| greghaynes | ok | 04:50 |
| lifeless | (since two-node configs suffer split brain) | 04:50 |
| greghaynes | Yea, was just reading up on how it clusters | 04:50 |
| lifeless | greghaynes: I agree with spamaps 'move init-done checks to look inthe DB instead' | 04:50 |
| * greghaynes will play with percona a bit | 04:52 | |
| lifeless | greghaynes: I just think you won't be able to make the heat template succeed in any sane way until the db is clustered | 04:52 |
| greghaynes | Yep, totally | 04:53 |
| SpamapS | agreed | 04:55 |
| SpamapS | btw, percona-xtradb-cluster was uploaded to debian unstable today | 04:55 |
| SpamapS | but I think we'll probably just get it from Percona anyway. | 04:55 |
| SpamapS | I wonder if they ever made Fedora packages. | 04:56 |
| SpamapS | greghaynes: you best make an effort to look into fedora packages for percona xtradb before getting too far into playing | 04:56 |
| *** AaronGr has quit IRC | 04:56 | |
| greghaynes | ah, if that doesnt exist its not an option? | 04:57 |
| *** AaronGr_Zzz is now known as AaronGr | 04:57 | |
| greghaynes | Was making me think, if we assume the two db's are super close latency wise, is there not a sync replication that could suffice for mvp? | 04:58 |
| mordred | lifeless: just use drizzle | 04:59 |
| greghaynes | Do we have anyone around who knows about that? | 04:59 |
| greghaynes | :p | 04:59 |
| SpamapS | mordred: memmmmorrrieeeesss | 05:00 |
| lifeless | greghaynes: they have a binary tarball you can just download and use | 05:01 |
| SpamapS | greghaynes: You might be able to find mariadb galera packages for fedora | 05:01 |
| lifeless | greghaynes: will work on any platform | 05:01 |
| SpamapS | oh there's a static binary tarball? | 05:01 |
| lifeless | greghaynes: I discussed this with flamingspork before he switched | 05:01 |
| greghaynes | aha | 05:01 |
| lifeless | mordred: patches accepted, but for this we did a design discussion @ HK and was decided then :) | 05:02 |
| *** CaptTofu has joined #tripleo | 05:07 | |
| *** CaptTofu has quit IRC | 05:11 | |
| *** killer_prince has quit IRC | 05:14 | |
| *** killer_prince has joined #tripleo | 05:14 | |
| *** cd-undercloud has joined #tripleo | 05:48 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 05:48 |
| *** cd-undercloud has quit IRC | 05:48 | |
| *** rwsu has quit IRC | 06:07 | |
| *** rwsu has joined #tripleo | 06:07 | |
| *** killer_prince has quit IRC | 06:24 | |
| *** rwsu has quit IRC | 06:25 | |
| *** cadenzajon has joined #tripleo | 06:33 | |
| *** cadenzajon has quit IRC | 06:37 | |
| *** rwsu has joined #tripleo | 06:37 | |
| *** saju_m has joined #tripleo | 06:38 | |
| *** coolsvap has joined #tripleo | 06:40 | |
| *** saju_m has quit IRC | 06:43 | |
| *** saju_m has joined #tripleo | 06:50 | |
| *** saju_m has quit IRC | 06:51 | |
| *** CaptTofu has joined #tripleo | 07:07 | |
| *** cd-undercloud has joined #tripleo | 07:09 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 07:09 |
| *** cd-undercloud has quit IRC | 07:09 | |
| *** CaptTofu has quit IRC | 07:12 | |
| *** mrunge has joined #tripleo | 07:14 | |
| *** e0ne has joined #tripleo | 07:34 | |
| *** viktors has joined #tripleo | 07:34 | |
| *** e0ne has quit IRC | 07:35 | |
| *** saju_m has joined #tripleo | 07:37 | |
| *** d0ugal has joined #tripleo | 07:41 | |
| *** d0ugal has joined #tripleo | 07:41 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Make load-image check image checksum https://review.openstack.org/71662 | 07:47 |
| *** rlandy has joined #tripleo | 07:49 | |
| *** jprovazn has joined #tripleo | 07:50 | |
| *** saju_m has quit IRC | 07:52 | |
| *** bauzas has joined #tripleo | 08:01 | |
| *** killer_prince has joined #tripleo | 08:07 | |
| *** mrunge has quit IRC | 08:23 | |
| *** mrunge has joined #tripleo | 08:23 | |
| SpamapS | stevebaker: commented on https://review.openstack.org/#/c/72533/ | 08:26 |
| SpamapS | stevebaker: thanks for it btw! | 08:26 |
| rpodolyaka | morning tripleo | 08:26 |
| *** e0ne has joined #tripleo | 08:27 | |
| *** jtomasek has joined #tripleo | 08:28 | |
| *** rdopieralski has joined #tripleo | 08:30 | |
| *** cd-undercloud has joined #tripleo | 08:30 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 08:30 |
| *** cd-undercloud has quit IRC | 08:30 | |
| *** hashar has joined #tripleo | 08:33 | |
| *** killer_prince has quit IRC | 08:34 | |
| *** killer_prince has joined #tripleo | 08:35 | |
| *** gcha has joined #tripleo | 08:35 | |
| *** hashar has quit IRC | 08:38 | |
| *** hashar has joined #tripleo | 08:40 | |
| *** ifarkas has joined #tripleo | 08:41 | |
| *** hashar_ has joined #tripleo | 08:41 | |
| *** hashar has quit IRC | 08:45 | |
| *** shardy_afk is now known as shardy | 08:48 | |
| *** e0ne_ has joined #tripleo | 08:49 | |
| *** pblaho has joined #tripleo | 08:50 | |
| *** saju_m has joined #tripleo | 08:51 | |
| *** e0ne has quit IRC | 08:53 | |
| *** dtantsur has joined #tripleo | 08:54 | |
| jprovazn | lifeless: ping | 08:56 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Update URL for Tuskar installation instructions https://review.openstack.org/72389 | 08:57 |
| *** jistr has joined #tripleo | 08:58 | |
| *** markmc has joined #tripleo | 09:00 | |
| *** hashar_ is now known as hashar | 09:06 | |
| *** CaptTofu has joined #tripleo | 09:08 | |
| *** jcoufal has joined #tripleo | 09:12 | |
| *** CaptTofu has quit IRC | 09:13 | |
| Ng | ttx: morning | 09:16 |
| Ng | derp | 09:16 |
| Ng | morning | 09:16 |
| *** ramishra has joined #tripleo | 09:19 | |
| lifeless | jprovazn: hi | 09:21 |
| *** athomas has joined #tripleo | 09:22 | |
| GheRivero | morning all | 09:23 |
| jprovazn | lifeless: hi, I started playing with xtradb cluster, a potential dib element for this would conflict with existing mysql element so I wondered if it would make sense to extend DIB with "a or b element" feature when resolving dependencies or if different approach should be used | 09:24 |
| lifeless | greghaynes: ^ jprovazn is poking at that too - you should talk | 09:24 |
| jprovazn | ah, cool | 09:24 |
| lifeless | jprovazn: so, in principle yes, a or b is a thing we should do | 09:24 |
| lifeless | jprovazn: but | 09:24 |
| lifeless | jprovazn: I'd just make the existing mysql element install xtradb cluster | 09:25 |
| lifeless | jprovazn: we can use that for N=1 clusters, no ? | 09:25 |
| jprovazn | lifeless: yes, we can | 09:25 |
| lifeless | xtradb is *a* mysql :) | 09:25 |
| jprovazn | lifeless: but I don't like the approach of enforcing xtradb cluster TBH | 09:26 |
| jprovazn | lifeless: e.g. because of missing packages | 09:26 |
| lifeless | jprovazn: tell me about the conflicts in detail | 09:26 |
| lifeless | jprovazn: are they different config file? | 09:26 |
| lifeless | jprovazn: xtradb cluster can be packaged for any distro that wants to | 09:26 |
| jprovazn | lifeless: no - just additional config settings are required for xtradb + some postconfig setup | 09:26 |
| greghaynes | Hello again jprovazn :) | 09:27 |
| jprovazn | greghaynes: Hi :) | 09:27 |
| lifeless | jprovazn: so, if the difference is just settings, I would say that you could make the mysql element take an option for what flavour of mysql to install | 09:27 |
| lifeless | jprovazn: or perhaps have a mysql-config element + a mysql-mysql, a mysql-mariadb, a mysql-xtradb etc. | 09:28 |
| jprovazn | lifeless: I see, yea - it might work | 09:28 |
| lifeless | jprovazn: but - that all said - while I support diversity, I don't see this flavour of db server versions as being distinct enough to worry about | 09:28 |
| lifeless | jprovazn: so personally, I'd do xtradb, which was agreed on in HK; and we can iterate on package presence in distros separately. | 09:29 |
| jprovazn | lifeless: ok | 09:29 |
| lifeless | jprovazn: I'd *much* rather be able to say 'the db is HA, end of story' than have different instructions... | 09:29 |
| lifeless | jprovazn: for 'oh yeah, on this platform, from this vendor, it's HA via pacemaker & DRDB, and on this other platform, it's HA via galera' | 09:30 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Show overcloud detail with right cond https://review.openstack.org/72604 | 09:30 |
| lifeless | jprovazn: at least for now- when someone truely cares about the difference, it would be patches-accepted time. What do you think ? | 09:30 |
| jprovazn | lifeless: makes sense to me | 09:30 |
| jprovazn | greghaynes: lifeless: so I'll just proceed with updating the existing mysql element. greghaynes how far did you get with xtradb and what are your thoughts about this so far? | 09:32 |
| jprovazn | lifeless: greghaynes: so far makeing 2 nodes overcloud cluster worked for me, but i had some issues with initial SST sync (mostly about debugging xtrabackup scripts) | 09:33 |
| *** lucasagomes has joined #tripleo | 09:33 | |
| greghaynes | I havent really made any progress, was just reading up on it. Have mostly been playing with the multi control node heat template | 09:33 |
| lifeless | jprovazn: 2? I thought you needed an odd # for quorum ? | 09:34 |
| greghaynes | I do think its not a bad default for the tripleo story, not sure what the value add for supporting non ha is in that use case. | 09:34 |
| lifeless | greghaynes: super small clouds | 09:35 |
| jprovazn | lifeless: actually 2-nodes worked for me, I just had to pick one of them as "master" | 09:36 |
| lifeless | jprovazn: but failover - won't it split brain ? | 09:36 |
| *** coolsvap has quit IRC | 09:36 | |
| *** boris-42_ has quit IRC | 09:36 | |
| openstackgerrit | A change was merged to openstack/tuskar: Bind the Overcloud ID to the Overcloud Role https://review.openstack.org/72432 | 09:38 |
| lifeless | ok, gnight. | 09:38 |
| greghaynes | gn | 09:38 |
| jprovazn | lifeless: not sure, what you mean | 09:38 |
| jprovazn | lifeless: good night | 09:38 |
| greghaynes | jprovazn: AIUI the network partition case could result in split brain | 09:39 |
| greghaynes | this is what I was just reading up on though | 09:39 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Adjust neutron package install for Fedora https://review.openstack.org/71667 | 09:39 |
| greghaynes | kind of seems like the two node case should be a sync-replicated master/slave with VIP | 09:40 |
| SpamapS | two node galera has one node running the extra voting daemon thing | 09:40 |
| SpamapS | greghaynes: but what happens when you scale to 3? Total metamorphosis? | 09:41 |
| greghaynes | 3 or more you can do xtradb stuff. I assume it doesnt work for 2 because it needs some form of quorom | 09:41 |
| SpamapS | 2 node galera can work. It is vulnerable to split brain via its own usual split-brain avoidance mechanisms (quorum) but you can use alternate methods to move the "master" around. Otherwise, in 2 node clusters, if split brain occurs, one commits hari-kari because it won't have quorum | 09:42 |
| SpamapS | it works with 2 | 09:42 |
| SpamapS | you run an extra non-functional galera daemon thing that just votes but doesn't actually serve | 09:43 |
| SpamapS | that way the one with the extra thing has quorum | 09:43 |
| SpamapS | the other one shuts down | 09:43 |
| greghaynes | I think were saying the same thing, I just dont know it in mysql ecosystem :) | 09:43 |
| SpamapS | so if you are going to update those 2, you need to make sure the one you're not updating has the voting thing running | 09:43 |
| greghaynes | basically move master and STONITH | 09:43 |
| SpamapS | right | 09:43 |
| SpamapS | the usual 2 node story | 09:43 |
| greghaynes | sweet | 09:44 |
| *** kwwii_ has joined #tripleo | 09:44 | |
| SpamapS | http://www.mysqlperformanceblog.com/2012/07/25/percona-xtradb-cluster-failure-scenarios-with-only-2-nodes/ | 09:45 |
| jprovazn | http://www.mysqlperformanceblog.com/2012/07/25/percona-xtradb-cluster-failure-scenarios-with-only-2-nodes/ | 09:45 |
| jprovazn | heh | 09:45 |
| jprovazn | :) | 09:45 |
| lifeless | so the problem with 2 nodes IMNSHO is that 50% of the time, a failure results in an offline cloud | 09:45 |
| SpamapS | http://www.codership.com/wiki/doku.php?id=galera_arbitrator | 09:46 |
| lifeless | I'm inclined to argue that we support 1 (use backups) and 3 (you can tolerate a single node failure). | 09:46 |
| lifeless | and skip 2 (50% of the time you can tolerate a single node failure) | 09:46 |
| jprovazn | sounds good | 09:46 |
| lifeless | I did care about 2 a while back | 09:47 |
| SpamapS | lifeless: if you have an out of band mechanism for moving the arbitrator to a working node then I believe that is reduced to "50% of the time a failure results in a small downtime window while out of band thing recovers" | 09:47 |
| lifeless | my thinking has changed :) | 09:47 |
| greghaynes | 2 seems like as special a case as 1 and with very little value ad vs 3 :) | 09:48 |
| SpamapS | but yeah, just have 3 | 09:48 |
| lifeless | SpamapS: 'downtime until a system administrator has a) figured out whats wrong, b) ensured that the master really is offline and won't come cruising back with voted in logs, c) read up on manual failure rules, d) executed. | 09:48 |
| lifeless | SpamapS: != 'small downtime window' | 09:49 |
| SpamapS | not really. YOu could have multiple media communication between the two nodes, allowing you to trust that the other side is down (like we used to do with heartbeat and a null modem cable...) | 09:49 |
| lifeless | SpamapS: that doesn't tell you the other side is down | 09:50 |
| lifeless | SpamapS: it tells you that multiple media communication has failed | 09:50 |
| lifeless | SpamapS: and the otherside might be down. | 09:50 |
| lifeless | SpamapS: or | 09:50 |
| lifeless | SpamapS: the idiot that partitioned your LAN managed to cut the other cable too. | 09:50 |
| SpamapS | if you can reach your gateway, but you can't reach then via the switch or the direct cable connection, then you have two votes for "other side is down" and one vote for "we are up" | 09:50 |
| SpamapS | s/then/the other node/ | 09:51 |
| lifeless | SpamapS: no you don't :) | 09:51 |
| SpamapS | eh? | 09:51 |
| lifeless | SpamapS: you need to get much more creative your 'ways the network can fuck up' playbook | 09:51 |
| SpamapS | you going to tell me heartbeat doesn't work? | 09:51 |
| *** cd-undercloud has joined #tripleo | 09:51 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 09:51 |
| *** cd-undercloud has quit IRC | 09:51 | |
| lifeless | SpamapS: I'm telling you its possible to have two nodes on the same LAN, mutually unable to talk, and a heartbeat failure at the same time. | 09:52 |
| lifeless | SpamapS: and then you have a real mess to cleanup | 09:52 |
| SpamapS | so you have the switch failing, _AND_ your unbelievably simple null modem cable failing at the exact same moment? | 09:52 |
| SpamapS | Possible, yes. Likely?? | 09:53 |
| SpamapS | Anyway, I've had the null modem fail, and the switch fail, but never both. | 09:53 |
| SpamapS | and the TCP stack fail | 09:53 |
| SpamapS | ahh linux 2.2 ... mi amor | 09:53 |
| lifeless | SpamapS: so, I'm saying that the consequences of a split brain are really hard to cleanup | 09:53 |
| SpamapS | definitely | 09:54 |
| greghaynes | +1 | 09:54 |
| lifeless | SpamapS: and that engineering in the possibility seems unwise | 09:54 |
| greghaynes | 1..3...infinity? | 09:54 |
| lifeless | SpamapS: and giving you examples of how your 'but X might work' don't prevent it. | 09:54 |
| lifeless | SpamapS: and saying I'd like to not engineer it in. | 09:54 |
| SpamapS | prevention is not the same as swaying the odds in your favor | 09:54 |
| SpamapS | something that you end up doing a lot when resources are tight | 09:55 |
| SpamapS | lifeless: agreed | 09:55 |
| lifeless | SpamapS: If and when someone comes to us and says hand on heart, I want to use OpenStack, I want to deploy via TripleO, I can't tolerate downtime, but I can't afford a third pizza box for my overcloud control plane... | 09:55 |
| lifeless | SpamapS: then, sure. their problem :) | 09:56 |
| lifeless | SpamapS: IMO though, restricted resources should imply 'and accepts downtime' | 09:56 |
| lifeless | since downtime is money. | 09:56 |
| SpamapS | lifeless: so we could just do the arbitrator for 2 nodes, with a big red warning label and strong admonitions to get 3 or else. | 09:56 |
| lifeless | -> they should do backups, probably point in time replicated logs | 09:56 |
| lifeless | SpamapS: we could. I don't understand why we would. | 09:57 |
| lifeless | SpamapS: I'm not sure if you're just waxing pedantic, or you have a real use case. | 09:57 |
| lifeless | SpamapS: given its 2am, I'm suspecting waxing pedantic | 09:57 |
| SpamapS | I have a desire to have a better answer than no. | 09:58 |
| SpamapS | thats all :) | 09:58 |
| lifeless | SpamapS: what is the source of the desire? | 09:58 |
| SpamapS | Guilt? | 09:58 |
| lifeless | SpamapS: ... | 09:58 |
| SpamapS | Empathy? | 09:58 |
| SpamapS | Tradition mostly I think. :) | 09:58 |
| lifeless | SpamapS: ah. So shoot *that* in the head. | 09:59 |
| lifeless | Here is my proposal for super resource constrained. Use 1 control node and streaming backups to the undercloud. | 09:59 |
| lifeless | if the node dies, nova rebuild it onto new hardware and restore the backup. | 09:59 |
| SpamapS | and with 3, no streaming backups? | 09:59 |
| lifeless | after you go buy a new server you cheapskate. | 10:00 |
| lifeless | with 3, backups become orthogonal | 10:00 |
| lifeless | still a part of hygenic deployments IMO. | 10:00 |
| greghaynes | How are we going to do the rolling updates for the 3node xtradb without some time where we risk split brain due to even quorom votes in network partition? | 10:00 |
| greghaynes | eek, sorry for asking at 2am | 10:00 |
| lifeless | bugs can wipe data etc. | 10:00 |
| lifeless | greghaynes: so it won't split brain | 10:00 |
| lifeless | greghaynes: it will shut down. | 10:00 |
| greghaynes | The whole cluster? | 10:00 |
| SpamapS | greghaynes: see my latest openstack-dev post :) | 10:00 |
| lifeless | greghaynes: yes | 10:01 |
| lifeless | greghaynes: with a 3 node cluster, any 2 can cooperate to be up. | 10:01 |
| SpamapS | http://lists.openstack.org/pipermail/openstack-dev/2014-February/026816.html | 10:01 |
| SpamapS | specifically | 10:01 |
| lifeless | greghaynes: if we redeploy 1 and another dies, then there are no 2-node pairs, so everything goes into 'I'm solo, sob' mode. | 10:01 |
| lifeless | SpamapS: btw, I think that Also in the case of a Galera DB where we may even be running | 10:02 |
| lifeless | degraded, we want to ensure that we have quorum before proceeding. | 10:02 |
| lifeless | is a mis-model of the problem :) | 10:02 |
| lifeless | SpamapS: heat should have 2 nodes that have reported in as 'ok' via wait conditions | 10:02 |
| lifeless | SpamapS: your suggestion that we block doesn't prevent the race condition | 10:03 |
| SpamapS | huh | 10:03 |
| lifeless | SpamapS: we could have all nodes up (or, quorum + 1 - which is I think what you meant, not quorum) | 10:03 |
| SpamapS | It's a graceful avoidance | 10:03 |
| SpamapS | not race prevention | 10:03 |
| lifeless | SpamapS: at the time that heat asks | 10:03 |
| lifeless | SpamapS: and then, a node fails as heat starts the deploy | 10:03 |
| SpamapS | you're about to reboot the box | 10:03 |
| lifeless | SpamapS: I may have misunderstood | 10:04 |
| SpamapS | anything _COULD_ happen... but I'm going to say "let's not make a known problem worse" | 10:04 |
| lifeless | SpamapS: it sounded like you were suggesting having the ability to cancel heat deploys (vs just taking your time getting ready) | 10:04 |
| SpamapS | That is for the user whether to cancel it or to delay it. | 10:05 |
| SpamapS | In galera land if we're degraded I do suggest we just delay until we aren't degraded. | 10:05 |
| SpamapS | but in nova if we can't evacuate workloads.. we're screwed... cancel it. | 10:05 |
| lifeless | SpamapS: that will mean we need different code for 1-hypervisor deploys vs 2+ | 10:06 |
| lifeless | SpamapS: because how will a 1-hupervisor node know that its not meant to be able to evacuate itself | 10:06 |
| SpamapS | that is always true when you run degraded | 10:06 |
| lifeless | SpamapS: lets talk tomorrow | 10:06 |
| *** lazy_prince has joined #tripleo | 10:06 | |
| lifeless | SpamapS: I fear over-thinking | 10:06 |
| SpamapS | there has to be an assertion "my uptime guarantee is 0" | 10:06 |
| lifeless | SpamapS: sure; tomorrow though? | 10:07 |
| lifeless | I need sleep. | 10:07 |
| lifeless | Mr Owl :) | 10:07 |
| SpamapS | mmk or just write down how you want it so I don't have to keep stabbing at the idea you have already. | 10:08 |
| SpamapS | I mean | 10:08 |
| SpamapS | you're right.. ;) | 10:08 |
| SpamapS | we forgot to make the T-shirts and all | 10:09 |
| lifeless | SpamapS: LOL | 10:09 |
| lifeless | SpamapS: no, is 11pm | 10:09 |
| lifeless | SpamapS: I may well be wrong | 10:09 |
| SpamapS | it's just easier if you state the right idea, and let us spin in circles on our bad variations of it, than us spinning in circles not even in the general vicinity of right :) | 10:09 |
| lifeless | SpamapS: which is why I haven't replied to the mail yet; I need to think it through too | 10:09 |
| *** xuhanp has quit IRC | 10:12 | |
| lifeless | SpamapS: actually scratch it, I agree that there should be errors, even if I may eventually disagree about why. | 10:13 |
| lifeless | SpamapS: heat has timeouts. Instance scripts can cause update errors merely by being slow. | 10:13 |
| lifeless | SpamapS: so either we say 'no timeout', or errors will be an intrinsic part, and if they are intrinsic, there's no reason not to embrace them. | 10:14 |
| *** saju_m has quit IRC | 10:25 | |
| *** saju_m has joined #tripleo | 10:26 | |
| *** jrist has quit IRC | 10:30 | |
| *** jrist has joined #tripleo | 10:42 | |
| *** hashar is now known as hasharAway | 10:45 | |
| *** max_lobur_afk is now known as max_lobur | 10:46 | |
| *** lazy_prince has quit IRC | 10:50 | |
| *** hasharAway has quit IRC | 10:57 | |
| *** CaptTofu has joined #tripleo | 10:57 | |
| *** CaptTofu has quit IRC | 11:02 | |
| *** cd-undercloud has joined #tripleo | 11:13 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 11:13 |
| *** cd-undercloud has quit IRC | 11:13 | |
| openstackgerrit | A change was merged to openstack/tuskar: Enable gating on E711 and E712 https://review.openstack.org/72295 | 11:14 |
| *** boris-42_ has joined #tripleo | 11:16 | |
| *** e0ne_ has quit IRC | 11:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: keystone package install support (openSUSE) https://review.openstack.org/71122 | 11:47 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix python-ceilometerclient package install https://review.openstack.org/71136 | 11:50 |
| *** e0ne has joined #tripleo | 11:50 | |
| *** lazy_prince has joined #tripleo | 11:51 | |
| *** mrunge has quit IRC | 11:52 | |
| *** e0ne_ has joined #tripleo | 11:59 | |
| *** d0ugal has quit IRC | 12:03 | |
| *** e0ne has quit IRC | 12:03 | |
| *** d0ugal has joined #tripleo | 12:04 | |
| *** d0ugal has joined #tripleo | 12:04 | |
| *** mrunge has joined #tripleo | 12:07 | |
| *** gcha has quit IRC | 12:07 | |
| *** tserong has quit IRC | 12:22 | |
| *** tserong has joined #tripleo | 12:22 | |
| *** tserong has joined #tripleo | 12:22 | |
| *** akuznetsov has joined #tripleo | 12:30 | |
| *** lucasagomes is now known as lucas-hungry | 12:30 | |
| *** cd-undercloud has joined #tripleo | 12:39 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 12:39 |
| *** cd-undercloud has quit IRC | 12:39 | |
| openstackgerrit | Ronelle Landy proposed a change to openstack/tuskar: Modifies I578b4e9f238590ea245b827bc75d252568d194fe to add ssl, swift elements to the template created. Also includes parameters to pass to 'heat.create_stack' or 'heat update_stack'. Co-Authored-By: marios <marios@redhat.com> https://review.openstack.org/72642 | 12:58 |
| openstackgerrit | Ronelle Landy proposed a change to openstack/tuskar: Modification to I578b4e9f238590ea245b827bc75d252568d194fe Adds ssl, swift elements to the template created. Also includes parameters to pass to 'heat.create_stack' or 'heat update_stack'. Co-Authored-By: marios <marios@redhat.com> https://review.openstack.org/72642 | 13:01 |
| jistr | heya | 13:07 |
| jistr | i just reported this https://bugs.launchpad.net/tripleo/+bug/1278861 | 13:07 |
| uvirtbot | Launchpad bug 1278861 in tripleo "Devtest timeout waiting on undercloud" [High,Triaged] | 13:07 |
| jistr | there is a workaround when running locally | 13:07 |
| jistr | but it breaks Jenkins | 13:07 |
| jistr | ---> should i change the importance to critical? | 13:08 |
| *** weshay has joined #tripleo | 13:09 | |
| *** gcha has joined #tripleo | 13:09 | |
| slagle | jistr: is there anything in the heat or nova logs on the seed that might help diagnose? | 13:17 |
| jistr | slagle: i already reprovisioned the machine | 13:18 |
| *** mrunge has quit IRC | 13:20 | |
| *** hashar has joined #tripleo | 13:21 | |
| *** kwwii_ has quit IRC | 13:21 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Fixing a nodes table https://review.openstack.org/72648 | 13:22 |
| *** ramishra has quit IRC | 13:23 | |
| *** jdob has joined #tripleo | 13:26 | |
| jistr | slagle: btw i vaguely recall searching for logs but not finding any. Are they in /var/log/nova and /var/log/heat on the seed? | 13:27 |
| slagle | jistr: no, you can use journalctl | 13:34 |
| slagle | e.g., journalctl --full -u nova-compute | 13:34 |
| jistr | slagle: ah, thanks | 13:36 |
| *** rbrady has joined #tripleo | 13:37 | |
| openstackgerrit | Ronelle Landy proposed a change to openstack/tuskar: WIP: Using merge.py from tuskar to generate overcloud.yaml https://review.openstack.org/52045 | 13:39 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Don't hardcode environment.d https://review.openstack.org/72654 | 13:41 |
| *** lucas-hungry is now known as lucasagomes | 13:44 | |
| *** CaptTofu has joined #tripleo | 13:44 | |
| *** ramishra has joined #tripleo | 13:52 | |
| *** noslzzp has joined #tripleo | 13:52 | |
| *** ramishra_ has joined #tripleo | 13:55 | |
| *** julim has joined #tripleo | 13:57 | |
| *** ramishra has quit IRC | 13:58 | |
| *** ramishra has joined #tripleo | 13:59 | |
| *** lblanchard has joined #tripleo | 14:01 | |
| *** dprince has joined #tripleo | 14:02 | |
| *** ramishra_ has quit IRC | 14:02 | |
| *** cd-undercloud has joined #tripleo | 14:03 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:03 |
| *** cd-undercloud has quit IRC | 14:03 | |
| *** morazi has joined #tripleo | 14:06 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-incubator: Cleanup RC files in .gitignore. https://review.openstack.org/72520 | 14:15 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Remove unneeded service mapping for mysql on openSUSE https://review.openstack.org/71914 | 14:17 |
| *** pblaho has quit IRC | 14:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Install systemd service files into /usr/lib/ https://review.openstack.org/71900 | 14:19 |
| *** ifarkas has quit IRC | 14:24 | |
| *** ifarkas has joined #tripleo | 14:25 | |
| *** saju_m has quit IRC | 14:27 | |
| *** saju_m has joined #tripleo | 14:28 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: initialize Keystone database on openSUSE https://review.openstack.org/71265 | 14:30 |
| openstackgerrit | Dougal Matthews proposed a change to openstack/python-tuskarclient: Add CLI support for Overclouds and Overcloud Roles https://review.openstack.org/72306 | 14:31 |
| *** lazy_prince has quit IRC | 14:35 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Make the MIRROR_TARGET directory if it didn't exist https://review.openstack.org/71531 | 14:38 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add package install support to cinder https://review.openstack.org/70271 | 14:40 |
| *** jprovazn has quit IRC | 14:49 | |
| *** jprovazn has joined #tripleo | 14:51 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Implement the deployment overview and progress page https://review.openstack.org/72671 | 14:53 |
| *** matty_dubs|gone is now known as matty_dubs | 14:57 | |
| *** jrist has quit IRC | 14:59 | |
| *** CaptTofu has quit IRC | 15:00 | |
| *** sgrasley has quit IRC | 15:25 | |
| openstackgerrit | Ronelle Landy proposed a change to openstack/tuskar: WIP: Using merge.py from tuskar to generate overcloud.yaml https://review.openstack.org/52045 | 15:26 |
| *** cd-undercloud has joined #tripleo | 15:27 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 15:27 |
| *** cd-undercloud has quit IRC | 15:27 | |
| *** jcoufal_ has joined #tripleo | 15:33 | |
| *** rdopieralski has quit IRC | 15:35 | |
| *** boris-42_ has quit IRC | 15:36 | |
| *** jcoufal has quit IRC | 15:36 | |
| *** dtantsur has quit IRC | 15:36 | |
| *** noslzzp has quit IRC | 15:36 | |
| *** jprovazn has quit IRC | 15:37 | |
| *** noslzzp has joined #tripleo | 15:40 | |
| *** dtantsur has joined #tripleo | 15:41 | |
| openstackgerrit | Jay Dobies proposed a change to openstack/tuskar: Fix delete cascade for attribute and counts https://review.openstack.org/72464 | 15:41 |
| *** jrist has joined #tripleo | 15:42 | |
| *** coolsvap has joined #tripleo | 15:42 | |
| *** saju_m has quit IRC | 15:45 | |
| *** saju_m has joined #tripleo | 15:45 | |
| *** jrist has quit IRC | 15:47 | |
| openstackgerrit | Dougal Matthews proposed a change to openstack/python-tuskarclient: Add CLI support for Overclouds and Overcloud Roles https://review.openstack.org/72306 | 15:49 |
| *** geerdest has joined #tripleo | 15:55 | |
| *** mrunge has joined #tripleo | 15:56 | |
| *** rdopieralski has joined #tripleo | 16:02 | |
| *** jrist has joined #tripleo | 16:02 | |
| *** rdopieralski has quit IRC | 16:02 | |
| *** victor_lowther_ has quit IRC | 16:02 | |
| *** rdopieralski has joined #tripleo | 16:03 | |
| *** dtantsur has quit IRC | 16:04 | |
| *** victor_lowther_ has joined #tripleo | 16:05 | |
| *** jrist has quit IRC | 16:06 | |
| *** jrist has joined #tripleo | 16:12 | |
| *** jrist has quit IRC | 16:13 | |
| *** ramishra has quit IRC | 16:15 | |
| *** jcoufal_ is now known as jcoufal | 16:15 | |
| *** CaptTofu has joined #tripleo | 16:16 | |
| *** ramishra has joined #tripleo | 16:16 | |
| *** dtantsur has joined #tripleo | 16:17 | |
| *** jrist has joined #tripleo | 16:17 | |
| *** mrunge has quit IRC | 16:21 | |
| *** bnemec has quit IRC | 16:31 | |
| *** rdopieralski has quit IRC | 16:32 | |
| *** ramishra_ has joined #tripleo | 16:32 | |
| *** bnemec has joined #tripleo | 16:34 | |
| *** ramishra has quit IRC | 16:34 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar: Update Install doc https://review.openstack.org/72697 | 16:35 |
| *** jistr has quit IRC | 16:47 | |
| *** boris-42_ has joined #tripleo | 16:52 | |
| *** matty_dubs is now known as matty_dubs|lunch | 16:52 | |
| *** max_lobur has quit IRC | 16:52 | |
| *** cd-undercloud has joined #tripleo | 16:53 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 16:53 |
| *** cd-undercloud has quit IRC | 16:53 | |
| *** bauzas has quit IRC | 16:58 | |
| *** e0ne_ has quit IRC | 16:59 | |
| *** cwolferh has joined #tripleo | 17:02 | |
| *** noslzzp has quit IRC | 17:05 | |
| openstackgerrit | Dougal Matthews proposed a change to openstack/python-tuskarclient: Add CLI support for Overclouds and Overcloud Roles https://review.openstack.org/72306 | 17:06 |
| *** athomas has quit IRC | 17:07 | |
| *** hashar has quit IRC | 17:07 | |
| *** dtantsur has quit IRC | 17:07 | |
| *** markmc has quit IRC | 17:08 | |
| *** cwolferh has quit IRC | 17:10 | |
| *** cwolferh has joined #tripleo | 17:11 | |
| *** akuznetsov has quit IRC | 17:11 | |
| *** akuznetsov has joined #tripleo | 17:17 | |
| *** jcoufal has quit IRC | 17:17 | |
| *** CaptTofu has quit IRC | 17:19 | |
| *** ifarkas has quit IRC | 17:20 | |
| *** max_lobur has joined #tripleo | 17:21 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Adding real event-list api https://review.openstack.org/72413 | 17:22 |
| *** viktors has quit IRC | 17:31 | |
| *** gothicmindfood has joined #tripleo | 17:34 | |
| *** max_lobur is now known as max_lobur_afk | 17:36 | |
| *** gothicmindfood has left #tripleo | 17:37 | |
| *** CaptTofu has joined #tripleo | 17:43 | |
| *** bauzas has joined #tripleo | 17:46 | |
| *** matty_dubs|lunch is now known as matty_dubs | 17:51 | |
| *** noslzzp has joined #tripleo | 17:52 | |
| *** ccrouch has quit IRC | 17:58 | |
| *** mrunge has joined #tripleo | 17:59 | |
| lifeless | morning | 18:17 |
| *** cd-undercloud has joined #tripleo | 18:18 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 18:18 |
| *** cd-undercloud has quit IRC | 18:18 | |
| *** ccrouch has joined #tripleo | 18:19 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar: Update Install doc https://review.openstack.org/72697 | 18:22 |
| *** lucasagomes has quit IRC | 18:33 | |
| *** epim has joined #tripleo | 18:37 | |
| *** bauzas has quit IRC | 18:38 | |
| *** gcha has quit IRC | 18:41 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar: Created initial data for Tuskar https://review.openstack.org/72734 | 18:45 |
| *** e0ne has joined #tripleo | 18:49 | |
| lifeless | SpamapS: got a sec ? | 18:53 |
| SpamapS | lifeless: I do! | 18:55 |
| lifeless | SpamapS: great! | 18:55 |
| lifeless | so this stuff | 18:55 |
| lifeless | watch_server_url: | 18:55 |
| lifeless | Fn::Join: | 18:55 |
| lifeless | - '' | 18:55 |
| lifeless | - - http:// | 18:55 |
| lifeless | - {"Fn::Select": [ 0, {"Fn::Select": [ "ctlplane", {"Fn::GetAtt": [ undercloud, networks ]} ]}]} | 18:55 |
| lifeless | - ":8003" | 18:55 |
| lifeless | in undercloud-source | 18:55 |
| lifeless | I'm trying to figure out how it determines that its looking up the notcompute ip address | 18:55 |
| lifeless | like | 18:55 |
| lifeless | I want to make this be the default value for a parameter | 18:56 |
| lifeless | is 'undercloud' there a network ref, or the server definition reference ? | 18:56 |
| SpamapS | lifeless: undercloud is the resource we're getting the networks from | 18:57 |
| SpamapS | lifeless: I think _technically_ we need to make that {Ref: undercloud} | 18:58 |
| lifeless | ok | 18:58 |
| SpamapS | no | 18:58 |
| SpamapS | Fn::GetAtt takes names | 18:58 |
| lifeless | I will have a patch for you to review in a sec | 18:58 |
| SpamapS | sorry ignore that technically | 18:58 |
| *** pblaho has joined #tripleo | 18:58 | |
| *** jcoufal has joined #tripleo | 18:59 | |
| *** jistr has joined #tripleo | 18:59 | |
| *** rpodolyaka1 has joined #tripleo | 18:59 | |
| SpamapS | also in HOT that Fn::Select stuff becomes get_attribute [ undercloud, networks.ctlplane.0 ] | 18:59 |
| lifeless | why can't we use get_attribute in the current templates? | 19:00 |
| SpamapS | because they're cfn | 19:00 |
| lifeless | not an answer :) | 19:00 |
| SpamapS | merge.py | 19:01 |
| SpamapS | doesn't speak HOT | 19:01 |
| SpamapS | been hoping we can avoid teaching it HOT | 19:01 |
| *** olaph has joined #tripleo | 19:01 | |
| SpamapS | should we be meeting? | 19:01 |
| lifeless | ohcrapyes | 19:02 |
| lifeless | SpamapS: can you ? I have to pack up a machine for technician to pick up | 19:02 |
| *** ramishra_ has quit IRC | 19:02 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Permit overriding the undercloud controller ip. https://review.openstack.org/72740 | 19:03 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Make it possible to do physical networks for VMs. https://review.openstack.org/71721 | 19:03 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Allow manual hosts entry exports in overcloud. https://review.openstack.org/71725 | 19:03 |
| lifeless | SpamapS: and tell me if ^ is legit. I think it is. | 19:03 |
| lifeless | https://review.openstack.org/72740 specifically | 19:03 |
| *** cadenzajon has joined #tripleo | 19:04 | |
| SpamapS | ok will start the meeting | 19:05 |
| *** televiper has joined #tripleo | 19:09 | |
| *** michchap has joined #tripleo | 19:12 | |
| *** michchap_ has quit IRC | 19:14 | |
| *** jprovazn has joined #tripleo | 19:15 | |
| *** coolsvap has quit IRC | 19:17 | |
| *** jrist has quit IRC | 19:20 | |
| openstackgerrit | Jonathan Brownell proposed a change to openstack/os-apply-config: fixed syntax error in documentation (missing "}" in mustache template example) https://review.openstack.org/72741 | 19:21 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add username to rabbit in local heat metadata https://review.openstack.org/71446 | 19:22 |
| rpodolyaka1 | shadower: I believe all you need to know is here https://wiki.openstack.org/wiki/TripleO/ReleaseManagement | 19:24 |
| rpodolyaka1 | shadower: still need to cleanup my scripts for automation of this. going to do this week and then share them | 19:25 |
| rpodolyaka1 | shadower: though it takes me more time to actually read the changes and figure out what version part must be bumped :) | 19:27 |
| shadower | rpodolyaka1: ah, fantastic, thanks! | 19:28 |
| rpodolyaka1 | shadower: np | 19:28 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Show overcloud detail with right cond https://review.openstack.org/72604 | 19:36 |
| *** mrunge has quit IRC | 19:39 | |
| *** bauzas has joined #tripleo | 19:41 | |
| *** cd-undercloud has joined #tripleo | 19:42 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 19:42 |
| *** cd-undercloud has quit IRC | 19:42 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar: Created initial data for Tuskar https://review.openstack.org/72734 | 19:48 |
| *** saju_m has quit IRC | 19:49 | |
| *** saju_m has joined #tripleo | 19:50 | |
| *** markmc has joined #tripleo | 19:53 | |
| *** jprovazn is now known as jprovazn_afk | 19:57 | |
| *** pblaho has quit IRC | 19:59 | |
| *** jistr has quit IRC | 20:02 | |
| *** hashar has joined #tripleo | 20:03 | |
| *** saju_m has quit IRC | 20:04 | |
| *** saju_m has joined #tripleo | 20:04 | |
| *** cadenzajon has quit IRC | 20:16 | |
| *** cadenzajon has joined #tripleo | 20:17 | |
| *** jcoufal has quit IRC | 20:19 | |
| *** spzala has joined #tripleo | 20:20 | |
| *** rpodolyaka1 has quit IRC | 20:20 | |
| *** epim has quit IRC | 20:22 | |
| *** jcoufal has joined #tripleo | 20:25 | |
| *** rpodolyaka1 has joined #tripleo | 20:27 | |
| lifeless | SpamapS: ok, time to abandon debian :P | 20:29 |
| *** jcoufal has quit IRC | 20:31 | |
| *** saju_m has quit IRC | 20:32 | |
| *** ramishra has joined #tripleo | 20:34 | |
| *** ramishra has quit IRC | 20:39 | |
| *** pblaho has joined #tripleo | 20:44 | |
| *** epim has joined #tripleo | 20:48 | |
| SpamapS | lifeless: seriously | 20:51 |
| SpamapS | lifeless: tho.. for what? Gentoo? ;) | 20:51 |
| pleia2 | SpamapS: arch | 20:51 |
| SpamapS | uses systemd? | 20:51 |
| *** rlandy has quit IRC | 20:52 | |
| pleia2 | ah that :) | 20:52 |
| SpamapS | lifeless: here's a suggestion to fork it that I think may actually be taken seriously. | 20:52 |
| SpamapS | there's | 20:52 |
| SpamapS | not here's | 20:52 |
| lifeless | pleia2: SpamapS: NixOS ? | 20:53 |
| lifeless | dprince: oh hai | 20:53 |
| lifeless | dprince: how goes the RH region recabling ? | 20:54 |
| dprince | lifeless: still waiting. The team had some personal issues to deal with so it has been bad timing for us unfortunately. | 20:55 |
| *** jprovazn_afk has quit IRC | 20:55 | |
| lifeless | dprince: ok, hope they are happier soon | 20:56 |
| lifeless | dprince: I want a time slice to talk dhcp-all-interfaces again | 20:56 |
| lifeless | dprince: I've been mulling it over :) | 20:57 |
| dprince | lifeless: yep. me too. I sent you a brief status update that I sent to Mark/Derek this morning. | 20:57 |
| lifeless | dprince: oh, let me look that up | 20:57 |
| dprince | lifeless: not much I can do other than bug people... which I hate to do in this specific case | 20:57 |
| dprince | lifeless: regarding the dhcp-all-interfaces thing I can talk now, or tomorrow at this time if you'd like | 20:58 |
| lifeless | 10m from now ok ? | 20:58 |
| lifeless | just like to get some food in my face hole | 20:58 |
| dprince | lifeless: yep | 20:58 |
| openstackgerrit | Jonathan Brownell proposed a change to openstack/os-apply-config: fixed syntax error in docs (mustache example) https://review.openstack.org/72741 | 21:00 |
| *** jdob_ has joined #tripleo | 21:01 | |
| *** ccrouch has quit IRC | 21:02 | |
| *** cd-undercloud has joined #tripleo | 21:05 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 21:05 |
| *** cd-undercloud has quit IRC | 21:05 | |
| *** e0ne has quit IRC | 21:07 | |
| lifeless | ok | 21:08 |
| lifeless | dprince: - hangout? | 21:08 |
| lifeless | anyone else interested in network layer modelling/bringup for ndoes? | 21:08 |
| dprince | lifeless: yes. lets do it. | 21:09 |
| *** sgrasley has joined #tripleo | 21:12 | |
| *** jdob_ has quit IRC | 21:15 | |
| *** rbrady has quit IRC | 21:19 | |
| peoplemerge | I hit an issue the other night not being able to log in to my new undercloud as stack:stack. Without hacking the image, how can I set this beforehand? | 21:24 |
| peoplemerge | Reading devtest_variables, I'm not seeing it | 21:24 |
| *** markmc has quit IRC | 21:24 | |
| *** rpodolyaka1 has quit IRC | 21:25 | |
| *** ccrouch has joined #tripleo | 21:28 | |
| lifeless | peoplemerge: grep for stack-user | 21:29 |
| *** ramishra has joined #tripleo | 21:34 | |
| peoplemerge | lifeless: thx, trying DIB_COMMON_ELEMENTS | 21:39 |
| *** ramishra has quit IRC | 21:39 | |
| * bnemec checks stack deploy status | 21:56 | |
| * bnemec forgot to actually start the stack deploy :-( | 21:56 | |
| bnemec | Multitasking is a terrible, terrible thing | 21:56 |
| *** lblanchard has quit IRC | 22:13 | |
| *** boris-42_ has quit IRC | 22:13 | |
| *** NearlyFunctional has quit IRC | 22:18 | |
| *** e0ne has joined #tripleo | 22:18 | |
| *** e0ne has quit IRC | 22:22 | |
| *** noslzzp has quit IRC | 22:24 | |
| *** jtomasek has quit IRC | 22:26 | |
| *** jrist has joined #tripleo | 22:26 | |
| *** cd-undercloud has joined #tripleo | 22:28 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 22:28 |
| *** cd-undercloud has quit IRC | 22:28 | |
| *** dprince has quit IRC | 22:28 | |
| lifeless | okies | 22:29 |
| lifeless | lets see | 22:29 |
| *** noslzzp has joined #tripleo | 22:31 | |
| SpamapS | peoplemerge: btw, it is 'stackuser' not 'stack-user' | 22:32 |
| SpamapS | peoplemerge: and it looks like overcloud defaults to having stackuser, but undercloud doesn't. Doh. | 22:33 |
| lifeless | thats a regression I think | 22:34 |
| lifeless | would explain stuff :) | 22:34 |
| *** NearlyFunctional has joined #tripleo | 22:35 | |
| *** ramishra has joined #tripleo | 22:35 | |
| SpamapS | well ok so I misspoke | 22:36 |
| SpamapS | devtes_variables sets the default to stackuser | 22:36 |
| SpamapS | devtest_overcloud also sets it to stackuser if it is unset | 22:36 |
| SpamapS | If you just source variables and then run devtest_undercloud, you should have a stack user. | 22:37 |
| SpamapS | peoplemerge: ^^ | 22:37 |
| SpamapS | btw we are up to 109,000 tokens on cd-undercloud, but index length is quite a bit more sane: | 22:39 |
| SpamapS | Data_length: 2097233920 | 22:39 |
| SpamapS | Index_length: 41680896 | 22:39 |
| SpamapS | lifeless: good call on Dynamic. | 22:39 |
| SpamapS | We're also doing a good job pruning them | 22:39 |
| SpamapS | Data_free: 44040192 | 22:39 |
| *** ramishra has quit IRC | 22:40 | |
| lifeless | 2G wtf :( | 22:40 |
| SpamapS | 44M of free space in the data file that grows in 8M increments means we're probably at the steady state for the table | 22:40 |
| SpamapS | Rows: 109363 | 22:40 |
| SpamapS | Avg_row_length: 19176 | 22:40 |
| SpamapS | lifeless: PKI | 22:40 |
| SpamapS | Not sure why the catalog goes in the token | 22:41 |
| SpamapS | I get why we _return it to the user_ when they get tokens | 22:41 |
| SpamapS | but why we're storing it... no idea. | 22:41 |
| *** noslzzp has quit IRC | 22:42 | |
| SpamapS | was just noticing getting a token takes 0.33s | 22:43 |
| SpamapS | $ nova --os-cache --timing hypervisor-stats | 22:45 |
| SpamapS | Please set a password for your new keyring: | 22:45 |
| SpamapS | Please confirm the password: | 22:45 |
| SpamapS | Error: blank passwords aren't allowed. | 22:45 |
| SpamapS | 1: tell me that on the first try. 2: wtf, why not? | 22:45 |
| hewbrocca | SpamapS: because they don't contain an uppercase character, a number, or a special character of course! | 22:47 |
| *** noslzzp has joined #tripleo | 22:47 | |
| *** hashar has quit IRC | 22:47 | |
| lifeless | SpamapS: use a file keyring | 22:48 |
| *** ramishra has joined #tripleo | 22:51 | |
| lifeless | SpamapS: so how goes update? | 22:52 |
| *** matty_dubs is now known as matty_dubs|gone | 22:52 | |
| *** ramishra has quit IRC | 22:53 | |
| *** ramishra has joined #tripleo | 22:53 | |
| *** CaptTofu has quit IRC | 22:53 | |
| *** noslzzp has quit IRC | 22:54 | |
| lifeless | ok, expenses finally submitted. OMG the pain. | 22:57 |
| *** ramishra has quit IRC | 22:58 | |
| *** pblaho has quit IRC | 22:59 | |
| peoplemerge | I've gotta back out an undercloud again. I can't stop heat from deploying because a node is BUILD - deleting. How to I force that delete again? | 23:06 |
| greghaynes | IIRC nova delete the node | 23:08 |
| lifeless | restart nova-compute | 23:09 |
| peoplemerge | ahh | 23:09 |
| lifeless | tis bug | 23:09 |
| peoplemerge | greghaynes: I had tried that | 23:09 |
| *** noslzzp has joined #tripleo | 23:09 | |
| lifeless | someone should fix, except Ironic. | 23:09 |
| peoplemerge | lifeless: thx | 23:09 |
| SpamapS | lifeless: update, well I'm working out the design and trying to fit in where hot-software-config fits since it looks like it is coming along | 23:12 |
| lifeless | SpamapS: do you need a teddy bear? | 23:13 |
| SpamapS | lifeless: right now I need stevebaker.. but once I get the info from him I might. | 23:13 |
| *** ramishra has joined #tripleo | 23:15 | |
| *** noslzzp has quit IRC | 23:16 | |
| *** cadenzajon has quit IRC | 23:17 | |
| *** noslzzp has joined #tripleo | 23:25 | |
| lifeless | ok | 23:26 |
| lifeless | I'm going to poke at the ovs vlan stuff shortly | 23:27 |
| *** jdob has quit IRC | 23:40 | |
| *** victor_lowther_ has quit IRC | 23:40 | |
| *** julim has quit IRC | 23:40 | |
| *** victor_lowther_ has joined #tripleo | 23:41 | |
| *** noslzzp has quit IRC | 23:47 | |
| *** bauzas has quit IRC | 23:47 | |
| *** noslzzp has joined #tripleo | 23:51 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-heat-templates: Add username parameter for rabbit https://review.openstack.org/71430 | 23:52 |
| *** cd-undercloud has joined #tripleo | 23:53 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 23:53 |
| *** cd-undercloud has quit IRC | 23:53 | |
| *** noslzzp has quit IRC | 23:57 | |
| *** weshay has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!