| *** olaph has quit IRC | 00:02 | |
| lifeless | devananda: did anyone else eat with us in Stts | 00:02 |
|---|---|---|
| lifeless | s/Stts/Stats/ | 00:02 |
| lifeless | StevenK: same diff | 00:03 |
| devananda | lifeless: the dinner after the pizza place? | 00:03 |
| devananda | lifeless: no - just the three of us | 00:03 |
| lifeless | devananda: yeah, who were the three :) | 00:03 |
| lifeless | devananda: you, me, and ??? (tentatively think its clint, not sure :)) | 00:03 |
| lifeless | StevenK: ok, so I'm going to deploy an undercloud with a class B or something to ensure we have space | 00:04 |
| lifeless | StevenK: how do you change the hypervisor etc ? | 00:05 |
| devananda | yes, clint | 00:05 |
| StevenK | lifeless: The hypervisor only needs changing if it's fake. If you're doing it on a CI machine, then you don't have to. | 00:05 |
| lifeless | SpamapS: ^ | 00:05 |
| lifeless | StevenK: uhm | 00:05 |
| devananda | also - ugh, expense reports :( | 00:06 |
| lifeless | StevenK: so the idea is to test with 2000 processes polling | 00:06 |
| lifeless | StevenK: I rather suspect I don't have resources for 2000 emulated VMs today. | 00:06 |
| StevenK | Ah ha | 00:06 |
| SpamapS | Must always remember to take pictures of people when you pay | 00:06 |
| SpamapS | Works every time. | 00:06 |
| greghaynes | they want photos now? | 00:06 |
| SpamapS | hah yeah | 00:06 |
| SpamapS | and stool samples | 00:06 |
| StevenK | I scribble names on the back of receipts | 00:06 |
| greghaynes | still probably less pain than doing EEM | 00:07 |
| StevenK | It tends to work | 00:07 |
| lifeless | right, now its time to book travel for my next trip. sheese. | 00:07 |
| SpamapS | I'm calling a moritorium until the mid-cycle | 00:08 |
| SpamapS | and even that.. I intend to cut short | 00:08 |
| lifeless | 2.75 hours | 00:10 |
| lifeless | ugh | 00:10 |
| lifeless | I want my life back | 00:10 |
| SpamapS | lifeless: I believe the concur people have some mobile apps for doing expense reporting in real-time | 00:10 |
| SpamapS | lifeless: the drudgery is in sitting down documenting everything days after it mattered. ;) | 00:11 |
| lifeless | they do if you let the HP sysadmins have remote wipe on your device. | 00:11 |
| lifeless | SpamapS: which I will not. | 00:11 |
| lifeless | since the same acls to do that give you unlimited access... | 00:11 |
| *** matsuhashi has quit IRC | 00:13 | |
| StevenK | lifeless: For that case, you need to connect to the seed after it's been built and change nova.conf like http://paste.openstack.org/show/82502/ and then restart nova-* on the seed | 00:14 |
| SpamapS | lifeless: hah, if only your phone could run a VM. :) | 00:14 |
| StevenK | My main problem with expenses is waiting the n days for EEM to load in transactions from the corp amex | 00:15 |
| StevenK | I think it was just over an hour to deal with expenses from the summit | 00:16 |
| *** nati_uen_ has quit IRC | 00:16 | |
| *** nati_ueno has joined #tripleo | 00:17 | |
| *** TravT has quit IRC | 00:18 | |
| lifeless | right, lunch, then this heat scaling test, then networking for CI | 00:18 |
| lifeless | SpamapS: you're doing a powerpoint on mitigation, per earlier ? | 00:18 |
| SpamapS | lifeless: In principle I have agreed to this. In practice, I have spent half the day asleep or running errands to make up for lost travel time. | 00:21 |
| *** nati_ueno has quit IRC | 00:21 | |
| lifeless | SpamapS: sure, I guess I mean, can I trust that something will appear in my inbox tomorrow, wednesday @ latest? | 00:25 |
| SpamapS | lifeless: lets recap. We need to see answers on how to mitigate ERRORs, or scale problems, or both, in Heat? | 00:29 |
| SpamapS | lifeless: the ML thread has not really produced anything helpful. | 00:30 |
| *** nati_ueno has joined #tripleo | 00:32 | |
| lifeless | SpamapS: we need one-or-more approaches to mitigate the STACK_FAIL on (common) errors. | 00:33 |
| lifeless | SpamapS: what was the ML thread subject? | 00:34 |
| lifeless | SpamapS: short term approaches - things we can help folk deliver in the next 4-5 weeks. | 00:34 |
| *** chuckC has quit IRC | 00:34 | |
| SpamapS | lifeless: "Short term scaling strategies for large Heat stacks" | 00:35 |
| lifeless | ah I see it | 00:36 |
| SpamapS | lifeless: responses are a) wait for update-failure-recovery (Juno-2 expectation) and some stuff about handling transient client errors better. | 00:36 |
| SpamapS | err there's a b) there but I forgot to type it | 00:36 |
| lifeless | :) | 00:37 |
| *** ddieterly has joined #tripleo | 00:38 | |
| SpamapS | lifeless: I'm a bit worried about abandon/adopt given that it breaks metadata updates | 00:38 |
| lifeless | yeah | 00:38 |
| lifeless | so chunking seems to have agreedment | 00:39 |
| SpamapS | yeah chunking may even make sense long term. | 00:39 |
| lifeless | + help test zanes retry patches | 00:39 |
| lifeless | I'm not sure it does, it seems like chunking is a way of manually distributing work, but thats a different discussion | 00:39 |
| lifeless | as a mitigation strategy, no debate | 00:39 |
| SpamapS | lifeless: https://review.openstack.org/#/q/status:open+project:openstack/heat+branch:master+topic:typeless-function-plugins,n,z <-- the prelude to the retry patches (doesn't actually implement the retry, but sets the table for it) .... | 00:39 |
| lifeless | so I think what we need to communicate is | 00:40 |
| lifeless | - for a given mitigation | 00:40 |
| lifeless | - what needs to be done in broad terms | 00:41 |
| lifeless | - who is / will do / can do it | 00:41 |
| SpamapS | ooo the whole stream is +A'd now.. nice | 00:41 |
| lifeless | - what are we doing w.r.t. it | 00:41 |
| lifeless | and then capture the space | 00:42 |
| *** nati_ueno has quit IRC | 00:46 | |
| *** nati_ueno has joined #tripleo | 00:47 | |
| *** chuckC has joined #tripleo | 00:50 | |
| *** nati_ueno has quit IRC | 00:51 | |
| lifeless | stevebaker: IIUC your metadata memoisation stuff hasn't landed yet, right ? | 00:58 |
| *** xuhaiwei has quit IRC | 01:02 | |
| *** nati_ueno has joined #tripleo | 01:07 | |
| *** nosnos has joined #tripleo | 01:09 | |
| *** saurabhs has left #tripleo | 01:10 | |
| *** nati_ueno has quit IRC | 01:23 | |
| SpamapS | lifeless: see inbox | 01:24 |
| SpamapS | lifeless: mitigations in presentation form | 01:24 |
| lifeless | \o/ | 01:24 |
| *** nati_ueno has joined #tripleo | 01:24 | |
| *** nati_ueno has quit IRC | 01:24 | |
| *** nati_ueno has joined #tripleo | 01:25 | |
| *** nati_uen_ has joined #tripleo | 01:26 | |
| *** nati_uen_ has quit IRC | 01:26 | |
| *** nati_uen_ has joined #tripleo | 01:26 | |
| *** nati_uen_ has quit IRC | 01:26 | |
| *** nati_ueno has quit IRC | 01:29 | |
| *** morganfainberg is now known as morganfainberg_Z | 01:41 | |
| lifeless | ok, barring any ZOMGs, I'm locked in for june 22nd through 27th in SFO | 01:47 |
| lifeless | / sunnyvale | 01:47 |
| lifeless | right, next thing on the todo list | 01:47 |
| greghaynes | wow, so this tcpdump for tftp is interesting | 01:52 |
| greghaynes | we get packet A, ack A, get packet B, keep acking A... eventually we get B again and ack B | 01:53 |
| *** jang has joined #tripleo | 01:53 | |
| greghaynes | and both sides see this happen | 01:54 |
| StevenK | Sounds like ACKs are geting swallowed | 01:54 |
| StevenK | lifeless: Wild Palms again? | 01:55 |
| lifeless | StevenK: DIAF | 01:58 |
| StevenK | Haha | 01:59 |
| *** yamahata has joined #tripleo | 02:02 | |
| *** ramishra has joined #tripleo | 02:17 | |
| dprince | eukj~. | 02:19 |
| dprince | ~. | 02:19 |
| dprince | ~. | 02:19 |
| *** dprince has quit IRC | 02:19 | |
| SpamapS | slagle: https://review.openstack.org/#/c/94523/ <-- I responded there asking for mercy. ;) | 02:21 |
| greghaynes | oh, "that reviewer" | 02:21 |
| SpamapS | haha | 02:21 |
| *** shakamunyi has quit IRC | 02:31 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 03:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 03:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| lifeless | StevenK: SpamapS: ok, so once I have this cloud, how do I get all the polling processes up | 03:12 |
| lifeless | ? | 03:12 |
| *** edmund has joined #tripleo | 03:18 | |
| *** nosnos has quit IRC | 03:20 | |
| *** vinsh has quit IRC | 03:26 | |
| *** eghobo has quit IRC | 03:47 | |
| *** nosnos has joined #tripleo | 04:00 | |
| elynn | Hi all, Could you please to review this patch? https://review.openstack.org/#/c/90887/ | 04:03 |
| StevenK | lifeless: Sorry, I was buying parts of lunch, assembling it and noming. After the cloud was built, I get a little fuzzy, since it turned into me running what SpamapS told me to against the stack. :-/ | 04:11 |
| StevenK | elynn: Hasn't the kpartx call been pointed out as possibly unneeded for a few patchsets? | 04:12 |
| *** pcrews has quit IRC | 04:14 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Ignore pm_password if it's >255 characters https://review.openstack.org/95441 | 04:19 |
| lifeless | sigh, lennart, sigh | 04:20 |
| *** edmund has quit IRC | 04:20 | |
| StevenK | Remove netlink from udev because it might be useful to people that don't use systemd? Yeah. | 04:20 |
| lifeless | the bit I was sighing at was the *entirely* reasonable loose coupling / backwards compat aspect and then lennary going 'no, we expect lockstep upgrades'. | 04:21 |
| StevenK | Yeah | 04:21 |
| StevenK | Because piecemeal upgrades of kernel and then userspace aren't a thing at all | 04:22 |
| *** jang has quit IRC | 04:22 | |
| lifeless | I am -so- looking forward to 10 years from now when folk have relearnt history. | 04:22 |
| StevenK | lifeless: Do you want https://review.openstack.org/#/c/95657/ to go through CI again? I was hoping for a +2/+A since the os-cloud-config-docs job branch has landed to see what happens when it runs. | 04:23 |
| lifeless | StevenK: do you think it should ? | 04:24 |
| StevenK | lifeless: I'm happy enough for it to go through check before gate, which will happen as soon as you comment/vote. | 04:25 |
| jogo | lifeless: https://review.openstack.org/96308 appears to be working (at least partially) | 04:26 |
| greghaynes | Can I get some review love on https://review.openstack.org/#/c/97294/ - theres a patch stack depending on it being merged | 04:27 |
| greghaynes | lifeless: ^ also modifies a patch you landed | 04:28 |
| StevenK | lifeless: Re: Lennart, according to TrinaryLogic: what he's really saying is "I want systemd to be part of the kernel but linus won't let me." | 04:29 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash https://review.openstack.org/96308 | 04:30 |
| lifeless | StevenK: thats because Linus cares about users. | 04:30 |
| *** eghobo has joined #tripleo | 04:42 | |
| *** rameshg87 has joined #tripleo | 04:47 | |
| *** tzumainn has quit IRC | 04:54 | |
| *** pcrews has joined #tripleo | 04:56 | |
| elynn | StevenK, Hi, I haven't remove that part yet, could you tell me how does kpartx work? | 05:08 |
| elynn | StevenK, I don't really know if I remove kpartx will bring what problems. | 05:09 |
| StevenK | elynn: Then test it? Remove the kpartx call and see if anything breaks. | 05:10 |
| lifeless | jogo: cool | 05:10 |
| elynn | StevenK, so you mean that kpartx in ubuntu img is useless? | 05:11 |
| lifeless | hmm | 05:11 |
| lifeless | jogo: | 05:11 |
| lifeless | 2014-06-03 04:57:52.649 | tar: /home/jenkins/workspace/check-tripleo-ironic-seed-precise/logs/seed_var_log: Cannot open: No such file or directory | 05:11 |
| lifeless | 2014-06-03 04:57:52.649 | tar: Error is not recoverable: exiting now | 05:11 |
| lifeless | jogo: we might want to switch to simple rsync or something | 05:12 |
| StevenK | elynn: What I mean is that the Ubuntu images and Fedora images are constructed differently | 05:12 |
| elynn | StevenK, hmm, I will have a try! | 05:13 |
| elynn | thx StevenK | 05:13 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Clean up all outstanding ReST errors and warnings https://review.openstack.org/97129 | 05:18 |
| *** rakesh_hs has joined #tripleo | 05:25 | |
| *** rdopiera has joined #tripleo | 05:39 | |
| lifeless | adam_g: I imagine you're about done for the day :)... where's it at? | 05:40 |
| *** jcoufal has joined #tripleo | 05:41 | |
| greghaynes | lifeless: re the tftp? | 05:41 |
| lifeless | yes | 05:41 |
| greghaynes | tried completely separate jump host + seed, still seeing issue | 05:41 |
| lifeless | ok | 05:41 |
| lifeless | SlickNik: re https://wiki.openstack.org/wiki/Trove/Replication-And-Clustering-With-Nodes-5#Example:_MySQL | 05:42 |
| lifeless | SlickNik: it seems unsuitable for galera | 05:42 |
| lifeless | SlickNik: but perhaps I'm missing something | 05:42 |
| greghaynes | also left a little note on my trying to confirm its networkey here https://bugs.launchpad.net/tripleo/+bug/1325762 | 05:43 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 05:43 |
| lifeless | greghaynes: you say https://review.openstack.org/#/c/97294/ conflicts with a patch of mine ? | 05:43 |
| greghaynes | not conflicts, it modifies what you did with bootstack | 05:44 |
| greghaynes | that one already landed though :/ | 05:44 |
| lifeless | ok | 05:44 |
| lifeless | fine by nme | 05:44 |
| lifeless | needs another core vote | 05:44 |
| lifeless | SpamapS: if you have jetlag... https://review.openstack.org/#/c/97294/ | 05:45 |
| *** mrunge has joined #tripleo | 05:45 | |
| *** mrunge has quit IRC | 05:45 | |
| *** mrunge has joined #tripleo | 05:45 | |
| StevenK | Oh blah | 05:45 |
| StevenK | I had the comment page for that review open, didn't hit submit | 05:45 |
| *** gcha has joined #tripleo | 05:54 | |
| lifeless | greghaynes: so https://review.openstack.org/#/c/86435/ still failed :( | 05:54 |
| greghaynes | with https://review.openstack.org/#/c/97294/ ? | 05:55 |
| *** lazy_prince has joined #tripleo | 05:55 | |
| lifeless | greghaynes: shouldn't matter | 05:56 |
| lifeless | greghaynes: since two absent keys have the same value :) | 05:56 |
| greghaynes | there is a check if they dont exist to not return you are master | 05:57 |
| lifeless | ah | 05:57 |
| greghaynes | otherwise bad things happen (TM) when multiple people try to become master before metadata server is up | 05:57 |
| lifeless | because of os-collect-configs fail-open behaviour | 05:57 |
| lifeless | we need to fix that | 05:57 |
| lifeless | 2014-05-30 23:53:19.419 | tar: /var/log/journal/7245d72979598396f6413f4ac8029f7f/system.journal: file changed as we read it | 06:03 |
| lifeless | ugh | 06:03 |
| lifeless | tar is just the wrong tool I think | 06:03 |
| greghaynes | yep | 06:03 |
| clarkb | uhm journalctl? | 06:04 |
| clarkb | pretty sure there are examples over in d-g for fedora | 06:04 |
| lifeless | clarkb: it affects ubuntu too | 06:04 |
| clarkb | ubuntu is using systemd? | 06:05 |
| lifeless | clarkb: any system where logs are being written to and tar isn't fast enough to avoid the race | 06:05 |
| greghaynes | we just get other logs being written to | 06:05 |
| clarkb | lifeless: oh sure | 06:05 |
| clarkb | d-g deals withthat too iirc | 06:05 |
| clarkb | it does a sleep, sync, then cp | 06:05 |
| clarkb | after stopping services | 06:06 |
| lifeless | clarkb: thats one way of solving it | 06:06 |
| lifeless | clarkb: how do they avoid things writing to syslog ? | 06:06 |
| lifeless | clarkb: or does d-g not grab syslog ? | 06:06 |
| clarkb | lifeless: it does grab it. iirc it does a dance before and after etsing where it stops syslog and moves things around | 06:07 |
| lifeless | clarkb: heh, so I think we can do better. | 06:07 |
| clarkb | sure | 06:07 |
| clarkb | but this works fine | 06:07 |
| lifeless | clarkb: e.g. rsync. Then tar / compress. | 06:07 |
| clarkb | there is no reason for that though | 06:07 |
| clarkb | you don't gain anything | 06:07 |
| lifeless | clarkb: we're dealing with multiple nodes. SSH handshakes are slow. Resources are expensive. | 06:07 |
| lifeless | clarkb: doing a one-touch no-waiting approach has a lot of gain. | 06:08 |
| lifeless | clarkb: but you've clearly dealt with it before. Why do you say we won't gain anything ? | 06:08 |
| clarkb | lifeless: because stopping rsyslog and doing a copy isn't expensive | 06:09 |
| clarkb | and gets you the same result | 06:09 |
| lifeless | clarkb: we'd need to stop all the openstack services, on multiple machines | 06:10 |
| lifeless | clarkb: rabbit and mysql etc; and to do that without spurious errors we'd need to do it in toplogical service dependency order. | 06:10 |
| clarkb | you need to regardless | 06:10 |
| lifeless | clarkb: we do? Why? | 06:10 |
| clarkb | because you are recycling test envs | 06:10 |
| lifeless | clarkb: we're deploying brand new machines every time | 06:11 |
| lifeless | clarkb: reusing the 'disk' sure | 06:11 |
| clarkb | right but killing those nodes is going to stop service gracefully | 06:11 |
| clarkb | its the same cpu time | 06:11 |
| lifeless | no its not | 06:11 |
| SpamapS | lifeless: what fail open? | 06:11 |
| lifeless | there's some disconnect here. | 06:11 |
| SpamapS | lifeless: "os-collect-config's fail open behavior" ? | 06:12 |
| SpamapS | lifeless: https://bugs.launchpad.net/os-collect-config none of those seem like a fail-open. ? | 06:13 |
| lifeless | SpamapS: we run o-r-c when collectors have failed | 06:13 |
| SpamapS | lifeless: ah, yes, would you report that? Pretty easy to correct I think. | 06:13 |
| lifeless | SpamapS: we also, relatedly, run all of o-r-c when various deployments aren't present, but thats new feature work | 06:13 |
| lifeless | SpamapS: I'm armpit deep in context & baby time right now; could you capture it ? | 06:14 |
| lifeless | SpamapS: otherwise I'll get to it when I get a tuit | 06:14 |
| SpamapS | Yeah I"ll report | 06:15 |
| SpamapS | lifeless: the running w/o deployments thing has troubled me as well | 06:15 |
| SpamapS | lifeless: it makes me wonder about using a dependency system instead of ordering | 06:15 |
| lifeless | SpamapS: a design doc for that would help | 06:15 |
| SpamapS | because really, what we're saying is, if there's no mysql configuration, there shouldn't be a mysql post-configuration or migration. | 06:16 |
| SpamapS | but thats the easy case. The hard one is when there are o-r-c scripts that assume there is a working mysql database. | 06:17 |
| SpamapS | since it might be local, or it might be remote | 06:17 |
| greghaynes | I would hope most things that depend on a working mysql database dont depend on a local mysql database? | 06:19 |
| *** kevinbenton has quit IRC | 06:19 | |
| SpamapS | https://bugs.launchpad.net/os-collect-config/+bug/1325818 | 06:21 |
| uvirtbot | Launchpad bug 1325818 in os-collect-config "os-collect-config should fail when any collectors fail" [High,Triaged] | 06:21 |
| SpamapS | greghaynes: hopefully, but as is often the case, things start local and move remote only when we need to scale out massively | 06:21 |
| GheRivero | morning all | 06:21 |
| lifeless | clarkb: so at the moment we just STNITH all the nodes before the next test starts. | 06:22 |
| clarkb | you essentially kill -9 it? | 06:25 |
| *** proffalken has quit IRC | 06:25 | |
| lifeless | we pull the power plug | 06:25 |
| clarkb | I guess if you are that mean then sure. I figured you would be using nova to do it for you though which is a bit more graful aiui | 06:26 |
| lifeless | we can't | 06:26 |
| lifeless | machines with no instance deployed to them are not power managed by nova-bm (and we still support nova-bm) | 06:26 |
| lifeless | Ironic will power off (STNITH) machines which are powered on when they are registered with Ironic | 06:27 |
| lifeless | but this happens in the *next test* | 06:27 |
| lifeless | we don't shut anything down during the current test | 06:28 |
| clarkb | ok thats different than how we do it | 06:28 |
| clarkb | and I would expect that with multi node devstack we would continue to shut things down | 06:28 |
| clarkb | but I will leave that to mattoliverau to sort out | 06:29 |
| lifeless | we will be doing reboots and stuff as we get upgrade testing in place | 06:29 |
| *** proffalken has joined #tripleo | 06:29 | |
| lifeless | clarkb: should we do it differently ? | 06:33 |
| lifeless | clarkb: what benefit (given we're testing that we deploy ;)) does shutting everything down have? | 06:33 |
| clarkb | lifeless: its simple | 06:33 |
| clarkb | you don't have to consider races at all | 06:34 |
| lifeless | clarkb: It doesn't seem simple to me | 06:34 |
| lifeless | clarkb: to shut down gracefully we have to hit all nodes in a repeating loop to take down services from the top to the bottom | 06:35 |
| clarkb | lifeless: so that I don't understand | 06:35 |
| clarkb | why can't you just stop the services? | 06:35 |
| lifeless | clarkb: if we shut down galera, keystone etc will all start throwing errors. | 06:35 |
| clarkb | at the end of testing you don't care about the cloud running during shutdown | 06:35 |
| lifeless | clarkb: but if we don't shut down galera, we'll have the race condition with logs to deal with. | 06:36 |
| lifeless | clarkb: errors in logs are something that devstack tests for (and I think its a good initiative) so spurious errors are bad | 06:36 |
| greghaynes | can we pull the power plug, mount the fs, read the logs? | 06:37 |
| SlickNik | lifeless: The mysql example at https://wiki.openstack.org/wiki/Trove/Replication-And-Clustering-With-Nodes-5#Example:_MySQL specifically illustrates how mysql async replication will work with the proposed clustering design. | 06:37 |
| lifeless | clarkb: so we'd need to shutdown all high end services - nova-* except for conductor, neutron-agent etc etc | 06:37 |
| lifeless | clarkb: then do another pass for the next layer down - nova-conductor, neutron-server etc | 06:37 |
| lifeless | clarkb: then keystone | 06:37 |
| lifeless | clarkb: then galera and rabbit | 06:37 |
| SlickNik | lifeless: Provisioning a galera cluster would be more akin to the Cassandra example at https://wiki.openstack.org/wiki/Trove/Replication-And-Clustering-With-Nodes-5#Example:_Cassandra | 06:37 |
| lifeless | SlickNik: it would be easier for a noob like me to approach if the example called out how its going to be used | 06:38 |
| lifeless | SlickNik: (and why- I mean, does anyone run non-galera clustered mysql these days?!) | 06:38 |
| *** nati_ueno has joined #tripleo | 06:38 | |
| SlickNik | lifeless: Agreed, it seems worthwhile to call out provisioning galera as its own scenario. Will get that into the specs / docs. | 06:39 |
| *** jprovazn has joined #tripleo | 06:40 | |
| SpamapS | lifeless: sent you updated mitigation slides | 06:42 |
| openstackgerrit | Lin Tan proposed a change to openstack/diskimage-builder: Correct the wrong rename in rhel element https://review.openstack.org/96114 | 06:42 |
| lifeless | SpamapS: cool, looking | 06:42 |
| *** e0ne has joined #tripleo | 06:44 | |
| *** ddieterly has quit IRC | 06:44 | |
| lifeless | SpamapS: why won't "Will not work with future plans for online updates (Signalling, rolling updates, etc" ? | 06:44 |
| *** ddieterly has joined #tripleo | 06:44 | |
| lifeless | SpamapS: (its the only bit that I didn't understand) | 06:45 |
| SpamapS | lifeless: we'll have to orchestrate per-stack updates for computes in the chunking example I gave. | 06:47 |
| SpamapS | lifeless: as in, we have to update the compute stack 0, and then 1, and then 2 .. there's no cohesive policy. | 06:47 |
| SpamapS | lifeless: signalling is the quorum-stopping that we want to do.. we'll have to work really hard and do secondary updates to make that work right in separate stacks. | 06:48 |
| lifeless | SpamapS: it seems to me it will all work, as long as: | 06:48 |
| SpamapS | Work, yes. Reusable? No. | 06:48 |
| lifeless | - things we want to be global we export *and* import from each stack (e.g. hostname scatter-gather) | 06:48 |
| SpamapS | we're effectively writing our own mini-convergence engine | 06:49 |
| lifeless | - our meta layer does updates in a well defined order (e.g. we have a DAG of stacks) | 06:49 |
| lifeless | yes | 06:49 |
| lifeless | it sucks | 06:49 |
| lifeless | but I'm trying to understand why these features /won't work/ | 06:49 |
| SpamapS | I support it btw. I just want to make sure it is clear that we are forking to get around upstream's complexity. | 06:49 |
| lifeless | IIUC its 'there is code needed to make them work' | 06:49 |
| SpamapS | I'm wary to get too far down the road... | 06:50 |
| * SpamapS eyes merge.py while fondling the hilt of his broadsword | 06:50 | |
| lifeless | I'm going to be presenting this 4? 5? layer of mgmt up | 06:50 |
| lifeless | if we say 'won't work' | 06:50 |
| lifeless | when its nuanced | 06:51 |
| *** ramishra has quit IRC | 06:51 | |
| lifeless | I am worried that will lead to mitigate-by-using-chef or something | 06:51 |
| SpamapS | "Will be superseded by ..." may be o-k then. | 06:51 |
| SpamapS | chef wouldn't solve anything | 06:51 |
| SpamapS | ansible would | 06:51 |
| SpamapS | or salt-cloud | 06:51 |
| lifeless | how about 'rolling update and signalling will not natively cross stacks - development required to enable those features | 06:52 |
| lifeless | or something ? | 06:52 |
| SpamapS | Yeah thats fine. | 06:52 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/os-cloud-config: Allow setup services endpoints https://review.openstack.org/92383 | 06:57 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-specs: Promote HEAT_ENV https://review.openstack.org/94910 | 06:57 |
| lxsli | Not urgent, but input would be nice on: https://review.openstack.org/96428 | 06:58 |
| *** ramishra has joined #tripleo | 06:59 | |
| StevenK | lxsli: That's nice and impossible to review, since Gerrit shows the svg as XML | 06:59 |
| StevenK | Which is ... handy of it | 06:59 |
| SpamapS | StevenK: git review -d | 06:59 |
| StevenK | s/review,/review using the web UI,/ | 06:59 |
| StevenK | Pedant. :-P | 06:59 |
| lxsli | StevenK: if you check my last comment, I give a handy link :) | 06:59 |
| SpamapS | even if it showed it to you | 06:59 |
| SpamapS | :-P | 06:59 |
| greghaynes | or http://docs-draft.openstack.org/28/96428/2/check/gate-tripleo-incubator-docs/05d5a68/doc/build/html/userguide.html | 07:00 |
| greghaynes | ah, looks like thats whats in the comments | 07:00 |
| lifeless | lxsli: nice stuff | 07:00 |
| SpamapS | totally | 07:00 |
| SpamapS | if I wasn't passing out I'd review immediately | 07:00 |
| SpamapS | but will do it first thing tomorrow | 07:00 |
| *** pblaho has joined #tripleo | 07:00 | |
| lxsli | That'd be grand, cheers | 07:01 |
| * SpamapS makes good on promise.. and passes out | 07:01 | |
| SpamapS | immediately | 07:01 |
| lifeless | adam_g: greghaynes: https://review.openstack.org/#/c/93844/ | 07:02 |
| elynn | Hi StevenK, are you there? | 07:03 |
| *** vinsh has joined #tripleo | 07:06 | |
| *** e0ne has quit IRC | 07:11 | |
| *** e0ne has joined #tripleo | 07:14 | |
| greghaynes | lifeless: I misunderstood about situation re tftp from before - adam_g says he was blocked on the setuptools issue so we havent actually tried with a new jumphost | 07:15 |
| GheRivero | greghaynes: I'm testing n43 | 07:16 |
| adam_g | hey. | 07:16 |
| jprovazn | where lives the code which tars logfiles into e.g. 192.0.2.3_logs.tar.xz when running gate tests? I'd like to include there /mnt/state/var/log/mysql path too | 07:16 |
| GheRivero | ow with the new seed | 07:16 |
| greghaynes | GheRivero: ooo | 07:16 |
| adam_g | so i brought up a second host to act as a new seed / jump host, but DIB was failing to build the seed blocked on the setuptools fix being synced to local pypi mirror | 07:16 |
| jprovazn | ah | 07:17 |
| GheRivero | the seed booted so I guess the setuptools issue is fixed now | 07:17 |
| jprovazn | toci_devtest.sh | 07:17 |
| adam_g | GheRivero, nice | 07:17 |
| *** yfaykya has quit IRC | 07:17 | |
| *** yfaykya has joined #tripleo | 07:18 | |
| *** eghobo has quit IRC | 07:18 | |
| *** andreaf has joined #tripleo | 07:22 | |
| *** yfaykya has quit IRC | 07:22 | |
| *** jtomasek has joined #tripleo | 07:23 | |
| *** yfaykya has joined #tripleo | 07:24 | |
| *** andreaf has quit IRC | 07:26 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack-infra/tripleo-ci: Add mysql logs https://review.openstack.org/97421 | 07:27 |
| *** yfaykya has quit IRC | 07:28 | |
| *** yfaykya has joined #tripleo | 07:30 | |
| *** yfaykya has quit IRC | 07:35 | |
| *** yfaykya has joined #tripleo | 07:36 | |
| *** martyntaylor has joined #tripleo | 07:41 | |
| *** yfaykya has quit IRC | 07:41 | |
| *** ramishra has quit IRC | 07:42 | |
| *** yfaykya has joined #tripleo | 07:42 | |
| *** yfaykya has quit IRC | 07:47 | |
| *** yfaykya has joined #tripleo | 07:48 | |
| *** yfaykya has quit IRC | 07:53 | |
| *** yfaykya has joined #tripleo | 07:55 | |
| *** yfaykya has quit IRC | 07:59 | |
| *** e0ne has quit IRC | 08:00 | |
| *** yfaykya has joined #tripleo | 08:01 | |
| *** ramishra has joined #tripleo | 08:01 | |
| *** e0ne has joined #tripleo | 08:03 | |
| *** jistr has joined #tripleo | 08:03 | |
| *** yfaykya has quit IRC | 08:05 | |
| *** yfaykya has joined #tripleo | 08:07 | |
| *** yfaykya has quit IRC | 08:11 | |
| *** dtantsur|afk is now known as dtantsur | 08:13 | |
| *** yfaykya has joined #tripleo | 08:13 | |
| lparth | hello, any chance someone could please approve this patch? Thanks. https://review.openstack.org/#/c/92400/ | 08:17 |
| *** rlandy has joined #tripleo | 08:17 | |
| *** yfaykya has quit IRC | 08:18 | |
| *** yfaykya has joined #tripleo | 08:19 | |
| *** derekh_ has joined #tripleo | 08:20 | |
| *** nati_ueno has quit IRC | 08:21 | |
| shadower | lparth: the check-tripleo-undercloud-precise test failed (so can't approve now), but it doesn't seem to be related to the patch | 08:22 |
| shadower | lparth: try adding a comment that says "recheck no bug". That will re-run the tests | 08:22 |
| jprovazn | lparth: shadower: I reran tests on it already | 08:22 |
| lparth | thanks shadower, jprovazn | 08:23 |
| openstackgerrit | Alexis Lee proposed a change to openstack-infra/tripleo-ci: Fix long lines https://review.openstack.org/97429 | 08:24 |
| shadower | cool | 08:24 |
| *** yfaykya has quit IRC | 08:24 | |
| *** yfaykya has joined #tripleo | 08:25 | |
| *** lucasagomes has joined #tripleo | 08:26 | |
| *** andreaf has joined #tripleo | 08:28 | |
| jistr | if some core can spare a while to do an easy review: https://review.openstack.org/#/c/96759/ | 08:28 |
| jistr | until we merge it, all other patches to Tuskar are blocked (CI is broken on Tuskar) | 08:29 |
| *** yfaykya has quit IRC | 08:30 | |
| *** yfaykya has joined #tripleo | 08:31 | |
| jprovazn | jistr: sure | 08:32 |
| jistr | jprovazn: thanks! | 08:32 |
| *** yfaykya has quit IRC | 08:36 | |
| *** yfaykya has joined #tripleo | 08:37 | |
| *** IvanBerezovskiy has joined #tripleo | 08:39 | |
| *** yfaykya has quit IRC | 08:42 | |
| *** yfaykya has joined #tripleo | 08:44 | |
| *** e0ne has quit IRC | 08:44 | |
| *** e0ne has joined #tripleo | 08:45 | |
| *** ramishra has quit IRC | 08:47 | |
| *** jp_at_hp has joined #tripleo | 08:48 | |
| *** yfaykya has quit IRC | 08:48 | |
| *** yfaykya has joined #tripleo | 08:50 | |
| *** yfaykya has quit IRC | 08:54 | |
| *** pblaho has quit IRC | 08:55 | |
| *** yfaykya has joined #tripleo | 08:56 | |
| *** akrivoka has joined #tripleo | 08:59 | |
| *** pblaho has joined #tripleo | 08:59 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 09:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 09:00 |
| *** yfaykya has quit IRC | 09:00 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Move bootstrap-host out from bootstack https://review.openstack.org/97294 | 09:01 |
| *** petertoft has joined #tripleo | 09:01 | |
| *** yfaykya has joined #tripleo | 09:02 | |
| *** yfaykya has quit IRC | 09:07 | |
| *** yfaykya has joined #tripleo | 09:08 | |
| *** yfaykya has quit IRC | 09:13 | |
| *** pelix has joined #tripleo | 09:14 | |
| *** yfaykya has joined #tripleo | 09:14 | |
| *** yfaykya has quit IRC | 09:19 | |
| *** yfaykya has joined #tripleo | 09:20 | |
| *** yfaykya has quit IRC | 09:24 | |
| *** ramishra has joined #tripleo | 09:25 | |
| *** bogdando has joined #tripleo | 09:26 | |
| *** yfaykya has joined #tripleo | 09:26 | |
| lifeless | derekh_: morning | 09:27 |
| derekh_ | lifeless: good morning, hows things today | 09:27 |
| openstackgerrit | A change was merged to openstack/os-cloud-config: Switch to ssh_key_contents for ironic https://review.openstack.org/95657 | 09:29 |
| derekh_ | lifeless: am currently testing a CI simulation script on R2 , nothing very complicated, runs overcloud job on a cloud X number of times, Y jobs running simultaneously , so we can then take a look and see how many passed/failed | 09:31 |
| *** yfaykya has quit IRC | 09:31 | |
| *** yfaykya has joined #tripleo | 09:33 | |
| lifeless | derekh_: current blocker for deploying a new undercloud is the vlan support | 09:36 |
| lifeless | derekh_: cool | 09:36 |
| lifeless | derekh_: I'm thinking perhaps I'll deploy the undercloud with no external endpoint initially; it would force jump host use to get at it, but get us online faster. | 09:37 |
| *** yfaykya has quit IRC | 09:38 | |
| derekh_ | lifeless: sounds ok to me (I usually ssh to undercloud to use the endpoints anyways) | 09:38 |
| lifeless | ok, I'll update the patches and stuff tomorrow | 09:38 |
| lifeless | derekh_: btw | 09:39 |
| *** yfaykya has joined #tripleo | 09:39 | |
| lifeless | derekh_: the tar log copier is about 5-10 failures a day | 09:39 |
| lifeless | derekh_: I filed a bug tagged ci for it, and submitted a recheck signature | 09:39 |
| lifeless | and with that, gnight | 09:39 |
| derekh_ | lifeless: ok, found it https://bugs.launchpad.net/tripleo/+bug/1325815 , I have seen the warning about the file changing while it was being read but never had a problem opening the tarball | 09:41 |
| uvirtbot | Launchpad bug 1325815 in tripleo "tar:... file changed as we read it" [High,Triaged] | 09:41 |
| derekh_ | lifeless: will take a look | 09:41 |
| *** yfaykya has quit IRC | 09:43 | |
| *** yfaykya has joined #tripleo | 09:45 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 09:45 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 09:46 |
| *** ifarkas has joined #tripleo | 09:46 | |
| *** giulivo has joined #tripleo | 09:49 | |
| *** yfaykya has quit IRC | 09:50 | |
| *** yfaykya has joined #tripleo | 09:51 | |
| *** yfaykya has quit IRC | 09:55 | |
| *** yfaykya has joined #tripleo | 09:57 | |
| *** IvanBerezovskiy has quit IRC | 10:00 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 10:01 |
| *** ramishra has quit IRC | 10:01 | |
| *** yfaykya has quit IRC | 10:02 | |
| *** yfaykya has joined #tripleo | 10:03 | |
| *** yfaykya has quit IRC | 10:08 | |
| *** yfaykya has joined #tripleo | 10:09 | |
| *** e0ne_ has joined #tripleo | 10:10 | |
| *** e0ne has quit IRC | 10:14 | |
| *** yfaykya has quit IRC | 10:14 | |
| *** yfaykya has joined #tripleo | 10:15 | |
| *** yfaykya has quit IRC | 10:20 | |
| *** yfaykya has joined #tripleo | 10:21 | |
| *** yfaykya has quit IRC | 10:26 | |
| *** ramishra has joined #tripleo | 10:27 | |
| *** yfaykya has joined #tripleo | 10:28 | |
| *** yfaykya has quit IRC | 10:32 | |
| *** yfaykya has joined #tripleo | 10:34 | |
| *** IvanBerezovskiy has joined #tripleo | 10:36 | |
| *** yfaykya has quit IRC | 10:38 | |
| *** IvanBerezovskiy has quit IRC | 10:39 | |
| *** yfaykya has joined #tripleo | 10:40 | |
| *** yfaykya has quit IRC | 10:45 | |
| *** yfaykya has joined #tripleo | 10:46 | |
| *** yfaykya has quit IRC | 10:50 | |
| *** yamahata has quit IRC | 10:52 | |
| *** yfaykya has joined #tripleo | 10:52 | |
| *** e0ne_ has quit IRC | 10:54 | |
| *** yfaykya has quit IRC | 10:57 | |
| *** yfaykya has joined #tripleo | 10:58 | |
| *** yfaykya has quit IRC | 11:03 | |
| *** yfaykya has joined #tripleo | 11:04 | |
| *** yfaykya has quit IRC | 11:09 | |
| *** rameshg87 has left #tripleo | 11:10 | |
| *** yfaykya has joined #tripleo | 11:10 | |
| *** rcarrillocruz has quit IRC | 11:13 | |
| *** rcarrillocruz has joined #tripleo | 11:14 | |
| *** yfaykya has quit IRC | 11:15 | |
| *** yfaykya has joined #tripleo | 11:17 | |
| *** morazi has joined #tripleo | 11:18 | |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: testenv-worker: use signals to enable/disable jobs https://review.openstack.org/87971 | 11:18 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Add bin/testenv-ctl stop_jobs/start_jobs script https://review.openstack.org/87972 | 11:21 |
| *** yfaykya has quit IRC | 11:21 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix jshint warnings https://review.openstack.org/97467 | 11:21 |
| *** lucasagomes is now known as lucas-hungry | 11:22 | |
| *** tzumainn has joined #tripleo | 11:22 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Sequence deploys - compute after control plane. https://review.openstack.org/93806 | 11:22 |
| *** yfaykya has joined #tripleo | 11:23 | |
| *** IvanBerezovskiy has joined #tripleo | 11:24 | |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Download the fedora base image from a good URL https://review.openstack.org/93058 | 11:25 |
| *** yfaykya has quit IRC | 11:27 | |
| *** jp_at_hp1 has joined #tripleo | 11:27 | |
| *** jp_at_hp has quit IRC | 11:28 | |
| *** dtantsur is now known as dtantsur|lunch | 11:28 | |
| *** yfaykya has joined #tripleo | 11:29 | |
| *** morazi has quit IRC | 11:29 | |
| *** yfaykya has quit IRC | 11:34 | |
| *** matsuhashi has joined #tripleo | 11:34 | |
| *** yfaykya has joined #tripleo | 11:35 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Allow install mariadb from RDO repository https://review.openstack.org/87295 | 11:37 |
| *** e0ne has joined #tripleo | 11:38 | |
| *** yamahata has joined #tripleo | 11:38 | |
| *** yfaykya has quit IRC | 11:39 | |
| *** IvanBerezovskiy has quit IRC | 11:40 | |
| *** yfaykya has joined #tripleo | 11:41 | |
| *** e0ne_ has joined #tripleo | 11:41 | |
| *** e0ne has quit IRC | 11:42 | |
| *** morazi has joined #tripleo | 11:42 | |
| *** yfaykya has quit IRC | 11:46 | |
| *** yfaykya has joined #tripleo | 11:47 | |
| *** lazy_prince has quit IRC | 11:47 | |
| *** weshay has joined #tripleo | 11:49 | |
| *** ramishra has quit IRC | 11:50 | |
| *** ramishra has joined #tripleo | 11:50 | |
| *** ramishra has quit IRC | 11:50 | |
| *** ramishra has joined #tripleo | 11:51 | |
| *** yfaykya has quit IRC | 11:51 | |
| *** dtantsur|lunch is now known as dtantsur | 11:53 | |
| *** yfaykya has joined #tripleo | 11:53 | |
| *** yfaykya has quit IRC | 11:58 | |
| *** yfaykya has joined #tripleo | 11:59 | |
| *** rlandy has quit IRC | 12:00 | |
| *** rlandy has joined #tripleo | 12:00 | |
| *** yfaykya has quit IRC | 12:04 | |
| *** yfaykya has joined #tripleo | 12:06 | |
| *** yfaykya has quit IRC | 12:10 | |
| *** IvanBerezovskiy has joined #tripleo | 12:11 | |
| *** yfaykya has joined #tripleo | 12:12 | |
| *** yfaykya has quit IRC | 12:16 | |
| *** yfaykya has joined #tripleo | 12:18 | |
| *** jdob has joined #tripleo | 12:20 | |
| *** yfaykya has quit IRC | 12:22 | |
| *** mrunge has quit IRC | 12:22 | |
| *** yfaykya has joined #tripleo | 12:24 | |
| *** ramishra has quit IRC | 12:26 | |
| *** ramishra has joined #tripleo | 12:26 | |
| *** yfaykya has quit IRC | 12:29 | |
| *** yfaykya has joined #tripleo | 12:30 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Move to software-config for the undercloud. https://review.openstack.org/93319 | 12:33 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Add passthrough configuration glue. https://review.openstack.org/91810 | 12:34 |
| *** yfaykya has quit IRC | 12:35 | |
| *** ramishra has quit IRC | 12:36 | |
| *** yfaykya has joined #tripleo | 12:36 | |
| *** weshay has quit IRC | 12:36 | |
| *** weshay has joined #tripleo | 12:37 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix jshint warnings https://review.openstack.org/97467 | 12:40 |
| *** yfaykya has quit IRC | 12:41 | |
| *** yfaykya has joined #tripleo | 12:42 | |
| *** jang has joined #tripleo | 12:43 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Make disk bus type configurable https://review.openstack.org/92400 | 12:43 |
| *** ddieterly has quit IRC | 12:44 | |
| *** julim has joined #tripleo | 12:46 | |
| *** yfaykya has quit IRC | 12:47 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix jshint warnings https://review.openstack.org/97467 | 12:47 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Move to software-config for the undercloud. https://review.openstack.org/93319 | 12:48 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-heat-templates: Add initial support for galera clustering https://review.openstack.org/83883 | 12:48 |
| *** yfaykya has joined #tripleo | 12:48 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/93253 | 12:50 |
| *** jcoufal has quit IRC | 12:52 | |
| *** jcoufal has joined #tripleo | 12:53 | |
| *** lucas-hungry is now known as lucasagomes | 12:53 | |
| *** yfaykya has quit IRC | 12:53 | |
| *** ramishra has joined #tripleo | 12:54 | |
| *** yfaykya has joined #tripleo | 12:55 | |
| *** dprince has joined #tripleo | 12:56 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Add passthrough configuration glue. https://review.openstack.org/91810 | 12:56 |
| *** jistr has quit IRC | 12:57 | |
| *** yfaykya has quit IRC | 12:59 | |
| *** jistr has joined #tripleo | 13:01 | |
| *** yfaykya has joined #tripleo | 13:01 | |
| *** pblaho has quit IRC | 13:02 | |
| *** yfaykya has quit IRC | 13:05 | |
| *** openstackgerrit has quit IRC | 13:06 | |
| *** openstackgerrit has joined #tripleo | 13:07 | |
| *** yfaykya has joined #tripleo | 13:07 | |
| *** ddieterly has joined #tripleo | 13:11 | |
| *** yfaykya has quit IRC | 13:11 | |
| *** matsuhashi has quit IRC | 13:12 | |
| *** nosnos has quit IRC | 13:12 | |
| *** yfaykya has joined #tripleo | 13:13 | |
| *** yfaykya has quit IRC | 13:17 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Add svc-map element. https://review.openstack.org/96770 | 13:18 |
| *** yfaykya has joined #tripleo | 13:19 | |
| openstackgerrit | Robert Parker proposed a change to openstack/tripleo-heat-templates: Setup SSL for Ceilometer https://review.openstack.org/96257 | 13:22 |
| *** matty_dubs|gone is now known as matty_dubs | 13:23 | |
| *** yfaykya has quit IRC | 13:23 | |
| *** yfaykya has joined #tripleo | 13:25 | |
| *** edmund has joined #tripleo | 13:28 | |
| *** nati_ueno has joined #tripleo | 13:29 | |
| *** yfaykya has quit IRC | 13:30 | |
| *** ramishra has quit IRC | 13:31 | |
| *** yfaykya has joined #tripleo | 13:31 | |
| *** jprovazn has quit IRC | 13:35 | |
| tzumainn | jistr, quick question about your multiple overcloud spec, when you have a moment? | 13:35 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Configurable /mnt/state https://review.openstack.org/94876 | 13:35 |
| *** yfaykya has quit IRC | 13:36 | |
| *** ramishra has joined #tripleo | 13:36 | |
| *** jprovazn has joined #tripleo | 13:37 | |
| *** yfaykya has joined #tripleo | 13:37 | |
| jistr | tzumainn: sec, i'll just dock my laptop | 13:41 |
| *** jistr has quit IRC | 13:41 | |
| *** yfaykya has quit IRC | 13:41 | |
| *** jistr has joined #tripleo | 13:42 | |
| *** pblaho has joined #tripleo | 13:42 | |
| jistr | tzumainn: shoot | 13:42 |
| tzumainn | jistr, thanks! simple question - should your spec be updated to match jdob's new understanding of tuskar as a planning service? | 13:42 |
| *** Anuj_ has joined #tripleo | 13:43 | |
| *** yfaykya has joined #tripleo | 13:44 | |
| Anuj_ | Hi is this the Openstack room? | 13:44 |
| tzumainn | because presumably that would imply we need some level of name-checking on the planning side which (I think) might not exist as it stands? | 13:44 |
| rdopiera | Anuj_: this is not a room, it's an irc channel, and it's not called #openstack, it's called #tripleo | 13:44 |
| jistr | tzumainn: hmm i guess so... thinking about it, if we change tuskar into a planning service, then the problem i was trying to solve will disappear :D | 13:44 |
| jistr | tzumainn: what do you mean by name checking? | 13:45 |
| tzumainn | well, we don't want two planned clouds to have the same name, right? | 13:45 |
| *** Anuj_ has quit IRC | 13:45 | |
| jistr | tzumainn: right, that would probably be solved by a uniqueness constraint on the planned cloud names (and when we switch to multi-tenancy, it would be uniqueness per tenant), but that probably doesn't warrant a blueprint... | 13:47 |
| jistr | jdob you broke my blueprint | 13:48 |
| tzumainn | jistr, right, but it's something that we don't have right now - does it warrant mentioning in the spec? | 13:48 |
| *** yfaykya has quit IRC | 13:48 | |
| jdob | jistr: I paved the way for it to be more awesome | 13:48 |
| tzumainn | these are honest questions, not rhetorical btw : ) | 13:48 |
| jistr | jdob: ok :D | 13:48 |
| *** yfaykya has joined #tripleo | 13:50 | |
| tzumainn | jistr, anyway, let me know if you think the spec should be updated for jdob's updates, or if you think the basic understanding is still essentially the same | 13:51 |
| *** shakamunyi has joined #tripleo | 13:51 | |
| *** IvanBerezovskiy has quit IRC | 13:53 | |
| *** vinsh has quit IRC | 13:54 | |
| *** yfaykya has quit IRC | 13:55 | |
| *** yfaykya has joined #tripleo | 13:56 | |
| *** vinsh has joined #tripleo | 13:56 | |
| jistr | tzumainn: yeah it warrants mentioning somewhere, i'm just now wondering if there's anything more to the multiple overclouds spec than this constraint, if we go the "tuskar as a planning service way", then maybe it's better to add a sentence to jdob's blueprint that "Cloud names must be globally unique, when we implement multi-tenancy, they must be unique per tenant." rather than have a separate blueprint to state this sentence :) The multiple | 13:57 |
| jistr | overclouds blueprint essentially dealt with Tuskar<->Heat interaction, but if there's going to be none, then there's nothing to deal with probably. I need to think this more through, if supporting multiple clouds means something significant for the planning service, but my first guess is it doesn't. | 13:57 |
| tzumainn | jistr, okay, that's fair enough! | 13:57 |
| jdob | jistr: ya, I'm thinking the same thing, it's not so much a spec as it is just covered by mine | 13:59 |
| tzumainn | look guys, I simplified your lives by killing a spec | 13:59 |
| tzumainn | you can send me flowers later | 13:59 |
| *** jdob_ has joined #tripleo | 14:00 | |
| *** edmund has quit IRC | 14:00 | |
| *** yfaykya has quit IRC | 14:00 | |
| *** yfaykya has joined #tripleo | 14:02 | |
| jistr | excellent, thanks tzumainn >:D | 14:02 |
| *** jcoufal has quit IRC | 14:03 | |
| jistr | but seriously, good job pointing that out, the sooner someone notices things like this, the better :) | 14:03 |
| *** yfaykya has quit IRC | 14:06 | |
| *** yfaykya has joined #tripleo | 14:08 | |
| tzumainn | jdob, question for you if you have a moment | 14:10 |
| jdob | hit me | 14:11 |
| *** jdob_ has quit IRC | 14:11 | |
| tzumainn | so, tuskar has the plan template, heat has the deployed template | 14:11 |
| tzumainn | the ui presumably has to parse both | 14:11 |
| tzumainn | is the ui parsing them, or is there a library that does it? | 14:11 |
| jdob | heat has classes for an object modeling of a template, but i'm not sure how easy they are to access | 14:12 |
| *** yfaykya has quit IRC | 14:12 | |
| jdob | seems like something they should provide anyway as a utility | 14:12 |
| jdob | so my vote is that in some capacity, heat provides that | 14:12 |
| jdob | even if we have to get someone here to contribute it | 14:13 |
| tzumainn | okay | 14:13 |
| tzumainn | it feels like after the tuskar specs have general agreement, it might be worth doing a gap analysis between the UI and api, just to make sure things aren't falling between the cracks and we have common understanding | 14:14 |
| *** ramishra has quit IRC | 14:14 | |
| *** yfaykya has joined #tripleo | 14:14 | |
| *** rdopiera has quit IRC | 14:16 | |
| jistr | tzumainn: i thought UI would work with Tuskar API on the level of "gimme 5 computes", then Tuskar API would give UI/CLI the full cloud template and we feed it to Heat... or is there a need to parse the templates? | 14:16 |
| *** nealph has joined #tripleo | 14:17 | |
| nealph | lsmola:can you take a quick look at https://review.openstack.org/#/c/94730/ and let me know if the patchset you downloaded was the latest? I've not been able to re-create your issue with the latest patchset. | 14:18 |
| *** yfaykya has quit IRC | 14:19 | |
| *** yfaykya has joined #tripleo | 14:20 | |
| jistr | nealph: afaik he's on PTO today, but should be at work tomorrow | 14:20 |
| nealph | jistr:thanks for the heads up...will try him then. | 14:21 |
| *** IvanBerezovskiy has joined #tripleo | 14:23 | |
| tzumainn | jistr, hm - well, actually, I'm not entirely sure; I had thought we'd need to be able to (for example) parse out the roles being used | 14:28 |
| *** andreaf has quit IRC | 14:29 | |
| *** nati_ueno has quit IRC | 14:29 | |
| jistr | tzumainn: ah ok... well on the planning side you should be able to get it directly from Tuskar API in JSON... on the currently-deployed side (Heat) i'm not sure if it's possible | 14:31 |
| *** jprovazn has quit IRC | 14:34 | |
| tzumainn | jistr, that seems like a gap to me - presumably when the UI is displaying information about a deployed overcloud, it's looking at Heat and not Tuskar, right? | 14:34 |
| *** jprovazn has joined #tripleo | 14:34 | |
| jistr | tzumainn: yes, that's what i meant | 14:36 |
| *** jprovazn has quit IRC | 14:36 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Remove support for legacy testenv-client https://review.openstack.org/97518 | 14:38 |
| *** TravT has joined #tripleo | 14:39 | |
| jistr | so one way around it would be to version the planned cloud, and keep track on the heat stack of: 1) what tuskar cloud was used to create the heat stack 2) what version it was. Then you could fetch info from Tuskar about that version of tuskar cloud (even if it's not the most recent one) instead of trying to parse a heat template. | 14:39 |
| jistr | tzumainn: ^ | 14:39 |
| tzumainn | isn't there the danger that either tuskar or heat is updated independently of the other? | 14:41 |
| *** spzala has joined #tripleo | 14:43 | |
| jistr | tzumainn: yes there is. It would be an issue even if we proxied calls from tuskar to heat. Someone could just go and talk to heat directly anyway. I wonder if we need to worry about this though. E.g. when one deploys a heat stack they can directly ssh into the machine and change its contents completely, e.g. make it a minecraft server even though heat deployed it with a template/image that contained mysql server. | 14:48 |
| jdob | tzumainn: there's a bunch of dangers like that | 14:48 |
| jistr | i kinda hope that people wouldn't do that :) | 14:48 |
| jdob | right, what jistr said, we can never actually stop people from going directly to the underlying services | 14:48 |
| jdob | there's a few hundred ways someone could screw things up if they wanted :) | 14:48 |
| *** rakesh_hs has quit IRC | 14:55 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 15:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 15:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| *** jcoufal has joined #tripleo | 15:02 | |
| *** jcoufal has quit IRC | 15:03 | |
| *** jcoufal has joined #tripleo | 15:03 | |
| tzumainn | jdob, jistr, right, which is why I thought we were going directly to heat for information about the deployed stack | 15:03 |
| *** geerdest has joined #tripleo | 15:11 | |
| *** ramishra has joined #tripleo | 15:15 | |
| *** ramishra has quit IRC | 15:19 | |
| *** robsparker has quit IRC | 15:22 | |
| *** ccrouch1 has joined #tripleo | 15:23 | |
| *** ccrouch has quit IRC | 15:24 | |
| *** ddieterly has quit IRC | 15:25 | |
| *** mrunge has joined #tripleo | 15:25 | |
| * jistr bbl for weekly meeting | 15:25 | |
| *** rdopiera has joined #tripleo | 15:25 | |
| *** jistr has quit IRC | 15:25 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update bootstack to use sysctl-set-value. https://review.openstack.org/84600 | 15:33 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Name sysctl.d files with .conf https://review.openstack.org/97540 | 15:33 |
| *** ramishra has joined #tripleo | 15:36 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Remove uneeded code from pkg-map https://review.openstack.org/97542 | 15:39 |
| *** morganfainberg_Z is now known as morganfainberg | 15:43 | |
| *** eghobo has joined #tripleo | 15:44 | |
| *** andreaf has joined #tripleo | 15:46 | |
| *** nati_ueno has joined #tripleo | 15:47 | |
| *** ramishra has quit IRC | 15:49 | |
| *** IvanBerezovskiy has left #tripleo | 15:49 | |
| *** rlandy has quit IRC | 15:51 | |
| *** nati_uen_ has joined #tripleo | 15:52 | |
| *** nati_ue__ has joined #tripleo | 15:53 | |
| *** nati_ue__ has quit IRC | 15:53 | |
| *** nati_ue__ has joined #tripleo | 15:54 | |
| *** ddieterly has joined #tripleo | 15:54 | |
| *** nati_ueno has quit IRC | 15:55 | |
| *** nati_uen_ has quit IRC | 15:57 | |
| *** matty_dubs is now known as matty_dubs|lunch | 15:57 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-specs: Tripleo CI improvements https://review.openstack.org/95026 | 15:59 |
| *** derekh_ has quit IRC | 16:00 | |
| *** e0ne_ has quit IRC | 16:00 | |
| GheRivero | Signing error: Unable to load certificate - ensure you have configured PKI with "keystone-manage pki_setup" | 16:02 |
| GheRivero | anyone has faced something similar? | 16:02 |
| *** martyntaylor has left #tripleo | 16:02 | |
| *** julim has quit IRC | 16:02 | |
| *** e0ne has joined #tripleo | 16:03 | |
| *** jrist-afk is now known as jrist | 16:04 | |
| *** e0ne has quit IRC | 16:05 | |
| goneri | hi, do you have an idea why the patch is not merged? https://review.openstack.org/#/c/93206/ ? | 16:05 |
| *** cwolferh has joined #tripleo | 16:06 | |
| *** pelix has quit IRC | 16:08 | |
| *** pelix has joined #tripleo | 16:09 | |
| pblaho | goneri: b/c it needs to get approved | 16:10 |
| goneri | oh, indeed, the Workflow+1 is missing. thanks. | 16:11 |
| pblaho | goneri: we do not approve patches if CI is failing even if it is not problem with that patch | 16:11 |
| pblaho | goneri: I will do it now | 16:12 |
| goneri | pblaho: thanks Petr | 16:12 |
| *** dtantsur is now known as dtantsur|afk | 16:12 | |
| pblaho | goneri: done | 16:12 |
| pblaho | np... | 16:12 |
| *** pelix has quit IRC | 16:13 | |
| pblaho | goneri: but it will take some time till it will land... there is long queue on gate | 16:14 |
| *** pelix has joined #tripleo | 16:14 | |
| *** yamahata has quit IRC | 16:16 | |
| openstackgerrit | Dougal Matthews proposed a change to openstack/tripleo-specs: TripleO Template and Deployment Plan Storage https://review.openstack.org/97553 | 16:25 |
| *** julim has joined #tripleo | 16:45 | |
| *** ramishra has joined #tripleo | 16:46 | |
| *** nati_ue__ has quit IRC | 16:46 | |
| *** mrunge has quit IRC | 16:49 | |
| *** fungi has joined #tripleo | 16:51 | |
| fungi | anybody happen to know why we're getting BadRequest: Can not find requested image (HTTP 400) (Request-ID: req-8066372f-8a25-4ca9-982a-5fe3a3476210) when nodepool is attempting to launch nodes in tripleo-test-cloud-rh1? | 16:53 |
| *** robsparker has joined #tripleo | 16:53 | |
| fungi | are there ongoing storage issues there or anything? | 16:53 |
| *** pblaho has quit IRC | 16:53 | |
| fungi | our active (latest) images there according to nodepool's list are 9820141c-bdd0-4b46-b0c1-e70b9de51dbd and ddce1d58-12c7-49b0-a506-5e8f9b294ae4 which were added about 2 hours ago | 16:56 |
| fungi | lifeless: devananda: Ng: SpamapS: jogo: GheRivero: derekh: dprince: slagle: ^ | 16:57 |
| *** rdopiera has quit IRC | 17:00 | |
| fungi | ahh, yep, glance image-list does not show 9820141c-bdd0-4b46-b0c1-e70b9de51dbd as existing at all | 17:01 |
| fungi | so we're failing to build f20 nodes there as of the past couple hours | 17:01 |
| dprince | fungi: So I think rh1 (the HP rack) is down at the moment | 17:01 |
| fungi | i can try to delete that one from nodepool's list so that it doesn't keep trying to use it | 17:01 |
| dprince | fungi: or at least it should be... | 17:01 |
| *** ddieterly has quit IRC | 17:01 | |
| fungi | dprince: rh1 is the rh rack. hp1 is the hp rack (which is apparently also sometimes called r1) | 17:02 |
| *** ifarkas has quit IRC | 17:02 | |
| dprince | fungi: ah, gotcha. Let me check the image listing real quick | 17:02 |
| SpamapS | heh.. I just realized I don't know how to login to the rh1 region at all | 17:03 |
| SpamapS | $ ssh heat-admin@ci-overcloud.rh1.tripleo.org | 17:03 |
| SpamapS | ssh: connect to host ci-overcloud.rh1.tripleo.org port 22: Connection refused | 17:03 |
| SpamapS | Not the same was as R1 ;) | 17:03 |
| dprince | fungi: so that image was apparently deleted today | 17:04 |
| fungi | fun! | 17:04 |
| *** lucasagomes is now known as lucas-dinner | 17:04 | |
| dprince | fungi: http://paste.openstack.org/show/82641/ | 17:04 |
| fungi | nodepool things it added it 2.19 hours ago and that it should be in a ready state | 17:04 |
| fungi | thinks | 17:05 |
| fungi | that's right about the time it should have gone ready. checking the debug logs in nodepool for additional clues | 17:05 |
| *** akrivoka has quit IRC | 17:05 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Move to software-config for the undercloud. https://review.openstack.org/93319 | 17:06 |
| *** ddieterly has joined #tripleo | 17:12 | |
| fungi | dprince: aha! smoking gun... http://paste.openstack.org/show/82646/ | 17:12 |
| fungi | double-barrel too. something broke in your cloud which caused the image to go into an error state, and something is wrong with nodepoold that it didn't remove that broken image itself | 17:13 |
| dprince | fungi: let me check the logs... | 17:13 |
| *** ramishra has quit IRC | 17:15 | |
| dprince | fungi: TRACE glance.api.v1.upload_utils Got error from Swift: put_object('glance', '9820141c-bdd0-4b46-b0c1-e70b9de51dbd', ...) failure and no ability to reset contents for reupload. | 17:15 |
| fungi | neat! | 17:16 |
| SpamapS | disk full? | 17:16 |
| dprince | SpamapS: no | 17:16 |
| *** petertoft has quit IRC | 17:17 | |
| SpamapS | thats just always my first assumption on anything that uses disks | 17:17 |
| *** ccrouch1 has quit IRC | 17:18 | |
| fungi | disks do get full, that's their primary defining characteristic | 17:18 |
| Shrews | pcrews: Welcome!!!!! | 17:19 |
| Shrews | pcrews: can't believe we suckered you in | 17:19 |
| dprince | fungi: I just uploaded 3 images with no issues. | 17:19 |
| dprince | fungi: can we just rekick it? | 17:20 |
| dprince | fungi: I'm wondering if this was a fluke of some sort (not sure what yet...) | 17:20 |
| fungi | dprince: i can just delete the image id from the nodepool end and it will start using the old image, then i can ask it to try to build another new one | 17:20 |
| fungi | i was mainly making sure i didn't destroy any lingering state we needed to hunt down the nodepool failure to correct for that condition | 17:21 |
| dprince | fungi: yeah, lets go with that for now. I was trying things as a different tenant (not using the nodepool account itself). if it reoccurs I'll try that account next | 17:21 |
| SpamapS | Shrews: we offered him all the cupcakes he can eat and bitchfaces he can drink. | 17:22 |
| dprince | fungi: I've got to run off for a bit, should be back online in an hour or so... | 17:22 |
| fungi | pcrews: i second Shrews's welcoming | 17:22 |
| Shrews | SpamapS: I would say I won't let you corrupt him, but I'm afraid he'll corrupt you :) | 17:23 |
| * dprince fungi or just email me dprince@redhat.com if you prefer | 17:23 | |
| fungi | dprince: okay, mostly just wanted to make sure there was actually something broken on the far end too. i think we've got enough details for the infra side now | 17:23 |
| dprince | fungi: cool | 17:24 |
| * pcrews thanks fungi and tells Shrews to let him enjoy the illusion for a little while longer ;) | 17:24 | |
| *** dprince has quit IRC | 17:24 | |
| *** shakamunyi has quit IRC | 17:24 | |
| *** giulivo has quit IRC | 17:26 | |
| *** matty_dubs|lunch is now known as matty_dubs | 17:27 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 17:28 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Make a separation between --heat-env https://review.openstack.org/93091 | 17:28 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Use seed ip instead of gateway ip for ssh-keygen. https://review.openstack.org/94219 | 17:29 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 17:29 |
| *** nati_ueno has joined #tripleo | 17:36 | |
| *** eghobo has quit IRC | 17:39 | |
| *** ccrouch has joined #tripleo | 17:50 | |
| *** cody-somerville has quit IRC | 17:51 | |
| *** jprovazn has joined #tripleo | 17:58 | |
| *** mrunge has joined #tripleo | 17:59 | |
| *** vinsh has quit IRC | 18:00 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/os-collect-config: Fix coverage report generation with tox https://review.openstack.org/97579 | 18:01 |
| *** nati_ueno has quit IRC | 18:02 | |
| *** nati_uen_ has joined #tripleo | 18:02 | |
| *** jang1 has joined #tripleo | 18:03 | |
| *** ddieterly has quit IRC | 18:04 | |
| *** julim has quit IRC | 18:05 | |
| *** ddieterly has joined #tripleo | 18:05 | |
| *** cody-somerville has joined #tripleo | 18:07 | |
| *** julim has joined #tripleo | 18:07 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/os-apply-config: Fix coverage report generation https://review.openstack.org/97583 | 18:07 |
| *** ifarkas has joined #tripleo | 18:13 | |
| *** shakamunyi has joined #tripleo | 18:20 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Deploy cloud hypervisor type https://review.openstack.org/94586 | 18:21 |
| *** shakamunyi has quit IRC | 18:24 | |
| *** shakamunyi has joined #tripleo | 18:25 | |
| *** shakayumi has joined #tripleo | 18:26 | |
| *** shakayumi has quit IRC | 18:26 | |
| *** olaph has joined #tripleo | 18:27 | |
| *** ddieterly has quit IRC | 18:33 | |
| *** mrunge has quit IRC | 18:34 | |
| SpamapS | adam_g: https://bugs.launchpad.net/diskimage-builder/+bug/1316475 <-- what is your current workaround for that? | 18:34 |
| uvirtbot | Launchpad bug 1316475 in diskimage-builder "[SRU] CloudSigma DS for causes hangs when serial console present" [Critical,In progress] | 18:34 |
| lifeless | o/ | 18:35 |
| SpamapS | adam_g: I want to land https://review.openstack.org/#/c/91864/ (reinstates trusty as default) but not until that workaround is present | 18:35 |
| adam_g | SpamapS, https://review.openstack.org/#/c/95598/ | 18:35 |
| SpamapS | ah did we miss the bug linkage there or something? | 18:35 |
| SpamapS | adam_g: ty | 18:37 |
| adam_g | SpamapS, must have. original bug was filed against tripleo-incubator, but not tagged as diskimage-builder till after patch was up? | 18:37 |
| SpamapS | ahh | 18:37 |
| SpamapS | yeah | 18:37 |
| SpamapS | silly bug tasks | 18:37 |
| SpamapS | adam_g: I wonder though if we shouldn't also have ubuntu depend on that element and have it set the default to Ec2.. | 18:38 |
| SpamapS | at least, until the SRU lands in 14.04 | 18:38 |
| *** nati_uen_ has quit IRC | 18:39 | |
| adam_g | SpamapS, we could.. my plan was to push a tripleo-incubator patch to set it in devtest | 18:39 |
| *** nati_ueno has joined #tripleo | 18:39 | |
| adam_g | SpamapS, or we just make Ec2 the default for that element | 18:39 |
| *** ddieterly has joined #tripleo | 18:49 | |
| *** nati_ueno has quit IRC | 18:53 | |
| morganfainberg | lifeless, is there a thread on that topic we discussed a couple weeks ago? | 18:53 |
| morganfainberg | lifeless, the user_id vs "friendly" lookup? | 18:54 |
| morganfainberg | lifeless, i'm digging through trying to find it so i can poke it as needed from an identity stand point (user_ids are not friendly for config files imo, but there are arguments to use them) | 18:54 |
| *** nati_uen_ has joined #tripleo | 18:55 | |
| lifeless | there was yes | 18:55 |
| lifeless | uhm | 18:55 |
| lifeless | FIIK the thread title | 18:55 |
| *** rpodolyaka1 has joined #tripleo | 18:56 | |
| morganfainberg | my google FU wasn't turning things up on this front...but i might be missing some key elementat that would make it show up | 18:56 |
| lifeless | morganfainberg: [openstack-dev] [Heat] [Keystone] [TripleO] Making use of domains by name - policy and API issues? | 18:58 |
| lifeless | Inbox | 18:58 |
| lifeless | x | 18:58 |
| lifeless | 'clint keystone domain unique' found it for me :) | 18:58 |
| morganfainberg | lifeless, awesome, i'll read up on that and breath some life into it (if it's not too scary to do so) | 18:59 |
| morganfainberg | thanks! | 18:59 |
| * morganfainberg might just make some other recommendations to get from where we are to where we need to be | 18:59 | |
| *** dprince has joined #tripleo | 19:00 | |
| lifeless | meeting time! | 19:00 |
| *** pelix has quit IRC | 19:00 | |
| *** jp_at_hp1 has quit IRC | 19:01 | |
| *** jistr has joined #tripleo | 19:02 | |
| *** derekh_ has joined #tripleo | 19:02 | |
| dprince | fungi: how is nodepool? | 19:02 |
| fungi | the new image seemed to build, but we have other fires to deal with now | 19:03 |
| *** e0ne has joined #tripleo | 19:03 | |
| *** petertoft has joined #tripleo | 19:19 | |
| *** shakamunyi has quit IRC | 19:21 | |
| *** petertoft has quit IRC | 19:21 | |
| *** shakamunyi has joined #tripleo | 19:24 | |
| *** e0ne has quit IRC | 19:25 | |
| *** vinsh has joined #tripleo | 19:28 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Deploy cloud hypervisor type https://review.openstack.org/94586 | 19:32 |
| *** jml has joined #tripleo | 19:35 | |
| *** e0ne has joined #tripleo | 19:39 | |
| *** bcrochet has quit IRC | 19:41 | |
| *** bcrochet has joined #tripleo | 19:41 | |
| *** bcrochet has quit IRC | 19:45 | |
| *** bcrochet has joined #tripleo | 19:48 | |
| *** pblaho has joined #tripleo | 19:49 | |
| *** sballe has joined #tripleo | 19:53 | |
| *** e0ne has quit IRC | 19:57 | |
| *** rpodolyaka1 has left #tripleo | 20:01 | |
| *** bcrochet has quit IRC | 20:02 | |
| *** jtomasek has quit IRC | 20:03 | |
| derekh_ | lifeless: I think that tar ball creation error message is a red herring, details added to bug | 20:03 |
| lifeless | derekh_: ok | 20:04 |
| lifeless | derekh_: I see, sure. | 20:04 |
| *** bcrochet has joined #tripleo | 20:05 | |
| *** derekh_ has quit IRC | 20:08 | |
| *** weshay has quit IRC | 20:10 | |
| *** bcrochet has quit IRC | 20:14 | |
| *** shakamunyi has quit IRC | 20:14 | |
| *** shakamunyi has joined #tripleo | 20:14 | |
| *** jcoufal has quit IRC | 20:15 | |
| *** ifarkas has quit IRC | 20:15 | |
| *** pblaho has quit IRC | 20:17 | |
| *** bcrochet has joined #tripleo | 20:19 | |
| *** jistr has quit IRC | 20:20 | |
| *** bcrochet has quit IRC | 20:21 | |
| *** ddieterly has quit IRC | 20:21 | |
| *** bcrochet has joined #tripleo | 20:23 | |
| *** weshay has joined #tripleo | 20:25 | |
| *** nati_uen_ has quit IRC | 20:28 | |
| *** ddieterly has joined #tripleo | 20:33 | |
| *** sballe has quit IRC | 20:35 | |
| *** julim has quit IRC | 20:37 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-specs: Unit Testing TripleO Projects https://review.openstack.org/95887 | 20:43 |
| *** mrunge has joined #tripleo | 20:45 | |
| *** nati_ueno has joined #tripleo | 20:47 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Move rabbitmq-server cluster port https://review.openstack.org/91704 | 20:54 |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 21:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326016 | 21:00 |
| *** nati_uen_ has joined #tripleo | 21:00 | |
| uvirtbot | Launchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** nati_ueno has quit IRC | 21:00 | |
| *** jdob has quit IRC | 21:00 | |
| *** jprovazn has quit IRC | 21:02 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-image-elements: Adjust rpc thread pool for Ironic https://review.openstack.org/93463 | 21:03 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-incubator: Allow overcloud fixed range to be configurable https://review.openstack.org/97626 | 21:05 |
| *** jml has quit IRC | 21:07 | |
| *** jml has joined #tripleo | 21:09 | |
| *** jml has quit IRC | 21:11 | |
| *** jml has joined #tripleo | 21:13 | |
| stevebaker | lifeless: attribute memoization hasn't landed yet - its still in the merge queue | 21:15 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tripleo-incubator: Allow overcloud fixed range to be configurable https://review.openstack.org/97626 | 21:15 |
| *** bcrochet is now known as bcrochet|g0ne | 21:22 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script https://review.openstack.org/86435 | 21:24 |
| *** shakamunyi has quit IRC | 21:26 | |
| *** mrunge has quit IRC | 21:27 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:32 | |
| SpamapS | stevebaker: hasn't it been there for like.. a week? | 21:32 |
| *** jang1 has quit IRC | 21:32 | |
| greghaynes | Looks like it has a +A https://review.openstack.org/#/c/94755/ | 21:34 |
| *** weshay has quit IRC | 21:34 | |
| greghaynes | Can I get a +A on https://review.openstack.org/#/c/89925/ (tests passed) | 21:35 |
| *** morazi has quit IRC | 21:37 | |
| SpamapS | greghaynes: from what I see, zuul is a bit.. overwhelmed.. so it may be a while, but +A'd nonetheless. | 21:39 |
| greghaynes | poor zuul | 21:40 |
| *** marun is now known as marun_afk | 21:40 | |
| lifeless | SpamapS: the polling stress test | 21:45 |
| lifeless | SpamapS: StevenK and I are confused about how to spin up the pollers | 21:46 |
| SpamapS | lifeless: scrape info from resource-metadata | 21:46 |
| lifeless | SpamapS: was there a script to to that? I don't fancy doing 2K by hand | 21:47 |
| SpamapS | lifeless: yes | 21:47 |
| SpamapS | it was an unholy one liner | 21:47 |
| lifeless | can has ? | 21:47 |
| SpamapS | StevenK did it | 21:47 |
| SpamapS | and I thought broke it out a bit and put it in a review somewhere | 21:47 |
| SpamapS | called like "kill-heat" | 21:48 |
| lifeless | ah | 21:48 |
| lifeless | he blamed you | 21:48 |
| lifeless | you blame him. Great. | 21:48 |
| lifeless | :) | 21:48 |
| SpamapS | no doubt I conceived of the monster | 21:48 |
| SpamapS | but he sewed up all the bits and found me a brain | 21:48 |
| *** gcha has quit IRC | 21:48 | |
| SpamapS | Apparently, the monster escaped | 21:48 |
| SpamapS | lifeless: I know for sure I don't have it socked away somewhere | 21:49 |
| SpamapS | and now I need to run out for some errands. | 21:49 |
| lifeless | kk | 21:49 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash https://review.openstack.org/96308 | 21:50 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash https://review.openstack.org/96308 | 21:54 |
| *** nati_uen_ has quit IRC | 21:54 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Only use Ec2 cloud-init data source for Ubuntu https://review.openstack.org/97634 | 21:56 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Reinstate Trusty as default for Ubuntu https://review.openstack.org/91864 | 21:58 |
| *** geerdest has quit IRC | 21:59 | |
| adam_g | SpamapS, ^ updated with dependencies that will work around the cloud-init issue till fixed | 21:59 |
| *** geerdest has joined #tripleo | 21:59 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Add os-is-bootstrap-host element and script https://review.openstack.org/86435 | 22:01 |
| *** nati_ueno has joined #tripleo | 22:02 | |
| SpamapS | adam_g: +2's around | 22:03 |
| * SpamapS disappears for a bit | 22:03 | |
| adam_g | doesnt seem like there is a cloud-init SRU rushing into the queue, so probably good idea to merge that in the short term | 22:04 |
| greghaynes | adam_g: What do you think about making the CLOUD_INIT_DATASOURCES fix check for DIB_RELEASE=trusty? | 22:05 |
| lifeless | greghaynes: surely thats the ubuntu elements perogative? | 22:06 |
| greghaynes | yes, I mean the fix in the ubuntu element | 22:06 |
| greghaynes | https://review.openstack.org/#/c/97634/1/elements/ubuntu/environment.d/10-cloud-init-datasources.bash | 22:06 |
| lifeless | ack | 22:06 |
| *** jml has quit IRC | 22:06 | |
| adam_g | greghaynes, sure | 22:08 |
| adam_g | greghaynes, well, actually no. as it is, cloud-init-datasources requires the environment variable be set to at least one source. if its conditional, it can't depend on the element | 22:09 |
| *** jml has joined #tripleo | 22:09 | |
| adam_g | happy to reconsider that requirement, tho | 22:09 |
| greghaynes | hrm. probably not worth it then | 22:11 |
| lifeless | adam_g: -1 on it | 22:13 |
| greghaynes | argh, it is in dib-elements though, and those are supposed to be general usage-ish | 22:13 |
| adam_g | lifeless, ? | 22:13 |
| lifeless | adam_g: the environment.d stuff runs in order | 22:14 |
| lifeless | you have not supplied a gap between the cloud-init-nocloud elements position and the ubuntu one | 22:14 |
| adam_g | oh, right | 22:14 |
| lifeless | or rather, the gap you have isn't clear enough to avoid confusion (and I haven't strictly checked that there is such a gap in fact...) | 22:14 |
| adam_g | thanks | 22:14 |
| lifeless | did nick's suggestions for tftp help? | 22:16 |
| adam_g | lifeless, which? | 22:22 |
| lifeless | I dunno | 22:22 |
| lifeless | apparently ng and nick chatted last night | 22:22 |
| lifeless | but i haven't seen any comms from Ng about it... | 22:22 |
| adam_g | im no sure. things are stable again | 22:22 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash https://review.openstack.org/96308 | 22:23 |
| lifeless | adam_g: cool! but we don't know what changed? | 22:23 |
| adam_g | but im not sure thats because we rotated out the old seed host /w potentially problematic hardware, or because new firmware was pushed out everywhere | 22:23 |
| adam_g | im not convinced it is firmware related.. the original seed host is still showing the same symptoms | 22:23 |
| Ng | lifeless: I upgraded the mellanox/raid firmwares to the same versions that 306 is running, but GheRivero had already had a successful run before I did that | 22:23 |
| Ng | so..... erp? | 22:24 |
| adam_g | as is one other node in the rack that has been excluded from our deploys isnce the beginning | 22:24 |
| *** nati_ueno has quit IRC | 22:24 | |
| adam_g | Ng, im about 90% into my second successful run right now on 405, and have another going on 306 | 22:24 |
| Ng | (27 rounds of firmware upgrades was not fun, but if someone needs to do it on any of the other nodes that aren't in nodes.json, the newer seed host has ~cetest/fw/hell.tgz with all the parts you need, and a script in it called inferis.sh that will automate everything bar saying "yes" twice | 22:25 |
| Ng | adam_g: nice | 22:25 |
| *** vinsh has quit IRC | 22:27 | |
| lifeless | Ng: why not bake that into the deploy ramdisk ? | 22:27 |
| Ng | lifeless: I'm not at all opposed to doing that sort of thing, but I wanted to just get this done so we could get back to testing deploys. The mellanox firmware would definitely be doable in the ramdisk. the RAID fw is a bit trickier in that it expects you to type yes twice and is one of those hateful self-extracting shell archives, so will be pretty fiendish to modify | 22:29 |
| lifeless | ack | 22:29 |
| lifeless | Ng: expect? | 22:29 |
| Ng | lifeless: I knew you'd say that ;) | 22:29 |
| Ng | I mean, sure, we could shove expect into the ramdisk too, why not! | 22:30 |
| TheJulia | Ng: sed -i could possibly be used to modify the script to remove the text entry | 22:30 |
| Ng | TheJulia: I suspect it\s not directly in the script, those SHAR things tend to just be an extractor script with a giant wad of data at the end, with all of the actuall installer scripts inside that :/ | 22:30 |
| Ng | but I'm sure we could make it work | 22:30 |
| TheJulia | :\ ugh, well if its in the script it would be easy, but yeah, otherwise its expect :( | 22:31 |
| Ng | lifeless: I sort of think this would make more sense as a separate image that ironic would boot either as part of the initial validation of a node, or at later instruction. I guess it's probably down to the time for that extra reboot, vs the overhead of a bigger ramdisk | 22:32 |
| *** nati_ueno has joined #tripleo | 22:32 | |
| lifeless | Ng: so, maybe, but we can work with what we've got, no ? | 22:32 |
| lifeless | IPA will apparently solve everything (though I think its a mistake to centralise but thats a different discussion) | 22:33 |
| Ng | Ironic Pale Ale | 22:33 |
| lifeless | Ng: note that if we have two different ramdisks, its not just a new boot, its also two copies of the kernel being transferred etc etc | 22:33 |
| Ng | we'd probably also want to test things like the safety of repeated runs of the fw upgrade scripts (i.e. do they detect there is nothing to do and exit cleanly) | 22:34 |
| Ng | hmm | 22:34 |
| lifeless | Ng: indeed, idempotency ftw | 22:34 |
| jogo | lifeless: so rudamentry heat scale testing in gate | 22:38 |
| jogo | lifeless: while infra is backed up thought I would dive into some non-patch intensive things | 22:39 |
| jogo | SpamapS: ^ | 22:39 |
| lifeless | cool | 22:40 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-specs: TripleO on OpenStack https://review.openstack.org/92642 | 22:40 |
| jogo | lifeless: so last I heard some folks had some proof of concept code | 22:41 |
| jogo | lifeless: do you know who it was? | 22:41 |
| lifeless | jogo: null virt driver, a stock tripleo template to deploy N hypervisor, and then a script to scrape the heat resource metadata and run os-collect-config once per machine | 22:42 |
| Ng | it was StevenK, right? | 22:42 |
| Ng | he mentioned at the summit that he was doing tremendously evil things for lifeless | 22:42 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Extract mysql reset files from reset-db https://review.openstack.org/93104 | 22:43 |
| lifeless | well spamaps, but yes | 22:43 |
| jogo | so is there any PoC code out there? | 22:43 |
| lifeless | jogo: apparently there is a review up with the scraping code | 22:44 |
| lifeless | the rest of it, no. | 22:44 |
| lifeless | it was done by hand | 22:44 |
| jogo | lifeless: link? | 22:44 |
| lifeless | jogo: I say apparently because I haven't seen it | 22:45 |
| lifeless | jogo: have a look in tripleo-incubator perhaps? for things by stevenk | 22:45 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization https://review.openstack.org/93032 | 22:45 |
| *** nati_uen_ has joined #tripleo | 22:47 | |
| Ng | adam_g: is that 405 run still holding steady? | 22:48 |
| adam_g | Ng, yes, its completed now, just trying to ping the user instance | 22:48 |
| Ng | ooh | 22:48 |
| jogo | ahh https://review.openstack.org/#/c/93819/ | 22:49 |
| *** nati_ueno has quit IRC | 22:50 | |
| *** jp_at_hp has joined #tripleo | 22:58 | |
| greghaynes | SpamapS: https://review.openstack.org/#/c/93554/9 re: why are they parameters, the ports will need to be used in more than one element (e.g. haproxy and app) and since there arent heat variables I assumed parameters would be better than hard coding numbers multiple places | 22:59 |
| *** TravT has quit IRC | 22:59 | |
| *** bcrochet|g0ne is now known as bcrochet | 23:01 | |
| lifeless | gothicmindfood: so g+ ? | 23:08 |
| lifeless | gothicmindfood: oh, I see, uhm I don't have john's g+ | 23:09 |
| *** ddieterly has quit IRC | 23:14 | |
| *** lathiat_ has joined #tripleo | 23:16 | |
| *** yfaykya_ has joined #tripleo | 23:16 | |
| *** dtantsur has joined #tripleo | 23:20 | |
| *** spiffxp- has joined #tripleo | 23:20 | |
| *** lathiat has quit IRC | 23:20 | |
| *** spiffxp has quit IRC | 23:20 | |
| *** peoplemerge has quit IRC | 23:20 | |
| *** dtantsur|afk has quit IRC | 23:20 | |
| *** andreaf has quit IRC | 23:20 | |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Run free https://review.openstack.org/93634 | 23:20 |
| *** andreaf has joined #tripleo | 23:20 | |
| *** yfaykya_ has quit IRC | 23:21 | |
| lifeless | wat | 23:21 |
| lifeless | oh heh | 23:21 |
| *** yfaykya_ has joined #tripleo | 23:22 | |
| *** peoplemerge has joined #tripleo | 23:23 | |
| greghaynes | hahaha | 23:23 |
| *** yfaykya has quit IRC | 23:24 | |
| *** yfaykya_ has quit IRC | 23:26 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Only use Ec2 cloud-init data source for Ubuntu https://review.openstack.org/97634 | 23:27 |
| *** geerdest has quit IRC | 23:27 | |
| *** yfaykya has joined #tripleo | 23:28 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Reinstate Trusty as default for Ubuntu https://review.openstack.org/91864 | 23:30 |
| *** yfaykya has quit IRC | 23:33 | |
| *** yfaykya has joined #tripleo | 23:34 | |
| *** yfaykya has quit IRC | 23:39 | |
| *** yfaykya has joined #tripleo | 23:41 | |
| openstackgerrit | A change was merged to openstack/tuskar: Regenerate tuskar.conf.sample https://review.openstack.org/96759 | 23:42 |
| *** andreaf has quit IRC | 23:42 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Add haproxy element to overcloud-control image https://review.openstack.org/93880 | 23:43 |
| *** yfaykya has quit IRC | 23:45 | |
| *** saurabhs has joined #tripleo | 23:46 | |
| *** yfaykya has joined #tripleo | 23:47 | |
| saurabhs | hey guys what is correct way of copying a file on image without it being run through os-apply-config | 23:47 |
| saurabhs | I want to copy a mysql config which is a jinja template, but if I configure it to be part of os-apply-config element then the template is messed up | 23:48 |
| *** lucas-dinner has quit IRC | 23:51 | |
| *** yfaykya has quit IRC | 23:51 | |
| *** yfaykya has joined #tripleo | 23:53 | |
| *** dprince has quit IRC | 23:53 | |
| openstackgerrit | A change was merged to openstack/os-collect-config: Use json equality rather than raw text https://review.openstack.org/94548 | 23:57 |
| *** yfaykya has quit IRC | 23:57 | |
| *** yfaykya has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!