| lifeless | mordred: btw - reinstall redphone :) | 00:01 |
|---|---|---|
| *** nati_ueno has quit IRC | 00:05 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/tripleo-ci: Use /opt/git instead of /opt/stack/new https://review.openstack.org/89974 | 00:05 |
| mordred | lifeless: merry christmas ^^ | 00:05 |
| *** epim has quit IRC | 00:06 | |
| *** epim has joined #tripleo | 00:10 | |
| *** epim has quit IRC | 00:12 | |
| StevenK | Hm | 00:14 |
| StevenK | I think that won't work, since the devtest scripts won't know about the openstack/ in the directory, for setting things like PATH | 00:15 |
| * mordred cries a little | 00:16 | |
| lifeless | StevenK: hmm ? | 00:17 |
| StevenK | mordred: You could accidently ln -s $TRIPLEO_ROOT/openstack/tripleo-incubator to $TRIPLEO_ROOT/tripleo-incubator | 00:17 |
| *** epim has joined #tripleo | 00:18 | |
| *** eguz has joined #tripleo | 00:19 | |
| *** eghobo has quit IRC | 00:23 | |
| *** epim has quit IRC | 00:29 | |
| *** fungi has quit IRC | 00:30 | |
| vinsh | greghaynes, http://paste.openstack.org/show/76870/ | 00:30 |
| vinsh | this thing. | 00:30 |
| vinsh | 3 nodes. all running keepalived. control0 prio300 control1 prio200 control2 prio300 | 00:31 |
| vinsh | sudo service keepalived stop on all of them | 00:31 |
| vinsh | This is a tcp dump with the service stopped... you can see the advertisements.. but its filled with a load of 100 vs 201 advertisements... that don't seem to have a labeled source | 00:32 |
| vinsh | its the virtual ip.. advertising to vrrp "vrrp 192.0.2.254 > 224.0.0.18" | 00:36 |
| vinsh | wth. | 00:36 |
| *** nati_ueno has joined #tripleo | 00:36 | |
| openstackgerrit | Monty Taylor proposed a change to openstack/tripleo-incubator: Handle two-level-deep git repo structure https://review.openstack.org/89977 | 00:39 |
| mordred | StevenK: how about that ^^ ? | 00:39 |
| *** nati_ueno has quit IRC | 00:40 | |
| StevenK | mordred: Did you test that? :-) | 00:42 |
| StevenK | Because I think it's going to die horribly. | 00:43 |
| lifeless | greghaynes: you might enjoy the workflow vs convergence thing now | 00:47 |
| lifeless | clarkb: do you know where we are at on the f20 starvation thing? | 00:50 |
| clarkb | no idea | 00:50 |
| *** kiall has quit IRC | 00:58 | |
| *** proffalken has quit IRC | 00:58 | |
| *** Shrews has quit IRC | 00:58 | |
| *** Shrews has joined #tripleo | 01:00 | |
| *** proffalken has joined #tripleo | 01:00 | |
| *** rlandy has quit IRC | 01:01 | |
| *** kiall has joined #tripleo | 01:02 | |
| *** nati_ueno has joined #tripleo | 01:10 | |
| lifeless | ugh | 01:15 |
| lifeless | 2014-04-23 03:51:00.874 | No distributions at all found for oslo.config>=1.2.0 (from -r /opt/stack/nova/requirements.txt (line 32)) | 01:15 |
| lifeless | 2014-04-23 03:51:00.874 | Storing debug log for failure in /root/.pip/pip.log | 01:15 |
| *** TravT has quit IRC | 01:15 | |
| lifeless | cody-somerville: oh hai | 01:19 |
| lifeless | cody-somerville: around ? | 01:19 |
| cody-somerville | lifeless: Heya :) | 01:19 |
| *** yamahata_ has joined #tripleo | 01:19 | |
| *** yamahata has joined #tripleo | 01:20 | |
| mordred | lifeless: if I want to reference $(lsb_release -sc) in an element - (trusty) is that something already abstracted? | 01:24 |
| StevenK | $DIB_RELEASE perhaps? | 01:26 |
| lifeless | cody-somerville: heya | 01:27 |
| mordred | StevenK: awesome. thanks! | 01:27 |
| *** fandi has quit IRC | 01:27 | |
| *** nati_ueno has quit IRC | 01:28 | |
| *** eguz has quit IRC | 01:33 | |
| vinsh | btw.. anyone following the keepalived on 3 control node saga.. it all works.. if you choose br-ex as the keepalived interface.. but not if you chose eth0. | 01:33 |
| vinsh | so. progress-ish. | 01:33 |
| mordred | StevenK: ok - you're my new goto person on questions. lucky you | 01:35 |
| lifeless | bbiab | 01:36 |
| mordred | StevenK: https://review.openstack.org/#/c/88479/3/modules/openstack_project/files/nodepool/elements/openstack-repos/extra-data.d/50-create-repo-list | 01:36 |
| *** nosnos has joined #tripleo | 01:36 | |
| mordred | StevenK: if I have a python script in an element script | 01:36 |
| mordred | it doesn't seem to so much be getting env vars | 01:36 |
| mordred | am I just stupid? | 01:37 |
| StevenK | mordred: I'm struggling to see how it's nailed together | 01:47 |
| mordred | StevenK: I thinkI figured it out | 01:48 |
| StevenK | mordred: Then I was delighted I could help? | 01:48 |
| mordred | StevenK: the simple problem was not that the sript wasn't seeing the env var - it was that I was joining two paths that both start with '/' | 01:49 |
| mordred | which ends up with fail | 01:50 |
| StevenK | Ah yeah | 01:50 |
| StevenK | I've hit that myself before | 01:50 |
| StevenK | os.path.join('/foo', '/bar') doesn't end with cute puppies and rainbows. | 01:50 |
| mordred | so this: Caching gerrit from git://git.openstack.org/openstack-infra/gerrit.git in /home/mordred/.cache/image-create/source-repositories/gerrit_68bdd7e0a4a59330c382812459e112857f1361da | 01:50 |
| mordred | if, in a day, there are new refs in that repo | 01:51 |
| mordred | do we re-clone? or do we fetch and move the dir? | 01:51 |
| clarkb | mordred: fetching is tricky because by default it won't fetch all the things | 01:52 |
| clarkb | but that is probably ok for caching | 01:52 |
| mordred | clarkb: btw - you should look in dib in elements/cache-url/bin/cache-url | 01:53 |
| adam_g | has the bump of default release from saucy -> trusty caused problems for anyone else? | 01:53 |
| mordred | clarkb: we should totally appropriate that code for the places in infra/config where we're downloading a url and wanting to NOT take action if the url contents have not changed :) | 01:53 |
| mordred | lifeless: btw - not that you'll be surprised or anything - but your suggestion to have an extra-data script write out a source-repositories file worked like a champ | 02:00 |
| *** weshay has quit IRC | 02:02 | |
| *** jrist has quit IRC | 02:02 | |
| *** jrist has joined #tripleo | 02:09 | |
| lifeless | mordred: great | 02:19 |
| mordred | lifeless: and I think I'm really close to having the image elements part of the project done | 02:23 |
| *** nati_ueno has joined #tripleo | 02:26 | |
| lifeless | StevenK: have you submitted your pyconau talk proposal? | 02:40 |
| lifeless | StevenK: I need brainstorm help for mine :/ | 02:40 |
| StevenK | lifeless: I have not | 02:41 |
| lifeless | StevenK: duuuude | 02:41 |
| lifeless | jogo: have you ? and anteaya I think you were planning on one too ? | 02:41 |
| clarkb | its too bad it happens during seattle summer otherwise I totally would | 02:44 |
| StevenK | clarkb: Seattle has one week of summer? :-P | 02:44 |
| clarkb | StevenK: exactly | 02:44 |
| clarkb | I will have family visiting too | 02:45 |
| clarkb | but yes first week of august is summer | 02:45 |
| StevenK | lifeless: I have an idea from SpamapS, which sounds pretty good. How can I help? | 02:45 |
| *** newell has quit IRC | 02:45 | |
| *** NobodyCam has quit IRC | 02:46 | |
| StevenK | clarkb: MAD did a send up of Northern Exposure, if you remember that show from the 90s. One bit of that send up was, "There's an awful lot of snow around, when is summer?" and the reply: "I don't rightly remember. Last year it was Tuesday." | 02:46 |
| *** morganfainberg is now known as morganfainberg_Z | 02:47 | |
| *** yamahata has quit IRC | 02:57 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1311631 | 03:00 |
| uvirtbot | Launchpad bug 1311631 in tripleo "rabbitmq os-refresh config fails on the master due to mis-parsing of nodes list" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 03:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| mordred | lifeless: if you saw this: | 03:04 |
| mordred | /tmp/image.N7JkNvoL/hooks/extra-data.d/98-source-repositories: line 104: pushd: /tmp/image.N7JkNvoL/mnt/root/config: Permission denied | 03:04 |
| mordred | lifeless: would you have an immediate reaction as to "zomg, that's clear a fail in XXXX" | 03:04 |
| lifeless | extra-data runs outside the chroot | 03:05 |
| lifeless | as such its not running as root | 03:05 |
| lifeless | you probably want to sudo there. | 03:05 |
| lifeless | I think. | 03:05 |
| lifeless | thats the path I'd look at first anyhow | 03:06 |
| lifeless | as well as obviously making sure that root/config is indeed a directory | 03:06 |
| mordred | but - I mean, it's a source-repostories thing | 03:07 |
| mordred | do I need to make an extra-data script that chmod's /root first? | 03:08 |
| mordred | and then a 99 that chmod's it back? | 03:08 |
| mordred | the intent is that I want it to clone a repo and put it in /root/config | 03:08 |
| mordred | you know - how about I just try something else | 03:08 |
| lifeless | so you should be able to | 03:16 |
| lifeless | I'd dig in a little and see whats going on | 03:16 |
| lifeless | it should be totally possible and straight forward | 03:16 |
| *** geerdest has quit IRC | 03:19 | |
| cody-somerville | ugh, realizing one of the earbuds in your headset is no longer working just before a long call is the worst. :( | 03:29 |
| lifeless | ouch | 03:33 |
| *** ramishra has joined #tripleo | 03:46 | |
| *** Rakesh6 has joined #tripleo | 03:49 | |
| *** vinsh is now known as vinsh_away | 03:53 | |
| *** nati_ueno has quit IRC | 03:55 | |
| mordred | lifeless: are install.ds' interleaved of sequential? | 03:57 |
| lifeless | sequential today | 03:57 |
| lifeless | we reserve the right to be more parallel in future but only for same-numbered things | 03:57 |
| mordred | so ALL of one element's stuff runs, then aother | 03:57 |
| mordred | or, rather, all of one elements' install.d runs, then the next, yeah? | 03:58 |
| *** eghobo has joined #tripleo | 04:01 | |
| * StevenK tries to expand his 66 word abstract | 04:10 | |
| lifeless | no | 04:13 |
| lifeless | all 66-foo runs | 04:13 |
| lifeless | then all 67-foo runs | 04:13 |
| lifeless | across all elements | 04:14 |
| lifeless | this lets elements drop new stuff inbetween bits of another element | 04:14 |
| lifeless | mordred: ^ | 04:15 |
| lifeless | StevenK: cool | 04:15 |
| lifeless | StevenK: got any other ideas? | 04:15 |
| *** yamahata has joined #tripleo | 04:17 | |
| StevenK | lifeless: I was pondering about os-cloud-config and friends | 04:18 |
| lifeless | ulp, pickup C time | 04:21 |
| *** lazy_prince has joined #tripleo | 04:25 | |
| *** morazi has quit IRC | 04:27 | |
| *** yamahata has quit IRC | 04:48 | |
| *** yamahata has joined #tripleo | 04:49 | |
| mordred | lifeless: ok. that's what I thought happened | 04:49 |
| mordred | but then it was weird | 04:49 |
| *** nati_ueno has joined #tripleo | 04:52 | |
| adam_g | hmph https://bugs.launchpad.net/diskimage-builder/+bug/1311987 | 05:03 |
| uvirtbot | Launchpad bug 1311987 in diskimage-builder "deploy-ironic ramdisks do not boot: Failed to execute /init (error -28)" [Undecided,New] | 05:03 |
| greghaynes | adam_g: was waiting for the big bug to show | 05:04 |
| adam_g | greghaynes, :) | 05:04 |
| StevenK | I thought it was /sbin/init, is /init an initramfs thing? | 05:04 |
| greghaynes | it says /sbin/init in trace | 05:05 |
| adam_g | greghaynes, that makes 2 bugs blocking the ironic + devstack functional from fully passing. once it is, i'd like to get it added the pipeline for diskimage-builder to catch those sorts | 05:05 |
| greghaynes | I assume you mean ironic + devtest | 05:06 |
| *** rwsu has quit IRC | 05:07 | |
| *** nati_ueno has quit IRC | 05:07 | |
| greghaynes | adam_g: Thats just running with USE_IRONIC=1? | 05:08 |
| adam_g | greghaynes, no, not devtest. devstack | 05:08 |
| greghaynes | oh | 05:08 |
| adam_g | http://logs.openstack.org/96/87396/8/check/check-tempest-dsvm-virtual-ironic/556a91a | 05:08 |
| *** nosnos has quit IRC | 05:08 | |
| adam_g | its the standard tempest + devstack check /w nova configured to use the ironic driver | 05:09 |
| lifeless | adam_g: is that a trusty ramdisk thing? cause saucy ironic ramdisk works for me | 05:13 |
| adam_g | lifeless, yes. saucy works fine for me too | 05:13 |
| adam_g | lifeless, the bump to trusty merged last night. noticed busted stuff today | 05:14 |
| lifeless | ok | 05:17 |
| lifeless | so that might be a more-systemd thing | 05:17 |
| StevenK | Why? Trusty is still upstart | 05:18 |
| *** e0ne has joined #tripleo | 05:22 | |
| *** rwsu has joined #tripleo | 05:23 | |
| StevenK | My devstack-gate change finally passed Jenkins | 05:23 |
| clarkb | StevenK: which one? | 05:25 |
| lifeless | StevenK: hah, systemd-logind for instances | 05:25 |
| lifeless | StevenK: in saucy, replaces pm stuff... | 05:25 |
| StevenK | clarkb: https://review.openstack.org/86435 | 05:25 |
| StevenK | lifeless: Sad. Face. | 05:25 |
| StevenK | clarkb: No, ignore me, that isn't the link. | 05:26 |
| StevenK | clarkb: https://review.openstack.org/#/c/89978/ | 05:26 |
| clarkb | :P I was going to say, that isn't devstack-gate | 05:26 |
| *** e0ne has quit IRC | 05:26 | |
| clarkb | I am still trying to get sdague's removal of bc through | 05:26 |
| lifeless | StevenK: using occ as an arbitrary message bus for instance? | 05:26 |
| lifeless | devananda: btw | 05:26 |
| lifeless | devananda: is occ the agent backplane for ^ or something custom? | 05:26 |
| StevenK | clarkb: Oh, thanks | 05:27 |
| StevenK | lifeless: Hm, that sounds like an idea. | 05:27 |
| StevenK | lifeless: http://paste.openstack.org/show/176EKsX7HUwX4oMqghyd/ is my current abstract. Can you proof-read? | 05:30 |
| lifeless | I don't know | 05:30 |
| lifeless | I'll try | 05:30 |
| lifeless | StevenK: ok; instant thoughts | 05:31 |
| lifeless | there's no title | 05:31 |
| lifeless | the first sentence doesn't hook me | 05:31 |
| lifeless | and there's no cohesive theme except lessons-learnt | 05:31 |
| StevenK | lifeless: Title is a seperate field, currently I have "TripleO, git, pip, venvs, tox, oh my" | 05:31 |
| *** nati_ueno has joined #tripleo | 05:33 | |
| *** nosnos has joined #tripleo | 05:35 | |
| *** saurabhs has left #tripleo | 05:35 | |
| *** eghobo has quit IRC | 05:39 | |
| *** eghobo has joined #tripleo | 05:39 | |
| *** eguz has joined #tripleo | 05:46 | |
| lifeless | StevenK: sure, I get that. So what do you think the theme is? | 05:47 |
| StevenK | lifeless: "Lessons learnt, and where to go next" was what I was thinking | 05:48 |
| *** eghobo has quit IRC | 05:50 | |
| *** nati_ueno has quit IRC | 05:58 | |
| *** jcoufal has joined #tripleo | 06:02 | |
| *** mrunge has joined #tripleo | 06:04 | |
| *** nati_ueno has joined #tripleo | 06:05 | |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install the "classic" icinga interface https://review.openstack.org/87223 | 06:10 |
| openstackgerrit | Matthew Macdonald-Wallace proposed a change to openstack/tripleo-image-elements: Install check_mk server https://review.openstack.org/87226 | 06:11 |
| *** jprovazn has joined #tripleo | 06:23 | |
| *** jeremydei has quit IRC | 06:31 | |
| lifeless | StevenK: scars earnt and thumbs ruled? | 06:34 |
| StevenK | lifeless: Heh, yes | 06:36 |
| * StevenK has finally convinced his laptop to upgrade to Trusty | 06:38 | |
| *** akrivoka has joined #tripleo | 06:44 | |
| *** ramishra has quit IRC | 06:45 | |
| *** ramishra has joined #tripleo | 06:46 | |
| *** rha has joined #tripleo | 06:47 | |
| rcarrillocruz | morning | 06:51 |
| *** ifarkas has joined #tripleo | 06:59 | |
| *** lsmola has joined #tripleo | 07:08 | |
| *** e0ne has joined #tripleo | 07:11 | |
| *** eguz has quit IRC | 07:13 | |
| *** e0ne has quit IRC | 07:16 | |
| *** lucas-dinner has quit IRC | 07:17 | |
| *** e0ne has joined #tripleo | 07:17 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-heat-templates: Introduce configurable virtual ip in templates https://review.openstack.org/89556 | 07:17 |
| *** bauzas has joined #tripleo | 07:21 | |
| *** ramishra has quit IRC | 07:24 | |
| *** ifarkas has quit IRC | 07:25 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Include default configuration for Sphinx https://review.openstack.org/89780 | 07:26 |
| *** ifarkas has joined #tripleo | 07:26 | |
| *** d0ugal has quit IRC | 07:27 | |
| *** olaph has quit IRC | 07:28 | |
| *** d0ugal has joined #tripleo | 07:29 | |
| *** d0ugal has quit IRC | 07:29 | |
| *** d0ugal has joined #tripleo | 07:29 | |
| *** mrunge has quit IRC | 07:35 | |
| *** rpodolyaka has joined #tripleo | 07:37 | |
| rpodolyaka | morning | 07:44 |
| *** rpodolyaka has quit IRC | 07:47 | |
| *** hashar has joined #tripleo | 07:47 | |
| *** rpodolyaka has joined #tripleo | 07:47 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/diskimage-builder: Map openjdk-7-jre-headless to RHEL+SUSE https://review.openstack.org/89531 | 07:49 |
| *** andreaf has joined #tripleo | 07:50 | |
| *** akuznetsov has quit IRC | 07:51 | |
| *** akuznetsov has joined #tripleo | 07:52 | |
| *** mrunge has joined #tripleo | 07:52 | |
| *** rcarrill` has joined #tripleo | 07:53 | |
| *** rcarrillocruz has quit IRC | 07:56 | |
| *** andreaf has quit IRC | 08:06 | |
| *** pblaho has joined #tripleo | 08:11 | |
| *** e0ne has quit IRC | 08:15 | |
| *** e0ne has joined #tripleo | 08:17 | |
| *** nosnos has quit IRC | 08:18 | |
| *** derekh has joined #tripleo | 08:18 | |
| *** jistr has joined #tripleo | 08:21 | |
| *** lucasagomes has joined #tripleo | 08:23 | |
| *** darraghb has joined #tripleo | 08:24 | |
| *** athomas has joined #tripleo | 08:39 | |
| *** andreaf has joined #tripleo | 08:40 | |
| *** markmc has joined #tripleo | 08:42 | |
| *** jtomasek has joined #tripleo | 08:45 | |
| *** viktors|afk is now known as viktors | 08:45 | |
| lifeless | StevenK: tchaypo: reminder: I'm in town wed through sunday; want to hack together work days? | 08:45 |
| lifeless | StevenK: dinner with sarah would rock; friday night or saturday would work - you choose | 08:46 |
| *** nati_ueno has quit IRC | 08:46 | |
| lifeless | tchaypo: we're likely doing yum cha too sunday am | 08:46 |
| *** martyntaylor has joined #tripleo | 08:48 | |
| *** e0ne_ has joined #tripleo | 08:48 | |
| *** dkehn has quit IRC | 08:49 | |
| StevenK | lifeless: Yes, I do -- where did you want to hack? | 08:51 |
| *** e0ne has quit IRC | 08:51 | |
| StevenK | lifeless: I'll get back to you re: dinner when I meet up with Sarah | 08:52 |
| lifeless | StevenK: lets see; have laptop, can travel. Staying at holiday inn darling harbour. | 08:52 |
| StevenK | lifeless: I can train out to you, you can train out to me, or we can meet at the HP office | 08:55 |
| *** dkehn has joined #tripleo | 08:56 | |
| lifeless | HP might work; but perhaps gatecrashing tchaypo's place might split the difference and have enough tablespace. Dunno - lets get his imput too. | 08:57 |
| StevenK | lifeless: I don't see how, given tchaypo is a lot closer to Darling Harbour :-) | 08:58 |
| *** untriaged-bot has joined #tripleo | 09:01 | |
| untriaged-bot | Untriaged bugs so far: | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1311987 | 09:01 |
| uvirtbot | Launchpad bug 1311987 in diskimage-builder "deploy-ironic ramdisks do not boot: Failed to execute /init (error -28)" [Undecided,New] | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1311986 | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1311631 | 09:01 |
| uvirtbot | Launchpad bug 1311986 in diskimage-builder "deploy-ironic ramdisks do not boot: Failed to execute /init (error -28)" [Undecided,New] | 09:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 09:01 |
| uvirtbot | Launchpad bug 1311631 in tripleo "rabbitmq os-refresh config fails on the master due to mis-parsing of nodes list" [Undecided,In progress] | 09:01 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 09:01 |
| *** untriaged-bot has quit IRC | 09:01 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Defaults TRIPLEO_ROOT to base of the current clone https://review.openstack.org/89838 | 09:05 |
| StevenK | Hmmm. Not sure why I have two screensavers post Trusty upgrade. | 09:07 |
| StevenK | lifeless: Friday night for dinner | 09:08 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 09:08 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Only use SSH keys for power mgmt if using VMs https://review.openstack.org/89787 | 09:08 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/tuskar-ui: Replace Integer to Number in Type check https://review.openstack.org/89789 | 09:08 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Only use SSH keys for power mgmt if using VMs https://review.openstack.org/89787 | 09:11 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Only use SSH keys for power mgmt if using VMs https://review.openstack.org/89787 | 09:12 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Only use SSH keys for power mgmt if using VMs https://review.openstack.org/89787 | 09:14 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 09:15 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 09:16 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 09:18 |
| *** hashar has quit IRC | 09:19 | |
| lifeless | StevenK: rendering at the same time ? | 09:31 |
| *** giulivo has joined #tripleo | 09:35 | |
| *** jprovazn has quit IRC | 09:36 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Init code for Instack Undercloud https://review.openstack.org/87951 | 09:38 |
| *** tzumainn has quit IRC | 09:45 | |
| *** tzumainn has joined #tripleo | 09:45 | |
| *** giulivo has quit IRC | 09:47 | |
| howleyt | Hi, anyone know when the support for scaling of overcloud control nodes should land? | 09:50 |
| howleyt | I realise that I can play with scaling myself by adding --scale controller to the make call, and figuring out the correct overcloud-source. yaml Just interested to know how far away it is from landing upstream. | 09:52 |
| *** hashar has joined #tripleo | 09:52 | |
| *** rcarrillocruz has joined #tripleo | 09:54 | |
| *** rcarrill` has quit IRC | 09:56 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Only use SSH keys for power mgmt if using VMs https://review.openstack.org/89787 | 09:57 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/tuskar-ui: Replace Integer to Number in Type check https://review.openstack.org/89789 | 10:10 |
| *** hashar has quit IRC | 10:10 | |
| *** jprovazn has joined #tripleo | 10:12 | |
| *** nosnos has joined #tripleo | 10:15 | |
| *** e0ne_ has quit IRC | 10:20 | |
| openstackgerrit | stephen-mulcahy proposed a change to openstack/tripleo-incubator: Support custom PyPI mirrors for setup-clienttools https://review.openstack.org/90049 | 10:32 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Replace Integer to Number in Type check https://review.openstack.org/89789 | 10:34 |
| *** jcoufal has quit IRC | 10:41 | |
| *** darraghb has quit IRC | 10:48 | |
| *** darraghb has joined #tripleo | 10:48 | |
| viktors | lifeless: hello | 10:55 |
| smulcahy | is there any obvious issue with https://review.openstack.org/90049 ? | 10:56 |
| smulcahy | It's not clear if this needs to do anything magical for easy_install edge cases or whether pip options get properly propagated down to underlying tools | 10:57 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-incubator: Adding undercloud ceilometer https://review.openstack.org/86917 | 10:59 |
| *** e0ne has joined #tripleo | 11:05 | |
| *** e0ne has quit IRC | 11:10 | |
| *** e0ne has joined #tripleo | 11:12 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 11:12 |
| *** ramishra has joined #tripleo | 11:34 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-incubator: Adding undercloud ceilometer https://review.openstack.org/86917 | 11:51 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 12:02 |
| *** hashar has joined #tripleo | 12:07 | |
| *** jdob has joined #tripleo | 12:16 | |
| anteaya | lifeless: I have submitted a pyconau talk proposal, yes | 12:24 |
| anteaya | lifeless: thanks for asking | 12:24 |
| anteaya | has everyone in tripleo voted in the tc election? | 12:24 |
| anteaya | vote vote vote | 12:24 |
| anteaya | and ask a friend | 12:24 |
| derekh | Two clouds == twice as much fun, compute node 1 and 0 on rh ci-overcloud down.... | 12:28 |
| derekh | Have rebooted compute1, os-collect-config problem, looking into it | 12:29 |
| *** rlandy has joined #tripleo | 12:31 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Include default configuration for Sphinx https://review.openstack.org/89780 | 12:38 |
| *** nosnos has quit IRC | 12:42 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 12:44 |
| Ng | morning | 12:45 |
| *** weshay has joined #tripleo | 12:52 | |
| *** spzala has joined #tripleo | 12:54 | |
| *** athomas has quit IRC | 12:58 | |
| *** NobodyCam has joined #tripleo | 12:58 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Add pacemaker and corosync as tripleo elements https://review.openstack.org/86069 | 13:01 |
| *** absubram__ has joined #tripleo | 13:01 | |
| *** athomas has joined #tripleo | 13:07 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Use Tuskar endpoint from Keystone https://review.openstack.org/90074 | 13:09 |
| *** Rakesh6 has quit IRC | 13:09 | |
| openstackgerrit | Om Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails. https://review.openstack.org/79289 | 13:09 |
| *** dprince has joined #tripleo | 13:15 | |
| *** morazi has joined #tripleo | 13:19 | |
| *** julim has joined #tripleo | 13:20 | |
| *** ramishra_ has joined #tripleo | 13:21 | |
| *** ramishra has quit IRC | 13:22 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:24 | |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 13:30 |
| *** gilliard_ has joined #tripleo | 13:31 | |
| *** lazy_prince has quit IRC | 13:35 | |
| dprince | derekh: So the teworker signals, thing. Thanks for the feedback about my thread leak... | 13:39 |
| derekh | dprince: yup | 13:39 |
| *** marun has joined #tripleo | 13:39 | |
| gilliard_ | Hi - what's the state of people's thoughts about CI on baremetal? | 13:40 |
| gilliard_ | has there been a discussion on the ML? I guess there's a lot to worry about w.r.t security. | 13:40 |
| dprince | derekh: Tinkering with it. So how do we feel about using globals in that script? I can either add a few globals, or do a more invasive refactoring to better support Threading for the signals feature | 13:40 |
| derekh | dprince: no prob, I like the idea so if can get it work asap great | 13:41 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-image-elements: sort the element-deps to please dib-lint https://review.openstack.org/90083 | 13:41 |
| derekh | dprince: I'm good with either, if the globals avoids a big refactor then go for it | 13:41 |
| dprince | derekh: let me push that real quick and see what you think | 13:42 |
| derekh | dprince: | 13:42 |
| derekh | dprince: ok | 13:42 |
| dprince | derekh: Another question for you. Do we have a good way to query what each testenv-worker process is doing? i.e. what Job number, etc it is doing? | 13:43 |
| *** CLOUDOUTAGE has joined #tripleo | 13:43 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- both ci-overclouds currently down https://etherpad.openstack.org/p/cloud-outage | 13:43 |
| *** CLOUDOUTAGE has quit IRC | 13:43 | |
| dprince | derekh: I was thinking we could have the testenv-worker modify its cmdline so that the process listing told us more meaningful info in that regard. | 13:43 |
| dprince | derekh: So something like a bin/testenv-ctl would allow us to stop/start/ and query the workers for what they are doing | 13:44 |
| derekh | dprince: it logs what was sent to it from the client, we could have the client send some better info | 13:45 |
| derekh | dprince: and also make it available through testenv-ctl | 13:45 |
| dprince | derekh: exactly. I more asking if something like this would be useful. For me... I think it would help visualize problem jobs where they occur. | 13:46 |
| *** akuznetsov has quit IRC | 13:46 | |
| dprince | derekh: we could for example put the job start time in there too... | 13:46 |
| derekh | dprince: yup, it normall parse the console logs to see where a job ran | 13:47 |
| derekh | dprince: yup, good idea | 13:47 |
| derekh | I'm working on the RH rack if anybody can take a look at HP rack it would be great, looks like we haven't had any passing tests in a while goodsquishy.com/downloads/tripleo-jobs.html | 13:53 |
| *** vinsh_away is now known as vinsh | 13:53 | |
| *** geerdest has joined #tripleo | 14:01 | |
| *** mrunge has quit IRC | 14:02 | |
| *** akuznetsov has joined #tripleo | 14:05 | |
| *** CLOUDOUTAGE has joined #tripleo | 14:14 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- both ci-overclouds currently down https://etherpad.openstack.org/p/cloud-outage | 14:14 |
| *** CLOUDOUTAGE has quit IRC | 14:14 | |
| *** rcarrill` has joined #tripleo | 14:25 | |
| *** rcarrillocruz has quit IRC | 14:27 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Set admin user tenant to 'admin' https://review.openstack.org/87302 | 14:27 |
| *** funzo has quit IRC | 14:27 | |
| *** funzo has joined #tripleo | 14:28 | |
| *** funzo is now known as Guest18818 | 14:28 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add metadata header https://review.openstack.org/82038 | 14:29 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add allow_empty to metadata header https://review.openstack.org/89597 | 14:29 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Allow multiple passes of template evaluation https://review.openstack.org/88308 | 14:29 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add mode to metadata header https://review.openstack.org/89598 | 14:29 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add cinder-lio element https://review.openstack.org/78463 | 14:31 |
| ccrouch1 | bnemec: \o/ | 14:33 |
| bnemec | ccrouch1: Indeed :-) | 14:34 |
| *** jtomasek has quit IRC | 14:35 | |
| *** Guest18818 is now known as funzo | 14:37 | |
| *** TravT has joined #tripleo | 14:37 | |
| Ng | hmm, cloud outrage eh | 14:38 |
| jang | good lord. A week on holiday and EVERYTHING CHANGES | 14:42 |
| jang | time to dust off some patches. | 14:43 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Make time out 60 mins like the Heat default. https://review.openstack.org/90099 | 14:45 |
| *** CLOUDOUTAGE has joined #tripleo | 14:45 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- both ci-overclouds currently down https://etherpad.openstack.org/p/cloud-outage | 14:45 |
| *** CLOUDOUTAGE has quit IRC | 14:45 | |
| *** hashar has quit IRC | 14:49 | |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/tripleo-ci: testenv-worker: use signals to enable/disable jobs https://review.openstack.org/87971 | 14:53 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/tripleo-ci: Add testenv stop/start processing scripts. https://review.openstack.org/87972 | 14:53 |
| *** ramishra_ has quit IRC | 14:53 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1311987 | 15:00 |
| uvirtbot | Launchpad bug 1311987 in diskimage-builder "deploy-ironic ramdisks do not boot: Failed to execute /init (error -28)" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1311631 | 15:00 |
| uvirtbot | Launchpad bug 1311631 in tripleo "rabbitmq os-refresh config fails on the master due to mis-parsing of nodes list" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 15:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1309027 | 15:00 |
| uvirtbot | Launchpad bug 1309027 in tripleo "rabbitmq nodes repeatedly reporting the master node as being 'down' then 'up'" [Undecided,Incomplete] | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add metadata header https://review.openstack.org/82038 | 15:06 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add allow_empty to metadata header https://review.openstack.org/89597 | 15:06 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Allow multiple passes of template evaluation https://review.openstack.org/88308 | 15:06 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add mode to metadata header https://review.openstack.org/89598 | 15:06 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: Create haproxy chroot directory if necessary https://review.openstack.org/90110 | 15:06 |
| *** ramishra has joined #tripleo | 15:08 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add allow_empty to metadata header https://review.openstack.org/89597 | 15:11 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Allow multiple passes of template evaluation https://review.openstack.org/88308 | 15:11 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add mode to metadata header https://review.openstack.org/89598 | 15:11 |
| *** gilliard_ has quit IRC | 15:12 | |
| *** CLOUDOUTAGE has joined #tripleo | 15:16 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- both ci-overclouds currently down https://etherpad.openstack.org/p/cloud-outage | 15:16 |
| *** CLOUDOUTAGE has quit IRC | 15:16 | |
| *** ramishra has quit IRC | 15:20 | |
| *** rcarrill` is now known as rcarrillocruz | 15:25 | |
| *** markmc has quit IRC | 15:25 | |
| NobodyCam | anyone seeing issues with DIB_REPOREF_*? I'm getting fatal: Couldn't find remote ref refs/changes/07/88307/6 /n fatal: The remote end hung up unexpectedly | 15:30 |
| NobodyCam | all of a sudden | 15:30 |
| *** dshulyak has quit IRC | 15:31 | |
| *** dshulyak has joined #tripleo | 15:33 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-heat-templates: Introduce configurable virtual ip in templates https://review.openstack.org/89556 | 15:46 |
| *** CLOUDOUTAGE has joined #tripleo | 15:47 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- both ci-overclouds currently down https://etherpad.openstack.org/p/cloud-outage | 15:47 |
| *** CLOUDOUTAGE has quit IRC | 15:47 | |
| *** akrivoka has quit IRC | 15:54 | |
| SpamapS | why does it say both overclouds are down? | 15:55 |
| SpamapS | I see running VMs and jobs? | 15:55 |
| Ng | crap, I've tried to get into that like 10 times and been distracted by calls | 15:55 |
| Ng | SpamapS: the bot just quotes the wikipage | 15:56 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-incubator: Add keepalived to overcloud, control vip to heat https://review.openstack.org/89613 | 15:56 |
| Ng | derekh: did the outrage engine just get started up again this afternoon? | 15:56 |
| SpamapS | possibly | 15:56 |
| Ng | I do keep losing my ssh to the undercloud machine, which is profoundly annoying | 15:59 |
| Ng | I guess it's my turn to have the shitty networking issue | 16:00 |
| *** lsmola has quit IRC | 16:00 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Updating ceilometer endpoint setup https://review.openstack.org/90132 | 16:04 |
| *** cwolferh has joined #tripleo | 16:04 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Updating ceilometer endpoint setup on overcloud deploy https://review.openstack.org/90132 | 16:04 |
| *** ramishra has joined #tripleo | 16:05 | |
| SpamapS | Ng: I've been getting sporadic ping failures so the insanity continues. :-P | 16:07 |
| openstackgerrit | Tom Hancock proposed a change to openstack/tripleo-incubator: Dont wait for a failed update https://review.openstack.org/90133 | 16:07 |
| derekh | SpamapS: Ng both were down, I've gotten the RH one back running, but have been on the phone ever sense | 16:07 |
| derekh | SpamapS: Ng the bot was always running but only spams irc when you uncomment the ircmessage: lline on the therpad | 16:08 |
| derekh | SpamapS: Ng so basically all jobs were failing, but now its most http://goodsquishy.com/downloads/tripleo-jobs.html | 16:09 |
| Ng | ah, doh | 16:09 |
| derekh | I'm off the phone now so will start looking into it again but will be leaving in an hour so if anybody else had time it would be great | 16:09 |
| *** eghobo has joined #tripleo | 16:11 | |
| SpamapS | looks like more compute nodes down | 16:11 |
| *** bauzas has quit IRC | 16:12 | |
| SpamapS | 8 again | 16:12 |
| * derekh is thinking we should just clean slate the hp ci-overcloud , we have done so much messing on these its hard to know whats what | 16:12 | |
| *** hashar has joined #tripleo | 16:12 | |
| SpamapS | derekh: agreed | 16:12 |
| *** csd has joined #tripleo | 16:12 | |
| SpamapS | maybe try to save the db | 16:12 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:13 | |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/tripleo-image-elements: Add rally element https://review.openstack.org/90135 | 16:14 |
| *** viktors is now known as viktors|afk | 16:14 | |
| derekh | I think lifeless wanted to finish scripting the whole process, has anybody started this? I got notes from the last time I did one but I wouldn't call it a script | 16:14 |
| SpamapS | heh | 16:15 |
| SpamapS | isn't that.. called TripleO ? | 16:15 |
| SpamapS | "scripting the deployment of OpenStack" ? ;) | 16:15 |
| SpamapS | one might laugh right now and say all we have are tools.. :) | 16:16 |
| derekh | SpamapS: :-) | 16:17 |
| derekh | the post deployment setup of test networks / infra quotas etc... | 16:17 |
| *** CLOUDOUTAGE has joined #tripleo | 16:17 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 16:17 |
| *** CLOUDOUTAGE has quit IRC | 16:17 | |
| derekh | Oh ya, I'm going to start say R1 = HP rack, and R2 = RH rack | 16:18 |
| *** ramishra has quit IRC | 16:19 | |
| *** pblaho has quit IRC | 16:19 | |
| *** athomas has quit IRC | 16:19 | |
| *** ramishra has joined #tripleo | 16:22 | |
| *** eguz has joined #tripleo | 16:23 | |
| *** newell has joined #tripleo | 16:24 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Remove superfluous double pound in _variables https://review.openstack.org/87425 | 16:24 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Update swift's selinux policies https://review.openstack.org/85539 | 16:24 |
| *** eghobo has quit IRC | 16:27 | |
| *** athomas has joined #tripleo | 16:27 | |
| SpamapS | Ok, I'm investigating novacompute8 being down again | 16:28 |
| SpamapS | Ng: I think we're going to need to report a ticket on that box. Has locked up hard twice now. | 16:28 |
| derekh | SpamapS: ok, I'll push up a WIP script to prepare a new ci-overcloud before I go (it will not work and will be messy but should be a start) | 16:29 |
| *** ramishra has quit IRC | 16:33 | |
| openstackgerrit | Duncan Thomas proposed a change to openstack/tripleo-heat-templates: Make size of loop device used by cinder LVM configurable https://review.openstack.org/88377 | 16:35 |
| SpamapS | wait.. novacompute8 is _not_ locked up | 16:35 |
| SpamapS | it just looks really slow | 16:35 |
| *** jistr has quit IRC | 16:40 | |
| SpamapS | weird.. can't seem to log in to novacompute8 w/ stack user | 16:42 |
| dprince | SpamapS: are we building w/ that element? | 16:44 |
| SpamapS | dprince: ci-overcloud yes | 16:45 |
| SpamapS | otherwise we'd have no way to login via ILO | 16:45 |
| SpamapS | AFAIK, ssh passwords are still disabled | 16:45 |
| Ng | SpamapS: ooi are you losing ssh sessions to the rack, or just getting lag? | 16:45 |
| Ng | (talking to NOC folk atm) | 16:46 |
| SpamapS | Ok so I'm seeing general networks uckage | 16:46 |
| dprince | Spamaps: Right. I use it in my own setups as well. But if for example we override DIB_COMMON_ELEMENTS with a custom value it could be getting dropped. | 16:46 |
| SpamapS | Ng: # mtr spamaps.org | 16:46 |
| SpamapS | Failed to resolve host: Name or service not known | 16:46 |
| dprince | Spamaps: just saying it is worth checking the DIB logs perhaps... | 16:46 |
| SpamapS | oh wait | 16:46 |
| SpamapS | Ng: ignore that | 16:46 |
| *** marun_ has joined #tripleo | 16:46 | |
| *** marun has quit IRC | 16:47 | |
| *** marun_ is now known as marun | 16:47 | |
| SpamapS | Ng: I am seeing network starts and stops | 16:47 |
| Ng | ok | 16:47 |
| SpamapS | Ng: so I don't ever lose connection, but I definitely see things slow down or stop working. Seems like general packet loss | 16:48 |
| Ng | from here it's consistently stoppy enough that idle sessions are timing out, and interactive lag is getting as high as 5s | 16:48 |
| SpamapS | Ng: just experienced it to the bastion | 16:48 |
| Ng | SpamapS: ok, thanks. you're a handy test case in that you're by far the closest to the rack :) | 16:48 |
| *** CLOUDOUTAGE has joined #tripleo | 16:48 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 16:48 |
| *** CLOUDOUTAGE has quit IRC | 16:48 | |
| Ng | I am infinity lightparsecs away ;) | 16:48 |
| SpamapS | Ng: aye | 16:48 |
| SpamapS | Ng: my IRC host is 11 hops away | 16:49 |
| SpamapS | my home network is 19 hops | 16:49 |
| *** hashar has quit IRC | 16:50 | |
| SpamapS | Ng: seeing the effect right now on bastion | 16:51 |
| SpamapS | Ng: from my home network which shows _no_ packet loss | 16:52 |
| Ng | SpamapS: right, that's the weird thing, no packet loss, but super wonky connections | 16:53 |
| SpamapS | Ng: http://paste.openstack.org/show/76970/ | 16:54 |
| Ng | I'd be guessing that we're going through an overloaded router/switch, but I have basically zero insight into how our network is structured, other than that it's pretty weird | 16:54 |
| SpamapS | Ng: 10% | 16:54 |
| SpamapS | Ng: and now it is snappy | 16:54 |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-image-elements: Add script to prepare a ci-overcloud https://review.openstack.org/90146 | 16:54 |
| derekh | lifeless: SpamapS: Ng just a dump of what I have ^^ for ci cloud preparation, will clean it up tomorrow but if you are redeploying you might find it helpful | 16:55 |
| SpamapS | Ng: pinging the default route.. I see spikes up to 3-6ms often | 16:55 |
| SpamapS | derekh: I think we're going to diagnose the network first | 16:55 |
| derekh | SpamapS: ok, makes sense | 16:56 |
| SpamapS | Ng: 93 packets transmitted, 83 received, 10% packet loss, time 92054ms | 16:56 |
| SpamapS | Ng: thats just pinging the default route from bastion | 16:56 |
| *** shardy is now known as shardy_afk | 16:57 | |
| SpamapS | Ng: 100 packets transmitted, 82 received, 18% packet loss, time 19958ms | 16:57 |
| SpamapS | rtt min/avg/max/mdev = 0.741/1.786/9.281/1.823 ms | 16:57 |
| SpamapS | Ng: definitely something local to the rack | 16:57 |
| SpamapS | whether it be the switch or the default route | 16:57 |
| SpamapS | or the nics themselves | 16:58 |
| Ng | SpamapS: groovy, that's super useful to know | 16:58 |
| Ng | I'll be opening a Net ticket just as soon as I can punch my way into Jira | 16:58 |
| *** jpeeler has quit IRC | 16:58 | |
| *** jpeeler has joined #tripleo | 16:59 | |
| *** jpeeler has joined #tripleo | 16:59 | |
| jang | SpamapS: is that on mellanox? | 16:59 |
| *** matty_dubs|lunch is now known as matty_dubs | 16:59 | |
| mordred | Ng: while you're filing tickets, you should tell them to put in the cache controller | 17:00 |
| SpamapS | jang: yes, with Trusty kernel | 17:00 |
| SpamapS | jang: which has the 2.x drivers | 17:00 |
| Ng | mordred: the whatnow? | 17:00 |
| mordred | SpamapS: I believe your rack does not have any hw caching on the raid controllers, right? | 17:01 |
| SpamapS | mordred: correct | 17:01 |
| mordred | apparently it's a known thing and can be remedied by a known solution | 17:01 |
| *** e0ne has quit IRC | 17:01 | |
| mordred | it just has to be requested | 17:01 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Remove quotas on seed and undercloud's nova https://review.openstack.org/87126 | 17:01 |
| Ng | oh, we don't have the bbwc modules? | 17:01 |
| mordred | right | 17:02 |
| Ng | awesome | 17:02 |
| SpamapS | watching ping -c 100 -i 0.2 $defaultroute run ... it pauses almost exactly 1 time per second | 17:03 |
| *** darraghb has quit IRC | 17:04 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Allows user-specified PXE timeout for overcloud https://review.openstack.org/86605 | 17:06 |
| *** martyntaylor has quit IRC | 17:07 | |
| SpamapS | Ng: it has gotten much worse now | 17:08 |
| SpamapS | hopefully because they're doing something | 17:09 |
| mordred | hey guys - if I put a script in a bin/ dir in an element and I want to use it in an install.d script | 17:09 |
| mordred | is that a thing I do? | 17:09 |
| mordred | because it doesn't seem to work that way really | 17:09 |
| SpamapS | mordred: you'd need to install it first | 17:11 |
| SpamapS | mordred: bin is not magical | 17:11 |
| mordred | SpamapS: any examples of doing that I could look at? | 17:12 |
| NobodyCam | has anyone been able to successfully use DIB_REPOLOCATION_ & DIB_REPOREF_ today? | 17:12 |
| *** morganfainberg_Z is now known as morganfainberg | 17:12 | |
| SpamapS | mordred: install -m 0755 -o root -g root $(dirname $0)/../bin/foo /usr/local/bin | 17:13 |
| mordred | SpamapS: in where? in pre-install.d ? | 17:13 |
| mordred | like, I'm struggling with the layer context here for a sec? | 17:14 |
| SpamapS | mordred: in install.d | 17:14 |
| mordred | SpamapS: how does install.d see it? | 17:14 |
| SpamapS | mordred: before you use the script, install the script | 17:14 |
| mordred | I mean, by what path can install.d reference the script? | 17:14 |
| mordred | if it's not in the image context already | 17:14 |
| SpamapS | $(dirname $0)/.. <-- the root of your element | 17:15 |
| *** derekh has quit IRC | 17:15 | |
| SpamapS | actually, that is the root of all elements | 17:15 |
| SpamapS | as they're all copied into one space.. but for what you want.. that works | 17:15 |
| *** lucasagomes has quit IRC | 17:15 | |
| SpamapS | Ng: bastion seems dead | 17:16 |
| Ng | SpamapS: crap, I was about to ask if you still have a shell on it, to get me its default route IP, because I can't log in :/ | 17:16 |
| Ng | but yes, I agree, it seems to be unreachable | 17:16 |
| SpamapS | Ng: 138.35.77.1 | 17:16 |
| SpamapS | Ng: it is actually occasionally responding | 17:16 |
| SpamapS | Proto Recv-Q Send-Q Local Address Foreign Address State | 17:17 |
| SpamapS | tcp 0 1392 184.105.137.237:39345 138.35.77.10:22 ESTABLISHED | 17:17 |
| SpamapS | Ng: taking about 30s to flush that Send-Q | 17:17 |
| SpamapS | ahh and it's back | 17:17 |
| SpamapS | [108211.278851] ipmitool[9540]: segfault at 160 ip 0000000000415265 sp 00007fff2ccf1cb0 error 6 in ipmitool[400000+9a000] | 17:17 |
| SpamapS | Ng: ok so I tweaked 'ethtool -K em3 gro off' and everything went to hell.. 'gro on' brought it back | 17:18 |
| SpamapS | Ng: so it may be that newer drivers _must_ have GRO | 17:19 |
| SpamapS | I think that is a red herring though. | 17:19 |
| *** CLOUDOUTAGE has joined #tripleo | 17:19 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 17:19 |
| *** CLOUDOUTAGE has quit IRC | 17:19 | |
| SpamapS | with it back on.. pings still stalling every few seconds | 17:19 |
| SpamapS | could be the router throttling pings tho | 17:19 |
| Ng | yeah pings are a terrible diagnostic tool | 17:20 |
| Ng | I sort of discounted the occasional ipmitool segfaults, they're happening pretty rarely and really shouldn't cause the kernel to fail to network | 17:20 |
| SpamapS | Ng: well long Send-Q's suggest it is not ping's fault | 17:21 |
| SpamapS | as in, the router is actually sucking | 17:21 |
| SpamapS | Ng: so is NOC looking into things? | 17:21 |
| SpamapS | jang: [ 10.461980] mlx4_en: Mellanox ConnectX HCA Ethernet driver v2.2-1 (Feb 2014) | 17:23 |
| SpamapS | jang: I believe that is the driver that works o-k, yes? | 17:23 |
| Ng | SpamapS: they are | 17:23 |
| SpamapS | Ng: ok, I'll lay off for a bit | 17:24 |
| *** panda has quit IRC | 17:30 | |
| *** panda has joined #tripleo | 17:31 | |
| *** akuznetsov has quit IRC | 17:32 | |
| *** akuznetsov has joined #tripleo | 17:32 | |
| mordred | andybody know what an equiv of linux-headers-generic on fedora might be? | 17:35 |
| *** eguz has quit IRC | 17:35 | |
| *** eghobo has joined #tripleo | 17:36 | |
| greghaynes | kernel-headers I think | 17:37 |
| greghaynes | https://apps.fedoraproject.org/packages/s/kernel-headers | 17:37 |
| greghaynes | Only because I checked before making that comment :p | 17:37 |
| *** epim has joined #tripleo | 17:40 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/os-apply-config: Use FHS-compliant default template location https://review.openstack.org/89668 | 17:42 |
| openstackgerrit | Ben Nemec proposed a change to openstack/os-refresh-config: Use FHS-compliant default base directory https://review.openstack.org/89667 | 17:42 |
| *** jtomasek has joined #tripleo | 17:43 | |
| mordred | greghaynes: thanks! | 17:43 |
| mordred | greghaynes: (I'm slowly reading and working through your review comments btw - thanks - they were very helful) | 17:43 |
| mordred | greghaynes: I assume I want to make a dib patch that does the kernel-headers mapping, right? | 17:43 |
| greghaynes | Awesome | 17:43 |
| greghaynes | Yep | 17:43 |
| openstackgerrit | Monty Taylor proposed a change to openstack/diskimage-builder: Add a mapping for kernel headers https://review.openstack.org/90164 | 17:45 |
| SpamapS | they're going to bounce the switch R1 is on | 17:46 |
| * Ng untypes that | 17:47 | |
| Ng | expected downtime of the switch is 3 mins | 17:47 |
| *** jprovazn has quit IRC | 17:47 | |
| mordred | greghaynes: how do I avoid running 10-support-physical-hardware? | 17:48 |
| *** CLOUDOUTAGE has joined #tripleo | 17:50 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 17:50 |
| *** CLOUDOUTAGE has quit IRC | 17:50 | |
| greghaynes | mordred: hrmm, looks like wed need to make that configurable somehow. probably by env var | 17:51 |
| mordred | greghaynes: ok. I can ignore for now - I don't think it's hurting anything | 17:51 |
| *** nati_ueno has joined #tripleo | 17:52 | |
| mordred | greghaynes: there's a thign like install-packages for ensuring services are/are not running right? | 17:53 |
| slagle | there's a os-svc-{enable,restart} | 17:55 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/tripleo-ci: testenv-worker: use signals to enable/disable jobs https://review.openstack.org/87971 | 17:55 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/tripleo-ci: Add testenv-ctl to stop/start job processing https://review.openstack.org/87972 | 17:55 |
| *** lucas-afk has joined #tripleo | 17:55 | |
| slagle | don't think there is a disable or stop though | 17:55 |
| *** andreaf has quit IRC | 18:02 | |
| *** jcoufal has joined #tripleo | 18:03 | |
| *** e0ne has joined #tripleo | 18:03 | |
| mordred | (in this case, I've installed a package, but because $debian_insanity, it's started the service - but the service should not be running) | 18:05 |
| *** ifarkas has quit IRC | 18:05 | |
| SpamapS | mordred: the image build disables invoke-rc.d during image build | 18:05 |
| mordred | ok. then maybe my problem is something else | 18:06 |
| SpamapS | mordred: disabling services is not abstracted between OS's though, so you will need to have a conditional based on the output of dib-init-system | 18:06 |
| *** absubram__ has quit IRC | 18:06 | |
| mordred | well, I want the service enabled when I boot the image | 18:06 |
| SpamapS | ok it will be | 18:06 |
| mordred | I just don't want it running during image build | 18:06 |
| SpamapS | yeah, invoke-rc.d will refuse to start it | 18:06 |
| *** nati_ueno has quit IRC | 18:06 | |
| mordred | great. | 18:06 |
| * mordred tries again | 18:06 | |
| mordred | SpamapS: I figure you guys have most likely run in to and dealt with 99% of the things I'm doing, which is why I keep asking question - I pretty much assume there is an answer and I just don't know about it | 18:09 |
| mordred | with the one exception of that what I really want to do is magically cache all of the things puppet wants to install in teh image and you haven't solved that for me yet :) | 18:09 |
| *** athomas has quit IRC | 18:11 | |
| SpamapS | mordred: why does puppet want to install things after image build? | 18:12 |
| dprince | mordred: you just need to build your images with a local package mirror in them, thats all | 18:13 |
| *** nati_ueno has joined #tripleo | 18:13 | |
| mordred | SpamapS: it doesn't | 18:14 |
| *** nati_ueno has quit IRC | 18:14 | |
| mordred | SpamapS: it wants to install things during image build | 18:14 |
| mordred | dprince: aroo? | 18:14 |
| mordred | dprince: that sounds like perhaps it's a thing I want | 18:14 |
| SpamapS | mordred: ah, so you need something more sophisticated that a squid? | 18:17 |
| * SpamapS just had an image of a person walking into the book store from Portlandia and asking for a book about cephalapods.. "Ah.. so you need something more sophisticated than a squid?" | 18:18 | |
| *** jcoufal has quit IRC | 18:19 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack/diskimage-builder: Revert default Ubuntu release back to Saucy https://review.openstack.org/90177 | 18:19 |
| SpamapS | adam_g: nooooooooooooooooooooooooooooooooooooooooooooooooooooooo | 18:20 |
| adam_g | SpamapS, :| | 18:20 |
| adam_g | i'm happy to work on figuring out whats wrong with trusty, but in the meantime... | 18:20 |
| SpamapS | Actually that begs the question | 18:20 |
| SpamapS | why aren't we building ramdisks in CI? | 18:20 |
| adam_g | SpamapS, im not sure. aside from the trusty issue, im waiting on one merge to devstack before we have a functional ironic devstack gate check that exercises DIB and would have caught that. i think it should be part of the DIB check pipeline | 18:21 |
| *** CLOUDOUTAGE has joined #tripleo | 18:21 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 18:21 |
| *** CLOUDOUTAGE has quit IRC | 18:21 | |
| *** e0ne has quit IRC | 18:21 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: sort the element-deps to please dib-lint https://review.openstack.org/90083 | 18:22 |
| *** darraghb has joined #tripleo | 18:22 | |
| *** e0ne has joined #tripleo | 18:22 | |
| *** nati_ueno has joined #tripleo | 18:22 | |
| mordred | SpamapS: I don't understand what you're saying - are you saying that the answer is "run a squid on your laptop" - or that there is a feature or dib called "local package mirror" that I have not enabled? | 18:23 |
| SpamapS | mordred: run a squid on your laptop | 18:24 |
| *** epim has quit IRC | 18:24 | |
| SpamapS | mordred: but there is a debian-mirror element that will setup an image which, on boot, will build and expose a debian/ubuntu/etc. mirror | 18:25 |
| *** darraghb has quit IRC | 18:25 | |
| mordred | nah - neither of those thigns are what I'm looking for here - but thanks! | 18:25 |
| *** rcarrill` has joined #tripleo | 18:25 | |
| SpamapS | mordred: you want to prepopulate /var/cache/apt ? | 18:25 |
| mordred | yeah | 18:25 |
| SpamapS | mordred: that could very easily be done | 18:25 |
| mordred | yah- I'll work on that once this bad-boy is done | 18:26 |
| bnemec | So I'm curious - what's our policy on the package mappings for distros not in the gate? | 18:26 |
| SpamapS | cleanup.d -> copy the contents somewhere with a name that is locatable using the same arguments | 18:26 |
| SpamapS | extra-data.d -> look for that locatable name and pull it into the image | 18:26 |
| bnemec | For example, opensuse and rhel both have map-packages too. | 18:26 |
| mordred | SpamapS: yup! | 18:26 |
| SpamapS | bnemec: get 'em in the gate. ;) | 18:26 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: MySQL element - correct os-svc-restart arguments https://review.openstack.org/90179 | 18:27 |
| *** epim has joined #tripleo | 18:27 | |
| bnemec | SpamapS: Works for me. Just wanted to make sure I wasn't off base there. :-) | 18:27 |
| *** rcarrillocruz has quit IRC | 18:27 | |
| *** jml has quit IRC | 18:28 | |
| mordred | SpamapS: right now I'm going for functional. once that's done, I'll start working on efficiency boost | 18:28 |
| SpamapS | bnemec: I also think we're not in a position to start kicking things out because they're not in the gate, until we get ourselves _fully_ into the gate and we have stable CI | 18:28 |
| *** blamar_ has joined #tripleo | 18:29 | |
| SpamapS | mordred: hah, we said that a year ago for TripleO ... "we'll optimize.. someday..." ;) | 18:29 |
| *** blamar_ is now known as blamar | 18:29 | |
| mordred | SpamapS: oh - I'll optimize - it'll just likely be saturday :) | 18:29 |
| bnemec | SpamapS: Yeah, as someone who just added support for rhel7 I'm not looking to kick anything out for not being in CI. ;-) | 18:30 |
| *** darraghb has joined #tripleo | 18:30 | |
| *** derekh has joined #tripleo | 18:31 | |
| NobodyCam | fyi reverting this: https://github.com/openstack/diskimage-builder/commit/f8416b1fe5905d8e9796aba51372ce33556bdfb5 seems to have fixed me issues with DIB_REPOLOCATION_ & DIB_REPOREF_ | 18:31 |
| *** jml has joined #tripleo | 18:31 | |
| *** dshulyak_ has joined #tripleo | 18:32 | |
| *** ramishra has joined #tripleo | 18:34 | |
| *** darraghb has quit IRC | 18:34 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Move files common for mysql and mariadb under mysql-common https://review.openstack.org/86870 | 18:36 |
| *** ramishra has quit IRC | 18:39 | |
| *** martyntaylor has joined #tripleo | 18:47 | |
| *** hashar has joined #tripleo | 18:49 | |
| *** nati_ueno has quit IRC | 18:49 | |
| Ng | NobodyCam: nuts, I really liked that commit | 18:49 |
| NobodyCam | could just be my env... | 18:49 |
| NobodyCam | if no one else is having issues | 18:50 |
| NobodyCam | :-p | 18:50 |
| *** sballe has quit IRC | 18:50 | |
| NobodyCam | but it broke my ablity to use DIB_REPOLOCATION_ & DIB_REPOREF_ | 18:50 |
| NobodyCam | reverting it seems to have things working again ... at least for me | 18:51 |
| *** martyntaylor has left #tripleo | 18:51 | |
| *** CLOUDOUTAGE has joined #tripleo | 18:52 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 18:52 |
| *** CLOUDOUTAGE has quit IRC | 18:52 | |
| *** ramishra has joined #tripleo | 18:55 | |
| *** ramishra has quit IRC | 19:00 | |
| *** sballe has joined #tripleo | 19:00 | |
| *** BadCub has joined #tripleo | 19:02 | |
| *** epim has quit IRC | 19:03 | |
| *** StevenK has quit IRC | 19:07 | |
| *** StevenK has joined #tripleo | 19:07 | |
| *** victor_lowther has quit IRC | 19:08 | |
| *** victor_lowther has joined #tripleo | 19:09 | |
| *** blamar has quit IRC | 19:12 | |
| *** CLOUDOUTAGE has joined #tripleo | 19:23 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 19:23 |
| *** CLOUDOUTAGE has quit IRC | 19:23 | |
| *** nati_ueno has joined #tripleo | 19:30 | |
| *** dshulyak_ has quit IRC | 19:32 | |
| *** saurabhs has joined #tripleo | 19:36 | |
| SpamapS | I reset ILO on novacompute8 ... since it has not been working. | 19:38 |
| *** sdague has quit IRC | 19:41 | |
| *** sdague has joined #tripleo | 19:42 | |
| lifeless | run roh | 19:43 |
| SpamapS | lifeless: yeah, so novacompute8, which I thought was locked up yesterday, just had a very unresponsive ilo | 19:44 |
| SpamapS | lifeless: are ilos their own distinct port or do they share with the mellanox? | 19:44 |
| Ng | distinct port | 19:44 |
| SpamapS | The ilo for that box, even after reset, is almost totally unusable | 19:44 |
| Ng | lifeless: just to catch you up a bit, we've been having fun with the NOC folk, resetting switches and suchlike, to try and get to the bottom of the weird network issue | 19:44 |
| SpamapS | you can hit enter 10 times and only one enter will be registered | 19:44 |
| Ng | s/NOC/NetworkEngineering/ | 19:45 |
| lifeless | mordred: padakges? I run an apt-mirror mirror | 19:45 |
| lifeless | SpamapS: distinct port and distinct CPU running distinct OS | 19:45 |
| lifeless | ok | 19:46 |
| lifeless | so today is a public holiday here; while there are folk on deck I'm going to honour that :) | 19:46 |
| SpamapS | heh, I know what an ilo is. But some models make the ilo share w/ the first nic. | 19:46 |
| lifeless | if at EOD USA its still dead jim I'll jump on it | 19:46 |
| lifeless | SpamapS: sorry, wasn't quite sure of the question :P | 19:46 |
| SpamapS | ok I'm on novacompute8 now.. | 19:47 |
| *** hashar has quit IRC | 19:47 | |
| SpamapS | can't ping gateway | 19:48 |
| SpamapS | but can ping some other compute nodes | 19:48 |
| *** dprince has quit IRC | 19:51 | |
| SpamapS | hm | 19:52 |
| SpamapS | I wonder if we have a bad NIC | 19:52 |
| SpamapS | link down/up fixed the problem | 19:52 |
| SpamapS | lifeless: so we were toying with the idea of deploying a new ci-overcloud | 19:53 |
| *** geerdest has quit IRC | 19:53 | |
| lifeless | SpamapS: mellanox driver version on compute8 ? | 19:53 |
| openstackgerrit | Jufang Wang proposed a change to openstack/tripleo-image-elements: configure keystone with apache https://review.openstack.org/89744 | 19:53 |
| lifeless | SpamapS: yes! new cloud needs finished deployment code because I'm damned if I want another round of manual fixups | 19:54 |
| lifeless | SpamapS: however if you want a close-eyes-its-horrible-but-have-to-do-it we could upgrade to trusty | 19:54 |
| *** CLOUDOUTAGE has joined #tripleo | 19:54 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 19:54 |
| *** CLOUDOUTAGE has quit IRC | 19:54 | |
| lifeless | SpamapS: ... manually | 19:54 |
| lifeless | which is: | 19:55 |
| lifeless | - sudo do-release-upgrade | 19:55 |
| lifeless | - copy the initrd and vmlinuz to kernel & ramdisk of the instance uuid on the undercloud tftproot | 19:55 |
| lifeless | - reboot | 19:55 |
| SpamapS | lifeless: Right.. I would prefer if we could avoid that. | 19:56 |
| SpamapS | lifeless: and also, novacompute8 has old mellanox | 19:56 |
| *** ramishra has joined #tripleo | 19:56 | |
| lifeless | SpamapS: dmesg - and sa_allocs or cmd_alloc or whatever it is we see ? | 19:57 |
| *** julim has quit IRC | 19:58 | |
| SpamapS | [26463.763436] hpsa 0000:06:00.0: cmd_alloc returned NULL! | 19:59 |
| SpamapS | that is about 20 hours ago | 19:59 |
| lifeless | taaaa daaaaaa | 20:00 |
| SpamapS | lifeless: I'm more inclined to upgrade mellanox driver manually and focus on how we could actually do a roll out of trusty using our own tools than do-r-u'ing them all | 20:00 |
| SpamapS | lifeless: adam_g tells me that trusty is breaking ironic ramdisk btw | 20:00 |
| adam_g | SpamapS, actually im looking at that now | 20:01 |
| *** ramishra has quit IRC | 20:01 | |
| derekh | lifeless: I started a script to prepare the ci-overcloud earlier I should get it finished tomorrow (more of a brain dump at the moment) https://review.openstack.org/#/c/90146/ | 20:01 |
| adam_g | SpamapS, seems to work fine on machines /w >= ~650MB of memory, but failing /w my default 512MB nodes | 20:01 |
| SpamapS | interesting | 20:02 |
| SpamapS | our ramdisk isn't.. that big is it? | 20:02 |
| SpamapS | -rw-r--r-- 1 clint clint 81M Apr 24 11:55 deploy-ramdisk.initramfs | 20:03 |
| SpamapS | not sure that -28 == 28 == ENOSPC | 20:03 |
| adam_g | SpamapS, is that a trusty or saucy? | 20:03 |
| SpamapS | adam_g: trusty | 20:03 |
| adam_g | SpamapS, re ENOSPC, increasing memory on the VM fixes the issue. so.. | 20:04 |
| SpamapS | adam_g: yeah that certainly points to it .. doh | 20:05 |
| SpamapS | [166854.436481] hpsa 0000:06:00.0: cmd_alloc returned NULL! | 20:06 |
| SpamapS | and just got one | 20:06 |
| *** epim has joined #tripleo | 20:09 | |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: MySQL Element - Support Debian libssl1.0.0 paths https://review.openstack.org/90223 | 20:11 |
| mordred | lifeless: if I ran an apt-mirror mirror on my laptop, how would I funnel activity in the image build through that but not have my laptop's apt-mirror thing be configured in the image so that when I boot it in a cloud and not my laptop apt still works? | 20:12 |
| SpamapS | mordred: elements/apt-mirror | 20:12 |
| SpamapS | errr | 20:13 |
| SpamapS | apt-sources, and apt-conf | 20:13 |
| SpamapS | actually neither of those cleans up | 20:14 |
| SpamapS | mordred: I don't think anybody has made an apt source changer that changes it back to the default after | 20:15 |
| SpamapS | mordred: note that there is considerable effort put into utilizing http_proxy but then not storing it in the image | 20:16 |
| *** epim has quit IRC | 20:16 | |
| SpamapS | ok lunch time | 20:16 |
| mordred | SpamapS: yeah - I think this is still too much and not enough at the same time | 20:18 |
| mordred | btw: | 20:19 |
| mordred | For APT based systems, if you require specific options for apt operations, | 20:19 |
| mordred | you can define `DIB_APT_CONF` with your favorite apt.conf to override it, | 20:19 |
| mordred | before running devtest.sh. | 20:19 |
| mordred | that's in elements/apt-conf/README.md | 20:19 |
| mordred | in diskimage-builder | 20:19 |
| mordred | but devtest is a script in tripleo-incubator - and it's possible people might be using diskimage-builder for other things | 20:20 |
| mordred | just saying :) | 20:20 |
| greghaynes | hah, good point | 20:20 |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Disable splashimage for legacy grub. https://review.openstack.org/84973 | 20:20 |
| *** akrivoka has joined #tripleo | 20:21 | |
| greghaynes | what, no splashimage!? | 20:22 |
| mordred | wow. | 20:22 |
| Ng | gerrit needs to give me a +OMGYES for patches that remove serial/textmode issues | 20:22 |
| SpamapS | mordred: Agreed. I tend to -1 those changes if I notice. | 20:23 |
| mordred | SpamapS: http_proxy support really blows | 20:23 |
| SpamapS | mordred: eh? | 20:23 |
| * mordred just tried to run dib while on the corp network - already giving up and I'll just do it later | 20:23 | |
| mordred | oh. wait | 20:23 |
| mordred | nevermind | 20:23 |
| mordred | I actually suck | 20:23 |
| SpamapS | mordred: for corp network, I have two lines in my squid.conf that I have to change | 20:23 |
| mordred | I'm not actually on the corp network | 20:23 |
| SpamapS | # Both of these for times on HP corporate network | 20:24 |
| SpamapS | #cache_peer web-proxy.cup.hp.com parent 8080 0 default | 20:24 |
| SpamapS | #never_direct allow all | 20:24 |
| SpamapS | the cup.hp.com is actually not needed btw | 20:24 |
| SpamapS | web-proxy always chooses the closest one | 20:24 |
| mordred | I'm tethered to my phone, but I'm sitting in the office so I set the proxy settings | 20:24 |
| SpamapS | mordred: :) | 20:24 |
| SpamapS | mordred: so what I do is just always have everything on my laptop pointed at my local squid | 20:24 |
| *** CLOUDOUTAGE has joined #tripleo | 20:24 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 20:24 |
| *** CLOUDOUTAGE has quit IRC | 20:24 | |
| SpamapS | mordred: and then when on corp network, I uncomment those things, and then my local squid goes through corporate evil | 20:25 |
| SpamapS | anyway, tummy rumbling | 20:25 |
| SpamapS | and only 35 minutes of indian buffet left | 20:25 |
| Ng | mordred: devtest runs through tethering? brave :) | 20:25 |
| SpamapS | Ng: been there, done htat. ;) | 20:26 |
| *** scott-millward has joined #tripleo | 20:27 | |
| mordred | Ng: not devtest | 20:28 |
| mordred | Ng: I'm working on getting nodepool to use dib instead of the template/snap method we use right now | 20:28 |
| mordred | so I'm the new crazy customer in the corner with a whole new set of use cases | 20:28 |
| Ng | aha :) | 20:29 |
| Ng | SpamapS: Vince is ready to reload the switch fabric. will take 10 mins. I say go for it, what say you? | 20:29 |
| *** epim has joined #tripleo | 20:31 | |
| mordred | ok. I AM doing dib tethered through my phone | 20:34 |
| mordred | and, sadly, I think it's working better than my home wifi | 20:34 |
| pleia2 | :\ | 20:34 |
| *** weshay has quit IRC | 20:34 | |
| lifeless | SpamapS: okydoky | 20:35 |
| Ng | mordred: if only you lived in the centre of the technology universe or something :/ | 20:35 |
| mordred | Ng: right? | 20:37 |
| *** lucas-afk has quit IRC | 20:37 | |
| Ng | oh, duh, SpamapS went to fill his face with indian food | 20:38 |
| Ng | unilateral decision time! | 20:38 |
| Ng | we're upgrading the ci cloud's switch fabric :) | 20:38 |
| *** derekh has quit IRC | 20:39 | |
| *** marun has quit IRC | 20:40 | |
| lifeless | Ng: schwaeet | 20:40 |
| clarkb | mordred: have you tried not using wireless N? or another AP? | 20:43 |
| clarkb | sometimes there are weird interactions between particular drivers and devices and APs. I once had a router that would hard crash if a particular intel chipset spoke to it on wireless N | 20:44 |
| Ng | the easy solution is to just buy apple computers and APs ;) | 20:45 |
| Ng | I mean HP computers and APs! We probably make APs, right? | 20:45 |
| lifeless | clarkb: many people had that :) | 20:47 |
| clarkb | lifeless: it is good to know I wasnt' the only one | 20:47 |
| lifeless | Ng: we probably subcontract that to OEMs | 20:47 |
| Ng | lifeless: sure, but it increases the chances that someone has actually tested them together | 20:49 |
| mordred | SpamapS: ugh | 20:50 |
| mordred | err: /Stage[main]/Iptables/Service[iptables]: Failed to call refresh: Could not start Service[iptables]: Execution of '/etc/init.d/iptables-persistent start' returned 1: at /opt/git/openstack-infra/config/modules/iptables/manifests/init.pp:32 | 20:50 |
| mordred | SpamapS: the invoke-rc.d policy thing doesn't prevent my puppet from doing things :( | 20:50 |
| mordred | clarkb: so ... unbound is screwing me in dib - because it's installing, and it's installing an /etc/resolv.conf - but it's not actually running the service | 20:52 |
| *** jdob has quit IRC | 20:52 | |
| mordred | so subsequent operations can't do dns resolution anymore | 20:52 |
| *** lucas-afk has joined #tripleo | 20:53 | |
| greghaynes | sounds like unbound element is needed | 20:53 |
| mordred | well, put we manage unbound via puppet | 20:53 |
| greghaynes | I can see people wanting that | 20:53 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Add package uninstall support https://review.openstack.org/90228 | 20:53 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Support declarative package installs/uninstalls https://review.openstack.org/90229 | 20:53 |
| clarkb | mordred: oh right because it says resolve against localhost | 20:53 |
| mordred | this is the fun place we're in here - is the mixing of tools | 20:53 |
| mordred | clarkb: :) | 20:53 |
| lifeless | mordred: ouch, puppet is being very naughty there | 20:53 |
| clarkb | mordred: maybe you need to have for extra.d run a resolver on localhost? | 20:53 |
| clarkb | lifeless: why is that naughty? | 20:53 |
| mordred | clarkb: he's talking about the error above | 20:54 |
| clarkb | oh | 20:54 |
| mordred | which is fine, I can fix that one | 20:54 |
| lifeless | clarkb: the iptables-persistent thing | 20:54 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Support declarative package installs/uninstalls https://review.openstack.org/90229 | 20:54 |
| *** CLOUDOUTAGE has joined #tripleo | 20:55 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 20:55 |
| *** CLOUDOUTAGE has quit IRC | 20:55 | |
| Ng | if nothing else, it's a bit annoying that puppt is calling the init script directly, instead of using service(1) or whatever | 20:56 |
| Ng | at least that could be hobbled during builds | 20:56 |
| mordred | yah. no, that one is one that's easy to fix | 20:56 |
| mordred | the unbound issue is much harder | 20:56 |
| mordred | because I need the node, when it boots up, to be prepped with the unbound - but dib is preventing unbound from running during the rest of image build | 20:57 |
| *** ramishra has joined #tripleo | 20:57 | |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Add package uninstall support https://review.openstack.org/90228 | 20:57 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Support declarative package installs/uninstalls https://review.openstack.org/90229 | 20:57 |
| mordred | and I could, as greghaynes points out, do an unbound element - and could even parameterized the puppet install of unbound out | 20:57 |
| mordred | but then I'm managing that in two places | 20:57 |
| * mordred cries | 20:57 | |
| greghaynes | Yea.. seems like you really need to do some config post-boot with that one so not sure of a better way :/ | 20:58 |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1311987 | 21:00 |
| uvirtbot | Launchpad bug 1311987 in diskimage-builder "deploy-ironic ramdisks do not boot: Failed to execute /init (error -28)" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1311631 | 21:00 |
| uvirtbot | Launchpad bug 1311631 in tripleo "rabbitmq os-refresh config fails on the master due to mis-parsing of nodes list" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 21:00 |
| lifeless | what does unbound specifically do that breaks things? | 21:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1309027 | 21:00 |
| uvirtbot | Launchpad bug 1309027 in tripleo "rabbitmq nodes repeatedly reporting the master node as being 'down' then 'up'" [Undecided,Incomplete] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| lifeless | like, shouldn't it be resolvconf integrated, so it dtrt at runtime ? | 21:00 |
| * lifeless probably isn't helping | 21:01 | |
| lifeless | and I need to run | 21:01 |
| lifeless | ciao :) | 21:01 |
| clarkb | we stopped doing resolveconf because it was broken and not on centos | 21:01 |
| clarkb | iirc | 21:01 |
| clarkb | would need to reread git log | 21:01 |
| *** ramishra has quit IRC | 21:02 | |
| *** jml has quit IRC | 21:16 | |
| *** jml has joined #tripleo | 21:17 | |
| *** hashar has joined #tripleo | 21:18 | |
| *** akrivoka has quit IRC | 21:18 | |
| *** e0ne has quit IRC | 21:21 | |
| *** yamahata has quit IRC | 21:24 | |
| *** epim has quit IRC | 21:26 | |
| *** CLOUDOUTAGE has joined #tripleo | 21:27 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 21:27 |
| *** CLOUDOUTAGE has quit IRC | 21:27 | |
| *** epim has joined #tripleo | 21:30 | |
| *** nati_ueno has quit IRC | 21:33 | |
| * SpamapS returns | 21:37 | |
| *** hashar has quit IRC | 21:40 | |
| SpamapS | mordred: I think wherever you cross paths with the fairly broken thing that is "puppet does build-y things at run time".. you're going to end up having to do things differently. | 21:41 |
| hewbrocca | and run-y things at build time | 21:41 |
| hewbrocca | just to make matters worse | 21:41 |
| bnemec | github recommends I name the new dib-run-parts repo bearded-octo-ninja :-) | 21:41 |
| SpamapS | mordred: the whole point of starting with "not puppet/chef/salt" in TripleO is that we are forced to do build things at build time, and config things at config time. | 21:41 |
| * bnemec is tempted to take its suggestion | 21:41 | |
| SpamapS | hewbrocca: that's the worst really | 21:41 |
| SpamapS | bnemec: doooooooiitt | 21:41 |
| hewbrocca | it's a horrible nightmare | 21:42 |
| * hewbrocca no puppet fan | 21:42 | |
| SpamapS | hewbrocca: worse still, state-y things at build time. | 21:42 |
| Ng | ensure => failure | 21:43 |
| SpamapS | #winning | 21:43 |
| hewbrocca | LOL | 21:44 |
| *** nibalizer has joined #tripleo | 21:48 | |
| nibalizer | greghaynes: hi | 21:48 |
| greghaynes | hai | 21:49 |
| mordred | SpamapS: right. so, refactoring teh whole system is out of scope | 21:50 |
| mordred | right now, the ONLY thing I'm doing it changing image creation tools | 21:50 |
| mordred | rewriting the puppet is not on the table, for better or for worse | 21:50 |
| *** e0ne has joined #tripleo | 21:52 | |
| *** derekh has joined #tripleo | 21:52 | |
| SpamapS | mordred: you don't have to refactor the whole system. The bits of puppet that are doing build-time things could be separated and deactivated during engine builds though. | 21:53 |
| mordred | SpamapS: right - thign is - we use the unbound module on ALL of our servers | 21:54 |
| mordred | SpamapS: not just the slaves | 21:54 |
| mordred | so changes to it affect thigns that aren't related to dib thigns | 21:54 |
| SpamapS | I'm rusty on my puppet .. but you can conditionally do pieces, right? | 21:54 |
| SpamapS | So like "if $BUILDING_IMAGE then noop else do as usual " ? | 21:55 |
| mordred | https://review.openstack.org/90234 | 21:55 |
| mordred | you mean like this? | 21:55 |
| SpamapS | yes! :) | 21:55 |
| mordred | I grok what you're saying, and am trying to get it sanely | 21:55 |
| SpamapS | is that even something you could plug into facter? | 21:55 |
| mordred | but what I DON'T want to do is have a bunch of craptastic puppet with a billion conditionals for if dib / if not dib | 21:56 |
| mordred | because that's very hard to reason about | 21:56 |
| mordred | I think this one is potentially not aweful | 21:56 |
| SpamapS | mordred: https://www.youtube.com/watch?v=sw2B9knw58U <-- and now you've arrived at the same conclusion she did.. | 21:57 |
| greghaynes | Does the puppet unbound code actually install the resolv.conf or is that just a result of the unbound package? | 21:57 |
| *** e0ne has quit IRC | 21:57 | |
| mordred | greghaynes: our puppet module does | 21:57 |
| mordred | so the workaroudn here is either: | 21:57 |
| greghaynes | :( Was thinking maybe you could get clever and install unbound via element before puppet does but there goes that idea | 21:57 |
| *** ramishra has joined #tripleo | 21:58 | |
| *** CLOUDOUTAGE has joined #tripleo | 21:58 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 21:58 |
| *** CLOUDOUTAGE has quit IRC | 21:58 | |
| mordred | a) block the resolv.conf install in puppet and then follow it up in a 99 install.d script | 21:58 |
| mordred | b) run a local resolver on my laptop so that the resolv.conf file doesn't break anything | 21:58 |
| derekh | mordred: incase it helps, at one stage we were using puppet to build RDO based images with packages, we had to do some weird stuff like swap out certain binaries during image builds to make it work with the openstack puppet modules https://github.com/agroup/tripleo-puppet-elements/blob/master/elements/puppet/install.d/99-zuppet#L22 | 21:58 |
| mordred | since the nodepool server where this will run for real will have a resolver | 21:58 |
| hewbrocca | fortunatly then we stopped doing that | 21:59 |
| *** ramishra has quit IRC | 22:02 | |
| SpamapS | mordred: a)... except, no cleanup.. wouldn't puppet run at first boot and make things the way they should be? | 22:04 |
| *** geerdest has joined #tripleo | 22:05 | |
| *** saurabhs has quit IRC | 22:05 | |
| SpamapS | mordred: that's why I was wondering about facter. If we could have a facter plugin which just set a variable "building_image" and then we remove that facter plugin.. your conditionals become very straight forward "this is something for doing / not doing at build time. | 22:05 |
| SpamapS | mordred: instead of 1000 different conditionals.. you just have one.. sprinkled through the puppet where it makes sense. | 22:06 |
| mordred | SpamapS: a) no. we dont' run puppet on boot at all | 22:06 |
| mordred | SpamapS: we only run puppet at image build time | 22:06 |
| SpamapS | oh lol of course | 22:06 |
| mordred | and never again | 22:06 |
| SpamapS | btw, have I told you that w/ the Ubuntu font, lol looks like a ghost chasing somebody ... lol===---... | 22:07 |
| SpamapS | ...---===/o/ <-- running away | 22:07 |
| mordred | the main thing is taht one of the reasons we're doing that is that we've gota bunch of base modules we need on the slaves AND on our long-lived servers whicha re puppet agent managemd | 22:07 |
| SpamapS | mordred: ok so you're just dealing with an entirely different build environment, not a difference between build/runtime. | 22:07 |
| mordred | that's right | 22:08 |
| * SpamapS gaining sense | 22:08 | |
| mordred | so if we start putting conditionals in those, we're increasing complexity in other areas | 22:08 |
| SpamapS | then yes, the cleanup at the end makes sense. Or maybe same thing, facter plugin, but you just run puppet again w/o the build variable set. | 22:08 |
| *** yamahata has joined #tripleo | 22:09 | |
| *** saurabhs has joined #tripleo | 22:10 | |
| *** ccrouch1 has quit IRC | 22:10 | |
| mordred | meh - that would require writing ruby | 22:10 |
| hewbrocca | (now you have 2 problems) | 22:10 |
| clarkb | if you do that when do you puppet the extra stuff? | 22:11 |
| clarkb | defeats the purpose to run puppet post boot | 22:11 |
| SpamapS | mordred: 3 lines or so | 22:11 |
| SpamapS | mordred: or you can use an executable with facter-customfacts-plugin | 22:12 |
| SpamapS | clarkb: if you do what? | 22:13 |
| clarkb | if you distinguish between build and not build | 22:13 |
| clarkb | when does not build puppet run? | 22:13 |
| SpamapS | clarkb: end of build :) | 22:13 |
| clarkb | so you do two passes? thats ugh | 22:14 |
| SpamapS | post-install.d/99-zpuppet-postbuild | 22:14 |
| clarkb | not worried about the dib side | 22:14 |
| clarkb | worried about the puppet side | 22:15 |
| SpamapS | clarkb: that's necessary because puppet isn't in control of everything. | 22:15 |
| *** e0ne has joined #tripleo | 22:16 | |
| mordred | clarkb: what are your thoughts on the resolver? | 22:17 |
| mordred | clarkb: I'm not sure what to do locally with that that won't screw network-manager | 22:17 |
| mordred | and I'd rather not have to say "you have to completely re-work your ubuntu to be able to build an infra slave image" | 22:17 |
| *** e0ne_ has joined #tripleo | 22:18 | |
| clarkb | mordred: nc port forward? | 22:19 |
| clarkb | though that requires root, but dib does too right? | 22:19 |
| SpamapS | dib requires sudo | 22:20 |
| clarkb | `nc -lk -p 53 127.0.1.1 53` something like that may work? | 22:20 |
| clarkb | not for udp though | 22:20 |
| SpamapS | wait what? | 22:20 |
| clarkb | SpamapS: you have a local resolver | 22:20 |
| derekh | fwiw we also has to set a factor fact to differentiate between build and run time puppet, the runtime puppet ran in a os-refresh-config script https://github.com/agroup/tripleo-puppet-elements/blob/master/elements/puppet/common.pp | 22:21 |
| clarkb | you just need the chroot to ee it on 127.0.0.1 | 22:21 |
| *** ccrouch has joined #tripleo | 22:21 | |
| *** e0ne has quit IRC | 22:21 | |
| SpamapS | yeah just .. that seems evil.. ;) | 22:21 |
| mordred | derekh: yeah - we're trying very hard to not refactor our puppet at all right now | 22:21 |
| SpamapS | well.. | 22:21 |
| clarkb | mordred: ya no refactor is good *and* I think its a bug to runtime puppet | 22:22 |
| SpamapS | your build host is the problem I think.. not dib per se | 22:22 |
| mordred | my build host is my laptop | 22:22 |
| mordred | so this is what I'm saying though | 22:22 |
| mordred | my build host is bog-standard trusty | 22:22 |
| SpamapS | dib wants unfettered sudo... | 22:22 |
| mordred | it has it | 22:22 |
| *** e0ne_ has quit IRC | 22:22 | |
| SpamapS | just surprised that is o-k ;) | 22:23 |
| mordred | SpamapS: we don't have a choice do we? | 22:23 |
| SpamapS | Run in a throw-away build VM. | 22:23 |
| mordred | ok. hang on | 22:23 |
| mordred | seriously | 22:23 |
| derekh | mordred: yup, fair enough, swapping binaries to make puppet thing it was starting things during image build was how we handled things we could refactor in the puppet modules, then swaped them back after image build | 22:23 |
| SpamapS | That's the other choice. | 22:23 |
| mordred | this has gotten truly ludicrou | 22:23 |
| mordred | ludicrous | 22:23 |
| mordred | I want to build a glance image | 22:23 |
| derekh | *think | 22:23 |
| mordred | on my laptop | 22:24 |
| mordred | I do NOT want to have to spin up a VM to do that | 22:24 |
| mordred | because that's just the most ridiculous thing I've ever heard | 22:24 |
| SpamapS | mordred: just spelling out that you're giving the glance elements root on your box. | 22:24 |
| mordred | what? | 22:24 |
| mordred | I'm not doing tripleo | 22:24 |
| SpamapS | oh sorry | 22:24 |
| * SpamapS OTP | 22:24 | |
| * mordred throws wet cat at SpamapS | 22:25 | |
| mordred | derekh: nod | 22:25 |
| *** lucas-afk has quit IRC | 22:25 | |
| *** CLOUDOUTAGE has joined #tripleo | 22:29 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 22:29 |
| *** CLOUDOUTAGE has quit IRC | 22:29 | |
| *** epim has quit IRC | 22:30 | |
| *** epim has joined #tripleo | 22:38 | |
| greghaynes | SpamapS: Im trying to grab a network uuid in heat, I imagine its some sub-property of {"Fn::Select": [ "ctlplane", {"Fn::GetAtt": [controller0, networks]} ]}... any idea how I can inspect the resource tree to see what that actually returns? | 22:42 |
| *** derekh has quit IRC | 22:45 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Fix /var/lib/mysql selinux labeling https://review.openstack.org/90245 | 22:53 |
| *** epim has quit IRC | 22:58 | |
| *** ramishra has joined #tripleo | 22:58 | |
| *** CLOUDOUTAGE has joined #tripleo | 23:00 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 23:00 |
| *** CLOUDOUTAGE has quit IRC | 23:00 | |
| *** dkehn_ has joined #tripleo | 23:00 | |
| *** clarkb has quit IRC | 23:01 | |
| *** clarkb has joined #tripleo | 23:01 | |
| *** dkehn has quit IRC | 23:03 | |
| *** dkehn has joined #tripleo | 23:03 | |
| *** ramishra has quit IRC | 23:03 | |
| *** dkehnx has quit IRC | 23:04 | |
| *** lifeless has quit IRC | 23:04 | |
| *** CLOUDOUTAGE has joined #tripleo | 23:04 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud partially down https://etherpad.openstack.org/p/cloud-outage | 23:04 |
| *** CLOUDOUTAGE has quit IRC | 23:04 | |
| *** epim has joined #tripleo | 23:05 | |
| *** lifeless has joined #tripleo | 23:05 | |
| SpamapS | aaand novacompute9 paniced | 23:08 |
| *** CaptTof__ has quit IRC | 23:08 | |
| SpamapS | mellanox.. | 23:08 |
| SpamapS | grah | 23:08 |
| *** CaptTofu has joined #tripleo | 23:09 | |
| *** epim has quit IRC | 23:11 | |
| *** CaptTofu has quit IRC | 23:14 | |
| *** e0ne has joined #tripleo | 23:18 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: indent: replace tab by 4 spaces https://review.openstack.org/88530 | 23:19 |
| *** eguz has joined #tripleo | 23:19 | |
| *** eguz has quit IRC | 23:19 | |
| *** ccrouch has quit IRC | 23:22 | |
| *** e0ne has quit IRC | 23:22 | |
| *** eghobo has quit IRC | 23:23 | |
| *** saurabhs has left #tripleo | 23:27 | |
| *** CaptTofu has joined #tripleo | 23:29 | |
| SpamapS | mmm fresh new devtest | 23:34 |
| SpamapS | [2014-04-24 16:03:01] Total runtime: 5012 s | 23:34 |
| SpamapS | [2014-04-24 16:03:01] ramdisk : 136 s | 23:34 |
| SpamapS | [2014-04-24 16:03:01] seed : 1041 s | 23:34 |
| SpamapS | [2014-04-24 16:03:01] undercloud : 1189 s | 23:34 |
| SpamapS | [2014-04-24 16:03:01] overcloud : 2614 s | 23:34 |
| *** CLOUDOUTAGE has joined #tripleo | 23:35 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- R1 ci-overcloud partially down https://etherpad.openstack.org/p/cloud-outage | 23:35 |
| *** CLOUDOUTAGE has quit IRC | 23:35 | |
| *** e0ne has joined #tripleo | 23:43 | |
| *** e0ne has quit IRC | 23:47 | |
| *** eghobo has joined #tripleo | 23:55 | |
| *** yamahata has quit IRC | 23:57 | |
| *** ramishra has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!