| *** TravT has quit IRC | 00:10 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Fix cirros MD5 fails https://review.openstack.org/92764 | 00:11 |
|---|---|---|
| *** nati_ueno has joined #tripleo | 00:12 | |
| * tchaypo goes to gate for the 5th time this week | 00:12 | |
| *** michchap has quit IRC | 00:21 | |
| *** michchap has joined #tripleo | 00:21 | |
| *** matsuhashi has joined #tripleo | 00:23 | |
| *** killer_prince has quit IRC | 00:25 | |
| *** nati_ueno has quit IRC | 00:29 | |
| *** chuckC has quit IRC | 00:29 | |
| *** michchap_ has joined #tripleo | 00:34 | |
| *** michchap has quit IRC | 00:36 | |
| *** nati_ueno has joined #tripleo | 00:37 | |
| *** Penick has joined #tripleo | 00:38 | |
| *** andreaf_ has joined #tripleo | 00:38 | |
| *** saurabhs has quit IRC | 00:40 | |
| *** zackf has joined #tripleo | 00:41 | |
| *** andreaf has quit IRC | 00:41 | |
| *** Penick has quit IRC | 00:43 | |
| *** xuhaiwei has joined #tripleo | 00:48 | |
| *** andreaf_ has quit IRC | 00:50 | |
| *** nati_ueno has quit IRC | 01:02 | |
| *** nati_ueno has joined #tripleo | 01:03 | |
| StevenK | Yay, expenses ... | 01:07 |
| greghaynes | aw, crap | 01:08 |
| greghaynes | need to do that | 01:08 |
| clarkb | :P I did them on tuesday | 01:08 |
| StevenK | All of my amex transactions have finally hit eem | 01:08 |
| clarkb | greghaynes: you are now a favorite in my expense reporting | 01:08 |
| StevenK | Haha | 01:08 |
| greghaynes | clarkb: dawwww | 01:09 |
| greghaynes | Ill remember for next summit | 01:09 |
| clarkb | uh oh | 01:09 |
| * StevenK has a dinner receipt with greghaynes on it too | 01:10 | |
| greghaynes | Yep. This is how you win at expenses | 01:10 |
| StevenK | By having every one of your workmates do them? | 01:10 |
| greghaynes | Precisely | 01:11 |
| greghaynes | The only winning move is not to play | 01:11 |
| StevenK | So far, nothing wants a receipt attached so far | 01:14 |
| adam_g | so | 01:14 |
| *** nati_ueno has quit IRC | 01:14 | |
| adam_g | https://launchpad.net/bugs/1316475 looks like a cloud-init issue | 01:14 |
| uvirtbot | Launchpad bug 1316475 in tripleo "trusty hang on first boot post deploy" [Critical,Triaged] | 01:14 |
| *** petertoft has joined #tripleo | 01:19 | |
| *** blamar_ has joined #tripleo | 01:30 | |
| *** blamar has quit IRC | 01:31 | |
| *** blamar_ is now known as blamar | 01:31 | |
| *** lazy_prince has joined #tripleo | 01:32 | |
| *** lazy_prince is now known as killer_prince | 01:32 | |
| StevenK | Now to scan in receipts | 01:34 |
| *** nosnos has joined #tripleo | 01:35 | |
| *** nati_ueno has joined #tripleo | 01:45 | |
| StevenK | greghaynes: "Your" expense claim has been submitted | 01:48 |
| *** dkehn_ has joined #tripleo | 01:49 | |
| *** nati_ueno has quit IRC | 01:49 | |
| clarkb | StevenK: when we travel internationally we tend to have to send in phsical receipts. Do you ahve to do that when its international for you or is all US centric? | 01:51 |
| StevenK | clarkb: They've all said "Digital preferred" for me | 01:51 |
| *** nati_ueno has joined #tripleo | 01:51 | |
| StevenK | So far I've only attached digital copies and filed the paper ones here, and haven't had any problems | 01:52 |
| StevenK | lifeless: I also had no problems attaching files to expenses with Firefox 29 | 01:53 |
| clarkb | StevenK: same here, worked fine | 01:54 |
| *** nati_ueno has quit IRC | 01:54 | |
| *** petertoft has quit IRC | 01:54 | |
| *** lazy_prince has joined #tripleo | 01:58 | |
| *** killer_prince has quit IRC | 01:58 | |
| *** dkehnx has quit IRC | 01:58 | |
| *** jpeeler has quit IRC | 01:58 | |
| *** lazy_prince is now known as killer_prince | 01:58 | |
| *** jpeeler has joined #tripleo | 02:00 | |
| *** dkehn_ is now known as dkehnx | 02:02 | |
| openstackgerrit | Ethan Lynn proposed a change to openstack/diskimage-builder: Add Ubuntu local image support https://review.openstack.org/90887 | 02:12 |
| *** nati_ueno has joined #tripleo | 02:14 | |
| lifeless | StevenK: they may well have fixed it by now ;) | 02:18 |
| StevenK | Heh, that's true. | 02:19 |
| * StevenK waits for the check-tripleo queue to die down | 02:19 | |
| lifeless | StevenK: 'two things' - subprocess to API, and no wait to waiting | 02:20 |
| StevenK | I did wonder if that's you meant | 02:21 |
| StevenK | Which is a bit of a pain to split up, but I can do that | 02:21 |
| StevenK | But should be far better than reordering bzr pipelines | 02:25 |
| StevenK | (Where I used to get so annoyed, that I would re-make the pipeline from scratch to save frustration) | 02:27 |
| *** nati_ueno has quit IRC | 02:32 | |
| *** julim has quit IRC | 02:32 | |
| *** killer_prince has quit IRC | 02:44 | |
| *** killer_prince has joined #tripleo | 02:48 | |
| mordred | clarkb: for international travel, the need for receipts varys by target country | 02:50 |
| mordred | for instance... | 02:50 |
| * mordred apologizes to non-hp people in channel | 02:50 | |
| mordred | HP wants receipts for a ₤3 coffee in england, but I can buy a €100 dinner in spain with no receipt | 02:51 |
| pleia2 | mordred: can we get an apology too? | 02:56 |
| pleia2 | ;D | 02:56 |
| pleia2 | crazy receipt policies | 02:56 |
| mordred | pleia2: ;) | 02:57 |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 03:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1321563 | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 03:00 |
| uvirtbot | Launchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New] | 03:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 03:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 03:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 03:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 03:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| *** lazy_prince has joined #tripleo | 03:02 | |
| *** killer_prince has quit IRC | 03:02 | |
| *** lazy_prince is now known as killer_prince | 03:02 | |
| *** nosnos has quit IRC | 03:03 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Simplify creation/cleanup of testenvs https://review.openstack.org/92734 | 03:04 |
| lifeless | http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/cs_utils.py#L19 is confidence inducing | 03:14 |
| *** morazi has joined #tripleo | 03:15 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-specs: TripleO on OpenStack https://review.openstack.org/92642 | 03:17 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Simplify creation/cleanup of testenvs https://review.openstack.org/92734 | 03:27 |
| *** matsuhashi has quit IRC | 03:29 | |
| *** eghobo has joined #tripleo | 03:32 | |
| *** matsuhashi has joined #tripleo | 03:36 | |
| StevenK | lifeless: Now I have to review my lunch | 03:40 |
| *** matsuhashi has quit IRC | 03:43 | |
| *** matsuhashi has joined #tripleo | 03:43 | |
| *** matsuhas_ has joined #tripleo | 03:45 | |
| *** matsuhashi has quit IRC | 03:45 | |
| *** nosnos has joined #tripleo | 03:45 | |
| *** csd has quit IRC | 03:55 | |
| *** akuznetsov has joined #tripleo | 03:59 | |
| *** OldCrowEW has joined #tripleo | 03:59 | |
| *** killer_prince has quit IRC | 04:04 | |
| *** lazy_prince has joined #tripleo | 04:05 | |
| *** lazy_prince is now known as killer_prince | 04:05 | |
| *** killer_prince has quit IRC | 04:14 | |
| *** nati_ueno has joined #tripleo | 04:28 | |
| *** edmund has quit IRC | 04:28 | |
| *** eghobo has quit IRC | 04:28 | |
| *** edmund has joined #tripleo | 04:28 | |
| *** matsuhas_ has quit IRC | 04:34 | |
| *** marun has quit IRC | 04:40 | |
| *** lazy_prince has joined #tripleo | 04:40 | |
| *** lazy_prince is now known as killer_prince | 04:40 | |
| *** morganfainberg is now known as morganfainberg_Z | 04:42 | |
| *** marun has joined #tripleo | 04:42 | |
| *** matsuhashi has joined #tripleo | 04:43 | |
| *** marun has quit IRC | 04:44 | |
| *** eghobo has joined #tripleo | 04:44 | |
| *** akuznetsov has quit IRC | 05:01 | |
| *** lazy_prince3 has joined #tripleo | 05:08 | |
| greghaynes | Yes, I was just reading that | 05:09 |
| *** dshulyak_ has joined #tripleo | 05:09 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Switch from subprocess to ironicclient https://review.openstack.org/93964 | 05:10 |
| *** OldCrowEW has quit IRC | 05:10 | |
| greghaynes | Does it just assume that if you have a ttyS1 theres that type of service connected on the other end? | 05:10 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Loop for 600 seconds rather than checking ironic https://review.openstack.org/95066 | 05:10 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Switch from subprocess to novaclient https://review.openstack.org/94779 | 05:10 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Loop for 600 seconds rather than checking nova-bm https://review.openstack.org/95067 | 05:11 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Switch from subprocess to keystoneclient https://review.openstack.org/94782 | 05:11 |
| *** edmund has quit IRC | 05:11 | |
| *** edmund has joined #tripleo | 05:11 | |
| tchaypo | okay. | 05:13 |
| *** eghobo has quit IRC | 05:15 | |
| *** eguz has joined #tripleo | 05:15 | |
| *** eghobo has joined #tripleo | 05:15 | |
| *** eghobo has quit IRC | 05:15 | |
| * StevenK waits for the flood of gerrit mail to cease | 05:15 | |
| *** eguz has quit IRC | 05:20 | |
| *** edmund1 has joined #tripleo | 05:21 | |
| *** edmund has quit IRC | 05:23 | |
| greghaynes | wow, I think it does | 05:24 |
| StevenK | greghaynes: I keep punching the 'set -o pipefail' comments because bnemec has been trying to land his branch to enforce that for tie for some time, and I'd like to make his job easier. | 05:26 |
| greghaynes | StevenK: Yep, and I agree on anything new, but I just am not sure I should change that for an already existing file with just a one line change - seems like an easy way to introduce a bug from something undocumented in commit msg | 05:27 |
| greghaynes | That shouldnt conflict with his change either since that file already exists | 05:28 |
| StevenK | The file is short enough that git might mark it as needing conflict resolution | 05:28 |
| StevenK | But I see your point | 05:28 |
| greghaynes | :( if it does id be up for basing off him | 05:29 |
| greghaynes | becuase that totally is annoying | 05:29 |
| greghaynes | maybe we should just land his change | 05:29 |
| *** shakayumi has joined #tripleo | 05:30 | |
| greghaynes | ooo https://review.openstack.org/#/c/83929/ | 05:30 |
| *** ramishra has joined #tripleo | 05:31 | |
| tchaypo | once again on the plane, they told me that "transmitting devices" must not be switched on, but "please feel free to join our in-flight wifi" | 05:32 |
| StevenK | Heh heh | 05:33 |
| greghaynes | Its a trap! | 05:33 |
| StevenK | tchaypo: Did they also ask passengers to not congregate in any common areas, like the toilets, or their seats ... ? :-P | 05:33 |
| *** jprovazn has joined #tripleo | 05:34 | |
| tchaypo | not for today's domestic flight | 05:34 |
| tchaypo | they did on the way back from atlanta | 05:34 |
| tchaypo | then they keep the seatbelt sign on for so long that as soon as it goes off half the plane is congregating around the toilet door | 05:34 |
| StevenK | Heh, it's only announced on Qantas for flights to the US, not out of | 05:35 |
| greghaynes | My flight from atlanta had an awesome anouncement of "We cannot give you permission to go to use the bathroom if the seatbelt sign is on. If you get up and do it I wont tackle you, but I cant give you permission to do it. Find the grey area." | 05:35 |
| StevenK | Is that when you point out the carpets are grey? | 05:36 |
| greghaynes | haha | 05:36 |
| *** eghobo has joined #tripleo | 05:42 | |
| *** tzumainn has quit IRC | 05:42 | |
| *** lazy_prince3 has quit IRC | 05:51 | |
| *** edmund has joined #tripleo | 05:54 | |
| *** edmund1 has quit IRC | 05:54 | |
| *** akuznetsov has joined #tripleo | 05:56 | |
| *** ramishra has quit IRC | 05:57 | |
| greghaynes | ok, just replicated that trusty hang thing in a vm | 05:58 |
| greghaynes | this is amazing | 05:58 |
| *** eghobo has quit IRC | 05:58 | |
| greghaynes | You just cannot have a second serial port unless its hooked up to CloudSigma | 05:58 |
| greghaynes | or it hangs | 05:58 |
| greghaynes | As a launchpad noob - is normal etiquitte to open a new bug on https://launchpad.net/cloud-init or add cloud-init as an affected project? | 05:59 |
| *** chuckC has joined #tripleo | 06:04 | |
| *** akuznetsov has quit IRC | 06:07 | |
| *** nati_ueno has quit IRC | 06:08 | |
| greghaynes | oh, nvm, was already added | 06:09 |
| StevenK | greghaynes: Usually, you add another bugtask, unless doing so will cause timeouts :-P | 06:10 |
| StevenK | (Which is like >8 or so) | 06:10 |
| clarkb | StevenK and in those cases you figure out how to do it via email | 06:12 |
| greghaynes | thats webscale | 06:13 |
| *** akuznetsov has joined #tripleo | 06:13 | |
| StevenK | clarkb: affects foo -- but that may cause random timeouts when changing any bugtasks state | 06:14 |
| StevenK | Because notifying in request is such awesomeness | 06:14 |
| *** jtomasek has joined #tripleo | 06:19 | |
| *** jtomasek has quit IRC | 06:26 | |
| *** mrunge has joined #tripleo | 06:27 | |
| *** dshulyak_ has quit IRC | 06:27 | |
| *** edmund has quit IRC | 06:29 | |
| tchaypo | expense report almost complete | 06:32 |
| tchaypo | woobles~ | 06:33 |
| StevenK | Haha | 06:33 |
| *** nati_ueno has joined #tripleo | 06:41 | |
| *** lazy_prince3 has joined #tripleo | 06:48 | |
| *** e0ne has joined #tripleo | 06:57 | |
| *** jcoufal has joined #tripleo | 07:00 | |
| *** lazy_prince3 is now known as lazy_prince | 07:02 | |
| *** e0ne has quit IRC | 07:09 | |
| *** killer_prince has quit IRC | 07:12 | |
| *** e0ne has joined #tripleo | 07:12 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options https://review.openstack.org/94907 | 07:16 |
| *** killer_prince has joined #tripleo | 07:21 | |
| *** e0ne_ has joined #tripleo | 07:22 | |
| *** e0ne has quit IRC | 07:25 | |
| *** jtomasek has joined #tripleo | 07:28 | |
| *** ifarkas has joined #tripleo | 07:29 | |
| *** gcha has joined #tripleo | 07:35 | |
| *** nati_ueno has quit IRC | 07:39 | |
| *** zackf has quit IRC | 07:39 | |
| *** jistr has joined #tripleo | 07:43 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Allow multiple binds per service in haproxy https://review.openstack.org/89925 | 07:49 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Prevent duplication of net_binds per service https://review.openstack.org/95097 | 07:49 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-incubator: Adding Undercloud Ceilometer config element https://review.openstack.org/94637 | 07:49 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tripleo-incubator: Generating of password for SNMPd https://review.openstack.org/94838 | 07:49 |
| *** pelix has joined #tripleo | 07:50 | |
| *** rwsu has quit IRC | 07:58 | |
| *** derekh_ has joined #tripleo | 08:00 | |
| derekh_ | ci still in trouble :-( http://goodsquishy.com/downloads/tripleo-jobs.html | 08:08 |
| *** akuznetsov has quit IRC | 08:09 | |
| derekh_ | SpamapS: if your still there, want me to take over on anything? | 08:12 |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-incubator: Generate overcloud keystone keys/certs https://review.openstack.org/95101 | 08:12 |
| *** yamahata has joined #tripleo | 08:13 | |
| derekh_ | looks like we are only running about 8 jobs at a time most of which are failing :-( | 08:13 |
| derekh_ | lifeless: SpamapS R1 ran 177 jobs in the last 10 hours, only 5 passed | 08:15 |
| derekh_ | brb | 08:15 |
| *** akuznetsov has joined #tripleo | 08:27 | |
| *** akrivoka has joined #tripleo | 08:27 | |
| *** ifarkas has quit IRC | 08:27 | |
| *** jp_at_hp has joined #tripleo | 08:32 | |
| lifeless | derekh_: :( | 08:35 |
| derekh_ | lifeless: trying to dig into it at the moment, the jobs that passed were all a few hours ago, non passed since then | 08:36 |
| *** CLOUDOUTAGE has joined #tripleo | 08:37 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 | 08:37 |
| *** CLOUDOUTAGE has quit IRC | 08:37 | |
| *** CLOUDOUTAGE has joined #tripleo | 08:39 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 08:39 |
| *** CLOUDOUTAGE has quit IRC | 08:39 | |
| lifeless | derekh_: thanks | 08:40 |
| lifeless | derekh_: I've been crook for two days, just coming out of it now | 08:40 |
| derekh_ | lifeless: :-( , no prob | 08:41 |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/tuskar: Don't display passwords when listing overclouds https://review.openstack.org/94648 | 08:43 |
| *** lucasagomes has joined #tripleo | 08:45 | |
| *** matsuhashi has quit IRC | 08:48 | |
| *** akrivoka has quit IRC | 08:56 | |
| *** akrivoka has joined #tripleo | 08:56 | |
| *** pblaho has joined #tripleo | 08:58 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 09:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1321563 | 09:00 |
| uvirtbot | Launchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 09:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 09:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 09:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 09:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 09:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| *** shakayumi has quit IRC | 09:01 | |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-incubator: Add the select-cloud script https://review.openstack.org/82807 | 09:01 |
| derekh_ | ssh to overcloud controller is very jumpy, somtimes stalling for over a minutes becoming responsive again | 09:04 |
| rcarrill` | haven't played with devtest for some time, it now complains about TRIPLEO_ROOT pointing to a location that does not contain a tripleo checkout | 09:06 |
| rcarrill` | so now devtest does not download tripleo-incubator itself to .cache? | 09:06 |
| *** rcarrill` is now known as rcarrillocruz | 09:06 | |
| rcarrillocruz | (morning) | 09:06 |
| *** e0ne_ has quit IRC | 09:08 | |
| *** e0ne has joined #tripleo | 09:09 | |
| *** CLOUDOUTAGE has joined #tripleo | 09:10 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 09:10 |
| *** CLOUDOUTAGE has quit IRC | 09:10 | |
| *** e0ne_ has joined #tripleo | 09:11 | |
| Kiall_ | Anyone able to comment on https://review.openstack.org/#/c/93031/ ? Tiny patch with multiple +2's, and a seemingly spurious f20 failure :( | 09:11 |
| *** Kiall_ is now known as Kiall | 09:12 | |
| *** athomas has joined #tripleo | 09:13 | |
| *** giulivo has joined #tripleo | 09:13 | |
| *** e0ne has quit IRC | 09:15 | |
| *** shardy_afk is now known as shardy | 09:15 | |
| *** matsuhashi has joined #tripleo | 09:16 | |
| derekh_ | Kiall: that f20 failure is because a regression was commited causing the f20 job to fail pretty much all of yesterday , thats fixed now but one of the ci racks is having other problem so don't recheck until its fixed | 09:20 |
| derekh_ | Ng SpamapS lifeless so ssh to overcloud controller is freezing on and off for around a minute each time, would the mlx driver need to be reloaded on it ? | 09:21 |
| derekh_ | if it does, whats the commands ? modprobe -r mlx4_en ; modprobe mlx4_en | 09:21 |
| *** jcoufal has quit IRC | 09:23 | |
| *** jcoufal has joined #tripleo | 09:23 | |
| lifeless | derekh_: something like that; its probably in history. I thought we'd upgraded it to trusty though :( | 09:31 |
| lifeless | derekh_: which shouldn't need such shenanigans | 09:31 |
| derekh_ | lifeless: VERSION="13.10, Saucy Salamander" | 09:32 |
| lifeless | ohhh gnar | 09:37 |
| *** CLOUDOUTAGE has joined #tripleo | 09:41 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 09:41 |
| *** CLOUDOUTAGE has quit IRC | 09:41 | |
| *** nosnos has quit IRC | 09:45 | |
| lsmola | derekh_: hello, have you seen this? http://pastebin.com/H2Ze849g | 09:46 |
| lsmola | derekh_: seems like this is failing https://github.com/openstack/tripleo-incubator/blob/master/scripts/register-nodes#L34 | 09:47 |
| derekh_ | lsmola: it doesn't look familiar probably worth looking at the nova logs | 09:48 |
| lsmola | derekh_: because there is a certificate passed as pm_password, is that alright? | 09:48 |
| lsmola | derekh_: it works when I just delete pm_user and pm_password | 09:48 |
| lsmola | derekh_: could it be there is a wrong character in the certificate? | 09:49 |
| derekh_ | lsmola: off the top of my head I'm not sure what supposed to be passed in, I don't have a running devtest at the moment to check but will start one now | 09:50 |
| lsmola | derekh_: ok, cool | 09:50 |
| *** ccorrigan has quit IRC | 09:55 | |
| *** athomas has quit IRC | 09:57 | |
| lsmola | derekh_: seems like there should be this rsa key, so only thing that comes to my mind is that there has to be some bad character http://pastebin.com/MQXFuSbw | 09:59 |
| lsmola | derekh_: so that nova baremetal returns 500 | 09:59 |
| lsmola | derekh_: weird, seems like this is there for a long time and I never had problems with that :-) I have fresh f20 maybe that is the cause | 10:01 |
| *** ifarkas has joined #tripleo | 10:01 | |
| *** athomas has joined #tripleo | 10:04 | |
| *** akuznetsov has quit IRC | 10:05 | |
| *** rlandy has joined #tripleo | 10:07 | |
| *** matsuhashi has quit IRC | 10:12 | |
| *** CLOUDOUTAGE has joined #tripleo | 10:12 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 10:12 |
| *** CLOUDOUTAGE has quit IRC | 10:12 | |
| derekh_ | lsmola: yup, my pm_password is a private key | 10:12 |
| lsmola | derekh_: do you see the failure? | 10:12 |
| derekh_ | lsmola: nope mine is ticking along , I'll kill it and get uptodate everything now | 10:13 |
| lsmola | derekh_: I have fresh f20 from the beaker, if that helps with anything :-) | 10:14 |
| derekh_ | lsmola: ahh ok, i don't have that, can't update mine at the moment either | 10:15 |
| lsmola | derekh_: btw. could you try it with the private key I posted? | 10:15 |
| *** matsuhas_ has joined #tripleo | 10:15 | |
| derekh_ | lsmola: ok, will do | 10:15 |
| lsmola | derekh_: ok, that should eliminate if there is not just some a bad char | 10:16 |
| derekh_ | lsmola: it should be in id_rsa_virt_power right? | 10:16 |
| lsmola | derekh_: yep https://github.com/openstack/tripleo-incubator/blame/fd1d619f2fc94f221650c02f158427a9d2944157/scripts/devtest_testenv.sh# | 10:17 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-incubator: Promote HEAT_ENV over env vars https://review.openstack.org/95130 | 10:17 |
| lsmola | derekh_: eh bad link :-) https://github.com/openstack/tripleo-incubator/blob/fd1d619f2fc94f221650c02f158427a9d2944157/scripts/devtest_testenv.sh#L193 | 10:18 |
| *** chuckC has quit IRC | 10:19 | |
| derekh_ | lsmola: is " seed 1 2048 60" at the end of that key file, thats not right is it ? | 10:19 |
| lsmola | derekh_: nope that are the other parameters | 10:19 |
| derekh_ | lsmola: actually its problem just being output | 10:19 |
| *** openstackstatus has quit IRC | 10:20 | |
| *** openstack has joined #tripleo | 10:21 | |
| *** openstackstatus has joined #tripleo | 10:22 | |
| *** ChanServ sets mode: +v openstackstatus | 10:22 | |
| *** lathiat has quit IRC | 10:23 | |
| lsmola | derekh_: ok let me know if it fails, it should be like in 4 minutes | 10:24 |
| *** lathiat has joined #tripleo | 10:24 | |
| derekh_ | lsmola: will do, its building the new seed now | 10:24 |
| lsmola | derekh_: ok | 10:24 |
| *** andreaf has joined #tripleo | 10:26 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options https://review.openstack.org/94907 | 10:28 |
| *** ccorrigan has joined #tripleo | 10:34 | |
| *** akuznetsov has joined #tripleo | 10:36 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Move user's group membership setup in separate script https://review.openstack.org/93954 | 10:38 |
| *** wfoster is now known as wfoster_afk | 10:42 | |
| *** CLOUDOUTAGE has joined #tripleo | 10:43 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 10:43 |
| *** CLOUDOUTAGE has quit IRC | 10:43 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Avoid recreating the VENV in setup-clienttools if one exists https://review.openstack.org/93957 | 10:44 |
| derekh_ | lsmola: ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-b8c24ddc-6ab9-4c0d-a38f-ae059b8760d2) | 10:48 |
| derekh_ | lsmola: gomma try with a different key now | 10:48 |
| lsmola | derekh_: hm, ok | 10:49 |
| lsmola | derekh_: seems like the key encoded in json gets some weird characters http://pastebin.com/7xVAhP0v | 10:50 |
| lsmola | derekh_: like \ | 10:50 |
| derekh_ | lsmola: does it work for you if you use a different key ? | 10:53 |
| lsmola | derekh_: I am not trying it without the pm_password and user params, i don't need them for bm_poseur right? | 10:54 |
| lsmola | derekh_: so possibly yes | 10:54 |
| lsmola | derekh_: does it work for you? | 10:54 |
| derekh_ | lsmola: it got that error ^^ with your key, now trying with a new one | 10:54 |
| lsmola | derekh_: seems like a bad char that leads to some crazy parameters being send to nova-baremetal cli | 10:55 |
| *** matsuhas_ has quit IRC | 10:55 | |
| *** matsuhashi has joined #tripleo | 10:57 | |
| *** matsuhashi has quit IRC | 10:58 | |
| *** matsuhashi has joined #tripleo | 10:58 | |
| xuhaiwei | +Ng:can i ask you a question now? | 10:58 |
| *** matsuhashi has quit IRC | 10:58 | |
| *** matsuhashi has joined #tripleo | 10:59 | |
| *** akrivoka has quit IRC | 10:59 | |
| *** akrivoka has joined #tripleo | 11:00 | |
| xuhaiwei | I am doing devtest, and i am confused about the nova hypervisor-stats, when the 'count' is 0, it means no baremetal node is registered? | 11:07 |
| *** tzumainn has joined #tripleo | 11:07 | |
| *** CLOUDOUTAGE has joined #tripleo | 11:14 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 11:14 |
| *** CLOUDOUTAGE has quit IRC | 11:14 | |
| *** ccorrigan has quit IRC | 11:23 | |
| *** akrivoka has quit IRC | 11:23 | |
| derekh_ | lsmola: same error with new key, must be some other problem, sorry not getting time to look into it properly | 11:32 |
| giulivo | derekh_, I'm seeing that same error now , reproduced it locally | 11:33 |
| lsmola | derekh_: hm weird, though you didn't see the error with your original key right? | 11:33 |
| derekh_ | lsmola: that was before I updated all the repositories | 11:34 |
| lsmola | derekh_: ah, ok | 11:34 |
| giulivo | lsmola, derekh_ here is what I see in the logs: (DataError) (1406, "Data too long for c | 11:36 |
| giulivo | olumn 'pm_password' at row 1") | 11:36 |
| lsmola | giulivo: cool | 11:37 |
| lsmola | giulivo: that explains it, seems like new constraint in nova bm? | 11:37 |
| giulivo | but I'm not sure how/where to fix... maybe I can try updating the column size manually first? | 11:37 |
| giulivo | also I wonder if the pm_password length is really supposed to change depending on the key? | 11:38 |
| giulivo | pm_password | varchar(255) | YES | | NULL | | | 11:38 |
| *** andreaf_ has joined #tripleo | 11:39 | |
| *** lucasagomes is now known as lucas-hungry | 11:39 | |
| lsmola | giulivo: I thing it's caused by constraint in nova | 11:40 |
| lsmola | giulivo: afaik Mysql will just cut the data and doesn't raise any error :-) | 11:40 |
| derekh_ | lsmola: giulivo: any chance this commit changed the behaviour https://review.openstack.org/#/c/89778/ ? | 11:42 |
| *** CLOUDOUTAGE has joined #tripleo | 11:45 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 11:45 |
| *** CLOUDOUTAGE has quit IRC | 11:45 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/os-cloud-config: Allow setup services endpoints https://review.openstack.org/92383 | 11:45 |
| openstackgerrit | Om Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support on PXE failures. https://review.openstack.org/79289 | 11:45 |
| *** mrunge has quit IRC | 11:47 | |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel https://review.openstack.org/79873 | 11:49 |
| lsmola | giulivo: weird it should be text https://github.com/openstack/nova/blob/2efd3faa3e07fdf16c2d91c16462e7e1e3f33e17/nova/virt/baremetal/db/sqlalchemy/models.py#L46 | 11:51 |
| lsmola | derekh_: not sure, I don't see anything there with a direct impact | 11:51 |
| giulivo | lsmola, address user password all are varchar for me | 11:52 |
| lsmola | giulivo: https://github.com/openstack/nova/blob/2efd3faa3e07fdf16c2d91c16462e7e1e3f33e17/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/001_init.py#L35 | 11:53 |
| lsmola | giulivo: hmm | 11:53 |
| giulivo | lsmola, yah that is what I get indeed | 11:54 |
| lsmola | giulivo: eh, so on model they set is as text, but on migration there is varchar, fuu | 11:54 |
| giulivo | but looks like its always been like that from 3rd of feb | 11:54 |
| lsmola | giulivo: yeah, it must be that just the validation is new | 11:55 |
| *** wfoster_afk is now known as wfoster | 11:55 | |
| lsmola | giulivo: previously it was probably just ignored | 11:55 |
| *** andreaf_ has quit IRC | 11:55 | |
| giulivo | lsmola, also my virt_power key is 1679 chars, how can it ever possibly fit in 255? | 11:57 |
| giulivo | I mean even it mysql was just truncating, then ssh to virt host should have failed | 11:59 |
| lsmola | giulivo: hm these are just IPMI credentials, which are not used for bm_poseur | 11:59 |
| lsmola | giulivo: but on real hardware yeah, i would say it should fail for them | 12:00 |
| giulivo | ah yeah libvirt driver has an option which points to the keyfile | 12:01 |
| lsmola | giulivo: hmm, unless the migrations were broken before | 12:01 |
| lsmola | giulivo: and there was actually text | 12:02 |
| giulivo | so if I'm reading it right, given the libvirt virtual power driver gets the key from the config file | 12:03 |
| *** akrivoka has joined #tripleo | 12:03 | |
| giulivo | we just shouldn't paste the rsa key into bm_password | 12:04 |
| *** akrivoka has joined #tripleo | 12:04 | |
| lsmola | giulivo: now it takes it from here https://github.com/openstack/tripleo-incubator/blob/fd1d619f2fc94f221650c02f158427a9d2944157/scripts/devtest_testenv.sh#L193 | 12:06 |
| lsmola | giulivo: so I guess we should just generate some smaller password for IPMI | 12:06 |
| lsmola | giulivo: did you create critical bug for this? | 12:06 |
| giulivo | nope I'm still in the process of gathering some infos | 12:07 |
| lsmola | giulivo: will you or should I? | 12:08 |
| giulivo | so if I get it right, what you suggest to do is not cat the key into ssh-key | 12:08 |
| lsmola | giulivo: well, I am not sure if we should land a patch for nova-baremetal | 12:08 |
| giulivo | but nova-bm seems to be doing okay | 12:09 |
| giulivo | probably ipmi drivers don't even support to find an rsa key into the password field | 12:09 |
| lsmola | giulivo: the validation error comes from nova bm right? | 12:09 |
| giulivo | yeah but validation looks fine to me | 12:09 |
| giulivo | it seems it is us who are using the pm_password field improperly | 12:10 |
| lsmola | giulivo: it could be | 12:10 |
| *** e0ne_ has quit IRC | 12:10 | |
| lsmola | giulivo: let me check Ironic DB schema | 12:10 |
| giulivo | lsmola, I think you're right when saying that the we shouldn't cat the key contents in ssh-key | 12:12 |
| lsmola | giulivo: hm so Ironic stores it here https://github.com/openstack/ironic/blob/master/ironic/db/sqlalchemy/models.py#L155 | 12:14 |
| lsmola | giulivo: which i assume will be text or bigger | 12:15 |
| lsmola | giulivo: so we need possibly just a fix for nova bm | 12:15 |
| lsmola | giulivo: I guess putting there smaller password on our side should be fine | 12:15 |
| *** CLOUDOUTAGE has joined #tripleo | 12:16 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 12:16 |
| *** CLOUDOUTAGE has quit IRC | 12:16 | |
| lsmola | giulivo: will you create the bug once you will gather all the info? | 12:16 |
| giulivo | yes I can open the bug | 12:16 |
| giulivo | I'm not sure about the fix instead | 12:16 |
| lsmola | giulivo: cool, thank you | 12:16 |
| giulivo | I'll link that to you here | 12:16 |
| lsmola | giulivo: ok | 12:17 |
| giulivo | I'm still unsure about why we should paste the contents of the key into pm_password | 12:17 |
| giulivo | are the power drivers shared amongst novabm and ironic? | 12:17 |
| lsmola | giulivo: no you either deploy with nova-bm or ironic | 12:19 |
| *** matsuhashi has quit IRC | 12:20 | |
| lsmola | giulivo: I am not really sure, if they are trying that on real Baremetals with IPMI, they need to fill there real IPMI credentials | 12:20 |
| giulivo | so the nova virtual_power_driver gets the key from a file not from pm_password | 12:21 |
| *** OldCrowEW has joined #tripleo | 12:21 | |
| *** matsuhashi has joined #tripleo | 12:21 | |
| giulivo | agreed so we only populate pm_password when running on real hardware, in which case the user will have to provide the password somehow | 12:21 |
| lsmola | giulivo: uh, not sure I understand? :-) | 12:21 |
| giulivo | so we'd still don't want to paste the contents of the private key in pm_password | 12:21 |
| lsmola | giulivo: possibly not, it will be also quickest fix to just generate some smaller password and put it there | 12:23 |
| *** shakayumi has joined #tripleo | 12:23 | |
| lsmola | giulivo: hopefully somebody will comment the bug with the correct solution :-) | 12:23 |
| *** dprince has joined #tripleo | 12:24 | |
| *** andreaf has quit IRC | 12:25 | |
| giulivo | here it is https://bugs.launchpad.net/tripleo/+bug/1322599 | 12:26 |
| uvirtbot | Launchpad bug 1322599 in tripleo "nova baremetal-node-create fails with HTTP 500" [Undecided,New] | 12:26 |
| *** matsuhashi has quit IRC | 12:26 | |
| giulivo | I can't triage though | 12:26 |
| giulivo | derekh_, ^^ | 12:26 |
| derekh_ | giulivo: triaged as critical since 3 of us hit it now | 12:27 |
| *** lazy_prince has quit IRC | 12:32 | |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-image-elements: Don't install openvswitch dkms modules in trusty https://review.openstack.org/95151 | 12:37 |
| *** jdob has joined #tripleo | 12:37 | |
| *** andreaf_ has joined #tripleo | 12:37 | |
| *** matsuhashi has joined #tripleo | 12:37 | |
| *** matsuhashi has quit IRC | 12:38 | |
| *** matsuhashi has joined #tripleo | 12:38 | |
| *** matsuhashi has quit IRC | 12:38 | |
| *** matsuhashi has joined #tripleo | 12:39 | |
| *** casanch1 has joined #tripleo | 12:40 | |
| *** e0ne has joined #tripleo | 12:41 | |
| *** matsuhashi has quit IRC | 12:43 | |
| *** lucas-hungry is now known as lucasagomes | 12:43 | |
| *** CLOUDOUTAGE has joined #tripleo | 12:47 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 12:47 |
| *** CLOUDOUTAGE has quit IRC | 12:47 | |
| giulivo | lsmola, I think derekh_ is right, look here, traditional mode was False by default before https://review.openstack.org/#/c/89778/ | 12:48 |
| giulivo | and with traditional mode to False it was silently truncating | 12:49 |
| derekh_ | giulivo: try reverting and rerunning devtest | 12:49 |
| *** jcoufal has quit IRC | 12:53 | |
| *** jcoufal has joined #tripleo | 12:54 | |
| Ng | morning | 12:57 |
| *** athomas has quit IRC | 12:57 | |
| *** yamahata has quit IRC | 12:59 | |
| *** jcoufal has quit IRC | 13:00 | |
| *** jcoufal has joined #tripleo | 13:01 | |
| TheJulia | good morning :) | 13:02 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/diskimage-builder: Add support for source-repos gerrit refs https://review.openstack.org/90890 | 13:02 |
| Ng | adam_g: nice work tracking down the trusty hang bug! | 13:05 |
| *** athomas has joined #tripleo | 13:05 | |
| *** gcha has quit IRC | 13:06 | |
| *** gcha has joined #tripleo | 13:07 | |
| *** julim has joined #tripleo | 13:09 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Change horizon binding address to local-ipv4 in haproxy case https://review.openstack.org/91089 | 13:11 |
| *** CLOUDOUTAGE has joined #tripleo | 13:17 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 13:17 |
| *** CLOUDOUTAGE has quit IRC | 13:17 | |
| *** jrist has quit IRC | 13:28 | |
| *** jcoufal has quit IRC | 13:32 | |
| *** ddieterly has joined #tripleo | 13:34 | |
| *** jrist has joined #tripleo | 13:39 | |
| *** CLOUDOUTAGE has joined #tripleo | 13:48 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 13:48 |
| *** CLOUDOUTAGE has quit IRC | 13:48 | |
| giulivo | ping dprince | 13:56 |
| dprince | giulivo: hi | 13:59 |
| giulivo | dprince, I think I need some help figuring what is the real purpose of ssh-key in our TE_DATAFILE | 13:59 |
| giulivo | I'm working on https://bugs.launchpad.net/tripleo/+bug/1322599 | 13:59 |
| uvirtbot | Launchpad bug 1322599 in tripleo "nova baremetal-node-create fails with HTTP 500" [Critical,Triaged] | 13:59 |
| giulivo | I noticed you did some work with that in devtest_seed | 14:00 |
| giulivo | I'm not sure though why we keep passing the actual contents of the rsa private key around , from $TE_DATAFILE to both the seed and the undercloud (via heat) | 14:00 |
| dprince | giulivo: the ssh-key is ultimately used by the virtual power driver | 14:01 |
| dprince | giulivo: it allows the seed VM to issue "power" commands for other fake VMs | 14:01 |
| giulivo | yeah I understand what it does but it is not getting the rsa key from there | 14:02 |
| giulivo | it is just reading the keyfile configured as virtual_power_host_key | 14:02 |
| dprince | giulivo: looking at the ticket... | 14:03 |
| giulivo | thanks :) | 14:04 |
| giulivo | I think I'm missing what is the real "user" of our ssh-key , I just wanted to keep that intact while passing an empty string as pm_password when we use nova-bm | 14:05 |
| dprince | giulivo: I think we should revert that Nova commit first (before making any changes) | 14:07 |
| *** shakayumi has quit IRC | 14:07 | |
| dprince | giulivo: The TRADITIONAL changes to me seems to be something that services (Nova) should opt into | 14:07 |
| dprince | giulivo: not have it forced upon them by oslo.... | 14:07 |
| rpodolyaka1 | hmm, virtual power driver doesn't seem to be using pm_password | 14:08 |
| *** rwsu has joined #tripleo | 14:08 | |
| dprince | giulivo: we can still rework things on our side too but I'm not the biggest fan of that DB change | 14:08 |
| giulivo | rpodolyaka1, indeed I think it is not... while it would be nice if it actually could, but currently it doesn't | 14:09 |
| rpodolyaka1 | dprince: actually, it used too... | 14:09 |
| rpodolyaka1 | dprince: but later it was decided that it's better to default to TRADITIONAL mode here so that we'd catch such errors as early in the development cycle as possible | 14:09 |
| *** vinsh has joined #tripleo | 14:10 | |
| rpodolyaka1 | dprince: I think it's similar to how we enforce mysql innodb storage engine to be used | 14:10 |
| bnemec | People get upset when you silently corrupt their data, people get upset if you log a warning that you might silently corrupt their data, and people get upset when you refuse to corrupt their data. | 14:11 |
| bnemec | It's really a no-win situation. :-) | 14:11 |
| *** rpodolyaka1 is now known as rpodolyaka | 14:11 | |
| dprince | bnemec: right, so why change it? This is very similar to the Nova v2 on v3 discussion with regards to validating requests. | 14:12 |
| bnemec | dprince: See "silent data corruption". This wasn't "working" before either. | 14:13 |
| dprince | bnemec: v2 doesn't currently validate all requests, but the port to make it run on top of the v3 code would add those validations. Useful, sure. Breaking for some people, almost certainly. | 14:13 |
| rpodolyaka | aren't we just passing to nova the data that it's not going to use? | 14:14 |
| dprince | If we don't need the key then by all means lets remove it | 14:14 |
| *** athomas has quit IRC | 14:15 | |
| mordred | rpodolyaka: ++ | 14:15 |
| bnemec | Team call time. biab | 14:15 |
| mordred | like, we can set traditional in the gate, so that we can catch bad data error places - and if someone wants to deploy not that way - well, I mean, weird, but whatevs | 14:15 |
| * rpodolyaka is not surprised column type is Text() in models definitions and VARCHAR(255) in the migration script... | 14:16 | |
| *** athomas has joined #tripleo | 14:16 | |
| giulivo | rpodolyaka, because of the init script it seems https://github.com/openstack/nova/blob/2efd3faa3e07fdf16c2d91c16462e7e1e3f33e17/nova/virt/baremetal/db/sqlalchemy/migrate_repo/versions/001_init.py#L35 | 14:16 |
| dprince | mordred: my point is that just like some of the API proposals this may cause unexpected changes. | 14:16 |
| mordred | dprince: oh, totally. you can't just switch to traditional in an existing codebase without actually auditing and verifying that you're not already broken | 14:17 |
| dprince | mordred: I do like the idea of stepping towards this | 14:17 |
| mordred | and figuring out a plan to fix it | 14:17 |
| mordred | it would be interesting to spin up a non-voting job that ran mysql with traditional on and see what, if anything, breaks hard | 14:17 |
| rpodolyaka | giulivo: yeah, unfortunately we still have such bugs in our code. I hope my test for detecting this will land to oslo.db soon :P | 14:18 |
| bnemec | Also, it is possible to disable traditional mode still. | 14:18 |
| bnemec | I'd rather fix the bug, but as a temporary workaround that would be an option. | 14:18 |
| *** CLOUDOUTAGE has joined #tripleo | 14:19 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 14:19 |
| *** CLOUDOUTAGE has quit IRC | 14:19 | |
| bnemec | https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/options.py#L42 | 14:19 |
| dprince | bnemec: right, previously it was opt-in. Now its opt-out. | 14:19 |
| rpodolyaka | and we can just stop passing pm_password values when virtual power driver is used (as nova doesn't use it) | 14:20 |
| dprince | rpodolyaka: we can do that too | 14:20 |
| giulivo | rpodolyaka, bnemec dprince ^^ this is what I would try to do actually | 14:20 |
| rpodolyaka | and as for using of TRADITIONAL mode in general, as bnemec said it's no-win situation, but IMO, we'd better default to have it enabled and fix bugs early in the development cycle, even it can be painful for us... | 14:22 |
| bnemec | dprince: It was opt-in for a long time...and nobody did. | 14:22 |
| bnemec | And what rpodolyaka said. :-) | 14:22 |
| openstackgerrit | Jay Dobies proposed a change to openstack/tripleo-specs: Initial spec for the Tuskar Juno REST API changes https://review.openstack.org/94720 | 14:22 |
| *** edmund has joined #tripleo | 14:23 | |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Enable rsync daemon on swift-storage https://review.openstack.org/95173 | 14:25 |
| derekh_ | rlandy: If your interested some of the CI work I things we need to work on https://review.openstack.org/#/c/95026/ (It may change as people add comments/ideas) | 14:27 |
| rlandy | derekh_: thanks ... reading | 14:28 |
| giulivo | rpodolyaka, just to make sure, ironic doesn't use pm_password cause has its own db, right? | 14:37 |
| openstackgerrit | Lorcan Browne proposed a change to openstack/tripleo-image-elements: Add swiftclient library to swift https://review.openstack.org/95176 | 14:37 |
| *** zackf has joined #tripleo | 14:37 | |
| *** chuckC has joined #tripleo | 14:40 | |
| *** chuckC has quit IRC | 14:41 | |
| *** akrivoka has quit IRC | 14:42 | |
| *** jprovazn is now known as jprovazn_afk | 14:46 | |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-image-elements: Explicitly add the parted package. https://review.openstack.org/95177 | 14:47 |
| *** CLOUDOUTAGE has joined #tripleo | 14:50 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 14:50 |
| *** CLOUDOUTAGE has quit IRC | 14:50 | |
| *** hashar has joined #tripleo | 14:50 | |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Store swift account, container & object data on /mnt filesystem https://review.openstack.org/89847 | 14:52 |
| *** yamahata has joined #tripleo | 14:52 | |
| openstackgerrit | Lorcan Browne proposed a change to openstack/tripleo-image-elements: Add check_mk swift proxy diagnostic https://review.openstack.org/94607 | 14:53 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Avoid pasting ssh-key into pm_password https://review.openstack.org/95181 | 14:56 |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 15:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1321563 | 15:00 |
| uvirtbot | Launchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 15:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 15:00 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/diskimage-builder: Add support for source-repos gerrit refs https://review.openstack.org/90890 | 15:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 15:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 15:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 15:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Avoid pasting ssh-key into pm_password https://review.openstack.org/95181 | 15:00 |
| derekh_ | I seem to remember us trying to remove the DIB dependency on busybox at one stage, did I imagine that? | 15:11 |
| *** ifarkas has quit IRC | 15:17 | |
| *** rlandy has quit IRC | 15:19 | |
| *** saurabhs has joined #tripleo | 15:21 | |
| *** CLOUDOUTAGE has joined #tripleo | 15:21 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 15:21 |
| *** CLOUDOUTAGE has quit IRC | 15:21 | |
| *** chuckC has joined #tripleo | 15:27 | |
| greghaynes | oh look, our old friend cloudouta | 15:27 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-incubator: Avoid pasting ssh-key into pm_password https://review.openstack.org/95181 | 15:29 |
| *** TravT has joined #tripleo | 15:35 | |
| *** hashar has quit IRC | 15:37 | |
| *** hashar has joined #tripleo | 15:42 | |
| openstackgerrit | Flint Calvin proposed a change to openstack/tripleo-incubator: Making the necessary changes so that the value of metering_secret gets set appropriately in /etc/ceilometer/ceilometer.conf. https://review.openstack.org/94201 | 15:51 |
| *** eghobo has joined #tripleo | 15:51 | |
| *** CLOUDOUTAGE has joined #tripleo | 15:52 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 15:52 |
| *** CLOUDOUTAGE has quit IRC | 15:52 | |
| giulivo | actually, the virtual power driver module doesn't even care about pm_user and pm_host | 15:53 |
| giulivo | but I don't think there would be a point in updating it cause the development happens in ironic right? | 15:54 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Check for set -o pipefail https://review.openstack.org/83929 | 15:55 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Set -o pipefail new scripts https://review.openstack.org/95203 | 15:55 |
| SpamapS | hey what a surprise | 15:57 |
| SpamapS | R1 down | 15:57 |
| SpamapS | again | 15:57 |
| *** gcha has quit IRC | 16:01 | |
| SpamapS | derekh_: grrrr ... so ci-overcloud's controller has the newer mellanox driver.. supposedly | 16:02 |
| SpamapS | derekh_: that does not bode well | 16:03 |
| derekh_ | SpamapS: it was still on saucy, didn't look at driver version, its back up now everything I did is in the etherpad | 16:03 |
| derekh_ | SpamapS: was hopeing nodepool would just start using it | 16:04 |
| derekh_ | SpamapS: but that doesn't seem to have happened | 16:04 |
| derekh_ | I guess its at quota... | 16:05 |
| *** jistr has quit IRC | 16:05 | |
| derekh_ | SpamapS: exactly 40 instances tripleo-* in various states | 16:05 |
| SpamapS | derekh_: I'm thinking we need to write a dashboard for nodepool :) | 16:11 |
| SpamapS | derekh_: so we know what ones we can delete | 16:11 |
| SpamapS | derekh_: yesterday there were 27 h1.large's that nodepool didn't know about | 16:11 |
| SpamapS | derekh_: I can't get to the controller right now | 16:11 |
| *** morganfainberg_Z is now known as morganfainberg | 16:12 | |
| *** marun has joined #tripleo | 16:12 | |
| derekh_ | SpamapS: weird, I'm ssh'd in but ssh response stalls occasionally | 16:12 |
| derekh_ | SpamapS: ya a dashboard would be great | 16:13 |
| SpamapS | derekh_: that is classic mellanox fail behavior. | 16:13 |
| *** e0ne has quit IRC | 16:14 | |
| derekh_ | SpamapS: new ssh connections from the undercloud controller seem to be working | 16:14 |
| *** CLOUDOUTAGE has joined #tripleo | 16:23 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 16:23 |
| *** CLOUDOUTAGE has quit IRC | 16:23 | |
| *** asparks has joined #tripleo | 16:25 | |
| *** cwolferh has quit IRC | 16:26 | |
| *** akuznetsov has quit IRC | 16:31 | |
| SpamapS | derekh_: this may be something else. ILO is dropping connections too | 16:31 |
| derekh_ | SpamapS: ahh yes, it was for me too, sorry I forgot to mention it | 16:32 |
| *** jp_at_hp has quit IRC | 16:33 | |
| SpamapS | derekh_: thats a different hardware port entirely, but likely the same switch fabric.. | 16:33 |
| derekh_ | SpamapS: so possibly whatever is causing that is also causing connections to drop to VMs | 16:34 |
| SpamapS | aye | 16:34 |
| derekh_ | SpamapS: gotta run in coupld of minutes if jobs start running on R1 again you can look at http://goodsquishy.com/downloads/s_tripleo-jobs.html to see pass rates (gives you a rough idea if things are going ok) | 16:35 |
| SpamapS | derekh_: I think I'm going to take it to the NOC | 16:36 |
| SpamapS | derekh_: yesterday things were flying through | 16:36 |
| *** cwolferh has joined #tripleo | 16:36 | |
| derekh_ | SpamapS: they were all failing though | 16:36 |
| *** dkehn_ has joined #tripleo | 16:39 | |
| SpamapS | derekh_: I didn't say they were passing, they were flying through ;) | 16:39 |
| derekh_ | :-) | 16:39 |
| SpamapS | I'm honestly about to quit this job as CI cloud admin | 16:39 |
| SpamapS | It is awful. | 16:39 |
| * SpamapS -> Heat | 16:39 | |
| derekh_ | I'm starting to think we should drop all but a single overcloud job and only run on R2 , should give us some breathing room to redeploy and properly debug R1 | 16:39 |
| dprince | derekh_: not a horrible idea considering | 16:39 |
| SpamapS | +1 | 16:39 |
| SpamapS | I want to burn R1 | 16:39 |
| SpamapS | though I will backup the db | 16:39 |
| SpamapS | In fact I was thinking of just doing nova rebuilds and keeping it alive like that | 16:40 |
| derekh_ | we can maybe trash out some ideas on tuesday meeting | 16:40 |
| *** dkehnx has quit IRC | 16:40 | |
| *** dkehn has quit IRC | 16:40 | |
| SpamapS | I will be in UTC+1230 on tuesday... | 16:41 |
| *** dkehn has joined #tripleo | 16:41 | |
| derekh_ | plenty of coffee will get you through it :-) | 16:41 |
| derekh_ | ok gotta run | 16:42 |
| *** derekh_ has quit IRC | 16:42 | |
| SpamapS | if R1 is failing jobs and not running anything | 16:42 |
| SpamapS | actually | 16:42 |
| SpamapS | let's stop | 16:42 |
| SpamapS | I don't think this is absolutely mellanox failure | 16:42 |
| SpamapS | I mean, we need to update anyway | 16:43 |
| SpamapS | but I wonder if this is just good old fashioned routing fail.. seems like 10.10 <-> 10.10 is fine | 16:43 |
| *** lucasagomes is now known as lucas-afk | 16:44 | |
| *** asparks has quit IRC | 16:44 | |
| *** dkehn_ is now known as dkehnx | 16:45 | |
| *** bnemec is now known as beekneemech | 16:49 | |
| SpamapS | hm and now SSH to ci-overcloud.tripleo.org is working again | 16:50 |
| *** nati_ueno has joined #tripleo | 16:50 | |
| *** CLOUDOUTAGE has joined #tripleo | 16:54 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 16:54 |
| *** CLOUDOUTAGE has quit IRC | 16:54 | |
| *** akuznetsov has joined #tripleo | 16:57 | |
| *** nati_uen_ has joined #tripleo | 17:03 | |
| *** pblaho has quit IRC | 17:04 | |
| *** nati_ueno has quit IRC | 17:06 | |
| *** marun has quit IRC | 17:06 | |
| devananda | hi guys! how do you feel about gatign disk-image-builder on ironic? | 17:07 |
| devananda | i realized a few days ago it's asymmetric right now | 17:07 |
| *** nati_uen_ has quit IRC | 17:07 | |
| devananda | as in, ironic is implicitly using dib (to build the deploy ramdisk) but changes in dib aren't checking whether they break ironic | 17:08 |
| greghaynes | SpamapS: working again as in magically started working or did something change? | 17:08 |
| *** nati_ueno has joined #tripleo | 17:08 | |
| *** marun has joined #tripleo | 17:08 | |
| *** akuznetsov has quit IRC | 17:10 | |
| beekneemech | devananda: Makes sense to me. | 17:11 |
| SpamapS | greghaynes: magic | 17:11 |
| SpamapS | greghaynes: and now it's down again | 17:11 |
| greghaynes | :_( | 17:14 |
| devananda | https://review.openstack.org/95220 <-- adding ironic test to dib's check/gate | 17:18 |
| SpamapS | opening a JIRA ticket with the hpcloud NOC.. something is quite broken | 17:20 |
| SpamapS | I think it's an arp thing | 17:23 |
| SpamapS | if I ping the external IP from inside the rack, external connections start flowing | 17:24 |
| greghaynes | two hosts with same IP? | 17:24 |
| SpamapS | oh very well could be | 17:24 |
| SpamapS | or an accidental bridge | 17:24 |
| *** CLOUDOUTAGE has joined #tripleo | 17:24 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 17:24 |
| *** CLOUDOUTAGE has quit IRC | 17:24 | |
| greghaynes | yea, sporatic packet blackholing and then coming back is standard arp cache battling | 17:25 |
| SpamapS | hm | 17:25 |
| *** jprovazn_afk has quit IRC | 17:30 | |
| *** jang1 has joined #tripleo | 17:31 | |
| openstackgerrit | Fabio Giannetti proposed a change to openstack/tripleo-image-elements: Ceilometer Service Update/Upgrade in TripleO https://review.openstack.org/94500 | 17:32 |
| *** nati_ueno has quit IRC | 17:35 | |
| SpamapS | greghaynes: so it's not a duplicate host.. removing the ip and arping for it does not reveal a second box | 17:35 |
| SpamapS | must be a bridging loop | 17:35 |
| greghaynes | aye | 17:35 |
| SpamapS | oohhh hmm... | 17:36 |
| SpamapS | I see 10.10.16 traffic on vlan25 | 17:36 |
| *** athomas has quit IRC | 17:40 | |
| *** john3213 has joined #tripleo | 17:47 | |
| *** e0ne has joined #tripleo | 17:49 | |
| *** e0ne has quit IRC | 17:52 | |
| *** e0ne has joined #tripleo | 17:52 | |
| *** john3213 has left #tripleo | 17:52 | |
| *** e0ne has quit IRC | 17:55 | |
| *** CLOUDOUTAGE has joined #tripleo | 17:55 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 17:55 |
| *** CLOUDOUTAGE has quit IRC | 17:55 | |
| SpamapS | I wonder if they stole our IP range or something | 18:00 |
| *** rcarrill` has joined #tripleo | 18:05 | |
| *** rcarrillocruz has quit IRC | 18:06 | |
| *** marun has quit IRC | 18:08 | |
| *** e0ne has joined #tripleo | 18:11 | |
| *** OldCrowEW has quit IRC | 18:14 | |
| *** OldCrowEW has joined #tripleo | 18:16 | |
| *** dprince has quit IRC | 18:16 | |
| Ng | we've had bridging loops at least twice before, and scratched our heads and gone to them | 18:26 |
| Ng | the most recent bonkers thing, they were seeing the same mac on two ports, but upgrading the switch firmware and rebooting it, made it go away, so we chalked that up to a switch firmware bug | 18:26 |
| *** CLOUDOUTAGE has joined #tripleo | 18:26 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 18:26 |
| *** CLOUDOUTAGE has quit IRC | 18:26 | |
| *** OldCrowEW has quit IRC | 18:30 | |
| *** e0ne has quit IRC | 18:30 | |
| *** e0ne has joined #tripleo | 18:31 | |
| *** e0ne has quit IRC | 18:35 | |
| *** OldCrowEW has joined #tripleo | 18:37 | |
| SpamapS | Ng: this is suspiciously like that | 18:38 |
| SpamapS | Ng: I'm getting nowhere with my JIRA ticket | 18:38 |
| SpamapS | But we're definitely completely down as connections drop in and out constantly | 18:39 |
| Ng | SpamapS: oh? | 18:39 |
| Ng | SpamapS: as in, they're not helping you on the ticket? | 18:39 |
| SpamapS | Ng: yeah, been ignored by NOC and Network | 18:39 |
| SpamapS | ticket is sitting (NET-4002) | 18:39 |
| *** akuznetsov has joined #tripleo | 18:43 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Add Swift roles required to poll for Swift usage https://review.openstack.org/94730 | 18:48 |
| *** andreaf_ has quit IRC | 18:48 | |
| *** jogo is now known as flashgordon | 18:52 | |
| *** CLOUDOUTAGE has joined #tripleo | 18:57 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 18:57 |
| *** CLOUDOUTAGE has quit IRC | 18:57 | |
| *** pelix has quit IRC | 18:59 | |
| *** jdob has quit IRC | 19:08 | |
| *** akuznetsov has quit IRC | 19:14 | |
| *** dshulyak_ has joined #tripleo | 19:19 | |
| *** lsmola has quit IRC | 19:20 | |
| *** chuckC has quit IRC | 19:28 | |
| *** CLOUDOUTAGE has joined #tripleo | 19:28 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- Jobs failing on R1 https://etherpad.openstack.org/p/cloud-outage | 19:28 |
| *** CLOUDOUTAGE has quit IRC | 19:28 | |
| *** e0ne has joined #tripleo | 19:40 | |
| * SpamapS goes on meatspace errands for a while | 19:40 | |
| *** nati_ueno has joined #tripleo | 19:54 | |
| *** dshulyak_ has quit IRC | 20:20 | |
| lifeless | devananda: we'd be happy to have symmetric gating | 20:22 |
| *** dshulyak_ has joined #tripleo | 20:22 | |
| devananda | lifeless: great :) | 20:22 |
| *** dshulyak_ has quit IRC | 20:23 | |
| lifeless | there is some impl complexity which we're going through with heat as well related to whether you consume and test releases or not | 20:23 |
| lifeless | since dib is a thing that itself consumes things | 20:23 |
| *** dshulyak_ has joined #tripleo | 20:23 | |
| lifeless | but I suspect you just need the baseline dib, not tripleo-image-elements, right? | 20:23 |
| devananda | right | 20:24 |
| *** OldCrowEW has quit IRC | 20:26 | |
| *** chuckC has joined #tripleo | 20:26 | |
| *** OldCrowEW has joined #tripleo | 20:26 | |
| *** dshulyak_ has quit IRC | 20:28 | |
| -openstackstatus- NOTICE: Gerrit will be offline for about 20 minutes in order to rename some projects starting at 21:00 UTC. | 20:34 | |
| *** nati_uen_ has joined #tripleo | 20:36 | |
| *** nati_ueno has quit IRC | 20:39 | |
| *** nati_ueno has joined #tripleo | 20:43 | |
| *** nati_uen_ has quit IRC | 20:45 | |
| *** casanch1_ has joined #tripleo | 20:46 | |
| *** nati_uen_ has joined #tripleo | 20:46 | |
| *** nati_ueno has quit IRC | 20:48 | |
| *** casanch1 has quit IRC | 20:49 | |
| *** casanch1_ has quit IRC | 20:51 | |
| *** nati_uen_ has quit IRC | 20:53 | |
| *** nati_uen_ has joined #tripleo | 20:55 | |
| stevebaker | lifeless, I spent yesterday looking a building local pip mirrors for dib devstack gating. I think there will be push-back on building a comprehensive mirror. Pip installing from pypi.openstack.org has worked plenty fast for devstack gating so they'll want to do the same for image building | 20:57 |
| *** nati_ueno has joined #tripleo | 20:58 | |
| *** hashar has joined #tripleo | 20:59 | |
| stevebaker | lifeless, I tried using the pypi element to have a local mirror only of os-*-config git-built packages, but the resulting image still installed from released versions. | 20:59 |
| stevebaker | lifeless, tl;dr I *really* prefer my original approach ;) https://review.openstack.org/#/c/92035/ | 21:00 |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1321943 | 21:00 |
| uvirtbot | Launchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1321563 | 21:00 |
| uvirtbot | Launchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1318767 | 21:00 |
| uvirtbot | Launchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1314978 | 21:00 |
| uvirtbot | Launchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1315355 | 21:00 |
| uvirtbot | Launchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1316675 | 21:00 |
| uvirtbot | Launchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1320090 | 21:00 |
| uvirtbot | Launchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** AaronGr has left #tripleo | 21:00 | |
| *** nati_uen_ has quit IRC | 21:00 | |
| *** jtomasek has quit IRC | 21:11 | |
| *** dshulyak has quit IRC | 21:14 | |
| *** bogdando has quit IRC | 21:15 | |
| *** dshulyak has joined #tripleo | 21:16 | |
| *** bogdando has joined #tripleo | 21:16 | |
| *** zackf has quit IRC | 21:19 | |
| *** edmund has quit IRC | 21:21 | |
| lifeless | stevebaker: the local mirror will only have 2-3 pacakges in it | 21:31 |
| lifeless | stevebaker: its not a mirror really, its a repo | 21:31 |
| *** hashar has quit IRC | 21:33 | |
| *** hashar has joined #tripleo | 21:34 | |
| -openstackstatus- NOTICE: Gerrit is offline in order to rename some projects. ETA: 22:00. | 21:36 | |
| *** ChanServ changes topic to "Gerrit is offline in order to rename some projects. ETA: 22:00." | 21:36 | |
| *** nati_ueno has quit IRC | 21:37 | |
| stevebaker | lifeless, I did build a local repo which appeared to be valid, but the elements still installed from downloads - not sure what is going on there | 21:38 |
| Lotus907efi | anyone who can tell me where the data that is supposed to be available at http://169.254.169.254/2009-04-04/meta-data/instance-id should come from? | 21:39 |
| Lotus907efi | I have a group of overcloud systems where the cloud-init runs are not successful because although that IP is on the systems, the data that is supposed to be there is not there | 21:39 |
| Lotus907efi | I am trying to diagnose why the heat create of the overcloud failed | 21:40 |
| *** giulivo has quit IRC | 21:42 | |
| *** Penick has joined #tripleo | 21:45 | |
| *** weshay has quit IRC | 21:51 | |
| *** hashar has quit IRC | 21:51 | |
| *** jang1 has quit IRC | 21:59 | |
| *** yamahata has quit IRC | 22:10 | |
| *** lucas-afk has quit IRC | 22:12 | |
| *** lucas-afk has joined #tripleo | 22:13 | |
| Lotus907efi | From the keystone.log file on the undercloud: ERROR keystone.common.wsgi [-] object of type 'NoneType' has no len() | 22:13 |
| *** openstackgerrit has quit IRC | 22:14 | |
| *** yamahata has joined #tripleo | 22:14 | |
| *** openstackgerrit has joined #tripleo | 22:15 | |
| *** OldCrowEW has quit IRC | 22:16 | |
| *** openstackstatus has quit IRC | 22:18 | |
| *** openstack has joined #tripleo | 22:19 | |
| *** openstackstatus has joined #tripleo | 22:20 | |
| *** ChanServ sets mode: +v openstackstatus | 22:20 | |
| *** nati_ueno has joined #tripleo | 22:23 | |
| *** nati_ueno has quit IRC | 22:26 | |
| *** ChanServ changes topic to "https://etherpad.openstack.org/p/tripleo-ci-r1-trusty | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 22:29 | |
| *** ddieterly has quit IRC | 22:34 | |
| *** akuznetsov has joined #tripleo | 22:37 | |
| *** akuznetsov has quit IRC | 22:47 | |
| *** TravT has quit IRC | 22:56 | |
| *** morganfainberg is now known as morganfainberg_Z | 23:01 | |
| *** eguz has joined #tripleo | 23:02 | |
| *** eghobo has quit IRC | 23:06 | |
| *** nati_ueno has joined #tripleo | 23:09 | |
| *** nati_ueno has quit IRC | 23:13 | |
| *** ewindisch has quit IRC | 23:24 | |
| *** nati_ueno has joined #tripleo | 23:26 | |
| *** tzumainn has quit IRC | 23:49 | |
| *** e0ne has quit IRC | 23:57 | |
| SpamapS | Lotus907efi: that IP will be answered by neutron-metadata-agent | 23:57 |
| SpamapS | Lotus907efi: which will look up the source IP, and map that to the currently attached nova instance. | 23:58 |
| Lotus907efi | well, apparently that is not running | 23:58 |
| Lotus907efi | Greg says I have a keystone problem on the undercloud | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!