*** e0ne has quit IRC | 00:02 | |
*** CaptTofu has joined #tripleo | 00:04 | |
derekh | trying to avoid the java console, how does one go about using the text console ? | 00:04 |
---|---|---|
*** kevinbenton is now known as kevin | 00:05 | |
*** kevin is now known as kevinbenton | 00:05 | |
SpamapS | derekh: ssh to it | 00:06 |
SpamapS | derekh: that is the same auth details as we use for ipmi | 00:06 |
derekh | SpamapS: ahh ok, got it thanks | 00:10 |
derekh | hmmm, same problem as last time, SYN/ACK not making it back to connecting host | 00:13 |
derekh | lifeless: I see you rebooted last time, assuming I can't figure anything out in the next few minutes will do the same | 00:18 |
greghaynes | derekh: IIRC http://lists.openstack.org/pipermail/openstack-dev/2014-February/028009.html was why the reboot happened | 00:23 |
openstackgerrit | james Polley proposed a change to openstack/tripleo-incubator: Add a missing line of config to the sample squid config https://review.openstack.org/76715 | 00:23 |
*** matsuhashi has joined #tripleo | 00:23 | |
greghaynes | In case you hadnt already ready that | 00:24 |
derekh | greghaynes: yup, thats what I'm seeing at the moment | 00:24 |
derekh | greghaynes: thanks | 00:24 |
derekh | rebooting .... | 00:27 |
*** saurabhs1 has quit IRC | 00:29 | |
*** rbrady has joined #tripleo | 00:31 | |
*** saurabhs has joined #tripleo | 00:32 | |
*** UtahDave has joined #tripleo | 00:32 | |
*** saurabhs has quit IRC | 00:35 | |
rbrady | SpamapS: ping | 00:36 |
*** cd-undercloud has joined #tripleo | 00:40 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 00:40 |
*** cd-undercloud has quit IRC | 00:40 | |
*** shakayumi has joined #tripleo | 00:41 | |
*** shakayumi has quit IRC | 00:41 | |
*** shakayumi has joined #tripleo | 00:42 | |
fungi | looks like derekh is on the ball | 00:43 |
fungi | i was about to give you a heads up, but you're clearly hammering on the problem already for a while. best of luck! | 00:44 |
*** bauzas has quit IRC | 00:44 | |
derekh | fungi: same issue as last time, server power cycled the host when it comes back will see what I have to deal with | 00:44 |
derekh | fungi: thanks, I should have done the same for you, sorry | 00:45 |
fungi | sounds like the sort of fun i'm glad i no longer have to deal with myself | 00:45 |
fungi | no worries. just glad you know--it's not really adversely impacting us | 00:45 |
*** dshurtle has joined #tripleo | 00:46 | |
rbrady | derekh: do you have a second for a heat template question? | 00:47 |
derekh | rbrady: ask away, | 00:47 |
*** saurabhs has joined #tripleo | 00:49 | |
rbrady | lifeless wanted me to use the value from neutron.ovs.local_ip in a conf file template, but use that value to get/make the subnet. e.g. value is 192.0.2.7 and I need to change it to 192.0.2.0/24 | 00:49 |
*** jp_at_hp has joined #tripleo | 00:50 | |
rbrady | derekh: I haven't figured out how that is possible in a mustache template | 00:51 |
rbrady | derekh: any guidance to point me in the right direction? | 00:51 |
derekh | rbrady: I don't think it is possible in a mustache template | 00:51 |
*** weshay has joined #tripleo | 00:52 | |
derekh | rbrady: maybe it is possible to use a function available in the heat template or just to pass the value you need into the heat template | 00:52 |
*** saurabhs has left #tripleo | 00:53 | |
*** jp_at_hp has quit IRC | 00:53 | |
derekh | rbrady: or an executable os-apply-config template but I've never done one of those | 00:54 |
rbrady | derekh: executable os-apply-config template? | 00:54 |
derekh | rbrady: https://github.com/openstack/os-apply-config/blob/master/os_apply_config/apply_config.py#L99-L100 | 00:55 |
*** e0ne has joined #tripleo | 00:57 | |
*** dshurtle has quit IRC | 00:58 | |
*** dshurtle has joined #tripleo | 00:58 | |
*** dshurtle has quit IRC | 00:58 | |
*** dshurtle has joined #tripleo | 00:58 | |
*** dshurtle has quit IRC | 00:58 | |
*** dshurtle has joined #tripleo | 00:59 | |
*** e0ne has quit IRC | 01:02 | |
*** dshurtle has quit IRC | 01:02 | |
*** dshurtle has joined #tripleo | 01:03 | |
*** dshurtle has quit IRC | 01:08 | |
*** dshurtle has joined #tripleo | 01:09 | |
adam_g | hmm.. devtest_undercloud.sh failing for me with http://paste.ubuntu.com/7002765/ | 01:12 |
adam_g | is that a tmpfs + lack of RAM issue i've seen mentioned? | 01:12 |
rbrady | derekh: thanks for the insight, going to give the executable template a try | 01:20 |
*** UtahDave has quit IRC | 01:21 | |
derekh | rbrady: np, I've never used it just seen it in the code but worth a try, I'm not sure if we are using it anywhere else yet | 01:21 |
rbrady | adam_g: looks like you've hit this bug: https://bugs.launchpad.net/tripleo/+bug/1285269 | 01:22 |
adam_g | rbrady, ah, could be | 01:23 |
adam_g | thanks | 01:23 |
rbrady | adam_g: np | 01:24 |
*** rbrady has quit IRC | 01:33 | |
*** rbrady has joined #tripleo | 01:40 | |
*** nosnos has joined #tripleo | 01:46 | |
*** cd-undercloud has joined #tripleo | 01:48 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 01:48 |
*** cd-undercloud has quit IRC | 01:48 | |
openstackgerrit | Dustin J. Mitchell proposed a change to openstack/diskimage-builder: Update README formatting and content https://review.openstack.org/75480 | 01:55 |
*** newell has quit IRC | 01:56 | |
*** e0ne has joined #tripleo | 01:57 | |
*** e0ne has quit IRC | 02:02 | |
*** dkehn__ is now known as dkehn | 02:15 | |
*** dshurtle has quit IRC | 02:26 | |
*** jcooley_ has quit IRC | 02:37 | |
derekh | everything seems back up on ci-overcloud but vm's not getting ip's | 02:41 |
derekh | any ideas out there | 02:42 |
*** julim has joined #tripleo | 02:44 | |
*** cody-somerville has joined #tripleo | 02:47 | |
derekh | lifeless, devananda, Ng, SpamapS_, jog0_, GheRivero, derekh, dprince, slagle I got all services back up, vm's still don't seem to be getting IP's so I've mist something, anybody want to take over ? | 02:49 |
derekh | fungi: I gotta go, VM's not getting net access, will try and sort it out in a few hours if nobody else has picked up on it before then | 02:56 |
*** e0ne has joined #tripleo | 02:57 | |
*** derekh is now known as derekh_afk | 02:57 | |
*** e0ne has quit IRC | 03:01 | |
*** CaptTofu has quit IRC | 03:04 | |
*** lblanchard has joined #tripleo | 03:14 | |
*** cd-undercloud has joined #tripleo | 03:26 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 03:26 |
*** cd-undercloud has quit IRC | 03:26 | |
*** cd-undercloud has joined #tripleo | 03:28 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 03:28 |
*** cd-undercloud has quit IRC | 03:28 | |
*** ramishra has joined #tripleo | 03:34 | |
*** cd-undercloud has joined #tripleo | 03:41 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 03:41 |
*** cd-undercloud has quit IRC | 03:41 | |
*** coolsvap has quit IRC | 03:42 | |
*** weshay has quit IRC | 03:56 | |
*** e0ne has joined #tripleo | 03:57 | |
*** e0ne has quit IRC | 04:01 | |
*** jcooley_ has joined #tripleo | 04:17 | |
*** cd-undercloud has joined #tripleo | 04:28 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 04:28 |
*** cd-undercloud has quit IRC | 04:28 | |
openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Call virsh managedsave-remove https://review.openstack.org/75729 | 04:30 |
*** saurabhs has joined #tripleo | 04:35 | |
*** jcooley_ has quit IRC | 04:46 | |
*** lblanchard has quit IRC | 04:47 | |
*** jcooley_ has joined #tripleo | 04:52 | |
openstackgerrit | Richard Su proposed a change to openstack/diskimage-builder: Additional swift storage service mappings https://review.openstack.org/76753 | 04:55 |
*** e0ne has joined #tripleo | 04:57 | |
*** jcooley_ has quit IRC | 04:59 | |
*** matsuhashi has quit IRC | 05:00 | |
*** cd-undercloud has joined #tripleo | 05:01 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 05:01 |
*** cd-undercloud has quit IRC | 05:01 | |
*** e0ne has quit IRC | 05:02 | |
*** coolsvap has joined #tripleo | 05:04 | |
*** CaptTofu has joined #tripleo | 05:05 | |
openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: rsync configurations to support multi-node swift https://review.openstack.org/76105 | 05:06 |
*** CaptTofu has quit IRC | 05:09 | |
*** coolsvap has quit IRC | 05:16 | |
*** coolsvap has joined #tripleo | 05:17 | |
*** sdake_ has joined #tripleo | 05:19 | |
*** vkozhukalov has joined #tripleo | 05:36 | |
*** cd-undercloud has joined #tripleo | 05:46 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 05:46 |
*** cd-undercloud has quit IRC | 05:46 | |
*** cd-undercloud has joined #tripleo | 05:48 | |
cd-undercloud | ************** overcloud complete status=6 ************ | 05:48 |
*** cd-undercloud has quit IRC | 05:48 | |
*** killer_prince has quit IRC | 05:54 | |
*** sballe has joined #tripleo | 05:55 | |
*** e0ne has joined #tripleo | 05:57 | |
openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Use xtradb-cluster in place of mysql https://review.openstack.org/75540 | 05:57 |
*** sballe has quit IRC | 05:59 | |
*** e0ne has quit IRC | 06:01 | |
*** killer_prince has joined #tripleo | 06:03 | |
*** rpodolyaka1 has joined #tripleo | 06:07 | |
*** lazy_prince has joined #tripleo | 06:11 | |
*** sdake_ has quit IRC | 06:13 | |
*** vkozhukalov has quit IRC | 06:18 | |
openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: Design fiction porting undercloud-bm to software-config https://review.openstack.org/72533 | 06:21 |
*** andreaf has joined #tripleo | 06:23 | |
openstackgerrit | Steve Baker proposed a change to openstack/tripleo-heat-templates: Design fiction porting undercloud-bm to software-config https://review.openstack.org/72533 | 06:26 |
*** cd-undercloud has joined #tripleo | 06:38 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 06:38 |
*** cd-undercloud has quit IRC | 06:38 | |
*** saju_m has joined #tripleo | 06:48 | |
*** pblaho has joined #tripleo | 06:56 | |
*** e0ne has joined #tripleo | 06:57 | |
killer_prince | hi all, i am trying to create a RHEL 6.5 golden image using diskimage-builder with a custom element which has dependency on os-refresh-config. but it keeps failing. here are the logs http://paste.openstack.org/show/70123/. Can someone please help me..? | 06:58 |
*** e0ne has quit IRC | 07:02 | |
*** CaptTofu has joined #tripleo | 07:05 | |
*** CaptTofu has quit IRC | 07:11 | |
*** jtomasek has joined #tripleo | 07:15 | |
*** cd-undercloud has joined #tripleo | 07:17 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 07:17 |
*** cd-undercloud has quit IRC | 07:17 | |
*** cd-undercloud has joined #tripleo | 07:18 | |
cd-undercloud | ************** overcloud complete status=6 ************ | 07:18 |
*** cd-undercloud has quit IRC | 07:18 | |
lifeless | derekh_afk: looking | 07:30 |
lifeless | derekh_afk: neutron has flip-flopped which agent ID it's using, again. | 07:30 |
lifeless | derekh_afk: should be up now for l3, moving onto DHCP | 07:31 |
*** sirushti is now known as shortstop | 07:32 | |
*** rpodolyaka1 has left #tripleo | 07:32 | |
*** jprovazn has joined #tripleo | 07:32 | |
lifeless | derekh_afk: probably wants a rolling restart on compute | 07:40 |
lifeless | ci-overcloud back up | 07:42 |
lifeless | StevenK: did you get setup-baremetal beaten into shape? | 07:42 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Create stack in right order https://review.openstack.org/76267 | 07:45 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Better error handling of Overcloud actions https://review.openstack.org/76207 | 07:45 |
*** mrunge has joined #tripleo | 07:51 | |
*** vkozhukalov has joined #tripleo | 07:53 | |
*** jcooley_ has joined #tripleo | 07:56 | |
*** e0ne has joined #tripleo | 07:57 | |
*** e0ne has quit IRC | 08:02 | |
*** cd-undercloud has joined #tripleo | 08:05 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 08:05 |
*** cd-undercloud has quit IRC | 08:05 | |
*** saju_m has quit IRC | 08:10 | |
*** saju_m has joined #tripleo | 08:10 | |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Create stack in right order https://review.openstack.org/76267 | 08:14 |
openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Different Flavors for Tuskar roles https://review.openstack.org/75895 | 08:15 |
marios | ping - re https://review.openstack.org/#/c/76267/ <-- is it WIP? | 08:20 |
marios | lsmola: ^^^ | 08:20 |
lsmola | marios: yes, but feel free to review :-) | 08:21 |
marios | :) | 08:21 |
marios | lsmola: thx, wasn't sure | 08:22 |
lsmola | marios: I am not figting with tox, it wants to regenerate config.sample for some reason | 08:22 |
marios | lsmola: lets talk about this if we have call today. do you think its worth pulling out the stack_id fix so we can land that at least? | 08:22 |
*** ramishra has quit IRC | 08:23 | |
*** ramishra has joined #tripleo | 08:23 | |
lsmola | marios: hm, I guess it can go together | 08:23 |
lsmola | marios: i kind of depends on the different order of calling | 08:24 |
lsmola | marios: otherwise we would have to call Overcloud update on the end | 08:24 |
*** saju_m has quit IRC | 08:25 | |
lsmola | marios: do you have also issue with generate.sample, when you run tox -ept27 ? | 08:25 |
*** saju_m has joined #tripleo | 08:26 | |
marios | lsmola: haven't seen that one yet | 08:26 |
lsmola | marios: ImportError: No module named stevedore.named | 08:26 |
lsmola | marios: hm probably my bad :-) | 08:29 |
lsmola | marios: I need to update test requirements | 08:29 |
*** al has quit IRC | 08:29 | |
lsmola | marios: seems tox can't handle that | 08:30 |
*** al has joined #tripleo | 08:30 | |
*** bauzas has joined #tripleo | 08:30 | |
*** jistr has joined #tripleo | 08:30 | |
*** rdopieralski has joined #tripleo | 08:30 | |
*** rlandy has joined #tripleo | 08:36 | |
*** jcooley_ has quit IRC | 08:39 | |
*** jrist has quit IRC | 08:41 | |
*** saurabhs has quit IRC | 08:41 | |
rpodolyaka | morning all | 08:42 |
lsmola | morning | 08:43 |
lsmola | marios: btw. do we need to still clone the templates and put the patch in config? when we have the templates in requirements? | 08:43 |
*** cd-undercloud has joined #tripleo | 08:46 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 08:46 |
*** cd-undercloud has quit IRC | 08:46 | |
marios | lsmola: yeah because the templates themselves aren't packaged as part of the tripleo-heat-templates | 08:46 |
lsmola | marios: ha, ok | 08:46 |
*** shardy_afk is now known as shardy | 08:46 | |
lsmola | marios: I thought it is packaging the whole repo | 08:47 |
marios | lsmola: i also want to revisit the 'pulling from git master' - we didn't publish to pypi because 'it will go away' | 08:47 |
marios | lsmola: but for now is fine | 08:47 |
*** e0ne has joined #tripleo | 08:47 | |
marios | lsmola: you can see in tripleo-heat-templates setup.cfg this is where the package is described | 08:47 |
lsmola | marios: ok | 08:47 |
openstackgerrit | A change was merged to openstack/tuskar-ui: Add overcloud_role method to Node https://review.openstack.org/76348 | 08:48 |
openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Make the node details page prettier https://review.openstack.org/75860 | 08:48 |
openstackgerrit | A change was merged to openstack/tuskar-ui: Update overcloud role page according to wireframes https://review.openstack.org/75812 | 08:51 |
*** saju_m has quit IRC | 08:52 | |
openstackgerrit | A change was merged to openstack/tuskar-ui: Actually get node profiles from the API https://review.openstack.org/74762 | 08:53 |
openstackgerrit | Yongli He proposed a change to openstack/tuskar-ui: Exception message should not be localize https://review.openstack.org/76797 | 08:54 |
*** jrist has joined #tripleo | 08:54 | |
openstackgerrit | Yongli He proposed a change to openstack/tuskar: Exception message should not be localize https://review.openstack.org/76799 | 08:55 |
*** hashar has joined #tripleo | 08:55 | |
lifeless | marios: I don't think t-h-t is going anywhere fast | 08:57 |
lifeless | marios: changing as the python logic moves to heat, yes but the package + template, noooo | 08:57 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Create stack in right order https://review.openstack.org/76267 | 08:57 |
openstackgerrit | A change was merged to openstack/tuskar-ui: Make the node details page prettier https://review.openstack.org/75860 | 08:58 |
marios | lifeless: right, and agree | 08:59 |
*** ifarkas has joined #tripleo | 09:01 | |
*** martyntaylor has joined #tripleo | 09:04 | |
*** CaptTofu has joined #tripleo | 09:06 | |
*** martyntaylor has quit IRC | 09:09 | |
*** martyntaylor has joined #tripleo | 09:10 | |
*** CaptTofu has quit IRC | 09:11 | |
*** hashar_ has joined #tripleo | 09:12 | |
lsmola | lifeless: marios so it make sense to turn it into proper egg, right? templates included | 09:12 |
derekh_afk | lifeless: thanks, when the controller came back the neutron services came up with .novalocal in hostname | 09:15 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Create stack in right order https://review.openstack.org/76267 | 09:15 |
*** hashar has quit IRC | 09:15 | |
derekh_afk | lifeless: I changed hostname back and restarted those services and they started reporting in withou .nocalocal | 09:15 |
derekh_afk | lifeless: what did I miss? | 09:15 |
*** cd-undercloud has joined #tripleo | 09:27 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 09:27 |
*** cd-undercloud has quit IRC | 09:27 | |
*** ramishra has quit IRC | 09:29 | |
*** cd-undercloud has joined #tripleo | 09:29 | |
cd-undercloud | ************** overcloud complete status=7 ************ | 09:29 |
*** cd-undercloud has quit IRC | 09:29 | |
openstackgerrit | A change was merged to openstack/tripleo-image-elements: Revert "Workaround broken Neutron requirements." https://review.openstack.org/76474 | 09:32 |
lifeless | derekh_afk: In my report on the first incident I noted they'd move to .novalocal | 09:33 |
lifeless | derekh_afk: and filed a bug | 09:33 |
lifeless | derekh_afk: and moved the network etc to the new agent instance | 09:33 |
lifeless | derekh_afk: so when you switched agent back around, you disabled the working agent and enabled the one without the network configured | 09:33 |
*** e0ne has quit IRC | 09:34 | |
*** andreaf has quit IRC | 09:34 | |
derekh_afk | lifeless: ahh crap, I missread your report (I did read it) | 09:34 |
*** e0ne has joined #tripleo | 09:34 | |
lifeless | derekh_afk: neutron net-list-on-dhcp-agent + neutron router-list-on-l3-agent to diagnose this | 09:34 |
*** derekh_afk is now known as derekh | 09:34 | |
derekh | lifeless: ok, thanks, will do better next time | 09:35 |
lifeless | derekh: heh, I think you did fine | 09:36 |
lifeless | derekh: we need more ops docs | 09:37 |
derekh | lifeless: yup | 09:37 |
lifeless | derekh: you communicated clearly; you didn't delay, but did look closely. all good stuff. | 09:37 |
lifeless | derekh: were we down 14 hours? no. :) | 09:37 |
lifeless | derekh: I haven't done a rolling restart of nova-compute, I think that will be needed to clear some of the error status VMs, but... I think we should try to gather some details to add into the bug this time | 09:38 |
StevenK | lifeless: I *think* I have. It now works for new mode setups, tomorrow mornings plan is to take an axe to creates-nodes and testenv to create a legacy mode JSON and see if setup-baremetal still loves me. | 09:38 |
lifeless | since its not so overwhelming as last time | 09:38 |
StevenK | lifeless: Bah, sprinkle in correct spellings to taste. | 09:38 |
*** lucasagomes has joined #tripleo | 09:38 | |
lifeless | StevenK: git checkout origin/master; devtest_testenv.sh ? | 09:39 |
derekh | lifeless: I would have said 8 hours, when I sent my first message it was only down a few minutes | 09:39 |
StevenK | lifeless: It isn't quite that simple, but close. | 09:39 |
lifeless | derekh: well, still < 14 :) | 09:39 |
derekh | lifeless: as I was using it at the time and lost connectivity to my vm | 09:39 |
lifeless | derekh: I was getting drunk with dad at the time ;) | 09:39 |
StevenK | lifeless: It is probably going to be git co -- <testenv/create-nodes> in the cache and then throw devtest.sh at the wall | 09:40 |
* derekh needs a few drink too :-) | 09:40 | |
lifeless | StevenK: th[ats not a combo that we use though | 09:40 |
lifeless | StevenK: I think you want --use-existing + a testenv, not mixed scripts | 09:41 |
StevenK | I would have preferred to have setup-baremetal run against the undercloud for the new setup, but it didn't come up | 09:41 |
StevenK | lifeless: Why, testenv didn't change much, and I want a properly populated JSON with node-macs. | 09:42 |
lifeless | simpler to reproduce what CI does | 09:42 |
lifeless | which is latest code, but reuse envirnomnet | 09:42 |
openstackgerrit | A change was merged to openstack/tuskar: Better error handling of Overcloud actions https://review.openstack.org/76207 | 09:43 |
lifeless | StevenK: look at what toci_gate actually runs | 09:43 |
lifeless | StevenK: I mean, fundamentally - up to you, but I'd clean-env,d evtest_testenv in trunk, swich to my feature and run with --existing-environment | 09:44 |
*** ramishra has joined #tripleo | 09:45 | |
*** jp_at_hp has joined #tripleo | 09:47 | |
lifeless | jp_at_hp: could you try https://review.openstack.org/#/c/76636/ | 09:52 |
*** d0ugal has quit IRC | 09:54 | |
*** mrunge is now known as mrunge_away | 09:57 | |
mordred | Ng, lifeless: the rabbit cluster work is started by someone? or tbd? | 10:08 |
lifeless | nearly done | 10:09 |
lifeless | jprovazn <--- | 10:09 |
*** pbelanyi has joined #tripleo | 10:09 | |
*** jeremydei has joined #tripleo | 10:09 | |
*** athomas has quit IRC | 10:09 | |
jprovazn | mordred: lifeless: yep. reviews welcome: https://review.openstack.org/#/c/70238/ and then https://review.openstack.org/#/c/68392/ | 10:10 |
mordred | jprovazn: cool! thanks | 10:11 |
jprovazn | np | 10:11 |
*** Hefeweizen has quit IRC | 10:13 | |
*** cd-undercloud has joined #tripleo | 10:17 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 10:17 |
*** cd-undercloud has quit IRC | 10:17 | |
*** akrivoka has joined #tripleo | 10:28 | |
*** d0ugal has joined #tripleo | 10:31 | |
killer_prince | i created a new element and then created a os-refresh-config/configure.d and placed a script under this directory. I also defined dependency on os-refresh-config for this new element. when i am building the image using this element, i am not able to find this script on the image under /opt/stack/os-config-refresh/configure.d | 10:32 |
killer_prince | any idea what i missed in the whole step ? | 10:32 |
*** athomas has joined #tripleo | 10:33 | |
lifeless | have a look in /opt/stack/os-refresh-config/configure.d :) | 10:35 |
*** saju_m has joined #tripleo | 10:35 | |
killer_prince | lifeless | 10:36 |
lifeless | killer_prince: | 10:36 |
killer_prince | there is no such directory when image is booted | 10:37 |
lifeless | interesting | 10:37 |
lifeless | uhm | 10:37 |
lifeless | the build log should give enough info to debug this | 10:37 |
lifeless | or you can drop into a shell in the build | 10:37 |
killer_prince | lifeless: I am checking the build log now.. | 10:37 |
openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Proof of concept of error handling in the API https://review.openstack.org/76819 | 10:40 |
killer_prince | ln -s /opt/stack/venvs/os-refresh-config/bin/os-refresh-config /usr/local/bin/os-refresh-config | 10:43 |
killer_prince | for d in pre-configure.d configure.d migration.d post-configure.d | 10:43 |
killer_prince | install -m 0755 -o root -g root -d /opt/stack/os-config-refresh/pre-configure.d | 10:43 |
killer_prince | for d in pre-configure.d configure.d migration.d post-configure.d | 10:43 |
killer_prince | install -m 0755 -o root -g root -d /opt/stack/os-config-refresh/configure.d | 10:44 |
lifeless | sorery yeah I meant os-config-refresh. my figures btrayed me | 10:44 |
killer_prince | all these are there but the script that i placed in my custom element is not there.. | 10:44 |
killer_prince | let me pass the log contents... | 10:45 |
lifeless | killer_prince: its idnight here | 10:46 |
lifeless | killer_prince: I hope someone will look at it, but I won't) too late! | 10:47 |
*** jprovazn has quit IRC | 10:47 | |
*** CaptTofu has joined #tripleo | 10:48 | |
killer_prince | lifeless: np.. | 10:48 |
openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Proof of concept of error handling in the API https://review.openstack.org/76819 | 10:51 |
*** CaptTofu has quit IRC | 10:52 | |
killer_prince | here are the logs http://paste.openstack.org/show/70190/ | 10:54 |
*** CaptTofu has joined #tripleo | 10:56 | |
*** cd-undercloud has joined #tripleo | 10:57 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 10:57 |
*** cd-undercloud has quit IRC | 10:57 | |
*** jcooley_ has joined #tripleo | 11:01 | |
openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Better apt-sources docs. https://review.openstack.org/73035 | 11:02 |
openstackgerrit | lifeless proposed a change to openstack/diskimage-builder: Improve local-config proxy handling. https://review.openstack.org/76636 | 11:02 |
*** jcooley_ has quit IRC | 11:11 | |
jang | wrt. sunnyvale hackathon next week: does anyone know if there's transport lined up twixt the hotel and the venue? | 11:14 |
*** jprovazn has joined #tripleo | 11:14 | |
*** nosnos has quit IRC | 11:14 | |
*** ramishra has quit IRC | 11:28 | |
*** ramishra has joined #tripleo | 11:28 | |
openstackgerrit | Dmitry Tantsur proposed a change to openstack/tuskar-ui: Manage image ID's for node profile https://review.openstack.org/75823 | 11:30 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Using flavors from Roles https://review.openstack.org/76826 | 11:31 |
openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the overcloud views tests https://review.openstack.org/76161 | 11:33 |
*** cd-undercloud has joined #tripleo | 11:35 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 11:35 |
*** cd-undercloud has quit IRC | 11:35 | |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Using flavors from Roles https://review.openstack.org/76826 | 11:36 |
openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Proof of concept of error handling in the API https://review.openstack.org/76819 | 11:38 |
killer_prince | i created a new element and then created a os-refresh-config/configure.d and placed a script under this directory. I also defined dependency on os-refresh-config for this new element. when i am building the image using this element, i am not able to find this script on the image under /opt/stack/os-config-refresh/configure.d | 11:47 |
killer_prince | any idea what i missed in the whole step ? | 11:47 |
killer_prince | here are the logs http://paste.openstack.org/show/70190/ | 11:47 |
*** jcooley_ has joined #tripleo | 11:53 | |
killer_prince | okay. i found that there is a configure.d at / | 11:53 |
killer_prince | and my scripts do exist there.. | 11:54 |
killer_prince | is this expected or some kind of bug in diskimage-builder | 11:54 |
*** jcooley_ has quit IRC | 11:59 | |
*** martyntaylor has left #tripleo | 12:01 | |
slagle | killer_prince: what's the name of the new element you added? (looking through that log) | 12:03 |
*** cd-undercloud has joined #tripleo | 12:04 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 12:04 |
*** cd-undercloud has quit IRC | 12:04 | |
killer_prince | slagle: localboot | 12:08 |
slagle | killer_prince: according to that log, the image build failed for a different reason | 12:08 |
slagle | missing gcc in the image | 12:08 |
slagle | what image type are you trying to build? rhel? | 12:08 |
killer_prince | yes.. | 12:09 |
*** hashar_ has quit IRC | 12:09 | |
killer_prince | slagle: sorry, i pasted the wrong logs.. let me paste it again.. | 12:10 |
slagle | derekh: are we merging stuff? or still blocked on CI? | 12:11 |
slagle | i saw a couple of commits flow through w/o successful runs... | 12:11 |
killer_prince | slagle: http://paste.openstack.org/show/70214/ | 12:11 |
derekh | slagle: Stuff should be back working again, zuul seems to be running jobs but a lot are queued, | 12:15 |
derekh | slagle: it doesn't seem to be running as many jobs as it could, there is a lot of vm;s in an error state since the outage | 12:15 |
derekh | slagle: so maybe it has hit its quota, gonna restart nova-compute on each compute node to see if that help get the nodes out of the error start | 12:16 |
slagle | killer_prince: line 1350 is the error | 12:16 |
slagle | killer_prince: missing arparse for some reason, so the os-refresh-config scripts did not get copied onto the image | 12:17 |
slagle | killer_prince: looks like a valid bug. argparse should be added to os-refresh-config's requirements.txt | 12:17 |
derekh | slagle: Done, lets give it a few minutes and see if it helps | 12:19 |
killer_prince | So can i log a bug..? or is it specific to RHEL images only.. | 12:21 |
openstackgerrit | James Slagle proposed a change to openstack/os-refresh-config: Add argparse to requirements.txt https://review.openstack.org/76844 | 12:23 |
*** mrunge_away is now known as mrunge | 12:23 | |
slagle | killer_prince: ^^ that patch should help | 12:23 |
derekh | 12:24 | |
slagle | derekh: ok, cool | 12:26 |
derekh | slagle: that seems to have help, looks like zuul is spinning up more instances and the error ones are being delete, so I think we're back waiting on CI, I stall havn't figured out why the overcloud job is failing got a bit distracted | 12:26 |
openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add set -eux https://review.openstack.org/76847 | 12:29 |
*** CaptTofu has quit IRC | 12:30 | |
killer_prince | slagle: how do i test it as it gets pulled during the build.. any ideas..? | 12:32 |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Fix of Overcloud Update https://review.openstack.org/76850 | 12:33 |
slagle | killer_prince: you need to set some environment variables that the source-repositories element will use | 12:33 |
slagle | killer_prince: take a look at https://github.com/openstack/diskimage-builder/tree/master/elements/source-repositories/README.md | 12:34 |
slagle | killer_prince: about half way down the page there is an example of using a change from a review in gerrit | 12:34 |
killer_prince | slagle: okay.. thanks | 12:36 |
*** CaptTofu has joined #tripleo | 12:40 | |
*** weshay has joined #tripleo | 12:44 | |
*** cd-undercloud has joined #tripleo | 12:45 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 12:45 |
*** cd-undercloud has quit IRC | 12:45 | |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Working scaling https://review.openstack.org/76854 | 12:48 |
*** saju_m has quit IRC | 12:53 | |
openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Working scaling https://review.openstack.org/76854 | 12:56 |
*** morazi has joined #tripleo | 13:09 | |
*** hashar has joined #tripleo | 13:11 | |
*** hashar_ has joined #tripleo | 13:12 | |
*** ohadlevy_ is now known as ohadlevy | 13:13 | |
*** ohadlevy has joined #tripleo | 13:13 | |
*** cd-undercloud has joined #tripleo | 13:15 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 13:15 |
*** cd-undercloud has quit IRC | 13:15 | |
*** hashar has quit IRC | 13:16 | |
*** gcha has joined #tripleo | 13:24 | |
openstackgerrit | A change was merged to openstack/tripleo-incubator: Avoid races with Neutron by waiting for it. https://review.openstack.org/75754 | 13:24 |
*** jdob has joined #tripleo | 13:25 | |
Ng | morning | 13:25 |
killer_prince | slagle: i tried DIB_REPOLOCATION_os\-refresh\-config=https://review.openstack.org/openstack/os-refresh-config | 13:27 |
killer_prince | slagle: but i get bash: DIB_REPOLOCATION_os-refresh-config=https://review.openstack.org/openstack/os-refresh-config: No such file or directory | 13:27 |
killer_prince | slagle: am i doing it in the right way..? | 13:28 |
openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add fedora-rdo-icehouse element https://review.openstack.org/73411 | 13:28 |
*** lucasagomes is now known as lucas-hungry | 13:28 | |
slagle | killer_prince: it should be DIB_REPOLOCATION_os_refresh_config and DIB_REPOREF_os_refresh_config (replace dashes with underscores) | 13:29 |
*** jang has quit IRC | 13:29 | |
killer_prince | slagle: aha.. k.. | 13:30 |
slagle | killer_prince: appears that the README doesn't mention that's required...could be a good doc update patch :) | 13:30 |
killer_prince | slagle: yup.. | 13:30 |
openstackgerrit | A change was merged to openstack/tripleo-image-elements: Fix typo https://review.openstack.org/73655 | 13:31 |
openstackgerrit | A change was merged to openstack/tripleo-image-elements: Install lvm2 package for cinder https://review.openstack.org/73656 | 13:32 |
*** andreaf has joined #tripleo | 13:33 | |
*** ramishra has quit IRC | 13:40 | |
*** jcooley_ has joined #tripleo | 13:40 | |
*** ramishra has joined #tripleo | 13:42 | |
openstackgerrit | Dmitry Tantsur proposed a change to openstack/tuskar-ui: Prevent node profile of deployed role from being deleted https://review.openstack.org/76864 | 13:46 |
*** jcooley_ has quit IRC | 13:46 | |
*** hashar_ is now known as hashar | 13:49 | |
* Ng still pretty unwell today, so will not be super useful. got a doctor's appointment in a couple of hours to see if they can medicate me at all (feeling a lot like my plague is building into an ear infection, yay) | 13:51 | |
*** dprince has joined #tripleo | 13:51 | |
rpodolyaka | get well soon, Ng! | 13:52 |
*** cd-undercloud has joined #tripleo | 13:53 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 13:53 |
*** cd-undercloud has quit IRC | 13:53 | |
*** noslzzp has joined #tripleo | 13:53 | |
slagle | don't share too many details, TSA might deny you entry at the border | 13:55 |
*** lblanchard has joined #tripleo | 13:58 | |
*** sballe has joined #tripleo | 14:02 | |
*** tzumainn has quit IRC | 14:03 | |
*** tzumainn has joined #tripleo | 14:03 | |
*** hewbrocca has joined #tripleo | 14:10 | |
killer_prince | slagle: i used DIB_REPOLOCATION_os_refresh_config and DIB_REPOREF_os_refresh_config but i still get same error about argparse | 14:14 |
rpodolyaka | killer_prince: isn't os-refresh-config installed by pip rather than by source-repository element? | 14:17 |
killer_prince | rpodolyaka: yes.. you are right. but slagle suggested me to use this. | 14:20 |
slagle | os-refresh-config doesn't use source-repository? | 14:22 |
killer_prince | may be we can check in the logs that i provided a while back.. | 14:23 |
*** ramishra has quit IRC | 14:23 | |
slagle | i checked, it doesn't | 14:24 |
slagle | so, i'm not sure how to test this review | 14:24 |
slagle | likely there is not an exposed way | 14:24 |
rpodolyaka | slagle: indeed, 8a2a8c71741348a3b2a77bfa077c68ea84a918bb | 14:24 |
rpodolyaka | (https://review.openstack.org/#/c/66352/) | 14:24 |
openstackgerrit | Dmitry Tantsur proposed a change to openstack/tuskar-ui: Prevent node profile of deployed role from being deleted https://review.openstack.org/76864 | 14:24 |
*** CaptTofu has quit IRC | 14:25 | |
rpodolyaka | slagle: killer_prince: from what I see, we can only fix this in os-refresh-config and then cut a new release | 14:25 |
rpodolyaka | slagle: killer_prince: or try an ugly hack to test it - modify os-refresh-config element to install it from gerrit review :) | 14:26 |
killer_prince | rpodolyaka: and when is that gonna happen.. | 14:26 |
rpodolyaka | killer_prince: we cut new releases each week, so I can just release it today, if needed | 14:26 |
killer_prince | rpodolyaka: would be great if you could do that. I need this change for some custom element that I am writing now.. its kind of blocker for me.. | 14:27 |
*** cd-undercloud has joined #tripleo | 14:29 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 14:29 |
*** cd-undercloud has quit IRC | 14:29 | |
*** lucas-hungry is now known as lucasagomes | 14:30 | |
rpodolyaka | killer_prince: no problem, we just need two core reviewers to get slagle's fix approved | 14:31 |
killer_prince | rpodolyaka: thanks a lot.. | 14:32 |
*** jcooley_ has joined #tripleo | 14:35 | |
*** ftcjeff has joined #tripleo | 14:36 | |
fungi | derekh: so it looks like you've got the ci cloud up and running again in the past couple hours... i see a handful of nodes nodepool thinks are in there for a very long time, so i'll get those cleaned up to open up a little more capacity for you | 14:39 |
derekh | fungi: cool, thanks do I need to delete vm's in error state ? | 14:39 |
fungi | derekh: well, i'll submit deletes through nodepool for any nodes in your cloud in any one state for more than 4 hours and we'll see where that leaves us. it will at least submit nova deletes for them if they exist on the provider end | 14:40 |
derekh | fungi: ok | 14:41 |
*** jcooley_ has quit IRC | 14:41 | |
fungi | over a dozen at least 4 hours in delete or ready state (some up to ~15 hours) | 14:42 |
fungi | oh, actually there was one in delete for over 35 hours, and another over 28 | 14:43 |
fungi | once those are clear i'll look at any which are in any state other than used for at least an hour (that will allow us to avoid killing nodes which are actively running jobs) | 14:43 |
derekh | 15 hours is probably when the outage started. ok thanks | 14:44 |
fungi | assuming that basically anything which has been building, deleting or in a ready state for an hour is probably broken | 14:44 |
derekh | sounds about right | 14:46 |
*** e0ne_ has joined #tripleo | 14:46 | |
*** CaptTofu has joined #tripleo | 14:48 | |
*** spzala has joined #tripleo | 14:49 | |
*** e0ne has quit IRC | 14:49 | |
openstackgerrit | A change was merged to openstack/tuskar-ui: Manage image ID's for node profile https://review.openstack.org/75823 | 14:52 |
*** ramishra has joined #tripleo | 14:54 | |
*** ramishra has quit IRC | 14:56 | |
*** ramishra has joined #tripleo | 14:57 | |
*** matty_dubs|gone is now known as matty_dubs | 15:00 | |
*** ramishra has quit IRC | 15:02 | |
killer_prince | yeah.. so now i can see the scripts there under /opt/stack/os-config-refresh/configure.d/ but how do I know if they were executed.. any log file.. | 15:05 |
slagle | killer_prince: they'll be executed when the image boots. os-refresh-config is triggered by the os-collect-config service | 15:06 |
*** ramishra has joined #tripleo | 15:07 | |
slagle | i *think* there is a /var/log/os-collect-config.log | 15:07 |
killer_prince | i have the instance running... so is there any log that i can check.. | 15:07 |
slagle | is this rhel 6? | 15:07 |
killer_prince | i do not have that file in running instance.. | 15:07 |
*** noslzzp has quit IRC | 15:07 | |
killer_prince | slagle: yes.. | 15:07 |
*** noslzzp has joined #tripleo | 15:08 | |
*** cd-undercloud has joined #tripleo | 15:08 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 15:08 |
*** cd-undercloud has quit IRC | 15:08 | |
slagle | typically, i would say check the stdout of the os-collect-config service, but rhel 6 does not use systemd, so i'm not sure where the stdout has gone... | 15:08 |
killer_prince | RHEL 6.5 x86_64 to be precise.. | 15:08 |
killer_prince | i am asking as the changes it was supposed to make did not happen.. so is there any way i can run it manually and check what went wrong.. | 15:09 |
killer_prince | sorry.. i am new to this and so lots of dumb doubts.. | 15:10 |
slagle | i think the issue is that we only install a systemd service file for os-collect-config, which is obviously not going to work on rhel 6 | 15:10 |
slagle | you can run 'os-collect-config' as root to trigger it | 15:11 |
slagle | os-collect-config --one-time to just run it once | 15:11 |
killer_prince | so i have one doubt.. the scripts that we put under os-refresh-config is run by os-collect-config.. correct..? | 15:13 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Ceilo/Ironic/Tuskar: don't hard code venv dir https://review.openstack.org/76887 | 15:13 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Heat/Horizon/Tempest: don't hard code venv dir https://review.openstack.org/76888 | 15:13 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Neutron: don't hard code the venv dir https://review.openstack.org/76626 | 15:14 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Link db sync utilities to /usr/local/bin... https://review.openstack.org/76579 | 15:14 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Glance/Cinder/Keystone/Swift: don't hard code venv https://review.openstack.org/76666 | 15:14 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Nova: don't hard code the venv dir https://review.openstack.org/76619 | 15:14 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Stop using the os-svc-install -n option. https://review.openstack.org/76604 | 15:14 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Drop the os-svc-install -n -c options. https://review.openstack.org/76605 | 15:14 |
slagle | killer_prince: os-collect-config runs os-refresh-config, which runs the scripts under /opt/stack/os-config-refresh | 15:14 |
killer_prince | I see that os-collect-config is quering for user data and instance meta data.. | 15:15 |
*** blamar_ has joined #tripleo | 15:16 | |
*** rdopiera1ski has joined #tripleo | 15:16 | |
*** SpamapS_ has joined #tripleo | 15:16 | |
*** dprince_ has joined #tripleo | 15:16 | |
*** vipuls has joined #tripleo | 15:21 | |
*** dprince has quit IRC | 15:21 | |
*** gcha has quit IRC | 15:21 | |
*** jprovazn has quit IRC | 15:21 | |
*** athomas has quit IRC | 15:21 | |
*** d0ugal has quit IRC | 15:21 | |
*** rdopieralski has quit IRC | 15:21 | |
*** blamar has quit IRC | 15:21 | |
*** SpamapS has quit IRC | 15:21 | |
*** vipul has quit IRC | 15:21 | |
*** andreaf has quit IRC | 15:21 | |
*** blamar_ is now known as blamar | 15:21 | |
*** ramishra has quit IRC | 15:21 | |
*** ramishra has joined #tripleo | 15:22 | |
*** funzo has quit IRC | 15:22 | |
*** funzo_ has joined #tripleo | 15:22 | |
killer_prince | i also tried running os-refresh-config and it executed my scripts.. but os-collect-config did not execute os-refresh-config. to confirm i tried grep os-refresh-config on os-collect-config /opt/stack/venvs/os-collect-config but didnt get any thing.. am i missing anything..? | 15:23 |
*** mrunge has quit IRC | 15:24 | |
slagle | it's configured in /etc/os-collect-config.conf | 15:27 |
*** CaptTofu has quit IRC | 15:28 | |
*** jprovazn has joined #tripleo | 15:28 | |
*** CaptTofu has joined #tripleo | 15:28 | |
*** athomas has joined #tripleo | 15:28 | |
*** d0ugal has joined #tripleo | 15:28 | |
killer_prince | slagle: aha.. so there its hidden.. | 15:29 |
*** gcha has joined #tripleo | 15:29 | |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Neutron: don't hard code the venv dir https://review.openstack.org/76626 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Link db sync utilities to /usr/local/bin... https://review.openstack.org/76579 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Heat/Horizon/Tempest: don't hard code venv dir https://review.openstack.org/76888 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Glance/Cinder/Keystone/Swift: don't hard code venv https://review.openstack.org/76666 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Ceilo/Ironic/Tuskar: don't hard code venv dir https://review.openstack.org/76887 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Nova: don't hard code the venv dir https://review.openstack.org/76619 | 15:29 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Stop using the os-svc-install -n -c options. https://review.openstack.org/76604 | 15:30 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Drop the os-svc-install -n -c options. https://review.openstack.org/76605 | 15:30 |
*** hashar has quit IRC | 15:30 | |
*** hashar has joined #tripleo | 15:32 | |
*** CaptTofu has quit IRC | 15:32 | |
*** funzo_ is now known as funzo | 15:33 | |
*** killer_prince has quit IRC | 15:41 | |
*** lazy_prince is now known as killer_prince | 15:41 | |
*** ramishra has quit IRC | 15:41 | |
*** ramishra has joined #tripleo | 15:42 | |
*** martyntaylor has joined #tripleo | 15:42 | |
*** panda has joined #tripleo | 15:45 | |
*** cd-undercloud has joined #tripleo | 15:46 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 15:46 |
*** cd-undercloud has quit IRC | 15:46 | |
*** panda_ has quit IRC | 15:48 | |
derekh | so now the the cloud is up and running again, looks like network access from the VM's is extreemly slow, so expect things to be failing | 15:52 |
derekh | e.g. this job has been running for about 2 hours https://jenkins03.openstack.org/job/check-tripleo-overcloud-precise/42/consoleFull | 15:52 |
derekh | and still hasn't finished download the cloud image.... | 15:53 |
*** CaptTofu has joined #tripleo | 15:53 | |
derekh | this looks like what we hit a while back with a MTU problem https://bugs.launchpad.net/neutron/+bug/1270646 but setting the MTU on vm's doesn't seem to have an effect | 15:54 |
derekh | eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc pfifo_fast state UP qlen 1000 | 15:55 |
*** ramishra has quit IRC | 16:00 | |
derekh | lifeless, devananda, Ng , SpamapS_ , jog0 , GheRivero , derekh , dprince_ , slagle ^^^ any ideas ? | 16:00 |
jog0 | derekh: sorry my network foo is very weak | 16:01 |
derekh | jog0: as is mine | 16:01 |
*** andreaf has joined #tripleo | 16:08 | |
*** coolsvap has quit IRC | 16:10 | |
slagle | derekh: i've never seen slow network as a symptom of that bug, only dropped packets due to exceeding mtu size | 16:15 |
slagle | and for that, i typically *increase* mtu to 1542 | 16:15 |
slagle | but, i don't think that applies in this case | 16:16 |
derekh | slagle: ok, tried it just incase didn't help | 16:21 |
*** bauzas has quit IRC | 16:22 | |
cody-somerville | lifeless: Are you around? | 16:22 |
*** cd-undercloud has joined #tripleo | 16:25 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 16:25 |
*** cd-undercloud has quit IRC | 16:25 | |
*** jcooley_ has joined #tripleo | 16:29 | |
*** cd-undercloud has joined #tripleo | 16:31 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 16:31 |
*** cd-undercloud has quit IRC | 16:31 | |
*** vkozhukalov has quit IRC | 16:36 | |
*** jcooley_ has quit IRC | 16:36 | |
*** jcooley_ has joined #tripleo | 16:38 | |
*** vkozhukalov has joined #tripleo | 16:39 | |
*** rdopiera1ski has quit IRC | 16:54 | |
*** pbelanyi has quit IRC | 16:56 | |
*** d0ugal has quit IRC | 16:58 | |
*** morazi has quit IRC | 16:58 | |
*** jcooley_ has quit IRC | 16:59 | |
*** jistr has quit IRC | 17:00 | |
*** morazi has joined #tripleo | 17:00 | |
*** killer_prince is now known as lazy_prince | 17:01 | |
*** vkozhukalov has quit IRC | 17:05 | |
*** e0ne_ has quit IRC | 17:06 | |
*** lblanchard has quit IRC | 17:06 | |
*** andreaf has quit IRC | 17:08 | |
*** jwalls has joined #tripleo | 17:08 | |
*** jp_at_hp has quit IRC | 17:08 | |
*** ftcjeff has quit IRC | 17:08 | |
*** jp_at_hp has joined #tripleo | 17:09 | |
openstackgerrit | Dustin J. Mitchell proposed a change to openstack/diskimage-builder: Update README formatting and content https://review.openstack.org/75480 | 17:10 |
*** martyntaylor has quit IRC | 17:13 | |
*** jcooley_ has joined #tripleo | 17:17 | |
*** sdake_ has joined #tripleo | 17:24 | |
*** bauzas has joined #tripleo | 17:26 | |
*** matty_dubs is now known as matty_dubs|lunch | 17:26 | |
derekh | lifeless, devananda, Ng , SpamapS_ , jog0 , GheRivero , derekh , dprince_ , slagle gotta run, didn't make much progress https://etherpad.openstack.org/p/cloud-outage | 17:42 |
*** jcooley_ has quit IRC | 17:45 | |
*** lblanchard has joined #tripleo | 17:45 | |
*** derekh has quit IRC | 17:46 | |
SpamapS_ | ugh | 17:48 |
*** SpamapS_ is now known as SpamapS | 17:48 | |
*** cd-undercloud has joined #tripleo | 17:48 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 17:48 |
*** cd-undercloud has quit IRC | 17:48 | |
*** SpamapS has quit IRC | 17:48 | |
*** SpamapS has joined #tripleo | 17:48 | |
*** rlandy is now known as rlandy|bbl | 17:52 | |
SpamapS | 64 bytes from 10.10.16.172: icmp_seq=1 ttl=64 time=0.577 ms | 17:55 |
SpamapS | 64 bytes from 10.10.16.172: icmp_seq=1 ttl=64 time=0.599 ms (DUP!) | 17:55 |
*** rsacharya has quit IRC | 17:55 | |
SpamapS | weird | 17:55 |
*** rsacharya has joined #tripleo | 17:58 | |
*** cadenzajon has joined #tripleo | 18:00 | |
*** hashar has quit IRC | 18:00 | |
*** gcha has quit IRC | 18:01 | |
*** andreaf has joined #tripleo | 18:03 | |
*** jp_at_hp has quit IRC | 18:06 | |
*** jcooley_ has joined #tripleo | 18:18 | |
*** pblaho has quit IRC | 18:18 | |
*** andreaf has quit IRC | 18:21 | |
*** akrivoka has quit IRC | 18:21 | |
openstackgerrit | Dmitry Tantsur proposed a change to openstack/tuskar-ui: Prevent node profile of deployed role from being deleted https://review.openstack.org/76864 | 18:21 |
SpamapS | what is 10.10.16.172 anyway? The old seed? | 18:21 |
* SpamapS is a bit confused why DHCP is setting a DNS server | 18:22 | |
*** gcha has joined #tripleo | 18:22 | |
*** shakayumi has joined #tripleo | 18:22 | |
openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-incubator: Stop devtest if user was not in libvirtd group https://review.openstack.org/76926 | 18:24 |
*** rsacharya has left #tripleo | 18:25 | |
*** athomas has quit IRC | 18:25 | |
*** cd-undercloud has joined #tripleo | 18:27 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 18:27 |
*** cd-undercloud has quit IRC | 18:27 | |
*** shakayumi has quit IRC | 18:29 | |
*** shakayumi has joined #tripleo | 18:30 | |
*** shakayumi has quit IRC | 18:30 | |
*** saurabhs has joined #tripleo | 18:30 | |
* SpamapS struggles to bend brain to map jenkins -> instances | 18:32 | |
lifeless | just toggled gro off on the ci-overcloud | 18:32 |
lifeless | should address network performance | 18:33 |
SpamapS | lifeless: I just turned it off about 15 minutes | 18:33 |
SpamapS | ago | 18:33 |
* SpamapS noted it in the etherpad like a good boy ;) | 18:33 | |
*** cadenzajon_ has joined #tripleo | 18:33 | |
SpamapS | lifeless: thank god you're here.. so how do I map tripleo-precise-tripleo-test-cloud-1728731 to an instance in ci-overcloud ? | 18:33 |
lifeless | SpamapS: ah haven't looked there yet, just saw derekh's pings | 18:33 |
lifeless | nova show tripleo-precise-tripleo-test-cloud-1728731 | 18:33 |
lifeless | as admin | 18:33 |
SpamapS | it.. can't be.. that easy | 18:33 |
SpamapS | ERROR: No server with a name or ID of 'tripleo-precise-tripleo-test-cloud-1728731' exists. | 18:34 |
lifeless | use the uuid | 18:34 |
*** matty_dubs|lunch is now known as matty_dubs | 18:34 | |
SpamapS | nova list --all-tenants grepping for it works | 18:34 |
lifeless | nov show $(nova list --all-tenants | awk '/tripleo-precise-tripleo-test-cloud-1728731/ {print $2}') | 18:34 |
*** cadenzajon has quit IRC | 18:35 | |
*** e0ne has joined #tripleo | 18:36 | |
*** saju_m has joined #tripleo | 18:37 | |
SpamapS | lifeless: and the job executing on that VM did in fact work out. | 18:39 |
SpamapS | first green \o/ | 18:39 |
SpamapS | lifeless: so I think I saw you asking how we can get gro forced off.. | 18:39 |
*** athomas has joined #tripleo | 18:39 | |
SpamapS | lifeless: we can just put in an if-up.d script, and I think sysconfig has something similar that they can set in variables. | 18:40 |
*** saju_m has quit IRC | 18:41 | |
*** d0ugal has joined #tripleo | 18:41 | |
*** d0ugal has quit IRC | 18:41 | |
*** d0ugal has joined #tripleo | 18:41 | |
lifeless | SpamapS: no, I wasn't asking that | 18:46 |
lifeless | SpamapS: https://etherpad.openstack.org/p/tripleo-network-configuration | 18:47 |
SpamapS | oh | 18:47 |
lifeless | might have given you that impression | 18:47 |
SpamapS | mmmk | 18:47 |
lifeless | its also a substantial scaling issue to have it off | 18:47 |
lifeless | SpamapS: hello metadata :) | 18:47 |
SpamapS | lifeless: right we're putting more work on the kernel right? But isn't this basically a GRE/OpenVswitch limitation? | 18:48 |
lifeless | SpamapS: see the df bit bug | 18:48 |
*** shakayumi has joined #tripleo | 18:50 | |
*** shakayumi has joined #tripleo | 18:51 | |
*** shakayumi has quit IRC | 18:51 | |
SpamapS | lifeless: I really don't understand the bug, but the symptom, as I'm understanding it, is that we are setting DF and thus causing our packets to be dropped without recourse? | 18:51 |
*** saju_m has joined #tripleo | 18:51 | |
*** shakayumi has joined #tripleo | 18:51 | |
*** shakayumi has quit IRC | 18:52 | |
*** shakayumi has joined #tripleo | 18:52 | |
*** shakayumi has quit IRC | 18:53 | |
jdob | having an issue on my devtest setup: https://gist.github.com/jdob/9256555 does that look familiar to anyone? | 18:53 |
*** shakayumi has joined #tripleo | 18:53 | |
*** vkozhukalov has joined #tripleo | 18:55 | |
*** cd-undercloud has joined #tripleo | 19:04 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 19:04 |
*** cd-undercloud has quit IRC | 19:04 | |
*** cwolferh has quit IRC | 19:13 | |
*** lblanchard has quit IRC | 19:19 | |
*** cwolferh has joined #tripleo | 19:20 | |
*** saju_m has quit IRC | 19:35 | |
*** cd-undercloud has joined #tripleo | 19:37 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 19:37 |
*** cd-undercloud has quit IRC | 19:37 | |
*** cd-undercloud has joined #tripleo | 19:42 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 19:42 |
*** cd-undercloud has quit IRC | 19:42 | |
*** akuznetsov has quit IRC | 19:43 | |
*** akuznetsov has joined #tripleo | 19:43 | |
bnemec | jdob: I sometimes have issues with connectivity to 192.0.2.1. If you ping it do you get something about Next-Hop? | 19:45 |
*** jang1 has joined #tripleo | 19:50 | |
*** rlandy|bbl is now known as rlandy | 19:50 | |
openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: rsync configurations to support multi-node swift https://review.openstack.org/76105 | 19:51 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add -i options to os-svc-install/os-svc-daemon... https://review.openstack.org/76952 | 19:59 |
*** lblanchard has joined #tripleo | 20:00 | |
*** dshurtle has joined #tripleo | 20:01 | |
*** hashar has joined #tripleo | 20:10 | |
*** bauzas has quit IRC | 20:14 | |
jdob | bnemec: yes! | 20:14 |
jdob | From 192.168.122.80: icmp_seq=2 Redirect Host(New nexthop: 192.168.122.1) | 20:14 |
bnemec | jdob: Okay, I'm not entirely sure what causes that, but I think re-creating the route generally fixes it for me. | 20:17 |
jdob | hrm, I had tried recreating the route | 20:17 |
lifeless | your traffic is being picked up on the wrong interface | 20:19 |
lifeless | so its telling you where the router is rather than routing it | 20:19 |
*** cd-undercloud has joined #tripleo | 20:20 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 20:20 |
*** cd-undercloud has quit IRC | 20:20 | |
lifeless | one reason it might think its teh wrong interface is because the seed doesn't have a better route to 192.0.2.0/24 | 20:21 |
lifeless | so you need to recreate the route as bnemec says - on the seed node, not on the host | 20:21 |
jdob | man is my networking knowledge way too limited | 20:24 |
lifeless | jdob: live n learn :) | 20:27 |
jdob | this reminds me of how I got started in computers as a kid, I'd break my dad's and have to learn how to fix it before he killed me | 20:27 |
jdob | lifeless: when you say seen node, you mean the undercloud guest right? | 20:29 |
lifeless | seed | 20:29 |
jdob | er, crap, typo. | 20:29 |
*** epim has joined #tripleo | 20:31 | |
*** newell has joined #tripleo | 20:38 | |
*** jtomasek has quit IRC | 20:42 | |
*** dshurtle has quit IRC | 20:47 | |
jdob | there we go :D | 20:54 |
jdob | \o/ | 20:54 |
jdob | thanks lifeless and bnemec | 20:55 |
*** ifarkas has quit IRC | 20:56 | |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-daemon -i. https://review.openstack.org/76963 | 21:00 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update elements to use os-svc-install -i. https://review.openstack.org/76964 | 21:00 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in OPENSTACK_VENV_DIR default if set https://review.openstack.org/76965 | 21:00 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS... https://review.openstack.org/76966 | 21:00 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 21:00 |
*** cd-undercloud has joined #tripleo | 21:00 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 21:00 |
*** cd-undercloud has quit IRC | 21:00 | |
*** cd-undercloud has joined #tripleo | 21:02 | |
cd-undercloud | ************** overcloud complete status=7 ************ | 21:02 |
*** cd-undercloud has quit IRC | 21:02 | |
*** e0ne has quit IRC | 21:05 | |
*** rbrady is now known as rbrady-bbl | 21:09 | |
fungi | lifeless: devananda: Ng: SpamapS_: jog0_: GheRivero: derekh: dprince: slagle: reviewing scrollback, it's unclear to me whether you expect ci-overcloud should be up or not. just in case you believe it should, the nodepool server is still getting... | 21:09 |
fungi | requests.exceptions.ConnectionError: HTTPSConnectionPool(host='ci-overcloud.tripleo.org', port=13000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 110] Connection timed out) | 21:09 |
fungi | and indeed, 'telnet ci-overcloud.tripleo.org 13000' times out for me as well | 21:12 |
lifeless | it should be up | 21:21 |
lifeless | nuts, I ahve some elocal but will look shortly | 21:22 |
fungi | k | 21:23 |
*** spzala has quit IRC | 21:24 | |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in OPENSTACK_VENV_DIR default if set https://review.openstack.org/76965 | 21:26 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS... https://review.openstack.org/76966 | 21:26 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 21:26 |
*** dprince_ has quit IRC | 21:28 | |
*** vipuls is now known as vipul | 21:28 | |
*** d0ugal has quit IRC | 21:29 | |
*** athomas has quit IRC | 21:32 | |
*** shakayumi has quit IRC | 21:35 | |
lifeless | fungi: https://etherpad.openstack.org/p/cloud-outage | 21:37 |
fungi | such fun! it's your old friend colonel panic | 21:40 |
fungi | died from "not much information" | 21:40 |
*** shakayumi has joined #tripleo | 21:41 | |
*** shakayumi has quit IRC | 21:41 | |
fungi | this is the second time in 24 hours, presumably? or did derekh recover it from a different failure earlier today? | 21:41 |
lifeless | second on 24h yes | 21:44 |
lifeless | Ng was looking into a bios upgrade | 21:44 |
lifeless | then got sick | 21:44 |
lifeless | and I've not had cycles to pursue that | 21:44 |
*** d0ugal has joined #tripleo | 21:47 | |
*** d0ugal has quit IRC | 21:47 | |
*** d0ugal has joined #tripleo | 21:47 | |
* lifeless stabs the ci control node harder | 21:51 | |
*** cd-undercloud has joined #tripleo | 21:51 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 21:51 |
*** cd-undercloud has quit IRC | 21:51 | |
*** jprovazn has quit IRC | 21:53 | |
*** cd-undercloud has joined #tripleo | 21:54 | |
cd-undercloud | ************** overcloud complete status=128 ************ | 21:54 |
*** cd-undercloud has quit IRC | 21:54 | |
lifeless | hah | 21:56 |
lifeless | ilo do your thing | 21:56 |
lifeless | 'power off'... | 21:56 |
lifeless | minute later | 21:56 |
lifeless | power still on | 21:56 |
lifeless | power off hard time | 21:57 |
tchaypo | okay, i have a registered token. now its time to set up the VPN | 21:57 |
openstackgerrit | Jonathan Brownell proposed a change to openstack/diskimage-builder: Fix syntax error in GRUB_OPTS env var handling https://review.openstack.org/76983 | 22:01 |
*** rlandy has quit IRC | 22:04 | |
*** jdob has quit IRC | 22:07 | |
lifeless | netboot failure | 22:07 |
tchaypo | StevenK: figured out how to trust the CA | 22:10 |
*** e0ne has joined #tripleo | 22:16 | |
*** e0ne has quit IRC | 22:21 | |
*** shardy is now known as shardy_afk | 22:23 | |
StevenK | tchaypo: Oooh | 22:25 |
*** gcha has quit IRC | 22:31 | |
*** rbrady-bbl is now known as rbrady | 22:35 | |
*** spzala has joined #tripleo | 22:35 | |
*** cd-undercloud has joined #tripleo | 22:38 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 22:38 |
*** cd-undercloud has quit IRC | 22:38 | |
*** bauzas has joined #tripleo | 22:40 | |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS... https://review.openstack.org/76966 | 22:41 |
openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 22:41 |
*** lblanchard has quit IRC | 22:46 | |
*** vkozhukalov has quit IRC | 22:47 | |
*** al has quit IRC | 22:48 | |
*** CaptTofu has quit IRC | 22:48 | |
*** jang1 has quit IRC | 22:49 | |
*** al has joined #tripleo | 22:50 | |
*** hashar has quit IRC | 22:55 | |
*** al has quit IRC | 22:57 | |
*** al has joined #tripleo | 22:57 | |
*** al has quit IRC | 22:57 | |
*** CaptTofu has joined #tripleo | 22:59 | |
*** al has joined #tripleo | 23:02 | |
tchaypo | so here's a neat trick | 23:04 |
tchaypo | I spotted https://review.openstack.org/#/c/71996/12/stack.sh and wondered if it would help me track down the errors I'm having running devtest | 23:04 |
tchaypo | it turns out that with set -o errexit, every time I run devtest_setup.sh, tmux exits | 23:04 |
lifeless | lol | 23:05 |
SpamapS | doh | 23:06 |
greghaynes | wierd, I do all my stuff in tmux | 23:06 |
greghaynes | and havent had that | 23:06 |
tchaypo | ever tried set -o errexit? | 23:06 |
greghaynes | oh, its new in that rev | 23:06 |
greghaynes | this sounds like fun :) | 23:06 |
*** matty_dubs is now known as matty_dubs|gone | 23:09 | |
tchaypo | It seems like it's this line | 23:14 |
tchaypo | if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi | 23:14 |
SpamapS | well yes, there's an error, so it is exiting with an error. | 23:14 |
*** cd-undercloud has joined #tripleo | 23:14 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 23:14 |
*** cd-undercloud has quit IRC | 23:14 | |
tchaypo | not even that. | 23:15 |
tchaypo | If I'm running with set -o errexit and I execute " '[' 0 '!=' 0 ']' ", tmux goes away | 23:15 |
tchaypo | oh, of course - because I'm running that in my shell and so my shell exits. ie, what you said. | 23:16 |
tchaypo | yep, and that makes perfect sense then. If I was using ./devtest_variables.sh, errexit would be set in a subshell. Because we source it, it gets set in my top-level shell. Once there's an error inside the script, the script exits with an error, which then exits the top-level shell. | 23:18 |
SpamapS | tchaypo: yeah, sourcing scripts is the devil :) | 23:18 |
tchaypo | So in short - if anyone suggests set -o errexit for devtest, we'll give them an embarrassed look and tell them we can't. | 23:19 |
tchaypo | no, that's not right, we can. | 23:20 |
tchaypo | we just can't set it in anything that gets sourced | 23:20 |
tchaypo | devtest_setup.sh already does set -eux | 23:21 |
lifeless | right | 23:24 |
lifeless | this is why I pushed hard to get away from sourcing so much | 23:24 |
lifeless | variables needs to be sourced, by definition | 23:24 |
lifeless | nothing else, in princple | 23:24 |
*** CaptTofu has quit IRC | 23:24 | |
lifeless | well, cloud creds etc | 23:24 |
lifeless | nothing complex | 23:24 |
SpamapS | lifeless: I'm on board with "no more variables" btw. | 23:27 |
SpamapS | lifeless: give me a config file over a sourced thing any day :) | 23:27 |
tchaypo | +1 | 23:27 |
tchaypo | even if it's a config file that gets sourced inside the script | 23:27 |
SpamapS | hah yeah | 23:27 |
tchaypo | sorry, i missed the air-quotes around "config file" | 23:28 |
greghaynes | A file with var=val is kinda like a config file ;) | 23:28 |
tchaypo | We have a turing-complete config language, yay! | 23:28 |
greghaynes | haha | 23:28 |
greghaynes | I did notice we chose mustache for the templating, I havent ever seen that outside of web shops | 23:30 |
*** mahipal has joined #tripleo | 23:30 | |
lifeless | greghaynes: we're a web shop :P | 23:31 |
greghaynes | touche | 23:31 |
openstackgerrit | Jonathan Brownell proposed a change to openstack/diskimage-builder: Fix spelling error in "vm" element README.md https://review.openstack.org/77007 | 23:32 |
*** spzala_ has joined #tripleo | 23:37 | |
*** sdake__ has joined #tripleo | 23:40 | |
*** sseago__ has joined #tripleo | 23:41 | |
*** SpamapS_ has joined #tripleo | 23:43 | |
*** spzala has quit IRC | 23:45 | |
*** sdake_ has quit IRC | 23:45 | |
*** SpamapS has quit IRC | 23:45 | |
*** sseago_ has quit IRC | 23:45 | |
*** openstackgerrit has quit IRC | 23:45 | |
*** jwalls has quit IRC | 23:45 | |
*** jwalls has joined #tripleo | 23:45 | |
*** jwalls has quit IRC | 23:49 | |
*** cd-undercloud has joined #tripleo | 23:53 | |
cd-undercloud | ************** overcloud complete status=1 ************ | 23:53 |
*** cd-undercloud has quit IRC | 23:53 | |
*** SpamapS_ is now known as SpamapS | 23:53 | |
*** SpamapS has quit IRC | 23:53 | |
*** SpamapS has joined #tripleo | 23:53 | |
StevenK | lifeless: Can I steal spot number #4? | 23:56 |
lifeless | sure | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!