Wednesday, 2014-08-13

tchaypookay, t-h-t is now clean00:01
tchaypot-i-e is now sad00:03
*** marun has joined #tripleo00:05
tchaypoin 104368, which looks like it needs a rebase00:05
openstackgerritJames Polley proposed a change to openstack/tripleo-image-elements: Support new virtual_interfaces definition for VIPs  https://review.openstack.org/10436800:06
openstackgerritlifeless proposed a change to openstack/tripleo-heat-templates: Make Swift replica count configurable.  https://review.openstack.org/11371600:10
tchaypoI'm starting to enjoy this middle-ear thing00:13
openstackgerritlifeless proposed a change to openstack/tripleo-heat-templates: Make Swift replica count configurable.  https://review.openstack.org/11372300:13
tchayposounds coming in my left ear get split into 3 - i hear harmonys slightly above and below the actual pitch00:13
tchayposo now when i whistle it sounds like I'm playing a harmonica00:14
*** sdake has quit IRC00:16
tchaypo122 BM_NETWORK_CIDR=$(os-apply-config -m $TE_DATAFILE --key baremetal-network.cidr --type raw --key-default '192.0.2.0/24')00:22
tchaypo123 BM_VLAN_SEED_TAG=$(OS_CONFIG_FILES=$TE_DATAFILE os-apply-config --key baremetal-network.seed.public_vlan.tag --type netaddress --key-default '')00:22
tchaypowe seem to have made a change in how we call o-a-c and pass the name of $TE_DATAFILE00:22
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Add security group rules to tripleo-ci/default  https://review.openstack.org/10916100:23
tchaypoit looks like -m is the new method, so I'll update 103757 to match00:24
lifelesstchaypo: I did that already00:25
lifelessoh00:25
lifelessI didn't aggressively do it00:25
lifelessalso there is the output of a bad resolution around line 19700:26
tchaypoof scripts/devtest_seed.sh?00:27
openstackgerritHaneef Ali proposed a change to openstack/tripleo-image-elements: Configures keystone with apache  https://review.openstack.org/10230400:29
lifelessyes00:29
lifelessI think00:29
tchaypothe lines where checks $BUILD_ONLY and then may exit 0 ?00:30
lifelessno00:30
lifelessthe duplicate oac call00:30
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: We deploy CI clouds HA, use HA storage in swift.  https://review.openstack.org/11373000:32
tchaypooh00:32
tchaypoROUTE_DEV=...00:32
tchaypothat's 224 in the copy I'm looking at00:33
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Add public VLAN support.  https://review.openstack.org/10375700:34
lifelessslagle: greghaynes: SpamapS: bnemec: https://review.openstack.org/#/c/103449/ is ready for +2s00:35
*** ci-overcloud has joined #tripleo00:36
ci-overcloud************** ci-overcloud complete status=1 ************00:36
*** ci-overcloud has quit IRC00:36
lifelessok, now I have 2 replicas00:36
lifelesstime to fixup the glance status :/00:36
tchaypoand i have prep_source_repos working nicely00:36
lifelesstchaypo: cool, so - deploy thy seed00:37
tchaypo*updates https://etherpad.openstack.org/p/tripleo-ci-hp2-build*00:37
lifelesstchaypo: have you added that to the list of etherpads ?00:38
lifelessoh yay glance00:38
lifelessHTTPInternalServerError (HTTP 500): Unable to delete image ffb40bec-3851-43b6-9db1-a2f048c8faf800:38
lifelesscan't delete image when its already gone from swift :/00:39
tchaypowheee00:40
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Because wrong doesn't work.  https://review.openstack.org/11373200:41
lifelessoh and we don't assert keytone updates either yet00:43
* lifeless pivots everything back to 15500:43
tchaypotweaking repo_refs again?00:44
lifelessyes00:44
lifelessbut not related to hp200:44
tchaypoi see you've made changes to the swift repos since last time i got hp2 in sync..00:44
lifelessyes00:44
lifelessthat change isn't relevant to you yet00:45
lifelessit will be00:45
stevebakeryus FAILED status due to 'StackValidationFailed: Property error : SmokeServer: key_name Value must be a string'00:46
lifelessstevebaker: \o/00:46
tchaypo*consults painkiller diary*00:47
tchaypookay, it looks like the aspirin isn't due for another 30 minutes, so I'll leave the paracetemol for the same time...00:48
*** ci-overcloud has joined #tripleo00:48
ci-overcloud************** ci-overcloud complete status=1 ************00:48
*** ci-overcloud has quit IRC00:48
tchaypoI'm taking 3x aspirins 3x day for 5 entire days, but 2x paracetemol "as required, max 4x/day", which likely means I'll be off them entirely by the 3rd day..00:49
* tchaypo kicks off seed build00:56
lifelesstchaypo: so is that etherpad in the list of therpads?00:56
tchaypothere's a list of etherpads?00:58
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-image-elements: Drop all direct usage of systemctl enable  https://review.openstack.org/11318100:59
*** xuhaiwei has joined #tripleo01:00
tchaypoStevenK: I've been waiting for you to tell me I'm being terrible01:00
lifelesstchaypo: yes01:02
lifelesstchaypo: in an etherpad01:02
lifelesstchaypo: https://etherpad.openstack.org/p/tripleo-etherpads01:02
StevenKtchaypo: But you aren't?01:03
tchaypolifeless: on that topic, where's the link to the hp1 rebuild etherpad? I can never find that one when I'm after it01:03
tchaypoStevenK: you haven't seen https://pypi.python.org/pypi/fuggit yet?01:03
StevenKtchaypo: I did01:03
lifelesstchaypo: line 1201:04
tchaypolifeless: ah. the bit where it says "ci-hp1-rebuild" should have given it away01:04
lifelessI believe so01:06
lifelesslol01:09
lifeless+ FLOATINGIPID=1f24d732-8848-4213-ae72-a50f4c62ef0001:09
lifeless+ neutron floatingip-associate 1f24d732-8848-4213-ae72-a50f4c62ef00 id 01887cf8-0724-4e58-bfd5-56f2532668aa 020c930d-ec37-478a-b68d-c9276c400e48 037ecf7a-a608-44b4-b703-56e8da6c0616 046ae112-1b9a-4312-8850-b48b7d215c0e 05973632-d123-4565-aaa7-58b247509f35 0798e7df-7129-40c2-b469-6d6193770582 0c1b4c70-c833-4593-93a8-cac941c753ac 0d08e5c8-0743-4c05-8b8b-7f1e016dcab3 0ef7e29f-f71c-4df9-972f-49f346df2d7d 10876b9c-e7ac-4dd8-9de6-51a0553b01:09
lifeless....01:09
lifelessmore01:09
*** nosnos has joined #tripleo01:10
*** yamahata has joined #tripleo01:10
*** cwolferh has quit IRC01:12
tchaypoExpected a MAC address but received 80-C1-6E-80-D4-18.01:21
StevenKtchaypo: Haha01:22
*** ramishra has joined #tripleo01:26
*** penick has quit IRC01:26
tchaypocomparing against the testenv.json on hp1, it has mac addresses in the form XX:XX:XX instead of XX-XX-XX01:33
tchayposo... what made this be all silly on hp2?01:33
*** andreaf has quit IRC01:35
*** andreaf has joined #tripleo01:35
tchaypoah. it's in that format in machine-information-tab.txt and in nodes.json01:36
lifelesstchaypo: so the hp2 data was given to us in a different format01:36
lifelesstchaypo: so probably it needs its migration script tweaked01:37
StevenK"The standard (IEEE 802) format for printing MAC-48 addresses in human-friendly form is six groups of two hexadecimal digits, separated by hyphens (-) or colons (:) ..."01:38
*** andreaf has quit IRC01:40
*** andreaf has joined #tripleo01:41
*** andreaf_ has joined #tripleo01:44
*** andreaf has quit IRC01:44
*** andreaf_ has quit IRC01:45
*** andreaf_ has joined #tripleo01:45
*** andreaf_ has quit IRC01:46
*** andreaf_ has joined #tripleo01:46
*** andreaf_ has quit IRC01:46
*** andreaf_ has joined #tripleo01:47
*** ramishra has quit IRC01:47
*** ramishra has joined #tripleo01:48
lifelesstchaypo: so the seed failed to come up? or the node registration after it came up errored?01:51
*** ramishra has quit IRC01:52
*** eguz has joined #tripleo01:53
*** eguz has quit IRC01:53
*** weshay_afk has quit IRC01:56
*** eghobo has quit IRC01:57
tchaypothe latter02:22
tchaypoor at last, not the former. the seed is up, it's the call to setup-baremetal that failed02:22
tchaypohere we go02:23
tchaypothe import script seems to be called mknodes02:23
*** andreaf_ has quit IRC02:24
*** andreaf_ has joined #tripleo02:24
*** Yanyanhu has joined #tripleo02:29
tchaypookay, re-running devtest_seed.sh02:32
tchaypoand now seems like a good moment to take a break. Will be back after I head out, get lunch, and head to UNSW, so probably 2-3 hours02:34
lifelesshmm, bighammer isn't deleting things02:37
*** andreaf_ has quit IRC02:53
*** andreaf_ has joined #tripleo02:54
*** andreaf_ has quit IRC02:54
*** andreaf_ has joined #tripleo02:54
*** ccrouch has joined #tripleo02:55
*** andreaf_ has quit IRC02:55
*** andreaf_ has joined #tripleo02:55
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126303:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604003:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589803:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784803:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]03:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]03:00
*** untriaged-bot has quit IRC03:00
*** rainya has joined #tripleo03:11
*** rushiagr_away is now known as rushiagr03:16
*** jml has quit IRC03:20
*** jml has joined #tripleo03:23
rainyag'day lifeless!03:28
*** shardy has quit IRC03:31
lifelessrainya: oh hai!03:32
rainyalifeless, my brother in law is from melbourne and says "oh hai"03:33
rainyai thought it was just a him thing03:33
lifelessits australian I suspect03:33
lifelessI'm not sure where I picked it up03:33
StevenKI use it a fair amount03:34
rainyai thought rich got it from anime03:34
*** nosnos has quit IRC03:35
*** ramishra has joined #tripleo03:39
*** akuznetsov has joined #tripleo03:39
*** marun has quit IRC03:43
*** marun has joined #tripleo03:56
* StevenK grumbles at os-svc-enable 03:59
*** eghobo has joined #tripleo04:00
*** rushiagr is now known as rushiagr_away04:01
*** jroll|dupe has joined #tripleo04:03
tchaypohrm.04:04
tchaypommon.http"04:04
tchaypo                                                                                                                                                                       │E04:04
tchayponot helpful04:04
tchaypoExpected a MAC address but received 80-C1-6E-80-D4-18.04:04
tchaypookay, i thought testenv.json would be rebuilt from the new nodes.json but...04:05
tchaypoof course that happens in devtest_testenv.sh not devtest_seed.sh04:05
lifelesstchaypo: did you run devtest_testenv.sh appropriately? :)04:05
*** jroll has quit IRC04:05
*** jroll|dupe is now known as jroll04:05
*** andreaf_ has quit IRC04:12
*** andreaf has joined #tripleo04:12
*** andreaf has quit IRC04:15
*** vinsh has joined #tripleo04:16
*** rainya has quit IRC04:25
*** nosnos has joined #tripleo04:32
*** eghobo has quit IRC04:37
*** marun has quit IRC04:39
*** eghobo has joined #tripleo04:40
*** loki184 has joined #tripleo04:52
*** eghobo has quit IRC04:52
*** dsneddon has quit IRC05:05
StevenKlifeless: Can I bug you to review and +A https://review.openstack.org/111536 and https://review.openstack.org/111606 so I can propose a -config branch that cleans os-cloud-config's job template +2/-15 ?05:10
*** ramishra has quit IRC05:15
*** ramishra has joined #tripleo05:15
*** marun has joined #tripleo05:17
*** ramishra has quit IRC05:20
*** marun has quit IRC05:21
*** marun has joined #tripleo05:22
*** loki184 has quit IRC05:23
*** marun has quit IRC05:29
*** k4n0 has joined #tripleo05:30
*** noslzzp has quit IRC05:31
*** rlandy has joined #tripleo05:37
*** pensu has joined #tripleo05:42
*** bmahalakshmi has joined #tripleo05:45
*** pcrews has quit IRC05:47
*** akuznetsov has quit IRC05:54
*** killer_prince is now known as lazy_prince05:55
*** akuznetsov has joined #tripleo05:58
*** dshulyak_ has joined #tripleo06:01
*** dshulyak_ has quit IRC06:05
*** ramishra has joined #tripleo06:06
*** dtantsur|afk is now known as dtantsur06:08
*** Azher has joined #tripleo06:11
*** pradeep has joined #tripleo06:12
*** loki184 has joined #tripleo06:13
*** akuznetsov has quit IRC06:20
*** akuznetsov has joined #tripleo06:20
*** ramishra has quit IRC06:23
*** ramishra has joined #tripleo06:23
*** ramishra has quit IRC06:28
*** ramishra has joined #tripleo06:29
*** pradeep has quit IRC06:30
*** rdopieralski has joined #tripleo06:33
openstackgerritA change was merged to openstack/tripleo-image-elements: Use pacemaker for running ceilometer central agent  https://review.openstack.org/10539706:34
pensuHi everyone, I have set up tripleo using RDO instack....now, my overcloud is working fine except the "nodes" link in infrastructure tab, it says,"An unexpected error has occurred", any idea what could cause this error?06:35
*** jcoufal has joined #tripleo06:40
openstackgerritA change was merged to openstack/tripleo-image-elements: Use pacemaker to run neutron l3 agent and metadata agent  https://review.openstack.org/11104906:45
*** Yanyanhu has quit IRC06:48
*** Yanyanhu has joined #tripleo06:49
lifelesshttps://review.openstack.org/#/c/103449/ is deeply desirous of +2's06:52
*** pradeep has joined #tripleo06:52
lifelesspensu: have a look in the horizon logs?06:52
openstackgerritA change was merged to openstack/os-cloud-config: Add __future__ import to register-nodes  https://review.openstack.org/11153606:53
openstackgerritA change was merged to openstack/tuskar: Add scaling logic to the deployment plan  https://review.openstack.org/11066406:56
*** lazy_prince is now known as killer_prince06:56
*** jprovazn has joined #tripleo06:58
pensulifeless: I tried that and interestingly /var/log/horizon/ is an empty directory!06:59
*** killer_prince is now known as lazy_prince06:59
lifelesspensu: try in the apache2 logs dir07:00
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Small fixes to support Python 3  https://review.openstack.org/11160607:00
pensulifeless: there are no apache2 logs either :(07:02
tchaypolifeless: no, no I did not. I'm very clever.07:10
* tchaypo is going to cheat and steal the last invokation of devtest_testenv from history07:13
lifelesstchaypo: I'm not sure it wil be there,.... but you need to be supplying both a nodes list and a bm networks config07:14
tchaypoyep. and tweaking the paths to point at them.07:15
lifelesstchaypo: well, it accepts paths :>07:15
tchaypoand it setns NODE_CNT=0, which I wouldn't have thought of, but I assume is because we don't need any extra nodes generated, we just use the ones in node.json07:15
lifelesswhat does ?07:15
tchaypoNODE_CNT=0 devtest_testenv.sh --nodes nodes.json --bm-networks tripleo-image-elements/elements/tripleo-cd/configs/hp2_networks.json $TE_DATAFILE07:16
tchaypothe commandline I stole from history07:16
lifelesshmm07:16
lifelessI would have expected us to not make nodes if a nodes json is supplied07:17
lifelessthat might be a bug to fix :)07:17
* tchaypo checks07:17
tchayponope.07:18
tchaypoNODE_CNT default to 15 if it's not already set07:18
tchaypoand then we just pass that value unomdifted into07:18
tchaypocreate-nodes07:18
tchaypoit's a simple tweak for us to add though, and it does seem intuitive.07:18
lifelessthough, that said,StevenK had a patch to separate out the bm network and nodes maniuplation of the testenv, from the creation-of-a-test-env07:20
lifelessso I'd coordinate with him before changing stuff07:20
tchaypoStevenK: pokey pokey07:20
*** akuznetsov has quit IRC07:26
tchaypoa different answer would be to say07:26
tchaypothat vm deinfitions are cheap anyway, that's why we are comfortable setting 15 as the default even though we don't use that many07:27
tchayposo maybe having it making those practically-free-15 by default isn't really something to worry about07:27
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Don't create extra nodes if nodes file is provided  https://review.openstack.org/11379407:27
tchaypoStevenK: ^^^07:28
lifelesstchaypo: I think what I'm getting at is that running something called 'testenv' when you don't want to create a testenv is bogus07:28
*** sballe has joined #tripleo07:29
tchaypoit's not called "Create testenv", just "testenv"; to me that suggests "prepare the testenv"07:30
tchaypowhich could involve creating things, but could also just involve parsing the testenv.json and making sure it's valid07:30
tchaypofeel free to nix the change :)07:30
tchaypomeanwhile, devtest_seed.sh just finished, I done made some flavours.07:31
tchaypoand now it's time for me to do my german homework before class.07:31
StevenKtchaypo: If nodes are passed in, I would just set NODE_CNT=0 TBH07:31
tchaypowhy the hell does https://pypi.python.org/pypi/fuggit show 277 downloads?07:32
tchaypoI'm going to assume that's how many people have run bandersnatch in the last 24 hours07:32
StevenKPerhaps. I know my mirror grabbed it this morning07:32
tchaypoStevenK: that doesn't seem right to me. What if I have one baremetal node that I want to use as the compute, but I need to make VMs for the controlplane?07:33
tchaypoI think 0 is a sensible default but I think we should allow the user to mix-and-match according to their needs07:34
*** rwsu has quit IRC07:34
StevenKtchaypo: Perhaps set NODE_DEFAULT=15 ; if NODES then NODE_DEFAULT=0  and then NODE_CNT=${NODE_CNT:-$NODE_DEFAULT}  ?07:37
*** shardy has joined #tripleo07:37
tchaypoyou could write that more tersely07:37
tchaypobut it would also be unreadable perl-level line noise, so I'm not going to suggest that we do07:38
StevenKtchaypo: I was being terse for one IRC line. At least four lines in shell ? :-)07:39
tchaypoStevenK: can you leave a commment on the review for me to look at later? Ich muss wirklich die Hausaufgaben anfangen07:39
tchaypowell no, I think I could do it in two, but it wouldn't be readable. 4 lines sounds about right :(07:40
StevenKtchaypo: Ich werde07:40
tchaypobis später!07:40
lazy_princeexit07:41
lazy_princesorry.. wrong window..07:41
shadowerlifeless: belated pong (from Monday, was on vacation)07:44
loki184lifeless: This patch requires yours +1 https://review.openstack.org/#/c/110562/ Please look at it07:47
*** Yanyanhu has quit IRC07:48
lifelesstchaypo: mix and match isn't supported by the contract today07:49
lifelesstchaypo: that would be something new07:49
lifelesstchaypo: (since we don't put the created nodes into the testenv)07:50
*** sballe has quit IRC07:51
*** rwsu has joined #tripleo07:52
*** Yanyanhu has joined #tripleo07:53
*** ramishra has quit IRC07:55
*** ramishra has joined #tripleo07:55
*** xuhaiwei has quit IRC08:00
*** ramishra has quit IRC08:00
*** derekh has joined #tripleo08:04
*** jang2 has joined #tripleo08:06
*** jml has quit IRC08:06
*** jistr has joined #tripleo08:08
*** akrivoka has joined #tripleo08:09
*** jml has joined #tripleo08:09
*** sballe has joined #tripleo08:11
derekhlifeless: did you see my message about glance getting failures connecting to http://10.10.16.155:5000/v2.0/tokens , does that need to be changed ?08:12
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Handle double path registration  https://review.openstack.org/11354608:12
derekhlifeless: I'll try an reproduce the swift failure again this morning but I'm only here for a few hours today so not sure how far I'll get08:13
lifelessderekh: I didn't but I saw it and fixed it08:13
lifelessderekh: I passed 500 80 with bighammer today08:14
lifelessderekh: now doing 3 3 with the exit 0 taken out08:14
lifelessderekh: I've moved your _derekh.sh version onto the main one08:14
derekhlifeless: ok, great, so did you redeploy swift ?08:14
lifelessderekh: I added patches to make swift have 2 replicas to avoid this in future08:14
derekhlifeless: ok08:15
lifelessderekh: deleted the images, and ran prepare-ci-cloud again to upload new ones08:15
derekhlifeless: got ya08:15
lifelessderekh: oh yeah, glance refused to delete them08:15
lifelessderekh: because errors from swift, I marked them deleted=1 via sql08:15
derekhlifeless: yup, I tried to delete them yesterday also and couldn't08:16
derekhlifeless: ok08:16
*** gfidente has joined #tripleo08:18
*** ifarkas has joined #tripleo08:20
lifelessall failed at running the actual job :/08:24
*** lucas-dinner is now known as lucasagomes08:24
lifelessERROR (ClientException): Unknown Error (HTTP 503)08:24
lifelessfrom nova list08:24
lifelessderekh: if you can look into that, that would be create :)08:25
derekhlifeless: will do08:26
lifelessthat was -108:26
lifeless-2 got to the oc failed one of the three oc nodes08:26
lifelessthre failed one of the oc nodes too08:27
derekhlifeless: so they mad some progress :-)08:28
lifelessderekh: yeah08:30
lifelessderekh: we may be finding that the eventlet race condition is more prevalent on these hypervisors08:31
lifelessderekh: in which case pulling that in as a hotfix will be important08:31
lifelessderekh: adam_g has a patch up to do that08:31
lifelessderekh: (as an element)08:31
derekhlifeless: we already are I thought, lemme check08:31
*** sballe has quit IRC08:32
derekhlifeless: http://git.openstack.org/cgit/openstack/tripleo-image-elements/commit/?id=0011e19c15cee09c21a1ec5f772a480a677201b908:33
*** bmahalakshmi has quit IRC08:33
*** athomas has quit IRC08:33
lifelessderekh: ah cool08:34
*** bmahalakshmi has joined #tripleo08:34
*** athomas has joined #tripleo08:35
derekhlifeless: frick it08:35
*** athomas has quit IRC08:35
lifelessderekh: thats a form of southern cooking right ?08:35
derekhlifeless: the TE's probably need to be redployed with08:35
*** athomas has joined #tripleo08:36
derekhlifeless: https://review.openstack.org/#/c/110492/08:36
lifelessderekh: ah08:36
lifelessderekh: why did two of the oc nodes succedin that case?08:37
derekhlifeless: I see them in the BUILD state08:37
lifelessoh08:37
lifelessso you think its not detecting the failure08:37
*** lazy_prince is now known as killer_prince08:38
derekhlifeless: yup, possibly, we don't have the logs to know for sure,08:38
derekhlifeless: but I do know for sure that we need that patch on the TE's08:39
derekhlifeless: unless they have rebuilt with it since the sprint08:39
lifelessderekh: I have not08:39
lifelessderekh: so add to repo_refs/repo_refs.yaml08:39
*** dsneddon has joined #tripleo08:41
lifelessderekh: ok pull that into our trees; I haven't rebuild the image or anything - I'll leave that to you08:41
lifelessderekh: pulled08:41
lifelessderekh: (see window 0 history for what I did)08:42
derekhlifeless: ok, will do and will take a look for the nova list error while its building08:42
*** rameshg87 has joined #tripleo08:44
*** ramishra has joined #tripleo08:44
*** ramishra has quit IRC08:44
*** akuznetsov has joined #tripleo08:45
derekhdeploy-testenv running08:52
derekhlifeless: the error doing nova list may be one of our intermitent ci failures, I'm guessing this one https://bugs.launchpad.net/tripleo/+bug/132952808:55
uvirtbotLaunchpad bug 1329528 in tripleo "seed cloud cannot signal 'ready' - we guess at readiness and then race with os-collect-config" [High,In progress]08:55
derekhlifeless: that or a time mismatch between jenkins node and TE, I'll check that once their back up08:56
lifelessderekh: I thought we had ntp configured now in our config ?08:56
derekhlifeless: did we? checking...08:57
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Corrected heat_opts name to tuskar_opts  https://review.openstack.org/11266708:59
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135623309:00
uvirtbotLaunchpad bug 1356233 in tripleo "Error in deploying a new node" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126309:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604009:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589809:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784809:00
*** untriaged-bot has quit IRC09:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]09:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]09:00
derekhlifeless: its part of the heat template but I don;t think its passed in, should it be the one from elements/tripleo-cd/configs/hp1_ci_env.json ?09:02
lifelessI don't think we're using that file yet..09:04
lifelessoh misread09:04
lifelessyes09:04
lifelessthat ntp server09:04
lifelessthats the uc09:04
derekhlifeless: ok, will add a patch for it and redeploy09:06
*** jp_at_hp has joined #tripleo09:07
*** pblaho has joined #tripleo09:08
*** sballe has joined #tripleo09:10
lifelessjp_at_hp: https://review.openstack.org/#/c/113431/ needs a review09:10
*** pblaho has quit IRC09:11
*** pblaho has joined #tripleo09:11
openstackgerritDerek Higgins proposed a change to openstack/tripleo-image-elements: Configure NTP on hp1 TE's  https://review.openstack.org/11381709:14
*** rameshg87_ has joined #tripleo09:15
*** pelix has joined #tripleo09:17
*** rameshg87 has quit IRC09:18
jp_at_hphmm - I just rechecked that lifeless, but I noticed that across 2 runs it has passed all 3 voting tests...09:18
jp_at_hpcode lgtm, so +2 is added.09:19
lifelessjp_at_hp: thanks, when it hits ci nirvana I'll add a +A09:20
*** sballe has quit IRC09:20
lifelessjp_at_hp: if you're in an approving mood https://review.openstack.org/#/c/105878/09:21
jp_at_hplifeless: still on hols, just had to get some expenses in, so logging off now - maybe I'll take a quick look though...09:21
*** bmahalakshmi has quit IRC09:22
lifelessjp_at_hp: ah, ok :) - thanks for the peek !09:22
*** sballe has joined #tripleo09:22
*** sballe_ has joined #tripleo09:24
openstackgerritA change was merged to openstack/tripleo-heat-templates: Change overcloud to use VIP for MySQL  https://review.openstack.org/10587809:27
*** sballe has quit IRC09:27
openstackgerritCian O'Driscoll proposed a change to openstack/tripleo-image-elements: Allow single node mariadb clusters to restart  https://review.openstack.org/11382309:31
*** jp_at_hp has quit IRC09:40
openstackgerrittom-howley proposed a change to openstack/tripleo-image-elements: CORE-1217 Configure haproxy logging  https://review.openstack.org/11382709:45
*** sballe__ has joined #tripleo09:46
*** andreaf has joined #tripleo09:47
*** sballe_ has quit IRC09:50
*** Yanyanhu has quit IRC09:50
*** pensu has quit IRC10:00
*** jeblair has quit IRC10:00
*** SpamapS has quit IRC10:00
*** bnemec has quit IRC10:00
*** pblaho is now known as pblaho|afk10:01
*** sballe has joined #tripleo10:06
*** rameshg87__ has joined #tripleo10:07
*** sballe__ has quit IRC10:07
*** pensu has joined #tripleo10:07
*** jeblair has joined #tripleo10:07
*** SpamapS has joined #tripleo10:07
*** bnemec has joined #tripleo10:07
*** rameshg87_ has quit IRC10:09
*** rameshg87_ has joined #tripleo10:09
openstackgerritdave-mcnally proposed a change to openstack/tripleo-image-elements: Adding configuration options for stunnel.  https://review.openstack.org/11287010:11
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Fix race between apache2 starting and o-r-c configuring apache2  https://review.openstack.org/11203910:12
*** rameshg87__ has quit IRC10:12
openstackgerritdave-mcnally proposed a change to openstack/tripleo-image-elements: Adding configuration options for stunnel.  https://review.openstack.org/11287010:14
openstackgerritdave-mcnally proposed a change to openstack/tripleo-heat-templates: Adding default stunnel configuration options.  https://review.openstack.org/11331310:14
*** jcoufal has quit IRC10:19
*** ByPasS has quit IRC10:20
*** rameshg87_ has quit IRC10:20
*** sballe_ has joined #tripleo10:21
*** sballe has quit IRC10:21
*** dtantsur is now known as dtantsur|lunch10:23
*** pensu has quit IRC10:23
*** jeblair has quit IRC10:23
*** SpamapS has quit IRC10:23
*** bnemec has quit IRC10:23
*** sballe_ has quit IRC10:25
*** sballe has joined #tripleo10:26
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324010:30
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making setup endpoint idempotent  https://review.openstack.org/11282510:30
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create roles idempotent  https://review.openstack.org/11383910:30
*** pensu has joined #tripleo10:30
*** jeblair has joined #tripleo10:30
*** SpamapS has joined #tripleo10:30
*** bnemec has joined #tripleo10:30
*** bmahalakshmi has joined #tripleo10:33
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324010:35
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create roles idempotent  https://review.openstack.org/11383910:35
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making setup endpoint idempotent  https://review.openstack.org/11282510:35
*** rameshg87 has joined #tripleo10:36
*** markmc has joined #tripleo10:36
*** sballe has quit IRC10:36
openstackgerritLadislav Smola proposed a change to openstack/tripleo-incubator: Switch to os-cloud-config register-endpoints  https://review.openstack.org/11384110:37
*** sballe has joined #tripleo10:37
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add svc-map to cinder-api  https://review.openstack.org/10419810:39
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add initial svc-map support  https://review.openstack.org/10419610:39
*** yamahata has quit IRC10:42
*** sballe_ has joined #tripleo10:44
*** sballe has quit IRC10:44
*** Azher has quit IRC10:47
*** morazi has quit IRC10:51
*** pblaho|afk is now known as pblaho11:03
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add initial svc-map support  https://review.openstack.org/10419611:05
*** rameshg87 has quit IRC11:06
*** loki184 has quit IRC11:06
*** rameshg87 has joined #tripleo11:06
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add initial svc-map support  https://review.openstack.org/10419611:07
*** jcoufal has joined #tripleo11:09
*** derekh has quit IRC11:09
*** loki184 has joined #tripleo11:13
*** rameshg87 has quit IRC11:17
pensulifeless: To do a new deployment, do I need to remove the previous one? I mean, the previous stack?11:19
*** lucasagomes is now known as lucas-hungry11:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create roles idempotent  https://review.openstack.org/11383911:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create tenant idempotent  https://review.openstack.org/11385211:22
*** sballe_ has quit IRC11:24
*** rameshg87 has joined #tripleo11:31
*** pblaho is now known as pblaho|afk11:31
*** andreaf_ has joined #tripleo11:32
*** andreaf has quit IRC11:34
*** andreaf_ has quit IRC11:38
*** pblaho|afk is now known as pblaho11:39
*** noslzzp has joined #tripleo11:40
*** morazi has joined #tripleo11:47
*** nosnos has quit IRC11:54
openstackgerritAna Krivokapic proposed a change to openstack/tuskar-ui: Replace mocked BareMetal node data with real API calls  https://review.openstack.org/11385811:55
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Migrate from common code to oslo.db  https://review.openstack.org/10454811:56
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add svc-map to cinder-api  https://review.openstack.org/10419812:03
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324012:04
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create tenant idempotent  https://review.openstack.org/11385212:04
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Making create roles idempotent  https://review.openstack.org/11383912:04
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make setup endpoint idempotent  https://review.openstack.org/11282512:04
openstackgerritA change was merged to openstack/tuskar-ui: Expose IP address of a node instance  https://review.openstack.org/11347212:05
*** sseago has quit IRC12:08
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Migrate from common code to oslo.db  https://review.openstack.org/10454812:09
*** jdob has joined #tripleo12:15
*** dprince has joined #tripleo12:15
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Migrate from common code to oslo.db  https://review.openstack.org/10454812:16
*** boris-42 has quit IRC12:21
pblahod0ugal: was it this line? "set_defaults(cfg.CONF)"12:21
d0ugalpblaho: heh, good spot. I was just about to ask if there is a better way to do that12:22
*** lucas-hungry is now known as lucasagomes12:22
d0ugalpblaho: trying to load the config from oslo.db12:23
d0ugalpblaho: the import_opt stuff doesn't work anymore.12:24
d0ugaljdob: https://review.openstack.org/#/c/112997/12:26
d0ugaljdob: That review of yours has started to error, I have no idea why.12:26
jdobugh12:26
*** weshay has joined #tripleo12:27
jdobthis review has been out there for a week passing and not merged, and now its failing12:27
d0ugaljdob: wanted to point you to it before you based even more reviews on it :)12:27
d0ugalYeah12:27
d0ugalSeems like a legit Python unittest error tho'12:27
d0ugaljdob: see my comment, I guess it's something intermittent. The tests pass locally for me.12:29
jdobthats even weirder, since its not like they are timing tests or anything12:29
jdobya, pass for me too12:29
*** bmahalakshmi has quit IRC12:31
*** boris-42 has joined #tripleo12:32
*** rameshg87 has quit IRC12:34
*** sballe has joined #tripleo12:34
pblahod0ugal, jdob: pass for me locally too12:34
*** rameshg87 has joined #tripleo12:35
*** pensu has quit IRC12:35
d0ugalpblaho: damn! these things normally fail for you :)12:35
*** jml has quit IRC12:37
*** jml has joined #tripleo12:38
pblahod0ugal: I know... that is the reason I tried to run it :-)12:39
*** sballe_ has joined #tripleo12:41
*** sballe has quit IRC12:44
*** rameshg87 has quit IRC12:48
*** julim has joined #tripleo12:50
*** noslzzp has quit IRC12:51
*** julim has quit IRC12:52
*** julim has joined #tripleo12:53
openstackgerritPetr Blaho proposed a change to openstack/tuskar: Changes DELETE Role from Plan to use role_uuid  https://review.openstack.org/11386912:54
*** rlandy_ has joined #tripleo13:05
openstackgerritPetr Blaho proposed a change to openstack/tripleo-specs: Modifies API spec to use uuid for Role endpoints  https://review.openstack.org/11387413:09
*** rlandy_ has quit IRC13:11
*** rlandy_ has joined #tripleo13:12
*** andreaf has joined #tripleo13:14
*** rlandy_ has quit IRC13:17
*** rlandy_ has joined #tripleo13:20
*** dsneddon has quit IRC13:20
*** noslzzp has joined #tripleo13:21
*** lblanchard has joined #tripleo13:21
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324013:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create tenant idempotent  https://review.openstack.org/11385213:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create roles idempotent  https://review.openstack.org/11383913:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make setup endpoint idempotent  https://review.openstack.org/11282513:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Allow setup services endpoints  https://review.openstack.org/11388213:22
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create keystone endpoint idempotent  https://review.openstack.org/11388313:22
*** lblanchard has quit IRC13:22
*** rlandy_ has quit IRC13:24
*** sballe_ has quit IRC13:24
*** rlandy_ has joined #tripleo13:25
*** andreaf_ has joined #tripleo13:25
*** dtantsur|lunch is now known as dtantsur13:26
*** lblanchard has joined #tripleo13:27
*** rlandy_ has quit IRC13:27
*** andreaf has quit IRC13:28
*** rushiagr_away is now known as rushiagr13:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create tenant idempotent  https://review.openstack.org/11385213:30
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create keystone endpoint idempotent  https://review.openstack.org/11388313:30
*** rlandy_ has joined #tripleo13:31
*** andreaf_ has quit IRC13:31
*** andreaf has joined #tripleo13:32
*** rlandy_ has quit IRC13:33
*** rlandy_ has joined #tripleo13:34
*** andreaf has quit IRC13:34
*** rlandy_ has quit IRC13:34
*** andreaf has joined #tripleo13:35
*** rlandy_ has joined #tripleo13:35
*** rlandy_ has quit IRC13:39
*** pcrews has joined #tripleo13:42
*** jprovazn has quit IRC13:46
gfidenteso while reviewing I came across this change https://review.openstack.org/#/c/113652/13:48
gfidenteI realized glance-api currently expects glance-registry to be available on local-ipv4 that is why it works without the patch13:48
openstackgerrittom-howley proposed a change to openstack/tripleo-image-elements: Configure haproxy logging  https://review.openstack.org/11382713:48
gfidenteI wonder if it would make any sense to change that though, cause in glance V2 API registry is deprectated and doesn't need the additional -registry service running at all13:49
gfidentealternatively we should put haproxy in front of -registry and make glance-api point to the VIP for that too... any strong opinion on the matter>13:49
gfidente?13:50
*** sballe has joined #tripleo13:51
*** andreaf_ has joined #tripleo13:52
*** marun has joined #tripleo13:53
*** sballe has quit IRC13:53
*** andreaf has quit IRC13:55
*** sballe has joined #tripleo13:56
jdobd0ugal, pblaho: figured out the issue with that patch, fixing now13:57
lsmolalifeless: SpamapS hola, could you look on https://review.openstack.org/#/c/111985/13:57
openstackgerritJay Dobies proposed a change to openstack/tuskar: Adds logic layer for handling plans  https://review.openstack.org/11299713:58
lsmolalifeless: SpamapS I have the idempotent calls done, so I don't want it to be blocked on Spec, thank you13:58
gfidentelsmola, I was looking at the idempotent patches too13:59
SpamapSlsmola: Heat can't really do admin-y setup things13:59
gfidentebut for LDAP, you're assuming users are in the directory while roles and membership is still in the DB13:59
SpamapSlsmola: You need an orchestration endpoint to talk to Heat...13:59
SpamapSand setup-endpoints ads it..14:00
gfidentelsmola, do we plan to cover the scenario where roles and membership also are stored in the directory?14:00
lsmolaSpamapS: last time you +2 the spec, what has changed? :-)14:00
lsmolaSpamapS: seems like doing that from Software config is the easiest way now, since the calls will be idempotent, we can easily switch to something more advanced in K14:01
*** andreaf_ has quit IRC14:02
*** sballe has quit IRC14:02
*** andreaf_ has joined #tripleo14:02
SpamapSlsmola: oh like that maybe it will work. :-P14:02
lsmolaSpamapS: :-)14:02
lsmolaSpamapS: we need some solution for J14:03
lsmolaSpamapS: making orchestration endpoint will take a lot of planning I presume14:03
SpamapSlsmola: sorry, I was on vacation. It is like an eternity since then. ;)14:03
lsmolaSpamapS: +barbican I guess, so we can sanely get passwords from that14:04
lsmolaSpamapS: yeah I got you :-) I've been out almost for a month, that was a hard way back :-D14:04
lsmolagfidente: not sure if I get the question14:05
lsmolaSpamapS: so will the spec be ok liek this for J? I guess the orchestration endpoint would be worth a design session at least14:06
lsmolagfidente: what roles and membership are you referring to?14:07
gfidentelsmola, let me find the submission so we can talk about the actual code which is easier14:08
lsmolagfidente: ok14:08
lsmolagfidente: I was just trying to make feature parity with init code that was in devtest scripts now and convert it to idempotent calls14:08
lsmolagfidente: so everything that was in devtest should be in os-cloud-config now14:09
gfidentelsmola, indeed I suspect the changes for keystone/ldap integration I saw were not from you now14:09
lsmolagfidente: except one roel ceilometer needs, will add that14:09
gfidenteand my question was about that keystone/ldap stuff so sorry for the confusion14:10
lsmolagfidente: nope, I checked them, they had a problem with user create no being idempotent14:10
lsmolagfidente: ok14:10
*** jprovazn has joined #tripleo14:11
*** rlandy_ has joined #tripleo14:15
*** openstackgerrit has quit IRC14:16
*** rlandy__ has joined #tripleo14:17
*** openstackgerrit has joined #tripleo14:17
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.context  https://review.openstack.org/11290414:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.fixture  https://review.openstack.org/11290514:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.log_handler  https://review.openstack.org/11290614:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.network_utils  https://review.openstack.org/11290714:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.cliutils  https://review.openstack.org/11290814:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused/unfinished tuskar.manager  https://review.openstack.org/11320514:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.rpc  https://review.openstack.org/11289814:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.service  https://review.openstack.org/11289914:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.processutils  https://review.openstack.org/11290014:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.notifier  https://review.openstack.org/11285814:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.loopingcall  https://review.openstack.org/11290114:18
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.threadgroup  https://review.openstack.org/11290214:19
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.periodic_task  https://review.openstack.org/11285614:19
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.eventlet_backdoor  https://review.openstack.org/11290314:19
lsmolad0ugal: wuuu :-)14:19
*** rlandy_ has quit IRC14:19
d0ugallsmola: lol, they should all pass now14:19
lsmolaSpamapS: or by orchestration endpoint you probably mean just heat, but previously we were talking about some endpoint that would do the configuration14:20
lsmolaSpamapS: I have to find the old conversation14:20
*** pradipta has joined #tripleo14:20
*** akrivoka has quit IRC14:24
SpamapSlsmola: yeah I'm sorry I totally purged all of that context. :-P14:24
lsmolaSpamapS: np :-)14:24
SpamapSlsmola: ah thanks for the reminder, ok. :)14:25
*** akrivoka has joined #tripleo14:25
* lsmola will be back in 40 minutes14:26
*** cwolferh has joined #tripleo14:31
*** noslzzp has quit IRC14:31
*** akrivoka has quit IRC14:32
*** akrivoka has joined #tripleo14:32
*** noslzzp has joined #tripleo14:34
*** julim has quit IRC14:36
*** julim has joined #tripleo14:37
*** sseago has joined #tripleo14:38
*** sdake has joined #tripleo14:46
*** sdake has quit IRC14:46
*** sdake has joined #tripleo14:46
*** sballe has joined #tripleo14:49
*** jprovazn is now known as jprovazn_afk14:49
openstackgerritPhil Neal proposed a change to openstack/tripleo-image-elements: Add elements for Ceilometer alarm services  https://review.openstack.org/11364714:56
gfidenteregebro, as mentioned yesterday, rwsu is your guy... https://review.openstack.org/#/c/108986/14:56
regebrogfidente: Ah, I might take the time to test that in the near future. Right now I'm just using Ubuntu instead. :-)14:58
*** pblaho is now known as pblaho|afk14:58
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126315:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604015:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589815:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]15:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784815:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135623315:00
uvirtbotLaunchpad bug 1356233 in tripleo "Error in deploying a new node" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135628315:00
uvirtbotLaunchpad bug 1356283 in tripleo "check-tripleo-novabm-overcloud-f20-nonha gate job is setting USE_IRONIC=0" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633915:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
openstackgerritJay Dobies proposed a change to openstack/tripleo-specs: Corrected configuration value request example  https://review.openstack.org/11391915:00
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in fedora  https://review.openstack.org/10844115:01
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: package-installs for pre-install.d/post-install.d  https://review.openstack.org/10843715:01
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in more elements  https://review.openstack.org/10876215:01
*** dsneddon has joined #tripleo15:01
*** ramishra has joined #tripleo15:03
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in fedora  https://review.openstack.org/10844115:05
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: package-installs for pre-install.d/post-install.d  https://review.openstack.org/10843715:05
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in more elements  https://review.openstack.org/10876215:05
*** funzo has quit IRC15:10
*** funzo has joined #tripleo15:10
*** pblaho|afk is now known as pblaho15:10
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Added a test to verify the storage date fields  https://review.openstack.org/11392515:14
*** cody-somerville has quit IRC15:18
*** rlandy has quit IRC15:18
*** yamahata has joined #tripleo15:19
*** rlandy__ has quit IRC15:22
openstackgerritA change was merged to openstack/tuskar: Fix the only_latest flag in the store list methods  https://review.openstack.org/11327815:24
*** ByPasS has joined #tripleo15:26
openstackgerritChristian Berendt proposed a change to openstack/tripleo-specs: Rename bash8 to bashate  https://review.openstack.org/11393315:27
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-incubator: Base decision to use MariaDB on NODE_DIST  https://review.openstack.org/11355715:34
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-incubator: Use percona on debian systems  https://review.openstack.org/10941515:34
*** rushiagr is now known as rushiagr_away15:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: mariadb: call os-svc-enable  https://review.openstack.org/11394315:47
*** dprince has quit IRC15:55
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Migrate from common code to oslo.db  https://review.openstack.org/10454815:55
d0ugalpblaho: ^ I think that's a slightly better way to do it :)15:55
*** akrivoka has quit IRC15:56
*** ByPasS has quit IRC15:57
*** spzala has joined #tripleo15:57
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394615:59
*** eghobo has joined #tripleo15:59
pblahod0ugal: that looks good.... so all issues solved? sample config and dbsync?16:00
d0ugalpblaho: yup, I think so16:00
d0ugalCI doesn't verify dbsync, but it will for the rest16:01
d0ugalI just done a dbsync locally16:01
*** k4n0 has quit IRC16:01
*** rushiagr_away is now known as rushiagr16:03
pblahod0ugal: awesome16:04
d0ugalpblaho: Now I think I can finally start doing what I set out to do, make the tests run against MySQL :)16:05
*** jyoti-ranjan has joined #tripleo16:09
pblahod0ugal: :-)16:09
jyoti-ranjanI need assistance for DIB element.16:12
jyoti-ranjanI need to copy set of files to specific location. For e.g. I have a DIB element where we have folder addon/scripts/hello.txt and want the file hello.txt to be copied to /home/ubuntu/Jyoti.16:12
jyoti-ranjanIs it possible? I think yes. To achieve this what should be correct cp command?16:13
*** dprince has joined #tripleo16:18
slagle4616:18
slagleww16:19
*** jcoufal has quit IRC16:20
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-incubator: Idempotency for keystone initialization  https://review.openstack.org/10617516:21
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394616:21
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-image-elements: Keystone/LDAP integration  https://review.openstack.org/10616716:21
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-image-elements: Keystone LDAP domain configuration  https://review.openstack.org/10811716:22
*** ramishra has quit IRC16:23
*** yamahata has quit IRC16:26
openstackgerritAlexis Lee proposed a change to openstack/tripleo-image-elements: Symlink to backup dir until ephemeral storage available  https://review.openstack.org/11395716:26
*** rdopieralski has quit IRC16:27
dprincebnemec greghaynes slagle jdob can you guys weigh in again on https://review.openstack.org/#/c/101825/16:32
*** dshulyak_ has joined #tripleo16:34
*** dshulyak_ has quit IRC16:35
openstackgerritDougal Matthews proposed a change to openstack/tuskar: WIP MySQL and PostgreSQL testing  https://review.openstack.org/11395816:35
devanandais tripleo CI cloud using ironic for the physical layer (rebuild UC nodes) yet?16:35
*** andreaf_ has quit IRC16:36
*** dshulyak_ has joined #tripleo16:37
*** akrivoka has joined #tripleo16:42
*** pradeep has quit IRC16:44
dprincedevananda: the new HP cloud is, but is isn't yet live (plugged into jenkins)16:45
*** gfidente has left #tripleo16:45
devanandadprince: ack. remind me - hw many nodes is that?16:45
*** gfidente has joined #tripleo16:45
dprincedevananda: for the rh1 cloud we are still using Nova BM though, built it out several months ago16:45
dprincedevananda: I think hp1 is like 40 nodes16:46
dprincedevananda: let me double check16:46
devanandaI mean the "new HP cloud"16:46
dprincedevananda: yeah, Ng would know that 16:46
devanandaI've heard of "few hundred nodes" but dont know if that's the one you referring to16:46
dprinceNg: how many nodes is in the HP2 zone?16:46
Ngdprince: devananda: I believe it has 60 nodes atm, with 40 more on the way. I don't think it's actually deployed yet. I was starting on that before moving to other things and I think tchaypo is looking at it now16:49
devanandaNg: ack, thanks. was just looking to see if ironic was used anywhere in tripleo yet (at the physical layer)16:50
*** sballe has quit IRC16:50
gfidentethis passed https://review.openstack.org/#/c/112367/ , so we can have haproxy reload back again :)16:51
dprincedevananda: I only use Ironic at this point, I see no reason to keep using nova bm when Ironic already works, and has better API/CLI tools16:53
jdobdprince: will take a look16:53
devanandadprince: ++16:53
openstackgerritLennart Regebro proposed a change to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397016:53
*** ByPasS has joined #tripleo16:53
devanandadprince: last i heard, the build out of tripleo CI underclouds was still done with noavbm, which you have confirmed16:54
devanandawhich makes me sad, since I can't point at it and say "look! ironic is in use in another cloud"16:54
hewbroccadevananda: as of this week we are using only Ironic w/instack IIUC16:55
dprincedevananda: yeah, sorry about that. What you can say is that all new TripleO clouds are using it. the only reason we are sticking with nova-bm on RH1 is for stability16:55
devanandadprince: what does "all new" mean? I haven't kept up with the new regions - do you have a wiki listing which runs what?16:56
devanandaalso, i find it odd that you'd say nova-bm is more stable in some way16:56
dprincedevananda: all new, means that when we rebuild rh1 (as in make a new cloud) we'll use Ironic16:56
devanandaah16:57
dprincedevananda: the plan is get HP1 stable, switch CI to it, make sure things are cool, disable RH1, rebuild w/ Ironic on RH116:58
openstackgerritJay Dobies proposed a change to openstack/tripleo-specs: Corrected configuration value request example  https://review.openstack.org/11391917:00
*** chuckC has quit IRC17:01
*** bmahalakshmi has joined #tripleo17:03
*** akrivoka has quit IRC17:05
*** yamahata has joined #tripleo17:11
*** jyoti-ranjan has left #tripleo17:14
*** yamahata has quit IRC17:18
*** pradeep has joined #tripleo17:21
*** markmc has quit IRC17:27
*** loki184 has joined #tripleo17:29
*** eguz has joined #tripleo17:31
SpamapShm17:31
SpamapStrying to debug a devtest failure17:31
SpamapSnone of the pacemaker/corosync managed things seem to be working17:32
SpamapSwhere do they log stdout/stderr from the things they run?17:34
*** eghobo has quit IRC17:35
*** lucasagomes is now known as lucas-dinner17:36
*** pradipta is now known as pradipta_away17:37
*** ccrouch has quit IRC17:43
*** pelix has quit IRC17:44
bnemeclifeless: greghaynes: https://review.openstack.org/#/c/112367/ passed CI after the rebase.  Ready for a +2/+A.17:49
*** loki184 has quit IRC17:49
openstackgerritA change was merged to openstack/tripleo-image-elements: Revert "Revert "Adding haproxy reload check""  https://review.openstack.org/11236717:50
bnemecgreghaynes: Thanks!17:50
*** chuckC has joined #tripleo17:54
SpamapSanyone else using devtest and seeing the overcloud controller at 20+ load average?17:56
SpamapSlots of CPU time going to corosync17:57
*** ccrouch has joined #tripleo17:58
jprovazn_afkSpamapS, hm, looks good on my setup, pacemaker for neutron+ceilometer patches were merged recently so that probably causes it, anything interesting in corosync logs?17:59
*** jprovazn_afk is now known as jprovazn17:59
*** penick has joined #tripleo18:00
*** penick has quit IRC18:00
SpamapSjprovazn: I don't know where corosync logs18:00
SpamapSbut there's nothing in /var/log/corosync18:00
*** saurabhs has joined #tripleo18:01
SpamapSjprovazn: so its working fine for you, with the defaults.. as in, 2G vms?18:01
jprovaznSpamapS, it's fine on fedora 64bit 4G18:01
SpamapS4G?18:02
SpamapSthe default is 2G at the moment18:02
SpamapSand I suspect it is way too low18:02
*** penick has joined #tripleo18:02
* jprovazn checks devtest_variables18:03
*** pradeep has quit IRC18:03
jprovaznSpamapS, ##    For 64bit it is better to create VMs with more memory and storage because of18:04
jprovazn##    increased memory footprint (3GB minimum - we suggest 4GB)::18:04
jprovazntripleo-incubator/scripts/devtest_testenv.sh18:04
SpamapSLinux overcloud-controller0-xl3r25htimep 3.13.0-33-generic #58-Ubuntu SMP Tue Jul 29 16:47:17 UTC 2014 i686 i686 i686 GNU/Linux18:05
*** loki184 has joined #tripleo18:05
SpamapSMem:          2015       1840        175         53         13        14118:05
jprovaznSpamapS, trying with 2G i386 images now if I hit the same issue18:05
SpamapSI literally just ran 'devtest.sh --trash-my-machine' after setting TRIPLEO_ROOT ... this is what I got.. an unusable cloud18:05
SpamapS10s to get a token18:06
jprovaznSpamapS, :( 2G is apparently not enough18:07
*** jistr has quit IRC18:08
openstackgerritErik Colnick proposed a change to openstack/tripleo-specs: TripleO DVR enablement  https://review.openstack.org/11398318:09
*** pblaho has quit IRC18:09
openstackgerritJay Dobies proposed a change to openstack/tuskar: Adds logic layer for handling plans  https://review.openstack.org/11299718:10
jprovaznSpamapS, it will take me a while to rebuild locally, but I suppose there are 2 options: 1) revert the 2 patches 2) bump default memory value18:12
openstackgerritJay Dobies proposed a change to openstack/tuskar: Adds logic layer for handling plans and roles  https://review.openstack.org/11299718:12
openstackgerritA change was merged to openstack-infra/tripleo-ci: Extract audit.log from archive  https://review.openstack.org/11337018:13
jprovaznSpamapS, bumping default memory value looks like something we will have to do anyway18:13
*** loki184 has quit IRC18:14
jprovaznFolks, would you have any objections if I send a patch which sets NODE_MEM=2572MB (+500M) for i386 arch?18:17
greghaynesjprovazn: derekh is the one who will probably want to discuss that18:18
greghaynessince its really an issue for our CI18:18
greghaynesBut IMO if it doesnt work as is then +1 we should fix that18:19
jprovaznhm, derek is not here... dprince ^ how much pain to CI would it cause ti increase NODE_MEM to 2.5G?18:25
*** loki184 has joined #tripleo18:27
*** yaling has quit IRC18:27
openstackgerritJay Dobies proposed a change to openstack/tuskar: Add set value functionality to plans manager  https://review.openstack.org/11398618:29
jprovaznlifeless, ^ what do you think about increasing memory for i386?18:31
*** jang3 has joined #tripleo18:35
*** lucasagomes_ has joined #tripleo18:35
*** akuznets_ has joined #tripleo18:35
*** julim_ has joined #tripleo18:36
*** spzala has quit IRC18:36
*** Hefeweiz1n has joined #tripleo18:36
*** sseago_ has joined #tripleo18:37
*** loki184 has quit IRC18:39
*** sseago has quit IRC18:41
*** julim has quit IRC18:41
*** akuznetsov has quit IRC18:41
*** lucas-dinner has quit IRC18:41
*** openstackgerrit has quit IRC18:41
*** jang has quit IRC18:41
*** Hefeweizen has quit IRC18:41
*** dshulyak has quit IRC18:41
*** dshulyak_ is now known as dshulyak18:41
*** penick has quit IRC18:45
*** penick has joined #tripleo18:46
*** jistr has joined #tripleo18:47
*** dshulyak1 has joined #tripleo18:49
*** openstackgerrit has joined #tripleo18:49
*** penick has quit IRC18:52
*** ByPasS has quit IRC18:53
*** ByPasS has joined #tripleo18:53
*** eguz has quit IRC18:56
*** rushiagr is now known as rushiagr_away18:56
openstackgerritPhil Neal proposed a change to openstack/tripleo-image-elements: Add elements for Ceilometer alarm services  https://review.openstack.org/11364718:58
dprincejprovazn: re NODE_MEM to 2.5G, So there was this idea that we might could over-allocate. So it might be do-able perhaps19:01
*** jang2 has quit IRC19:01
*** eghobo has joined #tripleo19:02
dprincejprovazn: as far as pain goes it may require rebuilding the TE worker nodes so not a show stopper or anything, we should be outfitted to rebuild them now without having to destroy the job queue19:02
*** penick has joined #tripleo19:03
dprincejprovazn: I think there was talk of changing the sizes anyways to switch back to testing on x86_64 so perhaps this is something to consider in that push19:03
*** spzala has joined #tripleo19:07
*** bmahalakshmi has quit IRC19:08
*** jang1 has quit IRC19:09
*** dshulyak has quit IRC19:14
*** dshulyak has joined #tripleo19:20
jprovazndprince, thanks for the info. What sizes did you plan to be able to switch back to testing on 64bit?19:21
jprovazns/did you plan/did you plan to use/19:22
dprincejprovazn: I think 3 gigs was proposed19:24
jprovazndprince, for 64bit only or both for 32bit and 64bit?19:26
dprincejprovazn: 64bit instead of 32bit, we wouldn't run both I don't think19:27
jprovazndprince, hm, NODE_MEM is by default set in tripleo-incubator/scripts/devtest_testenv.sh, is it set also somewhere else for CI?19:31
dprincejprovazn: I think we use the defaults for now19:33
*** dtantsur is now known as dtantsur|afk19:33
*** jang3 has quit IRC19:35
jprovazndprince, I'm not sure then if/how to take into account switching to 64bit in CI tests in the patch which would bump default memory for i386 to 2.5G? Do you mean setting default value to 3G instead?19:36
SpamapSI am getting massive instability using the defaults right now19:40
SpamapSmy cloud 'splodes19:40
*** marun has quit IRC19:41
*** dshulyak has quit IRC19:41
greghayneshttps://review.openstack.org/#/c/94727/ might be the 3gig proposal youre thinking of19:41
*** eghobo has quit IRC19:42
*** eghobo has joined #tripleo19:42
SpamapSman.. every time I start over on a new box for building devtest I forget.. gotta have at the very least squid. :-P19:43
*** eghobo has quit IRC19:45
openstackgerritJay Dobies proposed a change to openstack/tuskar: Fix for scaling default  https://review.openstack.org/11400219:45
jprovazngreghaynes, thanks - this looks like an old patch - suggested mem for 64bit was increased meantime to 4G19:46
greghayneshah19:46
greghayneswell, thats good19:46
jprovazngreghaynes, so the issue is basically with 386 now which defaults to 2G19:47
greghaynesyep19:47
openstackgerritJay Dobies proposed a change to openstack/tuskar: Fix for roles with dashes in the name  https://review.openstack.org/11400419:48
SpamapSmuch happier cloud w/ 4G nodes now19:49
greghaynesRAM, its what clouds crave!19:49
openstackgerritJay Dobies proposed a change to openstack/tuskar: WIP: Wires up REST API to underlying Tuskar logic  https://review.openstack.org/11398819:50
*** andreaf_ has joined #tripleo19:53
*** andreaf_ is now known as andreaf19:53
*** andreaf has quit IRC19:57
*** andreaf has joined #tripleo20:01
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: WIP Adds Satellite 6 Element  https://review.openstack.org/11400920:02
lifelessjprovazn: if we need to we need to20:02
*** lblanchard has quit IRC20:03
openstackgerritA change was merged to openstack/tripleo-heat-templates: Make removing nodes from scaled items possible.  https://review.openstack.org/11343120:05
openstackgerritJan Provaznik proposed a change to openstack/tripleo-incubator: Increase default RAM for 32bit to 2.5G  https://review.openstack.org/11401120:06
openstackgerritlifeless proposed a change to openstack/tripleo-heat-templates: Make turning debug on globally easy.  https://review.openstack.org/11277820:09
*** marun has joined #tripleo20:10
*** dshulyak has joined #tripleo20:12
lifelesslsmola: I think your thing needs way more thought20:15
*** jistr has quit IRC20:16
lifelesslsmola: ping! did that review, think we have a bunch of talking to do20:23
lifelessSpamapS: ^ you'll want to be involved, I'm sure20:23
SpamapSsomething is terribly wrong w/ neutron-l3-agent and neutron-metadata-agent20:26
SpamapSeven w/ enough RAM, they are not running on the controller20:26
SpamapSand I am corosync-stupid, so I don't know how to tell why20:26
SpamapSI believe the problem is that nothing has managed to os-svc-enable -n them20:27
SpamapSindeed, whatever moved them to corosync, removed their os-svc-enable lines20:27
*** julim_ has quit IRC20:28
*** jang has joined #tripleo20:28
lifelessoh20:28
lifelessI thought the metadata agent was active-active20:29
lifelessin fact I was surwe of it20:29
lifelessand we have dvr now20:29
lifelessjprovazn: morning!20:29
lifelessSpamapS: so AIUI pacemaker is meant to own startin and stopping them20:30
lifelessSpamapS: it passed CI20:31
SpamapSlifeless: the problem is, os-svc-daemon delegates enabled/disabled to os-svc-enable20:31
lifelessSpamapS: its got a dependency on https://review.openstack.org/#/c/111055/20:32
SpamapSlifeless: not sure how. neutron-l3-agent simply will not start, until you run 'os-svc-enable -n neutron-l3-agent' .. and nothing runs that20:32
lifelessSpamapS: so if you updated your image but not your template20:32
lifelesspacemaker and corosync wouldn't be running20:32
SpamapSI built fresh from "all-trunk" this morning20:32
jprovaznlifeless, good morning to NZ (I was just about going to bed ;)20:32
SpamapSbrand new blank box20:32
lifelessjprovazn: hi!20:33
lifelessjprovazn: I wan20:33
lifelessbah20:33
lifelessjprovazn: I wanted to ask, couple of things - a) is the nova-scheduler on your pacemaker hit-list20:33
jprovaznlifeless, no20:33
lifelessjprovazn: and secondly, it looks like the patches that landed aren't upgrade-compatible20:33
jprovaznlifeless, re: nova-scheduler - what is the reason for a/p?20:33
lifelessjprovazn: I think we need to make them handle pacemaker's data missing, because thats how folk with a running cloud are going to have to upgrade (see the revert HOT thread)20:33
lifelessjprovazn: the scheduler has massive races, with multiple schedulers you end up dispatching based on the same in-memory view for multiple seconds20:34
lifelessjprovazn: can trivially overwhelm machines20:34
jprovaznlifeless, right, but there is a host_Size var which mitigates this issue20:34
SpamapS9ae3c521 removed os-svc-enable ... I really don't know how this can be working at all. :-/20:34
jprovaznlifeless, in combination with retry filter, a/a should be fine20:35
lifelessjprovazn: I'm told by nova folk its better to run 1 with failover, at least at the moment20:35
*** dshulyak has quit IRC20:35
SpamapSlifeless: pacemaker is running. It reports success from the 'start' command, but the monitoring is showing that the daemons are not running. Because they aren't.. because /etc/init/os-svc-enable.conf is stopping the job20:36
jprovaznlifeless, hm, well, let me check here, but anyway putting it under pacemaker is not a big deal20:36
lifelessjprovazn: so, how much work is it to make 9ae3c52104a2098b2566fe77e5420ec93b5875f7 work when the template doesn't have the metadata for pacemaker20:36
lifelessjprovazn: e.g. by degrading to a/a20:36
* jprovazn reads back for the second request20:37
jprovaznah20:37
jprovaznthis is a minor thing20:37
lifelessjprovazn: oh yeah, b) was that the l3 agent changes aren't backwards comaptible20:37
lifelessjprovazn: in a few ways I guess20:37
lifelessjprovazn: firstly, if you upgrade a cloud that already has three agents20:37
lifelessyou'll end up with one20:37
lifelessand 2/3rds of the networks will stop working20:37
lifelesssecondly, if the template doesn't have pacemaker details yet (and for upgrades, it looks like we're going to have to deploy a new set of images with the old template, to get newer heat out there)20:38
lifelessit won't work20:38
jprovaznlifeless, there can be an additonal metadata which defines for services it it should run in a/a (--clone) or not20:38
SpamapSso what we need is on upstart systems to enable the job but set it to manual20:38
SpamapSdid it pass Ubuntu CI?20:39
lifelessSpamapS: no - thats nonvoting at the moment20:39
jprovaznSpamapS, all patches passed tests, otherwise it wouldn't be merged20:39
lifelessjprovazn: they didn't20:39
lifelesscheck-tripleo-ironic-overcloud-precise-haFAILURE in 32m 14s (non-voting)20:39
SpamapScheck-tripleo-ironic-overcloud-precise-ha FAILURE in 2h 12m 59s (non-voting)20:39
lifelessso I think we need to revert this20:39
jprovaznoh, so I meant different patch20:40
lifelesssorry SpamapS pasted the right one20:40
lifelessI think we need to revert it and fix the ubuntu issue, and the backwards compat stuff20:40
SpamapSsame symptom, demo instance is not pingable20:40
SpamapSok you're going to revert? I have a patch for the os-svc-enable20:41
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Revert "Use pacemaker to run neutron l3 agent and metadata agent"  https://review.openstack.org/11401920:42
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Revert "Use pacemaker to run neutron l3 agent and metadata agent"  https://review.openstack.org/11401920:42
lifelesshttps://review.openstack.org/#/c/114019/20:42
lifelessjprovazn: we might be able to bring it in as an option rather than changing the existing behaviour20:43
lifelessSpamapS: you tell me - I put the reasons in the revert commit message20:44
SpamapSlifeless: I don't know all the broken things, but "it doesn't work on Ubuntu" is the one affecting me.20:44
lifelessSpamapS: so ci clouds are ha a/a today20:45
lifelessat least the hpN ones20:45
lifelessthis will break them20:45
lifelessand it will break every helion commercial install out there20:45
lifeless(not that helion is privileged, just as an example of HA deploys)20:46
jprovaznlifeless, ok, will look at it tomorrow20:47
lifelessjprovazn: sorry :/ wish I'd caught that in review - but I doubt I'd have caught the ubuntu thing either20:48
SpamapSlifeless: http://paste.ubuntu.com/8039338/ there's the workaround for Ubuntu20:48
jprovaznnp, reverting it seems to be the only option at this point20:48
lifelessk - care to add your +2 on that20:49
lifelessah you have20:49
lifelessI'll +A when/if it passes CI20:49
SpamapSlifeless: the ultimate fix is, I think, to make os-svc-daemon use the manual override instead of the os-svc-enable.conf upstart job20:49
*** jprovazn has quit IRC20:50
tchaypoto answer an earlier question from jprovazn (unless it was dprince) - no, CI doesn't use devtest_testenv.sh, there's a similar script in the toci repo that gets used instead20:50
lifelessbbiab time for morning-with-Cynthia20:51
tchayposo bumping default ram isn't an issue for ci directly afaict20:51
*** stevebaker has quit IRC20:52
*** stevebaker has joined #tripleo20:52
tchaypobut I am guessing CI would like to be testing something like the default setup a user would run20:52
tchaypothe bigger issue in my mind is that it means the minimal seed+undercloud+3*overcloud setup now needs 2.5Gb more ram20:54
*** marun has quit IRC20:56
SpamapSmoar20:56
*** spzala has quit IRC20:57
dprincetchaypo: exactly, last time I checked we called devtest_testenv.sh directly in TOCI21:00
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126321:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604021:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589821:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784821:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]21:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135623321:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135628321:00
uvirtbotLaunchpad bug 1356233 in tripleo "Error in deploying a new node" [Undecided,New]21:00
uvirtbotLaunchpad bug 1356283 in tripleo "check-tripleo-novabm-overcloud-f20-nonha gate job is setting USE_IRONIC=0" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633921:00
*** untriaged-bot has quit IRC21:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,New]21:00
*** dprince has quit IRC21:00
*** jdob has quit IRC21:01
*** eghobo has joined #tripleo21:02
tchaypodprince: by "directly" i think you mean "by hand, once, at the time we're creating the environment, which then gets re-used for each subsequent run"?21:06
*** jang has quit IRC21:10
*** ccrouch has quit IRC21:12
*** otoolee- has quit IRC21:12
*** rushiagr_away has quit IRC21:12
*** ByPasS has quit IRC21:13
*** ccrouch1 has joined #tripleo21:13
*** ByPasS has joined #tripleo21:13
*** rushiagr_away has joined #tripleo21:15
*** jtomasek_ has quit IRC21:15
*** otoolee- has joined #tripleo21:17
*** marun has joined #tripleo21:17
*** uberj has joined #tripleo21:29
uberjhas anyone else seen dib fail to create a fedora image (19 or 20) due to systemd failing to be upgraded? https://pastebin.mozilla.org/5967093 <-- what I see. the 'install-packages -u' in elements/base/install.d/99-up-to-date is failing for some reason21:32
uberjoh sad, my pastebin got truncated :(21:32
lifelessback21:33
lifelesstchaypo: we create it from an orc script21:34
lifelessuberj: I don't recall seeing that in CI21:34
uberjhttp://paste.openstack.org/show/94639/21:35
uberjthere, thats an abridged version21:35
tchaypolifeless: unpacking what I think you mean - we create the test environment from an os-refresh-config script, we don't call _testenv.sh directly?21:36
lifeless23/500 passed on hp121:36
lifelesswe have a ways to go21:36
lifelesstchaypo: I suggest you use the source :)21:36
lifelesstchaypo: in tripleo-ci21:36
tchaypoI did, and it doesn't mention _testenv.sh21:37
tchaypohttps://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_devtest.sh right?21:37
*** jang has joined #tripleo21:37
lifelessno21:38
greghaynesuberj: seems like youre just running with an old base image, and apparently the systemd upgrade from package does not work within chroot21:38
lifelesstchaypo: try applying git grep testenv.sh21:39
clarkbgreghaynes: would you expect it to work? systemd probably tries to fork itself and reinit21:40
clarkbwithout actually reiniting21:40
greghaynesI definitely would not expect it to work21:40
greghayneseven on a normal system :p21:40
lifelessI'd expect it to work21:40
uberjgreghaynes: I'll try clearing the cache and running it again21:40
lifelessits a fairly common use case after all21:40
tchaypoah, I jumped straight into the o-r-c directory and ignored the bin directory in the element.21:41
tchaypoSince that's an element, I assume it must be run by d-i-b during an image build, but that seems really strange to me. Is that what happens?21:43
tchaypoor does this element get used differently?21:43
greghaynesuberj: id just drop into the chroot right before that phase and play around with upgrading that package by hand21:46
greghaynessee what explides21:46
uberjgreghaynes: how does one set a break point in dib?21:46
lifelesstchaypo: os-refresh-config scripts do not run during image builds21:47
greghaynesthere might be a better way, but I use "bash"21:47
lifelesstchaypo: I think we've had that discussion before21:47
lifelesstchaypo: so I suggest you look at the implementation of the os-refresh-config element in tie21:47
tchaypoyep. it's the layering that confuses me.21:47
tchaypothe dib element drops the o-r-c bits onto the image and then they run after the image boots21:48
uberjgreghaynes: oh I see the problem, you have 'set -e' at the top of that update element ;)21:48
* uberj tries the bash trick21:48
lifelessuberj: 'problem' ? ;)21:48
uberjuhg, also, intermittently https://raw.githubusercontent.com/pypa/pip/master/contrib/get-pip.py fails to download21:49
lifelessthat is the bane of our existence21:49
tchaypoyeah, github is the ugh21:49
uberjoh, so it happens to you too?21:49
uberjI don't feel special anymore21:49
bkeroAhh installing software from python scripts fetched from master branches of live repositories.21:53
bkeroIt's like someone thought it was a good idea.21:53
tchaypobkero: welcome to openstack!21:53
tchaypo:)21:53
tchaypoWhat do we call the node we run devtest.sh on?21:54
bkeroa trashed machine21:54
greghaynestchaypo: jump host sometimes21:54
tchaypoexcuse me21:54
greghaynesalso have heard bastion used21:54
greghaynesboth are wonderfully overloaded terms :)21:55
tchaypoI have no coffee, so there's no risk of getting it on a keyboard, so I can't properly appreciate bkero's humor, brb21:55
tchayposo I was thinking in a more generic sense21:55
tchaypoI think "bastion" or "jump host" applies to the specific machines we use for the CD cloud21:55
uberjgreghaynes: https://bugzilla.redhat.com/show_bug.cgi?id=64865421:55
uvirtbotuberj: Error: Could not parse XML returned by bugzilla.redhat.com: HTTP Error 404: Not Found21:55
tchaypobut if a user downloads it and starts running on their desktop, i don't think we'd call it a bastion21:56
greghaynesuberj: nice!21:56
greghaynesweve run into that type of issue before21:56
uberjI wonder if that is my issue21:56
greghaynesuberj: basically, dont run on tmpfs21:56
uberjokay21:56
uberjuh, theres a flag for that21:56
tchaypouberj: don't be silly21:56
tchaypouberj: that issue is from 2010, how could it possible affect you now?21:57
uberjI don't know21:57
uberjlet me see if removing the tmpfs does things21:57
uberjwell, I *am* running centos6.521:57
uberjas the host21:57
tchaypo(did my sarcasm come across, or do i need more coffee?)21:58
*** penick has quit IRC21:58
uberjtchaypo: It didn't, but now I'm laughing21:59
slaglelifeless: were you going to mail the list about the spec approval process?21:59
lifelessyes22:00
* uberj adjusts his cynicism detector22:00
slaglelifeless: k, thx. i couldn't recall who was doing that22:00
* slagle should have looked in meeting minutes22:00
lifeless:)22:00
*** weshay has quit IRC22:01
*** jang has quit IRC22:02
uberjwhat base os are you running dib CI on?22:02
*** vinsh has quit IRC22:03
lifelessuberj: xen or kvm running a VM which is ubuntu or fedora which then runs dib22:03
openstackgerritA change was merged to openstack/tripleo-incubator: Switch to only one top-level heading per doc  https://review.openstack.org/10443422:03
uberjokay22:03
uberjgreghaynes: lifeless tchaypo --no-tmpfs fixed my issue22:09
uberjnow lets see if the image is functional22:10
*** sseago_ has quit IRC22:11
tchaypoOMG22:12
tchaypo104434 is from the start of July. I'd forgotten that existed.22:12
*** morazi has quit IRC22:12
tchaypothat should help our stats a bit22:12
*** rwsu has quit IRC22:21
*** penick has joined #tripleo22:23
*** rwsu has joined #tripleo22:35
*** penick has quit IRC22:37
tchaypowhich is handy because I'm writing the email about the stats at the moment..22:37
*** penick has joined #tripleo22:37
*** noslzzp has quit IRC22:39
*** penick has quit IRC22:39
*** dguerri`afk is now known as dguerri22:40
*** jang has joined #tripleo22:47
*** andreaf has quit IRC22:53
bnemectchaypo: Yeah, there were actually a bunch of fairly simple reviews sitting in our 15 oldest list.22:56
*** chuckC has quit IRC22:58
*** chuckC has joined #tripleo22:58
*** andreaf has joined #tripleo22:59
*** penick has joined #tripleo22:59
*** chuckC has quit IRC23:01
*** lucasagomes_ has quit IRC23:03
*** jang has quit IRC23:03
*** andreaf_ has joined #tripleo23:04
*** andreaf has quit IRC23:07
bnemecSo, I don't think we ever actually set CI to i386: https://review.openstack.org/#/c/106342/23:09
bnemecBut it's ready to merge if we want to.23:09
*** dguerri is now known as dguerri`afk23:11
tchaypojogo: are you around? cinerama is having some trouble bending nova to her will, and she's asking questions that go way over my head23:11
cineramatchaypo, jogo: i may have got it, let me try one thing23:12
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-cloud-config: Updated from global requirements  https://review.openstack.org/10658923:23
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-collect-config: Updated from global requirements  https://review.openstack.org/10659023:23
openstackgerritOpenStack Proposal Bot proposed a change to openstack/tuskar-ui: Updated from global requirements  https://review.openstack.org/11407523:25
lifelesscinerama: vat is the question ?23:25
*** pradeep has joined #tripleo23:28
*** jang has joined #tripleo23:31
openstackgerritA change was merged to openstack/diskimage-builder: Don't try to install if packages is empty  https://review.openstack.org/10700823:36
*** chuckC has joined #tripleo23:37
lifelesstchaypo: so shall we progress hp2 ?23:40
tchaypoyes, that's what i was about to ask23:42
tchaypothere's a seed running and we need to check that *consults notes*23:42
tchaypobm-networks and nodes were correct23:42
openstackgerritA change was merged to openstack/tripleo-image-elements: Revert "Use pacemaker to run neutron l3 agent and metadata agent"  https://review.openstack.org/11401923:45
tchaypoI'm not sure if I'm checking the right things23:55
tchaypobut br-ctlplane has the IP the spreadsheet says it should have23:55
tchaypolikewise int_public23:55
cineramalifeless: oh hi! so i'm trying to do a nova rebuild with "--preserve-ephemeral" and it's saying "The current driver does not support preserving ephemeral partitions"23:56
cineramalifeless: was just checking i hadn't made a more obvious mistake23:56
lifelesscinerama: what driver are you using ?23:56
lifelesscinerama: I'mm betting you're trying to rebuild a libvirt driver instance, not an ironic one23:56
cineramalifeless: i have USE_IRONIC set to 123:57
lifelesscinerama: lets start with first principles23:57
cineramalifeless: that's probably a really good idea23:58
lifelesscinerama: whats your OS_AUTH_URL23:58
lifelessactually, I need to ask a meta question23:58
lifelesswhere do you see this issue ?23:58
cineramalifeless: when i do 'nova show' after i issue the rebuild23:58
lifelessok23:58
lifelessso your OS_AUTH_URL will let us identify the keystnoe you're talking to23:59
lifelessand from that the nova you're talking to23:59
*** pradeep has quit IRC23:59
lifelessoh23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!