Tuesday, 2016-11-01

openstackgerritMarkos Chandras proposed openstack/diskimage-builder: lib: common-functions: Fix tmpfs umounting  https://review.openstack.org/39200200:09
*** maticue has quit IRC00:15
*** artom_ has joined #tripleo00:17
*** artom has quit IRC00:20
*** noslzzp has quit IRC00:20
*** trozet has quit IRC00:20
*** trozet has joined #tripleo00:21
*** noslzzp has joined #tripleo00:21
*** gfidente|afk has quit IRC00:21
*** jrist has quit IRC00:21
*** jrist has joined #tripleo00:21
*** jrist has joined #tripleo00:21
*** gfidente|afk has joined #tripleo00:22
*** egafford has quit IRC00:24
*** egafford has joined #tripleo00:33
*** egafford has quit IRC00:38
*** akshai has quit IRC00:51
*** bana_k has quit IRC00:52
*** bana_k has joined #tripleo00:54
*** bana_k has quit IRC01:01
dhill_I don't understand why it fails : https://review.openstack.org/388920 ... any hints?01:08
*** chlong has quit IRC01:09
*** rlandy has quit IRC01:17
openstackgerritMerged openstack/diskimage-builder: Don't set tracing in environment files  https://review.openstack.org/38897201:43
*** bfournie1 has left #tripleo01:44
*** dmacpher has joined #tripleo01:45
*** dougbtv has joined #tripleo01:47
*** bfournie has joined #tripleo01:49
openstackgerritMerged openstack/diskimage-builder: Don't log datestamp by default in functional tests  https://review.openstack.org/39012601:55
*** b00tcat has quit IRC02:01
*** limao has joined #tripleo02:17
*** ramishra has joined #tripleo02:40
*** rdopiera has quit IRC02:53
*** bkopilov has quit IRC02:58
*** bkopilov_ has quit IRC02:58
*** rdopiera has joined #tripleo03:02
*** rdopiera has quit IRC03:07
*** bana_k has joined #tripleo03:08
*** rdopiera has joined #tripleo03:09
*** ramishra has quit IRC03:21
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544903:30
*** bana_k has quit IRC03:43
*** links has joined #tripleo03:52
*** bana_k has joined #tripleo04:01
*** ramishra has joined #tripleo04:06
*** bana_k has quit IRC04:14
*** dmacpher has quit IRC04:15
*** jeckersb is now known as jeckersb_gone04:18
*** bkopilov has joined #tripleo04:19
*** bkopilov_ has joined #tripleo04:19
*** tzumainn has quit IRC04:25
*** dmacpher has joined #tripleo04:28
*** akshai has joined #tripleo04:35
*** akshai has quit IRC04:36
*** dsariel has quit IRC04:45
*** bana_k has joined #tripleo05:06
*** limao has quit IRC05:14
openstackgerritNoam Angel proposed openstack/diskimage-builder: add support for SUSE in dhcp-all-interfaces  https://review.openstack.org/39177205:15
*** limao has joined #tripleo05:15
*** limao has quit IRC05:15
*** limao has joined #tripleo05:16
*** limao has quit IRC05:20
*** bana_k has quit IRC05:27
*** dsariel has joined #tripleo05:27
*** bana_k has joined #tripleo05:54
*** ebalduf has quit IRC05:55
*** ramishra_ has joined #tripleo06:00
*** flepied1 has quit IRC06:01
*** ramishra has quit IRC06:02
*** dsariel has quit IRC06:16
*** limao has joined #tripleo06:16
*** limao has quit IRC06:21
*** rcernin has joined #tripleo06:29
*** bana_k has quit IRC06:30
*** florianf has joined #tripleo06:30
*** florianf has quit IRC06:31
*** mcornea has joined #tripleo06:31
*** florianf has joined #tripleo06:31
*** limao has joined #tripleo06:44
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates: Add replacepkgs to the manual ovs upgrade workaround and fix a typo  https://review.openstack.org/39082606:58
*** bvandenh has quit IRC07:03
*** florianf has quit IRC07:03
*** akuznetsov has joined #tripleo07:07
*** paramite has joined #tripleo07:08
openstackgerritMerged openstack/diskimage-builder: Changed the home-page of diskimage-builder in setup.cfg  https://review.openstack.org/39149607:11
*** florianf has joined #tripleo07:12
*** jprovazn has joined #tripleo07:16
*** dmacpher has quit IRC07:16
paramiteHi guys, can I do reverify on https://review.openstack.org/#/c/390802/ myself or somebody from core has to?07:16
*** jtomasek_ has joined #tripleo07:16
paramitemeh, sorry for the noise, I saw +1 on place where it was not :)07:19
paramiteBut anyway, can we merge the patch? :)07:19
*** hjensas has joined #tripleo07:19
*** hjensas has quit IRC07:19
*** hjensas has joined #tripleo07:19
*** tesseract has joined #tripleo07:21
*** tesseract is now known as Guest1438107:21
*** bvandenh has joined #tripleo07:23
*** dmacpher has joined #tripleo07:31
*** akuznetsov has quit IRC07:31
*** akuznetsov has joined #tripleo07:31
*** jtomasek_ has quit IRC07:36
*** yamahata has quit IRC07:40
*** mhenkel has joined #tripleo07:43
*** ohamada has joined #tripleo07:59
*** ohamada has quit IRC08:00
d0ugalparamite: ideally we need the ovb-ha job to pass too08:04
*** chandankumar has joined #tripleo08:12
*** ebarrera has joined #tripleo08:13
*** hewbrocca_afk is now known as hewbrocca08:17
*** flepied has joined #tripleo08:17
*** akuznetsov has quit IRC08:18
*** zoli_gone-proxy is now known as zoliXXL08:19
*** florianf has quit IRC08:24
*** ccamacho has quit IRC08:24
*** ohamada has joined #tripleo08:28
*** zoliXXL is now known as zoli_gone-proxy08:30
*** aufi has joined #tripleo08:32
*** jtomasek_ has joined #tripleo08:34
*** zoli_gone-proxy is now known as zoliXXL08:37
*** florianf has joined #tripleo08:38
*** lucas-afk is now known as lucasagomes08:40
*** dixiaoli has joined #tripleo08:43
*** dixiaoli has quit IRC08:46
*** social has joined #tripleo08:54
*** athomas has joined #tripleo08:59
*** ohamada has quit IRC08:59
*** liverpooler has joined #tripleo09:00
*** flepied has quit IRC09:09
*** jaosorior has joined #tripleo09:15
*** jpich has joined #tripleo09:18
jaosoriorsshnaidm: ping09:23
sshnaidmjaosorior, hi09:23
jaosoriorsshnaidm: hey dude, I saw that the job merged :D09:23
sshnaidmjaosorior, yeah, finally :)09:23
sshnaidmjaosorior, did you run experimental on your patch?09:24
jaosoriorsshnaidm: well, I could but it won't do anything yet09:24
*** jbadiapa has quit IRC09:24
sshnaidmjaosorior, just for testing09:24
jaosoriorsshnaidm: any idea how I can provision just one node to be09:25
jaosoriorsshnaidm: ok done09:25
jaosoriorsshnaidm: so yeah, one node so I can install FreeIPA there?09:25
jaosoriorsshnaidm:  I kinda confused the workflow with the multinode, where the node are provisioned already09:25
jaosoriorso now I guess I have to do it via ironic also09:25
*** dtantsur|afk is now known as dtantsur09:26
sshnaidmjaosorior, yeah, in ovb it's with ironic09:26
jaosoriorsshnaidm: fuck, I don't see the new job in the zuul queue09:26
*** akrivoka has joined #tripleo09:27
*** panda|Zz is now known as panda|pto|PH09:28
*** milan has joined #tripleo09:28
sshnaidmjaosorior, telnet://66.187.229.202:1988509:29
sshnaidmjaosorior, everything is good :)09:29
sshnaidmjaosorior, it's experimental-tripleo pipeline09:29
jaosoriorsshnaidm: sshnaidm couldn't access that address :/09:30
jaosoriornor with telnet, nor with the browser, not sure what I'm supposed to do with it09:30
*** abregman has joined #tripleo09:30
sshnaidmjaosorior, running in console "telnet 66.187.229.202 19885"09:31
sshnaidmjaosorior, it's weird, I can connect and see job running09:31
sshnaidmjaosorior, anyway, it works, now you can start configuring it09:32
jaosoriorsshnaidm: true that, thanks man!09:32
sshnaidmjaosorior, regarding ovb I'm not sure what is best here.. Don't you have heat template or separate role for this server?09:33
jaosoriorsshnaidm: nope09:33
jaosoriorI can get a heat template though09:33
jaosoriorbut no role09:33
jaosoriorsshnaidm: it's a bit awkward, cause even the undercloud will be registered as an IPA client (well, FreeIPA client... now ironic-python-agent uses IPA too x_x)09:34
jaosoriorthe IPA acronym is overused09:34
*** derekh has joined #tripleo09:35
openstackgerritLucas Alvares Gomes proposed openstack/tripleo-quickstart: Force the use of python/pip version 2  https://review.openstack.org/36801309:35
*** zoliXXL is now known as zoli|wfh09:36
sshnaidmjaosorior, found now: https://adam.younglogic.com/2016/09/deploying-server-on-ironic-node-baseline/09:36
sshnaidmjaosorior, seems like ayoung blog09:36
jaosoriorsshnaidm: was thinking about using this https://github.com/resurrexit/freeipa-heat-template/blob/master/freeipa_server.yaml09:36
sshnaidmjaosorior, it seems nice09:38
sshnaidmjaosorior, either deploying server with ironic and then install there manually09:38
jaosoriorsshnaidm: yeah, an intern in our team came up with it, I just have to tweak it and remove some hardcoded stuff09:38
jaosoriorsshnaidm: any idea what the DNS is for the hypervisors in OVB?09:38
sshnaidmjaosorior, 8.8.8.8 and undercloud(?)09:39
sshnaidmjaosorior, let me check..09:39
jaosoriorI need the one for the undercloud09:40
sshnaidmjaosorior, 8.8.8.8 , 8.8.4.4 and some 10.1.8.10 - but for overcloud images only09:42
jaosoriorany idea where the 10.1.8.10 comes from?09:43
sshnaidmjaosorior, rh1 cloud internal I suppose09:43
sshnaidmjaosorior, it's better to use google dns I think09:43
jaosoriorsshnaidm: right, so I want to add a DNS entry, so the overcloud nodes know the address of the FreeIPA instance09:44
jaosoriorsshnaidm: cause all they get is the hostname09:44
sshnaidmjaosorior, /etc/hosts ?09:45
jaosoriorsshnaidm: yeah... I guess that's the only way at the moment09:46
*** oshvartz has joined #tripleo09:47
*** jaosorior has quit IRC09:48
*** jaosorior has joined #tripleo09:48
*** limao has quit IRC09:50
*** limao has joined #tripleo09:50
*** limao has quit IRC09:55
panda|pto|PHsshnaidm: I found the root cause, but I don't know to who I can talk to to fix it, and I don't know how to write a selinux policy (and where)09:55
sshnaidmpanda|pto|PH, is it about running redis with systemd?09:56
sshnaidmpanda|pto|PH, or just running redis..?09:56
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173809:57
jaosoriorpanda|pto|PH: gotta talk to rhallisey, but he's not online yet09:57
jaosoriorpanda|pto|PH: also, if you know the root cause and can reproduce it, you gotta run in permissive mode, then grep the audit log for the service you know that's causing issues, and pass those audit logs to rhallisey09:58
jaosoriorhe'll use audit2allow to create a policy09:58
openstackgerritMerged openstack/instack-undercloud: Enable telemetry services by default on undercloud  https://review.openstack.org/38670209:58
jaosoriorsshnaidm: how did you get the IP that I need to use to check the progress of that specific job?09:58
sshnaidmjaosorior, from http://status.openstack.org/zuul/09:59
sshnaidmjaosorior, find you job running there and copy link09:59
sshnaidmjaosorior, it's in experimental-tripleo pipeline10:00
jaosoriorsshnaidm: thanks dude!10:00
*** owalsh has joined #tripleo10:00
*** mgould|afk is now known as mgould10:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for Cinder API  https://review.openstack.org/39209010:07
*** dsariel has joined #tripleo10:08
panda|pto|PHjaosorior: already done the permissive part, redis is not allowed to create the socket when in enforcing mode, but in permissive is ok, I already have the denied in audit log10:08
panda|pto|PHjaosorior: https://bugs.launchpad.net/tripleo/+bug/163796110:09
openstackLaunchpad bug 1637961 in tripleo "periodic HA master job pingtest times out" [High,Confirmed] - Assigned to Gabriele Cerami (gcerami)10:09
jaosoriorpanda|pto|PH: where's the denied audit log?10:11
*** null_ref has quit IRC10:11
panda|pto|PHjaosorior: http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-ha/0ae3179/logs/overcloud-controller-0/var/log/audit/audit.txt.gz10:11
panda|pto|PHjaosorior: search for "denied"10:11
*** dsariel_ has joined #tripleo10:12
jaosoriorpanda|pto|PH: aah great, didn't see that in the laucnhpad bug10:12
jaosoriorpanda|pto|PH: alright, so now it's a matter of getting rhallisey to take a look at it10:12
jaosoriorhe'll add that to the openstack-selinux-rules package10:12
panda|pto|PHah, that's what I weas looking for :) thanks. I have a name, and a package name :)10:13
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable internal TLS for Cinder API  https://review.openstack.org/39209510:15
*** dsariel has quit IRC10:15
openstackgerritNoam Angel proposed openstack/diskimage-builder: add support for SUSE in dhcp-all-interfaces  https://review.openstack.org/39177210:16
*** shardy has joined #tripleo10:18
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable TLS in the internal network for Nova API  https://review.openstack.org/38849510:18
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable internal TLS for Nova API  https://review.openstack.org/38850310:19
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Enable internal TLS for MySQL  https://review.openstack.org/37847110:19
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable TLS in the internal networkf or Mysql  https://review.openstack.org/37853710:20
*** akrivoka has quit IRC10:22
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable internal TLS for Nova API  https://review.openstack.org/38850310:22
*** thrash|g0ne is now known as thrash10:23
*** anton has quit IRC10:24
jistrjaosorior: hi, when you have some time, can you review these please? https://review.openstack.org/#/q/topic:bug/163560610:26
*** anton has joined #tripleo10:27
*** chandankumar has quit IRC10:28
*** chandankumar has joined #tripleo10:28
*** oshvartz has quit IRC10:30
jaosoriorjistr: sure thing10:30
openstackgerritSteven Hardy proposed openstack-infra/tripleo-ci: Use parameter_merge_strategies for multinode env  https://review.openstack.org/38461810:32
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173810:32
*** akrivoka has joined #tripleo10:36
*** rodrigods has quit IRC10:37
*** rodrigods has joined #tripleo10:37
*** anton has quit IRC10:38
*** anton has joined #tripleo10:38
*** akuznetsov has joined #tripleo10:41
*** chandankumar has quit IRC10:43
*** chandankumar has joined #tripleo10:43
jaosoriorshardy: hey dude, I remember it's possible to deploy a setup with only one controller, but that uses the pacemaker environment too, do you remember how to do it?10:45
*** oshvartz has joined #tripleo10:45
shardyjaosorior: you just include this environment I think:10:47
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/environments/puppet-pacemaker.yaml10:47
*** akuznetsov has quit IRC10:47
jaosoriorshardy: so basically I do the same as I would usually do to deploy an HA setup but don't specify the controller count?10:48
shardyjaosorior: yup, I believe that's right10:51
jaosoriorshardy: thanks10:51
*** dprince has joined #tripleo10:56
*** chlong has joined #tripleo10:56
*** oshvartz has quit IRC10:57
*** jkilpatr has quit IRC10:57
*** zoli|wfh is now known as zoli|lunch11:02
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable internal TLS for Cinder API  https://review.openstack.org/39209511:06
*** liverpooler has quit IRC11:07
*** liverpooler has joined #tripleo11:07
*** bkopilov has quit IRC11:07
*** bkopilov_ has quit IRC11:07
*** liverpooler has quit IRC11:10
*** oshvartz has joined #tripleo11:11
jaosoriorif someone has time, could you review these commits? https://review.openstack.org/#/q/topic:barbican-integration they'11:15
jaosoriorthey're adding barbican to the overcloud11:15
adarazslucasagomes: hi Lucas. re: https://review.openstack.org/#/c/368013/6 -- why do we need the options to specify python and pip if there's only one valid value for them?11:15
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates: Change nova ram_allocation_ratio to match puppet-nova  https://review.openstack.org/39210811:16
lucasagomesadarazs, well, it could be "python" for distros with python 2.7 as the default interpreter, and "python2.7" for distros with python3 as the default interpreter11:19
lucasagomesadarazs, my first patch I was trying to lookup for the right version in code but it was asked to add the options11:20
adarazslucasagomes: oh. okay. followup question: if you create a virtenv with python2 won't the pip also be python2 based? or it might happen that there's no pip for python2 installed?11:21
adarazslucasagomes: re: str(sys.version_info[0]) + str(sys.version_info[1]) == "27" -- it seems if you use sys.version_info >= (2,7) instead that works fine. not sure what kind of python black magic is that but it seems to give back the proper true/false for me with various values :)11:22
openstackgerritMerged openstack/puppet-tripleo: NFS mounting for Glance file backend  https://review.openstack.org/38928411:23
lucasagomesadarazs, I think the pip should work depending on which version of python interpreter it's created, that's true11:24
lucasagomesadarazs, I can give it a go locally11:24
lucasagomesadarazs, oh, I can try to the version_info as well11:25
*** liverpooler has joined #tripleo11:25
adarazsI read it on stackoverflow so it must be true :)11:25
lucasagomesgoing to mark that patch as WIP and give it a go soon-ish11:25
lucasagomesadarazs, heh right :-D11:26
lucasagomesadarazs, mind adding those comments there ?11:26
lucasagomesthere == in the patch11:26
adarazssure.11:26
lucasagomesty!11:26
*** jeckersb_gone is now known as jeckersb11:27
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173811:28
adarazsdone.11:29
*** ohamada has joined #tripleo11:32
*** jkilpatr has joined #tripleo11:34
*** pradk has joined #tripleo11:35
*** dsariel_ has quit IRC11:36
jtomasekshardy: is it ok to consider DefaultPasswords, ServiceNetMap, EndpointMap parameters internal and explicitly exclude them in GUI?11:37
*** fultonj has joined #tripleo11:38
shardyjtomasek: DefaultPasswords yes, but not ServiceNetMap, as that's the interface operators use to assign a service to a specific network11:39
shardyjtomasek: in network/service_net_map.j2.yaml we expose two parameters, ServiceNetMapDefaults (which mostly I would not expect operators to modify), and ServiceNetMap (which overrides ServiceNetMapDefaults)11:40
jtomasekshardy: ok, and EndpointMap?11:41
shardyjtomasek: I'd say yes for now ignore EndpointMap as it's generated by a script from endpoint_data.yaml11:41
jaosoriordprince: can you check these commits? They're for getting barbican to the overcloud https://review.openstack.org/#/q/topic:barbican-integration11:41
shardyso we'd need a new workflow to allow operators to modify that11:41
dprincejaosorior: ack11:41
jtomasekshardy: ok, thanks11:41
shardyjtomasek: I'm not sure the best way to handle ServiceNetMap - we pass that parameter around in a few places, but the UI probably wants to treat those as internal, but expose a ServiceNetMap interface _somewhere_ populated with the ServiceNetMapDefaults (or perhaps just a manual override that's pased via ServiceNetMap for now)11:43
jaosoriordprince: thanks11:47
*** tobias_fiberdata has joined #tripleo11:48
jaosoriorshardy, jistr: if you have time :D https://review.openstack.org/#/q/topic:barbican-integration11:49
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: swift/proxy: remove swift::proxy::ceilometer::rabbit_host  https://review.openstack.org/39186411:49
EmilienMhello11:50
jaosoriorit's not production ready but it's a start. We still need to make a DogTag profile to have a real backend for barbican.11:50
jaosoriorEmilienM: hey dude, how's it going?11:50
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: nova/libvirt: add missing ports for live-migration  https://review.openstack.org/39187611:50
EmilienMjaosorior: excellent and you11:50
jaosoriorEmilienM: a bit flu-ish. but drinking vitamins and coffee helps11:51
EmilienMlooks like CI ovb/ha is flappy11:52
*** anton is now known as aarapov11:52
EmilienMif not broken11:53
jtomasekshardy: good point, I think this may be included in network configuration work. I'll keep displaying that parameter at each service parameters list for now, untill we have a better way to let user configure ServiceNetMap11:53
*** aarapov is now known as anton11:53
EmilienMConnectFailure: resources.server1: Unable to establish connection to http://10.0.0.14:8774/v2.1/os-volumes_boot: ('Connection aborted.', BadStatusLine("''",))11:53
jaosoriordafuq11:53
EmilienMsomeone already saw that ^ ?11:53
jaosoriorEmilienM: is that going on in the pingtest?11:54
*** dsariel has joined #tripleo11:54
EmilienMjaosorior: yes11:55
EmilienMI'm investigating in logstash11:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Parameters listing updates  https://review.openstack.org/39185511:56
jaosoriorEmilienM: can you pass a commit that shows that issue? gotta check the cinder logs11:56
thrashdprince: taking a look at the composable undercloud... Looks like the undercloud deploy needs to either process the jinja templates itself, or use the mistral action to do so11:56
EmilienMjaosorior: well, take a recent ha job on tripleo.org/cistatus.html11:56
jaosoriorEmilienM: s/cinder/nova11:56
dprincethrash: hey, yes! exactly11:56
EmilienMsounds like it happens since a promotion11:57
thrashdprince: I spent some time yesterday investgating running mistral the same as we do heat. should be possible.11:57
EmilienMit might be something in Cinder11:57
dprincethrash: I meant to look into it last week in BCN but of course didn't get a chance. Would be happy for you to help right this...11:57
dprincethrash: write even11:57
dprincethrash: write it right11:57
thrashlol11:57
thrashdprince: I'll keep hacking at the mistral thing. I think that's the right way to go.11:58
EmilienMjaosorior: it happenned first on October 27th11:58
jaosoriorEmilienM: during the summit O_O11:58
dprincethrash: actually, I was leaning towards a local version of the jinja template function11:58
thrashdprince: :D11:59
shardyI think a script that runs the j2 rendering would also be useful for t-h-t development11:59
jtomasekflorianf: could you please check pending GUI patches so we manage to backport them before next newton tripleo release (tomorrow afternoon)11:59
dprincethrash: adding Mistral, while potentially useful, would better fall towards a later feature I think11:59
florianfjtomasek: yep, on it.12:00
dprincethrash: unless of course we had other things in mind to do w/ Mistral via the UC installer bits12:00
EmilienMjaosorior: I'm digging nova logs12:00
jtomasekflorianf: thanks!12:00
thrashdprince: ok... either one shouldn't be too monumental. We can always refactor to the mistral action.12:00
jaosoriorshardy: what does the jinja rendering at the moment? if that's done via mistral one could just run that task individually, no?12:01
thrashjaosorior: it is in mistral, but that requires mistral. :D12:01
shardyjaosorior: yes it's done by a mistral action, but in the composable undercloud case there is no mistral atm12:01
shardyand it's sometimes a little clearer when errors happen if you just run j2 locally12:01
shardyso a script that calls the exact same code as the mistral action would be useful, I think12:01
openstackgerritMarios Andreou proposed openstack/tripleo-specs: Composable Service Upgrades  https://review.openstack.org/39211612:02
mariosshardy: first pass ^^^ not much detail yet :) mainly cos we don't got it :)12:02
shardymarios: ack, thanks will check it out after lunch12:02
thrashshardy: a refactor of the jinja processing into a utility would certainly be useful. For the undercloud deploy and an external script12:03
jaosoriorEmilienM: where did you see that BadStatus error? I haven't found it yet.12:03
thrashshardy: I'll start there.12:03
*** rbowen has joined #tripleo12:03
*** fzdarsky has joined #tripleo12:03
*** pkovar has joined #tripleo12:03
shardythrash: ack sounds good - note rbrady did start splitting up the mistral actions, not sure if that's a useful starting point or not12:04
shardyhttps://review.openstack.org/#/c/370868/812:04
thrashshardy: I'll take a look.12:04
EmilienMjaosorior: http://logs.openstack.org/64/391064/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/2d15c11/console.html#_2016-10-31_10_19_54_30488512:04
*** fzdarsky is now known as fzdarsky|afk12:05
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Fix default for barbican documentation  https://review.openstack.org/39211912:09
jaosoriorjistr: ^^12:09
fultonjhttps://review.openstack.org/#/c/391064/ has gotten two +2s. Could someone +1 the workflow so it can get merged?12:10
*** chandankumar has quit IRC12:10
jistrjaosorior: wfm, +2+A12:11
*** zaneb has joined #tripleo12:11
EmilienMjaosorior: I see nothing in nova/cinder logs12:12
EmilienMlooks like boot from volume doesn't work or is unstable12:12
openstackgerritMerged openstack/puppet-tripleo: Fixes transparent binding to OpenDaylight in HA Proxy  https://review.openstack.org/39166012:12
jaosoriorEmilienM: only intance of that I found is in the heat logs, but yeah, same here, I still find nothing in the nova or cinder volumes. I'm trying to search in the system.journal12:14
jaosoriorshardy: would it be possible to get someone from the heat team to add support for httpd deployment of heat in tripleo?12:14
jaosoriorshardy: would need the puppet-heat work and then to add in tripleo12:14
*** tzumainn has joined #tripleo12:15
*** lucasagomes is now known as lucas-hungry12:16
*** fzdarsky|afk has quit IRC12:16
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Update openstack-puppet-modules dependencies  https://review.openstack.org/39212312:17
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge  https://review.openstack.org/39212412:18
*** pkovar has quit IRC12:21
EmilienMjaosorior: I hold on a little bit debug now, I saw a recent promotion last night, I'll see if we still have this problem later12:21
jaosoriorEmilienM: I still don't find anything... is it possible to to turn selinux off for the images? might want to test with that12:22
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173812:23
*** noslzzp has quit IRC12:23
openstackgerritMerged openstack/tripleo-heat-templates: Re-add NFS backend for Glance  https://review.openstack.org/38928512:23
*** maeca1 has joined #tripleo12:23
*** jkilpatr_ has joined #tripleo12:23
*** jkilpatr has quit IRC12:24
*** noslzzp has joined #tripleo12:24
*** akuznetsov has joined #tripleo12:25
*** jayg|g0n3 is now known as jayg12:27
*** maticue has joined #tripleo12:29
*** abregman has quit IRC12:30
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates: Re-add NFS backend for Glance  https://review.openstack.org/39212812:31
*** zoli|lunch is now known as zoli|wfh12:31
*** tobias-fiberdata has joined #tripleo12:32
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120912:33
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4  https://review.openstack.org/37120912:33
openstackgerritJiri Stransky proposed openstack/puppet-tripleo: NFS mounting for Glance file backend  https://review.openstack.org/39213112:34
*** lhinds has quit IRC12:35
*** tobias_fiberdata has quit IRC12:35
*** jcoufal has joined #tripleo12:36
*** pkovar has joined #tripleo12:37
*** jerrygb_ has quit IRC12:38
*** akshai has joined #tripleo12:39
*** lhinds has joined #tripleo12:39
*** rlandy has joined #tripleo12:41
*** lhinds has quit IRC12:42
*** lhinds has joined #tripleo12:43
*** chandankumar has joined #tripleo12:44
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Update openstack-puppet-modules dependencies  https://review.openstack.org/39212312:45
openstackgerritMerged openstack/diskimage-builder: Remove RedHat grub workaround install  https://review.openstack.org/38858012:49
*** abregman has joined #tripleo12:51
*** jerrygb has joined #tripleo12:52
*** ramishra_ has quit IRC12:53
openstackgerritTim Rozet proposed openstack/puppet-tripleo: Fixes transparent binding to OpenDaylight in HA Proxy  https://review.openstack.org/39214212:55
*** derekh has quit IRC12:56
shadowerhey folks, what's the deal with specs and blueprints? When to use which?12:56
*** akuznetsov has quit IRC12:57
shadowerI'd like to add two changes to the validations API (have them optionally run in the deployment workflows and allowing multiple sources of validations & their updating)12:57
EmilienMshadower: specs are useful when we need to discuss about code design or new policies12:57
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: Rework gnocchi-upgrade to run in a separate upgrade step  https://review.openstack.org/38864912:57
EmilienMshadower: blueprints are usually some tasks that doesn't require much design, and allow developers to quickly implement new features12:58
EmilienMif you're creating a new API or system, a spec is better, as the team needs to agree on design12:58
shadowerEmilienM: cool, thanks. Looks like I sholud write up a couple of specs, then12:58
*** ayoung has quit IRC12:58
shadoweryeah12:58
openstackgerritMerged openstack/puppet-tripleo: Add barbican profile  https://review.openstack.org/35244712:58
openstackgerritMerged openstack/puppet-tripleo: Add barbican profile rspec testing  https://review.openstack.org/38809912:59
*** akuznetsov has joined #tripleo13:00
jaosoriorEmilienM: damn dude, I'm quite stumped... haven't found what's up with CI :/13:02
jaosoriorEmilienM: have you found anything?13:02
EmilienMnothing13:03
openstackgerritDmitry Tantsur proposed openstack/tripleo-common: Stop trying to power off nodes right after registering  https://review.openstack.org/39214813:03
dtantsurwhat's the correct lp project to use for bugs: tripleo or tripleo-common?13:04
EmilienMdtantsur: tripleo13:04
dtantsurack13:04
* dtantsur fixes his bug13:04
*** akuznetsov has quit IRC13:04
*** ramishra has joined #tripleo13:06
*** fultonj has quit IRC13:06
*** fultonj has joined #tripleo13:08
*** abregman_ has joined #tripleo13:10
openstackgerritMerged openstack/tripleo-heat-templates: Add replacepkgs to the manual ovs upgrade workaround and fix a typo  https://review.openstack.org/39082613:11
*** rhallisey has joined #tripleo13:13
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates: nova/libvirt: add missing ports for live-migration  https://review.openstack.org/39187613:13
*** abregman has quit IRC13:14
*** links has quit IRC13:14
openstackgerritJames Slagle proposed openstack-infra/tripleo-ci: Undercloud upgrade for mitaka and newton  https://review.openstack.org/38128613:16
*** lblanchard has joined #tripleo13:16
*** lucas-hungry is now known as lucasagomes13:18
*** zoli|wfh is now known as zoli|afk13:21
jaosoriorslagle: are you around?13:23
openstackgerritDmitry Tantsur proposed openstack/tripleo-common: Stop trying to power off nodes right after registering  https://review.openstack.org/39214813:25
openstackgerritEmilien Macchi proposed openstack/tripleo-specs: Propose TripleO Squads policy  https://review.openstack.org/38520113:27
*** morazi has joined #tripleo13:28
EmilienMmeeting in 30 min, feel free to add off items topics https://etherpad.openstack.org/p/tripleo-meeting-items13:29
jaosoriordtantsur: I'm a bit confused with https://review.openstack.org/#/c/392148/ the commit message mentions that powering off the nodes is useless but then that's being done in the workflow, what's up?13:31
slaglejaosorior: yes13:31
jaosoriorslagle: hey, do you happen to know what the current blocker is for getting multinode jobs to deploy net-iso?13:31
dtantsurjaosorior, people get confused when seeing the nodes in "power on" state :)13:31
dtantsurjaosorior, I'm fine with just dropping it otherwise13:31
jaosoriordtantsur: having that on the workflow is fine for me, I was just a bit confused on the commit message, and maybe you had some extra context for me to understand13:32
*** egafford has joined #tripleo13:32
slaglejaosorior: there is no blocker that i know of as of now13:32
*** fragatina has joined #tripleo13:32
dtantsurjaosorior, lemme try to rework the commit message13:32
*** fragatina has quit IRC13:32
slaglejaosorior: it's just a matter of someone doing it13:32
*** fragatina has joined #tripleo13:33
jaosoriorslagle: and basically it would be single-nic-with-vlans, right?13:33
jaosoriorslagle: or do we get multiple nics with multinode?13:33
slaglejaosorior: i'd probably start with single nics13:34
EmilienMjaosorior: right, we have found multinode job was sometimes not catching bugs we cought in ovb13:34
slaglejaosorior: and the jobs need to also be updated to actually use os-net-config13:34
jaosoriorslagle: what do they use at the moment if os-net-config is not being used?13:34
openstackgerritOpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements  https://review.openstack.org/38994513:35
*** morazi has quit IRC13:35
slaglejaosorior: they use a combination of a function from devstack-gate to create vxlan tunnels initially, and then on oc deployment, the nic templates just run an ip command to add the ip addresses to the already configured interfaces13:35
*** absubram has joined #tripleo13:36
openstackgerritDmitry Tantsur proposed openstack/tripleo-common: Power off new nodes when making them available, not right after enrolling  https://review.openstack.org/39214813:36
dtantsurjaosorior, this might be better ^^^13:36
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-validations: Updated from global requirements  https://review.openstack.org/39117013:36
jaosoriordtantsur: that works for me, thank you13:37
jaosoriorslagle: oh, I see13:37
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Only show deprecated args for overcloud deploy  https://review.openstack.org/39216613:39
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173813:40
openstackgerritJuan Antonio Osorio Robles proposed openstack/puppet-tripleo: Fix default for barbican documentation  https://review.openstack.org/39211913:48
*** morazi has joined #tripleo13:48
*** fragatina has quit IRC13:48
openstackgerritMerged openstack/tripleo-specs: Add promotion-blocker tag  https://review.openstack.org/38809413:53
openstackgerritMerged openstack/tripleo-specs: Remove spec-lite tag  https://review.openstack.org/38809513:53
*** ebalduf has joined #tripleo13:53
*** cdearborn has joined #tripleo13:55
*** chem has quit IRC13:56
openstackgerritNoam Angel proposed openstack/diskimage-builder: add support for ubuntu and InfiniBand indhcp-all-interfaces  https://review.openstack.org/39177213:56
openstackgerritNoam Angel proposed openstack/diskimage-builder: add element cloud-init-allow-password-authentication  https://review.openstack.org/39176513:56
openstackgerritNoam Angel proposed openstack/diskimage-builder: redhat-common add option to select networking service NetworkManager/network  https://review.openstack.org/39217013:56
*** derekh has joined #tripleo13:57
EmilienMtripleo meeting is about to start on #openstack-meeting-alt, feel free to join !13:58
*** jerrygb_ has joined #tripleo14:00
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart: Fix tempest runner in devmode  https://review.openstack.org/39131214:02
*** jerrygb has quit IRC14:02
openstackgerritNoam Angel proposed openstack/diskimage-builder: redhat-common add option to select networking service NetworkManager/network  https://review.openstack.org/39217014:03
*** liverpooler has quit IRC14:05
*** anton has quit IRC14:06
*** anton has joined #tripleo14:09
*** anton has quit IRC14:09
*** anton has joined #tripleo14:10
*** anton has quit IRC14:12
*** anton has joined #tripleo14:12
*** anton has quit IRC14:14
*** anton has joined #tripleo14:15
*** jprovazn has quit IRC14:16
*** zoli|afk is now known as zoli|wfh14:17
*** jprovazn has joined #tripleo14:18
lucasagomestrown, hi there... what happened with tripleo-quickstart/config/config_general/devmode.yml ? Was it replaced by minimal.yml ?14:19
openstackgerritNoam Angel proposed openstack/diskimage-builder: simplify ARCH param for rhel/centos param can be x86_64 and amd64  https://review.openstack.org/39217614:20
*** ayoung has joined #tripleo14:23
lucasagomestrown, nvm, I found it https://review.openstack.org/#/c/369571/14:23
*** jkilpatr_ has quit IRC14:31
*** jerrygb has joined #tripleo14:31
openstackgerritDmitry Tantsur proposed openstack/tripleo-common: Do not try "manage" actions on nodes that are not in "enroll" state  https://review.openstack.org/39217914:33
*** jerrygb_ has quit IRC14:33
openstackgerritMerged openstack/tripleo-ui: Include nested parameters in service parameters  https://review.openstack.org/39094614:36
openstackgerritSven Anderson proposed openstack/puppet-tripleo: Implement Nova ec2api profile  https://review.openstack.org/33656214:38
ansiwenEmilienM: can we move forward here? https://review.openstack.org/#/c/336562/14:39
*** jkilpatr has joined #tripleo14:40
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud: Prevent pxe_ilo driver from guessing the boot mode to use  https://review.openstack.org/39218114:43
dtantsurlucasagomes, fyi ^^^14:43
lucasagomesdtantsur, damn, yes!14:44
openstackgerritMerged openstack/tripleo-quickstart: Stop using deprecated network range  https://review.openstack.org/34344314:45
jaosoriorEmilienM: there seems to be nothing in the audit logs, only strange thing is this from httpd (note that cinder runs over it) type=ANOM_ABEND msg=audit(1478006873.887:1495): auid=4294967295 uid=166 gid=166 ses=4294967295 subj=system_u:system_r:httpd_t:s0 pid=7701 comm="httpd" reason="memory violation" sig=1114:46
EmilienMansiwen: will look shortly14:48
EmilienMjaosorior: I'll look after mtg14:48
*** jerrygb_ has joined #tripleo15:00
*** jerrygb has quit IRC15:03
weshayanyone try the containerized compute deployment? wondering if there is info on registering the atomic image to ironic15:03
openstackgerritJiri Tomasek proposed openstack/tripleo-ui: Parameters listing updates  https://review.openstack.org/39185515:04
openstackgerritFlorian Fuchs proposed openstack/tripleo-ui: Use mistral action to create new containers  https://review.openstack.org/39218915:04
jristjtomasek: can you also look at https://review.openstack.org/#/c/390854/ ?15:06
jristwe need it for validations15:06
jristshadower: any progress on ^15:06
jristjtomasek: haha I was caught up :)15:07
jristjtomasek: then two new ones within the last 3 minutes15:07
* jtomasek looks15:07
shadowerjrist: I've just replied to steve's comment. Will update the patch & retest15:07
openstackgerritDmitry Tantsur proposed openstack/tripleo-common: Power off new nodes when making them available, not right after enrolling  https://review.openstack.org/39214815:13
*** paramite has quit IRC15:17
*** ramishra has quit IRC15:17
jkilpatrEmilienM, do you know why bulk introspection is being depreciated? Most docs/people still use it.15:18
*** yamahata has joined #tripleo15:18
*** paramite has joined #tripleo15:19
*** ramishra has joined #tripleo15:19
jtomasekd0ugal: how does the CLI upload files to swift plan?15:20
jpichjkilpatr: openstack overcloud node introspect is meant to replace the bulk introspection command, AFAIUI15:21
*** akuznetsov has joined #tripleo15:21
d0ugaljtomasek: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/workflows/plan_management.py#L22-L3515:22
d0ugaljtomasek: tl;dr - it creates a tarball, uploads it and swift extracts it15:22
jkilpatrjpich, I can see that from the message, but the node introspect command means you are responsible for running it against each node, sure you can hack together a quick bash forloop. or move all nodes to manageable and use the --all-manageable option but its not a drop in replacement.15:22
*** athomas has quit IRC15:22
*** chandankumar has quit IRC15:23
jtomasekd0ugal: oh, I see, ok15:23
d0ugaljkilpatr: Doesn't the previous command only introspect manageable nodes?15:24
jpichjkilpatr: the background for moving away from bulk introspection was described in http://lists.openstack.org/pipermail/openstack-dev/2016-May/095456.html15:24
jpichd0ugal: It moves available nodes back into manageable15:24
*** paramite has quit IRC15:24
d0ugaljpich: ah15:25
jkilpatrjpich, thanks I was looking for reasoning like this.15:25
jpichjkilpatr: Hope that helps15:26
*** milan has quit IRC15:28
*** zaneb has quit IRC15:28
*** athomas has joined #tripleo15:29
openstackgerritMerged openstack/tripleo-ui: Fix deployment success output  https://review.openstack.org/38927015:29
jkilpatrjpich, yes it does. I was looking to add scale features to these commands, namely --retires N and --batch N, one would retry failed introspection the other would batch N to perform at once.15:30
jkilpatrcurrently once you top 20 or so nodes bulk introspection is hopeless and it starts being problematic long before that.15:30
*** akuznetsov has quit IRC15:31
*** absubram has quit IRC15:31
openstackgerritMerged openstack/tripleo-ui: Improve parameters loading  https://review.openstack.org/39181315:32
*** kjw3 has joined #tripleo15:33
*** dhill_ has quit IRC15:34
*** dhill_ has joined #tripleo15:34
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173815:35
shardyshadower: hey did you have a patch that builds a dynamic inventory for ansible (for overcloud nodes) ?15:35
*** absubram has joined #tripleo15:36
jtomasekhonza: you can use BaseHttpRequestErrorHandler to try to handle stuff there but again, problem is as you mentioned - apis behave inconsistently15:37
jtomasekhonza: ApiServices are supposed to only return promises for api calls, not to do the api error handling15:38
jtomasekhonza: I know that this is not very DRY on the actions side, but I'd prefer to keep it that way until the apis we use stabilize and then we could extract actions which trigger api calls into redux middleware15:40
*** zaneb has joined #tripleo15:40
*** paramite has joined #tripleo15:41
jpichjkilpatr: dtantsur is probably a good person to bounce ideas off wrt improving the introspection workflow15:42
*** bana_k has joined #tripleo15:42
* dtantsur hides15:42
dtantsurjkilpatr, jpich, note that "introspection start" and "introspection bulk start" are completely different commands, coming from different projects15:43
dtantsurhowever, batches support is built into ironic-inspector, but iirc only enabled for ssh drivers15:43
*** pcaruana has joined #tripleo15:44
*** aufi has quit IRC15:44
jkilpatrdtantsur, I'm aware took the time monday to map out the entire call path for the bulk commands vs normal introspection15:46
dtantsurheh, I can imagine15:47
jkilpatrdtantsur, I used the entire whiteboard and then some. Anyways I'm in contact with the ironic team too, trying to decide at what point in the chain I want to implement this and see how each project feels about the idea.15:48
jkilpatralso have to keep from hiding new flaws on the CI level, but provide robust end user commands that actually work.15:48
*** dhill_ has quit IRC15:50
*** Guest14381 has quit IRC15:51
*** akshai has quit IRC15:54
*** akshai has joined #tripleo15:56
shadowershardy: https://github.com/openstack/tripleo-validations/blob/master/scripts/tripleo-ansible-inventory15:56
shardyshadower: thanks!15:57
*** dhill_ has joined #tripleo16:00
openstackgerritMerged openstack/puppet-tripleo: Fixes transparent binding to OpenDaylight in HA Proxy  https://review.openstack.org/39214216:03
*** owalsh_ has joined #tripleo16:04
*** owalsh has quit IRC16:04
*** rcernin has quit IRC16:06
openstackgerritAdriano Petrich proposed openstack/tripleo-heat-templates: GATE TEST, please ignore  https://review.openstack.org/36544916:06
*** milan has joined #tripleo16:07
*** artom__ has joined #tripleo16:07
*** pcaruana has quit IRC16:07
*** owalsh_ is now known as owalsh16:08
*** artom_ has quit IRC16:08
*** artom__ is now known as artom16:09
*** myoung is now known as myoung|food16:11
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart: Fix documentation for accessing overcloud nodes  https://review.openstack.org/39222016:11
weshaybah.. can't find the containerized compute experimental job, /me looking for logs..16:13
*** chandankumar has joined #tripleo16:13
weshayI just kicked off experimental but it's not in the list16:13
*** akshai has quit IRC16:13
*** ebarrera has quit IRC16:13
*** akshai has joined #tripleo16:17
*** jaosorior has quit IRC16:17
EmilienMweshay: can someone look on https://review.openstack.org/#/c/391209/ ?16:18
weshayaye16:19
*** trozet has quit IRC16:21
*** trozet has joined #tripleo16:21
*** bana_k has quit IRC16:27
honzajtomasek: If the the API call fail due to authentication, we redirect to the login page.  For all other outcomes, we resolve the promise and "pass it on" to the specific API class.16:35
honzajtomasek: it's kind of like middleware16:35
jtomasekhonza: I understand, my concern is that it skips whole api error handling infrastructure which we already have in place and adds another logic that does it, in a place which is not meant to do that.16:38
honzajtomasek: there is only thing that needs to happen when you encounter a 40116:38
honzajtomasek: *shrug*16:39
jtomasekhonza: you will have to put there a logic to figure out what api service we were calling and handle the api error specific to that api as apis are inconsistent. this is exactly what ApiErrorHandlers do16:39
jtomasekhonza: you mentioned that mistral is different16:39
honzajtomasek: yes, you're right16:39
jtomasekhonza: in general all you'd need to do is add dispatch(loginActions.logoutUser) to case 401 in ApiErrorHandlers16:40
honzajtomasek: sounds good16:41
*** jerrygb has joined #tripleo16:41
jtomasekhonza: only ptoblem for now is that we'll need to pass dispatch as parameter to ApiErrorHandler to be able to dispatch action from there. once we move this to middlewares we'll have dispatch available without doing it16:42
*** mcornea has quit IRC16:42
*** jerrygb_ has quit IRC16:43
*** jerrygb_ has joined #tripleo16:44
*** jerrygb has quit IRC16:46
*** mlupton has joined #tripleo16:52
*** myoung|food is now known as myoung16:54
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: WIP TLS everywhere job  https://review.openstack.org/39173816:58
*** bana_k has joined #tripleo16:59
*** akshai has quit IRC16:59
*** jpich has quit IRC17:01
openstackgerritMerged openstack/tripleo-ui: Save the result of direct action calls in Mistral  https://review.openstack.org/38227717:03
*** mlupton has quit IRC17:06
*** aufi has joined #tripleo17:16
*** derekh has quit IRC17:18
*** jerrygb has joined #tripleo17:25
openstackgerritMerged openstack/tripleo-heat-templates: Add Barbican to the overcloud  https://review.openstack.org/35245817:27
*** mlupton has joined #tripleo17:27
*** jerrygb_ has quit IRC17:28
*** akshai has joined #tripleo17:29
*** Goneri has joined #tripleo17:36
*** egafford has quit IRC17:39
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225317:40
*** shardy has quit IRC17:44
*** hewbrocca is now known as hewbrocca_afk17:46
*** akuznetsov has joined #tripleo17:46
*** zoli|wfh is now known as zoli|gone17:49
*** zoli|gone is now known as zoli_gone-proxy17:50
openstackgerritDougal Matthews proposed openstack/tripleo-docs: [WIP] Mistral API Documentation  https://review.openstack.org/35868517:52
d0ugaltzumainn: ^ I have done some more work on the docs17:52
d0ugaltzumainn: not complete yet, but updated the commit message with things I still want to add (some of these might be better added in a follow up to avoid this turning into a book-in-a-patch)17:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225317:53
*** jtomasek_ has quit IRC17:53
d0ugaltzumainn: feedback would be much appreciated :)17:53
tzumainnd0ugal, okay, I'll take a look soon - thanks!17:54
*** akshai has quit IRC17:55
*** egafford has joined #tripleo17:56
*** mgould is now known as mgould|afk18:00
*** athomas has quit IRC18:02
*** akuznetsov has quit IRC18:02
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure we update ceph and composable nodes  https://review.openstack.org/39225918:04
*** akuznetsov has joined #tripleo18:05
openstackgerritLukas Bezdicka proposed openstack/tripleo-heat-templates: Ensure we update ceph and composable nodes  https://review.openstack.org/39226018:06
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225318:07
*** mhenkel has quit IRC18:07
tzumainnd0ugal, left some comments, looks great overall!18:07
*** lucasagomes is now known as lucas-afk18:09
*** dtantsur is now known as dtantsur|afk18:13
*** akshai has joined #tripleo18:15
*** aufi has quit IRC18:16
slagleEmilienM: do we know what's wrong with the ovb-ha job?18:22
EmilienMslagle: I was investigating it18:22
EmilienMso this morning I woke up and saw the job mostly red everywhere18:22
EmilienMI think it's a bad promotion18:22
EmilienMslagle: I see 2 errors with ping test18:23
EmilienMthey happen randomly18:23
EmilienM1) pingtest stack timeout18:24
EmilienM2) pingtest stack error when creating the Nova::Service from volume18:24
EmilienMNova::Server sorry18:24
*** abregman_ has quit IRC18:24
EmilienMslagle: I looked in logstack and 2) started on October 27th18:24
EmilienMlooking at logstash again, looks like 1) started yesterday18:25
*** anton has quit IRC18:26
slagleare there bugs filed for both issues?18:26
EmilienMnothing in logstash showed me that was due to a patch we merged18:26
*** anton has joined #tripleo18:26
EmilienMI think it's a promotion, and a bug in OpenStack somewhere18:26
EmilienMslagle: not yet, as I'm still debugging the what is failing18:26
EmilienMslagle: I'm creating a launchpad now18:27
EmilienMI also see failures on non-ha18:27
EmilienMClientException: resources.volume1: Gateway Time-out (HTTP 504)18:28
EmilienMok that's a 3)18:28
EmilienMthey might be all related18:28
EmilienMall 3 are in pingtest, and related to nova/cinder somehow18:28
EmilienMI'm filing a single bug now18:28
EmilienMslagle: https://bugs.launchpad.net/tripleo/+bug/163835018:32
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)18:32
EmilienMovb-nonha looks pretty stable though18:36
EmilienMI saw some failures but not a lot18:36
EmilienMwhile ovb-ha is really in bad shape18:36
slagleit might be a heat/keystone error18:37
slaglelooking at http://logs.openstack.org/86/381286/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f17ac81/18:37
slaglethe heat-engine.log on controller 2 shows:18:38
slagleDomain admin client authentication f    ailed18:38
slaglehttp://logs.openstack.org/86/381286/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f17ac81/logs/overcloud-controller-2/var/log/heat/heat-engine.txt.gz#_2016-11-01_15_07_47_20318:38
*** akuznetsov has quit IRC18:39
EmilienMDomain admin client authentication failed18:39
EmilienMindeed18:39
EmilienMdid we have it before? /me checks in old logs18:39
openstackgerritRyan Hallisey proposed openstack/tripleo-heat-templates: Add a heat output describing where roles are located  https://review.openstack.org/32782218:40
*** maeca1 has quit IRC18:40
*** maeca1 has joined #tripleo18:40
slagleEmilienM: in the keystone.log: Could not find domain: heat_stack18:41
EmilienMI check 2 weeks ago, we didn't have this log18:42
EmilienM(in heat-engine)18:42
EmilienMslagle: link?18:42
EmilienM2016-11-01 15:07:47.189 28358 WARNING keystone.auth.controllers [req-5e7fad87-3dfa-47e1-aa82-02b651d0e86d - - - - -] Could not find domain: heat_stack18:43
slagleEmilienM: http://logs.openstack.org/86/381286/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f17ac81/logs/overcloud-controller-2/var/log/keystone/keystone.txt.gz#_2016-11-01_15_07_47_18918:43
EmilienMok I got it18:43
EmilienMI'm going to logstash it18:43
slagleat the same time as the error in heat-engine.log18:43
EmilienMhow I'm afraid logstash won't find it18:44
EmilienMbecause it doesn't look in this file I think18:44
EmilienMdamn18:45
EmilienMpuppet should create the domain18:46
*** egafford has quit IRC18:48
*** egafford has joined #tripleo18:49
*** pkovar has quit IRC18:55
*** nyechiel has joined #tripleo18:55
EmilienMNov  1 15:01:51 localhost os-collect-config: #033[mNotice: /Stage[main]/Heat::Keystone::Auth/Keystone_role[heat_stack_user]/ensure: created18:58
EmilienM2016-11-01 15:07:47.189 28358 WARNING keystone.auth.controllers [req-5e7fad87-3dfa-47e1-aa82-02b651d0e86d - - - - -] Could not find domain: heat_stack18:59
EmilienMoh wait, heat_stack_user not heat_stack domain18:59
*** chandankumar has quit IRC19:00
slagleare we configuring heat with the wrong domain name?19:02
EmilienMperhaps19:02
openstackgerritMerged openstack-infra/tripleo-ci: Undercloud upgrade for mitaka and newton  https://review.openstack.org/38128619:02
EmilienMwe did some changes recently19:02
EmilienMme and Steve were fixing a bug about composable services when heat is separated from keystone19:03
EmilienMI'm looking into the patch again19:03
slaglewell, this is strange19:03
slagleheat.conf on controller 0 is different from the others19:03
slaglecontroller 1 and 2 have:19:04
slaglestack_user_domain_name = heat_stack19:04
slaglestack_domain_admin = heat_stack_domain_admin19:04
slaglebut controller 0 does not have those 2 lines19:04
slaglethis might explain why the failure is transient, it depens on which heat-engine it gets picked up by in ha19:05
EmilienMstack_user_domain_name etc are missing19:05
EmilienMproblem could be here: https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/keystone.pp#L154-L17319:06
EmilienMmwhahaha: on top of your head, could you check that code?19:06
EmilienMspecially https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/keystone.pp#L15919:06
*** Goneri has quit IRC19:06
mwhahahahmm19:07
EmilienMlaunchpad 163113019:08
openstackLaunchpad bug 1631130 in tripleo "Deployment fails when Keystone and Heat services are deployed on different roles nodes" [High,Fix released] https://launchpad.net/bugs/1631130 - Assigned to Emilien Macchi (emilienm)19:08
EmilienMso we tried to fix that ^19:08
mwhahahamight be an ordering thing19:08
EmilienMso we did two things: https://review.openstack.org/#/c/386118/ and https://review.openstack.org/#/c/386115/19:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/163835019:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)19:10
mwhahahaEmilienM: so the os_service_default would result in the missing config items, so i guess the question is when does the base heat profile get included19:11
EmilienMmwhahaha: in CI, we shouldn't be in the "else"19:12
EmilienMsince keystone & heat are collocated19:12
mwhahahabut it's a parse order thing19:12
mwhahahaor at least it could be19:13
mwhahahaEmilienM: Nov  1 15:01:51 localhost os-collect-config: #033[mNotice: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_user_domain_name]/ensure: removed#033[0m19:15
mwhahahait got removed after it was created19:15
mwhahahahttp://logs.openstack.org/86/381286/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f17ac81/logs/overcloud-controller-0/var/log/messages19:15
mwhahahaNov  1 14:29:22 localhost os-collect-config: #033[mNotice: /Stage[main]/Heat::Keystone::Domain/Heat_config[DEFAULT/stack_domain_admin]/ensure: created#033[0m19:16
EmilienMwtf19:16
mwhahahait was removed 30 mins later on a subseqent run19:16
mwhahahaso we do hit that code or hit something else that removed it19:16
EmilienMso created at step 319:17
EmilienMand removed at step 519:18
mwhahahaso here's the thing19:19
EmilienMso we have 2 problems, it should be created at step 5 (it's created earlier) and it's removed at step 5 (while it should be created)19:19
mwhahahatipleo::profile::base::heat doest not include heat::ekystone::domain for any steps except 3 and 419:19
*** egafford has quit IRC19:20
mwhahahahttps://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/heat.pp#L56-L6219:20
mwhahahaso we will always be in that else19:20
mwhahahaer that else would never conflict19:20
mwhahahathis code makes my brain hurt19:21
EmilienMwell no19:21
EmilienMwe check if tripleo::profile::base::heat is defined19:21
EmilienMwhich should be the case19:21
EmilienMmwhahaha: sorry, I wrote that thing :(19:22
mwhahahaEmilienM: i'm just thinking because of the include vs class19:22
EmilienMthe include should create the resource since manage_* are true in puppet-heat by defeault19:23
mwhahahaif tripleo::profile::base::heat actually was already defined and it included class { '::heat::keystone::domain': } the include would be a noop in profile base keystone19:23
mwhahahawe really need to just do the user creation seperate from the configuration update19:24
mwhahahaand make the code right19:24
EmilienMmwhahaha: could it be an ordering issue where the class is defined *after* keystone?19:24
mwhahahaEmilienM: yes19:24
EmilienMso if defined doesn't work19:24
EmilienMok, so I missed that part19:25
mwhahahaif defined is dangerous19:25
EmilienMmy bad :(19:25
mwhahahawe should really not use it for anything19:25
mwhahahabecause it is dependent on catalog ordering19:25
EmilienMmwhahaha: safest way would be to create new parameters19:25
mwhahahaand since our catalogs may be dynamic in nature it's hard to ensure ordering19:25
EmilienMright19:25
EmilienMin fact, I was just looking for a way to detect if heat & keystone were collocated19:26
mwhahahauser creation should be it's own class and we should just include them all with keystone19:26
mwhahahathe configuration should be done seperately19:26
EmilienMright but we only need this user etc if we have heat deployed in the cloud19:27
mwhahahashort term, just add a manage_config to heat::keystone::domain19:27
mwhahahalong term, user creation should be it's own class19:27
mwhahahasimilar to heat::keystone::auth19:28
*** bnemec has joined #tripleo19:28
mwhahahaor just disable the manage_* and do that in tripleo19:28
mwhahaha-^ probably more realistic19:28
EmilienMmwhahaha: outside puppet you mean?19:29
mwhahahano do it in  puppet, but do it in tripleo classes19:29
EmilienMah19:29
EmilienMmakes sense19:29
mwhahahaso there's no reason we can't just do a keyston_user in tripleo profiles19:29
EmilienMmwhahaha: in our case, how do you detect that both keystone & heat are collocated though?19:30
*** dsariel has quit IRC19:30
mwhahahawho cares19:30
mwhahahajust disable the user creation in heat19:30
mwhahahaand create the user in the keystone profile19:30
mwhahahaif heat is enabled19:31
mwhahahawhy does it matter if they are collocated?19:31
mwhahahathe goal was to not leak the password right?19:31
EmilienMmwhahaha: yes19:31
mwhahahaif you disable user creation, there's no need to call keystone::heat::domain in the keystone profile19:32
mwhahahasec i'll propose patch19:32
*** ohamada has quit IRC19:33
*** egafford has joined #tripleo19:33
*** paramite has quit IRC19:39
*** rcernin has joined #tripleo19:41
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: gnocchi statsd should be able to send data to port 8125  https://review.openstack.org/39228719:43
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Create heat user in keystone profile  https://review.openstack.org/39228819:44
mwhahahaEmilienM: -^19:44
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Create heat user in keystone profile  https://review.openstack.org/39228819:45
EmilienMmwhahaha: looking19:46
mwhahahaEmilienM: so either something like that or we add in a manage_config to heat::keystone::domain and use that instead. I'm not sure which is best19:47
EmilienMmwhahaha: mhh this thing works for me19:50
*** mlupton has quit IRC19:51
EmilienMconfig is handled at step 3 and 4 anyway, right?19:51
mwhahahayea19:51
EmilienMactually I'm wondering if we should remove this thing and configure it up to step 3 and more19:51
mwhahahabut shouldn't we be configuring the user before starting the services?19:51
mwhahaha:D19:51
EmilienMtrue, but it should not be a big deal until we actually use heat19:52
EmilienMin your patch, I would also change manifests/profile/base/heat.pp19:52
EmilienMto configure declare ::heat::keystone::domain in "if $step >= 4" maybe19:53
EmilienMit sounds dangerous to me to ignore it at step 519:53
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225319:54
EmilienMmwhahaha: wdyt?19:56
mwhahahayea19:57
*** Goneri has joined #tripleo19:57
mwhahahaEmilienM: so there's also https://review.openstack.org/#/c/392292/19:57
mwhahahaif we want to go that rought19:57
mwhahahaer route19:57
mwhahahai think that might be cleaner but not sure the scope of this change (if it needs backporting)19:58
EmilienMmwhahaha: I prefer puppet-heat patch for longterm support19:59
EmilienMmwhahaha: I think logic should go in upstream modules19:59
EmilienMbut that's me19:59
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225320:00
mwhahahaso here's the sticking point20:00
mwhahahawhen heat & keystone are combined you have to know20:00
mwhahahasince it's the same class we can't use them together20:00
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225320:00
mwhahahaunless we calculate the manage_* flags via hiera and know that heat and keystone are combined20:00
EmilienMmwhahaha: tc meeting, brb20:02
*** mlupton has joined #tripleo20:05
*** florianf has quit IRC20:06
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/163835020:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)20:10
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates: gnocchi statsd should be able to send data to port 8125  https://review.openstack.org/39228720:11
*** dprince has quit IRC20:12
openstackgerritNoam Angel proposed openstack/diskimage-builder: add element cloud-init-allow-password-authentication  https://review.openstack.org/39176520:13
*** nyechiel has quit IRC20:17
openstackgerritNoam Angel proposed openstack/diskimage-builder: add support for ubuntu and InfiniBand indhcp-all-interfaces  https://review.openstack.org/39177220:18
openstackgerritNoam Angel proposed openstack/diskimage-builder: add support for SUSE in dhcp-all-interfaces  https://review.openstack.org/39177220:19
*** jayg is now known as jayg|g0n320:19
*** maeca1 has quit IRC20:20
openstackgerritgreghaynes proposed openstack/diskimage-builder: Perform package install outside of debootstrap  https://review.openstack.org/39225320:21
openstackgerritwes hayutin proposed openstack/tripleo-specs: update the index for ocata  https://review.openstack.org/38820820:21
*** yamahata has quit IRC20:22
openstackgerritNoam Angel proposed openstack/diskimage-builder: simplify ARCH param for rhel/centos param can be x86_64 and amd64  https://review.openstack.org/39217620:23
*** paramite has joined #tripleo20:23
*** nyechiel has joined #tripleo20:27
openstackgerritAlex Schultz proposed openstack/puppet-tripleo: Create heat user in keystone profile  https://review.openstack.org/39228820:32
openstackgerritNoam Angel proposed openstack/diskimage-builder: simplify ARCH param for rhel/centos param can be x86_64 and amd64  https://review.openstack.org/39217620:34
openstackgerritNoam Angel proposed openstack/diskimage-builder: redhat-common add option to select networking service NetworkManager/network  https://review.openstack.org/39217020:35
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Remove endpoints creation from post-deployment config  https://review.openstack.org/39230220:38
*** egafford has quit IRC20:38
*** nyechiel has quit IRC20:40
*** rhallisey has quit IRC20:40
*** jkilpatr has quit IRC20:41
*** jprovazn has quit IRC20:42
*** mlupton has quit IRC20:47
*** oneswig has joined #tripleo20:54
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Remove endpoints creation from post-deployment config  https://review.openstack.org/39230220:55
EmilienMmwhahaha: back!20:56
mwhahahak waiting to see if ci passes (after i fixed the user role instead of role)20:56
EmilienMmwhahaha: I guess your first solution with puppet-tripleo is simplier20:57
EmilienMhttps://review.openstack.org/#/c/392288/20:57
EmilienMyou also changed the step20:57
EmilienMmwhahaha: should we make heat_admin_password required in hiera?20:58
EmilienMyou put 'changeme' if undef20:58
mwhahahawell that's the default of the puppet-heat class20:59
mwhahahaso if it's not in hiera it gets changeme20:59
EmilienMoh well, default in puppet-heat is same20:59
EmilienMright20:59
mwhahahawe probably should make it required :D but i want to see if this works first20:59
EmilienMmwhahaha: do we need Class['::keystone::roles::admin'] before creating the domain/user?20:59
mwhahahano just the user role i believe20:59
EmilienMk21:00
*** paramite has quit IRC21:00
*** dsariel has joined #tripleo21:00
dtrainorWhen I have a deployment that fails because I have a Controller that runs in to the infamous "No valid host was found" error, will that leave any other node that depends on a Controller, like a Compute, in the CREATE_ABORTED status?  Is that what aborts the Compute deploy, or would I have seen a separate error for the Compute deployment failure?21:01
dtrainorThe reason I ask is because I'm looking at the CREATE_ABORTED status of this Compute node, and the Compute resource has a required_by of a pretty inclusive list of resources21:02
openstackgerritAna Krivokapic proposed openstack/python-tripleoclient: Remove endpoints creation from post-deployment config  https://review.openstack.org/39230221:05
EmilienMslagle: can you add https://review.openstack.org/#/c/392288 in your backlog? if it's loog to you, I'll approve it tonight if CI pass consistently and if heat config is good again on all controller nodes.21:06
EmilienMs/loog/looks good/ - this is me re-inventing english21:07
slagleEmilienM: honestly, i could only review that for syntax21:08
slagleEmilienM: maybe capture what it fixes somewhere21:08
slaglei dont understand what the root cause ended up being21:09
slaglewere we creating the user, then deleting it?21:09
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/163835021:10
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)21:10
*** ooolpbot has quit IRC21:10
EmilienMslagle: I made a mistake by using "if Defined" in Puppet which is a bad thing. We're having an orchestration issue where users is randomly managed21:12
*** absubram has quit IRC21:12
*** morazi has quit IRC21:13
EmilienMslagle: what mwhahaha does in https://review.openstack.org/#/c/392288/ is to separate heat config and keystone resource in a better way I did21:13
*** Goneri has quit IRC21:14
*** mlupton has joined #tripleo21:18
*** mlupton has quit IRC21:25
openstackgerritJames Slagle proposed openstack/python-tripleoclient: Fix nodes count check on stack-update with custom roles  https://review.openstack.org/39231321:25
openstackgerritJames Slagle proposed openstack/python-tripleoclient: Fix nodes count check on stack-update with custom roles  https://review.openstack.org/39231321:26
*** mlupton has joined #tripleo21:27
*** mlupton has quit IRC21:27
*** mlupton has joined #tripleo21:29
*** abregman has joined #tripleo21:29
*** lblanchard has quit IRC21:31
*** akrivoka has quit IRC21:31
openstackgerritJames Slagle proposed openstack/python-tripleoclient: Fix nodes count check on stack-update with custom roles  https://review.openstack.org/39231321:38
*** isq has joined #tripleo21:41
*** jerrygb has quit IRC21:46
*** abregman has quit IRC21:49
*** jcoufal has quit IRC21:52
*** jkilpatr has joined #tripleo21:55
*** thrash is now known as thrash|g0ne21:57
*** jerrygb has joined #tripleo22:04
*** oneswig has quit IRC22:05
*** akshai has quit IRC22:05
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates: Add Neutron network type and VLAN ranges to network-environment.yaml  https://review.openstack.org/39232522:05
*** jerrygb has quit IRC22:06
tdasilvaEmilienM: hi! wondering if you could help me out figure out something about swift configuration in tripleo. in my overcloud deployment i see the option object_post_as_copy = false22:06
tdasilvabut here https://github.com/openstack/puppet-swift/blob/master/manifests/proxy/copy.pp it is set as true22:07
tdasilvawhere else could it be set as false?22:07
*** mlupton has quit IRC22:07
tdasilvaEmilienM: the other place I looked was here: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/swift-proxy.yaml but it doesn't list any copy options there at all22:09
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/163835022:10
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)22:10
*** ooolpbot has quit IRC22:10
*** mlupton has joined #tripleo22:20
openstackgerritMerged openstack/diskimage-builder: elements: simple-init: Remove SUSE interfaces  https://review.openstack.org/38815822:20
*** rcernin has quit IRC22:31
*** yamahata has joined #tripleo22:32
*** dsariel has quit IRC22:32
mwhahahatdasilva: is that the default in the packaging maybe?22:32
mwhahahatdasilva: cause i don't see a reference to swift::proxy::copy anywhere in the heat templates22:33
tdasilvamwhahaha: the default is set to true in swift, but i just found this https://github.com/openstack/puppet-swift/blob/master/spec/classes/swift_proxy_copy_spec.rb22:37
mwhahahatdasilva: that's just a unit test22:37
EmilienMI'm back22:37
EmilienMmwhahaha: looks like ha job failed22:37
tdasilvamwhahaha: lol, as you can see i know nothing about puppet22:38
mwhahahaEmilienM: 503 for something, looking22:38
EmilienMhttp://logs.openstack.org/88/392288/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/7db9856/console.html#_2016-11-01_22_13_43_98654522:38
EmilienM401 Unauthorized: This server could not verify that you are authorized to access the document you requested. Either you supplied the wrong credentials (e.g., bad password), or your browser does not understand how to supply the credentials required. (HTTP 401)22:38
EmilienMduring glance image upload22:38
mwhahahaEmilienM: what creds are used  for the glance upload?22:39
*** jkilpatr has quit IRC22:40
*** jkilpatr has joined #tripleo22:40
tdasilvamwhahaha: oh, sorry, you said packaging and I understood upstream source. so yeah, the default in upstream source is true, but it is possible that the packaging changes that to false22:41
mwhahahatdasilva: https://github.com/rdo-packages/swift-distgit/blob/rpm-master/proxy-server.conf#L8222:43
tdasilvamwhahaha: perfect, thanks!22:43
mwhahahatdasilva: packaging sets it as false and tripleo never corrects it. you could do a custom template to trigger a swift::proxy::copy invocation for your deploy if you want it true22:43
tdasilvamwhahaha: suppose i wanted to add another option to the copy filter? how would i got about it?22:44
mwhahahatdasilva: as an extra thing or by default? I'm not a swift expert so what exactly do you need to add?22:45
tdasilvamwhahaha: well..i was just using copy as an example, but what i need to add is an option to staticweb middleware for this bug: https://bugzilla.redhat.com/show_bug.cgi?id=138330522:47
openstackbugzilla.redhat.com bug 1383305 in openstack-tripleo "Please configure the url_base /etc/swift/proxy-server.conf" [Unspecified,Assigned] - Assigned to thiago22:47
mwhahahaand i just had a conversation about swift configs today heh22:47
tdasilvaso i tried to add that here: https://github.com/openstack/puppet-swift/blob/master/manifests/proxy/staticweb.pp and here https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/swift-proxy.yaml22:48
mwhahahaso for tripleo, that means update puppet-swift to do whatever you want and then include it in tripleo22:48
mwhahahatdasilva: if you're looking at adding something to the staicweb filter then yea that'd be the right spot22:49
*** [1]cdearborn has joined #tripleo22:50
tdasilvamwhahaha: the way i went about it was to hack directly in my undercloud and do a overcloud deploy to test it, but i don't see the changes in the overcloud once it is deployed22:50
mwhahahatdasilva: did you build the overcloud images with your change included?22:51
*** panda|pto|PH is now known as panda|Zz22:51
mwhahahathere's some environmental vars you can use to do that but that's kinda outside my expertise22:52
tdasilvamwhahaha: no...i just made the changes directly to /etc/puppet and /usr/share/openstack-tripleo-heat-templates/22:53
mwhahahatdasilva: i think what you need is to add the url_base to swift::proxy::staticweb, and update the THT with hieradata to set the url_base22:53
mwhahahatdasilva: after you deployed or before? i just know to get those puppet changes in you have to include them when you build the overcloud images22:53
tdasilvamwhahaha: yeah, i think that's what i did, just wanted to actually test it out before sending git-review22:53
tdasilvabefore i deployed, but i didn't do any image rebuilding22:54
mwhahahayea that's probably why it didn't end up on the overcloud22:54
tdasilvamwhahaha: just did this: http://tripleo.org/basic_deployment/basic_deployment_cli.html#redeploy-the-overcloud22:55
mwhahahatdasilva: http://hardysteven.blogspot.com/2016/08/tripleo-deploy-artifacts-and-puppet.html22:55
tdasilvamwhahaha: cool, i will check that out!22:56
mwhahahagood luck, if you need help with the puppet-swift updates just ask over in #puppet-openstack22:56
EmilienMmwhahaha: overcloudrc iirc23:02
EmilienMI'm doing recheck23:02
*** cdearborn has quit IRC23:03
mwhahahaEmilienM: unrecognized fernet token or something23:03
EmilienMmwhahaha: I'm looking heat config now23:04
*** mlupton has quit IRC23:04
EmilienMparams are good !!23:04
EmilienMon controller0 and 123:04
EmilienMso it seems like we have good config now23:04
mwhahahaEmilienM: http://logs.openstack.org/88/392288/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/7db9856/logs/overcloud-controller-2/var/log/glance/api.txt.gz23:04
mwhahahalooks like we have some floating keystone issue or something23:05
*** dmacpher has quit IRC23:05
mwhahahaEmilienM: looks like glance is doing v2.0 auth and other stuff is doing v3, is this expected?23:07
EmilienMmwhahaha: no idea23:09
EmilienMlooks like another issue (again)23:09
mwhahahaoh this must be for the glance middleware23:09
mwhahahacause it's glance posting for a token23:09
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/163835023:10
openstackLaunchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server" [Critical,In progress] - Assigned to Emilien Macchi (emilienm)23:10
*** ooolpbot has quit IRC23:10
mwhahahathere's a whole bunch of 401s and 404s to /v3/auth/tokens on controller-023:10
mwhahahameh these ligsd23:10
mwhahahas/ligsd/logs23:10
mwhahahaEmilienM: aren't the fernet keys supposed to be the same on each controller?23:15
mwhahahacause they aren't23:15
*** dmacpher has joined #tripleo23:18
mwhahahaEmilienM: we enabled fernet keys by default in puppet-keystone, are we not properly configuring fernet in tripleo?23:21
* mwhahaha facepalms23:23
EmilienMmwhahaha: what???23:26
mwhahahaso we enabled fernet by default in puppet-keystone which is fine for single node controllers. I thought tripleo handled fernet key generation.23:27
mwhahahait doesn't23:27
EmilienMyes we do23:27
mwhahahanot in the hieradata23:27
EmilienMwe generate with tripleoclient23:27
mwhahahathere are no fernet keys23:27
EmilienMI wrote that23:27
mwhahahaits not working23:27
EmilienMit was working23:27
EmilienMI can show you logs of when it worked, keys were same23:27
mwhahahawell it doesn't work anymore :D23:27
mwhahahaall i've got is credential keys in the heiradata23:27
mwhahahano fernet keys23:28
mwhahahaand the fernet keys are all different23:28
mwhahahaon the nodes23:28
mwhahahahttp://paste.openstack.org/show/587591/23:28
EmilienMwait23:28
EmilienMI'm stupid23:28
EmilienMI did credentials work23:29
EmilienMnot fernets23:29
EmilienMok I need to sleep23:29
mwhahahak so i proposed puppet-keystone revert23:29
EmilienMI'm approving your revert...23:29
EmilienMif you don't mind23:29
mwhahahathat's fine23:29
EmilienMk23:29
EmilienMayoung did work for fernet23:29
EmilienMbut seems like it's not finished23:29
* mwhahaha gets the bus out and prepares to drive over the world23:29
ayoungFor multi node controllers we can't just generate keys in the Keystone server23:30
EmilienMmwhahaha: https://review.openstack.org/#/c/383956/23:30
ayoungthe Tripleo approach is to generate them on the undercloud and push them out to the overcloud23:30
mwhahahaayoung: yea the problem is i thought we were already doing that23:31
ayoungnot yet23:31
mwhahahait's ocata, that stuff landed in mitaka :D23:31
ayoungmwhahaha, I really don;t know how to solve that problem in the general Puppet case23:31
ayoungnot the puppet changes they didn't23:31
mwhahahaayoung: they did today, cause i thought tripleo was doing fernet key generation in tripleo client already23:32
ayoungsorry23:32
mwhahahaayoung: there's the disconnect and we gate the puppet-keystone a single controller so we didn't know23:32
ayoungmwhahaha, looking for a spec that might be relevant one sec23:32
*** cwolferh has quit IRC23:32
ayounghttp://git.openstack.org/cgit/openstack/keystone-specs/tree/specs/keystone/ocata/fernet-key-store.rst23:33
EmilienMwhy don't we do like what I did for credentials for now?23:33
ayoungthe goal there is to use an external key manager23:33
EmilienMI thought we agreed on that ayoung ?23:33
ayoungEmilienM, yeah, and I thought I posted that...23:33
ayounghttps://review.openstack.org/#/c/371247/23:33
EmilienMocata is super short, we won't make your keystone blueprint + tripleo integration23:33
mwhahahaayoung: can we just do an MVP to get on fernet and work on expanding that later like everyone else?23:34
ayoungMVP?23:34
mwhahahaminimal viable product23:34
mwhahahaaka do the simple stuff first and expand23:34
ayoungah, I thought you meant most valuable player.  We have one of those.  It is EmilienM23:34
mwhahahado like we do for credentials and then expand to add rotation and such23:34
ayoungmwhahaha, so, yeah, that review should be the basis.  What happened today?23:35
mwhahahaayoung: we landed fernet by default in puppet-keystone23:35
mwhahahaand since we're not managing fernet keys in tripleo at all, it blew up23:35
ayoungand what happened?  THings break?23:35
mwhahahayea cause you can't auth on a multiple controller cluster when all the keys are different23:35
mwhahahatokens only work 1/3 the time23:35
ayoungwell, that should be a solvable problem23:36
ayoungmwhahaha, I think we have things out of order23:36
ayoungbefore we can make Fernet default, we need synchronized keys23:36
mwhahahaayoung: but we have a method for doing that via tripleoclient23:36
ayoungWelll, sort of.23:37
mwhahahawhy is that not in place already is kinda my question23:37
ayoungWere we using it?23:37
mwhahahano23:37
mwhahahathat's how we got here23:37
ayoungmwhahaha, maybe cuz I didn;'t write it yet23:37
ayounglet me see...23:37
mwhahahasorry i'm confused cause this was stuff that landed elsewhere in mitaka23:38
ayounghttps://review.openstack.org/#/c/383956/23:38
mwhahahaso i wasn't aware tripleo was lacking this bit23:38
ayoungHmmm that looks wrongish23:38
mwhahahaalso because we dont' explicitly define uuid, we got nailed when upstream changed23:38
ayoungAh23:38
ayoungOK so we need that one one...is it sufficient, though...23:39
ayoungI seem to recall needing a tripleoclient change too23:39
mwhahahaanyway we can solve this tomorrow23:39
mwhahahanow we know that tripleo doesn't do fernet :D23:39
ayoungEmilienM, when you did credentials, did that require a python-tripleoclient change, or was that just something else I am misrememberating?23:40
* mwhahaha wanders off for a bit23:40
EmilienMayoung: tripleoclient23:41
EmilienMI have to go now or she kills me and we don't want that23:41
openstackgerritJames Slagle proposed openstack/python-tripleoclient: Fix nodes count check on stack-update with custom roles  https://review.openstack.org/39231323:44
*** kjw3 has quit IRC23:47
*** dmacpher has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!