Wednesday, 2018-02-07

*** rloo has quit IRC00:06
*** retr0h has quit IRC00:13
*** mkrai has quit IRC00:13
*** csmart has quit IRC00:13
*** bfournie has joined #openstack-ironic00:22
*** r-daneel has quit IRC00:23
*** ijw has joined #openstack-ironic00:24
*** rpioso is now known as rpioso|afk00:29
*** ijw has quit IRC00:29
openstackgerritMerged openstack/ironic master: Rework exception handling on deploy failures in conductor  https://review.openstack.org/53112000:44
*** hshiina has joined #openstack-ironic00:46
*** mkrai has joined #openstack-ironic00:49
*** hamzy_ has quit IRC00:50
*** Goneri has joined #openstack-ironic00:56
*** tiendc has joined #openstack-ironic00:58
*** phuongnh has joined #openstack-ironic01:06
*** ijw has joined #openstack-ironic01:11
*** hieulq has joined #openstack-ironic01:11
*** rbudden has joined #openstack-ironic01:12
*** Goneri has quit IRC01:12
*** absubram has joined #openstack-ironic01:13
*** hamzy_ has joined #openstack-ironic01:13
*** ijw has quit IRC01:17
*** tuanla____ has joined #openstack-ironic01:17
*** yangyapeng has quit IRC01:20
*** yangyapeng has joined #openstack-ironic01:20
*** links has joined #openstack-ironic01:28
*** retr0h has joined #openstack-ironic01:28
*** csmart has joined #openstack-ironic01:28
*** links has quit IRC01:29
*** rbudden has quit IRC01:33
*** absubram has quit IRC01:39
*** ijw has joined #openstack-ironic01:39
*** ijw has quit IRC01:44
*** ijw has joined #openstack-ironic01:47
openstackgerritMerged openstack/ironic master: Zuul: Remove project name  https://review.openstack.org/54119001:52
*** ijw has quit IRC01:52
*** masber has quit IRC01:59
*** jlabarre has quit IRC02:05
*** links has joined #openstack-ironic02:12
openstackgerritShivanand Tendulker proposed openstack/ironic master: Add validate_rescue() method to boot interface  https://review.openstack.org/54095402:20
*** crushil has joined #openstack-ironic02:27
*** nmathew has joined #openstack-ironic02:32
*** nmathew has quit IRC02:33
*** nmathew has joined #openstack-ironic02:33
*** tuanla____ has quit IRC02:33
*** phuongnh has quit IRC02:33
*** tiendc has quit IRC02:33
*** zshi has quit IRC02:34
*** phuongnh has joined #openstack-ironic02:34
*** tiendc has joined #openstack-ironic02:34
*** tuanla____ has joined #openstack-ironic02:34
*** zshi has joined #openstack-ironic02:34
*** nmathew has quit IRC02:51
openstackgerritMerged openstack/bifrost master: Fix upper-constraints use  https://review.openstack.org/54053802:52
openstackgerritMerged openstack/bifrost master: Fix non-venv keystone handling  https://review.openstack.org/49891002:54
*** mkrai has quit IRC02:54
*** csmart has quit IRC02:55
*** retr0h has quit IRC02:55
*** zhenguo has joined #openstack-ironic03:01
openstackgerritwangdequn proposed openstack/ironic master: Correct grammar, duplicate the found  https://review.openstack.org/54152503:08
*** ijw has joined #openstack-ironic03:09
*** ijw has quit IRC03:14
*** namnh has joined #openstack-ironic03:18
openstackgerritwangdequn proposed openstack/ironic master: correct grammar, duplicate the found  https://review.openstack.org/54153003:24
*** jaganathan has joined #openstack-ironic03:39
*** crushil has quit IRC03:49
*** ianychoi_ has quit IRC03:49
*** ianychoi_ has joined #openstack-ironic03:50
*** tiendc has quit IRC03:53
*** tiendc has joined #openstack-ironic03:59
*** openstackgerrit has quit IRC04:04
*** exodusftw has quit IRC04:08
TheJuliadtantsur|afk: sambetts|afk: I'm convinced https://review.openstack.org/#/c/541378/ helps, but does not fix the main issue we are started encountering on stable/pike. High level details in the whiteboard with another example. I'm going to crash, I've been staring at this for far too many hours. Whiteboard updated with more details.04:13
patchbotpatch 541378 - ironic (stable/pike) - [Devstack] Replace tap with veth04:13
*** exodusftw has joined #openstack-ironic04:15
*** zhangfei has joined #openstack-ironic04:22
*** hshiina2 has joined #openstack-ironic04:29
*** harlowja has quit IRC04:32
*** zhangfei_ has joined #openstack-ironic04:33
*** hshiina has quit IRC04:33
*** tuanla____ has quit IRC04:40
*** phuongnh has quit IRC04:41
*** namnh has quit IRC04:42
*** tiendc has quit IRC04:44
*** openstackgerrit has joined #openstack-ironic05:07
openstackgerritMerged openstack/ironic master: Modify error quotation marks  https://review.openstack.org/54114605:07
openstackgerritMerged openstack/ironic master: Correct grammar, duplicate the found  https://review.openstack.org/54152505:07
*** zhenguo has quit IRC05:10
*** harlowja has joined #openstack-ironic05:37
*** devananda has quit IRC05:40
*** olivierbourdon38 has joined #openstack-ironic05:41
*** zhangfei_ has quit IRC05:52
*** mkrai has joined #openstack-ironic05:57
*** retr0h has joined #openstack-ironic06:02
*** ccollett has joined #openstack-ironic06:07
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071306:07
*** charcol-v2 has quit IRC06:09
*** e0ne has joined #openstack-ironic06:11
*** csmart has joined #openstack-ironic06:13
*** marios has joined #openstack-ironic06:13
*** e0ne has quit IRC06:17
*** ptx0 has quit IRC06:20
*** ptx0 has joined #openstack-ironic06:23
*** olivierbourdon38 has quit IRC06:23
*** ptx0 has quit IRC06:42
*** mjura has joined #openstack-ironic06:44
*** e0ne has joined #openstack-ironic06:48
*** dpawar has joined #openstack-ironic06:49
*** olivierbourdon38 has joined #openstack-ironic06:50
*** ptx0 has joined #openstack-ironic06:51
*** hjensas has quit IRC06:52
*** harlowja has quit IRC06:52
*** e0ne has quit IRC06:54
*** dpawar has quit IRC06:57
*** vsaienk0 has joined #openstack-ironic07:02
openstackgerritJohn L. Villalovos proposed openstack/ironic-python-agent master: When ouputting to /dev/null should open in write mode  https://review.openstack.org/54158407:04
*** olivierbourdon38 has quit IRC07:16
*** AlexeyAbashkin has joined #openstack-ironic07:22
*** dpawar has joined #openstack-ironic07:23
*** ccollett has quit IRC07:25
*** jtomasek has joined #openstack-ironic07:40
*** mrunge has joined #openstack-ironic07:40
*** mrunge_ has quit IRC07:42
*** AlexeyAbashkin has quit IRC07:42
*** olivierbourdon38 has joined #openstack-ironic07:46
*** vsaienk0 has quit IRC07:48
*** rcernin has quit IRC07:48
*** weshay has quit IRC07:50
*** hshiina3 has joined #openstack-ironic07:50
*** rnoriega has quit IRC07:51
*** lhinds has quit IRC07:51
*** hjensas has joined #openstack-ironic07:51
*** hjensas has joined #openstack-ironic07:51
*** hshiina3 is now known as hshiina07:53
*** hshiina2 has quit IRC07:54
*** AlexeyAbashkin has joined #openstack-ironic07:54
*** vsaienk0 has joined #openstack-ironic07:57
*** amarao has joined #openstack-ironic07:58
*** rnoriega has joined #openstack-ironic08:02
*** weshay has joined #openstack-ironic08:04
*** lhinds has joined #openstack-ironic08:04
*** ianychoi_ has quit IRC08:14
*** ianychoi_ has joined #openstack-ironic08:19
*** threestrands has quit IRC08:22
*** tesseract has joined #openstack-ironic08:24
*** ianychoi_ has quit IRC08:28
openstackgerritShivanand Tendulker proposed openstack/ironic-python-agent master: Add documentation for rescue mode  https://review.openstack.org/43162208:28
*** ianychoi_ has joined #openstack-ironic08:30
*** pcaruana has joined #openstack-ironic08:38
*** ianychoi_ has quit IRC08:45
*** ianychoi_ has joined #openstack-ironic08:46
openstackgerritHironori Shiina proposed openstack/ironic master: Update iRMC document for classic driver deprecation  https://review.openstack.org/54117109:02
*** athomas has joined #openstack-ironic09:03
openstackgerritMerged openstack/bifrost master: Zuul: Remove project name  https://review.openstack.org/54101509:08
*** athomas has quit IRC09:14
openstackgerritHironori Shiina proposed openstack/ironic master: Update iRMC document for classic driver deprecation  https://review.openstack.org/54117109:16
*** rbartal has joined #openstack-ironic09:17
openstackgerritHironori Shiina proposed openstack/ironic master: Update iRMC document for classic driver deprecation  https://review.openstack.org/54117109:20
*** e0ne has joined #openstack-ironic09:26
openstackgerritgaofei proposed openstack/bifrost master: Fix unreachable link in document  https://review.openstack.org/54163409:27
*** olivierbourdon38 has quit IRC09:28
*** mgoddard_ has joined #openstack-ironic09:28
*** hshiina is now known as hshiina|afk09:30
*** hshiina|afk has quit IRC09:33
*** derekh has joined #openstack-ironic09:44
openstackgerritwangdequn proposed openstack/ironic master: correct grammar, duplicate the found  https://review.openstack.org/54164009:45
openstackgerritMerged openstack/ironic master: correct grammar, duplicate the found  https://review.openstack.org/54153009:45
*** hw_wutianwei has quit IRC09:54
*** hw_wutianwei has joined #openstack-ironic09:56
*** pcaruana has quit IRC09:57
*** namnh has joined #openstack-ironic09:57
*** vsaienk0 has quit IRC10:09
*** sambetts|afk is now known as sambetts10:10
*** pcaruana has joined #openstack-ironic10:13
*** strigazi_ has joined #openstack-ironic10:16
*** hw_wutianwei has quit IRC10:17
*** vsaienk0 has joined #openstack-ironic10:18
*** stendulker has joined #openstack-ironic10:18
*** hw_wutianwei has joined #openstack-ironic10:18
*** strigazi has quit IRC10:19
*** strigazi_ is now known as strigazi10:19
*** ianychoi_ has quit IRC10:21
*** ianychoi_ has joined #openstack-ironic10:24
*** dtantsur|afk is now known as dtantsur10:26
dtantsurmorning ironic10:26
stendulkerGood morning dtantsur :)10:27
pas-hamorning dtantsur stendulker10:28
stendulkerGood morning pas-ha :)10:28
*** gcb has quit IRC10:29
*** jaganathan has quit IRC10:30
dtantsuro/10:31
sambettsMorning dtantsur, pas-ha, stendulker10:31
stendulkersambetts: Good morning :)10:32
stendulkerdtantsur: I went through you comments for https://review.openstack.org/#/c/54095410:32
patchbotpatch 540954 - ironic - Add validate_rescue() method to boot interface10:32
stendulkerdtantsur: I think, I'm bit confused...10:33
stendulkerdtantsur: Are you suggesting retaining the existing AgentRescue.validate() code as is and add a additional check of boot.validate_rescue() to it ?10:33
dtantsurcorrect10:34
stendulkerBut the rescue_ramdisk and rescue_kernel used by PXE are being validated in AgentRescue.validate()10:34
dtantsurstendulker: I guess it should be moved to PXEBoot.validate_rescue as well10:36
dtantsurthe rescue interface itself IIRC only uses rescue_password in reality10:36
stendulkerdtantsur: yes.10:36
stendulkerdtantsur: Currently we do not make call to boot.validate in conductor for rescue operation https://github.com/openstack/ironic/blob/master/ironic/conductor/manager.py#L580-L596.10:38
*** athomas has joined #openstack-ironic10:38
stendulkerAlso regarding ilo patch. We would need the ilo/rescue.py to implement rescue.get_properties()10:39
dtantsurno, we wouldn't10:39
dtantsuryour get_properties does not differ from AgentRescue.get_properties10:40
dtantsurcurrently your patch incorrectly pulls in ilo_rescue_iso from the boot interface10:40
stendulkerso you mean rescue_kernel, rescue_ramdisk or ilo_rescue_iso belongs to boot interface?10:40
*** trungnv has joined #openstack-ironic10:41
sambettsthey should yes, they are tied to the method of booting, e.g. rescue_kernel/rescue_ramdisk are pxe things, ilo_rescue_iso is ILOVirtualMedia things10:41
stendulkerok. Currently we have put rescue_kernel/rescue_ramdisk into AgentRescuehttps://github.com/openstack/ironic/blob/master/ironic/drivers/modules/agent.py#L61-L6710:42
stendulkerThen I will have to move this into pxe.py10:43
stendulkerAlso perform boot interface validation into conductor/manager.py10:44
dtantsuryep10:44
*** lucas-afk is now known as lucasagomes10:45
stendulkerThen we would not need common/rescue.py as proposed in the patch.10:45
stendulkerThank you. Will post the patch with the changes.10:45
*** trungnv has quit IRC10:52
dtantsurthanks!10:54
*** ianychoi_ has quit IRC11:05
*** ianychoi_ has joined #openstack-ironic11:07
*** stendulker_ has joined #openstack-ironic11:07
*** stendulker has quit IRC11:09
*** pcaruana has quit IRC11:09
*** vsaienk0 has quit IRC11:12
*** stendulker_ has quit IRC11:14
openstackgerritSam Betts proposed openstack/ironic stable/ocata: Use zuul.override_checkout  instead of custom branch_override var  https://review.openstack.org/54083011:17
*** dmellado has quit IRC11:21
peitolmMorning all11:22
dtantsurmorning peitolm11:23
*** dmellado has joined #openstack-ironic11:27
*** vsaienk0 has joined #openstack-ironic11:28
*** namnh has quit IRC11:28
peitolmdtantsur: I was talking about the issue with mimetypes from yesterday, with a colleague this morning, and i suddenly thought, what happens if a user has two nodes called node and node.json,  (this may be an edge case on the edge case),11:31
dtantsurpeitolm: funny, I thought about it just today :)11:31
dtantsurwell, we have to maintain compatibility in *this* patch11:31
dtantsurmeaning, "node.json" will fetch "node" :(11:32
dtantsurmy follow-up plan is to create a microversion that reversed the order11:32
dtantsurthen deprecate this whole thing and remove it in some other microversion11:32
dtantsurwdyt?11:32
openstackgerritDmitry Tantsur proposed openstack/ironic master: Do not pass credentials to the ramdisk on cleaning  https://review.openstack.org/54168311:33
dtantsursambetts, mind reviewing ^^ please? it's security hardening11:33
dtantsurwe've been working on it in private, so should be looking good, but it never hurts to get another opinion11:33
peitolmif the mime type wasn't being parsed, what would the user expect? that node and node.json are two different things, can we determine somehow if pecan has done a mimetype match?11:34
sambettsdtantsur: at first glance it looks good11:34
peitolmin fact, is the raw url available to us? could we look at that, and use that in extremis11:34
dtantsurpeitolm: the current API contract is for /v1/nodes/test.json means /v1/nodes/test with Content-Type: Application/Json11:36
dtantsurs/means/to mean/11:36
dtantsurwhich is sad, but I don't think we can work around the problem without breaking it11:36
peitolmanother thought, we know if .json has been stripped correct, could do we something like, if .json stripped, check if node.json exists in preference11:36
peitolmi guess that's your followup microversion11:37
dtantsurI do it, but it cannot solve the problem of "test" vs "test.json"11:37
dtantsuryes, with a microversion I'll change it11:37
peitolmo.k. if stripped, check both, if both exist, return with .json,11:37
*** vsaienk0 has quit IRC11:38
dtantsuryes, that will be the *new* version11:38
dtantsurin the old version we have to do s/with/without/ still :)11:38
dtantsur:) -> :(11:38
peitolm*nods* i need more coffee, i've been an hour and a half without11:38
dtantsur++11:38
peitolmso to summarise, the correct behaviour is that node and node.json should both be able to exist, and be accessible, if asked for node.json and it doesn't exist, it should fail, (where at the moment, it returns the result for node)11:40
dtantsurpeitolm: https://etherpad.openstack.org/p/ironic-node-mime11:44
openstackgerritMerged openstack/ironic master: Only set default network interface flat if enabled in config  https://review.openstack.org/53583011:45
openstackgerritMerged openstack/ironic-inspector master: Remove sample policy and config files  https://review.openstack.org/54040211:45
*** charcol has quit IRC11:47
*** dmellado has quit IRC11:48
*** vsaienk0 has joined #openstack-ironic11:51
*** yangyapeng has quit IRC11:57
*** dmellado has joined #openstack-ironic11:59
*** ianychoi_ has quit IRC12:00
peitolmdtantsur: cool, a new toy :)12:01
*** vsaienk0 has quit IRC12:01
*** etingof has quit IRC12:04
openstackgerritDmitry Tantsur proposed openstack/ironic master: Stop guessing mime types based on URLs  https://review.openstack.org/54128912:07
*** vsaienk0 has joined #openstack-ironic12:10
*** yangyapeng has joined #openstack-ironic12:12
*** pcaruana has joined #openstack-ironic12:13
*** yangyapeng has quit IRC12:16
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add 10.1 to the release mapping in preparation for the release  https://review.openstack.org/54169412:17
*** etingof has joined #openstack-ironic12:17
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add 10.1 and queens to the release mapping  https://review.openstack.org/54169412:19
*** dpawar has quit IRC12:19
*** dpawar has joined #openstack-ironic12:20
openstackgerritDmitry Tantsur proposed openstack/ironic master: Add 10.1 and queens to the release mapping  https://review.openstack.org/54169412:22
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix extra double-quote  https://review.openstack.org/54169612:22
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: Fix devstack example  https://review.openstack.org/54169712:23
*** dpawar has quit IRC12:24
*** AlexeyAbashkin has quit IRC12:25
*** yangyapeng has joined #openstack-ironic12:29
*** zshi has quit IRC12:29
*** zshi has joined #openstack-ironic12:33
*** AlexeyAbashkin has joined #openstack-ironic12:40
*** hw_wutianwei has quit IRC12:43
*** hw_wutianwei has joined #openstack-ironic12:44
openstackgerritIlya Etingof proposed openstack/sushy-tools master: Abstract away the virtualization management interface  https://review.openstack.org/52332112:45
openstackgerritDmitry Tantsur proposed openstack/ironic master: Handle case when a glance image contains no data  https://review.openstack.org/53118012:46
openstackgerritDmitry Tantsur proposed openstack/ironic stable/pike: Do not pass credentials to the ramdisk on cleaning  https://review.openstack.org/54170312:47
openstackgerritDmitry Tantsur proposed openstack/ironic stable/ocata: Do not pass credentials to the ramdisk on cleaning  https://review.openstack.org/54170412:48
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: DNM - testing tripleo  https://review.openstack.org/54170512:52
openstackgerritHarald Jensås proposed openstack/ironic master: DNM - Depends-On to test tripleo gate  https://review.openstack.org/54170612:52
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: Fix devstack example  https://review.openstack.org/54169712:59
*** vsaienk0 has quit IRC13:14
openstackgerritMerged openstack/bifrost master: Fix unreachable link in document  https://review.openstack.org/54163413:14
*** olivierbourdon38 has joined #openstack-ironic13:16
*** vsaienk0 has joined #openstack-ironic13:18
*** sekharvajjula has joined #openstack-ironic13:21
*** zhangfei has quit IRC13:25
*** jlabarre has joined #openstack-ironic13:31
*** shekarvajjula has joined #openstack-ironic13:32
*** AlexeyAbashkin has quit IRC13:34
*** AlexeyAbashkin has joined #openstack-ironic13:35
*** nicodemos has joined #openstack-ironic13:37
*** hjensas has quit IRC13:37
*** jcoufal has joined #openstack-ironic13:39
*** AlexeyAbashkin has quit IRC13:40
*** zhangfei has joined #openstack-ironic13:41
*** rloo has joined #openstack-ironic13:48
openstackgerritMerged openstack/ironic-ui stable/queens: Update .gitreview for stable/queens  https://review.openstack.org/54131713:52
*** shekarvajjula has quit IRC13:53
openstackgerritDmitry Tantsur proposed openstack/networking-baremetal master: Make the agent distributed using hashring and notifications  https://review.openstack.org/52470913:54
openstackgerritDmitry Tantsur proposed openstack/networking-baremetal master: Add unit tests for member manager  https://review.openstack.org/54121613:55
*** AlexeyAbashkin has joined #openstack-ironic13:55
rloogood morning everyone, dtantsur, sambetts13:57
dtantsurmorning rloo13:57
dtantsurrloo: https://review.openstack.org/541683 is ready for approval13:57
patchbotpatch 541683 - ironic - Do not pass credentials to the ramdisk on cleaning13:57
rloosambetts: this doesn't look good, i assume we still want the dependent patches to land? https://review.openstack.org/#/c/540355/13:57
patchbotpatch 540355 - ironic-tempest-plugin - Switch to override-checkout for stable branch jobs13:57
rloodtantsur: oh, looking...13:58
sambettsrloo: yeah I hit a bug in zuul v3 with my latest stable/ocata dependent patch13:58
sambettsrloo: basically you can't have comments in the job that have quote marks in them13:58
sambettsotherwise it blows us13:59
sambettsup*13:59
*** ijw has joined #openstack-ironic13:59
rloosambetts: whoa, maybe need a pep8/linter to check that :-(13:59
sambettsI've fixed it I was just waiting for zuul to respond on the dependent before I rechecked that one13:59
sambettsrloo: I've pushed a fix up to zuul13:59
rloosambetts: thx13:59
sambettsso hopfully that gets reviewed soon13:59
TheJuliaGood morning everyone14:00
sambettsrloo: the dependant patches should be good to go now14:00
rloomorning TheJulia14:00
dtantsurmorning TheJulia14:01
rloosambetts: ok, will review those soon14:01
*** SerenaFeng has joined #openstack-ironic14:01
dtantsurrloo, TheJulia, the backports are ready when the master one merges https://review.openstack.org/#/q/topic:bug/174483614:01
peitolmMorning rloo, TheJulia14:01
rloomorning peitolm14:03
TheJuliadtantsur: I noticed :)14:03
dtantsur:)14:03
*** ijw has quit IRC14:03
TheJuliahas anyone got braincells to help try and figure out what is going on with grenade today?14:03
*** r-daneel has joined #openstack-ironic14:04
sambettsTheJulia: I can try :)14:04
*** Goneri has joined #openstack-ironic14:04
TheJuliaI managed to reproduce in an all in one node performing a pull-up14:05
TheJuliajust... thinking maybe a patch is needed to keep tempest retrying for... oh... ever.14:05
TheJuliaor.... I could shutdown the conductor.....14:05
openstackgerritMerged openstack/ironic stable/pike: [Devstack] Replace tap with veth  https://review.openstack.org/54137814:05
TheJuliathat would at least leave the VMs up with networking and everything else can fail14:06
sambettsTheJulia: so now the veth fix has been backported we're still seeing the issue?14:07
TheJuliayes14:07
sambetts:(14:07
TheJuliaI left rough notes of what I found last night on the whiteboard14:08
*** links has quit IRC14:09
olivierbourdon38I am confused about an error I am having when I try to add the openstack CLI fromhttps://review.openstack.org/#/c/540713/7/playbooks/roles/bifrost-keystone-install/tasks/install.yml line 76 and onwards14:10
patchbotpatch 540713 - bifrost - Fix for proper deployment when keystone enabled an...14:10
olivierbourdon38The error I am getting is: Could not satisfy constraints for 'python-openstackclient': installation from path or url cannot be constrained to a version14:11
dtantsurmy guess was that in the CI (at least) zuul can insert requirements in the form of file:///opt/stack/new/python-openstackclient14:11
rloohas anyone looked to see why ironic-tempest-dsvm-pxe_ipmitool-postgres has been failing? http://ci-watch.tintri.com/project?project=ironic&time=7+days14:11
dtantsurwhich cannot be constrained14:11
olivierbourdon38which I now understand because I am trying to install from locally cloned repo into path /opt/stack/python-openstackclient14:11
dtantsurrloo: TheJulia (?) mentioned glance problem14:11
*** mjturek has joined #openstack-ironic14:12
rloodtantsur: ok, so we're good with it failing for now? I can ignore?14:12
*** r-daneel has quit IRC14:12
*** rloo has quit IRC14:12
olivierbourdon38whereas there is already an entry in /opt/stack/requirements/requirents.txt which states: python-openstackclient===3.14.014:12
olivierbourdon38therefore the error14:12
*** rloo has joined #openstack-ironic14:12
dtantsurrloo: I think we can ignore postgres *for now*14:13
olivierbourdon38however couldn't this also occur with the keystone git repo code above14:13
rloodtantsur: ok :)14:13
dtantsurolivierbourdon38: keystone is not in global-requirements (nor are any other services)14:13
olivierbourdon38currently it succeed because there is no entry for keystone in requirements.txt14:13
dtantsurcorrect. so the contraints are only applied to keystone dependencies, but not to keystone *itself*14:13
*** sekharvajjula has quit IRC14:13
*** shekarvajjula has joined #openstack-ironic14:13
olivierbourdon38ok I think I get the idea, thanks14:14
dtantsuryou're welcome14:14
*** e0ne has quit IRC14:17
*** ianychoi has joined #openstack-ironic14:17
* rloo updated whiteboard with status of postgres job. now on to bigger and better things...14:18
sambettsTheJulia: I'm sure I've seen an issue with the VIPs like this before, I think if I remember correctly it was an issue with the MTU we had configured on the links (but I thought we fixed that)14:23
sambettsagain maybe we forgot the backport :/14:23
*** hjensas has joined #openstack-ironic14:23
TheJuliaIt feels very dejavu-ish14:23
TheJuliamtu is 145014:23
TheJuliai think14:24
mjturekgm ironic!14:25
sambettso/ mjturek14:25
openstackgerritOlivier Bourdon proposed openstack/bifrost master: WIP: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071314:31
*** r-daneel has joined #openstack-ironic14:31
sambettsTheJulia: looking at the logs you linked on the whiteboard, the baremetal boots up and does the deploy OK, then reboots to boot into the guest image and never makes it to the OS14:33
sambettsTheJulia: which would be why it isn't responding to the pings/arps etc from the VIP14:34
sambettsTheJulia: it seems like its trying to boot from PXE into the guest image and is failing14:34
*** rbudden has joined #openstack-ironic14:35
sambettsTheJulia: but thats expected because it its now in the tenant network and we don't support PXE there14:35
*** lucasagomes is now known as lucas-hungry14:35
sambettsTheJulia: looking at the vbmc logs I never see a call to switch the boot device to HDD for the VMs14:36
TheJuliabut why now?14:37
*** baha has joined #openstack-ironic14:39
sambettsthat is the question, the networking for the VMs on reboot is working though, because they are getting an IP address from the tenant network DHCP14:40
*** vsaienk0 has quit IRC14:43
*** vsaienk0 has joined #openstack-ironic14:43
TheJuliasame version of ovs between working/not working14:44
sambettsyeah, I don't think the networking is the issue, that actually seems to be doing what it should be14:44
sambettsbut the VM is sat in this state: http://logs.openstack.org/32/540432/2/check/ironic-grenade-dsvm-multinode-multitenant/199b2f4/logs/old/ironic-bm-logs/node-1_no_ansi_2018-02-05-12:50:36.txt.gz14:45
sambettsI wonder if there was a change in libvirt or something where it used to boot from hdd if it failed to PXE14:46
sambettsbut now it doesn't14:46
TheJuliasame version of libvirt14:46
sambettshmmm14:46
* TheJulia checked last night14:46
TheJuliaalways the same three tests...14:47
sambettshave we got a link to logs for this job last time it passed?14:47
TheJuliaone moment14:47
dtantsurTheJulia, rloo, https://review.openstack.org/#/c/541694/14:47
patchbotpatch 541694 - ironic - Add 10.1 and queens to the release mapping14:47
sambettsTheJulia: have we changed the type of image used for the job recently?14:49
TheJuliasambetts: http://logs.openstack.org/42/526442/2/check/ironic-grenade-dsvm-multinode-multitenant/7a88329/logs/ this passed on stable/pike about a week ago14:49
TheJuliaalthough we should likely be looking at http://logs.openstack.org/42/526442/2/check/ironic-grenade-dsvm/035fa33/ since that is entirely pike14:50
rloodtantsur: looking...14:50
TheJuliasambetts: another master job that worked on stable/pike grenade https://review.openstack.org/#/c/537650/14:52
patchbotpatch 537650 - ironic - Be more sane about cleaning (MERGED)14:52
*** athomas has quit IRC14:52
*** zhangfei has quit IRC14:53
*** dpawar has joined #openstack-ironic14:55
sambettsTheJulia: looking at the devstack log, we set the default boot option for ironic to netboot which is wrong for a multi-tenant deployment14:56
*** dpawar has quit IRC14:56
*** e0ne has joined #openstack-ironic14:56
sambettsthis might have always been wrong and we might have been relying on the wholedisk image type to force it to boot option local14:56
TheJuliasambetts: well yes however that is also our default operating behvior14:56
sambettsbut this is a multi-tenant job, boot option net is never going to work14:57
sambettsshouldn't we be configuring it correctly, and seeing if that fixes the issue?14:58
sambettseither that or we need to make sure that we are setting boot_option=local on the nodes as we enroll them14:59
TheJuliaagreed, I noticed none being set. Worth a try at least, a direct patch to stable/pike for now?15:00
olivierbourdon38anybody could help with finding out what is wrong in http://logs.openstack.org/13/540713/8/check/bifrost-integration-dibipa-debian-centos-7/ef448a2/job-output.txt.gz or is it just a transient failure ?15:00
TheJuliadib dependency failure I guess http://logs.openstack.org/13/540713/8/check/bifrost-integration-dibipa-debian-centos-7/ef448a2/job-output.txt.gz#_2018-02-07_14_43_26_79337615:01
*** marios has quit IRC15:02
sambettsTheJulia: I think I know whats going on...15:03
*** dpawar has joined #openstack-ironic15:03
sambettsTheJulia: and whats made the change15:03
* sambetts just confirming my theory15:03
*** SerenaFeng has quit IRC15:05
openstackgerritJulia Kreger proposed openstack/ironic stable/pike: Test switching to local default boot  https://review.openstack.org/54174115:05
sambettsTheJulia: making a change to the job on stable/pike won't work15:06
openstackgerritShivanand Tendulker proposed openstack/ironic-python-agent master: Add documentation for rescue mode  https://review.openstack.org/43162215:06
sambettsTheJulia: the grenade job runs the master playbook15:06
TheJuliadoh, yeah15:06
*** vsaienk0 has quit IRC15:06
sambettsTheJulia: and I think thats whats broken things15:06
sambettsTheJulia: because we've change the grenade job playbook to use hardware types15:06
sambettsbut its actually running the pike devstack plugin15:07
TheJuliathat would do it15:07
sambettsso master grenade playbook local.conf + pike devstack plugin15:07
sambetts== boom15:07
sambettsTheJulia: https://review.openstack.org/#/c/536875/15:08
patchbotpatch 536875 - ironic - Switch the CI to hardware types (MERGED)15:08
sambettsthat patch ^15:08
*** dsneddon has quit IRC15:09
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071315:09
dtantsurso, how many times have I broken things with that patch? :)15:09
TheJuliaoh15:10
TheJuliawell direct deploy traditionally implied always local boot15:10
TheJuliaunless the patch to change that landed...15:10
sambettsyeah I thought that was the case too15:11
* rloo thinks we don't land any of dtantsur's features so close to release :)15:11
TheJuliabut the conductor logs are also indicates none set, so it goes to default15:11
TheJuliawe must have a bug somewhere in stable/pike that we're hitting15:12
sambettsthere is a lot of logic in the stable/pike devstack plugin that regexs on the node driver being agent_*15:12
openstackgerritDmitry Tantsur proposed openstack/ironic master: Clean up release notes before a release  https://review.openstack.org/54174415:14
dtantsureveryone's (esp. rloo's) favourite part ^^^15:14
sambettsthere is also no logic in the stable/pike driver for reading the value of IRONIC_DEFAULT_DEPLOY_INTERFACE15:14
rloo'favorite' cuz it means we're going to release soon. the only bright side :)15:14
sambettsso its using the default for the IPMI hardware type15:15
sambettswhich I'm guessing isn't direct?15:15
sambettshence why its no using local15:15
dtantsuryep, the default is iscsi15:15
dtantsurmeh15:16
sambettsand the ugly beasty reveals itself15:16
* TheJulia adds "change default boot mode" to the list for the cycle15:16
dtantsurso, do we just need to return this grenade to classic drivers? that should be easy15:16
dtantsurTheJulia++++++++++++++++++++++++++++++++++15:16
sambettsyeah15:16
sambettsI'll put up a partial revert of your patch to restore the old grenade local.conf15:17
sambettsand we've got to remember that the grenade job is special because the local.conf in there is for the "old" part of the grenade15:17
sambettsrun15:17
rloosambetts: with a note that we can't change it to hw type until Rocky15:18
TheJuliasambetts: if you put a giant warning note in as well, then +++++15:18
rloosambetts: the more comments there the better15:18
dtantsurfunny enough, I've hit a similar problem with inspector grenade, but I did not link the two events15:18
* dtantsur needs a PTO?15:19
rloonope, we all need to figure out how to do this w/o trying to cram in sooo much stuff at the end15:20
TheJuliapto +++++15:20
* TheJulia needs PTO too15:20
*** vsaienk0 has joined #openstack-ironic15:21
peitolmPTO?15:21
dtantsurTheJulia: in very little time you're going to lose the right to have PTOs :D15:22
peitolmPurple thingy objects?15:23
dtantsurLOL15:23
dtantsurpaid time off, but I like your idea better15:23
*** AlexeyAbashkin has quit IRC15:24
*** AlexeyAbashkin has joined #openstack-ironic15:24
TheJuliadtantsur: translation, I should take a few days off sooner rather than later15:25
peitolmin some countries a Purple thingy object might be currency15:25
*** rbartal has quit IRC15:25
dtantsurTheJulia: true15:25
*** absubram has joined #openstack-ironic15:25
* TheJulia is liking the return of silly to the channel15:25
peitolmmy last PTO got me started with openstack....15:26
dtantsurwow, how?15:26
* peitolm bows15:26
peitolmI had some time with the kids at school, the wife away and some free time, so i built an openstack lab15:26
dtantsurthat's a curious idea how to spend free time :D15:27
dtantsur(says someone who's building an openstack SDK in rust in his spare time)15:27
dtantsurpeitolm: anyway, now I'm curious: why an openstack lab?15:28
peitolmwell, usually i'm too beat in the evenings to hack around with tech, so end up on the games console, watching mindless TV, or playing Clash of Clans, (or all three)15:28
peitolmwhy openstack? well, i can build HPC compute clusters, and i've had a play with IBM cloud private, so why not try openstack, seems to be getting a lot of visibilty, so i was curious15:30
*** absubram has quit IRC15:30
peitolmgernally it boils down to, welll, why not15:30
dtantsurthat's quite a reason, I agree15:30
*** absubram has joined #openstack-ironic15:31
peitolmi'm considering investigating it further for work, but wanted to play at home first, i wanted to understand how it (mostly) fits together15:31
*** lucas-hungry is now known as lucasagomes15:32
dtantsurand how is it looking so far?15:32
dtantsurcores, if we land https://review.openstack.org/#/c/541584/ and https://review.openstack.org/#/c/537968/ we'll be ready for IPA release (modulo reno?)15:32
patchbotpatch 541584 - ironic-python-agent - When ouputting to /dev/null should open in write mode15:32
patchbotpatch 537968 - ironic-python-agent - Replace curly quotes with straight quotes15:32
peitolmdtantsur: well, the simple version with vm deployment on a hypervisor worked,15:33
peitolmi'm stuck getting ironic working though, which is why i'm here, TheJulia has been most helpful with where to look, and patient enough with what i'm sure are stupid questions15:34
*** dpawar has quit IRC15:34
dtantsurmost of "stupid" questions may in reality be a lack of documentation on our side15:35
*** dpawar has joined #openstack-ironic15:35
dtantsurwhich is something where contributions are always welcome ;)15:35
openstackgerritOpenStack Release Bot proposed openstack/ironic-inspector stable/queens: Update .gitreview for stable/queens  https://review.openstack.org/54174815:35
openstackgerritOpenStack Release Bot proposed openstack/ironic-inspector stable/queens: Update UPPER_CONSTRAINTS_FILE for stable/queens  https://review.openstack.org/54174915:35
openstackgerritOpenStack Release Bot proposed openstack/ironic-inspector master: Update reno for stable/queens  https://review.openstack.org/54175015:35
peitolmunfortunately, some of that was true, (and some of it is because i'm using the pike release as packaged by ubuntu), so i think i got lost between sets of documentation15:35
rloodtantsur: wrt release notes, I feel like we should have one features note about rescue, with subpoints is ok. but right now, it is 3 main points15:36
rloodtantsur: and the three points aren't even together15:36
dtantsurrloo: I'm fine with that. what I did not like is that the list heading was talking about API15:36
peitolmnow i'm not sure if i'm stuck because i'm hitting a bug, or some side-affect of the translation from HV&VM setup to ironic15:36
rloodtantsur: heh, i didn't even look at your changes, am just reading http://logs.openstack.org/44/541744/1/check/build-openstack-releasenotes/adc6973/html/unreleased.html :)15:37
dtantsurrloo: which three btw? do you want the validate_rescue change to be merged into "features"?15:37
rloodtantsur: also, for prelude, is it worth mentioning traits there. and/or rescue (even though we haven't totally finished)?15:38
dtantsurNot sure. For me prelude is something "omg big and important"15:38
dtantsurrescue is not finished, dunno how important traits are for people right now..15:38
rloodtantsur: 'Adds rescue_interface field...', 'Adds version 1.38 of the Bare Metal API...', 'Adds new configuration options for the rescue feature'15:38
dtantsurs/people/consumers/15:38
rloodtantsur: can't admin start using traits for scheduling with nova?15:39
dtantsurthey totally can, I'm just not sure it's worth a prelude15:39
dtantsurditto for routed networks support15:39
*** dpawar has quit IRC15:39
rloodtantsur: here's 9.1.0 prelude: https://docs.openstack.org/releasenotes/ironic/pike.html#id715:40
dtantsurah, of this style. yes, I can do something alike15:40
peitolmq: is pike the current stable, or is that queens?15:41
dtantsurpeitolm: pike is15:41
dtantsurwe're releasing queens final really soon though15:42
rloodtantsur: also for prelude, don't know if we want to mention stuff from 10.0, eg ansible deploy interface.15:42
peitolmok. a followup then, if pike is the current stable, then what is latest? (as far as documentation is concerned)15:43
rloopeitolm: latest == master branch15:43
peitolmo.k. so latest is really "unstable" or "development"15:44
rloopeitolm: no unstable code here, development is acceptable :)15:45
peitolmo.k.15:45
*** vsaienk0 has quit IRC15:45
dtantsurrloo: in Pike we did, so I guess so15:46
openstackgerritSam Betts proposed openstack/ironic master: Partial revert of "Switch the CI to hardware types"  https://review.openstack.org/54175515:46
sambettsTheJulia, dtantsur ^15:46
TheJuliaI was just going to ask about that :)15:46
TheJuliasambetts: dtantsur should we keep swift settings... granted nothing else related was backported so maybe not15:48
*** snasir has quit IRC15:49
dtantsurto be honest, I'd prefer we only change what is needed to fix the job15:49
dtantsurbut I won't block it based on that15:50
TheJuliaworks for me15:50
*** snasir has joined #openstack-ironic15:50
openstackgerritIlya Etingof proposed openstack/sushy-tools master: Abstract away the virtualization management interface  https://review.openstack.org/52332115:50
openstackgerritJulia Kreger proposed openstack/ironic master: Return multinode multitenant to voting status  https://review.openstack.org/53773915:50
*** devananda has joined #openstack-ironic15:50
dtantsursambetts: let's maybe only change the drivers back? I'd prefer a oneline diff, not a revert :)15:51
TheJuliasambetts: jlvillal has asked why on the subject, likely just a slightly more verbose commit message15:52
jlvillalTheJulia, +115:53
jlvillal:)15:53
*** dsneddon has joined #openstack-ironic15:53
openstackgerritMerged openstack/ironic master: Do not pass credentials to the ramdisk on cleaning  https://review.openstack.org/54168315:55
sambettsat some point we've got to convert all this to non-legacy jobs15:55
TheJuliaI was actually thinking about that in the shower a little while aog15:56
dtantsurtotally, but I'd prefer to keep them clean anyway15:56
*** SerenaFeng has joined #openstack-ironic15:56
TheJuliado we want to merge the vif detach locking change?15:57
NobodyCamGood Morning Ironic'ers16:00
openstackgerritSam Betts proposed openstack/ironic master: Revert grenade jobs to classic drivers  https://review.openstack.org/54175516:00
sambettsupdated ^16:00
*** SerenaFeng has quit IRC16:01
dtantsurmorning NobodyCam16:01
NobodyCam:) happy hump day dtantsur :)16:01
*** ricardoas has joined #openstack-ironic16:02
dtantsurrloo: do you have more comments on the release notes? if not, I'll post an update16:03
rloodtantsur: yup, i do. give me a few minutes, i have a few more16:03
*** mjura has quit IRC16:03
*** pcaruana has quit IRC16:03
dtantsurack16:03
*** SerenaFeng has joined #openstack-ironic16:06
openstackgerritMerged openstack/networking-baremetal master: Make the agent distributed using hashring and notifications  https://review.openstack.org/52470916:11
openstackgerritMerged openstack/ironic master: Fix handling of 'timeout' parameter to power methods  https://review.openstack.org/54015016:14
openstackgerritMerged openstack/ironic master: Use zuul.override_checkout  instead of custom branch_override var  https://review.openstack.org/54043216:14
openstackgerritMerged openstack/ironic-python-agent master: Add documentation for rescue mode  https://review.openstack.org/43162216:14
dtantsurrloo: thanks! to be honest, I don't think I have enough patience to group vendor stuff :)16:17
rloodtantsur: i know :-( it would be nicer though. but hey, i suppose they could do it themselves...16:17
rloodtantsur: i have to stop spending brain cells thinking about how to make things nicer for the reader!16:18
*** dpawar has joined #openstack-ironic16:18
rloosambetts: i didn't +2 cuz I have a question: https://github.com/openstack/ironic/blob/master/doc/source/install/configure-networking.rst16:19
rloosambetts: i'm blind, i see your comment now16:20
openstackgerritMerged openstack/networking-baremetal master: Add unit tests for member manager  https://review.openstack.org/54121616:21
openstackgerritSam Betts proposed openstack/ironic master: Revert grenade jobs to classic drivers  https://review.openstack.org/54175516:22
jlvillalsambetts, FYI: merge conflict on ^^^16:23
sambettsyup... fixing :)16:23
jlvillal:)16:23
jlvillalsambetts, I think that merge conflict is that fault of sambetts' zuul.override_checkout patch merging ;)16:24
jlvillals/that fault/the fault/16:24
sambettsyup16:25
jlvillalI'll let you complain to him :)16:25
sambettsits my own fault ;)16:25
rloojlvillal: sigh, that sambetts, always causing trouble16:25
rloodtantsur: should we update commit msg, or are you OK w/o? https://review.openstack.org/#/c/540830/4/playbooks/legacy/tempest-dsvm-ironic-ipa-partition-bios-pxe_ipmitool-tinyipa/run.yaml16:25
patchbotpatch 540830 - ironic (stable/ocata) - Use zuul.override_checkout  instead of custom bran...16:25
dtantsurrloo: since it does not touch production code, I'm kinda okay with it16:26
dtantsuror like: I'd update it, but I don't want to wait 2-3 more hours for the check queue16:26
rloodtantsur: ok, at least there are comments there if anyone is curious. and yeah, if we had lots of time etc16:26
*** ijw has joined #openstack-ironic16:26
openstackgerritSam Betts proposed openstack/ironic master: Revert grenade jobs to classic drivers  https://review.openstack.org/54175516:27
*** ijw has quit IRC16:28
*** ijw has joined #openstack-ironic16:28
*** jaganathan has joined #openstack-ironic16:33
*** dpawar has quit IRC16:34
*** dpawar has joined #openstack-ironic16:36
*** amarao has quit IRC16:37
*** absubram has quit IRC16:37
openstackgerritJulia Kreger proposed openstack/ironic master: Return multinode multitenant to voting status  https://review.openstack.org/53773916:37
openstackgerritMerged openstack/ironic-inspector stable/ocata: Remove use of construct lib FieldError exception  https://review.openstack.org/54127116:39
*** dpawar has quit IRC16:40
*** dprince has joined #openstack-ironic16:43
*** zhangfei has joined #openstack-ironic16:43
*** zhangfei has quit IRC16:45
jrollgood morning16:45
sambettso/ jroll16:45
* jroll sees you all have done everything already this morning16:45
sambettsits certainly been a busy morning ;)16:46
jrollI'm basically not here today, internal conf thing16:46
dtantsurmorning jroll16:46
TheJuliagood morning jroll16:47
jroll\o16:47
NobodyCammorning jroll16:47
*** baha has quit IRC16:48
rloogood morning jroll, NobodyCam, jlvillal16:49
NobodyCamGood morning rloo :)16:49
*** e0ne has quit IRC16:49
jlvillalGood morning rloo jroll NobodyCam TheJulia sambetts dtantsur and everyone else16:49
NobodyCamhey hey jlvillal top of the morning to ya16:49
jrollfeels like everyone is here today \o/16:50
sambetts\o/16:51
TheJuliawoot, one +2 on 53444116:55
*** jaganathan has quit IRC16:56
*** dprince has quit IRC16:59
*** absubram has joined #openstack-ironic16:59
jrollthat's the +2 we really needed on it, too :)17:00
* jroll reviews now17:00
*** absubram has quit IRC17:00
*** SerenaFeng has quit IRC17:01
openstackgerritIlya Etingof proposed openstack/sushy-tools master: Abstract away the virtualization management interface  https://review.openstack.org/52332117:02
TheJuliaindeed17:02
*** SerenaFeng has joined #openstack-ironic17:05
rlooTheJulia: wrt 534441 do we want a reno?17:08
*** rpioso|afk is now known as rpioso17:08
*** SerenaFeng has quit IRC17:09
*** harlowja has joined #openstack-ironic17:11
dtantsursimple patch for approval, I think the last for IPA queens: https://review.openstack.org/#/c/541584/17:12
patchbotpatch 541584 - ironic-python-agent - When ouputting to /dev/null should open in write mode17:12
dtantsurTheJulia: how is bifrost looking? I can leave it up to you to submit the release later today though17:13
TheJuliarloo: I guess we ought to17:14
TheJuliadtantsur: I can release it later, I've not had a chance to look today, but I think I just needed nobodycam to do a review or two17:15
dtantsurpas-ha: wanna release ironic-staging-drivers and create stable/queens?17:15
dtantsurTheJulia: ack17:15
TheJuliarloo: how much detail are you thinking would be good regarding a release note?17:16
* NobodyCam opens review queue17:16
TheJuliaNobodyCam: <317:16
rlooTheJulia: i hadn't thought about it.17:16
TheJuliaNobodyCam: do I owe you coffee now?17:16
* rloo thinks which crunching on a carrot...17:16
* TheJulia just started a bagel toasting17:16
NobodyCam:)17:16
dtantsuranyone wants to try to fit https://review.openstack.org/#/c/541289/ in the release?17:17
patchbotpatch 541289 - ironic - Stop guessing mime types based on URLs17:17
dtantsurmay not be a trivial addition though :)17:17
*** harlowja has quit IRC17:17
rlooTheJulia: something like 'Fixes an issue where the Compute service failed to delete the instance because the VIF couldn't be deleted?17:18
rloodtantsur: it is on my list. i was deciding whether to do that or Julia's first. I can still be swayed :)17:18
dtantsurrloo: Julia's goes first IMO17:18
rloodtantsur: are you trying to do a release today or tomorrow?17:18
dtantsurtomorrow morning (my) is fine17:19
rloooh, i think staging drivers will break. someone needs to add timeout parameter.17:19
dtantsur*I* will not do a release today anyway, a bit too late. Julia may. if she does not - I will do tomorrow.17:19
rloopas-ha: ^^ wrt staging drivers. i'll try to push up a patch later today after reviewin17:20
rloodtantsur, TheJulia: let's do it tomorrow then :)17:20
TheJuliathen both I think :)17:20
dtantsurIPA can go in today, I suspect, when everything merges17:21
rpiosoGood morning, all17:22
*** vsaienk0 has joined #openstack-ironic17:24
*** AlexeyAbashkin has quit IRC17:26
*** tesseract has quit IRC17:27
*** crushil_ has quit IRC17:28
dtantsurmorning rpioso17:28
rpiosoHey dtantsur :)17:29
openstackgerritDmitry Tantsur proposed openstack/ironic master: Clean up release notes before a release  https://review.openstack.org/54174417:31
dtantsurrloo: let's try this ^^^17:31
dtantsurcan someone please take over this patch and address the comments, if any?17:32
* rloo tries to be happy about that17:32
dtantsurI have to go and we need it landed before branching17:32
rloodtantsur: yup, two things. the renos and that mimetype patch of yours. don't worry. we'll take care of them.17:32
dtantsurcool, thanks!17:33
*** dtantsur is now known as dtantsur|afk17:33
dtantsur|afko/17:33
*** absubram has joined #openstack-ironic17:33
rloobye dtantsur|afk!17:33
*** vsaienk0 has quit IRC17:33
NobodyCamhave a good night dtantsur|afk17:36
openstackgerritMerged openstack/bifrost master: Fix for proper deployment when keystone enabled and minimal OS  https://review.openstack.org/54071317:39
rpiosoGood evening, dtantsur|afk17:39
*** baha has joined #openstack-ironic17:39
*** baha1 has joined #openstack-ironic17:40
openstackgerritJulia Kreger proposed openstack/ironic master: Don't try to lock for vif detach  https://review.openstack.org/53444117:41
TheJuliarloo: sambetts ^^^17:41
rlooTheJulia: I'm still working on the previous revision. stuck on the unit tests. (WHICH I HATE)17:41
TheJulia:(17:42
*** baha has quit IRC17:42
*** baha1 is now known as baha17:42
rlooTheJulia: commented, I am not convinced the unit tests are correct :-(17:47
TheJuliaugh17:48
rlooTheJulia: this is why it takes me so long to review. I don't like reviewing unit tests, so i have to force myself to actually look and understand them.17:48
TheJuliayay scope creep17:48
*** snasir has quit IRC17:49
*** crushil has joined #openstack-ironic17:49
*** crushil_ has joined #openstack-ironic17:49
TheJuliamakes sense to be very explicit in-line and document it17:50
TheJuliaI'm not sure I'm groking your test comments17:51
rlooTheJulia: which ones? all of them?17:52
openstackgerritMerged openstack/ironic master: Add 10.1 and queens to the release mapping  https://review.openstack.org/54169417:52
TheJuliacommented17:53
TheJuliatl;dr is the reservation is present and the test ensures that that the task still executes when the node is locked17:54
TheJuliaas it would be during a long running task17:54
TheJuliawoot, it looks like the multinode test works, it is actually upgrading17:55
sambetts\o/17:56
rlooTheJulia: ah, i commented -- maybe rename the test then. and a comment. or maybe it was just me that didn't grok it17:56
rlooTheJulia: YAY wrt multinode test!17:56
TheJuliaI can rename it17:56
TheJuliaI have a meeting in a couple minutes, will try to push a new version soon17:57
rlooTheJulia: no worries. we have time.17:57
rloowhen did grenade start failing in pike? https://review.openstack.org/#/c/541703/17:58
patchbotpatch 541703 - ironic (stable/pike) - Do not pass credentials to the ramdisk on cleaning17:58
rloomultinode grenade. did we just break something?17:58
*** derekh has quit IRC17:59
*** olivierbourdon38 has quit IRC17:59
*** e0ne has joined #openstack-ironic18:04
*** crushil_ has quit IRC18:06
TheJuliawelp, looks like no meeting18:08
* TheJulia finishes updates18:08
*** mgoddard_ has quit IRC18:08
*** jaganathan has joined #openstack-ironic18:09
TheJuliarloo: re test name, I would interpret it that the test is trying to obtain an exclusive lock, I'll try to think of something more clear18:10
*** devananda has quit IRC18:10
rlooTheJulia: 'foo' is ok with me, as long as there is a comment :) Long test names aren't that friendly either.18:10
*** jaganathan has quit IRC18:10
*** harlowja has joined #openstack-ironic18:11
TheJuliarloo: heh, yeah :) Call it purple!18:13
rlooTheJulia: :D18:13
* TheJulia feels an april fools prank coming on18:13
*** harlowja has quit IRC18:18
TheJuliarloo: I believe that would be the first instance I've seen of it fail on pike18:18
TheJuliawell, recently18:18
*** mjturek has quit IRC18:18
TheJuliarloo: unit tests running.....18:18
*** baha has quit IRC18:18
rlooTheJulia: i am wondering if I should just do a recheck. my other thought is the change we just did to fix the ironic-tempest-plugin issue. but i haven't actually looked into it18:19
TheJuliadmitry already did18:19
TheJuliarecheck that is18:20
TheJuliaIt should still be looked at though18:20
rlooTheJulia: oh good (about the recheck)18:20
openstackgerritMerged openstack/ironic stable/ocata: Do not pass credentials to the ramdisk on cleaning  https://review.openstack.org/54170418:20
rlooTheJulia: I just looked at dmitry's https://review.openstack.org/#/c/541289/. If you have a chance, would you look. I'm now worried about merging it in w/o more discussion18:20
patchbotpatch 541289 - ironic - Stop guessing mime types based on URLs18:20
TheJuliaugh18:21
TheJuliainternal server errors now on grenade18:21
* TheJulia wonders if we ran out of memory18:21
rlooTheJulia: :-((((((18:21
*** etingof has quit IRC18:21
TheJuliasegfaulting python is back!18:23
TheJuliahttp://logs.openstack.org/55/541755/4/check/ironic-grenade-dsvm-multinode-multitenant/9453b6b/logs/syslog.txt.gz#_Feb_07_18_05_5218:23
openstackgerritJulia Kreger proposed openstack/ironic master: Don't try to lock for vif detach  https://review.openstack.org/53444118:24
TheJuliarloo: http://logs.openstack.org/03/541703/1/gate/ironic-grenade-dsvm-multinode-multitenant/abbf674/job-output.txt.gz#_2018-02-07_17_29_40_332690 is quite possibly due to something like vifs not being unlocked in time or just performance issues (this was on stable/pike)18:26
rlooTheJulia: we will want to backport your vif change18:27
*** r-daneel_ has joined #openstack-ironic18:28
rlooTheJulia: the segfaulting python is the bigger worry now.18:28
TheJuliaI'm going to stack my change to re-work the upgrade on top of the partial grenade revert since segfaulting python18:28
TheJuliaoh18:28
TheJuliathat required nova to land a patch18:29
*** r-daneel has quit IRC18:29
*** r-daneel_ is now known as r-daneel18:29
*** rpioso is now known as rpioso|eat18:31
* rpioso|eat needs to carbo-load for cleaning up today's snowfall.18:32
*** baha has joined #openstack-ironic18:32
jlvillalTheJulia, Any objection to me +A on https://review.openstack.org/#/c/541755/  ?18:38
patchbotpatch 541755 - ironic - Revert grenade jobs to classic drivers18:38
*** etingof has joined #openstack-ironic18:39
openstackgerritSam Betts proposed openstack/ironic master: Ensure we unbind flat network ports and clear BM mac addresses  https://review.openstack.org/53998118:41
* sambetts ---> home 18:41
sambettsnight all o/18:41
sambettscya'll tomorrow18:41
*** sambetts is now known as sambetts|afk18:41
TheJuliajlvillal: none18:42
openstackgerritJulia Kreger proposed openstack/ironic master: Rework upgrade to restart nova  https://review.openstack.org/53559618:42
TheJuliajlvillal: rloo: I have no idea if ^^^^ is going to fix our nova pains. Another thought... we could run a nova-compute and nova-conductor on both nodes...18:43
TheJuliaif the primary node's install dies during the upgrade.. *shrug*18:43
TheJuliathoughts?18:43
* rloo thinks that rolling upgrades was not worth it18:44
TheJuliait is actually kind of a good test in that it is hte most possible racey test we could have in the gate where everything could possibly go wrong!18:45
rlooTheJulia: yes, it is a great test. Too bad it fails though :D18:46
TheJuliaheh18:46
jlvillalTheJulia, As a note, multinode still fails with: https://review.openstack.org/#/c/541755/18:46
patchbotpatch 541755 - ironic - Revert grenade jobs to classic drivers18:46
jlvillalTheJulia, I noticed your patch on top of it.18:46
TheJuliayup18:46
TheJulianow nova-conductor segfaults!18:46
TheJulia*dance*18:47
jlvillalTheJulia, Ugh18:47
*** shekarvajjula has quit IRC18:48
rlooTheJulia: this is just for the normal grenade, not multinode: https://review.openstack.org/#/c/535596/6/devstack/upgrade/upgrade.sh18:48
patchbotpatch 535596 - ironic - Rework upgrade to restart nova18:48
*** shekarvajjula has joined #openstack-ironic18:48
openstackgerritHarald Jensås proposed openstack/networking-baremetal master: DNM - testing tripleo  https://review.openstack.org/54170518:48
rlooTheJulia: is normal grenade failing too?18:49
TheJuliarloo: no18:50
TheJuliadoh18:50
TheJuliaERRTOOMUCHMULTITASKING18:50
rlooTheJulia: oh. ok then, that explains why I'm confused. Yeah, I think it is ok to go a bit slower. i'm good if we release and multinode grenade is still failing. as long as we address in next week or so.18:51
rlooTheJulia: otherwise you're gonna get really burned out and we may make mistakes. NOT that we haven't already...18:52
TheJuliaI'm updating the patch now18:52
*** mgoddard_ has joined #openstack-ironic18:53
openstackgerritJulia Kreger proposed openstack/ironic master: Rework upgrade to restart nova  https://review.openstack.org/53559618:54
TheJuliaI'm hoping to take the next two days off, personally18:54
rlooTheJulia: good!18:56
*** mjturek has joined #openstack-ironic18:58
rlooTheJulia: wrt 535596. we haven't upgraded nova, just restarting. there could be a problem if the old nova doesn't work with new packages, right?18:58
TheJuliaanyway, crossing my fingers that that finally gets that job working18:58
TheJuliarloo: absolutely18:58
*** e0ne has quit IRC18:59
rlooTheJulia: so they may not have been in a broken state, but could be in a broken state after restarting. Or they could have been in a broken state, and work after restarting :-(18:59
rlooTheJulia: or they could be broken regardless ...19:00
TheJuliayup yup yup19:00
TheJuliagrenade: why we can't have nice things19:00
rlooWHO runs all their services on the same node and does an upgrade of them like grenade does it...19:00
TheJuliagreat question....19:01
rlooTheJulia: can we go on strike? No rolling upgrade testing if our test framework doesn't make sense?19:01
TheJuliarloo: works for me19:01
* TheJulia will bring coffee, and make signs19:01
rlooTheJulia: ok, two +2, +A. Done. something to discuss at PTG?19:01
TheJuliaand tea, lots of tea19:01
rlooTheJulia: how does anyone else test rolling upgrades?19:02
rlooPTG topic i think...19:02
TheJuliayeah...19:02
TheJuliaWe have tons of duplication too... I feel like we could almost use a "Here I'm running xyz s19:02
TheJuliaerr, xyz scenarios" and "here is upgrade testing"19:03
TheJuliaand have half the jobs19:03
rlooTheJulia: sigh. i'd say a ptg topic but we always talk about it, then it sits in subteam thing and we never raise priority so nothing gets done to try to combine.19:04
rlooTheJulia: i hear you might be the next PTL. Thanks for looking into ^^ :)19:04
TheJuliaheh :)19:06
*** Goneri has quit IRC19:09
*** e0ne has joined #openstack-ironic19:10
*** Goneri has joined #openstack-ironic19:10
*** mjturek has quit IRC19:11
jrollTheJulia: ++ on removing duplication19:12
jrollthat was part of the intention with standalone - test most things in isolation there, do minimal full stack testign19:12
jrollwe did the first half...19:13
*** r-daneel_ has joined #openstack-ironic19:13
*** r-daneel has quit IRC19:14
*** r-daneel_ is now known as r-daneel19:14
TheJuliayeah19:19
*** mjturek has joined #openstack-ironic19:20
*** fragatina has joined #openstack-ironic19:21
rloojroll: since you reviewed it, I commented. If you have thoughts. https://review.openstack.org/#/c/541289/19:24
patchbotpatch 541289 - ironic - Stop guessing mime types based on URLs19:24
jrollrloo: replied19:32
*** lucasagomes is now known as lucas-afk19:33
rloojroll: thx :)19:33
openstackgerritOlivier Bourdon proposed openstack/bifrost master: Fix extra double-quote  https://review.openstack.org/54169619:33
openstackgerritMerged openstack/ironic stable/pike: Use zuul.override_checkout  instead of custom branch_override var  https://review.openstack.org/54082919:34
openstackgerritMerged openstack/ironic-python-agent master: Replace curly quotes with straight quotes  https://review.openstack.org/53796819:34
openstackgerritMerged openstack/ironic stable/ocata: Use zuul.override_checkout  instead of custom branch_override var  https://review.openstack.org/54083019:34
*** mgoddard_ has quit IRC19:35
*** AlexeyAbashkin has joined #openstack-ironic19:37
rloooh, need another core to review this. https://review.openstack.org/#/c/534441/19:39
patchbotpatch 534441 - ironic - Don't try to lock for vif detach19:39
rloojlvillal, vdrok: you feel like reviewing ^^19:40
openstackgerritMerged openstack/ironic master: Handle case when a glance image contains no data  https://review.openstack.org/53118019:40
openstackgerritMerged openstack/ironic master: Update iRMC document for classic driver deprecation  https://review.openstack.org/54117119:40
*** mjturek has quit IRC19:41
*** mjturek has joined #openstack-ironic19:42
*** mjturek has quit IRC19:42
*** rpioso|eat is now known as rpioso19:42
vdrokWill take a look in a bit19:47
*** pcaruana has joined #openstack-ironic19:52
TheJuliarloo: posted a quick reply to one, the spelling mistake in the reno we should just fix before +A'ing, anyway will let vdrok review it19:59
rlooTheJulia: works for me20:00
rlooTheJulia, jlvillal, vdrok: the last 'big' patch for the release (I hope): https://review.openstack.org/#/c/541289/20:03
patchbotpatch 541289 - ironic - Stop guessing mime types based on URLs20:03
* rloo found out that eating Ben&Jerry icecream while reviewing made the experience that much better. Until the icecream was finished...20:04
* persia ponders the advantages of remunerating reviewers with endless ice cream conveyors, and how that affects freight charges for PTG attendance20:05
rloo:D20:06
*** pcaruana has quit IRC20:07
openstackgerritMerged openstack/ironic-inspector stable/queens: Update .gitreview for stable/queens  https://review.openstack.org/54174820:12
TheJuliarloo: lol20:14
TheJuliapersia: It is not a bad idea... as long as we have time to also exercise such that the nonexistant ice cream calories are not counted against us when we visit the doctor20:15
openstackgerritRuby Loo proposed openstack/ironic master: Clean up release notes before a release  https://review.openstack.org/54174420:19
*** fragatina has quit IRC20:20
persiaTheJulia: Ah, so the ice cream conveyor is powered by a treadmill at a standing desk?20:23
TheJuliapersia: Indeed, excess energy can also be leveraged to power CI20:24
persia:)20:24
TheJuliaThat would actually be something I would exercise for....  faster running CI20:24
*** jaganathan has joined #openstack-ironic20:26
openstackgerritRuby Loo proposed openstack/ironic-inspector master: Update reno for stable/queens  https://review.openstack.org/54175020:27
*** hw_wutianwei has quit IRC20:31
*** hw_wutianwei has joined #openstack-ironic20:32
*** AlexeyAbashkin has quit IRC20:33
*** mjturek has joined #openstack-ironic20:34
rloowow, this.is.an.old.patch: https://review.openstack.org/#/c/128701/20:34
patchbotpatch 128701 - ironic-python-agent - Refactor image writing20:34
TheJuliawow, yeah20:37
*** mjturek has quit IRC20:38
*** mjturek has joined #openstack-ironic20:39
rlooTheJulia: wrt 541289. I'm good with Closes-Bug, cuz I think we'll need an RFE to do the version thing, cuz it needs a version bump. So we coul use the same bug or a new one for that.20:41
rlooTheJulia: so we could edit it to be Partial if you want. I don't really care. It actually does fix the issue. except for that case with test & test.json.20:41
rlooTheJulia: I do think it is worth editing the reno to mention heartbeat endpoint. So i can update, just want to know if you want me to close the bug or not.20:43
TheJuliaI think it still falls under a bug, since it an be a bug and force a version to be updated by the very nature of bugs. I was just thinking from a tracking standpoint, but who knows what tomorrow brings so closes is also fine with me20:51
rlooTheJulia: thx, am going to update the reno and then approve.20:52
TheJuliawfm!20:52
TheJuliathanks!20:52
openstackgerritMerged openstack/ironic master: Revert grenade jobs to classic drivers  https://review.openstack.org/54175520:58
openstackgerritRuby Loo proposed openstack/ironic master: Stop guessing mime types based on URLs  https://review.openstack.org/54128920:58
*** mrunge_ has joined #openstack-ironic21:02
*** mrunge has quit IRC21:03
*** crushil has quit IRC21:12
openstackgerritVladyslav Drok proposed openstack/ironic master: Remove unnecessary lines from sample local.conf  https://review.openstack.org/54190021:12
openstackgerritVladyslav Drok proposed openstack/ironic master: Remove unnecessary lines from sample local.conf  https://review.openstack.org/54190021:14
*** mjturek has quit IRC21:16
TheJuliasigh, the sqlalchemy issue prevents us just restarting21:19
TheJuliaI suspect the only viable option is to just run a conductor and compute process on the second node.... and leave them running. We might have to shift some timeouts....21:19
vdrokTheJulia: left a question on detach patch21:20
TheJulialooking21:21
*** crushil has joined #openstack-ironic21:22
*** e0ne has quit IRC21:23
*** e0ne has joined #openstack-ironic21:27
TheJuliavdrok: replied21:27
openstackgerritMerged openstack/bifrost master: Fix extra double-quote  https://review.openstack.org/54169621:34
rpiosoPlease explain the difference between a Code-Review:-1 and Code-Review:-2.21:37
*** threestrands has joined #openstack-ironic21:39
jrollrpioso: -1: should not be merged now, or should not be merged in its current state. -2: this should never be merged, or I'm making it 100% sure that this does not get merged without my knowledge.21:39
jrollthat's how I tend to see it anyway21:39
jroll-2 is typically a procedural block, in short21:39
rpiosojroll: Thank you :)21:40
jrollrpioso: no problem. just so you know, -2s persist between patchsets, -1s do not. that's why -2 is often used for procedural things21:40
rpiosojroll: Interesting. I didn't realize that.21:41
TheJuliarpioso: For what it is worth, it is not uncommon for us to forget -2's, so if we need to get one removed just raise visibility21:44
rpiosoTheJulia: Thank you21:44
jroll++21:45
openstackgerritMerged openstack/ironic-tempest-plugin master: Switch to override-checkout for stable branch jobs  https://review.openstack.org/54035521:45
openstackgerritMerged openstack/ironic-python-agent master: When ouputting to /dev/null should open in write mode  https://review.openstack.org/54158421:45
rpiosojroll, TheJulia: Could you give a couple of examples that would cause you to assign -2?21:46
jrollrpioso: the most common is because the spec isn't merged21:46
* rpioso is not clear on procedural things21:46
jrollor rfe isn't approved21:46
rpiosojroll: So, not the quality of the code?21:47
TheJuliarpioso: no, not the quality of the code at all21:47
jrollrpioso: another interesting case is like... a patch is ready to merge, but it makes huge changes and we shouldn't merge it the day before feature freeze21:47
jrollnope21:47
rpiosoTheJulia, jroll: That helps. Thanks, again.21:48
jrollyou're welcome :)21:48
TheJuliaAnother case is when there is something breaking that has a +2's already and we need to block something from moving forward until we are able to discuss and resolve the issue.21:49
*** ijw has quit IRC21:51
rpiosoTheJulia: That makes sense, too.21:54
*** ijw has joined #openstack-ironic21:54
rlooso I think everything that we wanted to land, has landed or in the process of landing. (and this is probably easier than landing on the moon). the only outstanding thing is fixing multinode grenade.21:57
TheJuliarloo: I'm going to ponder it after running to the store21:58
TheJuliasince I think refreshment is needed21:58
*** Sukhdev has joined #openstack-ironic21:59
rlooTheJulia:  Enjoy, and don't worry about it. You've gone beyond the call of duty. The rest can wait. (Sez me!)21:59
TheJuliamaybe trying to launch something towards a another planetary system in kerbal space program21:59
TheJuliao/ Sukhdev22:00
SukhdevTheJulia Hi22:00
TheJuliarpioso: So the question comes to mind, have you encountered a -2 that needs attention?22:00
rpiosoTheJulia: Yes, but not in ironic. I'm a core in python-dracclient. I was not clear on the difference between the two.22:02
rpiosoTheJulia: Now I am :)22:02
TheJuliaAhh ok :)22:04
rloojroll: is this needed/wanted? https://review.openstack.org/#/c/540359/22:07
patchbotpatch 540359 - ironic - Don't use hardware types on Ocata CI22:07
*** r-daneel has quit IRC22:08
*** rbudden has quit IRC22:11
*** devananda has joined #openstack-ironic22:16
*** fragatina has joined #openstack-ironic22:19
openstackgerritMerged openstack/bifrost master: Fedora 25-27 support  https://review.openstack.org/47175022:20
*** fragatina has quit IRC22:20
*** fragatina has joined #openstack-ironic22:20
*** fragatina has quit IRC22:20
*** fragatina has joined #openstack-ironic22:21
*** fragatina has quit IRC22:21
*** fragatina has joined #openstack-ironic22:21
*** fragatina has quit IRC22:21
*** fragatina has joined #openstack-ironic22:22
*** rcernin has joined #openstack-ironic22:25
*** jlabarre has quit IRC22:32
*** crushil has quit IRC22:35
*** baha has quit IRC22:39
jrollrloo: good question, I need to check if sam's change fixed that CI22:44
*** e0ne has quit IRC22:47
rloojroll: it looks useful but sam's changes did fix the ironic-tempest-plugin tests22:48
jrollrloo: ok, it isn't needed then, I'll abandon. thanks22:49
*** Goneri has quit IRC22:50
rloojroll: ok thx!22:50
jroll:)22:50
*** crushil has joined #openstack-ironic22:51
*** pmannidi has joined #openstack-ironic22:55
*** crushil has quit IRC23:00
*** rbudden has joined #openstack-ironic23:03
*** edleafe- has joined #openstack-ironic23:06
*** edleafe has quit IRC23:08
openstackgerritMerged openstack/ironic master: Clean up release notes before a release  https://review.openstack.org/54174423:09
*** edleafe- is now known as edleafe23:10
*** edleafe has quit IRC23:21
*** Sukhdev has quit IRC23:23
*** edleafe has joined #openstack-ironic23:24
*** masber has joined #openstack-ironic23:27
*** Sukhdev has joined #openstack-ironic23:36
*** MattMan has quit IRC23:37
*** MattMan has joined #openstack-ironic23:38

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!