Thursday, 2019-03-28

*** betherly has joined #openstack-ironic00:28
*** betherly has quit IRC00:33
*** tiendc has joined #openstack-ironic01:00
*** betherly has joined #openstack-ironic01:01
*** betherly has quit IRC01:05
*** rloo has quit IRC01:08
*** gyee has quit IRC01:16
*** betherly has joined #openstack-ironic01:33
*** betherly has quit IRC01:37
*** jhesketh has joined #openstack-ironic02:12
openstackgerritKaifeng Wang proposed openstack/ironic master: Workaround for sendfile size limit  https://review.openstack.org/62211402:15
*** betherly has joined #openstack-ironic02:15
*** betherly has quit IRC02:20
*** dsneddon has quit IRC02:22
*** rh-jelabarre has quit IRC02:25
openstackgerritUros Orozel proposed openstack/ironic master: UEFI iPXE provisioning fails when ipxe_use_swift is enabled  https://review.openstack.org/64807402:34
*** dsneddon has joined #openstack-ironic02:48
*** whoami-rajat has joined #openstack-ironic02:58
*** betherly has joined #openstack-ironic03:08
*** betherly has quit IRC03:13
*** betherly has joined #openstack-ironic04:02
*** betherly has quit IRC04:07
*** Lucas_Gray has joined #openstack-ironic04:14
*** Lucas_Gray has quit IRC04:43
*** Lucas_Gray has joined #openstack-ironic04:45
*** rcernin has quit IRC04:47
*** mkrai has joined #openstack-ironic04:48
*** betherly has joined #openstack-ironic04:55
*** betherly has quit IRC04:59
*** rcernin has joined #openstack-ironic05:01
*** whoami-rajat has quit IRC05:07
*** jaypipes has quit IRC05:09
*** jaypipes has joined #openstack-ironic05:09
*** stendulker has joined #openstack-ironic05:11
openstackgerritNisha Agarwal proposed openstack/ironic master: RPC changes for Graphical Console  https://review.openstack.org/64082705:11
*** Lucas_Gray has quit IRC05:22
openstackgerritDigambar proposed openstack/ironic master: Modify the iDRAC driver to use realtime RAID creation  https://review.openstack.org/63490305:31
*** rachit7 has joined #openstack-ironic05:34
*** betherly has joined #openstack-ironic05:48
*** betherly has quit IRC05:52
*** rcernin has quit IRC06:03
*** whoami-rajat has joined #openstack-ironic06:31
*** betherly has joined #openstack-ironic06:40
*** betherly has quit IRC06:45
*** rcernin has joined #openstack-ironic06:47
*** rcernin has quit IRC06:47
*** rcernin has joined #openstack-ironic06:48
openstackgerritKaifeng Wang proposed openstack/ironic master: Workaround for sendfile size limit  https://review.openstack.org/62211407:03
*** e0ne has joined #openstack-ironic07:17
*** e0ne has quit IRC07:19
*** pcaruana has joined #openstack-ironic07:20
arne_wiebalckgood morning, ironic!07:33
*** rpittau|afk is now known as rpittau07:41
rpittaugood morning ironic! o/07:41
*** rcernin has quit IRC07:48
arne_wiebalckgood morning rpittau o/07:50
rpittauhey arne_wiebalck :)07:50
*** tssurya has joined #openstack-ironic08:03
*** rcernin has joined #openstack-ironic08:04
*** betherly has joined #openstack-ironic08:06
*** iurygregory_ has joined #openstack-ironic08:10
*** betherly has quit IRC08:11
*** iurygregory has quit IRC08:12
*** iurygregory_ is now known as iurygregory08:12
*** Lucas_Gray has joined #openstack-ironic08:13
*** rachit7 has quit IRC08:15
iurygregorymorning ironic o/08:22
rpittauhey iurygregory :)08:22
mkraiGM Ironic!08:54
*** Wryhder has joined #openstack-ironic08:54
*** Lucas_Gray has quit IRC08:57
*** Wryhder is now known as Lucas_Gray08:57
mkraiHi, is there any API in Nova that can map to our cleaning API?08:58
*** betherly has joined #openstack-ironic08:59
*** betherly has quit IRC09:04
*** S4ren has joined #openstack-ironic09:06
arne_wiebalckmkrai: What do you mean, trigger ironic cleaning via nova?09:06
mkraiarne_wiebalck: Right09:08
arne_wiebalckmkrai: delete in nova will trigger a clean in ironic, no?09:10
*** oanson has quit IRC09:10
mkraiarne_wiebalck: Explicitly cleaning only09:10
arne_wiebalckmkrai: like you have a node in ironic which does not have an instance and you'd like to clean it via nova?09:11
mkraiarne_wiebalck: Is it possible?09:13
mkraiNova would not know about the Ironic node in that case09:13
arne_wiebalckmkrai: that would have been my next question: how do you refer to an ironic node from nova?09:14
mkraiarne_wiebalck: Actually I am working on this feature which is manual deployment step09:14
mkraiWhich will be similar to clean API specifying deploy_steps09:14
*** diga has joined #openstack-ironic09:14
mkraiSo to rephrase my question, I am thinking how would we expose the manual deploy API from Nova09:15
mkraiarne_wiebalck: Sorry for the confusion09:15
digaHello Ironic09:16
digaarne_wiebalck rpittau iurygregory:  Hello09:17
iurygregoryo/09:17
arne_wiebalckmkrai: mgoddard would be the one to talk to about this09:18
mkraiarne_wiebalck: Ok thank you for your response :)09:18
mkraimgoddard: Hi!09:18
arne_wiebalckmkrai: FWIU, the deploy step framework will be more or less in analogy how the clean steps are done09:19
openstackgerritMark Goddard proposed openstack/bifrost stable/rocky: Use a virtualenv in openSUSE CI testing  https://review.openstack.org/64822109:19
rpittauhi diga :)09:19
openstackgerritMark Goddard proposed openstack/bifrost stable/rocky: Use a virtualenv in openSUSE CI testing  https://review.openstack.org/64822109:19
arne_wiebalckmkrai: but users will have the possibility to execute certain deploy steps when deploying an instance09:19
* arne_wiebalck better leaves the floor to mgoddard 09:20
arne_wiebalckhi diga09:20
digaarne_wiebalck: Hey09:20
mkraiarne_wiebalck: The use case is allow deploy steps after the node is provisioned09:20
mgoddardhi mkrai arne_wiebalck09:20
mkraiAnd the deploy template works while provisioning only09:20
mkraiHi mgoddard09:21
arne_wiebalckmkrai: yes, that's also a use case, I have a story for it ...09:21
digaarne_wiebalck:  I updated my patch, not sure reason behind why zuul is failing - https://review.openstack.org/#/c/634903/09:21
patchbotpatch 634903 - ironic - Modify the iDRAC driver to use realtime RAID creation - 11 patch sets09:21
* arne_wiebalck is looking for the active node deploy step story ...09:21
mkraimgoddard: I was discussing about the manual deploy step feature with arne_wiebalck09:21
digado I need recheck ?09:22
arne_wiebalckmkrai: https://storyboard.openstack.org/#!/story/200512909:22
mgoddardmkrai: just catching up09:22
mkraiarne_wiebalck: Here it is https://storyboard.openstack.org/#!/story/200512909:22
mgoddardmkrai: we don't really expose anything too baremetal-specific in nova09:22
mkraimgoddard: Ok I agree09:23
mgoddardmkrai: instead we try to abstract things to use the same interfaces as VMs09:23
mkraimgoddard: Yes so I am looking for similar interface in Nova09:23
*** betherly has joined #openstack-ironic09:23
mgoddardmkrai: this is for during deployment?09:23
mkraimgoddard: post deployment09:24
mgoddardmkrai: what is the use case?09:24
arne_wiebalckmkrai: coud you provide a use case example?09:24
mkraiarne_wiebalck: mgoddard Yes09:24
rpittaudiga: you can recheck, that error unfortunately sometimes happen in that job09:24
mkraiRun a deploy_step "do_nothing" post deployment via Nova09:24
*** lucasagomes has joined #openstack-ironic09:25
mgoddardmkrai: that doesn't sound very useful :p09:25
mkraithe do_nothing can be changing a boot parameter post deployment which might need a power recycle as well09:25
mkraimgoddard: Trying to elaborate :) Sorry09:25
mgoddardok, I think we have discussed this before09:25
arne_wiebalckmgoddard: mkrai : I think my story ^ has an example09:26
mkraimgoddard: Yes.09:26
mkraiCopying from the story,  A real-life example for such a required post-deploy configuration change is the change of BIOS settings to disable hyper-threading in order to address a security vulnerability.09:26
arne_wiebalckmgoddard: mkrai : changing BIOS settings post deployment09:26
digarpittau: Sure. Thanks man09:27
mgoddardthere is no such API currently. After ironic deploys an instance it does not really do much other than monitor and set the power state09:27
mkraimgoddard: Yes I know. I am talking about the post deploy step feature implementation :)09:27
*** betherly has quit IRC09:28
mgoddardI'm struggling to see how we would fit this into nova09:28
openstackgerritDigambar proposed openstack/ironic master: Modify the iDRAC driver to use realtime RAID creation  https://review.openstack.org/63490309:28
mkraimgoddard: If we implement this feature in Ironic. How do we expose via Nova?09:28
arne_wiebalckthe story does not suggest to expose this to nova09:28
mgoddardwe have deploy steps in ironic, could easily provide a list of steps to run09:28
mgoddardbut I don't see how we would map that to a concept in nova09:28
mkraimgoddard: I tried looking for similar interface in Nova and think rebuild might be a possible option09:29
arne_wiebalckmkrai: mgoddard: I don't think this should be exposed via nova.09:29
mgoddardmkrai: you could rebuild, but it would overwrite the image on the disk09:30
arne_wiebalckI am not sure the owner of an instance should have the rights to reconfigure the BIOS.09:30
mgoddardarne_wiebalck: with deploy templates they implicitly could do09:31
arne_wiebalckmgoddard: right, but in a controlled way09:31
mgoddardyes09:31
arne_wiebalckit's probably a grey area09:31
arne_wiebalckin our cloud we basically hand the instance over to the user09:31
arne_wiebalckthe user can do anything with it09:32
mgoddardeven the BMC?09:32
arne_wiebalckyes09:32
arne_wiebalckmostly for "backwards compatibility"09:32
mgoddarddo you reimage the BMC firmware afterwards?09:32
arne_wiebalckwe should :D09:32
arne_wiebalckwe plan to do that, yes09:33
mgoddardor could check checksums09:33
arne_wiebalckatm there are other DBs keeping track of credentials as well, so we need to make sure these are updated once we reset things09:33
arne_wiebalckwe plan to count the number of admins accounts and reset the pw09:34
arne_wiebalckso, in principle users can already now change BIOS settings (in our cloud)09:35
arne_wiebalckthe story I created was more for fleet-wide campaigns09:35
arne_wiebalckchange the BIOS on 1000 nodes09:35
mkraiarne_wiebalck: mgoddard Thank you for the discussion :)09:36
arne_wiebalckanyway, I think it'll be hard to do that via nova09:36
*** derekh has joined #openstack-ironic09:37
*** oanson has joined #openstack-ironic09:41
*** e0ne has joined #openstack-ironic09:47
digadtantsur|afk: Hi09:54
digaohh. looks like he is away09:54
*** priteau has joined #openstack-ironic09:55
openstackgerritDigambar proposed openstack/ironic stable/rocky: [WIP] Remove nic method dependencies from dracclient==1.5.0 from stable releases  https://review.openstack.org/64836009:55
*** dtantsur|afk is now known as dtantsur09:59
dtantsurhi diga10:00
dtantsurmorning ironic10:00
rpittauhi dtantsur :)10:01
dtantsureandersson: hi! the reasoning was that each hardware type has someone responsible for it (community for ipmi, <vendor name> for <vendor type>), etc. and we wanted these people to define what is compatible with what hardware type.10:01
dtantsurI do agree that networking interfaces are weird citizens here10:01
digadtantsur: Hi10:02
dtantsurI don't mind opening this topic again, but back then some people felt very strongly10:02
digadtantsur: I just pushed patch for removing dracclient 1.5.0 dependency from ironic stable version - https://review.openstack.org/#/c/648360/10:02
patchbotpatch 648360 - ironic (stable/rocky) - [WIP] Remove nic method dependencies from dracclie... - 1 patch set10:02
digadtantsur: Its in WIP now, working on unittests10:03
dtantsurgreat! I'll check it soon10:03
diga:)10:03
iurygregorymorning dtantsur10:17
dtantsurmgoddard: hi, did you have a chance to check my questions on https://review.openstack.org/#/c/647485/ ?10:18
patchbotpatch 647485 - ironic - Check for deploy.deploy deploy step in heartbeat - 1 patch set10:18
mgoddardsorry dtantsur, got lost in the noise. Looking now10:19
mgoddarddtantsur: I'm not sure about touching provisioning. I don't think it will hurt, but I also don't think it affects timeout, does it?10:23
dtantsurmgoddard: I think we use this to determine if IPA is still alive for a node. I may be horribly wrong though.10:23
mgoddarddtantsur: ok, you are right - deploy timeout is CONF.conductor.deploy_callback_timeout seconds since last provision_updated_at10:26
*** melwitt has quit IRC10:26
mgoddarddtantsur: so the missing case is when in DEPLOYWAIT and not self.in_deploy_deploy_step(task)?10:27
dtantsurmgoddard: yep, I think so10:29
mgoddarddtantsur: ack10:29
*** e0ne has quit IRC10:31
*** melwitt has joined #openstack-ironic10:32
*** e0ne has joined #openstack-ironic10:36
*** hoonetorg has quit IRC10:39
openstackgerritMark Goddard proposed openstack/ironic master: Check for deploy.deploy deploy step in heartbeat  https://review.openstack.org/64748510:41
*** mkrai has quit IRC10:42
openstackgerritMerged openstack/tenks master: Fix Zuul galaxy install task  https://review.openstack.org/64818510:52
*** hoonetorg has joined #openstack-ironic10:52
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Allow backfilling/assigning allocations  https://review.openstack.org/64817810:54
*** tiendc has quit IRC10:54
dtantsurmgoddard: sorry, another question10:58
*** e0ne has quit IRC10:58
mgoddarddtantsur: fire away10:59
*** e0ne has joined #openstack-ironic11:05
mgoddarddtantsur: good catch11:06
openstackgerritMark Goddard proposed openstack/ironic master: Check for deploy.deploy deploy step in heartbeat  https://review.openstack.org/64748511:06
mgoddardSo many values for a bool, it's sometimes hard to know which one to pick11:07
openstackgerritMerged openstack/tenks master: Ignore COMPUTE_NET_ATTACH_INTERFACE trait when waiting for placement  https://review.openstack.org/64814411:08
*** stendulker has quit IRC11:11
*** lennyb has quit IRC11:12
*** lennyb has joined #openstack-ironic11:14
dtantsurlol11:17
*** diga has quit IRC11:29
openstackgerritMerged openstack/sushy stable/stein: Change sushy devstack job to python3  https://review.openstack.org/64804511:30
*** Lucas_Gray has quit IRC11:37
*** w14161_1_ has quit IRC11:40
*** Lucas_Gray has joined #openstack-ironic11:49
*** jtomasek has joined #openstack-ironic11:51
*** cdearborn has quit IRC11:54
openstackgerritArne Wiebalck proposed openstack/ironic-specs master: Add details on target_raid_config for Software RAID support  https://review.openstack.org/64310011:56
*** jtomasek has quit IRC12:00
*** rh-jelabarre has joined #openstack-ironic12:01
*** Lucas_Gray has quit IRC12:09
*** Wryhder has joined #openstack-ironic12:09
*** whoami-rajat has quit IRC12:10
*** Wryhder is now known as Lucas_Gray12:10
*** trown|outtypewww is now known as trown12:11
openstackgerritIlya Etingof proposed openstack/ironic master: Add UEFI firmware to Redfish emulator config  https://review.openstack.org/64838212:17
*** whoami-rajat has joined #openstack-ironic12:20
*** w14161_1 has joined #openstack-ironic12:27
*** jtomasek has joined #openstack-ironic12:32
*** e0ne has quit IRC12:41
*** rloo has joined #openstack-ironic12:42
*** e0ne has joined #openstack-ironic12:43
TheJuliadtantsur|afk: any thoughts?12:45
TheJuliaerr, thoguhts on the discussion from last night re driver interfaces and overrides12:45
iurygregorydtantsur, the problem was that devstack-minimal was branched yesterday so they didnt have stable/stein   https://review.openstack.org/#/c/647774 https://zuul.openstack.org/status now shows ironicclient-functional and ironicclient-tempest \o/12:46
patchbotpatch 647774 - python-ironicclient (stable/stein) - Move to zuulv3 - 2 patch sets12:46
* TheJulia reads12:46
TheJuliaI suspect people feel less strongly at this poitn, fwiw12:46
*** jtomasek has quit IRC12:46
dtantsurTheJulia: oh, I posted something about it this morning. I feel less strongly as well, maybe we should relax restrictions at least around network/storage interfaces12:50
TheJuliaI think that is really reasonable12:53
*** jcoufal has joined #openstack-ironic12:53
dtantsurIt may be a bit funky from the coding point of view though :)12:57
jrollI agree allowing additional network/storage interfaces should be fine12:58
iurygregorymorning jroll o/13:05
iurygregorymorning TheJulia o/13:05
jrollmorning :)13:06
*** jtomasek has joined #openstack-ironic13:07
dtantsurmorning jroll13:08
jrollheya dtantsur \o13:08
*** jtomasek has quit IRC13:08
openstackgerritMerged openstack/ironic master: Workaround for sendfile size limit  https://review.openstack.org/62211413:09
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allocation API: backfilling allocations  https://review.openstack.org/64817813:10
dtantsurhuh, this ^^ proved bigger than I hoped for13:10
*** jtomasek has joined #openstack-ironic13:13
*** bfournie has joined #openstack-ironic13:18
*** jhesketh has quit IRC13:18
*** e0ne has quit IRC13:20
*** munimeha1 has joined #openstack-ironic13:20
*** derekh has quit IRC13:24
dtantsurTheJulia, jroll, please check https://review.openstack.org/#/c/645523/13:51
patchbotpatch 645523 - ironic (stable/stein) - Prepare stable/stein testing - 1 patch set13:51
*** baha has joined #openstack-ironic13:56
jrolldtantsur: +213:57
jrollthanks for doing that.13:57
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Allocation API: add tests on backfilling allocations  https://review.openstack.org/64840313:59
*** derekh has joined #openstack-ironic14:04
TheJuliadtantsur: thanks14:12
TheJuliatoo many meetings this week :(14:13
bdodddtantsur I added a reply to your comments in settings.py of my sushy patch set. If you and/or etingof could take a look and give me feedback it would be appreciated. https://review.openstack.org/#/c/643656/1/sushy/resources/settings.py14:13
patchbotpatch 643656 - sushy - Update model to support ApplyTime annotations - 1 patch set14:13
dtantsurwill do14:13
bdoddo/14:14
openstackgerritDmitry Tantsur proposed openstack/python-ironicclient master: Allocation API: support allocation backfilling  https://review.openstack.org/64840614:16
openstackgerritIlya Etingof proposed openstack/sushy-tools master: Fix Flask config load under WSGI  https://review.openstack.org/64840714:16
iurygregoryPeople, ironic can create nodes with windows images or nodes with dual boot (Linux/Windows)? my friend asked me this question and i told him would do some search XD14:19
dtantsuriurygregory: if it's a whole disk image, you can do anything (in theory)14:19
iurygregorydtantsur, ty!14:20
dtantsurI'd be curious to hear if they succeed or not :)14:20
*** jtomasek has quit IRC14:20
etingofbdodd's patch reminds me that sushy models are mostly suitable for GET'ing not for POST'ing...14:23
etingoffor example, ActionField object does not capture action parameters14:24
iurygregorydtantsur, I will let you know if he decides to use ironic o/14:25
*** jtomasek has joined #openstack-ironic14:26
etingofironic should take over their windows shop by reinstalling linux14:26
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allocation API: backfilling allocations  https://review.openstack.org/64817814:27
*** whoami-rajat has quit IRC14:30
dtantsurfolks, a quick poll: how stupid is it to accept names into a field called node_uuid?14:36
dtantsurif the alternative is to have two fields: node and node_uuid with just slightly different semantics?14:36
*** mbeierl has joined #openstack-ironic14:36
dtantsurand another alternative is to have one more place in our API where names are not accepted14:37
dtantsurcontext: https://storyboard.openstack.org/#!/story/200501414:37
*** e0ne has joined #openstack-ironic14:38
rpittaunot stupid, just not "correct"14:39
dtantsuryeah, I have 3 variants that I do not like :)14:39
rpittauxD14:39
dtantsurrpittau: which one would you prefer?14:40
rpittauhehhhhhh14:40
rpittauto be honest, I would keep them separated14:40
*** mbeierl has quit IRC14:41
dtantsurso, having both node_uuid and node, even though node will not be in the result of GET?14:41
rpittauavoid confusion, misunderstanding, headaches14:41
dtantsuror just having node in POST and node_uuid in GET?14:41
dtantsurit's not very restful, but probably the most intuitive14:42
rpittauit's a hard choice :/14:42
dtantsurright?14:43
rpittauI'd go for the most intuitive, but I'm being very practical here14:43
* dtantsur wonders if TheJulia has a minute for ^^^14:43
* TheJulia blinks14:43
TheJuliasorry meetings14:43
*** sthussey has joined #openstack-ironic14:44
TheJuliaugh, because of users expecting to be able to use the name as well, I think we kind of have to support it14:45
dtantsuryeah, we already have POST /v1/ports that only supports UUIDs, and it is pretty inconvenient14:45
TheJuliayeah, we've had a few people demand/ask it be fixed14:45
dtantsurTheJulia: so, would you have node_uuid accept names, or rather have 'node' in POST, while returning 'node_uuid' in GET?14:46
TheJulianode in post I think14:46
dtantsurokay, I guess I'll go with it14:47
*** mbeierl has joined #openstack-ironic14:57
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allocation API: backfilling allocations  https://review.openstack.org/64817815:00
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Allocation API: add tests on backfilling allocations  https://review.openstack.org/64840315:00
*** Lucas_Gray has quit IRC15:01
openstackgerritDmitry Tantsur proposed openstack/python-ironicclient master: Allocation API: support allocation backfilling  https://review.openstack.org/64840615:02
openstackgerritDmitry Tantsur proposed openstack/python-ironicclient master: Allocation API: support allocation backfilling  https://review.openstack.org/64840615:04
*** hamzy has quit IRC15:10
*** e0ne has quit IRC15:14
*** baha has quit IRC15:18
*** jtomasek has quit IRC15:19
*** e0ne has joined #openstack-ironic15:20
*** jtomasek has joined #openstack-ironic15:20
openstackgerritIlya Etingof proposed openstack/ironic master: WIP: Add Redfish Virtual Media support  https://review.openstack.org/63845315:27
*** jhesketh has joined #openstack-ironic15:28
*** jhesketh has quit IRC15:34
*** hamzy has joined #openstack-ironic15:35
*** e0ne has quit IRC15:40
openstackgerritMerged openstack/ironic stable/stein: Prepare stable/stein testing  https://review.openstack.org/64552315:42
*** gyee has joined #openstack-ironic16:12
*** jtomasek has quit IRC16:13
*** tssurya has quit IRC16:27
*** e0ne has joined #openstack-ironic16:27
*** whoami-rajat has joined #openstack-ironic16:32
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Allocation API: add tests on backfilling allocations  https://review.openstack.org/64840316:46
*** jtomasek has joined #openstack-ironic16:49
*** lucasagomes has quit IRC17:00
arne_wiebalckTheJulia: dtantsur : do we have a reference architecture document for larger deployments anywhere? Quickly looking I only found one for "small clouds" ...17:06
*** trown is now known as trown|lunch17:07
*** e0ne has quit IRC17:07
*** e0ne has joined #openstack-ironic17:08
arne_wiebalckBackground is that we plan to add quite some nodes over the next couple of months, starting in 2 weeks or so with an additional 600.17:08
arne_wiebalckAnd in addition to unknown issues, we already see that the resource tracker (one our 1 compute node) is getting to its limits ...17:09
arne_wiebalck... with the 1700 nodes we have currently in ironic.17:10
arne_wiebalckSo, one option we're discussing is to shard and scale out nova compute (which we had in the past, but then removed again).17:11
arne_wiebalckIn general, I'd be interested in feedback from deployers that are one order of magnitude bigger than we are, so ~10k nodes, to prepare for the next months :)17:14
*** e0ne has quit IRC17:17
dtantsurarne_wiebalck: we need input from someone like you to write such a document :)17:18
arne_wiebalckdtantsur: I was afraid of that answer ;)17:18
*** e0ne has joined #openstack-ironic17:21
rpittaugood night! o/17:24
*** rpittau is now known as rpittau|afk17:24
arne_wiebalckbye rpittau|afk o/17:24
eanderssondtantsur, for sure makes sense17:44
*** e0ne has quit IRC17:45
jrollheh, good answer dtantsur17:53
jrollarne_wiebalck: from my experience, I can totally recommend scaling out nova-compute, it should "just work" to spin more up17:53
jrollwe have a hash ring in there, they will figure out how to distribute management of nodes17:54
arne_wiebalckIIRC, James mentioned you were running with one nova-compute only, no?17:55
arne_wiebalckMaybe my memory does not serve me well here ...17:56
jrollyes, we're also on juno :)17:56
arne_wiebalck:-D17:56
jrollthis is from previous job experience, as well as writing the code to make nova-compute work that way :P17:56
arne_wiebalckjroll: juno is the reason you have only 1 nova-compute?17:56
jrollarne_wiebalck: part of it, yes17:57
jroll(we actually have two in active/passive)17:57
jroll(or maybe more)17:57
arne_wiebalckjroll: I see17:57
arne_wiebalckjroll: the thing getting to its limits now is the resource tracker17:57
arne_wiebalckit takes ~25 mins or so for one cycle17:57
arne_wiebalckand during that time it does not necessarily like to do other things such as scheduling instances :-S17:58
arne_wiebalckbut the resource tracker is serial17:58
jrollarne_wiebalck: right, so since newton(?) if you have multiple computes, each will only manage a subset of the ironic nodes. so e.g. 10k nodes and 5 n-cpu means each manages 2k nodes. the time to run the resource tracker is roughly linear with respect to the number of nodes.17:58
arne_wiebalckwhich is fine for "normal" compute nodes17:58
arne_wiebalckI was wondering if it would make sense to parallelise the RT, similar to what etingof  has done for ironic's power sync17:59
jrollit might, that's a conversation for nova folks :)18:00
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Allocation API: add tests on backfilling allocations  https://review.openstack.org/64840318:00
arne_wiebalckyeah ... probably such a parallelisation is not sensible in 99% of the deployments18:00
jrollI mean, it doesn't hurt in a VM deployment either18:01
arne_wiebalckI'm just worried that multiple n-cpu will bring some other "surprises"18:02
arne_wiebalckjroll: right18:02
arne_wiebalckjroll: or you simply configure the number of workers to 118:02
jrollarne_wiebalck: there should be minimal surprises, lemme find the commit message which has some notes18:02
jrollany surprises should be eventually consistent types of things18:02
*** derekh has quit IRC18:02
arne_wiebalckjroll: well, I think we had some surprises initially ... which is why we dropped to 1 n-cpu18:03
jrollinteresting, I'd be curious to hear more about those18:03
arne_wiebalckjroll: b/c we started with several n-cpu18:03
jrollthis has some notes: https://review.openstack.org/#/c/348443/18//COMMIT_MSG18:03
patchbotpatch 348443 - nova - Ironic: allow multiple compute services (MERGED) - 18 patch sets18:03
arne_wiebalckawesome, thx, I'll go through it18:04
arne_wiebalckjroll: I think some of the issues were related with the concurrent move to cells v218:04
arne_wiebalckjroll: and with the fact that we were early adopters18:04
arne_wiebalckjroll: many if not all issues have meanwhile been addressed I believe18:05
jrollah, cool18:05
arne_wiebalckjroll: our nova experts are actually pushing for scale-out :)18:05
*** trown|lunch is now known as trown18:06
arne_wiebalckjroll: thx for your input!18:06
jrollarne_wiebalck: no problem :)18:07
*** dtantsur is now known as dtantsur|afk18:08
dtantsur|afksee you18:08
arne_wiebalckdtantsur|afk: o/18:09
openstackgerritZane Bitter proposed openstack/ironic master: Docs: correct expected host format for drac_address  https://review.openstack.org/64849318:11
openstackgerritIlya Etingof proposed openstack/ironic master: Add systemd unit for sushy emulator in devstack  https://review.openstack.org/64482218:12
openstackgerritIlya Etingof proposed openstack/ironic master: Add UEFI firmware to Redfish emulator config  https://review.openstack.org/64838218:12
*** S4ren has quit IRC18:16
openstackgerritIlya Etingof proposed openstack/ironic master: Add systemd unit for sushy emulator in devstack  https://review.openstack.org/64482218:18
openstackgerritIlya Etingof proposed openstack/ironic master: Add UEFI firmware to Redfish emulator config  https://review.openstack.org/64838218:18
openstackgerritIlya Etingof proposed openstack/ironic master: Run IPMI and Redfish emulators as stack  https://review.openstack.org/64849618:18
openstackgerritIlya Etingof proposed openstack/ironic master: Inhibit automatic vbmcd start in devstack  https://review.openstack.org/64849818:26
openstackgerritIlya Etingof proposed openstack/ironic master: WIP: Add Redfish Virtual Media support  https://review.openstack.org/63845318:27
arne_wiebalckgood night everyone o/18:31
jroll\o18:33
*** jtomasek has quit IRC18:33
*** jtomasek has joined #openstack-ironic18:34
*** jtomasek has quit IRC18:35
*** MattMan_ has quit IRC19:08
*** MattMan_ has joined #openstack-ironic19:09
openstackgerritIlya Etingof proposed openstack/ironic master: WIP: Add Redfish Virtual Media support  https://review.openstack.org/63845319:10
openstackgerritIlya Etingof proposed openstack/ironic master: Improve VirtualBMC use in Devstack  https://review.openstack.org/64851019:10
mgoddardnow that's more like it - custom RAID config via deploy templates in under 11 minutes19:10
openstackgerritMerged openstack/ironic master: Docs: correct expected host format for drac_address  https://review.openstack.org/64849319:10
*** priteau has quit IRC19:27
*** e0ne has joined #openstack-ironic19:33
*** rcernin has quit IRC19:36
openstackgerritMark Goddard proposed openstack/ironic master: WIP: Add iDRAC RAID deploy steps  https://review.openstack.org/64173119:37
*** dsneddon has quit IRC19:38
*** jaypipes_ has joined #openstack-ironic19:54
*** jaypipes has quit IRC19:54
*** jtomasek has joined #openstack-ironic19:57
*** e0ne has quit IRC20:00
*** pcaruana has quit IRC20:08
*** brandor5 has joined #openstack-ironic20:21
*** efried has quit IRC20:22
brandor5hello everyone: I was wondering if ironic in newton can provision ppc64le systems, i tried googling but wasn't very successful20:22
*** dsneddon has joined #openstack-ironic20:23
*** jtomasek has quit IRC20:24
*** efried has joined #openstack-ironic20:26
*** dsneddon has quit IRC20:30
TheJuliabrandor5: I think back in newton some peopel may ahve been but I'm not sure. I know we more formalized support and addded capabilities more recently to help enable that more officially, but that was more like Rocky time frame.20:30
brandor5TheJulia: ok, thanks!20:30
TheJuliabrandor5: You may want to try early tomorrow and chat with mjturek20:30
TheJuliaOr maybe inquire with...20:31
TheJuliatrying to think of the name20:31
TheJuliatonyb: You may be able to help brandor520:31
brandor5cool, thanks for the pointers!20:31
TheJuliahappy to help20:32
tonybsay waht now?20:33
tonybbrandor5: I didn't try newton, but thinking about the chnages I landed in queens I'd say 'not without fiddling'20:34
tonybbrandor5: You'd need to use 'whole disk' images, You'll probably need DIB from a newer release than newton20:35
brandor5tonyb: ok, thanks for the heads up... we use whole disk by default so we're good with that20:37
tonybbrandor5: cool20:37
tonybI can't think of anything that'd stop it20:37
brandor5just trying to do some research now, i'll probably be back in a couple weeks when we're actually trying to make it happen :D20:38
tonybbrandor5: but I will do the thing where I point out that newton is well passed End-Of-Life so using a newer ironic would be good if you can20:38
tonybbrandor5: cool beans20:38
tonybbrandor5: ping me if you need help20:38
brandor5yep, we've got the upgrade on the map and will be jumping up to rocky this summer20:38
tonybbrandor5: I'm in Australia (UTC+1000) sodepending where you are overlap could be tricky20:39
brandor5unfortunately the ppc project is happening before that20:39
tonybbrandor5: Well rocky shoudl be good *and* you can use partition images \o/20:39
brandor5no worries20:39
tonybbrandor5: and it wil need to be ppc64le not ppc64(be)20:39
brandor5yep, that's what we have :)20:40
tonybcool20:40
*** efried has quit IRC20:44
*** hamzy has quit IRC20:54
*** openstackgerrit has quit IRC21:07
*** trown is now known as trown|outtypewww21:08
*** brandor5 has quit IRC21:31
*** jcoufal has quit IRC21:31
*** andrein has joined #openstack-ironic21:41
*** rcernin has joined #openstack-ironic21:44
*** rcernin has quit IRC21:55
*** rcernin has joined #openstack-ironic22:04
*** munimeha1 has quit IRC22:20
*** whoami-rajat has quit IRC22:52
*** S4ren has joined #openstack-ironic23:05
*** S4ren has quit IRC23:10
*** S4ren has joined #openstack-ironic23:11
*** S4ren has quit IRC23:16
eanderssonTheJulia, the yahoo way of doing it seem fine. We are just doing that.23:33
eanderssonI assumed it would make upgrade more difficult, but we have to change the driver anyway.23:33
eanderssonSince we are upgrading from Newton to Rocky.23:33
eanderssonI still think we should allow ops to override these constants somehow though (at least for network), but not sure how that would work.23:34
*** dsneddon has joined #openstack-ironic23:35
*** dsneddon has quit IRC23:40
*** gyee has quit IRC23:57
*** jhesketh has joined #openstack-ironic23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!