Thursday, 2019-01-31

*** TxGirlGeek has quit IRC00:01
*** sdake has joined #openstack-ironic00:05
*** dustinc has quit IRC00:12
*** sdake has quit IRC00:29
*** sdake has joined #openstack-ironic00:49
*** mjturek has quit IRC00:50
*** sdake has quit IRC01:09
*** jcoufal has quit IRC01:22
*** gyee has quit IRC01:33
*** _fragatina has quit IRC01:56
*** jcoufal has joined #openstack-ironic02:01
*** TxGirlGeek has joined #openstack-ironic02:02
*** hwoarang has quit IRC02:37
*** hwoarang has joined #openstack-ironic02:38
*** dims has quit IRC02:38
*** dims has joined #openstack-ironic02:55
*** sdake has joined #openstack-ironic03:00
*** mmethot has quit IRC03:14
*** mmethot has joined #openstack-ironic03:21
*** dnuka has joined #openstack-ironic03:29
dnukagood morning ironic03:30
*** gkadam-bmgr has joined #openstack-ironic03:34
*** _fragatina has joined #openstack-ironic04:18
*** TxGirlGeek has quit IRC04:18
*** _fragatina has quit IRC04:19
*** _fragatina has joined #openstack-ironic04:19
*** _fragatina has quit IRC04:21
*** _fragatina has joined #openstack-ironic04:21
*** _fragatina has quit IRC04:27
*** _fragatina has joined #openstack-ironic04:28
*** _fragatina has quit IRC04:31
*** _fragatina has joined #openstack-ironic04:31
*** _fragatina has quit IRC04:32
*** _fragatina has joined #openstack-ironic04:33
*** _fragatina has quit IRC04:41
*** _fragatina has joined #openstack-ironic04:42
*** _fragatina has quit IRC04:42
*** _fragatina has joined #openstack-ironic04:43
*** _fragatina has quit IRC04:44
*** _fragatina has joined #openstack-ironic04:44
*** rh-jelabarre has quit IRC05:07
*** _fragatina has quit IRC05:11
*** _fragatina has joined #openstack-ironic05:11
*** _fragatina has quit IRC05:16
*** _fragatina has joined #openstack-ironic05:16
*** _fragatina has quit IRC05:19
*** _fragatina has joined #openstack-ironic05:20
*** _fragatina has quit IRC05:38
*** _fragatina has joined #openstack-ironic05:39
*** e0ne has joined #openstack-ironic06:12
*** zzzeek has quit IRC06:33
*** zzzeek has joined #openstack-ironic06:37
*** moshele has joined #openstack-ironic06:58
*** moshele has quit IRC07:02
*** e0ne has quit IRC07:05
*** openstackgerrit has joined #openstack-ironic07:15
openstackgerritRiccardo Pittau proposed openstack/ironic master: Removing deprecated drac_host property  https://review.openstack.org/63390207:15
*** hamdyk has joined #openstack-ironic07:16
TheJuliaGood morning!07:17
dnukagood morning TheJulia07:18
TheJuliamanjeets: from what your describing, it sounds like your version of neutron is out of date.07:18
TheJuliadtantsur|afk: My asessment is that you are sane. I'm sorry.07:21
*** dnuka is now known as dnuka|brb07:22
arne_wiebalckgood morning, Ironic!07:24
*** dnuka|brb is now known as dnuka07:25
dnukahi arne_wiebalck o/07:25
arne_wiebalckhey dnuka o/07:26
arne_wiebalckTheJulia: Welcome to the Ironic earlybrids :)07:27
TheJulia:)07:27
TheJuliaI'm on a train from Brno to Praha at the moment :)07:28
dnuka:)07:29
*** jiapei has quit IRC07:35
*** jiapei has joined #openstack-ironic07:35
jiapeiGood afternoon everybody07:38
*** jcoufal has quit IRC07:39
dnukagood morning jiapei07:39
jiapeiHey morning dnuka07:39
TheJuliagood morning jiapei07:39
jiapeiGood morning TheJulia07:39
jiapeiPraha is really a beautiful city, it's on my wish list :)07:40
*** hwoarang has quit IRC07:41
*** hwoarang has joined #openstack-ironic07:44
arne_wiebalckgood morninf jiapei o/07:44
arne_wiebalcks/morninf/morning/07:44
arne_wiebalckTheJulia: Holidays or work?07:45
jiapeio/ Good morning arne_wiebalck07:45
TheJuliaarne_wiebalck: work07:48
TheJuliajiapei: sadly it will be a depart train, get on bus to the airport kind of visit.07:48
etingofTheJulia, don't feel bad about it, Praha is not that welcoming in the winter time... better come again in Spring o/07:49
dnukagood morning etingof o/07:49
etingofo/ dnuka07:50
jiapeiMorning etingof o/07:51
arne_wiebalckHey etingof o/07:52
etingofhey-hey ironic \o07:52
TheJuliaetingof: I will say the view of the countryside from the train is beautiful so far07:53
etingofTheJulia, it is, though it's better with all the chlorophyll cells up and running07:55
*** jcoufal has joined #openstack-ironic07:56
TheJulia:)07:56
TheJuliaSo before I go searching for all the things....  is anyone aware of things that require my immediate attention?08:04
*** rpittau has joined #openstack-ironic08:07
TheJuliao/ rpittau08:08
rpittaugood morning ironic! o/08:08
rpittauhey TheJulia :)08:08
dnukahi rpittau o/08:08
arne_wiebalckgood morning rpittau o/08:08
rpittauhi dnuka :)08:08
rpittauhey arne_wiebalck :)08:08
*** dnuka is now known as dnuka|brb08:09
openstackgerritMerged openstack/ironic-specs master: Better explain how do we know that the Smart NIC is ready to be programmed  https://review.openstack.org/63297108:14
*** sburette has joined #openstack-ironic08:36
*** amoralej|off is now known as amoralej08:49
*** dsneddon has quit IRC08:51
*** dsneddon has joined #openstack-ironic08:54
*** dnuka|brb is now known as dnuka08:57
TheJuliaIf anyone is bored, adding "owner" field support to python-ironicclient would be awesome09:02
*** iurygregory has joined #openstack-ironic09:05
etingofthat's not the 'description' field, right?09:05
iurygregorygood morning everyone o/09:05
rpittauhi iurygregory :)09:05
dnukaTheJulia, I'm interested :), but I'm not sure what you mean by "owner" field.09:06
dnukamorning iurygregory o/09:07
TheJuliadnuka: I added an "owner" field to the node object09:07
TheJuliabut never got to the client side of adding the ability to get/set in the library becasue of $TIME09:07
TheJuliaor lack their of time09:07
dnukaTheJulia, #link :)09:08
TheJuliaetingof: yeah, description is different. Owner is a dual purposed field right now but we only added information storage09:08
etingoffor reference -- https://review.openstack.org/#/c/63293509:08
TheJuliadnuka: the ironic change which merged?09:08
patchbotpatch 632935 - python-ironicclient - Support node description - 1 patch set09:08
TheJuliarpittau: I looked at the tinycore change for IPA. It looks like it is still running in the ramdisk with python2.7. see: http://logs.openstack.org/79/633479/5/check/ipa-tempest-wholedisk-bios-ipmi-direct-tinyipa-src/a878a2f/controller/logs/ironic-bm-logs/node-0_no_ansi_2019-01-29-12:56:40_log.txt.gz09:12
*** S4ren has joined #openstack-ironic09:13
TheJuliarpittau: looking at the commit message again, I see you intend to make it a separate patch, that the patch is just to prepare.... so I just workflow+1'ed it. :)09:14
rpittauTheJulia, that patch doesn't include pyuthon 3 :)09:14
rpittauTheJulia, yes, I ahve the py3 patch in the working, it should be ok but I was waiting for this one to merge first, didnt want to mixup too many things09:14
*** e0ne has joined #openstack-ironic09:15
*** S4ren has quit IRC09:16
*** S4ren has joined #openstack-ironic09:16
rpittauTheJulia, thanks!09:18
dnukaetingof, I need to create a same kind of patch like this one - https://review.openstack.org/#/c/632935, but for the "owner" field right? :)09:19
patchbotpatch 632935 - python-ironicclient - Support node description - 1 patch set09:19
TheJuliadnuka: exactly, that is if your up for it09:20
TheJuliahmm... why has train slowed to a crawl09:21
*** tssurya has joined #openstack-ironic09:21
*** dtantsur|afk is now known as dtantsur09:21
dtantsurmorning ironic09:21
dnukaTheJulia, I'll give it a try :)09:21
TheJuliadnuka: awesome09:22
dnukamorning dtantsur o/09:22
TheJuliadtantsur: Sorry your sane ;)09:22
dtantsurWUT? >_<09:22
* TheJulia giggles09:22
dtantsurhehe, good morning TheJulia. how's metaltubing going?09:22
TheJuliaIt goes, approaching Praha09:22
rpittauhi dtantsur :)09:22
dtantsur\o09:22
iurygregorymorning dtantsur and TheJulia o/09:23
TheJuliawhich means my laptopping is over until i park myself someplace after airport security09:23
TheJuliaugh, looks like https://review.openstack.org/#/c/628223/ got a transitory py27 failure09:28
patchbotpatch 628223 - ironic - Allocation API: conductor API (without HA and take... - 10 patch sets09:28
dtantsur:(09:29
TheJuliaIts still in the gate queue :(09:29
dtantsurokie, I'll recheck later09:29
TheJuliaI'm hoping we can get the smartnic stuff queued up next week if we can get allocation merged, since we need to get api things merging09:30
dtantsurwell, we need a 2nd +2 for the API patch *quick glance at mgoddard*09:31
* mgoddard pops up head09:31
TheJuliaOkay, closing laptop and putting it away. I doub;t I'll have any sort of inflight wifi... so might not get on until later :(09:31
TheJuliaAt least this is a short flight09:32
mgoddarddtantsur: I'll take a look at some point today. Knee deep in Tenks right now09:32
dtantsurcool!09:36
openstackgerritIury Gregory Melo Ferreira proposed openstack/python-ironicclient master: Move to zuulv3  https://review.openstack.org/63301009:42
*** trungnv has quit IRC09:47
*** derekh has joined #openstack-ironic09:48
openstackgerritMerged openstack/ironic master: Remove uses of logger name "oslo_messaging"  https://review.openstack.org/63325309:55
*** dougsz has joined #openstack-ironic09:57
openstackgerritMerged openstack/ironic master: Add is_smartnic to Port data model  https://review.openstack.org/62964010:24
openstackgerritMerged openstack/python-ironicclient master: Support for conductors exposed from API  https://review.openstack.org/62403210:24
openstackgerritMerged openstack/ironic-python-agent master: Build tinyipa with tinycore 8.x  https://review.openstack.org/63347910:24
dnukacongrats rpittau :)10:24
rpittauoh nice :)10:25
rpittauthanks dnuka10:25
openstackgerritM V P Nitesh proposed openstack/ironic master: Provides mount point as cinder requires it to attach volume  https://review.openstack.org/63345910:25
dnuka:)10:25
rpittautime to go for py310:25
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Support using JSON-RPC instead of oslo.messaging  https://review.openstack.org/63305210:26
openstackgerritMark Goddard proposed openstack/tenks master: Allow setting node and volume name prefixes per-spec  https://review.openstack.org/63390010:26
dtantsurhjensas: congrats on the neutron patch finally merging :) I guess we need it backported?10:28
*** jistr is now known as jistr|chat10:31
hjensasdtantsur: I pushed backports to rocky and queens upstream. It did'nt rebase cleanly on Pike.10:34
*** rnoriega has quit IRC10:35
hjensasdtantsur: I think we might not need it for pre-Queens. We did'nt have routed networks support in Ironic at that point (if I remember correctly). And updating the MAC is kind of only Ironic usecase.10:36
*** rnoriega has joined #openstack-ironic10:36
dtantsurcool, thanks!10:36
hjensasdtantsur: Will I see you at the beer event tomorrow?10:39
dtantsurhjensas: what's the beer event?10:40
dtantsurwe planned on just hanging out downtown, to be honest10:40
hjensasFosdem, there is usually somthing at Dillerium.10:40
etingofalmost delirium ^10:42
* iurygregory : delerium beer? <310:42
etingofperhaps delirium comes in Dillerium10:42
hjensasetingof: thanks, there is a bigger chance I'll find the place again now. :D10:43
*** e0ne has quit IRC10:46
dtantsurheh10:47
* etingof can't help but wonder, for how long ironic is being used in the wild?10:48
etingofinitial git commit dates back to 201010:48
dtantsuretingof: from Kilo in Red Hat, from Juno in Yahoo (now Oath) and Rackspace OnMetal (JayF may confirm)10:51
dtantsurbasically, Juno was the first working release :)10:51
etingof2014, give or take10:51
dtantsuryeah, late 201410:52
*** e0ne has joined #openstack-ironic10:52
dtantsurhmm, actually I seem to remember that Yahoo used nova-baremetal in Juno, not Ironic. maybe I'm wrong, let's ask rloo when she's here.10:52
*** priteau has joined #openstack-ironic10:53
etingofthat's close enough for code archaeology purpose10:56
* etingof could have done some more of the carbon dating in git commit log10:58
dnukafolks, a merge conflict occured even before submtting the very first patch to gerrit. Is this possible :)11:03
*** jistr|chat is now known as jistr11:05
TheJuliadtantsur: I think there was a migration. I know they did lots of downstream patches for $things11:06
dtantsuryeah, definitely not pure Juno :) And I don't think it was pure Kilo in our case.11:07
dtantsurdnuka: it can happen if you're based on an old commit, and the head of master has diverged too much.11:07
TheJuliaetingof: that initial commit is from initial repo states, since ironic started as the nova codebase which was massively re-written11:07
rpittaudnuka, yes that's possible :)11:07
dnukadtantsur, rpittau thanks :)11:08
TheJuliaI think rackspace was icehouse timeframe, but it would have been around around when they were trying to stay as close to master as possible11:09
dtantsurhistory :)11:09
TheJuliaA long time ago in a universe far far away... the bear metal bears came forth and deployed servers. They said it was Ironic.11:10
dtantsurTheJulia: I cannot stop boasting: look, a CI run without rabbitmq http://logs.openstack.org/52/633052/24/check/ironic-tempest-functional-python3/dbdd424/controller/logs/ :)11:11
dtantsurlol11:11
TheJuliadtantsur: enjoying that I see? :)11:11
dtantsura lot :)11:11
dtantsurkeystone does not need rabbit as well, apparently? this means we can switch bifrost (unless neutron is enabled)11:12
TheJuliaYeah, keystone is a direct API service11:13
TheJuliaThat would actually be kind of cool11:13
* TheJulia is excited now11:13
* dtantsur ponders a similar patch to ironic-inspector, now that we're planning API-worker split there11:13
TheJulialooks like the linchpin for the events work is going to be https://review.openstack.org/#/c/345934/811:21
patchbotpatch 345934 - python-ironicclient - Add Events support - 8 patch sets11:21
TheJuliaI guess that needs a reno11:21
*** e0ne has quit IRC11:24
openstackgerritMark Goddard proposed openstack/tenks master: Allow setting node and volume name prefixes per-spec  https://review.openstack.org/63390011:25
*** e0ne has joined #openstack-ironic11:34
*** rpittau has quit IRC11:34
*** rpittau has joined #openstack-ironic11:34
*** sdake has quit IRC11:41
*** sdake has joined #openstack-ironic11:48
openstackgerritHamdy Khader proposed openstack/ironic master: Add support for Smart NICs  https://review.openstack.org/58357311:50
openstackgerritHamdy Khader proposed openstack/ironic master: Expose is_smartnic in port API  https://review.openstack.org/63050811:50
TheJuliadtantsur: ugh http://logs.openstack.org/23/628223/10/gate/openstack-tox-py27/922cd7b/job-output.txt.gz#_2019-01-31_08_32_50_955893 <-- re allocations11:51
dtantsureeeehhhmmm11:51
dtantsurwe don't modify migrations in this patch, do we?11:52
TheJuliaI don't think so11:52
TheJuliadtantsur: nope11:53
TheJuliadtantsur: the rpc schema ver is bumped, but nothing db side. I wonder if it was just a fluke race... or if py27 will start to fail more often11:53
dtantsurTheJulia: wdyt about https://github.com/metalkube/dev-scripts/pull/17#discussion_r25263467111:58
TheJuliadtantsur: the two patches in the gate right now have passed py27 without issues11:58
* dtantsur suspects his bad luck..11:59
dtantsurI can try rebasing and seeing if it helps anything11:59
openstackgerritHamdy Khader proposed openstack/python-ironicclient master: Add is-smartnic port attribute to port command  https://review.openstack.org/62944911:59
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allocation API: conductor API (without HA and take over)  https://review.openstack.org/62822312:01
openstackgerritDmitry Tantsur proposed openstack/ironic master: Allocation API: REST API implementation  https://review.openstack.org/62926012:01
TheJuliadtantsur: sounds like an ironic issue. That being said, the enabled order should be ipxe,pxe based on preference loading.12:01
TheJuliaI've seen something super similar with iscsi deploy and fedora filesystem permissions12:02
TheJuliabut I can't place it right now in my memory12:02
*** amoralej is now known as amoralej|lunch12:02
* TheJulia goes and finds foods12:02
dtantsurderekh: could you check if the nodes really end up with boot_interface=ipxe (not pxe)?12:02
TheJuliaderekh: also, what is the deploy_interface ?12:03
dtantsurI think he does not change it, so iscsi?12:04
openstackgerritHamdy Khader proposed openstack/python-ironicclient master: Add 'hostname' to port's local link connection  https://review.openstack.org/62877312:05
derekhdtantsur: TheJulia node show has boot_interface=ipxe and deploy_interface=iscsi12:16
openstackgerritMerged openstack/ironic master: Make versioned notifications topics configurable  https://review.openstack.org/62981812:16
TheJuliaderekh: try direct please :)12:17
derekhTheJulia: will do12:18
TheJuliaThanks!12:18
dtantsurdunno if it will help. I rather suspect that we forgot to pass ipxe_enabled=True somewhere from the ipxe interface.12:18
dtantsurderekh: could you file an issue on the storyboard about your discovery?12:19
derekhdtantsur: yup, no prob12:19
TheJuliadtantsur: possible, but iscsi image download is slightly behavior in terms of image cache interaction12:20
TheJuliaAt least, that is what my memory thinks12:21
dtantsurTheJulia: you're right, but the failure happens on the ramdisk download, and it should be (?) the same for both deploy interfaces.12:21
dtantsuraaaand, another transient failure :( but the next patch passed all UT jobs12:21
TheJuliaUgh12:21
openstackgerritIury Gregory Melo Ferreira proposed openstack/python-ironicclient master: Move to zuulv3  https://review.openstack.org/63301012:22
TheJuliaYeah, should be the same path...12:22
derekhdtantsur: TheJulia http://paste.openstack.org/show/744301/12:22
TheJuliaMmm food12:22
dtantsurTheJulia: enjoy!12:24
dtantsurthanks derekh. yeah, something is wrong here, looking at the code now.12:26
*** rh-jelabarre has joined #openstack-ironic12:26
dtantsuraha, found it!12:27
openstackgerritDhanuka Warusadura proposed openstack/python-ironicclient master: Add node owner  https://review.openstack.org/63421512:28
dnukano more merge conflicts :)12:29
TheJulia\o/12:29
TheJulia*212:30
dnuka:)12:30
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Fix iPXE boot interface with ipxe_enabled=True  https://review.openstack.org/63421612:30
dtantsurderekh: do you think you could try ^^^?12:30
dtantsurTheJulia: ^^12:30
derekhdtantsur: will do https://storyboard.openstack.org/#!/story/200490512:32
TheJuliaI guess that would have different logic that far down for ipxe...12:34
TheJulia:(12:34
dtantsurI think the root of the issue is that we still use PXE paths for downloading images in this case12:36
*** bfournie has quit IRC12:45
openstackgerritM V P Nitesh proposed openstack/ironic master: Provides mount point as cinder requires it to attach volume  https://review.openstack.org/63345912:58
*** e0ne has quit IRC13:01
derekhdtantsur: the patch appears to work, in that my node now deploys, but then after it powers back on I get this http://paste.openstack.org/show/744305/ and it powers off again,13:02
derekhdtantsur: its getting a lot further so this problem may be unrelated13:03
derekhbbiab13:03
*** e0ne has joined #openstack-ironic13:04
openstackgerritMerged openstack/ironic master: Removing deprecated drac_host property  https://review.openstack.org/63390213:15
*** gkadam-bmgr is now known as gkadam13:15
dtantsurderekh: it's conductor crashing, we need to see why.13:16
dtantsurhmmmmm, wait13:16
dtantsurwe call from the conductor to the same conductor.. and it tells us there's no conductors?? WUT?13:17
dtantsurcan it be some race with creating a database?13:18
*** gkadam has quit IRC13:20
*** amoralej|lunch is now known as amoralej13:23
*** e0ne has quit IRC13:25
*** jistr is now known as jistr|call13:25
*** e0ne has joined #openstack-ironic13:29
dnukagood night folks o/13:30
*** dnuka has quit IRC13:30
*** e0ne has quit IRC13:30
*** jistr|call is now known as jistr13:31
*** rh-jelabarre has quit IRC13:31
*** e0ne has joined #openstack-ironic13:32
*** e0ne has quit IRC13:34
openstackgerritDmitry Tantsur proposed openstack/ironic master: Fix iPXE boot interface with ipxe_enabled=False  https://review.openstack.org/63421613:34
dtantsurthe complete patch ^^\13:35
*** rpittau has quit IRC13:35
*** e0ne has joined #openstack-ironic13:35
*** jcoufal has quit IRC13:37
arne_wiebalckRoot device hints FTW! Used them twice this week to boot machines which did not manage to boot after deployment: one with h/w RAID controller trying to expose the individual drives, the other with an attached disk array. \o/13:45
*** rpittau has joined #openstack-ironic13:46
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent master: Build tinyipa with python3  https://review.openstack.org/63423213:47
rpittauwifi gave up exactly when I pressed enter after git review :/13:47
*** pcaruana has quit IRC13:52
*** pcaruana has joined #openstack-ironic14:02
*** rloo has joined #openstack-ironic14:07
*** sdake has quit IRC14:08
*** e0ne has quit IRC14:13
*** diga has joined #openstack-ironic14:16
openstackgerritHamdy Khader proposed openstack/ironic master: Add support for Smart NICs  https://review.openstack.org/58357314:19
openstackgerritHamdy Khader proposed openstack/ironic master: Expose is_smartnic in port API  https://review.openstack.org/63050814:19
*** baha has joined #openstack-ironic14:26
*** zul has joined #openstack-ironic14:31
*** mjturek has joined #openstack-ironic14:33
mjturekanyone familiar with the baremetal basic ops test, after the node gets verified by SSH, right before cleaning starts, I see another neutron port come into existence.. Anyone know what that port might be for? It seems to be a virtual mac address14:35
*** sdake has joined #openstack-ironic14:39
*** e0ne has joined #openstack-ironic14:45
mjturekhmmm maybe should ask on the ML14:45
*** jcoufal has joined #openstack-ironic14:53
*** rh-jelabarre has joined #openstack-ironic14:54
openstackgerritMark Goddard proposed openstack/tenks master: Configure VM interfaces in macvtap passthrough mode  https://review.openstack.org/63424915:01
openstackgerritMark Goddard proposed openstack/tenks master: Use node state rather than 'cmd'  https://review.openstack.org/63425015:01
openstackgerritMark Goddard proposed openstack/tenks master: Default Ironic node properties are bogus  https://review.openstack.org/63425115:01
*** hamdyk has quit IRC15:05
*** TxGirlGeek has joined #openstack-ironic15:18
*** priteau has quit IRC15:19
mjturek\15:22
*** sdake has quit IRC15:24
openstackgerritRiccardo Pittau proposed openstack/ironic-python-agent master: Build tinyipa with python3  https://review.openstack.org/63423215:28
*** baha has quit IRC15:30
*** mjturek has quit IRC15:33
*** trown|outtypewww is now known as trown15:38
*** mjturek has joined #openstack-ironic15:41
*** priteau has joined #openstack-ironic15:43
*** robbbe has joined #openstack-ironic15:52
*** baha has joined #openstack-ironic15:58
*** _fragatina has quit IRC16:03
*** _fragatina has joined #openstack-ironic16:04
*** _fragatina has quit IRC16:04
*** _fragatina has joined #openstack-ironic16:05
*** arne_wiebalck_ has joined #openstack-ironic16:07
*** arne_wiebalck_ has quit IRC16:07
*** _fragatina has quit IRC16:10
*** robbbe has quit IRC16:19
*** sdake has joined #openstack-ironic16:20
*** robbbe has joined #openstack-ironic16:35
*** TxGirlGeek has quit IRC16:35
*** sthussey has joined #openstack-ironic16:39
*** etingof has left #openstack-ironic16:45
*** e0ne has quit IRC16:46
*** etingof has joined #openstack-ironic16:46
etingofdo we have a timestamp field on the node noting last power state reading? I do not see such thing so far16:47
dtantsurI don't think we do (I'd like to see it btw)16:48
* arne_wiebalck thinks this was discussed recently and the answer was no16:48
*** priteau has quit IRC16:50
etingofI am thinking to add such field and use it right away for prioritizing power status calls for the upcoming status (TheJulia's suggestion)16:58
etingofI am not sure if that would hurt performance... locking...?16:59
*** mjturek has quit IRC17:00
*** mjturek has joined #openstack-ironic17:00
dtantsurmmm, right, it will require going back to exclusive locks by default17:02
dtantsurwhich is probably why we don't do it17:02
*** TxGirlGeek has joined #openstack-ironic17:03
*** iurygregory has quit IRC17:06
*** _fragatina has joined #openstack-ironic17:08
etingofaha, but we do have in-memory dict holding retry counters - we could probably just sort nodes by its contents17:10
etingoffor power sync ordering17:10
*** sburette has quit IRC17:12
NobodyCamGood Morning Ironic'ers17:12
TheJuliaEh, but an exclusive lock is a good thing I guess for BMC power polling... But that is the only way to force last_updated to change between checks.17:12
* TheJulia looks out the window of the hotel lounge... Sees only darkness and feels confused by NobodyCam :)17:13
*** mjturek has quit IRC17:14
TheJuliaetingof: in memory really only seems the same path unless we have good reason to lock17:14
NobodyCamhehehehe17:14
NobodyCamGota luv UGT17:14
TheJuliaWhich would update the DB record.....17:14
TheJuliaIndeed.17:14
*** mjturek has joined #openstack-ironic17:15
*** tssurya has quit IRC17:17
* etingof is not sure how to interpret TheJulia's thoughts... let's blame NobodyCam's early morning for that o/17:18
*** dustinc has joined #openstack-ironic17:18
TheJuliaetingof: sorry, numberous threads trying to run that suggest I should go explore, exercise, or sleep.. likely all three.17:19
*** gyee has joined #openstack-ironic17:19
rpittauhi NobodyCam :)17:20
NobodyCamGood Morning rpittau :)17:22
dtantsurmorning NobodyCam17:25
dtantsuretingof: we use shared lock for checking the power state, only acquire the exclusive lock if we need to update it in the database (or force it on the node)17:26
etingofdtantsur, right, so if we need to update the timestamp on the node in the DB we'd have to upgrade the lock frequently...17:28
*** sdake has quit IRC17:28
*** dsneddon has quit IRC17:29
etingofdtantsur, is the in-memory power failure states suitable for ordering the nodes? would not it be lighter to use that then?17:29
dtantsuretingof: I'm not sure I get the last question. What ordering do you mean?17:30
NobodyCamHey hey dtantsur Good Morning17:30
etingofdtantsur, so the idea is to ponder previously failed nodes (on power status) to fail them faster and get them out of the way17:31
TheJuliaetingof: I think so, but only for over 300 or so nodes. If less than that then I say skip sorting17:31
manjeetsTheJulia, I had a bit old neutron (along with mech driver for a hardware switch) its like month old I believe, but I'll rebase it to master today17:31
manjeetsthanks for pointers17:32
etingofdtantsur, the root of the problem being nodecount may be too high for the greenlets we herd17:32
*** robbbe has quit IRC17:33
TheJuliamanjeets: vnic_type has been a thing for a few years.... But I might not understand the entire isaue17:33
TheJuliaIasue17:33
etingofthus, attempt to optimize the ordering17:33
* etingof is seeking the enzyme to destroy the rna of the devconf virus17:35
TheJuliaHeh17:36
TheJuliaA few folks in my class got whatever it was...17:36
manjeetsTheJulia, I have enabled mech driver in neutron for switch we are testing in my prior setup I enabled ovs,switch_mech and when i was boot nova instance with speical port it never always failed on binding provisioning port17:37
manjeetsso it never reached the point where it would call bind_port from switch_mech17:37
manjeetsthen i tried new configuration which didn't have provisioning network and I was able to see bind_port being executed from switch mech but was throwing error I mentioned above17:38
dtantsurTheJulia: do we need presence at the ops meetup? https://etherpad.openstack.org/p/BER-ops-meetup17:38
manjeetsthen tried just with ovs and error was still there on server boot17:38
* manjeets rebuilds devstack with ironic will post logs in a bit17:41
*** dsneddon has joined #openstack-ironic17:45
rpittaubye all! good evening! o/17:48
*** rpittau has quit IRC17:48
TheJuliamanjeets:  okay, still sounds weird, but logs will help17:50
TheJuliadtantsur: it would be good for there to be someone from ironic there to answer questions, listen to feedback17:52
dtantsurTheJulia: I can do it for obvious reasons :)17:52
TheJulia\o/17:53
*** dougsz has quit IRC17:53
*** derekh has quit IRC18:00
*** sdake has joined #openstack-ironic18:07
manjeetshttp://paste.openstack.org/show/744330/18:11
manjeetsTheJulia, I see this a lot in logs ^^ with ovs and other mech18:11
manjeetsnote: I'm still using a month old neutron18:12
manjeetsbut ironic and everything else is master18:12
*** bfournie has joined #openstack-ironic18:26
*** trown is now known as trown|lunch18:29
TheJuliamanjeets: I would suspect your ml2 driver is just not loading18:29
TheJuliaAnything in your neutron logs verifying that all your plugins are loading?18:30
manjeetsTheJulia, neutron logs are fine, I see bind_port getting executed from ml2 driver I have for hardware switch18:36
*** mjturek has quit IRC18:36
* TheJulia raises an eyebrow18:36
manjeetsthis happens with or without the hardware ml2 driver, I noticed this with ovs as well18:37
TheJuliaWell, OVS would never know what to do with a vnic_type of baremetal18:37
manjeetswait TheJulia I'm building devstack with ovs and new neutron will verify logs again18:37
TheJuliaOVS is just going to go "I don't know what to do with this" and return that error18:38
*** mjturek has joined #openstack-ironic18:38
*** amoralej is now known as amoralej|off18:39
*** dtantsur is now known as dtantsur|afk18:39
*** e0ne has joined #openstack-ironic18:39
dtantsur|afkon PTO starting tomorrow, see some of you on FOSDEM, the others - on Wednesday18:39
*** moshele has joined #openstack-ironic18:44
*** mjturek has quit IRC18:44
TheJuliadtantsur|afk: enjoy... Oh new sqlalchemy from the 25th....18:47
manjeetsah ohk TheJulia, since there's ovs agent but no agent for this mech driver could that be an issue ?18:48
*** e0ne has quit IRC18:48
TheJuliamanjeets: no, the ml2 drivers, as I understand it, register when they load into neutron-api... What they support18:50
manjeetsexactly that's what my understanding is18:51
*** TxGirlGeek has quit IRC18:51
TheJuliaSo to get that error, I wonder what driver your loading.18:52
manjeetsI am loading openvswitch,switch_mech18:52
manjeetshttps://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/drivers/mech_agent.py#L9118:52
TheJuliaOh18:53
*** TxGirlGeek has joined #openstack-ironic18:53
TheJuliaswitch_mech is?18:53
manjeetsthe driver I'm writing for hardware switch18:53
manjeetsI have added baremetal in supported vnics18:54
TheJuliaOh!18:55
TheJuliaOkay18:56
*** mjturek has joined #openstack-ironic18:56
*** moshele has quit IRC19:08
*** mjturek has quit IRC19:09
*** mjturek has joined #openstack-ironic19:10
*** moshele has joined #openstack-ironic19:16
TheJuliamanjeets: take a look at what netwoeking-generic-switch does in https://github.com/openstack/networking-generic-switch/blob/master/networking_generic_switch/generic_switch_mech.py in terms of supporting the baremetal vnic type19:17
*** mgoddard has quit IRC19:18
*** mgoddard has joined #openstack-ironic19:19
manjeetsTheJulia, so missing is_port_supported could be the reason ?19:19
manjeetsI think that'll fix the issue ?19:20
manjeetsthanks TheJulia19:20
TheJuliaI suspect so :)19:20
*** moshele has quit IRC19:20
openstackgerritMerged openstack/ironic master: Allocation API: conductor API (without HA and take over)  https://review.openstack.org/62822319:28
*** pcaruana has quit IRC19:30
*** trown|lunch is now known as trown19:43
*** dsneddon has quit IRC19:46
*** TxGirlGeek has quit IRC19:48
*** jcoufal has quit IRC19:50
*** rnoriega has quit IRC19:57
*** rnoriega has joined #openstack-ironic19:57
*** dsneddon has joined #openstack-ironic20:02
*** arne_wiebalck_ has joined #openstack-ironic20:02
*** arne_wiebalck_ has quit IRC20:03
*** mjturek has quit IRC20:03
*** arne_wiebalck_ has joined #openstack-ironic20:05
*** mjturek has joined #openstack-ironic20:05
*** TxGirlGeek has joined #openstack-ironic20:05
*** mjturek has quit IRC20:09
*** mjturek has joined #openstack-ironic20:10
*** mjturek has quit IRC20:34
*** TxGirlGeek has quit IRC21:00
*** jcoufal has joined #openstack-ironic21:02
openstackgerritArne Wiebalck proposed openstack/ironic master: Fix test for 'force_persistent_boot_device' (i)PXE driver_info option  https://review.openstack.org/63431521:12
openstackgerritArne Wiebalck proposed openstack/ironic master: Fix test for 'force_persistent_boot_device' (i)PXE driver_info option  https://review.openstack.org/63431521:16
*** arne_wiebalck_ has quit IRC21:17
*** TxGirlGeek has joined #openstack-ironic21:25
*** rh-jelabarre has quit IRC21:33
*** bfournie has quit IRC21:58
*** trown is now known as trown|outtypewww22:02
JayFetingof: dtantsur|afk: we started deploying Ironic, with our to-be-upstreamed agent driver, before Icehouse, even... I don't remember what the "H" release was, but probably that one or before22:03
*** baha has quit IRC22:08
*** jcoufal has quit IRC22:19
*** sdake has quit IRC22:30
*** S4ren has quit IRC22:34
tonybIs it normal for the ironic-neutron-agent-heartbeat.info message queue to have lots of un-ACKed messages?22:38
tonyb[root@director ~]# docker exec -it --user root rabbitmq rabbitmqctl list_queues | grep heartbeat22:38
tonybironic-neutron-agent-heartbeat.info     13723722:38
tonybthat number is slowly growing22:39
tonybThat's rocky so it may be differnt on master I'll try to spin up master next week sometime if needed22:40
eanderssonAnything in the baremetal agent logs?22:45
*** sdake has joined #openstack-ironic22:45
tonybeandersson: What kind of anything?22:48
eanderssonLike errors or similar22:48
tonybeandersson: I'm pretty sure I saw it logging the heartbeat22:49
tonybeandersson: not that I noticed22:49
tonybeandersson: I can remove the node and introspect it again and save the logs22:49
*** MattMan has quit IRC22:51
*** MattMan has joined #openstack-ironic22:52
*** bfournie has joined #openstack-ironic23:01
*** dustinc has quit IRC23:18
tonybeandersson: Nothing that looks like an error23:37
eanderssontonyb, are they all un'acked? or just not consumed?23:38
* tonyb admits he can't even find the producer or comsumer for that queue23:38
tonybeandersson: Not sure23:38
tonybeandersson: just that they keep growing which seemed strange23:38
eanderssonIs there a consumer attached to the queue?23:39
eanderssonI haven't actually used networking-baremetal yet myself23:39
eanderssonbut what happens if you restart the agent?23:39
eanderssonbecause it's supposed to be listening to that queue23:40
eanderssonafaik23:40
tonybYeah there aren't any consumers23:42
tonyb[root@director ~]# docker exec -it --user root rabbitmq rabbitmqctl list_queues name consumers | grep heartbeat23:42
tonybironic-neutron-agent-heartbeat.info     023:42
*** diga has quit IRC23:43
eanderssonhttp://git.openstack.org/cgit/openstack/networking-baremetal/tree/networking_baremetal/agent/ironic_neutron_agent.py#n5923:48
eanderssonSo this looks like it should be listening.23:48
eanderssonIt even looks like it is supposed to both listen and notify23:49
tonybOkay, I suspect that's my problem I can't find that anywhere23:50
* tonyb goes to look at the installer to workout where that's supposed to be/come from23:51
tonybeandersson: Thanks23:51

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!