Friday, 2016-05-06

*** ChrisAusten has joined #openstack-ironic00:05
*** chlong has quit IRC00:05
*** aginwala has quit IRC00:09
*** aginwala has joined #openstack-ironic00:12
*** garthb_ has quit IRC00:13
*** rloo has quit IRC00:16
*** fragatin_ has quit IRC00:17
*** dtantsur has joined #openstack-ironic00:22
*** fragatina has joined #openstack-ironic00:29
*** fragatina has quit IRC00:33
*** aginwala has quit IRC00:36
*** rloo has joined #openstack-ironic00:40
*** aginwala has joined #openstack-ironic00:40
*** fragatina has joined #openstack-ironic00:41
*** rloo has quit IRC00:41
*** piet has joined #openstack-ironic00:42
*** fragatina has quit IRC00:42
*** afaranha has quit IRC00:47
*** fragatina has joined #openstack-ironic00:51
*** mtanino has quit IRC00:52
*** fragatin_ has joined #openstack-ironic00:52
*** fragatin_ has quit IRC00:53
*** fragatin_ has joined #openstack-ironic00:53
*** hoangcx has joined #openstack-ironic00:54
*** fragatina has quit IRC00:55
*** fragatina has joined #openstack-ironic00:56
*** fragatin_ has quit IRC00:57
*** fragatina has quit IRC01:01
*** piet has quit IRC01:01
*** baoli_ has quit IRC01:09
*** baoli has joined #openstack-ironic01:10
*** chlong has joined #openstack-ironic01:31
*** fragatina has joined #openstack-ironic01:34
*** sacharya has joined #openstack-ironic01:47
*** Goneri has joined #openstack-ironic01:48
*** sacharya has quit IRC01:51
*** jcoufal has joined #openstack-ironic01:57
*** baoli has quit IRC01:57
*** keedya has joined #openstack-ironic01:59
*** jcoufal has quit IRC02:03
openstackgerritMerged openstack/ironic-python-agent: Stop using git:// and be nice to people behind proxy servers  https://review.openstack.org/31311502:05
*** aginwala has quit IRC02:13
openstackgerritMerged openstack/ironic: Pass environment through to create-node.sh  https://review.openstack.org/31310602:14
openstackgerritMerged openstack/ironic: Update resources only for specific node during deletion.  https://review.openstack.org/28749802:15
*** piet has joined #openstack-ironic02:20
*** joprovost has quit IRC02:22
*** fragatina has quit IRC02:24
*** keedya has quit IRC02:24
*** sacharya has joined #openstack-ironic02:35
*** piet has quit IRC02:37
*** absubram has joined #openstack-ironic02:53
*** absubram_ has joined #openstack-ironic02:54
*** piet has joined #openstack-ironic02:56
*** absubram has quit IRC02:58
*** absubram_ is now known as absubram02:58
*** daemontool_ has quit IRC03:04
*** chenke has joined #openstack-ironic03:05
chenkehave anybody use the inspur server as the baremetal server, i saw the server have loaded the deploy ramdisk and kernel but not execute it. Dose anybody solve this kind of issue?03:08
*** piet has quit IRC03:13
*** fragatina has joined #openstack-ironic03:16
*** fragatina has quit IRC03:16
*** fragatina has joined #openstack-ironic03:16
*** absubram has quit IRC03:26
*** amit213 has quit IRC03:33
*** amit213 has joined #openstack-ironic03:33
*** sivaramakrishna has joined #openstack-ironic03:46
*** vmud213 has joined #openstack-ironic03:53
*** mmitchell has joined #openstack-ironic03:58
*** mmitchell has left #openstack-ironic03:58
*** links has joined #openstack-ironic04:02
*** hamzy has joined #openstack-ironic04:06
*** ankkumar5 has joined #openstack-ironic04:08
*** praneshp has quit IRC04:09
*** sacharya has quit IRC04:19
*** piet has joined #openstack-ironic04:21
*** aginwala has joined #openstack-ironic04:28
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add portgroups to support LAG interfaces - API  https://review.openstack.org/20624404:51
*** wajdi has joined #openstack-ironic04:53
openstackgerritvinay kumar muddu proposed openstack/ironic-lib: Add support for BIOS local boot for GPT label  https://review.openstack.org/31161305:14
*** mbound has joined #openstack-ironic05:18
*** sacharya has joined #openstack-ironic05:20
*** sacharya has quit IRC05:25
*** piet has quit IRC05:27
openstackgerritvinay kumar muddu proposed openstack/ironic-lib: Add support for BIOS local boot for GPT label  https://review.openstack.org/31161305:28
*** ChrisAusten has quit IRC05:36
*** mbound has quit IRC05:43
*** yolanda has joined #openstack-ironic05:55
*** wajdi has quit IRC06:01
*** vishwanathj has quit IRC06:13
*** chenke has quit IRC06:19
*** e0ne has joined #openstack-ironic06:39
*** yolanda has quit IRC06:46
*** yolanda has joined #openstack-ironic06:47
*** sacharya has joined #openstack-ironic06:53
*** e0ne has quit IRC06:53
*** athomas has joined #openstack-ironic06:55
*** sacharya has quit IRC06:58
*** e0ne has joined #openstack-ironic06:59
*** e0ne has quit IRC07:00
*** yolanda has quit IRC07:02
openstackgerritAparna proposed openstack/proliantutils: Using python 'subprocess' to execute hpssacli commands  https://review.openstack.org/31325407:05
*** yolanda has joined #openstack-ironic07:06
*** hoangcx has quit IRC07:08
*** aginwala has quit IRC07:13
*** aginwala has joined #openstack-ironic07:14
*** e0ne has joined #openstack-ironic07:16
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add network drivers  https://review.openstack.org/28585207:20
*** ifarkas has joined #openstack-ironic07:21
*** yolanda has quit IRC07:21
*** chopmann has joined #openstack-ironic07:27
*** yolanda has joined #openstack-ironic07:32
betherlyo/ morning all!07:35
*** openstackstatus has quit IRC07:38
*** openstackstatus has joined #openstack-ironic07:39
*** ChanServ sets mode: +v openstackstatus07:39
*** ohamada has joined #openstack-ironic07:51
*** daemontool has joined #openstack-ironic07:53
*** chlong has quit IRC07:54
*** hoangcx has joined #openstack-ironic07:54
*** zzzeek has quit IRC08:00
*** zzzeek has joined #openstack-ironic08:02
*** e0ne has quit IRC08:07
*** dmk0202 has joined #openstack-ironic08:08
*** lucas-afk is now known as lucasagomes08:15
lucasagomesjlvillal, yeah I hit the resource creation error yesterday in grenade08:15
lucasagomesI will tkae a look at it08:15
lucasagomesbetherly, morning08:15
lucasagomesmorning all08:15
betherlyo/ lucasagomes :)08:18
*** e0ne has joined #openstack-ironic08:20
vmud213morning lucasagomes, betherly08:22
*** aginwala has quit IRC08:28
*** chopmann has quit IRC08:29
betherlyo/ vmud21308:30
alinebmorning all08:32
*** jistr has joined #openstack-ironic08:32
*** pcaruana has joined #openstack-ironic08:36
*** derekh has joined #openstack-ironic08:38
*** e0ne has quit IRC08:40
*** chenke has joined #openstack-ironic08:41
*** fragatina has quit IRC08:41
openstackgerritAline Bousquet proposed openstack/ironic-lib: Add support for metrics  https://review.openstack.org/30152608:42
*** e0ne has joined #openstack-ironic08:42
*** chopmann has joined #openstack-ironic08:45
lucasagomesvmud213, alineb morning08:46
lucasagomeshi all, https://review.openstack.org/#/c/308965/ has 2+2 already someone wants to give it a final review?08:47
*** fragatina has joined #openstack-ironic08:50
*** sacharya has joined #openstack-ironic08:54
*** tesseract has joined #openstack-ironic08:56
*** tesseract is now known as Guest4293008:56
*** sacharya has quit IRC08:59
*** alex_xu has quit IRC09:03
*** alex_xu has joined #openstack-ironic09:05
*** electrofelix has joined #openstack-ironic09:07
*** chenke_ has joined #openstack-ironic09:08
*** mkovacik has joined #openstack-ironic09:09
mkovacikmorning Ironic!09:10
vmud213lucasagomes: When you get a chance can you look into https://review.openstack.org/#/c/31161309:11
*** chenke has quit IRC09:12
*** irf has joined #openstack-ironic09:14
irfmorning Ironic !!!09:14
sambetts|afkMorning all09:14
*** sambetts|afk is now known as sambetts09:14
irfas per doc : http://docs.openstack.org/developer/ironic/deploy/troubleshooting.html#nova-returns-no-valid-host-was-found-error09:15
irfi have completed the first four points ...09:15
irfregarding fifth point ... i am not sure ...09:16
irfthe output of command : "nova hypervisor-stats" should show the status of hypervisor for Ironic node ?09:18
mkovaciksambetts, hi, how about the HA spec ;)09:18
mkovacikaarefiev, morning! ^ :)09:18
sambettsirf the output of the command nova hypervisor-stats should show you the count of all the resources avaialble to it from Ironic09:19
sambettsmkovacik: will check it out this morning, I've been distracted by tinyIPA the last couple of days hehe09:19
irfok .. so it means all the bare metal resources ?09:20
mkovaciksambetts, if not I'm reminding you on Monday for sure :P09:20
sambettsirf: so if the ironic node is availiable and usable by nova, the count of memory_mb will increase by the amount that ironic node has09:20
sambettsthe same for the other stats, free disk and cpus09:21
sambettsmkovacik: hehe ;)09:22
*** mgould has joined #openstack-ironic09:26
irfohh ok ..so is there any way where i can say that nova has undrstood my BM node and it has now taken it under considertation for compute .. actually i dont have any stats befor adding BM node so dont know wheather it has added in nova compute or not09:32
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add portgroups to support LAG interfaces - API  https://review.openstack.org/20624409:33
mgouldmorning all09:33
mkovacikmgould, morning!09:33
irfmorning mgould09:34
mgouldmorning mkovacik irf09:36
irfohh ok ..so is there any way where i can say that nova has undrstood my BM node and it has now taken it under considertation for compute .. actually i dont have any stats befor adding BM node so dont know wheather it has added in nova compute or not...09:37
dtantsurMorning Ironic09:40
irfhere is the output of command : http://pastebin.com/XwruKUQa09:41
mgouldmorning dtantsur09:49
mkovacikdtantsur, morning09:49
*** fragatina has quit IRC09:52
irfplease suggest ...10:02
sivaramakrishnaHi irf,10:04
sivaramakrishnahow many nodes do you have?10:04
sivaramakrishnaif  you do a nova hypervisor-list10:04
sivaramakrishnathe ironic node uuid show as 'up'10:04
irfi ahve just single machine10:06
sivaramakrishnacan you run nova hypervisor-list10:07
irfnova hypervisor-list command shoes just one node10:07
irfwhihc is my compute node10:07
irflet me show the output in pastebin10:07
*** ekarlso has quit IRC10:08
irfhttp://pastebin.com/dfs6x0Uh10:08
*** hoangcx has quit IRC10:08
sivaramakrishnahm..your ironic node uuid isn't appearing there10:09
sivaramakrishnacheck if the power sync up happened successfully10:09
mkovacikdtantsur, since I've been annoying with it aarefiev and sambetts already I have to do the same to you :P How about the HA spec? ;)10:10
dtantsurmkovacik, slowly getting to it :)10:10
dtantsurETOOMANYEMAILS10:10
irfi guess not10:11
mkovacikyes  | mail > /dev/null & 2&110:11
irfthe reason being .. i am seeing some errors in the nova-schedulr logs10:12
sivaramakrishnacheck ironic-conductor logs as well10:12
irfok10:13
irfirconic conductor logs does not show any logs related to the issue i m facing10:13
irflet me paste the output10:14
irfthis is nova scheduler logs : http://pastebin.com/5zXaTgZU10:14
mgouldmkovacik, is the HA inspector spec now basically stable?10:15
sivaramakrishnais your node in available state.. and what's the power state?10:15
sivaramakrishnaoutput ot ironic node-list10:15
irfnode is in available state and is power on10:18
irfironic node-list shows my bare metal node and it is in power on mode10:18
sambettsif you run "ironic node validate", whats the output?10:19
sambettsironic node validate <uuid>10:20
*** electrofelix has quit IRC10:20
*** dims has quit IRC10:20
mkovacikmgould, yup, the spec should be safe to read now ;)10:21
mgouldmkovacik, :-)10:23
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add network drivers  https://review.openstack.org/28585210:24
irfhere is the output of the comamnd : http://pastebin.com/LECAKayX10:24
irfironic node-validate10:24
sambettsirf: can you paste an ironic node-list and ironic node-show <uuid> for that node10:25
irfok10:25
sambettsbtw we have our own paste service, paste.openstack.org if you want to use it10:26
irfhere it is : http://pastebin.com/Yvx2PJyW10:28
irfnext time sure10:28
*** electrofelix has joined #openstack-ironic10:32
sambettsirf: are you trying to hook Ironic into a hybrid BM and VM env?10:32
* dtantsur just reviewed a huge portgroups patch10:34
sambettsdtantsur: the API patch?10:35
dtantsuryep10:35
* dtantsur switches to the network drivers one10:35
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add network drivers  https://review.openstack.org/28585210:35
*** alaski is now known as lascii10:36
irfsorry dod not get10:36
irfwhat is that ?10:36
sambettsare you trying to make nova control BM via Ironic and VMs?10:37
irfmy opesntack rest of the services are deployed on VMs along with Ironic service and BM node is differemt ...10:38
sivaramakrishnadtansur: thanks for the review; had rebased it to include get_patch_values fix..10:38
sivaramakrishnamade required changes in the port/portgroups10:39
mgouldirf: if you succeed, will users be able to start VMs using "nova boot"?10:39
*** MattMan has quit IRC10:40
*** MattMan has joined #openstack-ironic10:40
dtantsurjroll, left a bulk of questions on the network drivers patch re interaction with the driver composition10:41
sambettsdtantsur: you make some good points there ;)10:43
irfi never tried that10:43
dtantsurok, now to the HA spec10:43
sambettsirf: I can see in your nova hypervisor stats output that you have a nova compute serving VMs10:44
sambettsirf: with 5 of them running10:44
mgouldirf, sure, but is that part of what you're trying to achieve? I think not, but it seems you've accidentally made that happen...10:44
irfyes10:44
sambettsirf: do you have 2 nova computes running, one for vms and one for Ironic?10:45
irfnop10:45
irfi have just sigle compute node10:45
irfi have just single compute node and one controller node ...10:46
irfon controller node all services are deployed10:46
irfalong with Ironic10:46
irfnova scheduler is running on controller node and nova compute service is running on compute node10:47
sambettsfrom novas perspective, Ironic is a hypervisor, so you need to have a nova compute running with the ironic virt driver running to interact with Ironic10:47
sambettshttp://docs.openstack.org/developer/ironic/deploy/install-guide.html#configure-compute-to-use-the-bare-metal-service10:47
irfok ..10:48
irflet me chk this doc10:48
irfbut i have already configured the nova conf10:48
irffle10:48
irffile10:48
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Update the deploy drivers with network flipping logic  https://review.openstack.org/21326210:48
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Added operator documentation for ironic portgroups  https://review.openstack.org/22849610:49
sivaramakrishnavsaienko:can you look at https://review.openstack.org/#/c/256364/61 .. is it half done or something?10:51
*** Goneri has quit IRC10:52
vdroksivaramakrishna: he'll be back on tuesday, it's holidays in Ukraine. That patch is done, tho some people requested to move it out of chain. I'd leave it as is for now10:53
sivaramakrishnathanks vdrok..10:56
sivaramakrishnaleaving it as is for now, will try to resolve merge conflict after a while.10:57
*** dmk0202 has quit IRC10:57
*** dmk0202 has joined #openstack-ironic10:57
*** sivaramakrishna has quit IRC10:58
*** jtomasek has quit IRC10:59
sambettsI personally think it should be broken out of the networking patch chain, and it'll need reworking anyway, because the way we handle hwinfo in devstack has changed recently too10:59
lucasagomesvmud213, awesome fix!11:02
lucasagomesvmud213, left a comment/suggestion regarding the release note, I find that a bit misleading (since it does not mention local boot, where GPT + BIOS w/o it is fine)11:02
lucasagomesother than that it LGTM11:02
*** dims has joined #openstack-ironic11:02
vmud213lucasagomes: thanks for the review. Will look into the comments and get back to you soon :)11:03
sambettsmgould: https://review.openstack.org/#/c/313103/, tinyipa -> http://logs.openstack.org/03/313103/1/check/gate-tempest-dsvm-ironic-pxe_ipa-tinyipa-src-nv/dda65de/console.html, coreos -> http://logs.openstack.org/03/313103/1/check/gate-tempest-dsvm-ironic-pxe_ipa-src/24af878/console.html11:06
sambettsmgould: check out the difference in test run times11:06
mgouldsambetts, 337s versus 675s - awesome!11:08
sambettsmgould: yeah :D need more test results because of potential differences in the speed of the test slave its landed on, but looks promising11:09
mgouldhurrah for N=1 samples :-)11:10
sambetts:-P11:11
mgouldsambetts, that's with precompiled Python libs on tinyipa?11:14
sambettsmgould: yup, so the gate is slower than my local machine which run in 220-23011:15
sambettstotal job time was 35 minutes, which is 21 minutes of devstack + 5 minutes of test run + something11:16
* mgould wonders what's taking up the other 9 minutes11:17
* sambetts guesses its the time to setup the slave, and time to clean up / post results 11:18
mgouldcan we reduce devstack time by eg installing fewer services? I guess that means we won't discover harmful interactions with other services11:18
mgouldbut OTOH, we'd have our gate broken by other projects less often...11:18
sambettsmgould: we already reduce the number services quite a bit11:19
sambettsI'm not sure what the slow part of devstack is, because we're already caching the repos etc so we don't have to git clone everytime11:21
sambettsdevananda was saying that we shouldn't need to be running swift for either agent or iscsi deployment :/11:22
sambettsso we may be able to looking killing that service11:22
jroll20 minutes sounds about right for devstack11:23
jrollalso, morning11:23
vdrokmorning jroll11:24
vdroksambetts: I think it's better to run swift in devstack, people are using it11:24
jroll+1, for the agent driver11:25
jrollfor iscsi, don't think it's needed11:25
vdrokyep11:25
sambettsjroll: devananda seemed to think we didn't even need it for agent deploy11:26
sambetts:/ not sure though11:26
mgouldmorning jroll11:26
vdroksambetts: we can use direct http links, not swift temp urls, but still, lots of people will be using swift11:26
jrollsambetts: technically we could use apache11:26
jrollbut yeah that, I'd rather not break tempurls11:26
sambettsah ok11:26
jrollmajority of our timeouts are test/build timeouts anyway, not job timeouts11:27
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent: Remove check on carrier from the DHCP wait loop during inspection  https://review.openstack.org/31351111:29
dtantsurlucasagomes, jroll, mind checking ^^ please? gives me a lot of headache downstream :(11:29
dtantsur(this is a one-line patch)11:30
lucasagomesdtantsur, will do11:30
jrollFirefox can't find the server at file.rdu.redhat.com11:33
jroll:|11:33
* jroll trusts, +211:33
dtantsurouch, this is an internal server11:33
* dtantsur complains about people using too many internal things for no reason11:34
sambettswow... nice to see some nic drivers not playing the game with the carrier field...11:34
dtantsurjroll, the thing there is very simple: has_carrier=False for all NIC's in the posted back data. which is obviously false.11:34
dtantsuror it would not be able to post back any data11:34
jrolldtantsur: right, I gathered that :)11:35
jrolljust was curious to see11:35
*** alex_xu has quit IRC11:42
*** baoli has joined #openstack-ironic11:42
mgouldsambetts, carrier field?11:43
*** yolanda has quit IRC11:43
lucasagomesjroll, dtantsur just approved that11:44
dtantsurthnx11:44
jrollty lucas11:44
lucasagomesmgould, yeah, that's a flag for ethernet devices to say if the plug is connected to it or not11:44
sambetts /sys/class/net/eth0/carrier11:44
sambettsmgould: ^11:44
sambettsetc11:44
dtantsuryeah... and it does not seem to play well with something on our ramdisks... maybe DIB magic again, dunno (and don't want to dive into it tbh)11:45
mgouldlucasagomes, sambetts thanks!11:46
sambettsdtantsur: have you got the IPA logs from the run that was in the bugzilla, I'd like to see if carrier == 0 or is carrier == The device not up, because both lead to has_carrier=False11:46
dtantsursambetts, we don't log that unfortunately11:46
dtantsurbut IIRC when device is not up, it throws an error11:46
sambettsit drops a log.debug11:46
*** alex_xu has joined #openstack-ironic11:47
sambettshttps://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/hardware.py#L41811:47
*** baoli_ has joined #openstack-ironic11:47
lucasagomessambetts, device is not up is operstate11:48
lucasagomesI think carrier should be independent of it no?11:48
* lucasagomes googles11:48
sambettsif the device isn't up you can't read the carrier11:48
dtantsursambetts, no such debug logs11:48
*** yolanda has joined #openstack-ironic11:48
sambettshttps://linuxconfig.org/how-to-detect-whether-a-physical-cable-is-connected-to-network-card-slot-on-linux11:48
sambettshmm :/11:48
dtantsurso it must be just "0"11:49
mgouldsambetts, clearly the answer is "rear-facing webcam + OpenCV"11:49
sambettsdtantsur: or any other string thats not 1 :-P11:49
lucasagomesinteresting, I always thought it would be independent of whether the iface is up or down11:49
dtantsurwell, if it's possible, then yeah11:49
dtantsuraccording to https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-class-net it can only be 1 or 011:50
sambettsdtantsur: no idea :-P it appears that some drivers aren't playing by the rules, so it could be anything11:50
*** baoli has quit IRC11:50
sambettsmgould: heh11:51
dtantsurthat's why I'm just dropping the check: I don't want to guess in which way a vendor may screw it >_<11:51
dtantsurmgould++11:51
dtantsuroh wonderful, and if interface is not up yet, carrier is not available. so yeah, it's a good idea to drop this check, it seems11:53
lucasagomesfolks if you have some time, mind taking a look at https://review.openstack.org/#/c/312575/ (to enable pyghmi to be installed from source on devstack)11:54
lucasagomeswe have a patch in project config with 2+2 waiting for this change to enable a job in pyghmi's gate11:54
lucasagomesthanks11:54
mat128dtantsur, sambetts, mgould, lucasagomes: (good morning) about dropping the carrier check11:55
mat128does that mean an imposed wait time of X (60s I think?) when I have only one NIC active but 2 NICs in a machine?11:55
mat128That's usually how our machines end up during that phase11:56
dtantsurmat128, morning. unfortunately, it does. we found no reliable way to avoid that.11:56
*** thrash|g0ne is now known as thrash11:57
mgouldmorning mat12811:57
mat128what do you do in a multi-NIC scenario, put them all on the same network that has a DHCP? I found this to be unreliable because you were assigned 2 IPs from the same subnet and routing would be broken half the time11:57
dtantsurmat128, you can tweak the wait time, if you're sure your ramdisk does not have the problem in question11:57
mgouldmat128, why were you using that setup? It doesn't sound very useful...11:58
* mgould <- networking n00b11:58
mat128we have multiple NICs and do bonding on the tenant network, but during deploying/cleaning/inspecting/any time we need IPA, we only have one NIC active11:58
mgouldah, OK11:58
mat128mainly because you can't PXE boot with a bond11:58
mgouldgotcha11:58
mat128because you need both ends to cooperate on the modes/etc.11:59
mat128hence the "bonding" kernel module :)11:59
* mgould nods11:59
mat128I would have hoped the carrier thing was reliable :(11:59
dtantsurit's not :(11:59
dtantsuryou can play with https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/cmd/agent.py#L118 if you know your hardware and ramdisk11:59
mgouldlet me think through the operstate thing out loud...12:01
mgouldoperstate = 1 if the NIC is powered up, and 0 if it's off, right?12:01
mat128dtantsur: but if we merge your change as-is, and I know my cards reliably report carrier state, I am forced to wait for the callback, right?12:01
mat128s/callback/timeout/12:01
mgouldso in mat128's case, operstate=0 for one NIC and 1 for the other, and it will stay that way throughout introspection12:01
dtantsurmat128, even if your card are fine with carrier, DIB does some interesting magic in its ramdisks12:02
mat128I guess I can always check on a real machine12:02
dtantsurmgould, operstate is not 1/0, it has a lot of values12:02
mgouldbut we might have a situation where a NIC gets powered on half-way through introspection, so we can't in general ignore NICs with operstate=012:02
mgouldoh, right12:02
mgouldbah12:02
dtantsurmgould, check https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-class-net12:02
mat128only on DIB ones or even CoreOS and TinyCore ?12:02
dtantsurmat128, we use DIB in our products, so I can't say for sure. I doubt anyone uses tinyipa in production right now (sambetts?)12:03
lucasagomesmat128, dtantsur not sure if this element was/is used in the bash ramdisk12:03
lucasagomeshttps://github.com/openstack/diskimage-builder/blob/master/elements/dhcp-all-interfaces/install.d/dhcp-all-interfaces.sh12:03
lucasagomesbut it may be the one12:03
mat128we are still on CoreOS right now, can,t move to tinyipa because of the lack of a shell on the screen :( lol12:03
dtantsurlucasagomes, it is and was12:03
sambettsmat128: lack of shell ?12:04
dtantsurmat128, it might happen you can set this option to e.g. 5 (seconds), and live happily. or even drop it to 0, because I think coreos forces NIC/DHCP initialization before running anything12:04
lucasagomesright, cause it does have a loop to wait for the link there12:04
mat128with coreos you can ssh to it, not sure (havent really checked) tinyipa does it12:04
mgoulddtantsur, OK, but the point is that operstate can change during the course of introspection?12:04
dtantsurmgould, looks like everything can change during introspection :(12:05
mgoulddammit12:05
dtantsurmgould, mat128, here is how it looked for us: http://paste.openstack.org/show/496321/12:06
sambettsmat128: it doesn't have ssh yet, but it has a shell if you serial console/KVM to the node running it12:06
dtantsurnotice network data collection before DHCP12:06
sambettsok, so I still think we're being screwed over by the dhcp-all-interfaces script here...12:06
dtantsuryep. so maybe for coreos it's safe to set inspection_dhcp_wait_timeout=012:07
sambettsI think IPA is starting in parrallel with that script, and breaking all the things12:07
mat128sambetts: thought IPA was running in inittab or something, I'll check on my end, but if it does we might simply switch12:07
dtantsursambetts, most likely you're right12:07
* dtantsur hates DIB12:07
mat128lol12:07
sambettsdtantsur: maybe we can modify DIB ironic agent to make it run after that script?12:08
sambettsdtantsur: that script ifups all the interfaces etc12:08
dtantsursambetts, I've no idea how to do that tbh. the dhcp thing is pretty weird there12:09
*** rbudden has joined #openstack-ironic12:09
sambettsdtantsur: they use a .service or .init script to activate it based on which init system your building into12:10
sambettsdtantsur: https://github.com/openstack/diskimage-builder/blob/master/elements/dhcp-all-interfaces/install.d/dhcp-interface%40.service12:10
sambettsdtantsur: maybe we can add an After= to ironic-agent to make sure it happens after that?12:11
* lucasagomes go grab some lunch12:11
*** lucasagomes is now known as lucas-hungry12:11
*** dprince has joined #openstack-ironic12:11
sambettsin here -> https://github.com/openstack/diskimage-builder/blob/master/elements/ironic-agent/install.d/ironic-agent-source-install/ironic-python-agent.service12:12
* sambetts isn't super familiar with systemd units so :/ 12:12
dtantsursambetts, after will work only if this script reports it's finishing.. which is unlikely IMO12:14
sambettsthis script actually only brings up links where carrier = 1 -> https://github.com/openstack/diskimage-builder/blob/master/elements/dhcp-all-interfaces/install.d/dhcp-all-interfaces.sh12:15
mgouldsambetts, me neither, but AIUI providing that kind of dependency graph info is most of the point of systemd12:16
mgouldso it should definitely be possible12:16
dtantsurthere is no one dhcp-all-interfaces unit btw. it's some kind of a special systemd unit12:16
dtantsurso I've no idea. we already depend on network-online, and it does not help much12:16
sambettsdhcp-all-interfaces' unit also depends on that which is why we're starting at the same time12:17
sambettsthis is how it gets installed -> https://github.com/openstack/diskimage-builder/blob/master/elements/dhcp-all-interfaces/install.d/50-dhcp-all-interfaces12:17
dtantsurno, it depends on network. these are different12:17
*** trown|outtypewww is now known as trown12:17
mgoulddtantsur, https://github.com/openstack/diskimage-builder/blob/master/elements/ironic-agent/install.d/ironic-agent-source-install/ironic-python-agent.service doesn't mention network-online12:18
dtantsurmgould, this one is not used in production fwiw12:18
dtantsuras we don't use source install12:18
dtantsur(but worth fixing, yeah)12:18
* dtantsur -> lunch, brb12:18
mgouldah, OK12:18
* mgould -> also lunch12:19
*** ekarlso has joined #openstack-ironic12:19
*** alex_xu has quit IRC12:22
*** alex_xu has joined #openstack-ironic12:25
sambettsdtantsur: is that a custom package you ship then?12:32
mat128fwiw, carrier detection is reliable on CoreOS, and operstate is up/down as expected12:36
*** ankkumar5 has quit IRC12:36
mat128http://imgur.com/9TU2LV612:37
*** ankkumar5 has joined #openstack-ironic12:37
sambettsmat128: I'm pretty sure the unreliable nature comes from the DIB dhcp-all-interfaces script being used to bring up every interface on the node, and that running in parrallel with IPA so IPA gets weird results12:38
dtantsurmat128, it works for me too, but not for some of our folks12:38
dtantsursambetts, well, the RDO package, yeah12:38
sambettsits a race condition12:38
*** vmud213 has quit IRC12:39
sambettsor I think thats part of it12:39
mat128maybe we can have a "trust sysfs" config option until it is resolved?12:39
mat128instead of dropping the checks12:39
sambettsdropping the checks does seem wrong to me :/12:40
*** mat128 is now known as mat128|afk12:42
*** ankkumar5 has quit IRC12:42
*** irf has quit IRC12:42
*** caiobo has joined #openstack-ironic12:42
*** hshiina has joined #openstack-ironic12:44
*** Goneri has joined #openstack-ironic12:44
*** ayoung has quit IRC12:48
*** athomas has quit IRC12:49
*** alejandrito has joined #openstack-ironic12:52
dtantsursambetts, carrier check is broken, that's it.12:52
dtantsurwe might think of better checks, but this one proved to not work12:52
dtantsuror rather: one carrier=1 is reliable. carrier=0 means nothing specific.12:53
dtantsurs/one/only/12:53
caiobogood morning everybody :) how are you all doing?12:54
dtantsurmorning caiobo! we're arguing about networking on Friday, what can be better?12:54
jrolldtantsur: wanna talk about api versions instead? :D12:54
*** mat128|afk is now known as mat12812:55
dtantsurjroll, of course \o/ what do we version today? :)12:55
sambettsdtantsur: maybe we should make it if operstate == up and carrier == 1 wait else don't wait for it?12:55
jrolloh idk, just wanted to make your friday more fun :)12:55
caiobomorning dtantsur, nothing can be better haha12:55
*** sacharya has joined #openstack-ironic12:55
TheJuliajroll: that is evil, i like it :)12:55
sambettshehehe12:55
caiobojroll: +++12:56
dtantsursambetts, and we get people broken again, because at this moment carrier=0...12:56
dtantsurbut it does not mean that there is no carrier, just.. because12:56
dtantsurjroll, lets version drivers, wdyt?12:56
jrolldtantsur: don't give me any ideas :)12:56
mat128dtantsur: the drivers API or drivers themselves?12:57
dtantsursambetts, I mean, I agree that's unfortunate. I just don't want to guess when this carrier check will break the next time..12:57
sambettsI'd be super surprised if it really is being reliable ... :/12:57
TheJuliaI actually had some people ask why we were not versioning drivers12:57
sambettsunreliable12:57
dtantsurmat128, both?12:57
jrollversioning drivers is easy, move them all out and rely on pypi versioning12:57
mat128dtantsur: interesting12:57
*** thiagop-afk has joined #openstack-ironic12:58
dtantsursambetts, well, I have a guy with carrier=0 on all interfaces... and whatever dirty magic DIB does (and it does!), it should not be the case in my mind12:58
dtantsurI mean, I'm pretty sure this DHCP script does not unplug the cable :)12:58
*** thiagop-afk is now known as thiagop12:58
dtantsurjroll, awesome, lets do it!!12:58
sambettsdtantsur: carrier = 0 before or after dhcp-all-interfaces has ifuped the interface?12:58
mat128dtantsur: do we know what magic it does so that carrier ends up 0 but still plugged in12:58
thiagopTGIF! Morning, Ironicers!12:58
mat128o/ thiagop12:58
sambettsand actually carrier 0 or has_carrier=False?12:59
dtantsursambetts, 1. during the process; 2. I can only say that carrier!=1 :)12:59
dtantsurmorning thiagop, happy Friday12:59
sambettsdtantsur: during which process? inspection?13:00
mat128dtantsur: do you know which kernel module was used for these interfaces?13:00
mat128or network card model13:00
dtantsursambetts, as you said, we have dhcp-all-interfaces racing with IPA, and during that carrier!=1, which is a lie to me13:00
*** sacharya has quit IRC13:00
dtantsurmat128, mmm. no, but I can ask, if it matters13:01
*** ifarkas has quit IRC13:01
dtantsurbut I suspect we're just missing something in how linux works with network interfaces, tbh13:01
mat128well, either DIB does something that we can just reproduce / workaround, or it's something in the modules themselves13:01
*** jaypipes has joined #openstack-ironic13:02
*** yolanda has quit IRC13:02
mat128https://github.com/torvalds/linux/blob/master/net/core/net-sysfs.c#L18013:02
openstackgerritAparna proposed openstack/proliantutils: Using python 'subprocess' to execute hpssacli commands  https://review.openstack.org/31325413:03
*** athomas has joined #openstack-ironic13:04
sambettsIMO I think we should also be checking IF_OPER_UP on the interfaces in that loop too, otherwise we'll be waiting for timeout on nics that'll never get an IP because they are down13:05
dtantsursambetts, mat128, I think our problem here is that "carrier" in /sys is not a *hardware* property. it's a property of the interface, which is a software entity, that can be down or can be doing something13:05
dtantsursambetts, I'm open to any property that is derived from underlying hardware, not from a high-level object13:06
sambettsits as much a hw property as /sys/class/net/<nic>/address is13:06
*** piet has joined #openstack-ironic13:06
mat128https://www.kernel.org/doc/Documentation/networking/operstates.txt13:06
mat128IFF_LOWER_UP maybe?13:07
*** yolanda has joined #openstack-ironic13:07
dtantsursambetts, as we see, it's not. it depends on state of software (i.e. interface implementation)13:08
mat128interesting find: If I ifconfig XX down, then carrier returns "Invalid argument", after ifup, kernel says "registered PHC device on XX", then carrier is 0, once kernel reports "link becomes ready" then carrier becomes 113:08
dtantsurso, my point with this patch is to get rid of a check that does not work, making sure we can also backport it13:09
mat128so in that case I guess I could have carrier=0 on all interfaces, if it's in the middle if getting brought back up13:09
dtantsurthen we can iterate on how to improve that. sambetts, mat128, are you fine with that?13:09
mat128dtantsur: merging your patch as it means an imposed wait time for everyone, which bothers me a little bit13:10
dtantsurmat128, more than breaking people?13:10
mat128can we identify in which case they are broken? so far we only have "DIB does magic and it breaks"13:10
mat128I think there is more to it13:10
dtantsurmat128, I welcome any ideas :) it's a bit hard to debug a random failure on a ramdisk, y'know..13:12
mat128dtantsur: I perfectly understand that, don't worry :)13:12
dtantsur:) I'm totally fine with reverting it as soon as we have anything to replace. the last thing I want to do, though, is to dig into DIB13:13
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: SSH: add SSHIcsConsole class  https://review.openstack.org/30718113:14
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: IPMItool: add IPMIIcsConsole class  https://review.openstack.org/30171513:14
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: console utils: add methods for ironic console server  https://review.openstack.org/30171413:14
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: conductor manager: apply console-log related methods  https://review.openstack.org/30171713:14
openstackgerritAkira YOSHIYAMA proposed openstack/ironic: WIP: REST API: add an API controller for console log  https://review.openstack.org/30171813:14
*** jjohnson2 has joined #openstack-ironic13:15
*** cdearborn has joined #openstack-ironic13:17
*** mtanino has joined #openstack-ironic13:17
*** lucas-hungry is now known as lucasagomes13:18
*** piet has quit IRC13:19
*** piet has joined #openstack-ironic13:20
jrollfyi, gate is going to be clogged up for a bit13:21
jrollthere is suddenly tens of thousands of jobs in the queue :)13:21
lucasagomes:-(13:21
caiobohey guys, does all patches need release notes?13:23
vdrokheh, much monasca-analytics, wow :)13:23
jrollcaiobo: not all, but if it fixes a significant bug, adds a feature, has an upgrade impact, etc, it needs one13:24
dtantsurFriday gate \o/13:24
* dtantsur goes to the beach13:24
jroll++13:24
*** pcaruana has quit IRC13:25
* lucasagomes is thinking about removing the python-daemon depenency from VirtualBMC if the g-r ppl pushes back on it13:25
caiobojroll: oh I see, so even bugfix can have release notes. Thanks  :)13:26
jrollnp13:26
*** jcoufal has joined #openstack-ironic13:26
*** mtanino has quit IRC13:27
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Modified IronicNode mock to use single source of data  https://review.openstack.org/30792713:30
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Changed action from 'deploy' to 'active'  https://review.openstack.org/30792613:30
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: IronicNodeProvision resource  https://review.openstack.org/28245713:30
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Added styling for driver list  https://review.openstack.org/30173513:30
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Added provision state manipulation  https://review.openstack.org/30792813:30
*** joprovost has joined #openstack-ironic13:30
openstackgerritJarrod Johnson proposed openstack/pyghmi: Implement Python3 compatibility  https://review.openstack.org/31301213:32
*** jjohnson2_ has joined #openstack-ironic13:33
*** [4]cdearborn has joined #openstack-ironic13:33
*** jjohnson2 has quit IRC13:35
mkovacikdtantsur, mgould, sambetts thx for the reviews! :)13:43
*** ifarkas has joined #openstack-ironic13:43
*** wajdi has joined #openstack-ironic13:44
mgouldmkovacik, np13:44
wajdigood morning all13:46
*** absubram has joined #openstack-ironic13:46
*** cdearborn has quit IRC13:46
jlvillallucasagomes, Glad you hit the resource creation error too13:46
wajdijlvillal: hi! I missed a chunk of messages yesterday near the end of the day. How did your test run go for grenade?13:47
jlvillalI moved to using tinyipa13:47
*** absubram_ has joined #openstack-ironic13:47
jlvillalNo change in the failure. And did a little yak-shaving over 'short_source'13:47
lucasagomesjlvillal, :-( yeah I'm partially looking into it13:48
jlvillalsambetts, https://review.openstack.org/31313213:48
NobodyCamGood Morning Iroinc'ers and OFC ...13:49
NobodyCamTGIF :)13:49
wajdigood morning!13:49
mkovacik:D13:49
*** jaypipes has quit IRC13:50
NobodyCammorning wajdi mkovacik13:50
mkovacikNobodyCam, o/13:50
mgouldmorning NobodyCam13:50
mkovacikmorning!13:50
*** trown is now known as trown|outtypewww13:50
NobodyCammorning mgould13:50
NobodyCam:)13:50
thiagopmorning NobodyCam13:50
*** absubram has quit IRC13:50
*** absubram_ is now known as absubram13:50
NobodyCamhey hey thiagop morning13:51
jrolldoes anyone want to paste the resource creation error message anywhere?13:51
dtantsurmorning jlvillal, wajdi, NobodyCam13:53
wajdimorning dtantsur13:53
jlvillalGood morning dtantsur wajdi lucasagomes jroll NobodyCam thiagop mgould sambetts mkovacik and everyone else13:54
jlvillalmgould, Any luck with the grenade stuff13:54
mkovacikjlvillal, morning! :)13:54
jlvillalmgould, We should try to keep the etherpad updated each day with the current status13:54
jlvillalmgould, I saw "we" since you were kind enough to volunteer to be a cat herder for Grenade work too :)13:55
jlvillals/I saw/I say/13:55
mgouldjlvillal, yep13:55
* mgould is feeling extra-braindead today, one moment...13:56
*** dansmith is now known as snotdan13:56
mgouldjlvillal, I updated the Etherpad with the work I did with sambetts yesterday13:57
NobodyCammorning dtantsur jroll jlvillal13:57
jlvillalmgould, Great and thanks :)13:57
sambettsjlvillal: https://review.openstack.org/#/c/313103/13:57
jlvillaljroll, On the error. Line 18 & 19 of https://etherpad.openstack.org/p/ironic-newton-grenade-whiteboard13:58
NobodyCammorning sambetts13:58
sambettso/ NobodyCam TGIF13:58
jlvillaljroll, Is where I am at13:58
NobodyCamOh yes13:58
*** snotdan is now known as superdan13:58
jlvillalsambetts, Hopefully you saw the patch to use the qemu https://github.com repo instead of git://qeum-????13:59
jlvillalIt got merged yesterday13:59
*** jtomasek has joined #openstack-ironic13:59
sambettsjlvillal: no i didn't but thats awesome :D13:59
jlvillalHmmm. snotdan doesn't have the ring of superdan13:59
superdanjlvillal: indeed.. I was trying to be accurate, but.. it was a little gross :)14:00
jlvillalsuperdan, I had an idea of what was happening :)14:00
jlvillalsambetts, https://review.openstack.org/31311514:00
jrolljlvillal: ok cool14:00
*** ametts has joined #openstack-ironic14:00
*** Goneri has quit IRC14:01
superdanjlvillal: yeah, the "post summit sickness"14:01
jlvillalsuperdan, A lot of people at our San Antonio site seemed to have been hit as well :(14:01
superdanyeah, sucks14:01
* sambetts also got hit with summit sickness14:02
caiobomorning NobodyCam14:02
jlvillaljroll, I started a run right before I left work last night. Will be checking it when I get back in. I will update etherpad with any new info.14:02
jlvillalsorry sambetts :(14:02
jroll2016-05-03 17:41:55.650 | + /opt/stack/new/grenade/projects/60_nova/resources.sh:create:94 :   openstack ip floating add 172.24.5.6 nova_server114:02
jroll2016-05-03 17:41:57.241 | No nw_info cache associated with instance (HTTP 400) (Request-ID: req-08647e54-1512-4285-adfe-6e1e1b1089a1)14:02
jrollhmm, seems odd14:02
jrollI feel like we just need a delay there?14:03
jlvillaljroll, Maybe but I'm not sure. Previously I did some work to re-run that after the test pass failed. Just the resources.sh script. And it kept failing.14:03
NobodyCammorning caiobo14:03
jlvillalI will try again today.14:03
jrolljlvillal: I mean between the server create and the flip create14:04
jlvillaljroll, As an FYI, the 'vm-setup.sh' script works well on RackSpace VMs.  https://github.com/JohnVillalovos/devstack-gate-test14:04
* jroll will play around14:04
jrollew, VMs.14:04
jlvillalI tried it a couple days ago.14:04
jroll:P14:04
*** alejandrito has quit IRC14:05
*** catintheroof has joined #openstack-ironic14:05
jlvillaljroll, Ah okay on where the delay should be. I will check that out. Thanks14:05
jrolljlvillal: blah, it has a --wait so maybe not14:05
jrollidk, I'll play around14:06
*** ayoung has joined #openstack-ironic14:07
jjohnson2_lucasagomes, fyi, I did another patch.  I stopped doing range = xrange, so I'm no longer overwriting a builtin14:07
lucasagomesjjohnson2_, cool, will take a look soon-ish14:08
lucasagomesthanks for that14:08
* mgould -> relocate14:08
jjohnson2_lucasagomes, I left the other stuff as-is, since I'm also supporting some other projects where pulling in six would represent some silly work14:08
*** mgould has quit IRC14:08
*** mtanino has joined #openstack-ironic14:09
jjohnson2_long story, in no way justifiable, but easier for me to roll with it over these relatively little things14:09
*** mgould has joined #openstack-ironic14:11
lucasagomesjjohnson2_, yeah for the other stuff it's fine I guess, just the thing about overwriting the builtin wasn't very nice14:13
lucasagomesjjohnson2_, in case we have to deal with more compat problems we can rethink about using six14:14
*** Goneri has joined #openstack-ironic14:14
*** catintheroof has quit IRC14:15
*** vishwanathj has joined #openstack-ironic14:15
-openstackstatus- NOTICE: Zuul is currently recovering from a large number of changes, it will take a few hours until your job is processed. Please have patience and enjoy a great weekend!14:15
jjohnson2_lucasagomes, yeah, I have done very basic testing14:15
jjohnson2_lucasagomes, and I know that the server side only works in python2, and I have more to do14:16
jjohnson2_but I might have to park it for a few days while I handle some urgent requests...14:16
*** links has quit IRC14:17
lucasagomesjjohnson2_, yeah, it's all good14:18
lucasagomesthanks for looking into it btw14:19
* jroll git clone git://git.openstack.org/openstack-dev/devstack just to spite jlvillal :D14:19
jlvillaljroll, Good luck with that ;)14:19
*** jaypipes has joined #openstack-ironic14:20
lucasagomesmat128, fyi I've removed the python-daemon dependency from VirtualBMC as part of the refactor, since it's not present in the g-r and I only need a simple double-fork to detach the process from its parent/session14:21
* lucasagomes expect it to pass Jenkins now14:21
*** ChrisAusten has joined #openstack-ironic14:22
jjohnson2_is there a hard timeline for everything needing Python 3 support I should conspicuously be paying attention to?14:22
jrolljjohnson2_: last week14:24
* jroll is joking14:24
lucasagomesjjohnson2_, not that I know of, but pretty much all ironic projects do support python3 so14:24
jjohnson2_jroll, let me get my delorean14:24
jroll\o/14:24
lucasagomesexcept for pyghmi14:24
jrolllucasagomes: this is waiting on you jfyi https://bugs.launchpad.net/ironic/+bug/156113714:24
openstackLaunchpad bug 1561137 in Ironic "[RFE] Add support for operators on root device hints" [Wishlist,Confirmed] - Assigned to Lucas Alvares Gomes (lucasagomes)14:24
jrollwhen you have time, no rush14:24
lucasagomesjroll, no spec needed right?14:25
lucasagomesI think the plumbing patch for IPA already merged (/me checks)14:25
lucasagomesoh it didn't,14:26
lucasagomeshttps://review.openstack.org/#/c/296539/14:26
jrolllucasagomes: well, the last comment is "can you update the description to match ___" :P14:26
lucasagomesjroll, fair enough, will do it today14:26
*** r-daneel has joined #openstack-ironic14:26
jrollcool, thanks14:26
*** catintheroof has joined #openstack-ironic14:27
*** dprince has quit IRC14:37
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Created IronicNodePower resource  https://review.openstack.org/31357814:37
*** hshiina has quit IRC14:40
*** absubram has quit IRC14:41
*** e0ne has quit IRC14:43
*** e0ne has joined #openstack-ironic14:44
-openstackstatus- NOTICE: Zuul has been restarted. As a results, we only preserved patches in the gate queue. Be sure to recheck your patches in gerrit if needed.14:46
sambettsjroll: Does IPA really need i18n? babel is adding 21mb of bloat before compression, and ~10mb after14:48
jrollsambetts: no, who put that in there14:48
jrollheh, it isn't used14:48
sambettsits a dependecy of oslo.i18n14:49
openstackgerritJim Rollenhagen proposed openstack/ironic-python-agent: Remove oslo.i18n  https://review.openstack.org/31358614:49
sambettshahaha14:49
sambettsawesome! more shrinkage!14:49
jrolllet's make sure something else doesn't install it before landing14:50
krotscheckI'm trying to test power state management with the fake driver. Is the "unknown" power state a legitimate state? In a real deployment, when/how would that change?14:50
jrollkrotscheck: unknown is the state at enroll time, and when the bmc cannot be reached14:51
jrollis null in DB, None in api response14:51
krotscheckjroll: Right - so the fake driver has allowed me to advance a node to the active state, but the power state never changed.14:51
* dtantsur will remove https://github.com/rdo-packages/ironic-python-agent-distgit/blob/rpm-master/openstack-ironic-python-agent.spec#L64 then14:51
jrollkrotscheck: interesting14:52
krotscheckjroll: I'm guessing that's not intentional.14:52
sambettsI guess it makes sense though, if its fake then is it on or is it off, is the cat dead?14:53
jrollkrotscheck: here's the code https://github.com/openstack/ironic/blob/master/ironic/drivers/modules/fake.py#L3414:53
jrollkrotscheck: get_power_state is what ensures the bmc can be reached14:53
krotscheckjroll: hrm.... ok, so it just does what I ask it to.14:54
jrollkrotscheck: right, and then it should return what you did as current power state14:54
krotscheckNot super useful, but I think i can handle that.14:54
*** jaypipes has quit IRC14:55
jrollwell, what would you rather it returns :)14:56
*** caiobo is now known as caiobo-afk14:57
krotscheckjroll: well, it'd be better if it represented a real deployment more. So, say, if the usual place for power state to be resolved is when it moves into the maange state, then that's where that value should be set.14:57
krotscheckBut for now I can special case it ;_14:57
krotscheck:)14:57
*** caiobo-afk has quit IRC14:57
jrollkrotscheck: yeah I've thought about that14:57
*** rloo has joined #openstack-ironic14:59
* sambetts wonders if we should add a validation into that driver so that it prevents it moving to managable without being in either power on or power offf14:59
sambettsforcing you to set it15:00
krotschecksambetts: Is that something which the API client must do, or is that something done by the driver?15:01
sambettsI'm suggesting that we make it so that you need to do a set-power-state on that node while in enroll in order to move it to managable15:04
sambettsironic node-set-power-state ... etc15:04
krotschecksambetts: So the user would, after enrolling, have to set power state, and then set manageable? This implies that a managed node cannot be powered off.15:05
*** Guest42930 has quit IRC15:05
*** david-lyle has joined #openstack-ironic15:06
krotscheckDifferent question - I'm getting 400 invalid when I gtry to issue PUT /state/power reboot - I'm guessing there's no API driven way to manually perform a reboot?15:06
sambettsthere is the fake driver just doesn't support it15:07
vdroksambetts: hrm, I thought that a node in enroll == can not do power control in general case15:08
sambettshmmm :/15:09
sambettsthe fake driver is a hard one because its decided for testing, and basically just fudges everything15:10
*** jaypipes has joined #openstack-ironic15:11
*** mjturek1 has quit IRC15:12
openstackgerritMerged openstack/ironic: DevStack: Support to install pyghmi from source  https://review.openstack.org/31257515:12
krotscheckjroll: I see a 'reboot' power state in http://docs.openstack.org/developer/ironic/_modules/ironic/common/states.html, but no way to access it via the API. Is that correct?15:14
krotscheck(As in- ironic handles all the rebooting, you can't do it as a user)15:14
sambettsit should work, I'm pretty sure I've done it15:15
jrollkrotscheck: no, PUT /v1/nodes/uuid/states/power {target: reboot}15:15
jrollkrotscheck: fake driver might not support it, like sam said15:15
krotscheckoh, right15:16
krotscheckdrat15:16
krotscheckHrm. I don't suppose there's a way for me to ask whether a node's driver supports that.15:16
* krotscheck ponders15:17
*** ChrisAusten has quit IRC15:18
*** ChrisAusten has joined #openstack-ironic15:20
jrolljlvillal: do you have a grenade patchset going for "things to make this work"?15:20
jrollfound the create resource bug, I think15:20
jlvillaljroll: I have a bunch of patches in the 'update-projects.sh' file15:21
jlvillalAnd awesome :)15:21
wajdi^^ yeah that's what I did to add patches yesterday15:21
jrolljlvillal: well, there's a bug in projects/60_nova/resources.sh15:21
jrollwith neutron enabled15:21
jlvillaljroll: https://github.com/JohnVillalovos/devstack-gate-test/blob/master/ansible/roles/devstack-gate/files/update-projects.sh15:21
jrolloh my15:22
jrollokay, I'll spin a new patch for this specifically and you can put it there :)15:22
jlvillalLions and Tigers and Bears...15:22
*** ijw has joined #openstack-ironic15:22
jlvillaljroll: Thanks15:22
JayFdevananda: was out sick yesterday afternoon; re: manual cleaning -- I've never used it and have no access to anything that makes calls to it15:22
jlvillalJayF: Did you get the post-summit illness? :(15:23
JayFNo, just sorta a chronic thing that triggers from time to time15:23
jlvillalI've heard a lot of reports from other people too15:23
jlvillalJayF: I hope you feel better today :)15:23
NobodyCammorning JayF hope your feeling better this morning15:24
JayFme too :P15:24
JayFthanks15:24
jrollbbiab/b 17015:28
jroll...15:28
jrollbbiab.15:28
*** sacharya has joined #openstack-ironic15:31
*** ChrisAusten has quit IRC15:32
*** ChrisAusten has joined #openstack-ironic15:32
*** dmk0202 has quit IRC15:33
*** jtomasek has quit IRC15:33
dtantsurhave a great weekend15:33
NobodyCamyou out dtantsur15:34
*** dmk0202 has joined #openstack-ironic15:34
*** keedya has joined #openstack-ironic15:35
*** dtantsur is now known as dtantsur|afk15:35
*** jaypipes is now known as leakypipes15:36
*** catintheroof has quit IRC15:36
mgouldo/ dtantsur|afk15:36
*** absubram has joined #openstack-ironic15:36
lucasagomesjjohnson2_, FYI, the patch adding the ironic job in the pyghmi's gate have been merged15:38
*** jcoufal has quit IRC15:38
jjohnson2_lucasagomes, ok, guess I'll give it a try with recheck this python3 thing15:38
keedyagood morning o/15:38
lucasagomesjjohnson2_, I just did :-)15:38
jjohnson2_oh15:38
keedyadtantsur: thanks for the review :)15:39
*** dmk0202 has quit IRC15:39
lucasagomesit's non-voting from now, we can make it voting after we gather some data about it (to make sure it's stable)15:40
jjohnson2_well, it should fail that patch15:40
jjohnson2_that patch as-is breaks python2 execution of the server at the moment15:41
*** fragatina has joined #openstack-ironic15:41
*** rloo has quit IRC15:42
*** rloo has joined #openstack-ironic15:42
*** jcoufal has joined #openstack-ironic15:44
JayFI'm going to put up two patches to project-config today: 1 to make tinyipa the default ramdisk, and 1 to use ipxe for the majority of our jobs15:44
lucasagomesJayF, cool, I thought the ipxe one was a thing already15:46
devanandamorning, all15:47
JayFI have a patch changing the ironic default15:48
JayFin devstack15:48
JayFto enable ipxe15:48
JayFbut I think given how the gate jobs are factored we should do those separately15:48
*** Nisha has joined #openstack-ironic15:49
*** Nisha has quit IRC15:54
lucasagomesthis change https://review.openstack.org/#/c/308965/ has 2+2 already, someone wants to give it a last review (or not) ?15:54
*** rloo has quit IRC15:55
*** rloo has joined #openstack-ironic15:56
*** davideagnello has joined #openstack-ironic15:57
*** ChrisAusten has quit IRC15:57
vdroklucasagomes: hey, have you seen something like http://paste.openstack.org/show/496350/ ?16:00
vdrokvbmc command takes ~250 seconds to fail16:01
* lucasagomes checks16:01
vdrokI'm trying to run smoke on 3 tinyipa vms with vbmc16:01
*** mjturek1 has joined #openstack-ironic16:02
lucasagomesvdrok, oh that sucks... is it vbmc or ipmitool itself?16:02
* lucasagomes do some tests16:02
vdrokthat I don't know :)16:02
vdrokdoes vbmc have a separate log?16:03
*** yolanda has quit IRC16:03
sambettsgahhhh why does oslo.concurrency require Babel....16:03
lucasagomesvdrok, you can configure it yes, if you create a file ~/.vbmc/virtualbmc.conf with a [log] debug=True logfile=<path to your log file>16:04
lucasagomesvdrok, it will log all the actions running on the vbmc daemons16:04
vdroklucasagomes: thanks, will do16:04
lucasagomesvdrok, so apparently it's ipmitool, I just ran it against a port that is not being listened16:04
vdroklucasagomes: ipmitool itself seems to be ok when running from commandline at least16:04
lucasagomesand it's taking a lot of time16:04
lucasagomesvdrok, try it out, ipmitool -I lanplus -U admin -P password -H 127.0.0.1 -p 50000 -R 12 -N 5 power status16:04
lucasagomesI wonder if the vbmc daemon stopped listening for some reason (we need the logs for that)16:05
lucasagomesto know that*16:05
vdroklucasagomes: yep, that port is reported to be listened16:05
JayFI played with vbmc a bit in a highly scaled out onmetal devstack setup16:05
JayFand found a bit of slowness compared to ssh driver16:05
JayFI can reproduce that environment if you all wanna take a look at perf with 32/64 nodes16:06
*** derekh has quit IRC16:06
*** mjturek1 has quit IRC16:06
*** aginwala has joined #openstack-ironic16:06
lucasagomesJayF, oh that's cool to have some data like that16:06
*** mjturek1 has joined #openstack-ironic16:07
JayFI will start spinning one up once I'm done with these meetings and I can drop your key on16:08
vdroklucasagomes: aha, logging is enabled already :) I see 2016-05-06 18:51:08,184.184 24508 DEBUG VirtualBMC [-] Set boot device called for node-1 with boot device "network" there, no errors16:08
vdroktiming is close, so seems to be it16:08
*** johnthetubaguy has quit IRC16:08
lucasagomesvdrok, oh16:08
*** johnthetubaguy has joined #openstack-ironic16:08
lucasagomesvdrok, there's also this hardcoded timeout on pyghmi's session https://github.com/openstack/pyghmi/blob/65219f2353a62fe38eb4405d9ab70f13cc4f4af9/pyghmi/ipmi/bmc.py#L14716:09
vdroklucasagomes: you suggest to try pyghmi?16:10
lucasagomesvdrok, no, I mean vbmc uses pyghmi's on the background16:10
vdrokah16:10
lucasagomesI wonder if that 30s timeout (because it's a bit high) implicates in something16:10
lucasagomesperhaps making it configurable would be a good thing to have16:11
* lucasagomes sends a patch out to pyghmi16:11
*** piet has quit IRC16:11
*** stendulker has joined #openstack-ironic16:11
vdroklucasagomes: btw, I think pas-ha was having something similar, https://bugs.launchpad.net/ironic/+bug/1548652, but it was marked invalid by someone16:13
openstackLaunchpad bug 1548652 in Ironic "tmp file for ipmi password might get reused which leads to failure" [Undecided,Invalid] - Assigned to Ramamani Yeleswarapu (ramamani-yeleswarapu)16:13
vdrokjust recalled he said smth about it16:13
lucasagomesjlvillal, what errors you are seem on grenade? I'm now at this one here http://paste.openstack.org/show/496359/16:14
openstackgerritMario Villaplana proposed openstack/ironic: Add notification base classes and docs  https://review.openstack.org/29846116:14
jlvillallucasagomes: I think same.  jroll has proposed a patch16:14
lucasagomesjlvillal, before it was complaining about openvswitch-switch not being started, so before I run ./ironic-grenade.sh I've installed openvswitch-switch and started it manually16:14
lucasagomesjlvillal, oh right16:14
jlvillalJim Rollenhagen proposed openstack-dev/grenade: Allow creating floating ip address with Neutron enabled  https://review.openstack.org/31360016:15
lucasagomescool16:15
* lucasagomes looks16:15
jrollyeah, likely it16:15
jlvillallucasagomes: On a different note, I confused a Brazilian guy at work when I said 'Oy' at the coffee machine.16:15
jrolllucasagomes: check grenade log, if it's failing at floating ip create then that should do it16:15
jlvillalHe wondered if I was Brazilian for a moment :)16:15
lucasagomesvdrok, a-ha ok so it's ipmitool that is taking a long time to return (4minutes like it says in the bug), see: http://paste.openstack.org/show/496361/16:16
lucasagomesjlvillal, lol cool16:16
lucasagomesjroll, yeah, that's exactly the failure I'm seeing now, thanks for the patch16:17
wajdiisn't "No nw_info cache associated with instance" resolved by setting the IRONIC_RAMDISK_TYPE to tinyipa? When I ran my test yesterday after setting that, I did not receive that error anymore16:17
jrolllucasagomes: np, haven't tested with the full stack but that should do it16:18
jrollI think/hope :)16:18
lucasagomesyea, I will give it a run here16:18
jrollnw info cache thing is unrelated to my patch and if that's the issue then it won't help16:18
lucasagomesproblem is that it takes time :-)16:18
lucasagomesI usually run it and go do something else heh16:18
jlvillalwajdi: I don't know...16:18
jlvillallucasagomes: Same here.  A lot of time...16:19
jlvillal:(16:19
lucasagomesacutally we should look at how to improve that, we need to get faster results16:19
jrollright, that's why I'm not doing that and rather standing up devstack and running through it manually16:19
jlvillalAgreed! I'm just not sure how.16:19
lucasagomesI understand the scripts doesn't care becuase it runs on gate and it should just retunr the failure w/o retrying16:19
jrollwell, not doing it for every change16:19
lucasagomesbut damn, it takes an hour16:19
*** piet has joined #openstack-ironic16:19
lucasagomesjroll, yeah that's better for smoking tests but we are not there yet16:19
*** piet has quit IRC16:19
lucasagomesit fails before that16:19
jrolllucasagomes: well, that's what I did here, I found the failure and kinda went through the script manually16:20
*** frontrunner has joined #openstack-ironic16:21
* jroll takes his dog to the groomers, bbl16:22
*** e0ne has quit IRC16:23
*** klindgren has quit IRC16:24
*** klindgren has joined #openstack-ironic16:25
*** dprince has joined #openstack-ironic16:25
*** catintheroof has joined #openstack-ironic16:27
*** devananda has quit IRC16:28
*** mariojv has quit IRC16:28
*** elfosardo has quit IRC16:28
*** mtreinish has quit IRC16:28
*** stevebaker has quit IRC16:29
*** devananda has joined #openstack-ironic16:30
sambettslucasagomes: I really want to work out why devstack is slow as treacle16:30
sambettslucasagomes: 20+ minutes to install and start 10 or so services is crazy16:31
wajdiI'm sharing my latest run and the output I got w.r.t grenade: http://paste.openstack.org/show/496366/16:31
devanandaJayF: ok, np. I've just put a TODO in the docs, since I don't have any example code to use either for that16:31
jlvillalAnyone know if there is a slide deck on Ironic and what it does on a high level?16:33
jlvillalMy program manager is asking me...16:33
*** jaybeale has joined #openstack-ironic16:33
*** stevebaker has joined #openstack-ironic16:33
*** jaybeale has quit IRC16:34
sambetts\0/ latest run using my even further optimised tinyipa took 56 seconds from power on to lookup16:34
wajdi^^ \o/16:34
mgouldsambetts, woo!16:34
sambettsdown to 50mb now16:34
lucasagomessambetts, :-/ yeah it's slow16:35
*** mtreinish has joined #openstack-ironic16:35
lucasagomesit's procedural as well16:35
sambettslucasagomes: yeah I think thats what kills it16:35
mgouldsambetts, I have updated the etherpad accordingly :_)16:35
mgould:-)16:35
mgouldsambetts, did you manage to eliminate Babel?16:36
sambettsmgould: we can work out the time to transfer 1 mb now because we know how long it took to boot 59 mb and how long it took to boot 50mb and we can work out the difference I think right?16:36
*** mari0jv has joined #openstack-ironic16:36
*** chopmann has quit IRC16:36
sambettsmgould: No :( because its imported by other oslo stuff, but I've manually killed a bunch of the .dat files that are shipped as part of Babel leaving only en_US which has eliminated like 20mb16:37
sambettspre-compress16:37
*** mariojv has joined #openstack-ironic16:38
devanandasambetts: be mindful of making timing assumptions in the upstream test envs - there's a fair amount of variability between node providers and between test runs in general16:38
devanandasambetts: also, sub-minute boot is great progress!16:39
wajdimgould: are you still receiving: "Unable to locate package liberasurecode-dev"?16:39
wajdior is that old..16:39
openstackgerritLucas Alvares Gomes proposed openstack/pyghmi: DO NOT REVIEW, TESTING GATE  https://review.openstack.org/31363816:40
*** dmk0202 has joined #openstack-ironic16:40
*** elfosardo has joined #openstack-ironic16:40
*** mgould has quit IRC16:40
*** athomas has quit IRC16:45
*** aginwala has quit IRC16:46
klindgrenhello - I am trying to figure out what the status is of the nova -> ironic interaction re: scheduling.  Specifically re: availability-zones and the interaction between ironic and nova-compute.16:46
klindgrenhttp://lists.openstack.org/pipermail/openstack-dev/2015-December/082011.html16:46
sambettsdevananda: very true, even though I'm running a similar size VM for running my devstack mine tends complete a full BareMetalBasicOps about 20-30seconds faster than a couple of gate runs I've seen16:47
klindgrenis ^^ still the state of the things.  I was in the newton nova -> Ironic cross project talk where you were talking about raid, which from an operator view was painful.16:47
JayFklindgren: http://specs.openstack.org/openstack/ironic-specs/priorities/newton-priorities.html16:48
JayFklindgren: we have two of the needed APIs for that as priorities this cycle16:48
devanandaklindgren: not much has changed as far as nova/ironic scheduling *yet*. A lot of things are being designed / planned for this cycle.16:48
JayFklindgren: and the multiple compute hosts piece16:48
lucasagomessambetts, dtantsur|afk mat128 sorry on the carrier stuff I wasn't paying much attention on IRC, I see there's no consensus whether we should or should not rely on that16:49
lucasagomesso I've revoked the approval16:49
*** piet has joined #openstack-ironic16:49
lucasagomesapproval of that patch*16:49
klindgrenjayf, devananda  thanks16:49
*** electrofelix has quit IRC16:50
*** garthb has joined #openstack-ironic16:50
openstackgerritMichael Krotscheck proposed openstack/ironic-webclient: Power state may now be managed  https://review.openstack.org/31364116:50
*** Nisha has joined #openstack-ironic16:50
*** [4]cdearborn has quit IRC16:53
*** ChrisAusten has joined #openstack-ironic16:55
*** dmk0202 has quit IRC16:55
*** dmk0202 has joined #openstack-ironic16:57
*** mari0jv has left #openstack-ironic16:57
lucasagomesjroll, fwiw, your patch took me further in the grenade thing http://paste.openstack.org/show/496372/16:57
* lucasagomes will create some loop devices manually16:58
*** wajdi has quit IRC16:58
openstackgerritJohn L. Villalovos proposed openstack/ironic-specs: Place priorities list in reverse chronological order  https://review.openstack.org/31279417:00
jlvillallucasagomes: Sweet. Progress :)17:00
jlvillalI'm still running stuff...17:01
lucasagomesyeah17:01
*** dmk0202 has quit IRC17:01
* lucasagomes created 200 loop devices17:02
*** mjturek1 has quit IRC17:02
*** ijw_ has joined #openstack-ironic17:03
*** ijw has quit IRC17:06
*** jistr has quit IRC17:07
*** stendulker_ has joined #openstack-ironic17:12
JayFlucasagomes: around? still interested in poking at the big devstack box17:14
lucasagomesJayF, still around yeah, not for long tho :-/17:14
lucasagomesJayF, you have it setup there?17:14
JayFgot a key?17:14
*** wajdi has joined #openstack-ironic17:14
JayFyeah, I never tore down my old ones17:14
vdroklucasagomes: btw, I think depends-on on project-config patches does not work17:15
JayFvdrok: that's correct17:15
lucasagomesvdrok, :-( yeah it failed with the same error17:15
*** stendulker has quit IRC17:15
lucasagomesJayF, vdrok thanks... not sure how I test it17:15
JayFcan you link me that change?17:15
vdroklucasagomes: just merge it :)17:15
lucasagomesJayF, pm'ed17:16
lucasagomesJayF, https://review.openstack.org/31363817:16
wajdiwhen running grenade tests a second time, do you simply remove the workspace and run again? Or do you bring up a new stack o na clean vm?17:17
JayFlucasagomes: +1 to just merging that17:17
wajdiwant to make sure that my testing is valid. I have been removing the workspace and just running again.17:17
JayFlucasagomes: the p-c one you dep on17:17
*** jjohnson2 has joined #openstack-ironic17:18
lucasagomesthanks, I've set workflow to +0 and added a comment17:18
lucasagomeswill wait someone from -infra17:19
JayFyou link it to anyone there?17:19
* JayF will17:19
devanandaanyone tried calling the send_raw passthru method of the ipmitool drivers?17:19
*** jtomasek has joined #openstack-ironic17:21
*** jjohnson2_ has quit IRC17:22
jlvillalAs an FYI, I have pushed a new version of https://github.com/JohnVillalovos/devstack-gate-test with jroll's fix for an issue with resource phase create17:24
jlvillalNobodyCam: TheJulia cinerama lucasagomes JayF sambetts ^^^17:25
*** fragatina has quit IRC17:26
cineramathanks jlvillal17:26
*** fragatina has joined #openstack-ironic17:26
*** aginwala has joined #openstack-ironic17:28
*** fragatina has quit IRC17:34
*** [4]cdearborn has joined #openstack-ironic17:34
openstackgerritJarrod Johnson proposed openstack/pyghmi: Add Hardware inventory from Lenovo Agentless  https://review.openstack.org/31256817:35
*** aginwala has quit IRC17:37
*** klindgren has quit IRC17:40
*** irf has joined #openstack-ironic17:40
*** klindgren has joined #openstack-ironic17:41
mgagnehas anyone had an issue where nova-compute no longer report heartbeat for all hypervisors? compute_nodes.updated_at field in database is stalled until I restart the nova-compute service and nova-scheduler complains about stalled hypervisors.17:41
mgagne"has not been heard from in a while"17:42
*** lintan has quit IRC17:44
*** mjturek1 has joined #openstack-ironic17:44
*** lintan has joined #openstack-ironic17:44
vdroklucasagomes: I reopened that bug (1548652), with incomplete status, will try to investigate a bit on the weekend17:46
*** yolanda has joined #openstack-ironic17:46
JayFwin 3217:46
JayFwhoops17:46
vdrokgood night everyone! have a good weekend17:46
lucasagomesvdrok, thanks a lot17:47
lucasagomesnight17:47
*** aginwala has joined #openstack-ironic17:47
*** xhku has joined #openstack-ironic17:47
jlvillalvdrok: Dos ve dos17:48
* sambetts is heading off 17:53
*** sambetts is now known as sambetts|afk17:53
*** yolanda has quit IRC17:54
*** fragatina has joined #openstack-ironic17:55
*** stendulker_ has quit IRC17:55
*** fragatin_ has joined #openstack-ironic17:55
*** ohamada has quit IRC17:56
jlvillalsambetts|afk: Ciao17:57
*** Guest60954 has quit IRC17:57
*** yarkot1 has quit IRC17:57
*** akrish9 has joined #openstack-ironic17:58
*** fragatina has quit IRC17:59
*** Nisha has quit IRC18:01
*** joprovost has quit IRC18:05
*** joprovost has joined #openstack-ironic18:06
*** yarkot1 has joined #openstack-ironic18:07
*** lucasagomes is now known as lucas-afk18:07
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Ports  https://review.openstack.org/31318718:13
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Root and Nodes  https://review.openstack.org/31279518:13
devanandamoar docs18:14
*** jjohnson2 has quit IRC18:18
*** jjohnson2_ has joined #openstack-ironic18:18
JayFTIL about less -R -- it properly renders the colors in logfiles generated by devstack18:20
*** martiby has joined #openstack-ironic18:26
*** wajdi has quit IRC18:26
*** rloo has quit IRC18:30
*** irf has quit IRC18:36
*** piet has quit IRC18:38
*** e0ne has joined #openstack-ironic18:39
*** keedya has quit IRC18:40
*** catintheroof has quit IRC18:41
openstackgerritJarrod Johnson proposed openstack/pyghmi: Add Hardware inventory from Lenovo Agentless  https://review.openstack.org/31256818:42
mgagneso I'm still trying to investigate that issue I have with compute_nodes entries in nova database getting stalled with Ironic18:56
mgagnein our cloud/virtual cell, everything is fine, compute_nodes entries are updated (updated_at field)18:57
mgagnebut with baremetal, it's stalled18:57
*** aginwala has quit IRC19:03
*** sivaramakrishna has joined #openstack-ironic19:04
openstackgerritJarrod Johnson proposed openstack/pyghmi: Add Hardware inventory from Lenovo Agentless  https://review.openstack.org/31256819:06
*** wajdi has joined #openstack-ironic19:08
*** ifarkas has quit IRC19:08
*** aginwala has joined #openstack-ironic19:09
*** e0ne has quit IRC19:11
*** keedya has joined #openstack-ironic19:20
*** aginwala has quit IRC19:20
*** wajdi has quit IRC19:23
*** rcernin has joined #openstack-ironic19:26
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Add network drivers  https://review.openstack.org/28585219:26
*** chenke_ has quit IRC19:30
openstackgerritSivaramakrishna Garimella proposed openstack/ironic: Update the deploy drivers with network flipping logic  https://review.openstack.org/21326219:31
*** keedya_ has joined #openstack-ironic19:31
mgagneok found the issue19:32
mgagnethis line makes it so scheduler is not updated with latest compute node info if resource usage didn't change: https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L63219:33
*** yonglihe has quit IRC19:33
mgagnethis however means that ComputeFilter will complain since updated_at field will go outdated rapidly19:33
mgagnejroll: ^19:34
jrollmgagne: that's fun, file a nova bug?19:34
mgagnehas anyone encounter the bug yet?19:34
jrollis that the heartbeat though?19:34
*** absubram has quit IRC19:35
*** keedya has quit IRC19:35
jrollI haven't seen that afaik19:35
mgagneI'm not sure what is calling that thing19:35
mgagnemaybe there is 2 code path to update the hearbeat19:35
jrollcould be19:35
openstackgerritSam Betts proposed openstack/ironic-python-agent: TinyIPA: Shave off some file size from tinyipa ramdisk  https://review.openstack.org/31367519:35
sambetts|afkdown to ~49mb ^19:36
sambetts|afknight all19:36
jrollmgagne: I honestly have no idea how this works, I'll have to look later19:38
mgagnesure19:38
*** sivaramakrishna has quit IRC19:38
openstackgerritJarrod Johnson proposed openstack/pyghmi: Add Hardware inventory from Lenovo Agentless  https://review.openstack.org/31256819:46
*** piet has joined #openstack-ironic19:49
*** rloo has joined #openstack-ironic19:53
*** dprince has quit IRC19:59
*** mkovacik_ has joined #openstack-ironic20:05
*** dmk0202 has joined #openstack-ironic20:05
mgagnejroll: bug #157921320:05
openstackbug 1579213 in OpenStack Compute (nova) "ComputeFilter fails because compute node has not been heard from in a while" [Undecided,New] https://launchpad.net/bugs/157921320:05
*** joprovost has quit IRC20:05
*** wajdi has joined #openstack-ironic20:06
*** joprovost has joined #openstack-ironic20:06
*** aginwala has joined #openstack-ironic20:07
*** mkovacik has quit IRC20:07
*** joprovost has quit IRC20:11
*** mlavalle has joined #openstack-ironic20:14
*** xhku has quit IRC20:15
*** aginwala has quit IRC20:17
*** wajdi_ has joined #openstack-ironic20:19
*** wajdi has quit IRC20:19
*** keedya_ has quit IRC20:19
*** joprovost has joined #openstack-ironic20:20
*** keedya has joined #openstack-ironic20:24
*** garthb_ has joined #openstack-ironic20:27
*** sinval has quit IRC20:28
*** wajdi_ has quit IRC20:28
*** Sukhdev has joined #openstack-ironic20:29
*** garthb has quit IRC20:29
*** e0ne has joined #openstack-ironic20:29
*** openstackgerrit has quit IRC20:32
*** joprovost has quit IRC20:33
*** joprovost has joined #openstack-ironic20:33
*** openstackgerrit has joined #openstack-ironic20:33
*** xhku has joined #openstack-ironic20:33
*** joprovost has quit IRC20:38
*** xhku has quit IRC20:38
*** e0ne has quit IRC20:39
*** e0ne has joined #openstack-ironic20:40
*** e0ne has quit IRC20:40
*** Sukhdev has quit IRC20:40
*** rloo has quit IRC20:41
*** Sukhdev has joined #openstack-ironic20:41
devanandajroll: the adopt-node-state patch has 3 +2's -- unless you have a reason to hold it back, I'm happy with landing it20:47
openstackgerritJarrod Johnson proposed openstack/pyghmi: Add Hardware inventory from Lenovo Agentless  https://review.openstack.org/31256820:47
jrolldevananda: I haven't looked at it, I trust y'all20:47
*** baoli_ has quit IRC20:50
*** fragatin_ has quit IRC20:53
*** joprovost has joined #openstack-ironic20:54
*** dmk0202 has quit IRC20:58
*** Sukhdev has quit IRC21:02
*** baoli has joined #openstack-ironic21:06
*** jjohnson2_ has quit IRC21:11
*** joprovost has quit IRC21:11
openstackgerritMerged openstack/ironic: API: Check for reserved words when naming a node  https://review.openstack.org/30896521:14
*** baoli has quit IRC21:16
*** ayoung has quit IRC21:20
*** Sukhdev has joined #openstack-ironic21:23
*** ametts has quit IRC21:24
JayFis there a reason we run both a tinyipa and a tinyipa-src job on Ironic proper?21:26
JayFAbout to push a proposed change to make tinyipa the default, and it seems to me the -src is a job that should only be running on IPA, in the same way we did with CoreOS21:26
*** leakypipes has quit IRC21:35
*** jcoufal has quit IRC21:35
*** aginwala has joined #openstack-ironic21:38
*** aginwala has quit IRC21:40
*** fragatina has joined #openstack-ironic21:42
*** fragatin_ has joined #openstack-ironic21:43
*** aginwala has joined #openstack-ironic21:44
*** fragatina has quit IRC21:47
*** thiagop has quit IRC21:47
*** penick has joined #openstack-ironic21:47
*** davideagnello has quit IRC21:53
openstackgerritDevananda van der Veen proposed openstack/ironic: Update the api-ref documentation for Drivers  https://review.openstack.org/31370821:54
*** thrash is now known as thrash|wknd21:54
JayFjroll: around?21:54
JayFor anyone, really, who has interest and possibly insight into how we want to adjust jobs with tinyipa being the default21:55
openstackgerritClif Houck proposed openstack/ironic-specs: Add spec for image caching to nodes via agent driver  https://review.openstack.org/31059421:56
*** piet has quit IRC21:59
*** rbudden has quit IRC22:08
*** Sukhdev has quit IRC22:10
bodepdbifrost question. Considering patching the csv code in bifrost_inventory.py to support netmask and gateway, not sure if it's worth doing (I'd either add it there or just create my own custom templates to generate the json)22:10
JayFTheJulia: ^22:12
bodepdThe question is if a patch would be considered since the csv stuff is already a bit of a mess (and any patch would add columns making it messier)22:12
openstackgerritOpenStack Proposal Bot proposed openstack/bifrost: Updated from global requirements  https://review.openstack.org/31371222:15
JayFbodepd: TheJulia would be the best person to answer that, but I know generally they are accepting of new features and such22:16
bodepdAll in all, not a bad experience with bifrost. The main thing that bit me was switching to virbr0 to a real interface22:16
*** ayoung has joined #openstack-ironic22:16
JayFRFR Jay Faulkner proposed openstack-infra/project-config: Use TinyIPA and iPXE for most Ironic jobs https://review.openstack.org/31371722:16
bodepdJayF: sure. thanks. They've already accepted a patch from me, this is more of a question if that code is a dead-end or something worth contributing to.22:17
JayFMy statement would be that bifrost is young enough that I doubt any code is considered a dead end at this point22:17
openstackgerritOpenStack Proposal Bot proposed openstack/ironic: Updated from global requirements  https://review.openstack.org/31372922:17
openstackgerritOpenStack Proposal Bot proposed openstack/ironic-inspector: Updated from global requirements  https://review.openstack.org/31373022:17
JayFbut I don't use it ans as always can be wrong :)22:17
*** [4]cdearborn has quit IRC22:21
openstackgerritOpenStack Proposal Bot proposed openstack/python-ironicclient: Updated from global requirements  https://review.openstack.org/31375622:22
*** keedya has quit IRC22:24
jrollJayF: adjust jobs how22:29
*** ChrisAusten has quit IRC22:30
*** ChrisAusten has joined #openstack-ironic22:31
JayFkrotscheck: https://review.openstack.org/31371722:32
JayFerm22:32
JayFjroll: https://review.openstack.org/31371722:32
JayFthat's what I get for touch typing with tab completion, sorry krots22:32
*** david-lyle has quit IRC22:35
*** david-lyle has joined #openstack-ironic22:37
jrollJayF: at a glance that seems fine to me22:38
JayFit's not passing tests, there's a line here:22:38
JayF  - name: ^gate-tempest-dsvm-ironic-pxe_ipa-tinyipa.*$22:38
JayF    branch: ^(?!stable/(kilo|liberty)).*$22:38
JayFreferencing a job I removed that I think is failing tests22:38
JayFand it makes me wonder if these changes would break stable jobs, since clearly they're marked to not run the tinyipa job22:38
JayFso wondering if that means I have to leave the tinyipa job named tinyipa, and let the old job stay the exact same to not break stable jobs22:39
* JayF barely understands how this works22:39
jrollooo.22:39
jrollyeah, liberty/kilo ironic/devstack doesn't support tinyipa22:39
jrollI gotta bounce, but that's certainly something to think about22:40
JayFso do it that way, then?22:40
jrollyou could do something in the job def22:40
JayFlike if branch=, do it diff?22:40
JayFI'll lookup for an example of that22:40
jrollwith like... if ZUUL_REF == kilo or liberty: force coreos22:40
jrollif [ "$ZUUL_BRANCH" != "stable/kilo" -a "$ZUUL_BRANCH" != "stable/liberty" ] ; then22:40
jrollthere's already one in that section22:40
JayFas a note I'm out on Monday, so if anyone wants to poke this when I'm gone feel free22:40
JayFbut i'll fix that up22:40
jrollcool22:41
jrollalso I'd like something that's still !ipxe22:41
jroll(there might be already and I missed it)22:41
jrollbut really gotta go, see ya22:41
*** mkovacik__ has joined #openstack-ironic22:42
JayFI left pxe_ssh as !ipxe22:43
*** wajdi has joined #openstack-ironic22:44
*** mkovacik_ has quit IRC22:45
*** ChrisAusten has quit IRC22:47
*** wajdi_ has joined #openstack-ironic22:49
*** wajdi has quit IRC22:52
*** r-daneel has quit IRC22:52
*** wajdi_ is now known as wajdi22:55
openstackgerritClint 'SpamapS' Byrum proposed openstack/bifrost: Convert ironic.conf to template generated  https://review.openstack.org/30563422:57
*** aginwala has quit IRC23:12
*** daemontool has quit IRC23:12
*** ppiela has quit IRC23:20
*** garthb_ has quit IRC23:38
openstackgerritJay Faulkner proposed openstack/ironic: Make iPXE + TinyIPA the defaults for devstack  https://review.openstack.org/31303523:39
openstackgerritJay Faulkner proposed openstack/ironic: Make iPXE + TinyIPA the defaults for devstack  https://review.openstack.org/31303523:40
*** xhku has joined #openstack-ironic23:45
*** absubram has joined #openstack-ironic23:48
*** absubram_ has joined #openstack-ironic23:49
*** xhku has quit IRC23:49
*** absubram has quit IRC23:53
*** absubram_ is now known as absubram23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!