Tuesday, 2014-09-30

*** eghobo has joined #openstack-ironic00:03
*** todd_dsm has joined #openstack-ironic00:08
*** marcoemorais has quit IRC00:19
*** shakamunyi has joined #openstack-ironic00:21
*** marcoemorais has joined #openstack-ironic00:21
*** marcoemorais has quit IRC00:27
*** marcoemorais has joined #openstack-ironic00:27
*** rloo has quit IRC00:35
*** Kui has joined #openstack-ironic00:37
*** todd_dsm has quit IRC00:39
*** todd_dsm has joined #openstack-ironic00:40
*** marcoemorais has quit IRC00:43
*** shakayumi has joined #openstack-ironic00:43
*** shakamunyi has quit IRC00:43
*** rwsu has quit IRC00:49
*** harlowja_away is now known as harlowja00:49
*** chuckC has quit IRC00:53
*** todd_dsm has quit IRC00:58
*** marcoemorais has joined #openstack-ironic01:01
*** ChuckC_ has quit IRC01:04
*** athomas has quit IRC01:07
*** marcoemorais has quit IRC01:15
*** rwsu has joined #openstack-ironic01:15
*** eghobo has quit IRC01:16
*** marcoemorais has joined #openstack-ironic01:18
*** marcoemorais has quit IRC01:19
*** marcoemorais has joined #openstack-ironic01:20
*** marcoemorais has quit IRC01:24
*** pcrews has quit IRC01:25
*** marcoemorais has joined #openstack-ironic01:26
*** pcrews has joined #openstack-ironic01:27
openstackgerritYuiko Takada proposed a change to openstack/python-ironicclient: Add "ironic node-set-power-state" cmd unit test  https://review.openstack.org/12426701:30
*** mikedillion has joined #openstack-ironic01:33
*** ChuckC_ has joined #openstack-ironic01:34
*** chuckC has joined #openstack-ironic01:35
*** marcoemorais has quit IRC01:39
*** foexle_ has joined #openstack-ironic01:39
*** foexle has quit IRC01:43
*** mikedillion has quit IRC01:47
*** shakayumi has quit IRC01:49
*** dlaube has quit IRC01:53
*** nosnos has joined #openstack-ironic01:56
*** shakamunyi has joined #openstack-ironic02:16
*** shakamunyi has quit IRC02:49
*** ramineni has joined #openstack-ironic02:49
*** shakamunyi has joined #openstack-ironic02:49
*** shakayumi has joined #openstack-ironic02:52
*** shakamunyi has quit IRC02:56
*** pcrews has quit IRC02:56
*** shakayumi has quit IRC02:56
*** alexiz has joined #openstack-ironic02:57
*** pcrews has joined #openstack-ironic02:57
*** pcrews has quit IRC03:04
*** pcrews has joined #openstack-ironic03:21
*** nosnos has quit IRC03:22
*** nosnos has joined #openstack-ironic03:23
*** Poornima has joined #openstack-ironic03:24
*** harlowja is now known as harlowja_away03:26
*** nosnos has quit IRC03:27
*** penick has joined #openstack-ironic03:29
*** penick_ has joined #openstack-ironic03:29
*** harlowja_away is now known as harlowja03:32
*** penick has quit IRC03:33
*** penick_ is now known as penick03:33
*** rameshg87 has joined #openstack-ironic03:42
*** nikunj2512 has joined #openstack-ironic03:50
*** rameshg871 has joined #openstack-ironic03:59
*** pcrews has quit IRC03:59
*** rameshg87 has quit IRC04:01
*** rushiagr_away is now known as rushiagr04:01
*** marcoemorais has joined #openstack-ironic04:03
*** marcoemorais1 has joined #openstack-ironic04:05
*** marcoemorais has quit IRC04:08
*** rameshg871 has quit IRC04:09
*** nosnos has joined #openstack-ironic04:14
*** shikui_ has joined #openstack-ironic04:30
*** jrist has quit IRC04:32
*** Kui has quit IRC04:33
*** jrist has joined #openstack-ironic04:35
*** rushiagr is now known as rushiagr_away04:51
*** pensu has joined #openstack-ironic04:56
*** alexiz has quit IRC05:00
*** loki184 has joined #openstack-ironic05:01
*** rakesh_hs has joined #openstack-ironic05:13
*** rushiagr_away is now known as rushiagr05:32
*** penick has quit IRC05:59
*** harlowja is now known as harlowja_away06:05
*** rwsu has quit IRC06:09
*** k4n0 has joined #openstack-ironic06:27
*** nosnos has quit IRC06:39
*** nosnos has joined #openstack-ironic06:40
*** nosnos has quit IRC06:41
*** nosnos_ has joined #openstack-ironic06:41
GheRiveromorning all06:43
*** jcoufal has joined #openstack-ironic06:47
*** jcoufal has quit IRC06:47
openstackgerritAnusha Ramineni proposed a change to openstack/ironic: ilo* drivers to use only ilo credentials  https://review.openstack.org/12470406:47
openstackgerritOpenStack Proposal Bot proposed a change to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/12495306:47
*** GheRivero has quit IRC06:52
*** GheRivero has joined #openstack-ironic06:58
*** AJaeger has joined #openstack-ironic06:59
AJaegerdevananda, https://review.openstack.org/#/c/124953/ looks fine - the single french file has one string and that is translated...07:00
AJaegerdevananda, 124953 is the translation import07:00
AJaegerdevananda, but I don't think this is critical to integrate...07:00
*** igordcard has joined #openstack-ironic07:08
*** ifarkas has joined #openstack-ironic07:18
*** romcheg has joined #openstack-ironic07:20
*** foexle_ has quit IRC07:21
*** jcoufal has joined #openstack-ironic07:22
*** jcoufal has quit IRC07:23
*** jcoufal has joined #openstack-ironic07:23
*** ndipanov_gone is now known as ndipanov07:27
*** ifarkas has quit IRC07:32
*** jcoufal has quit IRC07:32
*** ifarkas has joined #openstack-ironic07:32
*** ifarkas has quit IRC07:33
*** ifarkas has joined #openstack-ironic07:33
*** jcoufal has joined #openstack-ironic07:34
*** jcoufal has quit IRC07:40
*** jcoufal has joined #openstack-ironic07:40
*** vinbs has joined #openstack-ironic07:48
*** athomas has joined #openstack-ironic07:55
*** jistr has joined #openstack-ironic07:55
*** MattMan has quit IRC07:56
*** MattMan has joined #openstack-ironic07:56
*** andreykurilin_ has joined #openstack-ironic07:57
*** lsmola has joined #openstack-ironic07:59
*** loki184 has quit IRC08:02
*** derekh has joined #openstack-ironic08:03
*** dtantsur|afk is now known as dtantsur08:04
dtantsurMorning Ironic08:04
GheRiveromorning all08:05
GheRiveroanybody knows what's going on with  check-tripleo-ironic-undercloud-precise-nonha?08:06
*** pensu has quit IRC08:14
*** pensu has joined #openstack-ironic08:15
*** loki184 has joined #openstack-ironic08:23
*** lucasagomes has joined #openstack-ironic08:23
romchegMorning everyone!08:28
*** ramineni1 has joined #openstack-ironic08:32
*** ramineni has quit IRC08:34
*** ramineni1 has quit IRC08:38
*** andreykurilin_ has quit IRC08:44
*** viktors|afk is now known as viktors08:44
*** marcoemorais1 has quit IRC08:47
*** foexle has joined #openstack-ironic08:49
dtantsurromcheg, GheRivero, morning08:54
dtantsurGheRivero, no idea, was not paying attention recently08:54
derekhHi all, the ironic jobs has been failing since last night bug 1375641 , the novabm jobs are ok, any ides ?09:17
dtantsurderekh, wow. anything interesting in ironic logs?09:25
dtantsur WARNING wsme.api [-] Client-side error: Node 3ced5637-d168-4d89-8074-f8002154e47c can not be updated while a state transition is in progress.09:26
dtantsurhmmm...09:26
dtantsurand completely nothing in conductor logs09:27
dtantsurlucasagomes, feel like joining our party ^^^?09:27
* lucasagomes reads09:28
lucasagomes1 sec just finishing a comment in a review09:28
*** yuanying has quit IRC09:28
openstackgerritDerek Higgins proposed a change to openstack/ironic: Revert "Conductor changes target_power_state before starting work"  https://review.openstack.org/12499009:29
*** yuanying has joined #openstack-ironic09:29
derekhdtantsur: just the log you pasted09:29
dtantsurderekh, I also suspect ^^^ is related09:29
derekhdtantsur: lucasagomes ^^ I'm trying that revert, it failed tripleo ci with the same error and merged around the same time the problem started09:30
lucasagomesderekh, oh! ack... lemme take a look at the logs too, but yeah let's revert to unbroke u guys09:31
derekhlucasagomes: Ok, we'll know in just under an hour, its not just us that are broke, ironic is too ;-)09:32
lucasagomesyeah indeed09:33
lucasagomes:(09:33
dtantsurupdated bug as affecting Ironic09:33
*** yuanying has quit IRC09:34
* dtantsur does not feel comfortable that we introduce so many serious changes just before rc1 :(09:34
derekhThe tripleo ci is fairly reliable these day, paying attention to the results should help09:35
dtantsurderekh, did we decide on making it voting some day?09:37
dtantsurthat would help a lot09:37
openstackgerritA change was merged to openstack/ironic: Add HashRingManager to wrap hash ring singleton  https://review.openstack.org/12095109:38
derekhdtantsur: ok, so this is where it gets complicated, technically it is voting but in a different queue to the other ci jobs (and not in gate) so it easily be ignored09:39
derekhdtantsur: but somehow enforcing the vote would be better09:39
dtantsurright09:39
lucasagomesyeah, if we could get it enforcing the vote in all projects that tripleo touches, otherwise if we do only in Ironic we will have the risk of get broken by things not related to ironic at all09:44
dtantsurlucasagomes, I think I detected error in the power state patch09:45
dtantsurnode['target_power_state'] = states.NOSTATE   should be https://review.openstack.org/#/c/124225/4/ironic/conductor/utils.py on line 8709:45
*** ramineni has joined #openstack-ironic09:45
lucasagomesdtantsur, oh, right, I'm looking into it now09:46
dtantsurderekh, lucasagomes lemme propose alternative patch and see of it fixes the problem09:46
lucasagomesdtantsur, oh! makes sense09:47
lucasagomesbecause it already has that target state but then it does nothing (just returns) without cleaning that up09:47
lucasagomestho the manager should unset that :/ lemme see09:48
openstackgerritDmitry Tantsur proposed a change to openstack/ironic: Always reset target_power_state in node_power_action  https://review.openstack.org/12499409:50
dtantsurlucasagomes, derekh ^^^09:51
dtantsurnow let's wait for the job to pass/not to pass09:52
lucasagomesyeah, it looks like that bug was there b409:53
lucasagomesodd09:53
dtantsurlucasagomes, no, previously target_power_state was set after that code10:00
dtantsurline 102 https://review.openstack.org/#/c/124225/4/ironic/conductor/utils.py10:00
dtantsurso on early return target_power_state was still NOSTATE10:01
dtantsurafter devananda's patch it's no longer the first place to set target_power_state10:01
lucasagomesdtantsur, +11110:01
lucasagomesmaybe we should put a big ass try...finally there to unset that10:03
lucasagomesgosh10:03
*** yuanying has joined #openstack-ironic10:08
dtantsurlucasagomes, let's check our guesses and unbreak the gate. then we can think how to solve it the best way10:11
lucasagomesyup10:11
lucasagomesyeah, I think it's grand your patch might fix it10:11
lucasagomesbut odd enough we have no unittests caughting that new behavior etc10:11
openstackgerritA change was merged to openstack/ironic: Stop running check_uptodate in the pep8 testenv  https://review.openstack.org/12491910:13
*** yuanying has quit IRC10:16
*** jcoufal has quit IRC10:23
*** jcoufal has joined #openstack-ironic10:24
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: For convention rename the first classmethod parameter cls  https://review.openstack.org/12500010:25
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: For convention rename the first classmethod parameter to cls  https://review.openstack.org/12500010:28
*** sabah has joined #openstack-ironic10:31
lucasagomesthe revert seems to have worked, let's wait for the alternative patch10:33
*** pelix has joined #openstack-ironic10:34
derekhlucasagomes: dtantsur the fix worked aswell https://review.openstack.org/#/c/124994/10:43
lucasagomesderekh, right, I think we should go with the fix then10:44
derekhlucasagomes: sounds ok to me10:44
*** Poornima has quit IRC10:45
lucasagomesderekh, ack! thanks10:45
dtantsurderekh, cool!10:49
dtantsurany cores around to approve the change?10:49
lucasagomesyuriyz, Haomeng|2 maybe10:49
yuriyzmorning Ironic10:50
dtantsuryuriyz, morning! we need your help approving https://review.openstack.org/#/c/124994/ to unbreak tripleo jobs10:51
dtantsurthanks10:53
lucasagomesmorning yuriyz10:55
*** romcheg has quit IRC10:56
*** lucasagomes is now known as lucas-hungry11:01
*** loki184 has quit IRC11:02
*** romcheg has joined #openstack-ironic11:07
*** ramineni has quit IRC11:08
*** yuanying has joined #openstack-ironic11:13
*** yuanying has quit IRC11:21
*** nikunj2512 has quit IRC11:23
*** mikal has quit IRC11:23
*** igordcard has quit IRC11:28
*** jistr is now known as jistr|english11:31
*** vdrok_ has joined #openstack-ironic11:31
*** mrda-vacation has quit IRC11:33
openstackgerritA change was merged to openstack/ironic: Always reset target_power_state in node_power_action  https://review.openstack.org/12499411:34
*** mikal has joined #openstack-ironic11:35
dtantsurlucas-hungry, derekh ^^^11:37
*** vinbs has quit IRC11:37
derekhdtantsur: great, thanks11:38
*** dhellmann has quit IRC11:48
*** dhellmann has joined #openstack-ironic11:50
*** lazy_prince2 has joined #openstack-ironic11:55
*** romcheg has quit IRC12:02
dtantsurshort walk, brb12:04
openstackgerritA change was merged to openstack/ironic: For convention rename the first classmethod parameter to cls  https://review.openstack.org/12500012:10
*** igordcard has joined #openstack-ironic12:11
openstackgerritA change was merged to openstack/ironic: Force the SSH commands to use their default language  https://review.openstack.org/12406112:16
*** yuanying has joined #openstack-ironic12:18
*** dprince has joined #openstack-ironic12:18
*** romcheg has joined #openstack-ironic12:26
*** yuanying has quit IRC12:27
*** foexle has quit IRC12:30
*** ifarkas has quit IRC12:33
*** ifarkas has joined #openstack-ironic12:35
lucas-hungrydtantsur, w00t12:39
*** lucas-hungry is now known as lucasagomes12:39
*** rushiagr is now known as rushiagr_away12:51
*** jistr|english is now known as jistr12:56
*** k4n0 has quit IRC13:01
*** shikui_ has quit IRC13:03
*** krtaylor has quit IRC13:06
*** krtaylor has joined #openstack-ironic13:07
*** lazy_prince2 has quit IRC13:09
*** rloo has joined #openstack-ironic13:13
*** ramineni has joined #openstack-ironic13:13
*** pensu has quit IRC13:15
*** igordcard has quit IRC13:16
*** igordcard has joined #openstack-ironic13:17
*** yuanying has joined #openstack-ironic13:23
*** linggao has joined #openstack-ironic13:24
*** rushiagr_away is now known as rushiagr13:24
*** ramineni has quit IRC13:27
*** pensu has joined #openstack-ironic13:30
*** yuanying has quit IRC13:32
*** ramineni has joined #openstack-ironic13:35
*** rushiagr is now known as rushiagr_away13:36
*** sabah has quit IRC13:37
jrollmorning ironic :)13:39
dtantsurjroll, morning13:41
lucasagomesjroll, morning13:45
*** jasondotstar has joined #openstack-ironic13:46
rloomorning jroll, afternoon dtantsur & lucasagomes13:47
*** DCWilliams_VA has joined #openstack-ironic13:47
dtantsurrloo, hi13:48
lucasagomesrloo, morning :)13:49
GheRiveromorning @ironic13:51
romchegMorning rloo, jroll!13:51
*** DCWilliams_VA has left #openstack-ironic13:53
rlooafternoon GheRivero, romcheg!13:53
*** athomas has quit IRC13:59
*** nosnos_ has quit IRC13:59
*** nosnos has joined #openstack-ironic13:59
*** nosnos has quit IRC14:04
*** ramineni1 has joined #openstack-ironic14:04
*** athomas has joined #openstack-ironic14:05
*** ramineni has quit IRC14:07
*** DCWilliams_VA has joined #openstack-ironic14:14
*** chuckC has quit IRC14:18
*** rushiagr_away is now known as rushiagr14:19
*** DCWilliams_VA has quit IRC14:19
MattManHi, is there a best practice means of getting nova's baremetal* CLI to use Ironic instead of the built in nova-baremetal ? trying this on Icehouse.14:19
lucasagomesMattMan, we have a proxy in J for that14:20
lucasagomesbut only for partial of the nova bm commands14:20
lucasagomesMattMan, https://review.openstack.org/#/c/12043314:20
devanandag'mornig, all14:21
MattManlucasagomes: thanks... I'll give that patch a shot.14:21
lucasagomesMattMan, cool, no worries14:23
lucasagomesdevananda, morning14:23
*** DCWilliams_VA has joined #openstack-ironic14:23
NobodyCamgood morning ironic14:24
openstackgerritA change was merged to openstack/ironic: Refactor PXE clean up tests  https://review.openstack.org/12213714:26
devanandalucasagomes: hi! on https://review.openstack.org/#/c/124493/12/ironic/conductor/manager.py -- I can add tests for "is_active_on", but I wrote this so that it wouldn't affect existing behavior of register / unregister14:26
*** ChuckC_ has quit IRC14:27
lucasagomesdevananda, right14:27
lucasagomesdevananda, so, for the glance thing, we need a admin token to be passed within the context to glance14:27
*** yuanying has joined #openstack-ironic14:28
lucasagomesdevananda, and about the tests, I think we should add some specially for that new update_existing behavior14:29
devanandalucasagomes: comments posted. from an API POV, the behavior of register/unregister didn't change14:31
lucasagomesdevananda, cheers, ack14:31
devanandalucasagomes: but I agree that there should be a unit test now for the online column's behavior14:31
lucasagomesdevananda, yeah14:32
lucasagomesdevananda, for the auth_token thing, should we get a admin auth_token with ks before submitting the request to glance? :/14:32
lucasagomesidk where I would get a auth-token from cause it's not a request, it's a periodic task14:33
*** jcoufal has quit IRC14:33
devanandaright, i'll poke at that in a minute. how are you testing it?14:33
lucasagomesdevananda, I've a devstack setup with a machine deployed14:34
lucasagomes2 conductors with diff configs "host" and the tftpboot dir14:35
lucasagomesI kill the conductor that is currently managing that node and wait the periodic task to run on the other conductor14:35
devanandagotcha14:35
lucasagomesI think the auth token is the only missing bit to get something kinda working14:36
lucasagomesI mean working14:36
devanandalucasagomes: so I think we need to set up the keystone_authtoken config option properly for this14:36
*** krtaylor has quit IRC14:36
lucasagomesdevananda, like we have in nova for the ironic cli?14:36
devanandaI dont know if we've been configuring glance access for that token, since it didn't need it14:36
devanandanot exactly, no14:36
devanandalike we have in ironic for swift and neutron access14:36
*** yuanying has quit IRC14:36
devanandalook at ironic/common/swift.py14:36
lucasagomesright, lemme take a look14:37
*** krtaylor has joined #openstack-ironic14:37
*** jcoufal has joined #openstack-ironic14:37
devanandalucasagomes: I think devstack is setting up that account correctly. devstack/lib/ironic:create_ironic_accounts()14:40
lucasagomesdevananda, yeah... it's an admin14:41
devanandalucasagomes: you should see in your /etc/irnic/ironic.conf settings for [keystone_authtoken] admin_user, admin_password, etc. so that should be enough -- we just need to elevate to that14:41
lucasagomesI think we just need to check within our code, before passing the context to the glance cli14:41
devanandayep14:41
lucasagomesif the token contains an auth-token14:41
lucasagomesin case the strategy is keystone ofc14:41
lucasagomesand if not, get one14:42
*** pcrews has joined #openstack-ironic14:42
lucasagomesdevananda, yeah, aight, looking into that now14:42
*** krtaylor has quit IRC14:42
*** openstackgerrit has quit IRC14:42
lucasagomesmakes me wonder how it never was needed, maybe because the images were public14:43
*** krtaylor has joined #openstack-ironic14:44
devanandathe normal path uses the nova context14:46
*** krtaylor has quit IRC14:47
*** ramineni1 has quit IRC14:49
NobodyCamdevananda: see pm14:51
*** DCWilliams_VA has quit IRC14:52
devanandaadam_g: https://review.openstack.org/#/c/124162/ merged into nova now, too.14:55
devanandaadam_g: so we ought to see more stability in the parallel tempest runs14:55
*** ramineni has joined #openstack-ironic14:55
*** DCWilliams_VA has joined #openstack-ironic14:56
*** ramineni1 has joined #openstack-ironic14:56
devanandalucasagomes: re the config option in https://review.openstack.org/#/c/124610/ - i pinged jroll yesterday, but i'm not sure we reached a conclusion. he seemed to want the config option14:56
*** krtaylor has joined #openstack-ironic14:57
devanandalucasagomes: namely because they had to tune the sync power state task.14:57
devanandajroll: any further thoughts on that?14:57
lucasagomesright let's discuss it... I don't think it may harm to offer a way to ppl tune ir up or down, or even disable the periodic task14:57
lucasagomesit doesn't affect the agent, if a cluster has only that driver running14:58
lucasagomeswhy should we have that periodic task14:58
devanandatrue14:59
*** ramineni has quit IRC15:00
*** todd_dsm has joined #openstack-ironic15:00
*** rakesh_hs has quit IRC15:00
*** rwsu has joined #openstack-ironic15:16
*** ChuckC_ has joined #openstack-ironic15:22
*** openstackgerrit has joined #openstack-ironic15:24
*** chuckC has joined #openstack-ironic15:25
rloohi devananda, do you have a minute to explain node.is_active_on? (wrt https://review.openstack.org/#/c/124493)15:27
rloodevananda: it is set to the conductor id when deploying, and unset when tearing down.15:27
*** jcoufal has quit IRC15:27
devanandarloo: yup15:27
rloodevananda: if deploy fails, would/should it be unset?15:27
rloodevananda: and if it is unset when tear down, how does that help with the comment 'clear... so that no other conductor will recreate the deploy env...'?15:28
rloodevananda: if it isn't set, won't another conductor recreate the deploy env? or maybe I'm confused.15:28
devanandaif a node doesn't have an active state, no conductor will rebuild it to active state15:29
devanandaif a ndoe has an active state, but the conductor it was last active on is dead15:29
devanandathen another conductor should rebuild its state15:29
devanandaso when is_active_on is set to a not-active conductor15:30
devanandawhat ever cnoductor that node currently maps to should take_over()15:30
rloodevananda: ahh. got it. the take_over scenario. thx.15:31
devanandaperhaps the column name should be changed? last_active_on? activity_managed_by? state_cached_on?15:31
*** arif-ali has joined #openstack-ironic15:31
rloodevananda: well, it is 'currently-active' or what it thinks it is currently-active-on?15:31
rloodevananda: yeah, so I guess I don't like the 'last' part.15:32
devanandait's not what the node thinks -- it is the id of the last recorded conductor to have prepared the environment15:32
*** coolsvap|afk is now known as coolsvap15:32
*** DCWilliams_VA has quit IRC15:33
*** yuanying has joined #openstack-ironic15:33
*** dtantsur is now known as dtantsur|afk15:34
rloodevananda: if this is only meant to track what conductor has local state for a node, maybe node.local_state_on?15:35
rloonode.conductor_with_local_state?15:36
*** krtaylor has quit IRC15:37
*** krtaylor has joined #openstack-ironic15:38
*** todd_dsm has quit IRC15:41
*** yuanying has quit IRC15:42
*** hemna has joined #openstack-ironic15:44
*** ndipanov is now known as ndipanov_gone15:51
openstackgerritLucas Alvares Gomes proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state  https://review.openstack.org/12461015:57
lucasagomesdevananda, ^15:57
lucasagomesdevananda, one thing, diff than other clients, the glance client doesn't seem to accept the username, password ...15:58
lucasagomeshttps://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/client.py#L25-L2915:58
lucasagomeslike we do for neutron https://github.com/openstack/python-neutronclient/blob/master/neutronclient/client.py#L87-L10215:58
lucasagomesso I get the auth_token with keystone and populate it in the context15:58
lucasagomesseems to be working, it regenerates all the configs, cache the images etc16:00
jrolldevananda, lucasagomes, on the periodic task... someone running the agent would still need that task in case a conductor crashes, no? the request still gets forwarded to a conductor, if it forwards to a conductor that isn't there, there will be problems16:01
devanandajroll: no16:01
*** ifarkas has quit IRC16:02
jrollbecause you're now basically storing the hash ring in the db aiui16:02
jrollmaybe not16:02
lucasagomesjroll, that part is already done16:02
lucasagomesjroll, if the conductor dies, the api will redirect the request to another conductor that has that driver and is active16:02
jrollok, so we're going to continue to calculate the ring on every request then?16:03
lucasagomeswhat we are doing is more the take over, to prepare the env for the new conductor to be able to manage the node it took over16:03
jrollright16:03
lucasagomesjroll, for now I think so, do we cache the hash_ring now16:03
devanandajroll: i dont want to, but to preserve as much functionality as possible at THIS point in the cycle, yes16:03
lucasagomestho*16:03
devanandawe do not cache the hash ring on the API side16:03
jrollok, then would be cool to disable the periodic task16:03
devanandaand when receiving RPC requests, conductors arenot rejecting things that dont map there16:03
devanandasince they aren't checking the mapping at that ppoint16:04
jrollI would also think it would be nice to make the interval configurable, if you're doing 100+ nodes per conductor the power loop can get very slow16:04
devanandawhen a conductor shuts down, it deregisters, so the API services will see that right away16:04
devanandawhen it crashes, there'll be a delay before API services see that it's offline16:04
devananda... /me confirms that16:04
jrollyeah, agree16:04
jrollthere's a heartbeat timeout16:05
lucasagomesyeah seems correct16:05
devanandayep16:05
devanandaget_active_driver_dict considers both the heartbeat and online fields16:05
lucasagomesdevananda, I can confirm that the images are being cached and the configs are generated correctly16:05
lucasagomesnow I'm a bit stuck with the neutron bit16:05
devanandanice!16:05
devanandaok, stuck on what?16:05
lucasagomesit does call update neutron, but it still pointing to the old tftpdir!16:06
devanandaoh16:06
jrollodd16:06
lucasagomesall I changed in my conf for the second conductor was host, tftp_master_path = /opt/stack/data/ironic/tftpboot2/master_images tftp_root = /opt/stack/data/ironic/tftpboot216:06
jrollcan you tell if that's what neutron is responding to dhcp requests with16:06
jroll?16:06
lucasagomesyou know if there's any other bit I need to change? or if there's a problem because both conductors are running on the same ip16:07
lucasagomesso neutron just ignore the update16:07
jrollidk, I would check neutron logs16:07
jrolland just to be sure, check that conductor two has restarted since you changed that conf :P16:07
devanandajroll: i think he's looking for the periodic task to update it16:08
devanandalucasagomes: neutron does have some issues with >1 thing sharing an IP16:08
devanandabut IDK if that's the problem here16:08
jrolldevananda: sounds like the task ran and called to neutron16:08
devanandalucasagomes: is conductor2 creating the new config files?16:09
devanandalucasagomes: nm, yhou said that :)16:09
lucasagomesyeah it has a new config file16:10
lucasagomeshmm I may need another vm to test it16:10
lucasagomesby looking at the logs it seems all good16:10
devanandaso yea, it won't change in tjhis case16:10
lucasagomeswill paste it for ye16:10
devanandalucasagomes: look at the options being sent to neutron16:10
devanandaironic/dhcp/neutron.py:13816:10
devanandaboot file name (pxelinux.0) and the IP16:11
devanandaall the routing beyond that is done by MAC address and symlinks on the conductor16:11
devanandaso to test this, we'd need conductors with different IPs16:11
NobodyCambrb16:12
lucasagomeshttp://paste.openstack.org/show/117187/16:12
lucasagomesthe error with the handler there is just a log interpolation, need to fix that too16:12
lucasagomesdevananda, yeah :( right I will need a new vm for that16:12
lucasagomesurgh16:12
lucasagomeswill take a while to me to confirm, today I've to leave a bit early (I'm moving houses so I need to go to the new place do some stuff)16:13
devanandado you have neutron's log from that time?16:13
devanandaack, nm16:13
lucasagomesdevananda, yup may have16:13
lucasagomes1 sec16:13
lucasagomesneutron log http://paste.openstack.org/show/117189/16:15
jrollmaybe look at /opt/stack/data/neutron/dhcp/e72cb063-425f-4afa-9f4e-8856199b9e5a/host16:16
jrollor everything in that uuid dir, even16:16
lucasagomesstack@vm:~/devstack$ cat /opt/stack/data/neutron/dhcp/e72cb063-425f-4afa-9f4e-8856199b9e5a/host16:17
lucasagomesfa:16:3e:a5:23:7b,host-10-1-0-3.openstacklocal,10.1.0.316:17
lucasagomesfa:16:3e:83:ab:40,host-10-1-0-1.openstacklocal,10.1.0.116:17
lucasagomes52:54:00:48:5e:6b,host-10-1-0-4.openstacklocal,10.1.0.4,set:574c6e0a-b9af-42d2-af29-bf88bddea2d616:17
lucasagomesthe dhcp options16:17
lucasagomesstack@vm:~/devstack$ cat /opt/stack/data/neutron/dhcp/e72cb063-425f-4afa-9f4e-8856199b9e5a/opts16:17
lucasagomestag:tag0,option:router,10.1.0.116:17
lucasagomestag:574c6e0a-b9af-42d2-af29-bf88bddea2d6,option:tftp-server,192.168.122.15616:17
lucasagomestag:574c6e0a-b9af-42d2-af29-bf88bddea2d6,option:bootfile-name,pxelinux.016:17
lucasagomestag:574c6e0a-b9af-42d2-af29-bf88bddea2d6,option:server-ip-address,192.168.122.15616:17
lucasagomestag:tag0,option:dns-server,10.1.0.3stack@vm:~/devstack$16:17
*** marcoemorais has joined #openstack-ironic16:17
lucasagomesmaybe we just need to change the tftpd process16:18
lucasagomesbut, a real test would need 2 diff nodes16:18
lucasagomesyeah that's the tftpd16:18
lucasagomes 7361 ?        Ss     0:00 in.tftpd -v -v -v -v -v --map-file /opt/stack/data/ironic/tftpboot/map-file /opt/stack/data/ironic/tftpboot16:18
jrollhmm yeah, maybe each conductor needs a tftpd16:18
lucasagomesyeah16:18
*** r-daneel has joined #openstack-ironic16:19
lucasagomesthey are running on the same node :/16:19
devanandarunning >1 conductor on the same host isn't supported, heh16:19
lucasagomesyup16:19
lucasagomesbut I couldn't find any way quicker to test :P16:19
devanandalucasagomes: it looks like your test has gone as far as it can with a single VM16:19
devanandacheers :)16:19
lucasagomesI can confirm that all the images and configs have been generated correctly16:19
jrollyeah16:19
lucasagomesyeah16:19
jrolllucasagomes: boot a node with ironic and run the conductor there16:19
jroll:P16:19
devanandaheh16:20
lucasagomeshah will do16:20
*** r-daneel has quit IRC16:20
lucasagomesaight if nobody confirms it today, tomorrow I will setup the right env to make it sure it works16:20
lucasagomes(or try to)16:21
Shrewsdevananda: hrm, is_active_on should be set immediately during deploy, but i'm not seeing that in my test. curious16:21
devanandalucasagomes: cheers, thanks again. I'm going to give it a shot today, as I'll be distracted by meetings tomorrow16:22
lucasagomesdevananda, right, no problem! lemme know the outcome (comment on the patch or something)16:22
*** viktors is now known as viktors|afk16:23
*** ramineni1 has quit IRC16:25
*** r-daneel has joined #openstack-ironic16:28
*** jistr has quit IRC16:28
*** marcoemorais has quit IRC16:31
*** marcoemorais has joined #openstack-ironic16:31
*** lazy_prince is now known as killer_prince16:35
Shrewsdevananda: ah, hrm. i think we need to update objects/node.py for the new column16:38
devanandaShrews: ah, yup16:38
*** yuanying has joined #openstack-ironic16:38
Shrewsgotta run out for a bit. bbs16:40
lucasagomesShrews, oh I put that on the following patch :/ we may need to move it to the previous one16:41
*** marcoemorais has quit IRC16:42
lucasagomesaight I will call it a day, have a great night everyone16:46
*** lucasagomes is now known as lucas-afk16:46
NobodyCamhave a good night lucas-afk16:47
*** yuanying has quit IRC16:47
*** athomas has quit IRC16:52
devanandaGheRivero: ping16:57
devanandavdrok: I was looking at 124400 and wondering why all the DB fixture code is in tests/base, not tests/db/base - but it looks just like legacy from when we forked from Nova16:58
*** derekh has quit IRC16:58
devanandavdrok: what do you think about separating the "needs admin context" from "needs a db fixture" tests? I think these are really seaprate things16:59
devanandaalso, running unit tests against mysql locally, I see only two INSERTs into one of our tables,a nd I dont see any ALTERs at all17:00
*** krtaylor has quit IRC17:01
*** marcoemorais has joined #openstack-ironic17:05
openstackgerritA change was merged to openstack/ironic: Imported Translations from Transifex  https://review.openstack.org/12495317:07
openstackgerritA change was merged to openstack/ironic: Remove useless variable in migration  https://review.openstack.org/12508917:16
*** harlowja_away is now known as harlowja17:18
*** todd_dsm has joined #openstack-ironic17:28
*** coolsvap is now known as coolsvap|afk17:35
*** hemna has quit IRC17:39
*** yuanying has joined #openstack-ironic17:43
*** rushiagr is now known as rushiagr_away17:48
*** rushiagr_away is now known as rushiagr17:51
*** yuanying has quit IRC17:52
*** lsmola has quit IRC17:54
*** pelix has quit IRC17:55
*** todd_dsm has quit IRC17:59
*** todd_dsm has joined #openstack-ironic18:06
*** todd_dsm has quit IRC18:14
adam_ganyone know why websockify is listed in ironics requirements.txt?18:24
NobodyCamadam_g: because openstack wanted it there: https://github.com/openstack/ironic/commit/9fe674e2dd31ac1fa94315d9f8b7cbca93c642f518:25
Shrewsdevananda: ok with me moving this https://review.openstack.org/#/c/124610/7/ironic/objects/node.py to 124493 for you?18:26
NobodyCambrb18:29
adam_gNobodyCam, hmm. silly18:31
*** vdrok_ has quit IRC18:31
devanandaNobodyCam: openstack bot only updates requrements we have. it doesn'ta dd new ones18:32
devanandaso someone wanted it there at some point18:32
devanandaShrews: yup, go for it18:32
Shrewsdevananda: looking closer, it's more code than that actually. not sure if we should move things around at this point18:33
devanandaShrews: i'm kinda side tracked by things right now, so poke away. lucas-afk also wanted some more unit tests on 12449318:33
devanandahmm18:34
*** openstackgerrit has quit IRC18:47
*** yuanying has joined #openstack-ironic18:48
*** openstackgerrit has joined #openstack-ironic18:49
*** yuanying has quit IRC18:56
*** andreykurilin_ has joined #openstack-ironic19:02
*** rushiagr is now known as rushiagr_away19:02
*** chuckC has quit IRC19:03
*** ChuckC_ is now known as ChuckC19:03
*** krtaylor has joined #openstack-ironic19:03
*** chuckC_ has joined #openstack-ironic19:03
*** blogan has joined #openstack-ironic19:03
blogandoes ironic use paste deploy?19:04
*** annegentle has quit IRC19:04
*** hemna has joined #openstack-ironic19:05
Shrewsblogan: no. pecan19:06
bloganShrews: Im pretty sure paste deploy can be used with pecan, though it might require some fenagling19:07
Shrewsblogan: oh. if that's true, i've seen no indication of its use19:08
bloganShrews: I'm just wondering if there was a big reason why ironic isn't using it19:08
Shrewsblogan: we use what all other openstack projects use, so your question isn't ironic specific19:09
jrollShrews: some projects use paste :P19:09
jrolle.g. neutron19:09
bloganShrews: well from the 3 or 4 openstack projects I've been looking at closely, they all use paste deploy19:09
Shrewsjroll: what all other "good" openstack projects use  :-P19:09
bloganlol19:09
blogani am totally fine with not using it19:09
jrollhow dare you make fun of neutron19:09
jroll:P19:10
* Shrews pastes jroll's tongue to the floor19:10
jrollyeah, idk why ironic doesn't use it, but I'm happy with that19:10
bloganso does it seem like the trend is to not use paste?19:10
jrolldon't really see what benefits it would bring us19:10
bloganjust like the trend seems to be going towards pecan and wsme19:10
jrollindeed19:11
jrollbut that's for actually running an api server, looks like paste deploy is just for finding/configuring them19:11
bloganso to get middleware, it looks like ironic is using the wrap_app keyword in pecan's make_app method, is that correct?19:11
jrollsomething like that19:12
blogani guess I see a reason to use paste when you want to add a configurable pipeline, and swap in middleware, but I should just hold off on using it until we actually need something like that19:13
bloganthanks for the info jroll and Shrews19:13
jrollnp19:14
jrollI feel like you can add middleware somewhat easily19:14
* jroll looks19:14
jrollmaybe not19:14
blogani'd need to research pecan's wrap_app functionality19:16
devanandablogan: can you describe why we would use pastedeploy as oposed to what we have today?19:23
devanandablogan: fwiw, we are using pecan/wsme right now, and afaik using paste deploy hasn't been suggested before19:24
blogandevananda: i'm not sure what you have today isn't able to do everything paste deploy can't, which is why I'm in here.  I'm also not a paste deploy expert or proponent, just gathering information as to why its not being used in ironic when it is in many other projects19:24
devanandaa quick read of the docs suggests it's a framework for loading wsgi apps19:25
devanandaso one might use pastedeploy to load ironic's wsgi app - -but irnic doesn't need to use it, per se. is that right?19:26
bloganyeah but also being able to chain together middleware, afaik19:26
bloganoh ironic doesn't need to use it to have a wsgi app, i think paste deploy might add some more easier control for an operator if they need extra middleware19:27
devanandagotcha19:28
devanandaso yea, we provide two methods of serving the ironic wsgi app today19:28
bloganim not saying one way is the right way though19:28
devanandaincluding as an apache mod_wsgi app19:29
bloganyeah and I think we'll do a similar thing19:31
blogani'm using ironic as a reference for a project I'm working on becasue we're using pecan and wsme19:31
*** dprince has quit IRC19:34
*** yuanying has joined #openstack-ironic19:53
*** yuanying has quit IRC20:03
*** shikui_ has joined #openstack-ironic20:15
*** Guest43089 has joined #openstack-ironic20:20
*** Guest43089 is now known as annegentle_20:20
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add "active" tracking to nodes and conductors  https://review.openstack.org/12449320:20
*** mordred is now known as jeblair20:21
*** jeblair is now known as mordred20:21
ShrewsDoes anyone know how to force git-review to download a change from gerritt (with the -d option) into a *differently* named branch?20:24
Shrewsmordred: know that by any chance? ^^^20:25
rlooShrews: I couldn't figure that out either.20:25
Shrewsi'm not sure it is possible looking at the git-review help20:26
Shrewswhich makes me a sad panda20:26
JayFShrews: you can cherry-pick commits out of it20:27
ShrewsJayF: i want to be able to 'git review' my changes back, though20:27
Shrewsi guess i could use twisted, git magic to make that happen, but don't want to20:28
Shrewshrm, wonder if i can rename an existing branch20:28
rlooShrews: you can rename the existing branch20:28
ShrewsFIRST!20:28
jrollShrews: you can git review -t topicname20:28
jrollinstead of using the branch name for topic20:29
rlooShrews: git branch -m20:29
jrolllike... when you push your changes to gerrit, that is20:29
rlooShrews may be first, but he was wondering, whereas I knew that it could be renamed ;)20:29
openstackgerritDavid Shrewsbury proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state  https://review.openstack.org/12461020:30
*** jasondotstar has quit IRC20:30
Shrewsw00t20:31
rlooShrews, I was just reviewing 124510. what did you do? :D20:31
Shrewsrloo: just left a comment explaining20:31
Shrewsrloo: basically, 124493 was incomplete b/c it needed code from 124610 to actually work20:32
Shrewsis_active_on was not being persisted to the database20:32
rlooShrews: yeah, that makes sense. Although I had a comment in that file. Will move it ;)20:32
*** blogan has left #openstack-ironic20:32
Shrewsrloo: i left a comment on 124493 that all comments still need to be addressed20:33
rloothx Shrews. Why didn't you address them too? :D20:33
*** pensu has quit IRC20:33
Shrewslucas-afk: took your advice and moved the node object code around ^^^^20:33
Shrewsrloo: i am lazy and incompetent20:34
rlooShrews:  not too incompetent. You addressed the comment I was going to add (about VERSION) ;)20:34
Shrewsmostly incompetent, then  :)20:35
rlooha ha. Hey, Shrews, what's the difference between node.reservation and node.is_active_on. wrt being 'internal'.20:36
Shrewsumm, not sure. i think internal attributes are ones we don't expose to the API20:37
Shrewsthat's what lucas' comment says at least20:38
rlooShrews: right, but why don't we want to expose it, but we expose reservation?20:38
Shrewsthe value is not signifcant to a user. it's a db row id number20:39
Shrewsperhaps if it were a uuid of the conductor....20:40
Shrewsas for reservation, i'm not sure what that one is, tbh20:40
rlooShrews: oh oh. So why aren't we using uuid, wouldn't that be more useful.20:40
Shrewsrloo: we don't want to use that for a foreign key. if we want to expose the uuid, we have to change our queries to do a join between nodes and conductors tables20:41
rlooShrews: oh. ugh. db stuff. ok thx.20:42
* Shrews throws out db jargon to scare rloo20:42
rloobut it could be useful for debugging, to have the uuid.20:42
rlooShrews, you need to throw harder; it didn't make it across the border.20:43
Shrewsprobably a future feature to expose that20:43
rlooyeah, need to leave stuff for the rest of us to do ;)20:43
jrollreservation is nice to have exposed20:49
jrollthat's the conductor hostname that currently has a lock20:49
*** linggao has quit IRC20:50
*** enterprisedc has quit IRC20:56
*** yuanying has joined #openstack-ironic20:58
rloojroll: I was just wondering whether it might be useful to expose is_active_on too. But that can be a future debate/change.20:59
devanandai thought briefly about using the hostname for is_active_on21:02
*** marcoemorais has quit IRC21:03
devanandaShrews: you want cherry-pick21:03
*** marcoemorais has joined #openstack-ironic21:03
devanandaSHAS=$(for p in $PATCH_SERIES; do git review -d $p; echo $(git log | head -1 | awk '{print $2}'); done);  git checkout master && (for p in $SHAS; do git cherry-pick $p; done)21:05
devanandasomething like that21:05
devanandareconstituting a patch tree where there are new branches is fun :)21:05
* devananda uses words wrong21:05
devanandareconstituting a patch series where some patches in the middle have new revisions is fun :)21:06
jrolldevananda: more simply, git checkout master && for p in $PATCH_SERIES; do git review -x $p; done21:06
jroll:)21:07
devananda-x ?21:07
jrollcherry picks straight from gerrit21:07
*** yuanying has quit IRC21:07
devanandaoooh that's new21:07
devanandathanks21:07
jrollI've been using it for months, not sure how new21:07
devanandahuh21:09
JayFOur original automation for applying unmerged patches downstream used -x21:10
JayFso it has to have been working for at least 4 months or more, right jroll21:10
jroll"automation" is an interesting word21:10
jrollyes21:10
*** hemna_ has joined #openstack-ironic21:10
JayFit would've been fully automatic21:10
JayFif only jroll were a robot21:10
jrollactually, that one liner I posted *was* the automation21:10
JayFhaha21:10
jrolljust s/$PATCH_SERIES/$(cat ironic-patches)21:10
*** hemna has quit IRC21:11
devanandalooks like about 8 months21:11
devanandahttps://github.com/openstack-infra/git-review/commit/e1e9d2ccfa65e610d8930b5d2f7f27cc7eebd49121:12
devanandaso yea, it's "new" :)21:12
*** krtaylor has quit IRC21:13
Shrewsdevananda: uhhh, your way is complex. i just renamed the current branch  :-P21:29
*** dhellmann is now known as dhellmann_21:30
*** andreykurilin_ has quit IRC21:42
*** hemna_ has quit IRC21:50
*** hemna_ has joined #openstack-ironic22:00
*** yuanying has joined #openstack-ironic22:03
*** krtaylor has joined #openstack-ironic22:09
*** yuanying has quit IRC22:12
*** romcheg1 has joined #openstack-ironic22:24
*** ryanpetrello has quit IRC22:41
*** ryanpetrello has joined #openstack-ironic22:47
*** romcheg1 has quit IRC22:49
*** yuanying has joined #openstack-ironic22:58
*** achanda has joined #openstack-ironic23:13
*** lucas-afk has quit IRC23:25
*** igordcard has quit IRC23:26
*** hemna__ has joined #openstack-ironic23:35
*** hemna_ has quit IRC23:36
*** yuanying has quit IRC23:46
*** yuanying has joined #openstack-ironic23:46
*** yuanying has quit IRC23:51
*** chuckC_ has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!