| *** achanda has quit IRC | 00:02 | |
| *** ChuckC has quit IRC | 00:09 | |
| *** rloo has quit IRC | 00:16 | |
| *** marcoemorais has quit IRC | 00:28 | |
| *** hemna__ has quit IRC | 00:35 | |
| *** hemna__ has joined #openstack-ironic | 00:40 | |
| *** ChuckC has joined #openstack-ironic | 00:43 | |
| *** yuanying has joined #openstack-ironic | 00:47 | |
| *** chuckC_ has joined #openstack-ironic | 00:51 | |
| *** yuanying has quit IRC | 00:55 | |
| *** r-daneel has quit IRC | 01:04 | |
| *** HenryG_ has joined #openstack-ironic | 01:22 | |
| *** HenryG has quit IRC | 01:24 | |
| *** hemna__ has quit IRC | 01:37 | |
| *** yuanying has joined #openstack-ironic | 01:52 | |
| *** nosnos has joined #openstack-ironic | 01:57 | |
| *** chuckC_ has quit IRC | 02:11 | |
| *** HenryG_ has quit IRC | 02:25 | |
| *** HenryG has joined #openstack-ironic | 02:41 | |
| *** harlowja is now known as harlowja_away | 02:42 | |
| *** ramineni has joined #openstack-ironic | 02:55 | |
| *** pcrews has quit IRC | 03:09 | |
| *** nosnos has quit IRC | 03:21 | |
| *** nosnos has joined #openstack-ironic | 03:22 | |
| *** rushiagr_away is now known as rushiagr | 03:26 | |
| *** nosnos has quit IRC | 03:26 | |
| *** nosnos has joined #openstack-ironic | 04:20 | |
| *** nikunj2512 has joined #openstack-ironic | 04:35 | |
| *** rushiagr is now known as rushiagr_away | 04:52 | |
| *** killer_prince is now known as lazy_prince | 05:00 | |
| *** rakesh_hs has joined #openstack-ironic | 05:09 | |
| *** smoriya has joined #openstack-ironic | 05:09 | |
| *** marcoemorais has joined #openstack-ironic | 05:14 | |
| *** marcoemorais1 has joined #openstack-ironic | 05:15 | |
| *** marcoemorais has quit IRC | 05:19 | |
| *** rushiagr_away is now known as rushiagr | 05:35 | |
| *** k4n0 has joined #openstack-ironic | 05:47 | |
| *** rushiagr is now known as rushiagr_away | 05:50 | |
| *** rushiagr_away is now known as rushiagr | 05:51 | |
| *** pensu has joined #openstack-ironic | 05:55 | |
| *** AJaeger has left #openstack-ironic | 05:58 | |
| *** dtantsur|afk is now known as dtantsur | 06:36 | |
| dtantsur | Morning Ironic | 06:36 |
|---|---|---|
| devananda | mornin :) | 06:37 |
| devananda | i'm about to hack out some changes to the hash ring patches, then pass out | 06:37 |
| dtantsur | heh, unusual to see you online in the morning :) | 06:40 |
| dtantsur | ok will review them today (though I can't really test them I guess) | 06:40 |
| devananda | dtantsur: what do you think of s/is_active_on/conductor_affinity/ | 06:42 |
| dtantsur | devananda, +1 to having word 'conductor' in it | 06:42 |
| *** ifarkas has joined #openstack-ironic | 06:44 | |
| dtantsur | devananda, re 'affinity'... it sounds like some constant property, 'is_active_on' sounds like something transient | 06:45 |
| devananda | yep | 06:45 |
| devananda | i'm not unsetting it any more :) | 06:45 |
| dtantsur | ah so I need to refresh my knowledge of your patches | 06:45 |
| devananda | rloo's comments convinced me, there's no need to unset it during tear down | 06:45 |
| dtantsur | (and also to wake up) | 06:45 |
| devananda | well, i haven't posted them yet :) | 06:45 |
| dtantsur | I see :) | 06:46 |
| devananda | it merely indicates the last conductor to have done something to that node | 06:46 |
| devananda | which may or may not indicate that there is some persistent local state on taht conductor | 06:46 |
| devananda | but what that really means is: if a node is mapped to the conductor which did NOT most recently maybe create some lcoal state, then it should try to do that | 06:47 |
| devananda | if affinity != mapping: do take_over() | 06:48 |
| GheRivero | devananda: pong | 06:49 |
| devananda | GheRivero: 17:00:24 < devananda> also, running unit tests against mysql locally, I see only two INSERTs into one of our tables,a nd I dont see any ALTERs at all | 06:50 |
| devananda | any thoughts on why I don't see ALTERs ? | 06:51 |
| GheRivero | no idea. Will check it now to see if I get the same behavior | 06:52 |
| *** marcoemorais1 has quit IRC | 07:01 | |
| openstackgerrit | Mark Atwood proposed a change to openstack/ironic: Cleans up some Sphinx rST warnings in Ironic https://review.openstack.org/125273 | 07:02 |
| dtantsur | devananda, makes sense | 07:06 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/ironic: Updated from global requirements https://review.openstack.org/125274 | 07:11 |
| GheRivero | devananda: indeed, there are only two inserts but i have multiple ALTERS | 07:17 |
| devananda | GheRivero: can you paste one? | 07:18 |
| GheRivero | ALTER TABLE nodes ADD COLUMN provision_updated_at DATETIME | 07:19 |
| GheRivero | http://paste.openstack.org/show/117373/ | 07:19 |
| *** jcoufal has joined #openstack-ironic | 07:20 | |
| devananda | GheRivero: I see. what db backend are you using // how is it configured? | 07:20 |
| GheRivero | you just need a mysql server with the account openstack_citest :openstack_citest | 07:22 |
| devananda | that's what I have | 07:22 |
| devananda | and I see alembic creating tables and such, but no ALTERs | 07:22 |
| GheRivero | I didn't try with pg | 07:26 |
| devananda | i'm using mysql | 07:26 |
| GheRivero | weird that you get the insert but not the alter | 07:28 |
| GheRivero | the first time i have some issues with the openstack_citest user permissions, but the tests were falling long way before doing anything | 07:29 |
| *** Haomeng has joined #openstack-ironic | 07:35 | |
| *** Haomeng|2 has quit IRC | 07:36 | |
| *** Haomeng|2 has joined #openstack-ironic | 07:39 | |
| *** Haomeng has quit IRC | 07:40 | |
| *** coolsvap|afk is now known as coolsvap | 07:48 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add "affinity" tracking to nodes and conductors https://review.openstack.org/124493 | 07:49 |
| *** andreykurilin_ has joined #openstack-ironic | 07:51 | |
| *** MattMan has quit IRC | 07:56 | |
| *** MattMan has joined #openstack-ironic | 07:57 | |
| *** jistr has joined #openstack-ironic | 08:02 | |
| *** ndipanov_gone is now known as ndipanov | 08:04 | |
| *** lsmola has joined #openstack-ironic | 08:04 | |
| *** viktors|afk is now known as viktors | 08:08 | |
| *** yuanying has quit IRC | 08:10 | |
| *** yuanying has joined #openstack-ironic | 08:11 | |
| *** yuanying has quit IRC | 08:15 | |
| *** lucasagomes has joined #openstack-ironic | 08:18 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state https://review.openstack.org/124610 | 08:20 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent https://review.openstack.org/118932 | 08:21 |
| * devananda sleeps | 08:22 | |
| *** coolsvap is now known as coolsvap|afk | 08:24 | |
| *** derekh has joined #openstack-ironic | 08:26 | |
| lucasagomes | devananda, g'night :) | 08:38 |
| *** igordcard has joined #openstack-ironic | 08:41 | |
| vdrok | morning ironic! | 08:55 |
| vdrok | devananda, will investigate db tests today | 08:56 |
| *** andreykurilin_ has quit IRC | 09:00 | |
| *** yuanying has joined #openstack-ironic | 09:12 | |
| *** loki184 has joined #openstack-ironic | 09:16 | |
| *** rushiagr is now known as rushiagr_away | 09:31 | |
| *** dtantsur is now known as dtantsur|lunch | 09:38 | |
| *** wendar has quit IRC | 09:38 | |
| *** wendar has joined #openstack-ironic | 09:38 | |
| *** yuanying has quit IRC | 09:46 | |
| openstackgerrit | A change was merged to openstack/ironic: Updated from global requirements https://review.openstack.org/125274 | 10:02 |
| *** tonycoffman has quit IRC | 10:17 | |
| *** sirushti has quit IRC | 10:17 | |
| *** sirushti has joined #openstack-ironic | 10:17 | |
| *** tonycoffman has joined #openstack-ironic | 10:18 | |
| openstackgerrit | Anusha Ramineni proposed a change to openstack/ironic: ilo* drivers to use only ilo credentials https://review.openstack.org/124704 | 10:19 |
| openstackgerrit | Julien Danjou proposed a change to openstack/ironic: Replace custom lazy loading by stevedore https://review.openstack.org/125090 | 10:32 |
| *** yuanying has joined #openstack-ironic | 10:42 | |
| *** ramineni has quit IRC | 11:00 | |
| *** loki184 has quit IRC | 11:02 | |
| *** nikunj2512 has quit IRC | 11:04 | |
| *** Haomeng has joined #openstack-ironic | 11:10 | |
| *** Haomeng|2 has quit IRC | 11:11 | |
| *** yuanying has quit IRC | 11:16 | |
| *** rakesh_hs has quit IRC | 11:25 | |
| openstackgerrit | A change was merged to openstack/ironic: Use DbTestCase as test base when context needed https://review.openstack.org/124400 | 11:29 |
| *** GheRivero has quit IRC | 11:30 | |
| *** GheRivero has joined #openstack-ironic | 11:30 | |
| *** GheRivero has quit IRC | 11:30 | |
| *** lazy_prince is now known as killer_prince | 11:31 | |
| *** GheRivero has joined #openstack-ironic | 11:31 | |
| *** yuanying has joined #openstack-ironic | 11:42 | |
| *** dtantsur|lunch is now known as dtantsur | 11:46 | |
| dtantsur | jroll, hi, what's the staus of agent jobs? devstack fails for them: http://logs.openstack.org/90/125090/2/check/check-tempest-dsvm-ironic-agent_ssh-nv/dfa94dd/logs/devstacklog.txt.gz | 11:46 |
| *** dprince has joined #openstack-ironic | 11:55 | |
| *** lucasagomes is now known as lucas-hungry | 12:07 | |
| *** pensu has quit IRC | 12:07 | |
| *** yuanying has quit IRC | 12:15 | |
| openstackgerrit | A change was merged to openstack/ironic: Replace custom lazy loading by stevedore https://review.openstack.org/125090 | 12:20 |
| Shrews | s'up, ironic? | 12:25 |
| *** igordcard has quit IRC | 12:27 | |
| dtantsur | Shrews, hi | 12:38 |
| *** romcheg has quit IRC | 12:38 | |
| *** igordcard has joined #openstack-ironic | 12:39 | |
| *** romcheg has joined #openstack-ironic | 12:44 | |
| *** andreykurilin_ has joined #openstack-ironic | 12:49 | |
| *** pensu has joined #openstack-ironic | 12:54 | |
| *** shikui_ has quit IRC | 12:54 | |
| *** andreykurilin_ has quit IRC | 12:57 | |
| *** rloo has joined #openstack-ironic | 13:05 | |
| *** yuanying has joined #openstack-ironic | 13:12 | |
| jroll | morning ironig | 13:12 |
| jroll | dtantsur: it's a known issue, tl;dr is that ramdisk doesn't load in 512MB RAM | 13:13 |
| jroll | dtantsur: but setting that higher isn't viable long term :( | 13:13 |
| dtantsur | jroll, morning, understood | 13:13 |
| *** lucas-hungry is now known as lucasagomes | 13:14 | |
| lucasagomes | jroll, morning | 13:14 |
| jroll | morning lucas | 13:14 |
| *** pensu has quit IRC | 13:17 | |
| Shrews | lucasagomes: i'm sure you saw, but making sure... moved your node object code from 124610 to 124493 and made you co-author on it. | 13:28 |
| lucasagomes | Shrews, yup! saw it | 13:28 |
| *** nosnos has quit IRC | 13:28 | |
| lucasagomes | thanks for moving it and adding me as co-author | 13:28 |
| lucasagomes | I'm adding tests to the hash ring and addressing comments | 13:28 |
| *** nosnos has joined #openstack-ironic | 13:29 | |
| lucasagomes | there's some other comments to address on 124493 too | 13:29 |
| lucasagomes | Shrews, and morning :) | 13:31 |
| Shrews | morning lucasagomes, dtantsur, jroll, and anyone i missed | 13:31 |
| *** pensu has joined #openstack-ironic | 13:32 | |
| jroll | heya Shrews :) | 13:32 |
| *** nosnos has quit IRC | 13:33 | |
| *** yuanying has quit IRC | 13:46 | |
| *** pelix has joined #openstack-ironic | 14:00 | |
| *** krtaylor has quit IRC | 14:01 | |
| *** dprince has quit IRC | 14:07 | |
| *** openstackgerrit has quit IRC | 14:18 | |
| *** openstackgerrit has joined #openstack-ironic | 14:19 | |
| *** ramineni has joined #openstack-ironic | 14:23 | |
| openstackgerrit | Anusha Ramineni proposed a change to openstack/ironic: ilo* drivers to use only ilo credentials https://review.openstack.org/124704 | 14:25 |
| *** krtaylor has joined #openstack-ironic | 14:25 | |
| *** dprince has joined #openstack-ironic | 14:37 | |
| *** ramineni has quit IRC | 14:37 | |
| *** dhellmann has joined #openstack-ironic | 14:38 | |
| *** dhellmann has quit IRC | 14:38 | |
| openstackgerrit | Vladyslav Drok proposed a change to openstack/ironic: Move database fixture to a separate test case https://review.openstack.org/125361 | 14:40 |
| *** dhellmann has joined #openstack-ironic | 14:41 | |
| *** yuanying has joined #openstack-ironic | 14:42 | |
| *** pcrews has joined #openstack-ironic | 14:47 | |
| *** openstackgerrit has quit IRC | 14:47 | |
| *** openstackgerrit has joined #openstack-ironic | 14:49 | |
| NobodyCam | morning ironic....almost | 14:50 |
| dtantsur | NobodyCam, hi! almost morning or almost ironic? :) | 14:53 |
| romcheg | Or almost NobodyCam? :-P | 14:53 |
| romcheg | Hi guys! | 14:53 |
| *** romcheg has left #openstack-ironic | 14:54 | |
| dtantsur | rom<TAB> :( | 14:55 |
| openstackgerrit | A change was merged to openstack/python-ironicclient: Add "ironic node-set-power-state" cmd unit test https://review.openstack.org/124267 | 14:56 |
| *** dhellmann_ has quit IRC | 14:57 | |
| lucasagomes | NobodyCam, morning | 15:00 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state https://review.openstack.org/124610 | 15:00 |
| lucasagomes | with tests now ^ | 15:00 |
| *** dhellmann_ has joined #openstack-ironic | 15:01 | |
| *** chuckC_ has joined #openstack-ironic | 15:02 | |
| *** openstackgerrit has quit IRC | 15:02 | |
| *** openstackgerrit has joined #openstack-ironic | 15:04 | |
| NobodyCam | Okay Now G.M.I. (Good MOrning Ironic) | 15:05 |
| *** dhellmann_ has quit IRC | 15:05 | |
| NobodyCam | morning dtantsur, rocheg lucasagomes, Shrews and rloo | 15:05 |
| Shrews | morning NobodyCam | 15:06 |
| *** dhellmann_ has joined #openstack-ironic | 15:06 | |
| rloo | Hi NobodyCam, dtantsur, lucasagomes, Shrews ;) | 15:06 |
| NobodyCam | :) | 15:07 |
| dtantsur | rloo, hi | 15:07 |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Use poll instead of threading.Event in heartbeat thread https://review.openstack.org/113343 | 15:09 |
| jroll | ^ this was a fun one | 15:10 |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Use poll instead of threading.Event in heartbeat thread https://review.openstack.org/113343 | 15:11 |
| lucasagomes | rloo, morning! | 15:13 |
| *** viktors is now known as viktors|afk | 15:13 | |
| *** yuanying has quit IRC | 15:15 | |
| *** mitz_ has quit IRC | 15:18 | |
| *** Poornima has joined #openstack-ironic | 15:18 | |
| NobodyCam | anyone happen to know where I could find the ipmi protocol specification | 15:19 |
| Shrews | jroll: you know that poll() call is blocking, right? unless you use eventlet to monkeypatch select lib somewhere | 15:19 |
| lucasagomes | NobodyCam, yup, 1 sec | 15:19 |
| jroll | Shrews: yeah, it's fine, that's a separate thread | 15:20 |
| lucasagomes | http://www.intel.com/content/www/us/en/servers/ipmi/ipmi-specifications.html | 15:20 |
| lucasagomes | NobodyCam, ^ | 15:20 |
| jroll | (I think that's fine) | 15:20 |
| NobodyCam | also anyone have a ipmi system that htey could past the output of: ipmitool -H 127.0.0.1 —vvvv —I lan chassis power off | 15:20 |
| jroll | Shrews: imbw, is that going to block from context switching? | 15:20 |
| jroll | NobodyCam: sure | 15:20 |
| NobodyCam | lucasagomes: Thank you :) | 15:21 |
| Shrews | jroll: not thread context switching. just blocks your one thread | 15:22 |
| jroll | Shrews: yeah, that's what I thought, that's fine | 15:22 |
| jroll | NobodyCam: though we've always used lanplus, not sure what -I lan does or if it will work | 15:23 |
| * jroll tries | 15:23 | |
| Shrews | jroll: cool. i don't know that code well, so just mentioning it | 15:23 |
| Shrews | s/well/at all/ | 15:23 |
| *** mitz_ has joined #openstack-ironic | 15:23 | |
| *** Poornima has quit IRC | 15:25 | |
| *** dhellmann_ has quit IRC | 15:26 | |
| openstackgerrit | Sam Betts proposed a change to openstack/ironic: Add a call to management.validate(task) https://review.openstack.org/125387 | 15:26 |
| lucasagomes | Shrews, are u taking a look at 124493 comments ? if not I will put a patch up to address them | 15:27 |
| Shrews | lucasagomes: no, but i was thinking about dtantsur's comment about DBDuplicateEntry | 15:28 |
| *** dhellmann_ has joined #openstack-ironic | 15:28 | |
| lucasagomes | Shrews, yeah :/ hmm did you find something? is that possible? | 15:28 |
| Shrews | lucasagomes: i'm not sure yet | 15:29 |
| lucasagomes | I haven't looked deep into it | 15:29 |
| lucasagomes | ok | 15:29 |
| jroll | NobodyCam: PM'd | 15:29 |
| jroll | ERROR (BadRequest): Multiple possible networks found, use a Network ID to be more specific. (HTTP 400) (Request-ID: req-d9987c01-0dcd-4a8f-89d0-a8557587c812) | 15:31 |
| jroll | just got this in our normal devstack setup | 15:31 |
| Shrews | lucasagomes: i did run the patch through devstack and it seems to work | 15:32 |
| jroll | is that something new or did I break it? | 15:32 |
| Shrews | jroll: new-ish | 15:32 |
| jroll | ugh | 15:32 |
| Shrews | use --nic net-id=private, iirc | 15:32 |
| Shrews | to nova boot | 15:32 |
| jroll | thanks | 15:32 |
| jroll | ugh, has to be uuid | 15:33 |
| *** dtantsur is now known as dtantsur|afk | 15:33 | |
| Shrews | oh, yeah :( | 15:33 |
| jroll | nova network-list | 15:33 |
| Shrews | jroll: NET=$(nova network-list | grep "private" | awk '{print $2}') | 15:33 |
| jroll | nice | 15:33 |
| * jroll adds to docs | 15:33 | |
| jroll | thanks | 15:33 |
| *** dhellmann has quit IRC | 15:34 | |
| *** dhellmann_ is now known as dhellmann | 15:34 | |
| *** jasondotstar has joined #openstack-ironic | 15:34 | |
| lucasagomes | Shrews, w00t! | 15:34 |
| lucasagomes | did you or deva tested the neutron bit yesterday? | 15:35 |
| NobodyCam | jroll: does inaplus work? | 15:35 |
| Shrews | lucasagomes: well, by work, i mean "conductor_affinity" gets set properly | 15:35 |
| jroll | NobodyCam: lanplus works for me, I have you output with just lan | 15:35 |
| lucasagomes | heh yeah | 15:35 |
| NobodyCam | jroll: could you paste both? | 15:35 |
| jroll | NobodyCam: I just closed everything :/ | 15:35 |
| Shrews | lucasagomes: neutron bit? | 15:35 |
| jroll | NobodyCam: give me a few | 15:35 |
| NobodyCam | jroll: no need to re open | 15:36 |
| lucasagomes | Shrews, yeah, on the take_over() | 15:36 |
| NobodyCam | this is good | 15:36 |
| lucasagomes | I could test it because I was doing my tests in only 1 vm | 15:36 |
| jroll | NobodyCam: I can if it helps | 15:36 |
| jroll | ok | 15:36 |
| Shrews | lucasagomes: oh, not yet (at least i haven't) | 15:36 |
| lucasagomes | so I got as far as checking that the images are cached and the configs are generated correctly... I see neutron being called but | 15:37 |
| lucasagomes | to test it we need another vm, 1 conductor per vm | 15:37 |
| lucasagomes | > 1 ir-cond in the same host is not supported | 15:37 |
| Shrews | lucasagomes: ah, hrm | 15:37 |
| lucasagomes | Shrews, right, no bothers | 15:37 |
| * Shrews wants our test environment setup with docker containers to test such things | 15:38 | |
| lucasagomes | Shrews, +1!!!! | 15:38 |
| lucasagomes | that would be awesome | 15:39 |
| Shrews | dox could eventually give us that... in time | 15:39 |
| lucasagomes | yush! | 15:39 |
| *** igordcard has quit IRC | 15:42 | |
| *** mikedillion has joined #openstack-ironic | 15:42 | |
| *** andreykurilin_ has joined #openstack-ironic | 15:46 | |
| *** todd_dsm has joined #openstack-ironic | 15:49 | |
| openstackgerrit | Sam Betts proposed a change to openstack/ironic: Add a call to management.validate(task) https://review.openstack.org/125387 | 15:56 |
| lucasagomes | Shrews, once you unprovisioned ur node, did the conductor_affinity got unset? | 15:56 |
| *** chuckC_ has quit IRC | 16:03 | |
| *** ifarkas has quit IRC | 16:04 | |
| *** k4n0 has quit IRC | 16:05 | |
| *** yuanying has joined #openstack-ironic | 16:12 | |
| NobodyCam | grrrrr....brb :-p | 16:14 |
| *** dprince_ has joined #openstack-ironic | 16:15 | |
| *** annegentle_ has quit IRC | 16:17 | |
| *** dprince has quit IRC | 16:17 | |
| *** mordred has quit IRC | 16:17 | |
| *** harlowja_away has quit IRC | 16:17 | |
| *** annegentle has joined #openstack-ironic | 16:17 | |
| *** openstackgerrit has quit IRC | 16:18 | |
| *** annegentle is now known as Guest84781 | 16:18 | |
| rloo | hi lucasagomes, wrt conductor_affinity (124493). | 16:18 |
| *** mordred has joined #openstack-ironic | 16:18 | |
| rloo | lucasagomes: that value is meant to stick even after a deploy is done. | 16:18 |
| rloo | lucasagomes: which is why it is never reset to None. | 16:18 |
| lucasagomes | rloo, so in the last review that deva put it up seems so | 16:18 |
| lucasagomes | but I don't think it should | 16:19 |
| lucasagomes | I think it should be removed once the node is undeployed | 16:19 |
| rloo | lucasagomes: yeah, my fault maybe. See his comments in patch 12. | 16:19 |
| lucasagomes | I left some comments there | 16:19 |
| lucasagomes | will see | 16:19 |
| *** openstackgerrit has joined #openstack-ironic | 16:19 | |
| rloo | lucasagomes: https://review.openstack.org/#/c/124493/12/ironic/conductor/manager.py | 16:20 |
| NobodyCam | more coffee needed | 16:21 |
| lucasagomes | rloo, right, yeah hmm | 16:22 |
| lucasagomes | well I see it's not needed, but I think that leaving it in the db is a bit misleading | 16:22 |
| rloo | lucasagomes: yeah. hmmm :-) I now wonder why we don't just set that when a node is registered/mapped to a conductor, regardless of whether a deploy was done or not. | 16:22 |
| lucasagomes | cause that will only get updated if the node *is deployed* when running the periodic task | 16:22 |
| rloo | lucasagomes: well, it isn't misleading. it depends on what you think the definition of that field is. | 16:22 |
| lucasagomes | rloo, well that the last conductor that have managed that node | 16:23 |
| rloo | lucasagomes: if the def is 'the last conductor to ... ', then it is still true even after deployment. the last conductor that did something hasn't changed. | 16:23 |
| lucasagomes | yeah | 16:23 |
| lucasagomes | I think that's the definition, but still hmmm | 16:23 |
| * lucasagomes thinking | 16:23 | |
| lucasagomes | I see ppl trying to use it as "get all the nodes managed by this conductor" | 16:24 |
| lucasagomes | and using that field | 16:24 |
| lucasagomes | but as the node is not deployed that field might be old | 16:24 |
| devananda | morning, all | 16:24 |
| lucasagomes | idk I've mixing feelings about it, I got think more | 16:24 |
| lucasagomes | devananda, morning | 16:24 |
| rloo | morning devananda. you're on the hot seat with your conductor_affinity ;) | 16:24 |
| devananda | lucasagomes: right now, that field could indicate an active deploy OR an active console session OR both | 16:25 |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state https://review.openstack.org/124610 | 16:25 |
| devananda | lucasagomes: in this case, people shouldn't use it --it's not exposed in the API,a nd I dont think it should be | 16:25 |
| rloo | or non-active. | 16:25 |
| NobodyCam | good morning devananda :) | 16:25 |
| lucasagomes | devananda, right, the console bit is another thing... cause it doesn't work in the take_over() | 16:25 |
| lucasagomes | I think that for this realease we may want to constraint the use of that field | 16:26 |
| lucasagomes | only for the deploy/undeploy bit | 16:26 |
| lucasagomes | and once we fix the console we can set the affinity there as part of that patch | 16:26 |
| devananda | sure | 16:26 |
| devananda | that's fine, too | 16:26 |
| lucasagomes | leaving a TODO/FIXME on the console part is ok | 16:26 |
| devananda | but we also still don't need to unset it :) | 16:26 |
| lucasagomes | but having partial fixes :/ | 16:26 |
| lucasagomes | yeah we don't, I have to think more about it | 16:27 |
| lucasagomes | my fear is ppl misusing it to find the nodes that controler X is managing and using that field | 16:27 |
| lucasagomes | but I might be being a bit paranoid here | 16:27 |
| JayF | I would absolutely misuse it in that way | 16:27 |
| lucasagomes | ^ | 16:28 |
| *** marcoemorais has joined #openstack-ironic | 16:28 | |
| lucasagomes | yeah so, by setting when deployed and unset when undeployed makes it simple | 16:28 |
| JayF | Out of date data is slightly better than no data at all | 16:28 |
| lucasagomes | at least for this release | 16:28 |
| lucasagomes | JayF, +1 | 16:28 |
| JayF | lucasagomes: for IPA, conductors are needed even for inactive nodes | 16:28 |
| lucasagomes | well... | 16:28 |
| JayF | lucasagomes: to handle heartbeats from running agents | 16:28 |
| devananda | lucasagomes: so a TODO or NOTE to change the default interval seems poor. standard is 60s. I think we leave it there, let people change it if needed. | 16:28 |
| devananda | JayF: so you would update that field even for heartbeats? | 16:29 |
| Shrews | lucasagomes: 'nova delete' leaves conductor_affinity set | 16:29 |
| Shrews | not sure if that's correct? | 16:29 |
| lucasagomes | Shrews, yeah we are talking bout it now | 16:29 |
| devananda | Shrews: that was my intent in rev8, yes | 16:29 |
| devananda | as I see it, there's never a reason to unset it | 16:29 |
| * Shrews reads sb | 16:30 | |
| JayF | devananda: I don't think we'd update it, just noting that whereas for pxe driver, the undeployed server is essentially just a paperweight, but with IPA+long running agents, it's not neccessarily | 16:30 |
| JayF | that being said, long running agents isn't in J | 16:30 |
| JayF | so have fun | 16:30 |
| JayF | lol | 16:30 |
| devananda | because there is never a time (except at the very start of the cluster ,or for new ndoes, when it's already NULL) that there was not previously a conductor to which this node was mapped | 16:30 |
| devananda | and the field indicates THAT | 16:30 |
| lucasagomes | devananda, leave as 60s default? yeah, well that's fine | 16:30 |
| devananda | "some time in the past, this conductor touched this node" | 16:30 |
| devananda | it's not intended to LIMIT the interaction to that conductor | 16:31 |
| devananda | and it's never checked / enforced | 16:31 |
| lucasagomes | right | 16:31 |
| devananda | if another conductor tries to manage that node (and gets the TaskManager lock) then great -- that's fine. it should update the conductor_affinity field when it's done | 16:31 |
| JayF | So for pxe driver, that would typically indicate, for instance, the conductor that pxe booted the instance? | 16:31 |
| lucasagomes | +1 yeah, maybe checking on the node lock if the affinity still correct | 16:31 |
| devananda | JayF: that last pxe booted it. even if the node is now off and has no instance. yes. | 16:32 |
| JayF | devananda: that makes sense, thanks :) | 16:32 |
| devananda | lucasagomes: no need to check it at all ever | 16:32 |
| devananda | that's my point | 16:32 |
| openstackgerrit | Sam Betts proposed a change to openstack/ironic: Add a call to management.validate(task) https://review.openstack.org/125387 | 16:32 |
| devananda | outside of "the ring changed, do I need to take_over()" | 16:32 |
| devananda | there is no need to read taht field | 16:32 |
| lucasagomes | yeah... I see it, I was most wondering on leaving that old data in the db | 16:33 |
| lucasagomes | and ppl building things on top of it... but yeah | 16:33 |
| lucasagomes | well we can leave it then | 16:33 |
| devananda | but it's not old data -- it has actual meaning | 16:33 |
| devananda | i'm sorry that I haven't communicated all this more clearly, and that it's left to the last minute like this. | 16:34 |
| lucasagomes | right, yeah "the last conductor that touched this node" | 16:34 |
| lucasagomes | devananda, it's grand... as I said I had to think more about it | 16:34 |
| lucasagomes | so talking about it helps with that | 16:34 |
| *** romcheg has joined #openstack-ironic | 16:34 | |
| * devananda needs to blog on how take over works. /me just need to blog. | 16:35 | |
| lucasagomes | lol | 16:35 |
| lucasagomes | g+? | 16:35 |
| * NobodyCam would read that blog | 16:35 | |
| rloo | why a blog? why not part of our doc? | 16:36 |
| devananda | it *also* should become doc'd | 16:37 |
| NobodyCam | anyone seen Jbjohnson on of late? | 16:37 |
| lucasagomes | NobodyCam, nop | 16:38 |
| NobodyCam | :-p | 16:38 |
| devananda | lucasagomes: so what's the next step. it looks like you've added a few revs to this -- should I take a look, or are you posting another? | 16:38 |
| devananda | I really really want to land *something* now. | 16:38 |
| lucasagomes | devananda, I added tests to it | 16:38 |
| devananda | so we can cut RC1 | 16:38 |
| lucasagomes | +1 | 16:39 |
| devananda | and unblock everything else :) | 16:39 |
| lucasagomes | devananda, the neutron bit still unsure tho, I didn't test it yet :( | 16:39 |
| devananda | I also think the reviews and questions have been super helpful | 16:39 |
| devananda | lucasagomes: ah, yea, i didn't get time yesterday | 16:39 |
| devananda | lucasagomes: i'll be in meetings again in an hour. might have time this afternoon... i hope | 16:40 |
| lucasagomes | devananda, ack, no problem | 16:40 |
| *** todd_dsm has quit IRC | 16:40 | |
| rloo | lucasagomes: does the periodic task stuff work with this new interpretation of conductor_affinity? (Am asking before I review.) | 16:41 |
| lucasagomes | devananda, so the reviews I added has tests, and one behavior change if u set that internval to 0 the periodic task will be non-op | 16:41 |
| lucasagomes | in case ppl want to disable it | 16:41 |
| devananda | oh, good. thanks! | 16:41 |
| lucasagomes | rloo, yup, the perirodic task filters for nodes associated with an instance | 16:41 |
| devananda | being able to disable it ++ | 16:41 |
| lucasagomes | rloo, and check the affinity, if it's not set to the conductor that is suppose to manage it it just skip that node | 16:42 |
| lucasagomes | if it's the right conductor it updates that field | 16:42 |
| lucasagomes | rloo, so I think it's all fine | 16:42 |
| rloo | lucasagomes: thx. will look after I grab some food! | 16:43 |
| devananda | Shrews: aren't the FK index names auto-generated? | 16:43 |
| lucasagomes | rloo, ack! thanks | 16:43 |
| devananda | Shrews: I just realized it is "iso_fk" which seems ... not auto generated | 16:43 |
| *** romcheg has quit IRC | 16:44 | |
| *** yuanying has quit IRC | 16:45 | |
| *** romcheg has joined #openstack-ironic | 16:46 | |
| NobodyCam | should we fix/land https://review.openstack.org/#/c/118614 before we cut rc? | 16:49 |
| *** derekh has quit IRC | 16:49 | |
| *** romcheg has quit IRC | 16:49 | |
| devananda | lucasagomes: I'm fixing up 124493, unless you're actively working on it, then I'll wait | 16:50 |
| lucasagomes | devananda, no, I started but didn't progress much | 16:50 |
| lucasagomes | I will have to go soon | 16:51 |
| lucasagomes | devananda, so, go ahead | 16:51 |
| devananda | k k | 16:51 |
| lucasagomes | aight I think I should go (still need to finish packing his stuff to move out) | 16:52 |
| lucasagomes | have a good night everyone | 16:52 |
| *** lucasagomes is now known as lucas-afk | 16:52 | |
| *** andreykurilin_ has quit IRC | 16:53 | |
| NobodyCam | have a good noght lucas-afk | 16:54 |
| *** jcoufal has quit IRC | 16:54 | |
| NobodyCam | night even | 16:54 |
| Shrews | devananda: they have to be named so that we can manually drop the FK before dropping the column in the migration script | 16:55 |
| devananda | Shrews: sure. i mean, i thought the names were generated systematicaly | 16:56 |
| devananda | there was some work in oslo to do that a year or two ago | 16:56 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/ironic: Update hacking version in test requirements https://review.openstack.org/125423 | 16:56 |
| Shrews | devananda: but what is the name we'd use when dropping the FK? is it guessable? | 16:56 |
| devananda | Shrews: i'm not saying this clearly :( | 16:57 |
| Shrews | devananda: please use new words :-P | 16:57 |
| devananda | Shrews: eg, http://lists.openstack.org/pipermail/openstack-dev/2013-December/022647.html | 16:58 |
| devananda | https://review.openstack.org/#/c/84444/ | 16:58 |
| devananda | "will have a name such as <tablename>_<columnname>_fk" | 16:58 |
| *** vdrok_ has joined #openstack-ironic | 16:59 | |
| *** vdrok__ has joined #openstack-ironic | 16:59 | |
| Shrews | devananda: from what i recall, it was "nodes_ibfk_2" or similar | 17:00 |
| *** vdrok__ has quit IRC | 17:00 | |
| devananda | that's ... not very helpful | 17:00 |
| Shrews | there is an existing nodes_ibfk_1 FK | 17:00 |
| devananda | right. auto generated names ftl | 17:01 |
| devananda | "ibfk_1" is terrible | 17:01 |
| devananda | nodes_conductor_affinity_fk is better | 17:01 |
| * devananda renames | 17:01 | |
| rloo | NobodyCam: wrt 118614, that's docn, it can be done after RC1 is cut. It'll show up whenever it is merged. | 17:02 |
| NobodyCam | :) rloo ack | 17:02 |
| Shrews | devananda: did you read note #3 on that email link? "Foreign keys shouldn't be created against nullable columns." We are doing that. | 17:05 |
| devananda | on or against? | 17:06 |
| Shrews | ah, against here. nm | 17:06 |
| devananda | we could make online non-NULLable | 17:07 |
| devananda | default True | 17:07 |
| devananda | I think that preserves the same behavior for new clusters. it might be better when upgrading an existing cluster | 17:07 |
| Shrews | that sounds reasonable for upgrade situations | 17:08 |
| *** rushiagr_away is now known as rushiagr | 17:10 | |
| *** jistr has quit IRC | 17:13 | |
| *** harlowja has joined #openstack-ironic | 17:15 | |
| *** killer_prince has quit IRC | 17:23 | |
| *** lsmola has quit IRC | 17:31 | |
| devananda | this is an odd error: InvalidRequestError: Object '<Conductor at 0x7f43aab5c650>' is already attached to session '341' (this is '342') | 17:35 |
| devananda | happens every time, if I select, then try to update that result, without an explicit session object | 17:36 |
| devananda | dtantsur|afk: ^ that's why we (currently) need a session | 17:36 |
| devananda | I suspect it's a bug, but am not going to dig into it further now | 17:36 |
| *** killer_prince has joined #openstack-ironic | 17:36 | |
| *** killer_prince is now known as lazy_prince | 17:37 | |
| *** yuanying has joined #openstack-ironic | 17:42 | |
| *** vdrok_ has quit IRC | 17:45 | |
| *** vdrok_ has joined #openstack-ironic | 17:46 | |
| *** eghobo has joined #openstack-ironic | 17:46 | |
| *** vdrok_ has quit IRC | 17:46 | |
| *** dlaube has joined #openstack-ironic | 18:01 | |
| *** tatyana has joined #openstack-ironic | 18:05 | |
| *** ChuckC has quit IRC | 18:05 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add "affinity" tracking to nodes and conductors https://review.openstack.org/124493 | 18:05 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add "affinity" tracking to nodes and conductors https://review.openstack.org/124493 | 18:09 |
| *** tatyana has quit IRC | 18:10 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state https://review.openstack.org/124610 | 18:11 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent https://review.openstack.org/118932 | 18:13 |
| devananda | rloo, lucas-afk: fixed up 124493 based on comments and our discussion. then rebased the others on top | 18:13 |
| devananda | afk for meetings and lunch... | 18:13 |
| rloo | thx devananda | 18:13 |
| *** yuanying has quit IRC | 18:16 | |
| *** rushiagr is now known as rushiagr_away | 18:19 | |
| *** ChuckC has joined #openstack-ironic | 18:25 | |
| *** pelix has quit IRC | 18:25 | |
| *** ChuckC has quit IRC | 18:25 | |
| *** ChuckC has joined #openstack-ironic | 18:25 | |
| *** chuckC_ has joined #openstack-ironic | 18:27 | |
| *** todd_dsm has joined #openstack-ironic | 18:27 | |
| *** Ng has quit IRC | 18:40 | |
| *** Ng has joined #openstack-ironic | 18:40 | |
| NobodyCam | brb | 18:59 |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Use poll instead of threading.Event in heartbeat thread https://review.openstack.org/113343 | 19:02 |
| openstackgerrit | Jim Rollenhagen proposed a change to openstack/ironic-python-agent: Force heartbeat immediately after async command completes https://review.openstack.org/125464 | 19:02 |
| *** yuanying has joined #openstack-ironic | 19:12 | |
| NobodyCam | brb | 19:39 |
| *** yuanying has quit IRC | 19:46 | |
| Shrews | I'm confused by https://review.openstack.org/125423. Does 'hacking' not get updated automatically? | 20:14 |
| Shrews | git log says it was updated automatically like that last Nov. did it change to manual at some point? | 20:16 |
| *** igordcard has joined #openstack-ironic | 20:27 | |
| *** romcheg has joined #openstack-ironic | 20:28 | |
| Shrews | fyi, hacking is a special case, as confirmed by infra | 20:34 |
| * Shrews gets lonely | 20:36 | |
| NobodyCam | :( | 20:36 |
| NobodyCam | so do we need to update hacking | 20:36 |
| NobodyCam | or will infra? | 20:37 |
| Shrews | NobodyCam: we should. i left a comment explaining why | 20:37 |
| * NobodyCam tries to cheer up Shrews | 20:37 | |
| *** rainya has joined #openstack-ironic | 20:37 | |
| NobodyCam | :) | 20:37 |
| *** romcheg has quit IRC | 20:38 | |
| jroll | wonder how badly IPA fails with later hacking | 20:38 |
| *** romcheg has joined #openstack-ironic | 20:38 | |
| *** romcheg has quit IRC | 20:38 | |
| *** yuanying has joined #openstack-ironic | 20:42 | |
| *** mikedillion has quit IRC | 20:51 | |
| *** todd_dsm has quit IRC | 20:55 | |
| *** marcoemorais has quit IRC | 21:02 | |
| *** andreykurilin_ has joined #openstack-ironic | 21:02 | |
| *** jasondotstar has quit IRC | 21:02 | |
| *** marcoemorais has joined #openstack-ironic | 21:02 | |
| *** marcoemorais has quit IRC | 21:03 | |
| NobodyCam | brb | 21:03 |
| *** marcoemorais has joined #openstack-ironic | 21:05 | |
| *** marcoemorais has quit IRC | 21:05 | |
| *** marcoemorais has joined #openstack-ironic | 21:05 | |
| *** marcoemorais has quit IRC | 21:06 | |
| *** marcoemorais has joined #openstack-ironic | 21:06 | |
| *** pensu has quit IRC | 21:11 | |
| * devananda returns from meetingland | 21:14 | |
| devananda | anyone around to do reviews on the RC blocking patches? | 21:14 |
| *** yuanying has quit IRC | 21:16 | |
| Shrews | yup | 21:17 |
| devananda | great | 21:17 |
| Shrews | devananda: i left a +1 on the first in the series (affinity). looking at the period task one now | 21:19 |
| * devananda looks at the first patch, sees a +1 and a -1 | 21:19 | |
| Shrews | the 2nd has all sorts of test failures. | 21:20 |
| devananda | urgh | 21:20 |
| devananda | k. passed everything locally | 21:21 |
| Shrews | some don't look related | 21:21 |
| devananda | rloo: re: your comments on https://review.openstack.org/#/c/124493/12..16/ironic/db/sqlalchemy/api.py - can you explain why a heartbeat shouldn't set online=True? | 21:21 |
| rloo | devananda: can you explain why it should? shouldn't it already be set properly? | 21:22 |
| devananda | rloo: it should indeed. but if it's not, should that conductor's heartbeats not be counted? | 21:22 |
| rloo | devananda: no, I don't think so. if it isn't, there's a bug or I'm not understanding something. | 21:23 |
| devananda | it would be a bug, yep | 21:23 |
| rloo | devananda: so I'd rather know about the bug and fix it, than to 'cover' it here. | 21:23 |
| devananda | and the result would be the conductor service is running, but gets removed from the hash ring | 21:23 |
| devananda | and doesn't log anything | 21:23 |
| devananda | actually - i can revert that part of the change | 21:24 |
| rloo | devananda: but the conductor service was offline? | 21:24 |
| devananda | and it will raise ConductorNotFound, whcih should at least get logged from the periodic task | 21:24 |
| devananda | I prefer a service to be self-correcting, rather than require an operator to notice the log entry | 21:24 |
| devananda | and I don't see any harm in it | 21:25 |
| devananda | if the conductor service is actually running and heartbeating, but somehow the DB is out of sync, why should it require an operator to come "fix" it? | 21:25 |
| devananda | why not just self-correct? | 21:25 |
| rloo | devananda: guess I'm worried about either a race condition or buggy code. So I'd rather not have some self-heal for something I'm not aware of. | 21:25 |
| Shrews | a network blip might hide one conductor. split brain type thing. we'd want it to come back when the network is fixed | 21:25 |
| jroll | self-healing++ | 21:26 |
| devananda | rloo: I would rather have it self-heal when it can. | 21:26 |
| rloo | but *when* does a conductor's online get set to False? | 21:26 |
| devananda | rloo: when it calls unregister | 21:26 |
| rloo | only with some change in the db directly or when unregister is called. | 21:26 |
| devananda | what if I'm trying to restart the service and somehow those messages get reversed | 21:26 |
| devananda | and now there's a running service but the DB record says "online=False" | 21:27 |
| devananda | rloo: my point isn't "recover from known failure" -- it is that this is a trivial way a service can recover from unknown failurse | 21:27 |
| devananda | now that the heartbeat is mroe than just the updated_at field, we should set both fields | 21:28 |
| rloo | devananda: Ok, if you see it that way. We should probably log a message in that case? if it was False?) | 21:28 |
| Shrews | devananda: actually, my comment makes me think... what if we do have a split-brain situation? wouldn't both conductors attempt to take over the other's nodes? | 21:29 |
| devananda | if we actually have two conductor services with the same *hostname*, all bets are off | 21:29 |
| rloo | devananda: what if an unregister happens, but while unregistering, the touch_conductor() gets called? | 21:30 |
| *** andreykurilin_ has quit IRC | 21:30 | |
| devananda | rloo: then, instead of that conductor appearing offline immediately, it will continue to appear to be online until the timeout is reached | 21:31 |
| devananda | rloo: and after that, it'll still appear offline | 21:31 |
| devananda | so the system will take a bit longer to converge on the correct state, but that's fine | 21:31 |
| devananda | see get_active_driver_dict -- it filters by both online=True and updated_at | 21:32 |
| devananda | Shrews: we're relying on the operator configuring each service with a unique hostname | 21:33 |
| rloo | devananda: sorry, i was thinking what if the dbapi.unregister_conductor gets called first (which sets online=False), then the touch_conductor() gets called which sets online=True. But maybe that will never happen. | 21:33 |
| devananda | Shrews: so a split brain here could happen eg. if using galera, and the heartbeat updates stop propagating across galera nodes | 21:33 |
| devananda | Shrews: so (this pool) and (that pool) of conductors each try to map all nodes onto their own half of the ring | 21:34 |
| rloo | devananda: anyway. If you feel strongly about adding online=True in touch_conductor (and jroll seems to like it too), I'd like to at least log if it was false->true. Since we're talking about something unexpected happening. | 21:35 |
| devananda | Shrews: that'd suck. let's not do that. also -- if the ironic DB gets that busy, it means someone is runing a *massive* cluster. | 21:35 |
| devananda | rloo: so, logging that will require issuing a SELECT query first. there isn't one right now. | 21:36 |
| jroll | rloo: I tend to like self-healing, I'm not very aware of the details for this right now | 21:36 |
| openstackgerrit | Josh Gachnang proposed a change to openstack/ironic-python-agent: Add command metrics to IPA API https://review.openstack.org/119981 | 21:36 |
| rloo | devananda: sigh. ok then. I don't think we want to issue a SELECT each time. | 21:36 |
| devananda | jroll: short version: heartbeat used to only set updated_at=now(). I'm adding another "online" field so we can track intentional shutdown vs. it-just-went-away | 21:36 |
| devananda | jroll: and I'm suggesting that the heartbeat just default to set "online=True" -- not that we expect it to have another value, but just in case | 21:37 |
| jroll | devananda: yeah, I think that's fine. we see the conductor is alive, mark it alive | 21:38 |
| jroll | right? | 21:38 |
| devananda | jroll: this is called only from the conductor's heartbeat periodic task | 21:38 |
| jroll | devananda: right, in that case we know it's alive | 21:38 |
| devananda | rloo: the code won't directly call unregister and then touch_conductor -- but if we were to move to a properly threaded db connector, then in theory, it might be possible | 21:39 |
| devananda | rloo: but I don't think that would actually result in a problem -- the cluster will still eventually notice that the conductor is offline | 21:40 |
| devananda | it's eventual state would be (online: True, updated_at: really-old-timestamp) | 21:40 |
| devananda | jroll: right | 21:40 |
| rloo | if we're ok with that eventual state, then that's fine. | 21:42 |
| openstackgerrit | A change was merged to openstack/ironic: Update hacking version in test requirements https://review.openstack.org/125423 | 21:42 |
| *** shikui_ has joined #openstack-ironic | 21:43 | |
| NobodyCam | devananda would we ever want to get a conductor that is offline? question comes from line 529 of https://review.openstack.org/#/c/124493/16/ironic/db/sqlalchemy/api.py | 21:44 |
| devananda | NobodyCam: you mean get a list of conductor services that are offline? | 21:45 |
| NobodyCam | gah /me luvs it when his irc forgets to scroll the screen | 21:45 |
| NobodyCam | devananda: yes | 21:46 |
| devananda | NobodyCam: I can't predict the future :) | 21:46 |
| Shrews | lousy PTL!!!! :-P | 21:48 |
| * Shrews hands devananda a Magic 8-Ball | 21:48 | |
| devananda | Shrews: THe 8-ball says: Something will definitely happen soon. | 21:49 |
| Shrews | http://www.ask8ball.net/ | 21:49 |
| devananda | rloo: so, instead of filtering on associated (since your'e correct, that's technically orthogonal to provision state, even though Nova sets it first) | 21:51 |
| NobodyCam | devananda: are you opposed to something like : http://paste.openstack.org/show/gwLgksRa7oGR7tH26hRP/ | 21:51 |
| rloo | devananda: wrt 124493, i'm ready to +2 if you can tell me what the plan is wrt unit tests. | 21:51 |
| devananda | rloo: you'd just use prov_state = ACTIVE ? | 21:51 |
| devananda | rloo: the plan is: I write them now :) | 21:52 |
| rloo | devananda: well, that's what is being checked after getting the lock | 21:52 |
| rloo | devananda: ok, so I'll wait for the tests then ;) | 21:52 |
| devananda | you have some good points on 124610 that need to be addressed | 21:53 |
| *** mordred has quit IRC | 21:54 | |
| *** mordred has joined #openstack-ironic | 21:54 | |
| devananda | rloo: actually, question for you abotu the tests | 21:55 |
| rloo | devananda: "I don't know" :-) | 21:55 |
| devananda | hehe | 21:56 |
| devananda | rloo: I think the only API change is adding "update_existing" parameter | 21:57 |
| devananda | rloo: are there other unit test changes you want to see? | 21:57 |
| rloo | devananda: I was right, I don't know. I didn't actually think about them. Just the lack of any seemed odd ;) Let me put on my thinking cap... | 21:58 |
| *** marcoemorais has quit IRC | 22:05 | |
| *** marcoemorais has joined #openstack-ironic | 22:06 | |
| *** marcoemorais has quit IRC | 22:06 | |
| *** marcoemorais has joined #openstack-ironic | 22:07 | |
| *** marcoemorais has quit IRC | 22:07 | |
| *** marcoemorais has joined #openstack-ironic | 22:07 | |
| *** mikedillion has joined #openstack-ironic | 22:10 | |
| *** yuanying has joined #openstack-ironic | 22:12 | |
| *** marcoemorais has quit IRC | 22:13 | |
| *** marcoemorais has joined #openstack-ironic | 22:13 | |
| rloo | devananda: is there or do we need to add a test for the new db changes? (in tests/db/sqlalchemy/test_migrations.py | 22:15 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add "affinity" tracking to nodes and conductors https://review.openstack.org/124493 | 22:15 |
| devananda | rloo: ^ | 22:15 |
| devananda | rloo: no. all migrations should be automatically tested | 22:16 |
| rloo | devananda: I looked quickly and I think you got them covered. I just added a comment (to the previous revision) with a few minor suggestions wrt tests. I have to go now but will review it 1-2 hrs from now I think. | 22:20 |
| *** rloo is now known as rloo_afk | 22:20 | |
| *** marcoemorais has quit IRC | 22:20 | |
| devananda | rloo_afk: thanks much | 22:20 |
| NobodyCam | brb | 22:23 |
| *** marcoemorais has joined #openstack-ironic | 22:31 | |
| NobodyCam | devananda: line 233 of https://review.openstack.org/#/c/124493/17/ironic/conductor/manager.py should we log something about unable to register conductor? | 22:32 |
| *** marcoemorais has quit IRC | 22:33 | |
| devananda | NobodyCam: it's already "shut down" so why log it again? | 22:33 |
| devananda | also, we shouldn't be adding more translatable strings now | 22:34 |
| devananda | also, changing whether it logs there is unrelated to the hash ring rebalancing | 22:34 |
| NobodyCam | ahh oh kay :) | 22:36 |
| jroll | devananda: you remember off hand the difference between nova reschedules / does not reschedule? | 22:40 |
| jroll | I remember it was a certain type of exception | 22:40 |
| jroll | unrelated, as much as I appreciate a string freeze, avoiding adding logging due to that sounds like a bad decision, if it's good loggin | 22:41 |
| jroll | g | 22:41 |
| jroll | ("why log it again" is a fine reason) | 22:42 |
| devananda | jroll: I don't disagree | 22:42 |
| jroll | ok :) | 22:42 |
| *** tatyana has joined #openstack-ironic | 22:42 | |
| devananda | jroll: but also, i feel frustrated in that we haven't fixed this yet | 22:42 |
| *** tatyana has quit IRC | 22:42 | |
| jroll | devananda: ha, indeed | 22:42 |
| jroll | devananda: can I help? just need code reviews? | 22:43 |
| * jroll blindly +A's everything | 22:43 | |
| devananda | and I"m getign frustrated with nit picks :) | 22:43 |
| devananda | jroll: yes. reviews are great | 22:43 |
| devananda | jroll: and if you can test it with two conductors on different IPs to amke sure it's working with Neutron properly, that'd be grand | 22:44 |
| devananda | i haven't had time, and neither has lucas | 22:44 |
| devananda | and that's the last functional piece we haven't tested | 22:44 |
| *** Guest84781 is now known as annegentle | 22:44 | |
| jroll | if I had a pxe driver environment I'd love to :( | 22:44 |
| *** yuanying has quit IRC | 22:45 | |
| devananda | :( | 22:47 |
| *** yuanying has joined #openstack-ironic | 22:48 | |
| jroll | devananda: I wonder if this would work with a devstack + a conductor on a separate VM | 22:50 |
| jroll | I guess it could | 22:51 |
| jroll | let me see what I can do | 22:51 |
| NobodyCam | 124493 reviewed | 22:51 |
| devananda | jroll: probably could. cheers | 22:53 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Add periodic task to rebuild conductor local state https://review.openstack.org/124610 | 22:53 |
| devananda | NobodyCam: tyvm | 22:54 |
| NobodyCam | lol I was just lookst at ^^^ did you happen to add a doc string to def _get_ksclient(): | 22:55 |
| NobodyCam | its an internal function and the name is clear enough ... | 22:55 |
| NobodyCam | but every other function has at least a one liner | 22:56 |
| jroll | idk what a ksclient is :P | 22:56 |
| NobodyCam | *every other function in that file | 22:56 |
| NobodyCam | :-p | 22:56 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack/ironic: Make hash ring mapping be more consistent https://review.openstack.org/118932 | 22:57 |
| jroll | devananda: mind rebasing 118932 real oh there you go :P | 22:57 |
| devananda | NobodyCam: lol | 22:58 |
| NobodyCam | is line 877 of https://review.openstack.org/#/c/124610/13/ironic/conductor/manager.py missing a _ | 22:59 |
| *** lucas-afk has quit IRC | 23:02 | |
| devananda | NobodyCam: nope | 23:04 |
| devananda | it's *not* translated | 23:04 |
| devananda | I can replace the , with % | 23:05 |
| jroll | oh lord, I need to mysql/rabbit across the internet :| | 23:05 |
| jroll | (to do this easily) | 23:05 |
| devananda | jroll: urg. yup | 23:05 |
| NobodyCam | for me it was the () that wraped the string that thru me | 23:05 |
| devananda | ah | 23:06 |
| Shrews | sorry, had to do the dinner thing. around now. | 23:08 |
| devananda | dhellmann: going through your email/'pad now | 23:09 |
| Shrews | devananda: although i'm listed as co-author on 124493, i'm willing to +2A that thing at this point if no one else is reviewing it | 23:09 |
| NobodyCam | WB Shrews | 23:09 |
| *** mikedillion has quit IRC | 23:09 | |
| NobodyCam | Shrews: rloo said she would look at it in a couple of hours | 23:10 |
| Shrews | NobodyCam: ack | 23:10 |
| devananda | Shrews: looks like rloo wanted unit tests, but otherwise +'d it | 23:11 |
| devananda | so ya, probably good to give her a chance to review the unit tests I added | 23:11 |
| Shrews | 124610 is looking good to me. not sure how to test that, though | 23:30 |
| * Shrews wants to add a +1.5 vote | 23:30 | |
| Shrews | brb | 23:31 |
| rloo_afk | I'm back but I would NOT be offended if you +2 and +A'd something in my absence! | 23:37 |
| *** rloo_afk is now known as rloo | 23:37 | |
| *** dprince_ has quit IRC | 23:39 | |
| *** romcheg has joined #openstack-ironic | 23:40 | |
| NobodyCam | devananda: correct me if I am wrong, if a node in is a state like deploying and the conductor handling the deploy dies, then affinity will not get updated untill the deploy actually times out and the p-task runs after the time out | 23:40 |
| *** romcheg has quit IRC | 23:48 | |
| *** mikedillion has joined #openstack-ironic | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!