* NobodyCam shakes fist at nova for `ironicclient.common.apiclient.exceptions.Conflict: Unable to attach VIF because VIF # is already attached to Ironic Port # | 17:57 | |
NobodyCam | ` | 17:57 |
---|---|---|
NobodyCam | * ~2000 nodes on three conductors | 17:58 |
NobodyCam | and ofc Happy Saturday | 18:01 |
TheJulia | NobodyCam: I guess a failed deploy getting rescheduled? | 18:31 |
TheJulia | or just a vif being re-used? | 18:32 |
NobodyCam | LOL I wish... DC power outge | 18:32 |
TheJulia | ouch | 18:32 |
NobodyCam | outage | 18:32 |
NobodyCam | ;p | 18:32 |
TheJulia | I believe the cern folks asked if we could just kill the auto-vif attach on start with the ironic virt driver but I don't know what the outcome was | 18:33 |
NobodyCam | I will happily add my +1 to the request | 18:35 |
TheJulia | hopefully it is just log noise? | 18:36 |
NobodyCam | the tuff part for us is that nova-compute doesn't hart beat until all that and the scheduler update is complete | 18:37 |
NobodyCam | so compute service is about 45 minutes per node | 18:37 |
NobodyCam | we have three | 18:37 |
NobodyCam | and seems like the only way to do is start one by one | 18:38 |
TheJulia | that should be able to be nooped on the vif side of things, I think, but yeah :( | 18:38 |
TheJulia | cern restarted their 20 conductors all at once and bad things happened | 18:38 |
TheJulia | and well, they pair conductors with nova-computes | 18:38 |
TheJulia | soooooo yeah | 18:39 |
NobodyCam | just started another compute | 18:39 |
NobodyCam | will post when it hart beats | 18:39 |
TheJulia | I've been hacking in support to have a mandatory number of days | 18:41 |
TheJulia | for the history entry stuffs | 18:41 |
NobodyCam | :dance: | 18:41 |
TheJulia | I likely need to try and get the API done this next week if I want to make library freeze | 18:42 |
TheJulia | not that it *really* matters with microversioned APIs | 18:42 |
NobodyCam | oh sure give us a reason to upgrade (again) | 18:42 |
TheJulia | isen't that our job as a community? Fix issues and give you reasons to upgrade? | 18:42 |
NobodyCam | heheh oh ya | 18:43 |
opendevreview | Julia Kreger proposed openstack/ironic master: Record node history and manage events in db https://review.opendev.org/c/openstack/ironic/+/803292 | 18:46 |
TheJulia | I'm asuming some actions at logical points but ^^^ is a solid start | 18:46 |
NobodyCam | looks like via check ended , send inventory | 18:54 |
NobodyCam | sending* | 18:54 |
NobodyCam | hart beat received | 18:57 |
NobodyCam | ~20 minutes | 18:57 |
TheJulia | eek | 19:05 |
TheJulia | not good | 19:05 |
NobodyCam | that's the fast one | 19:05 |
* NobodyCam :ducks: | 19:05 | |
TheJulia | anyone with braincells for nova internals that can fix that^^^^ | 19:08 |
opendevreview | Julia Kreger proposed openstack/ironic master: Fix upgrade logic to allow for bundled changes https://review.opendev.org/c/openstack/ironic/+/803532 | 22:18 |
opendevreview | Julia Kreger proposed openstack/ironic master: Implements node history: database https://review.opendev.org/c/openstack/ironic/+/768009 | 22:19 |
opendevreview | Julia Kreger proposed openstack/ironic master: Record node history and manage events in db https://review.opendev.org/c/openstack/ironic/+/803292 | 22:19 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!